Only 1% of Brits cared much about the EU before the 2016 Brexit vote