Brexiters would rather trust the wisdom of ordinary people than the opinion of experts