At the very most, just 23 Trump Tower residents voted for Donald Trump