The number of Congress members who won’t attend Donald Trump’s inauguration has surpassed 40