The hardest part of being a parent has nothing to do with raising kids