Frustrated with Neurologist's Medication Interference

I'm beyond frustrated with my Neurologist... he's given me medication that interferes with my birth control multiple times now and this last time it's caused me a pregnancy scare.. I got in contact with my neurologist center to tell them about the issue and my Neurologist told them to tell me that I'm going to have to change my birth control because the medication he gave me is "specifically to target my seizures" which isn't that what all anti-epileptic meds are supposed to do??? I wouldn't have an issue with this but the birth control I'm on is the only one I've had that doesn't cause weight gain and I'm recovering from anorexia & bulimia so weight gain from medicine is terrifying to me. I feel like my neurologist doesn't listen to me and yelled at me at my last appointment because I stopped taking a medicine he gave me that made my seizures worse (I went from having 5-6 a week to having about 5-7 a day and was exhausted) I don't know what to do doctors in my area hardly take my insurance and I can't afford to not have a Neurologist but I hate this man.