Should I Contact My Doctor About Stopping My Anti-Depressants?

I made a post maybe 2 weeks ago talking about how my doctor wanted to put me on fluoxetine. When I went into talk to him about having extreme anxiety at work (my first appointment in several years) he said that it was an anti-depressant but it would help my anxiety too which is why he wanted to put me on it but I have a very rapid heartbeat as it is and have spurts of my heart racing were I can't breathe well. I didn't like the medication within a week of me taking it so I didn't take it one day on purpose to see if it would help my migraines since I was getting extreme migraines from it as well (I also already have chronic migraines as well) And it did help my hand feel better and I believe after that I may be took it 2 more times and I haven't taken it since. It's been maybe 4 or 5 days since I've taken one and I really don't want to contact my doctors about it. they'll more than likely tell me to come in and last time I was in I felt like they were very inconvenienced to have me in (at least that's how they acted toward me). so I don't have a desire to call back and pay for anything else. since I've stopped taking them I've felt a ton better. I read if you take it long enough you can have withdrawals if you're not careful when getting off it and i don't like that at all. My overall question here is is it really necessary for me to go out of my way and contact my doctor's office after everything that I've already been through with them I will they're with them I wouldn't want to go in for another appointment or talk to my doctor so I just don't see a point in doing it. please leave advice below! ❤