Exploring Food Intolerance and Alternatives

Does anyone have an off/on relationship with their intolerance? A period time you can have ice-cream, cream, cheese, etc but for another period of time you can with no reaction? A doctor once said I was probably allergic to the protein in dairy (idk what it is but it starts with a C) but I still don’t understand. Then also, does anyone have the exact same reaction of alt milks they do to dairy? I have never had an issue with eating almonds raw or oats, but almond milk and oak milk have the exact same effect on me as cows milk would.