Maybe start with some beginners resistance band and joint strengthening exercises? I also recommend kinesiology tape to take pressure off certain muscles and joints.

Have you tried a shoulder brace? My hip was having the problem and keeping it in a brace helped it from moving too much. I also recommend physical therapy with a knowledgeable therapist that can help strengthen the muscles and tendons around it so it can hold itself in place better

I second the comment about trying resistance bands! I have problems with my shoulders too, they can get so bad that they pop out when I’m just standing or sleeping, but it goes away when I use resistance bands for a few weeks. I usually do front & lateral raises (which can also be done with body weight), and then the standard and overhead pull-apart.