The question here is if someone in your room is causing these issues by interacting with the player. When the owner of the room clicks on “permissions” on the left he/she can set the permissions for the player to “Moderators only” in which case only the owner can control the player. Try to enable this and see if it makes a difference. How many users do you have in your room?
You can learn more about permissions here: How to use Watch2Gether?