You don't need to take the batteries out of the remote! Go into the Settings, General, and then to the right go to Bixby Voice Settings, go down to Voice wake-up and go over to the right and turn it OFF. Problem solved!
I recently bought a 55" Q70R and mine does the same. I'll be playing a game and all of a sudden it just starts listening to voices in the room or the game I'm playing without pressing the remote button!
Can we turn Bixby off completely?
Have you tried the following steps to turn off Voice Wake-up? Press the Microphone button once, and then select Explore Now to go to the Explore Bixby screen. Select Settings and then select to turn off Voice Wake-up.