I believe I found a solution for music/games/movies (i.e. everything but phone calls). I had to go into the settings for my bluetooth headphones and disable the ability to use the headphones for calls. (Go to Bluetooth/under Paired Devices tap the gear next to the headphones/under "Use for" deselect "Calls").
The thing that tipped me off is the quality of the music sounded like a phone call. Mono, low bitrate.
I also use smartwatches. It seems related to dual audio and using multiple bluetooth devices at once but nothing I tried fixed the issue until I disabled the Calls functionality alltogether.
Has an actual solution come out for this? Im using an s20 ultra, doesn't have the dual audio option in settings. When I get in my car and connect to play music, it sounds awful. To get decent sound quality I have to disconnect from the watch, disconnect from the car and reconnect to the car. If the watch gets reconnected the quality goes back to being bad. Being disconnected from the watch is a "solution", but not one I think is okay for a phone that's $1400 and a $300 watch. Samsung support seems oblivious and just had me clear the cache partition, which didn't help.