Hey so I've been researching this. Apparently it's something they had to implement in the android 11 2nd beta. Do we know when this function will be coming to Samsung OneUi? It basically ensures that when a media player has stopped the service that the pesky media notification goes away and doesn't stay long after the app has been closed. It's a pet peeve of mine to not have a completely clean notification panel.
I was having this problem, but it corrected itself somehow. now, when I long press on the music/media notification shade widget, the "close" button actually works. maybe it was because I restarted the device? S20 +, full wipe.