I just had this fixed by my carrier. It's not an app issue or the need to download a new app. You have to request the carrier to allow you to use visual voicemail. I asked my carrier to figure it out and they fixed it in 5 min. The preinstalled samsung voicemail app has visual voicemail in it. So when they unlocked it on my phone suddenly it populated with voicemail and I never downloaded a new app.
If your carrier doesn't support visual though, then you may have to download an app, I'm unsure. This is how I fixed it on my phone.
My memory may be failing me, but I thought past Samsung devices had a built in visual voicemail but I can't seem to enable it on my Note 10 plus.
I have an unlocked S10 and couldn't get the icon. I chatted with AT&T technical support and I think they helped me resolve the issue. I don't know if it was them, because i was messing with a lot of things on my phone, but a notification came on my screen that a new download for the Samsung visual voicemail was available, so I downloaded it. Now i have the app and moved it to my home screen. Works great.
I have a unlocked note 10 plus and all I had to do was call AT&T support and have them add the IMEI of the phone to their system. This enabled voLTE and allowed the preloaded Voicemail app to work. You will need the IMEI number (found under About Phone>Status>IMEI info) and your SIM card number. If the voicemail asks for a password try your AT&T pin thats what worked for me.