I also have an unlocked Note 20 Ultra 5G on AT&T.
Have you confirmed that your plan at AT&T includes mobile hotspot? Also be aware that some AT&T plans only provided a certain amount of hotspot data per month.
Have you tried resetting the Network settings?
Have you tried wiping the device cache partition?