Not sure if this will work, but it's worth a try; go to Settings/Display and turn-on switch for "Touch Sensitivity".... I have that on my s20+ and assume you have the same. This is designed if you are using a screen protector. Hope it helps!
There are other topics already open on this issue. A large number of buyers, reported touchscreen issues for both 4G and 5G variants of the S20 FE. I am keeping an eye open on this issue as i also want to buy one of these models in the followin days, that if they manage to fix the problems.
There are a number of suggestions listed on reddit but none work for most of the users. I'd suggest you send it back and let them send you another unit. Somebody claimed that the second phone didn't have this problem so it might be a hardware problem.