I would try deleting the watch from the wear app and then adding it back via the connect watch to new phone feature on the watch. Settings/General/Transfer watch to new phone. When I switched my watch4 to my new phone, it did switch via smart switch, but it got confused. I had to go through the reconnection process.
