There's currently a bug where the clock goes back to the default size randomly.
But I tested turning off the always on display and it seems to have stopped. My clock has stayed the same size I set it to since.
So maybe turn off your always display and see if that helps.
Thank you for your Feedback. We appreciate your contribution to the Beta community.
One UI Beta Team
Opens in new window