I had to join the betas for both Google Messages and for Carrier Services apps on Google Play. After that RCS started working, including on beta 2. RCS is still not completely reliable, it is definitely a work in progress.
I figured it out. Go to Google Messages -> Settings -> Advanced -> Phone Number. Make sure it is not blank. Mine was and once I typed in my number it worked.