My new contacts are properly set to default to my google account address book. This works fine for new contacts manually entered.
However, if I am downloading a vCard from email or text it instead is saved to my samsung account address book.
On the advice of Samsung I did a factory reset and restore. For the first vCard download I got a pop up that asked about the storage location. I selected google account address book and it worked. However, for all subsequent vCard downloads there was no pop up window and I am again back to them being stored in the samsung account address book.
I can't figure out how to fix this and apparently Samsung can't either.