Yea, I just did when I looked at it again actually. I think it was the W3 cache plugin that I had installed. When I deactivated that, deleted the pages and then resynced the store, only one page was created. Thanks for looking!
It was weird because nothing was working wrong, but there were just a lot of different store pages.