I think Google has figured out where to obtain new raw data (in the sense that this is how people actually speak) to train Gemini on: Google Meet.

Gemini Transcripts are now fully integrated with all Google Workspace accounts (on by default). Businesses require documentation for liability claims, and AI is doing a better job than any human can ever do. Even hospitals are using it now to document team conversations.

This provides Gemini training data on how people actually talk (as opposed to YouTube videos they trained on) and on private data before it reaches the markets.

Terrible for privacy, but a genius move.