It seems like llms.txt https://directory.llmstxt.cloud is emerging as a standard here. Especially nice for niche stuff that you can't expect the llm to be well trained on.