In this study overall confidence in AI generated health information was low. In my experience AI generated medical information references blogs and a variety of websites instead of medical studies making them less useful than they have the potential to be.

https://jamanetwork.com/journals/jama/fullarticle/2831965w