There was an incident recently at the National Eating Disorder Association (NEDA) website, in which ChatGPT gave bad advice and was turned off:
https://www.cbsnews.com/news/eating-disorder-helpline-chatbot-disabled/ Based on my experience setting up the Bot at LENR-CANR.org, I expect some technical glitch caused the bot to malfunction and give out the wrong information for anorexia. I expect they uploaded the help-desk files at NEDA, and those files probably have the right information, but the bot did not find it. A lot can go wrong during setup. The data has to be structured in certain ways, and it has to be in the correct format. At LENR-CANR.org a glitch caused the Bot to lose half of the files. You have to understand and adjust various parameters such as the "AI temperature." Like all newly emerging software, ChatBots are unreliable and prone to glitches. There are no proper user manuals or installation instructions. A bot should be tested extensively before releasing it to the public. Especially before releasing it to patients who are worried and vulnerable. The larger question is: Should we be using Bots to hand out medical information and other potentially harmful information? I say yes, we should. It does not seem any different from posting web pages with medical advice, such as this one about anorexia at the Mayo Clinic: https://www.mayoclinic.org/diseases-conditions/anorexia-nervosa/symptoms-causes/syc-20353591 Then again, maybe we should not use Bots just yet. Two reasons: 1. They do not work well (as I said); 2. People have the mistaken impression they are intelligent, or even sentient. People may have more faith in what a Bot says than what is written in an ordinary web page at Mayo Clinic, or in a library book about diet and health. In a few years, people will be used to ChatBots and they will know that a Bot is just a convenient way to find and summarize what you want to know. They are not omniscient super-intelligent beings. Bots are extremely useful for many purposes. The other day I wanted to know how many watt-hours an AAA battery produces, and how big the electrodes are. The Bard Chatbot told me: A lithium AAA battery electrode is: - Diameter: 10.5 to 11.6 millimeters - Height: 4.8 to 5.6 millimeters - Positive electrode surface area: 0.1 to 0.2 square centimeters - Negative electrode surface area: 0.2 to 0.3 square centimeters Output is 2500 mAh or 4.5 Wh. It was faster and more convenient than scrolling through Google search finds. I assume it is right. I should check, though. If it was for a serious purpose, I would check.
