Hi Lauren, Thanks for attending the Future Audiences community call this week, and apologies that we ran out of time before I was able to answer your question about the ChatGPT plugin next steps – answering now :)
You wrote: “As for the Foundation’s ChatGPT plugin, I’m afraid I find it mostly unusable because it ignores everything after the first dozen paragraphs of all articles. That was listed as needing 3-4 days to fix on https://phabricator.wikimedia.org/T343932 a month ago. Do you know whether there are any plans to go ahead with that fix?” First: thank you for enabling the plugin and trying it out! We've just added a link to a survey that we're hoping will give us more feedback on whether/how the current plugin is meeting user needs. If you haven't already done so, I'd appreciate if you could take a few minutes to fill it out (you should see it in the footer of the plugin response). On your specific point about the 12-paragraph cutoff: yes, we recognize that this may lead to the plugin not finding relevant information in some cases (though from our qualitative coding of about 300 query responses in 6 languages, the plugin was able to return relevant information about 84% of the time[1]). However, as I tried to stress in my blog post,[2] this plugin is intended to be a quick experiment and not a fully-featured, permanently maintained product. We're tracking many possible optimizations, such as changes to the 12-paragraph cutoff, showing references, improving output quality, etc. But in order to go deep on optimization, we first need to get compelling evidence that we *should* invest in a product like this, on an external platform that we don't manage, longterm – because making that investment would require getting more resources (i.e., an actual feature development team, not just part-time R&D), which would be a nontrivial change to our annual plan. As the WMF staffer who would be making the case for that investment internally and to the community: I think while there are definitely more things we can use the plugin to learn, and it's always possible that usage of ChatGPT and the plugin may take off wildly, I don't personally feel comfortable making that recommendation at this moment. (All that said, we may be able to get some more ad hoc R&D resources to make some optimization tweaks as we try to learn more – that's what I'm currently aiming for, so stay tuned!) Please let me know if you have more questions or want to talk about plugin specifics, on-list, via email, or onwiki (the project is on Meta here: https://meta.wikimedia.org/wiki/Talk:Future_Audiences)! Best, Maryana 1. https://meta.wikimedia.org/wiki/Future_Audiences/Experiments:_conversational/generative_AI#Preliminary_results 2. https://diff.wikimedia.org/2023/07/13/exploring-paths-for-the-future-of-free-knowledge-new-wikipedia-chatgpt-plugin-leveraging-rich-media-social-apps-and-other-experiments/ _______________________________________________ Wikimedia-l mailing list -- wikimedia-l@lists.wikimedia.org, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l Public archives at https://lists.wikimedia.org/hyperkitty/list/wikimedia-l@lists.wikimedia.org/message/IPOVMFNRDSARHDFWWZJQWLIQ3FRU6CNW/ To unsubscribe send an email to wikimedia-l-le...@lists.wikimedia.org
