Hi all Great topic of conversation. Instats (instats.org) has done a significant amount of work on how to use AI responsibly, meaning both ethically and accurately in terms of outputs (so reproducibly as well). Consistent with Jacob's point, here are some of our findings:
1) Unsurprisingly, better models that offer more computing resources tend to perform better (e.g., ChatGPT 5 Pro versus their fast model ChatGPT 5 Instant), but; 2) There is significant variation in different models in terms of what they're good at (e.g., Gemini 2.5 Pro is a better writer than the comparable ChatGPT 5 Thinking, and the comparable Claude Opus 4.1 tends to be the best and most critical coder); 3) No matter which model you use and no matter how many computing resources you're able to recruit, great prompting will net the best results. This is a crucial point. Good prompts are best drafted in conversation with Gen AI. You can give it your context and tell it to help you write the prompt. This tends to net the best results. Make sure to tell it that you plan to include chapters from software user manuals or R package documentation in the prompt—and attach that documentation to the prompt. When the prompt is ready, you can use it to tackle your problem. Give only relevant and precise information in the prompt to avoid confusing the model and increasing its output variation too much. Hope this helps. For more info check out this free introduction to using AI for research workflows: https://instats.org/seminar/ai-tools-for-researchers-free-seminar-1 Best wishes Michael Zyphur Professor and Director Instats | instats.org Follow Instats: Bluesky <https://bsky.app/profile/instats.bsky.social> Linkedin <https://www.linkedin.com/company/81837504/admin/dashboard/> Facebook <https://www.facebook.com/InstatsTraining/> On Thu, 18 Sept 2025 at 17:52, Klaus Schliep <[email protected]> wrote: > We should make more effort to encourage students to ask questions on a > list like here! > > First, if I get several questions about a function it is often a sign to > improve the man pages. This feedback is very helpful for developers. And in > the best case it start new collaborations. > Also there are usually several ways to do an analysis. So if you get lucky > that several people engage you will see different perspectives on your > problem. > > Cheers, > Klaus > > > > Am 08.09.2025 um 16:55 schrieb Jacob Berv < > [email protected]>: > > > > I think your experience is pretty common, but I would add that the > latest frontier models are much smarter than they were even one year ago. I > think it’s worth trying again if you have acess. 'GPT5-Thinking’ (on the > $20/month plan) is incredibly good, and apparently GPT5-Pro is another > significant leap above it, but very hard to access at $200/month). > > > > The issue of being outdated/biased in the training data, as Brian > suggested, is also getting less severe, as these models can now go online > to find recent information (and will do so in an active way if you direct > them to). > > > > Of course I dont think we should be telling people what to do. I’m not > an evangelist for these things other than to share my experiences, which > have been overall positive, given enough practice. I do find that you need > to invest some time in learning how to talk to these models to get good > results, but I’m convinced that we are in a very good position to get > maximum utility out of them, since academics are perhaps accidentally very > good at meta-science things, like thinking about how to ask a question in > the best way. > > > > But you are right, if you don’t know what the ’shape of bullshit’ looks > like, then they can be dangerous and misleading, and especially so for a > junior person. > > > > Jake > > > >> On Sep 8, 2025, at 9:48 AM, Daniel Caetano <[email protected]> > wrote: > >> > >> Hello Jacob, > >> > >> I do not know the main reasons for the decrease in the conversations in > this list. About the use of LLM for searching information, it is getting > more and more impossible to avoid it (especially with Google feeding these > responses right at the top). However, I had too many cases of wrong answers > to convince me that we are not there yet with these tools. I got code > wrong, math wrong, and statistical concepts that were imprecise when > reading these results. > >> > >> The interesting thing is that the best answers these LLMs can produce > is when they can source information from places like Stack overflow and > this list serve! So, it might happen in the future that we will simply > reverse back to "the old ways". Right now, these LLMs are not really > helping me that much. > >> > >> Cheers, > >> > >> Daniel Caetano > >> > >>> On 9/8/25 06:00, [email protected] wrote: > >>> Send R-sig-phylo mailing list submissions to > >>> [email protected] > >>> > >>> To subscribe or unsubscribe via the World Wide Web, visit > >>> https://stat.ethz.ch/mailman/listinfo/r-sig-phylo > >>> or, via email, send a message with subject or body 'help' to > >>> [email protected] > >>> > >>> You can reach the person managing the list at > >>> [email protected] > >>> > >>> When replying, please edit your Subject line so it is more specific > >>> than "Re: Contents of R-sig-phylo digest..." > >>> > >>> > >>> Today's Topics: > >>> > >>> 1. Re: What kinds of posts is this list needed for in the age > >>> of AI? (Jacob Berv) > >>> > >>> ---------------------------------------------------------------------- > >>> > >>> Message: 1 > >>> Date: Sun, 7 Sep 2025 19:41:19 -0400 > >>> From: Jacob Berv <[email protected]> > >>> To: Hilmar Lapp <[email protected]> > >>> Cc: R Sig Phylo Listserv <[email protected]> > >>> Subject: Re: [R-sig-phylo] What kinds of posts is this list needed > >>> for in the age of AI? > >>> Message-ID: <[email protected]> > >>> Content-Type: text/plain; charset="utf-8" > >>> > >>> This is an interesting take. I wouldn’t be surprised if it accounts > for a drop off in most questions up to a pretty advanced level. I know that > my personal use of forums like stack overflow has gone to zero, but before > November 2023 was a daily visit. Even google for search is eroding, at > least for me personally. The frontier models already know the best > strategies for using search engines to find the most relevant information. > I have also had good luck in using LLMs to assist in building relatively > complex phylogenetic analysis workflows. Maybe this itself is worth some > discussion on here. > >>> > >>> > >>>> On Sep 2, 2025, at 5:57 PM, Hilmar Lapp <[email protected]> > wrote: > >>>> > >>>> (Re-titling, but it’s a follow-up to my previous post) > >>>> > >>>> Incidentally, I notice that the most recent Answer post to Evoldir is > from 2022 (https://evol.mcmaster.ca/brian/Answers.html). > >>>> > >>>> There’s no reason to speculate that people who ask questions have > somehow become rude and do not post a summary of answers anymore, so the > most likely (and arguably Occam’s razor) explanation seems to be that it’s > the asking of questions that has faded away on Evoldir as well. Indeed, the > “Other” category archive (https://evol.mcmaster.ca/brian/Other.html) > bears this out, at least for the last 4 months that the archive extends to. > >>>> > >>>> In that context, the asking of questions has persisted here for a > little longer, but in principle has taken the same trajectory. > >>>> > >>>> This decline in posted questions obviously coincides with the advent > of increasingly powerful LLMs. Correlation doesn’t imply causation, and I > don’t know whether this is what trainees are using now, but I know it’s > what I would try first. As a random experiment, I tried one of the more > recent questions that generated a number of responses here, on > model-averaging corHMM models. > >>>> > >>>> Original list post: > https://www.mail-archive.com/[email protected]/msg06023.html > >>>> My query to ChatGPT (free version): > https://chatgpt.com/share/68b76601-2a80-8001-98ad-7ef4417db345 > >>>> > >>>> You all judge the answer. > >>>> > >>>> Anyone interested in examining this with a little more rigor? > >>>> > >>>> -hilmar > >>>> > >>>>> On Aug 24, 2025, at 4:44 PM, Hilmar Lapp via R-sig-phylo < > [email protected]> wrote: > >>>>> > >>>>> Apologies for chiming in a little late here. I have to say I am (and > have been for a while) in favor of moving the course announcements out of > this mailing list into a channel of its own. The main reasons in my mind > include the following: > >>>>> > >>>>> - Having a separate channel for course announcements seems to be the > common mechanism for distribution lists with a wide reach (case in point > for our field is evoldir). This seems to work well – people interested in > receiving these announcements opt in, everybody else doesn’t need to take > (and validate, check, monitor etc) specific action. > >>>>> > >>>>> - Science has become and will continue to become more > interdisciplinary, more so than ever in the age of data deluges and AI/ML. > This can make it difficult to draw a good line between courses that are and > those that aren’t pertinent to this community. And across this community, > we’d probably choose different lines. > >>>>> > >>>>> - It’s a little hard to say right now whether the diminishing > frequency of on-topic Q&A-style threads is a reaction to course > announcements taking over more and more and arguably most of the traffic, > or an independent trend reflecting something else. In the past, we have > used frequency of posts as evidence supporting the list’s value and > usefulness. I still consider this very valuable information to have – if > this kind of forum is becoming or is much less useful than it was in the > past, we should be able to see this from the list traffic, ideally > unencumbered by confounding factors. Perhaps chatbots can now utilize the > information from the list archives (which are public) well enough that most > questions get answered this way rather than reaching the list as a post. > >>>>> > >>>>> -hilmar > >>>>> > >>>> -- > >>>> Hilmar Lapp -:- ORCID:0000-0001-9107-0714 < > https://orcid.org/0000-0001-9107-0714> -:- GitHub:hlapp < > https://github.com/hlapp> > >>>> > >>>> [[alternative HTML version deleted]] > >>>> > >>>> _______________________________________________ > >>>> R-sig-phylo mailing list - [email protected] > >>>> https://stat.ethz.ch/mailman/listinfo/r-sig-phylo > >>>> Searchable archive at > http://www.mail-archive.com/[email protected]/ > >>> > >>> > >>> > >>> ------------------------------ > >>> > >>> Subject: Digest Footer > >>> > >>> _______________________________________________ > >>> R-sig-phylo mailing list > >>> [email protected] > >>> https://stat.ethz.ch/mailman/listinfo/r-sig-phylo > >>> > >>> > >>> ------------------------------ > >>> > >>> End of R-sig-phylo Digest, Vol 209, Issue 2 > >>> ******************************************* > >> > >> _______________________________________________ > >> R-sig-phylo mailing list - [email protected] > >> https://stat.ethz.ch/mailman/listinfo/r-sig-phylo > >> Searchable archive at > http://www.mail-archive.com/[email protected]/ > > > > _______________________________________________ > > R-sig-phylo mailing list - [email protected] > > https://stat.ethz.ch/mailman/listinfo/r-sig-phylo > > Searchable archive at > http://www.mail-archive.com/[email protected]/ > > _______________________________________________ > R-sig-phylo mailing list - [email protected] > https://stat.ethz.ch/mailman/listinfo/r-sig-phylo > Searchable archive at > http://www.mail-archive.com/[email protected]/ > [[alternative HTML version deleted]] _______________________________________________ R-sig-phylo mailing list - [email protected] https://stat.ethz.ch/mailman/listinfo/r-sig-phylo Searchable archive at http://www.mail-archive.com/[email protected]/
