On 01/11/2025 15:33, Tim Bray via Silklist wrote:
On Nov 1, 2025 at 7:24:18 AM, Charles Haynes via Silklist < [email protected]> wrote:This is fascinating to me. It's clear that the intent is to treat AI as adversarial and to try to hinder it. What's not as clear to me is why.Because there is a massive backlash against GenAI among quite a wide range of people. A few reasons:
Well summarised!I think there's two things that have become intertwined, but can in principle be separated:
1) LLMs as a technology. Which is just one part of AI, and lots of other things (like: anything *else* done with a neural network) is being lumped together under "AI" and carried along with the flow, such as any company that does a bit of linear regression slapping "AI!!!!" all over it to hoover up investor megabucks, or anti-AI people hating on algorithmic generative art because they think it's wasting energy and exploiting other's work.
2) AI companies as an economic thing happening right now.The LLMs-as-a-technology are poorly understood by most people. Part of this is because the AI companies are trying to paint a picture that will guide investment towards them - they want everyone to think they're useful, and with more R&D will get much more useful! Part of this is because humans are not well prepared to judge the actual "intelligence" of a thing that seems very human - we make a lot of assumptions from our in-baked expectations. LLMs are eloquent speakers and can draw on a vast body of knowledge from their training sets, but are still not particularly *intelligent*; and the issue of hallucination is inherent to how they work (they "learn" by finding patterns and trends in their training set, rather than "remembering facts" per se, and answer questions by extrapolating those patterns - they *do not have* a thing like the human notion of facts) rather than a bug that they'll iron out with just a few more billions in R&D.
Lots of people are working hard on ways to make them useful, finding niches where their limitations aren't a problem, but the results so far still haven't been *spectacular*, and there doesn't seem a clear technical path to make more than incremental improvements at this point. Unless somebody has a true breakthrough - a new idea rather than development of the current tech - they're probably close to their peak utility, which means they're being used for some stuff - mainly on an experimental basis, but sometimes actually getting *some* benefit for real.
However, the AI companies are trying to pitch this as an early stage growth technology with vast room for improvement, for obvious reasons. And how much of the use they are getting right now will continue if the users have to pay the actual costs of running and training the things, rather than it being subsidised by investors at a massive loss, remains to be seen.
I'm a software engineer, and their utility in "vibe coding" is a matter of hot debate. Some claim it's great if you master the techniques of correctly prompting the things, others find the results are terrible, I've yet to see any really conclusive studies either way, but from what I hear the results tend to only be good if you're lucky.
I dunno. I think that if they truly had the claimed potential, we'd be seeing more of a benefit from it by now, as so many people are trying so hard to apply them. It's not like the early days of the Internet, where anybody who dipped their toes into it generally started to see cool opportunities, and the challenge was in explaining it to new people, and it just got better with network effects (until it started to rot). Instead, we have loads of people rushing in and pouring capital in, and the results just seem... underwhelming, nonetheless?
Maybe it could be made to work with a language with a highly power type system, like Idris, where one can ask the LLM to produce a program matching a specification and an automated system can check the program meets that specification (with the LLM needing to "show it's working", as languages like Idris require you to provide proofs that your program matches its specification) and make it fix it if it doesn't, in an unattended loop. That sounds great, but converting the specifications of what you want from the program into a *formal* specification amenable to automated type checking is a whole problem of its own, arguably not much simpler than writing a program in the first place. So we'll see.
-- Alaric Snell-Pym (M0KTN neé M7KIT) http://www.snell-pym.org.uk/alaric/
OpenPGP_signature.asc
Description: OpenPGP digital signature
-- Silklist mailing list [email protected] https://mailman.panix.com/listinfo.cgi/silklist
