Hi Dawid, Thank you for sharing the thoughts. I am just one of the hundreds of contributors in Lucene community, I find myself nodding to all and couldn't help but to share my two cents from a contributor's point of view.
*On your first # Trust, role models, credibility* I totally agree that credibility is the process of earning continuous trust in community. As a relatively nascent Lucene contributor but a seasoned Java programmer, I have below principles for myself when engaging with community in GenAI era: 1. *Ownership*. I value every engagement with the community, and responsibility goes first. I won't ship something modified by AI without human-in-the-loop review, oversight, testing from first-principle and deep understanding of what I do. I take full accountability for the artifacts shipped, the coding paradigm shift doesn't change this pillar. 2. *Respect*. Today, people are burnt out with AI verbosity and long output or even AI slop. The scarcest resource is committers' and reviewers' attention. I want to value people's attention budget, ship succinct, well-thoughted artifacts. 3. *Hold the bar.* I think engineering is not just about making things work, it is intellectually thoughtful design, an art. I want to keep improving my engineering taste and code aesthetic as I gain more knowledge and experience on Lucene. The code is an image of myself, I want it to be green, sustainable, and designly beautiful. 4. *Don't outsource thinking*. I think AI is an amplifier, empowers us, boosts productivity, but it is never a substitute. I won't approach a problem without thinking deeply myself first. One honest example is that when I helped https://github.com/apache/lucene/issues/16044, I created a JMH benchmark and let AI help me write the boilerplate code for pure mmap read without MemorySegmentIndexInput, but there was a critical miss, I didn't align the memory address for posix_madvise, the results were misleading. Only after @jimczi did a second round of deep look did we land on the miss. I am super ashamed of this (sorry Jim for taking your time..), if I did it entirely by hand in the very beginning, I wouldn't have made such a mistake. Another example is when I developed gpt-oss.java ( https://github.com/amzn/gpt-oss.java), the MXFP4 quantization in the MLP layer, only after consulting Project Panama maintainers and iterating intensively did I arrive at an innovative solution (see https://github.com/amzn/gpt-oss.java/blob/main/src/main/java/ParallelMatMul.java#L241), AI was doing badly on this and didn't help me at all. These are all like wake-up calls. In addition, on what it takes to become committer, I find this discussion in Jun from Apache Calcite's community quite useful, for reference https://lists.apache.org/thread/3hlc59mtm76py8plky0w5bv7qpgfdy39. In short, beyond code contributions, maybe reviews, judgement shown, and overall community engagement matter more now. *On your # Person-to-person discussions* This is a good callout. I think we should utilize dev@ more actively. For example, I recently raised an issue about the legacy in-memory term hash code segment https://github.com/apache/lucene/issues/16574, I should have posted to dev@ as this is about asking for expert guidance, this would be a better place. *On your # Programming, the LLMs and the future of Lucene* I hold the same sentiment that sometimes AI takes away the fun part of programming from me... Previously, deep dive at pace was an enjoyable process, but now anyway, I try to focus on not outsourcing what to do and design decisions. Your bzip2 example is a perfect case where AI can empower and think outside the box, the code is complex but super performant, this proves AI can do some surprising good things diversely within a vertical, scoped, deterministic, verifiable scenario. I think we could let AI open the mind and explore, for example, I try to focus on indexing part, since in indexing flamegraph, there are two main flat tops, one in tokenization, one in term hash accumulating, maybe we can let AI remove the flat tops by exploring aggressively. But when things need to be grounded, humans should understand every piece line of code and place taste and judgment, I agree on that. Another example where I find AI is useful is performance analysis and data deep dives. In https://github.com/apache/lucene/issues/11608#issuecomment-5265355538, I did proper scaffolding, ran the end-to-end wikipedia benchmarks, then gave AI the data to help generate the viz graphs and insights. That's very straightforward and useful for human to view and tell. Best, Neoremind On Fri, Aug 28, 2026 at 11:50 PM Gus Heck <[email protected]> wrote: > > Sorry, didn't quite finish my sentence before inadvertently sending: The > line I was pointing out was " No use of the output can contravene any > provision as stated in the License." Which seems to backhaul all the > restrictions of the use of the model onto the usage of it's output (though > I am not a lawyer, I could be wrong) > > On Fri, Aug 28, 2026 at 11:41 AM Gus Heck <[email protected]> wrote: > >> Some thoughts for those lacking the energy to read the full FAQ I linked: >> >> LLM Generated Content is fundamentally not copyrightable (whether code, >> text or images), and it may reproduce copyrighted materials used in its >> training. Therefore, clear identification/demarcation of generated code, >> and the specific tool that generated it is probably important. (The link >> contains detailed guidance rules near the bottom of the section). Some >> generation tools would be ineligible. Some versions of deep seek don't >> appear to be compatible with the definition of open source (unless you >> train your own model from scratch) because the model license has >> restrictions prohibiting specific types of uses >> <https://github.com/deepseek-ai/DeepSeek-Coder/blob/2f9fd85927c669dae3c0fbb2d607274023af243e/LICENSE-MODEL>. >> That would violate point 6 in the definition link included in the ASF >> guidance <https://opensource.org/osd>. Note the line in the license that >> says: >> >> >> >> >> >> On Fri, Aug 28, 2026 at 11:03 AM Gus Heck <[email protected]> wrote: >> >>> Key consideration: ASF legal guidance on copyright and LLM generated >>> code: >>> https://www.apache.org/legal/generative-tooling.html#include-in-contributions >>> >>> On Fri, Aug 28, 2026 at 7:32 AM Dawid Weiss <[email protected]> >>> wrote: >>> >>>> Hi everyone, >>>> >>>> This email has a few thoughts - loosely connected - and I'm sorry >>>> if it reads like a stream-of-consciousness log but I've meant >>>> to write it for quite some time and today is the tipping point for me >>>> (hello, >>>> Malcolm Gladwell). >>>> >>>> So. As I'm sure you've noticed, more and more code is being generated >>>> these days. I have some thoughts I wanted to share about it. >>>> >>>> # Trust, role models, credibility. >>>> >>>> LLMs change how the open source model works - not just the code >>>> submitted but how we (humans) lack the means to evaluate or assess the >>>> skills of those who submit those patches. In the past, you could build >>>> your assessment of what a person knows/ can do by looking at what >>>> they contributed. Even if the code they produced was by and large of >>>> different flavors (think Mike McCandless vs Rob vs Uwe), you could >>>> tell these were virtuoso programmers and build your trust in their >>>> judgement and opinion. >>>> >>>> Can this model be applied these days? If not - how do we select future >>>> committers? >>>> >>>> # Person-to-person discussions >>>> >>>> I've also noticed that there's less and less of the person-to-person >>>> discussions on the dev list. I am sentimental, maybe, but I miss the >>>> oddball conversations and jokes we used to have in the past. Maybe >>>> it's just the project going graybeard or maybe it's the sign of >>>> changing times but I can't help but think that it's became less of a >>>> forum for programming enthusiasts exchanging (wild) ideas and learning >>>> from each other and more of a corporate fix-this-add-this repository >>>> with some infrastructure built on top. Or maybe it's just people >>>> departing to more attractive languages/ projects - this can't be ruled >>>> out too. I'm not judging here, I'm just observing the reality. >>>> >>>> # Programming, the LLMs and the future of Lucene >>>> >>>> I really like programming - I've always liked it. But LLMs are getting >>>> so good at understanding code (and improving it) that I find myself >>>> more and more frequently in awe. Take a look at my recent CI failure >>>> investigations (intentionally done with LLM help)... or this >>>> patch/issue I (well) wrote last night - it rewrites bzip2 >>>> decompression and the output (in java) is faster than my native >>>> bunzip2... >>>> >>>> https://github.com/apache/commons-compress/pull/811 >>>> >>>> It is this wild. >>>> >>>> I don't want to go into the debate of whether programmers are still >>>> needed at all - I don't have an established opinion on this... yet. >>>> The (human) learning process is definitely affected by those tools - >>>> not only they make one lazy but the sheer amount of code LLMs can >>>> produce is so overwhelming that it's hard to keep up (and after a >>>> while you just trigger auto-mode...). >>>> >>>> This said... I've been wondering if - in the spirit of oddball and >>>> crazy things we used to do - we shouldn't create a branch and let LLMs >>>> rip through whatever they like (collectively) as long as they keep the >>>> tests, code and output right. There's a saying that LLMs enable >>>> "everything you ever wanted to do but thought too complex". If you >>>> take a look at that bzip2 patch, it certainly feels this way for me. >>>> >>>> So... maybe we (as people) should brainstorm some wild, hacky ideas on >>>> the mailing list and then have them implemented just to see what they >>>> outcome is like? Wouldn't it be fun? :) >>>> >>>> Looking forward to hearing from you. >>>> >>>> Dawid >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>>> >>> >>> -- >>> http://www.needhamsoftware.com (work) >>> https://a.co/d/b2sZLD9 (my fantasy fiction book) >>> >> >> >> -- >> http://www.needhamsoftware.com (work) >> https://a.co/d/b2sZLD9 (my fantasy fiction book) >> > > > -- > http://www.needhamsoftware.com (work) > https://a.co/d/b2sZLD9 (my fantasy fiction book) >
