Jeffrey Walton <[email protected]> writes: > I think it is a good idea to utilize LLMs to detect problems, > especially latent problems that have slipped through the cracks over > the years. However, there are downsides to LLMs, and I would look > into fixing the current processes while using the LLMs as a complement > to existing practices.
Agreed. > First, code should not be merged into Master until the CI tests have > successfully run. Commits that don't pass the CI test don't pass > through the security gate. The new code stays in a development branch > or testing fork until they pass the CI tests. This is how many (all?) > organizations with a mature SDLC operate. Well, we can't really do that. Our CI is in a separate repository. I believe this was a decision made by Bruno and Simon because it worked well previously for them. I do not think this was a consideration at the time, but I have become a fan of it because it avoids the supply chain and permission issues that plague "organizations with a mature SDLC" (see recent happenings associated with TeamPCP). > Second, LLMs have at least three costs. First is the deskilling that > happens when relying on them. [1,2]. Relying too much on LLMs will > have a negative effect on the talent contributing to the project. I haven't read those particular references, but it is a concern I have. I think it is a personal responsibility not to offload all your thinking to an LLM, though. > Second is the power consumed powering the algorithms, and its effect > on renewable energy.[3,4] The environmental and societal cost seems to > be high. Third is the credit system being used to buy LLM time and the > associated costs.[5] There is no free ride. And if the algorithms are > going to be invoked at each commit, that could be a disaster at scale. > Are there enough tokens available to free software to make LLM > scanning a feasible procedural requirement? I don't necessarily disagree, but I don't think this argument is very effective. Everyone knows that eating red meat is terrible for the environment, but few change their behavior accordingly. Collin
