Hello Adolf, > On 26 Jan 2026, at 19:07, Adolf Belka <[email protected]> wrote: > > Hi Michael & Stefan, > > On 26/01/2026 18:23, Michael Tremer wrote: >> Hello Stefan, >> Thanks for looking into this. >> I would suggest that for the cleanup project, it would be best to keep two >> versions of rust-syn. Obviously this will inflate the number of packages for >> now, but actually starting to update and therefore potentially add more >> dependencies does not sounds wise to me. We would mix up too many changes >> into one which is never good. > > I am pretty certain from my work on the python update, which required rust > updates, that you will find that you will need two different versions of > rust-syn as one rust module will require an older version and another rust > module will require a newer version and the two won't overlap. In my work > there were even some rust modules where I ended up with the latest version > plus two older versions being required.
I hope that at some point someone will be able to explain to me what the benefit is to ship an older version of a crate that will have some bugs that have been fixed in the newer version. And if this is regarding LTS or breaking changes, other libraries totally manage this... >> Please start a list with all those packages that we would have to have a >> look at once the cleanup has been completed and we will start with updating >> them. That way we should be able keep this cleaner and hopefully won’t >> introduce too many new dependencies. >> In the end, I guess we will have to run this cleanup more than just this >> time, because we never know which package has dropped any dependencies. This >> is however rather unlikely. >> Best, >> -Michael >>> On 26 Jan 2026, at 15:31, Stefan Schantl <[email protected]> wrote: >>> >>> Hello list it's me again, >>> >>> the build process now reached python-cryptography which requires rust- >>> asn1, which requires rust-ans1_derive, which did not build because of a >>> to new version of rust-syn. >>> >>> rust-asn1_derive (0.12.2) >>> [ 1 ][ FAIL ] >>> >>> make: Nothing to be done for 'download'. >>> make: Leaving directory '/home/ipfire-2.x/lfs' >>> make: Entering directory '/usr/src/lfs' >>> toml-0.8.19.tar.gz checksum OK >>> make: Nothing to be done for 'install'. >>> make: Leaving directory '/usr/src/lfs' >>> Jän 26 15:18:59: Building rust-asn1_derive make: Entering directory >>> '/home/ipfire-2.x/lfs' >>> make: Nothing to be done for 'download'. >>> make: Leaving directory '/home/ipfire-2.x/lfs' >>> make: Entering directory '/usr/src/lfs' >>> asn1_derive-0.12.2.tar.gz checksum OK >>> ====================================== Installing asn1_derive- >>> 0.12.2 ... >>> Install started; saving file list to /usr/src/lsalr ... >>> cd /usr/src/asn1_derive-0.12.2 && if [ -f Cargo.toml.orig ]; then \ >>> rm -f Cargo.toml.orig; \ >>> fi; \ >>> >>> cd /usr/src/asn1_derive-0.12.2 && mkdir -p /usr/src/asn1_derive- >>> 0.12.2/.cargo && echo "${CARGO_CONFIG}" > /usr/src/asn1_derive- >>> 0.12.2/.cargo/config && rm -f Cargo.lock >>> cd /usr/src/asn1_derive-0.12.2 && CARGOPATH=/usr/src/asn1_derive- >>> 0.12.2/.cargo RUSTC_BOOTSTRAP=1 cargo --offline build --release -Z >>> avoid-dev-deps -j12 >>> warning: `/usr/src/asn1_derive-0.12.2/.cargo/config` is deprecated >>> in favor of `config.toml` >>> | >>> = help: if you need to support cargo 1.38 or earlier, you can >>> symlink `config` to `config.toml` >>> error: failed to select a version for the requirement `syn = >>> "^1.0.58"` >>> candidate versions found which didn't match: 2.0.114 >>> location searched: directory source `/usr/share/cargo/registry` >>> (which is replacing registry `crates-io`) >>> required by package `asn1_derive v0.12.2 (/usr/src/asn1_derive- >>> 0.12.2)` >>> perhaps a crate was updated and forgotten to be re-vendored? >>> As a reminder, you're using offline mode (--offline) which can >>> sometimes cause surprising resolution failures, if this error is too >>> confusing you may wish to retry without `--offline`. >>> make: *** [rust-asn1_derive:78: /usr/src/log/asn1_derive-0.12.2] >>> Error 101 >>> make: Leaving directory '/usr/src/lfs' >>> >>> ERROR: Building rust-asn1_derive >>> [ FAIL ] >>> Check /home/ipfire-2.x/log_x86_64/_build.ipfire.log for errors if >>> applicable >>> [ FAIL ] >>> root@localhost:/home/ipfire-2.x# >>> >>> Currently there is an older version of the rust-syn packaged, which >>> would allow me to bypass this issue, but would violence the goal of >>> getting rid of unneccessary rust modules. >>> >>> Theoretically I also could update the asn1_derive crate to the latest >>> version but this may break building the next modules. >>> >>> May this could act as starting point for the python update, where all >>> the rust stuff also needs to be touched..... >>> >>> @Adolf, @Michael what do you think about that? >>> >>> Thanks in advance, >>> >>> -Stefan >>> >>> >>> >>>> Hello list, >>>> >>>> currently I'm working on cleaning up the rust packages. >>>> >>>> For these I disabled all rust modules in the make.sh file and perform >>>> a >>>> clean build as Michael suggested. >>>> >>>> At the moment I'm past the stage where "cbindgen" successfully has >>>> been >>>> build and have 103 rust modules (inlcluding there sub-dependencies) >>>> only for this one tool. >>>> >>>> An additional rust module is required to build suricata. This is >>>> because of patching the source code the required rust module is not >>>> part of their source tarball. >>>> >>>> This currently summs to 104 rust modules for the moment. >>>> >>>> I'm looking forward when python-cryptography kicks in its module >>>> whishes.... >>>> >>>> Best regards, >>>> >>>> -Stefan >>>>> Hello Adolf, >>>>> Hello Michael, >>>>> >>>>> I would give the rust cleanup a try in the next few days. >>>>> >>>>> Adolf may I can ask you to put your current state of the python >>>>> update >>>>> into a git repositry? > > As my last work from November last year was not completed and is based on an > older status now and also on python3.13 vs the current 3.14, I think it makes > more sense that once Stefan has completed the clean up of the rust modules, I > then take that as the starting point and go through all my changes as done > before until I get to the problem I experience with python-cryptography not > being able to find any of the rust modules required by python-maturin. That > has always been the point where I got stuck. Did you try to update python-cryptography first without touching Python and after that try the Python update? > At that time I will then put that git branch I am working on into my personal > IPFire git repo so that the two of you can look at it to see what I am doing > wrong at that stage. > > That way, I can still contribute with all the update steps that I can do but > hand it over when it gets to the step that has consistently beat me. > > Regards, > > Adolf. > > >>>>> >>>>> Thanks in advance, >>>>> >>>>> -Stefan >>>>> >>>>>> Hello Adolf, >>>>>> >>>>>>> On 23 Jan 2026, at 11:06, Adolf Belka <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>> Hi Michael, >>>>>>> >>>>>>> On 23/01/2026 11:31, Michael Tremer wrote: >>>>>>>> Hello Stefan, >>>>>>>> Hello list, >>>>>>>> Thank you for looking at this. Of course it is very important >>>>>>>> that we are able to stay on the latest version of Suricata. >>>>>>>> I have merged your monster of a patch so that we can move on >>>>>>>> for >>>>>>>> now, but I have a couple of bigger questions that we all >>>>>>>> should >>>>>>>> have a look at: >>>>>>>> Adolf has in the past spent a lot of time on updating Rust. >>>>>>>> This >>>>>>>> is all tapping into Python - or rather python-cryptography - >>>>>>>> having some Rust code that has further dependencies. In >>>>>>>> essence, >>>>>>>> it has been a huge headache to update this. Maybe Adolf even >>>>>>>> has >>>>>>>> some other words for this all. >>>>>>> >>>>>>> My words on this are that I have now tried multiple times to >>>>>>> get >>>>>>> a >>>>>>> new python update built. Each time I have done it a bit >>>>>>> different >>>>>>> but the end result has been the same and that is that python- >>>>>>> cryptography (which requires rust modules to be built) ends up >>>>>>> requiring python-maturin that requires more rust modules but at >>>>>>> the >>>>>>> end of this the python-cryptography fails to find the built >>>>>>> rust >>>>>>> modules. >>>>>>> >>>>>>> I have been stuck at this last point so many times that I have >>>>>>> realised that I am finding lots of reasons not to go and work >>>>>>> on >>>>>>> the python update. >>>>>>> That is not a good position and also python has now moved from >>>>>>> 3.13 >>>>>>> to 3.14 so things are moving away from me. >>>>>>> >>>>>>> I have come to the conclusion that someone else, more capable >>>>>>> than >>>>>>> me needs to have a go at the python update, so I am giving up >>>>>>> on >>>>>>> it >>>>>>> but will continue working on other things. >>>>>> >>>>>> Hmm okay, you sound like you are giving up on this :) I know how >>>>>> many >>>>>> hours (we probably need to measure those in days or even weeks) >>>>>> you >>>>>> have spent on this though. >>>>>> >>>>>> Let’s pool resources together and finally get this done. >>>>>> Hopefully >>>>>> this will be a smoother ride as a combined effort. >>>>>> >>>>>>>> Just building cbindgen has required a further ~98 Rust crates >>>>>>>> to >>>>>>>> be packaged. Often we have the same crate in different >>>>>>>> versions >>>>>>>> because other crates have pinned a specific version. In >>>>>>>> total, >>>>>>>> we >>>>>>>> currently have ~790 packages in IPFire. Out of those, there >>>>>>>> are >>>>>>>> 202 packages in the rust-* namespace. That is pretty much a >>>>>>>> quarter of the distribution. Although not a lot in size, this >>>>>>>> is >>>>>>>> a considerable maintenance burden. >>>>>>>> ClamAV and Suricata have (recently?) started to bundle all >>>>>>>> their >>>>>>>> Rust dependencies with their release tarballs. Although this >>>>>>>> is >>>>>>>> not a good thing for many other reasons, it will move the >>>>>>>> onus >>>>>>>> onto the upstream projects to provide whatever they need. If >>>>>>>> their dependencies (and the dependencies of their >>>>>>>> dependencies) >>>>>>>> explode, this is not really our problem any more as well as >>>>>>>> any >>>>>>>> supply chain problems. Great - within reason. >>>>>>>> That leaves us with only very few packages that would >>>>>>>> actually >>>>>>>> require any external Rust crates (Suricata is even configured >>>>>>>> to >>>>>>>> *exclusively* use their bundled crates): cbindgen as a new >>>>>>>> thing, >>>>>>>> python-cryptography, anything else? We might actually only >>>>>>>> need >>>>>>>> a >>>>>>>> fraction of the Rust crates that we currently have as the >>>>>>>> only >>>>>>>> packages that may actually tap into our locally built >>>>>>>> repository >>>>>>>> are only those two. >>>>>>> >>>>>>> Unfortunately there is the addon oci-python-sdk that uses >>>>>>> python- >>>>>>> cryptography. >>>>>> >>>>>> python-cryptography was on my list. oci-python-sdk only uses Rust >>>>>> indirectly through python-cryptography, right? >>>>>> >>>>>>>> Is anyone happy to give this all a try and cleanup any old >>>>>>>> Rust >>>>>>>> deps? That way, I hope we will have a much smoother ride >>>>>>>> moving >>>>>>>> forward with a Python update. >>>>>>> >>>>>>> I can take the current status, before Stefan's patches, and see >>>>>>> how >>>>>>> many existing rust modules can be removed. Anything that can be >>>>>>> removed is a step forward. >>>>>> >>>>>> Yes, I think we should try to shrink what we have now if that is >>>>>> possible at all. As most packages are bundling all Rust deps, >>>>>> there >>>>>> should be some we won’t need any more in the system. >>>>>> >>>>>> Then, we hopefully have much less to update/worry about in any >>>>>> other >>>>>> way when we start touching python-cryptography. >>>>>> >>>>>> So who is volunteering to do this? Commenting out all Rust >>>>>> packages, >>>>>> then build python-cryptography which will fail as it requires >>>>>> some >>>>>> Rust crates. Those will be there so they will only have to be >>>>>> commented in again. Once the package builds, we should then have >>>>>> a >>>>>> couple of packages still commented that we can drop. >>>>>> >>>>>>> I think a problem moving forward is that more python modules >>>>>>> are >>>>>>> ending up being a combination of python and rust as the >>>>>>> cryptography and maturin modules have already done. I have also >>>>>>> seen a lot of rust modules covering the same stuff as covered >>>>>>> by >>>>>>> python modules. So the future I think looks like it will >>>>>>> continue >>>>>>> to be very frustrating. >>>>>> >>>>>> Yes it does, but we will have to find a way whether we want it or >>>>>> not. >>>>>> >>>>>> -Michael >>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Adolf. >>>>>>> >>>>>>> >>>>>>>> All the best, >>>>>>>> -Michael >>>>>>>>> On 22 Jan 2026, at 17:38, Stefan Schantl >>>>>>>>> <[email protected]> wrote: >>>>>>>>> >>>>>>>>> Hello list followers, >>>>>>>>> >>>>>>>>> I'm currently updating rust and affected modules. >>>>>>>>> >>>>>>>>> This happends mainly because I'm trying to fix the >>>>>>>>> "suricata >>>>>>>>> cache >>>>>>>>> grows infinite" problem, which a lot of people are >>>>>>>>> affected. >>>>>>>>> >>>>>>>>> To archive this, I ported the patches from suricata main >>>>>>>>> development >>>>>>>>> branch to our used suricata version (8.0.3). >>>>>>>>> >>>>>>>>> To perform a full build, a new tool called cbindgen - which >>>>>>>>> is >>>>>>>>> a rust >>>>>>>>> to c bindings generator, is required. >>>>>>>>> >>>>>>>>> Sadly this tool is also written in rust and requires some >>>>>>>>> new >>>>>>>>> dependencies and a more up to date rust compiler. >>>>>>>>> >>>>>>>>> I hope to send a patchset for all this very soon to the >>>>>>>>> mailing >>>>>>>>> list. >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> >>>>>>>>> -Stefan
