Re: NSS and gskit are getting the axe
FWIW, I’m getting someone to make you an account on a system specifically for developing OSS stuff. They should send you an email soon. I’m also in contact with with someone from IBM about offering CI. They have a bunch of projects they’d like to offer resources to. They mention wanting to contact Daniel, but I’m getting them to post to the list here first. > On Jul 19, 2023, at 7:24 AM, Patrick Monnerat wrote: > > If you think you can do it, please act! As I said before, I personally gave > up trying to adapt the test environment after a long investigation and > building up a CI from scratch is out of my skills. > -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: NSS and gskit are getting the axe
On 7/19/23 03:26, Calvin Buckley wrote: Lack of easily available CI environments for i is by far the biggest pain point. I’ve been poking the IBM people I know to improve the situation. (The annoying thing it's harder to have a disposable environment, for reproducibility/security's sake. Even "just a box" is better than the current status though though.) WRT what Timothe said, I don’t think native Perl is useful (it’s highly unmaintained AFAIK, and platform differences will probably make porting the test runner annoying) , but you could use the AIX Perl to run tests against the ILE curl. But even then, once you have a CI box, you can at least run compile smoke tests. If you think you can do it, please act! As I said before, I personally gave up trying to adapt the test environment after a long investigation and building up a CI from scratch is out of my skills. There are many points you should know though: - PASE perl usage is problematic: see one of my previous mail in this thread. - The curl cli tool is not ported to ILE. The main reasons are: 1) EBCDIC (could be a QADRT program, yes), 2) Too many options to be supported in a *CMD (the primary goal was not a qshell usage). - gskit uses a certificate store rather than PEM files, thus specifying crypto objects to it differs from other backends. standard TLS test cases won't then be compatible. - A libtest makefile that compiles C test programs exists from the very early days of the OS400 port, but is now disabled (and unmaintained) for years. The idea behind it was to have what you call "smoke tests" and a first step to the (later abandoned) test environment implementation. - Remember we are currently targetting gskit, not only OS400 and/or compilation: as this backend is not available for other platforms, we cannot rely on cross test results for it. -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: NSS and gskit are getting the axe
Lack of easily available CI environments for i is by far the biggest pain point. I’ve been poking the IBM people I know to improve the situation. (The annoying thing it's harder to have a disposable environment, for reproducibility/security's sake. Even "just a box" is better than the current status though though.) WRT what Timothe said, I don’t think native Perl is useful (it’s highly unmaintained AFAIK, and platform differences will probably make porting the test runner annoying) , but you could use the AIX Perl to run tests against the ILE curl. But even then, once you have a CI box, you can at least run compile smoke tests. > On Jul 18, 2023, at 9:39 PM, Patrick Monnerat via curl-library > wrote: >> >>> You did not mention a CI. >> >> I do now. In 2023, not testing code regularly in CI is just lame. > This is probably true nowadays in our open-source word. I've never seen such > a practice or a standardized way of doing it on the OS400 world. I have to > admit that for 7 years, I'm not as involved in it as I was before! -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: NSS and gskit are getting the axe
On 7/19/23 01:11, Daniel Stenberg wrote: On Wed, 19 Jul 2023, Patrick Monnerat wrote: There's no CI for OS400 because 1) tests do no run on this platform; 2) is there an OS400 CI environment available ? I don't know. I don't see why there couldn't be? From replies from others (many thanks to them!), it seems it is feasible but is completely out of my skills, I'm afraid. For 16 years, the OS400 port has been tested manually to all users' satisfaction. I don't think that is true. Just about every release is shipped with broken os400/gskit builds because nobody tests curl+gskit in CI and not from git either. The fact we've had a bad situation for 16 years is not a very good motivation to keep it. This has occurred for 7 years only. And these are primarily compilation problems, not related to gskit. Back when we added gskit support we didn't test a single TLS backend in CI builds. Now we test all TLS backends except gskit. We would not dream of accepting a new TLS backend without it. If you have decided gskit cannot be saved from deprecation, please write it explicitly and avoid letting some "hope" on it: from your Jan 2 mail https://curl.se/mail/lib-2023-01/0003.html "If we get the gskit support in shape in time before August 2023, I think that is a working argument to *not* deprecate it then". I don't see it "in shape" now. I have repeatedly said since, many times, that gskit is up for deletion and yet not a single soul has spoken up about it. I don't sense a strong desire to keep it around. Well, the definition of "in shape" is missing! My feelings back to January was mainly TLS 1.3 and its crypto algorithms. I do have now such a version that compiles, but still untested regarding its functionalities. You did not mention a CI. I do now. In 2023, not testing code regularly in CI is just lame. This is probably true nowadays in our open-source word. I've never seen such a practice or a standardized way of doing it on the OS400 world. I have to admit that for 7 years, I'm not as involved in it as I was before! IMO, I now have done enough developments that are not accepted, or even ignored. You have done a lot, more than anyone I know, for OS400 and gskit in curl. This is not a dis against you or any other individual. It just happens that this area is *clearly* not valued very much by the community that uses it so it has always remained in this weird half-assed state. Struggling. Yes, as already mentioned in the January thread, the OS400 world is another planet where there are only users :-( I think we as a community has a responsibility to gradually enhance and improve curl and over time trim it and cut out code and parts of the project that can't keep up with this. Sure, providing we can reasonably use an infrastructure that allows it. In the OS400 context, the infrastructure is not reasonable for what we need and making it up by ourselves (whoever it is) is a project by itself. I think it can be saved. I just don't think the OS400 peeps are up for it. I see no evidence of such attempts - other than from you Patrick. I value your efforts, but I think it is rather lame that you as an individual should save the port for a commercial operating system that virtually not a single private person uses and everyone pays a lot of money for, but the companies themselves don't care for curl on their platform. But users do. And I know some other commercial company we do care about, that very frequently makes mistakes around our project and the communication around it! That's even worse :-( In my mind, what would save gskit in curl would the very least be someone setting up a CI for curl + gskit. Then refresh the code and support modern things like TLS 1.3. As said before, TLS 1.3 is ready. And again, before a CI we must have the test environment. But before August, this is just https://www.youtube.com/watch?v=nsOnWtCxLKI ! -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: NSS and gskit are getting the axe
On 7/19/23 00:52, Timothe Litt via curl-library wrote: With respect to CI: While I know little about OS400, Perldoc says that you *can *natively compile Perl with IBM's Visual Age compiler. Most CPAN modules are either pure Perl, portable XS (a bunch of C macros), or provide a pure Perl alternative. Whether the tests depend on any exceptions would have to be investigated. Perldoc also says: Perl on ILE There exists a port of Perl to the ILE environment. This port, however, is based quite an old release of Perl, Perl 5.00502 (August 1998). (As of July 2002 the latest release of Perl is 5.8.0, and even 5.6.1 has been out since April 2001.) If you need to run Perl on ILE, though, you may need this older port: http://www.cpan.org/ports/#os400 Note that any Perl release later than 5.00502 has not been ported to ILE. If you need to use Perl in the ILE environment, you may want to consider using Qp2RunPase() to call the PASE version of Perl. In qshell, the native (ILE) OS400 shell: $ > perl qsh: 001-0019 Error found searching for command perl. No such path or directory. $ It exists in the PASE environment however. PASE environment is approximately to OS400 what wine (https://www.winehq.org/) is to linux. In addition, the machine instruction language/codes are not the same. A bit like if you're emulating a processor on another one. PASE is based on ASCII, ILE on EBCDIC! As perl is essentially an interpreted language, you must have an interpreter in the target environment even if you run perl modules stored in their "precompiled" form. From a PASE perl, forking and scheduling an external program like our test system does will try to run an ILE binary as if it were a PASE program :-( I investigated the OS400 support of our test environment many years ago and finally gave up. -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: NSS and gskit are getting the axe
On Wed, 19 Jul 2023, Patrick Monnerat wrote: There's no CI for OS400 because 1) tests do no run on this platform; 2) is there an OS400 CI environment available ? I don't know. I don't see why there couldn't be? For 16 years, the OS400 port has been tested manually to all users' satisfaction. I don't think that is true. Just about every release is shipped with broken os400/gskit builds because nobody tests curl+gskit in CI and not from git either. The fact we've had a bad situation for 16 years is not a very good motivation to keep it. Back when we added gskit support we didn't test a single TLS backend in CI builds. Now we test all TLS backends except gskit. We would not dream of accepting a new TLS backend without it. If you have decided gskit cannot be saved from deprecation, please write it explicitly and avoid letting some "hope" on it: from your Jan 2 mail https://curl.se/mail/lib-2023-01/0003.html "If we get the gskit support in shape in time before August 2023, I think that is a working argument to *not* deprecate it then". I don't see it "in shape" now. I have repeatedly said since, many times, that gskit is up for deletion and yet not a single soul has spoken up about it. I don't sense a strong desire to keep it around. You did not mention a CI. I do now. In 2023, not testing code regularly in CI is just lame. IMO, I now have done enough developments that are not accepted, or even ignored. You have done a lot, more than anyone I know, for OS400 and gskit in curl. This is not a dis against you or any other individual. It just happens that this area is *clearly* not valued very much by the community that uses it so it has always remained in this weird half-assed state. Struggling. I think we as a community has a responsibility to gradually enhance and improve curl and over time trim it and cut out code and parts of the project that can't keep up with this. I think it can be saved. I just don't think the OS400 peeps are up for it. I see no evidence of such attempts - other than from you Patrick. I value your efforts, but I think it is rather lame that you as an individual should save the port for a commercial operating system that virtually not a single private person uses and everyone pays a lot of money for, but the companies themselves don't care for curl on their platform. In my mind, what would save gskit in curl would the very least be someone setting up a CI for curl + gskit. Then refresh the code and support modern things like TLS 1.3. -- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://curl.se/support.html -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: NSS and gskit are getting the axe
On Wed, Jul 19, 2023 at 12:30:02AM +0200, Patrick Monnerat via curl-library wrote: > No tests on OS400: would need perl among other features. These are available > under PASE which is an AIX emulation, but certainly not native OS400. If the tests won't even run there, then maybe you can convince our BDFL that a simple compile test would be good enough. That could be self-contained on OS490 and trivially feed to the autobuilds (https://curl.se/dev/builds.html) but that doesn't provide much visibility. It could also tie into another server that does the GitHub interfacing to bring the results into the GitHub PR interface, such as how Dagobert uses BuildBot to run Solaris builds (e.g. https://buildfarm.opencsw.org/buildbot/builders/curl-unthreaded-solaris10-sparc). You don't need to turn OS400 into a GitHub runner to accomplish this. Dan -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: NSS and gskit are getting the axe
With respect to CI: While I know little about OS400, Perldoc says that you *can *natively compile Perl with IBM's Visual Age compiler. Most CPAN modules are either pure Perl, portable XS (a bunch of C macros), or provide a pure Perl alternative. Whether the tests depend on any exceptions would have to be investigated. https://perldoc.perl.org/perlos400 GitHub actions supports self-hosted runners; I don't know what it would take to port the runner to OS400. https://github.com/actions/runner reports that the runner is mostly C# (95%), with a little JS (3.5%) and Shell (1.3%). If you do and supply a machine (VMs are fine), Actions can be told to schedule jobs on your runner. (And export suitable environment variables.) FWIW Timothe Litt ACM Distinguished Engineer -- This communication may not represent the ACM or my employer's views, if any, on the matters discussed. On 18-Jul-23 18:30, Patrick Monnerat via curl-library wrote: On 7/18/23 23:03, Daniel Stenberg wrote: On Tue, 18 Jul 2023, Patrick Monnerat via curl-library wrote: I have made some local changes to gskit, in hope to save it form deprecation (modernize, TLS 1.3, ALPN). It compiles fine, but unfortunately I can't test it as the certificate store access is denied on pub400. This is why I did not submit a PR for it yet. I'm a little tired of the fragile gskit situation. I'm willing to re-evaluate once we see a CI job that builds and tests curl with gskit. I want to (continuned to) raise the bar for what is an acceptable level for supported TLS libraries in curl. Because if nobody can make CI builds for gskit, then I don't think it is that important either. Yes, it is important because there are no alternative under OS400. This is not a gskit problem, rather an OS400 one. There's no CI for OS400 because 1) tests do no run on this platform; 2) is there an OS400 CI environment available ? No tests on OS400: would need perl among other features. These are available under PASE which is an AIX emulation, but certainly not native OS400. For 16 years, the OS400 port has been tested manually to all users' satisfaction. If you have decided gskit cannot be saved from deprecation, please write it explicitly and avoid letting some "hope" on it: from your Jan 2 mail https://curl.se/mail/lib-2023-01/0003.html "If we get the gskit support in shape in time before August 2023, I think that is a working argument to *not* deprecate it then". You did not mention a CI. IMO, I now have done enough developments that are not accepted, or even ignored. Patrick OpenPGP_signature Description: OpenPGP digital signature -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: NSS and gskit are getting the axe
On 7/18/23 23:03, Daniel Stenberg wrote: On Tue, 18 Jul 2023, Patrick Monnerat via curl-library wrote: I have made some local changes to gskit, in hope to save it form deprecation (modernize, TLS 1.3, ALPN). It compiles fine, but unfortunately I can't test it as the certificate store access is denied on pub400. This is why I did not submit a PR for it yet. I'm a little tired of the fragile gskit situation. I'm willing to re-evaluate once we see a CI job that builds and tests curl with gskit. I want to (continuned to) raise the bar for what is an acceptable level for supported TLS libraries in curl. Because if nobody can make CI builds for gskit, then I don't think it is that important either. Yes, it is important because there are no alternative under OS400. This is not a gskit problem, rather an OS400 one. There's no CI for OS400 because 1) tests do no run on this platform; 2) is there an OS400 CI environment available ? No tests on OS400: would need perl among other features. These are available under PASE which is an AIX emulation, but certainly not native OS400. For 16 years, the OS400 port has been tested manually to all users' satisfaction. If you have decided gskit cannot be saved from deprecation, please write it explicitly and avoid letting some "hope" on it: from your Jan 2 mail https://curl.se/mail/lib-2023-01/0003.html "If we get the gskit support in shape in time before August 2023, I think that is a working argument to *not* deprecate it then". You did not mention a CI. IMO, I now have done enough developments that are not accepted, or even ignored. Patrick -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: NSS and gskit are getting the axe
On Tue, 18 Jul 2023, Patrick Monnerat via curl-library wrote: I have made some local changes to gskit, in hope to save it form deprecation (modernize, TLS 1.3, ALPN). It compiles fine, but unfortunately I can't test it as the certificate store access is denied on pub400. This is why I did not submit a PR for it yet. I'm a little tired of the fragile gskit situation. I'm willing to re-evaluate once we see a CI job that builds and tests curl with gskit. I want to (continuned to) raise the bar for what is an acceptable level for supported TLS libraries in curl. Because if nobody can make CI builds for gskit, then I don't think it is that important either. -- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://curl.se/support.html -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: NSS and gskit are getting the axe
On 7/18/23 17:03, Calvin Buckley wrote: Do you just need RO access to the certificate store, or writes too? If so, I might be able to get you access to a less restricted system. Well, RO access would be a huge asset! It would allow testing with the standard certificate store, thus access to simple TLS servers. Testing with a client certificate requires to have write access not only to the certificate store but also to the DCM web gui and I can't imagine someone will give me this permission ! But I consider this last point as not important, since the real handling is in gskit itself, no recent change in curl for it and has been working for years. In fact, I also tried to use the DCM on pub400 with the idea to create my own certificate store, but the access is also restricted :-( Patrick -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: NSS and gskit are getting the axe
Do you just need RO access to the certificate store, or writes too? If so, I might be able to get you access to a less restricted system. > On Jul 18, 2023, at 11:50 AM, Patrick Monnerat via curl-library > wrote: > > > On 7/18/23 16:37, Calvin Buckley via curl-library wrote: >> I don’t have the time to become a curl maintainer, but I do have time >> every so often to check if GSKit and curl in general are working on i. >> >> Consider that a “don’t delete GSKit just yet.” NSS though? > > I have made some local changes to gskit, in hope to save it form deprecation > (modernize, TLS 1.3, ALPN). > > It compiles fine, but unfortunately I can't test it as the certificate store > access is denied on pub400. This is why I did not submit a PR for it yet. > > I you or someone else in the OS400 community volunteers for testing, I can > publish it. > > Else I'm still in hope I can get the certificate store unblocked... > > Patrick -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: NSS and gskit are getting the axe
On 7/18/23 16:37, Calvin Buckley via curl-library wrote: I don’t have the time to become a curl maintainer, but I do have time every so often to check if GSKit and curl in general are working on i. Consider that a “don’t delete GSKit just yet.” NSS though? I have made some local changes to gskit, in hope to save it form deprecation (modernize, TLS 1.3, ALPN). It compiles fine, but unfortunately I can't test it as the certificate store access is denied on pub400. This is why I did not submit a PR for it yet. I you or someone else in the OS400 community volunteers for testing, I can publish it. Else I'm still in hope I can get the certificate store unblocked... Patrick On Jul 18, 2023, at 8:41 AM, Daniel Stenberg via curl-library wrote: Hello, I have not received any loud complaints recently even after having repeatedly mentioned that this is going to happen in August 2023, so we now have two PRs pending: Remove NSS [1] and Remove gskit [2] Both backends are used sparesly in modern curl. In addition, the gskit one suffers badly from no tests/CI jobs. In total these two changes remove some 5,000 lines of code and docs. [1] = https://github.com/curl/curl/pull/11459 [2] = https://github.com/curl/curl/pull/11460 -- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://curl.se/support.html -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: NSS and gskit are getting the axe
I don’t have the time to become a curl maintainer, but I do have time every so often to check if GSKit and curl in general are working on i. Consider that a “don’t delete GSKit just yet.” NSS though? > On Jul 18, 2023, at 8:41 AM, Daniel Stenberg via curl-library > wrote: > > Hello, > > I have not received any loud complaints recently even after having repeatedly > mentioned that this is going to happen in August 2023, so we now have two PRs > pending: > > Remove NSS [1] and Remove gskit [2] > > Both backends are used sparesly in modern curl. In addition, the gskit one > suffers badly from no tests/CI jobs. > > In total these two changes remove some 5,000 lines of code and docs. > > [1] = https://github.com/curl/curl/pull/11459 > [2] = https://github.com/curl/curl/pull/11460 > > -- > > / daniel.haxx.se > | Commercial curl support up to 24x7 is available! > | Private help, bug fixes, support, ports, new features > | https://curl.se/support.html > -- > Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library > Etiquette: https://curl.se/mail/etiquette.html -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
NSS and gskit are getting the axe
Hello, I have not received any loud complaints recently even after having repeatedly mentioned that this is going to happen in August 2023, so we now have two PRs pending: Remove NSS [1] and Remove gskit [2] Both backends are used sparesly in modern curl. In addition, the gskit one suffers badly from no tests/CI jobs. In total these two changes remove some 5,000 lines of code and docs. [1] = https://github.com/curl/curl/pull/11459 [2] = https://github.com/curl/curl/pull/11460 -- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://curl.se/support.html -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
