Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-07-06 Thread Marc Espie
On Thu, Jun 24, 2021 at 02:56:16PM -0600, Theo de Raadt wrote:
> > I think the easiest path here is to incorporate the new upstream into a
> > port, unless someone is familiar with zlib and can cherrypick out the
> > commit(s) that resolve the issue. (I didn't find zlib in ports already.)
> 
> That is completely impossible.  It must be in base.  There are 3 copies
> in base -- userland, kernel, and bootblocks.  They must be kept in sync.

This is not even true, there is a 4th copy in perl's library.



Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-25 Thread Stuart Henderson
On 2021-06-25, Theo Buehler  wrote:
> If we want to go the cherry-picking route, here's a diff that fixes the
> test.csv.gz test case provided in the linked issue.

No objection to this (it won't make a future sync much harder, there
are already many changes in openbsd-zlib compared to the more common
version), but there are plenty of ports that do specifically ask for
a newer version that we have hacked away the version check, there is
probably some reason they have decided to require the version they do
(though many are so old probably nobody remembers by now).

>From the ports side a larger update would be welcome. FWIW my earlier
update is here: https://marc.info/?l=openbsd-tech=132810606729160=2
(it wasn't just "dump the new code in", I did merge OpenBSD changes).

Hacking around missing functions has introduced some problems in
ports development.  And different codepaths taken mean we hit bugs
in upstream software that are not seen by others. mail/notmuch is
an example where we found problems
(https://marc.info/?l=openbsd-ports=158654770825909=2) and
fixed/worked around them because it has a good regression testsuite;
that's not all that common in other ports so I'll be surprised if
there are no other issues latent in the tree (I guess it took some
time before the problem with R prpmpting this discussion showed up).
But ports can't answer Mark's question about z_off64_t.

> the copy in gnu/usr.bin/cvs is even older...

Fortunately that one's unused.




Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-24 Thread Matt Dowle
On Thu, Jun 24, 2021 at 10:41 PM Sebastien Marie  wrote:

> On Thu, Jun 24, 2021 at 08:04:37PM -0600, Matt Dowle wrote:
> >
> > > It is NOT 16 years old.  You keep saying that.  There is a different
> > development
> > process involved here which has upsides and downsides and which I don't
> > expect
> > you will understand.
> >
> > That's right. I don't understand.
> > Could you explain it then, or point me to a document that explains what
> > your development process is?
> > Putting two and two together, it seems that it is 16 years plus a bunch
> of
> > cherry picked bug fixes backported over a very many years. If that's what
> > you do, whilst I understand that can make some sense to keep patching
> say 5
> > year old libraries, at some point it becomes too old and too risky.
> >
>
> I am not sure to understand why our zlib version (which might be
> called a maintained fork from 16 years old version) would be more
> risky than pushing a newer version just because 'it is newer'.
>

There are limits here: 'new' and 'old' need to be defined. I agree with you
that upgrading to bleeding edge (releases under say 1 year old) regularly
may not be a good idea. I think that may be what you have in mind by 'new'.
But, in this particular case, the 'newest' version of zlib is 4 years old.
And, there have been no bug fixes since that release, too. That's quite
different to 'just because it is newer'. A lot of eyes and usage has been
on that zlib version 1.2.11 in the last 4 years.

The longer you continue a maintained fork, the bigger the time lag between
the patches you are taking from later versions and backporting them to 16
years ago. The bigger the time lag the risker it is. Because the bug fixes
in later versions of zlib were made and tested (by zlib devs, and people
using those later versions) with those fixes in those later versions of
zlib, not 1.2.3 from 16 years ago. The act of backporting bug fixes to
earlier versions is not risk free. There are a lot of eyes and dependencies
using the zlib versions in the form that they have been released. I guess
there are not so many eyes and usage of your maintained fork. These reasons
are why I said your maintained fork, in this particular case of the
baseline being 16 years ago, is riskier. Not because I was naive to assume
the latest one is always better. I did not say that all your maintained
forks are risker. I'm just saying there's a limit and having a 16-year old
baseline for a maintained fork is beyond that limit, and therefore riskier.

We are not hostile to make changes, but at least please told us what
> should be changed/adjusted and why it is important for your
> use-case. And if it doesn't hurt us too, changes will be done: patches
> are accepted.
>

I already replied on that point in this thread to Theo.

Best, Matt


> Thanks.
> --
> Sebastien Marie
>


Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-24 Thread Sebastien Marie
On Thu, Jun 24, 2021 at 08:04:37PM -0600, Matt Dowle wrote:
> 
> > It is NOT 16 years old.  You keep saying that.  There is a different
> development
> process involved here which has upsides and downsides and which I don't
> expect
> you will understand.
> 
> That's right. I don't understand.
> Could you explain it then, or point me to a document that explains what
> your development process is?
> Putting two and two together, it seems that it is 16 years plus a bunch of
> cherry picked bug fixes backported over a very many years. If that's what
> you do, whilst I understand that can make some sense to keep patching say 5
> year old libraries, at some point it becomes too old and too risky.
> 

I am not sure to understand why our zlib version (which might be
called a maintained fork from 16 years old version) would be more
risky than pushing a newer version just because 'it is newer'.

We are not hostile to make changes, but at least please told us what
should be changed/adjusted and why it is important for your
use-case. And if it doesn't hurt us too, changes will be done: patches
are accepted.

Thanks.
-- 
Sebastien Marie



Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-24 Thread Theo Buehler
On Thu, Jun 24, 2021 at 11:09:05AM -0600, Matt Dowle wrote:
> Hi,
> 
> Is it intentional or is there any good reason that OpenBSD 6.9 released May
> 2021 uses a 16 year old version of zlib (v1.2.3; July 2005)?  The latest
> version v1.2.11 (Jan 2017) is 4 years old.
> 
> Background here: https://github.com/Rdatatable/data.table/pull/5049

If we want to go the cherry-picking route, here's a diff that fixes the
test.csv.gz test case provided in the linked issue.

All of test.data.table() now passes:

All 9062 tests (last 2163) in tests/tests.Rraw.bz2 completed ok in 57.9s 
elapsed (49.9s cpu)

I haven't done further testing as deflateBound() is not used in the base
system.

This backports zlib's deflate.c diff between 1.2.3 and 1.2.3.1.  It
adjusts the calculation of the bound by calculating the wrapper length
instead of trying to bound it. The result can now be larger than the
previous bound.  It modifies the two copies of deflate.c in /usr/src
where the change would apply cleanly.  Perl's version has the change and
the copy in gnu/usr.bin/cvs is even older...

https://github.com/madler/zlib/commit/b1c19ca6d82c98a8be6cd9cad7a9c5fa5e8e634e#diff-7e5fd0aa55941ed3e3c9c564a713b3841f20f7defc549277cdf94f6c90882af8

Index: lib/libz/deflate.c
===
RCS file: /cvs/src/lib/libz/deflate.c,v
retrieving revision 1.11
diff -u -p -r1.11 deflate.c
--- lib/libz/deflate.c  27 Oct 2009 23:59:31 -  1.11
+++ lib/libz/deflate.c  25 Jun 2021 03:03:40 -
@@ -479,33 +479,65 @@ int ZEXPORT deflateTune(strm, good_lengt
  * resulting from using fixed blocks instead of stored blocks, which deflate
  * can emit on compressed data for some combinations of the parameters.
  *
- * This function could be more sophisticated to provide closer upper bounds
- * for every combination of windowBits and memLevel, as well as wrap.
- * But even the conservative upper bound of about 14% expansion does not
- * seem onerous for output buffer allocation.
+ * This function could be more sophisticated to provide closer upper bounds for
+ * every combination of windowBits and memLevel.  But even the conservative
+ * upper bound of about 14% expansion does not seem onerous for output buffer
+ * allocation.
  */
 uLong ZEXPORT deflateBound(strm, sourceLen)
 z_streamp strm;
 uLong sourceLen;
 {
 deflate_state *s;
-uLong destLen;
+uLong complen, wraplen;
+Bytef *str;
 
-/* conservative upper bound */
-destLen = sourceLen +
-  ((sourceLen + 7) >> 3) + ((sourceLen + 63) >> 6) + 11;
+/* conservative upper bound for compressed data */
+complen = sourceLen +
+  ((sourceLen + 7) >> 3) + ((sourceLen + 63) >> 6) + 5;
 
-/* if can't get parameters, return conservative bound */
+/* if can't get parameters, return conservative bound plus zlib wrapper */
 if (strm == Z_NULL || strm->state == Z_NULL)
-return destLen;
+return complen + 6;
 
-/* if not default parameters, return conservative bound */
+/* compute wrapper length */
 s = strm->state;
+switch (s->wrap) {
+case 0: /* raw deflate */
+wraplen = 0;
+break;
+case 1: /* zlib wrapper */
+wraplen = 6 + (s->strstart ? 4 : 0);
+break;
+case 2: /* gzip wrapper */
+wraplen = 18;
+if (s->gzhead != NULL) {/* user-supplied gzip header */
+if (s->gzhead->extra != NULL)
+wraplen += 2 + s->gzhead->extra_len;
+str = s->gzhead->name;
+if (str != NULL)
+do {
+wraplen++;
+} while (*str++);
+str = s->gzhead->comment;
+if (str != NULL)
+do {
+wraplen++;
+} while (*str++);
+if (s->gzhead->hcrc)
+wraplen += 2;
+}
+break;
+default:/* for compiler happiness */
+wraplen = 6;
+}
+
+/* if not default parameters, return conservative bound */
 if (s->w_bits != 15 || s->hash_bits != 8 + 7)
-return destLen;
+return complen + wraplen;
 
 /* default settings: return tight bound for that case */
-return compressBound(sourceLen);
+return compressBound(sourceLen) - 6 + wraplen;
 }
 
 /* =
Index: sys/lib/libz/deflate.c
===
RCS file: /cvs/src/sys/lib/libz/deflate.c,v
retrieving revision 1.3
diff -u -p -r1.3 deflate.c
--- sys/lib/libz/deflate.c  14 Mar 2016 23:08:06 -  1.3
+++ sys/lib/libz/deflate.c  25 Jun 2021 03:04:49 -
@@ -479,33 +479,65 @@ int ZEXPORT deflateTune(strm, good_lengt
  * resulting from using fixed blocks instead of stored blocks, which 

Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-24 Thread Matt Dowle
> So feisty.

Seriously?

On Thu, Jun 24, 2021 at 8:33 PM Theo de Raadt  wrote:

> Matt Dowle  wrote:
>
> > That's right. I don't understand.
> > Could you explain it then, or point me to a document that explains what
> > your development process is?
> > Putting two and two together, it seems that it is 16 years plus a bunch
> of
> > cherry picked bug fixes backported over a very many years. If that's what
> > you do, whilst I understand that can make some sense to keep patching
> say 5
> > year old libraries, at some point it becomes too old and too risky.
>
> So feisty.
>


Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-24 Thread Theo de Raadt
Matt Dowle  wrote:

> That's right. I don't understand.
> Could you explain it then, or point me to a document that explains what
> your development process is?
> Putting two and two together, it seems that it is 16 years plus a bunch of
> cherry picked bug fixes backported over a very many years. If that's what
> you do, whilst I understand that can make some sense to keep patching say 5
> year old libraries, at some point it becomes too old and too risky.

So feisty.



Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-24 Thread Matt Dowle
> I don't know either.
> That is what I am asking.

I'm not going to spend more time investigating a bug fix in zlib made 15
years ago. If that's what your policy is, then we have provided plentiful
pointers for you to do so.

> Yes, you keep saying we should just throw the new code in, and you keep
not explaining what the actual problem is.

You keep paraphrasing me: I didn't say "throw the new code in".

You complained it was "pages and pages", but I have linked to what the
problem is and what the fix was. That is not the same as me "not explaining
what the problem is".

You keep asking me to spend even more time on this, and not recognizing
that we have already sunk time into this.

> It is NOT 16 years old.  You keep saying that.  There is a different
development
process involved here which has upsides and downsides and which I don't
expect
you will understand.

That's right. I don't understand.
Could you explain it then, or point me to a document that explains what
your development process is?
Putting two and two together, it seems that it is 16 years plus a bunch of
cherry picked bug fixes backported over a very many years. If that's what
you do, whilst I understand that can make some sense to keep patching say 5
year old libraries, at some point it becomes too old and too risky.

Matt

On Thu, Jun 24, 2021 at 7:27 PM Theo de Raadt  wrote:

> Matt Dowle  wrote:
>
> > Theo,
> >
> > > Instead, we got pages and pages that summarize to "must
> > update", and doesn't explain what the bug is or what the fix is.
> >
> > > If only we had an explanation of what is actually wrong and needs
> fixing
> >
> > We know it was this news item from 1.2.3.1 (released 16 August 2006)
> > https://zlib.net/ChangeLog.txt :
> > - Take into account wrapper variations in deflateBound()
> > but I don't know which commit that was.
>
> I don't know either.
>
> That is what I am asking.
>
> But you don't know and keep repeating:
>
> > Yes it would be appreciated if OpenBSD upgraded to zlib 1.2.11 which at
> 4 years old
> > seems reasonably old. Using a 16-year old version of a library such as
> zlib, however,
> > seems too old to me: at that age it's starting to become unreasonable to
> expect other
> > open-source maintainers such as myself to support.
>
> Yes, you keep saying we should just throw the new code in, and you keep
> not explaining what the actual problem is.
>
> It is NOT 16 years old.  You keep saying that.  There is a different
> development
> process involved here which has upsides and downsides and which I don't
> expect
> you will understand.
>


Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-24 Thread Theo de Raadt
Matt Dowle  wrote:

> Theo,
> 
> > Instead, we got pages and pages that summarize to "must
> update", and doesn't explain what the bug is or what the fix is.
> 
> > If only we had an explanation of what is actually wrong and needs fixing
> 
> We know it was this news item from 1.2.3.1 (released 16 August 2006)
> https://zlib.net/ChangeLog.txt :
> - Take into account wrapper variations in deflateBound()
> but I don't know which commit that was.

I don't know either.

That is what I am asking.

But you don't know and keep repeating:

> Yes it would be appreciated if OpenBSD upgraded to zlib 1.2.11 which at 4 
> years old
> seems reasonably old. Using a 16-year old version of a library such as zlib, 
> however,
> seems too old to me: at that age it's starting to become unreasonable to 
> expect other
> open-source maintainers such as myself to support.

Yes, you keep saying we should just throw the new code in, and you keep
not explaining what the actual problem is.

It is NOT 16 years old.  You keep saying that.  There is a different development
process involved here which has upsides and downsides and which I don't expect
you will understand.



Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-24 Thread Matt Dowle
Theo,

> Instead, we got pages and pages that summarize to "must
update", and doesn't explain what the bug is or what the fix is.

> If only we had an explanation of what is actually wrong and needs fixing

We know it was this news item from 1.2.3.1 (released 16 August 2006)
https://zlib.net/ChangeLog.txt :
- Take into account wrapper variations in deflateBound()
but I don't know which commit that was.

We have already worked around it on our side.

Yes it would be appreciated if OpenBSD upgraded to zlib 1.2.11 which at 4
years old seems reasonably old. Using a 16-year old version of a library
such as zlib, however, seems too old to me: at that age it's starting to
become unreasonable to expect other open-source maintainers such as myself
to support.

Best, Matt

On Thu, Jun 24, 2021 at 3:46 PM Theo de Raadt  wrote:

> Dave Voutila  wrote:
>
> > Theo de Raadt writes:
> >
> > > Dave Voutila  wrote:
> > >
> > >> Theo de Raadt writes:
> > >>
> > >> >> I think the easiest path here is to incorporate the new upstream
> into a
> > >> >> port, unless someone is familiar with zlib and can cherrypick out
> the
> > >> >> commit(s) that resolve the issue. (I didn't find zlib in ports
> already.)
> > >> >
> > >> > That is completely impossible.  It must be in base.  There are 3
> copies
> > >> > in base -- userland, kernel, and bootblocks.  They must be kept in
> sync.
> > >>
> > >> Not saying to replace what's in base, but have a different version in
> > >> ports available for ports. I was thinking along the lines of egcc or
> > >> eopenssl in that the port co-exists with base and ports that need them
> > >> need tweaking to use them.
> > >
> > > You've got to be kidding.  In what world does it help to require -I and
> > > -l lines all over the place, or else everything breaks.
> >
> > I'm in 100% agreement it sucks and it's something I believe is already
> > done for ports that require OpenSSL. /shrug. My thinking was instead of
> > having to test all of base across all archs with any potential fix, we
> > could isolate the change to maybe the R port if other R packages or
> > whatever have run into this.
> >
> > But I'm not volunteering to do either so I'll stop beating this horse
> > now before it never walks again.
>
> This is crazy.  It is probably a 2-3 line fix.  If only we had an
> explanation of what is actually wrong and needs fixing...
>


Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-24 Thread Stuart Henderson
On 2021-06-24, Dave Voutila  wrote:
> I'm in 100% agreement it sucks and it's something I believe is already
> done for ports that require OpenSSL.

Only for ports which require OpenSSL and don't require a library
which pulls in LibreSSL. For example we are now stuck on updating
Postfix (thanks to an openssl dev there who successfully pushed
for requiring newer api only in openssl for now), and I am
concerned about future Python versions since there is a redhat
person there pushing for openssl only.

Once you link against these things you can no longer link against
a library using the equivalent thing from base.




Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-24 Thread Theo de Raadt
Dave Voutila  wrote:

> Theo de Raadt writes:
> 
> > Dave Voutila  wrote:
> >
> >> Theo de Raadt writes:
> >>
> >> >> I think the easiest path here is to incorporate the new upstream into a
> >> >> port, unless someone is familiar with zlib and can cherrypick out the
> >> >> commit(s) that resolve the issue. (I didn't find zlib in ports already.)
> >> >
> >> > That is completely impossible.  It must be in base.  There are 3 copies
> >> > in base -- userland, kernel, and bootblocks.  They must be kept in sync.
> >>
> >> Not saying to replace what's in base, but have a different version in
> >> ports available for ports. I was thinking along the lines of egcc or
> >> eopenssl in that the port co-exists with base and ports that need them
> >> need tweaking to use them.
> >
> > You've got to be kidding.  In what world does it help to require -I and
> > -l lines all over the place, or else everything breaks.
> 
> I'm in 100% agreement it sucks and it's something I believe is already
> done for ports that require OpenSSL. /shrug. My thinking was instead of
> having to test all of base across all archs with any potential fix, we
> could isolate the change to maybe the R port if other R packages or
> whatever have run into this.
> 
> But I'm not volunteering to do either so I'll stop beating this horse
> now before it never walks again.

This is crazy.  It is probably a 2-3 line fix.  If only we had an
explanation of what is actually wrong and needs fixing...



Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-24 Thread Dave Voutila


Theo de Raadt writes:

> Dave Voutila  wrote:
>
>> Theo de Raadt writes:
>>
>> >> I think the easiest path here is to incorporate the new upstream into a
>> >> port, unless someone is familiar with zlib and can cherrypick out the
>> >> commit(s) that resolve the issue. (I didn't find zlib in ports already.)
>> >
>> > That is completely impossible.  It must be in base.  There are 3 copies
>> > in base -- userland, kernel, and bootblocks.  They must be kept in sync.
>>
>> Not saying to replace what's in base, but have a different version in
>> ports available for ports. I was thinking along the lines of egcc or
>> eopenssl in that the port co-exists with base and ports that need them
>> need tweaking to use them.
>
> You've got to be kidding.  In what world does it help to require -I and
> -l lines all over the place, or else everything breaks.

I'm in 100% agreement it sucks and it's something I believe is already
done for ports that require OpenSSL. /shrug. My thinking was instead of
having to test all of base across all archs with any potential fix, we
could isolate the change to maybe the R port if other R packages or
whatever have run into this.

But I'm not volunteering to do either so I'll stop beating this horse
now before it never walks again.

-dv



Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-24 Thread Stuart Henderson
On 2021-06-24, Theo de Raadt  wrote:
>> I think the easiest path here is to incorporate the new upstream into a
>> port, unless someone is familiar with zlib and can cherrypick out the
>> commit(s) that resolve the issue. (I didn't find zlib in ports already.)
>
> That is completely impossible.  It must be in base.  There are 3 copies

Yes, things always mess up when different versions of a shared library
with the same name and most of the same API exist in base and ports.
(This is an ongoing problem with libressl/openssl).  Doing this for
zlib won't go well.

> in base -- userland, kernel, and bootblocks.  They must be kept in sync.

There's one in perl as well, it's newer already.

> It gets updated when things matter.  I wasn't aware a change had happened
> which matters.  The thread makes me unenthusiastic, because I cannot tell
> what changed that matters.  Updating it is not a trivial effort, because
> every bootblock needs testing.  I've done it twice...

We had various problems in ports, but last time I looked at updating
zlib (some years ago) it got derailed into a discussion about largefile
stuff and I lost interest. We've been trying to hack around it as best
we can in ports. There are probably some cases where we didn't update
ports because of it, and probably some where we're using static linked
bundled copies (I bet chromium does that).

We definitely ran into things wanting the "transparent write" mode and
gzclose_r / gzclose_w functions (hacked around e.g. in notmuch).
Think I recall seeing some things want gzoffset that probably
blocked updating them. I had a quick look on codesearch.debian.net
to see if I could figure out what's used but there are bundled copies
all over the place so it's near impossible to identify them.




Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-24 Thread Theo de Raadt
Matt Dowle  wrote:

> Hi,
> 
> Is it intentional or is there any good reason that OpenBSD 6.9 released May
> 2021 uses a 16 year old version of zlib (v1.2.3; July 2005)?  The latest
> version v1.2.11 (Jan 2017) is 4 years old.
> 
> Background here: https://github.com/Rdatatable/data.table/pull/5049
> 
> Best,
> Matt
> Maintainer of data.table in R

We'd be happy to fix the bug if there was an actual report about what
we should fix.  Instead, we got pages and pages that summarize to "must
update", and doesn't explain what the bug is or what the fix is.



Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-24 Thread Theo de Raadt
Dave Voutila  wrote:

> Theo de Raadt writes:
> 
> >> I think the easiest path here is to incorporate the new upstream into a
> >> port, unless someone is familiar with zlib and can cherrypick out the
> >> commit(s) that resolve the issue. (I didn't find zlib in ports already.)
> >
> > That is completely impossible.  It must be in base.  There are 3 copies
> > in base -- userland, kernel, and bootblocks.  They must be kept in sync.
> 
> Not saying to replace what's in base, but have a different version in
> ports available for ports. I was thinking along the lines of egcc or
> eopenssl in that the port co-exists with base and ports that need them
> need tweaking to use them.

You've got to be kidding.  In what world does it help to require -I and
-l lines all over the place, or else everything breaks.



Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-24 Thread Dave Voutila


Theo de Raadt writes:

>> I think the easiest path here is to incorporate the new upstream into a
>> port, unless someone is familiar with zlib and can cherrypick out the
>> commit(s) that resolve the issue. (I didn't find zlib in ports already.)
>
> That is completely impossible.  It must be in base.  There are 3 copies
> in base -- userland, kernel, and bootblocks.  They must be kept in sync.

Not saying to replace what's in base, but have a different version in
ports available for ports. I was thinking along the lines of egcc or
eopenssl in that the port co-exists with base and ports that need them
need tweaking to use them.

>
> It gets updated when things matter.  I wasn't aware a change had happened
> which matters.  The thread makes me unenthusiastic, because I cannot tell
> what changed that matters.  Updating it is not a trivial effort, because
> every bootblock needs testing.  I've done it twice...

I figured and want to be clear I am *not* volunteering for that
exercise! I don't see a change that matters for base.

-dv



Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-24 Thread Theo de Raadt
> I think the easiest path here is to incorporate the new upstream into a
> port, unless someone is familiar with zlib and can cherrypick out the
> commit(s) that resolve the issue. (I didn't find zlib in ports already.)

That is completely impossible.  It must be in base.  There are 3 copies
in base -- userland, kernel, and bootblocks.  They must be kept in sync.

It gets updated when things matter.  I wasn't aware a change had happened
which matters.  The thread makes me unenthusiastic, because I cannot tell
what changed that matters.  Updating it is not a trivial effort, because
every bootblock needs testing.  I've done it twice...



Re: Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-24 Thread Dave Voutila


Matt Dowle writes:

> Hi,
>
> Is it intentional or is there any good reason that OpenBSD 6.9 released May
> 2021 uses a 16 year old version of zlib (v1.2.3; July 2005)?  The latest
> version v1.2.11 (Jan 2017) is 4 years old.
>
> Background here: https://github.com/Rdatatable/data.table/pull/5049

Had a chat with Matt off-list and took a very terse look at zlib's
current status w.r.t. building just on amd64 -current.

I think the easiest path here is to incorporate the new upstream into a
port, unless someone is familiar with zlib and can cherrypick out the
commit(s) that resolve the issue. (I didn't find zlib in ports already.)

To be honest, the extend of my effort was:

1. clone zlib master branch from https://github.com/madler/zlib
2. try building with clang on amd64 -current...which fails.
3. try building with GCC 8.4 from ports...which "just works" (didn't
even need gmake).

To be clear: I didn't test if this actually fixed any issues.

Then any ports that need the newer zlib and can't use the one in base
can use the port.

For folks that maintain ports...does this sound feasible or am I
dreaming? (Assuming someone writes and maintains the port.)

-dv



Why 16 year old zlib 1.2.3 in OpenBSD 6.9 released May 2021 please?

2021-06-24 Thread Matt Dowle
Hi,

Is it intentional or is there any good reason that OpenBSD 6.9 released May
2021 uses a 16 year old version of zlib (v1.2.3; July 2005)?  The latest
version v1.2.11 (Jan 2017) is 4 years old.

Background here: https://github.com/Rdatatable/data.table/pull/5049

Best,
Matt
Maintainer of data.table in R