: I was referring to doing this with languages other than java.
:
: I'm also assuming that exceeding this limit is going to cause indirect
: hassles for users of lucene, e.g. breaking various security / supply
: chain tools. We know a lot of these are total crap but people in the
: corporate world have to suffer under them.
Just to be clear -- our 'Implementation-Version:' has been exceeding the
72 byte "single line" limit for a LOOOOONG time -- worrying about how
"security / supply chain" tools will handle parsing that line now seems
kind of silly...
If tools can't handle a line wrap in the 8.10 jars, then they haven't
been able to handle the line wrap since we switched from svn to git (when
the Implementation Version values switched from being based svn version#
to git sha)
The *ONLY* thing that's new here is where in the value the line wrap
happens (with 8.10.0 it happens in the middle of the SHA) and that our
smoketest tool isn't smart enough to parse the values properly.
This is not even the first time we've even had a conversation about the
smoke tester and Implementation Version line wraps: LUCENE-7023.
: Its super-easy to use a short hash here and avoid problems.
There is however an actual and practical downside to switching our
implementation version to using a "short" SHA, and that's that we would
lose the ability to garuntee that the information in the
Implementation-Version uniquely identifies what commit a given jar was
built from. Multiple commits with the same short(end) hash are possible
-- Multiple commits with identical (full) commits is not.
Folks may think that using git tags is useful enough for figuring this
out from official releases, but being able to look at the jar metadata
from arbitrary builds off of arbitrary branches and sanity check where
exactly they come from has been very useful to me for 10+ years.
: On Thu, Sep 16, 2021 at 3:03 AM Dawid Weiss <dawid.we...@gmail.com> wrote:
: >
: > Jar command doesn't have it, true. But it's fairly trivial to do, even
: > with an inline snippet like this one?
: >
: > public class PrintManifest {
: > public static void main(String[] jars) throws IOException {
: > for (var jar : jars) {
: > var manifest = new JarFile(Paths.get(jar).toFile()).getManifest();
: > var attrs = manifest.getMainAttributes();
: > System.out.println(jar + ": " +
attrs.getValue("Implementation-Version"));
: > }
: > }
: > }
: >
: > I have this in my lucene-core-9.0.0-SNAPSHOT.jar:
: >
: > Implementation-Version: 9.0.0-SNAPSHOT de45b68c909815ce5ea7b6b9e1a2ce337
: > 5b6334d [snapshot build, details omitted]
: >
: > and running:
: >
: > java PrintManifest.java lucene-core-9.0.0-SNAPSHOT.jar
: >
: > shows:
: >
: > lucene-core-9.0.0-SNAPSHOT.jar: 9.0.0-SNAPSHOT
: > de45b68c909815ce5ea7b6b9e1a2ce3375b6334d [snapshot build, details
: > omitted]
: >
: > This seems easier to me than trying to remember and keep the length of
: > that line shorter than an arbitrary limit.
: >
: > Dawid
: >
: >
: > On Wed, Sep 15, 2021 at 9:46 PM Robert Muir <rcm...@gmail.com> wrote:
: > >
: > > But its irrelevant that is "valid" when virtually no tools match it.
: > >
: > > In other words, I'd agree with you if the "jar" command had some
: > > ability to read these manifests and print stuff to stdout, e.g. if
: > > there was ANY interop at all here.
: > >
: > > But there isn't. So IMO it makes no sense to cause confusion and chaos
: > > by adding an unnecessarily long git commit hash.
: > >
: > > On Wed, Sep 15, 2021 at 3:26 PM Dawid Weiss <dawid.we...@gmail.com> wrote:
: > > >
: > > >
: > > > This is valid manifest line-breaking though... Can we read the manifest
properly on the smoke tester side somehow (for example, run a Java process that
reads and extracts the required attribute)? This way we wouldn't care about the
implementation details of how manifest wraps the lines (or escapes characters).
: > > >
: > > > D.
: > > >
: > > > On Wed, Sep 15, 2021 at 8:46 PM Mike Drob <md...@mdrob.com> wrote:
: > > >>
: > > >> The benchmark jar has the info we need… sort of. When I built it, it
has:
: > > >>
: > > >> Implementation-Version: 8.10.0 75a5061d3715cc5d93c4cbe4f1fa62bf035eea1
: > > >> 1 - mdrob - 2021-09-15 11:40:36
: > > >>
: > > >>
: > > >> and it’s looking for Implementation-Version: 8.10.0
75a5061d3715cc5d93c4cbe4f1fa62bf035eea11 on one line.
: > > >>
: > > >> Because 8.10 is a character longer than 8.9, we happen to wrap the
last character of the git commit sha. From the manifest spec:
: > > >>
: > > >> No line may be longer than 72 bytes (not characters), in its
UTF8-encoded form.
: > > >> If a value would make the initial line longer than this, it should be
continued
: > > >> on extra lines (each starting with a single SPACE).
: > > >>
: > > >> And we were already teetering on the edge of that limit. We'll run
into this problem again in a few years when we try to release version 10.0.0,
so solving it now has practical benefits down the line.
: > > >>
: > > >> There's a few options that I can come up with -
: > > >> 1. Use the short-hash when we generate the jar
: > > >> 2. Use the short-hash when we check the contents in the smoke test
: > > >> 3. Do some line join magic in the smoke test.
: > > >>
: > > >> I'm leaning towards number 1 as I feel that would still be unique
enough for our needs, but would like to hear from others as well.
: > > >>
: > > >> On Wed, Sep 15, 2021 at 9:46 AM Timothy potter <thelabd...@gmail.com>
wrote:
: > > >>>
: > > >>> can someone also please look into that benchmark jar issue?
: > > >>>
: > > >>> Sent from my iPhone
: > > >>>
: > > >>> On Sep 15, 2021, at 9:44 AM, Nhat Nguyen
<nhat.ngu...@elastic.co.invalid> wrote:
: > > >>>
: > > >>>
: > > >>> Thanks Mayya and Mike! I will backport it to the 8.10 branch.
: > > >>>
: > > >>> On Wed, Sep 15, 2021 at 10:12 AM Mike Drob <md...@mdrob.com> wrote:
: > > >>>>
: > > >>>> I think since Tim is out on vacation, it's probably not too late.
That looks like a good fix to have, do we know how long the bug has been
present?
: > > >>>>
: > > >>>> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova
<mayya.sharip...@elastic.co.invalid> wrote:
: > > >>>>>
: > > >>>>> Hello everyone,
: > > >>>>> We have discovered a bug and fixed a bug in Lucene sort
optimization (LUCENE-10106) and would like to merge it to Lucene 8.10 if it is
not too late.
: > > >>>>> I apologize for the inconvenience, the bug was discovered just
yesterday.
: > > >>>>>
: > > >>>>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter
<thelabd...@apache.org> wrote:
: > > >>>>>>
: > > >>>>>> Ahem ... unfortunately there will not be an 8.10 RC this week. I'm
: > > >>>>>> headed out on vacation tomorrow, back at keys on Monday, Sept 20
: > > >>>>>> unless someone else wants to pick up the RM duties before then?
: > > >>>>>>
: > > >>>>>> After failing the test suite at various places and other weirdness
: > > >>>>>> like .asc files not getting created, I finally got to the smoke
test
: > > >>>>>> part, which is now failing with:
: > > >>>>>>
: > > >>>>>> File
"/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
: > > >>>>>> line 176, in checkJARMetaData
: > > >>>>>> raise RuntimeError('%s is missing "%s" inside its
: > > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)' % \
: > > >>>>>> RuntimeError: JAR file
: > > >>>>>>
"/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar"
: > > >>>>>> is missing "Implementation-Version: 8.10.0
: > > >>>>>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its
: > > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)
: > > >>>>>>
: > > >>>>>> FWIW, I verified that the other Lucene JAR files have this line in
: > > >>>>>> them, such as core:
: > > >>>>>>
: > > >>>>>> Manifest-Version: 1.0
: > > >>>>>> Ant-Version: Apache Ant 1.9.15
: > > >>>>>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1)
: > > >>>>>> Extension-Name: org.apache.lucene
: > > >>>>>> Specification-Title: Lucene Search Engine: core
: > > >>>>>> Specification-Version: 8.10.0
: > > >>>>>> Specification-Vendor: The Apache Software Foundation
: > > >>>>>> Implementation-Title: org.apache.lucene
: > > >>>>>> Implementation-Version: 8.10.0
ecf5c747e6df418dd05a18af327c20051f0584d
: > > >>>>>> 7 - tjp - 2021-09-14 19:08:42
: > > >>>>>> Implementation-Vendor: The Apache Software Foundation
: > > >>>>>> X-Compile-Source-JDK: 8
: > > >>>>>> X-Compile-Target-JDK: 8
: > > >>>>>> Multi-Release: true
: > > >>>>>>
: > > >>>>>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya
: > > >>>>>> <ichattopadhy...@gmail.com> wrote:
: > > >>>>>> >
: > > >>>>>> > All the best, this is the worst step.
: > > >>>>>> >
: > > >>>>>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter,
<thelabd...@gmail.com> wrote:
: > > >>>>>> >>
: > > >>>>>> >> Building RC1 now ... stay tuned.
: > > >>>>>> >>
: > > >>>>>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter
<thelabd...@gmail.com> wrote:
: > > >>>>>> >> >
: > > >>>>>> >> > Thanks for the update Mike!
: > > >>>>>> >> >
: > > >>>>>> >> > I'm backporting SOLR-15620 right now and am cooking up a
quick PR for
: > > >>>>>> >> > SOLR-15621, which looks like an easy win for the issue
Cassandra
: > > >>>>>> >> > reported on Slack earlier today.
: > > >>>>>> >> >
: > > >>>>>> >> > Cheers,
: > > >>>>>> >> > Tim
: > > >>>>>> >> >
: > > >>>>>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob <md...@apache.org>
wrote:
: > > >>>>>> >> > >
: > > >>>>>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and
benchmarking
: > > >>>>>> >> > > both look pretty good, but I've got a few last unit tests
that I need
: > > >>>>>> >> > > to chase down. Hopefully taken care of by today or
tomorrow, I'll be
: > > >>>>>> >> > > sure to keep you updated though.
: > > >>>>>> >> > >
: > > >>>>>> >> > >
: > > >>>>>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter
<thelabd...@gmail.com> wrote:
: > > >>>>>> >> > > >
: > > >>>>>> >> > > > I found https://issues.apache.org/jira/browse/SOLR-15620
while testing
: > > >>>>>> >> > > > the schema designer. I haven't built the RC yet, so going
to see if I
: > > >>>>>> >> > > > can get this in today.
: > > >>>>>> >> > > >
: > > >>>>>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter
<thelabd...@apache.org> wrote:
: > > >>>>>> >> > > > >
: > > >>>>>> >> > > > > NOTICE:
: > > >>>>>> >> > > > >
: > > >>>>>> >> > > > > Branch branch_8_10 has been cut and versions updated to
8.11 on stable branch.
: > > >>>>>> >> > > > >
: > > >>>>>> >> > > > > Please observe the normal rules:
: > > >>>>>> >> > > > >
: > > >>>>>> >> > > > > * No new features may be committed to the branch.
: > > >>>>>> >> > > > >
: > > >>>>>> >> > > > > * Documentation patches, build patches and serious bug
fixes may be
: > > >>>>>> >> > > > > committed to the branch. However, you should submit
all patches you
: > > >>>>>> >> > > > > want to commit to Jira first to give others the
chance to review
: > > >>>>>> >> > > > > and possibly vote against the patch. Keep in mind
that it is our
: > > >>>>>> >> > > > > main intention to keep the branch as stable as
possible.
: > > >>>>>> >> > > > >
: > > >>>>>> >> > > > > * All patches that are intended for the branch should
first be committed
: > > >>>>>> >> > > > > to the unstable branch, merged into the stable
branch, and then into
: > > >>>>>> >> > > > > the current release branch.
: > > >>>>>> >> > > > >
: > > >>>>>> >> > > > > * Normal unstable and stable branch development may
continue as usual.
: > > >>>>>> >> > > > > However, if you plan to commit a big change to the
unstable branch
: > > >>>>>> >> > > > > while the branch feature freeze is in effect, think
twice: can't the
: > > >>>>>> >> > > > > addition wait a couple more days? Merges of bug fixes
into the branch
: > > >>>>>> >> > > > > may become more difficult.
: > > >>>>>> >> > > > >
: > > >>>>>> >> > > > > * Only Jira issues with Fix version 8.10 and priority
"Blocker" will delay
: > > >>>>>> >> > > > > a release candidate build.
: > > >>>>>> >> > > > > ----
: > > >>>>>> >> > > >
: > > >>>>>> >> > > >
---------------------------------------------------------------------
: > > >>>>>> >> > > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: > > >>>>>> >> > > > For additional commands, e-mail:
dev-h...@lucene.apache.org
: > > >>>>>> >> > > >
: > > >>>>>> >> > >
: > > >>>>>> >> > >
---------------------------------------------------------------------
: > > >>>>>> >> > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: > > >>>>>> >> > > For additional commands, e-mail: dev-h...@lucene.apache.org
: > > >>>>>> >> > >
: > > >>>>>> >>
: > > >>>>>> >>
---------------------------------------------------------------------
: > > >>>>>> >> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
: > > >>>>>> >> For additional commands, e-mail: dev-h...@solr.apache.org
: > > >>>>>> >>
: > > >>>>>>
: > > >>>>>>
---------------------------------------------------------------------
: > > >>>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: > > >>>>>> For additional commands, e-mail: dev-h...@lucene.apache.org
: > > >>>>>>
: > >
: > > ---------------------------------------------------------------------
: > > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
: > > For additional commands, e-mail: dev-h...@solr.apache.org
: > >
: >
: > ---------------------------------------------------------------------
: > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: > For additional commands, e-mail: dev-h...@lucene.apache.org
: >
:
: ---------------------------------------------------------------------
: To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
: For additional commands, e-mail: dev-h...@lucene.apache.org
:
:
-Hoss
http://www.lucidworks.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
