Ok, fine. D.
On Thu, Sep 16, 2021 at 12:00 PM Robert Muir <rcm...@gmail.com> wrote: > > The shit that mike's python tool is doing, i'd expect bugs like that. > If you honestly disagree with me, go fix his python code to obey "the > rules". Be sure you have "the rules" correct as well (count bytes, not > characters, count EOL, etc). > > Try to imagine how much software is going to get this wrong. Various > tools scanning for security problems that aren't written in java and > hence aren't using Jar class. Your security team isn't writing fucking > java code. Your ops team isn't writing fucking java code, so it really > does not matter if it is "easy in java". > > This is *exactly* what i mean when i say this crazy shit has NO > INTEROP. It is 100% pure garbage, java showing its true colors. > > On Thu, Sep 16, 2021 at 5:53 AM Robert Muir <rcm...@gmail.com> wrote: > > > > I was referring to doing this with languages other than java. > > > > I'm also assuming that exceeding this limit is going to cause indirect > > hassles for users of lucene, e.g. breaking various security / supply > > chain tools. We know a lot of these are total crap but people in the > > corporate world have to suffer under them. > > > > Its super-easy to use a short hash here and avoid problems. > > > > On Thu, Sep 16, 2021 at 3:03 AM Dawid Weiss <dawid.we...@gmail.com> wrote: > > > > > > Jar command doesn't have it, true. But it's fairly trivial to do, even > > > with an inline snippet like this one? > > > > > > public class PrintManifest { > > > public static void main(String[] jars) throws IOException { > > > for (var jar : jars) { > > > var manifest = new JarFile(Paths.get(jar).toFile()).getManifest(); > > > var attrs = manifest.getMainAttributes(); > > > System.out.println(jar + ": " + > > > attrs.getValue("Implementation-Version")); > > > } > > > } > > > } > > > > > > I have this in my lucene-core-9.0.0-SNAPSHOT.jar: > > > > > > Implementation-Version: 9.0.0-SNAPSHOT de45b68c909815ce5ea7b6b9e1a2ce337 > > > 5b6334d [snapshot build, details omitted] > > > > > > and running: > > > > > > java PrintManifest.java lucene-core-9.0.0-SNAPSHOT.jar > > > > > > shows: > > > > > > lucene-core-9.0.0-SNAPSHOT.jar: 9.0.0-SNAPSHOT > > > de45b68c909815ce5ea7b6b9e1a2ce3375b6334d [snapshot build, details > > > omitted] > > > > > > This seems easier to me than trying to remember and keep the length of > > > that line shorter than an arbitrary limit. > > > > > > Dawid > > > > > > > > > On Wed, Sep 15, 2021 at 9:46 PM Robert Muir <rcm...@gmail.com> wrote: > > > > > > > > But its irrelevant that is "valid" when virtually no tools match it. > > > > > > > > In other words, I'd agree with you if the "jar" command had some > > > > ability to read these manifests and print stuff to stdout, e.g. if > > > > there was ANY interop at all here. > > > > > > > > But there isn't. So IMO it makes no sense to cause confusion and chaos > > > > by adding an unnecessarily long git commit hash. > > > > > > > > On Wed, Sep 15, 2021 at 3:26 PM Dawid Weiss <dawid.we...@gmail.com> > > > > wrote: > > > > > > > > > > > > > > > This is valid manifest line-breaking though... Can we read the > > > > > manifest properly on the smoke tester side somehow (for example, run > > > > > a Java process that reads and extracts the required attribute)? This > > > > > way we wouldn't care about the implementation details of how manifest > > > > > wraps the lines (or escapes characters). > > > > > > > > > > D. > > > > > > > > > > On Wed, Sep 15, 2021 at 8:46 PM Mike Drob <md...@mdrob.com> wrote: > > > > >> > > > > >> The benchmark jar has the info we need… sort of. When I built it, it > > > > >> has: > > > > >> > > > > >> Implementation-Version: 8.10.0 > > > > >> 75a5061d3715cc5d93c4cbe4f1fa62bf035eea1 > > > > >> 1 - mdrob - 2021-09-15 11:40:36 > > > > >> > > > > >> > > > > >> and it’s looking for Implementation-Version: 8.10.0 > > > > >> 75a5061d3715cc5d93c4cbe4f1fa62bf035eea11 on one line. > > > > >> > > > > >> Because 8.10 is a character longer than 8.9, we happen to wrap the > > > > >> last character of the git commit sha. From the manifest spec: > > > > >> > > > > >> No line may be longer than 72 bytes (not characters), in its > > > > >> UTF8-encoded form. > > > > >> If a value would make the initial line longer than this, it should > > > > >> be continued > > > > >> on extra lines (each starting with a single SPACE). > > > > >> > > > > >> And we were already teetering on the edge of that limit. We'll run > > > > >> into this problem again in a few years when we try to release > > > > >> version 10.0.0, so solving it now has practical benefits down the > > > > >> line. > > > > >> > > > > >> There's a few options that I can come up with - > > > > >> 1. Use the short-hash when we generate the jar > > > > >> 2. Use the short-hash when we check the contents in the smoke test > > > > >> 3. Do some line join magic in the smoke test. > > > > >> > > > > >> I'm leaning towards number 1 as I feel that would still be unique > > > > >> enough for our needs, but would like to hear from others as well. > > > > >> > > > > >> On Wed, Sep 15, 2021 at 9:46 AM Timothy potter > > > > >> <thelabd...@gmail.com> wrote: > > > > >>> > > > > >>> can someone also please look into that benchmark jar issue? > > > > >>> > > > > >>> Sent from my iPhone > > > > >>> > > > > >>> On Sep 15, 2021, at 9:44 AM, Nhat Nguyen > > > > >>> <nhat.ngu...@elastic.co.invalid> wrote: > > > > >>> > > > > >>> > > > > >>> Thanks Mayya and Mike! I will backport it to the 8.10 branch. > > > > >>> > > > > >>> On Wed, Sep 15, 2021 at 10:12 AM Mike Drob <md...@mdrob.com> wrote: > > > > >>>> > > > > >>>> I think since Tim is out on vacation, it's probably not too late. > > > > >>>> That looks like a good fix to have, do we know how long the bug > > > > >>>> has been present? > > > > >>>> > > > > >>>> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova > > > > >>>> <mayya.sharip...@elastic.co.invalid> wrote: > > > > >>>>> > > > > >>>>> Hello everyone, > > > > >>>>> We have discovered a bug and fixed a bug in Lucene sort > > > > >>>>> optimization (LUCENE-10106) and would like to merge it to Lucene > > > > >>>>> 8.10 if it is not too late. > > > > >>>>> I apologize for the inconvenience, the bug was discovered just > > > > >>>>> yesterday. > > > > >>>>> > > > > >>>>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter > > > > >>>>> <thelabd...@apache.org> wrote: > > > > >>>>>> > > > > >>>>>> Ahem ... unfortunately there will not be an 8.10 RC this week. > > > > >>>>>> I'm > > > > >>>>>> headed out on vacation tomorrow, back at keys on Monday, Sept 20 > > > > >>>>>> unless someone else wants to pick up the RM duties before then? > > > > >>>>>> > > > > >>>>>> After failing the test suite at various places and other > > > > >>>>>> weirdness > > > > >>>>>> like .asc files not getting created, I finally got to the smoke > > > > >>>>>> test > > > > >>>>>> part, which is now failing with: > > > > >>>>>> > > > > >>>>>> File > > > > >>>>>> "/Users/tjp/.lucene-releases/8.10.0/lucene-solr/dev-tools/scripts/smokeTestRelease.py", > > > > >>>>>> line 176, in checkJARMetaData > > > > >>>>>> raise RuntimeError('%s is missing "%s" inside its > > > > >>>>>> META-INF/MANIFEST.MF (wrong git revision?)' % \ > > > > >>>>>> RuntimeError: JAR file > > > > >>>>>> "/Users/tjp/.lucene-releases/8.10.0/RC1/smoketest/unpack/lucene-8.10.0/benchmark/lucene-benchmark-8.10.0.jar" > > > > >>>>>> is missing "Implementation-Version: 8.10.0 > > > > >>>>>> ecf5c747e6df418dd05a18af327c20051f0584d7" inside its > > > > >>>>>> META-INF/MANIFEST.MF (wrong git revision?) > > > > >>>>>> > > > > >>>>>> FWIW, I verified that the other Lucene JAR files have this line > > > > >>>>>> in > > > > >>>>>> them, such as core: > > > > >>>>>> > > > > >>>>>> Manifest-Version: 1.0 > > > > >>>>>> Ant-Version: Apache Ant 1.9.15 > > > > >>>>>> Created-By: 1.8.0_265-b01 (AppleJDK-8.0.265.1.1) > > > > >>>>>> Extension-Name: org.apache.lucene > > > > >>>>>> Specification-Title: Lucene Search Engine: core > > > > >>>>>> Specification-Version: 8.10.0 > > > > >>>>>> Specification-Vendor: The Apache Software Foundation > > > > >>>>>> Implementation-Title: org.apache.lucene > > > > >>>>>> Implementation-Version: 8.10.0 > > > > >>>>>> ecf5c747e6df418dd05a18af327c20051f0584d > > > > >>>>>> 7 - tjp - 2021-09-14 19:08:42 > > > > >>>>>> Implementation-Vendor: The Apache Software Foundation > > > > >>>>>> X-Compile-Source-JDK: 8 > > > > >>>>>> X-Compile-Target-JDK: 8 > > > > >>>>>> Multi-Release: true > > > > >>>>>> > > > > >>>>>> On Tue, Sep 14, 2021 at 1:21 PM Ishan Chattopadhyaya > > > > >>>>>> <ichattopadhy...@gmail.com> wrote: > > > > >>>>>> > > > > > >>>>>> > All the best, this is the worst step. > > > > >>>>>> > > > > > >>>>>> > On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter, > > > > >>>>>> > <thelabd...@gmail.com> wrote: > > > > >>>>>> >> > > > > >>>>>> >> Building RC1 now ... stay tuned. > > > > >>>>>> >> > > > > >>>>>> >> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter > > > > >>>>>> >> <thelabd...@gmail.com> wrote: > > > > >>>>>> >> > > > > > >>>>>> >> > Thanks for the update Mike! > > > > >>>>>> >> > > > > > >>>>>> >> > I'm backporting SOLR-15620 right now and am cooking up a > > > > >>>>>> >> > quick PR for > > > > >>>>>> >> > SOLR-15621, which looks like an easy win for the issue > > > > >>>>>> >> > Cassandra > > > > >>>>>> >> > reported on Slack earlier today. > > > > >>>>>> >> > > > > > >>>>>> >> > Cheers, > > > > >>>>>> >> > Tim > > > > >>>>>> >> > > > > > >>>>>> >> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob > > > > >>>>>> >> > <md...@apache.org> wrote: > > > > >>>>>> >> > > > > > > >>>>>> >> > > Hi Tim, I'm still working on SOLR-15555, the code and > > > > >>>>>> >> > > benchmarking > > > > >>>>>> >> > > both look pretty good, but I've got a few last unit tests > > > > >>>>>> >> > > that I need > > > > >>>>>> >> > > to chase down. Hopefully taken care of by today or > > > > >>>>>> >> > > tomorrow, I'll be > > > > >>>>>> >> > > sure to keep you updated though. > > > > >>>>>> >> > > > > > > >>>>>> >> > > > > > > >>>>>> >> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter > > > > >>>>>> >> > > <thelabd...@gmail.com> wrote: > > > > >>>>>> >> > > > > > > > >>>>>> >> > > > I found > > > > >>>>>> >> > > > https://issues.apache.org/jira/browse/SOLR-15620 while > > > > >>>>>> >> > > > testing > > > > >>>>>> >> > > > the schema designer. I haven't built the RC yet, so > > > > >>>>>> >> > > > going to see if I > > > > >>>>>> >> > > > can get this in today. > > > > >>>>>> >> > > > > > > > >>>>>> >> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter > > > > >>>>>> >> > > > <thelabd...@apache.org> wrote: > > > > >>>>>> >> > > > > > > > > >>>>>> >> > > > > NOTICE: > > > > >>>>>> >> > > > > > > > > >>>>>> >> > > > > Branch branch_8_10 has been cut and versions updated > > > > >>>>>> >> > > > > to 8.11 on stable branch. > > > > >>>>>> >> > > > > > > > > >>>>>> >> > > > > Please observe the normal rules: > > > > >>>>>> >> > > > > > > > > >>>>>> >> > > > > * No new features may be committed to the branch. > > > > >>>>>> >> > > > > > > > > >>>>>> >> > > > > * Documentation patches, build patches and serious > > > > >>>>>> >> > > > > bug fixes may be > > > > >>>>>> >> > > > > committed to the branch. However, you should submit > > > > >>>>>> >> > > > > all patches you > > > > >>>>>> >> > > > > want to commit to Jira first to give others the > > > > >>>>>> >> > > > > chance to review > > > > >>>>>> >> > > > > and possibly vote against the patch. Keep in mind > > > > >>>>>> >> > > > > that it is our > > > > >>>>>> >> > > > > main intention to keep the branch as stable as > > > > >>>>>> >> > > > > possible. > > > > >>>>>> >> > > > > > > > > >>>>>> >> > > > > * All patches that are intended for the branch should > > > > >>>>>> >> > > > > first be committed > > > > >>>>>> >> > > > > to the unstable branch, merged into the stable > > > > >>>>>> >> > > > > branch, and then into > > > > >>>>>> >> > > > > the current release branch. > > > > >>>>>> >> > > > > > > > > >>>>>> >> > > > > * Normal unstable and stable branch development may > > > > >>>>>> >> > > > > continue as usual. > > > > >>>>>> >> > > > > However, if you plan to commit a big change to the > > > > >>>>>> >> > > > > unstable branch > > > > >>>>>> >> > > > > while the branch feature freeze is in effect, think > > > > >>>>>> >> > > > > twice: can't the > > > > >>>>>> >> > > > > addition wait a couple more days? Merges of bug > > > > >>>>>> >> > > > > fixes into the branch > > > > >>>>>> >> > > > > may become more difficult. > > > > >>>>>> >> > > > > > > > > >>>>>> >> > > > > * Only Jira issues with Fix version 8.10 and priority > > > > >>>>>> >> > > > > "Blocker" will delay > > > > >>>>>> >> > > > > a release candidate build. > > > > >>>>>> >> > > > > ---- > > > > >>>>>> >> > > > > > > > >>>>>> >> > > > --------------------------------------------------------------------- > > > > >>>>>> >> > > > To unsubscribe, e-mail: > > > > >>>>>> >> > > > dev-unsubscr...@lucene.apache.org > > > > >>>>>> >> > > > For additional commands, e-mail: > > > > >>>>>> >> > > > dev-h...@lucene.apache.org > > > > >>>>>> >> > > > > > > > >>>>>> >> > > > > > > >>>>>> >> > > --------------------------------------------------------------------- > > > > >>>>>> >> > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > > > >>>>>> >> > > For additional commands, e-mail: > > > > >>>>>> >> > > dev-h...@lucene.apache.org > > > > >>>>>> >> > > > > > > >>>>>> >> > > > > >>>>>> >> --------------------------------------------------------------------- > > > > >>>>>> >> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org > > > > >>>>>> >> For additional commands, e-mail: dev-h...@solr.apache.org > > > > >>>>>> >> > > > > >>>>>> > > > > >>>>>> --------------------------------------------------------------------- > > > > >>>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > > > >>>>>> For additional commands, e-mail: dev-h...@lucene.apache.org > > > > >>>>>> > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org > > > > For additional commands, e-mail: dev-h...@solr.apache.org > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > > For additional commands, e-mail: dev-h...@lucene.apache.org > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org