Re: Intellij build/test times

2024-06-08 Thread Dawid Weiss
> By the way, the
> classpath problems seem to occur with either method (gradle or
> intellij) when running entire suite - I just confused while switching
> back and forth. This is on main, haven't tried 9x recently
>

Some of these headaches are caused by Lucene's folder structure and have
been there forever - resources mixed with source classes. I don't know if
you can make intellij use a folder as a resource and as source directory at
the same time - I don't think it's possible. If so, tests that rely on
these resources will fail. It's been this way since I remember - nothing
has changed here.

There is also a lot of trickery involving modular paths etc. I don't think
it'll be easy to simulate this in Intellij. Then, 99% of test cases will
run just fine from intellij without any special hacks (I think)...

I'd say - run individual tests from intellij, add a test launching config
redirecting to gradle for the whole suite - it should also be faster this
way since tests will run in parallel between modules.

D.


Re: Intellij build/test times

2024-06-08 Thread Michael Sokolov
Indeed, it's when I run multiple tests that I see the problems.
Running single test classes seems to work OK. In the past I have been
able to run the entire test suite, but I agree this is less critical
than being able to debug single tests. Cursory internet search
indicates the problem is widespread and others propose using the same
plan - don't use gradle test runner in intellij. By the way, the
classpath problems seem to occur with either method (gradle or
intellij) when running entire suite - I just confused while switching
back and forth. This is on main, haven't tried 9x recently

On Fri, Jun 7, 2024 at 4:05 PM Dawid Weiss  wrote:
>
>
> Hi Mike,
>
> Are you trying to run all the tests from Lucene from IntelliJ? I admit I 
> haven't tried that... :) I usually use intellij for running/ debugging 
> isolated classes, then rerun the full suite from command line (increased 
> parallelism). I don't think everything will work - if something needs a 
> specific setup done by gradle tasks or has resources under src, where they're 
> not seed as resources by intellij and thus not copied - tough luck. But most 
> stuff should work.
>
> Running via gradle is slow for me not just with Lucene but also with other 
> projects... I can take a look but I'm pessimistic I can do any wonders here.
>
> Dawid
>
> On Fri, Jun 7, 2024 at 6:06 PM Michael Sokolov  wrote:
>>
>> I'm also getting errors like:
>>
>> Caused by: java.lang.ExceptionInInitializerError: Exception
>> java.lang.LinkageError: MemorySegmentIndexInputProvider is missing in
>> Lucene JAR file [in thread
>> "TEST-TestDemo.testDemo-seed#[872544629C2881C6]"]
>>
>> I wonder if this is due to some kind of module permissions thing
>> controlling the visibility of these symbols?
>>
>> On Fri, Jun 7, 2024 at 11:53 AM Michael Sokolov  wrote:
>> >
>> > hm I found FakeCharFilterFactory in src/test/META-INF.services -- it's
>> > in a "test sources root" folder and won't allow itself to be set as a
>> > resources folder? hm even after fiddling with this - I finally get to
>> > mark it as "test resources root" my test is still not passing. This
>> > can't be this hard!
>> >
>> > On Fri, Jun 7, 2024 at 11:44 AM Michael Sokolov  wrote:
>> > >
>> > > hmm so after playing around with this Intellij build for a bit I ran
>> > > into some trouble -- all the tests relying on SPI seemed to start
>> > > failing. So then I switched back to build with Gradle and rebuild the
>> > > project and these tests passed. Just to double check there wasn't some
>> > > strange stale build problem, I think switched back again to IntelliJ
>> > > builder and I still see the same failures; example is like:
>> > >
>> > > NOTE: reproduce with: gradlew test --tests
>> > > TestAnalysisSPILoader.testLookupCharFilter
>> > > -Dtests.seed=88A2DA17C6510A33 -Dtests.locale=en-PR
>> > > -Dtests.timezone=Etc/GMT-9 -Dtests.asserts=true
>> > > -Dtests.file.encoding=UTF-8
>> > >
>> > > java.lang.IllegalArgumentException: A SPI class of type
>> > > org.apache.lucene.analysis.CharFilterFactory with name 'Fake' does not
>> > > exist. You need to add the corresponding JAR file supporting this SPI
>> > > to your classpath. The current classpath supports the following names:
>> > > []
>> > >
>> > > I guess there must be some setup required in order to expose the SPI
>> > > resource files to the build? So I checked some of the resources
>> > > folders like lucene/analysis/common/src/resources and sure enough it
>> > > is labeled as a resources folder in intellij UI. So ... what am I
>> > > missing?
>> > >
>> > > On Fri, Jun 7, 2024 at 10:40 AM Michael Sokolov  
>> > > wrote:
>> > > >
>> > > > ok, life must be scary for developers on windows!
>> > > >
>> > > > On Fri, Jun 7, 2024 at 10:33 AM Dawid Weiss  
>> > > > wrote:
>> > > > >
>> > > > >
>> > > > > Certain regenerate tasks do require perl and python indeed.
>> > > > >
>> > > > > On Fri, Jun 7, 2024 at 2:23 PM Michael Sokolov  
>> > > > > wrote:
>> > > > >>
>> > > > >> While editing this CONTRIBUTING.md I found the following statement:
>> > > > >>
>> > > > >> Some build tasks (in particular `./gradlew check`) require Perl
>> > > > >> and Python 3.
>> > > > >>
>> > > > >> Is it actually true that we require Perl?
>> > > > >>
>> > > > >> On Fri, Jun 7, 2024 at 8:11 AM Michael Sokolov  
>> > > > >> wrote:
>> > > > >> >
>> > > > >> > So I'm glad we have a fix for this, but it's making me realize 
>> > > > >> > that
>> > > > >> > any new joiner that uses intellij (probably most of them?) will 
>> > > > >> > have
>> > > > >> > this problem and have no idea what to do about it. They will just
>> > > > >> > conclude - running Lucene tests in intellij sucks. If we revived 
>> > > > >> > that
>> > > > >> > intellij target maybe that would help - but .. you would have to 
>> > > > >> > know
>> > > > >> > to run it! So then I went to look at our project web page to see 
>> > > > >> > what
>> > > > >> > kind of developer docs we have that a new contributor might find.
>> > > >