[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17485183#comment-17485183
 ] 

ASF subversion and git services commented on LUCENE-10255:
--

Commit f103cca5652dc3aabcd469fd5f007c6828b3c695 in lucene's branch 
refs/heads/main from Dawid Weiss
[ https://gitbox.apache.org/repos/asf?p=lucene.git;h=f103cca ]

LUCENE-10255: Add the required unnamed modules in benchmarks subproject to 
module-info so that they are explicit.


> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Assignee: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 36h 10m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> {_}surprisingly difficult{_}... A PoC that almost passes all checks is here:
> -[https://github.com/dweiss/lucene/tree/jms-]
> [https://github.com/dweiss/lucene/tree/jms2]
> Here are my conclusions so far:
>  * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue. (yes, true because 
> gradle splits module resources and classes into separate folders, rendering 
> them unusable as an expanded module).
>  * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
>  * Dependencies between modules will effectively live in two places: in 
> gradle build files and in module-info files. -And they can go out of sync, 
> although it's probably easy to catch (since javac would complain about 
> missing classes during compilation, even if they're in module path).- (with 
> separate module and classpath configurations there is a possibility to verify 
> the consistency).
>  * Probably the biggest challenge (not covered in the PoC) are with our 
> custom javadoc and ecj linter tasks - they see the module-info.java and can't 
> cope with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix).
>  * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
>  * We will have to rethink the long-term strategy of how white-box tests 
> work. There are some guidelines here but all of them have some cons (IDEs 
> being confused). 
> [https://docs.gradle.org/current/userguide/java_testing.html#sec:java_testing_modular]
>  * it's pretty much impossible to exclude transitive dependencies from 
> modules we depend on - if they're not compile-time only (static) 
> requirements, they will have to be present on module path.
>  * supporting modules may or may not work in your IDE.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17463289#comment-17463289
 ] 

Dawid Weiss commented on LUCENE-10255:
--

I filed this simple example in the gradle issue tracker - it shows how even a 
simple project can be made to fail with modular setup there.

https://github.com/gradle/gradle/issues/19376

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Assignee: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 36h 10m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> -https://github.com/dweiss/lucene/tree/jms-
> https://github.com/dweiss/lucene/tree/jms2
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, -modules have 
> to be JARs. The effect of this is that what was previously a set of 
> directories from dependencies now has to be a JAR. What was previously an 
> incremental update of a single .class file now ripples throughout the build 
> recreating module JARs (ZIPs!)... I didn't realize it at first, but it's a 
> costly thing to do. I'm not even sure how IDEs handle this issue.- (not true)
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. -And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).- (with separate module 
> and classpath configurations there is a possibility to verify the 
> consistency).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> * We will have to rethink the long-term strategy of how white-box tests work. 
> There are some guidelines here but all of them have some cons (IDEs being 
> confused). 
> https://docs.gradle.org/current/userguide/java_testing.html#sec:java_testing_modular
> * it's pretty much impossible to exclude transitive dependencies from modules 
> we depend on - if they're not compile-time only (static) requirements, they 
> will have to be present on module path.
> * supporting modules may or may not work in your IDE.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461972#comment-17461972
 ] 

ASF subversion and git services commented on LUCENE-10255:
--

Commit 7e1f3fef699376cc6069a31a5498670080741e98 in lucene's branch 
refs/heads/branch_9x from Dawid Weiss
[ https://gitbox.apache.org/repos/asf?p=lucene.git;h=7e1f3fe ]

LUCENE-10255: add unsynced providers to the module.


> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Assignee: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 36h 10m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> -https://github.com/dweiss/lucene/tree/jms-
> https://github.com/dweiss/lucene/tree/jms2
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, -modules have 
> to be JARs. The effect of this is that what was previously a set of 
> directories from dependencies now has to be a JAR. What was previously an 
> incremental update of a single .class file now ripples throughout the build 
> recreating module JARs (ZIPs!)... I didn't realize it at first, but it's a 
> costly thing to do. I'm not even sure how IDEs handle this issue.- (not true)
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. -And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).- (with separate module 
> and classpath configurations there is a possibility to verify the 
> consistency).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> * We will have to rethink the long-term strategy of how white-box tests work. 
> There are some guidelines here but all of them have some cons (IDEs being 
> confused). 
> https://docs.gradle.org/current/userguide/java_testing.html#sec:java_testing_modular
> * it's pretty much impossible to exclude transitive dependencies from modules 
> we depend on - if they're not compile-time only (static) requirements, they 
> will have to be present on module path.
> * supporting modules may or may not work in your IDE.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461970#comment-17461970
 ] 

ASF subversion and git services commented on LUCENE-10255:
--

Commit c1c27d4ff409ee514f1681207c2fec0dacd54c3c in lucene's branch 
refs/heads/branch_9x from Dawid Weiss
[ https://gitbox.apache.org/repos/asf?p=lucene.git;h=c1c27d4 ]

LUCENE-10255: initial support for Java Modules (squashed).


> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Assignee: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 36h 10m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> -https://github.com/dweiss/lucene/tree/jms-
> https://github.com/dweiss/lucene/tree/jms2
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, -modules have 
> to be JARs. The effect of this is that what was previously a set of 
> directories from dependencies now has to be a JAR. What was previously an 
> incremental update of a single .class file now ripples throughout the build 
> recreating module JARs (ZIPs!)... I didn't realize it at first, but it's a 
> costly thing to do. I'm not even sure how IDEs handle this issue.- (not true)
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. -And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).- (with separate module 
> and classpath configurations there is a possibility to verify the 
> consistency).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> * We will have to rethink the long-term strategy of how white-box tests work. 
> There are some guidelines here but all of them have some cons (IDEs being 
> confused). 
> https://docs.gradle.org/current/userguide/java_testing.html#sec:java_testing_modular
> * it's pretty much impossible to exclude transitive dependencies from modules 
> we depend on - if they're not compile-time only (static) requirements, they 
> will have to be present on module path.
> * supporting modules may or may not work in your IDE.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461967#comment-17461967
 ] 

ASF subversion and git services commented on LUCENE-10255:
--

Commit d42db56babfe1bb93a5f34b064bfa11056716812 in lucene's branch 
refs/heads/main from Dawid Weiss
[ https://gitbox.apache.org/repos/asf?p=lucene.git;h=d42db56 ]

LUCENE-10255: initial support for Java Modules.


> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Assignee: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 36h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> -https://github.com/dweiss/lucene/tree/jms-
> https://github.com/dweiss/lucene/tree/jms2
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, -modules have 
> to be JARs. The effect of this is that what was previously a set of 
> directories from dependencies now has to be a JAR. What was previously an 
> incremental update of a single .class file now ripples throughout the build 
> recreating module JARs (ZIPs!)... I didn't realize it at first, but it's a 
> costly thing to do. I'm not even sure how IDEs handle this issue.- (not true)
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. -And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).- (with separate module 
> and classpath configurations there is a possibility to verify the 
> consistency).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> * We will have to rethink the long-term strategy of how white-box tests work. 
> There are some guidelines here but all of them have some cons (IDEs being 
> confused). 
> https://docs.gradle.org/current/userguide/java_testing.html#sec:java_testing_modular
> * it's pretty much impossible to exclude transitive dependencies from modules 
> we depend on - if they're not compile-time only (static) requirements, they 
> will have to be present on module path.
> * supporting modules may or may not work in your IDE.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17461966#comment-17461966
 ] 

ASF subversion and git services commented on LUCENE-10255:
--

Commit e0745c7b24b392f2657e207c45031238e2f5289a in lucene's branch 
refs/heads/main from Dawid Weiss
[ https://gitbox.apache.org/repos/asf?p=lucene.git;h=e0745c7 ]

LUCENE-10255: re-add utilities for debugging packages and services. These are 
not included by default to avoid unnecessary compilation overhead.


> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Assignee: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 36h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> -https://github.com/dweiss/lucene/tree/jms-
> https://github.com/dweiss/lucene/tree/jms2
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, -modules have 
> to be JARs. The effect of this is that what was previously a set of 
> directories from dependencies now has to be a JAR. What was previously an 
> incremental update of a single .class file now ripples throughout the build 
> recreating module JARs (ZIPs!)... I didn't realize it at first, but it's a 
> costly thing to do. I'm not even sure how IDEs handle this issue.- (not true)
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. -And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).- (with separate module 
> and classpath configurations there is a possibility to verify the 
> consistency).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> * We will have to rethink the long-term strategy of how white-box tests work. 
> There are some guidelines here but all of them have some cons (IDEs being 
> confused). 
> https://docs.gradle.org/current/userguide/java_testing.html#sec:java_testing_modular
> * it's pretty much impossible to exclude transitive dependencies from modules 
> we depend on - if they're not compile-time only (static) requirements, they 
> will have to be present on module path.
> * supporting modules may or may not work in your IDE.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17457143#comment-17457143
 ] 

Dawid Weiss commented on LUCENE-10255:
--

I've reorganized a lot of work, did the cleanups and discovered new problems 
along the way. A cleaned-up branch is here:

https://github.com/dweiss/lucene/tree/jms2
https://github.com/apache/lucene/pull/533

I may be reorganizing the commit graph from time to time (force-pushes) so that 
I can make each commit a reasonably large-scoped thing that fixes one thing. 
Currently they're fairly separate already. 

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 28h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, -modules have 
> to be JARs. The effect of this is that what was previously a set of 
> directories from dependencies now has to be a JAR. What was previously an 
> incremental update of a single .class file now ripples throughout the build 
> recreating module JARs (ZIPs!)... I didn't realize it at first, but it's a 
> costly thing to do. I'm not even sure how IDEs handle this issue.- (not true)
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. -And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).- (with separate module 
> and classpath configurations there is a possibility to verify the 
> consistency).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> * We will have to rethink the long-term strategy of how white-box tests work. 
> There are some guidelines here but all of them have some cons (IDEs being 
> confused). 
> https://docs.gradle.org/current/userguide/java_testing.html#sec:java_testing_modular
> * it's pretty much impossible to exclude transitive dependencies from modules 
> we depend on - if they're not compile-time only (static) requirements, they 
> will have to be present on module path.
> * supporting modules may or may not work in your IDE.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456388#comment-17456388
 ] 

Dawid Weiss commented on LUCENE-10255:
--

Finally! I think we have all the pieces that are needed to introduce proper 
java module support in Lucene. The change basically folds down to:
- separation of dependency configurations between classpath and module-path 
conventions,
- turning off the default module support in gradle (yeah, sorry - I can't make 
it work the way we need it to work) and collecting the module-path and 
classpath manually for javac and tests. This is not as invasive as it sounds - 
gradle is fairly elegant here.
- supporting test subprojects with full module support. I think these should be 
done as separate subprojects because otherwise Eclipse and many other IDEs will 
go tumbling down - multiple sourcesets with a module-info will confuse the hell 
out of them.

All of these things are working and validated in the draft PR at 
https://github.com/apache/lucene/pull/470 - thanks everyone for their support 
and persistence.

I will clean up the repetitive parts of the patch, clean up the unnecessary or 
experimental unrelated changes and will create a cleaned up PR soon. I think 
now, that everything is working, it should be downhill.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 21h 50m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, -modules have 
> to be JARs. The effect of this is that what was previously a set of 
> directories from dependencies now has to be a JAR. What was previously an 
> incremental update of a single .class file now ripples throughout the build 
> recreating module JARs (ZIPs!)... I didn't realize it at first, but it's a 
> costly thing to do. I'm not even sure how IDEs handle this issue.- (not true)
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. -And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).- (with separate module 
> and classpath configurations there is a possibility to verify the 
> consistency).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> * We will have to rethink the long-term strategy of how white-box tests work. 
> There are some guidelines here but all of them have some cons (IDEs being 
> confused). 
> https://docs.gradle.org/current/userguide/java_testing.html#sec:java_testing_modular
> * it's pretty much impossible to exclude transitive dependencies from modules 
> we depend on - if they're not compile-time only (static) requirements, they 
> will have to be present on module path.
> * supporting modules may or may not work in your IDE.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17452578#comment-17452578
 ] 

Dawid Weiss commented on LUCENE-10255:
--

I've added an automatic check verifying the consistency between services 
provided by the modular and classpath layer. It works fine - currently fails 
with:
{code}
org.apache.lucene.distribution.TestModularLayer > test suite's output saved to 
C:\Work\apache\lucene\lucene\distribution-tests\build\test-results\test\outputs\OUTPUT-org.apache.lucene.distribution.TestModularLayer.txt,
 copied below:
   > java.lang.AssertionError: [Modular providers of service 
org.apache.lucene.analysis.TokenFilterFactory in module: 
org.apache.lucene.analysis.common]
   > Expecting TreeSet:
   >   ["org.apache.lucene.analysis.ar.ArabicNormalizationFilterFactory",
...
   > but could not find the following element(s):
   >   ["org.apache.lucene.analysis.es.SpanishPluralStemFilterFactory"]
{code}

which is exactly right.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 11.5h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17451265#comment-17451265
 ] 

Uwe Schindler commented on LUCENE-10255:


Hi [~rfscholte], thank for the advice!

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Robert Scholte (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17451236#comment-17451236
 ] 

Robert Scholte commented on LUCENE-10255:
-

TLDR; unlike Maven, which has a GroupId and ArtifactId for uniquely identifying 
a versionless artifact, Java has the package to uniquely identify a class and a 
module name to uniquely identify a modular jar. Keeping these in sync will 
implicitly prevent you from split package issues.
Prevent situations like 
https://qz.com/646467/how-one-programmer-broke-the-internet-by-deleting-a-tiny-piece-of-code/
 and ensure there will never be any doubt about the module name, so my advice: 
start with org.apache.lucene

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450762#comment-17450762
 ] 

Uwe Schindler commented on LUCENE-10255:


Thanks [~sor] for confirmation.

We now have the following names generated from the gradle build:
{noformat}
> Task :showModuleNames
lucene-benchmark-10.0.0-SNAPSHOT.jar   -> 
org.apache.lucene.benchmark
lucene-backward-codecs-10.0.0-SNAPSHOT.jar -> 
org.apache.lucene.backward_codecs
lucene-classification-10.0.0-SNAPSHOT.jar  -> 
org.apache.lucene.classification
lucene-codecs-10.0.0-SNAPSHOT.jar  -> org.apache.lucene.codecs
lucene-core-10.0.0-SNAPSHOT.jar-> org.apache.lucene.core
lucene-demo-10.0.0-SNAPSHOT.jar-> org.apache.lucene.demo
lucene-expressions-10.0.0-SNAPSHOT.jar -> 
org.apache.lucene.expressions
lucene-facet-10.0.0-SNAPSHOT.jar   -> org.apache.lucene.facet
lucene-grouping-10.0.0-SNAPSHOT.jar-> org.apache.lucene.grouping
lucene-highlighter-10.0.0-SNAPSHOT.jar -> 
org.apache.lucene.highlighter
lucene-join-10.0.0-SNAPSHOT.jar-> org.apache.lucene.join
lucene-luke-10.0.0-SNAPSHOT.jar-> org.apache.lucene.luke
lucene-memory-10.0.0-SNAPSHOT.jar  -> org.apache.lucene.memory
lucene-misc-10.0.0-SNAPSHOT.jar-> org.apache.lucene.misc
lucene-monitor-10.0.0-SNAPSHOT.jar -> org.apache.lucene.monitor
lucene-queries-10.0.0-SNAPSHOT.jar -> org.apache.lucene.queries
lucene-queryparser-10.0.0-SNAPSHOT.jar -> 
org.apache.lucene.queryparser
lucene-replicator-10.0.0-SNAPSHOT.jar  -> 
org.apache.lucene.replicator
lucene-sandbox-10.0.0-SNAPSHOT.jar -> org.apache.lucene.sandbox
lucene-spatial-extras-10.0.0-SNAPSHOT.jar  -> 
org.apache.lucene.spatial_extras
lucene-spatial3d-10.0.0-SNAPSHOT.jar   -> 
org.apache.lucene.spatial3d
lucene-suggest-10.0.0-SNAPSHOT.jar -> org.apache.lucene.suggest
lucene-test-framework-10.0.0-SNAPSHOT.jar  -> 
org.apache.lucene.test_framework
lucene-analysis-common-10.0.0-SNAPSHOT.jar -> 
org.apache.lucene.analysis.common
lucene-analysis-icu-10.0.0-SNAPSHOT.jar-> 
org.apache.lucene.analysis.icu
lucene-analysis-kuromoji-10.0.0-SNAPSHOT.jar   -> 
org.apache.lucene.analysis.kuromoji
lucene-analysis-morfologik-10.0.0-SNAPSHOT.jar -> 
org.apache.lucene.analysis.morfologik
lucene-analysis-nori-10.0.0-SNAPSHOT.jar   -> 
org.apache.lucene.analysis.nori
lucene-analysis-opennlp-10.0.0-SNAPSHOT.jar-> 
org.apache.lucene.analysis.opennlp
lucene-analysis-phonetic-10.0.0-SNAPSHOT.jar   -> 
org.apache.lucene.analysis.phonetic
lucene-analysis-smartcn-10.0.0-SNAPSHOT.jar-> 
org.apache.lucene.analysis.smartcn
lucene-analysis-stempel-10.0.0-SNAPSHOT.jar-> 
org.apache.lucene.analysis.stempel
{noformat}

(see https://github.com/apache/lucene/pull/487)

At the moment it is automatic module names, but this issue is about fully 
modularizing.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since 

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Christian Stein (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450707#comment-17450707
 ] 

Christian Stein commented on LUCENE-10255:
--

FWIW, I agree with Uwe on the naming topic and want to add "prior art" samples 
from other `org.apache.*` project already shipping as Java modules with their 
module names with `org.apache.`: Derby, Felix, POI, Tomcat, and Wicket.

https://github.com/sormuras/modules/blob/be524907f29f60c7895b3cde62850a1937969ad7/com.github.sormuras.modules/com/github/sormuras/modules/modules.properties#L2480-L2551

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450690#comment-17450690
 ] 

Dawid Weiss commented on LUCENE-10255:
--

I accept your arguments, even if I disagree with them, Uwe. I provided a PR to 
change it already.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450680#comment-17450680
 ] 

Uwe Schindler commented on LUCENE-10255:


bq. Sorry for the silly question, but I'm trying to understand why you'd need 
to type it a million times. I too dislike the verbosity of java, but it is my 
understanding that you might only add it to a module-info.java, like once?

Exactly. And for our API users it is not understandable why you must write in 
the moudle-info.java {{requires lucene.core}} but in all java files {{import 
org.apach.lucene.xyz.*;}}. This is inconsistent. And there is the risk of 
clashes (although Lucene is very special, but we will see other third party 
modules then also name their modules like "lucene.foobar.xy", although they 
have nothing in common with Apache. We are an Apache project, so our package 
names, module names and maven artifact names should have the 
"org.apache.lucene" prefix.

This allows to consume in the way everybody knows: In java files for imports 
and when definig your dependencies in Maven or the requires directoives in Java 
modules.

bq. But because it is an API, I do care that it's easy for users to consume it 
with the module system. And that also includes making it easy to consume things 
like analyzers via SPI providers if they are using the module system.

Yes, and this will also work with module system. I tested it after adding 
correct "uses SPIBaseClass" statements to lucene-core's module-info.java. 
Theoretically, in addition we can hide everything from analyzers-common except 
the SPI

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450647#comment-17450647
 ] 

Dawid Weiss commented on LUCENE-10255:
--

It'll be a different prefix in module-info.java "requires xyz" statements and 
in command-line invocations of Luke. Also, it'll list Lucene module as 
"lucene.core@version" instead of "org.apache.lucene@version". I'll provide a PR 
to go back to the full-prefix - Uwe seems to be really determined that this is 
the right way (tm) of doing it.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Robert Muir (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450639#comment-17450639
 ] 

Robert Muir commented on LUCENE-10255:
--

my comment was a genuine question, as I don't yet understand how annoying this 
name will be to API users. I don't yet have any opinion on the color of the 
bikeshed :)

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450635#comment-17450635
 ] 

Dawid Weiss commented on LUCENE-10255:
--

I already typed "-m lucene.luke" what seems like half a million times while 
debugging stuff around the jms and gradle bugs. So I'm almost there.

Listen... I really don't like the full prefix but I really could care less 
about it if you all want to stick with the full domain name - let's just fix 
it, respin the release candidate and be done with it. I did announce the 
shorthand version on LUCENE-10234, perhaps I should have written an all-caps 
announcement but I didn't, sorry. Let's do it the way you like it, I really 
don't care THAT MUCH. I only care a little.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Robert Muir (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450630#comment-17450630
 ] 

Robert Muir commented on LUCENE-10255:
--

{quote}
Sorry but I remain unconvinced that typing a million times "org.apache." in 
various contexts wins you or me anything.
{quote}

Sorry for the silly question, but I'm trying to understand why you'd need to 
type it a million times. I too dislike the verbosity of java, but it is my 
understanding that you might only add it to a module-info.java, like once?

I think as far as specifying stuff on the commandline, its not a problem, as 
lucene isn't a commandline application but instead an API. The one app we 
really ship (luke) has a sh/bat to make it easy.

But because it is an API, I do care that it's easy for users to consume it with 
the module system. And that also includes making it easy to consume things like 
analyzers via SPI providers if they are using the module system. I just don't 
know what that looks like yet (due to my unfamiliarity with the module system), 
but I'd love to visually see the tradeoffs between say 'lucene.analysis.common' 
and 'org.apache.lucene.analysis.common' from an "API user" perspective.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450628#comment-17450628
 ] 

Dawid Weiss commented on LUCENE-10255:
--

https://issues.apache.org/jira/browse/LUCENE-10234

> These are so important changes that it should have been a post on mailing 
> list!

Sure. It wasn't a change though - it was an introduction of what wasn't there 
before at all.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450627#comment-17450627
 ] 

Uwe Schindler commented on LUCENE-10255:


bq. This was done with an announcement on the pull request

These are so important changes that it should have been a post on mailing list!

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450625#comment-17450625
 ] 

Uwe Schindler commented on LUCENE-10255:


bq. I just figured out that you already added Automatic Module names to the 9.0 
release, which are not even hardcoded, but derived through regular 
expressions/search-replace from the internal gradle project path. 

This is even more risky if we decide to remove the ":lucene" top level Gradle 
folder, then the module name changes and nobody will notice!

Everything that's relevant to source code of downstream users should be 
explicitly declared (either in module-info.java or in the manifest).

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450624#comment-17450624
 ] 

Dawid Weiss commented on LUCENE-10255:
--

Uwe... This was done with an announcement on the pull request and the issue. 
And it's also literally everywhere in the scripts you've reviewed ("-m 
lucene.luke"). 

If you really care so much about it and wish to change it to a full prefix we 
can still do it - 9.0 is not out yet.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450620#comment-17450620
 ] 

Uwe Schindler commented on LUCENE-10255:


Apache TIKA also uses module names according to the spec: 
https://github.com/apache/tika/blob/9d29536228860860549d89a052673d47c2af75ca/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-xmp-commons/pom.xml#L48

I just figured out that you already added Automatic Module names to the 9.0 
release, which are not even hardcoded, but derived through regular 
expressions/search replace. This was done completely without any announcement, 
so we have a Lucene release with broken names going out soon. I am glad that 
nobody takes care about modules at the moment...

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450613#comment-17450613
 ] 

Dawid Weiss commented on LUCENE-10255:
--

Mistakes of the youth... I remain unconvinced.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450607#comment-17450607
 ] 

Uwe Schindler commented on LUCENE-10255:


>From the list posted before there is also an example made you:  
>"com.carrotsearch.hppc" is module name of the maven artifact 
>"com.carrotsearch:hppc". This was exactly also my proposal for Lucene:

https://github.com/carrotsearch/hppc/blob/29ab369adac23a76acae1d08529654b2c2dc59e5/gradle/java/compiler.gradle#L24

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450604#comment-17450604
 ] 

Dawid Weiss commented on LUCENE-10255:
--

Sure, Uwe. I think I expressed my personal opinion. :) Some of our current 
module naming cannot be converted to modules (anything with a dash). If you 
want consistency then the first step would be to rename those modules in the 
repo.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450597#comment-17450597
 ] 

Uwe Schindler commented on LUCENE-10255:


We can disagree, for sure, but I'd like to get some more opinions. This MUST be 
a community decision. I gave my well educated opinion and invite everybody to 
read this blog post: 
https://sormuras.github.io/blog/2019-08-04-maven-coordinates-and-java-module-names.html;
 [~sor]  explains very well how a module name should look like.

The module names inside java/jdk are short, but the same is for package names. 
There is also the satement: The package names in every module *should* start 
with the module name (this is not always fully possible, but a good rule is 
that module name and package name should have a common prefix).

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450589#comment-17450589
 ] 

Dawid Weiss commented on LUCENE-10255:
--

You don't understand me, Uwe. I agree on maven central coordinates. I don't 
agree on full prefixes for module naming. I think "lucene." is unique enough. 
This is a subjective opinion and it's really no convincing me otherwise. If you 
want to push the full prefix - I'll live with it, but I don't agree it is 
necessary or useful or solves anything.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450583#comment-17450583
 ] 

Uwe Schindler commented on LUCENE-10255:


Hi,
please have an overview on Maven central and the good work done by 
[~sormu...@gmx.de]:

This table has module names and their artifact names extracted by a script from 
Maven central: 
https://github.com/sormuras/modules/blob/main/doc/Top1000-2020.txt.md (see also 
the repo: https://github.com/sormuras/modules)

When looking at the Top 1000, you will se that all module names that can be 
found on Maven Central use the package names / coordinate names. If you read 
the JLS, they recommend for packages and modules only "simple names" for small 
projects without large outreach.

Here is  conclusion what he recommends: 
https://sormuras.github.io/blog/2019-08-04-maven-coordinates-and-java-module-names.html

So I just repeat myself: Module names should really be unqiue. I don't care 
about 9.0, because its not officially announced, but when we enable the module 
system we should use unique names.

O should we rename also all packages in Lucene's sozurce code and strip off 
org.apache?

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450554#comment-17450554
 ] 

Dawid Weiss commented on LUCENE-10255:
--

Sorry but I remain unconvinced that typing a million times "org.apache." in 
various contexts wins you or me anything. Sure - maven coordinates are there as 
an example where this sort of makes sense (because all of the bazillion 
artifacts live under the same namespace tree). The module system is different  
though - there will be no name conflicts there if you shorten the module name 
to just "lucene". I don't see any gain in prefixing it with anything - the 
opposite, adding a prefix is a nuisance if the 'lucene' prefix is sufficiently 
unique to guarantee no conflicts with anything else. 

Even in the maven namespace some people opt for shorter prefixes (including 
various Apache commons libraries) [1].

[1] https://repo1.maven.org/maven2/commons-net/commons-net/



> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450472#comment-17450472
 ] 

Uwe Schindler commented on LUCENE-10255:


bq. java -m org.apache.lucene.core sounds way less attractive than just java -m 
lucene.core

This is IMHO no argument for shorter names: If your own project is using the 
module system then you have a module-info.java, too. Then you can start it 
without hassle and won't specify any extra options.

I would ask around if there's a standard already. I would really like to see 
consistent module names. "java", "jdk" prefix is different, because Java never 
had any modules names before, but Maven has/had package prefixes. There were 
discussions about this already on JDK mailing list together with Maven people, 
but I have to find them. I think Maven uses the artifact coordinates also for 
module name, but I am not 100% sure. Maybe [~rfscholte] has some more 
information what community standards have evolved.

-1 to use "lucene" as module name prefix, +1 to use "org.apache.lucene" as 
prefix.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450361#comment-17450361
 ] 

Dawid Weiss commented on LUCENE-10255:
--

Think this way: java's internal modules don't have the domain prefix either - 
they rely on the uniqueness of the first part (jdk., java.). I think this is 
sufficient. No need to be paranoid./

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450359#comment-17450359
 ] 

Dawid Weiss commented on LUCENE-10255:
--

I did that intentionally. I hate those long prefixes. They make life much more 
complicated and I don't think there's a risk of running into a conflict with 
anything existing... 

java -m org.apache.lucene.core sounds way less attractive than just java -m 
lucene.core.



> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17450327#comment-17450327
 ] 

Uwe Schindler commented on LUCENE-10255:


Hi,
I would like to bring in one more thing to investiagte: The current Lucene 
modules as of 9.0 are named without full reverse domain names. We should 
investigate on other ASF projects if there is a "standard" how to name modules. 
I don't like it that the Maven group:artifact name is totally different from 
the module name. IMHO the Lucene module should be named with 
"org.apache.lucene." instead of plain "lucene.X".

The log4j module uses this pattern already, and we should coordinate that. 
Maybe ASF has a standard already. I'd ask on the "Apache Commons" project to 
figure out how they plan to handle it.

Changing the current syntax of module name is not a problem, because except for 
Luke we don't expose the modules in our documentation.

As said before I am in favor to name the modules like "groupid.artifactid" 
based on Maven coordinates (append with "." inbetween).

Thoughts?

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Tomoko Uchida (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17449486#comment-17449486
 ] 

Tomoko Uchida commented on LUCENE-10255:


{quote}About compatibility with the to-be-released Lucene/Luke 9.0 and 
auto-modules: I opened a Lucene 8.8 index yesterday (I chose a random ZIP from 
the backwards tests of branch 9.0) in Luke 9.0.0 (non RC, fresh built) 
yesterday. So discovery with auto-modules worked. That's good.
{quote}
Thank you!

 

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17449455#comment-17449455
 ] 

Uwe Schindler commented on LUCENE-10255:


Hi,

About compatibility with the to-be-released Lucene/Luke 9.0 and auto-modules: I 
opened a Lucene 8.8 index yesterday (I chose a random ZIP from the backwards 
tests of branch 9.0) in Luke 9.0.0 (non RC, fresh built) yesterday. So 
discovery with auto-modules worked. That's good.

Of course with module-info files we need to declare the "uses" (in lucene-core) 
and "provides" clauses (all modules that export) clauses.

Uwe

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448834#comment-17448834
 ] 

Uwe Schindler commented on LUCENE-10255:


In short I agree with you on analyzers or revert of modules. 

Still before any decisions about Module system, I want to open an older index 
from 8.x and check if it works (requires SPI to work). Will try tomorrow, it's 
now too late.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448832#comment-17448832
 ] 

Uwe Schindler commented on LUCENE-10255:


LUCENE-10259 is a horrible experience on windows. PR is there and we agreed to 
fix it already. The Analyzer question was discussed and personally I see no big 
issues.

The version number in info box was just an observation, but problem is also 
known.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Tomoko Uchida (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448828#comment-17448828
 ] 

Tomoko Uchida commented on LUCENE-10255:


As I said - let's keep it as it is in 9.0. Please don't disturb the release for 
this nor revert the module-related stuff.

Could you please read my lengthy explanation if it matters to you:
 * I know *it works* although there could be minor glitches that come from 
(partially) adopting the module system.
 * It will be a sandbox to see how the adaptation of the module system affects 
an application.
 * I don't have enough time for now to do thorough tests, will catch the 
glitches in the following releases. I have little real-world experience around 
Jigsaw, but have basic knowledge about it.
 * Luke has a lot of major/minor features than it looks. When you make such 
infrastructure changes on Luke, please let me know if you are not willing to do 
thorough manual tests. It's totally okay and reasonable to use this app for 
experimentation and I don't ask you to do manual tests; so please, again, just 
let me know that and give me time to do so on your behalf.

[~jpountz] thanks, there is no critical issue that should be reported to RM in 
Luke. If so, I would cast a -1 instead of complaining here. Please go ahead!

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448775#comment-17448775
 ] 

Uwe Schindler commented on LUCENE-10255:


About the analyzers... I am thinking of a solution to replace the hack. 
Otherwise I tend to show only StandardAnalyzer in screenshot, but maybe change 
default to a carefully populated custom analyzer?

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448773#comment-17448773
 ] 

Uwe Schindler commented on LUCENE-10255:


We don't need to revert module system for luke's startup scripts in 9.0, but we 
should fix 2 issues:

- LUCENE-10259 (the whitespace issue) => blocker
- LUCENE-10260 (implementation version no longer shown in Luke) => this is 
minor but looks like related to modules, too (LucenePackage no longer returns 
impl version in module mode - have to investigate).

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448772#comment-17448772
 ] 

Dawid Weiss commented on LUCENE-10255:
--

> you know most Windows users have whitespace in their user name

Right... not me - that's why I didn't notice. Listen... there is also the 
possibility of just reverting all those module-related changes and going back 
to plain classpath. I don't mind (although I think it's a step backwards).

Uwe - please feel free to commit to this branch anything you think is 
appropriate, that's why I posted it - counting on your feedback.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448756#comment-17448756
 ] 

Uwe Schindler commented on LUCENE-10255:


(with full module system support as in Dawid's PR, the custom view does not 
work, too).

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448755#comment-17448755
 ] 

Uwe Schindler commented on LUCENE-10255:


The issue with Luke that Tomoko was referring to is this:
 !screenshot-1.png! 

It only shows StandardAnalyzer, no other analyzers. I don't think it's fatal, 
but not so nice. We may fix it as described by removing the analyzers or switch 
to the "custom" view by default:

 !screenshot-2.png! 

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448754#comment-17448754
 ] 

Uwe Schindler commented on LUCENE-10255:


See LUCENE-10259

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448751#comment-17448751
 ] 

Uwe Schindler commented on LUCENE-10255:


[~jpountz], there are another Bug in Luke's startup scripts. Because of missing 
quotes in both the Windows and Linux shell scripts, Luke does not start.

I will open blocker issue. I am currently testing the Lucene release and this 
was the first thing I noticed (you know most Windows users have whitespace in 
their user name, so when you download to desktop and double click, luke does 
not start at all).

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Adrien Grand (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448741#comment-17448741
 ] 

Adrien Grand commented on LUCENE-10255:
---

Tomoko, can you explain what is broken? If it makes Luke hardly usable for most 
users I'd vote for respinning.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448733#comment-17448733
 ] 

Uwe Schindler commented on LUCENE-10255:


The rules are simple: Code that calls "ServiceProvider.load() must have 
"uses..." in the module-info. For Lucene all the SPI stuff is done inside 
lucene.core (be sure and grep on the code for ServiceLoader), so we need to add 
an extry "uses..." for every SPI (Codec, PostingsFormat, TokenizerFactory, 
TokenFilterFactory - also for those we have no implementations in core, but we 
consume all when people use TokenFilterFactory.forName(...).

Should I commit to your branch? Maybe the Luke problems with CustomAnalyzer 
goes away then.

Implementation classes of services must be listed as "provides".

Of course all classes must be exported, but that's already done.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448721#comment-17448721
 ] 

Uwe Schindler commented on LUCENE-10255:


Yes we need to declare the SPI's we provide for external use, see java.base 
exaple:

https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/module-info.java#L368-L398

It says "uses" for all the SPI interfaces it exports. In addition the 
implementations are listed like in Dawid's PR.

So we must add:
{code:java}
  uses  org.apache.lucene.codecs.Codec;
{code}
(because our code consumes the Codec SPI provided by other modules).

and also the implementations (already done):
{code}
  provides org.apache.lucene.codecs.Codec with
  org.apache.lucene.codecs.lucene90.Lucene90Codec;
{code}

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448717#comment-17448717
 ] 

Uwe Schindler commented on LUCENE-10255:


If I have some time the next days I will read more about Services and Modules. 
There were some pitfalls which you don't notice easily unless you have many SPI 
splattered over several modules.

So we should first go and enable a real module-info for things like codecs 
module or backwards module. And then see what happens and add the correct 
provides/uses in combination with requires/exports. Because exporting does not 
make the SPI interface available for consumption, it must be marked as SPI 
first, so the module system knows "this interface" is a service provider. 
That's all missing (it just works because it is all self-contained).

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448714#comment-17448714
 ] 

Uwe Schindler commented on LUCENE-10255:


We won't have real modules in 9.0, so who cares about this bug? If the 
reflection hack shows StandardAnalyzer it's fine.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Tomoko Uchida (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448711#comment-17448711
 ] 

Tomoko Uchida commented on LUCENE-10255:


Please, no... I don't think there is not any good reason to stop ongoing 
release for this.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448708#comment-17448708
 ] 

Dawid Weiss commented on LUCENE-10255:
--

No worries, Tomoko. Thank you for taking your time and pointing at the problem! 
Uwe is on it. I'm sure if I send a box of candy to [~jpountz] he won't mind 
making a third respin of 9.0...

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Tomoko Uchida (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448705#comment-17448705
 ] 

Tomoko Uchida commented on LUCENE-10255:


{quote}There is still time to reiterate the release and modify those scripts to 
run Luke in classpath mode. Or fix the module declaration so that it sees what 
it needs to see. I'll take a look at this.
{quote}
I would keep it as is - anyway the reflection hack is not good and should be 
dropped. I can't take time for now, but I will make a follow-up on 9.1.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448704#comment-17448704
 ] 

Dawid Weiss commented on LUCENE-10255:
--

> Let's go tep by step. We have to also make sure that tests run with 
> dependencies as "modules".

This should be handled automagically by gradle... Much like compilation already 
is.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448702#comment-17448702
 ] 

Uwe Schindler commented on LUCENE-10255:


As far as I see: lucene.core must also declare "uses XyzFactory" because the 
cod calling ServiceLoader is there. This also enables it to "see" 
implementations from unrelated modules on classpath.

It works at moment, because lucene-core is a singleton modue and all it needs 
is inside. But the factories won't see codecs from other modules unless you say 
"uses".

Let's go tep by step. We have to also make sure that tests run with 
dependencies as "modules".

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448699#comment-17448699
 ] 

Dawid Weiss commented on LUCENE-10255:
--

>  As a fact, since 9.0 Luke's Analysis UI is broken from the users' perspective

There is still time to reiterate the release and modify those scripts to run 
Luke in classpath mode. Or fix the module declaration so that it sees what it 
needs to see. I'll take a look at this.

I don't think I will be able to explicitly shout out when something is changed 
that affects Luke - had I known about the problem, I'd have tried to fix it... 
(I ran Luke and toyed around but didn't click on each and every option). 

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448698#comment-17448698
 ] 

Uwe Schindler commented on LUCENE-10255:


This looks like helpful: 
https://www.infoq.com/articles/java11-aware-service-module/

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448697#comment-17448697
 ] 

Uwe Schindler commented on LUCENE-10255:


there's also "uses com.service.api.AnInterface;"

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Tomoko Uchida (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448695#comment-17448695
 ] 

Tomoko Uchida commented on LUCENE-10255:


I mean - I'll try to follow core/infrastructure changes and make necessary 
modifications on Luke GUI; meanwhile, I don't have sufficient spare time to 
follow every change. As a fact, since 9.0 Luke's Analysis UI is broken from the 
users' perspective, even if it is the right decision for the future of the 
project. I'd be glad that if you just let me know when you change luke module's 
infrastructure before it's not too late.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448694#comment-17448694
 ] 

Dawid Weiss commented on LUCENE-10255:
--

There is also require static, right? So that Luke could declare the optional 
requirement (that can be unsatisfied)?

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448693#comment-17448693
 ] 

Uwe Schindler commented on LUCENE-10255:


Oh I figured out, theoretically it should be enough to have it on module-path. 
No command line parameters needed. Let me play around :-)

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448689#comment-17448689
 ] 

Uwe Schindler commented on LUCENE-10255:


It will see the SPIs if the module is added to the runtime. This can be done 
directly inside module-info, or dynamically on command line.

If somebody writes an application and depends on both modules, you will see all 
SPIs. If you want it pluggable just add modules at runtime using commandline 
parameter or with the module system API (Luke could add an UI to select JAR 
files and then query their SPIs using the module reflection APIs).

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448681#comment-17448681
 ] 

Dawid Weiss commented on LUCENE-10255:
--

You overestimate my experience with the module system, Uwe... As far as I 
understand, prior to this PR, everything would work because automatic modules 
are in effect a loophole to get into the entire unnamed module space. After 
this PR - Luke won't see those SPIs (I think...). I just wanted something that 
is minimal, compiles and launches - I'm sure there are quirks looking to be 
addressed.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448675#comment-17448675
 ] 

Uwe Schindler commented on LUCENE-10255:


bq. You can still run it in classpath mode, right? Then it'll work like before.

Of course. I also noticed that Luke's module-info.java currently only depends 
on lucene-core, not analyzers-common, so you are limited anyways with analysis 
(only StandardTokenizer is provided by SPI no filters nothing...). What would 
be the best user experience here? Add some commandline option to add 
analyzers-common to module path, or add it by default (e.g. with 
{{--add-modules lucene.analyzers.common}})? Or how can you make it pluggable 
without falling back to classpath mode?

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448672#comment-17448672
 ] 

Dawid Weiss commented on LUCENE-10255:
--

You can still run it in classpath mode, right? Then it'll work like before.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Tomoko Uchida (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448670#comment-17448670
 ] 

Tomoko Uchida commented on LUCENE-10255:


{quote}Generally Luke should support CustomAnalyzer and build its analyzers 
using components based on SPI, which is of course a bit much work for quickly 
testing.
{quote}
Yes Luke has already supported CustomAnalyzer ever since I rewrote it (though I 
think few of you may know that). Its "Preset analyzers" serves the default of 
"Analysis" UI, but this is for convenience - we can drop it and replace the 
default with "Custom analyzers".

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448664#comment-17448664
 ] 

Uwe Schindler commented on LUCENE-10255:


bq. It works but there is one side effect - its "Analysis > Preset analyzers" 
no longer work since this had relied on reflection tricks that are not 
compatible with the module system. While I don't have any objection we also 
need to drop the zombie function (GUI layer) from Luke. I have little time for 
now but will try to find time for that.

I'd replace this by a hardcoded list as beginning.

Generally Luke should support CustomAnalyzer and build its analyzers using 
components based on SPI, which is of course a bit much work for quickly testing.

Another option is to add SPI functionality to analyzers, but I disagree with 
that. IMHO at some points those hardcoded analyzers (which is code duplication 
everywhere, one for each language) should go away and replaced by a factory 
class returning preconfigured CustomAnalyzer instances. E.g. like {{Analyzer 
ana = Analyzers.german()}}.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Tomoko Uchida (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448599#comment-17448599
 ] 

Tomoko Uchida commented on LUCENE-10255:


{quote}I converted Luke to a full module - it works
{quote}
Yes... I noticed that a few days ago when I saw the new Luke start scripts.

It works but there is one side effect - its "Analysis > Preset analyzers" no 
longer work since this had relied on reflection tricks that are not compatible 
with the module system. While I don't have any objection we also need to drop 
the zombie function (GUI layer) from Luke. I have little time for now but will 
try to find time for that.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448558#comment-17448558
 ] 

Dawid Weiss commented on LUCENE-10255:
--

I tweaked the build so that the changes required to existing tasks and 
infrastructure are minimal. I checked that Luke works (as a module) and I also 
converted lucene.core to a module... and things seems to work beautifully so 
far. In fact, you can immediately see the gain of the jms because the compiler 
shows you API glitches - package-private classes in arguments or return types 
of public API classes (I had to disable that lint check in the patch, for now).

It's quite fun.

https://github.com/apache/lucene/pull/470

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Dawid Weiss (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448241#comment-17448241
 ] 

Dawid Weiss commented on LUCENE-10255:
--

> Normally you have a module-info.java that is compiled by javac? Or is this 
> further additional metadata? 

This is additional metadata injected by 'jar'. It's actually kind of funny to 
look at the source code of that...
See "-module-version=" and "--main-class=" options here:
https://docs.oracle.com/en/java/javase/17/docs/specs/man/jar.html

> Because version numbers should not exist for modules, the JMS requires you to 
> encode it into module name. Like a module "org.apache.lucene.core.v9".

It's not really like this. Modules can have numbers; it's just they're not 
currently used for anything (you can't make a versioned dependency or enforce 
it). This is "outside of the scope of the jms". But you can see those versions, 
for example:

{code}
java --list-modules
...
jdk.sctp@14.0.1
jdk.security.auth@14.0.1
...
{code} 

The branch I created embeds Lucene's version into the created JAR (of Luke) as 
well.

I also thought about what you suggested - keep a separate source folder for 
just the module-info.java. It is an alternative that may be the easiest one to 
follow (until all subprojects have module-info.java information). I just 
thought it was... not elegant somehow. Seemed hacky... But given the complexity 
of workarounds for ecj and javadoc it may be a lesser evil - I'll look into 
this.

Robert - don't spent too much time on this; I'll reiterate with Uwe's 
suggestion - it seems like a cleaner alternative. When all subprojects have 
module-infos it may be easier to just move the tooling to use full modules 
(instead of the classpath).

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Robert Muir (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448213#comment-17448213
 ] 

Robert Muir commented on LUCENE-10255:
--

{quote}
Probably the biggest challenge (not covered in the PoC) are with our custom 
javadoc and ecj linter tasks - they see the module-info.java and can't cope 
with it. At the same time, there is no easy way to exclude that one particular 
file: ecj would have to accept a full set of sources (command argument limit 
will be a problem), javac can accept a full set of java sources (external file) 
but then it doesn't copy doc-files properly anymore (this is probably easier to 
fix).
{quote}

I'm checking out your branch, I will try to look into some of this part. I 
tried to prototype such stuff before (likely going not nearly as far as you 
did) and made some progress, although I can't remember what I did :)

I would expect that e.g. the javadoc task, we may have to change 
{{-sourcepath}} to {{--module-source-path}} and similar stuff like that. maybe 
similar stuff for ecj. I'm sure I am oversimplifying it.


> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448210#comment-17448210
 ] 

Uwe Schindler commented on LUCENE-10255:


bq. Probably the biggest challenge (not covered in the PoC) are with our custom 
javadoc and ecj linter tasks - they see the module-info.java and can't cope 
with it. At the same time, there is no easy way to exclude that one particular 
file: ecj would have to accept a full set of sources (command argument limit 
will be a problem), javac can accept a full set of java sources (external file) 
but then it doesn't copy doc-files properly anymore (this is probably easier to 
fix).

How about putting the module-info.java into a separate directory (maybe next to 
build.gradle) and tell javac to also read that file. So it will be invisible to 
all source processing tools like ecj or javadoc.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448207#comment-17448207
 ] 

Uwe Schindler commented on LUCENE-10255:


bq. A Java module contains metadata (such as the module version or main class) 
that is completely detached from any source file. These things live in a class 
bytecode of the compiled module-info; interestingly, there is no source-level 
way to specify it - these class attributes are injected by the 'jar' tool. 
Gradle has some fancy on-the-fly asm conversion filter that injects it.

Normally you have a module-info.java that is compiled by javac? Or is this 
further additional metadata? Because version numbers should not exist for 
modules, the JMS requires you to encode it into module name. Like a module 
"org.apache.lucene.core.v9".

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-10255) Fully embrace the java module system

[Uwe Schindler (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448197#comment-17448197
 ] 

Uwe Schindler commented on LUCENE-10255:


bq. There are differences at runtime that are hard to anticipate - for example 
resource lookups via class loader no longer work (I fixed this in Luke).

That was always an anti-pattern, should not be used anywhere. The problem of 
Luke was that some resources were in a different package. With module system 
you can only load resources from _same_ package that your class is in.

> Fully embrace the java module system
> 
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Dawid Weiss
>Priority: Major
>
> I've experimented a bit trying to move the code to the JMS. It is 
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any 
> higher-level tooling, including IDEs, I think). For starters, modules have to 
> be JARs. The effect of this is that what was previously a set of directories 
> from dependencies now has to be a JAR. What was previously an incremental 
> update of a single .class file now ripples throughout the build recreating 
> module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing 
> to do. I'm not even sure how IDEs handle this issue.
> * A Java module contains metadata (such as the module version or main class) 
> that is completely detached from any source file. These things live in a 
> class bytecode of the compiled module-info; interestingly, there is no 
> source-level way to specify it - these class attributes are injected by the 
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that 
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle 
> build files and in module-info files. And they can go out of sync, although 
> it's probably easy to catch (since javac would complain about missing classes 
> during compilation, even if they're in module path).
> * Probably the biggest challenge (not covered in the PoC) are with our custom 
> javadoc and ecj linter tasks - they see the module-info.java and can't cope 
> with it. At the same time, there is no easy way to exclude that one 
> particular file: ecj would have to accept a full set of sources (command 
> argument limit will be a problem), javac can accept a full set of java 
> sources (external file) but then it doesn't copy doc-files properly anymore 
> (this is probably easier to fix). 
> * There are differences at runtime that are hard to anticipate - for example 
> resource lookups via class loader no longer work (I fixed this in Luke).
> After poking a bit and trying it out I have to say I have mixed feelings 
> about moving to the JMS. On the one hand, many things are great - the module 
> path, module descriptors and access modes. On the other hand, the tooling 
> tricks required to make it all work make you shiver.
> If anybody wants to play/ improve things on that experimental branch (I 
> converted Luke to a full module - it works), please be my guest. I have to 
> sit on this and think whether it's something I really like or not.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org