[jira] [Comment Edited] (SOLR-13452) Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
[ https://issues.apache.org/jira/browse/SOLR-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920247#comment-16920247 ] Mark Miller edited comment on SOLR-13452 at 9/1/19 12:24 AM: - Development has shifted to https://github.com/apache/lucene-solr/tree/jira/SOLR-13452_gradle_6 which is up to date from a few days ago I believe. If anyone pushes something to the previous branch on an update, I'll cherry pick it to the latest branch for you if it helps. was (Author: markrmil...@gmail.com): Development has shifted to https://github.com/apache/lucene-solr/tree/jira/SOLR-13452_gradle_6 which is up from a few days ago I believe. If anyone pushes something to the previous branch on an update, I'll cherry pick it to the latest branch for you if it helps. > Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle. > - > > Key: SOLR-13452 > URL: https://issues.apache.org/jira/browse/SOLR-13452 > Project: Solr > Issue Type: Improvement > Components: Build >Reporter: Mark Miller >Assignee: Mark Miller >Priority: Major > Fix For: master (9.0) > > Attachments: gradle-build.pdf > > Time Spent: 10m > Remaining Estimate: 0h > > I took some things from the great work that Dat did in > [https://github.com/apache/lucene-solr/tree/jira/gradle] and took the ball a > little further. > > When working with gradle in sub modules directly, I recommend > [https://github.com/dougborg/gdub] > This gradle branch uses the following plugin for version locking, version > configuration and version consistency across modules: > [https://github.com/palantir/gradle-consistent-versions] > > https://github.com/apache/lucene-solr/tree/jira/SOLR-13452_gradle_6 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13452) Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
[
https://issues.apache.org/jira/browse/SOLR-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16914216#comment-16914216
]
Tomoko Uchida edited comment on SOLR-13452 at 8/23/19 2:18 PM:
---
When I looked into inter-module dependency problems on IDEA, I noticed that it
seems difficult to manage to keep our current IDEA module structure with Gradle
IDEA plugin.
I discarded (refactored) current structure defined in {{.idea/modules.xml}}
and instead, made use of the default structure generated by the Gradle plugin.
(Gradle IDEA plugin works well for me and I think it's much better than
maintaining ".iml" files for all Lucene/Solr modules on our own.)
By this change IDEA users will see:
1. module names have been changed (e.g., "backward-codecs" is now
"lucene-backward-codecs", "extraction" is "solr-contrib-extraction", and so on)
2. main / test modules had been merged into one module (e.g., "lucene-core"
and "lucene-core-tests" are now packed into "lucene-core", "solr-core" and
"solr-core-tests" are packed into "solr-core", and so on).
And you might notice 3. IDEA complains some circular dependencies those
haven't been reported so far (they had been cleverly suppressed by customized
module structure), but daily development shouldn't be affected by this.
I think unused IDEA configuration files can be removed when we remove all other
Ant related files.
For me IDEA setup is 90% ready now... still not perfect, some
fixes/improvements are needed.
[~markrmil...@gmail.com]: when I looked through build files, I noticed a typo.
[https://github.com/apache/lucene-solr/blob/jira/SOLR-13452_gradle_5/lucene/analysis/stempel/build.gradle#L22]
I think {{lucene-analyzers-stemple}} should be {{lucene-analyzers-stempel}} :)
was (Author: tomoko uchida):
When I looked into inter-module dependency problems on IDEA, I noticed that it
seems difficult to manage to keep our current IDEA module structure with Gradle
IDEA plugin.
I discarded (refactored) current structure defined in {{.idea/modules.xml}}
and instead, made use of the default structure generated by the Gradle plugin.
(Gradle IDEA plugin works well for me and I think it's much better than
maintaining ".iml" files for all Lucene/Solr modules on our own.)
By this change IDEA users will see:
1. module names have been changed (e.g., "backward-codecs" is now
"lucene-backward-codecs", "extraction" is "solr-contrib-extraction", and so on)
2. main / test modules have been merged into one module (e.g., "lucene-core"
and "lucene-core-tests" are now packed into "lucene-core", "solr-core" and
"solr-core-tests" are packed into "solr-core", and so on).
And you might notice 3. IDEA complains some circular dependencies those
haven't been reported so far (they have been cleverly suppressed by customized
module structure), but daily development shouldn't be affected by this.
I think unused IDEA configuration files can be removed when we remove all other
Ant related files.
For me IDEA setup is 90% ready now... still not perfect, some
fixes/improvements are needed.
> Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
> -
>
> Key: SOLR-13452
> URL: https://issues.apache.org/jira/browse/SOLR-13452
> Project: Solr
> Issue Type: Improvement
> Components: Build
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Major
> Fix For: master (9.0)
>
> Attachments: gradle-build.pdf
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I took some things from the great work that Dat did in
> [https://github.com/apache/lucene-solr/tree/jira/gradle] and took the ball a
> little further.
>
> When working with gradle in sub modules directly, I recommend
> [https://github.com/dougborg/gdub]
> This gradle branch uses the following plugin for version locking, version
> configuration and version consistency across modules:
> [https://github.com/palantir/gradle-consistent-versions]
>
> https://github.com/apache/lucene-solr/tree/jira/SOLR-13452_gradle_5
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13452) Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
[
https://issues.apache.org/jira/browse/SOLR-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16914216#comment-16914216
]
Tomoko Uchida edited comment on SOLR-13452 at 8/23/19 12:42 PM:
When I looked into inter-module dependency problems on IDEA, I noticed that it
seems difficult to manage to keep our current IDEA module structure with Gradle
IDEA plugin.
I discarded (refactored) current structure defined in {{.idea/modules.xml}}
and instead, made use of the default structure generated by the Gradle plugin.
(Gradle IDEA plugin works well for me and I think it's much better than
maintaining ".iml" files for all Lucene/Solr modules on our own.)
By this change IDEA users will see:
1. module names have been changed (e.g., "backward-codecs" is now
"lucene-backward-codecs", "extraction" is "solr-contrib-extraction", and so on)
2. main / test modules have been merged into one module (e.g., "lucene-core"
and "lucene-core-tests" are now packed into "lucene-core", "solr-core" and
"solr-core-tests" are packed into "solr-core", and so on).
And you might notice 3. IDEA complains some circular dependencies those
haven't been reported so far (they have been cleverly suppressed by customized
module structure), but daily development shouldn't be affected by this.
I think unused IDEA configuration files can be removed when we remove all other
Ant related files.
For me IDEA setup is 90% ready now... still not perfect, some
fixes/improvements are needed.
was (Author: tomoko uchida):
When I looked into inter-module dependency problems on IDEA, I noticed that it
seems difficult to manage to keep our current IDEA module structure with Gradle
IDEA plugin.
I discarded current structure (defined in {{.idea/modules.xml}}) and instead
made use of the default structure generated by the Gradle plugin. (Gradle IDEA
plugin works well for me and I think it's much better than maintaining ".iml"
files for all Lucene/Solr modules on our own.)
By this change IDEA users will see:
1. module names have been changed (e.g., "backward-codecs" is now
"lucene-backward-codecs", "extraction" is "solr-contrib-extraction", and so on)
2. main / test modules have been merged into one module (e.g., "lucene-core"
and "lucene-core-tests" are now packed into "lucene-core", "solr-core" and
"solr-core-tests" are packed into "solr-core", and so on).
And you might notice 3. IDEA complains some circular dependencies those
haven't been reported so far (they have been cleverly suppressed by customized
module structure), but daily development shouldn't be affected by this.
I think unused IDEA configuration files can be removed when we remove all other
Ant related files.
For me IDEA setup is 90% ready now... still not perfect, some
fixes/improvements are needed.
> Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
> -
>
> Key: SOLR-13452
> URL: https://issues.apache.org/jira/browse/SOLR-13452
> Project: Solr
> Issue Type: Improvement
> Components: Build
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Major
> Fix For: master (9.0)
>
> Attachments: gradle-build.pdf
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I took some things from the great work that Dat did in
> [https://github.com/apache/lucene-solr/tree/jira/gradle] and took the ball a
> little further.
>
> When working with gradle in sub modules directly, I recommend
> [https://github.com/dougborg/gdub]
> This gradle branch uses the following plugin for version locking, version
> configuration and version consistency across modules:
> [https://github.com/palantir/gradle-consistent-versions]
>
> https://github.com/apache/lucene-solr/tree/jira/SOLR-13452_gradle_5
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13452) Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
[
https://issues.apache.org/jira/browse/SOLR-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16914216#comment-16914216
]
Tomoko Uchida edited comment on SOLR-13452 at 8/23/19 12:40 PM:
When I looked into inter-module dependency problems on IDEA, I noticed that it
seems difficult to manage to keep our current IDEA module structure with Gradle
IDEA plugin.
I discarded current structure (defined in {{.idea/modules.xml}}) and instead
made use of the default structure generated by the Gradle plugin. (Gradle IDEA
plugin works well for me and I think it's much better than maintaining ".iml"
files for all Lucene/Solr modules on our own.)
By this change IDEA users will see:
1. module names have been changed (e.g., "backward-codecs" is now
"lucene-backward-codecs", "extraction" is "solr-contrib-extraction", and so on)
2. main / test modules have been merged into one module (e.g., "lucene-core"
and "lucene-core-tests" are now packed into "lucene-core", "solr-core" and
"solr-core-tests" are packed into "solr-core", and so on).
And you might notice 3. IDEA complains some circular dependencies those
haven't been reported so far (they have been cleverly suppressed by customized
module structure), but daily development shouldn't be affected by this.
I think unused IDEA configuration files can be removed when we remove all other
Ant related files.
For me IDEA setup is 90% ready now... still not perfect, some
fixes/improvements are needed.
was (Author: tomoko uchida):
When I looked into inter-module dependency problems on IDEA, I noticed that it
seems difficult to manage to keep our current IDEA module structure with Gradle
IDEA plugin.
I discarded current IDEA module structure (defined in {{.idea/modules.xml}})
and instead made use of the default structure generated by the Gradle plugin.
(Gradle IDEA plugin works well for me and I think it's much better than
maintaining ".iml" files for all Lucene/Solr modules.)
By this change IDEA users will see that
1. module names have been changed (e.g., "backward-codecs" is now
"lucene-backward-codecs", "extraction" is "solr-contrib-extraction", and so on)
2. separated main / test modules have been merged into one module (e.g.,
"lucene-core" and "lucene-core-tests" are now packed into "lucene-core",
"solr-core" and "solr-core-tests" are packed into "solr-core", and so on).
And you might notice 3. IDEA complains some circular dependency problems those
haven't been reported so far, but daily development shouldn't be affected by
this.
For me IDEA setup is 90% ready now... still not perfect, some
fixes/improvements are needed.
> Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
> -
>
> Key: SOLR-13452
> URL: https://issues.apache.org/jira/browse/SOLR-13452
> Project: Solr
> Issue Type: Improvement
> Components: Build
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Major
> Fix For: master (9.0)
>
> Attachments: gradle-build.pdf
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I took some things from the great work that Dat did in
> [https://github.com/apache/lucene-solr/tree/jira/gradle] and took the ball a
> little further.
>
> When working with gradle in sub modules directly, I recommend
> [https://github.com/dougborg/gdub]
> This gradle branch uses the following plugin for version locking, version
> configuration and version consistency across modules:
> [https://github.com/palantir/gradle-consistent-versions]
>
> https://github.com/apache/lucene-solr/tree/jira/SOLR-13452_gradle_5
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13452) Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
[ https://issues.apache.org/jira/browse/SOLR-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913088#comment-16913088 ] Ignacio Vera edited comment on SOLR-13452 at 8/22/19 7:55 AM: -- I had a look to the implementation and I found that the new Lucene modules monitor and Luke seems not to be included in the gradle build. And a question: which is the gradle command similar to ant precommit? was (Author: ivera): I had a look to the implementation and I found that the new Lucene modules monitor and Luke seems not to be included in the cradle build. And a question: which is the griddle command similar to ant precommit? > Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle. > - > > Key: SOLR-13452 > URL: https://issues.apache.org/jira/browse/SOLR-13452 > Project: Solr > Issue Type: Improvement > Components: Build >Reporter: Mark Miller >Assignee: Mark Miller >Priority: Major > Fix For: master (9.0) > > Attachments: gradle-build.pdf > > Time Spent: 10m > Remaining Estimate: 0h > > I took some things from the great work that Dat did in > [https://github.com/apache/lucene-solr/tree/jira/gradle] and took the ball a > little further. > > When working with gradle in sub modules directly, I recommend > [https://github.com/dougborg/gdub] > This gradle branch uses the following plugin for version locking, version > configuration and version consistency across modules: > [https://github.com/palantir/gradle-consistent-versions] > > https://github.com/apache/lucene-solr/tree/jira/SOLR-13452_gradle_5 -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13452) Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
[ https://issues.apache.org/jira/browse/SOLR-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16861487#comment-16861487 ] Joel Bernstein edited comment on SOLR-13452 at 6/11/19 8:50 PM: I think this sounds legit. Solr uses a very limited set of Calcite features and the test cases cover all of them. If the test cases are passing is that enough to be certain that the production release will run correctly with the same features? [~krisden], what are your thoughts on this? was (Author: joel.bernstein): I think this sounds legit. Solr uses a very limited set of Calcite features and the test cases cover all of them. If the test cases are passing is that enough to be certain that the production release will run correctly with the same features? > Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle. > - > > Key: SOLR-13452 > URL: https://issues.apache.org/jira/browse/SOLR-13452 > Project: Solr > Issue Type: Improvement > Components: Build >Reporter: Mark Miller >Assignee: Mark Miller >Priority: Major > Fix For: master (9.0) > > > I took some things from the great work that Dat did in > [https://github.com/apache/lucene-solr/tree/jira/gradle] and took the ball a > little further. > > When working with gradle in sub modules directly, I recommend > [https://github.com/dougborg/gdub] > This gradle branch uses the following plugin for version locking, version > configuration and version consistency across modules: > [https://github.com/palantir/gradle-consistent-versions] > > https://github.com/apache/lucene-solr/tree/jira/SOLR-13452_gradle_3 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13452) Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
[
https://issues.apache.org/jira/browse/SOLR-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16857576#comment-16857576
]
Mark Miller edited comment on SOLR-13452 at 6/6/19 11:52 AM:
-
{quote}One think I noticed is that you use File constructor a lot
{quote}
Yeah, I'm still fumbling with groovy and gradle as a Java guy. I'll do a pass
and try to convert file stuff.
FYI, the new unusedDeps task will give a printout like the following (example
on solr-core):
{noformat}
> gw unusedDeps
Our classpath dependency count 101
Our directly used dependency count 76
List of possibly unused jars - they may be used at runtime however
(Class.forName or something), this is not definitive.
We take our classpath dependenies, substract our direct dependencies and then
subtract dependencies used by our direct dependencies
asm-analysis-6.2.jar
asm-tree-6.2.jar
commons-beanutils-1.9.3.jar
kerb-crypto-1.0.1.jar
kerby-config-1.0.1.jar
kerby-pkix-1.0.1.jar
kerby-util-1.0.1.jar
lucene-spatial-8.1.0.jar
org.restlet.ext.servlet-2.3.0.jar
{noformat}
was (Author: markrmil...@gmail.com):
bq. One think I noticed is that you use File constructor a lot
Yeah, I'm still fumbling with groovy and gradle as a Java guy. I'll do a pass
and try to convert file stuff.
FYI, the new unusedDeps task will give a printout like the following:
{noformat}
> gw unusedDeps
Our classpath dependency count 101
Our directly used dependency count 76
List of possibly unused jars - they may be used at runtime however
(Class.forName or something), this is not definitive.
We take our classpath dependenies, substract our direct dependencies and then
subtract dependencies used by our direct dependencies
asm-analysis-6.2.jar
asm-tree-6.2.jar
commons-beanutils-1.9.3.jar
kerb-crypto-1.0.1.jar
kerby-config-1.0.1.jar
kerby-pkix-1.0.1.jar
kerby-util-1.0.1.jar
lucene-spatial-8.1.0.jar
org.restlet.ext.servlet-2.3.0.jar
{noformat}
> Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
> -
>
> Key: SOLR-13452
> URL: https://issues.apache.org/jira/browse/SOLR-13452
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Build
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Major
> Fix For: master (9.0)
>
>
> I took some things from the great work that Dat did in
> [https://github.com/apache/lucene-solr/tree/jira/gradle] and took the ball a
> little further.
>
> When working with gradle in sub modules directly, I recommend
> [https://github.com/dougborg/gdub]
> This gradle branch uses the following plugin for version locking, version
> configuration and version consistency across modules:
> [https://github.com/palantir/gradle-consistent-versions]
>
> https://github.com/apache/lucene-solr/tree/jira/SOLR-13452_gradle_3
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13452) Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
[ https://issues.apache.org/jira/browse/SOLR-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16849137#comment-16849137 ] Mark Miller edited comment on SOLR-13452 at 5/27/19 7:08 PM: - bq. Both approaches have their pros and cons. No worries. Right, any project could go either way, so it has more to do with what the community will allow or go with and what I can manage to get bogged down with here. Technical issues aside, it's not easy to make a big change like this with so many developers on the project. One thing that could def put weight towards using transitive with excludes is if we don't get full version resolution without transitive. For example, if zookeeper says it wants commons-io 2.4 and we have transitive false, when we pick our commons-io version, does it still take that into account? [~dweiss], [~thetaphi], do you know? If we did't get proper version resolution, I'd really argue we should change how we do things sooner rather than later. was (Author: markrmil...@gmail.com): bq. Both approaches have their pros and cons. No worries. Right, any project could go either way, so it has more to do with what the community will allow or go with and what I can manage to get bogged down with her. Technical issues aside, it's not easy to make a big change like this with so many developers on the project. One thing that could def put weight towards using transitive with excludes is if we don't get full version resolution without transitive. For example, if zookeeper says it wants commons-io 2.4 and we have transitive false, when we pick our commons-io version, does it still take that into account? [~dweiss], [~thetaphi], do you know? If we did't get proper version resolution, I'd really argue we should change how we do things sooner rather than later. > Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle. > - > > Key: SOLR-13452 > URL: https://issues.apache.org/jira/browse/SOLR-13452 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Reporter: Mark Miller >Priority: Major > > I took some things from the great work that Dat did in > [https://github.com/apache/lucene-solr/tree/jira/gradle] and took the ball a > little further. > > When working with gradle in sub modules directly, I recommend > [https://github.com/dougborg/gdub] > This gradle branch uses the following plugin for version locking, version > configuration and version consistency across modules: > [https://github.com/palantir/gradle-consistent-versions] > > https://github.com/apache/lucene-solr/tree/jira/SOLR-13452_gradle_2 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13452) Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
[
https://issues.apache.org/jira/browse/SOLR-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16845411#comment-16845411
]
Varun Thacker edited comment on SOLR-13452 at 5/22/19 1:39 AM:
---
When i run {{./gradlew runjflex}} StandardTokenizerImpl has a compile time
error - I believe we need to port this hack
[https://github.com/apache/lucene-solr/blob/c756b50ae4324b5692b2956c7a5d484122ac3016/lucene/common-build.xml#L2575]
over? I'll try to address this and post back.
I knew about this specifically because I recently ran into this while trying to
build a custom tokenizer
Just changing the skeleton.default file and removing the final modifier from
ZZ_BUFFERSIZE didn't work . It hadn't worked for me previously as well. I'll
look at it when I'm back in a few hours
was (Author: varunthacker):
When i run {{./gradlew runjflex}} StandardTokenizerImpl has a compile time
error - I believe we need to port this hack
[https://github.com/apache/lucene-solr/blob/c756b50ae4324b5692b2956c7a5d484122ac3016/lucene/common-build.xml#L2575]
over? I'll try to address this and post back.
I knew about this specifically because I recently ran into this while trying to
build a custom tokenizer
> Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
> -
>
> Key: SOLR-13452
> URL: https://issues.apache.org/jira/browse/SOLR-13452
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Build
>Reporter: Mark Miller
>Priority: Major
>
> I took some things from the great work that Dat did in
> [https://github.com/apache/lucene-solr/tree/jira/gradle] and took the ball a
> little further.
>
> When working with gradle in sub modules directly, I recommend
> [https://github.com/dougborg/gdub]
> This gradle branch uses the following plugin for version locking, version
> configuration and version consistency across modules:
> [https://github.com/palantir/gradle-consistent-versions]
> By default, dependencies are not transitive, but there is a special
> Configuration for adding dependencies on other project internal modules that
> are transitive to their direct external dependencies (their jar libs).
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13452) Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
[
https://issues.apache.org/jira/browse/SOLR-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16845411#comment-16845411
]
Varun Thacker edited comment on SOLR-13452 at 5/22/19 1:31 AM:
---
When i run {{./gradlew runjflex}} StandardTokenizerImpl has a compile time
error - I believe we need to port this hack
[https://github.com/apache/lucene-solr/blob/c756b50ae4324b5692b2956c7a5d484122ac3016/lucene/common-build.xml#L2575]
over? I'll try to address this and post back.
I knew about this specifically because I recently ran into this while trying to
build a custom tokenizer
was (Author: varunthacker):
When i run {{./gradlew runjflex}} this the StandardTokenizerImpl has a compile
time error - I believe we need to port this hack
[https://github.com/apache/lucene-solr/blob/c756b50ae4324b5692b2956c7a5d484122ac3016/lucene/common-build.xml#L2575]
over? I'll try to address this and post back.
I knew about this specifically because I recently ran into this while trying to
build a custom tokenizer
> Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
> -
>
> Key: SOLR-13452
> URL: https://issues.apache.org/jira/browse/SOLR-13452
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Build
>Reporter: Mark Miller
>Priority: Major
>
> I took some things from the great work that Dat did in
> [https://github.com/apache/lucene-solr/tree/jira/gradle] and took the ball a
> little further.
>
> When working with gradle in sub modules directly, I recommend
> [https://github.com/dougborg/gdub]
> This gradle branch uses the following plugin for version locking, version
> configuration and version consistency across modules:
> [https://github.com/palantir/gradle-consistent-versions]
> By default, dependencies are not transitive, but there is a special
> Configuration for adding dependencies on other project internal modules that
> are transitive to their direct external dependencies (their jar libs).
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13452) Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
[ https://issues.apache.org/jira/browse/SOLR-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16844245#comment-16844245 ] Mark Miller edited comment on SOLR-13452 at 5/20/19 8:59 PM: - The above commit let's you test the build using your local check out in docker. This is very basic at the moment and just runs a few targets. By doing this in docker we can control various isolation levels, a clean dependency cache environment and can install certain target pre-requisites like python. We could shorten the dependency gathering phase with an apt cache image or by sharing local caches with Docker if wanted to. The test is in buildSrc and implemented as a new target called buildTest (I have not tested that much yet, just running from eclipse directly so far). It's implemented as a Java junit test that calls out to bash. A test script is run and configured to fail the junit test if any command fails. The test has an assume that basically checks for unix and docker command availability. was (Author: markrmil...@gmail.com): The above commit let's you test the build using your local check out in docker. This is very basic at the moment and just runs a few targets. By doing this in docker we can control various isolation levels, a clean dependency cache environment and can install certain target pre-requisites like python. We could shorten the dependency gathering phase with an apt cache image or by sharing local caches with Docker if wanted to. The test is in buildSrc and implemented as a new target called testBuild (I have not tested that much yet, just running from eclipse directly so far). It's implemented as a Java junit test that calls out to bash. A test script is run and configured to fail the junit test if any command fails. The test has an assume that basically checks for unix and docker command availability. > Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle. > - > > Key: SOLR-13452 > URL: https://issues.apache.org/jira/browse/SOLR-13452 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Reporter: Mark Miller >Priority: Major > > I took some things from the great work that Dat did in > [https://github.com/apache/lucene-solr/tree/jira/gradle] and took the ball a > little further. > > When working with gradle in sub modules directly, I recommend > [https://github.com/dougborg/gdub] > This gradle branch uses the following plugin for version locking, version > configuration and version consistency across modules: > [https://github.com/palantir/gradle-consistent-versions] > By default, dependencies are not transitive, but there is a special > Configuration for adding dependencies on other project internal modules that > are transitive to their direct external dependencies (their jar libs). > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13452) Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
[
https://issues.apache.org/jira/browse/SOLR-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843252#comment-16843252
]
Mark Miller edited comment on SOLR-13452 at 5/18/19 9:43 PM:
-
Did some quick testing to ensure we are not regressing in multi-module build
performance:
{noformat}
with deps cached and warmed up (ant cmd run more than once and gradle daemon on)
ant clean compile-test
Total time: 1 minute 35 seconds
gw clean compileTestJava
BUILD SUCCESSFUL in 27s
cold run with no deps cached (gradle daemon on)
ant clean compile-test
Total time: 7 minutes 19 seconds
gw clean compileTestJava
BUILD SUCCESSFUL in 1m 7s
{noformat}
was (Author: markrmil...@gmail.com):
Did some quick testing to ensure we are not regressing in multi-module build
performance:
{noformat}
with deps cached and warmed up (ant cmd run more than once and gradle daemon on)
ant clean compile-java
Total time: 1 minute 35 seconds
gw clean compileTestJava
BUILD SUCCESSFUL in 27s
cold run with no deps cached (gradle daemon on)
ant clean compile-test
Total time: 7 minutes 19 seconds
gw clean compileTestJava
BUILD SUCCESSFUL in 1m 7s
{noformat}
> Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
> -
>
> Key: SOLR-13452
> URL: https://issues.apache.org/jira/browse/SOLR-13452
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Build
>Reporter: Mark Miller
>Priority: Major
>
> I took some things from the great work that Dat did in
> [https://github.com/apache/lucene-solr/tree/jira/gradle] and took the ball a
> little further.
>
> When working with gradle in sub modules directly, I recommend
> [https://github.com/dougborg/gdub]
> This gradle branch uses the following plugin for version locking, version
> configuration and version consistency across modules:
> [https://github.com/palantir/gradle-consistent-versions]
> By default, dependencies are not transitive, but there is a special
> Configuration for adding dependencies on other project internal modules that
> are transitive to their direct external dependencies (their jar libs).
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13452) Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
[
https://issues.apache.org/jira/browse/SOLR-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16840817#comment-16840817
]
Uwe Schindler edited comment on SOLR-13452 at 5/15/19 10:23 PM:
Hi Mark,
I am working on forbidden-apis but there are few additional problems:
(1) First you added an {{include 'build/src'}} in settings.gradle, although
this is not needed. The result of this is that the buildSrc dir is treated as a
separate subproject, so all the configurations are applied to it, too. This
should not be done. I removed it, project still compiles. If you keep the
include, you break the build plugin because it uses itsself to build itsself.
This fails e.g., forbiddenapis. On the other hand, it might be good to run
"rat" on it. But this should be configured in buildSrc directly.
Another option is to give the build source a separate name and exclude it from
the top-level config inside "subprojects" config. This is how Elasticsearch
does it.
(2) There is a problem with the "compileOnly" dependency in solr-core. As it's
compile only its also not exported to projects relying on it. As forbiddenapis
hardcodes it as a signatures file, it wont work. You already excluded solrj,
but it also affect other Solr modules that does not have it. IMHO, I'd change
the configuration to only exclude servlet-api.txt sgnatures if the servlet-api
compileOnly dependency is there.
(3) commons-io is missing. We need some trick to get the version number, but
that's also easy: (like in 2, ask the configuration for the version).
(4) System.out checking is also missing
I fixed (1) locally and working on (2). Once this works, I will look into the
test excldudes for the forked Hadoop test classes. That should be easy (just
change the forbiddenApisTest task in the Solr Core module and exclude
directories).
was (Author: thetaphi):
Hi Mark,
I am working on forbidden-apis but there are 2 additional problems:
(1) First you added an {{include 'build/src'}} in settings.gradle, although
this is not needed. The result of this is that the buildSrc dir is treated as a
separate subproject, so all the configurations are applied to it, too. This
should not be done. I removed it, project still compiles. If you keep the
include, you break the build plugin because it uses itsself to build itsself.
This fails e.g., forbiddenapis. On the other hand, it might be good to run
"rat" on it. But this should be configured in buildSrc directly.
Another option is to give the build source a separate name and exclude it from
the top-level config inside "subprojects" config. This is how Elasticsearch
does it.
(2) There is a problem with the "compileOnly" dependency in solr-core. As it's
compile only its also not exported to projects relying on it. As forbiddenapis
hardcodes it as a signatures file, it wont work. You already excluded solrj,
but it also affect other Solr modules that does not have it. IMHO, I'd change
the configuration to only exclude servlet-api.txt sgnatures if the servlet-api
compileOnly dependency is there.
(3) commons-io is missing. We need some trick to get the version number, but
that's also easy: (like in 2, ask the configuration for the version).
I fixed (1) locally and working on (2). Once this works, I will look into the
test excldudes for the forked Hadoop test classes. That should be easy (just
change the forbiddenApisTest task in the Solr Core module and exclude
directories).
> Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
> -
>
> Key: SOLR-13452
> URL: https://issues.apache.org/jira/browse/SOLR-13452
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Build
>Reporter: Mark Miller
>Priority: Major
>
> I took some things from the great work that Dat did in
> [https://github.com/apache/lucene-solr/tree/jira/gradle] and took the ball a
> little further.
>
> When working with gradle in sub modules directly, I recommend
> [https://github.com/dougborg/gdub]
> This gradle branch uses the following plugin for version locking, version
> configuration and version consistency across modules:
> [https://github.com/palantir/gradle-consistent-versions]
> By default, dependencies are not transitive, but there is a special
> Configuration for adding dependencies on other project internal modules that
> are transitive to their direct external dependencies (their jar libs).
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13452) Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
[ https://issues.apache.org/jira/browse/SOLR-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16840459#comment-16840459 ] Mark Miller edited comment on SOLR-13452 at 5/15/19 2:24 PM: - I think Groovy seems more widely known and used with Gradle currently and one of the primary motivations here is to use something that most devs will be familiar with and that has great support and tooling. If that changes, we would/could consider moving in the future I would imagine. was (Author: markrmil...@gmail.com): I think Goovy seems more widely known and used with Gradle currently and one of the primary motivations here is to use something that most devs will be familiar with and that has great support and tooling. If that changes, we would/cloud consider moving in the future I would imagine. > Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle. > - > > Key: SOLR-13452 > URL: https://issues.apache.org/jira/browse/SOLR-13452 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Build >Reporter: Mark Miller >Priority: Major > > I took some things from the great work that Dat did in > [https://github.com/apache/lucene-solr/tree/jira/gradle] and took the ball a > little further. > > When working with gradle in sub modules directly, I recommend > [https://github.com/dougborg/gdub] > This gradle branch uses the following plugin for version locking, version > configuration and version consistency across modules: > [https://github.com/palantir/gradle-consistent-versions] > By default, dependencies are not transitive, but there is a special > Configuration for adding dependencies on other project internal modules that > are transitive to their direct external dependencies (their jar libs). > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13452) Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
[
https://issues.apache.org/jira/browse/SOLR-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16837747#comment-16837747
]
Tomoko Uchida edited comment on SOLR-13452 at 5/11/19 3:43 AM:
---
Hi [~markrmil...@gmail.com],
thank you for working for this.
{quote}I got eclipse working fairly well for me, I don't use idea though, so I
imagine some work there.
{quote}
I use IntelliJ IDEA. I think I can help with this after all work is done.
(Please give a mention to me.)
was (Author: tomoko uchida):
Hi [~markrmil...@gmail.com],
thank you for working this.
bq. I got eclipse working fairly well for me, I don't use idea though, so I
imagine some work there.
I use IntelliJ IDEA. I think I can help with this after all work is done.
(Please give a mention to me.)
> Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
> -
>
> Key: SOLR-13452
> URL: https://issues.apache.org/jira/browse/SOLR-13452
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Build
>Reporter: Mark Miller
>Priority: Major
>
> I took some things from the great work that Dat did in
> https://github.com/apache/lucene-solr/tree/jira/gradle and took the ball a
> little further.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
