[jira] [Commented] (LUCENE-7061) fix remaining api issues with XYZPoint classes
[
https://issues.apache.org/jira/browse/LUCENE-7061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15199426#comment-15199426
]
Matt Davis commented on LUCENE-7061:
+1 and -1 for start and end exclusive for Int and Long makes sense but what
would be the pattern for Float and Double?
> fix remaining api issues with XYZPoint classes
> --
>
> Key: LUCENE-7061
> URL: https://issues.apache.org/jira/browse/LUCENE-7061
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Robert Muir
> Fix For: master, 6.0
>
> Attachments: LUCENE-7061.patch
>
>
> There are still some major problems today:
> XYZPoint.newRangeQuery has "brain damage" from variable length terms. nulls
> for open ranges make no sense: these are fixed-width types and instead you
> can use things like Integer.maxValue. Removing the nulls is safe, as we can
> just switch to primitive types instead of boxed types.
> XYZPoint.newRangeQuery requires boolean arrays for inclusive/exclusive, but
> thats just more brain damage. If you want to exclude an integer, you just
> subtract 1 from it and other simple stuff.
> For the apis, this means Instead of:
> {code}
> public static Query newRangeQuery(String field, Long lowerValue, boolean
> lowerInclusive, Long upperValue, boolean upperInclusive);
>
> public static Query newMultiRangeQuery(String field, Long[] lowerValue,
> boolean lowerInclusive[], Long[] upperValue, boolean upperInclusive[]);
> {code}
> we have:
> {code}
> public static Query newRangeQuery(String field, long lowerValue, long
> upperValue);
> public static Query newRangeQuery(String field, long[] lowerValue, long[]
> upperValue);
> {code}
> PointRangeQuery is horribly complex due to these nulls and boolean arrays,
> and need not be. Now it only works "inclusive" and is much simpler.
> XYZPoint.newSetQuery throws IOException, just creating the query. This is
> very confusing and unnecessary (no i/o happens).
> LatLonPoint's bounding box query is not inclusive like the other geo. And the
> test does not fail!
> I also found a few missing checks here and there while cleaning up.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6990) Mongo 3.x Directory support
[ https://issues.apache.org/jira/browse/LUCENE-6990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127504#comment-15127504 ] Matt Davis commented on LUCENE-6990: https://github.com/lumongo/lumongo/wiki/Distributed-Directory > Mongo 3.x Directory support > --- > > Key: LUCENE-6990 > URL: https://issues.apache.org/jira/browse/LUCENE-6990 > Project: Lucene - Core > Issue Type: Improvement > Components: core/store >Affects Versions: 5.4 >Reporter: Michele Renda >Priority: Minor > > I would like to setup the lucene store to a MongoDB installation. > It already exists an old project which gives this possiblity: > https://github.com/rstiller/mongo-lucene > Unfortunately this project is stuck to the old version of Mondo DB and does > not run anymore. Newest version of MongoDB now uses new classes like > GridFSBuckets and MongoDatabase (replacing the old classes GridFS and > MongoDB). > Here an example of usage: > http://mongodb.github.io/mongo-java-driver/3.2/driver/reference/gridfs/ > Can you please implement a MongoDBDirectory? > Because I am not sure which team (Lucene or MongoDB one) can better take care > of this request, I am opening a copy of this request to the other team too > (see external issue URL). > Best regards -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6990) Mongo 3.x Directory support
[ https://issues.apache.org/jira/browse/LUCENE-6990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127500#comment-15127500 ] Matt Davis commented on LUCENE-6990: http://lumongo.org/ > Mongo 3.x Directory support > --- > > Key: LUCENE-6990 > URL: https://issues.apache.org/jira/browse/LUCENE-6990 > Project: Lucene - Core > Issue Type: Improvement > Components: core/store >Affects Versions: 5.4 >Reporter: Michele Renda >Priority: Minor > > I would like to setup the lucene store to a MongoDB installation. > It already exists an old project which gives this possiblity: > https://github.com/rstiller/mongo-lucene > Unfortunately this project is stuck to the old version of Mondo DB and does > not run anymore. Newest version of MongoDB now uses new classes like > GridFSBuckets and MongoDatabase (replacing the old classes GridFS and > MongoDB). > Here an example of usage: > http://mongodb.github.io/mongo-java-driver/3.2/driver/reference/gridfs/ > Can you please implement a MongoDBDirectory? > Because I am not sure which team (Lucene or MongoDB one) can better take care > of this request, I am opening a copy of this request to the other team too > (see external issue URL). > Best regards -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6664) Replace SynonymFilter with SynonymGraphFilter
[
https://issues.apache.org/jira/browse/LUCENE-6664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934080#comment-14934080
]
Matt Davis commented on LUCENE-6664:
Could this be a candidate for 6.0 (breaking back-compat)?
> Replace SynonymFilter with SynonymGraphFilter
> -
>
> Key: LUCENE-6664
> URL: https://issues.apache.org/jira/browse/LUCENE-6664
> Project: Lucene - Core
> Issue Type: New Feature
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Attachments: LUCENE-6664.patch, LUCENE-6664.patch, LUCENE-6664.patch,
> LUCENE-6664.patch, usa.png, usa_flat.png
>
>
> Spinoff from LUCENE-6582.
> I created a new SynonymGraphFilter (to replace the current buggy
> SynonymFilter), that produces correct graphs (does no "graph
> flattening" itself). I think this makes it simpler.
> This means you must add the FlattenGraphFilter yourself, if you are
> applying synonyms during indexing.
> Index-time syn expansion is a necessarily "lossy" graph transformation
> when multi-token (input or output) synonyms are applied, because the
> index does not store {{posLength}}, so there will always be phrase
> queries that should match but do not, and then phrase queries that
> should not match but do.
> http://blog.mikemccandless.com/2012/04/lucenes-tokenstreams-are-actually.html
> goes into detail about this.
> However, with this new SynonymGraphFilter, if instead you do synonym
> expansion at query time (and don't do the flattening), and you use
> TermAutomatonQuery (future: somehow integrated into a query parser),
> or maybe just "enumerate all paths and make union of PhraseQuery", you
> should get 100% correct matches (not sure about "proper" scoring
> though...).
> This new syn filter still cannot consume an arbitrary graph.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5755) Explore alternative build systems
[ https://issues.apache.org/jira/browse/LUCENE-5755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14050971#comment-14050971 ] Matt Davis commented on LUCENE-5755: Sample forbidden api ant called from gradle, not sure it is good form but it does work for a trivial example: https://gist.github.com/mdavis95/94217e972c879d430028 Explore alternative build systems - Key: LUCENE-5755 URL: https://issues.apache.org/jira/browse/LUCENE-5755 Project: Lucene - Core Issue Type: Task Reporter: Dawid Weiss Assignee: Dawid Weiss Priority: Minor I am dissatisfied with how ANT and submodules currently work in Lucene/ Solr. It's not even the tool's fault; it seems Lucene builds just hit the borders of what it can do, especially in terms of submodule dependencies etc. I don't think Maven will help much too, given certain things I'd like to have in the build (for example collect all tests globally for a single execution phase at the end of the build, to support better load-balancing). I'd like to explore Gradle as an alternative. This task is a notepad for thoughts and experiments. An example of a complex (?) gradle build is javafx, for example. http://hg.openjdk.java.net/openjfx/8/master/rt/file/f89b7dc932af/build.gradle -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5755) Explore alternative build systems
[
https://issues.apache.org/jira/browse/LUCENE-5755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14049094#comment-14049094
]
Matt Davis commented on LUCENE-5755:
Gradle does use which ever junit version you specific. For example:
dependencies {
testCompile 'junit:junit:[4.10,)'
}
And because of the annotation RunWith(RandomizedRunner.class) and the
configured dependency to
'com.carrotsearch.randomizedtesting:randomizedtesting-runner:2.1.3' it uses the
RandomizedRunner from CarrotSearch. As I said before it also launches a
separate JVM. The actually issue is that it switches the system.out before
each test to capture the output. See the following classes:
https://github.com/gradle/gradle/blob/master/subprojects/plugins/src/main/groovy/org/gradle/api/internal/tasks/testing/processors/CaptureTestOutputTestResultProcessor.java
https://github.com/gradle/gradle/blob/master/subprojects/plugins/src/main/groovy/org/gradle/api/internal/tasks/testing/junit/JUnitTestClassProcessor.java
https://github.com/gradle/gradle/blob/master/subprojects/plugins/src/main/groovy/org/gradle/api/internal/tasks/testing/junit/JULRedirector.java
https://github.com/gradle/gradle/blob/master/subprojects/core/src/main/groovy/org/gradle/logging/internal/DefaultStandardOutputRedirector.java
https://github.com/gradle/gradle/blob/master/subprojects/plugins/src/main/groovy/org/gradle/api/internal/tasks/testing/junit/JUnitTestFramework.java
https://github.com/gradle/gradle/blob/master/subprojects/plugins/src/main/groovy/org/gradle/api/tasks/testing/Test.java
If I disable the System.out check
(https://github.com/mdavis95/lucene-solr/commit/1b8798894e9f2c52b905eb5f259ef4108bb0b6d7),
and allow lucene benchmark to find alg files through a hack
(https://github.com/mdavis95/lucene-solr/commit/6be7c4a3b68fdc9ee749674d101a1c75b2f725be).
A whole lot of tests start to past.
For the lucene core I have 783 test successfull, 518 skipped, and 5 failures.
The only 5 that failed were in TestVirtualMethod.
java.lang.RuntimeException: Suite class
org.apache.lucene.util.TestVirtualMethod$TestClass1 should be public.
Explore alternative build systems
-
Key: LUCENE-5755
URL: https://issues.apache.org/jira/browse/LUCENE-5755
Project: Lucene - Core
Issue Type: Task
Reporter: Dawid Weiss
Assignee: Dawid Weiss
Priority: Minor
I am dissatisfied with how ANT and submodules currently work in Lucene/ Solr.
It's not even the tool's fault; it seems Lucene builds just hit the borders
of what it can do, especially in terms of submodule dependencies etc.
I don't think Maven will help much too, given certain things I'd like to have
in the build (for example collect all tests globally for a single execution
phase at the end of the build, to support better load-balancing).
I'd like to explore Gradle as an alternative. This task is a notepad for
thoughts and experiments.
An example of a complex (?) gradle build is javafx, for example.
http://hg.openjdk.java.net/openjfx/8/master/rt/file/f89b7dc932af/build.gradle
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5755) Explore alternative build systems
[ https://issues.apache.org/jira/browse/LUCENE-5755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14049245#comment-14049245 ] Matt Davis commented on LUCENE-5755: Understood. I was not aware of Junit4. I thought you meant Junit version 4. Gradle does support a few things along those the lines of parallelism: http://www.gradle.org/docs/current/dsl/org.gradle.api.tasks.testing.Test.html maxParallelForks The maximum number of forked test processes to execute in parallel. The default value is 1 (no parallel test execution). forkEvery The maximum number of test classes to execute in a forked test process. The forked test process will be restarted when this limit is reached. The default value is 0 (no maximum). It however doesn't isolate the working directories, doesn't have global seeding, or load balancing as far as I know. Seems to me that it would be good to push these features upstream into gradle because they sound useful to other projects but it is outside my ability. Filtering inner clases worked for me. Thanks. https://github.com/mdavis95/lucene-solr/commit/53217bccff2a14efa58951d8b3c0d20635f6 Explore alternative build systems - Key: LUCENE-5755 URL: https://issues.apache.org/jira/browse/LUCENE-5755 Project: Lucene - Core Issue Type: Task Reporter: Dawid Weiss Assignee: Dawid Weiss Priority: Minor I am dissatisfied with how ANT and submodules currently work in Lucene/ Solr. It's not even the tool's fault; it seems Lucene builds just hit the borders of what it can do, especially in terms of submodule dependencies etc. I don't think Maven will help much too, given certain things I'd like to have in the build (for example collect all tests globally for a single execution phase at the end of the build, to support better load-balancing). I'd like to explore Gradle as an alternative. This task is a notepad for thoughts and experiments. An example of a complex (?) gradle build is javafx, for example. http://hg.openjdk.java.net/openjfx/8/master/rt/file/f89b7dc932af/build.gradle -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5755) Explore alternative build systems
[ https://issues.apache.org/jira/browse/LUCENE-5755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14049383#comment-14049383 ] Matt Davis commented on LUCENE-5755: I have all Lucene test case compiling on my fork with the system out check suppressed. Solr is untouched and javacc. I am using transitive dependencies but that is an easy fix. I disagree that these features are not features that would useful for gradle to implement natively. The following seem pretty useful in general but I will let someone from gradle decide that: 1)We use a special SecurityManager that also prevents to escape the test's working dir or to prevent that a tests calls System.halt(). This security manager relies on the Junit4-Runner. 2)Also the runner not only parallelizes, it also keeps statistics about the performance of tests to reorder them on next run, so slower tests don't leave the last JVM run longer just because the running tests take too long 3)Another important thing is that the runner randomizes the test execution order, which is important to prevent bugs that are caused by tests leaving state that influence others. Things like forbidden-apis make sense to call from ant tasks but it is easy to make a gradle plugin out of them. As a side note, I think making eclipse projects with gradle is going to be problem unless i am missing something. lucene-core test depends on lucene-test-framework and lucene-test-framework depends back on lucene-core. I will leave this to others that are more knowledgeable about lucene to continue. Explore alternative build systems - Key: LUCENE-5755 URL: https://issues.apache.org/jira/browse/LUCENE-5755 Project: Lucene - Core Issue Type: Task Reporter: Dawid Weiss Assignee: Dawid Weiss Priority: Minor I am dissatisfied with how ANT and submodules currently work in Lucene/ Solr. It's not even the tool's fault; it seems Lucene builds just hit the borders of what it can do, especially in terms of submodule dependencies etc. I don't think Maven will help much too, given certain things I'd like to have in the build (for example collect all tests globally for a single execution phase at the end of the build, to support better load-balancing). I'd like to explore Gradle as an alternative. This task is a notepad for thoughts and experiments. An example of a complex (?) gradle build is javafx, for example. http://hg.openjdk.java.net/openjfx/8/master/rt/file/f89b7dc932af/build.gradle -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5755) Explore alternative build systems
[ https://issues.apache.org/jira/browse/LUCENE-5755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045920#comment-14045920 ] Matt Davis commented on LUCENE-5755: I have managed to get all lucene modules to compile under gradle. All test cases fail due to the system out issue. It will compile successfully with ./gradlew build -x test but will fail with a ./gradlew build due to the test cases. It is not running javacc either but my guess is that this would due the trick https://github.com/RobertFischer/gradle-javacc-plugin. I have not validated that the output it is generating is at all valid. Solr modules are not compiling yet either. If anyone wants to play around with it: https://github.com/apache/lucene-solr/pull/59 Explore alternative build systems - Key: LUCENE-5755 URL: https://issues.apache.org/jira/browse/LUCENE-5755 Project: Lucene - Core Issue Type: Task Reporter: Dawid Weiss Assignee: Dawid Weiss Priority: Minor I am dissatisfied with how ANT and submodules currently work in Lucene/ Solr. It's not even the tool's fault; it seems Lucene builds just hit the borders of what it can do, especially in terms of submodule dependencies etc. I don't think Maven will help much too, given certain things I'd like to have in the build (for example collect all tests globally for a single execution phase at the end of the build, to support better load-balancing). I'd like to explore Gradle as an alternative. This task is a notepad for thoughts and experiments. An example of a complex (?) gradle build is javafx, for example. http://hg.openjdk.java.net/openjfx/8/master/rt/file/f89b7dc932af/build.gradle -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5755) Explore alternative build systems
[ https://issues.apache.org/jira/browse/LUCENE-5755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042068#comment-14042068 ] Matt Davis commented on LUCENE-5755: As far as I can tell gradle alway runs tests in a separate JVM (http://www.gradle.org/docs/current/javadoc/org/gradle/api/tasks/testing/Test.html) but maybe I am missing something here. Unfortunately, I do not have a Gradle build for Lucene. I work a project storing Lucene indexes into MongoDB and test the MongoDB Lucene directory with a handleful of Lucene test cases and this is how I noticed the issue. I don't think anything I have done would be too much help other than maybe the publish to maven (https://github.com/lumongo/lumongo/blob/master/maven.gradle) which turned out to be more of a pain that I thought. I am happy to help / give a port a try if wanted. Explore alternative build systems - Key: LUCENE-5755 URL: https://issues.apache.org/jira/browse/LUCENE-5755 Project: Lucene - Core Issue Type: Task Reporter: Dawid Weiss Assignee: Dawid Weiss Priority: Minor I am dissatisfied with how ANT and submodules currently work in Lucene/ Solr. It's not even the tool's fault; it seems Lucene builds just hit the borders of what it can do, especially in terms of submodule dependencies etc. I don't think Maven will help much too, given certain things I'd like to have in the build (for example collect all tests globally for a single execution phase at the end of the build, to support better load-balancing). I'd like to explore Gradle as an alternative. This task is a notepad for thoughts and experiments. An example of a complex (?) gradle build is javafx, for example. http://hg.openjdk.java.net/openjfx/8/master/rt/file/f89b7dc932af/build.gradle -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5755) Explore alternative build systems
[ https://issues.apache.org/jira/browse/LUCENE-5755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041557#comment-14041557 ] Matt Davis commented on LUCENE-5755: I get errors due to the System.out checks compiling Lucene tests cases due to Gradle stealing System.out in my project: java.lang.AssertionError: Something has changed System.out to: org.gradle.util.LinePerThreadBufferingOutputStream Explore alternative build systems - Key: LUCENE-5755 URL: https://issues.apache.org/jira/browse/LUCENE-5755 Project: Lucene - Core Issue Type: Task Reporter: Dawid Weiss Assignee: Dawid Weiss Priority: Minor I am dissatisfied with how ANT and submodules currently work in Lucene/ Solr. It's not even the tool's fault; it seems Lucene builds just hit the borders of what it can do, especially in terms of submodule dependencies etc. I don't think Maven will help much too, given certain things I'd like to have in the build (for example collect all tests globally for a single execution phase at the end of the build, to support better load-balancing). I'd like to explore Gradle as an alternative. This task is a notepad for thoughts and experiments. An example of a complex (?) gradle build is javafx, for example. http://hg.openjdk.java.net/openjfx/8/master/rt/file/f89b7dc932af/build.gradle -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-5755) Explore alternative build systems
[ https://issues.apache.org/jira/browse/LUCENE-5755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041557#comment-14041557 ] Matt Davis edited comment on LUCENE-5755 at 6/24/14 1:07 AM: - I get errors due to the System.out checks compiling Lucene tests cases due to Gradle stealing System.out in my project: java.lang.AssertionError: Something has changed System.out to: org.gradle.util.LinePerThreadBufferingOutputStream http://forums.gradle.org/gradle/topics/capturing_output_in_unit_test_and_verifying_it was (Author: mdavis95): I get errors due to the System.out checks compiling Lucene tests cases due to Gradle stealing System.out in my project: java.lang.AssertionError: Something has changed System.out to: org.gradle.util.LinePerThreadBufferingOutputStream Explore alternative build systems - Key: LUCENE-5755 URL: https://issues.apache.org/jira/browse/LUCENE-5755 Project: Lucene - Core Issue Type: Task Reporter: Dawid Weiss Assignee: Dawid Weiss Priority: Minor I am dissatisfied with how ANT and submodules currently work in Lucene/ Solr. It's not even the tool's fault; it seems Lucene builds just hit the borders of what it can do, especially in terms of submodule dependencies etc. I don't think Maven will help much too, given certain things I'd like to have in the build (for example collect all tests globally for a single execution phase at the end of the build, to support better load-balancing). I'd like to explore Gradle as an alternative. This task is a notepad for thoughts and experiments. An example of a complex (?) gradle build is javafx, for example. http://hg.openjdk.java.net/openjfx/8/master/rt/file/f89b7dc932af/build.gradle -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
