Re: revisit payloads API in DocsAndPositionsEnum

2012-08-11 Thread Shai Erera
I agree Shai On Sun, Aug 12, 2012 at 9:18 AM, Robert Muir wrote: > That would definitely be separate. I looked into this: the problem is > things like LUCENE-4219. > > So the current behavior for payload-using span queries (at least > span-near with payloads) is wrong, they score differently de

Re: revisit payloads API in DocsAndPositionsEnum

2012-08-11 Thread Robert Muir
That would definitely be separate. I looked into this: the problem is things like LUCENE-4219. So the current behavior for payload-using span queries (at least span-near with payloads) is wrong, they score differently depending upon whether you next() or advance() them (which is horrible!), so I c

Re: revisit payloads API in DocsAndPositionsEnum

2012-08-11 Thread Shai Erera
Looks good. Perhaps separately, what do you think about doing the same to Spans.isPayloadAvailable/getPayload? Shai On Sun, Aug 12, 2012 at 8:56 AM, Robert Muir wrote: > Here's a patch: http://pastebin.com/d2DdWxJp > > On Sat, Aug 11, 2012 at 1:35 PM, Simon Willnauer > wrote: > > +1 this make

Re: revisit payloads API in DocsAndPositionsEnum

2012-08-11 Thread Robert Muir
Here's a patch: http://pastebin.com/d2DdWxJp On Sat, Aug 11, 2012 at 1:35 PM, Simon Willnauer wrote: > +1 this makes lots of sense > > simon > > On Sat, Aug 11, 2012 at 7:28 PM, Michael McCandless > wrote: >> +1 >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >> >> On Sat, Aug 11,

Re: remove FieldsEnum?

2012-08-11 Thread Shai Erera
Ah I see. Shai On Sun, Aug 12, 2012 at 8:32 AM, Robert Muir wrote: > JIRA has been down for a while now. We can't just stop development > because of this: personally I will just continue working and sending > patches to the list for review. > > If someone wants, they can create JIRAs for these

Re: remove FieldsEnum?

2012-08-11 Thread Robert Muir
JIRA has been down for a while now. We can't just stop development because of this: personally I will just continue working and sending patches to the list for review. If someone wants, they can create JIRAs for these things after the fact: but I'm not letting it get in my way. On Sun, Aug 12, 20

Re: remove FieldsEnum?

2012-08-11 Thread Shai Erera
Is there a JIRA issue for it? The patch is large and the issue seems important enough to be handled through JIRA? Shai On Sun, Aug 12, 2012 at 4:53 AM, Robert Muir wrote: > Here's a patch: http://pastebin.com/PQGSgBtT > > On Sat, Aug 11, 2012 at 9:29 AM, Michael McCandless > wrote: > > +1 > >

Re: remove FieldsEnum?

2012-08-11 Thread Robert Muir
Here's a patch: http://pastebin.com/PQGSgBtT On Sat, Aug 11, 2012 at 9:29 AM, Michael McCandless wrote: > +1 > > Mike McCandless > > http://blog.mikemccandless.com > > > On Sat, Aug 11, 2012 at 9:27 AM, Robert Muir wrote: >> FieldsEnum seems kinda awkward, can we just have something like >> Iter

Jenkins build is back to normal : Lucene-trunk-Linux-Java7-64-test-only #1443

2012-08-11 Thread builder
See - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org

Build failed in Jenkins: Lucene-trunk-Linux-Java7-64-test-only #1442

2012-08-11 Thread builder
See -- [...truncated 908 lines...] [junit4:junit4] Suite: org.apache.lucene.util.TestUnicodeUtil [junit4:junit4] Completed on J1 in 0.99s, 4 tests [junit4:junit4] [junit4:junit4] Suite: org.apache.lucene.index.TestDocumentWriter [junit4:junit4] Completed

[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.7.0_05) - Build # 202 - Failure!

2012-08-11 Thread Policeman Jenkins Server
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Windows/202/ Java: 32bit/jdk1.7.0_05 -client -XX:+UseParallelGC 1 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler Error Message: ERROR: SolrIndexSearcher opens=76 closes=75 Stack Trac

[JENKINS] Lucene-Solr-tests-only-4.x-nightly - Build # 1 - Failure

2012-08-11 Thread Apache Jenkins Server
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-4.x-nightly/1/ 1 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler Error Message: ERROR: SolrIndexSearcher opens=76 closes=75 Stack Trace: java.lang.AssertionError: ERROR: SolrIndexSearcher

[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0-ea-b49) - Build # 368 - Failure!

2012-08-11 Thread Policeman Jenkins Server
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux/368/ Java: 32bit/jdk1.8.0-ea-b49 -server -XX:+UseSerialGC 1 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler Error Message: ERROR: SolrIndexSearcher opens=76 closes=75 Stack Trac

Re: (LUCENE-3892) Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, Simple9/16/64, etc.)

2012-08-11 Thread Michael McCandless
On Sat, Aug 11, 2012 at 10:31 AM, Robert Muir wrote: > I'm having a tough time remembering what these packed ints options do > (I thought the perf boost from allowing overhead came from upgrading > to the next byte boundary?) Upgrading to the next byte boundary, or using PACKED_SINGLE_BLOCK when

Re: revisit payloads API in DocsAndPositionsEnum

2012-08-11 Thread Simon Willnauer
+1 this makes lots of sense simon On Sat, Aug 11, 2012 at 7:28 PM, Michael McCandless wrote: > +1 > > Mike McCandless > > http://blog.mikemccandless.com > > > On Sat, Aug 11, 2012 at 10:08 AM, Robert Muir wrote: >> The payloads api is really confusing: >> >> /** Returns the payload at this po

Re: revisit payloads API in DocsAndPositionsEnum

2012-08-11 Thread Michael McCandless
+1 Mike McCandless http://blog.mikemccandless.com On Sat, Aug 11, 2012 at 10:08 AM, Robert Muir wrote: > The payloads api is really confusing: > > /** Returns the payload at this position, or null if no >* payload was indexed. Only call this once per >* position. You should not mo

Re: (LUCENE-3892) Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, Simple9/16/64, etc.)

2012-08-11 Thread Robert Muir
I'm having a tough time remembering what these packed ints options do (I thought the perf boost from allowing overhead came from upgrading to the next byte boundary?) Anyway: again I'm a little concerned about the wikipedia benchmark here for this purpose. For e.g. structured content from databas

revisit payloads API in DocsAndPositionsEnum

2012-08-11 Thread Robert Muir
The payloads api is really confusing: /** Returns the payload at this position, or null if no * payload was indexed. Only call this once per * position. You should not modify anything (neither * members of the returned BytesRef nor bytes in the * byte[]). */ public abstract By

Re: remove FieldsEnum?

2012-08-11 Thread Michael McCandless
+1 Mike McCandless http://blog.mikemccandless.com On Sat, Aug 11, 2012 at 9:27 AM, Robert Muir wrote: > FieldsEnum seems kinda awkward, can we just have something like > Iterable on Fields? > > so the consuming code i think would be easier. > > for (String field : fields) { > Terms terms = f

remove FieldsEnum?

2012-08-11 Thread Robert Muir
FieldsEnum seems kinda awkward, can we just have something like Iterable on Fields? so the consuming code i think would be easier. for (String field : fields) { Terms terms = fields.terms(field); ... } I don't like that there are two ways to get the terms for a field today, so there is dupli

Re: (LUCENE-3892) Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, Simple9/16/64, etc.)

2012-08-11 Thread Michael McCandless
Here are my results ... base = DEFAULT (0.2 acceptable overhead), competitor = compact (0.0 overhead, ie PACKED): Dual Xeon x5680: TaskQPS base StdDev base QPS compact StdDev compact Pct diff Prefix3 77.855.86 72.792.24 -15% -4%

Re: LUCENE-1888: payloads on term vectors

2012-08-11 Thread Michael McCandless
+1 Mike McCandless http://blog.mikemccandless.com On Sat, Aug 11, 2012 at 9:01 AM, Robert Muir wrote: > See patch here: http://pastebin.com/aK0bbTMA > > I think its ready. File format changes are backwards compatible, etc. > > -- > lucidimagination.com > > -

Re: don't allow negatives in the positions file

2012-08-11 Thread Michael McCandless
+1, patch looks great. Mike McCandless http://blog.mikemccandless.com On Sat, Aug 11, 2012 at 8:59 AM, Robert Muir wrote: > Hello, see the linked patch: > > http://pastebin.com/7JAaJ3EN > > Because of an ancient bug in lucene 2.4.0, we still allow -1 as a > position. But this doesnt even work t

LUCENE-1888: payloads on term vectors

2012-08-11 Thread Robert Muir
See patch here: http://pastebin.com/aK0bbTMA I think its ready. File format changes are backwards compatible, etc. -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-m

don't allow negatives in the positions file

2012-08-11 Thread Robert Muir
Hello, see the linked patch: http://pastebin.com/7JAaJ3EN Because of an ancient bug in lucene 2.4.0, we still allow -1 as a position. But this doesnt even work today (i created such an index, and phrase queries etc dont work because tons of lucene code assumes positions are >= 0). Additionally, t

RE: Restructuring of Jenkins finished

2012-08-11 Thread Uwe Schindler
> - ant jenkins-clover: this one runs the clover tasks. Be sure to enable - > Drun.clover=true and -Dtests.jvms=1 (it's not yet done automatically, I may do > this using subant in a later stage). In the Jenkins config you also have to select > ANT_OPTS with maany memory -Xmx1536M, otherwise rep

Restructuring of Jenkins finished

2012-08-11 Thread Uwe Schindler
Hi, Robert and I opened LUCENE-4187 to restructure the Jenkins builds and to get rid of shell scripts, which make it hard to configure Jenkins on non-ASF machines. I ported all shell scripts over to top-level build.xml targets: - ant jenkins-hourly: this one runs the hourly test runs (it also doe

Re: [IMPORTANT] Thread leak patch

2012-08-11 Thread Michael McCandless
On Sat, Aug 11, 2012 at 6:31 AM, Dawid Weiss wrote: > This is important to avoid frustrations, please read until the end. > I've been working in the background for some time now and I think I am > ready to commit the long-planned runner changes. It includes: > > - timeout control (entire suite mu

[IMPORTANT] Thread leak patch

2012-08-11 Thread Dawid Weiss
Hi. This is important to avoid frustrations, please read until the end. I've been working in the background for some time now and I think I am ready to commit the long-planned runner changes. It includes: - timeout control (entire suite must not last longer than 10 minutes), - thread leak control

Build failed in Jenkins: Lucene-trunk-Linux-Java6-64 #191

2012-08-11 Thread builder
See Changes: [uschindler] LUCENE-4187: simplier pattern [uschindler] LUCENE-4187: exclude backwards (when it comes) [uschindler] LUCENE-4187: Nuke remaining shell scripts [uschindler] LUCENE-4187: Add jenkins clover task [uschindler] LUCENE-4187: Remove remaining clover reference [uschindle

Build failed in Jenkins: Lucene-trunk-Linux-Java7-64 #114

2012-08-11 Thread builder
See Changes: [uschindler] LUCENE-4187: simplier pattern [uschindler] LUCENE-4187: exclude backwards (when it comes) [uschindler] LUCENE-4187: Nuke remaining shell scripts [uschindler] LUCENE-4187: Add jenkins clover task [uschindler] LUCENE-4187: Remove remaining clover reference [uschindle

[JENKINS] Lucene-Solr-tests-only-trunk-java7 - Build # 3041 - Failure

2012-08-11 Thread Apache Jenkins Server
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk-java7/3041/ 1 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler Error Message: ERROR: SolrIndexSearcher opens=78 closes=77 Stack Trace: java.lang.AssertionError: ERROR: SolrIndexSearc