date:20140121

2014-01-21 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878322#comment-13878322
 ] 

Steve Rowe commented on LUCENE-5411:


Committed to trunk and branch_4x.

Unfortunately, the version reported by JFlex 1.5.0 is still "1.5.0-SNAPSHOT".  
Not sure whether 1.5.0 can be redeployed on Maven Central - if not, there will 
have to be a 1.5.1 release.

I'll keep this issue open until the SNAPSHOT problem is resolved. 

> Upgrade to released JFlex 1.5.0
> ---
>
> Key: LUCENE-5411
> URL: https://issues.apache.org/jira/browse/LUCENE-5411
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: general/build
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Minor
> Fix For: 5.0, 4.7
>
> Attachments: LUCENE-5411.patch
>
>
> The JFlex 1.5.0 release will be officially announced shortly.  The jar is 
> already on Maven Central.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5411) Upgrade to released JFlex 1.5.0


[ 
https://issues.apache.org/jira/browse/LUCENE-5411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878307#comment-13878307
 ] 

ASF subversion and git services commented on LUCENE-5411:
-

Commit 1560260 from [~steve_rowe] in branch 'dev/trunk'
[ https://svn.apache.org/r1560260 ]

LUCENE-5411: Upgrade to released JFlex 1.5.0; stop requiring a locally built 
JFlex snapshot jar.

> Upgrade to released JFlex 1.5.0
> ---
>
> Key: LUCENE-5411
> URL: https://issues.apache.org/jira/browse/LUCENE-5411
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: general/build
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Minor
> Fix For: 5.0, 4.7
>
> Attachments: LUCENE-5411.patch
>
>
> The JFlex 1.5.0 release will be officially announced shortly.  The jar is 
> already on Maven Central.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5411) Upgrade to released JFlex 1.5.0


 [ 
https://issues.apache.org/jira/browse/LUCENE-5411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated LUCENE-5411:
---

Attachment: LUCENE-5411.patch

Patch (mostly stolen from the forbidden-api setup) switching to using 
{{}} to download the JFlex 1.5.0 jar from Maven Central and use 
it from there, rather than the previously required locally-built-from-source 
JFlex jar.

FYI, after the switch, I had to increase the JVM max mem in my {{ANT_OPTS}} 
environment variable from {{-Xmx1g}} to {{-Xmx1040m}} in order to get JFlex to 
generate {{UAX29URLEmailTokenizerImpl.java}} without OOMing.

Committing shortly.


> Upgrade to released JFlex 1.5.0
> ---
>
> Key: LUCENE-5411
> URL: https://issues.apache.org/jira/browse/LUCENE-5411
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: general/build
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Minor
> Fix For: 5.0, 4.7
>
> Attachments: LUCENE-5411.patch
>
>
> The JFlex 1.5.0 release will be officially announced shortly.  The jar is 
> already on Maven Central.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5377) Lucene 4.6 cause mixed version segments info file(.si) wrong


[ 
https://issues.apache.org/jira/browse/LUCENE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878294#comment-13878294
 ] 

Littlestar commented on LUCENE-5377:


I debug into it, Lucene 4.6's new format of segments.gen(segments_N) record bad 
segments info.


> Lucene 4.6 cause mixed version segments info file(.si) wrong
> 
>
> Key: LUCENE-5377
> URL: https://issues.apache.org/jira/browse/LUCENE-5377
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index, modules/facet
>Affects Versions: 4.6
> Environment: windows/linux
>Reporter: Littlestar
> Attachments: TestCore_45.java, TestCore_46.java
>
>
> my old facet index create by Lucene version=4.2
> use indexChecker ok.
> now I upgrade to Lucene 4.6 and put some new records to index.
> then reopen index, some files in indexdir missing
> no .si files.
> I debug into it,  new version format of segments.gen(segments_N) record bad 
> segments info.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5377) Lucene 4.6 cause mixed version segments info file(.si) wrong


 [ 
https://issues.apache.org/jira/browse/LUCENE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Littlestar updated LUCENE-5377:
---

Summary: Lucene 4.6 cause mixed version segments info file(.si) wrong  
(was: Lucene upgrade cause mixed version segments info file(.si) wrong)

> Lucene 4.6 cause mixed version segments info file(.si) wrong
> 
>
> Key: LUCENE-5377
> URL: https://issues.apache.org/jira/browse/LUCENE-5377
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index, modules/facet
>Affects Versions: 4.6
> Environment: windows/linux
>Reporter: Littlestar
> Attachments: TestCore_45.java, TestCore_46.java
>
>
> my old facet index create by Lucene version=4.2
> use indexChecker ok.
> now I upgrade to Lucene 4.6 and put some new records to index.
> then reopen index, some files in indexdir missing
> no .si files.
> I debug into it,  new version format of segments.gen(segments_N) record bad 
> segments info.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-5411) Upgrade to released JFlex 1.5.0

Steve Rowe created LUCENE-5411:
--

 Summary: Upgrade to released JFlex 1.5.0
 Key: LUCENE-5411
 URL: https://issues.apache.org/jira/browse/LUCENE-5411
 Project: Lucene - Core
  Issue Type: Improvement
  Components: general/build
Reporter: Steve Rowe
Assignee: Steve Rowe
Priority: Minor
 Fix For: 5.0, 4.7


The JFlex 1.5.0 release will be officially announced shortly.  The jar is 
already on Maven Central.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5594) Enable using extended field types with prefix queries for non-default encoded strings

2014-01-21 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878286#comment-13878286
 ] 

Steve Rowe commented on SOLR-5594:
--

bq. Once I figure out (or know) how to fix the EOL issue, I'll run ant 
precommit to fix any further issues on that front.

In case you haven't figured it out yet, you just run {{svn propset 
svn:eol-style native }}, where {{}} is the (list of) file(s) 
that don't have the {{svn:eol-style}} property set on them.  This info should 
be on Lucene's and Solr's HowToContribute wiki pages, though I see it's not 
there yet.

{quote}
Figured that out and ran it, only to run into the same issue twice over:
lucene-trunk2/lucene/common-build.xml:1851: java.lang.OutOfMemoryError: Java 
heap space
{quote}

Line 1851 in {{lucene/common-build.xml}} runs the JTidy Ant task.  According to 
the comment on line 341:

{code:xml}
  
{code}

You can give arguments to the JVM that runs {{ant}} via the environment 
variable {{ANT_OPTS}}.  Here's mine (on Win7+Cygwin under bash):

{noformat}
export ANT_OPTS=-Xmx1100M -XX:MaxPermSize=256m -Dpython.exe=python2.6.exe 
-Dpython32.exe=python3.2m.exe
{noformat}

I doubt the JTidy Ant task requires 1100MB, but the JFlex Ant task OOMs with 
anything less than 1040MB on my box when it generates 
{{UAX29URLEmailTokenizerImpl.java}} (via {{ant jflex}} under 
{{lucene/analysis/common/}}, so that's why I have it set that high.

> Enable using extended field types with prefix queries for non-default encoded 
> strings
> -
>
> Key: SOLR-5594
> URL: https://issues.apache.org/jira/browse/SOLR-5594
> Project: Solr
>  Issue Type: Improvement
>  Components: query parsers, Schema and Analysis
>Affects Versions: 4.6
>Reporter: Anshum Gupta
>Assignee: Anshum Gupta
>Priority: Minor
> Attachments: SOLR-5594-branch_4x.patch, SOLR-5594.patch, 
> SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch, 
> SOLR-5594.patch
>
>
> Enable users to be able to use prefix query with custom field types with 
> non-default encoding/decoding for queries more easily. e.g. having a custom 
> field work with base64 encoded query strings.
> Currently, the workaround for it is to have the override at getRewriteMethod 
> level. Perhaps having the prefixQuery also use the calling FieldType's 
> readableToIndexed method would work better.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Areek Zillur as Lucene/Solr committer!

2014-01-21 Thread Han Jiang

Congratulations and welcome Areek!


On Wed, Jan 22, 2014 at 12:57 PM, Shalin Shekhar Mangar <
shalinman...@gmail.com> wrote:

> Welcome Areek!
>
> On Wed, Jan 22, 2014 at 2:00 AM, Areek Zillur  wrote:
> > Thanks Robert! I am very pleased to be a committer to Lucene/Solr!
> >
> > I am originally from Dhaka, Bangladesh. I am currently a 4th year
> Computer
> > Engineering
> > student at University of Waterloo in Canada. I was fortunate enough to
> have
> > multiple internships
> > all over North America through the university's co-op program.
> > I was first introduced to Lucene/Solr in one of my work-terms at A9 and
> > loved it.
> >
> > I really enjoy the open-source development and the friendliness of the
> > community
> > behind Lucene/Solr.
> > In my free time, I enjoy working on working on my  recreational
> algorithmic
> > trading system and learning
> > new programming languages.
> >
> > I hope to continue to work on Lucene/Solr and learn a lot more from the
> > community!
> >
> > Thanks,
> >
> > Areek Zillur
> >
> >
> > On Tue, Jan 21, 2014 at 11:41 AM, Yonik Seeley 
> > wrote:
> >>
> >> Welcome Areek!
> >>
> >> -Yonik
> >> http://heliosearch.com -- making solr shine
> >>
> >> On Tue, Jan 21, 2014 at 2:26 PM, Robert Muir  wrote:
> >> > I'm pleased to announce that Areek Zillur has accepted to join our
> ranks
> >> > as
> >> > a committer.
> >> >
> >> > Areek has been improving suggester support in Lucene and Solr,
> including
> >> > a
> >> > revamped Solr component slated for the 4.7 release. [1]
> >> >
> >> > Areek, it is tradition that you introduce yourself with a brief bio.
> >> >
> >> > Once your account is setup, you should then be able to add yourself to
> >> > the
> >> > who we are page on the website as well.
> >> >
> >> > Congratulations and welcome!
> >> >
> >> > [1] https://issues.apache.org/jira/browse/SOLR-5378
> >> >
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> >> For additional commands, e-mail: dev-h...@lucene.apache.org
> >>
> >
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>


-- 
Han Jiang

Team of Search Engine and Web Mining,
School of Electronic Engineering and Computer Science,
Peking University, China

Re: Welcome Areek Zillur as Lucene/Solr committer!

2014-01-21 Thread Shalin Shekhar Mangar

Welcome Areek!

On Wed, Jan 22, 2014 at 2:00 AM, Areek Zillur  wrote:
> Thanks Robert! I am very pleased to be a committer to Lucene/Solr!
>
> I am originally from Dhaka, Bangladesh. I am currently a 4th year Computer
> Engineering
> student at University of Waterloo in Canada. I was fortunate enough to have
> multiple internships
> all over North America through the university's co-op program.
> I was first introduced to Lucene/Solr in one of my work-terms at A9 and
> loved it.
>
> I really enjoy the open-source development and the friendliness of the
> community
> behind Lucene/Solr.
> In my free time, I enjoy working on working on my  recreational algorithmic
> trading system and learning
> new programming languages.
>
> I hope to continue to work on Lucene/Solr and learn a lot more from the
> community!
>
> Thanks,
>
> Areek Zillur
>
>
> On Tue, Jan 21, 2014 at 11:41 AM, Yonik Seeley 
> wrote:
>>
>> Welcome Areek!
>>
>> -Yonik
>> http://heliosearch.com -- making solr shine
>>
>> On Tue, Jan 21, 2014 at 2:26 PM, Robert Muir  wrote:
>> > I'm pleased to announce that Areek Zillur has accepted to join our ranks
>> > as
>> > a committer.
>> >
>> > Areek has been improving suggester support in Lucene and Solr, including
>> > a
>> > revamped Solr component slated for the 4.7 release. [1]
>> >
>> > Areek, it is tradition that you introduce yourself with a brief bio.
>> >
>> > Once your account is setup, you should then be able to add yourself to
>> > the
>> > who we are page on the website as well.
>> >
>> > Congratulations and welcome!
>> >
>> > [1] https://issues.apache.org/jira/browse/SOLR-5378
>> >
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>



-- 
Regards,
Shalin Shekhar Mangar.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Closed] (LUCENE-3565) StandardQueryParser generates incorrect query for groups containing one term if using default operator AND

2014-01-21 Thread Adriano Crestani (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adriano Crestani closed LUCENE-3565.


Resolution: Not A Problem

> StandardQueryParser generates incorrect query for groups containing one term 
> if using default operator AND
> --
>
> Key: LUCENE-3565
> URL: https://issues.apache.org/jira/browse/LUCENE-3565
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/queryparser
>Affects Versions: 3.4
>Reporter: Trejkaz
>  Labels: queryparser
>
> The following test demonstrates a bug in StandardQueryParser when 
> DEFAULT_OPERATOR is set to AND:
> {code}
> @Test
> public void testDefaultOperatorSingleGrouped() throws Exception
> {
> StandardQueryParser parser = new StandardQueryParser();
> 
> parser.getQueryConfigHandler().set(StandardQueryConfigHandler.ConfigurationKeys.DEFAULT_OPERATOR,
>  StandardQueryConfigHandler.Operator.AND);
> BooleanQuery expectedQuery = new BooleanQuery();
> BooleanQuery innerQuery1 = new BooleanQuery();
> innerQuery1.add(new TermQuery(new Term("text", "a")), 
> BooleanClause.Occur.MUST);
> expectedQuery.add(innerQuery1, BooleanClause.Occur.SHOULD);
> BooleanQuery innerQuery2 = new BooleanQuery();
> innerQuery2.add(new TermQuery(new Term("text", "b")), 
> BooleanClause.Occur.MUST);
> expectedQuery.add(innerQuery2, BooleanClause.Occur.SHOULD);
> Query actualQuery = parser.parse("(a) OR (b)", "text");
> assertEquals("Wrong query was generated", expectedQuery, actualQuery);
> }
> {code}
> BooleanSingleChildOptimizationQueryNodeProcessor appears to be responsible 
> because if I remove it, the correct query is generated (however, doing so 
> breaks a number of other tests of our own.)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3565) StandardQueryParser generates incorrect query for groups containing one term if using default operator AND

2014-01-21 Thread Adriano Crestani (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878241#comment-13878241
 ] 

Adriano Crestani commented on LUCENE-3565:
--

I don't think the suggested test case is correct. It's not wrong either. The 
problem is that the query parsers (both classic and standard) return an 
optimized query. In my opinion, (+text:a) OR (+text:b) is the same as text:a OR 
text:b, but the latter is optimized.

> StandardQueryParser generates incorrect query for groups containing one term 
> if using default operator AND
> --
>
> Key: LUCENE-3565
> URL: https://issues.apache.org/jira/browse/LUCENE-3565
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/queryparser
>Affects Versions: 3.4
>Reporter: Trejkaz
>  Labels: queryparser
>
> The following test demonstrates a bug in StandardQueryParser when 
> DEFAULT_OPERATOR is set to AND:
> {code}
> @Test
> public void testDefaultOperatorSingleGrouped() throws Exception
> {
> StandardQueryParser parser = new StandardQueryParser();
> 
> parser.getQueryConfigHandler().set(StandardQueryConfigHandler.ConfigurationKeys.DEFAULT_OPERATOR,
>  StandardQueryConfigHandler.Operator.AND);
> BooleanQuery expectedQuery = new BooleanQuery();
> BooleanQuery innerQuery1 = new BooleanQuery();
> innerQuery1.add(new TermQuery(new Term("text", "a")), 
> BooleanClause.Occur.MUST);
> expectedQuery.add(innerQuery1, BooleanClause.Occur.SHOULD);
> BooleanQuery innerQuery2 = new BooleanQuery();
> innerQuery2.add(new TermQuery(new Term("text", "b")), 
> BooleanClause.Occur.MUST);
> expectedQuery.add(innerQuery2, BooleanClause.Occur.SHOULD);
> Query actualQuery = parser.parse("(a) OR (b)", "text");
> assertEquals("Wrong query was generated", expectedQuery, actualQuery);
> }
> {code}
> BooleanSingleChildOptimizationQueryNodeProcessor appears to be responsible 
> because if I remove it, the correct query is generated (however, doing so 
> breaks a number of other tests of our own.)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-5099) QueryNode should have the ability to detach from its node parent

2014-01-21 Thread Adriano Crestani (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adriano Crestani resolved LUCENE-5099.
--

   Resolution: Fixed
Fix Version/s: 4.7
Lucene Fields:   (was: New)

> QueryNode should have the ability to detach from its node parent
> 
>
> Key: LUCENE-5099
> URL: https://issues.apache.org/jira/browse/LUCENE-5099
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/queryparser
>Affects Versions: 3.6.2, 4.3.1
>Reporter: Adriano Crestani
>Assignee: Adriano Crestani
> Fix For: 4.7
>
> Attachments: LUCENE-5099_adrianocrestani_2014-01-19.patch, 
> LUCENE-5099_adrianocrestani_2014-01-19_4x.patch
>
>
> QueryNodeProcessorImpl should always return the root of the tree after 
> processing, so it needs to make the parent is set to null before returning. 
> Otherwise, the previous parent is leaked and never removed.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5099) QueryNode should have the ability to detach from its node parent


[ 
https://issues.apache.org/jira/browse/LUCENE-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878221#comment-13878221
 ] 

ASF subversion and git services commented on LUCENE-5099:
-

Commit 1560246 from [~adriano_crestani] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1560246 ]

LUCENE-5099 - QueryNode should have the ability to detach from its node parent

> QueryNode should have the ability to detach from its node parent
> 
>
> Key: LUCENE-5099
> URL: https://issues.apache.org/jira/browse/LUCENE-5099
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/queryparser
>Affects Versions: 3.6.2, 4.3.1
>Reporter: Adriano Crestani
>Assignee: Adriano Crestani
> Attachments: LUCENE-5099_adrianocrestani_2014-01-19.patch, 
> LUCENE-5099_adrianocrestani_2014-01-19_4x.patch
>
>
> QueryNodeProcessorImpl should always return the root of the tree after 
> processing, so it needs to make the parent is set to null before returning. 
> Otherwise, the previous parent is leaked and never removed.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5099) QueryNode should have the ability to detach from its node parent

2014-01-21 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878220#comment-13878220
 ] 

ASF subversion and git services commented on LUCENE-5099:
-

Commit 1560245 from [~adriano_crestani] in branch 'dev/trunk'
[ https://svn.apache.org/r1560245 ]

LUCENE-5099 - QueryNode should have the ability to detach from its node parent

> QueryNode should have the ability to detach from its node parent
> 
>
> Key: LUCENE-5099
> URL: https://issues.apache.org/jira/browse/LUCENE-5099
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/queryparser
>Affects Versions: 3.6.2, 4.3.1
>Reporter: Adriano Crestani
>Assignee: Adriano Crestani
> Attachments: LUCENE-5099_adrianocrestani_2014-01-19.patch, 
> LUCENE-5099_adrianocrestani_2014-01-19_4x.patch
>
>
> QueryNodeProcessorImpl should always return the root of the tree after 
> processing, so it needs to make the parent is set to null before returning. 
> Otherwise, the previous parent is leaked and never removed.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5650) When mixing adds and deletes, it appears there is a corner case where peersync can bring back a deleted update.


[ 
https://issues.apache.org/jira/browse/SOLR-5650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878217#comment-13878217
 ] 

Mark Miller commented on SOLR-5650:
---

Just saw another fail where the control had a doc the cluster did not. This 
issue may have helped hide it previously, don't remember seeing it before. 
Actually looks like a legit fail. Looks like a single leader in a shard 
accepted an update and was killed. The update is gone, and so the loss makes 
sense in the current system. You need support for the param that tells how many 
replicas to the update must get to for success to protect against this.

> When mixing adds and deletes, it appears there is a corner case where 
> peersync can bring back a deleted update.
> ---
>
> Key: SOLR-5650
> URL: https://issues.apache.org/jira/browse/SOLR-5650
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 5.0, 4.7
>
> Attachments: SOLR-5650.patch, SOLR-5650.patch, solr.log.tar.gz
>
>




--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5594) Enable using extended field types with prefix queries for non-default encoded strings


[ 
https://issues.apache.org/jira/browse/SOLR-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878195#comment-13878195
 ] 

Anshum Gupta commented on SOLR-5594:


Ran "ant check-forbidden-apis" successfully.

> Enable using extended field types with prefix queries for non-default encoded 
> strings
> -
>
> Key: SOLR-5594
> URL: https://issues.apache.org/jira/browse/SOLR-5594
> Project: Solr
>  Issue Type: Improvement
>  Components: query parsers, Schema and Analysis
>Affects Versions: 4.6
>Reporter: Anshum Gupta
>Assignee: Anshum Gupta
>Priority: Minor
> Attachments: SOLR-5594-branch_4x.patch, SOLR-5594.patch, 
> SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch, 
> SOLR-5594.patch
>
>
> Enable users to be able to use prefix query with custom field types with 
> non-default encoding/decoding for queries more easily. e.g. having a custom 
> field work with base64 encoded query strings.
> Currently, the workaround for it is to have the override at getRewriteMethod 
> level. Perhaps having the prefixQuery also use the calling FieldType's 
> readableToIndexed method would work better.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5377) Lucene upgrade cause mixed version segments info file(.si) wrong


 [ 
https://issues.apache.org/jira/browse/LUCENE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Littlestar updated LUCENE-5377:
---

Component/s: modules/facet
Summary: Lucene upgrade cause mixed version segments info file(.si) 
wrong  (was: Lucene mixed version segments cause segment info file(.si) wrong)

> Lucene upgrade cause mixed version segments info file(.si) wrong
> 
>
> Key: LUCENE-5377
> URL: https://issues.apache.org/jira/browse/LUCENE-5377
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index, modules/facet
>Affects Versions: 4.6
> Environment: windows/linux
>Reporter: Littlestar
> Attachments: TestCore_45.java, TestCore_46.java
>
>
> my old facet index create by Lucene version=4.2
> use indexChecker ok.
> now I upgrade to Lucene 4.6 and put some new records to index.
> then reopen index, some files in indexdir missing
> no .si files.
> I debug into it,  new version format of segments.gen(segments_N) record bad 
> segments info.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5594) Enable using extended field types with prefix queries for non-default encoded strings


[ 
https://issues.apache.org/jira/browse/SOLR-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878156#comment-13878156
 ] 

Anshum Gupta commented on SOLR-5594:


Figured that out and ran it, only to run into the same issue twice over:
lucene-trunk2/lucene/common-build.xml:1851: java.lang.OutOfMemoryError: Java 
heap space

is this a known issue or am I missing something here?

> Enable using extended field types with prefix queries for non-default encoded 
> strings
> -
>
> Key: SOLR-5594
> URL: https://issues.apache.org/jira/browse/SOLR-5594
> Project: Solr
>  Issue Type: Improvement
>  Components: query parsers, Schema and Analysis
>Affects Versions: 4.6
>Reporter: Anshum Gupta
>Assignee: Anshum Gupta
>Priority: Minor
> Attachments: SOLR-5594-branch_4x.patch, SOLR-5594.patch, 
> SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch, 
> SOLR-5594.patch
>
>
> Enable users to be able to use prefix query with custom field types with 
> non-default encoding/decoding for queries more easily. e.g. having a custom 
> field work with base64 encoded query strings.
> Currently, the workaround for it is to have the override at getRewriteMethod 
> level. Perhaps having the prefixQuery also use the calling FieldType's 
> readableToIndexed method would work better.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5594) Enable using extended field types with prefix queries for non-default encoded strings


 [ 
https://issues.apache.org/jira/browse/SOLR-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anshum Gupta updated SOLR-5594:
---

Attachment: SOLR-5594.patch

* randomized the doc counts more (more like a 2:1 distribution).
* Checked for query equality across qparsers.

I haven't been able to run ant precommit yet so have just fixed "new Random()" 
for now.
Once I figure out (or know) how to fix the EOL issue, I'll run ant precommit to 
fix any further issues on that front.

> Enable using extended field types with prefix queries for non-default encoded 
> strings
> -
>
> Key: SOLR-5594
> URL: https://issues.apache.org/jira/browse/SOLR-5594
> Project: Solr
>  Issue Type: Improvement
>  Components: query parsers, Schema and Analysis
>Affects Versions: 4.6
>Reporter: Anshum Gupta
>Assignee: Anshum Gupta
>Priority: Minor
> Attachments: SOLR-5594-branch_4x.patch, SOLR-5594.patch, 
> SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch, 
> SOLR-5594.patch
>
>
> Enable users to be able to use prefix query with custom field types with 
> non-default encoding/decoding for queries more easily. e.g. having a custom 
> field work with base64 encoded query strings.
> Currently, the workaround for it is to have the override at getRewriteMethod 
> level. Perhaps having the prefixQuery also use the calling FieldType's 
> readableToIndexed method would work better.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5407) Deadlock? while indexing in cascaded threads


 [ 
https://issues.apache.org/jira/browse/LUCENE-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luis Filipe Nassif updated LUCENE-5407:
---

Description: Apparently I found a deadlock problem with IndexWriter in a 
cascaded thread design to add documents (I am working on an application 
integrating Tika, which has the capability to add embedded documents to the 
index as independent documents as they are found). The attached code 
illustrates the problem. Sometimes it stops processing, at least one of the 
threads remains in WAITING state. It must be executed no more than 5 times in 
my environment to trigger the problem.  (was: Apparently I found a deadlock 
problem with IndexWriter using Reader Fields in a cascaded thread design to add 
documents (I am working on an application integrating Tika, which has the 
capability to add embedded documents to the index as independent documents as 
they are found). The attached code illustrates the problem. Sometimes it stops 
processing, at least one of the threads remains in WAITING state. It must be 
executed no more than 5 times in my environment to trigger the problem.)
Summary: Deadlock? while indexing in cascaded threads  (was: Deadlock? 
while indexing reader fields in cascaded threads)

> Deadlock? while indexing in cascaded threads
> 
>
> Key: LUCENE-5407
> URL: https://issues.apache.org/jira/browse/LUCENE-5407
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.6
> Environment: Windows 7 64 bits, JRE 1.7.0_25 64 bits
>Reporter: Luis Filipe Nassif
> Attachments: Test.java, thread_dump.txt
>
>
> Apparently I found a deadlock problem with IndexWriter in a cascaded thread 
> design to add documents (I am working on an application integrating Tika, 
> which has the capability to add embedded documents to the index as 
> independent documents as they are found). The attached code illustrates the 
> problem. Sometimes it stops processing, at least one of the threads remains 
> in WAITING state. It must be executed no more than 5 times in my environment 
> to trigger the problem.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5594) Enable using extended field types with prefix queries for non-default encoded strings


[ 
https://issues.apache.org/jira/browse/SOLR-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878121#comment-13878121
 ] 

Anshum Gupta commented on SOLR-5594:


I’ve tried Mac EOL styles(\r) , *nix style EOL (\n) and  OS dependent (which 
should also be mac style) but in vain.

> Enable using extended field types with prefix queries for non-default encoded 
> strings
> -
>
> Key: SOLR-5594
> URL: https://issues.apache.org/jira/browse/SOLR-5594
> Project: Solr
>  Issue Type: Improvement
>  Components: query parsers, Schema and Analysis
>Affects Versions: 4.6
>Reporter: Anshum Gupta
>Assignee: Anshum Gupta
>Priority: Minor
> Attachments: SOLR-5594-branch_4x.patch, SOLR-5594.patch, 
> SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch
>
>
> Enable users to be able to use prefix query with custom field types with 
> non-default encoding/decoding for queries more easily. e.g. having a custom 
> field work with base64 encoded query strings.
> Currently, the workaround for it is to have the override at getRewriteMethod 
> level. Perhaps having the prefixQuery also use the calling FieldType's 
> readableToIndexed method would work better.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Areek Zillur as Lucene/Solr committer!

2014-01-21 Thread Mark Miller

Welcome Areek!

- Mark 



On Jan 21, 2014, 8:05:32 PM, Erick Erickson  wrote: 
Welcome aboard Areek!

On Tue, Jan 21, 2014 at 7:33 PM, Joel Bernstein  wrote:
> Congratulations Areek!
>
> Joel Bernstein
> Search Engineer at Heliosearch
>
>
> On Tue, Jan 21, 2014 at 5:47 PM, Dawid Weiss 
> wrote:
>>
>> Congratulations and welcome, Areek!
>>
>> On Tue, Jan 21, 2014 at 11:43 PM, Christian Moen  wrote:
>> > Great! Congrats and welcome, Areek!
>> >
>> > Best,
>> > Christian
>> >
>> > On Jan 22, 2014, at 4:26 AM, Robert Muir  wrote:
>> >
>> > I'm pleased to announce that Areek Zillur has accepted to join our ranks
>> > as
>> > a committer.
>> >
>> > Areek has been improving suggester support in Lucene and Solr, including
>> > a
>> > revamped Solr component slated for the 4.7 release. [1]
>> >
>> > Areek, it is tradition that you introduce yourself with a brief bio.
>> >
>> > Once your account is setup, you should then be able to add yourself to
>> > the
>> > who we are page on the website as well.
>> >
>> > Congratulations and welcome!
>> >
>> > [1] https://issues.apache.org/jira/browse/SOLR-5378
>> >
>> >
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Areek Zillur as Lucene/Solr committer!

2014-01-21 Thread Erick Erickson

Welcome aboard Areek!

On Tue, Jan 21, 2014 at 7:33 PM, Joel Bernstein  wrote:
> Congratulations Areek!
>
> Joel Bernstein
> Search Engineer at Heliosearch
>
>
> On Tue, Jan 21, 2014 at 5:47 PM, Dawid Weiss 
> wrote:
>>
>> Congratulations and welcome, Areek!
>>
>> On Tue, Jan 21, 2014 at 11:43 PM, Christian Moen  wrote:
>> > Great! Congrats and welcome, Areek!
>> >
>> > Best,
>> > Christian
>> >
>> > On Jan 22, 2014, at 4:26 AM, Robert Muir  wrote:
>> >
>> > I'm pleased to announce that Areek Zillur has accepted to join our ranks
>> > as
>> > a committer.
>> >
>> > Areek has been improving suggester support in Lucene and Solr, including
>> > a
>> > revamped Solr component slated for the 4.7 release. [1]
>> >
>> > Areek, it is tradition that you introduce yourself with a brief bio.
>> >
>> > Once your account is setup, you should then be able to add yourself to
>> > the
>> > who we are page on the website as well.
>> >
>> > Congratulations and welcome!
>> >
>> > [1] https://issues.apache.org/jira/browse/SOLR-5378
>> >
>> >
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5594) Enable using extended field types with prefix queries for non-default encoded strings


[ 
https://issues.apache.org/jira/browse/SOLR-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878082#comment-13878082
 ] 

Anshum Gupta commented on SOLR-5594:


Thanks for the patience Hoss.

* I get what you're trying to put forward about asserting the queries to be 
equal. I've made those changes and I'm trying to get the test to pass for now 
(which isn't happening for now).
* The numDocs for each of those would be different, but yes, by adding what you 
suggested it would only become stronger. I'll add that too.

When I run "ant precommit", I just get "The following files are missing 
svn:eol-style" (perhaps it stops right there) followed by a list of files.



> Enable using extended field types with prefix queries for non-default encoded 
> strings
> -
>
> Key: SOLR-5594
> URL: https://issues.apache.org/jira/browse/SOLR-5594
> Project: Solr
>  Issue Type: Improvement
>  Components: query parsers, Schema and Analysis
>Affects Versions: 4.6
>Reporter: Anshum Gupta
>Assignee: Anshum Gupta
>Priority: Minor
> Attachments: SOLR-5594-branch_4x.patch, SOLR-5594.patch, 
> SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch
>
>
> Enable users to be able to use prefix query with custom field types with 
> non-default encoding/decoding for queries more easily. e.g. having a custom 
> field work with base64 encoded query strings.
> Currently, the workaround for it is to have the override at getRewriteMethod 
> level. Perhaps having the prefixQuery also use the calling FieldType's 
> readableToIndexed method would work better.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Areek Zillur as Lucene/Solr committer!

2014-01-21 Thread Joel Bernstein

Congratulations Areek!

Joel Bernstein
Search Engineer at Heliosearch


On Tue, Jan 21, 2014 at 5:47 PM, Dawid Weiss
wrote:

> Congratulations and welcome, Areek!
>
> On Tue, Jan 21, 2014 at 11:43 PM, Christian Moen  wrote:
> > Great! Congrats and welcome, Areek!
> >
> > Best,
> > Christian
> >
> > On Jan 22, 2014, at 4:26 AM, Robert Muir  wrote:
> >
> > I'm pleased to announce that Areek Zillur has accepted to join our ranks
> as
> > a committer.
> >
> > Areek has been improving suggester support in Lucene and Solr, including
> a
> > revamped Solr component slated for the 4.7 release. [1]
> >
> > Areek, it is tradition that you introduce yourself with a brief bio.
> >
> > Once your account is setup, you should then be able to add yourself to
> the
> > who we are page on the website as well.
> >
> > Congratulations and welcome!
> >
> > [1] https://issues.apache.org/jira/browse/SOLR-5378
> >
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

[JENKINS-MAVEN] Lucene-Solr-Maven-4.x #569: POMs out of sync

2014-01-21 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-Maven-4.x/569/

1 tests failed.
FAILED:  org.apache.solr.cloud.OverseerRolesTest.testDistribSearch

Error Message:
could not set the new overseer

Stack Trace:
java.lang.AssertionError: could not set the new overseer
at 
__randomizedtesting.SeedInfo.seed([58C023A7A589ABC9:D926ADBFD2D6CBF5]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.OverseerRolesTest.addOverseerRole2ExistingNodes(OverseerRolesTest.java:122)
at 
org.apache.solr.cloud.OverseerRolesTest.doTest(OverseerRolesTest.java:88)




Build Log:
[...truncated 52223 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-4.x/build.xml:482: 
The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-4.x/build.xml:176: 
The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-4.x/extra-targets.xml:77:
 Java returned: 1

Total time: 129 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5509) ChaosMonkeyNothingIsSafeTest rare fails due to TOLEADER retries.


[ 
https://issues.apache.org/jira/browse/SOLR-5509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878069#comment-13878069
 ] 

Mark Miller commented on SOLR-5509:
---

It wasn't, I've seen it since. But now I'm thinking it's probably a variation 
of SOLR-5650, but all the replicas just happen to be in sync.

> ChaosMonkeyNothingIsSafeTest rare fails due to TOLEADER retries.
> 
>
> Key: SOLR-5509
> URL: https://issues.apache.org/jira/browse/SOLR-5509
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 5.0, 4.7
>
> Attachments: cmns-test-cloud-off-by1-control-2.log
>
>
> {noformat}
>[junit4]   2> 41386 T28 C21 P57194 oasup.LogUpdateProcessor.finish 
> [collection1] webapp= path=/update params={wt=javabin&CONTROL=TRUE&version=2} 
> {add=[50086 (1452880907553734656)]} 0 142
>[junit4]   2> 42009 T133 C27 P60411 oasup.LogUpdateProcessor.finish 
> [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[50086 
> (1452880908206997504)]} 0 254
>[junit4]   2> 42323 T27 C21 P57194 oasup.LogUpdateProcessor.finish 
> [collection1] webapp= path=/update params={wt=javabin&CONTROL=TRUE&version=2} 
> {delete=[50086 (-1452880908537298944)]} 0 2
>[junit4]   2> 42327 T131 C27 P60411 oasup.LogUpdateProcessor.finish 
> [collection1] webapp= path=/update params={wt=javabin&version=2} 
> {delete=[50086 (-1452880908542541824)]} 0 1
>[junit4]   2> 42622 T132 C27 P60411 oasup.LogUpdateProcessor.finish 
> [collection1] webapp= path=/update 
> params={update.distrib=TOLEADER&wt=javabin&version=2} {add=[50086 
> (1452880908850823168)]} 0 1
>[junit4]   2> 42623 T48 C22 P57136 oasup.LogUpdateProcessor.finish 
> [collection1] webapp= path=/update params={wt=javabin&version=2} 
> {add=[50086]} 0 1223
>[junit4]   2> ## Only in cloudDocList: [{id=50086}]
>[junit4]   2>  cloudClient 
> :{numFound=1,start=0,docs=[SolrDocument{id=50086, 
> _version_=1452880908850823168}]}
> h
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5594) Enable using extended field types with prefix queries for non-default encoded strings

2014-01-21 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878055#comment-13878055
 ] 

Hoss Man commented on SOLR-5594:


Anshum: this all looks great, except for 3 things...

* my point about testing that the querys produced by all the parsers are 
.equals() was about the queries produced by each parser being equal _to each 
other_.  The point is regardless of which parser you use, the resulting queries 
should be the same because we now delegate to the FieldType.  So instead of 
distinct testQuerySimple, testQueryLucene, testQueryPrefix methods each using a 
single parser, we should have a test that shows something like this...{code}
assertQueryEquals(req,
  "{!lucene df=swap_foo_bar_in_prefix_query}foo*",
  "{!simple f=swap_foo_bar_in_prefix_query}foo*",
  "{!prefix f=swap_foo_bar_in_prefix_query}foo",
  ...);
assertQueryEquals(req,
  "{!lucene}int_prefix_as_range:[42 TO *]",
  "{!lucene}int_prefix_as_range:42*",
  "{!simple}int_prefix_as_range:[42 TO *]",
  "{!simple}int_prefix_as_range:42*",
  "{!prefix f=int_prefix_as_range}42",
  ...);
{code}...you see what i mean?
* {{new Random()}} is not allowed - you need to use {{random()}} inherited from 
the test baseclass. you can use "ant precommit" to help spot mistakes like this.
* taking a second look at your assertions about swap_foo_bar_in_prefix_query, i 
don't actually think they are very strong, since (unless i'm missing 
something?) the number of docs containing "foo" prefixes and the number of docs 
containing "bar" prefixes will be the same.  you should change the math so that 
one is less common then the other, and mix in some values that don't match 
either prefix as well.  (ie: let the field be multivalued, and if the id is a 
multiple of 3, add a foo term, if it's a multiple of 7 add a bar term, and if 
it's a multiple of 11 add a zzz term - then assert the expected counts based on 
the total number of docs)


> Enable using extended field types with prefix queries for non-default encoded 
> strings
> -
>
> Key: SOLR-5594
> URL: https://issues.apache.org/jira/browse/SOLR-5594
> Project: Solr
>  Issue Type: Improvement
>  Components: query parsers, Schema and Analysis
>Affects Versions: 4.6
>Reporter: Anshum Gupta
>Assignee: Anshum Gupta
>Priority: Minor
> Attachments: SOLR-5594-branch_4x.patch, SOLR-5594.patch, 
> SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch
>
>
> Enable users to be able to use prefix query with custom field types with 
> non-default encoding/decoding for queries more easily. e.g. having a custom 
> field work with base64 encoded query strings.
> Currently, the workaround for it is to have the override at getRewriteMethod 
> level. Perhaps having the prefixQuery also use the calling FieldType's 
> readableToIndexed method would work better.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5650) When mixing adds and deletes, it appears there is a corner case where peersync can bring back a deleted update.


[ 
https://issues.apache.org/jira/browse/SOLR-5650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878037#comment-13878037
 ] 

ASF subversion and git services commented on SOLR-5650:
---

Commit 1560223 from [~markrmil...@gmail.com] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1560223 ]

SOLR-5650: When a replica becomes a leader, only peer sync with other replicas 
that last published an ACTIVE state.

> When mixing adds and deletes, it appears there is a corner case where 
> peersync can bring back a deleted update.
> ---
>
> Key: SOLR-5650
> URL: https://issues.apache.org/jira/browse/SOLR-5650
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 5.0, 4.7
>
> Attachments: SOLR-5650.patch, SOLR-5650.patch, solr.log.tar.gz
>
>




--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5407) Deadlock? while indexing reader fields in cascaded threads

2014-01-21 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878032#comment-13878032
 ] 

Luis Filipe Nassif commented on LUCENE-5407:


Actually the pre-processing approach could be done, but Tika does text and 
embedded doc extraction at the same time and we don't want to repeat part of 
the processing.

Beside this, I expected Lucene should work in this "thread-chaining" setup.

> Deadlock? while indexing reader fields in cascaded threads
> --
>
> Key: LUCENE-5407
> URL: https://issues.apache.org/jira/browse/LUCENE-5407
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.6
> Environment: Windows 7 64 bits, JRE 1.7.0_25 64 bits
>Reporter: Luis Filipe Nassif
> Attachments: Test.java, thread_dump.txt
>
>
> Apparently I found a deadlock problem with IndexWriter using Reader Fields in 
> a cascaded thread design to add documents (I am working on an application 
> integrating Tika, which has the capability to add embedded documents to the 
> index as independent documents as they are found). The attached code 
> illustrates the problem. Sometimes it stops processing, at least one of the 
> threads remains in WAITING state. It must be executed no more than 5 times in 
> my environment to trigger the problem.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5650) When mixing adds and deletes, it appears there is a corner case where peersync can bring back a deleted update.


[ 
https://issues.apache.org/jira/browse/SOLR-5650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878030#comment-13878030
 ] 

ASF subversion and git services commented on SOLR-5650:
---

Commit 1560221 from [~markrmil...@gmail.com] in branch 'dev/trunk'
[ https://svn.apache.org/r1560221 ]

SOLR-5650: When a replica becomes a leader, only peer sync with other replicas 
that last published an ACTIVE state.

> When mixing adds and deletes, it appears there is a corner case where 
> peersync can bring back a deleted update.
> ---
>
> Key: SOLR-5650
> URL: https://issues.apache.org/jira/browse/SOLR-5650
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 5.0, 4.7
>
> Attachments: SOLR-5650.patch, SOLR-5650.patch, solr.log.tar.gz
>
>




--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5652) Heisenbug in DistribCursorPagingTest: "walk already seen ..."

2014-01-21 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878020#comment-13878020
 ] 

Hoss Man commented on SOLR-5652:



bq. The next steps I can think of: ...

Done.

It also occurred to me that even though the test is just looping over the all 
the sort fields, and even though the requests are *suppose* to be completely 
independent of each other; the ordering of the sort fields _was_ based on a Map 
iteration -- which might explain why the seeds don't reproduce for other people 
on other OSs/JVMs (presuming that there is a bug in the code itself, and that 
bug is dependent on some past behavior).  So in addition to the extra logging i 
mentioned wanting to add, the code that gets the list of all sort field names 
now explicitly sorts those names, then shuffles them using the tests 
{{random()}} ... so hopefully the failures will now be more easy to reproduce.

> Heisenbug in DistribCursorPagingTest: "walk already seen ..."
> -
>
> Key: SOLR-5652
> URL: https://issues.apache.org/jira/browse/SOLR-5652
> Project: Solr
>  Issue Type: Bug
>Reporter: Hoss Man
>Assignee: Hoss Man
> Attachments: jenkins.thetaphi.de_Lucene-Solr-4.x-MacOSX_1200.log.txt, 
> jenkins.thetaphi.de_Lucene-Solr-4.x-MacOSX_1217.log.txt
>
>
> Twice now, Uwe's jenkins has encountered a "walk already seen ..." assertion 
> failure from DistribCursorPagingTest that I've been unable to fathom, let 
> alone reproduce.
> Using this as a tracking issue to try and make sense of it.
> Summary of things noticed so far (in 2 failures):
> * So far only seen on http://jenkins.thetaphi.de
> * So far only seen on MacOSX
> * So far only seen on branch 4x
> * So far seen on both Java6 and Java7
> * fails occured in first block of randomized testing: 
> ** we've indexed a small number of randomized docs
> ** we're explicitly looping over every field and sorting in both directions
> * fails were both when sorting on one of the "\*_dv_last desc" fields 
> (docValues=true, sortMissingLast=true) 
> ** sort on same field asc has always worked fine just before this (fields are 
> in arbitrary order, but "asc" always tried before "desc")
> ** sorting on some other random fields has sometimes been tried before this 
> and worked
> (specifics of each failure seen in the wild recorded in comments)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5652) Heisenbug in DistribCursorPagingTest: "walk already seen ..."


[ 
https://issues.apache.org/jira/browse/SOLR-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878017#comment-13878017
 ] 

ASF subversion and git services commented on SOLR-5652:
---

Commit 1560220 from hoss...@apache.org in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1560220 ]

SOLR-5652: Make test more deterministicly random, and increase logging to try 
and track down strange test failure (merge r1560219)

> Heisenbug in DistribCursorPagingTest: "walk already seen ..."
> -
>
> Key: SOLR-5652
> URL: https://issues.apache.org/jira/browse/SOLR-5652
> Project: Solr
>  Issue Type: Bug
>Reporter: Hoss Man
>Assignee: Hoss Man
> Attachments: jenkins.thetaphi.de_Lucene-Solr-4.x-MacOSX_1200.log.txt, 
> jenkins.thetaphi.de_Lucene-Solr-4.x-MacOSX_1217.log.txt
>
>
> Twice now, Uwe's jenkins has encountered a "walk already seen ..." assertion 
> failure from DistribCursorPagingTest that I've been unable to fathom, let 
> alone reproduce.
> Using this as a tracking issue to try and make sense of it.
> Summary of things noticed so far (in 2 failures):
> * So far only seen on http://jenkins.thetaphi.de
> * So far only seen on MacOSX
> * So far only seen on branch 4x
> * So far seen on both Java6 and Java7
> * fails occured in first block of randomized testing: 
> ** we've indexed a small number of randomized docs
> ** we're explicitly looping over every field and sorting in both directions
> * fails were both when sorting on one of the "\*_dv_last desc" fields 
> (docValues=true, sortMissingLast=true) 
> ** sort on same field asc has always worked fine just before this (fields are 
> in arbitrary order, but "asc" always tried before "desc")
> ** sorting on some other random fields has sometimes been tried before this 
> and worked
> (specifics of each failure seen in the wild recorded in comments)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5410) Add fuzziness support to SimpleQueryParser

2014-01-21 Thread Lee Hinman (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878015#comment-13878015
 ] 

Lee Hinman commented on LUCENE-5410:


I would like to work on this also, if that's alright.

> Add fuzziness support to SimpleQueryParser
> --
>
> Key: LUCENE-5410
> URL: https://issues.apache.org/jira/browse/LUCENE-5410
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/queryparser
>Affects Versions: 4.7
>Reporter: Lee Hinman
>Priority: Minor
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> It would be nice to add fuzzy query support to the {{SimpleQueryParser}} so 
> that:
> {{foo~2}}
> generates a {{FuzzyQuery}} with an max edit distance of 2 and:
> {{"foo bar"~2}}
> generates a {{PhraseQuery}} with a slop of 2.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-5410) Add fuzziness support to SimpleQueryParser

2014-01-21 Thread Lee Hinman (JIRA)

Lee Hinman created LUCENE-5410:
--

 Summary: Add fuzziness support to SimpleQueryParser
 Key: LUCENE-5410
 URL: https://issues.apache.org/jira/browse/LUCENE-5410
 Project: Lucene - Core
  Issue Type: Improvement
  Components: core/queryparser
Affects Versions: 4.7
Reporter: Lee Hinman
Priority: Minor


It would be nice to add fuzzy query support to the {{SimpleQueryParser}} so 
that:

{{foo~2}}

generates a {{FuzzyQuery}} with an max edit distance of 2 and:

{{"foo bar"~2}}

generates a {{PhraseQuery}} with a slop of 2.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5652) Heisenbug in DistribCursorPagingTest: "walk already seen ..."

2014-01-21 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878011#comment-13878011
 ] 

ASF subversion and git services commented on SOLR-5652:
---

Commit 1560219 from hoss...@apache.org in branch 'dev/trunk'
[ https://svn.apache.org/r1560219 ]

SOLR-5652: Make test more deterministicly random, and increase logging to try 
and track down strange test failure

> Heisenbug in DistribCursorPagingTest: "walk already seen ..."
> -
>
> Key: SOLR-5652
> URL: https://issues.apache.org/jira/browse/SOLR-5652
> Project: Solr
>  Issue Type: Bug
>Reporter: Hoss Man
>Assignee: Hoss Man
> Attachments: jenkins.thetaphi.de_Lucene-Solr-4.x-MacOSX_1200.log.txt, 
> jenkins.thetaphi.de_Lucene-Solr-4.x-MacOSX_1217.log.txt
>
>
> Twice now, Uwe's jenkins has encountered a "walk already seen ..." assertion 
> failure from DistribCursorPagingTest that I've been unable to fathom, let 
> alone reproduce.
> Using this as a tracking issue to try and make sense of it.
> Summary of things noticed so far (in 2 failures):
> * So far only seen on http://jenkins.thetaphi.de
> * So far only seen on MacOSX
> * So far only seen on branch 4x
> * So far seen on both Java6 and Java7
> * fails occured in first block of randomized testing: 
> ** we've indexed a small number of randomized docs
> ** we're explicitly looping over every field and sorting in both directions
> * fails were both when sorting on one of the "\*_dv_last desc" fields 
> (docValues=true, sortMissingLast=true) 
> ** sort on same field asc has always worked fine just before this (fields are 
> in arbitrary order, but "asc" always tried before "desc")
> ** sorting on some other random fields has sometimes been tried before this 
> and worked
> (specifics of each failure seen in the wild recorded in comments)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-5594) Enable using extended field types with prefix queries for non-default encoded strings


[ 
https://issues.apache.org/jira/browse/SOLR-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877980#comment-13877980
 ] 

Anshum Gupta edited comment on SOLR-5594 at 1/21/14 11:16 PM:
--

Patch with the following changes:
* Added back SolrQueryParserBase.newPrefixQuery so that it doesn't break 
backward compatibility.
* Added some javadocs to the test classes
* Changed the names of custom fields in the new test schema.
* Added tests that show {{\{!prefix\}}}, {{\{!simple\}}}, and {{\{!lucene\}}} 
all produce queries that are .equals() for custom field.


was (Author: anshumg):
Patch with the following changes:
* Added back SolrQueryParserBase.newPrefixQuery so that it doesn't break 
backward compatibility.
* Added some javadocs to the test classes
* Changed the names of custom fields in the new test schema.
* Added tests that show {!prefix}, {!simple}, and {!lucene} all produce queries 
that are .equals() for custom field.

> Enable using extended field types with prefix queries for non-default encoded 
> strings
> -
>
> Key: SOLR-5594
> URL: https://issues.apache.org/jira/browse/SOLR-5594
> Project: Solr
>  Issue Type: Improvement
>  Components: query parsers, Schema and Analysis
>Affects Versions: 4.6
>Reporter: Anshum Gupta
>Assignee: Anshum Gupta
>Priority: Minor
> Attachments: SOLR-5594-branch_4x.patch, SOLR-5594.patch, 
> SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch
>
>
> Enable users to be able to use prefix query with custom field types with 
> non-default encoding/decoding for queries more easily. e.g. having a custom 
> field work with base64 encoded query strings.
> Currently, the workaround for it is to have the override at getRewriteMethod 
> level. Perhaps having the prefixQuery also use the calling FieldType's 
> readableToIndexed method would work better.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5594) Enable using extended field types with prefix queries for non-default encoded strings


 [ 
https://issues.apache.org/jira/browse/SOLR-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anshum Gupta updated SOLR-5594:
---

Attachment: SOLR-5594.patch

Patch with the following changes:
* Added back SolrQueryParserBase.newPrefixQuery so that it doesn't break 
backward compatibility.
* Added some javadocs to the test classes
* Changed the names of custom fields in the new test schema.
* Added tests that show {!prefix}, {!simple}, and {!lucene} all produce queries 
that are .equals() for custom field.

> Enable using extended field types with prefix queries for non-default encoded 
> strings
> -
>
> Key: SOLR-5594
> URL: https://issues.apache.org/jira/browse/SOLR-5594
> Project: Solr
>  Issue Type: Improvement
>  Components: query parsers, Schema and Analysis
>Affects Versions: 4.6
>Reporter: Anshum Gupta
>Assignee: Anshum Gupta
>Priority: Minor
> Attachments: SOLR-5594-branch_4x.patch, SOLR-5594.patch, 
> SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch
>
>
> Enable users to be able to use prefix query with custom field types with 
> non-default encoding/decoding for queries more easily. e.g. having a custom 
> field work with base64 encoded query strings.
> Currently, the workaround for it is to have the override at getRewriteMethod 
> level. Perhaps having the prefixQuery also use the calling FieldType's 
> readableToIndexed method would work better.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5650) When mixing adds and deletes, it appears there is a corner case where peersync can bring back a deleted update.


[ 
https://issues.apache.org/jira/browse/SOLR-5650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877975#comment-13877975
 ] 

Mark Miller commented on SOLR-5650:
---

You will still have the possibility of bringing back documents on startup by 
syncing with a replica that is out of date by less than 100 updates on cluster 
startup. The number of recent docs that would come back is fairly small though, 
and the chances of it happening pretty slim. We may be able to do more around 
that as well.

> When mixing adds and deletes, it appears there is a corner case where 
> peersync can bring back a deleted update.
> ---
>
> Key: SOLR-5650
> URL: https://issues.apache.org/jira/browse/SOLR-5650
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 5.0, 4.7
>
> Attachments: SOLR-5650.patch, SOLR-5650.patch, solr.log.tar.gz
>
>




--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5650) When mixing adds and deletes, it appears there is a corner case where peersync can bring back a deleted update.


 [ 
https://issues.apache.org/jira/browse/SOLR-5650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-5650:
--

Attachment: SOLR-5650.patch

> When mixing adds and deletes, it appears there is a corner case where 
> peersync can bring back a deleted update.
> ---
>
> Key: SOLR-5650
> URL: https://issues.apache.org/jira/browse/SOLR-5650
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 5.0, 4.7
>
> Attachments: SOLR-5650.patch, SOLR-5650.patch, solr.log.tar.gz
>
>




--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5476) Overseer Role for nodes


[ 
https://issues.apache.org/jira/browse/SOLR-5476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877964#comment-13877964
 ] 

Mark Miller commented on SOLR-5476:
---

This test is still failing on almost all of my local test runs.

> Overseer Role for nodes
> ---
>
> Key: SOLR-5476
> URL: https://issues.apache.org/jira/browse/SOLR-5476
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
>Assignee: Noble Paul
> Fix For: 5.0, 4.7
>
> Attachments: SOLR-5476.patch, SOLR-5476.patch, SOLR-5476.patch, 
> SOLR-5476.patch, SOLR-5476.patch
>
>
> In a very large cluster the Overseer is likely to be overloaded.If the same 
> node is a serving a few other shards it can lead to OverSeer getting slowed 
> down due to GC pauses , or simply too much of work  . If the cluster is 
> really large , it is possible to dedicate high end h/w for OverSeers
> It works as a new collection admin command
> command=addrole&role=overseer&node=192.168.1.5:8983_solr
> This results in the creation of a entry in the /roles.json in ZK which would 
> look like the following
> {code:javascript}
> {
> "overseer" : ["192.168.1.5:8983_solr"]
> }
> {code}
> If a node is designated for overseer it gets preference over others when 
> overseer election takes place. If no designated servers are available another 
> random node would become the Overseer.
> Later on, if one of the designated nodes are brought up ,it would take over 
> the Overseer role from the current Overseer to become the Overseer of the 
> system



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Areek Zillur as Lucene/Solr committer!

2014-01-21 Thread Dawid Weiss

Congratulations and welcome, Areek!

On Tue, Jan 21, 2014 at 11:43 PM, Christian Moen  wrote:
> Great! Congrats and welcome, Areek!
>
> Best,
> Christian
>
> On Jan 22, 2014, at 4:26 AM, Robert Muir  wrote:
>
> I'm pleased to announce that Areek Zillur has accepted to join our ranks as
> a committer.
>
> Areek has been improving suggester support in Lucene and Solr, including a
> revamped Solr component slated for the 4.7 release. [1]
>
> Areek, it is tradition that you introduce yourself with a brief bio.
>
> Once your account is setup, you should then be able to add yourself to the
> who we are page on the website as well.
>
> Congratulations and welcome!
>
> [1] https://issues.apache.org/jira/browse/SOLR-5378
>
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5650) When mixing adds and deletes, it appears there is a corner case where peersync can bring back a deleted update.


 [ 
https://issues.apache.org/jira/browse/SOLR-5650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-5650:
--

Attachment: SOLR-5650.patch

> When mixing adds and deletes, it appears there is a corner case where 
> peersync can bring back a deleted update.
> ---
>
> Key: SOLR-5650
> URL: https://issues.apache.org/jira/browse/SOLR-5650
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 5.0, 4.7
>
> Attachments: SOLR-5650.patch, solr.log.tar.gz
>
>




--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Areek Zillur as Lucene/Solr committer!

2014-01-21 Thread Christian Moen

Great! Congrats and welcome, Areek!

Best,
Christian

On Jan 22, 2014, at 4:26 AM, Robert Muir  wrote:

> I'm pleased to announce that Areek Zillur has accepted to join our ranks as a 
> committer.
> 
> Areek has been improving suggester support in Lucene and Solr, including a 
> revamped Solr component slated for the 4.7 release. [1]
> 
> Areek, it is tradition that you introduce yourself with a brief bio.
> 
> Once your account is setup, you should then be able to add yourself to the 
> who we are page on the website as well.
> 
> Congratulations and welcome!
> 
> [1] https://issues.apache.org/jira/browse/SOLR-5378
>

[jira] [Commented] (SOLR-5652) Heisenbug in DistribCursorPagingTest: "walk already seen ..."

2014-01-21 Thread Uwe Schindler (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877914#comment-13877914
 ] 

Uwe Schindler commented on SOLR-5652:
-

Heisenbug on Policeman Jenkins :-)

> Heisenbug in DistribCursorPagingTest: "walk already seen ..."
> -
>
> Key: SOLR-5652
> URL: https://issues.apache.org/jira/browse/SOLR-5652
> Project: Solr
>  Issue Type: Bug
>Reporter: Hoss Man
>Assignee: Hoss Man
> Attachments: jenkins.thetaphi.de_Lucene-Solr-4.x-MacOSX_1200.log.txt, 
> jenkins.thetaphi.de_Lucene-Solr-4.x-MacOSX_1217.log.txt
>
>
> Twice now, Uwe's jenkins has encountered a "walk already seen ..." assertion 
> failure from DistribCursorPagingTest that I've been unable to fathom, let 
> alone reproduce.
> Using this as a tracking issue to try and make sense of it.
> Summary of things noticed so far (in 2 failures):
> * So far only seen on http://jenkins.thetaphi.de
> * So far only seen on MacOSX
> * So far only seen on branch 4x
> * So far seen on both Java6 and Java7
> * fails occured in first block of randomized testing: 
> ** we've indexed a small number of randomized docs
> ** we're explicitly looping over every field and sorting in both directions
> * fails were both when sorting on one of the "\*_dv_last desc" fields 
> (docValues=true, sortMissingLast=true) 
> ** sort on same field asc has always worked fine just before this (fields are 
> in arbitrary order, but "asc" always tried before "desc")
> ** sorting on some other random fields has sometimes been tried before this 
> and worked
> (specifics of each failure seen in the wild recorded in comments)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5652) Heisenbug in DistribCursorPagingTest: "walk already seen ..."


 [ 
https://issues.apache.org/jira/browse/SOLR-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-5652:
---

Attachment: jenkins.thetaphi.de_Lucene-Solr-4.x-MacOSX_1217.log.txt
jenkins.thetaphi.de_Lucene-Solr-4.x-MacOSX_1200.log.txt

Full logs from the two builds that have failed so far

> Heisenbug in DistribCursorPagingTest: "walk already seen ..."
> -
>
> Key: SOLR-5652
> URL: https://issues.apache.org/jira/browse/SOLR-5652
> Project: Solr
>  Issue Type: Bug
>Reporter: Hoss Man
>Assignee: Hoss Man
> Attachments: jenkins.thetaphi.de_Lucene-Solr-4.x-MacOSX_1200.log.txt, 
> jenkins.thetaphi.de_Lucene-Solr-4.x-MacOSX_1217.log.txt
>
>
> Twice now, Uwe's jenkins has encountered a "walk already seen ..." assertion 
> failure from DistribCursorPagingTest that I've been unable to fathom, let 
> alone reproduce.
> Using this as a tracking issue to try and make sense of it.
> Summary of things noticed so far (in 2 failures):
> * So far only seen on http://jenkins.thetaphi.de
> * So far only seen on MacOSX
> * So far only seen on branch 4x
> * So far seen on both Java6 and Java7
> * fails occured in first block of randomized testing: 
> ** we've indexed a small number of randomized docs
> ** we're explicitly looping over every field and sorting in both directions
> * fails were both when sorting on one of the "\*_dv_last desc" fields 
> (docValues=true, sortMissingLast=true) 
> ** sort on same field asc has always worked fine just before this (fields are 
> in arbitrary order, but "asc" always tried before "desc")
> ** sorting on some other random fields has sometimes been tried before this 
> and worked
> (specifics of each failure seen in the wild recorded in comments)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3565) StandardQueryParser generates incorrect query for groups containing one term if using default operator AND

2014-01-21 Thread Trejkaz (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877912#comment-13877912
 ] 

Trejkaz commented on LUCENE-3565:
-

Seems to have been fixed in 3.6.2.


> StandardQueryParser generates incorrect query for groups containing one term 
> if using default operator AND
> --
>
> Key: LUCENE-3565
> URL: https://issues.apache.org/jira/browse/LUCENE-3565
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/queryparser
>Affects Versions: 3.4
>Reporter: Trejkaz
>  Labels: queryparser
>
> The following test demonstrates a bug in StandardQueryParser when 
> DEFAULT_OPERATOR is set to AND:
> {code}
> @Test
> public void testDefaultOperatorSingleGrouped() throws Exception
> {
> StandardQueryParser parser = new StandardQueryParser();
> 
> parser.getQueryConfigHandler().set(StandardQueryConfigHandler.ConfigurationKeys.DEFAULT_OPERATOR,
>  StandardQueryConfigHandler.Operator.AND);
> BooleanQuery expectedQuery = new BooleanQuery();
> BooleanQuery innerQuery1 = new BooleanQuery();
> innerQuery1.add(new TermQuery(new Term("text", "a")), 
> BooleanClause.Occur.MUST);
> expectedQuery.add(innerQuery1, BooleanClause.Occur.SHOULD);
> BooleanQuery innerQuery2 = new BooleanQuery();
> innerQuery2.add(new TermQuery(new Term("text", "b")), 
> BooleanClause.Occur.MUST);
> expectedQuery.add(innerQuery2, BooleanClause.Occur.SHOULD);
> Query actualQuery = parser.parse("(a) OR (b)", "text");
> assertEquals("Wrong query was generated", expectedQuery, actualQuery);
> }
> {code}
> BooleanSingleChildOptimizationQueryNodeProcessor appears to be responsible 
> because if I remove it, the correct query is generated (however, doing so 
> breaks a number of other tests of our own.)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5526) Query parser extends standard cause NPE on Solr startup


[ 
https://issues.apache.org/jira/browse/SOLR-5526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877902#comment-13877902
 ] 

Hoss Man commented on SOLR-5526:


Hey Vitaliy,

For the most part, these tests look great -- just a few minor things i think we 
should try to clean up...

* QParserPlugin.standardPlugins's javadoc needs to point out the importance of 
these names being static & final so people aren't surprised by these  tests 
when new parsers are added in the future.
* TestStandardQParsers is doing something sufficiently odd that it really needs 
some javadocs explaining why it exists (ie: mention the class loading problems 
associated if there is a standardPlugin that has a non-static, non-final name, 
with an {{@see}} this issue, {{@see QParserPlugin.standardPlugins}}, etc...)
* we should probably make TestStandardQParsers assert that the static & final 
name it finds in each class matches the name associated in 
QParserPlugin.standardPlugins.
* solrconfig-query-parser-init.xml has a cut & paste comment referring to an 
unrelated test.
* TestInitQParser should have a javadoc comment explaining what the point of 
the test is
* TestInitQParser should actaully do a query using the "fail" parser registered 
in the config, to help future-proof us against someone unwittingly changing the 
test config in a way that defeats the point of the test.

...if you want to take a crack at this that would be great, if not i'll try to 
make some time later this week.


> Query parser extends standard cause NPE on Solr startup
> ---
>
> Key: SOLR-5526
> URL: https://issues.apache.org/jira/browse/SOLR-5526
> Project: Solr
>  Issue Type: Bug
>  Components: query parsers
>Affects Versions: 4.5.1, 4.6, 5.0
> Environment: Linux, Java 1.7.0_45
>Reporter: Nikolay Khitrin
>Priority: Blocker
> Attachments: NPE_load_trace, SOLR-5526-final-names.patch, 
> SOLR-5526-final-names.patch, SOLR-5526-tests.patch, SOLR-5526.patch, 
> SOLR-5526.patch
>
>
> Adding any custom query parser extending standard one with non-final field 
> {{NAME}} lead to messy {{NullPointerException}} during Solr startup.
> Definition of standard parsers is located in  
> {{QParserPlugin.standardPlugins}} static array. The array contains names from 
> static {{NAME}} fields and classes for each plugin.   
> But all of listed parsers are derived from {{QParserPlugin}}, so we have 
> circular dependency of static fields.
> Normally, class loader start initializing {{QParserPlugin}} before all listed 
> plugins in {{SolrCore.initQParsers}}, and array elements referenced to 
> {{NAME}} plugins' fields are filled properly.
> Custom parsers are instantiated before standard parsers. And when we subclass 
> plugin with non-final {{NAME}} field and add it to Solr via 
> {{solrconfig.xml}}, class loader start loading our class before 
> {{QParserPlugin}}. Because {{QParserPlugin}} is a superclass for plugin, it 
> must be initialized before subclasses, and static dereferencing cause null 
> elements in {{standardPlugins}} array because it filled before {{NAME}} field 
> of loading plugin's superclass.
> How to reproduce:
> # Checkout Solr (trunk or stable)
> # Add the following line to solr/example/solr/collection1/conf/solrconfig.xml
>   {{}}
> # Call ant run-example in solr folder
> Possible workarounds:
> * dev-workaround: add {{int workaround = 
> QParserPlugin.standardPlugins.length;}} as a first line to
>   {{SolrCore.initQParsers}}
> * user-workaround: add plugin with final {{NAME}} field (edismax) to 
> solrconfig.xml  before subclasses of standard plugins. 
>   {{ class="solr.search.ExtendedDismaxQParserPlugin"/>}}
>   
> Possible fix:
> Move {{standardPlugins}} to new final class to break circular dependency.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Areek Zillur as Lucene/Solr committer!

2014-01-21 Thread Jan Høydahl

Welcome Areek!

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

21. jan. 2014 kl. 20:26 skrev Robert Muir :

> I'm pleased to announce that Areek Zillur has accepted to join our ranks as a 
> committer.
> 
> Areek has been improving suggester support in Lucene and Solr, including a 
> revamped Solr component slated for the 4.7 release. [1]
> 
> Areek, it is tradition that you introduce yourself with a brief bio.
> 
> Once your account is setup, you should then be able to add yourself to the 
> who we are page on the website as well.
> 
> Congratulations and welcome!
> 
> [1] https://issues.apache.org/jira/browse/SOLR-5378
>

Re: Welcome Areek Zillur as Lucene/Solr committer!

2014-01-21 Thread Alan Woodward

Welcome Areek!

Alan Woodward
www.flax.co.uk


On 21 Jan 2014, at 20:30, Areek Zillur wrote:

> Thanks Robert! I am very pleased to be a committer to Lucene/Solr!
> 
> I am originally from Dhaka, Bangladesh. I am currently a 4th year Computer 
> Engineering
> student at University of Waterloo in Canada. I was fortunate enough to have 
> multiple internships 
> all over North America through the university's co-op program.
> I was first introduced to Lucene/Solr in one of my work-terms at A9 and 
> loved it.
> 
> I really enjoy the open-source development and the friendliness of the 
> community 
> behind Lucene/Solr.
> In my free time, I enjoy working on working on my  recreational algorithmic 
> trading system and learning 
> new programming languages.
> 
> I hope to continue to work on Lucene/Solr and learn a lot more from the 
> community!
> 
> Thanks,
> 
> Areek Zillur
> 
> 
> On Tue, Jan 21, 2014 at 11:41 AM, Yonik Seeley  wrote:
> Welcome Areek!
> 
> -Yonik
> http://heliosearch.com -- making solr shine
> 
> On Tue, Jan 21, 2014 at 2:26 PM, Robert Muir  wrote:
> > I'm pleased to announce that Areek Zillur has accepted to join our ranks as
> > a committer.
> >
> > Areek has been improving suggester support in Lucene and Solr, including a
> > revamped Solr component slated for the 4.7 release. [1]
> >
> > Areek, it is tradition that you introduce yourself with a brief bio.
> >
> > Once your account is setup, you should then be able to add yourself to the
> > who we are page on the website as well.
> >
> > Congratulations and welcome!
> >
> > [1] https://issues.apache.org/jira/browse/SOLR-5378
> >
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
> 
>

RE: Welcome Areek Zillur as Lucene/Solr committer!

2014-01-21 Thread Uwe Schindler

Welcome Areek!

 

-

Uwe Schindler

H.-H.-Meier-Allee 63, D-28213 Bremen

  http://www.thetaphi.de

eMail: u...@thetaphi.de

 

From: Robert Muir [mailto:rcm...@gmail.com] 
Sent: Tuesday, January 21, 2014 8:26 PM
To: dev@lucene.apache.org
Subject: Welcome Areek Zillur as Lucene/Solr committer!

 

I'm pleased to announce that Areek Zillur has accepted to join our ranks as a 
committer.

Areek has been improving suggester support in Lucene and Solr, including a 
revamped Solr component slated for the 4.7 release. [1]

Areek, it is tradition that you introduce yourself with a brief bio.

Once your account is setup, you should then be able to add yourself to the who 
we are page on the website as well.

Congratulations and welcome!

[1] https://issues.apache.org/jira/browse/SOLR-5378

Re: Welcome Areek Zillur as Lucene/Solr committer!

2014-01-21 Thread Adrien Grand

Welcome Areek!

On Tue, Jan 21, 2014 at 9:30 PM, Areek Zillur  wrote:
> Thanks Robert! I am very pleased to be a committer to Lucene/Solr!
>
> I am originally from Dhaka, Bangladesh. I am currently a 4th year Computer
> Engineering
> student at University of Waterloo in Canada. I was fortunate enough to have
> multiple internships
> all over North America through the university's co-op program.
> I was first introduced to Lucene/Solr in one of my work-terms at A9 and
> loved it.
>
> I really enjoy the open-source development and the friendliness of the
> community
> behind Lucene/Solr.
> In my free time, I enjoy working on working on my  recreational algorithmic
> trading system and learning
> new programming languages.
>
> I hope to continue to work on Lucene/Solr and learn a lot more from the
> community!
>
> Thanks,
>
> Areek Zillur
>
>
> On Tue, Jan 21, 2014 at 11:41 AM, Yonik Seeley 
> wrote:
>>
>> Welcome Areek!
>>
>> -Yonik
>> http://heliosearch.com -- making solr shine
>>
>> On Tue, Jan 21, 2014 at 2:26 PM, Robert Muir  wrote:
>> > I'm pleased to announce that Areek Zillur has accepted to join our ranks
>> > as
>> > a committer.
>> >
>> > Areek has been improving suggester support in Lucene and Solr, including
>> > a
>> > revamped Solr component slated for the 4.7 release. [1]
>> >
>> > Areek, it is tradition that you introduce yourself with a brief bio.
>> >
>> > Once your account is setup, you should then be able to add yourself to
>> > the
>> > who we are page on the website as well.
>> >
>> > Congratulations and welcome!
>> >
>> > [1] https://issues.apache.org/jira/browse/SOLR-5378
>> >
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>



-- 
Adrien

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5409) ToParentBlockJoinCollector.getTopGroups returns empty Groups

2014-01-21 Thread Peng Cheng (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peng Cheng updated LUCENE-5409:
---

Description: 
A bug is observed to cause unstable results returned by the getTopGroups 
function of class ToParentBlockJoinCollector.

In the scorer generation stage, the ToParentBlockJoinCollector will 
automatically rewrite all the associated ToParentBlockJoinQuery (and their 
subqueries), and save them into its in-memory Look-up table, namely joinQueryID 
(see enroll() method for detail). Unfortunately, in the getTopGroups method, 
the new ToParentBlockJoinQuery parameter is not rewritten (at least users are 
not expected to do so). When the new one is searched in the old lookup table 
(considering the impact of rewrite() on hashCode()), the lookup will largely 
fail and eventually end up with a topGroup collection consisting of only empty 
groups (their hitCounts are guaranteed to be zero).

An easy fix would be to rewrite the original BlockJoinQuery before invoking 
getTopGroups method. However, the computational cost of this is not optimal. A 
better but slightly more complex solution would be to save unrewrited Queries 
into the lookup table.

  was:
In the scorer generation stage, the ToParentBlockJoinCollector will 
automatically rewrite all the associated ToParentBlockJoinQuery (and their 
subqueries), and save them into its in-memory Look-up table, namely joinQueryID 
(see enroll() method for detail). Unfortunately, in the getTopGroups method, 
the new ToParentBlockJoinQuery parameter is not rewritten (at least users are 
not expected to do so). When the new one is searched in the old lookup table 
(considering the impact of rewrite() on hashCode()), the result (namely _slot) 
will always fail and eventually end up with a topGroup collection consisting of 
only empty groups (their hitCounts are guaranteed to be zero).

An easy fix would be to rewrite the original BlockJoinQuery before invoking 
getTopGroups method. However, the computational cost of this is not optimal. A 
better but slightly more complex solution would be to save unrewrited Queries 
into the lookup table.


> ToParentBlockJoinCollector.getTopGroups returns empty Groups
> 
>
> Key: LUCENE-5409
> URL: https://issues.apache.org/jira/browse/LUCENE-5409
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/search
>Affects Versions: 4.6
> Environment: Ubuntu 12.04
>Reporter: Peng Cheng
>Priority: Critical
> Fix For: 4.7
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> A bug is observed to cause unstable results returned by the getTopGroups 
> function of class ToParentBlockJoinCollector.
> In the scorer generation stage, the ToParentBlockJoinCollector will 
> automatically rewrite all the associated ToParentBlockJoinQuery (and their 
> subqueries), and save them into its in-memory Look-up table, namely 
> joinQueryID (see enroll() method for detail). Unfortunately, in the 
> getTopGroups method, the new ToParentBlockJoinQuery parameter is not 
> rewritten (at least users are not expected to do so). When the new one is 
> searched in the old lookup table (considering the impact of rewrite() on 
> hashCode()), the lookup will largely fail and eventually end up with a 
> topGroup collection consisting of only empty groups (their hitCounts are 
> guaranteed to be zero).
> An easy fix would be to rewrite the original BlockJoinQuery before invoking 
> getTopGroups method. However, the computational cost of this is not optimal. 
> A better but slightly more complex solution would be to save unrewrited 
> Queries into the lookup table.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-5409) ToParentBlockJoinCollector.getTopGroups returns empty Groups

2014-01-21 Thread Peng Cheng (JIRA)

Peng Cheng created LUCENE-5409:
--

 Summary: ToParentBlockJoinCollector.getTopGroups returns empty 
Groups
 Key: LUCENE-5409
 URL: https://issues.apache.org/jira/browse/LUCENE-5409
 Project: Lucene - Core
  Issue Type: Bug
  Components: core/search
Affects Versions: 4.6
 Environment: Ubuntu 12.04
Reporter: Peng Cheng
Priority: Critical
 Fix For: 4.7


In the scorer generation stage, the ToParentBlockJoinCollector will 
automatically rewrite all the associated ToParentBlockJoinQuery (and their 
subqueries), and save them into its in-memory Look-up table, namely joinQueryID 
(see enroll() method for detail). Unfortunately, in the getTopGroups method, 
the new ToParentBlockJoinQuery parameter is not rewritten (at least users are 
not expected to do so). When the new one is searched in the old lookup table 
(considering the impact of rewrite() on hashCode()), the result (namely _slot) 
will always fail and eventually end up with a topGroup collection consisting of 
only empty groups (their hitCounts are guaranteed to be zero).

An easy fix would be to rewrite the original BlockJoinQuery before invoking 
getTopGroups method. However, the computational cost of this is not optimal. A 
better but slightly more complex solution would be to save unrewrited Queries 
into the lookup table.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5407) Deadlock? while indexing reader fields in cascaded threads


[ 
https://issues.apache.org/jira/browse/LUCENE-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877850#comment-13877850
 ] 

Luis Filipe Nassif commented on LUCENE-5407:


Thank you for taking a look. Yes, Lucene indexes concurrently really well if I 
feed the docs to parallel threads for indexing with the same indexWriter. Even 
if some indexing thread becomes blocked while reading some doc (eg: slow 
network, infinite loop while extracting text from corrupted doc), the other 
indexing threads continue doing the work. So I thought this kind of thread 
chaining shoud also work fine... Why is Thread-1 blocked here if in a parallel 
setup the indexing threads do not block if one of them blocks while reading 
some doc? 

The use case for this design is that we have terabytes of data to process from 
slow 7200rpm discs and we don't want to pre-process, process and post-process 
the docs. It is better to read each doc only once and do all the processing 
with it, taking advantage of disc and system cache, for example.

> Deadlock? while indexing reader fields in cascaded threads
> --
>
> Key: LUCENE-5407
> URL: https://issues.apache.org/jira/browse/LUCENE-5407
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.6
> Environment: Windows 7 64 bits, JRE 1.7.0_25 64 bits
>Reporter: Luis Filipe Nassif
> Attachments: Test.java, thread_dump.txt
>
>
> Apparently I found a deadlock problem with IndexWriter using Reader Fields in 
> a cascaded thread design to add documents (I am working on an application 
> integrating Tika, which has the capability to add embedded documents to the 
> index as independent documents as they are found). The attached code 
> illustrates the problem. Sometimes it stops processing, at least one of the 
> threads remains in WAITING state. It must be executed no more than 5 times in 
> my environment to trigger the problem.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5405) Exception strategy for analysis improved

2014-01-21 Thread Michael McCandless (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877841#comment-13877841
 ] 

Michael McCandless commented on LUCENE-5405:


+1 to include field name in the infoStream logging ... anything to help in 
debugging analysis problems.

> Exception strategy for analysis improved
> 
>
> Key: LUCENE-5405
> URL: https://issues.apache.org/jira/browse/LUCENE-5405
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Benson Margulies
>
> SOLR-5623 included some conversation about the dilemmas of exception 
> management and reporting in the analysis chain. 
> I've belatedly become educated about the infostream, and this situation is a 
> job for it. The DocInverterPerField can note exceptions in the analysis 
> chain, log out to the infostream, and then rethrow them as before. No 
> wrapping, no muss, no fuss.
> There are comments on this JIRA from a more complex prior idea that readers 
> might want to ignore.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5526) Query parser extends standard cause NPE on Solr startup

2014-01-21 Thread Vitaliy Zhovtyuk (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vitaliy Zhovtyuk updated SOLR-5526:
---

Attachment: SOLR-5526.patch

Combined:
Missed final NAME field fixes. 
Test reproducing NPE on Solr start-up
Test checking final and static NAME field for all standard parsers

> Query parser extends standard cause NPE on Solr startup
> ---
>
> Key: SOLR-5526
> URL: https://issues.apache.org/jira/browse/SOLR-5526
> Project: Solr
>  Issue Type: Bug
>  Components: query parsers
>Affects Versions: 4.5.1, 4.6, 5.0
> Environment: Linux, Java 1.7.0_45
>Reporter: Nikolay Khitrin
>Priority: Blocker
> Attachments: NPE_load_trace, SOLR-5526-final-names.patch, 
> SOLR-5526-final-names.patch, SOLR-5526-tests.patch, SOLR-5526.patch, 
> SOLR-5526.patch
>
>
> Adding any custom query parser extending standard one with non-final field 
> {{NAME}} lead to messy {{NullPointerException}} during Solr startup.
> Definition of standard parsers is located in  
> {{QParserPlugin.standardPlugins}} static array. The array contains names from 
> static {{NAME}} fields and classes for each plugin.   
> But all of listed parsers are derived from {{QParserPlugin}}, so we have 
> circular dependency of static fields.
> Normally, class loader start initializing {{QParserPlugin}} before all listed 
> plugins in {{SolrCore.initQParsers}}, and array elements referenced to 
> {{NAME}} plugins' fields are filled properly.
> Custom parsers are instantiated before standard parsers. And when we subclass 
> plugin with non-final {{NAME}} field and add it to Solr via 
> {{solrconfig.xml}}, class loader start loading our class before 
> {{QParserPlugin}}. Because {{QParserPlugin}} is a superclass for plugin, it 
> must be initialized before subclasses, and static dereferencing cause null 
> elements in {{standardPlugins}} array because it filled before {{NAME}} field 
> of loading plugin's superclass.
> How to reproduce:
> # Checkout Solr (trunk or stable)
> # Add the following line to solr/example/solr/collection1/conf/solrconfig.xml
>   {{}}
> # Call ant run-example in solr folder
> Possible workarounds:
> * dev-workaround: add {{int workaround = 
> QParserPlugin.standardPlugins.length;}} as a first line to
>   {{SolrCore.initQParsers}}
> * user-workaround: add plugin with final {{NAME}} field (edismax) to 
> solrconfig.xml  before subclasses of standard plugins. 
>   {{ class="solr.search.ExtendedDismaxQParserPlugin"/>}}
>   
> Possible fix:
> Move {{standardPlugins}} to new final class to break circular dependency.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Areek Zillur as Lucene/Solr committer!

2014-01-21 Thread Kranti Parisa

Welcome Areek and thanks for sharing your journey, so far, with Solr!

Thanks,
Kranti K. Parisa
http://www.linkedin.com/in/krantiparisa



On Tue, Jan 21, 2014 at 3:43 PM, Michael McCandless <
luc...@mikemccandless.com> wrote:

> Welcome Areek!
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Tue, Jan 21, 2014 at 3:30 PM, Areek Zillur  wrote:
> > Thanks Robert! I am very pleased to be a committer to Lucene/Solr!
> >
> > I am originally from Dhaka, Bangladesh. I am currently a 4th year
> Computer
> > Engineering
> > student at University of Waterloo in Canada. I was fortunate enough to
> have
> > multiple internships
> > all over North America through the university's co-op program.
> > I was first introduced to Lucene/Solr in one of my work-terms at A9 and
> > loved it.
> >
> > I really enjoy the open-source development and the friendliness of the
> > community
> > behind Lucene/Solr.
> > In my free time, I enjoy working on working on my  recreational
> algorithmic
> > trading system and learning
> > new programming languages.
> >
> > I hope to continue to work on Lucene/Solr and learn a lot more from the
> > community!
> >
> > Thanks,
> >
> > Areek Zillur
> >
> >
> > On Tue, Jan 21, 2014 at 11:41 AM, Yonik Seeley 
> > wrote:
> >>
> >> Welcome Areek!
> >>
> >> -Yonik
> >> http://heliosearch.com -- making solr shine
> >>
> >> On Tue, Jan 21, 2014 at 2:26 PM, Robert Muir  wrote:
> >> > I'm pleased to announce that Areek Zillur has accepted to join our
> ranks
> >> > as
> >> > a committer.
> >> >
> >> > Areek has been improving suggester support in Lucene and Solr,
> including
> >> > a
> >> > revamped Solr component slated for the 4.7 release. [1]
> >> >
> >> > Areek, it is tradition that you introduce yourself with a brief bio.
> >> >
> >> > Once your account is setup, you should then be able to add yourself to
> >> > the
> >> > who we are page on the website as well.
> >> >
> >> > Congratulations and welcome!
> >> >
> >> > [1] https://issues.apache.org/jira/browse/SOLR-5378
> >> >
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> >> For additional commands, e-mail: dev-h...@lucene.apache.org
> >>
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Welcome Areek Zillur as Lucene/Solr committer!

2014-01-21 Thread Michael McCandless

Welcome Areek!

Mike McCandless

http://blog.mikemccandless.com


On Tue, Jan 21, 2014 at 3:30 PM, Areek Zillur  wrote:
> Thanks Robert! I am very pleased to be a committer to Lucene/Solr!
>
> I am originally from Dhaka, Bangladesh. I am currently a 4th year Computer
> Engineering
> student at University of Waterloo in Canada. I was fortunate enough to have
> multiple internships
> all over North America through the university's co-op program.
> I was first introduced to Lucene/Solr in one of my work-terms at A9 and
> loved it.
>
> I really enjoy the open-source development and the friendliness of the
> community
> behind Lucene/Solr.
> In my free time, I enjoy working on working on my  recreational algorithmic
> trading system and learning
> new programming languages.
>
> I hope to continue to work on Lucene/Solr and learn a lot more from the
> community!
>
> Thanks,
>
> Areek Zillur
>
>
> On Tue, Jan 21, 2014 at 11:41 AM, Yonik Seeley 
> wrote:
>>
>> Welcome Areek!
>>
>> -Yonik
>> http://heliosearch.com -- making solr shine
>>
>> On Tue, Jan 21, 2014 at 2:26 PM, Robert Muir  wrote:
>> > I'm pleased to announce that Areek Zillur has accepted to join our ranks
>> > as
>> > a committer.
>> >
>> > Areek has been improving suggester support in Lucene and Solr, including
>> > a
>> > revamped Solr component slated for the 4.7 release. [1]
>> >
>> > Areek, it is tradition that you introduce yourself with a brief bio.
>> >
>> > Once your account is setup, you should then be able to add yourself to
>> > the
>> > who we are page on the website as well.
>> >
>> > Congratulations and welcome!
>> >
>> > [1] https://issues.apache.org/jira/browse/SOLR-5378
>> >
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Areek Zillur as Lucene/Solr committer!

2014-01-21 Thread Areek Zillur

Thanks Robert! I am very pleased to be a committer to Lucene/Solr!

I am originally from Dhaka, Bangladesh. I am currently a 4th year Computer
Engineering
student at University of Waterloo in Canada. I was fortunate enough to have
multiple internships
all over North America through the university's co-op program.
I was first introduced to Lucene/Solr in one of my work-terms at A9 and
loved it.

I really enjoy the open-source development and the friendliness of the
community
behind Lucene/Solr.
In my free time, I enjoy working on working on my  recreational algorithmic
trading system and learning
new programming languages.

I hope to continue to work on Lucene/Solr and learn a lot more from the
community!

Thanks,

Areek Zillur

On Tue, Jan 21, 2014 at 11:41 AM, Yonik Seeley wrote:

> Welcome Areek!
>
> -Yonik
> http://heliosearch.com -- making solr shine
>
> On Tue, Jan 21, 2014 at 2:26 PM, Robert Muir  wrote:
> > I'm pleased to announce that Areek Zillur has accepted to join our ranks
> as
> > a committer.
> >
> > Areek has been improving suggester support in Lucene and Solr, including
> a
> > revamped Solr component slated for the 4.7 release. [1]
> >
> > Areek, it is tradition that you introduce yourself with a brief bio.
> >
> > Once your account is setup, you should then be able to add yourself to
> the
> > who we are page on the website as well.
> >
> > Congratulations and welcome!
> >
> > [1] https://issues.apache.org/jira/browse/SOLR-5378
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

[jira] [Commented] (LUCENE-5405) Exception strategy for analysis improved

2014-01-21 Thread Robert Muir (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877789#comment-13877789
 ] 

Robert Muir commented on LUCENE-5405:
-

Benson: this patch looks great.

I'll wait a bit for any feedback and test it out later today. 

> Exception strategy for analysis improved
> 
>
> Key: LUCENE-5405
> URL: https://issues.apache.org/jira/browse/LUCENE-5405
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Benson Margulies
>
> SOLR-5623 included some conversation about the dilemmas of exception 
> management and reporting in the analysis chain. 
> I've belatedly become educated about the infostream, and this situation is a 
> job for it. The DocInverterPerField can note exceptions in the analysis 
> chain, log out to the infostream, and then rethrow them as before. No 
> wrapping, no muss, no fuss.
> There are comments on this JIRA from a more complex prior idea that readers 
> might want to ignore.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Welcome Areek Zillur as Lucene/Solr committer!

2014-01-21 Thread Robert Muir

I'm pleased to announce that Areek Zillur has accepted to join our ranks as
a committer.

Areek has been improving suggester support in Lucene and Solr, including a
revamped Solr component slated for the 4.7 release. [1]

Areek, it is tradition that you introduce yourself with a brief bio.

Once your account is setup, you should then be able to add yourself to the
who we are page on the website as well.

Congratulations and welcome!

[1] https://issues.apache.org/jira/browse/SOLR-5378

Re: Welcome Areek Zillur as Lucene/Solr committer!

2014-01-21 Thread Yonik Seeley

Welcome Areek!

-Yonik
http://heliosearch.com -- making solr shine

On Tue, Jan 21, 2014 at 2:26 PM, Robert Muir  wrote:
> I'm pleased to announce that Areek Zillur has accepted to join our ranks as
> a committer.
>
> Areek has been improving suggester support in Lucene and Solr, including a
> revamped Solr component slated for the 4.7 release. [1]
>
> Areek, it is tradition that you introduce yourself with a brief bio.
>
> Once your account is setup, you should then be able to add yourself to the
> who we are page on the website as well.
>
> Congratulations and welcome!
>
> [1] https://issues.apache.org/jira/browse/SOLR-5378
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Areek Zillur as Lucene/Solr committer!

2014-01-21 Thread Steve Rowe

Congrats and welcome, Areek!

Steve
On Jan 21, 2014 2:26 PM, "Robert Muir"  wrote:

> I'm pleased to announce that Areek Zillur has accepted to join our ranks
> as a committer.
>
> Areek has been improving suggester support in Lucene and Solr, including a
> revamped Solr component slated for the 4.7 release. [1]
>
> Areek, it is tradition that you introduce yourself with a brief bio.
>
> Once your account is setup, you should then be able to add yourself to the
> who we are page on the website as well.
>
> Congratulations and welcome!
>
> [1] https://issues.apache.org/jira/browse/SOLR-5378
>
>

[jira] [Created] (LUCENE-5408) GeometryStrategy -- match geometries in DocValues

2014-01-21 Thread David Smiley (JIRA)

David Smiley created LUCENE-5408:

Summary: GeometryStrategy -- match geometries in DocValues
Key: LUCENE-5408
URL: https://issues.apache.org/jira/browse/LUCENE-5408
Project: Lucene - Core
Issue Type: New Feature
Components: modules/spatial
Reporter: David Smiley
Assignee: David Smiley
Fix For: 4.7

I've started work on a new SpatialStrategy implementation I'm tentatively
calling GeometryStrategy. It's similar to the [JtsGeoStrategy in
Spatial-Solr-Sandbox|https://github.com/ryantxu/spatial-solr-sandbox/tree/master/LSE/src/main/java/org/apache/lucene/spatial/pending/jts]
but a little different in the details -- certainly faster. Using Spatial4j
0.4's BinaryCodec, it'll serialize the shape to bytes (for polygons this in
internally WKB format) and the strategy will put it in a BinaryDocValuesField.
In practice the shape is likely a polygon but it needn't be. Then I'll
implement a Filter that returns a DocIdSetIterator that evaluates a given
document passed via advance(docid)) to see if the query shape matches a shape
in DocValues. It's improper usage for it to be used in a situation where it
will evaluate every document id via nextDoc(). And in practice the DocValues
format chosen should be a disk resident one since each value tends to be kind
of big.

This spatial strategy in and of itself has no _index_; it's O(N) where N is the
number of documents that get passed thru it. So it should be placed last in
the query/filter tree so that the other queries limit the documents it needs to
see. At a minimum, another query/filter to use in conjunction is another
SpatialStrategy like RecursivePrefixTreeStrategy.

Eventually once the PrefixTree grid encoding has a little bit more metadata, it
will be possible to further combine the grid & this strategy in such a way that
many documents won't need to be checked against the serialized geometry.

--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5407) Deadlock? while indexing reader fields in cascaded threads

2014-01-21 Thread Mikhail Khludnev (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877747#comment-13877747
 ] 

Mikhail Khludnev commented on LUCENE-5407:
--

bq. The test does not recreate IndexWriter instances, It is created once in the 
main thread
Yep. My fault. 

Ok. I reproduced this case. I think the problem is that

"main" 
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)

at java.io.PipedReader.read(PipedReader.java:309)
- locked <0x0007ea28ada8> (a java.io.PipedReader)
at 
org.apache.lucene.index.TestIndex2$ParsingReader.read(TestIndex2.java:85)
at .
at 
org.apache.lucene.analysis.util.FilteringTokenFilter.incrementToken(FilteringTokenFilter.java:55)

at 
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1179)
at org.apache.lucene.index.TestIndex2.doIndex(TestIndex2.java:45)

"main" obtains some locks in indexWriter, blocked by the reader and waits for 
closing the stream, which is blocked on obtaining those indexWriter locks.

"Thread-1" ...
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x0007e9ad1b80> (a 
java.util.concurrent.locks.ReentrantLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at 
org.apache.lucene.index.DocumentsWriterFlushControl.obtainAndLock(DocumentsWriterFlushControl.java:445)
...
at 
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1179)
at 
org.apache.lucene.index.TestIndex2$ParsingReader.run(TestIndex2.java:99)

it sounds like you expect "reenterability" (j2ee meme) from indexWriter and 
Lucene Analysis, which is never promised to be so. Sad.
Overall, I don't think that obtaining any locks under Lucene Analysis API is a 
good idea. 

I'm not aware of Tika, but I suppose you need to somehow pre-process the docs 
and feed them into Lucene one-by-one or in parallel. Once again, Lucene indexes 
concurrently really well, but without thread-chaining. 





> Deadlock? while indexing reader fields in cascaded threads
> --
>
> Key: LUCENE-5407
> URL: https://issues.apache.org/jira/browse/LUCENE-5407
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.6
> Environment: Windows 7 64 bits, JRE 1.7.0_25 64 bits
>Reporter: Luis Filipe Nassif
> Attachments: Test.java, thread_dump.txt
>
>
> Apparently I found a deadlock problem with IndexWriter using Reader Fields in 
> a cascaded thread design to add documents (I am working on an application 
> integrating Tika, which has the capability to add embedded documents to the 
> index as independent documents as they are found). The attached code 
> illustrates the problem. Sometimes it stops processing, at least one of the 
> threads remains in WAITING state. It must be executed no more than 5 times in 
> my environment to trigger the problem.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5650) When mixing adds and deletes, it appears there is a corner case where peersync can bring back a deleted update.


[ 
https://issues.apache.org/jira/browse/SOLR-5650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877743#comment-13877743
 ] 

Mark Miller commented on SOLR-5650:
---

One good improvement:

For the case were the leader is a replacement, only peer sync with those nodes 
that have last published ACTIVE.

> When mixing adds and deletes, it appears there is a corner case where 
> peersync can bring back a deleted update.
> ---
>
> Key: SOLR-5650
> URL: https://issues.apache.org/jira/browse/SOLR-5650
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 5.0, 4.7
>
> Attachments: solr.log.tar.gz
>
>




--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [VOTE] Release Lucene/Solr 4.6.1 RC2

2014-01-21 Thread Andi Vajda



+1

PyLucene built from branch_4x r1559610 built and passed its tests.

Andi..

On Sun, 19 Jan 2014, Mark Miller wrote:


Please vote to release the following artifacts:

http://people.apache.org/~markrmiller/lucene_solr_4_6_1r1559610/

Here is my +1.

SUCCESS! [0:51:51.964657]

--
- Mark



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-5652) Heisenbug in DistribCursorPagingTest: "walk already seen ..."

Hoss Man created SOLR-5652:
--

 Summary: Heisenbug in DistribCursorPagingTest: "walk already seen 
..."
 Key: SOLR-5652
 URL: https://issues.apache.org/jira/browse/SOLR-5652
 Project: Solr
  Issue Type: Bug
Reporter: Hoss Man
Assignee: Hoss Man


Twice now, Uwe's jenkins has encountered a "walk already seen ..." assertion 
failure from DistribCursorPagingTest that I've been unable to fathom, let alone 
reproduce.

Using this as a tracking issue to try and make sense of it.

Summary of things noticed so far (in 2 failures):
* So far only seen on http://jenkins.thetaphi.de
* So far only seen on MacOSX
* So far only seen on branch 4x
* So far seen on both Java6 and Java7
* fails occured in first block of randomized testing: 
** we've indexed a small number of randomized docs
** we're explicitly looping over every field and sorting in both directions
* fails were both when sorting on one of the "\*_dv_last desc" fields 
(docValues=true, sortMissingLast=true) 
** sort on same field asc has always worked fine just before this (fields are 
in arbitrary order, but "asc" always tried before "desc")
** sorting on some other random fields has sometimes been tried before this and 
worked


(specifics of each failure seen in the wild recorded in comments)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5652) Heisenbug in DistribCursorPagingTest: "walk already seen ..."


[ 
https://issues.apache.org/jira/browse/SOLR-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877675#comment-13877675
 ] 

Hoss Man commented on SOLR-5652:


http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-MacOSX/1200/
Revision: 1558588
Using Java: 64bit/jdk1.6.0 -XX:-UseCompressedOops -XX:+UseParallelGC

{noformat}
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=DistribCursorPagingTest -Dtests.method=testDistribSearch 
-Dtests.seed=B7F9EFE0FC392581 -Dtests.slow=true -Dtests.locale=es_MX 
-Dtests.timezone=America/Tortola -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 29.3s | DistribCursorPagingTest.testDistribSearch <<<
   [junit4]> Throwable #1: java.lang.AssertionError: walk already seen: 39
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([B7F9EFE0FC392581:361F61F88B6645BD]:0)
   [junit4]>at 
org.apache.solr.cloud.DistribCursorPagingTest.assertFullWalkNoDups(DistribCursorPagingTest.java:628)
   [junit4]>at 
org.apache.solr.cloud.DistribCursorPagingTest.doRandomSortsOnLargeIndex(DistribCursorPagingTest.java:465)
   [junit4]>at 
org.apache.solr.cloud.DistribCursorPagingTest.doTest(DistribCursorPagingTest.java:86)
   [junit4]>at 
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
   [junit4]>at java.lang.Thread.run(Thread.java:695)
{noformat}

After this I made teh following commits to tryand help track down the bug, in 
the hopes it would uncover some shard out of sync problems (which is the only 
explanation i could think of)...

https://svn.apache.org/r1558939
https://svn.apache.org/r1558945


> Heisenbug in DistribCursorPagingTest: "walk already seen ..."
> -
>
> Key: SOLR-5652
> URL: https://issues.apache.org/jira/browse/SOLR-5652
> Project: Solr
>  Issue Type: Bug
>Reporter: Hoss Man
>Assignee: Hoss Man
>
> Twice now, Uwe's jenkins has encountered a "walk already seen ..." assertion 
> failure from DistribCursorPagingTest that I've been unable to fathom, let 
> alone reproduce.
> Using this as a tracking issue to try and make sense of it.
> Summary of things noticed so far (in 2 failures):
> * So far only seen on http://jenkins.thetaphi.de
> * So far only seen on MacOSX
> * So far only seen on branch 4x
> * So far seen on both Java6 and Java7
> * fails occured in first block of randomized testing: 
> ** we've indexed a small number of randomized docs
> ** we're explicitly looping over every field and sorting in both directions
> * fails were both when sorting on one of the "\*_dv_last desc" fields 
> (docValues=true, sortMissingLast=true) 
> ** sort on same field asc has always worked fine just before this (fields are 
> in arbitrary order, but "asc" always tried before "desc")
> ** sorting on some other random fields has sometimes been tried before this 
> and worked
> (specifics of each failure seen in the wild recorded in comments)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5652) Heisenbug in DistribCursorPagingTest: "walk already seen ..."


[ 
https://issues.apache.org/jira/browse/SOLR-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877683#comment-13877683
 ] 

Hoss Man commented on SOLR-5652:


http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-MacOSX/1217/
Revision: 1559847
Using Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseSerialGC


{noformat}
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=DistribCursorPagingTest -Dtests.method=testDistribSearch 
-Dtests.seed=F09D8E3EF23506C2 -Dtests.slow=true -Dtests.locale=sr_RS 
-Dtests.timezone=Australia/Tasmania -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 21.5s | DistribCursorPagingTest.testDistribSearch <<<
   [junit4]> Throwable #1: java.lang.AssertionError: walk already seen: 93, 
don't know why; q=id:93 gives: 
{responseHeader={status=0,QTime=7},response={numFound=1,start=0,maxScore=4.555348,docs=[SolrDocument{id=93,
 long=5077, long_last=5077, long_first=5077, long_dv_last=5077, 
long_dv_first=5077, float=-3.6574272E8, float_last=-3.6574272E8, 
float_first=-3.6574272E8, float_dv_last=-3.6574272E8, 
float_dv_first=-3.6574272E8, double=-1.3713607226255326E9, 
double_last=-1.3713607226255326E9, double_first=-1.3713607226255326E9, 
double_dv_last=-1.3713607226255326E9, double_dv_first=-1.3713607226255326E9, 
str=˜˺ʵ, str_last=˜˺ʵ, str_first=˜˺ʵ, str_dv_last=˜˺ʵ, str_dv_first=˜˺ʵ, 
bin=LfuliMaoJJG5866cs8lYmtS89ZDH2owXyi2QPp9kw6zpPlrrT4UAZw==, 
bin_last=LfuliMaoJJG5866cs8lYmtS89ZDH2owXyi2QPp9kw6zpPlrrT4UAZw==, 
bin_first=LfuliMaoJJG5866cs8lYmtS89ZDH2owXyi2QPp9kw6zpPlrrT4UAZw==, 
bin_dv_last=LfuliMaoJJG5866cs8lYmtS89ZDH2owXyi2QPp9kw6zpPlrrT4UAZw==, 
bin_dv_first=LfuliMaoJJG5866cs8lYmtS89ZDH2owXyi2QPp9kw6zpPlrrT4UAZw==, 
_version_=1457794677110472704}]}}
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([F09D8E3EF23506C2:717B0026856A66FE]:0)
   [junit4]>at 
org.apache.solr.cloud.DistribCursorPagingTest.assertFullWalkNoDups(DistribCursorPagingTest.java:636)
   [junit4]>at 
org.apache.solr.cloud.DistribCursorPagingTest.doRandomSortsOnLargeIndex(DistribCursorPagingTest.java:465)
   [junit4]>at 
org.apache.solr.cloud.DistribCursorPagingTest.doTest(DistribCursorPagingTest.java:86)
   [junit4]>at 
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
   [junit4]>at java.lang.Thread.run(Thread.java:744)
{noformat}

One thing that jumps out at me here is that in this failure the doc in question 
(93) doesn't have a value for the int_dv_last field being sorted on - no 
concrete info on whether that was true in the first failure as well.

The next steps I can think of:
* get the explicit sort criteria into the assertion failure so it's easier to 
verify w/o digging through the logs
* add more logging so every time a doc is seen in one of these cursor walks, we 
log it's sort value -- i'm still not convinced this problem isn't a general 
shard inconsistency problem



> Heisenbug in DistribCursorPagingTest: "walk already seen ..."
> -
>
> Key: SOLR-5652
> URL: https://issues.apache.org/jira/browse/SOLR-5652
> Project: Solr
>  Issue Type: Bug
>Reporter: Hoss Man
>Assignee: Hoss Man
>
> Twice now, Uwe's jenkins has encountered a "walk already seen ..." assertion 
> failure from DistribCursorPagingTest that I've been unable to fathom, let 
> alone reproduce.
> Using this as a tracking issue to try and make sense of it.
> Summary of things noticed so far (in 2 failures):
> * So far only seen on http://jenkins.thetaphi.de
> * So far only seen on MacOSX
> * So far only seen on branch 4x
> * So far seen on both Java6 and Java7
> * fails occured in first block of randomized testing: 
> ** we've indexed a small number of randomized docs
> ** we're explicitly looping over every field and sorting in both directions
> * fails were both when sorting on one of the "\*_dv_last desc" fields 
> (docValues=true, sortMissingLast=true) 
> ** sort on same field asc has always worked fine just before this (fields are 
> in arbitrary order, but "asc" always tried before "desc")
> ** sorting on some other random fields has sometimes been tried before this 
> and worked
> (specifics of each failure seen in the wild recorded in comments)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5407) Deadlock? while indexing reader fields in cascaded threads


 [ 
https://issues.apache.org/jira/browse/LUCENE-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luis Filipe Nassif updated LUCENE-5407:
---

Attachment: thread_dump.txt

> Deadlock? while indexing reader fields in cascaded threads
> --
>
> Key: LUCENE-5407
> URL: https://issues.apache.org/jira/browse/LUCENE-5407
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.6
> Environment: Windows 7 64 bits, JRE 1.7.0_25 64 bits
>Reporter: Luis Filipe Nassif
> Attachments: Test.java, thread_dump.txt
>
>
> Apparently I found a deadlock problem with IndexWriter using Reader Fields in 
> a cascaded thread design to add documents (I am working on an application 
> integrating Tika, which has the capability to add embedded documents to the 
> index as independent documents as they are found). The attached code 
> illustrates the problem. Sometimes it stops processing, at least one of the 
> threads remains in WAITING state. It must be executed no more than 5 times in 
> my environment to trigger the problem.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5505) LoggingInfoStream not usabe in a multi-core setup

2014-01-21 Thread Shikhar Bhushan (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shikhar Bhushan updated SOLR-5505:
--

Fix Version/s: 4.7

> LoggingInfoStream not usabe in a multi-core setup
> -
>
> Key: SOLR-5505
> URL: https://issues.apache.org/jira/browse/SOLR-5505
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 4.6
>Reporter: Shikhar Bhushan
> Fix For: 4.7
>
> Attachments: SOLR-5505.patch, SOLR-5505.patch
>
>
> {{LoggingInfoStream}} that was introduced in SOLR-4977 does not log any core 
> context.
> Previously this was possible by encoding this into the infoStream's file path.
> This means in a multi-core setup it is very hard to distinguish between the 
> infoStream messages for different cores.
> {{LoggingInfoStream}} should be automatically configured to prepend the core 
> name to log messages.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5637) Per-request cache statistics

2014-01-21 Thread Shikhar Bhushan (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shikhar Bhushan updated SOLR-5637:
--

Fix Version/s: 4.7

> Per-request cache statistics
> 
>
> Key: SOLR-5637
> URL: https://issues.apache.org/jira/browse/SOLR-5637
> Project: Solr
>  Issue Type: New Feature
>Reporter: Shikhar Bhushan
>Priority: Minor
> Fix For: 4.7
>
> Attachments: SOLR-5367.patch, SOLR-5367.patch
>
>
> We have found it very useful to have information on the number of cache hits 
> and misses for key Solr caches (filterCache, documentCache, etc.) at the 
> request level.
> This is currently implemented in our codebase using custom {{SolrCache}} 
> implementations.
> I am working on moving to maintaining stats in the {{SolrRequestInfo}} 
> thread-local, and adding hooks in get() methods of SolrCache implementations. 
> This will be glued up using the {{DebugComponent}} and can be requested using 
> a "debug.cache" parameter.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5648) SolrCore#getStatistics() should nest open searchers' stats

2014-01-21 Thread Shikhar Bhushan (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shikhar Bhushan updated SOLR-5648:
--

Fix Version/s: 4.7

> SolrCore#getStatistics() should nest open searchers' stats
> --
>
> Key: SOLR-5648
> URL: https://issues.apache.org/jira/browse/SOLR-5648
> Project: Solr
>  Issue Type: Task
>Reporter: Shikhar Bhushan
>Priority: Minor
> Fix For: 4.7
>
> Attachments: SOLR-5648.patch, oldestSearcherStaleness.gif, 
> openSearchers.gif
>
>
> {{SolrIndexSearcher}} leaks are a notable cause of garbage collection issues 
> in codebases with custom components.
> So it is useful to be able to access monitoring information about what 
> searchers are currently open, and in turn access their stats e.g. 
> {{openedAt}}.
> This can be nested via {{SolrCore#getStatistics()}} which has a 
> {{_searchers}} collection of all open searchers.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b123) - Build # 9080 - Failure!

2014-01-21 Thread Policeman Jenkins Server

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9080/
Java: 32bit/jdk1.8.0-ea-b123 -server -XX:+UseG1GC

1 tests failed.
REGRESSION:  org.apache.lucene.index.TestFlushByRamOrCountsPolicy.testFlushByRam

Error Message:
Captured an uncaught exception in thread: Thread[id=230, name=Thread-185, 
state=RUNNABLE, group=TGRP-TestFlushByRamOrCountsPolicy]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=230, name=Thread-185, state=RUNNABLE, 
group=TGRP-TestFlushByRamOrCountsPolicy]
Caused by: java.lang.RuntimeException: java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([E10D7BABE9843C23]:0)
at 
org.apache.lucene.index.TestFlushByRamOrCountsPolicy$IndexThread.run(TestFlushByRamOrCountsPolicy.java:329)
Caused by: java.lang.AssertionError
at 
org.apache.lucene.index.ByteSliceReader.readByte(ByteSliceReader.java:73)
at org.apache.lucene.store.DataInput.readVInt(DataInput.java:108)
at 
org.apache.lucene.index.FreqProxTermsWriterPerField.flush(FreqProxTermsWriterPerField.java:453)
at 
org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:85)
at org.apache.lucene.index.TermsHash.flush(TermsHash.java:116)
at org.apache.lucene.index.DocInverter.flush(DocInverter.java:53)
at 
org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:81)
at 
org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:465)
at 
org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:506)
at 
org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:616)
at 
org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2828)
at 
org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2986)
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2953)
at 
org.apache.lucene.index.TestFlushByRamOrCountsPolicy$IndexThread.run(TestFlushByRamOrCountsPolicy.java:325)




Build Log:
[...truncated 481 lines...]
   [junit4] Suite: org.apache.lucene.index.TestFlushByRamOrCountsPolicy
   [junit4]   2> jan 21, 2014 5:57:56 PM 
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
 uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: 
Thread[Thread-185,5,TGRP-TestFlushByRamOrCountsPolicy]
   [junit4]   2> java.lang.RuntimeException: java.lang.AssertionError
   [junit4]   2>at 
__randomizedtesting.SeedInfo.seed([E10D7BABE9843C23]:0)
   [junit4]   2>at 
org.apache.lucene.index.TestFlushByRamOrCountsPolicy$IndexThread.run(TestFlushByRamOrCountsPolicy.java:329)
   [junit4]   2> Caused by: java.lang.AssertionError
   [junit4]   2>at 
org.apache.lucene.index.ByteSliceReader.readByte(ByteSliceReader.java:73)
   [junit4]   2>at 
org.apache.lucene.store.DataInput.readVInt(DataInput.java:108)
   [junit4]   2>at 
org.apache.lucene.index.FreqProxTermsWriterPerField.flush(FreqProxTermsWriterPerField.java:453)
   [junit4]   2>at 
org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:85)
   [junit4]   2>at 
org.apache.lucene.index.TermsHash.flush(TermsHash.java:116)
   [junit4]   2>at 
org.apache.lucene.index.DocInverter.flush(DocInverter.java:53)
   [junit4]   2>at 
org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:81)
   [junit4]   2>at 
org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:465)
   [junit4]   2>at 
org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:506)
   [junit4]   2>at 
org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:616)
   [junit4]   2>at 
org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2828)
   [junit4]   2>at 
org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2986)
   [junit4]   2>at 
org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2953)
   [junit4]   2>at 
org.apache.lucene.index.TestFlushByRamOrCountsPolicy$IndexThread.run(TestFlushByRamOrCountsPolicy.java:325)
   [junit4]   2> 
   [junit4]   1> FAILED exc:
   [junit4]   1> java.lang.AssertionError
   [junit4]   1>at 
org.apache.lucene.index.ByteSliceReader.readByte(ByteSliceReader.java:73)
   [junit4]   1>at 
org.apache.lucene.store.DataInput.readVInt(DataInput.java:108)
   [junit4]   1>at 
org.apache.lucene.index.FreqProxTermsWriterPerField.flush(FreqProxTermsWriterPerField.java:453)
   [junit4]   1>at 
org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:85)
   [junit4]   1>at 
org.apache.lucene.index.TermsHash.flush(TermsHash.java:116)
   [junit4]   1>at 
org.apache.lucene.index.DocInverter.flush(DocInverter.java:53)
   [junit4]

[jira] [Updated] (SOLR-5477) Async execution of OverseerCollectionProcessor tasks

2014-01-21 Thread Shalin Shekhar Mangar (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar updated SOLR-5477:


Attachment: SOLR-5477.patch

This is Anshum's patch brought in sync with latest trunk.

> Async execution of OverseerCollectionProcessor tasks
> 
>
> Key: SOLR-5477
> URL: https://issues.apache.org/jira/browse/SOLR-5477
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
>Assignee: Anshum Gupta
> Attachments: SOLR-5477-CoreAdminStatus.patch, SOLR-5477.patch, 
> SOLR-5477.patch
>
>
> Typical collection admin commands are long running and it is very common to 
> have the requests get timed out.  It is more of a problem if the cluster is 
> very large.Add an option to run these commands asynchronously
> add an extra param async=true for all collection commands
> the task is written to ZK and the caller is returned a task id. 
> as separate collection admin command will be added to poll the status of the 
> task
> command=status&id=7657668909
> if id is not passed all running async tasks should be listed
> A separate queue is created to store in-process tasks . After the tasks are 
> completed the queue entry is removed. OverSeerColectionProcessor will perform 
> these tasks in multiple threads



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5648) SolrCore#getStatistics() should nest open searchers' stats

2014-01-21 Thread Gregg Donovan (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877553#comment-13877553
 ] 

Gregg Donovan commented on SOLR-5648:
-

We use this patch to to graph the number open searchers and the oldest searcher 
in Ganglia. It's very useful to have an overview of searcher health and for 
detecting possible leaks. 

!openSearchers.gif|Count of open searchers!

!oldestSearcherStaleness.gif|Age of oldest searcher!

> SolrCore#getStatistics() should nest open searchers' stats
> --
>
> Key: SOLR-5648
> URL: https://issues.apache.org/jira/browse/SOLR-5648
> Project: Solr
>  Issue Type: Task
>Reporter: Shikhar Bhushan
>Priority: Minor
> Attachments: SOLR-5648.patch, oldestSearcherStaleness.gif, 
> openSearchers.gif
>
>
> {{SolrIndexSearcher}} leaks are a notable cause of garbage collection issues 
> in codebases with custom components.
> So it is useful to be able to access monitoring information about what 
> searchers are currently open, and in turn access their stats e.g. 
> {{openedAt}}.
> This can be nested via {{SolrCore#getStatistics()}} which has a 
> {{_searchers}} collection of all open searchers.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5648) SolrCore#getStatistics() should nest open searchers' stats

2014-01-21 Thread Gregg Donovan (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gregg Donovan updated SOLR-5648:


Attachment: openSearchers.gif
oldestSearcherStaleness.gif

Ganglia graphs from new searcher statistics.

> SolrCore#getStatistics() should nest open searchers' stats
> --
>
> Key: SOLR-5648
> URL: https://issues.apache.org/jira/browse/SOLR-5648
> Project: Solr
>  Issue Type: Task
>Reporter: Shikhar Bhushan
>Priority: Minor
> Attachments: SOLR-5648.patch, oldestSearcherStaleness.gif, 
> openSearchers.gif
>
>
> {{SolrIndexSearcher}} leaks are a notable cause of garbage collection issues 
> in codebases with custom components.
> So it is useful to be able to access monitoring information about what 
> searchers are currently open, and in turn access their stats e.g. 
> {{openedAt}}.
> This can be nested via {{SolrCore#getStatistics()}} which has a 
> {{_searchers}} collection of all open searchers.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5407) Deadlock? while indexing reader fields in cascaded threads


[ 
https://issues.apache.org/jira/browse/LUCENE-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877543#comment-13877543
 ] 

Luis Filipe Nassif commented on LUCENE-5407:


The test does not recreate IndexWriter instances, It is created once in the 
main thread. And addDocument(doc) returns in the main thread only after 
pipedWriter is closed in the secondary thread (so pipedReader returns -1), then 
indexWriter is closed after adding all docs. I will post a thread dump.

> Deadlock? while indexing reader fields in cascaded threads
> --
>
> Key: LUCENE-5407
> URL: https://issues.apache.org/jira/browse/LUCENE-5407
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.6
> Environment: Windows 7 64 bits, JRE 1.7.0_25 64 bits
>Reporter: Luis Filipe Nassif
> Attachments: Test.java
>
>
> Apparently I found a deadlock problem with IndexWriter using Reader Fields in 
> a cascaded thread design to add documents (I am working on an application 
> integrating Tika, which has the capability to add embedded documents to the 
> index as independent documents as they are found). The attached code 
> illustrates the problem. Sometimes it stops processing, at least one of the 
> threads remains in WAITING state. It must be executed no more than 5 times in 
> my environment to trigger the problem.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-5594) Enable using extended field types with prefix queries for non-default encoded strings


[ 
https://issues.apache.org/jira/browse/SOLR-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877521#comment-13877521
 ] 

Anshum Gupta edited comment on SOLR-5594 at 1/21/14 3:15 PM:
-

Hoss: Thanks for taking that up before I got back.

bq. Yes, this would also be identical to range query behaviour: newRangeQuery 
also delegates to the field type, and the protected method is there for 
subclasses.
My changes are Solr specific. I'm not sure how would it play with the Solr 
QParsers (Solr extended fields) if I moved the changes to Lucene instead of 
Solr.



was (Author: anshumg):
Thanks for taking that up before I got back.

bq. Yes, this would also be identical to range query behaviour: newRangeQuery 
also delegates to the field type, and the protected method is there for 
subclasses.
My changes are Solr specific. I'm not sure how would it play with the Solr 
QParsers (Solr extended fields) if I moved the changes to Lucene instead of 
Solr.


> Enable using extended field types with prefix queries for non-default encoded 
> strings
> -
>
> Key: SOLR-5594
> URL: https://issues.apache.org/jira/browse/SOLR-5594
> Project: Solr
>  Issue Type: Improvement
>  Components: query parsers, Schema and Analysis
>Affects Versions: 4.6
>Reporter: Anshum Gupta
>Assignee: Anshum Gupta
>Priority: Minor
> Attachments: SOLR-5594-branch_4x.patch, SOLR-5594.patch, 
> SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch
>
>
> Enable users to be able to use prefix query with custom field types with 
> non-default encoding/decoding for queries more easily. e.g. having a custom 
> field work with base64 encoded query strings.
> Currently, the workaround for it is to have the override at getRewriteMethod 
> level. Perhaps having the prefixQuery also use the calling FieldType's 
> readableToIndexed method would work better.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5594) Enable using extended field types with prefix queries for non-default encoded strings


[ 
https://issues.apache.org/jira/browse/SOLR-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877522#comment-13877522
 ] 

Anshum Gupta commented on SOLR-5594:


[~hossman_luc...@fucit.org] : I'll make the recommended changes and put up 
another patch.

> Enable using extended field types with prefix queries for non-default encoded 
> strings
> -
>
> Key: SOLR-5594
> URL: https://issues.apache.org/jira/browse/SOLR-5594
> Project: Solr
>  Issue Type: Improvement
>  Components: query parsers, Schema and Analysis
>Affects Versions: 4.6
>Reporter: Anshum Gupta
>Assignee: Anshum Gupta
>Priority: Minor
> Attachments: SOLR-5594-branch_4x.patch, SOLR-5594.patch, 
> SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch
>
>
> Enable users to be able to use prefix query with custom field types with 
> non-default encoding/decoding for queries more easily. e.g. having a custom 
> field work with base64 encoded query strings.
> Currently, the workaround for it is to have the override at getRewriteMethod 
> level. Perhaps having the prefixQuery also use the calling FieldType's 
> readableToIndexed method would work better.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5594) Enable using extended field types with prefix queries for non-default encoded strings


[ 
https://issues.apache.org/jira/browse/SOLR-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877521#comment-13877521
 ] 

Anshum Gupta commented on SOLR-5594:


Thanks for taking that up before I got back.

bq. Yes, this would also be identical to range query behaviour: newRangeQuery 
also delegates to the field type, and the protected method is there for 
subclasses.
My changes are Solr specific. I'm not sure how would it play with the Solr 
QParsers (Solr extended fields) if I moved the changes to Lucene instead of 
Solr.


> Enable using extended field types with prefix queries for non-default encoded 
> strings
> -
>
> Key: SOLR-5594
> URL: https://issues.apache.org/jira/browse/SOLR-5594
> Project: Solr
>  Issue Type: Improvement
>  Components: query parsers, Schema and Analysis
>Affects Versions: 4.6
>Reporter: Anshum Gupta
>Assignee: Anshum Gupta
>Priority: Minor
> Attachments: SOLR-5594-branch_4x.patch, SOLR-5594.patch, 
> SOLR-5594.patch, SOLR-5594.patch, SOLR-5594.patch
>
>
> Enable users to be able to use prefix query with custom field types with 
> non-default encoding/decoding for queries more easily. e.g. having a custom 
> field work with base64 encoded query strings.
> Currently, the workaround for it is to have the override at getRewriteMethod 
> level. Perhaps having the prefixQuery also use the calling FieldType's 
> readableToIndexed method would work better.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-5377) Lucene mixed version segments cause segment info file(.si) wrong


[ 
https://issues.apache.org/jira/browse/LUCENE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877492#comment-13877492
 ] 

Littlestar edited comment on LUCENE-5377 at 1/21/14 2:54 PM:
-

reproduce steps:
1. run TestCore_45 in luence 4.5.1, it will generate a index 
TestCore_45/data/taxo.
2. copy TestCore_45/data/taxo into TestCore_46/data/taxo.
3. run TestCore_46 in luence 4.6.0.
you will see the following errors.
Exception in thread "main" java.io.FileNotFoundException: xception in thread 
"main" java.io.FileNotFoundException: 
G:\JavaWorkspace\bug_test\Lucene_46\data\taxo\_2.si (.)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.(RandomAccessFile.java:233)
at 
org.apache.lucene.store.FSDirectory$FSIndexInput.(FSDirectory.java:388)
at 
org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.(SimpleFSDirectory.java:103)
at 
org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:58)
at 
org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Lucene40SegmentInfoReader.java:51)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:340)
at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:404)
at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:843)
at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:694)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:400)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.readCommitData(DirectoryTaxonomyWriter.java:138)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:222)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:331)
at TestCore_46$1.(TestCore_46.java:27)
at TestCore_46.open(TestCore_46.java:27)
at TestCore_46.main(TestCore_46.java:75)


==


was (Author: cnstar9988):
reproduce steps:
1. run TestCore_45 in luence 4.5.1, it will generate a index 
TestCore_45/data/taxo.
2. copy TestCore_45/data/taxo in TestCore_46/data/taxo.
3. run TestCore_46 in luence 4.6.0.
you will see the following errors.
Exception in thread "main" java.io.FileNotFoundException: xception in thread 
"main" java.io.FileNotFoundException: 
G:\JavaWorkspace\bug_test\Lucene_46\data\taxo\_2.si (.)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.(RandomAccessFile.java:233)
at 
org.apache.lucene.store.FSDirectory$FSIndexInput.(FSDirectory.java:388)
at 
org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.(SimpleFSDirectory.java:103)
at 
org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:58)
at 
org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Lucene40SegmentInfoReader.java:51)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:340)
at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:404)
at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:843)
at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:694)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:400)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.readCommitData(DirectoryTaxonomyWriter.java:138)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:222)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:331)
at TestCore_46$1.(TestCore_46.java:27)
at TestCore_46.open(TestCore_46.java:27)
at TestCore_46.main(TestCore_46.java:75)


==

> Lucene mixed version segments cause segment info file(.si) wrong
> 
>
> Key: LUCENE-5377
> URL: https://issues.apache.org/jira/browse/LUCENE-5377
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.6
> Environment: windows/linux
>Reporter: Littlestar
> Attachments: TestCore_45.java, TestCore_46.java
>
>
> my old facet index create by Lucene version=4.2
> use indexChecker ok.
> now I upgrade to Lucene 4.6 and put some new records to index.
> then reopen index, some files in indexdir missing
> no .si files.
> I debug into it,  new version format of segments.gen(segments_N) record bad 
> segments info.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-

[jira] [Comment Edited] (LUCENE-5377) Lucene mixed version segments cause segment info file(.si) wrong


[ 
https://issues.apache.org/jira/browse/LUCENE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877492#comment-13877492
 ] 

Littlestar edited comment on LUCENE-5377 at 1/21/14 2:20 PM:
-

reproduce steps:
1. run TestCore_45 in luence 4.5.1, it will generate a index 
TestCore_45/data/taxo.
2. copy TestCore_45/data/taxo in TestCore_46/data/taxo.
3. run TestCore_46 in luence 4.6.0.
you will see the following errors.
Exception in thread "main" java.io.FileNotFoundException: xception in thread 
"main" java.io.FileNotFoundException: 
G:\JavaWorkspace\bug_test\Lucene_46\data\taxo\_2.si (.)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.(RandomAccessFile.java:233)
at 
org.apache.lucene.store.FSDirectory$FSIndexInput.(FSDirectory.java:388)
at 
org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.(SimpleFSDirectory.java:103)
at 
org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:58)
at 
org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Lucene40SegmentInfoReader.java:51)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:340)
at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:404)
at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:843)
at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:694)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:400)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.readCommitData(DirectoryTaxonomyWriter.java:138)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:222)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:331)
at TestCore_46$1.(TestCore_46.java:27)
at TestCore_46.open(TestCore_46.java:27)
at TestCore_46.main(TestCore_46.java:75)


==


was (Author: cnstar9988):
reproduce steps:
1. run TestCore_45 in luence 4.5.1, it generate a index TestCore_45/data/taxo.
2. copy data/taxo in TestCore_46/data/taxo.
3. run TestCore_46 in luence 4.6.0.
you will see the following errors.
Exception in thread "main" java.io.FileNotFoundException: xception in thread 
"main" java.io.FileNotFoundException: 
G:\JavaWorkspace\bug_test\Lucene_46\data\taxo\_2.si (.)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.(RandomAccessFile.java:233)
at 
org.apache.lucene.store.FSDirectory$FSIndexInput.(FSDirectory.java:388)
at 
org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.(SimpleFSDirectory.java:103)
at 
org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:58)
at 
org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Lucene40SegmentInfoReader.java:51)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:340)
at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:404)
at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:843)
at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:694)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:400)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.readCommitData(DirectoryTaxonomyWriter.java:138)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:222)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:331)
at TestCore_46$1.(TestCore_46.java:27)
at TestCore_46.open(TestCore_46.java:27)
at TestCore_46.main(TestCore_46.java:75)


==

> Lucene mixed version segments cause segment info file(.si) wrong
> 
>
> Key: LUCENE-5377
> URL: https://issues.apache.org/jira/browse/LUCENE-5377
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.6
> Environment: windows/linux
>Reporter: Littlestar
> Attachments: TestCore_45.java, TestCore_46.java
>
>
> my old facet index create by Lucene version=4.2
> use indexChecker ok.
> now I upgrade to Lucene 4.6 and put some new records to index.
> then reopen index, some files in indexdir missing
> no .si files.
> I debug into it,  new version format of segments.gen(segments_N) record bad 
> segments info.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsu

[jira] [Updated] (LUCENE-5377) Lucene mixed version segments cause segment info file(.si) wrong


 [ 
https://issues.apache.org/jira/browse/LUCENE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Littlestar updated LUCENE-5377:
---

Attachment: TestCore_46.java
TestCore_45.java

reproduce steps:
1. run TestCore_45 in luence 4.5, it generate a index TestCore_45/data/taxo.
2. copy data/taxo in TestCore_46/data/taxo.
3. run TestCore_46.
you will see the following errors.
Exception in thread "main" java.io.FileNotFoundException: xception in thread 
"main" java.io.FileNotFoundException: 
G:\JavaWorkspace\bug_test\Lucene_46\data\taxo\_2.si (.)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.(RandomAccessFile.java:233)
at 
org.apache.lucene.store.FSDirectory$FSIndexInput.(FSDirectory.java:388)
at 
org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.(SimpleFSDirectory.java:103)
at 
org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:58)
at 
org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Lucene40SegmentInfoReader.java:51)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:340)
at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:404)
at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:843)
at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:694)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:400)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.readCommitData(DirectoryTaxonomyWriter.java:138)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:222)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:331)
at TestCore_46$1.(TestCore_46.java:27)
at TestCore_46.open(TestCore_46.java:27)
at TestCore_46.main(TestCore_46.java:75)


==

> Lucene mixed version segments cause segment info file(.si) wrong
> 
>
> Key: LUCENE-5377
> URL: https://issues.apache.org/jira/browse/LUCENE-5377
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.6
> Environment: windows/linux
>Reporter: Littlestar
> Attachments: TestCore_45.java, TestCore_46.java
>
>
> my old facet index create by Lucene version=4.2
> use indexChecker ok.
> now I upgrade to Lucene 4.6 and put some new records to index.
> then reopen index, some files in indexdir missing
> no .si files.
> I debug into it,  new version format of segments.gen(segments_N) record bad 
> segments info.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-5377) Lucene mixed version segments cause segment info file(.si) wrong


[ 
https://issues.apache.org/jira/browse/LUCENE-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877492#comment-13877492
 ] 

Littlestar edited comment on LUCENE-5377 at 1/21/14 2:18 PM:
-

reproduce steps:
1. run TestCore_45 in luence 4.5.1, it generate a index TestCore_45/data/taxo.
2. copy data/taxo in TestCore_46/data/taxo.
3. run TestCore_46 in luence 4.6.0.
you will see the following errors.
Exception in thread "main" java.io.FileNotFoundException: xception in thread 
"main" java.io.FileNotFoundException: 
G:\JavaWorkspace\bug_test\Lucene_46\data\taxo\_2.si (.)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.(RandomAccessFile.java:233)
at 
org.apache.lucene.store.FSDirectory$FSIndexInput.(FSDirectory.java:388)
at 
org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.(SimpleFSDirectory.java:103)
at 
org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:58)
at 
org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Lucene40SegmentInfoReader.java:51)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:340)
at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:404)
at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:843)
at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:694)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:400)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.readCommitData(DirectoryTaxonomyWriter.java:138)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:222)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:331)
at TestCore_46$1.(TestCore_46.java:27)
at TestCore_46.open(TestCore_46.java:27)
at TestCore_46.main(TestCore_46.java:75)


==


was (Author: cnstar9988):
reproduce steps:
1. run TestCore_45 in luence 4.5, it generate a index TestCore_45/data/taxo.
2. copy data/taxo in TestCore_46/data/taxo.
3. run TestCore_46.
you will see the following errors.
Exception in thread "main" java.io.FileNotFoundException: xception in thread 
"main" java.io.FileNotFoundException: 
G:\JavaWorkspace\bug_test\Lucene_46\data\taxo\_2.si (.)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.(RandomAccessFile.java:233)
at 
org.apache.lucene.store.FSDirectory$FSIndexInput.(FSDirectory.java:388)
at 
org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.(SimpleFSDirectory.java:103)
at 
org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:58)
at 
org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Lucene40SegmentInfoReader.java:51)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:340)
at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:404)
at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:843)
at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:694)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:400)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.readCommitData(DirectoryTaxonomyWriter.java:138)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:222)
at 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.(DirectoryTaxonomyWriter.java:331)
at TestCore_46$1.(TestCore_46.java:27)
at TestCore_46.open(TestCore_46.java:27)
at TestCore_46.main(TestCore_46.java:75)


==

> Lucene mixed version segments cause segment info file(.si) wrong
> 
>
> Key: LUCENE-5377
> URL: https://issues.apache.org/jira/browse/LUCENE-5377
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.6
> Environment: windows/linux
>Reporter: Littlestar
> Attachments: TestCore_45.java, TestCore_46.java
>
>
> my old facet index create by Lucene version=4.2
> use indexChecker ok.
> now I upgrade to Lucene 4.6 and put some new records to index.
> then reopen index, some files in indexdir missing
> no .si files.
> I debug into it,  new version format of segments.gen(segments_N) record bad 
> segments info.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@luc

[jira] [Commented] (LUCENE-5407) Deadlock? while indexing reader fields in cascaded threads

2014-01-21 Thread Mikhail Khludnev (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877482#comment-13877482
 ] 

Mikhail Khludnev commented on LUCENE-5407:
--

I don't think so.

Why do you recreates indexwriter in this test, even if you have a reason, it's 
definitely not the case for which it was designed. 
Also, I see that ParsingReader.run() calls indexWriter.addDocument(doc); when 
indexWriter might be already closed in the main thread.
Anyway, please post thread dump with the deadlock that clarifies a lot.

> Deadlock? while indexing reader fields in cascaded threads
> --
>
> Key: LUCENE-5407
> URL: https://issues.apache.org/jira/browse/LUCENE-5407
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.6
> Environment: Windows 7 64 bits, JRE 1.7.0_25 64 bits
>Reporter: Luis Filipe Nassif
> Attachments: Test.java
>
>
> Apparently I found a deadlock problem with IndexWriter using Reader Fields in 
> a cascaded thread design to add documents (I am working on an application 
> integrating Tika, which has the capability to add embedded documents to the 
> index as independent documents as they are found). The attached code 
> illustrates the problem. Sometimes it stops processing, at least one of the 
> threads remains in WAITING state. It must be executed no more than 5 times in 
> my environment to trigger the problem.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5651) Make labels clickable and the output wrapped in UI

2014-01-21 Thread Artem Lukanin (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Artem Lukanin updated SOLR-5651:


Attachment: SOLR-5651_clickable_labels.patch

added a space after checkboxes

> Make labels clickable and the output wrapped in UI
> --
>
> Key: SOLR-5651
> URL: https://issues.apache.org/jira/browse/SOLR-5651
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 4.6
>Reporter: Artem Lukanin
> Fix For: 4.6.1
>
> Attachments: SOLR-5651_clickable_labels.patch, 
> SOLR-5651_clickable_labels.patch
>
>
> Now it is practically impossible to tick off check boxes in the UI, because 
> the label text is wrapped in a fancy A tag without HREF.
> Also it is very hard to read the output on the Query menu when 
> debugQuery=true. It is impossible to scroll the window to the right. It would 
> be better if the text in the PRE tag was wrapped.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5526) Query parser extends standard cause NPE on Solr startup

2014-01-21 Thread Shalin Shekhar Mangar (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-5526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877476#comment-13877476
 ] 

Shalin Shekhar Mangar commented on SOLR-5526:
-

Thanks Vitaliy. Can you please combine the two patches i.e. the test and making 
the NAME fields final?

Just a tip for the future: Just have the same names for each iteration of the 
patch. Jira will automatically grey out old patches and keep the latest one 
highlighted. It also helps reviewers in figuring out which is the the latest 
patch.

> Query parser extends standard cause NPE on Solr startup
> ---
>
> Key: SOLR-5526
> URL: https://issues.apache.org/jira/browse/SOLR-5526
> Project: Solr
>  Issue Type: Bug
>  Components: query parsers
>Affects Versions: 4.5.1, 4.6, 5.0
> Environment: Linux, Java 1.7.0_45
>Reporter: Nikolay Khitrin
>Priority: Blocker
> Attachments: NPE_load_trace, SOLR-5526-final-names.patch, 
> SOLR-5526-final-names.patch, SOLR-5526-tests.patch, SOLR-5526.patch
>
>
> Adding any custom query parser extending standard one with non-final field 
> {{NAME}} lead to messy {{NullPointerException}} during Solr startup.
> Definition of standard parsers is located in  
> {{QParserPlugin.standardPlugins}} static array. The array contains names from 
> static {{NAME}} fields and classes for each plugin.   
> But all of listed parsers are derived from {{QParserPlugin}}, so we have 
> circular dependency of static fields.
> Normally, class loader start initializing {{QParserPlugin}} before all listed 
> plugins in {{SolrCore.initQParsers}}, and array elements referenced to 
> {{NAME}} plugins' fields are filled properly.
> Custom parsers are instantiated before standard parsers. And when we subclass 
> plugin with non-final {{NAME}} field and add it to Solr via 
> {{solrconfig.xml}}, class loader start loading our class before 
> {{QParserPlugin}}. Because {{QParserPlugin}} is a superclass for plugin, it 
> must be initialized before subclasses, and static dereferencing cause null 
> elements in {{standardPlugins}} array because it filled before {{NAME}} field 
> of loading plugin's superclass.
> How to reproduce:
> # Checkout Solr (trunk or stable)
> # Add the following line to solr/example/solr/collection1/conf/solrconfig.xml
>   {{}}
> # Call ant run-example in solr folder
> Possible workarounds:
> * dev-workaround: add {{int workaround = 
> QParserPlugin.standardPlugins.length;}} as a first line to
>   {{SolrCore.initQParsers}}
> * user-workaround: add plugin with final {{NAME}} field (edismax) to 
> solrconfig.xml  before subclasses of standard plugins. 
>   {{ class="solr.search.ExtendedDismaxQParserPlugin"/>}}
>   
> Possible fix:
> Move {{standardPlugins}} to new final class to break circular dependency.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5651) Make labels clickable and the output wrapped in UI

2014-01-21 Thread Artem Lukanin (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Artem Lukanin updated SOLR-5651:


Attachment: SOLR-5651_clickable_labels.patch

The patch resolves the issue.

> Make labels clickable and the output wrapped in UI
> --
>
> Key: SOLR-5651
> URL: https://issues.apache.org/jira/browse/SOLR-5651
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 4.6
>Reporter: Artem Lukanin
> Fix For: 4.6.1
>
> Attachments: SOLR-5651_clickable_labels.patch
>
>
> Now it is practically impossible to tick off check boxes in the UI, because 
> the label text is wrapped in a fancy A tag without HREF.
> Also it is very hard to read the output on the Query menu when 
> debugQuery=true. It is impossible to scroll the window to the right. It would 
> be better if the text in the PRE tag was wrapped.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-5651) Make labels clickable and the output wrapped in UI

2014-01-21 Thread Artem Lukanin (JIRA)

Artem Lukanin created SOLR-5651:
---

 Summary: Make labels clickable and the output wrapped in UI
 Key: SOLR-5651
 URL: https://issues.apache.org/jira/browse/SOLR-5651
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.6
Reporter: Artem Lukanin
 Fix For: 4.6.1


Now it is practically impossible to tick off check boxes in the UI, because the 
label text is wrapped in a fancy A tag without HREF.
Also it is very hard to read the output on the Query menu when debugQuery=true. 
It is impossible to scroll the window to the right. It would be better if the 
text in the PRE tag was wrapped.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5407) Deadlock? while indexing reader fields in cascaded threads


 [ 
https://issues.apache.org/jira/browse/LUCENE-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luis Filipe Nassif updated LUCENE-5407:
---

Attachment: Test.java

> Deadlock? while indexing reader fields in cascaded threads
> --
>
> Key: LUCENE-5407
> URL: https://issues.apache.org/jira/browse/LUCENE-5407
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.6
> Environment: Windows 7 64 bits, JRE 1.7.0_25 64 bits
>Reporter: Luis Filipe Nassif
> Attachments: Test.java
>
>
> Apparently I found a deadlock problem with IndexWriter using Reader Fields in 
> a cascaded thread design to add documents (I am working on an application 
> integrating Tika, which has the capability to add embedded documents to the 
> index as independent documents as they are found). The attached code 
> illustrates the problem. Sometimes it stops processing, at least one of the 
> threads remains in WAITING state. It must be executed no more than 5 times in 
> my environment to trigger the problem.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-5407) Deadlock? while indexing reader fields in cascaded threads