Introduction to PyLucene Community and some doubts
Hello Everyone, I am Vishrut Mehta, currently a third year students at IIIT Hyderabad, India. I have been contributing to Open Source since two years and also have contributed to organizations like E-cidadania, Sahana Software Foundation, Gnome, etc. I am very interested in Search engines and search related libraries. I need some help from the community, I am currently working on a project which deals with the follow issue - Need to search within any uploaded documents(like .pdf, .doc, etc) from the userand need to search text or strings within those documents. Can anyone help me for this, it would be a great help ?! Thanks You! Regards, -- *Vishrut Mehta* International Institute of Information Technology, Gachibowli,Hyderabad-500032
Re: Introduction to PyLucene Community and some doubts
Hello sir, Thank you for the quick reply. I want to integrate this functionality with web2py, So i would need to stick with python and Pylucene. So the method you are saying is like, extracting text from all the document using different python libraries, and then Indexing the data, then Search the string, is this correct ? And also, I have other doubt also, look here - http://lucene.apache.org/pylucene/features.html in the middle of the page, it is wriiten to import any function, i need to call like: from lucene import StandardAnalyzer but here, its not like this - http://svn.apache.org/viewvc/lucene/pylucene/trunk/samples/IndexFiles.py?view=markup its written : from org.apache.lucene.analysis.standard import StandardAnalyzer So, can u explain me why such a difference ? Thanks you!! Regards, Vishrut Mehta On Tue, Jun 11, 2013 at 3:10 PM, Vinay Modi vinaym...@gmail.com wrote: Vishrut, Have a look at Apache Nutch. If you want to stick with Lucene, you will need to do lot of stuff yourself. See Python libraries for pdf and doc etc, read the text from those file using these libraries and then use Lucene to index and search. Regards, *Vinay Modi **Celebrating 150th Birth Anniversary of Swami Vivekananda * www.vivekananda150jayanti.org The information contained in this e-mail message and any attachment(s) is intended only for the person or entity to which it is addressed and may contain information which is confidential, privileged and/or exempt from disclosure under applicable law. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this e-mail message in error, please contact the sender and delete it or otherwise destroy all copies of this transmission. On 11-06-2013 15:35, Vishrut Mehta wrote: Hello Everyone, I am Vishrut Mehta, currently a third year students at IIIT Hyderabad, India. I have been contributing to Open Source since two years and also have contributed to organizations like E-cidadania, Sahana Software Foundation, Gnome, etc. I am very interested in Search engines and search related libraries. I need some help from the community, I am currently working on a project which deals with the follow issue - Need to search within any uploaded documents(like .pdf, .doc, etc) from the userand need to search text or strings within those documents. Can anyone help me for this, it would be a great help ?! Thanks You! Regards, -- *Vishrut Mehta* International Institute of Information Technology, Gachibowli,Hyderabad-500032
Re: Introduction to PyLucene Community and some doubts
Hi, I suggest you have a look at Apache TIKA: http://tika.apache.org You can easily call a java -jar tika.jar command via python tools like os.popen and convert files in various formats to text. There's even a python wrapper based on JCC but I'm not sure if that's still maintained: http://redmine.djity.net/projects/pythontika/wiki Regards, Thomas -- Am 11.06.2013 um 12:05 schrieb Vishrut Mehta vishrut.mehta...@gmail.com: Hello Everyone, I am Vishrut Mehta, currently a third year students at IIIT Hyderabad, India. I have been contributing to Open Source since two years and also have contributed to organizations like E-cidadania, Sahana Software Foundation, Gnome, etc. I am very interested in Search engines and search related libraries. I need some help from the community, I am currently working on a project which deals with the follow issue - Need to search within any uploaded documents(like .pdf, .doc, etc) from the userand need to search text or strings within those documents. Can anyone help me for this, it would be a great help ?! Thanks You! Regards, -- *Vishrut Mehta* International Institute of Information Technology, Gachibowli,Hyderabad-500032
Re: jcc/sources/functions.cpp:20:23: error: arpa/inet.h: No such file or directory
Samantha, you may want to try a per-built binary of JCC for windows: there are version for win32 and py26 and py27 available here: http://code.google.com/a/apache-extras.org/p/pylucene-extra/downloads/list These eggs were built using Python, Java, ant and MSVC (Microsoft Visual Studio 9.0) - plus cygwin for the 'make' part. I can recommend this build chain especially as Python on Windows was also built using MSVC. Regards, Thomas -- Am 10.06.2013 um 20:08 schrieb Samantha Williamson 19scwilliamso...@gmail.com: Documentation on this project is so poor, I have no idea where to start. I found one archived thread from this mail list that covers the error, but it claims that the issue was fixed. Am I missing something here? I'm not using PyLucene with this, is that the issue?
Future of pylucene-extra
Hi, the pylucene-extra project started some time ago with the goal to provide pre-built PyLucene and JCC eggs on several OS/Python/Java combos [1]. In fact we collected 32 eggs since June 2011 - including versions for Windows and Mac OSX. Now that Google announced their EOL support for Downloads in Google Code projects [2]mit's time to think about the future of the pylucene-extra project. So before I start to move files and setup a new home for pylucene-extras let me ask you, the PyLucene users (well developers that use the library), if there is still interest in such a project. The download numbers in the Google Code project site indicate at least some interest, but some human feedback is certainly more valuable. Finally the question is if there's anyone else interested to contribute. We started with a bunch of enthusiasts who wanted to contribute eggs for different platforms, but currently I'm the only one who provides eggs - and only for Windows. I must admit however that building PyLucene on Linux or MacOSX is not a big deal anymore - windows is another story of course. So, pleas raise you hands if you want to see the pylucene-extra project survive and think it's still useful. Also let us know if you want to contribute. Looking forward to your feedback! Regards, Thomas [1] http://code.google.com/a/apache-extras.org/p/pylucene-extra/ [2] http://google-opensource.blogspot.de/2013/05/a-change-to-google-code-download-service.html
[jira] [Commented] (SOLR-3076) Solr should support block joins
[ https://issues.apache.org/jira/browse/SOLR-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680292#comment-13680292 ] Daniel Soriano commented on SOLR-3076: -- But paths in patches start with a/ and b/, and patches are applied into both subdirs. I don't understand how apply this patch file to trunk. I have done two trunk copies starting with a and b, but --dry-run notices that changes are applied into both. What's the trick? Solr should support block joins --- Key: SOLR-3076 URL: https://issues.apache.org/jira/browse/SOLR-3076 Project: Solr Issue Type: New Feature Reporter: Grant Ingersoll Fix For: 5.0, 4.4 Attachments: 27M-singlesegment-histogram.png, 27M-singlesegment.png, bjq-vs-filters-backward-disi.patch, bjq-vs-filters-illegal-state.patch, child-bjqparser.patch, dih-3076.patch, dih-config.xml, parent-bjq-qparser.patch, parent-bjq-qparser.patch, Screen Shot 2012-07-17 at 1.12.11 AM.png, SOLR-3076-childDocs.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-7036-childDocs-solr-fork-trunk-patched, solrconf-bjq-erschema-snippet.xml, solrconfig.xml.patch, tochild-bjq-filtered-search-fix.patch Lucene has the ability to do block joins, we should add it to Solr. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-5051) Incorrect abbreviation synonyms treating in WordDelimiterFilter
Artem Lukanin created LUCENE-5051: - Summary: Incorrect abbreviation synonyms treating in WordDelimiterFilter Key: LUCENE-5051 URL: https://issues.apache.org/jira/browse/LUCENE-5051 Project: Lucene - Core Issue Type: Bug Affects Versions: 4.3, 4.3.1 Reporter: Artem Lukanin If there are 2 abbreviation synonyms in the stream, they are not treated as synonyms after splitting by dots in WordDelimiterFilter. Correct treatment: before and after WordDelimiterFilter: codepre tokens: wi fi wireles network positions: 1 21 2 /pre/code Incorrect treatment: before WordDelimiterFilter: codepre tokens: wi.fi. wireles.network. positions: 1 1 /pre/code after WordDelimiterFilter: codepre tokens: wi fi wireles network positions: 1 23 4 /pre/code but should be: codepre tokens: wi fi wireles network positions: 1 21 2 /pre/code See a patch for Solr 4.3.1 configs, which demonstrates the bug if wi.fi. router is analyzed in name_syn field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5051) Incorrect abbreviation synonyms treating in WordDelimiterFilter
[ https://issues.apache.org/jira/browse/LUCENE-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680296#comment-13680296 ] Artem Lukanin commented on LUCENE-5051: --- Correct treatment: before and after WordDelimiterFilter: codepre tokens: wi fi wireles network positions: 1 21 2 /pre/code Incorrect treatment: before WordDelimiterFilter: codepre tokens: wi.fi. wireles.network. positions: 1 1 /pre/code after WordDelimiterFilter: codepre tokens: wi fi wireles network positions: 1 2 34 /pre/code but should be: codepre tokens: wi fi wireles network positions: 1 21 2 /pre/code Incorrect abbreviation synonyms treating in WordDelimiterFilter --- Key: LUCENE-5051 URL: https://issues.apache.org/jira/browse/LUCENE-5051 Project: Lucene - Core Issue Type: Bug Affects Versions: 4.3, 4.3.1 Reporter: Artem Lukanin If there are 2 abbreviation synonyms in the stream, they are not treated as synonyms after splitting by dots in WordDelimiterFilter. Correct treatment: before and after WordDelimiterFilter: codepre tokens: wi fi wireles network positions: 1 21 2 /pre/code Incorrect treatment: before WordDelimiterFilter: codepre tokens: wi.fi. wireles.network. positions: 1 1 /pre/code after WordDelimiterFilter: codepre tokens: wi fi wireles network positions: 1 23 4 /pre/code but should be: codepre tokens: wi fi wireles network positions: 1 21 2 /pre/code See a patch for Solr 4.3.1 configs, which demonstrates the bug if wi.fi. router is analyzed in name_syn field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-5051) Incorrect abbreviation synonyms treating in WordDelimiterFilter
[
https://issues.apache.org/jira/browse/LUCENE-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680296#comment-13680296
]
Artem Lukanin edited comment on LUCENE-5051 at 6/11/13 8:23 AM:
Correct treatment:
before and after WordDelimiterFilter:
{code}
tokens: wi fi wireles network
positions: 1 21 2
{code}
Incorrect treatment:
before WordDelimiterFilter:
{code}
tokens: wi.fi. wireles.network.
positions: 1 1
{code}
after WordDelimiterFilter:
{code}
tokens: wi fi wireles network
positions: 1 2 34
{code}
but should be:
{code}
tokens: wi fi wireles network
positions: 1 21 2
{code}
was (Author: alukanin):
Correct treatment:
before and after WordDelimiterFilter:
codepre
tokens: wi fi wireles network
positions: 1 21 2
/pre/code
Incorrect treatment:
before WordDelimiterFilter:
codepre
tokens: wi.fi. wireles.network.
positions: 1 1
/pre/code
after WordDelimiterFilter:
codepre
tokens: wi fi wireles network
positions: 1 2 34
/pre/code
but should be:
codepre
tokens: wi fi wireles network
positions: 1 21 2
/pre/code
Incorrect abbreviation synonyms treating in WordDelimiterFilter
---
Key: LUCENE-5051
URL: https://issues.apache.org/jira/browse/LUCENE-5051
Project: Lucene - Core
Issue Type: Bug
Affects Versions: 4.3, 4.3.1
Reporter: Artem Lukanin
If there are 2 abbreviation synonyms in the stream, they are not treated as
synonyms after splitting by dots in WordDelimiterFilter.
Correct treatment:
before and after WordDelimiterFilter:
codepre
tokens: wi fi wireles network
positions: 1 21 2
/pre/code
Incorrect treatment:
before WordDelimiterFilter:
codepre
tokens: wi.fi. wireles.network.
positions: 1 1
/pre/code
after WordDelimiterFilter:
codepre
tokens: wi fi wireles network
positions: 1 23 4
/pre/code
but should be:
codepre
tokens: wi fi wireles network
positions: 1 21 2
/pre/code
See a patch for Solr 4.3.1 configs, which demonstrates the bug if wi.fi.
router is analyzed in name_syn field.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Issue Comment Deleted] (LUCENE-5051) Incorrect abbreviation synonyms treating in WordDelimiterFilter
[
https://issues.apache.org/jira/browse/LUCENE-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Lukanin updated LUCENE-5051:
--
Comment: was deleted
(was: Correct treatment:
before and after WordDelimiterFilter:
{code}
tokens: wi fi wireles network
positions: 1 21 2
{code}
Incorrect treatment:
before WordDelimiterFilter:
{code}
tokens: wi.fi. wireles.network.
positions: 1 1
{code}
after WordDelimiterFilter:
{code}
tokens: wi fi wireles network
positions: 1 2 34
{code}
but should be:
{code}
tokens: wi fi wireles network
positions: 1 21 2
{code})
Incorrect abbreviation synonyms treating in WordDelimiterFilter
---
Key: LUCENE-5051
URL: https://issues.apache.org/jira/browse/LUCENE-5051
Project: Lucene - Core
Issue Type: Bug
Affects Versions: 4.3, 4.3.1
Reporter: Artem Lukanin
If there are 2 abbreviation synonyms in the stream, they are not treated as
synonyms after splitting by dots in WordDelimiterFilter.
Correct treatment:
before and after WordDelimiterFilter:
{code}
tokens: wi fi wireles network
positions: 1 21 2
{code}
Incorrect treatment:
before WordDelimiterFilter:
{code}
tokens: wi.fi. wireles.network.
positions: 1 1
{code}
after WordDelimiterFilter:
{code}
tokens: wi fi wireles network
positions: 1 23 4
{code}
but should be:
{code}
tokens: wi fi wireles network
positions: 1 21 2
{code}
See a patch for Solr 4.3.1 configs, which demonstrates the bug if wi.fi.
router is analyzed in name_syn field.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5051) Incorrect abbreviation synonyms treating in WordDelimiterFilter
[
https://issues.apache.org/jira/browse/LUCENE-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Lukanin updated LUCENE-5051:
--
Description:
If there are 2 abbreviation synonyms in the stream, they are not treated as
synonyms after splitting by dots in WordDelimiterFilter.
Correct treatment:
before and after WordDelimiterFilter:
{code}
tokens: wi fi wireles network
positions: 1 21 2
{code}
Incorrect treatment:
before WordDelimiterFilter:
{code}
tokens: wi.fi. wireles.network.
positions: 1 1
{code}
after WordDelimiterFilter:
{code}
tokens: wi fi wireles network
positions: 1 23 4
{code}
but should be:
{code}
tokens: wi fi wireles network
positions: 1 21 2
{code}
See a patch for Solr 4.3.1 configs, which demonstrates the bug if wi.fi.
router is analyzed in name_syn field.
was:
If there are 2 abbreviation synonyms in the stream, they are not treated as
synonyms after splitting by dots in WordDelimiterFilter.
Correct treatment:
before and after WordDelimiterFilter:
codepre
tokens: wi fi wireles network
positions: 1 21 2
/pre/code
Incorrect treatment:
before WordDelimiterFilter:
codepre
tokens: wi.fi. wireles.network.
positions: 1 1
/pre/code
after WordDelimiterFilter:
codepre
tokens: wi fi wireles network
positions: 1 23 4
/pre/code
but should be:
codepre
tokens: wi fi wireles network
positions: 1 21 2
/pre/code
See a patch for Solr 4.3.1 configs, which demonstrates the bug if wi.fi.
router is analyzed in name_syn field.
Incorrect abbreviation synonyms treating in WordDelimiterFilter
---
Key: LUCENE-5051
URL: https://issues.apache.org/jira/browse/LUCENE-5051
Project: Lucene - Core
Issue Type: Bug
Affects Versions: 4.3, 4.3.1
Reporter: Artem Lukanin
If there are 2 abbreviation synonyms in the stream, they are not treated as
synonyms after splitting by dots in WordDelimiterFilter.
Correct treatment:
before and after WordDelimiterFilter:
{code}
tokens: wi fi wireles network
positions: 1 21 2
{code}
Incorrect treatment:
before WordDelimiterFilter:
{code}
tokens: wi.fi. wireles.network.
positions: 1 1
{code}
after WordDelimiterFilter:
{code}
tokens: wi fi wireles network
positions: 1 23 4
{code}
but should be:
{code}
tokens: wi fi wireles network
positions: 1 21 2
{code}
See a patch for Solr 4.3.1 configs, which demonstrates the bug if wi.fi.
router is analyzed in name_syn field.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5051) Incorrect abbreviation synonyms treating in WordDelimiterFilter
[
https://issues.apache.org/jira/browse/LUCENE-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Lukanin updated LUCENE-5051:
--
Attachment: incorrect_synonym_treating_sample.patch
a patch for Solr configs, which shows the bug
Incorrect abbreviation synonyms treating in WordDelimiterFilter
---
Key: LUCENE-5051
URL: https://issues.apache.org/jira/browse/LUCENE-5051
Project: Lucene - Core
Issue Type: Bug
Affects Versions: 4.3, 4.3.1
Reporter: Artem Lukanin
Attachments: incorrect_synonym_treating_sample.patch
If there are 2 abbreviation synonyms in the stream, they are not treated as
synonyms after splitting by dots in WordDelimiterFilter.
Correct treatment:
before and after WordDelimiterFilter:
{code}
tokens: wi fi wireles network
positions: 1 21 2
{code}
Incorrect treatment:
before WordDelimiterFilter:
{code}
tokens: wi.fi. wireles.network.
positions: 1 1
{code}
after WordDelimiterFilter:
{code}
tokens: wi fi wireles network
positions: 1 23 4
{code}
but should be:
{code}
tokens: wi fi wireles network
positions: 1 21 2
{code}
See a patch for Solr 4.3.1 configs, which demonstrates the bug if wi.fi.
router is analyzed in name_syn field.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3076) Solr should support block joins
[
https://issues.apache.org/jira/browse/SOLR-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680302#comment-13680302
]
Jan Høydahl commented on SOLR-3076:
---
It's probably a git patch. Try {{patch -p1}} as mentioned here
http://wiki.apache.org/solr/HowToContribute#Working_With_Patches
Solr should support block joins
---
Key: SOLR-3076
URL: https://issues.apache.org/jira/browse/SOLR-3076
Project: Solr
Issue Type: New Feature
Reporter: Grant Ingersoll
Fix For: 5.0, 4.4
Attachments: 27M-singlesegment-histogram.png, 27M-singlesegment.png,
bjq-vs-filters-backward-disi.patch, bjq-vs-filters-illegal-state.patch,
child-bjqparser.patch, dih-3076.patch, dih-config.xml,
parent-bjq-qparser.patch, parent-bjq-qparser.patch, Screen Shot 2012-07-17 at
1.12.11 AM.png, SOLR-3076-childDocs.patch, SOLR-3076.patch, SOLR-3076.patch,
SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch,
SOLR-3076.patch, SOLR-7036-childDocs-solr-fork-trunk-patched,
solrconf-bjq-erschema-snippet.xml, solrconfig.xml.patch,
tochild-bjq-filtered-search-fix.patch
Lucene has the ability to do block joins, we should add it to Solr.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.7.0_21) - Build # 6016 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6016/ Java: 64bit/jdk1.7.0_21 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC All tests passed Build Log: [...truncated 27604 lines...] [javadoc] Generating Javadoc [javadoc] Javadoc execution [javadoc] Loading source files for package org.apache.lucene... [javadoc] Loading source files for package org.apache.lucene.analysis... [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6 [javadoc] Loading source files for package org.apache.lucene.analysis.tokenattributes... [javadoc] Loading source files for package org.apache.lucene.codecs... [javadoc] Loading source files for package org.apache.lucene.codecs.compressing... [javadoc] Loading source files for package org.apache.lucene.codecs.lucene3x... [javadoc] Loading source files for package org.apache.lucene.codecs.lucene40... [javadoc] Loading source files for package org.apache.lucene.codecs.lucene41... [javadoc] Loading source files for package org.apache.lucene.codecs.lucene42... [javadoc] Loading source files for package org.apache.lucene.codecs.perfield... [javadoc] Loading source files for package org.apache.lucene.document... [javadoc] Loading source files for package org.apache.lucene.index... [javadoc] Loading source files for package org.apache.lucene.search... [javadoc] Loading source files for package org.apache.lucene.search.payloads... [javadoc] Loading source files for package org.apache.lucene.search.similarities... [javadoc] Loading source files for package org.apache.lucene.search.spans... [javadoc] Loading source files for package org.apache.lucene.store... [javadoc] Loading source files for package org.apache.lucene.util... [javadoc] Loading source files for package org.apache.lucene.util.automaton... [javadoc] Loading source files for package org.apache.lucene.util.fst... [javadoc] Loading source files for package org.apache.lucene.util.mutable... [javadoc] Loading source files for package org.apache.lucene.util.packed... [javadoc] Constructing Javadoc information... [javadoc] Standard Doclet version 1.7.0_21 [javadoc] Building tree for all the packages and classes... [javadoc] Generating /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/docs/core/org/apache/lucene/search/package-summary.html... [javadoc] Copying file /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/core/src/java/org/apache/lucene/search/doc-files/nrq-formula-1.png to directory /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/docs/core/org/apache/lucene/search/doc-files... [javadoc] Copying file /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/core/src/java/org/apache/lucene/search/doc-files/nrq-formula-2.png to directory /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/docs/core/org/apache/lucene/search/doc-files... [javadoc] Building index for all the packages and classes... [javadoc] Building index for all classes... [javadoc] Generating /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/docs/core/help-doc.html... [javadoc] 1 warning [...truncated 33 lines...] [javadoc] Generating Javadoc [javadoc] Javadoc execution [javadoc] Loading source files for package org.apache.lucene.analysis.ar... [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6 [javadoc] Loading source files for package org.apache.lucene.analysis.bg... [javadoc] Loading source files for package org.apache.lucene.analysis.br... [javadoc] Loading source files for package org.apache.lucene.analysis.ca... [javadoc] Loading source files for package org.apache.lucene.analysis.charfilter... [javadoc] Loading source files for package org.apache.lucene.analysis.cjk... [javadoc] Loading source files for package org.apache.lucene.analysis.cn... [javadoc] Loading source files for package org.apache.lucene.analysis.commongrams... [javadoc] Loading source files for package org.apache.lucene.analysis.compound... [javadoc] Loading source files for package org.apache.lucene.analysis.compound.hyphenation... [javadoc] Loading source files for package org.apache.lucene.analysis.core... [javadoc] Loading source files for package org.apache.lucene.analysis.cz... [javadoc] Loading source files for package org.apache.lucene.analysis.da... [javadoc] Loading source files for package org.apache.lucene.analysis.de... [javadoc] Loading source files for package org.apache.lucene.analysis.el... [javadoc] Loading source files for package org.apache.lucene.analysis.en... [javadoc] Loading source files for package org.apache.lucene.analysis.es... [javadoc] Loading source files for package org.apache.lucene.analysis.eu... [javadoc] Loading source files for package org.apache.lucene.analysis.fa... [javadoc] Loading source files for package org.apache.lucene.analysis.fi... [javadoc] Loading source files
[jira] [Updated] (SOLR-4837) Add defType like parameter for filter query and facet.query.
[ https://issues.apache.org/jira/browse/SOLR-4837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shingo Sasaki updated SOLR-4837: Attachment: SOLR-4837.patch This patch adds two parameters: fq.defType and facet.query.defType. Add defType like parameter for filter query and facet.query. Key: SOLR-4837 URL: https://issues.apache.org/jira/browse/SOLR-4837 Project: Solr Issue Type: Improvement Reporter: Shingo Sasaki Attachments: SOLR-4837.patch The parameter defType of QueryComponent does not affect filter queries (=fq) and facet queries, but there are situations that queries are parsed equally. * All queries is parsed by query parser of defType. * Add new parameters, fq.defType / facet.defType. I do not know which is appropriate. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: lucene-solr pull request: Lucene solr 4 3
Typically, people will open up a JIRA and attach the patch in order to get it into the system, could you go ahead and do that? See: http://wiki.apache.org/solr/HowToContribute Best Erick On Mon, Jun 10, 2013 at 4:35 PM, thrykol g...@git.apache.org wrote: GitHub user thrykol opened a pull request: https://github.com/apache/lucene-solr/pull/10 Lucene solr 4 3 Logging pages threw a JS error when logging has not been initialized. Cleaned up logic to handle un-initialized logging. You can merge this pull request into a Git repository by running: $ git pull https://github.com/thrykol/lucene-solr lucene_solr_4_3 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/lucene-solr/pull/10.patch - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 552 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/552/ Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseSerialGC 1 tests failed. REGRESSION: org.apache.lucene.facet.search.TestDrillSideways.testEmptyIndex Error Message: Stack Trace: java.lang.NullPointerException at __randomizedtesting.SeedInfo.seed([77627EDBF39A97CA:F391723E2213B233]:0) at org.apache.lucene.facet.search.DrillSidewaysScorer.score(DrillSidewaysScorer.java:83) at org.apache.lucene.search.AssertingScorer.score(AssertingScorer.java:131) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:616) at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:93) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:301) at org.apache.lucene.facet.search.DrillSideways.search(DrillSideways.java:264) at org.apache.lucene.facet.search.DrillSideways.search(DrillSideways.java:436) at org.apache.lucene.facet.search.TestDrillSideways.testEmptyIndex(TestDrillSideways.java:1170) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) at java.lang.Thread.run(Thread.java:722) Build Log: [...truncated 6805
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4057 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4057/ All tests passed Build Log: [...truncated 28696 lines...] BUILD FAILED /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:385: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:60: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/build.xml:250: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/build.xml:558: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:2107: java.net.SocketException: Connection reset by peer at java.net.PlainSocketImpl.socketSetOption(Native Method) at java.net.AbstractPlainSocketImpl.setOption(AbstractPlainSocketImpl.java:269) at java.net.Socket.setTcpNoDelay(Socket.java:940) at sun.security.ssl.BaseSSLSocketImpl.setTcpNoDelay(BaseSSLSocketImpl.java:330) at sun.net.www.http.HttpClient.openServer(HttpClient.java:390) at sun.net.www.http.HttpClient.openServer(HttpClient.java:473) at sun.net.www.protocol.https.HttpsClient.init(HttpsClient.java:270) at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:327) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:931) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177) at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153) at org.apache.tools.ant.taskdefs.Get$GetThread.openConnection(Get.java:660) at org.apache.tools.ant.taskdefs.Get$GetThread.get(Get.java:579) at org.apache.tools.ant.taskdefs.Get$GetThread.run(Get.java:569) Total time: 66 minutes 25 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Recording test results Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-4641) Schema should throw exception on illegal field parameters
is there a solution for it, we also use own custom properties?! -- View this message in context: http://lucene.472066.n3.nabble.com/jira-Commented-SOLR-4641-Schema-should-throw-exception-on-illegal-field-parameters-tp4063164p4069623.html Sent from the Lucene - Java Developer mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5038) Don't call MergePolicy / IndexWriter during DWPT Flush
[ https://issues.apache.org/jira/browse/LUCENE-5038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Willnauer updated LUCENE-5038: Attachment: LUCENE-5038.patch here is an updated patch. I removed _setUseCompoundFile_ from the MPs now we only have the ratio on the MP and setUseCF on IWC. Yet, given that patch I think we can safely move the CFS related settings from LogMP and TieredMP into MergePolicy or add an intermediate class. The are all identical and we should consolidate this. Any objections if I pull them up to MP? Don't call MergePolicy / IndexWriter during DWPT Flush -- Key: LUCENE-5038 URL: https://issues.apache.org/jira/browse/LUCENE-5038 Project: Lucene - Core Issue Type: Improvement Components: core/index Affects Versions: 4.3 Reporter: Simon Willnauer Assignee: Simon Willnauer Fix For: 4.4 Attachments: LUCENE-5038.patch, LUCENE-5038.patch, LUCENE-5038.patch, LUCENE-5038.patch We currently consult the indexwriter - merge policy to decide if we need to write CFS or not which is bad in many ways. - we should call mergepolicy only during merges - we should never sync on IW during DWPT flush - we should be able to make the decision if we need to write CFS or not before flush, ie. we could write parts of the flush directly to CFS or even start writing stored fields directly. - in the NRT case it might make sense to write all flushes to CFS to minimize filedescriptors independent of the index size. I wonder if we can use a simple boolean for this in the IWC and get away with not consulting merge policy. This would simplify concurrency a lot here already. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-4641) Schema should throw exception on illegal field parameters
From Steve's reply:
One thing you can do right now as an alternative to custom {{field}}
attributes is child elements, e.g.:
{code:xml}
field indexed=true multiValued=true name=f1 stored=true type=string
MYPARAMVALUE/MYPARAM !-- Maven property style --
custom MYPARAM=VALUE/ !-- Alternative syntax; element name
could be anything you want --
...
/field
{code}
Is the above sufficient for your use cases?
On Tue, Jun 11, 2013 at 8:20 AM, uwe72 uwe.clem...@exxcellent.de wrote:
is there a solution for it, we also use own custom properties?!
--
View this message in context:
http://lucene.472066.n3.nabble.com/jira-Commented-SOLR-4641-Schema-should-throw-exception-on-illegal-field-parameters-tp4063164p4069623.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-4916) Add support to write and read Solr index files and transaction log files to and from HDFS.
Mark Miller created SOLR-4916: - Summary: Add support to write and read Solr index files and transaction log files to and from HDFS. Key: SOLR-4916 URL: https://issues.apache.org/jira/browse/SOLR-4916 Project: Solr Issue Type: New Feature Reporter: Mark Miller Assignee: Mark Miller -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-4816) Add document routing to CloudSolrServer
[ https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680393#comment-13680393 ] Joel Bernstein edited comment on SOLR-4816 at 6/11/13 2:07 PM: --- Mark, there were a couple of changes I still wanted to make to this ticket: 1) Add a switch to turn on/off threading. 2) Add a thread pool rather then spawning new threads each request. 3) Add a few more tests. But before I dive in I wanted to be sure we're on the same page. Does this design satisfy the back compat issue for the response and exceptions? Are there other show stoppers in this design/implementation that need to be addressed? was (Author: joel.bernstein): Mark, there were a couple of changes I still wanted to make to this ticket: 1) Add a switch to turn on/off threading. 2) Add a thread pool rather then spawning new threads each request. 3) Add a few more tests. But before I dive I wanted to be sure we're on the same page. Does this design satisfy the back compat issue for the response and exceptions? Are there other show stoppers in this design/implementation that need to be addressed? Add document routing to CloudSolrServer --- Key: SOLR-4816 URL: https://issues.apache.org/jira/browse/SOLR-4816 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.3 Reporter: Joel Bernstein Assignee: Mark Miller Priority: Minor Fix For: 5.0, 4.4 Attachments: SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816-sriesenberg.patch This issue adds the following enhancements to CloudSolrServer's update logic: 1) Document routing: Updates are routed directly to the correct shard leader eliminating document routing at the server. 2) Parallel update execution: Updates for each shard are executed in a separate thread so parallel indexing can occur across the cluster. 3) Javabin transport: Update requests are sent via javabin transport. These enhancements should allow for near linear scalability on indexing throughput. Usage: CloudSolrServer cloudClient = new CloudSolrServer(zkAddress); SolrInputDocument doc1 = new SolrInputDocument(); doc1.addField(id, 0); doc1.addField(a_t, hello1); SolrInputDocument doc2 = new SolrInputDocument(); doc2.addField(id, 2); doc2.addField(a_t, hello2); UpdateRequest request = new UpdateRequest(); request.add(doc1); request.add(doc2); request.setAction(AbstractUpdateRequest.ACTION.OPTIMIZE, false, false); NamedList response = cloudClient.request(request); // Returns a backwards compatible condensed response. //To get more detailed response down cast to RouteResponse: CloudSolrServer.RouteResponse rr = (CloudSolrServer.RouteResponse)response; NamedList responses = rr.getRouteResponse(); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4816) Add document routing to CloudSolrServer
[ https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680393#comment-13680393 ] Joel Bernstein commented on SOLR-4816: -- Mark, there were a couple of changes I still wanted to make to this ticket: 1) Add a switch to turn on/off threading. 2) Add a thread pool rather then spawning new threads each request. 3) Add a few more tests. But before I dive I wanted to be sure we're on the same page. Does this design satisfy the back compat issue for the response and exceptions? Are there other show stoppers in this design/implementation that need to be addressed? Add document routing to CloudSolrServer --- Key: SOLR-4816 URL: https://issues.apache.org/jira/browse/SOLR-4816 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.3 Reporter: Joel Bernstein Assignee: Mark Miller Priority: Minor Fix For: 5.0, 4.4 Attachments: SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816-sriesenberg.patch This issue adds the following enhancements to CloudSolrServer's update logic: 1) Document routing: Updates are routed directly to the correct shard leader eliminating document routing at the server. 2) Parallel update execution: Updates for each shard are executed in a separate thread so parallel indexing can occur across the cluster. 3) Javabin transport: Update requests are sent via javabin transport. These enhancements should allow for near linear scalability on indexing throughput. Usage: CloudSolrServer cloudClient = new CloudSolrServer(zkAddress); SolrInputDocument doc1 = new SolrInputDocument(); doc1.addField(id, 0); doc1.addField(a_t, hello1); SolrInputDocument doc2 = new SolrInputDocument(); doc2.addField(id, 2); doc2.addField(a_t, hello2); UpdateRequest request = new UpdateRequest(); request.add(doc1); request.add(doc2); request.setAction(AbstractUpdateRequest.ACTION.OPTIMIZE, false, false); NamedList response = cloudClient.request(request); // Returns a backwards compatible condensed response. //To get more detailed response down cast to RouteResponse: CloudSolrServer.RouteResponse rr = (CloudSolrServer.RouteResponse)response; NamedList responses = rr.getRouteResponse(); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4816) Add document routing to CloudSolrServer
[ https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680409#comment-13680409 ] Mark Miller commented on SOLR-4816: --- 1 - yes, I think we need the option of using just one thread. I also think that should be the default until 5x. 2 - yes, I think a thread pool is the way to go 3 - always will get a +1 from me on more tests bq. Does this design satisfy the back compat issue for the response and exceptions? I have to review closer - it's on my short list. I think if we can use one thread and the responses have the same format by default, I won't have much concern. bq. Are there other show stoppers in this design/implementation I won't know till I look closer, but I don't doubt we will get this in. Add document routing to CloudSolrServer --- Key: SOLR-4816 URL: https://issues.apache.org/jira/browse/SOLR-4816 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.3 Reporter: Joel Bernstein Assignee: Mark Miller Priority: Minor Fix For: 5.0, 4.4 Attachments: SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816-sriesenberg.patch This issue adds the following enhancements to CloudSolrServer's update logic: 1) Document routing: Updates are routed directly to the correct shard leader eliminating document routing at the server. 2) Parallel update execution: Updates for each shard are executed in a separate thread so parallel indexing can occur across the cluster. 3) Javabin transport: Update requests are sent via javabin transport. These enhancements should allow for near linear scalability on indexing throughput. Usage: CloudSolrServer cloudClient = new CloudSolrServer(zkAddress); SolrInputDocument doc1 = new SolrInputDocument(); doc1.addField(id, 0); doc1.addField(a_t, hello1); SolrInputDocument doc2 = new SolrInputDocument(); doc2.addField(id, 2); doc2.addField(a_t, hello2); UpdateRequest request = new UpdateRequest(); request.add(doc1); request.add(doc2); request.setAction(AbstractUpdateRequest.ACTION.OPTIMIZE, false, false); NamedList response = cloudClient.request(request); // Returns a backwards compatible condensed response. //To get more detailed response down cast to RouteResponse: CloudSolrServer.RouteResponse rr = (CloudSolrServer.RouteResponse)response; NamedList responses = rr.getRouteResponse(); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 552 - Still Failing!
Committed a fix. Shai On Tue, Jun 11, 2013 at 2:56 PM, Policeman Jenkins Server jenk...@thetaphi.de wrote: Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/552/ Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseSerialGC 1 tests failed. REGRESSION: org.apache.lucene.facet.search.TestDrillSideways.testEmptyIndex Error Message: Stack Trace: java.lang.NullPointerException at __randomizedtesting.SeedInfo.seed([77627EDBF39A97CA:F391723E2213B233]:0) at org.apache.lucene.facet.search.DrillSidewaysScorer.score(DrillSidewaysScorer.java:83) at org.apache.lucene.search.AssertingScorer.score(AssertingScorer.java:131) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:616) at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:93) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:301) at org.apache.lucene.facet.search.DrillSideways.search(DrillSideways.java:264) at org.apache.lucene.facet.search.DrillSideways.search(DrillSideways.java:436) at org.apache.lucene.facet.search.TestDrillSideways.testEmptyIndex(TestDrillSideways.java:1170) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[jira] [Commented] (SOLR-4909) Solr and IndexReader Re-opening on Replication Slave
[ https://issues.apache.org/jira/browse/SOLR-4909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680414#comment-13680414 ] Mark Miller commented on SOLR-4909: --- It seems like it would be nicer if the solution worked for 'repeaters' as well, but that could solve things for the slave case - the likely side effects are probably confusion if a user see the numbers are different and they should align - i dont like that so much. But currently you have to commit or reopen the writer to pick up the newest commit. Solr and IndexReader Re-opening on Replication Slave Key: SOLR-4909 URL: https://issues.apache.org/jira/browse/SOLR-4909 Project: Solr Issue Type: Improvement Components: replication (java), search Affects Versions: 4.3 Reporter: Michael Garski Fix For: 5.0, 4.4 Attachments: SOLR-4909-demo.patch I've been experimenting with caching filter data per segment in Solr using a CachingWrapperFilter FilteredQuery within a custom query parser (as suggested by [~yo...@apache.org] in SOLR-3763) and encountered situations where the value of getCoreCacheKey() on the AtomicReader for each segment can change for a given segment on disk when the searcher is reopened. As CachingWrapperFilter uses the value of the segment's getCoreCacheKey() as the key in the cache, there are situations where the data cached on that segment is not reused when the segment on disk is still part of the index. This affects the Lucene field cache and field value caches as well as they are cached per segment. When Solr first starts it opens the searcher's underlying DirectoryReader in StandardIndexReaderFactory.newReader by calling DirectoryReader.open(indexDir, termInfosIndexDivisor), and the reader is subsequently reopened in SolrCore.openNewSearcher by calling DirectoryReader.openIfChanged(currentReader, writer.get(), true). The act of reopening the reader with the writer when it was first opened without a writer results in the value of getCoreCacheKey() changing on each of the segments even though some of the segments have not changed. Depending on the role of the Solr server, this has different effects: * On a SolrCloud node or free-standing index and search server the segment cache is invalidated during the first DirectoryReader reopen - subsequent reopens use the same IndexWriter instance and as such the value of getCoreCacheKey() on each segment does not change so the cache is retained. * For a master-slave replication set up the segment cache invalidation occurs on the slave during every replication as the index is reopened using a new IndexWriter instance which results in the value of getCoreCacheKey() changing on each segment when the DirectoryReader is reopened using a different IndexWriter instance. I can think of a few approaches to alter the re-opening behavior to allow reuse of segment level caches in both cases, and I'd like to get some input on other ideas before digging in: * To change the cloud node/standalone first commit issue it might be possible to create the UpdateHandler and IndexWriter before the DirectoryReader, and use the writer to open the reader. There is a comment in the SolrCore constructor by [~yo...@apache.org] that the searcher should be opened before the update handler so that may not be an acceptable approach. * To change the behavior of a slave in a replication set up, one solution would be to not open a writer from the SnapPuller when the new index is retrieved if the core is enabled as a slave only. The writer is needed on a server configured as a master slave that is functioning as a replication repeater so downstream slaves can see the changes in the index and retrieve them. I'll attach a unit test that demonstrates the behavior of reopening the DirectoryReader and it's effects on the value of getCoreCacheKey. My assumption is that the behavior of Lucene during the various reader reopen operations is correct and that the changes are necessary on the Solr side of things. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4792) stop shipping a war in 5.0
[ https://issues.apache.org/jira/browse/SOLR-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680415#comment-13680415 ] Mark Miller commented on SOLR-4792: --- This issue is simply about not shipping a war anymore - I think people should create new JIRA issues for other ideas around this change. stop shipping a war in 5.0 -- Key: SOLR-4792 URL: https://issues.apache.org/jira/browse/SOLR-4792 Project: Solr Issue Type: Task Components: Build Reporter: Robert Muir Assignee: Robert Muir Fix For: 5.0 Attachments: SOLR-4792.patch see the vote on the developer list. This is the first step: if we stop shipping a war then we are free to do anything we want. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4910) solr.xml persistence is completely broken
[
https://issues.apache.org/jira/browse/SOLR-4910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680416#comment-13680416
]
Mark Miller commented on SOLR-4910:
---
bq. The bulk of the effort in what I did tonight was figuring out how to verify
that all the nodes and attributes in one xml file are identical to another.
Nice! That will be useful.
solr.xml persistence is completely broken
-
Key: SOLR-4910
URL: https://issues.apache.org/jira/browse/SOLR-4910
Project: Solr
Issue Type: Bug
Affects Versions: 5.0, 4.4
Reporter: Erick Erickson
Assignee: Erick Erickson
Priority: Blocker
Attachments: SOLR-4910.patch, SOLR-4910.patch
I'm working on SOLR-4862 (persisting a created core doesn't preserve some
values) and at least compared to 4.3 code, persisting to solr.xml is
completely broken.
I learned to hate persistence while working on SOLR-4196 etc. and I'm glad
it's going away. I frequently got lost in implicit properties (they're easy
to persist and shouldn't be), what should/shouldn't be persisted (e.g. the
translated ${var:default} or the original), and it was a monster, so don't
think I'm nostalgic for the historical behavior.
Before I dive back in I want to get some idea whether or not the current
behavior was intentional or not, I don't want to go back into that junk only
to undo someone else's work.
Creating a new core (collection2 in my example) with persistence turned on in
solr.xml for instance changes the original definition for collection1 (stock
4.x as of tonight) from this:
core name=collection1 instanceDir=collection1 shard=${shard:}
collection=${collection:collection1} config=${solrconfig:solrconfig.xml}
schema=${schema:schema.xml}
coreNodeName=${coreNodeName:}/
to this:
core loadOnStartup=true shard=${shard:} instanceDir=collection1/
transient=false name=collection1 dataDir=data/
collection=${collection:collection1}
property name=name value=collection1/
property name=config value=solrconfig.xml/
property name=solr.core.instanceDir value=solr/collection1//
property name=transient value=false/
property name=schema value=schema.xml/
property name=loadOnStartup value=true/
property name=solr.core.schemaName value=schema.xml/
property name=solr.core.name value=collection1/
property name=solr.core.dataDir value=data//
property name=instanceDir value=collection1//
property name=solr.core.configName value=solrconfig.xml/
/core
So, there are two questions:
1 what is correct for 4.x?
2 do we care at all about 5.x?
As much as I hate to say it, I think that we need to go back to the 4.3
behavior. It might be as simple as not persisting in the property tags
anything already in the original definition. Not quite sure what to put where
in the newly-created core though, I suspect that the compact core + attribs
would be best (assuming there's no property tag already in the definition.
I really hate the mix of attributes on the core tag and property tags,
wish we had one or the other
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [VOTE] Lucene Solr 4.3.1 RC1
+1 -Yonik http://lucidworks.com On Mon, Jun 10, 2013 at 3:16 AM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: http://people.apache.org/~shalin/staging_area/lucene-solr-4.3.1-RC1-rev1491148/ Smoke tester is happy for me (osx). Here's my +1! -- Regards, Shalin Shekhar Mangar. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5038) Don't call MergePolicy / IndexWriter during DWPT Flush
[ https://issues.apache.org/jira/browse/LUCENE-5038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680428#comment-13680428 ] Shai Erera commented on LUCENE-5038: Patch looks good. +1 to pull setNoCFSRatio to MP. It's a generic setting. We can default to 1.0 (always) in MP in order to not break MP impls out there. Also, Can you add documentation to IWC.setUseCFS to see MP.setNoCFSRatio for controlling CFS for merged segments? Don't call MergePolicy / IndexWriter during DWPT Flush -- Key: LUCENE-5038 URL: https://issues.apache.org/jira/browse/LUCENE-5038 Project: Lucene - Core Issue Type: Improvement Components: core/index Affects Versions: 4.3 Reporter: Simon Willnauer Assignee: Simon Willnauer Fix For: 4.4 Attachments: LUCENE-5038.patch, LUCENE-5038.patch, LUCENE-5038.patch, LUCENE-5038.patch We currently consult the indexwriter - merge policy to decide if we need to write CFS or not which is bad in many ways. - we should call mergepolicy only during merges - we should never sync on IW during DWPT flush - we should be able to make the decision if we need to write CFS or not before flush, ie. we could write parts of the flush directly to CFS or even start writing stored fields directly. - in the NRT case it might make sense to write all flushes to CFS to minimize filedescriptors independent of the index size. I wonder if we can use a simple boolean for this in the IWC and get away with not consulting merge policy. This would simplify concurrency a lot here already. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: lucene indexing and field configuration or schema
Hi John, On Mon, Jun 10, 2013 at 8:17 PM, John Wang john.w...@gmail.com wrote: We found having the schema information up front allows us flexibilities in designing our posting list, also makes the indexingchain logic much simpler. Can you give examples of the kind of decisions that you are able to make by having the schema up-front? -- Adrien - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: lucene-solr pull request: Lucene solr 4 3
: Typically, people will open up a JIRA and attach the patch in order to : get it into the system, could you go ahead and do that? See: : http://wiki.apache.org/solr/HowToContribute Erick: pull request emails like this are automaticly generated by the git.apache.org - github.org bridge put in place by Infra anytime someone with a fork on github hit's the Pull Request button. The user involved is not neccessarily subscribed to this list, and the From email address is a black box (g...@git.apache.org) conceptually, pull request emails should be treated the same as someone anonymously emailing the URL of a patch to the list. : On Mon, Jun 10, 2013 at 4:35 PM, thrykol g...@git.apache.org wrote: : GitHub user thrykol opened a pull request: : : https://github.com/apache/lucene-solr/pull/10 : : Lucene solr 4 3 : : Logging pages threw a JS error when logging has not been initialized. Cleaned up logic to handle un-initialized logging. : : You can merge this pull request into a Git repository by running: : : $ git pull https://github.com/thrykol/lucene-solr lucene_solr_4_3 : : Alternatively you can review and apply these changes as the patch at: : : https://github.com/apache/lucene-solr/pull/10.patch -Hoss - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: lucene-solr pull request: Lucene solr 4 3
Related: https://issues.apache.org/jira/browse/LEGAL-156 We can accept github pull requests. -Yonik http://lucidworks.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4816) Add document routing to CloudSolrServer
[ https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680453#comment-13680453 ] Shawn Heisey commented on SOLR-4816: The work I've been doing (slowly) on SOLR-4715 overlaps with this issue. I've been thinking that I need to divide it into bite-size tasks for my own purposes, but that could possibly help here too. If I concentrate first on giving CloudSolrServer a way to set the writer to binary, that would reduce the complexity of this patch quite a bit. Thoughts? Add document routing to CloudSolrServer --- Key: SOLR-4816 URL: https://issues.apache.org/jira/browse/SOLR-4816 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.3 Reporter: Joel Bernstein Assignee: Mark Miller Priority: Minor Fix For: 5.0, 4.4 Attachments: SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816-sriesenberg.patch This issue adds the following enhancements to CloudSolrServer's update logic: 1) Document routing: Updates are routed directly to the correct shard leader eliminating document routing at the server. 2) Parallel update execution: Updates for each shard are executed in a separate thread so parallel indexing can occur across the cluster. 3) Javabin transport: Update requests are sent via javabin transport. These enhancements should allow for near linear scalability on indexing throughput. Usage: CloudSolrServer cloudClient = new CloudSolrServer(zkAddress); SolrInputDocument doc1 = new SolrInputDocument(); doc1.addField(id, 0); doc1.addField(a_t, hello1); SolrInputDocument doc2 = new SolrInputDocument(); doc2.addField(id, 2); doc2.addField(a_t, hello2); UpdateRequest request = new UpdateRequest(); request.add(doc1); request.add(doc2); request.setAction(AbstractUpdateRequest.ACTION.OPTIMIZE, false, false); NamedList response = cloudClient.request(request); // Returns a backwards compatible condensed response. //To get more detailed response down cast to RouteResponse: CloudSolrServer.RouteResponse rr = (CloudSolrServer.RouteResponse)response; NamedList responses = rr.getRouteResponse(); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-4816) Add document routing to CloudSolrServer
[ https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680453#comment-13680453 ] Shawn Heisey edited comment on SOLR-4816 at 6/11/13 4:38 PM: - The work I've been doing (slowly) on SOLR-4715 overlaps with this issue. I've been thinking that I need to divide it into bite-size tasks for my own purposes, but that could possibly help here too. If I concentrate first on giving LBHttpSolrServer an easy way to set the writer to binary, that would reduce the complexity of this patch quite a bit. Thoughts? was (Author: elyograg): The work I've been doing (slowly) on SOLR-4715 overlaps with this issue. I've been thinking that I need to divide it into bite-size tasks for my own purposes, but that could possibly help here too. If I concentrate first on giving CloudSolrServer a way to set the writer to binary, that would reduce the complexity of this patch quite a bit. Thoughts? Add document routing to CloudSolrServer --- Key: SOLR-4816 URL: https://issues.apache.org/jira/browse/SOLR-4816 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.3 Reporter: Joel Bernstein Assignee: Mark Miller Priority: Minor Fix For: 5.0, 4.4 Attachments: SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816-sriesenberg.patch This issue adds the following enhancements to CloudSolrServer's update logic: 1) Document routing: Updates are routed directly to the correct shard leader eliminating document routing at the server. 2) Parallel update execution: Updates for each shard are executed in a separate thread so parallel indexing can occur across the cluster. 3) Javabin transport: Update requests are sent via javabin transport. These enhancements should allow for near linear scalability on indexing throughput. Usage: CloudSolrServer cloudClient = new CloudSolrServer(zkAddress); SolrInputDocument doc1 = new SolrInputDocument(); doc1.addField(id, 0); doc1.addField(a_t, hello1); SolrInputDocument doc2 = new SolrInputDocument(); doc2.addField(id, 2); doc2.addField(a_t, hello2); UpdateRequest request = new UpdateRequest(); request.add(doc1); request.add(doc2); request.setAction(AbstractUpdateRequest.ACTION.OPTIMIZE, false, false); NamedList response = cloudClient.request(request); // Returns a backwards compatible condensed response. //To get more detailed response down cast to RouteResponse: CloudSolrServer.RouteResponse rr = (CloudSolrServer.RouteResponse)response; NamedList responses = rr.getRouteResponse(); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4792) stop shipping a war in 5.0
[ https://issues.apache.org/jira/browse/SOLR-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680464#comment-13680464 ] Noble Paul commented on SOLR-4792: -- bq. I think people should create new JIRA issues for other ideas around this change. I shall open an issue for the Solr as a standalone java app. stop shipping a war in 5.0 -- Key: SOLR-4792 URL: https://issues.apache.org/jira/browse/SOLR-4792 Project: Solr Issue Type: Task Components: Build Reporter: Robert Muir Assignee: Robert Muir Fix For: 5.0 Attachments: SOLR-4792.patch see the vote on the developer list. This is the first step: if we stop shipping a war then we are free to do anything we want. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-4917) Logging UI Javascript errors
Shane created SOLR-4917: --- Summary: Logging UI Javascript errors Key: SOLR-4917 URL: https://issues.apache.org/jira/browse/SOLR-4917 Project: Solr Issue Type: Bug Components: web gui Affects Versions: 4.3 Reporter: Shane Fix For: 4.3 When logging has not been configured, the logging/level UI throw some JavaScript errors. I submitted a PR via GitHub. The GitHub patch can be found at https://github.com/apache/lucene-solr/pull/10.patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-4918) Accept router as a collection create param and persist it
Noble Paul created SOLR-4918: Summary: Accept router as a collection create param and persist it Key: SOLR-4918 URL: https://issues.apache.org/jira/browse/SOLR-4918 Project: Solr Issue Type: Improvement Components: SolrCloud Reporter: Noble Paul Assignee: Noble Paul Currently the router is implicit and it is confusing to users how how cloud is sharded -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5038) Don't call MergePolicy / IndexWriter during DWPT Flush
[ https://issues.apache.org/jira/browse/LUCENE-5038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680499#comment-13680499 ] Simon Willnauer commented on LUCENE-5038: - bq. Patch looks good. +1 to pull setNoCFSRatio to MP. It's a generic setting. We can default to 1.0 (always) in MP in order to not break MP impls out there. Also, Can you add documentation to IWC.setUseCFS to see MP.setNoCFSRatio for controlling CFS for merged segments? I will use 1.0 as the default I guess I will also pull up _setMaxCFSSegmentSizeMB_ and friends since they are redundant as well. I agree default should be 1.0 on MP but the TMP and LMP use their defaults or do I miss something? Don't call MergePolicy / IndexWriter during DWPT Flush -- Key: LUCENE-5038 URL: https://issues.apache.org/jira/browse/LUCENE-5038 Project: Lucene - Core Issue Type: Improvement Components: core/index Affects Versions: 4.3 Reporter: Simon Willnauer Assignee: Simon Willnauer Fix For: 4.4 Attachments: LUCENE-5038.patch, LUCENE-5038.patch, LUCENE-5038.patch, LUCENE-5038.patch We currently consult the indexwriter - merge policy to decide if we need to write CFS or not which is bad in many ways. - we should call mergepolicy only during merges - we should never sync on IW during DWPT flush - we should be able to make the decision if we need to write CFS or not before flush, ie. we could write parts of the flush directly to CFS or even start writing stored fields directly. - in the NRT case it might make sense to write all flushes to CFS to minimize filedescriptors independent of the index size. I wonder if we can use a simple boolean for this in the IWC and get away with not consulting merge policy. This would simplify concurrency a lot here already. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3076) Solr should support block joins
[ https://issues.apache.org/jira/browse/SOLR-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680500#comment-13680500 ] Daniel Soriano commented on SOLR-3076: -- That has been perfect!! Thanks everyone. Now, block joins are running. Solr should support block joins --- Key: SOLR-3076 URL: https://issues.apache.org/jira/browse/SOLR-3076 Project: Solr Issue Type: New Feature Reporter: Grant Ingersoll Fix For: 5.0, 4.4 Attachments: 27M-singlesegment-histogram.png, 27M-singlesegment.png, bjq-vs-filters-backward-disi.patch, bjq-vs-filters-illegal-state.patch, child-bjqparser.patch, dih-3076.patch, dih-config.xml, parent-bjq-qparser.patch, parent-bjq-qparser.patch, Screen Shot 2012-07-17 at 1.12.11 AM.png, SOLR-3076-childDocs.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-7036-childDocs-solr-fork-trunk-patched, solrconf-bjq-erschema-snippet.xml, solrconfig.xml.patch, tochild-bjq-filtered-search-fix.patch Lucene has the ability to do block joins, we should add it to Solr. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5038) Don't call MergePolicy / IndexWriter during DWPT Flush
[ https://issues.apache.org/jira/browse/LUCENE-5038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680505#comment-13680505 ] Shai Erera commented on LUCENE-5038: TMP and LMP should use their defaults. I was suggesting a default for MP so that you don't need to make the method abstract and potentially break existing MP impls out there. 1.0 sounds like a good default. Don't call MergePolicy / IndexWriter during DWPT Flush -- Key: LUCENE-5038 URL: https://issues.apache.org/jira/browse/LUCENE-5038 Project: Lucene - Core Issue Type: Improvement Components: core/index Affects Versions: 4.3 Reporter: Simon Willnauer Assignee: Simon Willnauer Fix For: 4.4 Attachments: LUCENE-5038.patch, LUCENE-5038.patch, LUCENE-5038.patch, LUCENE-5038.patch We currently consult the indexwriter - merge policy to decide if we need to write CFS or not which is bad in many ways. - we should call mergepolicy only during merges - we should never sync on IW during DWPT flush - we should be able to make the decision if we need to write CFS or not before flush, ie. we could write parts of the flush directly to CFS or even start writing stored fields directly. - in the NRT case it might make sense to write all flushes to CFS to minimize filedescriptors independent of the index size. I wonder if we can use a simple boolean for this in the IWC and get away with not consulting merge policy. This would simplify concurrency a lot here already. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Artifacts-trunk - Build # 2300 - Failure
Build: https://builds.apache.org/job/Lucene-Artifacts-trunk/2300/ No tests ran. Build Log: [...truncated 6102 lines...] BUILD FAILED /usr/home/hudson/hudson-slave/workspace/Lucene-Artifacts-trunk/lucene/build.xml:558: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Artifacts-trunk/lucene/common-build.xml:2107: java.net.ConnectException: Operation timed out at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) at sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:160) at sun.net.NetworkClient.doConnect(NetworkClient.java:180) at sun.net.www.http.HttpClient.openServer(HttpClient.java:378) at sun.net.www.http.HttpClient.openServer(HttpClient.java:473) at sun.net.www.protocol.https.HttpsClient.init(HttpsClient.java:270) at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:327) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:931) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177) at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153) at org.apache.tools.ant.taskdefs.Get$GetThread.openConnection(Get.java:660) at org.apache.tools.ant.taskdefs.Get$GetThread.get(Get.java:579) at org.apache.tools.ant.taskdefs.Get$GetThread.run(Get.java:569) Total time: 6 minutes 30 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Publishing Javadoc Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Solr field-collapsing should be able to use Lucene's BlockGroupingCollector
In Lucene it is possible to use the BlockGroupingCollector for grouping in order to take advantage of indexing document blocks ( IndexWriter.addDocuments()http://lucene.apache.org/core/4_3_0/core/org/apache/lucene/index/IndexWriter.html?is-external=true#addDocuments%28java.lang.Iterable%29 ). With SOLR-3076 and SOLR-3535, it is possible to index document blocks. I would like to have an option to use the BlockGroupingCollector with Solr field-collapsing/grouping.Should I open a JIRA issue, or is there something that would prevent using Lucene's BlockGroupingCollector with Solr? Tom
[jira] [Updated] (SOLR-4679) HTML line breaks (br) are removed during indexing; causes wrong search results
[ https://issues.apache.org/jira/browse/SOLR-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hoss Man updated SOLR-4679: --- Attachment: SOLR-4679__weird_TIKA-1134.patch patch includes a test demonstrating hte problem in Solr, and an example of how we could work around this in SolrContentHandler – but i don't think the workarround is a good idea ... not w/o a lot more careful thought about how all that extra ignorblae whitespace might affect people (not just from html docs, but from any other types of docs where Tika produces ignorable whitespace sax events) HTML line breaks (br) are removed during indexing; causes wrong search results Key: SOLR-4679 URL: https://issues.apache.org/jira/browse/SOLR-4679 Project: Solr Issue Type: Bug Components: update Affects Versions: 4.2 Environment: Windows Server 2008 R2, Java 6, Tomcat 7 Reporter: Christoph Straßer Attachments: external.htm, SOLR-4679__weird_TIKA-1134.patch, Solr_HtmlLineBreak_Linz_NotFound.png, Solr_HtmlLineBreak_Vienna.png HTML line breaks (br, BR, br/, ...) seem to be removed during extraction of content from HTML-Files. They need to be replaced with a empty space. Test-File: html head titleTest mit HTML-Zeilenschaltungen/title /head p word1brword2br/ Some other words, a special name like linzbrand another special name - vienna /p /html The Solr-content-attribute contains the following text: Test mit HTML-Zeilenschaltungen word1word2 Some other words, a special name like linzand another special name - vienna So we are not able to find the word linz. We use the ExtractingRequestHandler to put content into Solr. (wiki.apache.org/solr/ExtractingRequestHandler) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-5052) bitset codec for off heap filters
Mikhail Khludnev created LUCENE-5052: Summary: bitset codec for off heap filters Key: LUCENE-5052 URL: https://issues.apache.org/jira/browse/LUCENE-5052 Project: Lucene - Core Issue Type: New Feature Components: core/codecs Reporter: Mikhail Khludnev Fix For: 5.0 Colleagues, When we filter we don’t care any of scoring factors i.e. norms, positions, tf, but it should be fast. The obvious way to handle this is to decode postings list and cache it in heap (CachingWrappingFilter, Solr’s DocSet). Both of consuming a heap and decoding as well are expensive. Let’s write a posting list as a bitset, if df is greater than segment's maxdocs/8 (what about skiplists? and overall performance?). Beside of the codec implementation, the trickiest part to me is to design API for this. How we can let the app know that a term query don’t need to be cached in heap, but can be held as an mmaped bitset? WDYT? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-4919) Allow setting ResponseParser and RequestWriter on LBHttpSolrServer
Shawn Heisey created SOLR-4919: -- Summary: Allow setting ResponseParser and RequestWriter on LBHttpSolrServer Key: SOLR-4919 URL: https://issues.apache.org/jira/browse/SOLR-4919 Project: Solr Issue Type: Improvement Components: clients - java Affects Versions: 4.3 Reporter: Shawn Heisey Assignee: Shawn Heisey Priority: Minor Fix For: 5.0, 4.4 Patch to allow setting parser/writer on LBHttpSolrServer. Will only work if no server objects exist within. Part of larger issue SOLR-4715. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-4908) SolrContentHandler procuces glued words when extracting html
[
https://issues.apache.org/jira/browse/SOLR-4908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man resolved SOLR-4908.
Resolution: Duplicate
resolving as a Dup of SOLR-4679, but thank you so much for your investigation
into the root cause ... that really helps.
{quote}
The Tika XHTMLContentHandler issues ignoreableWhitspace events with a newline
in the character stream when a br tag is encountered.
The SolrContentHandler should be modified to add the ignorable whitespace to
the content.
{quote}
I don't believe this modification to SolrContentHandler would actually make
sense -- the fact that br tags in html *only* produce ignorableWhitespace
events in the resulting XHTML SAX stream seems like a bug in Tika, so i've
opened TIKA-1134 to try to get to the bottom of it.
SolrContentHandler procuces glued words when extracting html
Key: SOLR-4908
URL: https://issues.apache.org/jira/browse/SOLR-4908
Project: Solr
Issue Type: Bug
Components: contrib - Solr Cell (Tika extraction)
Affects Versions: 4.3
Environment: Windows 7, 64bit, Solr 4.3 example
Reporter: Markus Schuch
Attachments: tika-test.html
The SolrContentHandler produces glued words when extracting html
for html documents like:
{code}
htmlhead/headbodygluedbr/words/body/html
{code}
This was solved in Tika [TIKA-343] but the problem occurs when using the
extraction handler because the SolrContentHandler discards
ignoreableWhitespace.
The Tika XHTMLContentHandler issues ignoreableWhitspace events with a newline
in the character stream when a br tag is encountered.
The SolrContentHandler should be modified to add the ignorable whitespace to
the content.
Reproduction Steps:
# POST the html example file from the attachtments to
http://localhost:8983/solr/update/extract?literal.id=html-test-1commit=true
(e.g. with curl or HTTP Requester Plugin in Firefox)
# Query for the document
http://localhost:8983/solr/collection1/select?q=id%3A%22html-test-1%22fl=contentwt=xmlindent=true
# Look for the field content, which contains the word Shouldnotbeglued
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5050) CompressingStoredFieldsReader should close the index file as soon as it has been read
[
https://issues.apache.org/jira/browse/LUCENE-5050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680560#comment-13680560
]
Commit Tag Bot commented on LUCENE-5050:
[branch_4x commit] jpountz
http://svn.apache.org/viewvc?view=revisionrevision=1491909
LUCENE-5050: Close the stored fields and term vectors index files eagerly
(merged from r1491889).
CompressingStoredFieldsReader should close the index file as soon as it has
been read
-
Key: LUCENE-5050
URL: https://issues.apache.org/jira/browse/LUCENE-5050
Project: Lucene - Core
Issue Type: Improvement
Affects Versions: 4.3.1
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
Fix For: 5.0, 4.4
Attachments: LUCENE-5050.patch
Although CompressingStoredFieldsReader loads the stored fields index into
memory, it only closes the index file in {{close()}}. Closing at the end of
the constructor should help save file descriptors.
The same idea applies to CompressingTermVectorsReader.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-4919) Allow setting ResponseParser and RequestWriter on LBHttpSolrServer
[ https://issues.apache.org/jira/browse/SOLR-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Heisey updated SOLR-4919: --- Attachment: SOLR-4919.patch Attached patch. In addition to adding methods for setting parser/writer, it removes the unnecessary 'throws MalformedURLException'. I might need to leave that in for 4.x, would appreciate some advice there. Allow setting ResponseParser and RequestWriter on LBHttpSolrServer -- Key: SOLR-4919 URL: https://issues.apache.org/jira/browse/SOLR-4919 Project: Solr Issue Type: Improvement Components: clients - java Affects Versions: 4.3 Reporter: Shawn Heisey Assignee: Shawn Heisey Priority: Minor Fix For: 5.0, 4.4 Attachments: SOLR-4919.patch Patch to allow setting parser/writer on LBHttpSolrServer. Will only work if no server objects exist within. Part of larger issue SOLR-4715. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-5050) CompressingStoredFieldsReader should close the index file as soon as it has been read
[
https://issues.apache.org/jira/browse/LUCENE-5050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrien Grand resolved LUCENE-5050.
--
Resolution: Fixed
Thanks Robert.
CompressingStoredFieldsReader should close the index file as soon as it has
been read
-
Key: LUCENE-5050
URL: https://issues.apache.org/jira/browse/LUCENE-5050
Project: Lucene - Core
Issue Type: Improvement
Affects Versions: 4.3.1
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
Fix For: 5.0, 4.4
Attachments: LUCENE-5050.patch
Although CompressingStoredFieldsReader loads the stored fields index into
memory, it only closes the index file in {{close()}}. Closing at the end of
the constructor should help save file descriptors.
The same idea applies to CompressingTermVectorsReader.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2849) Solr maven dependencies: logging
[
https://issues.apache.org/jira/browse/SOLR-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680602#comment-13680602
]
Steve Rowe commented on SOLR-2849:
--
bq. So rather than force them to exclude slf4j-jdk14 (which is annoying), it
could be marked as optional.
I see {{ant dist}} under {{solr/}} still puts the {{slf4j-log4j12}} jar into
{{dist/solrj-lib/}}. As I've said previously on this issue, I think the Maven
config should mirror the Ant config: until we stop distributing the
{{slf4j-log4j12}} jar as a solrj dependency, I don't think we should mark it as
optional in the Maven config.
Solr maven dependencies: logging
Key: SOLR-2849
URL: https://issues.apache.org/jira/browse/SOLR-2849
Project: Solr
Issue Type: Improvement
Components: Build
Affects Versions: 4.0-ALPHA
Reporter: David Smiley
Assignee: Steve Rowe
Priority: Trivial
Fix For: 3.5, 4.0-ALPHA
Attachments: SOLR-2849_maven_dependencies.patch,
SOLR-2849_maven_dependencies.patch, SOLR-2849_maven_dependencies.patch,
SOLR-2849_maven_dependencies.patch, SOLR-2849_maven_dependencies.patch,
SOLR-2849_maven_dependencies.patch
I was looking at my maven based project's Solr-core dependencies (trunk), and
observed some issues that I think should be fixed in Solr's maven poms. I ran
{{mvn dependency:tree}} -- the output is further below. There are two
changes I see needed, related to logging:
* slf4j-jdk14 should be runtime scope, and optional.
* httpclient depends on commons-logging. Exclude this dependency from the
httpclient dependency, and add a dependency on jcl-over-slf4j with compile
scope.
* Zookeeper depends on Log4j, unfortunately. There is an issue to change this
to SLF4J: ZOOKEEPER-850. In the mean time we should exclude it and use
log4j-over-slf4j with compile scope, at the solrj pom.
As an aside, it's unfortunate to see all those velocity dependencies. It
even depends on struts -- seriously?! I hope solritas gets put back into a
contrib sometime: SOLR-2588
Steve, if you'd like to me to create the patch, I will.
{code}
[INFO] +- org.apache.solr:solr-core:jar:4.0-SNAPSHOT:compile
[INFO] | +- org.apache.solr:solr-solrj:jar:4.0-SNAPSHOT:compile
[INFO] | | \- org.apache.zookeeper:zookeeper:jar:3.3.3:compile
[INFO] | | +- log4j:log4j:jar:1.2.15:compile
[INFO] | | | \- javax.mail:mail:jar:1.4:compile
[INFO] | | | \- javax.activation:activation:jar:1.1:compile
[INFO] | | \- jline:jline:jar:0.9.94:compile
[INFO] | +- org.apache.solr:solr-noggit:jar:4.0-SNAPSHOT:compile
[INFO] | +-
org.apache.lucene:lucene-analyzers-phonetic:jar:4.0-SNAPSHOT:compile
[INFO] | +- org.apache.lucene:lucene-highlighter:jar:4.0-SNAPSHOT:compile
[INFO] | +- org.apache.lucene:lucene-memory:jar:4.0-SNAPSHOT:compile
[INFO] | +- org.apache.lucene:lucene-misc:jar:4.0-SNAPSHOT:compile
[INFO] | +- org.apache.lucene:lucene-queryparser:jar:4.0-SNAPSHOT:compile
[INFO] | | \- org.apache.lucene:lucene-sandbox:jar:4.0-SNAPSHOT:compile
[INFO] | | \- jakarta-regexp:jakarta-regexp:jar:1.4:compile
[INFO] | +- org.apache.lucene:lucene-spatial:jar:4.0-SNAPSHOT:compile
[INFO] | +- org.apache.lucene:lucene-suggest:jar:4.0-SNAPSHOT:compile
[INFO] | +- org.apache.lucene:lucene-grouping:jar:4.0-SNAPSHOT:compile
[INFO] | +- org.apache.solr:solr-commons-csv:jar:4.0-SNAPSHOT:compile
[INFO] | +- commons-codec:commons-codec:jar:1.4:compile
[INFO] | +- commons-fileupload:commons-fileupload:jar:1.2.1:compile
[INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] | | \- commons-logging:commons-logging:jar:1.0.4:compile
[INFO] | +- commons-io:commons-io:jar:1.4:compile
[INFO] | +- org.apache.velocity:velocity:jar:1.6.4:compile
[INFO] | | +- commons-collections:commons-collections:jar:3.2.1:compile
[INFO] | | \- oro:oro:jar:2.0.8:compile
[INFO] | +- org.apache.velocity:velocity-tools:jar:2.0:compile
[INFO] | | +- commons-beanutils:commons-beanutils:jar:1.7.0:compile
[INFO] | | +- commons-digester:commons-digester:jar:1.8:compile
[INFO] | | +- commons-chain:commons-chain:jar:1.1:compile
[INFO] | | +- commons-validator:commons-validator:jar:1.3.1:compile
[INFO] | | +- dom4j:dom4j:jar:1.1:compile
[INFO] | | +- sslext:sslext:jar:1.2-0:compile
[INFO] | | +- org.apache.struts:struts-core:jar:1.3.8:compile
[INFO] | | | \- antlr:antlr:jar:2.7.2:compile
[INFO] | | +- org.apache.struts:struts-taglib:jar:1.3.8:compile
[INFO] | | \- org.apache.struts:struts-tiles:jar:1.3.8:compile
[INFO] | +- org.slf4j:slf4j-jdk14:jar:1.6.1:compile
[INFO] | \- org.codehaus.woodstox:wstx-asl:jar:3.2.7:runtime
{code}
--
This message is automatically generated by JIRA.
If you think it was sent
[jira] [Commented] (LUCENE-5047) Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open()
[
https://issues.apache.org/jira/browse/LUCENE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680642#comment-13680642
]
Robert Muir commented on LUCENE-5047:
-
{quote}
A possible fix for now may be to try/catch for the new Exceptions in the 2
directory impls (there is also a try-with-resources there) and rethrow as FNFE.
{quote}
In my opinion this is wrong to do: the bug is things like NFS-special-cases in
lucene core that are catching these exceptions. Such code needs to be fixed to
also catch the nio2 exceptions!
Trunk's NIOFSDir and MMapDir throw different Exception on file not found
cases because of use of FileChannel.open()
---
Key: LUCENE-5047
URL: https://issues.apache.org/jira/browse/LUCENE-5047
Project: Lucene - Core
Issue Type: Bug
Components: core/store
Affects Versions: 5.0
Reporter: Uwe Schindler
Assignee: Uwe Schindler
Fix For: 5.0
In trunk, we use FileChannel.open() when opening files from NIOFSDir and
MMapDirectory. This new method (all new APIs in Java 7!) throw
java.nio.file.NoSuchFileException if a file does not exist. The old-style
FileNotFoundException is only thrown by old APIs.
We have to decide how to correctly document the expected behaviour in the
abstract Directory interface and how we fix this (Directory should clearly
state *which* IOException type is to be thrown if we really depend on the
specific type).
The new exceptions are more specific, so you get the most correct exception
(it may happen with old apis that you get FNFE if you open a file where you
have no access because it is locked...). With Java 7's FileChannel.open() you
get a separate and correct Exception.
A possible fix for now may be to try/catch for the new Exceptions in the 2
directory impls (there is also a try-with-resources there) and rethrow as
FNFE.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5047) Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open()
[ https://issues.apache.org/jira/browse/LUCENE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated LUCENE-5047: Attachment: LUCENE-5047.patch ok, i grepped and reviewed FileNotFoundException, i think the only issue is the NFS hack (in two places) Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open() --- Key: LUCENE-5047 URL: https://issues.apache.org/jira/browse/LUCENE-5047 Project: Lucene - Core Issue Type: Bug Components: core/store Affects Versions: 5.0 Reporter: Uwe Schindler Assignee: Uwe Schindler Fix For: 5.0 Attachments: LUCENE-5047.patch In trunk, we use FileChannel.open() when opening files from NIOFSDir and MMapDirectory. This new method (all new APIs in Java 7!) throw java.nio.file.NoSuchFileException if a file does not exist. The old-style FileNotFoundException is only thrown by old APIs. We have to decide how to correctly document the expected behaviour in the abstract Directory interface and how we fix this (Directory should clearly state *which* IOException type is to be thrown if we really depend on the specific type). The new exceptions are more specific, so you get the most correct exception (it may happen with old apis that you get FNFE if you open a file where you have no access because it is locked...). With Java 7's FileChannel.open() you get a separate and correct Exception. A possible fix for now may be to try/catch for the new Exceptions in the 2 directory impls (there is also a try-with-resources there) and rethrow as FNFE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5047) Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open()
[ https://issues.apache.org/jira/browse/LUCENE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680668#comment-13680668 ] Michael McCandless commented on LUCENE-5047: +1, patch looks good! Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open() --- Key: LUCENE-5047 URL: https://issues.apache.org/jira/browse/LUCENE-5047 Project: Lucene - Core Issue Type: Bug Components: core/store Affects Versions: 5.0 Reporter: Uwe Schindler Assignee: Uwe Schindler Fix For: 5.0 Attachments: LUCENE-5047.patch In trunk, we use FileChannel.open() when opening files from NIOFSDir and MMapDirectory. This new method (all new APIs in Java 7!) throw java.nio.file.NoSuchFileException if a file does not exist. The old-style FileNotFoundException is only thrown by old APIs. We have to decide how to correctly document the expected behaviour in the abstract Directory interface and how we fix this (Directory should clearly state *which* IOException type is to be thrown if we really depend on the specific type). The new exceptions are more specific, so you get the most correct exception (it may happen with old apis that you get FNFE if you open a file where you have no access because it is locked...). With Java 7's FileChannel.open() you get a separate and correct Exception. A possible fix for now may be to try/catch for the new Exceptions in the 2 directory impls (there is also a try-with-resources there) and rethrow as FNFE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5047) Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open()
[ https://issues.apache.org/jira/browse/LUCENE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680675#comment-13680675 ] Uwe Schindler commented on LUCENE-5047: --- Looks good. We should look for other fnfe outside of core. Did Shai already fix his test problem? Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open() --- Key: LUCENE-5047 URL: https://issues.apache.org/jira/browse/LUCENE-5047 Project: Lucene - Core Issue Type: Bug Components: core/store Affects Versions: 5.0 Reporter: Uwe Schindler Assignee: Uwe Schindler Fix For: 5.0 Attachments: LUCENE-5047.patch In trunk, we use FileChannel.open() when opening files from NIOFSDir and MMapDirectory. This new method (all new APIs in Java 7!) throw java.nio.file.NoSuchFileException if a file does not exist. The old-style FileNotFoundException is only thrown by old APIs. We have to decide how to correctly document the expected behaviour in the abstract Directory interface and how we fix this (Directory should clearly state *which* IOException type is to be thrown if we really depend on the specific type). The new exceptions are more specific, so you get the most correct exception (it may happen with old apis that you get FNFE if you open a file where you have no access because it is locked...). With Java 7's FileChannel.open() you get a separate and correct Exception. A possible fix for now may be to try/catch for the new Exceptions in the 2 directory impls (there is also a try-with-resources there) and rethrow as FNFE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-4919) Allow setting ResponseParser and RequestWriter on LBHttpSolrServer
[ https://issues.apache.org/jira/browse/SOLR-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Heisey updated SOLR-4919: --- Attachment: SOLR-4919.patch Updated patch that actually works. Tests pass on trunk. It does change some method signatures so they don't include throwing an exception, is that a bad thing to do for 4x? Allow setting ResponseParser and RequestWriter on LBHttpSolrServer -- Key: SOLR-4919 URL: https://issues.apache.org/jira/browse/SOLR-4919 Project: Solr Issue Type: Improvement Components: clients - java Affects Versions: 4.3 Reporter: Shawn Heisey Assignee: Shawn Heisey Priority: Minor Fix For: 5.0, 4.4 Attachments: SOLR-4919.patch, SOLR-4919.patch Patch to allow setting parser/writer on LBHttpSolrServer. Will only work if no server objects exist within. Part of larger issue SOLR-4715. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4816) Add document routing to CloudSolrServer
[ https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680682#comment-13680682 ] Shawn Heisey commented on SOLR-4816: Adding setters to LBHttpSolrServer ready to go in SOLR-4919. Add document routing to CloudSolrServer --- Key: SOLR-4816 URL: https://issues.apache.org/jira/browse/SOLR-4816 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.3 Reporter: Joel Bernstein Assignee: Mark Miller Priority: Minor Fix For: 5.0, 4.4 Attachments: SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816-sriesenberg.patch This issue adds the following enhancements to CloudSolrServer's update logic: 1) Document routing: Updates are routed directly to the correct shard leader eliminating document routing at the server. 2) Parallel update execution: Updates for each shard are executed in a separate thread so parallel indexing can occur across the cluster. 3) Javabin transport: Update requests are sent via javabin transport. These enhancements should allow for near linear scalability on indexing throughput. Usage: CloudSolrServer cloudClient = new CloudSolrServer(zkAddress); SolrInputDocument doc1 = new SolrInputDocument(); doc1.addField(id, 0); doc1.addField(a_t, hello1); SolrInputDocument doc2 = new SolrInputDocument(); doc2.addField(id, 2); doc2.addField(a_t, hello2); UpdateRequest request = new UpdateRequest(); request.add(doc1); request.add(doc2); request.setAction(AbstractUpdateRequest.ACTION.OPTIMIZE, false, false); NamedList response = cloudClient.request(request); // Returns a backwards compatible condensed response. //To get more detailed response down cast to RouteResponse: CloudSolrServer.RouteResponse rr = (CloudSolrServer.RouteResponse)response; NamedList responses = rr.getRouteResponse(); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5047) Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open()
[ https://issues.apache.org/jira/browse/LUCENE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680684#comment-13680684 ] Shai Erera commented on LUCENE-5047: Yes I did and committed to trunk only. Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open() --- Key: LUCENE-5047 URL: https://issues.apache.org/jira/browse/LUCENE-5047 Project: Lucene - Core Issue Type: Bug Components: core/store Affects Versions: 5.0 Reporter: Uwe Schindler Assignee: Uwe Schindler Fix For: 5.0 Attachments: LUCENE-5047.patch In trunk, we use FileChannel.open() when opening files from NIOFSDir and MMapDirectory. This new method (all new APIs in Java 7!) throw java.nio.file.NoSuchFileException if a file does not exist. The old-style FileNotFoundException is only thrown by old APIs. We have to decide how to correctly document the expected behaviour in the abstract Directory interface and how we fix this (Directory should clearly state *which* IOException type is to be thrown if we really depend on the specific type). The new exceptions are more specific, so you get the most correct exception (it may happen with old apis that you get FNFE if you open a file where you have no access because it is locked...). With Java 7's FileChannel.open() you get a separate and correct Exception. A possible fix for now may be to try/catch for the new Exceptions in the 2 directory impls (there is also a try-with-resources there) and rethrow as FNFE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4919) Allow setting ResponseParser and RequestWriter on LBHttpSolrServer
[ https://issues.apache.org/jira/browse/SOLR-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680685#comment-13680685 ] Mark Miller commented on SOLR-4919: --- We tend to be a bit more lax with the java apis than the http apis for back compat - especially since a lot of the java apis could still use improvement. Consider the affect of the change, is it worth the small hassle, and document well in CHANGES.txt. Allow setting ResponseParser and RequestWriter on LBHttpSolrServer -- Key: SOLR-4919 URL: https://issues.apache.org/jira/browse/SOLR-4919 Project: Solr Issue Type: Improvement Components: clients - java Affects Versions: 4.3 Reporter: Shawn Heisey Assignee: Shawn Heisey Priority: Minor Fix For: 5.0, 4.4 Attachments: SOLR-4919.patch, SOLR-4919.patch Patch to allow setting parser/writer on LBHttpSolrServer. Will only work if no server objects exist within. Part of larger issue SOLR-4715. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4792) stop shipping a war in 5.0
[
https://issues.apache.org/jira/browse/SOLR-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680690#comment-13680690
]
Steve Rowe commented on SOLR-4792:
--
{{ant validate-maven-dependencies}} is failing now (and has failed a couple of
Jenkins runs), because the {{-validate-maven-dependencies}} target in
{{solr/build.xml}} calls {{-validate-maven-dependencies}} in {{solr/webapp/}},
but there is no longer a {{pom.xml}} file for that dir, so the target, and the
build, fails.
When I add the following to {{solr/webapp/build.xml}}, the build succeeds:
{code:xml}
!-- nothing to do --
target name=-validate-maven-dependencies/
{code}
Committing shortly.
stop shipping a war in 5.0
--
Key: SOLR-4792
URL: https://issues.apache.org/jira/browse/SOLR-4792
Project: Solr
Issue Type: Task
Components: Build
Reporter: Robert Muir
Assignee: Robert Muir
Fix For: 5.0
Attachments: SOLR-4792.patch
see the vote on the developer list.
This is the first step: if we stop shipping a war then we are free to do
anything we want.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4919) Allow setting ResponseParser and RequestWriter on LBHttpSolrServer
[ https://issues.apache.org/jira/browse/SOLR-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680689#comment-13680689 ] Shawn Heisey commented on SOLR-4919: In a later issue I intend to deprecate the constructors on various SolrServer implementations that currently take a parser and replace them with constructors that take a parser and a writer, but that can be done after this issue goes in. Allow setting ResponseParser and RequestWriter on LBHttpSolrServer -- Key: SOLR-4919 URL: https://issues.apache.org/jira/browse/SOLR-4919 Project: Solr Issue Type: Improvement Components: clients - java Affects Versions: 4.3 Reporter: Shawn Heisey Assignee: Shawn Heisey Priority: Minor Fix For: 5.0, 4.4 Attachments: SOLR-4919.patch, SOLR-4919.patch Patch to allow setting parser/writer on LBHttpSolrServer. Will only work if no server objects exist within. Part of larger issue SOLR-4715. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-4919) Allow setting ResponseParser and RequestWriter on LBHttpSolrServer
[ https://issues.apache.org/jira/browse/SOLR-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680689#comment-13680689 ] Shawn Heisey edited comment on SOLR-4919 at 6/11/13 9:23 PM: - In a later issue I intend to deprecate the constructors on various SolrServer implementations that currently take a parser and replace them with constructors that take a parser and a writer, but that can be done after SOLR-4816 goes in. was (Author: elyograg): In a later issue I intend to deprecate the constructors on various SolrServer implementations that currently take a parser and replace them with constructors that take a parser and a writer, but that can be done after this issue goes in. Allow setting ResponseParser and RequestWriter on LBHttpSolrServer -- Key: SOLR-4919 URL: https://issues.apache.org/jira/browse/SOLR-4919 Project: Solr Issue Type: Improvement Components: clients - java Affects Versions: 4.3 Reporter: Shawn Heisey Assignee: Shawn Heisey Priority: Minor Fix For: 5.0, 4.4 Attachments: SOLR-4919.patch, SOLR-4919.patch Patch to allow setting parser/writer on LBHttpSolrServer. Will only work if no server objects exist within. Part of larger issue SOLR-4715. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4792) stop shipping a war in 5.0
[ https://issues.apache.org/jira/browse/SOLR-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680697#comment-13680697 ] Commit Tag Bot commented on SOLR-4792: -- [trunk commit] sarowe http://svn.apache.org/viewvc?view=revisionrevision=1491970 SOLR-4792: Skip target '-validate-maven-dependencies' in solr/webapp/ stop shipping a war in 5.0 -- Key: SOLR-4792 URL: https://issues.apache.org/jira/browse/SOLR-4792 Project: Solr Issue Type: Task Components: Build Reporter: Robert Muir Assignee: Robert Muir Fix For: 5.0 Attachments: SOLR-4792.patch see the vote on the developer list. This is the first step: if we stop shipping a war then we are free to do anything we want. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-4792) stop shipping a war in 5.0
[
https://issues.apache.org/jira/browse/SOLR-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680690#comment-13680690
]
Steve Rowe edited comment on SOLR-4792 at 6/11/13 9:29 PM:
---
{{ant validate-maven-dependencies}} is failing now (and has failed a couple of
Jenkins runs), because the {{-validate-maven-dependencies}} target in
{{solr/build.xml}} calls {{-validate-maven-dependencies}} in {{solr/webapp}},
but there is no longer a {{pom.xml}} file for that dir, so the target, and the
build, fails.
When I add the following to {{solr/webapp/build.xml}}, the build succeeds:
{code:xml}
!-- nothing to do --
target name=-validate-maven-dependencies/
{code}
Committing shortly.
was (Author: steve_rowe):
{{ant validate-maven-dependencies}} is failing now (and has failed a couple
of Jenkins runs), because the {{-validate-maven-dependencies}} target in
{{solr/build.xml}} calls {{-validate-maven-dependencies}} in {{solr/webapp/}},
but there is no longer a {{pom.xml}} file for that dir, so the target, and the
build, fails.
When I add the following to {{solr/webapp/build.xml}}, the build succeeds:
{code:xml}
!-- nothing to do --
target name=-validate-maven-dependencies/
{code}
Committing shortly.
stop shipping a war in 5.0
--
Key: SOLR-4792
URL: https://issues.apache.org/jira/browse/SOLR-4792
Project: Solr
Issue Type: Task
Components: Build
Reporter: Robert Muir
Assignee: Robert Muir
Fix For: 5.0
Attachments: SOLR-4792.patch
see the vote on the developer list.
This is the first step: if we stop shipping a war then we are free to do
anything we want.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-4919) Allow setting ResponseParser and RequestWriter on LBHttpSolrServer
[ https://issues.apache.org/jira/browse/SOLR-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Heisey updated SOLR-4919: --- Attachment: SOLR-4919.patch New patch. Included note on upgrading in CHANGES.txt. Allow setting ResponseParser and RequestWriter on LBHttpSolrServer -- Key: SOLR-4919 URL: https://issues.apache.org/jira/browse/SOLR-4919 Project: Solr Issue Type: Improvement Components: clients - java Affects Versions: 4.3 Reporter: Shawn Heisey Assignee: Shawn Heisey Priority: Minor Fix For: 5.0, 4.4 Attachments: SOLR-4919.patch, SOLR-4919.patch, SOLR-4919.patch Patch to allow setting parser/writer on LBHttpSolrServer. Will only work if no server objects exist within. Part of larger issue SOLR-4715. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-4919) Allow setting ResponseParser and RequestWriter on LBHttpSolrServer
[ https://issues.apache.org/jira/browse/SOLR-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Heisey updated SOLR-4919: --- Attachment: SOLR-4919.patch Left a word out of the new CHANGES.txt note in previous patch. Fixed in new patch. Allow setting ResponseParser and RequestWriter on LBHttpSolrServer -- Key: SOLR-4919 URL: https://issues.apache.org/jira/browse/SOLR-4919 Project: Solr Issue Type: Improvement Components: clients - java Affects Versions: 4.3 Reporter: Shawn Heisey Assignee: Shawn Heisey Priority: Minor Fix For: 5.0, 4.4 Attachments: SOLR-4919.patch, SOLR-4919.patch, SOLR-4919.patch, SOLR-4919.patch Patch to allow setting parser/writer on LBHttpSolrServer. Will only work if no server objects exist within. Part of larger issue SOLR-4715. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5047) Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open()
[ https://issues.apache.org/jira/browse/LUCENE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-5047: -- Attachment: LUCENE-5047.patch Reviewed the whole Lucene + Solr code. Some tests were catching only FNFE, I fixed those like replicator. I also made MockDirWrapper randomly choose one or the other exception if a missing file is to be emulated. Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open() --- Key: LUCENE-5047 URL: https://issues.apache.org/jira/browse/LUCENE-5047 Project: Lucene - Core Issue Type: Bug Components: core/store Affects Versions: 5.0 Reporter: Uwe Schindler Assignee: Uwe Schindler Fix For: 5.0 Attachments: LUCENE-5047.patch, LUCENE-5047.patch In trunk, we use FileChannel.open() when opening files from NIOFSDir and MMapDirectory. This new method (all new APIs in Java 7!) throw java.nio.file.NoSuchFileException if a file does not exist. The old-style FileNotFoundException is only thrown by old APIs. We have to decide how to correctly document the expected behaviour in the abstract Directory interface and how we fix this (Directory should clearly state *which* IOException type is to be thrown if we really depend on the specific type). The new exceptions are more specific, so you get the most correct exception (it may happen with old apis that you get FNFE if you open a file where you have no access because it is locked...). With Java 7's FileChannel.open() you get a separate and correct Exception. A possible fix for now may be to try/catch for the new Exceptions in the 2 directory impls (there is also a try-with-resources there) and rethrow as FNFE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3076) Solr should support block joins
[ https://issues.apache.org/jira/browse/SOLR-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Burton-West updated SOLR-3076: -- Attachment: SOLR-3076.patch Patch against trunk (SVN style, patch -p0) that adds testXML(), which illustrates XML block indexing syntax and exercises the XMLLoader Solr should support block joins --- Key: SOLR-3076 URL: https://issues.apache.org/jira/browse/SOLR-3076 Project: Solr Issue Type: New Feature Reporter: Grant Ingersoll Fix For: 5.0, 4.4 Attachments: 27M-singlesegment-histogram.png, 27M-singlesegment.png, bjq-vs-filters-backward-disi.patch, bjq-vs-filters-illegal-state.patch, child-bjqparser.patch, dih-3076.patch, dih-config.xml, parent-bjq-qparser.patch, parent-bjq-qparser.patch, Screen Shot 2012-07-17 at 1.12.11 AM.png, SOLR-3076-childDocs.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-7036-childDocs-solr-fork-trunk-patched, solrconf-bjq-erschema-snippet.xml, solrconfig.xml.patch, tochild-bjq-filtered-search-fix.patch Lucene has the ability to do block joins, we should add it to Solr. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5047) Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open()
[ https://issues.apache.org/jira/browse/LUCENE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680743#comment-13680743 ] Robert Muir commented on LUCENE-5047: - +1 to commit this, especially with the MockDir tests, I like it. Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open() --- Key: LUCENE-5047 URL: https://issues.apache.org/jira/browse/LUCENE-5047 Project: Lucene - Core Issue Type: Bug Components: core/store Affects Versions: 5.0 Reporter: Uwe Schindler Assignee: Uwe Schindler Fix For: 5.0 Attachments: LUCENE-5047.patch, LUCENE-5047.patch In trunk, we use FileChannel.open() when opening files from NIOFSDir and MMapDirectory. This new method (all new APIs in Java 7!) throw java.nio.file.NoSuchFileException if a file does not exist. The old-style FileNotFoundException is only thrown by old APIs. We have to decide how to correctly document the expected behaviour in the abstract Directory interface and how we fix this (Directory should clearly state *which* IOException type is to be thrown if we really depend on the specific type). The new exceptions are more specific, so you get the most correct exception (it may happen with old apis that you get FNFE if you open a file where you have no access because it is locked...). With Java 7's FileChannel.open() you get a separate and correct Exception. A possible fix for now may be to try/catch for the new Exceptions in the 2 directory impls (there is also a try-with-resources there) and rethrow as FNFE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5047) Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open()
[ https://issues.apache.org/jira/browse/LUCENE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-5047: -- Attachment: LUCENE-5047.patch Slightly improved javadocs: I stated clearly in Directory which Exceptions may be thrown. It was partly done for fileLength, but not for openInput/openSlicer. I will backport these changes to 4.x (the javadocs ones) and add a warning that custom dirs in 4x may only throw FNFE and not the java 7 one. Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open() --- Key: LUCENE-5047 URL: https://issues.apache.org/jira/browse/LUCENE-5047 Project: Lucene - Core Issue Type: Bug Components: core/store Affects Versions: 5.0 Reporter: Uwe Schindler Assignee: Uwe Schindler Fix For: 5.0 Attachments: LUCENE-5047.patch, LUCENE-5047.patch, LUCENE-5047.patch In trunk, we use FileChannel.open() when opening files from NIOFSDir and MMapDirectory. This new method (all new APIs in Java 7!) throw java.nio.file.NoSuchFileException if a file does not exist. The old-style FileNotFoundException is only thrown by old APIs. We have to decide how to correctly document the expected behaviour in the abstract Directory interface and how we fix this (Directory should clearly state *which* IOException type is to be thrown if we really depend on the specific type). The new exceptions are more specific, so you get the most correct exception (it may happen with old apis that you get FNFE if you open a file where you have no access because it is locked...). With Java 7's FileChannel.open() you get a separate and correct Exception. A possible fix for now may be to try/catch for the new Exceptions in the 2 directory impls (there is also a try-with-resources there) and rethrow as FNFE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5047) Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open()
[ https://issues.apache.org/jira/browse/LUCENE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680762#comment-13680762 ] Commit Tag Bot commented on LUCENE-5047: [trunk commit] uschindler http://svn.apache.org/viewvc?view=revisionrevision=1491992 LUCENE-5047: Handle NoSuchFileException of Java 7 like FileNotFoundException when opeining index files; document this in Directory. Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open() --- Key: LUCENE-5047 URL: https://issues.apache.org/jira/browse/LUCENE-5047 Project: Lucene - Core Issue Type: Bug Components: core/store Affects Versions: 5.0 Reporter: Uwe Schindler Assignee: Uwe Schindler Fix For: 5.0 Attachments: LUCENE-5047.patch, LUCENE-5047.patch, LUCENE-5047.patch In trunk, we use FileChannel.open() when opening files from NIOFSDir and MMapDirectory. This new method (all new APIs in Java 7!) throw java.nio.file.NoSuchFileException if a file does not exist. The old-style FileNotFoundException is only thrown by old APIs. We have to decide how to correctly document the expected behaviour in the abstract Directory interface and how we fix this (Directory should clearly state *which* IOException type is to be thrown if we really depend on the specific type). The new exceptions are more specific, so you get the most correct exception (it may happen with old apis that you get FNFE if you open a file where you have no access because it is locked...). With Java 7's FileChannel.open() you get a separate and correct Exception. A possible fix for now may be to try/catch for the new Exceptions in the 2 directory impls (there is also a try-with-resources there) and rethrow as FNFE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5047) Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open()
[ https://issues.apache.org/jira/browse/LUCENE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680767#comment-13680767 ] Commit Tag Bot commented on LUCENE-5047: [branch_4x commit] uschindler http://svn.apache.org/viewvc?view=revisionrevision=1491998 Backport Javadocs changes from: LUCENE-5047: Handle NoSuchFileException of Java 7 like FileNotFoundException when opeining index files; document this in Directory Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open() --- Key: LUCENE-5047 URL: https://issues.apache.org/jira/browse/LUCENE-5047 Project: Lucene - Core Issue Type: Bug Components: core/store Affects Versions: 5.0 Reporter: Uwe Schindler Assignee: Uwe Schindler Fix For: 5.0 Attachments: LUCENE-5047.patch, LUCENE-5047.patch, LUCENE-5047.patch In trunk, we use FileChannel.open() when opening files from NIOFSDir and MMapDirectory. This new method (all new APIs in Java 7!) throw java.nio.file.NoSuchFileException if a file does not exist. The old-style FileNotFoundException is only thrown by old APIs. We have to decide how to correctly document the expected behaviour in the abstract Directory interface and how we fix this (Directory should clearly state *which* IOException type is to be thrown if we really depend on the specific type). The new exceptions are more specific, so you get the most correct exception (it may happen with old apis that you get FNFE if you open a file where you have no access because it is locked...). With Java 7's FileChannel.open() you get a separate and correct Exception. A possible fix for now may be to try/catch for the new Exceptions in the 2 directory impls (there is also a try-with-resources there) and rethrow as FNFE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-5047) Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open()
[ https://issues.apache.org/jira/browse/LUCENE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler resolved LUCENE-5047. --- Resolution: Fixed Committed the latest patch and rewritten/backported the javadocs for 4.x Trunk's NIOFSDir and MMapDir throw different Exception on file not found cases because of use of FileChannel.open() --- Key: LUCENE-5047 URL: https://issues.apache.org/jira/browse/LUCENE-5047 Project: Lucene - Core Issue Type: Bug Components: core/store Affects Versions: 5.0 Reporter: Uwe Schindler Assignee: Uwe Schindler Fix For: 5.0 Attachments: LUCENE-5047.patch, LUCENE-5047.patch, LUCENE-5047.patch In trunk, we use FileChannel.open() when opening files from NIOFSDir and MMapDirectory. This new method (all new APIs in Java 7!) throw java.nio.file.NoSuchFileException if a file does not exist. The old-style FileNotFoundException is only thrown by old APIs. We have to decide how to correctly document the expected behaviour in the abstract Directory interface and how we fix this (Directory should clearly state *which* IOException type is to be thrown if we really depend on the specific type). The new exceptions are more specific, so you get the most correct exception (it may happen with old apis that you get FNFE if you open a file where you have no access because it is locked...). With Java 7's FileChannel.open() you get a separate and correct Exception. A possible fix for now may be to try/catch for the new Exceptions in the 2 directory impls (there is also a try-with-resources there) and rethrow as FNFE. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-4909) Solr and IndexReader Re-opening on Replication Slave
[ https://issues.apache.org/jira/browse/SOLR-4909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Garski updated SOLR-4909: - Attachment: SOLR-4909_fix.patch Attached a fix that corrects the issue. Here is an overview of the fix: Simply having the slave not have a writer instance is not feasible as then old commit points would continue to pile up on the slave, eventually exhausting disk space. The writer is required to delete the old commit points via the deletion policy, and the only way for the writer to be aware of the new commit points retrieved from the master is to open a new writer, and if the reader is reopened with a different writer instance the segment-level caches are lost. To change this behavior the reader is disconnected from the writer when explicitly configured to do so with a new parameter in the indexConfig section of the solr config named createReaderFromWriter which defaults to true to make the current behavior the default. If the value is explicitly set to false, which would normally only be done in a read-only slave, the reader is always initialized and re-opened from a directory instance and not a writer instance. There is logic in SolrCore.openNewSearcher to open a new reader rather than re-open should the underlying directory instance in the current reader not match that of the new index writer as that means that a full copy of the index was downloaded into a new directory, as would happen during replication if the slave's version was ahead of the master's. The patch was created on the lucene_solr_4_3 branch with all tests passing I can create versions for other branches if needed. Solr and IndexReader Re-opening on Replication Slave Key: SOLR-4909 URL: https://issues.apache.org/jira/browse/SOLR-4909 Project: Solr Issue Type: Improvement Components: replication (java), search Affects Versions: 4.3 Reporter: Michael Garski Fix For: 5.0, 4.4 Attachments: SOLR-4909-demo.patch, SOLR-4909_fix.patch I've been experimenting with caching filter data per segment in Solr using a CachingWrapperFilter FilteredQuery within a custom query parser (as suggested by [~yo...@apache.org] in SOLR-3763) and encountered situations where the value of getCoreCacheKey() on the AtomicReader for each segment can change for a given segment on disk when the searcher is reopened. As CachingWrapperFilter uses the value of the segment's getCoreCacheKey() as the key in the cache, there are situations where the data cached on that segment is not reused when the segment on disk is still part of the index. This affects the Lucene field cache and field value caches as well as they are cached per segment. When Solr first starts it opens the searcher's underlying DirectoryReader in StandardIndexReaderFactory.newReader by calling DirectoryReader.open(indexDir, termInfosIndexDivisor), and the reader is subsequently reopened in SolrCore.openNewSearcher by calling DirectoryReader.openIfChanged(currentReader, writer.get(), true). The act of reopening the reader with the writer when it was first opened without a writer results in the value of getCoreCacheKey() changing on each of the segments even though some of the segments have not changed. Depending on the role of the Solr server, this has different effects: * On a SolrCloud node or free-standing index and search server the segment cache is invalidated during the first DirectoryReader reopen - subsequent reopens use the same IndexWriter instance and as such the value of getCoreCacheKey() on each segment does not change so the cache is retained. * For a master-slave replication set up the segment cache invalidation occurs on the slave during every replication as the index is reopened using a new IndexWriter instance which results in the value of getCoreCacheKey() changing on each segment when the DirectoryReader is reopened using a different IndexWriter instance. I can think of a few approaches to alter the re-opening behavior to allow reuse of segment level caches in both cases, and I'd like to get some input on other ideas before digging in: * To change the cloud node/standalone first commit issue it might be possible to create the UpdateHandler and IndexWriter before the DirectoryReader, and use the writer to open the reader. There is a comment in the SolrCore constructor by [~yo...@apache.org] that the searcher should be opened before the update handler so that may not be an acceptable approach. * To change the behavior of a slave in a replication set up, one solution would be to not open a writer from the SnapPuller when the new index is retrieved if the core is enabled as a slave only. The writer is needed on a server
[jira] [Commented] (SOLR-4909) Solr and IndexReader Re-opening on Replication Slave
[ https://issues.apache.org/jira/browse/SOLR-4909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680821#comment-13680821 ] Mark Miller commented on SOLR-4909: --- bq. the reader is always initialized and re-opened from a directory instance and not a writer instance. Have to consider this carefully considering SOLR-4764 likely aims to drop opening from a directory at all. Solr and IndexReader Re-opening on Replication Slave Key: SOLR-4909 URL: https://issues.apache.org/jira/browse/SOLR-4909 Project: Solr Issue Type: Improvement Components: replication (java), search Affects Versions: 4.3 Reporter: Michael Garski Fix For: 5.0, 4.4 Attachments: SOLR-4909-demo.patch, SOLR-4909_fix.patch I've been experimenting with caching filter data per segment in Solr using a CachingWrapperFilter FilteredQuery within a custom query parser (as suggested by [~yo...@apache.org] in SOLR-3763) and encountered situations where the value of getCoreCacheKey() on the AtomicReader for each segment can change for a given segment on disk when the searcher is reopened. As CachingWrapperFilter uses the value of the segment's getCoreCacheKey() as the key in the cache, there are situations where the data cached on that segment is not reused when the segment on disk is still part of the index. This affects the Lucene field cache and field value caches as well as they are cached per segment. When Solr first starts it opens the searcher's underlying DirectoryReader in StandardIndexReaderFactory.newReader by calling DirectoryReader.open(indexDir, termInfosIndexDivisor), and the reader is subsequently reopened in SolrCore.openNewSearcher by calling DirectoryReader.openIfChanged(currentReader, writer.get(), true). The act of reopening the reader with the writer when it was first opened without a writer results in the value of getCoreCacheKey() changing on each of the segments even though some of the segments have not changed. Depending on the role of the Solr server, this has different effects: * On a SolrCloud node or free-standing index and search server the segment cache is invalidated during the first DirectoryReader reopen - subsequent reopens use the same IndexWriter instance and as such the value of getCoreCacheKey() on each segment does not change so the cache is retained. * For a master-slave replication set up the segment cache invalidation occurs on the slave during every replication as the index is reopened using a new IndexWriter instance which results in the value of getCoreCacheKey() changing on each segment when the DirectoryReader is reopened using a different IndexWriter instance. I can think of a few approaches to alter the re-opening behavior to allow reuse of segment level caches in both cases, and I'd like to get some input on other ideas before digging in: * To change the cloud node/standalone first commit issue it might be possible to create the UpdateHandler and IndexWriter before the DirectoryReader, and use the writer to open the reader. There is a comment in the SolrCore constructor by [~yo...@apache.org] that the searcher should be opened before the update handler so that may not be an acceptable approach. * To change the behavior of a slave in a replication set up, one solution would be to not open a writer from the SnapPuller when the new index is retrieved if the core is enabled as a slave only. The writer is needed on a server configured as a master slave that is functioning as a replication repeater so downstream slaves can see the changes in the index and retrieve them. I'll attach a unit test that demonstrates the behavior of reopening the DirectoryReader and it's effects on the value of getCoreCacheKey. My assumption is that the behavior of Lucene during the various reader reopen operations is correct and that the changes are necessary on the Solr side of things. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4909) Solr and IndexReader Re-opening on Replication Slave
[ https://issues.apache.org/jira/browse/SOLR-4909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680838#comment-13680838 ] Michael Garski commented on SOLR-4909: -- bq. SOLR-4764 likely aims to drop opening from a directory at all. Could SOLR-4764 use the same config logic to determine how to open/re-open the reader? Default behavior would be to open the reader from the writer (as necessary for NRT), but explicitly configured non-NRT instances would not open from the writer. Short of adding a way to re-open an index writer on a new commit point without resulting in dumping existing segment caches, I'm not sure how else the replication slave case can be addressed. Solr and IndexReader Re-opening on Replication Slave Key: SOLR-4909 URL: https://issues.apache.org/jira/browse/SOLR-4909 Project: Solr Issue Type: Improvement Components: replication (java), search Affects Versions: 4.3 Reporter: Michael Garski Fix For: 5.0, 4.4 Attachments: SOLR-4909-demo.patch, SOLR-4909_fix.patch I've been experimenting with caching filter data per segment in Solr using a CachingWrapperFilter FilteredQuery within a custom query parser (as suggested by [~yo...@apache.org] in SOLR-3763) and encountered situations where the value of getCoreCacheKey() on the AtomicReader for each segment can change for a given segment on disk when the searcher is reopened. As CachingWrapperFilter uses the value of the segment's getCoreCacheKey() as the key in the cache, there are situations where the data cached on that segment is not reused when the segment on disk is still part of the index. This affects the Lucene field cache and field value caches as well as they are cached per segment. When Solr first starts it opens the searcher's underlying DirectoryReader in StandardIndexReaderFactory.newReader by calling DirectoryReader.open(indexDir, termInfosIndexDivisor), and the reader is subsequently reopened in SolrCore.openNewSearcher by calling DirectoryReader.openIfChanged(currentReader, writer.get(), true). The act of reopening the reader with the writer when it was first opened without a writer results in the value of getCoreCacheKey() changing on each of the segments even though some of the segments have not changed. Depending on the role of the Solr server, this has different effects: * On a SolrCloud node or free-standing index and search server the segment cache is invalidated during the first DirectoryReader reopen - subsequent reopens use the same IndexWriter instance and as such the value of getCoreCacheKey() on each segment does not change so the cache is retained. * For a master-slave replication set up the segment cache invalidation occurs on the slave during every replication as the index is reopened using a new IndexWriter instance which results in the value of getCoreCacheKey() changing on each segment when the DirectoryReader is reopened using a different IndexWriter instance. I can think of a few approaches to alter the re-opening behavior to allow reuse of segment level caches in both cases, and I'd like to get some input on other ideas before digging in: * To change the cloud node/standalone first commit issue it might be possible to create the UpdateHandler and IndexWriter before the DirectoryReader, and use the writer to open the reader. There is a comment in the SolrCore constructor by [~yo...@apache.org] that the searcher should be opened before the update handler so that may not be an acceptable approach. * To change the behavior of a slave in a replication set up, one solution would be to not open a writer from the SnapPuller when the new index is retrieved if the core is enabled as a slave only. The writer is needed on a server configured as a master slave that is functioning as a replication repeater so downstream slaves can see the changes in the index and retrieve them. I'll attach a unit test that demonstrates the behavior of reopening the DirectoryReader and it's effects on the value of getCoreCacheKey. My assumption is that the behavior of Lucene during the various reader reopen operations is correct and that the changes are necessary on the Solr side of things. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4655) The Overseer should assign node names by default.
[ https://issues.apache.org/jira/browse/SOLR-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680840#comment-13680840 ] Mark Miller commented on SOLR-4655: --- Okay, I think something changed from 4.3 to 4.x that affects this - my other branch that had tests passing against 4.3 also has shard tests failing after updating to trunk. Don't know what it is yet, but some new information. I'll dig in and try and figure it out - this patch is pretty useful for SOLR-4916. The Overseer should assign node names by default. - Key: SOLR-4655 URL: https://issues.apache.org/jira/browse/SOLR-4655 Project: Solr Issue Type: Improvement Components: SolrCloud Reporter: Mark Miller Assignee: Mark Miller Fix For: 4.4 Attachments: SOLR-4655.patch, SOLR-4655.patch, SOLR-4655.patch, SOLR-4655.patch, SOLR-4655.patch, SOLR-4655.patch, SOLR-4655.patch, SOLR-4655.patch Currently we make a unique node name by using the host address as part of the name. This means that if you want a node with a new address to take over, the node name is misleading. It's best if you set custom names for each node before starting your cluster. This is cumbersome though, and cannot currently be done with the collections API. Instead, the overseer could assign a more generic name such as nodeN by default. Then you can easily swap in another node with no pre planning and no confusion in the name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4916) Add support to write and read Solr index files and transaction log files to and from HDFS.
[ https://issues.apache.org/jira/browse/SOLR-4916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680854#comment-13680854 ] Mark Miller commented on SOLR-4916: --- I'm working on getting a first rev for this in line with trunk and 4x. Patrick Hunt and I did the bulk of the implementation while Greg Chanan added the support for talking to a kerberos'd hdfs. We borrowed the HdfsDirectory from Apache Blur and contributed some code back to them. Initially, it's in Solr because that was easiest and satisfied my initial goals - however, someone that knows Lucene modules a bit better than me might want to take on moving it later on, with the idea that we might even collaborate with the Apache Blur guys on it in one location. I'm close to having an initial patch. I have to take care of 2 test fails that I think are related to the changes in SOLR-4655, and investigate a test fail in TestCloudManagedSchema. Add support to write and read Solr index files and transaction log files to and from HDFS. -- Key: SOLR-4916 URL: https://issues.apache.org/jira/browse/SOLR-4916 Project: Solr Issue Type: New Feature Reporter: Mark Miller Assignee: Mark Miller -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-4363) Inconsistent coreLoadThreads attributes in solr.xml between read/write
[
https://issues.apache.org/jira/browse/SOLR-4363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Erickson reassigned SOLR-4363:
Assignee: Erick Erickson
Inconsistent coreLoadThreads attributes in solr.xml between read/write
Key: SOLR-4363
URL: https://issues.apache.org/jira/browse/SOLR-4363
Project: Solr
Issue Type: Bug
Affects Versions: 4.1
Reporter: Patanachai Tangchaisin
Assignee: Erick Erickson
Priority: Minor
Solr is reading coreLoadThreads from an solr element in solr.xml
However, when persistent is enabled in solr.xml, Solr inserts
coreLoadThreads attribute to a wrong element.
Before start solr
{code}
solr persistent=true coreLoadThreads=2
cores host=localhost adminPath=/admin/cores hostPort=8983
hostContext=solr
.
/solr
{code}
After start solr
{code}
solr persistent=true
cores host=localhost adminPath=/admin/cores coreLoadThreads=2
hostPort=8080 hostContext=solr
.
/solr
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-3900) LogWatcher Config Not Persisted
[ https://issues.apache.org/jira/browse/SOLR-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson reassigned SOLR-3900: Assignee: Erick Erickson LogWatcher Config Not Persisted Key: SOLR-3900 URL: https://issues.apache.org/jira/browse/SOLR-3900 Project: Solr Issue Type: Bug Components: multicore Reporter: Michael Garski Assignee: Erick Erickson Priority: Minor Fix For: 4.4 When the solr.xml file is set to persistent=true, the logging element that contains the LogWatcher configuration is not persisted to the new solr.xml file that is written when managing the cores via core admin. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4919) Allow setting ResponseParser and RequestWriter on LBHttpSolrServer
[ https://issues.apache.org/jira/browse/SOLR-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680904#comment-13680904 ] Mark Miller commented on SOLR-4919: --- Doing a quick review of the latest patch... QQ: the CHANGES entry says CloudSolrServer and LBHttpSolrServer methods no longer claim to throw MalformedURLException. But, that exception is removed from the sig of LBHttpSolrServer#addSolrServer but is also added to the sig of LBHttpSolrServer#removeSolrServer. Shouldn't we keep it or remove it from *all* methods? Allow setting ResponseParser and RequestWriter on LBHttpSolrServer -- Key: SOLR-4919 URL: https://issues.apache.org/jira/browse/SOLR-4919 Project: Solr Issue Type: Improvement Components: clients - java Affects Versions: 4.3 Reporter: Shawn Heisey Assignee: Shawn Heisey Priority: Minor Fix For: 5.0, 4.4 Attachments: SOLR-4919.patch, SOLR-4919.patch, SOLR-4919.patch, SOLR-4919.patch Patch to allow setting parser/writer on LBHttpSolrServer. Will only work if no server objects exist within. Part of larger issue SOLR-4715. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4919) Allow setting ResponseParser and RequestWriter on LBHttpSolrServer
[ https://issues.apache.org/jira/browse/SOLR-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680907#comment-13680907 ] Mark Miller commented on SOLR-4919: --- Shouldn't we just require setting the parser/writer in the constructor? I'm not sure I like this setter methods that only work under certain conditions - it seems much cleaner to require those in the constructor. I think we should change that, but as it is, the access of aliveServers.size should really be wrapped in a synchronized(aliveServers) block. Allow setting ResponseParser and RequestWriter on LBHttpSolrServer -- Key: SOLR-4919 URL: https://issues.apache.org/jira/browse/SOLR-4919 Project: Solr Issue Type: Improvement Components: clients - java Affects Versions: 4.3 Reporter: Shawn Heisey Assignee: Shawn Heisey Priority: Minor Fix For: 5.0, 4.4 Attachments: SOLR-4919.patch, SOLR-4919.patch, SOLR-4919.patch, SOLR-4919.patch Patch to allow setting parser/writer on LBHttpSolrServer. Will only work if no server objects exist within. Part of larger issue SOLR-4715. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4919) Allow setting ResponseParser and RequestWriter on LBHttpSolrServer
[ https://issues.apache.org/jira/browse/SOLR-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680912#comment-13680912 ] Shawn Heisey commented on SOLR-4919: Thanks for the review. I do have a patch that changes the constructors, I was just trying to get this in so the patch for SOLR-4816 could be simplified. Allow setting ResponseParser and RequestWriter on LBHttpSolrServer -- Key: SOLR-4919 URL: https://issues.apache.org/jira/browse/SOLR-4919 Project: Solr Issue Type: Improvement Components: clients - java Affects Versions: 4.3 Reporter: Shawn Heisey Assignee: Shawn Heisey Priority: Minor Fix For: 5.0, 4.4 Attachments: SOLR-4919.patch, SOLR-4919.patch, SOLR-4919.patch, SOLR-4919.patch Patch to allow setting parser/writer on LBHttpSolrServer. Will only work if no server objects exist within. Part of larger issue SOLR-4715. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4919) Allow setting ResponseParser and RequestWriter on LBHttpSolrServer
[ https://issues.apache.org/jira/browse/SOLR-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680920#comment-13680920 ] Shawn Heisey commented on SOLR-4919: I see what I did wrong with removeSolrServer. I was a little zealous. :) I will also incorporate the new constructors. My original goal with SOLR-4715 was to mirror (as much as possible) what can be done now with HttpSolrServer. In pursuit of that goal, I'm inclined to keep the setters, but I could be talked out of it if it's really considered a bad idea. Allow setting ResponseParser and RequestWriter on LBHttpSolrServer -- Key: SOLR-4919 URL: https://issues.apache.org/jira/browse/SOLR-4919 Project: Solr Issue Type: Improvement Components: clients - java Affects Versions: 4.3 Reporter: Shawn Heisey Assignee: Shawn Heisey Priority: Minor Fix For: 5.0, 4.4 Attachments: SOLR-4919.patch, SOLR-4919.patch, SOLR-4919.patch, SOLR-4919.patch Patch to allow setting parser/writer on LBHttpSolrServer. Will only work if no server objects exist within. Part of larger issue SOLR-4715. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4224) refactor JavaBinCodec input stream definition to enhance reuse
[ https://issues.apache.org/jira/browse/SOLR-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680943#comment-13680943 ] Shalin Shekhar Mangar commented on SOLR-4224: - bq. Is this not fixed in trunk? It is. See the comment by the commit tag bot. refactor JavaBinCodec input stream definition to enhance reuse -- Key: SOLR-4224 URL: https://issues.apache.org/jira/browse/SOLR-4224 Project: Solr Issue Type: Improvement Components: clients - java Reporter: Patrick Hunt Assignee: Mark Miller Priority: Minor Fix For: 5.0, 4.4 Attachments: SOLR-4224.patch JavaBinCodec currently requires use of the concrete FastInputStream when unmarshalling a record. A JavaBinCodec API that takes an interface or abstract implementation would allow greater reuse. In my particular case I am trying to use JavaBinCodec to marshal/unmarshal from an data source that doesn't allow buffering. The semantics are such that I can read only a single record from the input source. The buffering in FastInputStream is reading information contained in the second record. No state other than the input data source itself is available to cache the FastInputStream between calls. As a result I'm losing the second record. I would like to provide an InputStream/DataInput that doesn't do any buffering. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jrockit-jdk1.6.0_37-R28.2.5-4.1.0) - Build # 6028 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6028/
Java: 32bit/jrockit-jdk1.6.0_37-R28.2.5-4.1.0 -XnoOpt
1 tests failed.
FAILED:
org.apache.lucene.spatial.prefix.SpatialOpRecursivePrefixTreeTest.testContains
{#0 seed=[FE6FCEAB450EDECB:9C74451EA3CA7D1F]}
Error Message:
Shouldn't match I #2:ShapePair(Rect(minX=30.0,maxX=57.0,minY=-13.0,maxY=8.0) ,
Rect(minX=242.0,maxX=252.0,minY=-92.0,maxY=14.0))
Q:Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)
Stack Trace:
java.lang.AssertionError: Shouldn't match I
#2:ShapePair(Rect(minX=30.0,maxX=57.0,minY=-13.0,maxY=8.0) ,
Rect(minX=242.0,maxX=252.0,minY=-92.0,maxY=14.0))
Q:Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)
at
__randomizedtesting.SeedInfo.seed([FE6FCEAB450EDECB:9C74451EA3CA7D1F]:0)
at org.junit.Assert.fail(Assert.java:93)
at
org.apache.lucene.spatial.prefix.SpatialOpRecursivePrefixTreeTest.fail(SpatialOpRecursivePrefixTreeTest.java:287)
at
org.apache.lucene.spatial.prefix.SpatialOpRecursivePrefixTreeTest.doTest(SpatialOpRecursivePrefixTreeTest.java:273)
at
org.apache.lucene.spatial.prefix.SpatialOpRecursivePrefixTreeTest.testContains(SpatialOpRecursivePrefixTreeTest.java:101)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:738)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:774)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:683)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:44)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
[JENKINS] Lucene-Solr-4.x-MacOSX (64bit/jdk1.7.0) - Build # 539 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-MacOSX/539/ Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseParallelGC All tests passed Build Log: [...truncated 327 lines...] [junit4:junit4] ERROR: JVM J0 ended with an exception, command line: /Library/Java/JavaVirtualMachines/jdk1.7.0_21.jdk/Contents/Home/jre/bin/java -XX:+UseCompressedOops -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/heapdumps -Dtests.prefix=tests -Dtests.seed=DA7D99534C182BC1 -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=4.4 -Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.slow=true -Dtests.asserts.gracious=false -Dtests.multiplier=1 -DtempDir=. -Djava.io.tmpdir=. -Djunit4.tempDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/build/core/test/temp -Dclover.db.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/build/clover/db -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Djava.security.policy=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/tools/junit4/tests.policy -Dlucene.version=4.4-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Dfile.encoding=ISO-8859-1 -classpath /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/build/test-framework/classes/java:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/build/codecs/classes/java:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/test-framework/lib/junit-4.10.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/test-framework/lib/randomizedtesting-runner-2.0.10.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/build/core/classes/java:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/build/core/classes/test:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/Users/jenkins/.ant/lib/ivy-2.3.0.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_21.jdk/Contents/Home/lib/tools.jar:/Users/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.10.jar -ea:org.apache.lucene... -ea:org.apache.solr... com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -flush -eventsfile
[jira] [Commented] (SOLR-4763) Performance issue when using group.facet=true
[
https://issues.apache.org/jira/browse/SOLR-4763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680951#comment-13680951
]
Miriam C commented on SOLR-4763:
This bug leaves no choice but to choose between facets and groups.
Is This going to be fixed?
Performance issue when using group.facet=true
-
Key: SOLR-4763
URL: https://issues.apache.org/jira/browse/SOLR-4763
Project: Solr
Issue Type: Bug
Affects Versions: 4.2
Reporter: Alexander Koval
I do not know whether this is bug or not. But calculating facets with
{{group.facet=true}} is too slow.
I have query that:
{code}
matches: 730597,
ngroups: 24024,
{code}
1. All queries with {{group.facet=true}}:
{code}
QTime: 5171
facet: {
time: 4716
{code}
2. Without {{group.facet}}:
* First query:
{code}
QTime: 3284
facet: {
time: 3104
{code}
* Next queries:
{code}
QTime: 230,
facet: {
time: 76
{code}
So I think with {{group.facet=true}} Solr doesn't use cache to calculate
facets.
Is it possible to improve performance of facets when {{group.facet=true}}?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
