[ANNOUNCE] Apache PyLucene 3.4.0
I am pleased to announce the availability of Apache PyLucene 3.4.0. Apache PyLucene, a subproject of Apache Lucene, is a Python extension for accessing Apache Lucene Core. Its goal is to allow you to use Lucene's text indexing and searching capabilities from Python. It is API compatible with the latest version of Lucene Core, 3.4.0. This release contains a number of bug fixes and improvements. Details can be found in the changes files: http://svn.apache.org/repos/asf/lucene/pylucene/tags/pylucene_3_4_0/CHANGES http://svn.apache.org/repos/asf/lucene/pylucene/trunk/jcc/CHANGES Apache PyLucene is available from the following download page: http://www.apache.org/dyn/closer.cgi/lucene/pylucene/pylucene-3.4.0-1-src.tar.gz When downloading from a mirror site, please remember to verify the downloads using signatures found on the Apache site: http://www.apache.org/dist/lucene/pylucene/KEYS For more information on Apache PyLucene, visit the project home page: http://lucene.apache.org/pylucene Andi..
[jira] [Updated] (SOLR-2780) Facet count problem : Multi-Select Faceting After grouping results
[
https://issues.apache.org/jira/browse/SOLR-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramzi Alqrainy updated SOLR-2780:
-
Component/s: (was: clients - php)
search
Facet count problem : Multi-Select Faceting After grouping results
---
Key: SOLR-2780
URL: https://issues.apache.org/jira/browse/SOLR-2780
Project: Solr
Issue Type: Bug
Components: search
Affects Versions: 4.0
Reporter: Ramzi Alqrainy
Priority: Critical
Attachments: SOLR-2780.patch
Dear All ,
Kindly note that I am using Solr 4.0 and Kindly note that group.truncate=true
calculates facet counts that based on the most relevant document of each
group matching the query.
But when I used Multi-Select Faceting [Tagging and excluding Filters] , the
solr can't calculate the facet after grouping the results and select multi
facet.
http://127.0.0.1:8983/solr/select/?facet=truesort=score+desc,+rate+desc,total_of_reviews+descfacet.limit=-1bf=sum%28product%28atan%28total_of_reviews%29,50%29,product%28rate,10%29%29^4group.field=place_idfacet.field={!ex%3Dce}cat_enfacet.field={!ex%3Dce}cat_arfacet.field={!ex%3Dir}iregionfacet.field={!ex%3Dir}region_enfacet.field={!ex%3Dir}region_arfacet.field={!ex%3Drr}rratefacet.field=place_statusfacet.field=theme_enfacet.field=icityfacet.field={!ex%3Dce}icatfacet.field={!ex%3Dsce}isubcatfacet.field={!ex%3Dsce}subcat_enfacet.field={!ex%3Dsce}subcat_arqt=/spellfq=place_status:1fq=icity:1fq=cat_en:%28%22Restaurants%22%29group.format=simplegroup.ngroups=truefacet.mincount=1qf=title_ar^24+title_en^24+cat_ar^10+cat_en^10++review^20hl.fl=reviewjson.nl=mapwt=jsondefType=edismaxrows=10spellcheck.accuracy=0.6start=0q=smartgroup.truncate=truegroup=trueindent=on
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-2780) Facet count problem : Multi-Select Faceting After grouping results
[
https://issues.apache.org/jira/browse/SOLR-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martijn van Groningen updated SOLR-2780:
Affects Version/s: 3.3
3.4
Fix Version/s: 4.0
3.5
Facet count problem : Multi-Select Faceting After grouping results
---
Key: SOLR-2780
URL: https://issues.apache.org/jira/browse/SOLR-2780
Project: Solr
Issue Type: Bug
Components: search
Affects Versions: 3.3, 3.4, 4.0
Reporter: Ramzi Alqrainy
Priority: Critical
Fix For: 3.5, 4.0
Attachments: SOLR-2780.patch
Dear All ,
Kindly note that I am using Solr 4.0 and Kindly note that group.truncate=true
calculates facet counts that based on the most relevant document of each
group matching the query.
But when I used Multi-Select Faceting [Tagging and excluding Filters] , the
solr can't calculate the facet after grouping the results and select multi
facet.
http://127.0.0.1:8983/solr/select/?facet=truesort=score+desc,+rate+desc,total_of_reviews+descfacet.limit=-1bf=sum%28product%28atan%28total_of_reviews%29,50%29,product%28rate,10%29%29^4group.field=place_idfacet.field={!ex%3Dce}cat_enfacet.field={!ex%3Dce}cat_arfacet.field={!ex%3Dir}iregionfacet.field={!ex%3Dir}region_enfacet.field={!ex%3Dir}region_arfacet.field={!ex%3Drr}rratefacet.field=place_statusfacet.field=theme_enfacet.field=icityfacet.field={!ex%3Dce}icatfacet.field={!ex%3Dsce}isubcatfacet.field={!ex%3Dsce}subcat_enfacet.field={!ex%3Dsce}subcat_arqt=/spellfq=place_status:1fq=icity:1fq=cat_en:%28%22Restaurants%22%29group.format=simplegroup.ngroups=truefacet.mincount=1qf=title_ar^24+title_en^24+cat_ar^10+cat_en^10++review^20hl.fl=reviewjson.nl=mapwt=jsondefType=edismaxrows=10spellcheck.accuracy=0.6start=0q=smartgroup.truncate=truegroup=trueindent=on
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1895) ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time
[
https://issues.apache.org/jira/browse/SOLR-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13107738#comment-13107738
]
Koji Sekiguchi commented on SOLR-1895:
--
Thank you for correcting my test, Karl!
I think I found a mismatch in baforeClass(). The comment says that s-d13 is:
{code}
// | share| document
// |--|--
// | allow | deny | allow | deny
// -+---+--+---+--
// s-d13| 1,2,3 | 1, 3 | |
// -+---+--+---+--
{code}
but the code is:
{code}
assertU(adoc(id, s-d13,
allow_token_document, token1,
allow_token_document, token2,
allow_token_document, token3,
deny_token_share, token1,
deny_token_share, token3));
{code}
Can you correct them?
ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search
time
--
Key: SOLR-1895
URL: https://issues.apache.org/jira/browse/SOLR-1895
Project: Solr
Issue Type: New Feature
Components: SearchComponents - other
Reporter: Karl Wright
Labels: document, security, solr
Fix For: 3.5, 4.0
Attachments: LCFSecurityFilter.java, LCFSecurityFilter.java,
LCFSecurityFilter.java, LCFSecurityFilter.java, SOLR-1895.patch,
SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch
I've written an LCF SearchComponent which filters returned results based on
access tokens provided by LCF's authority service. The component requires
you to configure the appropriate authority service URL base, e.g.:
!-- LCF document security enforcement component --
searchComponent name=lcfSecurity class=LCFSecurityFilter
str
name=AuthorityServiceBaseURLhttp://localhost:8080/lcf-authority-service/str
/searchComponent
Also required are the following schema.xml additions:
!-- Security fields --
field name=allow_token_document type=string indexed=true
stored=false multiValued=true/
field name=deny_token_document type=string indexed=true
stored=false multiValued=true/
field name=allow_token_share type=string indexed=true
stored=false multiValued=true/
field name=deny_token_share type=string indexed=true stored=false
multiValued=true/
Finally, to tie it into the standard request handler, it seems to need to run
last:
requestHandler name=standard class=solr.SearchHandler default=true
arr name=last-components
strlcfSecurity/str
/arr
...
I have not set a package for this code. Nor have I been able to get it
reviewed by someone as conversant with Solr as I would prefer. It is my
hope, however, that this module will become part of the standard Solr 1.5
suite of search components, since that would tie it in with LCF nicely.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2351) Allow the MoreLikeThis component to accept filters and use the already parsed query from previous stages (if applicable) as seed.
[ https://issues.apache.org/jira/browse/SOLR-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13107747#comment-13107747 ] Sameh Hamza commented on SOLR-2351: --- I've successfully applied that patch and compiled, could you please explain what files/folders should i upload to live server to apply and how can I test it. Many thanks Sameh Allow the MoreLikeThis component to accept filters and use the already parsed query from previous stages (if applicable) as seed. - Key: SOLR-2351 URL: https://issues.apache.org/jira/browse/SOLR-2351 Project: Solr Issue Type: Improvement Components: MoreLikeThis Reporter: Amit Nithian Priority: Minor Fix For: 3.5, 4.0 Attachments: mlt.patch Currently the MLT component doesn't accept filter queries specified on the URL which my application needed (I needed to restrict similar results by a lat/long bounding box). This patch also attempts to solve the issue of allowing the boost functions of the dismax to be used in the MLT component by using the query object created by the QueryComponent to OR with the query created by the MLT as part of the final query. In a blank dismax query with no query/phrase clauses, this works although a separate BF definition/parsing would be ideal. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-2782) Parameter inconsistency in SolrJ: ints versus longs
Parameter inconsistency in SolrJ: ints versus longs --- Key: SOLR-2782 URL: https://issues.apache.org/jira/browse/SOLR-2782 Project: Solr Issue Type: Bug Components: clients - java Affects Versions: 3.3 Reporter: Willis Blackburn Priority: Minor I've noticed that SolrDocumentsList uses long for fields like numFound and start. But the methods SolrQuery.setRows and setStart only accept ints, which leads to the possibility of a query being unable to request some of the documents found by a query. I realize that this is an edge case, since not many Lucene indexes will have 2 billion+ documents? But it is an inconsistency and requires that I at some point reduce a long to an int in code, which always feels wrong. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-trunk - Build # 10580 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/10580/ 1 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.update.AutoCommitTest Error Message: java.lang.AssertionError: directory of test was not closed, opened from: org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:33) Stack Trace: java.lang.RuntimeException: java.lang.AssertionError: directory of test was not closed, opened from: org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:33) at org.apache.lucene.util.LuceneTestCase.afterClassLuceneTestCaseJ4(LuceneTestCase.java:470) at org.apache.lucene.util.LuceneTestCase.checkResourcesAfterClass(LuceneTestCase.java:528) at org.apache.lucene.util.LuceneTestCase.afterClassLuceneTestCaseJ4(LuceneTestCase.java:438) Build Log (for compile errors): [...truncated 7902 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-3.x-java7 - Build # 474 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-3.x-java7/474/ 1 tests failed. REGRESSION: org.apache.solr.TestDistributedGrouping.testDistribSearch Error Message: java.lang.AssertionError: Some threads threw uncaught exceptions! Stack Trace: java.lang.RuntimeException: java.lang.AssertionError: Some threads threw uncaught exceptions! at org.apache.lucene.util.LuceneTestCase.tearDown(LuceneTestCase.java:574) at org.apache.solr.SolrTestCaseJ4.tearDown(SolrTestCaseJ4.java:96) at org.apache.solr.BaseDistributedSearchTestCase.tearDown(BaseDistributedSearchTestCase.java:160) at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:147) at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:50) at org.apache.lucene.util.LuceneTestCase.checkUncaughtExceptionsAfter(LuceneTestCase.java:602) at org.apache.lucene.util.LuceneTestCase.tearDown(LuceneTestCase.java:546) Build Log (for compile errors): [...truncated 14199 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-tests-only-3.x-java7 - Build # 474 - Failure
I'm not able to reproduce this failure locally with the following command: ant test -Dtestcase=TestDistributedGrouping -Dtestmethod=testDistribSearch -Dtests.seed=4f7ef1931134ac6:-6aad51ee2de9b078:1761e11b17577f5 -Dtests.multiplier=5 Someone else has an idea why this failure occurs? The test did pass yesterday. On 19 September 2011 14:58, Apache Jenkins Server jenk...@builds.apache.org wrote: Build: https://builds.apache.org/job/Lucene-Solr-tests-only-3.x-java7/474/ 1 tests failed. REGRESSION: org.apache.solr.TestDistributedGrouping.testDistribSearch Error Message: java.lang.AssertionError: Some threads threw uncaught exceptions! Stack Trace: java.lang.RuntimeException: java.lang.AssertionError: Some threads threw uncaught exceptions! at org.apache.lucene.util.LuceneTestCase.tearDown(LuceneTestCase.java:574) at org.apache.solr.SolrTestCaseJ4.tearDown(SolrTestCaseJ4.java:96) at org.apache.solr.BaseDistributedSearchTestCase.tearDown(BaseDistributedSearchTestCase.java:160) at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:147) at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:50) at org.apache.lucene.util.LuceneTestCase.checkUncaughtExceptionsAfter(LuceneTestCase.java:602) at org.apache.lucene.util.LuceneTestCase.tearDown(LuceneTestCase.java:546) Build Log (for compile errors): [...truncated 14199 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Met vriendelijke groet, Martijn van Groningen - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-1979) Create LanguageIdentifierUpdateProcessor
[ https://issues.apache.org/jira/browse/SOLR-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Høydahl updated SOLR-1979: -- Attachment: SOLR-1979.patch Some further improvements: * Default fallback language if none set is now to avoid nullpointer exception * All individually detected languages are now added to langsField array * More tests Create LanguageIdentifierUpdateProcessor Key: SOLR-1979 URL: https://issues.apache.org/jira/browse/SOLR-1979 Project: Solr Issue Type: New Feature Components: contrib - LangId, update Reporter: Jan Høydahl Assignee: Jan Høydahl Priority: Minor Labels: UpdateProcessor Fix For: 3.5, 4.0 Attachments: SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch Language identification from document fields, and mapping of field names to language-specific fields based on detected language. Wrap the Tika LanguageIdentifier in an UpdateProcessor. See user documentation at http://wiki.apache.org/solr/LanguageDetection -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-1979) Create LanguageIdentifierUpdateProcessor
[ https://issues.apache.org/jira/browse/SOLR-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Høydahl updated SOLR-1979: -- Attachment: SOLR-1979.patch Added link to Wiki in example update chain in solrconfig Create LanguageIdentifierUpdateProcessor Key: SOLR-1979 URL: https://issues.apache.org/jira/browse/SOLR-1979 Project: Solr Issue Type: New Feature Components: contrib - LangId, update Reporter: Jan Høydahl Assignee: Jan Høydahl Priority: Minor Labels: UpdateProcessor Fix For: 3.5, 4.0 Attachments: SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch Language identification from document fields, and mapping of field names to language-specific fields based on detected language. Wrap the Tika LanguageIdentifier in an UpdateProcessor. See user documentation at http://wiki.apache.org/solr/LanguageDetection -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2585) Context-Sensitive Spelling Suggestions Collations
[ https://issues.apache.org/jira/browse/SOLR-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13107880#comment-13107880 ] James Dyer commented on SOLR-2585: -- Robert, I was thinking of maybe eventually submitting some refactorings as a follow-up to this issue. But if you want, we could do some things first then come back to this. Here were my initial thoughts, none of which are very well-though-out at this point... 1. Maybe move FileBasedSpellChecker to Lucene for consistency (each spell checker in Solr refers to a Spell checker in Lucene). Also, this makes it available to Lucene users. 2. Perhaps SpellingOptions could somehow be deleted. 3. If the Lucene Spell Checkers all inherited a common interface and/or Abstract Class, all of the *SolrSpellChecker classes could probably be reduced to 1 class (or 1 parent class with just a few overrides here and there...) (I know you feel we're not ready for this, but we could annotate the Lucene parent (class and/or interface) like this for now @lucene.internal - external users should use the appropriate subclass directly / @lucene.experimental - this [class|interface] may change or be removed in a future version). 4. Clarify the code in SpellCheckComponent. Wasn't thinking about this now, but I do see where you're coming from, especially with the distributed code in finishStage. I think there is some code duplication between finishStage (distributed) and process (non-dist / 1st stage dist) that can maybe be eliminated. Probably some good code comments would help de-mystify this too. Maybe rename a method or two for additional clarity. 5. Now that you point out that instanceof check in finishStage, we probably should write a test case with DirectSpellChecker in a distributed environment. Possibly a revamped (set of) *SolrSpellChecker class(es) could eliminate the need for such checks? 6. I think SpellingParams should be for parameters the user can put in their query. I'm not sure you can do this with accuracy. This one should probably be somewhere else as this is a SearchComponent config param, not a request param. Maybe there are others like this. Context-Sensitive Spelling Suggestions Collations --- Key: SOLR-2585 URL: https://issues.apache.org/jira/browse/SOLR-2585 Project: Solr Issue Type: Improvement Components: spellchecker Affects Versions: 4.0 Reporter: James Dyer Priority: Minor Attachments: SOLR-2585.patch, SOLR-2585.patch, SOLR-2585.patch, SOLR-2585.patch, SOLR-2585.patch Solr currently cannot offer what I'm calling here a context-sensitive spelling suggestion. That is, if a user enters one or more words that have docFrequency 0, but nevertheless are misspelled, then no suggestions are offered. Currently, Solr will always consider a word correctly spelled if it is in the index and/or dictionary, regardless of context. This issue patch add support for context-sensitive spelling suggestions. See SpellCheckCollatorTest.testContextSensitiveCollate() for a the typical use case for this functionality. This tests both using IndexBasedSepllChecker and DirectSolrSpellChecker. Two new Spelling Parameters were added: - spellcheck.alternativeTermCount - The count of suggestions to return for each query term existing in the index and/or dictionary. Presumably, users will want fewer suggestions for words with docFrequency0. Also setting this value turns on context-sensitive spell suggestions. - spellcheck.maxResultsForSuggest - The maximum number of hits the request can return in order to both generate spelling suggestions and set the correctlySpelled element to false. For example, if this is set to 5 and the user's query returns 5 or fewer results, the spellchecker will report correctlySpelled=false and also offer suggestions (and collations if requested). Setting this greater than zero is useful for creating did-you-mean suggestions for queries that return a low number of hits. I have also included a test using shards. See additions to DistributedSpellCheckComponentTest. In Lucene, SpellChecker.java can already support this functionality (by passing a null IndexReader and field-name). The DirectSpellChecker, however, needs a minor enhancement. This gives the option to allow DirectSpellChecker to return suggestions for all query terms regardless of frequency. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3439) add checks/asserts if you search across a closed reader
[ https://issues.apache.org/jira/browse/LUCENE-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-3439: --- Attachment: LUCENE-3439.patch Patch, adding missing ensureOpens, adding volatile to IW's close/closing bools, and fixing the scary bug where we invoke del policy (and possibly delete file) on a closed writer. add checks/asserts if you search across a closed reader --- Key: LUCENE-3439 URL: https://issues.apache.org/jira/browse/LUCENE-3439 Project: Lucene - Java Issue Type: Bug Reporter: Robert Muir Attachments: LUCENE-3439.patch, LUCENE-3439_test.patch if you try to search across a closed reader (and/or searcher too), there are no checks, not even assertions statements. this results in crazy scary stacktraces deep inside places like FSTs/various term dictionary implementations etc. In some situations, depending on codec, you wont even get an error (i'm sure its fun when you try to retrieve the stored fields!) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (LUCENE-3439) add checks/asserts if you search across a closed reader
[ https://issues.apache.org/jira/browse/LUCENE-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless reassigned LUCENE-3439: -- Assignee: Michael McCandless add checks/asserts if you search across a closed reader --- Key: LUCENE-3439 URL: https://issues.apache.org/jira/browse/LUCENE-3439 Project: Lucene - Java Issue Type: Bug Reporter: Robert Muir Assignee: Michael McCandless Attachments: LUCENE-3439.patch, LUCENE-3439_test.patch if you try to search across a closed reader (and/or searcher too), there are no checks, not even assertions statements. this results in crazy scary stacktraces deep inside places like FSTs/various term dictionary implementations etc. In some situations, depending on codec, you wont even get an error (i'm sure its fun when you try to retrieve the stored fields!) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-2763) Extracting update request handler throws exception and returns 400 when zero-length file posted using multipart form post
[
https://issues.apache.org/jira/browse/SOLR-2763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Høydahl updated SOLR-2763:
--
Attachment: SOLR-2763.patch
Reproduced and - i think - fixed with attached patch. Please verify.
Extracting update request handler throws exception and returns 400 when
zero-length file posted using multipart form post
-
Key: SOLR-2763
URL: https://issues.apache.org/jira/browse/SOLR-2763
Project: Solr
Issue Type: Bug
Components: update
Affects Versions: 1.4.1, 3.1, 3.2, 3.3
Reporter: Karl Wright
Attachments: SOLR-2763.patch
When zero-length documents are posted to the extracting update request
handler, and the method used for posting is multipart form encoding, then you
get a 400 error returned and the following exception to stderr:
Sep 14, 2011 3:45:45 AM org.apache.solr.common.SolrException log
SEVERE: org.apache.solr.common.SolrException: missing content stream
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:50)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at
org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:238)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1360)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Sep 14, 2011 3:45:45 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update/extract params={id=123} status=400
QTime=300
Other ways of indexing zero-length data do not produce this error.
A curl command that will reproduce the problem easily is as follows:
curl -location -F id=123 -F file=@hello.txt
http://localhost:8983/solr/update/extract
... assuming hello.txt is a zero-length file.
This ticket is related to CONNECTORS-254.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-trunk - Build # 10586 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/10586/ 1 tests failed. REGRESSION: org.apache.solr.core.TestJmxIntegration.testJmxOnCoreReload Error Message: Number of registered MBeans is not the same as info registry size expected:52 but was:50 Stack Trace: junit.framework.AssertionFailedError: Number of registered MBeans is not the same as info registry size expected:52 but was:50 at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:148) at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:50) at org.apache.solr.core.TestJmxIntegration.testJmxOnCoreReload(TestJmxIntegration.java:134) at org.apache.lucene.util.LuceneTestCase$2$1.evaluate(LuceneTestCase.java:611) Build Log (for compile errors): [...truncated 7822 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2763) Extracting update request handler throws exception and returns 400 when zero-length file posted using multipart form post
[
https://issues.apache.org/jira/browse/SOLR-2763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108093#comment-13108093
]
Jan Høydahl commented on SOLR-2763:
---
And a more correct curl line to reproduce is:
{code}
curl -location -F literal.id=123 -F file=@empty.txt
http://localhost:8983/solr/update/extract;
{code}
This one fails before the patch and succeeds after, but you have to commit
afterwards to see the change of course..
Extracting update request handler throws exception and returns 400 when
zero-length file posted using multipart form post
-
Key: SOLR-2763
URL: https://issues.apache.org/jira/browse/SOLR-2763
Project: Solr
Issue Type: Bug
Components: update
Affects Versions: 1.4.1, 3.1, 3.2, 3.3
Reporter: Karl Wright
Attachments: SOLR-2763.patch
When zero-length documents are posted to the extracting update request
handler, and the method used for posting is multipart form encoding, then you
get a 400 error returned and the following exception to stderr:
Sep 14, 2011 3:45:45 AM org.apache.solr.common.SolrException log
SEVERE: org.apache.solr.common.SolrException: missing content stream
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:50)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at
org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:238)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1360)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Sep 14, 2011 3:45:45 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update/extract params={id=123} status=400
QTime=300
Other ways of indexing zero-length data do not produce this error.
A curl command that will reproduce the problem easily is as follows:
curl -location -F id=123 -F file=@hello.txt
http://localhost:8983/solr/update/extract
... assuming hello.txt is a zero-length file.
This ticket is related to CONNECTORS-254.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-2783) Please make DocSet extend Serializable
Please make DocSet extend Serializable
--
Key: SOLR-2783
URL: https://issues.apache.org/jira/browse/SOLR-2783
Project: Solr
Issue Type: Wish
Environment: Any.
Reporter: Sujit Pal
Priority: Minor
We have built a custom EHCache backed implementation of SolrCache that allows
us to spill over the cache to disk and have it persistent across Solr restarts.
To allow disk spillover we need the key and value of the cache to be
Serializable. So our SolrCache implementation signature is like this:
{code}
public class EhCacheSolrCache implements SolrCacheSerializable,Serializable {
...
}
{/code}
One of the things we are caching are DocSets (specifically BitDocSets).
Currently we are wrapping it into a Serializable class of our own:
{code}
public class SerializableBitDocSet extends BitDocSet implements Serializable {
private static final long serialVersionUID = 3723685897599896159L;
public SerializableBitDocSet() {
super();
}
public SerializableBitDocSet(OpenBitSet obs) {
super(obs);
}
{/code}
and when getting or putting into the cache, we convert to the Serializable
version using the deprecated method getBits().
{code}
SerializableBitDocSet docset = new
SerializableBitDocSet(newSearcher.getDocSet(q).getBits());
newSearcher.getCache(CACHED_FACETS_BITSETS_CACHENAME).put(facetQuery, docset);
{/code}
On the get, since a SerializableBitDocSet is a BitDocSet there is no extra work.
Two issues:
1) we are using a deprecated getBits() call to make the conversion - this is
likely to disappear in a later version.
2) the approach is hacky, since we have to potentially subclass other bitset
like structures to make them Serializable as well.
It would be nice if DocSet (the parent interface) can be made Serializable.
This would allow Solr users to use this as a cache value for disk-persistent
caches without any hackery.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-2783) Please make DocSet extend Serializable
[
https://issues.apache.org/jira/browse/SOLR-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sujit Pal updated SOLR-2783:
Description:
We have built a custom EHCache backed implementation of SolrCache that allows
us to spill over the cache to disk and have it persistent across Solr restarts.
To allow disk spillover we need the key and value of the cache to be
Serializable. So our SolrCache implementation signature is like this:
{code}
public class EhCacheSolrCache implements SolrCacheSerializable,Serializable {
...
}
{code}
One of the things we are caching are DocSets (specifically BitDocSets).
Currently we are wrapping it into a Serializable class of our own:
{code}
public class SerializableBitDocSet extends BitDocSet implements Serializable {
private static final long serialVersionUID = 3723685897599896159L;
public SerializableBitDocSet() {
super();
}
public SerializableBitDocSet(OpenBitSet obs) {
super(obs);
}
{code}
and when getting or putting into the cache, we convert to the Serializable
version using the deprecated method getBits().
{code}
SerializableBitDocSet docset = new
SerializableBitDocSet(newSearcher.getDocSet(q).getBits());
newSearcher.getCache(CACHED_FACETS_BITSETS_CACHENAME).put(facetQuery, docset);
{code}
On the get, since a SerializableBitDocSet is a BitDocSet there is no extra work.
Two issues:
1) we are using a deprecated getBits() call to make the conversion - this is
likely to disappear in a later version.
2) the approach is hacky, since we have to potentially subclass other bitset
like structures to make them Serializable as well.
It would be nice if DocSet (the parent interface) can be made Serializable.
This would allow Solr users to use this as a cache value for disk-persistent
caches without any hackery.
was:
We have built a custom EHCache backed implementation of SolrCache that allows
us to spill over the cache to disk and have it persistent across Solr restarts.
To allow disk spillover we need the key and value of the cache to be
Serializable. So our SolrCache implementation signature is like this:
{code}
public class EhCacheSolrCache implements SolrCacheSerializable,Serializable {
...
}
{/code}
One of the things we are caching are DocSets (specifically BitDocSets).
Currently we are wrapping it into a Serializable class of our own:
{code}
public class SerializableBitDocSet extends BitDocSet implements Serializable {
private static final long serialVersionUID = 3723685897599896159L;
public SerializableBitDocSet() {
super();
}
public SerializableBitDocSet(OpenBitSet obs) {
super(obs);
}
{/code}
and when getting or putting into the cache, we convert to the Serializable
version using the deprecated method getBits().
{code}
SerializableBitDocSet docset = new
SerializableBitDocSet(newSearcher.getDocSet(q).getBits());
newSearcher.getCache(CACHED_FACETS_BITSETS_CACHENAME).put(facetQuery, docset);
{/code}
On the get, since a SerializableBitDocSet is a BitDocSet there is no extra work.
Two issues:
1) we are using a deprecated getBits() call to make the conversion - this is
likely to disappear in a later version.
2) the approach is hacky, since we have to potentially subclass other bitset
like structures to make them Serializable as well.
It would be nice if DocSet (the parent interface) can be made Serializable.
This would allow Solr users to use this as a cache value for disk-persistent
caches without any hackery.
Please make DocSet extend Serializable
--
Key: SOLR-2783
URL: https://issues.apache.org/jira/browse/SOLR-2783
Project: Solr
Issue Type: Wish
Environment: Any.
Reporter: Sujit Pal
Priority: Minor
We have built a custom EHCache backed implementation of SolrCache that allows
us to spill over the cache to disk and have it persistent across Solr
restarts. To allow disk spillover we need the key and value of the cache to
be Serializable. So our SolrCache implementation signature is like this:
{code}
public class EhCacheSolrCache implements SolrCacheSerializable,Serializable
{
...
}
{code}
One of the things we are caching are DocSets (specifically BitDocSets).
Currently we are wrapping it into a Serializable class of our own:
{code}
public class SerializableBitDocSet extends BitDocSet implements Serializable {
private static final long serialVersionUID = 3723685897599896159L;
public SerializableBitDocSet() {
super();
}
public SerializableBitDocSet(OpenBitSet obs) {
super(obs);
}
{code}
and when getting or putting into the cache, we convert to the Serializable
version using the deprecated method getBits().
{code}
SerializableBitDocSet docset = new
SerializableBitDocSet(newSearcher.getDocSet(q).getBits());
[jira] [Commented] (SOLR-2763) Extracting update request handler throws exception and returns 400 when zero-length file posted using multipart form post
[
https://issues.apache.org/jira/browse/SOLR-2763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108204#comment-13108204
]
Koji Sekiguchi commented on SOLR-2763:
--
Looks good!
Extracting update request handler throws exception and returns 400 when
zero-length file posted using multipart form post
-
Key: SOLR-2763
URL: https://issues.apache.org/jira/browse/SOLR-2763
Project: Solr
Issue Type: Bug
Components: update
Affects Versions: 1.4.1, 3.1, 3.2, 3.3
Reporter: Karl Wright
Attachments: SOLR-2763.patch
When zero-length documents are posted to the extracting update request
handler, and the method used for posting is multipart form encoding, then you
get a 400 error returned and the following exception to stderr:
Sep 14, 2011 3:45:45 AM org.apache.solr.common.SolrException log
SEVERE: org.apache.solr.common.SolrException: missing content stream
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:50)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at
org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:238)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1360)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Sep 14, 2011 3:45:45 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update/extract params={id=123} status=400
QTime=300
Other ways of indexing zero-length data do not produce this error.
A curl command that will reproduce the problem easily is as follows:
curl -location -F id=123 -F file=@hello.txt
http://localhost:8983/solr/update/extract
... assuming hello.txt is a zero-length file.
This ticket is related to CONNECTORS-254.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2778) Revise distributed code inside SearchComponents
[ https://issues.apache.org/jira/browse/SOLR-2778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108316#comment-13108316 ] Jason Rutherglen commented on SOLR-2778: Sweet-ness.com! Revise distributed code inside SearchComponents --- Key: SOLR-2778 URL: https://issues.apache.org/jira/browse/SOLR-2778 Project: Solr Issue Type: Improvement Reporter: Martijn van Groningen The distributed code inside search components such as QueryComponent and FacetComponent is complex. By structuring responsibilities the code becomes less complex and easier to understand. There is already a start for this that was part of distributed grouping (SOLR-2066). The following concepts were developed inside QueryComponent for SOLR-2066: * ShardRequestFactory is responsible for creating requests to shards in the cluster based on the incoming request from the client. * ShardResultTransformer. Transforming a NamedList response from the client in for example SearchGroup or TopDocs instance. * ShardResponseProcessor. Basically merges the shard responses. The ShardReponseProcessor uses a ShardResultTransformer to transform the shard response into a native structure (SearchGroup / TopGroups). These concepts are now only used for distributed grouping, but I think can also be used for non grouped distributed search. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-1895) ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time
[ https://issues.apache.org/jira/browse/SOLR-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Sekiguchi updated SOLR-1895: - Attachment: SOLR-1895.patch I changed doc ids in test code (my intention in the first patch was that id implies its permissions, and now permissions have been changed, so they should be modified). Also I added allow/deny fields (commented out) in example schema.xml. I think this is ready to go! ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time -- Key: SOLR-1895 URL: https://issues.apache.org/jira/browse/SOLR-1895 Project: Solr Issue Type: New Feature Components: SearchComponents - other Reporter: Karl Wright Labels: document, security, solr Fix For: 3.5, 4.0 Attachments: LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch I've written an LCF SearchComponent which filters returned results based on access tokens provided by LCF's authority service. The component requires you to configure the appropriate authority service URL base, e.g.: !-- LCF document security enforcement component -- searchComponent name=lcfSecurity class=LCFSecurityFilter str name=AuthorityServiceBaseURLhttp://localhost:8080/lcf-authority-service/str /searchComponent Also required are the following schema.xml additions: !-- Security fields -- field name=allow_token_document type=string indexed=true stored=false multiValued=true/ field name=deny_token_document type=string indexed=true stored=false multiValued=true/ field name=allow_token_share type=string indexed=true stored=false multiValued=true/ field name=deny_token_share type=string indexed=true stored=false multiValued=true/ Finally, to tie it into the standard request handler, it seems to need to run last: requestHandler name=standard class=solr.SearchHandler default=true arr name=last-components strlcfSecurity/str /arr ... I have not set a package for this code. Nor have I been able to get it reviewed by someone as conversant with Solr as I would prefer. It is my hope, however, that this module will become part of the standard Solr 1.5 suite of search components, since that would tie it in with LCF nicely. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1895) ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time
[ https://issues.apache.org/jira/browse/SOLR-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108320#comment-13108320 ] Koji Sekiguchi commented on SOLR-1895: -- bq. Is there really anything specific to ManifoldCF here? Perhaps this could just be AccessTokenSecurityFilter extends SearchComponent? I think so? I think it is specific MCF and allow/deny token security model provided by AD/Windows. ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time -- Key: SOLR-1895 URL: https://issues.apache.org/jira/browse/SOLR-1895 Project: Solr Issue Type: New Feature Components: SearchComponents - other Reporter: Karl Wright Labels: document, security, solr Fix For: 3.5, 4.0 Attachments: LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch I've written an LCF SearchComponent which filters returned results based on access tokens provided by LCF's authority service. The component requires you to configure the appropriate authority service URL base, e.g.: !-- LCF document security enforcement component -- searchComponent name=lcfSecurity class=LCFSecurityFilter str name=AuthorityServiceBaseURLhttp://localhost:8080/lcf-authority-service/str /searchComponent Also required are the following schema.xml additions: !-- Security fields -- field name=allow_token_document type=string indexed=true stored=false multiValued=true/ field name=deny_token_document type=string indexed=true stored=false multiValued=true/ field name=allow_token_share type=string indexed=true stored=false multiValued=true/ field name=deny_token_share type=string indexed=true stored=false multiValued=true/ Finally, to tie it into the standard request handler, it seems to need to run last: requestHandler name=standard class=solr.SearchHandler default=true arr name=last-components strlcfSecurity/str /arr ... I have not set a package for this code. Nor have I been able to get it reviewed by someone as conversant with Solr as I would prefer. It is my hope, however, that this module will become part of the standard Solr 1.5 suite of search components, since that would tie it in with LCF nicely. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1895) ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time
[ https://issues.apache.org/jira/browse/SOLR-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108330#comment-13108330 ] Chris Male commented on SOLR-1895: -- bq. I think this is ready to go! I think we can tidy this up further. - Lets dump the constructor since it just calls super() - Can we refactor the default manifold URL to a constant? - Same with the default timeout period - Some LOG.info calls are commented out, lets just delete them. If someone needs them, they can add them in themselves. - Is the performance of using BooleanFilter consisting of QueryWrapperFilters and WildcardQueries, really better than just having a BQ? Having fewer levels of indirection when the Queries are executed seems beneficial. - Lets dump the process(ResponseBuilder) override, it does nothing. - As I earlier commented, can we have a 1st class notion of a SecurityToken? Having just Strings today seems limited bq. I think so? I think it is specific MCF and allow/deny token security model provided by AD/Windows. I don't really see anything specific to MCF here, apart from the URL. I agree it defines a certain security model but by overriding getAccessTokens, I could source the tokens from anywhere. I could have a plaintext file in my solr installation where I read them from. ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time -- Key: SOLR-1895 URL: https://issues.apache.org/jira/browse/SOLR-1895 Project: Solr Issue Type: New Feature Components: SearchComponents - other Reporter: Karl Wright Labels: document, security, solr Fix For: 3.5, 4.0 Attachments: LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch I've written an LCF SearchComponent which filters returned results based on access tokens provided by LCF's authority service. The component requires you to configure the appropriate authority service URL base, e.g.: !-- LCF document security enforcement component -- searchComponent name=lcfSecurity class=LCFSecurityFilter str name=AuthorityServiceBaseURLhttp://localhost:8080/lcf-authority-service/str /searchComponent Also required are the following schema.xml additions: !-- Security fields -- field name=allow_token_document type=string indexed=true stored=false multiValued=true/ field name=deny_token_document type=string indexed=true stored=false multiValued=true/ field name=allow_token_share type=string indexed=true stored=false multiValued=true/ field name=deny_token_share type=string indexed=true stored=false multiValued=true/ Finally, to tie it into the standard request handler, it seems to need to run last: requestHandler name=standard class=solr.SearchHandler default=true arr name=last-components strlcfSecurity/str /arr ... I have not set a package for this code. Nor have I been able to get it reviewed by someone as conversant with Solr as I would prefer. It is my hope, however, that this module will become part of the standard Solr 1.5 suite of search components, since that would tie it in with LCF nicely. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1895) ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time
[ https://issues.apache.org/jira/browse/SOLR-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108334#comment-13108334 ] Erik Hatcher commented on SOLR-1895: I've just glanced at the patch, so forgive me if I'm off-base, but the examples of solrconfig here show the query component coming *before* the mcf component. Is that right? Shouldn't mcf come first to set the constraints for the query component's work? Also, what about using a PostFilter for these numerous wildcard queries, so that they are evaluated only on docs that match the rest of the query constraints? I'm a little weary of adding the MCF dependency to Solr core though (yes, I know it doesn't require MCF for compilation or run-time, but depends on MCF's security scheme). What about MCF maintaining this filter as a Solr plugin rather than it going into the core of Solr? ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time -- Key: SOLR-1895 URL: https://issues.apache.org/jira/browse/SOLR-1895 Project: Solr Issue Type: New Feature Components: SearchComponents - other Reporter: Karl Wright Labels: document, security, solr Fix For: 3.5, 4.0 Attachments: LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch I've written an LCF SearchComponent which filters returned results based on access tokens provided by LCF's authority service. The component requires you to configure the appropriate authority service URL base, e.g.: !-- LCF document security enforcement component -- searchComponent name=lcfSecurity class=LCFSecurityFilter str name=AuthorityServiceBaseURLhttp://localhost:8080/lcf-authority-service/str /searchComponent Also required are the following schema.xml additions: !-- Security fields -- field name=allow_token_document type=string indexed=true stored=false multiValued=true/ field name=deny_token_document type=string indexed=true stored=false multiValued=true/ field name=allow_token_share type=string indexed=true stored=false multiValued=true/ field name=deny_token_share type=string indexed=true stored=false multiValued=true/ Finally, to tie it into the standard request handler, it seems to need to run last: requestHandler name=standard class=solr.SearchHandler default=true arr name=last-components strlcfSecurity/str /arr ... I have not set a package for this code. Nor have I been able to get it reviewed by someone as conversant with Solr as I would prefer. It is my hope, however, that this module will become part of the standard Solr 1.5 suite of search components, since that would tie it in with LCF nicely. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1895) ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time
[ https://issues.apache.org/jira/browse/SOLR-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108340#comment-13108340 ] Koji Sekiguchi commented on SOLR-1895: -- Thank you for reviewing the patch, Chris and Male! I'll update the patch to incorporate some of your comment. For now: bq. the examples of solrconfig here show the query component coming before the mcf component. Is that right? Shouldn't mcf come first to set the constraints for the query component's work? As the security filter works at prepare phase, this is right. bq. I'm a little weary of adding the MCF dependency to Solr core though (yes, I know it doesn't require MCF for compilation or run-time, but depends on MCF's security scheme). I agree, so I placed it in contrib/auth at first time. bq. What about MCF maintaining this filter as a Solr plugin rather than it going into the core of Solr? I'd like to hear about it from Karl. :) ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time -- Key: SOLR-1895 URL: https://issues.apache.org/jira/browse/SOLR-1895 Project: Solr Issue Type: New Feature Components: SearchComponents - other Reporter: Karl Wright Labels: document, security, solr Fix For: 3.5, 4.0 Attachments: LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch I've written an LCF SearchComponent which filters returned results based on access tokens provided by LCF's authority service. The component requires you to configure the appropriate authority service URL base, e.g.: !-- LCF document security enforcement component -- searchComponent name=lcfSecurity class=LCFSecurityFilter str name=AuthorityServiceBaseURLhttp://localhost:8080/lcf-authority-service/str /searchComponent Also required are the following schema.xml additions: !-- Security fields -- field name=allow_token_document type=string indexed=true stored=false multiValued=true/ field name=deny_token_document type=string indexed=true stored=false multiValued=true/ field name=allow_token_share type=string indexed=true stored=false multiValued=true/ field name=deny_token_share type=string indexed=true stored=false multiValued=true/ Finally, to tie it into the standard request handler, it seems to need to run last: requestHandler name=standard class=solr.SearchHandler default=true arr name=last-components strlcfSecurity/str /arr ... I have not set a package for this code. Nor have I been able to get it reviewed by someone as conversant with Solr as I would prefer. It is my hope, however, that this module will become part of the standard Solr 1.5 suite of search components, since that would tie it in with LCF nicely. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Issue Comment Edited] (SOLR-1895) ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time
[ https://issues.apache.org/jira/browse/SOLR-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108340#comment-13108340 ] Koji Sekiguchi edited comment on SOLR-1895 at 9/20/11 3:02 AM: --- Thank you for reviewing the patch, Chris and Erik! I'll update the patch to incorporate some of your comment. For now: bq. the examples of solrconfig here show the query component coming before the mcf component. Is that right? Shouldn't mcf come first to set the constraints for the query component's work? As the security filter works at prepare phase, this is right. bq. I'm a little weary of adding the MCF dependency to Solr core though (yes, I know it doesn't require MCF for compilation or run-time, but depends on MCF's security scheme). I agree, so I placed it in contrib/auth at first time. bq. What about MCF maintaining this filter as a Solr plugin rather than it going into the core of Solr? I'd like to hear about it from Karl. :) was (Author: koji): Thank you for reviewing the patch, Chris and Male! I'll update the patch to incorporate some of your comment. For now: bq. the examples of solrconfig here show the query component coming before the mcf component. Is that right? Shouldn't mcf come first to set the constraints for the query component's work? As the security filter works at prepare phase, this is right. bq. I'm a little weary of adding the MCF dependency to Solr core though (yes, I know it doesn't require MCF for compilation or run-time, but depends on MCF's security scheme). I agree, so I placed it in contrib/auth at first time. bq. What about MCF maintaining this filter as a Solr plugin rather than it going into the core of Solr? I'd like to hear about it from Karl. :) ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time -- Key: SOLR-1895 URL: https://issues.apache.org/jira/browse/SOLR-1895 Project: Solr Issue Type: New Feature Components: SearchComponents - other Reporter: Karl Wright Labels: document, security, solr Fix For: 3.5, 4.0 Attachments: LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch I've written an LCF SearchComponent which filters returned results based on access tokens provided by LCF's authority service. The component requires you to configure the appropriate authority service URL base, e.g.: !-- LCF document security enforcement component -- searchComponent name=lcfSecurity class=LCFSecurityFilter str name=AuthorityServiceBaseURLhttp://localhost:8080/lcf-authority-service/str /searchComponent Also required are the following schema.xml additions: !-- Security fields -- field name=allow_token_document type=string indexed=true stored=false multiValued=true/ field name=deny_token_document type=string indexed=true stored=false multiValued=true/ field name=allow_token_share type=string indexed=true stored=false multiValued=true/ field name=deny_token_share type=string indexed=true stored=false multiValued=true/ Finally, to tie it into the standard request handler, it seems to need to run last: requestHandler name=standard class=solr.SearchHandler default=true arr name=last-components strlcfSecurity/str /arr ... I have not set a package for this code. Nor have I been able to get it reviewed by someone as conversant with Solr as I would prefer. It is my hope, however, that this module will become part of the standard Solr 1.5 suite of search components, since that would tie it in with LCF nicely. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1895) ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time
[ https://issues.apache.org/jira/browse/SOLR-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108345#comment-13108345 ] Erik Hatcher commented on SOLR-1895: Koji - thanks for pointing out the prepare phase use. It's very odd, to me, to see it that way though. Is there a technical reason this needs to be done in the prepare phase rather than in the process phase? ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time -- Key: SOLR-1895 URL: https://issues.apache.org/jira/browse/SOLR-1895 Project: Solr Issue Type: New Feature Components: SearchComponents - other Reporter: Karl Wright Labels: document, security, solr Fix For: 3.5, 4.0 Attachments: LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch I've written an LCF SearchComponent which filters returned results based on access tokens provided by LCF's authority service. The component requires you to configure the appropriate authority service URL base, e.g.: !-- LCF document security enforcement component -- searchComponent name=lcfSecurity class=LCFSecurityFilter str name=AuthorityServiceBaseURLhttp://localhost:8080/lcf-authority-service/str /searchComponent Also required are the following schema.xml additions: !-- Security fields -- field name=allow_token_document type=string indexed=true stored=false multiValued=true/ field name=deny_token_document type=string indexed=true stored=false multiValued=true/ field name=allow_token_share type=string indexed=true stored=false multiValued=true/ field name=deny_token_share type=string indexed=true stored=false multiValued=true/ Finally, to tie it into the standard request handler, it seems to need to run last: requestHandler name=standard class=solr.SearchHandler default=true arr name=last-components strlcfSecurity/str /arr ... I have not set a package for this code. Nor have I been able to get it reviewed by someone as conversant with Solr as I would prefer. It is my hope, however, that this module will become part of the standard Solr 1.5 suite of search components, since that would tie it in with LCF nicely. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1895) ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time
[ https://issues.apache.org/jira/browse/SOLR-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108351#comment-13108351 ] Chris Male commented on SOLR-1895: -- I agree with Erik. Security should be the first step in the chain since it may have an impact on any of the later components. I kind of do agree that having this in Solr core is a little messy, especially since by default it depends on an external service. Yet at the same time there has been a real thirst from users for some out-of-box security system. At the moment the scope of this component is quite small, but down the line we might want a more comprehensive security system that this would just be part of? ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time -- Key: SOLR-1895 URL: https://issues.apache.org/jira/browse/SOLR-1895 Project: Solr Issue Type: New Feature Components: SearchComponents - other Reporter: Karl Wright Labels: document, security, solr Fix For: 3.5, 4.0 Attachments: LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch I've written an LCF SearchComponent which filters returned results based on access tokens provided by LCF's authority service. The component requires you to configure the appropriate authority service URL base, e.g.: !-- LCF document security enforcement component -- searchComponent name=lcfSecurity class=LCFSecurityFilter str name=AuthorityServiceBaseURLhttp://localhost:8080/lcf-authority-service/str /searchComponent Also required are the following schema.xml additions: !-- Security fields -- field name=allow_token_document type=string indexed=true stored=false multiValued=true/ field name=deny_token_document type=string indexed=true stored=false multiValued=true/ field name=allow_token_share type=string indexed=true stored=false multiValued=true/ field name=deny_token_share type=string indexed=true stored=false multiValued=true/ Finally, to tie it into the standard request handler, it seems to need to run last: requestHandler name=standard class=solr.SearchHandler default=true arr name=last-components strlcfSecurity/str /arr ... I have not set a package for this code. Nor have I been able to get it reviewed by someone as conversant with Solr as I would prefer. It is my hope, however, that this module will become part of the standard Solr 1.5 suite of search components, since that would tie it in with LCF nicely. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1895) ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time
[ https://issues.apache.org/jira/browse/SOLR-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108354#comment-13108354 ] Erik Hatcher commented on SOLR-1895: So maybe we put in a general SecuritySearchComponent into core that delegates its work to a SecurityFilterGenerator plugin that gets looked up through the resource loader mechanism and must be configured in solrconfig (with an out of the box - NoSecurityFilterGenerator/Factory or something like that). We want to allow a PostFilter to come into play here too, so looks like we want the security filter generation to return a Query, not a Filter. Maybe? ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time -- Key: SOLR-1895 URL: https://issues.apache.org/jira/browse/SOLR-1895 Project: Solr Issue Type: New Feature Components: SearchComponents - other Reporter: Karl Wright Labels: document, security, solr Fix For: 3.5, 4.0 Attachments: LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch I've written an LCF SearchComponent which filters returned results based on access tokens provided by LCF's authority service. The component requires you to configure the appropriate authority service URL base, e.g.: !-- LCF document security enforcement component -- searchComponent name=lcfSecurity class=LCFSecurityFilter str name=AuthorityServiceBaseURLhttp://localhost:8080/lcf-authority-service/str /searchComponent Also required are the following schema.xml additions: !-- Security fields -- field name=allow_token_document type=string indexed=true stored=false multiValued=true/ field name=deny_token_document type=string indexed=true stored=false multiValued=true/ field name=allow_token_share type=string indexed=true stored=false multiValued=true/ field name=deny_token_share type=string indexed=true stored=false multiValued=true/ Finally, to tie it into the standard request handler, it seems to need to run last: requestHandler name=standard class=solr.SearchHandler default=true arr name=last-components strlcfSecurity/str /arr ... I have not set a package for this code. Nor have I been able to get it reviewed by someone as conversant with Solr as I would prefer. It is my hope, however, that this module will become part of the standard Solr 1.5 suite of search components, since that would tie it in with LCF nicely. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1895) ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time
[ https://issues.apache.org/jira/browse/SOLR-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108356#comment-13108356 ] Koji Sekiguchi commented on SOLR-1895: -- bq. Is there a technical reason this needs to be done in the prepare phase rather than in the process phase? The idea of this security filter forcibly inserts security Filters before executing query. So I think it is obvious? Hmm, so about the order of search components, I think it should be placed at the last, because if it is not at last, theoretically the any last component can modify the inserted security Filters. ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time -- Key: SOLR-1895 URL: https://issues.apache.org/jira/browse/SOLR-1895 Project: Solr Issue Type: New Feature Components: SearchComponents - other Reporter: Karl Wright Labels: document, security, solr Fix For: 3.5, 4.0 Attachments: LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch I've written an LCF SearchComponent which filters returned results based on access tokens provided by LCF's authority service. The component requires you to configure the appropriate authority service URL base, e.g.: !-- LCF document security enforcement component -- searchComponent name=lcfSecurity class=LCFSecurityFilter str name=AuthorityServiceBaseURLhttp://localhost:8080/lcf-authority-service/str /searchComponent Also required are the following schema.xml additions: !-- Security fields -- field name=allow_token_document type=string indexed=true stored=false multiValued=true/ field name=deny_token_document type=string indexed=true stored=false multiValued=true/ field name=allow_token_share type=string indexed=true stored=false multiValued=true/ field name=deny_token_share type=string indexed=true stored=false multiValued=true/ Finally, to tie it into the standard request handler, it seems to need to run last: requestHandler name=standard class=solr.SearchHandler default=true arr name=last-components strlcfSecurity/str /arr ... I have not set a package for this code. Nor have I been able to get it reviewed by someone as conversant with Solr as I would prefer. It is my hope, however, that this module will become part of the standard Solr 1.5 suite of search components, since that would tie it in with LCF nicely. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Issue Comment Edited] (SOLR-1895) ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time
[ https://issues.apache.org/jira/browse/SOLR-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108356#comment-13108356 ] Koji Sekiguchi edited comment on SOLR-1895 at 9/20/11 4:10 AM: --- bq. Is there a technical reason this needs to be done in the prepare phase rather than in the process phase? The idea of this security filter forcibly inserts security Filters before executing query. So I think it is obvious? Hmm, so about the order of search components, I think it should be placed at the last, because if it is not at last, theoretically the any last component can modify or remove the inserted security Filters. was (Author: koji): bq. Is there a technical reason this needs to be done in the prepare phase rather than in the process phase? The idea of this security filter forcibly inserts security Filters before executing query. So I think it is obvious? Hmm, so about the order of search components, I think it should be placed at the last, because if it is not at last, theoretically the any last component can modify the inserted security Filters. ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time -- Key: SOLR-1895 URL: https://issues.apache.org/jira/browse/SOLR-1895 Project: Solr Issue Type: New Feature Components: SearchComponents - other Reporter: Karl Wright Labels: document, security, solr Fix For: 3.5, 4.0 Attachments: LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch I've written an LCF SearchComponent which filters returned results based on access tokens provided by LCF's authority service. The component requires you to configure the appropriate authority service URL base, e.g.: !-- LCF document security enforcement component -- searchComponent name=lcfSecurity class=LCFSecurityFilter str name=AuthorityServiceBaseURLhttp://localhost:8080/lcf-authority-service/str /searchComponent Also required are the following schema.xml additions: !-- Security fields -- field name=allow_token_document type=string indexed=true stored=false multiValued=true/ field name=deny_token_document type=string indexed=true stored=false multiValued=true/ field name=allow_token_share type=string indexed=true stored=false multiValued=true/ field name=deny_token_share type=string indexed=true stored=false multiValued=true/ Finally, to tie it into the standard request handler, it seems to need to run last: requestHandler name=standard class=solr.SearchHandler default=true arr name=last-components strlcfSecurity/str /arr ... I have not set a package for this code. Nor have I been able to get it reviewed by someone as conversant with Solr as I would prefer. It is my hope, however, that this module will become part of the standard Solr 1.5 suite of search components, since that would tie it in with LCF nicely. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1895) ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time
[ https://issues.apache.org/jira/browse/SOLR-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13108361#comment-13108361 ] Chris Male commented on SOLR-1895: -- I like what you're suggesting Erik. bq. Hmm, so about the order of search components, I think it should be placed at the last, because if it is not at last, theoretically the any last component can modify or remove the inserted security Filters. I'm not sure we should fight that. If someone wanted to modify the security Filters they could configure a component to come after the security component. I still feel having it last means we cannot use any information it adds to the Request in latter components. ManifoldCF SearchComponent plugin for enforcing ManifoldCF security at search time -- Key: SOLR-1895 URL: https://issues.apache.org/jira/browse/SOLR-1895 Project: Solr Issue Type: New Feature Components: SearchComponents - other Reporter: Karl Wright Labels: document, security, solr Fix For: 3.5, 4.0 Attachments: LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, LCFSecurityFilter.java, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch, SOLR-1895.patch I've written an LCF SearchComponent which filters returned results based on access tokens provided by LCF's authority service. The component requires you to configure the appropriate authority service URL base, e.g.: !-- LCF document security enforcement component -- searchComponent name=lcfSecurity class=LCFSecurityFilter str name=AuthorityServiceBaseURLhttp://localhost:8080/lcf-authority-service/str /searchComponent Also required are the following schema.xml additions: !-- Security fields -- field name=allow_token_document type=string indexed=true stored=false multiValued=true/ field name=deny_token_document type=string indexed=true stored=false multiValued=true/ field name=allow_token_share type=string indexed=true stored=false multiValued=true/ field name=deny_token_share type=string indexed=true stored=false multiValued=true/ Finally, to tie it into the standard request handler, it seems to need to run last: requestHandler name=standard class=solr.SearchHandler default=true arr name=last-components strlcfSecurity/str /arr ... I have not set a package for this code. Nor have I been able to get it reviewed by someone as conversant with Solr as I would prefer. It is my hope, however, that this module will become part of the standard Solr 1.5 suite of search components, since that would tie it in with LCF nicely. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
