Re: Custom Analyzer
Thanks for replying Erick! I executed this query jar -tf jarfile.jar and my classes were indeed there. Regarding the package I had this line written in my java file: package org.apache.lucene.analysis.mr I removed this line but still it was not working. Then I tried creating a small project (marathi.jar, containing the class files) and place it in the lib folder, I even set the CLASSPATH to the jar file, but still Solr is throwing me a ClassNotFoundException. Please help me out! -- View this message in context: http://lucene.472066.n3.nabble.com/Custom-Analyzer-tp2162710p2166185.html Sent from the Solr - User mailing list archive at Nabble.com.
Inner index stock big BLOB or separate in a DB ?
Hi, With big BLOB objects transcripted in Base64, anyone tried a performance test in query with a huge data (~M documents, ~500Gb) to compare 2 methods: - store directly theses BLOB-Base64 objects in Index by String field - separate these BLOB in a DB, then a URL index field point to associate record to mapping in result search Could you suggest which one is better? Thanks, Hong-Thai NGUYEN
Re: Inner index stock big BLOB or separate in a DB ?
Second option is better. Storing Big BLOB data in index will increase index size and it will create performance issues - Grijesh -- View this message in context: http://lucene.472066.n3.nabble.com/Inner-index-stock-big-BLOB-or-separate-in-a-DB-tp2166964p2167129.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Start solr unsuccessfully on Geronimo
Hi Pankaj and to whom it may concern - The crux and cure have been identified. This is because of missing jasper deployer in Geronimo 2.1.6, which are available in earlier Geronimo version Just download the following components, manually copy them to relevant folders. Done :) http://repo1.maven.org/maven2/org/apache/geronimo/modules/geronimo-jasper-builder/2.1.6/geronimo-jasper-builder-2.1.6.jar http://repo1.maven.org/maven2/org/apache/geronimo/configs/jasper-deployer/2.1.6/jasper-deployer-2.1.6.car Cheers and happy new year all Bac Hoang On 12/28/2010 5:38 PM, pankaj bhatt wrote: hi Bac, this is an Geronimo specific problem and it come due to the following reason: it may be that aur app server uses a particular attribute named 'org.apache.InstanceManager' when will destroy the JSP.. Can you try it by creating a simple war file of a simple jsp and try with it.. i presume the same error will come. it might be possible, you are not using Geronimo full distribution. Please refer to this: http://www.mail-archive.com/user@geronimo.apache.org/msg11891.html / Pankaj Bhatt. On Tue, Dec 28, 2010 at 3:26 PM, Bac Hoang bac.ho...@axonactive.vn mailto:bac.ho...@axonactive.vn wrote: Hello all - Could any one please shed a light to the hassle issue below when *start * Solr on Geronimo. === org.apache.jasper.JasperException: *java.lang.IllegalStateException: No org.apache.InstanceManager set in ServletContext* org.apache.jasper.servlet.JspServletWrapper.getServlet(JspServletWrapper.java:151) org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:324) org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313) org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260) javax.servlet.http.HttpServlet.service(HttpServlet.java:806) org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:286) === Additional info.: - deployed successfully but error when start. It starts well on Tomcat and Jetty - use the solr 1.4.1. - Geronimo 2.1.6 - OS: Unix and Win 7 (both got the same error) - other applications (war) run well on this Geronimo - jasper plugins are installed (jasper and jasper deployer) Thanks much!
Carrot2 clustering tool Beginner
Hi, I am new to carrot2 clustering tool. Can anyone Guide me related to this tool and how it can integrate with solr or lucene. Thanks! Seeking for your guidance.
Highlighter problem when using WordDelimiterFilter and term vectors
Hi, when using WordDelimiterFilterFactory in the fieldType definition and setting termVectors=true termPositions=true termOffsets=true on the field, Solr gives me the following response for the query request ?q=warmwasserspeicherversion=2.2indent=onhl=true lst name=highlighting lst name=id-1 arr name=content strsome text WarmemWarmWasserSpeicher/em here/str /arr /lst /lst As you can see, the highlighter does not work like expected (at least for me). If the term vectors are not stored into the index, I get the expected result strsome text emWarmWasserSpeicher/em here/str. I'm using Solr version 1.4.1 BTW, this problem does not occur when using the FastVectorHighlighter (after applying patches https://issues.apache.org/jira/browse/SOLR-1268) Any ideas? Uploaded document: add doc field name=idid-1/field field name=contentsome text WarmWasserSpeicher here/field /doc /add Field type definition: fieldType name=text class=solr.TextField positionIncrementGap=100 analyzer type=index tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.WordDelimiterFilterFactory generateWordParts=1 generateNumberParts=1 catenateWords=1 catenateNumbers=0 catenateAll=0 splitOnCaseChange=1/ filter class=solr.LowerCaseFilterFactory/ /analyzer analyzer type=query tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.WordDelimiterFilterFactory generateWordParts=1 generateNumberParts=1 catenateWords=0 catenateNumbers=0 catenateAll=0 splitOnCaseChange=0/ filter class=solr.LowerCaseFilterFactory/ /analyzer /fieldType Field definition: fields ... field name=content type=text indexed=true stored=true termVectors=true termPositions=true termOffsets=true/ /fields solrconf.xml: requestHandler name=dismax class=solr.SearchHandler default=true lst name=defaults bool name=tvtrue/bool str name=defTypedismax/str str name=qfcontent/str str name=mm1/str str name=hltrue/str str name=flscore/str /lst arr name=last-components strtvComponent/str /arr /requestHandler ... searchComponent name=tvComponent class=org.apache.solr.handler.component.TermVectorComponent/ Thanks, Oliver
Re: Carrot2 clustering tool Beginner
How about reading the wiki: http://wiki.apache.org/solr/ClusteringComponent On Thursday 30 December 2010 13:21:19 Isha Garg wrote: Hi, I am new to carrot2 clustering tool. Can anyone Guide me related to this tool and how it can integrate with solr or lucene. Thanks! Seeking for your guidance. -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350
Re: UPDATE query in deltaquery
Hi Erick!
Here is my DIH configuration:
dataConfig
dataSource name=jdbc driver=org.postgresql.Driver
url=jdbc:postgresql://${dataimporter.request.dbHost}:${dataimporter.request.dbPort}/${dataimporter.request.dbName}
user=${dataimporter.request.dbUser}
password=${dataimporter.request.dbPassword} autoCommit=false
transactionIsolation=TRANSACTION_READ_UNCOMMITTED
holdability=CLOSE_CURSORS_AT_COMMIT/
document
entity name='myEntity' dataSource='jdbc' pk='id'
query=' . '
deltaImportQuery=' . '
deltaQuery=' . '
/entity
/document
/dataConfig
I have tried two options for the deltaQuery:
UPDATE Global.Projects SET prj_lastSync = now() WHERE prj_id =
'2'; Throws a null pointer exception as described in the
previous email
The second option is a DB function that I am calling this way:
SELECT get_deltaimport_items AS id FROM
project.get_deltaimport_items(2, #39;project#39;);
The function inside executes the UPDATE query shown above and a SELECT
query for the ids.
The ids are returned ok, but the UPDATE has no effect on the database.
Cheers!
Juan M.
On Thu, Dec 30, 2010 at 1:32 AM, Erick Erickson erickerick...@gmail.com wrote:
Well, let's see the queries you're sending, and your DIH configuration.
Otherwise, we're just guessing...
Best
Erick
On Wed, Dec 29, 2010 at 9:58 PM, Juan Manuel Alvarez
naici...@gmail.comwrote:
Hi! I would like to ask you a question about using a deltaQuery in DIH.
I am syncing with a PostgreSQL database.
At first I was calling a function that made two queries: an UPDATE and a
SELECT.
The select result was properly returned, but the UPDATE query did not
made any changes,
so I tried calling the same function from a PostgreSQL client and
everything went OK.
So I tried calling a simple UPDATE query directly in the deltaQuery
and I receive a
NullPointerException that I traced to the line 251 of the
JdbcDataSource.java
colNames = readFieldNames(resultSet.getMetaData());
The question is: is there a way I can make the update query work in
the deltaQuery
or am I doing something wrong?
Happy new year
Cheers!
Juan M.
Re: Custom Analyzer
What does jar -tf your jar file here show you the actual classes in your jar are? You're still saying it doesn't work, without providing details that let us help. Imagine we're asking you for help. Does your message give enough info to suggest much? Best Erick On Wed, Dec 29, 2010 at 11:51 PM, nitishgarg nitishgarg1...@gmail.comwrote: Thanks for replying Erick! I executed this query jar -tf jarfile.jar and my classes were indeed there. Regarding the package I had this line written in my java file: package org.apache.lucene.analysis.mr I removed this line but still it was not working. Then I tried creating a small project (marathi.jar, containing the class files) and place it in the lib folder, I even set the CLASSPATH to the jar file, but still Solr is throwing me a ClassNotFoundException. Please help me out! -- View this message in context: http://lucene.472066.n3.nabble.com/Custom-Analyzer-tp2162710p2166185.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: UPDATE query in deltaquery
WARNING: DIH isn't my strong suit, I generally prefer doing things
in SolrJ. Mostly I asked for clarification so someone #else# who
actually knows DIH details could chime in...
That said, I'm a bit confused. As I understand it, you shouldn't
be UPDATEing anything in DIH, it's a select where documents
then get added to Solr by magic. Your post leads me to believe
that you're trying to change the database via DIH, is that at
all true?
This is based in part on
The ids are returned ok, but the UPDATE has no effect on the database
Or do you mean effect on the index? If the latter, then the select
would only have a chance of updating the IDs of the Solr documents...
At least I think that's close to reality...
Best
Erick
On Thu, Dec 30, 2010 at 7:52 AM, Juan Manuel Alvarez naici...@gmail.comwrote:
Hi Erick!
Here is my DIH configuration:
dataConfig
dataSource name=jdbc driver=org.postgresql.Driver
url=jdbc:postgresql://${dataimporter.request.dbHost}:${dataimporter.request.dbPort}/${dataimporter.request.dbName}
user=${dataimporter.request.dbUser}
password=${dataimporter.request.dbPassword} autoCommit=false
transactionIsolation=TRANSACTION_READ_UNCOMMITTED
holdability=CLOSE_CURSORS_AT_COMMIT/
document
entity name='myEntity' dataSource='jdbc' pk='id'
query=' . '
deltaImportQuery=' . '
deltaQuery=' . '
/entity
/document
/dataConfig
I have tried two options for the deltaQuery:
UPDATE Global.Projects SET prj_lastSync = now() WHERE prj_id =
'2'; Throws a null pointer exception as described in the
previous email
The second option is a DB function that I am calling this way:
SELECT get_deltaimport_items AS id FROM
project.get_deltaimport_items(2, #39;project#39;);
The function inside executes the UPDATE query shown above and a SELECT
query for the ids.
The ids are returned ok, but the UPDATE has no effect on the database.
Cheers!
Juan M.
On Thu, Dec 30, 2010 at 1:32 AM, Erick Erickson erickerick...@gmail.com
wrote:
Well, let's see the queries you're sending, and your DIH configuration.
Otherwise, we're just guessing...
Best
Erick
On Wed, Dec 29, 2010 at 9:58 PM, Juan Manuel Alvarez naici...@gmail.com
wrote:
Hi! I would like to ask you a question about using a deltaQuery in DIH.
I am syncing with a PostgreSQL database.
At first I was calling a function that made two queries: an UPDATE and a
SELECT.
The select result was properly returned, but the UPDATE query did not
made any changes,
so I tried calling the same function from a PostgreSQL client and
everything went OK.
So I tried calling a simple UPDATE query directly in the deltaQuery
and I receive a
NullPointerException that I traced to the line 251 of the
JdbcDataSource.java
colNames = readFieldNames(resultSet.getMetaData());
The question is: is there a way I can make the update query work in
the deltaQuery
or am I doing something wrong?
Happy new year
Cheers!
Juan M.
Re: UPDATE query in deltaquery
Erick:
Thanks for the quick response.
I can't use the timestamp for doing DIH, so I need to use a custom
field that I need to update one for each delta-import, so that is why
I need to execute an UPDATE on the deltaQuery.
Cheers!
Juan M.
On Thu, Dec 30, 2010 at 10:07 AM, Erick Erickson
erickerick...@gmail.com wrote:
WARNING: DIH isn't my strong suit, I generally prefer doing things
in SolrJ. Mostly I asked for clarification so someone #else# who
actually knows DIH details could chime in...
That said, I'm a bit confused. As I understand it, you shouldn't
be UPDATEing anything in DIH, it's a select where documents
then get added to Solr by magic. Your post leads me to believe
that you're trying to change the database via DIH, is that at
all true?
This is based in part on
The ids are returned ok, but the UPDATE has no effect on the database
Or do you mean effect on the index? If the latter, then the select
would only have a chance of updating the IDs of the Solr documents...
At least I think that's close to reality...
Best
Erick
On Thu, Dec 30, 2010 at 7:52 AM, Juan Manuel Alvarez
naici...@gmail.comwrote:
Hi Erick!
Here is my DIH configuration:
dataConfig
dataSource name=jdbc driver=org.postgresql.Driver
url=jdbc:postgresql://${dataimporter.request.dbHost}:${dataimporter.request.dbPort}/${dataimporter.request.dbName}
user=${dataimporter.request.dbUser}
password=${dataimporter.request.dbPassword} autoCommit=false
transactionIsolation=TRANSACTION_READ_UNCOMMITTED
holdability=CLOSE_CURSORS_AT_COMMIT/
document
entity name='myEntity' dataSource='jdbc' pk='id'
query=' . '
deltaImportQuery=' . '
deltaQuery=' . '
/entity
/document
/dataConfig
I have tried two options for the deltaQuery:
UPDATE Global.Projects SET prj_lastSync = now() WHERE prj_id =
'2'; Throws a null pointer exception as described in the
previous email
The second option is a DB function that I am calling this way:
SELECT get_deltaimport_items AS id FROM
project.get_deltaimport_items(2, #39;project#39;);
The function inside executes the UPDATE query shown above and a SELECT
query for the ids.
The ids are returned ok, but the UPDATE has no effect on the database.
Cheers!
Juan M.
On Thu, Dec 30, 2010 at 1:32 AM, Erick Erickson erickerick...@gmail.com
wrote:
Well, let's see the queries you're sending, and your DIH configuration.
Otherwise, we're just guessing...
Best
Erick
On Wed, Dec 29, 2010 at 9:58 PM, Juan Manuel Alvarez naici...@gmail.com
wrote:
Hi! I would like to ask you a question about using a deltaQuery in DIH.
I am syncing with a PostgreSQL database.
At first I was calling a function that made two queries: an UPDATE and a
SELECT.
The select result was properly returned, but the UPDATE query did not
made any changes,
so I tried calling the same function from a PostgreSQL client and
everything went OK.
So I tried calling a simple UPDATE query directly in the deltaQuery
and I receive a
NullPointerException that I traced to the line 251 of the
JdbcDataSource.java
colNames = readFieldNames(resultSet.getMetaData());
The question is: is there a way I can make the update query work in
the deltaQuery
or am I doing something wrong?
Happy new year
Cheers!
Juan M.
Sort Facet Query
Hi List, I got a little issue with sorting a FacetQuery. Currently I am doing something like that in SolrJ: SolrQuery q = new SolrQuery(myQuery); q.setFacetQuery(names:thomas);//want to see the count of thomas's documents. q.setFacetPrefix(short, th); I don't know any better example, but the result from all those facets should be returned in lexicographic order, not by count - so i can ensure that every constraint is returned at the same place. Any ideas? Thank you! -- View this message in context: http://lucene.472066.n3.nabble.com/Sort-Facet-Query-tp2167635p2167635.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: old index files not deleted on slave
We have tried all locktypes simple, single, native. But nothing worked. I have upgraded to Solr 1.4, and when i used the replication system of 1.4. It is working fine. Not sure why the scripts are not able to replicate the index on Linux, but java based replication is working. Thanks for the info. -- View this message in context: http://lucene.472066.n3.nabble.com/old-index-files-not-deleted-on-slave-tp2113493p2167789.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Sort Facet Query
Set facet.limit to -1 (globally or for that field). That will return all the facets, in lexicographical order. Stephen Duncan Jr www.stephenduncanjr.com On Thu, Dec 30, 2010 at 9:04 AM, Em mailformailingli...@yahoo.de wrote: Hi List, I got a little issue with sorting a FacetQuery. Currently I am doing something like that in SolrJ: SolrQuery q = new SolrQuery(myQuery); q.setFacetQuery(names:thomas);//want to see the count of thomas's documents. q.setFacetPrefix(short, th); I don't know any better example, but the result from all those facets should be returned in lexicographic order, not by count - so i can ensure that every constraint is returned at the same place. Any ideas? Thank you! -- View this message in context: http://lucene.472066.n3.nabble.com/Sort-Facet-Query-tp2167635p2167635.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Sort Facet Query
No http://wiki.apache.org/solr/SimpleFacetParameters#facet.sort On Thursday 30 December 2010 15:42:14 Stephen Duncan Jr wrote: Set facet.limit to -1 (globally or for that field). That will return all the facets, in lexicographical order. Stephen Duncan Jr www.stephenduncanjr.com On Thu, Dec 30, 2010 at 9:04 AM, Em mailformailingli...@yahoo.de wrote: Hi List, I got a little issue with sorting a FacetQuery. Currently I am doing something like that in SolrJ: SolrQuery q = new SolrQuery(myQuery); q.setFacetQuery(names:thomas);//want to see the count of thomas's documents. q.setFacetPrefix(short, th); I don't know any better example, but the result from all those facets should be returned in lexicographic order, not by count - so i can ensure that every constraint is returned at the same place. Any ideas? Thank you! -- View this message in context: http://lucene.472066.n3.nabble.com/Sort-Facet-Query-tp2167635p2167635.htm l Sent from the Solr - User mailing list archive at Nabble.com. -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350
Re: Sort Facet Query
Markus is right, it will return them by count. I think my question could me more general: How can I set limit, sort etc. for a Facet Query? -- View this message in context: http://lucene.472066.n3.nabble.com/Sort-Facet-Query-tp2167635p2167965.html Sent from the Solr - User mailing list archive at Nabble.com.
Saravanan Chinnadurai/Actionimages is out of the office.
I will be out of the office starting 30/12/2010 and will not return until 03/01/2011. Please email to itsta...@actionimages.com for any urgent issues. Action Images is a division of Reuters Limited and your data will therefore be protected in accordance with the Reuters Group Privacy / Data Protection notice which is available in the privacy footer at www.reuters.com Registered in England No. 145516 VAT REG: 397000555
RE: UPDATE query in deltaquery
This may sound silly, but are you sure the user you're using has
permissions to do the updates you want? Not sure about postgres but I
think some jdbc's require that the connection be defined as rw, maybe
you should try adding readOnly=false to your jdbc definition.
Ephraim Ofir
-Original Message-
From: Juan Manuel Alvarez [mailto:naici...@gmail.com]
Sent: Thursday, December 30, 2010 2:52 PM
To: solr-user@lucene.apache.org
Subject: Re: UPDATE query in deltaquery
Hi Erick!
Here is my DIH configuration:
dataConfig
dataSource name=jdbc driver=org.postgresql.Driver
url=jdbc:postgresql://${dataimporter.request.dbHost}:${dataimporter.req
uest.dbPort}/${dataimporter.request.dbName}
user=${dataimporter.request.dbUser}
password=${dataimporter.request.dbPassword} autoCommit=false
transactionIsolation=TRANSACTION_READ_UNCOMMITTED
holdability=CLOSE_CURSORS_AT_COMMIT/
document
entity name='myEntity' dataSource='jdbc' pk='id'
query=' . '
deltaImportQuery=' . '
deltaQuery=' . '
/entity
/document
/dataConfig
I have tried two options for the deltaQuery:
UPDATE Global.Projects SET prj_lastSync = now() WHERE prj_id =
'2'; Throws a null pointer exception as described in the
previous email
The second option is a DB function that I am calling this way:
SELECT get_deltaimport_items AS id FROM
project.get_deltaimport_items(2, #39;project#39;);
The function inside executes the UPDATE query shown above and a SELECT
query for the ids.
The ids are returned ok, but the UPDATE has no effect on the database.
Cheers!
Juan M.
On Thu, Dec 30, 2010 at 1:32 AM, Erick Erickson
erickerick...@gmail.com wrote:
Well, let's see the queries you're sending, and your DIH
configuration.
Otherwise, we're just guessing...
Best
Erick
On Wed, Dec 29, 2010 at 9:58 PM, Juan Manuel Alvarez
naici...@gmail.comwrote:
Hi! I would like to ask you a question about using a deltaQuery in
DIH.
I am syncing with a PostgreSQL database.
At first I was calling a function that made two queries: an UPDATE
and a
SELECT.
The select result was properly returned, but the UPDATE query did not
made any changes,
so I tried calling the same function from a PostgreSQL client and
everything went OK.
So I tried calling a simple UPDATE query directly in the deltaQuery
and I receive a
NullPointerException that I traced to the line 251 of the
JdbcDataSource.java
colNames = readFieldNames(resultSet.getMetaData());
The question is: is there a way I can make the update query work in
the deltaQuery
or am I doing something wrong?
Happy new year
Cheers!
Juan M.
RE: Issue using filter query with spellCheck component
The SpellCheckComponent in v1.4 does not use fq. All it does is take the keywords out of the q (or spellcheck.q) parameter and check them against the entire dictionary. If any keyword is not in the dictionary, it gives you a list of alternatives. The collate function then takes the query and substitutes the top alternative for each keyword with suggestions. There is no filtering. It doesn't even assure you the suggestions or the collation will be any better than the original query. If you were using fq to filter out secret information, spell checking might reveal the secrets to your user anyway. In 3.x, there is an additional feature that will check the collation against your index, including the fq parameters you sent. This will assure you that the Collation is going to return some hits if you re-query for it. This is probably closer to what you want. This does not affect the individual word suggestions, however, as it only works for the collation query. See http://wiki.apache.org/solr/SpellCheckComponent#spellcheck.collate for more information. This added 3.x functionality is also available in 1.4.1 as a patch. See https://issues.apache.org/jira/browse/SOLR-2010 James Dyer E-Commerce Systems Ingram Content Group (615) 213-4311 -Original Message- From: taimurAQ [mailto:taimur_qure...@hotmail.com] Sent: Thursday, December 30, 2010 12:20 AM To: solr-user@lucene.apache.org Subject: Issue using filter query with spellCheck component Hi, I am trying to use a filter query along with the spellcheck component but the resultant rows are completely ignoring the filter query parameter. I am using solrJ API to communicate with solr1.4. Here is a code snippet of how i am passing the query parameters: //start code snippet paramsForSpellCheck.set(q, *:*); paramsForSpellCheck.set(spellcheck.q, query); paramsForSpellCheck.set(rows, 0); paramsForSpellCheck.set(indent, on); paramsForSpellCheck.set(spellcheck, on); paramsForSpellCheck.set(spellcheck.build, true); paramsForSpellCheck.set(spellcheck.extendedResults, true); paramsForSpellCheck.set(spellcheck.dictionary, a_spell); paramsForSpellCheck.set(spellcheck.collate, true); paramsForSpellCheck.set(spellcheck.count, 3); paramsForSpellCheck.set(spellcheck.onlyMorePopular, false); paramsForSpellCheck.set(spellcheck.reload, true); paramsForSpellCheck.set(qt, standard); paramsForSpellCheck.set(fq, catalog_id: + catalogs); //end code snippet The issue that i am facing now is that the fq query parameter is not having any affect on the spellcheck query. Are there any known issues while using filter query with the spellcheck component? If not, what am i doing wrong? Thanks in advance. Regards, Taimur -- View this message in context: http://lucene.472066.n3.nabble.com/Issue-using-filter-query-with-spellCheck-component-tp2166322p2166322.html Sent from the Solr - User mailing list archive at Nabble.com.
StreamingUpdateSolrServer - connection refused
Hi all. I have designed a synchronizer that goes out to various databases, extracts some data, does some processing, and then uses the StreamingUpdateSolrServer to send the records to a Solr index. When everything is up, it works just fine. Now I'm trying to account for problems, like if the Solr index is down for some reason. I'm trying to get it so that I can keep track of items that were missed for synchronization, so that they can get replayed when the index comes back up. Before I was using the StreamingUpdateSolrServer, I'd get a message back immediately that there was a problem. Now that it streams, the error comes outside of the thread that I'm using to run the updates, so I can't catch the exception, so my thread isn't aware of the error, and just chugs along. Has anyone run into this problem? Is there anything built into the server so that I can know which records it was unable to send, and then pull them out in order to try running them again later? Any insight that anyone has would be greatly appreciated. Thanks! -- Christopher Gross
Re: Sort Facet Query
At the end of Marcus' link is facet.sort=false which will return in lexigraphical order (sometimes called index order). Best Erick On Thu, Dec 30, 2010 at 10:26 AM, Em mailformailingli...@yahoo.de wrote: Markus is right, it will return them by count. I think my question could me more general: How can I set limit, sort etc. for a Facet Query? -- View this message in context: http://lucene.472066.n3.nabble.com/Sort-Facet-Query-tp2167635p2167965.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: UPDATE query in deltaquery
Hi Ephraim! Thanks for the answer!
Actually the user has permissions to make UPDATE queries.
I changed the configuration to
dataSource name=jdbc driver=org.postgresql.Driver
url=jdbc:postgresql://${dataimporter.request.dbHost}:${dataimporter.request.dbPort}/${dataimporter.request.dbName}
user=${dataimporter.request.dbUser}
password=${dataimporter.request.dbPassword}
transactionIsolation=TRANSACTION_SERIALIZABLE
readOnly=false/
and I still get the same results.
Cheers!
Juan M.
On Thu, Dec 30, 2010 at 12:40 PM, Ephraim Ofir ephra...@icq.com wrote:
This may sound silly, but are you sure the user you're using has
permissions to do the updates you want? Not sure about postgres but I
think some jdbc's require that the connection be defined as rw, maybe
you should try adding readOnly=false to your jdbc definition.
Ephraim Ofir
-Original Message-
From: Juan Manuel Alvarez [mailto:naici...@gmail.com]
Sent: Thursday, December 30, 2010 2:52 PM
To: solr-user@lucene.apache.org
Subject: Re: UPDATE query in deltaquery
Hi Erick!
Here is my DIH configuration:
dataConfig
dataSource name=jdbc driver=org.postgresql.Driver
url=jdbc:postgresql://${dataimporter.request.dbHost}:${dataimporter.req
uest.dbPort}/${dataimporter.request.dbName}
user=${dataimporter.request.dbUser}
password=${dataimporter.request.dbPassword} autoCommit=false
transactionIsolation=TRANSACTION_READ_UNCOMMITTED
holdability=CLOSE_CURSORS_AT_COMMIT/
document
entity name='myEntity' dataSource='jdbc' pk='id'
query=' . '
deltaImportQuery=' . '
deltaQuery=' . '
/entity
/document
/dataConfig
I have tried two options for the deltaQuery:
UPDATE Global.Projects SET prj_lastSync = now() WHERE prj_id =
'2'; Throws a null pointer exception as described in the
previous email
The second option is a DB function that I am calling this way:
SELECT get_deltaimport_items AS id FROM
project.get_deltaimport_items(2, #39;project#39;);
The function inside executes the UPDATE query shown above and a SELECT
query for the ids.
The ids are returned ok, but the UPDATE has no effect on the database.
Cheers!
Juan M.
On Thu, Dec 30, 2010 at 1:32 AM, Erick Erickson
erickerick...@gmail.com wrote:
Well, let's see the queries you're sending, and your DIH
configuration.
Otherwise, we're just guessing...
Best
Erick
On Wed, Dec 29, 2010 at 9:58 PM, Juan Manuel Alvarez
naici...@gmail.comwrote:
Hi! I would like to ask you a question about using a deltaQuery in
DIH.
I am syncing with a PostgreSQL database.
At first I was calling a function that made two queries: an UPDATE
and a
SELECT.
The select result was properly returned, but the UPDATE query did not
made any changes,
so I tried calling the same function from a PostgreSQL client and
everything went OK.
So I tried calling a simple UPDATE query directly in the deltaQuery
and I receive a
NullPointerException that I traced to the line 251 of the
JdbcDataSource.java
colNames = readFieldNames(resultSet.getMetaData());
The question is: is there a way I can make the update query work in
the deltaQuery
or am I doing something wrong?
Happy new year
Cheers!
Juan M.
Re: UPDATE query in deltaquery
Hi Travis!
I am executing a function in de DB that has two queries: an UPDATE and
a SELECT, and I am getting the select results ok, but the update has
no effects, so it seems that that approach is not working.
Cheers!
Juan M.
On Thu, Dec 30, 2010 at 11:26 AM, Travis Low t...@4centurion.com wrote:
If you are getting a null pointer exception here:
colNames = readFieldNames(resultSet.getMetaData());
Then that implies the DIH code is written to expect a select statement. You
might be able to fool it with some SQL injection:
update blah set foo=bar where id=1234; select id from blah
But if that doesn't work then you may be out of luck.
cheers,
Travis
On Thu, Dec 30, 2010 at 8:26 AM, Juan Manuel Alvarez
naici...@gmail.comwrote:
Erick:
Thanks for the quick response.
I can't use the timestamp for doing DIH, so I need to use a custom
field that I need to update one for each delta-import, so that is why
I need to execute an UPDATE on the deltaQuery.
Cheers!
Juan M.
On Thu, Dec 30, 2010 at 10:07 AM, Erick Erickson
erickerick...@gmail.com wrote:
WARNING: DIH isn't my strong suit, I generally prefer doing things
in SolrJ. Mostly I asked for clarification so someone #else# who
actually knows DIH details could chime in...
That said, I'm a bit confused. As I understand it, you shouldn't
be UPDATEing anything in DIH, it's a select where documents
then get added to Solr by magic. Your post leads me to believe
that you're trying to change the database via DIH, is that at
all true?
This is based in part on
The ids are returned ok, but the UPDATE has no effect on the database
Or do you mean effect on the index? If the latter, then the select
would only have a chance of updating the IDs of the Solr documents...
At least I think that's close to reality...
Best
Erick
On Thu, Dec 30, 2010 at 7:52 AM, Juan Manuel Alvarez naici...@gmail.com
wrote:
Hi Erick!
Here is my DIH configuration:
dataConfig
dataSource name=jdbc driver=org.postgresql.Driver
url=jdbc:postgresql://${dataimporter.request.dbHost}:${dataimporter.request.dbPort}/${dataimporter.request.dbName}
user=${dataimporter.request.dbUser}
password=${dataimporter.request.dbPassword} autoCommit=false
transactionIsolation=TRANSACTION_READ_UNCOMMITTED
holdability=CLOSE_CURSORS_AT_COMMIT/
document
entity name='myEntity' dataSource='jdbc' pk='id'
query=' . '
deltaImportQuery=' . '
deltaQuery=' . '
/entity
/document
/dataConfig
I have tried two options for the deltaQuery:
UPDATE Global.Projects SET prj_lastSync = now() WHERE prj_id =
'2'; Throws a null pointer exception as described in the
previous email
The second option is a DB function that I am calling this way:
SELECT get_deltaimport_items AS id FROM
project.get_deltaimport_items(2, #39;project#39;);
The function inside executes the UPDATE query shown above and a SELECT
query for the ids.
The ids are returned ok, but the UPDATE has no effect on the database.
Cheers!
Juan M.
On Thu, Dec 30, 2010 at 1:32 AM, Erick Erickson
erickerick...@gmail.com
wrote:
Well, let's see the queries you're sending, and your DIH
configuration.
Otherwise, we're just guessing...
Best
Erick
On Wed, Dec 29, 2010 at 9:58 PM, Juan Manuel Alvarez
naici...@gmail.com
wrote:
Hi! I would like to ask you a question about using a deltaQuery in
DIH.
I am syncing with a PostgreSQL database.
At first I was calling a function that made two queries: an UPDATE
and a
SELECT.
The select result was properly returned, but the UPDATE query did not
made any changes,
so I tried calling the same function from a PostgreSQL client and
everything went OK.
So I tried calling a simple UPDATE query directly in the deltaQuery
and I receive a
NullPointerException that I traced to the line 251 of the
JdbcDataSource.java
colNames = readFieldNames(resultSet.getMetaData());
The question is: is there a way I can make the update query work in
the deltaQuery
or am I doing something wrong?
Happy new year
Cheers!
Juan M.
--
**
*Travis Low, Director of Development*
** t...@4centurion.com* *
*Centurion Research Solutions, LLC*
*14048 ParkEast Circle *•* Suite 100 *•* Chantilly, VA 20151*
*703-956-6276 *•* 703-378-4474 (fax)*
*http://www.centurionresearch.com* http://www.centurionresearch.com
**The information contained in this email message is confidential and
protected from disclosure. If you are not the intended recipient, any use
or dissemination of this communication, including attachments, is strictly
prohibited. If you received this email message in error, please delete it
and immediately notify the sender.
This email message and any attachments have been scanned and are
RE: UPDATE query in deltaquery
Does your function get_deltaimport_items perform the update first and then the
select? Does it make a difference if you change the order? Did you try omitting
the TRANSACTION_SERIALIZABLE part?
Ephraim Ofir
-Original Message-
From: Juan Manuel Alvarez [mailto:naici...@gmail.com]
Sent: Thursday, December 30, 2010 7:04 PM
To: solr-user@lucene.apache.org
Subject: Re: UPDATE query in deltaquery
Hi Ephraim! Thanks for the answer!
Actually the user has permissions to make UPDATE queries.
I changed the configuration to
dataSource name=jdbc driver=org.postgresql.Driver
url=jdbc:postgresql://${dataimporter.request.dbHost}:${dataimporter.request.dbPort}/${dataimporter.request.dbName}
user=${dataimporter.request.dbUser}
password=${dataimporter.request.dbPassword}
transactionIsolation=TRANSACTION_SERIALIZABLE
readOnly=false/
and I still get the same results.
Cheers!
Juan M.
On Thu, Dec 30, 2010 at 12:40 PM, Ephraim Ofir ephra...@icq.com wrote:
This may sound silly, but are you sure the user you're using has
permissions to do the updates you want? Not sure about postgres but I
think some jdbc's require that the connection be defined as rw, maybe
you should try adding readOnly=false to your jdbc definition.
Ephraim Ofir
-Original Message-
From: Juan Manuel Alvarez [mailto:naici...@gmail.com]
Sent: Thursday, December 30, 2010 2:52 PM
To: solr-user@lucene.apache.org
Subject: Re: UPDATE query in deltaquery
Hi Erick!
Here is my DIH configuration:
dataConfig
dataSource name=jdbc driver=org.postgresql.Driver
url=jdbc:postgresql://${dataimporter.request.dbHost}:${dataimporter.req
uest.dbPort}/${dataimporter.request.dbName}
user=${dataimporter.request.dbUser}
password=${dataimporter.request.dbPassword} autoCommit=false
transactionIsolation=TRANSACTION_READ_UNCOMMITTED
holdability=CLOSE_CURSORS_AT_COMMIT/
document
entity name='myEntity' dataSource='jdbc' pk='id'
query=' . '
deltaImportQuery=' . '
deltaQuery=' . '
/entity
/document
/dataConfig
I have tried two options for the deltaQuery:
UPDATE Global.Projects SET prj_lastSync = now() WHERE prj_id =
'2'; Throws a null pointer exception as described in the
previous email
The second option is a DB function that I am calling this way:
SELECT get_deltaimport_items AS id FROM
project.get_deltaimport_items(2, #39;project#39;);
The function inside executes the UPDATE query shown above and a SELECT
query for the ids.
The ids are returned ok, but the UPDATE has no effect on the database.
Cheers!
Juan M.
On Thu, Dec 30, 2010 at 1:32 AM, Erick Erickson
erickerick...@gmail.com wrote:
Well, let's see the queries you're sending, and your DIH
configuration.
Otherwise, we're just guessing...
Best
Erick
On Wed, Dec 29, 2010 at 9:58 PM, Juan Manuel Alvarez
naici...@gmail.comwrote:
Hi! I would like to ask you a question about using a deltaQuery in
DIH.
I am syncing with a PostgreSQL database.
At first I was calling a function that made two queries: an UPDATE
and a
SELECT.
The select result was properly returned, but the UPDATE query did not
made any changes,
so I tried calling the same function from a PostgreSQL client and
everything went OK.
So I tried calling a simple UPDATE query directly in the deltaQuery
and I receive a
NullPointerException that I traced to the line 251 of the
JdbcDataSource.java
colNames = readFieldNames(resultSet.getMetaData());
The question is: is there a way I can make the update query work in
the deltaQuery
or am I doing something wrong?
Happy new year
Cheers!
Juan M.
Re: UPDATE query in deltaquery
Hi Ephraim! Thanks again for taking the time to help me. Really appreciated =o)
The UPDATE was before the SELECT, but putting it after leads to the
same result, with or without the TRANSACTION_SERIALIZABLE.
Cheers!
Juan M.
2010/12/30 Ephraim Ofir ephra...@icq.com:
Does your function get_deltaimport_items perform the update first and then
the select? Does it make a difference if you change the order? Did you try
omitting the TRANSACTION_SERIALIZABLE part?
Ephraim Ofir
-Original Message-
From: Juan Manuel Alvarez [mailto:naici...@gmail.com]
Sent: Thursday, December 30, 2010 7:04 PM
To: solr-user@lucene.apache.org
Subject: Re: UPDATE query in deltaquery
Hi Ephraim! Thanks for the answer!
Actually the user has permissions to make UPDATE queries.
I changed the configuration to
dataSource name=jdbc driver=org.postgresql.Driver
url=jdbc:postgresql://${dataimporter.request.dbHost}:${dataimporter.request.dbPort}/${dataimporter.request.dbName}
user=${dataimporter.request.dbUser}
password=${dataimporter.request.dbPassword}
transactionIsolation=TRANSACTION_SERIALIZABLE
readOnly=false/
and I still get the same results.
Cheers!
Juan M.
On Thu, Dec 30, 2010 at 12:40 PM, Ephraim Ofir ephra...@icq.com wrote:
This may sound silly, but are you sure the user you're using has
permissions to do the updates you want? Not sure about postgres but I
think some jdbc's require that the connection be defined as rw, maybe
you should try adding readOnly=false to your jdbc definition.
Ephraim Ofir
-Original Message-
From: Juan Manuel Alvarez [mailto:naici...@gmail.com]
Sent: Thursday, December 30, 2010 2:52 PM
To: solr-user@lucene.apache.org
Subject: Re: UPDATE query in deltaquery
Hi Erick!
Here is my DIH configuration:
dataConfig
dataSource name=jdbc driver=org.postgresql.Driver
url=jdbc:postgresql://${dataimporter.request.dbHost}:${dataimporter.req
uest.dbPort}/${dataimporter.request.dbName}
user=${dataimporter.request.dbUser}
password=${dataimporter.request.dbPassword} autoCommit=false
transactionIsolation=TRANSACTION_READ_UNCOMMITTED
holdability=CLOSE_CURSORS_AT_COMMIT/
document
entity name='myEntity' dataSource='jdbc' pk='id'
query=' . '
deltaImportQuery=' . '
deltaQuery=' . '
/entity
/document
/dataConfig
I have tried two options for the deltaQuery:
UPDATE Global.Projects SET prj_lastSync = now() WHERE prj_id =
'2'; Throws a null pointer exception as described in the
previous email
The second option is a DB function that I am calling this way:
SELECT get_deltaimport_items AS id FROM
project.get_deltaimport_items(2, #39;project#39;);
The function inside executes the UPDATE query shown above and a SELECT
query for the ids.
The ids are returned ok, but the UPDATE has no effect on the database.
Cheers!
Juan M.
On Thu, Dec 30, 2010 at 1:32 AM, Erick Erickson
erickerick...@gmail.com wrote:
Well, let's see the queries you're sending, and your DIH
configuration.
Otherwise, we're just guessing...
Best
Erick
On Wed, Dec 29, 2010 at 9:58 PM, Juan Manuel Alvarez
naici...@gmail.comwrote:
Hi! I would like to ask you a question about using a deltaQuery in
DIH.
I am syncing with a PostgreSQL database.
At first I was calling a function that made two queries: an UPDATE
and a
SELECT.
The select result was properly returned, but the UPDATE query did not
made any changes,
so I tried calling the same function from a PostgreSQL client and
everything went OK.
So I tried calling a simple UPDATE query directly in the deltaQuery
and I receive a
NullPointerException that I traced to the line 251 of the
JdbcDataSource.java
colNames = readFieldNames(resultSet.getMetaData());
The question is: is there a way I can make the update query work in
the deltaQuery
or am I doing something wrong?
Happy new year
Cheers!
Juan M.
Re: Sort Facet Query
Yes, I understood. But what if I DON'T want to return ALL facet fields in index-order, but only 2 of 5? When faceting on fields I could just specify it, but how can I do so with a FacetQuery without making *all* Facets sorted in the same way? -- View this message in context: http://lucene.472066.n3.nabble.com/Sort-Facet-Query-tp2167635p2168762.html Sent from the Solr - User mailing list archive at Nabble.com.
Different behavior for q=goo.com vs q=@goo.com in queries?
Using Lucid's Solr 1.4 distribution, if I index my email inbox and then search it by passing in different email expressions, I notice that I get different results based on whether the '@' character is included, even though the character is present in every email address in the field I'm searching. For example, q=goo.com returns multiple items, as expected. However, q...@goo.com return no results. Since every address containing goo.com also contains @goo.com, I would expect the same number of results. I get this from both the Solr admin console and from my application, which URL-encodes the query. I Googled, searched the Wiki, and grepped the Pugh and Lucid books, but don't see anything about this. Ideas? Thanks! -- View this message in context: http://lucene.472066.n3.nabble.com/Different-behavior-for-q-goo-com-vs-q-goo-com-in-queries-tp2168935p2168935.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: StreamingUpdateSolrServer - connection refused
My current solution is to use the ping() function -- which doesn't run in a thread -- to test the connection before trying to send the data to the Solr index. It isn't elegant, but it works. If anyone has a better idea, I'd like to hear it. -- Chris On Thu, Dec 30, 2010 at 11:10 AM, Christopher Gross cogr...@gmail.comwrote: Hi all. I have designed a synchronizer that goes out to various databases, extracts some data, does some processing, and then uses the StreamingUpdateSolrServer to send the records to a Solr index. When everything is up, it works just fine. Now I'm trying to account for problems, like if the Solr index is down for some reason. I'm trying to get it so that I can keep track of items that were missed for synchronization, so that they can get replayed when the index comes back up. Before I was using the StreamingUpdateSolrServer, I'd get a message back immediately that there was a problem. Now that it streams, the error comes outside of the thread that I'm using to run the updates, so I can't catch the exception, so my thread isn't aware of the error, and just chugs along. Has anyone run into this problem? Is there anything built into the server so that I can know which records it was unable to send, and then pull them out in order to try running them again later? Any insight that anyone has would be greatly appreciated. Thanks! -- Christopher Gross
Re: Different behavior for q=goo.com vs q=@goo.com in queries?
What steps have you taken to figure out whether the contents of your index are what you think? I suspect that the fields you're indexing aren't being analyzed/tokenized quite the way you expect either at query time or index time (or maybe both!). Take a look at the admin/analysis page for the field you're indexing the data into. If that doesn't shed any light on the problem, please paste in the fieldType definition for the field in question, maybe another set of eyes can see the issue. Best Erick On Thu, Dec 30, 2010 at 2:11 PM, mrw mikerobertsw...@gmail.com wrote: Using Lucid's Solr 1.4 distribution, if I index my email inbox and then search it by passing in different email expressions, I notice that I get different results based on whether the '@' character is included, even though the character is present in every email address in the field I'm searching. For example, q=goo.com returns multiple items, as expected. However, q...@goo.com return no results. Since every address containing goo.com also contains @goo.com, I would expect the same number of results. I get this from both the Solr admin console and from my application, which URL-encodes the query. I Googled, searched the Wiki, and grepped the Pugh and Lucid books, but don't see anything about this. Ideas? Thanks! -- View this message in context: http://lucene.472066.n3.nabble.com/Different-behavior-for-q-goo-com-vs-q-goo-com-in-queries-tp2168935p2168935.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Different behavior for q=goo.com vs q=@goo.com in queries?
Basically, just what you've suggested. I did the field/query analysis piece with verbose output. Not entirely sure how to interpret the results, of course. Currently reading anything I can find on that. Thanks Erick Erickson wrote: What steps have you taken to figure out whether the contents of your index are what you think? I suspect that the fields you're indexing aren't being analyzed/tokenized quite the way you expect either at query time or index time (or maybe both!). Take a look at the admin/analysis page for the field you're indexing the data into. If that doesn't shed any light on the problem, please paste in the fieldType definition for the field in question, maybe another set of eyes can see the issue. Best Erick -- View this message in context: http://lucene.472066.n3.nabble.com/Different-behavior-for-q-goo-com-vs-q-goo-com-in-queries-tp2168935p2169478.html Sent from the Solr - User mailing list archive at Nabble.com.
post search using solrj
Hi. I am using solrj and it has been working fine. I now have a requirement to add more parameters. So many that I get a max URI exceeded error. Is there anyway using SolrQuery todo a http post so I don't have these issues? don
Re: post search using solrj
Hi Don, you could give the HTTP method to be used as a second argument to the QueryRequest constructor: [http://lucene.apache.org/solr/api/org/apache/solr/client/solrj/request/QueryRequest.html#QueryRequest(org.apache.solr.common.params.SolrParams,%20org.apache.solr.client.solrj.SolrRequest.METHOD)] -Sascha Don Hill wrote: Hi. I am using solrj and it has been working fine. I now have a requirement to add more parameters. So many that I get a max URI exceeded error. Is there anyway using SolrQuery todo a http post so I don't have these issues? don
Re: DIH for taxonomy faceting in Lucid webcast
: : NonFic/Science, how do I turn that into 0/NonFic : : : 1/NonFic/Science using the DIH? : : I don't have any specific suggestions for you -- i've never ... : Thanks Chris. : : What did you use to generate those encodings if not DIH? I've used this gereral approach several times -- in most of them i had other people to write the code for me once i spec'ed it out, and i don't know how they actually implemented it. IN two specific cases i do know of: one used custom loader code that was part of the publishing system for the authoritative data store (essentially building the individual terms in SolrInputDocuments and sending them to solr via SolrJ) and the other case i was doing a really super simple ingestor that batch processed directories of XML files, and used XSLT to transform them into Solr's XML format (i think the category breadcrumb fields relied heavily on using xsl:for-each/ in conjunction with the position() function) -Hoss
Re: dynamic fields revisited
solr/admin/analysis.jsp uses the Luke handler. You can browse facets and fields. On Wed, Dec 29, 2010 at 7:46 PM, Ahmet Arslan iori...@yahoo.com wrote: If I understand you correctly, for an INT dynamic field called *_int2 filled with field callled my_number_int2 during data import in a query, I will search in the index on the field called: my_number_int2 correct? Exactly. Using http://wiki.apache.org/solr/LukeRequestHandler you can retrieve real field names under *_int2, if thats help. -- Lance Norskog goks...@gmail.com
Re: post search using solrj
Another way is to create a requestHandler entry point in solrconfig.xml that includes lots of parameters in the defaults section. This way your URLs only have things that change. On Thu, Dec 30, 2010 at 3:12 PM, Sascha SZOTT sz...@gmx.de wrote: Hi Don, you could give the HTTP method to be used as a second argument to the QueryRequest constructor: [http://lucene.apache.org/solr/api/org/apache/solr/client/solrj/request/QueryRequest.html#QueryRequest(org.apache.solr.common.params.SolrParams,%20org.apache.solr.client.solrj.SolrRequest.METHOD)] -Sascha Don Hill wrote: Hi. I am using solrj and it has been working fine. I now have a requirement to add more parameters. So many that I get a max URI exceeded error. Is there anyway using SolrQuery todo a http post so I don't have these issues? don -- Lance Norskog goks...@gmail.com
exception obtaining write lock on startup
Hi, I'm getting this exception when I have 2 cores as masters. Seems like one of the cores obtains a lock (file) and then the other tries to obtain the same one. However, the first one is not deleted. How do I fix this? Dec 30, 2010 4:34:48 PM org.apache.solr.handler.ReplicationHandler inform WARNING: Unable to get IndexCommit on startup org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: Native fsl...@..\webapps\solr\tnsolr\data\index\lucene-fe3fc928a4bbfeb55082e49b32a70c10 -write.lock at org.apache.lucene.store.Lock.obtain(Lock.java:85) at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1565) at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1421) at org.apache.solr.update.SolrIndexWriter.init(SolrIndexWriter.java:19 1) at org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHand ler.java:98) at org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHa ndler2.java:173) at org.apache.solr.update.DirectUpdateHandler2.forceOpenWriter(DirectUpd ateHandler2.java:376) at org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler. Tri
Re: dynamic fields revisited
When my Solr guru gets back, we'll redo the schema and see what happens, thanks! Dennis Gearon Signature Warning It is always a good idea to learn from your own mistakes. It is usually a better idea to learn from others’ mistakes, so you do not have to make them yourself. from 'http://blogs.techrepublic.com.com/security/?p=4501tag=nl.e036' EARTH has a Right To Life, otherwise we all die. - Original Message From: Lance Norskog goks...@gmail.com To: solr-user@lucene.apache.org Sent: Thu, December 30, 2010 4:26:58 PM Subject: Re: dynamic fields revisited solr/admin/analysis.jsp uses the Luke handler. You can browse facets and fields. On Wed, Dec 29, 2010 at 7:46 PM, Ahmet Arslan iori...@yahoo.com wrote: If I understand you correctly, for an INT dynamic field called *_int2 filled with field callled my_number_int2 during data import in a query, I will search in the index on the field called: my_number_int2 correct? Exactly. Using http://wiki.apache.org/solr/LukeRequestHandler you can retrieve real field names under *_int2, if thats help. -- Lance Norskog goks...@gmail.com
Re: exception obtaining write lock on startup
This will not work. At all. You can only have one Solr core instance changing an index. On Thu, Dec 30, 2010 at 4:38 PM, Tri Nguyen tringuye...@yahoo.com wrote: Hi, I'm getting this exception when I have 2 cores as masters. Seems like one of the cores obtains a lock (file) and then the other tries to obtain the same one. However, the first one is not deleted. How do I fix this? Dec 30, 2010 4:34:48 PM org.apache.solr.handler.ReplicationHandler inform WARNING: Unable to get IndexCommit on startup org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: Native fsl...@..\webapps\solr\tnsolr\data\index\lucene-fe3fc928a4bbfeb55082e49b32a70c10 -write.lock at org.apache.lucene.store.Lock.obtain(Lock.java:85) at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1565) at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1421) at org.apache.solr.update.SolrIndexWriter.init(SolrIndexWriter.java:19 1) at org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHand ler.java:98) at org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHa ndler2.java:173) at org.apache.solr.update.DirectUpdateHandler2.forceOpenWriter(DirectUpd ateHandler2.java:376) at org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler. Tri -- Lance Norskog goks...@gmail.com
Advice on Exact Matching?
Hi, I am trying to make sure that when I search for text—regardless of what that text is—that I get an exact match. I'm *still* getting some issues, and this last mile is becoming very painful. The solr field, for which I'm setting this up on, is pasted below my explanation. I appreciate any help. Explanation: I'm crawling websites with Nutch. I'm performing some mechanical-turk-like filtering and term matching. The problem is, there's some very gnarly behavior in Solr due to any number of gotchas. If I want to find *all* Solr documents that match [id]somejunk\hi[/id] then life is instantly hell. Likewise, lots of whitespace in between words throws it off john says hello, how are you? I would love to be able to search for these exact phrases. If that's just not practical (I'm more than willing to live with a bloated search index), what would some other strategies be? There's no MapReduce in Solr; I could attempt to do Hadoop-streaming, but that's not very ideal for a variety of reasons. Solr Schema.xml, fieldType text (no, this is not used everywhere; only on 2 fields): fieldType name=textclass=solr.TextField positionIncrementGap=100 analyzer tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.WordDelimiterFilterFactory generateWordParts=1 generateNumberParts=1 catenateWords=1 catenateNumbers=1 catenateAll=1 splitOnCaseChange=1/ filter class=solr.EnglishPorterFilterFactory protected=protwords.txt/ filter class=solr.SynonymFilterFactory synonyms=synonyms.txt expand=true ignoreCase=true/ filter class=solr.LowerCaseFilterFactory/ /analyzer /fieldType Thank you, Scott Gonyea
Re: Advice on Exact Matching?
You can gain a lot of insight into this kind of thing with the admin/analysis page. Often the issue is that your tokenizing/ filtering isn't doing quite what you think. Try turning on the debug checkboxes on that page and seeing what tokens are generated at index and analysis page. In particular, WordDelimiterFactory is often a surprise in how it splits and recombines tokens. Including synonyms is another potential issue. Not to mention the EnglishPorterFilterFactory. If that's not helpful, could you paste some examples that you expect to match that don't? Best Erick On Thu, Dec 30, 2010 at 8:04 PM, Scott Gonyea sc...@aitrus.org wrote: Hi, I am trying to make sure that when I search for text—regardless of what that text is—that I get an exact match. I'm *still* getting some issues, and this last mile is becoming very painful. The solr field, for which I'm setting this up on, is pasted below my explanation. I appreciate any help. Explanation: I'm crawling websites with Nutch. I'm performing some mechanical-turk-like filtering and term matching. The problem is, there's some very gnarly behavior in Solr due to any number of gotchas. If I want to find *all* Solr documents that match [id]somejunk\hi[/id] then life is instantly hell. Likewise, lots of whitespace in between words throws it off john says hello, how are you? I would love to be able to search for these exact phrases. If that's just not practical (I'm more than willing to live with a bloated search index), what would some other strategies be? There's no MapReduce in Solr; I could attempt to do Hadoop-streaming, but that's not very ideal for a variety of reasons. Solr Schema.xml, fieldType text (no, this is not used everywhere; only on 2 fields): fieldType name=textclass=solr.TextField positionIncrementGap=100 analyzer tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.WordDelimiterFilterFactory generateWordParts=1 generateNumberParts=1 catenateWords=1 catenateNumbers=1 catenateAll=1 splitOnCaseChange=1/ filter class=solr.EnglishPorterFilterFactory protected=protwords.txt/ filter class=solr.SynonymFilterFactory synonyms=synonyms.txt expand=true ignoreCase=true/ filter class=solr.LowerCaseFilterFactory/ /analyzer /fieldType Thank you, Scott Gonyea
Question on long delta import
When using DIH my delta imports appear to finish quickly.. ie it says Indexing completed. Added/Updated: 95491 documents. Deleted 11148 documents. in a relatively short amount of time (~30mins). However the importMessage says A command is still running... for a really long time (~60mins). What is happening during this phase and how could I speed this up? Thanks!
