Re: Error when creating collection
Looking on the logs of the other server, I see this: SEVERE: org.apache.solr.common.SolrException: I was asked to wait on state recovering for 192.168.1.155:8983_solr but I still do not see the requested state. I see state: active live:true at org.apache.solr.handler.admin.CoreAdminHandler.handleWaitForStateAction(CoreAdminHandler.java:890) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:186) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:591) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:192) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141) - Original Message - From: "Mark Olsen" To: solr-user@lucene.apache.org Sent: Monday, May 19, 2014 3:54:30 PM Subject: Re: Error when creating collection Shawn, I set a shard ID however am still getting the same issue. The logs on the solr server are reporting the following: The request is being performed on 192.168.1.155 and it is timing out when contacting another server in the zookeeper group (192.168.1.152). There are no firewall rules in place and I am able to lynx from 192.168.1.155 to 192.168.1.152 on that port. SEVERE: Error while trying to recover. core=collection1:org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://192.168.1.152:8983/solr at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:412) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181) at org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:202) at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:346) at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:223) Caused by: java.net.SocketTimeoutException: Read timed out Thanks, Mark - Original Message - From: "Shawn Heisey" To: solr-user@lucene.apache.org Sent: Wednesday, May 14, 2014 12:43:00 PM Subject: Re: Error when creating collection On 5/13/2014 4:39 PM, Mark Olsen wrote: > I'm creating a collection via Java using this function call: > > String collection = "profile-2"; > CoreAdminRequest.Create createRequest = new CoreAdminRequest.Create(); > createRequest.setCoreName(collection); > createRequest.setCollection(collection); > createRequest.setInstanceDir(collection); > createRequest.setNumShards(1); > createRequest.process(server); > > It is timing out with this exception (from the solr.out logs): > > SEVERE: org.apache.solr.common.SolrException: Error CREATEing SolrCore > 'profile-2': Could not get shard_id for core: profile-2 > coreNodeName:192.168.1.152:8983_solr_profile-2 > at > org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:483) > > at > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:140) > > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > > at > org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:591) > > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:192) > > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141) > > ... > Caused by: org.apache.solr.common.SolrException: Could not get shard_id for > core: profile-2 coreNodeName:192.168.1.152:8983_solr_profile-2 > at > org.apache.solr.cloud.ZkController.doGetShardIdProcess(ZkController.java:1221) > > at org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1290) > at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:861) > > In a "development" environment the zookeeper/solr instances are running with > elevated permissions and this function worked without error. > In a "test" environment (which matches the "production" environment) the > permissions are more restricted. I made sure the group/owner of the > /usr/local/solr directory are set up to be the correct user. This is happening because you never set the shard ID. See the "Caused by" message above. There is a setShardID method on the class that you are using. I believe this would typically get something like "shard1" as a value. The user that runs Solr must typically have write permissions to the solr home and all of its descendants. Note that with the CoreAdminRequest class, you are not creating a collection. You are creating a core. If you want to cre
Re: Error when creating collection
Shawn, I set a shard ID however am still getting the same issue. The logs on the solr server are reporting the following: The request is being performed on 192.168.1.155 and it is timing out when contacting another server in the zookeeper group (192.168.1.152). There are no firewall rules in place and I am able to lynx from 192.168.1.155 to 192.168.1.152 on that port. SEVERE: Error while trying to recover. core=collection1:org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://192.168.1.152:8983/solr at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:412) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181) at org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:202) at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:346) at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:223) Caused by: java.net.SocketTimeoutException: Read timed out Thanks, Mark - Original Message - From: "Shawn Heisey" To: solr-user@lucene.apache.org Sent: Wednesday, May 14, 2014 12:43:00 PM Subject: Re: Error when creating collection On 5/13/2014 4:39 PM, Mark Olsen wrote: > I'm creating a collection via Java using this function call: > > String collection = "profile-2"; > CoreAdminRequest.Create createRequest = new CoreAdminRequest.Create(); > createRequest.setCoreName(collection); > createRequest.setCollection(collection); > createRequest.setInstanceDir(collection); > createRequest.setNumShards(1); > createRequest.process(server); > > It is timing out with this exception (from the solr.out logs): > > SEVERE: org.apache.solr.common.SolrException: Error CREATEing SolrCore > 'profile-2': Could not get shard_id for core: profile-2 > coreNodeName:192.168.1.152:8983_solr_profile-2 > at > org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:483) > > at > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:140) > > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > > at > org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:591) > > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:192) > > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141) > > ... > Caused by: org.apache.solr.common.SolrException: Could not get shard_id for > core: profile-2 coreNodeName:192.168.1.152:8983_solr_profile-2 > at > org.apache.solr.cloud.ZkController.doGetShardIdProcess(ZkController.java:1221) > > at org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1290) > at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:861) > > In a "development" environment the zookeeper/solr instances are running with > elevated permissions and this function worked without error. > In a "test" environment (which matches the "production" environment) the > permissions are more restricted. I made sure the group/owner of the > /usr/local/solr directory are set up to be the correct user. This is happening because you never set the shard ID. See the "Caused by" message above. There is a setShardID method on the class that you are using. I believe this would typically get something like "shard1" as a value. The user that runs Solr must typically have write permissions to the solr home and all of its descendants. Note that with the CoreAdminRequest class, you are not creating a collection. You are creating a core. If you want to create an entire collection (which will typically create at least two cores on different Solr instances), you need to use CollectionAdminRequest instead. https://lucene.apache.org/solr/4_8_0/solr-solrj/org/apache/solr/client/solrj/request/CollectionAdminRequest.Create.html http://wiki.apache.org/solr/SolrTerminology Thanks, Shawn
Error when creating collection
Solr version: 4.2.1 I'm creating a collection via Java using this function call: String collection = "profile-2"; CoreAdminRequest.Create createRequest = new CoreAdminRequest.Create(); createRequest.setCoreName(collection); createRequest.setCollection(collection); createRequest.setInstanceDir(collection); createRequest.setNumShards(1); createRequest.process(server); It is timing out with this exception (from the solr.out logs): SEVERE: org.apache.solr.common.SolrException: Error CREATEing SolrCore 'profile-2': Could not get shard_id for core: profile-2 coreNodeName:192.168.1.152:8983_solr_profile-2 at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:483) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:140) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:591) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:192) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141) ... Caused by: org.apache.solr.common.SolrException: Could not get shard_id for core: profile-2 coreNodeName:192.168.1.152:8983_solr_profile-2 at org.apache.solr.cloud.ZkController.doGetShardIdProcess(ZkController.java:1221) at org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1290) at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:861) In a "development" environment the zookeeper/solr instances are running with elevated permissions and this function worked without error. In a "test" environment (which matches the "production" environment) the permissions are more restricted. I made sure the group/owner of the /usr/local/solr directory are set up to be the correct user. Any insight into potential file permissions that I may be overlooking? Thank you, Mark
Re: Range query and join, oarse exception when parens are added
Chris,
Thank you for the detailed explanation, this helps a lot.
One of my current hurdles is my search system is in Java using Lucene Query
objects to construct a BooleanQuery which is then handed to Solr. Since Lucene
does not know about the LocalParams it's tricky to get them to play properly
when dealing with complex queries. My first solution was to prefix the
LocalParams to the field name, this worked fine until I ran the range query.
Changing to use the "v=" field of a LocalParam would work from the query
structure perspective, however getting that into a Lucene Query object will be
a fun exercise.
`Mark
- Original Message -
From: "Chris Hostetter"
To: solr-user@lucene.apache.org
Sent: Thursday, April 10, 2014 4:33:19 PM
Subject: Re: Range query and join, oarse exception when parens are added
Mark: first off, the details matter.
Nothing in your first email mae it clear that the {!join} query you were
refering to was not the entirety of your query param -- which is part of
the confusion and was a significant piece of Shawn's answer. Had you
posted the *exact* request you were sending (with all params) and the
full response you got, the root cause of your problem would have been a
lot more obvious to some folks very quickly.
As shawn mentioned, the {!...} syntax involves localparams and invoking a
nammed parser -- normaly this syntax is the *first* thig in a query
string, and causes the *entire* string to be parsed by that parser. this
is why something like this should work fine for you (please confirm if
it does not)...
q={!join from=member_profile_doc_id to=id}language_proficiency_id_number:[30
TO 50]
...because the "{!" at the begining of the param value tells solr to let
that parser (in this case "join") parse the entire string, using the
localparam options specified (from=member_profile_doc_id & to=id). the
"join" parser then delegates to the "lucene" parser for it's body
(language_proficiency_id_number:[30 TO 50])
However ... when the "{!" syntax is not the first thing in the param
value, what happens is that the default parser (lucene) is used -- the
lucene parser has a handy feature that supports looking for the "{!"
syntax nested inside of it ... which means you can do things like this...
q=+(+{!prefix f=foo}bar -{!term f=yak}wak) -aaa:zzz
...however there is an important caveat to this: when the "lucene" parser
is looking for the "{!" syntax to know when you want it to delegate to
another parser, how does it know if/when you intended for the input to
that nested parser to end?
Specifically, in the example above, how does it know if the input to the
prefix parser was ment to be "bar" or "bar -{!term f=yak}wak) -aaa:zzz"
The answer is that it's conservative and assumes the input to the nested
parser stops as soon as it sees something that looks like the end of a the
current clause: whitespace or an open/close parent for example.
Which brings us to your specific example...
: +({!join from=member_profile_doc_id to=id}language_noun:english
: ({!join from=member_profile_doc_id to=id}language_proficiency_id_number:30
: {!join from=member_profile_doc_id to=id}language_proficiency_id_number:40
: {!join from=member_profile_doc_id to=id}language_proficiency_id_number:50))
...in this case, when the lucene parser sees the nested {!join...} parsers
it has no problem, because it's conservative rulees about the end of the
clauses match up with what you expect given the simple term queries. if
you change those individual term queries to a range query however...
+({!join from=member_profile_doc_id to=id}language_noun:english
{!join from=member_profile_doc_id to=id}language_proficiency_id_number:[30 TO
50])
...in this case, the lucene parser sees the "{!join} syntax and delegates
to the join parser, but it assumes only the
"language_proficiency_id_number:[30" portion of the input is ment for that
parser, and hangs on to the "TO 50]" to parse ass additional clauses. the
join parse doesn't really care about the input, but when it delegates to
another nested instance of the lucene parser, the input
"language_proficiency_id_number:[30" isn't valid because it's the start of
an unfinished range query.
Does that make sense so far?
As for the solution:
when you use the "{!foo} syntax, the local param "v" can be used to
specify the main input to the nested parser instead of the usual
prefix-ish syntax -- and this scopes the input un-ambiguiously...
+({!join from=member_profile_doc_id to=id v='language_noun:english'}
{!join from=member_profile_doc_id to=id v='language_proficiency_id_number:[30
TO 50]'})
FWIW, you can also use param derefrencing it it helps make things easier
to read for you (and/or if you need to include nested quotes and don't
want to deal with the escaping)...
q=+({!join from=$from to=id v=$noun} {!join from=$from to=id v=$prof)}
from=member_profile_doc_id
$noun=language_noun:english
$prof=language_profici
Re: Range query and join, oarse exception when parens are added
On 4/8/2014 22:00 GMT Shawn Heisey wrote:
>On 4/8/2014 1:48 PM, Mark Olsen wrote:
>> Solr version 4.2.1
>>
>> I'm having an issue using a "join" query with a range query, but only when
>> the query
is wrapped in parens.
>>
>> This query works:
>>
>> {!join from=member_profile_doc_id to=id}language_proficiency_id_number:[30
>> TO 50]
>>
>> However this query does not (just wrapping with parens):
>>
>> ({!join from=member_profile_doc_id to=id}language_proficiency_id_number:[30
>> TO 50])
>
> The {!join...} part of that is a localParam. It must precede the entire
> query. If you want to add parens, here's how you would need to do it:
>
> {!join from=member_profile_doc_id to=id}(language_proficiency_id_number:[30
> TO 50])
>
> With the left parenthesis where you placed it, the localParam is
> considered part of the query itself. It becomes incorrect Solr syntax
> at that point.
>
> Thanks,
> Shawn
Shawn,
Thank you for the response and I apologize for the delayed reply.
If I do the query with the localParam as you show it works, however as my
queries become more complex with multiple terms and fields I'm finding it
difficult to get the join localParam to work.
For example, when I was first playing with the above range query, I was able to
get this to work with nested localParams within the parenthesis:
+({!join from=member_profile_doc_id to=id}language_noun:english
({!join from=member_profile_doc_id to=id}language_proficiency_id_number:30
{!join from=member_profile_doc_id to=id}language_proficiency_id_number:40
{!join from=member_profile_doc_id to=id}language_proficiency_id_number:50))
This would work, but as soon as I used the range query with the square brackets
it would fail, hence why I thought was an issue.
As I've been trying queries it seemed to me that I need to have the localParam
before each field:term pair to get results.
Another example, this first query yields the results that I am expecting. Note
the parenthesis wrapping each localParam and field:term pair. In this case
there are two documents that both have the same "member_profile_doc_id" that
are matching the two two queries (language and certification).
+({!join from=member_profile_doc_id to=id}language_noun:english)
+({!join from=member_profile_doc_id to=id}certification_authority_id_number:50)
If I move the localParams outside and no longer declare them with each
field:term pair then no results are returned, example query:
{!join from=member_profile_doc_id to=id}(+language_noun:english
+certification_authority_id_number:50)
Unfortunately the documentation for the join localParams only gives a simple
field:term example so I've had to experiement with more complex queries to
figure out how to get it to work.
`Mark
Range query and join, oarse exception when parens are added
Solr version 4.2.1
I'm having an issue using a "join" query with a range query, but only when the
query is wrapped in parens.
This query works:
{!join from=member_profile_doc_id to=id}language_proficiency_id_number:[30 TO
50]
However this query does not (just wrapping with parens):
({!join from=member_profile_doc_id to=id}language_proficiency_id_number:[30 TO
50])
Caused by: org.apache.solr.parser.ParseException: Encountered "" at line
1, column 34.
Was expecting one of:
"TO" ...
...
...
at
org.apache.solr.parser.QueryParser.generateParseException(QueryParser.java:651)
at
org.apache.solr.parser.QueryParser.jj_consume_token(QueryParser.java:533)
at org.apache.solr.parser.QueryParser.Term(QueryParser.java:346)
at org.apache.solr.parser.QueryParser.Clause(QueryParser.java:186)
at org.apache.solr.parser.QueryParser.Query(QueryParser.java:108)
at org.apache.solr.parser.QueryParser.TopLevelQuery(QueryParser.java:97)
at
org.apache.solr.parser.SolrQueryParserBase.parse(SolrQueryParserBase.java:160)
... 45 more
Is this a known bug in this version of Solr (4.2.1)?
Thank you,
Mark
