RE: Help needed on Solr Streaming Expressions

2016-06-06 Thread Hui Liu 
The only difference between document3 and document5 is document3 has no data in 
'shard2', after loading some data into shard2, the http command also worked:

http://localhost:8988/solr/document3/stream?expr=search(document3,zkHost="127.0.0.1:2181",q="*:*",fl="document_id,
 sender_msg_dest", sort="document_id asc",qt="/export")

my guess is the 'null pointer' error from the stack trace is caused by no data 
in the 'shard2'.

Regards,
Hui

-Original Message-
From: Hui Liu 
Sent: Monday, June 06, 2016 1:04 PM
To: solr-user@lucene.apache.org
Subject: RE: Help needed on Solr Streaming Expressions

Joel,

Thank you very much for your help, I tried the http command below with my 
existing 2 shards collection 'document3' (sorry I have a typo below should be 
document3 instead of document2), this time I got much better error:

{"result-set":{"docs":[
{"EXCEPTION":"Unable to construct instance of 
org.apache.solr.client.solrj.io.stream.CloudSolrStream","EOF":true}]}}

I attach the error stack trace from 'solr-8988-console.log' and 'solr.log' here 
in file 'solr_error.txt'.

However I continued and tried create another identical collection 'document5' 
with 2 shards and 2 replica using the same schema, this time the http URL 
worked!!! Maybe my previous collection 'document3' has some corruption? 

-- command to create collection 'document5':
solr create -c document5 -d new_doc_configs5 -p 8988 -s 2 -rf 2

-- command for stream expression:
http://localhost:8988/solr/document5/stream?expr=search(document5,zkHost="127.0.0.1:2181",q="*:*",fl="document_id,
 sender_msg_dest", sort="document_id asc",qt="/export")

-- result from browser:
{"result-set":{"docs":[
{"document_id":20346005172,"sender_msg_dest":"ZZ:035239425"},
{"document_id":20346005173,"sender_msg_dest":"ZZ:035239425"},
{"document_id":20346006403,"sender_msg_dest":"14:004321519IBMP"},
{"document_id":20346006406,"sender_msg_dest":"14:004321519IBMP"},
{"document_id":20346006741,"sender_msg_dest":"14:004321519IBMP"},
{"document_id":20346006743,"sender_msg_dest":"14:004321519IBMP"},
{"EOF":true,"RESPONSE_TIME":10}]}}

Do you think I can try the same in http using other 'Stream Decorators' such as 
'complement' and 'innerJoin'?

Regards,
Hui

-Original Message-
From: Joel Bernstein [mailto:joels...@gmail.com]
Sent: Monday, June 06, 2016 9:51 AM
To: solr-user@lucene.apache.org
Subject: Re: Help needed on Solr Streaming Expressions

Hi,

To eliminate any issues that might be happening due to curl, try running the 
command from your browser.

http://localhost:8988/solr/document2/stream?expr=search(document3,zkHost="
127.0.0.1:2181",q="*:*",fl="document_id, sender_msg_dest", sort="document_id 
asc",qt="/export")



I think most browsers will url encode the expression automatically, but you can 
url encode also using an online tool. Also you can remove the zkHost param and 
it should default to zkHost your solr is connected to.


If you still get an error take a look at the logs and post the full stack trace 
to this thread, which will help determine where the problem is.



Joel Bernstein
http://joelsolr.blogspot.com/

On Sun, Jun 5, 2016 at 2:11 PM, Hui Liu <h...@opentext.com> wrote:

> Hi,
>
>
>
>   I have Solr 6.0.0 installed on my PC (windows 7), I was 
> experimenting with ‘Streaming Expression’ feature by following steps 
> from this link:
> https://cwiki.apache.org/confluence/display/solr/Streaming+Expressions
> , but cannot get it to work, attached is my solrconfig.xml and 
> schema.xml, note I do have ‘export’ handler defined in my 
> ‘solrconfig.xml’ and enabled all fields as ‘docvalues’ in 
> ‘schema.xml’; I am using solr cloud and external zookeeper (also 
> installed on m PC), here is the command to start this 2-node Solr 
> cloud instance and to create the collection ‘document3’:
>
>
>
> -- start 2-node solr cloud instances:
>
> solr start -c -z 127.0.0.1:2181 -p 8988 -s solr3
>
> solr start -c -z 127.0.0.1:2181 -p 8989 -s solr4
>
>
>
> -- create the collection:
>
> solr create -c document3 -d new_doc_configs3 -p 8988 -s 2 -rf 2
>
>
>
>   after creating the collection I loaded a few documents 
> using ‘csv’ format and I was able to query it using ‘curl’ command from my PC:
>
>
>
> -- this works on my PC:
>
> curl
> http://localhost:8988/solr/document3/select?q=*:*=document_id+des
> c,sender_msg_dest+desc=document_id,sender_msg_dest,recip_msg_dest
>
>
>
>

RE: Help needed on Solr Streaming Expressions

2016-06-06 Thread Hui Liu 
Joel,

Thank you very much for your help, I tried the http command below with my 
existing 2 shards collection 'document3' (sorry I have a typo below should be 
document3 instead of document2), this time I got much better error:

{"result-set":{"docs":[
{"EXCEPTION":"Unable to construct instance of 
org.apache.solr.client.solrj.io.stream.CloudSolrStream","EOF":true}]}}

I attach the error stack trace from 'solr-8988-console.log' and 'solr.log' here 
in file 'solr_error.txt'.

However I continued and tried create another identical collection 'document5' 
with 2 shards and 2 replica using the same schema, this time the http URL 
worked!!! Maybe my previous collection 'document3' has some corruption? 

-- command to create collection 'document5':
solr create -c document5 -d new_doc_configs5 -p 8988 -s 2 -rf 2

-- command for stream expression:
http://localhost:8988/solr/document5/stream?expr=search(document5,zkHost="127.0.0.1:2181",q="*:*",fl="document_id,
 sender_msg_dest", sort="document_id asc",qt="/export")

-- result from browser:
{"result-set":{"docs":[
{"document_id":20346005172,"sender_msg_dest":"ZZ:035239425"},
{"document_id":20346005173,"sender_msg_dest":"ZZ:035239425"},
{"document_id":20346006403,"sender_msg_dest":"14:004321519IBMP"},
{"document_id":20346006406,"sender_msg_dest":"14:004321519IBMP"},
{"document_id":20346006741,"sender_msg_dest":"14:004321519IBMP"},
{"document_id":20346006743,"sender_msg_dest":"14:004321519IBMP"},
{"EOF":true,"RESPONSE_TIME":10}]}}

Do you think I can try the same in http using other 'Stream Decorators' such as 
'complement' and 'innerJoin'?

Regards,
Hui

-Original Message-
From: Joel Bernstein [mailto:joels...@gmail.com] 
Sent: Monday, June 06, 2016 9:51 AM
To: solr-user@lucene.apache.org
Subject: Re: Help needed on Solr Streaming Expressions

Hi,

To eliminate any issues that might be happening due to curl, try running the 
command from your browser.

http://localhost:8988/solr/document2/stream?expr=search(document3,zkHost="
127.0.0.1:2181",q="*:*",fl="document_id, sender_msg_dest", sort="document_id 
asc",qt="/export")



I think most browsers will url encode the expression automatically, but you can 
url encode also using an online tool. Also you can remove the zkHost param and 
it should default to zkHost your solr is connected to.


If you still get an error take a look at the logs and post the full stack trace 
to this thread, which will help determine where the problem is.



Joel Bernstein
http://joelsolr.blogspot.com/

On Sun, Jun 5, 2016 at 2:11 PM, Hui Liu <h...@opentext.com> wrote:

> Hi,
>
>
>
>   I have Solr 6.0.0 installed on my PC (windows 7), I was 
> experimenting with ‘Streaming Expression’ feature by following steps 
> from this link:
> https://cwiki.apache.org/confluence/display/solr/Streaming+Expressions
> , but cannot get it to work, attached is my solrconfig.xml and 
> schema.xml, note I do have ‘export’ handler defined in my 
> ‘solrconfig.xml’ and enabled all fields as ‘docvalues’ in 
> ‘schema.xml’; I am using solr cloud and external zookeeper (also 
> installed on m PC), here is the command to start this 2-node Solr 
> cloud instance and to create the collection ‘document3’:
>
>
>
> -- start 2-node solr cloud instances:
>
> solr start -c -z 127.0.0.1:2181 -p 8988 -s solr3
>
> solr start -c -z 127.0.0.1:2181 -p 8989 -s solr4
>
>
>
> -- create the collection:
>
> solr create -c document3 -d new_doc_configs3 -p 8988 -s 2 -rf 2
>
>
>
>   after creating the collection I loaded a few documents 
> using ‘csv’ format and I was able to query it using ‘curl’ command from my PC:
>
>
>
> -- this works on my PC:
>
> curl
> http://localhost:8988/solr/document3/select?q=*:*=document_id+des
> c,sender_msg_dest+desc=document_id,sender_msg_dest,recip_msg_dest
>
>
>
>   but when trying Streaming ‘search’ using curl, it does 
> not work, I tried with 3 different options: with zkHost, using 
> ‘export’, or using ‘select’, all getting the same error:
>
>
> curl: (6) Could not resolve host: sort=document_id asc,qt=
>
> {"result-set":{"docs":[
>
> {"EXCEPTION":null,"EOF":true}]}}
>
> -- different curl commands tried, all getting the same error above:
>
> curl --data-urlencode 
> 'expr=search(document3,zkHost="127.0.0.1:2181",q="*:*",fl="document_id
> , sender_msg_dest", sort="document_id asc&q

Re: Help needed on Solr Streaming Expressions

2016-06-06 Thread Joel Bernstein 
Hi,

To eliminate any issues that might be happening due to curl, try running
the command from your browser.

http://localhost:8988/solr/document2/stream?expr=search(document3,zkHost="
127.0.0.1:2181",q="*:*",fl="document_id, sender_msg_dest",
sort="document_id asc",qt="/export")



I think most browsers will url encode the expression automatically, but you
can url encode also using an online tool. Also you can remove the zkHost
param and it should default to zkHost your solr is connected to.


If you still get an error take a look at the logs and post the full stack
trace to this thread, which will help determine where the problem is.



Joel Bernstein
http://joelsolr.blogspot.com/

On Sun, Jun 5, 2016 at 2:11 PM, Hui Liu  wrote:

> Hi,
>
>
>
>   I have Solr 6.0.0 installed on my PC (windows 7), I was
> experimenting with ‘Streaming Expression’ feature by following steps from
> this link:
> https://cwiki.apache.org/confluence/display/solr/Streaming+Expressions,
> but cannot get it to work, attached is my solrconfig.xml and schema.xml,
> note I do have ‘export’ handler defined in my ‘solrconfig.xml’ and enabled
> all fields as ‘docvalues’ in ‘schema.xml’; I am using solr cloud and
> external zookeeper (also installed on m PC), here is the command to start
> this 2-node Solr cloud instance and to create the collection ‘document3’:
>
>
>
> -- start 2-node solr cloud instances:
>
> solr start -c -z 127.0.0.1:2181 -p 8988 -s solr3
>
> solr start -c -z 127.0.0.1:2181 -p 8989 -s solr4
>
>
>
> -- create the collection:
>
> solr create -c document3 -d new_doc_configs3 -p 8988 -s 2 -rf 2
>
>
>
>   after creating the collection I loaded a few documents using
> ‘csv’ format and I was able to query it using ‘curl’ command from my PC:
>
>
>
> -- this works on my PC:
>
> curl
> http://localhost:8988/solr/document3/select?q=*:*=document_id+desc,sender_msg_dest+desc=document_id,sender_msg_dest,recip_msg_dest
>
>
>
>   but when trying Streaming ‘search’ using curl, it does not
> work, I tried with 3 different options: with zkHost, using ‘export’, or
> using ‘select’, all getting the same error:
>
>
> curl: (6) Could not resolve host: sort=document_id asc,qt=
>
> {"result-set":{"docs":[
>
> {"EXCEPTION":null,"EOF":true}]}}
>
> -- different curl commands tried, all getting the same error above:
>
> curl --data-urlencode 
> 'expr=search(document3,zkHost="127.0.0.1:2181",q="*:*",fl="document_id,
> sender_msg_dest", sort="document_id asc",qt="/export")' "
> http://localhost:8988/solr/document2/stream;
>
>
>
> curl --data-urlencode 'expr=search(document3,q="*:*",fl="document_id,
> sender_msg_dest", sort="document_id asc",qt="/export")' "
> http://localhost:8988/solr/document2/stream;
>
>
>
> curl --data-urlencode 'expr=search(document3,q="*:*",fl="document_id,
> sender_msg_dest", sort="document_id asc",qt="/select",rows=10)' "
> http://localhost:8988/solr/document2/stream;
>
>
>
>   what am I doing wrong? Thanks for any help!
>
>
>
> Regards,
>
> Hui Liu
>