Re: Exception when processing streaming expression

2018-07-24 Thread Christian Spitzlay
Hi,


> Am 15.06.2018 um 14:54 schrieb Christian Spitzlay 
> :
> 
> 
>> Am 15.06.2018 um 01:23 schrieb Joel Bernstein :
>> 
>> We have to check the behavior of the innerJoin. I suspect that its closing
>> the second stream when the first stream his finished. This would cause a
>> broken pipe with the second stream. The export handler has specific code
>> that eats the broken pipe exception so it doesn't end up in the logs. The
>> select hander does not have this code.
> 
> Ah, I see.  The stack trace in my original mail has the "broken pipe" message:
> 
> [...]
> Caused by: java.io.IOException: Broken pipe
>   at java.base/sun.nio.ch.FileDispatcherImpl.writev0(Native Method)
> [...]



Should I open a Jira ticket about the innerJoin issue?




>> In general you never want to use the select handler and set the rows to
>> such a big number. If you have that many rows you'll want to use the export
>> and handler which is designed to export the entire result set.
> 
> 
> We started out with the export handler but we are updating documents using 
> streaming expressions and we had fields that had types 
> that do not support docValues, according to the documentation at
> https://lucene.apache.org/solr/guide/7_3/docvalues.html#enabling-docvalues
> 
> We switched to the select handler in some places and it worked. 
> We set the rows parameter to a large value:
> "If you want to tell Solr to return all possible results from the query 
> without an 
> upper bound, specify rows to be 1000 or some other ridiculously 
> large value that is higher than the possible number of rows that are 
> expected."
> From:
> https://wiki.apache.org/solr/CommonQueryParameters#rows



Since we have trouble switching back to the export handler,
do you have any ideas how we could temporarily keep this exception from 
filling the solr log file when I run my code?


Christian



--  

Christian Spitzlay
Diplom-Physiker,
Senior Software-Entwickler

Tel: +49 69 / 348739116
E-Mail: christian.spitz...@biologis.com

bio.logis Genetic Information Management GmbH
Altenhöferallee 3
60438 Frankfurt am Main

Geschäftsführung: Prof. Dr. med. Daniela Steinberger, Dipl.Betriebswirt Enrico 
Just
Firmensitz Frankfurt am Main, Registergericht Frankfurt am Main, HRB 97945
Umsatzsteuer-Identifikationsnummer DE293587677




Re: Exception when processing streaming expression

2018-06-15 Thread Christian Spitzlay


> Am 15.06.2018 um 01:23 schrieb Joel Bernstein :
> 
> We have to check the behavior of the innerJoin. I suspect that its closing
> the second stream when the first stream his finished. This would cause a
> broken pipe with the second stream. The export handler has specific code
> that eats the broken pipe exception so it doesn't end up in the logs. The
> select hander does not have this code.

Ah, I see.  The stack trace in my original mail has the "broken pipe" message:

[...]
Caused by: java.io.IOException: Broken pipe
at java.base/sun.nio.ch.FileDispatcherImpl.writev0(Native Method)
[...]


> In general you never want to use the select handler and set the rows to
> such a big number. If you have that many rows you'll want to use the export
> and handler which is designed to export the entire result set.


We started out with the export handler but we are updating documents using 
streaming expressions and we had fields that had types 
that do not support docValues, according to the documentation at
https://lucene.apache.org/solr/guide/7_3/docvalues.html#enabling-docvalues

We switched to the select handler in some places and it worked. 
We set the rows parameter to a large value:
"If you want to tell Solr to return all possible results from the query without 
an 
upper bound, specify rows to be 1000 or some other ridiculously 
large value that is higher than the possible number of rows that are expected."
From:
https://wiki.apache.org/solr/CommonQueryParameters#rows


Christian Spitzlay





Re: Exception when processing streaming expression

2018-06-14 Thread Joel Bernstein
We have to check the behavior of the innerJoin. I suspect that its closing
the second stream when the first stream his finished. This would cause a
broken pipe with the second stream. The export handler has specific code
that eats the broken pipe exception so it doesn't end up in the logs. The
select hander does not have this code.

In general you never want to use the select handler and set the rows to
such a big number. If you have that many rows you'll want to use the export
and handler which is designed to export the entire result set.

Joel Bernstein
http://joelsolr.blogspot.com/

On Thu, Jun 14, 2018 at 1:30 PM, Christian Spitzlay <
christian.spitz...@biologis.com> wrote:

> What does that mean exactly?  If I set the rows parameter to 10
> the exception still occurs.  AFAICT all this happens internally during the
> processing of the streaming expression.  Why wouldn't the select send
> the EOF tuple when it reaches the end of the documents?
> Or why wouldn't the receiving end wait for it to appear?
> Due to an incredibly low timeout used internally?
>
>
> Christian Spitzlay
>
>
>
> > Am 14.06.2018 um 19:18 schrieb Susmit :
> >
> > Hi,
> > This may be expected if one of the streams is closed early - does not
> reach to EOF tuple
> >
> > Sent from my iPhone
> >
> >> On Jun 14, 2018, at 9:53 AM, Christian Spitzlay <
> christian.spitz...@biologis.com> wrote:
> >>
> >> Here ist one I stripped down as far as I could:
> >>
> >> innerJoin(sort(search(kmm, q="sds_endpoint_uuid:(
> 2f927a0b\-fe38\-451e\-9103\-580914a77e82)", 
> fl="sds_endpoint_uuid,sds_to_endpoint_uuid",
> sort="sds_to_endpoint_uuid ASC", qt="/export"), by="sds_endpoint_uuid
> ASC"), search(kmm, q=ss_search_api_datasource:entity\:as_metadata,
> fl="sds_metadata_of_uuid", sort="sds_metadata_of_uuid ASC", qt="/select",
> rows=1), on="sds_endpoint_uuid=sds_metadata_of_uuid")
> >>
> >> The exception happens both via PHP (search_api_solr / Solarium) and via
> the Solr admin UI.
> >> (version: Solr 7.3.1 on macOS High Sierra 10.13.5)
> >>
> >> It seems to be related to the fact that the second stream uses
> "select“.
> >> - If I use "export“ the exception doesn’t occur.
> >> - If I set the rows parameter "low enough“ so I do not get any results
> >> the exception doesn’t occur either.
> >>
> >>
> >> BTW: Do you know of any tool for formatting and/or syntax highlighting
> >> these expressions?
> >>
> >>
> >> Christian Spitzlay
> >>
> >>
> >>
> >>
> >>
> >>> Am 13.06.2018 um 23:02 schrieb Joel Bernstein :
> >>>
> >>> Can your provide some example expressions that are causing these
> exceptions?
> >>>
> >>> Joel Bernstein
> >>> http://joelsolr.blogspot.com/
> >>>
> >>> On Wed, Jun 13, 2018 at 9:02 AM, Christian Spitzlay <
> >>> christian.spitz...@biologis.com> wrote:
> >>>
>  Hi,
> 
>  I am seeing a lot of (reproducible) exceptions in my solr log file
>  when I execute streaming expressions:
> 
>  o.a.s.s.HttpSolrCall  Unable to write response, client closed
> connection
>  or we are shutting down
>  org.eclipse.jetty.io.EofException
>   at org.eclipse.jetty.io.ChannelEndPoint.flush(
>  ChannelEndPoint.java:292)
>   at org.eclipse.jetty.io.WriteFlusher.flush(
> WriteFlusher.java:429)
>   at org.eclipse.jetty.io.WriteFlusher.write(
> WriteFlusher.java:322)
>   at org.eclipse.jetty.io.AbstractEndPoint.write(
>  AbstractEndPoint.java:372)
>   at org.eclipse.jetty.server.HttpConnection$SendCallback.
>  process(HttpConnection.java:794)
>  […]
>   at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(
>  EatWhatYouKill.java:131)
>   at org.eclipse.jetty.util.thread.ReservedThreadExecutor$
>  ReservedThread.run(ReservedThreadExecutor.java:382)
>   at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
>  QueuedThreadPool.java:708)
>   at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
>  QueuedThreadPool.java:626)
>   at java.base/java.lang.Thread.run(Thread.java:844)
>  Caused by: java.io.IOException: Broken pipe
>   at java.base/sun.nio.ch.FileDispatcherImpl.writev0(Native
> Method)
>   at java.base/sun.nio.ch.SocketDispatcher.writev(
>  SocketDispatcher.java:51)
>   at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:148)
>   at java.base/sun.nio.ch.SocketChannelImpl.write(
>  SocketChannelImpl.java:506)
>   at org.eclipse.jetty.io.ChannelEndPoint.flush(
>  ChannelEndPoint.java:272)
>   ... 69 more
> 
> 
>  I have read up on the exception message and found
>  http://lucene.472066.n3.nabble.com/Unable-to-write-
> response-client-closed-
>  connection-or-we-are-shutting-down-tt4350349.html#a4350947
>  but I don’t understand how an early client connect can cause what I am
>  seeing:
> 
>  What puzzles me is that the response has been delivered in full to the
>  client library, including 

Re: Exception when processing streaming expression

2018-06-14 Thread Christian Spitzlay
What does that mean exactly?  If I set the rows parameter to 10
the exception still occurs.  AFAICT all this happens internally during the 
processing of the streaming expression.  Why wouldn't the select send 
the EOF tuple when it reaches the end of the documents? 
Or why wouldn't the receiving end wait for it to appear?  
Due to an incredibly low timeout used internally?


Christian Spitzlay



> Am 14.06.2018 um 19:18 schrieb Susmit :
> 
> Hi, 
> This may be expected if one of the streams is closed early - does not reach 
> to EOF tuple
> 
> Sent from my iPhone
> 
>> On Jun 14, 2018, at 9:53 AM, Christian Spitzlay 
>>  wrote:
>> 
>> Here ist one I stripped down as far as I could:
>> 
>> innerJoin(sort(search(kmm, 
>> q="sds_endpoint_uuid:(2f927a0b\-fe38\-451e\-9103\-580914a77e82)", 
>> fl="sds_endpoint_uuid,sds_to_endpoint_uuid", sort="sds_to_endpoint_uuid 
>> ASC", qt="/export"), by="sds_endpoint_uuid ASC"), search(kmm, 
>> q=ss_search_api_datasource:entity\:as_metadata, fl="sds_metadata_of_uuid", 
>> sort="sds_metadata_of_uuid ASC", qt="/select", rows=1), 
>> on="sds_endpoint_uuid=sds_metadata_of_uuid")
>> 
>> The exception happens both via PHP (search_api_solr / Solarium) and via the 
>> Solr admin UI.
>> (version: Solr 7.3.1 on macOS High Sierra 10.13.5)
>> 
>> It seems to be related to the fact that the second stream uses "select“. 
>> - If I use "export“ the exception doesn’t occur.
>> - If I set the rows parameter "low enough“ so I do not get any results
>> the exception doesn’t occur either.
>> 
>> 
>> BTW: Do you know of any tool for formatting and/or syntax highlighting 
>> these expressions?
>> 
>> 
>> Christian Spitzlay
>> 
>> 
>> 
>> 
>> 
>>> Am 13.06.2018 um 23:02 schrieb Joel Bernstein :
>>> 
>>> Can your provide some example expressions that are causing these exceptions?
>>> 
>>> Joel Bernstein
>>> http://joelsolr.blogspot.com/
>>> 
>>> On Wed, Jun 13, 2018 at 9:02 AM, Christian Spitzlay <
>>> christian.spitz...@biologis.com> wrote:
>>> 
 Hi,
 
 I am seeing a lot of (reproducible) exceptions in my solr log file
 when I execute streaming expressions:
 
 o.a.s.s.HttpSolrCall  Unable to write response, client closed connection
 or we are shutting down
 org.eclipse.jetty.io.EofException
  at org.eclipse.jetty.io.ChannelEndPoint.flush(
 ChannelEndPoint.java:292)
  at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:429)
  at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:322)
  at org.eclipse.jetty.io.AbstractEndPoint.write(
 AbstractEndPoint.java:372)
  at org.eclipse.jetty.server.HttpConnection$SendCallback.
 process(HttpConnection.java:794)
 […]
  at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(
 EatWhatYouKill.java:131)
  at org.eclipse.jetty.util.thread.ReservedThreadExecutor$
 ReservedThread.run(ReservedThreadExecutor.java:382)
  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
 QueuedThreadPool.java:708)
  at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
 QueuedThreadPool.java:626)
  at java.base/java.lang.Thread.run(Thread.java:844)
 Caused by: java.io.IOException: Broken pipe
  at java.base/sun.nio.ch.FileDispatcherImpl.writev0(Native Method)
  at java.base/sun.nio.ch.SocketDispatcher.writev(
 SocketDispatcher.java:51)
  at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:148)
  at java.base/sun.nio.ch.SocketChannelImpl.write(
 SocketChannelImpl.java:506)
  at org.eclipse.jetty.io.ChannelEndPoint.flush(
 ChannelEndPoint.java:272)
  ... 69 more
 
 
 I have read up on the exception message and found
 http://lucene.472066.n3.nabble.com/Unable-to-write-response-client-closed-
 connection-or-we-are-shutting-down-tt4350349.html#a4350947
 but I don’t understand how an early client connect can cause what I am
 seeing:
 
 What puzzles me is that the response has been delivered in full to the
 client library, including the document with EOF.
 
 So Solr must have already processed the streaming expression and returned
 the result.
 It’s just that the log is filled with stacktraces of this exception that
 suggests something went wrong.
 I don’t understand why this happens when the query seems to have succeeded.
 
 
 Best regards,
 Christian
 
 
 
>> 



Re: Exception when processing streaming expression

2018-06-14 Thread Susmit
Hi, 
This may be expected if one of the streams is closed early - does not reach to 
EOF tuple

Sent from my iPhone

> On Jun 14, 2018, at 9:53 AM, Christian Spitzlay 
>  wrote:
> 
> Here ist one I stripped down as far as I could:
> 
> innerJoin(sort(search(kmm, 
> q="sds_endpoint_uuid:(2f927a0b\-fe38\-451e\-9103\-580914a77e82)", 
> fl="sds_endpoint_uuid,sds_to_endpoint_uuid", sort="sds_to_endpoint_uuid ASC", 
> qt="/export"), by="sds_endpoint_uuid ASC"), search(kmm, 
> q=ss_search_api_datasource:entity\:as_metadata, fl="sds_metadata_of_uuid", 
> sort="sds_metadata_of_uuid ASC", qt="/select", rows=1), 
> on="sds_endpoint_uuid=sds_metadata_of_uuid")
> 
> The exception happens both via PHP (search_api_solr / Solarium) and via the 
> Solr admin UI.
> (version: Solr 7.3.1 on macOS High Sierra 10.13.5)
> 
> It seems to be related to the fact that the second stream uses "select“. 
> - If I use "export“ the exception doesn’t occur.
> - If I set the rows parameter "low enough“ so I do not get any results
>  the exception doesn’t occur either.
> 
> 
> BTW: Do you know of any tool for formatting and/or syntax highlighting 
> these expressions?
> 
> 
> Christian Spitzlay
> 
> 
> 
> 
> 
>> Am 13.06.2018 um 23:02 schrieb Joel Bernstein :
>> 
>> Can your provide some example expressions that are causing these exceptions?
>> 
>> Joel Bernstein
>> http://joelsolr.blogspot.com/
>> 
>> On Wed, Jun 13, 2018 at 9:02 AM, Christian Spitzlay <
>> christian.spitz...@biologis.com> wrote:
>> 
>>> Hi,
>>> 
>>> I am seeing a lot of (reproducible) exceptions in my solr log file
>>> when I execute streaming expressions:
>>> 
>>> o.a.s.s.HttpSolrCall  Unable to write response, client closed connection
>>> or we are shutting down
>>> org.eclipse.jetty.io.EofException
>>>   at org.eclipse.jetty.io.ChannelEndPoint.flush(
>>> ChannelEndPoint.java:292)
>>>   at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:429)
>>>   at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:322)
>>>   at org.eclipse.jetty.io.AbstractEndPoint.write(
>>> AbstractEndPoint.java:372)
>>>   at org.eclipse.jetty.server.HttpConnection$SendCallback.
>>> process(HttpConnection.java:794)
>>> […]
>>>   at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(
>>> EatWhatYouKill.java:131)
>>>   at org.eclipse.jetty.util.thread.ReservedThreadExecutor$
>>> ReservedThread.run(ReservedThreadExecutor.java:382)
>>>   at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
>>> QueuedThreadPool.java:708)
>>>   at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
>>> QueuedThreadPool.java:626)
>>>   at java.base/java.lang.Thread.run(Thread.java:844)
>>> Caused by: java.io.IOException: Broken pipe
>>>   at java.base/sun.nio.ch.FileDispatcherImpl.writev0(Native Method)
>>>   at java.base/sun.nio.ch.SocketDispatcher.writev(
>>> SocketDispatcher.java:51)
>>>   at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:148)
>>>   at java.base/sun.nio.ch.SocketChannelImpl.write(
>>> SocketChannelImpl.java:506)
>>>   at org.eclipse.jetty.io.ChannelEndPoint.flush(
>>> ChannelEndPoint.java:272)
>>>   ... 69 more
>>> 
>>> 
>>> I have read up on the exception message and found
>>> http://lucene.472066.n3.nabble.com/Unable-to-write-response-client-closed-
>>> connection-or-we-are-shutting-down-tt4350349.html#a4350947
>>> but I don’t understand how an early client connect can cause what I am
>>> seeing:
>>> 
>>> What puzzles me is that the response has been delivered in full to the
>>> client library, including the document with EOF.
>>> 
>>> So Solr must have already processed the streaming expression and returned
>>> the result.
>>> It’s just that the log is filled with stacktraces of this exception that
>>> suggests something went wrong.
>>> I don’t understand why this happens when the query seems to have succeeded.
>>> 
>>> 
>>> Best regards,
>>> Christian
>>> 
>>> 
>>> 
> 


Re: Exception when processing streaming expression

2018-06-14 Thread Christian Spitzlay
Here ist one I stripped down as far as I could:

innerJoin(sort(search(kmm, 
q="sds_endpoint_uuid:(2f927a0b\-fe38\-451e\-9103\-580914a77e82)", 
fl="sds_endpoint_uuid,sds_to_endpoint_uuid", sort="sds_to_endpoint_uuid ASC", 
qt="/export"), by="sds_endpoint_uuid ASC"), search(kmm, 
q=ss_search_api_datasource:entity\:as_metadata, fl="sds_metadata_of_uuid", 
sort="sds_metadata_of_uuid ASC", qt="/select", rows=1), 
on="sds_endpoint_uuid=sds_metadata_of_uuid")

The exception happens both via PHP (search_api_solr / Solarium) and via the 
Solr admin UI.
(version: Solr 7.3.1 on macOS High Sierra 10.13.5)

It seems to be related to the fact that the second stream uses "select“. 
- If I use "export“ the exception doesn’t occur.
- If I set the rows parameter "low enough“ so I do not get any results
  the exception doesn’t occur either.


BTW: Do you know of any tool for formatting and/or syntax highlighting 
these expressions?


Christian Spitzlay





> Am 13.06.2018 um 23:02 schrieb Joel Bernstein :
> 
> Can your provide some example expressions that are causing these exceptions?
> 
> Joel Bernstein
> http://joelsolr.blogspot.com/
> 
> On Wed, Jun 13, 2018 at 9:02 AM, Christian Spitzlay <
> christian.spitz...@biologis.com> wrote:
> 
>> Hi,
>> 
>> I am seeing a lot of (reproducible) exceptions in my solr log file
>> when I execute streaming expressions:
>> 
>> o.a.s.s.HttpSolrCall  Unable to write response, client closed connection
>> or we are shutting down
>> org.eclipse.jetty.io.EofException
>>at org.eclipse.jetty.io.ChannelEndPoint.flush(
>> ChannelEndPoint.java:292)
>>at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:429)
>>at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:322)
>>at org.eclipse.jetty.io.AbstractEndPoint.write(
>> AbstractEndPoint.java:372)
>>at org.eclipse.jetty.server.HttpConnection$SendCallback.
>> process(HttpConnection.java:794)
>> […]
>>at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(
>> EatWhatYouKill.java:131)
>>at org.eclipse.jetty.util.thread.ReservedThreadExecutor$
>> ReservedThread.run(ReservedThreadExecutor.java:382)
>>at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
>> QueuedThreadPool.java:708)
>>at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
>> QueuedThreadPool.java:626)
>>at java.base/java.lang.Thread.run(Thread.java:844)
>> Caused by: java.io.IOException: Broken pipe
>>at java.base/sun.nio.ch.FileDispatcherImpl.writev0(Native Method)
>>at java.base/sun.nio.ch.SocketDispatcher.writev(
>> SocketDispatcher.java:51)
>>at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:148)
>>at java.base/sun.nio.ch.SocketChannelImpl.write(
>> SocketChannelImpl.java:506)
>>at org.eclipse.jetty.io.ChannelEndPoint.flush(
>> ChannelEndPoint.java:272)
>>... 69 more
>> 
>> 
>> I have read up on the exception message and found
>> http://lucene.472066.n3.nabble.com/Unable-to-write-response-client-closed-
>> connection-or-we-are-shutting-down-tt4350349.html#a4350947
>> but I don’t understand how an early client connect can cause what I am
>> seeing:
>> 
>> What puzzles me is that the response has been delivered in full to the
>> client library, including the document with EOF.
>> 
>> So Solr must have already processed the streaming expression and returned
>> the result.
>> It’s just that the log is filled with stacktraces of this exception that
>> suggests something went wrong.
>> I don’t understand why this happens when the query seems to have succeeded.
>> 
>> 
>> Best regards,
>> Christian
>> 
>> 
>> 



Re: Exception when processing streaming expression

2018-06-13 Thread Joel Bernstein
Can your provide some example expressions that are causing these exceptions?

Joel Bernstein
http://joelsolr.blogspot.com/

On Wed, Jun 13, 2018 at 9:02 AM, Christian Spitzlay <
christian.spitz...@biologis.com> wrote:

> Hi,
>
> I am seeing a lot of (reproducible) exceptions in my solr log file
> when I execute streaming expressions:
>
> o.a.s.s.HttpSolrCall  Unable to write response, client closed connection
> or we are shutting down
> org.eclipse.jetty.io.EofException
> at org.eclipse.jetty.io.ChannelEndPoint.flush(
> ChannelEndPoint.java:292)
> at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:429)
> at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:322)
> at org.eclipse.jetty.io.AbstractEndPoint.write(
> AbstractEndPoint.java:372)
> at org.eclipse.jetty.server.HttpConnection$SendCallback.
> process(HttpConnection.java:794)
> […]
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(
> EatWhatYouKill.java:131)
> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$
> ReservedThread.run(ReservedThreadExecutor.java:382)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
> QueuedThreadPool.java:708)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
> QueuedThreadPool.java:626)
> at java.base/java.lang.Thread.run(Thread.java:844)
> Caused by: java.io.IOException: Broken pipe
> at java.base/sun.nio.ch.FileDispatcherImpl.writev0(Native Method)
> at java.base/sun.nio.ch.SocketDispatcher.writev(
> SocketDispatcher.java:51)
> at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:148)
> at java.base/sun.nio.ch.SocketChannelImpl.write(
> SocketChannelImpl.java:506)
> at org.eclipse.jetty.io.ChannelEndPoint.flush(
> ChannelEndPoint.java:272)
> ... 69 more
>
>
> I have read up on the exception message and found
> http://lucene.472066.n3.nabble.com/Unable-to-write-response-client-closed-
> connection-or-we-are-shutting-down-tt4350349.html#a4350947
> but I don’t understand how an early client connect can cause what I am
> seeing:
>
> What puzzles me is that the response has been delivered in full to the
> client library, including the document with EOF.
>
> So Solr must have already processed the streaming expression and returned
> the result.
> It’s just that the log is filled with stacktraces of this exception that
> suggests something went wrong.
> I don’t understand why this happens when the query seems to have succeeded.
>
>
> Best regards,
> Christian
>
>
>


Exception when processing streaming expression

2018-06-13 Thread Christian Spitzlay
Hi,

I am seeing a lot of (reproducible) exceptions in my solr log file
when I execute streaming expressions:

o.a.s.s.HttpSolrCall  Unable to write response, client closed connection or we 
are shutting down
org.eclipse.jetty.io.EofException
at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:292)
at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:429)
at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:322)
at 
org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:372)
at 
org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:794)
[…]
at 
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.io.IOException: Broken pipe
at java.base/sun.nio.ch.FileDispatcherImpl.writev0(Native Method)
at 
java.base/sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51)
at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:148)
at 
java.base/sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:506)
at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:272)
... 69 more


I have read up on the exception message and found 
http://lucene.472066.n3.nabble.com/Unable-to-write-response-client-closed-connection-or-we-are-shutting-down-tt4350349.html#a4350947
but I don’t understand how an early client connect can cause what I am seeing:

What puzzles me is that the response has been delivered in full to the 
client library, including the document with EOF.

So Solr must have already processed the streaming expression and returned the 
result.
It’s just that the log is filled with stacktraces of this exception that 
suggests something went wrong.
I don’t understand why this happens when the query seems to have succeeded.


Best regards,
Christian