Re: fts_solr: Error: fts_solr: received invalid uid '0'

[Aki Tuomi via dovecot]

Lets hope 2.3.10 ...

Aki

> On 10/12/2019 20:57 Larry Rosenman via dovecot  wrote:
> 
> 
> Any news on this getting into a release?
> 
> 
> On Sun, Oct 13, 2019 at 2:24 AM Stephan Bosch  wrote:
> > 
> >  
> >  On 10/10/2019 04:05, Larry Rosenman via dovecot wrote:
> >  > Is there a way to get this as a patchset against 2.3.8?
> >  >
> >  
> >  https://github.com/stephanbosch/dovecot-core/tree/fix-solr-xml-parser-2.3.8
> >  
> >  This hasn't seen any QA yet, so handle with care.
> >  
> >  Regards,
> >  
> >  Stephan.
> >  
> >  >
> >  > On Tue, Oct 8, 2019 at 12:03 PM Stephan Bosch via dovecot 
> >  > mailto:dovecot@dovecot.org>> wrote:
> >  >
> >  >
> >  >
> >  > On 08/10/2019 02:52, Stephan Bosch via dovecot wrote:
> >  > >
> >  > >
> >  > > On 07/10/2019 15:13, Fabian Kuran via dovecot wrote:
> >  > >> Hi,
> >  > >>
> >  > >>> Am 23.09.2019 um 09:36 schrieb Filip Hanes
> >  > mailto:filip.ha...@ringier.sk>
> >  > >>> >>:
> >  > >>>
> >  > >>>
> >  > >>> št 19. 9. 2019 o 15:30 Fabian via dovecot  >  > 
> >  > >>> >>
> >  > napísal(a):
> >  > >>>
> >  > >>> Are there any other hints or tips regarding this „invalid uid
> >  > >>> ‚0‘"-message?
> >  > >>>
> >  > >>> from fts-solr source:
> >  > >>> case SOLR_XML_CONTENT_STATE_UID:
> >  > >>>
> >  > if (uint32_parse(str, len, &ctx->uid) < 0 || ctx->uid == 0) {
> >  > >>>
> >  > i_error("fts_solr: received invalid uid '%s'", t_strndup(str, len));
> >  >
> >  > >>>
> >  > >>> It means that plugin tries to parse '0' when in state of
> >  > parsing uid
> >  > >>> in xml response from solr or there is a bug in xml response
> >  > parser.
> >  > >>> I would enable rawlog_dir in config and look there for unusual
> >  > >>> response or extra tags near usual uid in xml response. For
> >  > example:
> >  > >>> |plugin { fts = solr fts_solr = url=...
> >  > >>> rawlog_dir=/var/log/fts-solr/ }|
> >  > >>>
> >  > >>
> >  > >> Thanks for this tip! The rawlog_dir option is not yet supported in
> >  > >> our version 2.3.4. What we could provide now would be the
> >  > >> XML-Response from Solr. The corresponding search query sometimes
> >  > >> causes this error. But Solr's answer seems to always remain the
> >  > same
> >  > >> - in my opinion this already indicates a bug of Dovecot?
> >  > >>
> >  > >> The XML response can be downloaded here:
> >  > >> https://owncloud.eideo.de/index.php/s/8oyJaf5HBK6zdbr
> >  > >
> >  > > Based on the XML response above, I investigated this problem
> >  > > thoroughly and determined that this is a pretty severe bug in
> >  > the Solr
> >  > > XML response parsing code. This occurs only when the response is
> >  > > rather large and the boundary between two read chunks falls in the
> >  > > middle of a numeric value (that happens to end in '0').
> >  > >
> >  > > Tracking internally as DOP-1470.
> >  > >
> >  > > I think this may also explain some of the weird reports about
> >  > invalid
> >  > > UIDs that we've seen in recent history on this mailing list.
> >  >
> >  > Preliminary fix is here:
> >  > https://github.com/stephanbosch/dovecot-core/commits/fix-solr-xml-parser
> >  >
> >  > It's big due to some code restructuring for making a proper unit
> >  > test. I
> >  > am not sure when this will be in a release.
> >  >
> >  > Regards,
> >  >
> >  > Stephan.
> >  >
> >  >
> >  >
> >  > -- 
> >  > Larry Rosenman http://www.lerctr.org/~ler
> >  > Phone: +1 214-642-9640 (c) E-Mail: larry...@gmail.com 
> >  > 
> >  > US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
> >  
> > 
> 
> 
> 
> -- 
> 
> Larry Rosenman http://www.lerctr.org/~ler
> Phone: +1 214-642-9640 (c) E-Mail:larry...@gmail.com
> US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106

Re: fts_solr: Error: fts_solr: received invalid uid '0'

[Larry Rosenman via dovecot]

Any news on this getting into a release?

On Sun, Oct 13, 2019 at 2:24 AM Stephan Bosch  wrote:

>
>
> On 10/10/2019 04:05, Larry Rosenman via dovecot wrote:
> > Is there a way to get this as a patchset against 2.3.8?
> >
>
> https://github.com/stephanbosch/dovecot-core/tree/fix-solr-xml-parser-2.3.8
>
> This hasn't seen any QA yet, so handle with care.
>
> Regards,
>
> Stephan.
>
> >
> > On Tue, Oct 8, 2019 at 12:03 PM Stephan Bosch via dovecot
> > mailto:dovecot@dovecot.org>> wrote:
> >
> >
> >
> > On 08/10/2019 02:52, Stephan Bosch via dovecot wrote:
> > >
> > >
> > > On 07/10/2019 15:13, Fabian Kuran via dovecot wrote:
> > >> Hi,
> > >>
> > >>> Am 23.09.2019 um 09:36 schrieb Filip Hanes
> > mailto:filip.ha...@ringier.sk>
> > >>>  >>>:
> > >>>
> > >>>
> > >>> št 19. 9. 2019 o 15:30 Fabian via dovecot  > 
> > >>> >>
> > napísal(a):
> > >>>
> > >>> Are there any other hints or tips regarding this „invalid uid
> > >>> ‚0‘"-message?
> > >>>
> > >>> from fts-solr source:
> > >>> case SOLR_XML_CONTENT_STATE_UID:
> > >>>
> > if (uint32_parse(str, len, &ctx->uid) < 0 || ctx->uid == 0) {
> > >>>
> >
>  i_error("fts_solr: received invalid uid '%s'", t_strndup(str, 
> len));
> >
> > >>>
> > >>> It means that plugin tries to parse '0' when in state of
> > parsing uid
> > >>> in xml response from solr or there is a bug in xml response
> > parser.
> > >>> I would enable rawlog_dir in config and look there for unusual
> > >>> response or extra tags near usual uid in xml response. For
> > example:
> > >>> |plugin { fts = solr fts_solr = url=...
> > >>> rawlog_dir=/var/log/fts-solr/ }|
> > >>>
> > >>
> > >> Thanks for this tip! The rawlog_dir option is not yet supported in
> > >> our version 2.3.4. What we could provide now would be the
> > >> XML-Response from Solr. The corresponding search query sometimes
> > >> causes this error. But Solr's answer seems to always remain the
> > same
> > >> - in my opinion this already indicates a bug of Dovecot?
> > >>
> > >> The XML response can be downloaded here:
> > >> https://owncloud.eideo.de/index.php/s/8oyJaf5HBK6zdbr
> > >
> > > Based on the XML response above, I investigated this problem
> > > thoroughly and determined that this is a pretty severe bug in
> > the Solr
> > > XML response parsing code. This occurs only when the response is
> > > rather large and the boundary between two read chunks falls in the
> > > middle of a numeric value (that happens to end in '0').
> > >
> > > Tracking internally as DOP-1470.
> > >
> > > I think this may also explain some of the weird reports about
> > invalid
> > > UIDs that we've seen in recent history on this mailing list.
> >
> > Preliminary fix is here:
> >
> https://github.com/stephanbosch/dovecot-core/commits/fix-solr-xml-parser
> >
> > It's big due to some code restructuring for making a proper unit
> > test. I
> > am not sure when this will be in a release.
> >
> > Regards,
> >
> > Stephan.
> >
> >
> >
> > --
> > Larry Rosenman http://www.lerctr.org/~ler
> > Phone: +1 214-642-9640 (c) E-Mail: larry...@gmail.com
> > 
> > US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
>
>

-- 
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 (c) E-Mail: larry...@gmail.com
US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106

Re: fts_solr: Error: fts_solr: received invalid uid '0'

[Stephan Bosch via dovecot]

On 10/10/2019 04:05, Larry Rosenman via dovecot wrote:

Is there a way to get this as a patchset against 2.3.8?



https://github.com/stephanbosch/dovecot-core/tree/fix-solr-xml-parser-2.3.8

This hasn't seen any QA yet, so handle with care.

Regards,

Stephan.



On Tue, Oct 8, 2019 at 12:03 PM Stephan Bosch via dovecot 
mailto:dovecot@dovecot.org>> wrote:




On 08/10/2019 02:52, Stephan Bosch via dovecot wrote:
>
>
> On 07/10/2019 15:13, Fabian Kuran via dovecot wrote:
>> Hi,
>>
>>> Am 23.09.2019 um 09:36 schrieb Filip Hanes
mailto:filip.ha...@ringier.sk>
>>> >>:
>>>
>>>
>>> št 19. 9. 2019 o 15:30 Fabian via dovecot mailto:dovecot@dovecot.org>
>>> >>
napísal(a):
>>>
>>>     Are there any other hints or tips regarding this „invalid uid
>>>     ‚0‘"-message?
>>>
>>> from fts-solr source:
>>> case SOLR_XML_CONTENT_STATE_UID:
>>>
if (uint32_parse(str, len, &ctx->uid) < 0 || ctx->uid == 0) {
>>>
i_error("fts_solr: received invalid uid '%s'", t_strndup(str, 
len));

>>>
>>> It means that plugin tries to parse '0' when in state of
parsing uid
>>> in xml response from solr or there is a bug in xml response
parser.
>>> I would enable rawlog_dir in config and look there for unusual
>>> response or extra tags near usual uid in xml response. For
example:
>>> |plugin { fts = solr fts_solr = url=...
>>> rawlog_dir=/var/log/fts-solr/ }|
>>>
>>
>> Thanks for this tip! The rawlog_dir option is not yet supported in
>> our version 2.3.4. What we could provide now would be the
>> XML-Response from Solr. The corresponding search query sometimes
>> causes this error. But Solr's answer seems to always remain the
same
>> - in my opinion this already indicates a bug of Dovecot?
>>
>> The XML response can be downloaded here:
>> https://owncloud.eideo.de/index.php/s/8oyJaf5HBK6zdbr
>
> Based on the XML response above, I investigated this problem
> thoroughly and determined that this is a pretty severe bug in
the Solr
> XML response parsing code. This occurs only when the response is
> rather large and the boundary between two read chunks falls in the
> middle of a numeric value (that happens to end in '0').
>
> Tracking internally as DOP-1470.
>
> I think this may also explain some of the weird reports about
invalid
> UIDs that we've seen in recent history on this mailing list.

Preliminary fix is here:
https://github.com/stephanbosch/dovecot-core/commits/fix-solr-xml-parser

It's big due to some code restructuring for making a proper unit
test. I
am not sure when this will be in a release.

Regards,

Stephan.



--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 (c) E-Mail: larry...@gmail.com 


US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106

Re: fts_solr: Error: fts_solr: received invalid uid '0'

[Larry Rosenman via dovecot]

Is there a way to get this as a patchset against 2.3.8?


On Tue, Oct 8, 2019 at 12:03 PM Stephan Bosch via dovecot <
dovecot@dovecot.org> wrote:

>
>
> On 08/10/2019 02:52, Stephan Bosch via dovecot wrote:
> >
> >
> > On 07/10/2019 15:13, Fabian Kuran via dovecot wrote:
> >> Hi,
> >>
> >>> Am 23.09.2019 um 09:36 schrieb Filip Hanes  >>> >:
> >>>
> >>>
> >>> št 19. 9. 2019 o 15:30 Fabian via dovecot  >>> > napísal(a):
> >>>
> >>> Are there any other hints or tips regarding this „invalid uid
> >>> ‚0‘"-message?
> >>>
> >>> from fts-solr source:
> >>> case SOLR_XML_CONTENT_STATE_UID:
> >>> if (uint32_parse(str, len, &ctx->uid) < 0 || ctx->uid == 0) {
> >>>
> i_error("fts_solr: received invalid uid '%s'", t_strndup(str, 
> len));
>
> >>>
> >>> It means that plugin tries to parse '0' when in state of parsing uid
> >>> in xml response from solr or there is a bug in xml response parser.
> >>> I would enable rawlog_dir in config and look there for unusual
> >>> response or extra tags near usual uid in xml response. For example:
> >>> |plugin { fts = solr fts_solr = url=...
> >>> rawlog_dir=/var/log/fts-solr/ }|
> >>>
> >>
> >> Thanks for this tip! The rawlog_dir option is not yet supported in
> >> our version 2.3.4. What we could provide now would be the
> >> XML-Response from Solr. The corresponding search query sometimes
> >> causes this error. But Solr's answer seems to always remain the same
> >> - in my opinion this already indicates a bug of Dovecot?
> >>
> >> The XML response can be downloaded here:
> >> https://owncloud.eideo.de/index.php/s/8oyJaf5HBK6zdbr
> >
> > Based on the XML response above, I investigated this problem
> > thoroughly and determined that this is a pretty severe bug in the Solr
> > XML response parsing code. This occurs only when the response is
> > rather large and the boundary between two read chunks falls in the
> > middle of a numeric value (that happens to end in '0').
> >
> > Tracking internally as DOP-1470.
> >
> > I think this may also explain some of the weird reports about invalid
> > UIDs that we've seen in recent history on this mailing list.
>
> Preliminary fix is here:
> https://github.com/stephanbosch/dovecot-core/commits/fix-solr-xml-parser
>
> It's big due to some code restructuring for making a proper unit test. I
> am not sure when this will be in a release.
>
> Regards,
>
> Stephan.
>
>

-- 
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 (c) E-Mail: larry...@gmail.com
US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106

Re: fts_solr: Error: fts_solr: received invalid uid '0'

[Stephan Bosch via dovecot]

On 08/10/2019 02:52, Stephan Bosch via dovecot wrote:



On 07/10/2019 15:13, Fabian Kuran via dovecot wrote:

Hi,

Am 23.09.2019 um 09:36 schrieb Filip Hanes >:



št 19. 9. 2019 o 15:30 Fabian via dovecot > napísal(a):


    Are there any other hints or tips regarding this „invalid uid
    ‚0‘"-message?

from fts-solr source:
case SOLR_XML_CONTENT_STATE_UID:
if (uint32_parse(str, len, &ctx->uid) < 0 || ctx->uid == 0) {
i_error("fts_solr: received invalid uid '%s'", t_strndup(str, len)); 

It means that plugin tries to parse '0' when in state of parsing uid 
in xml response from solr or there is a bug in xml response parser.
I would enable rawlog_dir in config and look there for unusual 
response or extra tags near usual uid in xml response. For example:
|plugin { fts = solr fts_solr = url=... 
rawlog_dir=/var/log/fts-solr/ }|




Thanks for this tip! The rawlog_dir option is not yet supported in 
our version 2.3.4. What we could provide now would be the 
XML-Response from Solr. The corresponding search query sometimes 
causes this error. But Solr's answer seems to always remain the same 
- in my opinion this already indicates a bug of Dovecot?


The XML response can be downloaded here: 
https://owncloud.eideo.de/index.php/s/8oyJaf5HBK6zdbr


Based on the XML response above, I investigated this problem 
thoroughly and determined that this is a pretty severe bug in the Solr 
XML response parsing code. This occurs only when the response is 
rather large and the boundary between two read chunks falls in the 
middle of a numeric value (that happens to end in '0').


Tracking internally as DOP-1470.

I think this may also explain some of the weird reports about invalid 
UIDs that we've seen in recent history on this mailing list.


Preliminary fix is here: 
https://github.com/stephanbosch/dovecot-core/commits/fix-solr-xml-parser


It's big due to some code restructuring for making a proper unit test. I 
am not sure when this will be in a release.


Regards,

Stephan.

Re: fts_solr: Error: fts_solr: received invalid uid '0'

[Stephan Bosch via dovecot]

On 07/10/2019 15:13, Fabian Kuran via dovecot wrote:

Hi,

Am 23.09.2019 um 09:36 schrieb Filip Hanes >:



št 19. 9. 2019 o 15:30 Fabian via dovecot > napísal(a):


Are there any other hints or tips regarding this „invalid uid
‚0‘"-message?

from fts-solr source:
case SOLR_XML_CONTENT_STATE_UID:
if (uint32_parse(str, len, &ctx->uid) < 0 || ctx->uid == 0) {
i_error("fts_solr: received invalid uid '%s'", t_strndup(str, len));
It means that plugin tries to parse '0' when in state of parsing uid 
in xml response from solr or there is a bug in xml response parser.
I would enable rawlog_dir in config and look there for unusual 
response or extra tags near usual uid in xml response. For example:

|plugin { fts = solr fts_solr = url=... rawlog_dir=/var/log/fts-solr/ }|



Thanks for this tip! The rawlog_dir option is not yet supported in our 
version 2.3.4. What we could provide now would be the XML-Response 
from Solr. The corresponding search query sometimes causes this error. 
But Solr's answer seems to always remain the same - in my opinion this 
already indicates a bug of Dovecot?


The XML response can be downloaded here: 
https://owncloud.eideo.de/index.php/s/8oyJaf5HBK6zdbr


Based on the XML response above, I investigated this problem thoroughly 
and determined that this is a pretty severe bug in the Solr XML response 
parsing code. This occurs only when the response is rather large and the 
boundary between two read chunks falls in the middle of a numeric value 
(that happens to end in '0').


Tracking internally as DOP-1470.

I think this may also explain some of the weird reports about invalid 
UIDs that we've seen in recent history on this mailing list.


Regards,

Stephan.

Re: fts_solr: Error: fts_solr: received invalid uid '0'

[Fabian Kuran via dovecot]

Hi,

> Am 23.09.2019 um 09:36 schrieb Filip Hanes :
> 
> 
> št 19. 9. 2019 o 15:30 Fabian via dovecot  > napísal(a):
> Are there any other hints or tips regarding this „invalid uid ‚0‘"-message?
> from fts-solr source:
> case SOLR_XML_CONTENT_STATE_UID:
> if (uint32_parse(str, len, &ctx->uid) < 0 || ctx->uid == 0) {
> i_error("fts_solr: received invalid uid '%s'", t_strndup(str, 
> len));
> It means that plugin tries to parse '0' when in state of parsing uid in xml 
> response from solr or there is a bug in xml response parser.
> I would enable rawlog_dir in config and look there for unusual response or 
> extra tags near usual uid in xml response. For example:
> plugin {
>   fts = solr
>   fts_solr = url=... rawlog_dir=/var/log/fts-solr/
> }
> 

Thanks for this tip! The rawlog_dir option is not yet supported in our version 
2.3.4. What we could provide now would be the XML-Response from Solr. The 
corresponding search query sometimes causes this error. But Solr's answer seems 
to always remain the same - in my opinion this already indicates a bug of 
Dovecot?

The XML response can be downloaded here: 
https://owncloud.eideo.de/index.php/s/8oyJaf5HBK6zdbr

Kind regards,
Fabian

Re: fts_solr: Error: fts_solr: received invalid uid '0'

[Daniel Miller via dovecot]

On 9/19/2019 6:28 AM, Fabian via dovecot wrote:


Thanks for your response! No we are not limiting Soli’s memory usage. After your tip, we've 
also upgraded the memory to 32GB. But the behavior remains the same. I have also already 
considered that Dovecot may index the UID incorrectly. But if I search the index directly, I 
don't find any entries with UID = 0, so I have no idea where this "fts_solr: received 
invalid uid '0"" message might come from.

In our test environment we actually indexed only one user. The user's mailbox 
contains about 100.000 mails. This means that there is not really much data in 
the index.

Are there any other hints or tips regarding this „invalid uid ‚0‘"-message?

Logfile:



Sep 16 08:35:27 mailservertest dovecot: imap(user01)<30204><+IjNzqWS2s2sEQoK>: 
Debug: http-client[1]: peer 172.17.10.12:8983: Creating 1 new connections to handle 
requests (already 0 usable, connecting t$


Your post has truncated the lines (right margin). Re-post with the full 
lines.


--
Daniel

Re: fts_solr: Error: fts_solr: received invalid uid '0'

[Fabian via dovecot]

Hi, 

> Am 16.09.2019 um 09:39 schrieb Daniel Miller :
> 
> On 9/13/2019 1:21 AM, Fabian via dovecot wrote:
>> Hi,
>> we are trying to add full text search functionality with Solr to our Doveoct 
>> setup. Our Versions:
>> OS: Debian 9
>> Tried versions:
>> - Dovecot 2.2.7 with Solr 3.6
>> - Dovecot 2.3.4 with Solr 8.2
>> (2.2.7 from offical Debian repository, 2.3.4 from backports)
>> Search is working mostly of the time perfrectly smooth. But sometimes 
>> following message appears in mail.err:
>> dovecot: imap(username)<16189>: Error: 
>> fts_solr: received invalid uid '0'
>> If this error occurs our webmail frontend delivers most of the time a 
>> timeout. Sometimes the search only takes really long.
>> Are  there any ideas why this error occurs? We are not able to reproduce the 
>> error in such a way that it would always be reproducible. However, we can 
>> reproduce the behavior in some form over and over again - but we do not know 
>> exactly what is decisive.
> 
> Are you limiting Solr's memory usage? How much available memory is on your 
> server?
> 
> To shortcut the conversation - if you don't have at least 16G of *free* RAM 
> it's time to upgrade. My own server has 32G installed - I used to have 16G. 
> My own Solr problems basically disappeared after adding RAM. And I only serve 
> a few users - my own mailstore is the largest as I keep most of my mails. If 
> you're serving 20+ users you'd probably benefit from doubling to at least 64G.
> 

Thanks for your response! No we are not limiting Soli’s memory usage. After 
your tip, we've also upgraded the memory to 32GB. But the behavior remains the 
same. I have also already considered that Dovecot may index the UID 
incorrectly. But if I search the index directly, I don't find any entries with 
UID = 0, so I have no idea where this "fts_solr: received invalid uid '0"" 
message might come from.

In our test environment we actually indexed only one user. The user's mailbox 
contains about 100.000 mails. This means that there is not really much data in 
the index.

Are there any other hints or tips regarding this „invalid uid ‚0‘"-message?

Logfile:
Sep 16 08:35:27 mailservertest dovecot: imap(user01)<30204><+IjNzqWS2s2sEQoK>: 
Debug: http-client[1]: peer 172.17.10.12:8983: Linked queue 
http://172.17.10.12:8983 (1 queues linked)
Sep 16 08:35:27 mailservertest dovecot: imap(user01)<30204><+IjNzqWS2s2sEQoK>: 
Debug: http-client[1]: queue http://172.17.10.12:8983: Started new connection 
to 172.17.10.12:8983
Sep 16 08:35:27 mailservertest dovecot: imap(user01)<30204><+IjNzqWS2s2sEQoK>: 
Debug: http-client[1]: request [Req1: GET 
http://172.17.10.12/solr/dovecot/select?wt=xml&fl=box,uid,score&rows=10&sort=b$
Sep 16 08:35:27 mailservertest dovecot: imap(user01)<30204><+IjNzqWS2s2sEQoK>: 
Debug: http-client[1]: Waiting for 1 requests to finish
Sep 16 08:35:27 mailservertest dovecot: imap(user01)<30204><+IjNzqWS2s2sEQoK>: 
Debug: http-client[1]: peer 172.17.10.12:8983: Creating 1 new connections to 
handle requests (already 0 usable, connecting t$
Sep 16 08:35:27 mailservertest dovecot: imap(user01)<30204><+IjNzqWS2s2sEQoK>: 
Debug: http-client[1]: peer 172.17.10.12:8983: Making new connection 1 of 1 (0 
connections exist, 0 pending)
Sep 16 08:35:27 mailservertest dovecot: imap(user01)<30204><+IjNzqWS2s2sEQoK>: 
Debug: http-client[1]: conn 172.17.10.12:8983 [0]: (172.17.10.12:8983): 
Connecting
Sep 16 08:35:27 mailservertest dovecot: imap(user01)<30204><+IjNzqWS2s2sEQoK>: 
Debug: http-client[1]: conn 172.17.10.12:8983 [0]: (172.17.10.12:8983): Waiting 
for connect (fd=26) to finish for max 0 msecs
Sep 16 08:35:27 mailservertest dovecot: imap(user01)<30204><+IjNzqWS2s2sEQoK>: 
Debug: http-client[1]: conn 172.17.10.12:8983 [0]: HTTP connection created (1 
parallel connections exist)
Sep 16 08:35:27 mailservertest dovecot: imap(user01)<30204><+IjNzqWS2s2sEQoK>: 
Debug: http-client[1]: conn 172.17.10.12:8983 [0]: (172.17.10.12:8983): Client 
connected (fd=26)
Sep 16 08:35:27 mailservertest dovecot: imap(user01)<30204><+IjNzqWS2s2sEQoK>: 
Debug: http-client[1]: conn 172.17.10.12:8983 [0]: Connected
Sep 16 08:35:27 mailservertest dovecot: imap(user01)<30204><+IjNzqWS2s2sEQoK>: 
Debug: http-client[1]: conn 172.17.10.12:8983 [0]: Ready for requests
Sep 16 08:35:27 mailservertest dovecot: imap(user01)<30204><+IjNzqWS2s2sEQoK>: 
Debug: http-client[1]: peer 172.17.10.12:8983: Successfully connected (1 
connections exist, 0 pending)
Sep 16 08:35:27 mailservertest dovecot: imap(user01)<30204><+IjNzqWS2s2sEQoK>: 
Debug: http-client: peer 172.17.10.12:8983: Successfully connected (1 
connections exist, 0 pending)
Sep 16 08:35:27 mailservertest dovecot: imap(user01)<30204><+IjNzqWS2s2sEQoK>: 
Debug: http-client[1]: peer 172.17.10.12:8983: Using 1 idle connections to 
handle 1 requests (1 total connections ready)
Sep 16 08:35:27 mailservertest dovecot: imap(user01)<30204><+IjNzqWS2s2sEQoK>: 
Debug: http-client[1]: queue http://172.17.10.12:8983: Con

Re: fts_solr: Error: fts_solr: received invalid uid '0'

[Daniel Miller via dovecot]

On 9/13/2019 1:21 AM, Fabian via dovecot wrote:

Hi,

we are trying to add full text search functionality with Solr to our Doveoct 
setup. Our Versions:
OS: Debian 9
Tried versions:
- Dovecot 2.2.7 with Solr 3.6
- Dovecot 2.3.4 with Solr 8.2
(2.2.7 from offical Debian repository, 2.3.4 from backports)

Search is working mostly of the time perfrectly smooth. But sometimes following 
message appears in mail.err:
dovecot: imap(username)<16189>: Error: fts_solr: 
received invalid uid '0'

If this error occurs our webmail frontend delivers most of the time a timeout. 
Sometimes the search only takes really long.

Are  there any ideas why this error occurs? We are not able to reproduce the 
error in such a way that it would always be reproducible. However, we can 
reproduce the behavior in some form over and over again - but we do not know 
exactly what is decisive.



Are you limiting Solr's memory usage? How much available memory is on 
your server?


To shortcut the conversation - if you don't have at least 16G of *free* 
RAM it's time to upgrade. My own server has 32G installed - I used to 
have 16G. My own Solr problems basically disappeared after adding RAM. 
And I only serve a few users - my own mailstore is the largest as I keep 
most of my mails. If you're serving 20+ users you'd probably benefit 
from doubling to at least 64G.


--
Daniel