Re: Possible data corruption in JavaBinCodec in Solr 8.3 during distributed update?

2019-11-23 Thread Noble Paul
Thanks Colvin. Can you share the details in the ticket? I plan to debug this today. It's unlikely to be a synchronization issue because serialization/deserialization usually happens in single thread. On Sun, Nov 24, 2019, 4:09 AM Colvin Cowie wrote: > https://issues.apache.org/jira/browse/SOLR

Re: Possible data corruption in JavaBinCodec in Solr 8.3 during distributed update?

2019-11-23 Thread Colvin Cowie
https://issues.apache.org/jira/browse/SOLR-13963 I'll see about modifying the test I have to fit in with the existing tests, and if there's a better option then open to whatever On Sat, 23 Nov 2019 at 16:43, Colvin Cowie wrote: > I've found the problem, JavaBinCodec has a CharArr,* arr*, which

Re: Possible data corruption in JavaBinCodec in Solr 8.3 during distributed update?

2019-11-23 Thread Colvin Cowie
I've found the problem, JavaBinCodec has a CharArr,* arr*, which is modified in two different locations, but only one of which is protected with a synchronized block getStringProvider(), which is used when you call getValue() rather than getRawValue() on the string based SolrInputFields, synchroni

Re: Possible data corruption in JavaBinCodec in Solr 8.3 during distributed update?

2019-11-21 Thread Colvin Cowie
*> the difference is because the _default config has the dynamic schema building in it, which I assume is pushing it down a different code path. * Also to add to that, I assumed initially that this just meant that it was working because the corrupted field names would just cause it to create a fie

Re: Possible data corruption in JavaBinCodec in Solr 8.3 during distributed update?

2019-11-21 Thread Colvin Cowie
I've been a bit snowed under, but I've found the difference is because the _default config has the dynamic schema building in it, which I assume is pushing it down a different code path. I'm using the vanilla Solr 8.3.0 binary8.3.0 2aa586909b911e66e1d8863aa89f173d69f86cd2 - ishan - 2019-10-25

Re: Possible data corruption in JavaBinCodec in Solr 8.3 during distributed update?

2019-11-21 Thread Jason Gerlowski
Very curious what the config change that's related to reproducing this looks like. Maybe it's something that is worth adding test-randomization around? Just thinking aloud.

Re: Possible data corruption in JavaBinCodec in Solr 8.3 during distributed update?

2019-11-20 Thread Noble Paul
Sure, looking forward to that thnaks On Thu, Nov 21, 2019 at 8:06 AM Colvin Cowie wrote: > > Hi, I'll share it when I'm back at work tomorrow. > > I've found that the issue appears to not be reproducible using the > solrconfig.xml in the _default example, so it must be something in ours (or > som

Re: Possible data corruption in JavaBinCodec in Solr 8.3 during distributed update?

2019-11-20 Thread Colvin Cowie
Hi, I'll share it when I'm back at work tomorrow. I've found that the issue appears to not be reproducible using the solrconfig.xml in the _default example, so it must be something in ours (or something missing from it) that is making the problem appear, in combination with the code change. Thank

Re: Possible data corruption in JavaBinCodec in Solr 8.3 during distributed update?

2019-11-20 Thread Noble Paul
Can you share the test please On Thu, Nov 21, 2019 at 7:02 AM Noble Paul wrote: > > Thanks Colvin, I'll take a look > > On Thu, Nov 21, 2019 at 4:24 AM Colvin Cowie > wrote: > > > > I've identified the change which has caused the problem to materialize, but > > it shouldn't itself cause a probl

Re: Possible data corruption in JavaBinCodec in Solr 8.3 during distributed update?

2019-11-20 Thread Noble Paul
Thanks Colvin, I'll take a look On Thu, Nov 21, 2019 at 4:24 AM Colvin Cowie wrote: > > I've identified the change which has caused the problem to materialize, but > it shouldn't itself cause a problem. > > https://github.com/apache/lucene-solr/commit/e45e8127d5c17af4e4b87a0a4eaf0afaf4f9ff4b#diff

Re: Possible data corruption in JavaBinCodec in Solr 8.3 during distributed update?

2019-11-20 Thread Colvin Cowie
I've identified the change which has caused the problem to materialize, but it shouldn't itself cause a problem. https://github.com/apache/lucene-solr/commit/e45e8127d5c17af4e4b87a0a4eaf0afaf4f9ff4b#diff-7f7f485122d8257bd5d3210c092b967fR52 for https://issues.apache.org/jira/browse/SOLR-13682 In w

Possible data corruption in JavaBinCodec in Solr 8.3 during distributed update?

2019-11-19 Thread Colvin Cowie
Hello Apologies for the lack of actual detail in this, we're still digging into it ourselves. I will provide more detail, and maybe some logs, once I have a better idea of what is actually happening. But I thought I might as well ask if anyone knows of changes that were made in the Solr 8.3 releas