Odg: Odg: Odg: Odg: Odg: Lucene upgrade

2020-01-07 Thread Mario Kevo
Hi all, Please could someone review #4395<https://github.com/apache/geode/pull/4395>. BR, Mario Šalje: Mario Kevo Poslano: 17. prosinca 2019. 14:30 Prima: Jason Huynh Kopija: geode Predmet: Odg: Odg: Odg: Odg: Odg: Lucene upgrade Hi Jason, Nice ca

Odg: Odg: Odg: Odg: Odg: Lucene upgrade

2019-12-17 Thread Mario Kevo
: Odg: Odg: Odg: Lucene upgrade Hi Mario, I think I see what is going on here. The logic for "reindex" code was a bit off ( it expected reindex features to be complete by a certain release). I have a PR on develop to adjust that calculation (https://github.com/apache/geode

Re: Odg: Odg: Odg: Odg: Lucene upgrade

2019-12-13 Thread Jason Huynh
gt; In tests it tried to execute a query after all members are rolled. > > BR, > Mario > > -- > *Šalje:* Jason Huynh > *Poslano:* 11. prosinca 2019. 23:08 > *Prima:* Mario Kevo > *Kopija:* geode > *Predmet:* Re: Odg: Odg: Odg: Lucene upgrade

Odg: Odg: Odg: Odg: Lucene upgrade

2019-12-12 Thread Mario Kevo
a query after all members are rolled. BR, Mario Šalje: Jason Huynh Poslano: 11. prosinca 2019. 23:08 Prima: Mario Kevo Kopija: geode Predmet: Re: Odg: Odg: Odg: Lucene upgrade Hi Mario, Is the same test failing? If it's a different test, could you tell us which

Odg: Odg: Odg: Lucene upgrade

2019-12-11 Thread Mario Kevo
number of entries in tests? BR, Mario Šalje: Jason Huynh Poslano: 6. prosinca 2019. 20:53 Prima: Mario Kevo Kopija: geode Predmet: Re: Odg: Odg: Lucene upgrade Hi Mario, I made a PR against your branch for some of the changes I had to do to get past

Re: Odg: Odg: Lucene upgrade

2019-12-06 Thread Jason Huynh
stener.process() if all members are upgraded, also changed > > test to verify lucene indexes only after all members are upgraded, but > got > > the same error with incompatibilities between lucene versions. > > Changes are visible on https://github.com/apache/geode/pull/4198. > &g

Odg: Odg: Lucene upgrade

2019-12-06 Thread Mario Kevo
one is unsuccessfully dispatched. I don't know why some events are successfully dispatched, some not. Do you have any idea? BR, Mario Šalje: Jason Huynh Poslano: 2. prosinca 2019. 18:32 Prima: geode Predmet: Re: Odg: Lucene upgrade Hi Mario, Sorry I reread the original em

Re: Odg: Lucene upgrade

2019-12-02 Thread Jason Huynh
; Mario > > > > Šalje: Xiaojian Zhou > Poslano: 7. studenog 2019. 18:27 > Prima: geode > Predmet: Re: Lucene upgrade > > Oh, I misunderstood option-1 and option-2. What I vote is Jason's option-1. > > On Thu, Nov 7, 2019 at 9:19 AM Jaso

Odg: Lucene upgrade

2019-12-02 Thread Mario Kevo
Šalje: Xiaojian Zhou Poslano: 7. studenog 2019. 18:27 Prima: geode Predmet: Re: Lucene upgrade Oh, I misunderstood option-1 and option-2. What I vote is Jason's option-1. On Thu, Nov 7, 2019 at 9:19 AM Jason Huynh wrote: > Gester, I don't think we need to write in the old format,

Re: Lucene upgrade

2019-11-07 Thread Xiaojian Zhou
Oh, I misunderstood option-1 and option-2. What I vote is Jason's option-1. On Thu, Nov 7, 2019 at 9:19 AM Jason Huynh wrote: > Gester, I don't think we need to write in the old format, we just need the > new format not to be written while old members can potentially read the > lucene files.

Re: Lucene upgrade

2019-11-07 Thread Jason Huynh
Gester, I don't think we need to write in the old format, we just need the new format not to be written while old members can potentially read the lucene files. Option 1 can be very similar to Dan's snippet of code. I think Option 2 is going to leave a lot of people unhappy when they get stuck

Re: Lucene upgrade

2019-11-06 Thread Xiaojian Zhou
Usually re-creating region and index are expensive and customers are reluctant to do it, according to my memory. We do have an offline reindex scripts or steps (written by Barry?). If that could be an option, they can try that offline tool. I saw from Mario's email, he said: "I didn't found a

Re: Lucene upgrade

2019-11-06 Thread Jacob Barrett
> On Nov 6, 2019, at 3:36 PM, Jason Huynh wrote: > > Jake - there is a side effect to this in that the user would have to > reimport all their data into the user defined region too. Client apps > would also have to know which of the regions to put into.. also, I may be > misunderstanding

Re: Lucene upgrade

2019-11-06 Thread Jason Huynh
Dan - LGTM check it in! ;-) (kidding of course) Jake - there is a side effect to this in that the user would have to reimport all their data into the user defined region too. Client apps would also have to know which of the regions to put into.. also, I may be misunderstanding this suggestion,

Re: Lucene upgrade

2019-11-06 Thread Jacob Barrett
> On Nov 6, 2019, at 2:16 PM, Jason Huynh wrote: > > Jake, -from my understanding, the implementation details of geode-lucene is > that we are using a partitioned region as a "file-system" for lucene > files. Yeah, I didn’t explain well. I mean to say literally create a new region for the

Re: Lucene upgrade

2019-11-06 Thread Dan Smith
> > 1.) We add some product code/lucene listener to detect whether we have old > versions of geode and if so, do not write to lucene on the newly updated > node until all versions are up to date. Elaborating on this option a little more, this might be as simple as something like the below at the

Re: Lucene upgrade

2019-11-06 Thread Jason Huynh
Jake, -from my understanding, the implementation details of geode-lucene is that we are using a partitioned region as a "file-system" for lucene files. As new servers are rolled, the issue is that the new servers have the new codec. As puts occur on the users data region, the async listeners are

Re: Lucene upgrade

2019-11-06 Thread Xiaojian Zhou
He tried to upgrade lucene version from current 6.6.4 to 8.2. There're some challenges. One challenge is the codec changed, which caused the format of index is also changed. That's why we did not implement it. If he resolved the coding challenges, then rolling upgrade will probably need option-2

Re: Lucene upgrade

2019-11-06 Thread Jacob Barrett
What about “versioning” the region that backs the indexes? Old servers with old license would continue to read/write to old region. New servers would start re-indexing with the new version. Given the async nature of the indexing would the mismatch in indexing for some period of time have an

Re: Lucene upgrade

2019-11-06 Thread Jason Huynh
Hi Mario, I think there are a few ways to accomplish what Dan was suggesting...Dan or other's, please chime in with more options/solutions. 1.) We add some product code/lucene listener to detect whether we have old versions of geode and if so, do not write to lucene on the newly updated node

Re: Lucene upgrade

2019-11-06 Thread Mario Kevo
Hi Dan, thanks for suggestions. I didn't found a way to write lucene in older format. They only support reading old format indexes with newer version by using lucene-backward- codec. Regarding to freeze writes to the lucene index, that means that we need to start locators and servers, create

Re: Lucene upgrade

2019-11-04 Thread Dan Smith
I think the issue probably has to do with doing a rolling upgrade from an old version of geode (with an old version of lucene) to the new version of geode. Geode's lucene integration works by writing the lucene index to a colocated region. So lucene index data that was generated on one server can

Lucene upgrade

2019-11-04 Thread Mario Kevo
Hi geode dev, I'm working on upgrade lucene to a newer version. ( https://issues.apache.org/jira/browse/GEODE-7309) I followed instruction from https://cwiki.apache.org/confluence/display/GEODE/Upgrading+to+Lucene+7.1.0 Also add some other changes that is needed for lucene 8.2.0. I found some