Re: Re[2]: Discuss idle_verify with moving partitions changes.
Zhenya, Vlad's message looks incorrectly. How about something like: "Not all partitions were checked. Check of partitions was skipped due to rebalancing is in progress. Please rerun idle_verify after finish rebalancing." пн, 23 мар. 2020 г. в 12:47, Zhenya Stanilovsky : > > Guys thank for quick response, Ivan what do you think about Vlad`s > proposal to add additional info like : > "Possible results are not consistent due to rebalance still in progress" ? > Thanks ! > > >Понедельник, 23 марта 2020, 12:30 +03:00 от Ivan Rakov < > ivan.glu...@gmail.com>: > > > >Zhenya, > > > >As for me, the current behavior of idle_verify looks correct. > >There's no sense in checking MOVING partitions (on which we explicitly > >inform user), however checking consistency between the rest of owners > still > >makes sense: they still can diverge and we can be aware of the presence of > >the conflicts sooner. > >In case cluster is not idle (in terms of user activities, not in terms of > >internal cluster processes like rebalancing), utility will fail as > expected. > > > >On Mon, Mar 23, 2020 at 11:23 AM Vladislav Pyatkov < > vpyat...@gridgain.com > > >wrote: > > > >> Hi Zhenya, > >> > >> I see your point. Need to show some message, because cluster is not idle > >> (rebalance is going). > >> When cluster not idle we cannot validate partitions honestly. After > several > >> minutes we can to get absolutely different result, without any client's > >> operation of cache happened. > >> > >> May be enough showing some message more clear for end user. For example: > >> "Result has not valid, rebalance is going." > >> > >> Another thing you meaning - issue in indexes, when rebalance is > following. > >> I think idex_validate should fail in this case, because indexes always > in > >> load during rebalance. > >> > >> > >> On Mon, Mar 23, 2020 at 10:20 AM Zhenya Stanilovsky > >> < arzamas...@mail.ru.invalid > wrote: > >> > >> > > >> > Igniters, i found that near idle check commands only shows partitions > in > >> > MOVING states as info in log and not take into account this fact as > >> > erroneous idle cluster state. > >> > control.sh --cache idle_verify, control.sh --cache validate_indexes > >> > --check-crc > >> > > >> > for example command would show something like : > >> > > >> > Arguments: --cache idle_verify --yes > >> > > >> > > >> > > >> > idle_verify task was executed with the following args: caches=[], > >> > excluded=[], cacheFilter=[DEFAULT] > >> > idle_verify check has finished, no conflicts have been found. > >> > Verification was skipped for 21 MOVING partitions: > >> > Skipped partition: PartitionKeyV2 [grpId=1544803905, grpName=default, > >> > partId=7] > >> > Partition instances: [PartitionHashRecordV2 [isPrimary=false, > >> > consistentId=gridCommandHandlerTest2, updateCntr=3, > >> partitionState=MOVING, > >> > state=MOVING]] .. and so on > >> > > >> > I found this erroneous and can lead to further cluster index > corruption, > >> > for example in case when only command OK result checked. > >> > > >> > If no objections would be here, i plan to inform about moving states > as > >> > not OK exit code too. > >> > > >> > > >> > >> > >> > >> -- > >> Vladislav Pyatkov > >> Architect-Consultant "GridGain Rus" Llc. > >> +7-929-537-79-60 > >> > > > > -- BR, Sergey Antonov
Re: Re[2]: Discuss idle_verify with moving partitions changes.
Partial results are consistent though. I'd add something like "Possible results are not full" instead. On Mon, Mar 23, 2020 at 12:47 PM Zhenya Stanilovsky wrote: > > Guys thank for quick response, Ivan what do you think about Vlad`s > proposal to add additional info like : > "Possible results are not consistent due to rebalance still in progress" ? > Thanks ! > > >Понедельник, 23 марта 2020, 12:30 +03:00 от Ivan Rakov < > ivan.glu...@gmail.com>: > > > >Zhenya, > > > >As for me, the current behavior of idle_verify looks correct. > >There's no sense in checking MOVING partitions (on which we explicitly > >inform user), however checking consistency between the rest of owners > still > >makes sense: they still can diverge and we can be aware of the presence of > >the conflicts sooner. > >In case cluster is not idle (in terms of user activities, not in terms of > >internal cluster processes like rebalancing), utility will fail as > expected. > > > >On Mon, Mar 23, 2020 at 11:23 AM Vladislav Pyatkov < > vpyat...@gridgain.com > > >wrote: > > > >> Hi Zhenya, > >> > >> I see your point. Need to show some message, because cluster is not idle > >> (rebalance is going). > >> When cluster not idle we cannot validate partitions honestly. After > several > >> minutes we can to get absolutely different result, without any client's > >> operation of cache happened. > >> > >> May be enough showing some message more clear for end user. For example: > >> "Result has not valid, rebalance is going." > >> > >> Another thing you meaning - issue in indexes, when rebalance is > following. > >> I think idex_validate should fail in this case, because indexes always > in > >> load during rebalance. > >> > >> > >> On Mon, Mar 23, 2020 at 10:20 AM Zhenya Stanilovsky > >> < arzamas...@mail.ru.invalid > wrote: > >> > >> > > >> > Igniters, i found that near idle check commands only shows partitions > in > >> > MOVING states as info in log and not take into account this fact as > >> > erroneous idle cluster state. > >> > control.sh --cache idle_verify, control.sh --cache validate_indexes > >> > --check-crc > >> > > >> > for example command would show something like : > >> > > >> > Arguments: --cache idle_verify --yes > >> > > >> > > >> > > >> > idle_verify task was executed with the following args: caches=[], > >> > excluded=[], cacheFilter=[DEFAULT] > >> > idle_verify check has finished, no conflicts have been found. > >> > Verification was skipped for 21 MOVING partitions: > >> > Skipped partition: PartitionKeyV2 [grpId=1544803905, grpName=default, > >> > partId=7] > >> > Partition instances: [PartitionHashRecordV2 [isPrimary=false, > >> > consistentId=gridCommandHandlerTest2, updateCntr=3, > >> partitionState=MOVING, > >> > state=MOVING]] .. and so on > >> > > >> > I found this erroneous and can lead to further cluster index > corruption, > >> > for example in case when only command OK result checked. > >> > > >> > If no objections would be here, i plan to inform about moving states > as > >> > not OK exit code too. > >> > > >> > > >> > >> > >> > >> -- > >> Vladislav Pyatkov > >> Architect-Consultant "GridGain Rus" Llc. > >> +7-929-537-79-60 > >> > > > >
Re[2]: Discuss idle_verify with moving partitions changes.
Guys thank for quick response, Ivan what do you think about Vlad`s proposal to add additional info like : "Possible results are not consistent due to rebalance still in progress" ? Thanks ! >Понедельник, 23 марта 2020, 12:30 +03:00 от Ivan Rakov : > >Zhenya, > >As for me, the current behavior of idle_verify looks correct. >There's no sense in checking MOVING partitions (on which we explicitly >inform user), however checking consistency between the rest of owners still >makes sense: they still can diverge and we can be aware of the presence of >the conflicts sooner. >In case cluster is not idle (in terms of user activities, not in terms of >internal cluster processes like rebalancing), utility will fail as expected. > >On Mon, Mar 23, 2020 at 11:23 AM Vladislav Pyatkov < vpyat...@gridgain.com > >wrote: > >> Hi Zhenya, >> >> I see your point. Need to show some message, because cluster is not idle >> (rebalance is going). >> When cluster not idle we cannot validate partitions honestly. After several >> minutes we can to get absolutely different result, without any client's >> operation of cache happened. >> >> May be enough showing some message more clear for end user. For example: >> "Result has not valid, rebalance is going." >> >> Another thing you meaning - issue in indexes, when rebalance is following. >> I think idex_validate should fail in this case, because indexes always in >> load during rebalance. >> >> >> On Mon, Mar 23, 2020 at 10:20 AM Zhenya Stanilovsky >> < arzamas...@mail.ru.invalid > wrote: >> >> > >> > Igniters, i found that near idle check commands only shows partitions in >> > MOVING states as info in log and not take into account this fact as >> > erroneous idle cluster state. >> > control.sh --cache idle_verify, control.sh --cache validate_indexes >> > --check-crc >> > >> > for example command would show something like : >> > >> > Arguments: --cache idle_verify --yes >> > >> > >> >> > idle_verify task was executed with the following args: caches=[], >> > excluded=[], cacheFilter=[DEFAULT] >> > idle_verify check has finished, no conflicts have been found. >> > Verification was skipped for 21 MOVING partitions: >> > Skipped partition: PartitionKeyV2 [grpId=1544803905, grpName=default, >> > partId=7] >> > Partition instances: [PartitionHashRecordV2 [isPrimary=false, >> > consistentId=gridCommandHandlerTest2, updateCntr=3, >> partitionState=MOVING, >> > state=MOVING]] .. and so on >> > >> > I found this erroneous and can lead to further cluster index corruption, >> > for example in case when only command OK result checked. >> > >> > If no objections would be here, i plan to inform about moving states as >> > not OK exit code too. >> > >> > >> >> >> >> -- >> Vladislav Pyatkov >> Architect-Consultant "GridGain Rus" Llc. >> +7-929-537-79-60 >>
Re: Discuss idle_verify with moving partitions changes.
Zhenya, As for me, the current behavior of idle_verify looks correct. There's no sense in checking MOVING partitions (on which we explicitly inform user), however checking consistency between the rest of owners still makes sense: they still can diverge and we can be aware of the presence of the conflicts sooner. In case cluster is not idle (in terms of user activities, not in terms of internal cluster processes like rebalancing), utility will fail as expected. On Mon, Mar 23, 2020 at 11:23 AM Vladislav Pyatkov wrote: > Hi Zhenya, > > I see your point. Need to show some message, because cluster is not idle > (rebalance is going). > When cluster not idle we cannot validate partitions honestly. After several > minutes we can to get absolutely different result, without any client's > operation of cache happened. > > May be enough showing some message more clear for end user. For example: > "Result has not valid, rebalance is going." > > Another thing you meaning - issue in indexes, when rebalance is following. > I think idex_validate should fail in this case, because indexes always in > load during rebalance. > > > On Mon, Mar 23, 2020 at 10:20 AM Zhenya Stanilovsky > wrote: > > > > > Igniters, i found that near idle check commands only shows partitions in > > MOVING states as info in log and not take into account this fact as > > erroneous idle cluster state. > > control.sh --cache idle_verify, control.sh --cache validate_indexes > > --check-crc > > > > for example command would show something like : > > > > Arguments: --cache idle_verify --yes > > > > > > > idle_verify task was executed with the following args: caches=[], > > excluded=[], cacheFilter=[DEFAULT] > > idle_verify check has finished, no conflicts have been found. > > Verification was skipped for 21 MOVING partitions: > > Skipped partition: PartitionKeyV2 [grpId=1544803905, grpName=default, > > partId=7] > > Partition instances: [PartitionHashRecordV2 [isPrimary=false, > > consistentId=gridCommandHandlerTest2, updateCntr=3, > partitionState=MOVING, > > state=MOVING]] .. and so on > > > > I found this erroneous and can lead to further cluster index corruption, > > for example in case when only command OK result checked. > > > > If no objections would be here, i plan to inform about moving states as > > not OK exit code too. > > > > > > > > -- > Vladislav Pyatkov > Architect-Consultant "GridGain Rus" Llc. > +7-929-537-79-60 >
Re: Discuss idle_verify with moving partitions changes.
Hi Zhenya, I see your point. Need to show some message, because cluster is not idle (rebalance is going). When cluster not idle we cannot validate partitions honestly. After several minutes we can to get absolutely different result, without any client's operation of cache happened. May be enough showing some message more clear for end user. For example: "Result has not valid, rebalance is going." Another thing you meaning - issue in indexes, when rebalance is following. I think idex_validate should fail in this case, because indexes always in load during rebalance. On Mon, Mar 23, 2020 at 10:20 AM Zhenya Stanilovsky wrote: > > Igniters, i found that near idle check commands only shows partitions in > MOVING states as info in log and not take into account this fact as > erroneous idle cluster state. > control.sh --cache idle_verify, control.sh --cache validate_indexes > --check-crc > > for example command would show something like : > > Arguments: --cache idle_verify --yes > > > idle_verify task was executed with the following args: caches=[], > excluded=[], cacheFilter=[DEFAULT] > idle_verify check has finished, no conflicts have been found. > Verification was skipped for 21 MOVING partitions: > Skipped partition: PartitionKeyV2 [grpId=1544803905, grpName=default, > partId=7] > Partition instances: [PartitionHashRecordV2 [isPrimary=false, > consistentId=gridCommandHandlerTest2, updateCntr=3, partitionState=MOVING, > state=MOVING]] .. and so on > > I found this erroneous and can lead to further cluster index corruption, > for example in case when only command OK result checked. > > If no objections would be here, i plan to inform about moving states as > not OK exit code too. > > -- Vladislav Pyatkov Architect-Consultant "GridGain Rus" Llc. +7-929-537-79-60
Discuss idle_verify with moving partitions changes.
Igniters, i found that near idle check commands only shows partitions in MOVING states as info in log and not take into account this fact as erroneous idle cluster state. control.sh --cache idle_verify, control.sh --cache validate_indexes --check-crc for example command would show something like : Arguments: --cache idle_verify --yes idle_verify task was executed with the following args: caches=[], excluded=[], cacheFilter=[DEFAULT] idle_verify check has finished, no conflicts have been found. Verification was skipped for 21 MOVING partitions: Skipped partition: PartitionKeyV2 [grpId=1544803905, grpName=default, partId=7] Partition instances: [PartitionHashRecordV2 [isPrimary=false, consistentId=gridCommandHandlerTest2, updateCntr=3, partitionState=MOVING, state=MOVING]] .. and so on I found this erroneous and can lead to further cluster index corruption, for example in case when only command OK result checked. If no objections would be here, i plan to inform about moving states as not OK exit code too.
