Re: Resurrect FairAffinityFunction
tandpoint, you can just fail all queries that have a >>> JOIN >>> > > from different cache groups, if at least one of the groups is using >>> Fair >>> > > Affinity. I am not sure why this would be hard. >>> > > >>> > > >>> > > > >>> > > > On Thu, Aug 10, 2017 at 12:16 AM, Valentin Kulichenko < >>> > > > valentin.kuliche...@gmail.com> wrote: >>> > > > >>> > > > > As far as I know, all logical caches with the same affinity >>> function >>> > > and >>> > > > > node filter will end up in the same group. If that's the case, I >>> like >>> > > the >>> > > > > idea. This is exactly what I was looking for. >>> > > > > >>> > > > > -Val >>> > > > > >>> > > > > On Wed, Aug 9, 2017 at 8:18 AM, Evgenii Zhuravlev < >>> > > > > e.zhuravlev...@gmail.com> >>> > > > > wrote: >>> > > > > >>> > > > > > Dmitriy, >>> > > > > > >>> > > > > > Yes, you're right. Moreover, it looks like a good practice to >>> > combine >>> > > > > > caches that will be used for collocated JOINs in one group >>> since it >>> > > > > reduces >>> > > > > > overall overhead. >>> > > > > > >>> > > > > > I think it's not a problem to add this restriction to the SQL >>> JOIN >>> > > > level >>> > > > > if >>> > > > > > we will decide to use this solution. >>> > > > > > >>> > > > > > Evgenii >>> > > > > > >>> > > > > > >>> > > > > > >>> > > > > > >>> > > > > > 2017-08-09 17:07 GMT+03:00 Dmitriy Setrakyan < >>> > dsetrak...@apache.org >>> > > >: >>> > > > > > >>> > > > > > > On Wed, Aug 9, 2017 at 6:28 AM, ezhuravl < >>> > e.zhuravlev...@gmail.com >>> > > > >>> > > > > > wrote: >>> > > > > > > >>> > > > > > > > Folks, >>> > > > > > > > >>> > > > > > > > I've started working on a https://issues.apache.org/ >>> > > > > > > > jira/browse/IGNITE-5836 >>> > > > > > > > ticket and found that the recently added feature with >>> > cacheGroups >>> > > > > doing >>> > > > > > > > pretty much the same that was described in this issue. >>> > CacheGroup >>> > > > > > > > guarantees >>> > > > > > > > that all caches within a group have same assignments since >>> they >>> > > > > share a >>> > > > > > > > single underlying 'physical' cache. >>> > > > > > > > >>> > > > > > > >>> > > > > > > > I think we can return FairAffinityFunction and add >>> information >>> > to >>> > > > its >>> > > > > > > > Javadoc that all caches with same AffinityFunction and >>> > NodeFilter >>> > > > > > should >>> > > > > > > be >>> > > > > > > > combined in cache group to avoid a problem with >>> inconsistent >>> > > > previous >>> > > > > > > > assignments. >>> > > > > > > > >>> > > > > > > > What do you guys think? >>> > > > > > > > >>> > > > > > > >>> > > > > > > Are you suggesting that we can only reuse the same >>> > > > FairAffinityFunction >>> > > > > > > across the logical caches within the same group? This would >>> mean >>> > > that >>> > > > > > > caches from the different groups cannot participate in JOINs >>> or >>> > > > > > collocated >>> > > > > > > compute. >>> > > > > > > >>> > > > > > > I think I like the idea, however, we need to make sure that >>> we >>> > > > enforce >>> > > > > > this >>> > > > > > > restriction, at least at the SQL JOIN level. >>> > > > > > > >>> > > > > > > Alexey G, Val, would be nice to hear your thoughts on this. >>> > > > > > > >>> > > > > > > >>> > > > > > > > >>> > > > > > > > Evgenii >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > -- >>> > > > > > > > View this message in context: http://apache-ignite- >>> > > > > > > > developers.2346864.n4.nabble.com/Resurrect- >>> > FairAffinityFunction- >>> > > > > > > > tp19987p20669.html >>> > > > > > > > Sent from the Apache Ignite Developers mailing list >>> archive at >>> > > > > > > Nabble.com. >>> > > > > > > > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > > >>> > >>> >> >> >
Re: Resurrect FairAffinityFunction
case, I >> like >> > > the >> > > > > idea. This is exactly what I was looking for. >> > > > > >> > > > > -Val >> > > > > >> > > > > On Wed, Aug 9, 2017 at 8:18 AM, Evgenii Zhuravlev < >> > > > > e.zhuravlev...@gmail.com> >> > > > > wrote: >> > > > > >> > > > > > Dmitriy, >> > > > > > >> > > > > > Yes, you're right. Moreover, it looks like a good practice to >> > combine >> > > > > > caches that will be used for collocated JOINs in one group >> since it >> > > > > reduces >> > > > > > overall overhead. >> > > > > > >> > > > > > I think it's not a problem to add this restriction to the SQL >> JOIN >> > > > level >> > > > > if >> > > > > > we will decide to use this solution. >> > > > > > >> > > > > > Evgenii >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > 2017-08-09 17:07 GMT+03:00 Dmitriy Setrakyan < >> > dsetrak...@apache.org >> > > >: >> > > > > > >> > > > > > > On Wed, Aug 9, 2017 at 6:28 AM, ezhuravl < >> > e.zhuravlev...@gmail.com >> > > > >> > > > > > wrote: >> > > > > > > >> > > > > > > > Folks, >> > > > > > > > >> > > > > > > > I've started working on a https://issues.apache.org/ >> > > > > > > > jira/browse/IGNITE-5836 >> > > > > > > > ticket and found that the recently added feature with >> > cacheGroups >> > > > > doing >> > > > > > > > pretty much the same that was described in this issue. >> > CacheGroup >> > > > > > > > guarantees >> > > > > > > > that all caches within a group have same assignments since >> they >> > > > > share a >> > > > > > > > single underlying 'physical' cache. >> > > > > > > > >> > > > > > > >> > > > > > > > I think we can return FairAffinityFunction and add >> information >> > to >> > > > its >> > > > > > > > Javadoc that all caches with same AffinityFunction and >> > NodeFilter >> > > > > > should >> > > > > > > be >> > > > > > > > combined in cache group to avoid a problem with inconsistent >> > > > previous >> > > > > > > > assignments. >> > > > > > > > >> > > > > > > > What do you guys think? >> > > > > > > > >> > > > > > > >> > > > > > > Are you suggesting that we can only reuse the same >> > > > FairAffinityFunction >> > > > > > > across the logical caches within the same group? This would >> mean >> > > that >> > > > > > > caches from the different groups cannot participate in JOINs >> or >> > > > > > collocated >> > > > > > > compute. >> > > > > > > >> > > > > > > I think I like the idea, however, we need to make sure that we >> > > > enforce >> > > > > > this >> > > > > > > restriction, at least at the SQL JOIN level. >> > > > > > > >> > > > > > > Alexey G, Val, would be nice to hear your thoughts on this. >> > > > > > > >> > > > > > > >> > > > > > > > >> > > > > > > > Evgenii >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > -- >> > > > > > > > View this message in context: http://apache-ignite- >> > > > > > > > developers.2346864.n4.nabble.com/Resurrect- >> > FairAffinityFunction- >> > > > > > > > tp19987p20669.html >> > > > > > > > Sent from the Apache Ignite Developers mailing list archive >> at >> > > > > > > Nabble.com. >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> > >
Re: Resurrect FairAffinityFunction
> > > > 2017-08-09 17:07 GMT+03:00 Dmitriy Setrakyan < > > dsetrak...@apache.org > > > >: > > > > > > > > > > > > > On Wed, Aug 9, 2017 at 6:28 AM, ezhuravl < > > e.zhuravlev...@gmail.com > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Folks, > > > > > > > > > > > > > > > > I've started working on a https://issues.apache.org/ > > > > > > > > jira/browse/IGNITE-5836 > > > > > > > > ticket and found that the recently added feature with > > cacheGroups > > > > > doing > > > > > > > > pretty much the same that was described in this issue. > > CacheGroup > > > > > > > > guarantees > > > > > > > > that all caches within a group have same assignments since > they > > > > > share a > > > > > > > > single underlying 'physical' cache. > > > > > > > > > > > > > > > > > > > > > > > I think we can return FairAffinityFunction and add > information > > to > > > > its > > > > > > > > Javadoc that all caches with same AffinityFunction and > > NodeFilter > > > > > > should > > > > > > > be > > > > > > > > combined in cache group to avoid a problem with inconsistent > > > > previous > > > > > > > > assignments. > > > > > > > > > > > > > > > > What do you guys think? > > > > > > > > > > > > > > > > > > > > > > Are you suggesting that we can only reuse the same > > > > FairAffinityFunction > > > > > > > across the logical caches within the same group? This would > mean > > > that > > > > > > > caches from the different groups cannot participate in JOINs or > > > > > > collocated > > > > > > > compute. > > > > > > > > > > > > > > I think I like the idea, however, we need to make sure that we > > > > enforce > > > > > > this > > > > > > > restriction, at least at the SQL JOIN level. > > > > > > > > > > > > > > Alexey G, Val, would be nice to hear your thoughts on this. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Evgenii > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > View this message in context: http://apache-ignite- > > > > > > > > developers.2346864.n4.nabble.com/Resurrect- > > FairAffinityFunction- > > > > > > > > tp19987p20669.html > > > > > > > > Sent from the Apache Ignite Developers mailing list archive > at > > > > > > > Nabble.com. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: Resurrect FairAffinityFunction
at was described in this issue. > CacheGroup > > > > > > > guarantees > > > > > > > that all caches within a group have same assignments since they > > > > share a > > > > > > > single underlying 'physical' cache. > > > > > > > > > > > > > > > > > > > > I think we can return FairAffinityFunction and add information > to > > > its > > > > > > > Javadoc that all caches with same AffinityFunction and > NodeFilter > > > > > should > > > > > > be > > > > > > > combined in cache group to avoid a problem with inconsistent > > > previous > > > > > > > assignments. > > > > > > > > > > > > > > What do you guys think? > > > > > > > > > > > > > > > > > > > Are you suggesting that we can only reuse the same > > > FairAffinityFunction > > > > > > across the logical caches within the same group? This would mean > > that > > > > > > caches from the different groups cannot participate in JOINs or > > > > > collocated > > > > > > compute. > > > > > > > > > > > > I think I like the idea, however, we need to make sure that we > > > enforce > > > > > this > > > > > > restriction, at least at the SQL JOIN level. > > > > > > > > > > > > Alexey G, Val, would be nice to hear your thoughts on this. > > > > > > > > > > > > > > > > > > > > > > > > > > Evgenii > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > View this message in context: http://apache-ignite- > > > > > > > developers.2346864.n4.nabble.com/Resurrect- > FairAffinityFunction- > > > > > > > tp19987p20669.html > > > > > > > Sent from the Apache Ignite Developers mailing list archive at > > > > > > Nabble.com. > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: Resurrect FairAffinityFunction
On Tue, Aug 15, 2017 at 10:35 PM, Vladimir Ozerov wrote: > Valya, > > And this is the problem! ) We have another implementation which minimizes > movements - rendezvous. Fair affinity should not bother about partition > migration. This is not a feature, but a bug in implementation of > FairAffinityFunction. Let's fix that and forget about topology versions. > Vladimir, you are wrong on this. If we did not care about redundant partition migrations, we would use a standard mod function to assign affinity. > > ср, 16 авг. 2017 г. в 3:23, Valentin Kulichenko < > valentin.kuliche...@gmail.com>: > > > Vladimir, > > > > I would let other guys confirm, but I believe the reason is that if it > > recalculates distribution every time from scratch, it would trigger too > > much redundant data movement during rebalancing. Fair function not only > > tries to provide best possible distribution, but also minimizes this data > > movement, and for this it uses previous distribution. > > > > -Val > > > > On Tue, Aug 15, 2017 at 1:12 PM, Vladimir Ozerov > > wrote: > > > > > I do not like the idea as it would make it very hard to reason about > > > whether your SQL will fail or not. Let's looks at the problem from the > > > different angle. I have this question for years - why in the world > *fair* > > > affinity function, whose only ultimate goal is to provide equal > partition > > > distribution, depends on it's own previous state? Can we re-design in a > > way > > > that it depends only on partition count and current topology state? > > > > > > On Thu, Aug 10, 2017 at 12:16 AM, Valentin Kulichenko < > > > valentin.kuliche...@gmail.com> wrote: > > > > > > > As far as I know, all logical caches with the same affinity function > > and > > > > node filter will end up in the same group. If that's the case, I like > > the > > > > idea. This is exactly what I was looking for. > > > > > > > > -Val > > > > > > > > On Wed, Aug 9, 2017 at 8:18 AM, Evgenii Zhuravlev < > > > > e.zhuravlev...@gmail.com> > > > > wrote: > > > > > > > > > Dmitriy, > > > > > > > > > > Yes, you're right. Moreover, it looks like a good practice to > combine > > > > > caches that will be used for collocated JOINs in one group since it > > > > reduces > > > > > overall overhead. > > > > > > > > > > I think it's not a problem to add this restriction to the SQL JOIN > > > level > > > > if > > > > > we will decide to use this solution. > > > > > > > > > > Evgenii > > > > > > > > > > > > > > > > > > > > > > > > > 2017-08-09 17:07 GMT+03:00 Dmitriy Setrakyan < > dsetrak...@apache.org > > >: > > > > > > > > > > > On Wed, Aug 9, 2017 at 6:28 AM, ezhuravl < > e.zhuravlev...@gmail.com > > > > > > > > wrote: > > > > > > > > > > > > > Folks, > > > > > > > > > > > > > > I've started working on a https://issues.apache.org/ > > > > > > > jira/browse/IGNITE-5836 > > > > > > > ticket and found that the recently added feature with > cacheGroups > > > > doing > > > > > > > pretty much the same that was described in this issue. > CacheGroup > > > > > > > guarantees > > > > > > > that all caches within a group have same assignments since they > > > > share a > > > > > > > single underlying 'physical' cache. > > > > > > > > > > > > > > > > > > > > I think we can return FairAffinityFunction and add information > to > > > its > > > > > > > Javadoc that all caches with same AffinityFunction and > NodeFilter > > > > > should > > > > > > be > > > > > > > combined in cache group to avoid a problem with inconsistent > > > previous > > > > > > > assignments. > > > > > > > > > > > > > > What do you guys think? > > > > > > > > > > > > > > > > > > > Are you suggesting that we can only reuse the same > > > FairAffinityFunction > > > > > > across the logical caches within the same group? This would mean > > that > > > > > > caches from the different groups cannot participate in JOINs or > > > > > collocated > > > > > > compute. > > > > > > > > > > > > I think I like the idea, however, we need to make sure that we > > > enforce > > > > > this > > > > > > restriction, at least at the SQL JOIN level. > > > > > > > > > > > > Alexey G, Val, would be nice to hear your thoughts on this. > > > > > > > > > > > > > > > > > > > > > > > > > > Evgenii > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > View this message in context: http://apache-ignite- > > > > > > > developers.2346864.n4.nabble.com/Resurrect- > FairAffinityFunction- > > > > > > > tp19987p20669.html > > > > > > > Sent from the Apache Ignite Developers mailing list archive at > > > > > > Nabble.com. > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: Resurrect FairAffinityFunction
Dima, It is not hard to implement. It is hard to reason on whether your query will fail or not. Moreover, cache groups is an antipattern for SQL, personally I do not want users to use it unless absolutely needed (large topologies, large number of caches). Also take in count that the same problem with different partition distribution holds for any two caches with different affinity functions, so the problem is not tied to FairAffinityFunction only. IMO correct fix should be as follows: 1) Add requirement that every affinity function must provide sensible implementation of equals/hashCode 2) Add "boolean deterministic()" method to affinity function interface; "true" means that function doesn't depend on any external things, such as topology history 3) Fail SQL if there are at least two PARTITIONED caches with different or non-deterministic affinity functions, and distributed joins are not enabled 4) Fix FairAffinityFunction and make it deterministic, it should not depend on previous state. Makes sense? On Wed, Aug 16, 2017 at 3:31 AM, Dmitriy Setrakyan wrote: > On Tue, Aug 15, 2017 at 1:12 PM, Vladimir Ozerov > wrote: > > > I do not like the idea as it would make it very hard to reason about > > whether your SQL will fail or not. Let's looks at the problem from the > > different angle. I have this question for years - why in the world *fair* > > affinity function, whose only ultimate goal is to provide equal partition > > distribution, depends on it's own previous state? Can we re-design in a > way > > that it depends only on partition count and current topology state? > > > > Vladimir, we must know previous state, otherwise the data partitions will > be randomly moving across the network every time a topology changes. > > From the SQL standpoint, you can just fail all queries that have a JOIN > from different cache groups, if at least one of the groups is using Fair > Affinity. I am not sure why this would be hard. > > > > > > On Thu, Aug 10, 2017 at 12:16 AM, Valentin Kulichenko < > > valentin.kuliche...@gmail.com> wrote: > > > > > As far as I know, all logical caches with the same affinity function > and > > > node filter will end up in the same group. If that's the case, I like > the > > > idea. This is exactly what I was looking for. > > > > > > -Val > > > > > > On Wed, Aug 9, 2017 at 8:18 AM, Evgenii Zhuravlev < > > > e.zhuravlev...@gmail.com> > > > wrote: > > > > > > > Dmitriy, > > > > > > > > Yes, you're right. Moreover, it looks like a good practice to combine > > > > caches that will be used for collocated JOINs in one group since it > > > reduces > > > > overall overhead. > > > > > > > > I think it's not a problem to add this restriction to the SQL JOIN > > level > > > if > > > > we will decide to use this solution. > > > > > > > > Evgenii > > > > > > > > > > > > > > > > > > > > 2017-08-09 17:07 GMT+03:00 Dmitriy Setrakyan >: > > > > > > > > > On Wed, Aug 9, 2017 at 6:28 AM, ezhuravl > > > > > wrote: > > > > > > > > > > > Folks, > > > > > > > > > > > > I've started working on a https://issues.apache.org/ > > > > > > jira/browse/IGNITE-5836 > > > > > > ticket and found that the recently added feature with cacheGroups > > > doing > > > > > > pretty much the same that was described in this issue. CacheGroup > > > > > > guarantees > > > > > > that all caches within a group have same assignments since they > > > share a > > > > > > single underlying 'physical' cache. > > > > > > > > > > > > > > > > > I think we can return FairAffinityFunction and add information to > > its > > > > > > Javadoc that all caches with same AffinityFunction and NodeFilter > > > > should > > > > > be > > > > > > combined in cache group to avoid a problem with inconsistent > > previous > > > > > > assignments. > > > > > > > > > > > > What do you guys think? > > > > > > > > > > > > > > > > Are you suggesting that we can only reuse the same > > FairAffinityFunction > > > > > across the logical caches within the same group? This would mean > that > > > > > caches from the different groups cannot participate in JOINs or > > > > collocated > > > > > compute. > > > > > > > > > > I think I like the idea, however, we need to make sure that we > > enforce > > > > this > > > > > restriction, at least at the SQL JOIN level. > > > > > > > > > > Alexey G, Val, would be nice to hear your thoughts on this. > > > > > > > > > > > > > > > > > > > > > > Evgenii > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > View this message in context: http://apache-ignite- > > > > > > developers.2346864.n4.nabble.com/Resurrect-FairAffinityFunction- > > > > > > tp19987p20669.html > > > > > > Sent from the Apache Ignite Developers mailing list archive at > > > > > Nabble.com. > > > > > > > > > > > > > > > > > > > > >
Re: Resurrect FairAffinityFunction
Valya, And this is the problem! ) We have another implementation which minimizes movements - rendezvous. Fair affinity should not bother about partition migration. This is not a feature, but a bug in implementation of FairAffinityFunction. Let's fix that and forget about topology versions. ср, 16 авг. 2017 г. в 3:23, Valentin Kulichenko < valentin.kuliche...@gmail.com>: > Vladimir, > > I would let other guys confirm, but I believe the reason is that if it > recalculates distribution every time from scratch, it would trigger too > much redundant data movement during rebalancing. Fair function not only > tries to provide best possible distribution, but also minimizes this data > movement, and for this it uses previous distribution. > > -Val > > On Tue, Aug 15, 2017 at 1:12 PM, Vladimir Ozerov > wrote: > > > I do not like the idea as it would make it very hard to reason about > > whether your SQL will fail or not. Let's looks at the problem from the > > different angle. I have this question for years - why in the world *fair* > > affinity function, whose only ultimate goal is to provide equal partition > > distribution, depends on it's own previous state? Can we re-design in a > way > > that it depends only on partition count and current topology state? > > > > On Thu, Aug 10, 2017 at 12:16 AM, Valentin Kulichenko < > > valentin.kuliche...@gmail.com> wrote: > > > > > As far as I know, all logical caches with the same affinity function > and > > > node filter will end up in the same group. If that's the case, I like > the > > > idea. This is exactly what I was looking for. > > > > > > -Val > > > > > > On Wed, Aug 9, 2017 at 8:18 AM, Evgenii Zhuravlev < > > > e.zhuravlev...@gmail.com> > > > wrote: > > > > > > > Dmitriy, > > > > > > > > Yes, you're right. Moreover, it looks like a good practice to combine > > > > caches that will be used for collocated JOINs in one group since it > > > reduces > > > > overall overhead. > > > > > > > > I think it's not a problem to add this restriction to the SQL JOIN > > level > > > if > > > > we will decide to use this solution. > > > > > > > > Evgenii > > > > > > > > > > > > > > > > > > > > 2017-08-09 17:07 GMT+03:00 Dmitriy Setrakyan >: > > > > > > > > > On Wed, Aug 9, 2017 at 6:28 AM, ezhuravl > > > > > wrote: > > > > > > > > > > > Folks, > > > > > > > > > > > > I've started working on a https://issues.apache.org/ > > > > > > jira/browse/IGNITE-5836 > > > > > > ticket and found that the recently added feature with cacheGroups > > > doing > > > > > > pretty much the same that was described in this issue. CacheGroup > > > > > > guarantees > > > > > > that all caches within a group have same assignments since they > > > share a > > > > > > single underlying 'physical' cache. > > > > > > > > > > > > > > > > > I think we can return FairAffinityFunction and add information to > > its > > > > > > Javadoc that all caches with same AffinityFunction and NodeFilter > > > > should > > > > > be > > > > > > combined in cache group to avoid a problem with inconsistent > > previous > > > > > > assignments. > > > > > > > > > > > > What do you guys think? > > > > > > > > > > > > > > > > Are you suggesting that we can only reuse the same > > FairAffinityFunction > > > > > across the logical caches within the same group? This would mean > that > > > > > caches from the different groups cannot participate in JOINs or > > > > collocated > > > > > compute. > > > > > > > > > > I think I like the idea, however, we need to make sure that we > > enforce > > > > this > > > > > restriction, at least at the SQL JOIN level. > > > > > > > > > > Alexey G, Val, would be nice to hear your thoughts on this. > > > > > > > > > > > > > > > > > > > > > > Evgenii > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > View this message in context: http://apache-ignite- > > > > > > developers.2346864.n4.nabble.com/Resurrect-FairAffinityFunction- > > > > > > tp19987p20669.html > > > > > > Sent from the Apache Ignite Developers mailing list archive at > > > > > Nabble.com. > > > > > > > > > > > > > > > > > > > > >
Re: Resurrect FairAffinityFunction
On Tue, Aug 15, 2017 at 1:12 PM, Vladimir Ozerov wrote: > I do not like the idea as it would make it very hard to reason about > whether your SQL will fail or not. Let's looks at the problem from the > different angle. I have this question for years - why in the world *fair* > affinity function, whose only ultimate goal is to provide equal partition > distribution, depends on it's own previous state? Can we re-design in a way > that it depends only on partition count and current topology state? > Vladimir, we must know previous state, otherwise the data partitions will be randomly moving across the network every time a topology changes. >From the SQL standpoint, you can just fail all queries that have a JOIN from different cache groups, if at least one of the groups is using Fair Affinity. I am not sure why this would be hard. > > On Thu, Aug 10, 2017 at 12:16 AM, Valentin Kulichenko < > valentin.kuliche...@gmail.com> wrote: > > > As far as I know, all logical caches with the same affinity function and > > node filter will end up in the same group. If that's the case, I like the > > idea. This is exactly what I was looking for. > > > > -Val > > > > On Wed, Aug 9, 2017 at 8:18 AM, Evgenii Zhuravlev < > > e.zhuravlev...@gmail.com> > > wrote: > > > > > Dmitriy, > > > > > > Yes, you're right. Moreover, it looks like a good practice to combine > > > caches that will be used for collocated JOINs in one group since it > > reduces > > > overall overhead. > > > > > > I think it's not a problem to add this restriction to the SQL JOIN > level > > if > > > we will decide to use this solution. > > > > > > Evgenii > > > > > > > > > > > > > > > 2017-08-09 17:07 GMT+03:00 Dmitriy Setrakyan : > > > > > > > On Wed, Aug 9, 2017 at 6:28 AM, ezhuravl > > > wrote: > > > > > > > > > Folks, > > > > > > > > > > I've started working on a https://issues.apache.org/ > > > > > jira/browse/IGNITE-5836 > > > > > ticket and found that the recently added feature with cacheGroups > > doing > > > > > pretty much the same that was described in this issue. CacheGroup > > > > > guarantees > > > > > that all caches within a group have same assignments since they > > share a > > > > > single underlying 'physical' cache. > > > > > > > > > > > > > > I think we can return FairAffinityFunction and add information to > its > > > > > Javadoc that all caches with same AffinityFunction and NodeFilter > > > should > > > > be > > > > > combined in cache group to avoid a problem with inconsistent > previous > > > > > assignments. > > > > > > > > > > What do you guys think? > > > > > > > > > > > > > Are you suggesting that we can only reuse the same > FairAffinityFunction > > > > across the logical caches within the same group? This would mean that > > > > caches from the different groups cannot participate in JOINs or > > > collocated > > > > compute. > > > > > > > > I think I like the idea, however, we need to make sure that we > enforce > > > this > > > > restriction, at least at the SQL JOIN level. > > > > > > > > Alexey G, Val, would be nice to hear your thoughts on this. > > > > > > > > > > > > > > > > > > Evgenii > > > > > > > > > > > > > > > > > > > > -- > > > > > View this message in context: http://apache-ignite- > > > > > developers.2346864.n4.nabble.com/Resurrect-FairAffinityFunction- > > > > > tp19987p20669.html > > > > > Sent from the Apache Ignite Developers mailing list archive at > > > > Nabble.com. > > > > > > > > > > > > > > >
Re: Resurrect FairAffinityFunction
Vladimir, I would let other guys confirm, but I believe the reason is that if it recalculates distribution every time from scratch, it would trigger too much redundant data movement during rebalancing. Fair function not only tries to provide best possible distribution, but also minimizes this data movement, and for this it uses previous distribution. -Val On Tue, Aug 15, 2017 at 1:12 PM, Vladimir Ozerov wrote: > I do not like the idea as it would make it very hard to reason about > whether your SQL will fail or not. Let's looks at the problem from the > different angle. I have this question for years - why in the world *fair* > affinity function, whose only ultimate goal is to provide equal partition > distribution, depends on it's own previous state? Can we re-design in a way > that it depends only on partition count and current topology state? > > On Thu, Aug 10, 2017 at 12:16 AM, Valentin Kulichenko < > valentin.kuliche...@gmail.com> wrote: > > > As far as I know, all logical caches with the same affinity function and > > node filter will end up in the same group. If that's the case, I like the > > idea. This is exactly what I was looking for. > > > > -Val > > > > On Wed, Aug 9, 2017 at 8:18 AM, Evgenii Zhuravlev < > > e.zhuravlev...@gmail.com> > > wrote: > > > > > Dmitriy, > > > > > > Yes, you're right. Moreover, it looks like a good practice to combine > > > caches that will be used for collocated JOINs in one group since it > > reduces > > > overall overhead. > > > > > > I think it's not a problem to add this restriction to the SQL JOIN > level > > if > > > we will decide to use this solution. > > > > > > Evgenii > > > > > > > > > > > > > > > 2017-08-09 17:07 GMT+03:00 Dmitriy Setrakyan : > > > > > > > On Wed, Aug 9, 2017 at 6:28 AM, ezhuravl > > > wrote: > > > > > > > > > Folks, > > > > > > > > > > I've started working on a https://issues.apache.org/ > > > > > jira/browse/IGNITE-5836 > > > > > ticket and found that the recently added feature with cacheGroups > > doing > > > > > pretty much the same that was described in this issue. CacheGroup > > > > > guarantees > > > > > that all caches within a group have same assignments since they > > share a > > > > > single underlying 'physical' cache. > > > > > > > > > > > > > > I think we can return FairAffinityFunction and add information to > its > > > > > Javadoc that all caches with same AffinityFunction and NodeFilter > > > should > > > > be > > > > > combined in cache group to avoid a problem with inconsistent > previous > > > > > assignments. > > > > > > > > > > What do you guys think? > > > > > > > > > > > > > Are you suggesting that we can only reuse the same > FairAffinityFunction > > > > across the logical caches within the same group? This would mean that > > > > caches from the different groups cannot participate in JOINs or > > > collocated > > > > compute. > > > > > > > > I think I like the idea, however, we need to make sure that we > enforce > > > this > > > > restriction, at least at the SQL JOIN level. > > > > > > > > Alexey G, Val, would be nice to hear your thoughts on this. > > > > > > > > > > > > > > > > > > Evgenii > > > > > > > > > > > > > > > > > > > > -- > > > > > View this message in context: http://apache-ignite- > > > > > developers.2346864.n4.nabble.com/Resurrect-FairAffinityFunction- > > > > > tp19987p20669.html > > > > > Sent from the Apache Ignite Developers mailing list archive at > > > > Nabble.com. > > > > > > > > > > > > > > >
Re: Resurrect FairAffinityFunction
I do not like the idea as it would make it very hard to reason about whether your SQL will fail or not. Let's looks at the problem from the different angle. I have this question for years - why in the world *fair* affinity function, whose only ultimate goal is to provide equal partition distribution, depends on it's own previous state? Can we re-design in a way that it depends only on partition count and current topology state? On Thu, Aug 10, 2017 at 12:16 AM, Valentin Kulichenko < valentin.kuliche...@gmail.com> wrote: > As far as I know, all logical caches with the same affinity function and > node filter will end up in the same group. If that's the case, I like the > idea. This is exactly what I was looking for. > > -Val > > On Wed, Aug 9, 2017 at 8:18 AM, Evgenii Zhuravlev < > e.zhuravlev...@gmail.com> > wrote: > > > Dmitriy, > > > > Yes, you're right. Moreover, it looks like a good practice to combine > > caches that will be used for collocated JOINs in one group since it > reduces > > overall overhead. > > > > I think it's not a problem to add this restriction to the SQL JOIN level > if > > we will decide to use this solution. > > > > Evgenii > > > > > > > > > > 2017-08-09 17:07 GMT+03:00 Dmitriy Setrakyan : > > > > > On Wed, Aug 9, 2017 at 6:28 AM, ezhuravl > > wrote: > > > > > > > Folks, > > > > > > > > I've started working on a https://issues.apache.org/ > > > > jira/browse/IGNITE-5836 > > > > ticket and found that the recently added feature with cacheGroups > doing > > > > pretty much the same that was described in this issue. CacheGroup > > > > guarantees > > > > that all caches within a group have same assignments since they > share a > > > > single underlying 'physical' cache. > > > > > > > > > > > I think we can return FairAffinityFunction and add information to its > > > > Javadoc that all caches with same AffinityFunction and NodeFilter > > should > > > be > > > > combined in cache group to avoid a problem with inconsistent previous > > > > assignments. > > > > > > > > What do you guys think? > > > > > > > > > > Are you suggesting that we can only reuse the same FairAffinityFunction > > > across the logical caches within the same group? This would mean that > > > caches from the different groups cannot participate in JOINs or > > collocated > > > compute. > > > > > > I think I like the idea, however, we need to make sure that we enforce > > this > > > restriction, at least at the SQL JOIN level. > > > > > > Alexey G, Val, would be nice to hear your thoughts on this. > > > > > > > > > > > > > > Evgenii > > > > > > > > > > > > > > > > -- > > > > View this message in context: http://apache-ignite- > > > > developers.2346864.n4.nabble.com/Resurrect-FairAffinityFunction- > > > > tp19987p20669.html > > > > Sent from the Apache Ignite Developers mailing list archive at > > > Nabble.com. > > > > > > > > > >
Re: Resurrect FairAffinityFunction
As far as I know, all logical caches with the same affinity function and node filter will end up in the same group. If that's the case, I like the idea. This is exactly what I was looking for. -Val On Wed, Aug 9, 2017 at 8:18 AM, Evgenii Zhuravlev wrote: > Dmitriy, > > Yes, you're right. Moreover, it looks like a good practice to combine > caches that will be used for collocated JOINs in one group since it reduces > overall overhead. > > I think it's not a problem to add this restriction to the SQL JOIN level if > we will decide to use this solution. > > Evgenii > > > > > 2017-08-09 17:07 GMT+03:00 Dmitriy Setrakyan : > > > On Wed, Aug 9, 2017 at 6:28 AM, ezhuravl > wrote: > > > > > Folks, > > > > > > I've started working on a https://issues.apache.org/ > > > jira/browse/IGNITE-5836 > > > ticket and found that the recently added feature with cacheGroups doing > > > pretty much the same that was described in this issue. CacheGroup > > > guarantees > > > that all caches within a group have same assignments since they share a > > > single underlying 'physical' cache. > > > > > > > > I think we can return FairAffinityFunction and add information to its > > > Javadoc that all caches with same AffinityFunction and NodeFilter > should > > be > > > combined in cache group to avoid a problem with inconsistent previous > > > assignments. > > > > > > What do you guys think? > > > > > > > Are you suggesting that we can only reuse the same FairAffinityFunction > > across the logical caches within the same group? This would mean that > > caches from the different groups cannot participate in JOINs or > collocated > > compute. > > > > I think I like the idea, however, we need to make sure that we enforce > this > > restriction, at least at the SQL JOIN level. > > > > Alexey G, Val, would be nice to hear your thoughts on this. > > > > > > > > > > Evgenii > > > > > > > > > > > > -- > > > View this message in context: http://apache-ignite- > > > developers.2346864.n4.nabble.com/Resurrect-FairAffinityFunction- > > > tp19987p20669.html > > > Sent from the Apache Ignite Developers mailing list archive at > > Nabble.com. > > > > > >
Re: Resurrect FairAffinityFunction
Dmitriy, Yes, you're right. Moreover, it looks like a good practice to combine caches that will be used for collocated JOINs in one group since it reduces overall overhead. I think it's not a problem to add this restriction to the SQL JOIN level if we will decide to use this solution. Evgenii 2017-08-09 17:07 GMT+03:00 Dmitriy Setrakyan : > On Wed, Aug 9, 2017 at 6:28 AM, ezhuravl wrote: > > > Folks, > > > > I've started working on a https://issues.apache.org/ > > jira/browse/IGNITE-5836 > > ticket and found that the recently added feature with cacheGroups doing > > pretty much the same that was described in this issue. CacheGroup > > guarantees > > that all caches within a group have same assignments since they share a > > single underlying 'physical' cache. > > > > > I think we can return FairAffinityFunction and add information to its > > Javadoc that all caches with same AffinityFunction and NodeFilter should > be > > combined in cache group to avoid a problem with inconsistent previous > > assignments. > > > > What do you guys think? > > > > Are you suggesting that we can only reuse the same FairAffinityFunction > across the logical caches within the same group? This would mean that > caches from the different groups cannot participate in JOINs or collocated > compute. > > I think I like the idea, however, we need to make sure that we enforce this > restriction, at least at the SQL JOIN level. > > Alexey G, Val, would be nice to hear your thoughts on this. > > > > > > Evgenii > > > > > > > > -- > > View this message in context: http://apache-ignite- > > developers.2346864.n4.nabble.com/Resurrect-FairAffinityFunction- > > tp19987p20669.html > > Sent from the Apache Ignite Developers mailing list archive at > Nabble.com. > > >
Re: Resurrect FairAffinityFunction
On Wed, Aug 9, 2017 at 6:28 AM, ezhuravl wrote: > Folks, > > I've started working on a https://issues.apache.org/ > jira/browse/IGNITE-5836 > ticket and found that the recently added feature with cacheGroups doing > pretty much the same that was described in this issue. CacheGroup > guarantees > that all caches within a group have same assignments since they share a > single underlying 'physical' cache. > > I think we can return FairAffinityFunction and add information to its > Javadoc that all caches with same AffinityFunction and NodeFilter should be > combined in cache group to avoid a problem with inconsistent previous > assignments. > > What do you guys think? > Are you suggesting that we can only reuse the same FairAffinityFunction across the logical caches within the same group? This would mean that caches from the different groups cannot participate in JOINs or collocated compute. I think I like the idea, however, we need to make sure that we enforce this restriction, at least at the SQL JOIN level. Alexey G, Val, would be nice to hear your thoughts on this. > > Evgenii > > > > -- > View this message in context: http://apache-ignite- > developers.2346864.n4.nabble.com/Resurrect-FairAffinityFunction- > tp19987p20669.html > Sent from the Apache Ignite Developers mailing list archive at Nabble.com. >
Re: Resurrect FairAffinityFunction
Folks, I've started working on a https://issues.apache.org/jira/browse/IGNITE-5836 ticket and found that the recently added feature with cacheGroups doing pretty much the same that was described in this issue. CacheGroup guarantees that all caches within a group have same assignments since they share a single underlying 'physical' cache. I think we can return FairAffinityFunction and add information to its Javadoc that all caches with same AffinityFunction and NodeFilter should be combined in cache group to avoid a problem with inconsistent previous assignments. What do you guys think? Evgenii -- View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/Resurrect-FairAffinityFunction-tp19987p20669.html Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
Re: Resurrect FairAffinityFunction
Create a ticket: https://issues.apache.org/jira/browse/IGNITE-5836 -Val On Tue, Jul 25, 2017 at 11:54 AM, Valentin Kulichenko < valentin.kuliche...@gmail.com> wrote: > Semyon, > > We had some improvements, but to knowledge fair affinity still provides > much better distribution (at least I haven't seen any results showing > otherwise). Please correct me if I'm wrong. > > Actually, I think it's not an issue with fair function in particular, but > rather a design flow in affinity manager. The exact same issue will exist > not only with fair function, but with ANY function that > uses AffinityFunctionContext#previousAssignment to calculate assignments. > And the context is provided from the outside, function has nothing to do > with it. > > So let's fix the root cause and bring innocent FairAF back :) > > -Val > > On Tue, Jul 25, 2017 at 1:07 AM, Semyon Boikov > wrote: > >> Valentin, >> >> As far as I know in 2.0 some changes were made in rendezvous function so >> now it can provide better result. Do you have some numbers for 2.0 so that >> we can compare rendezvous and fair affinity functions? >> >> Thanks >> >> On Tue, Jul 25, 2017 at 5:13 AM, wrote: >> >> > Agree with Val, we should bring it back. >> > >> > D. >> > >> > On Jul 24, 2017, 8:14 PM, at 8:14 PM, Valentin Kulichenko < >> > valentin.kuliche...@gmail.com> wrote: >> > >Guys, >> > > >> > >Some time ago we removed FairAffinityFunction from the project. >> > >However, my >> > >communication with users clearly shows that is was a rush decision. >> > >Distribution showed by Fair AF is much better than default and for some >> > >users it's extremely important. Basically, there are cases when >> > >rendezvous >> > >function is no-go. >> > > >> > >The reason for removal was that it was possible to get inconsistent >> > >results >> > >in case multiple caches were created on different topologies. However, >> > >I >> > >think this is fixable. As far as I understand, the only thing we need >> > >to do >> > >is to maintain a single AffinityFunctionContext for all the caches with >> > >same affinity function. Currently for each cache we have separate >> > >context >> > >which holds the state used by Fair AF. If the state is different, we >> > >have >> > >an issue. >> > > >> > >The only question is how to check whether two functions are the same or >> > >not. In case both cache node filter and backup filter are not >> > >configured, >> > >this is easy - if number of partitions and excludeNeighbors flag are >> > >equal >> > >for two functions, these functions are also equal. >> > > >> > >With filters it's a bit more complicated as these are custom >> > >implementations and in general case we don't know how to compare them. >> > >Although, to solve this problem, we can enforce user to implement >> > >equals() >> > >method for these implementation if Fair AF is used. >> > > >> > >I propose to make changes described above and bring Fair AF back. >> > > >> > >Thoughts? >> > > >> > >-Val >> > >> > >
Re: Resurrect FairAffinityFunction
Semyon, We had some improvements, but to knowledge fair affinity still provides much better distribution (at least I haven't seen any results showing otherwise). Please correct me if I'm wrong. Actually, I think it's not an issue with fair function in particular, but rather a design flow in affinity manager. The exact same issue will exist not only with fair function, but with ANY function that uses AffinityFunctionContext#previousAssignment to calculate assignments. And the context is provided from the outside, function has nothing to do with it. So let's fix the root cause and bring innocent FairAF back :) -Val On Tue, Jul 25, 2017 at 1:07 AM, Semyon Boikov wrote: > Valentin, > > As far as I know in 2.0 some changes were made in rendezvous function so > now it can provide better result. Do you have some numbers for 2.0 so that > we can compare rendezvous and fair affinity functions? > > Thanks > > On Tue, Jul 25, 2017 at 5:13 AM, wrote: > > > Agree with Val, we should bring it back. > > > > D. > > > > On Jul 24, 2017, 8:14 PM, at 8:14 PM, Valentin Kulichenko < > > valentin.kuliche...@gmail.com> wrote: > > >Guys, > > > > > >Some time ago we removed FairAffinityFunction from the project. > > >However, my > > >communication with users clearly shows that is was a rush decision. > > >Distribution showed by Fair AF is much better than default and for some > > >users it's extremely important. Basically, there are cases when > > >rendezvous > > >function is no-go. > > > > > >The reason for removal was that it was possible to get inconsistent > > >results > > >in case multiple caches were created on different topologies. However, > > >I > > >think this is fixable. As far as I understand, the only thing we need > > >to do > > >is to maintain a single AffinityFunctionContext for all the caches with > > >same affinity function. Currently for each cache we have separate > > >context > > >which holds the state used by Fair AF. If the state is different, we > > >have > > >an issue. > > > > > >The only question is how to check whether two functions are the same or > > >not. In case both cache node filter and backup filter are not > > >configured, > > >this is easy - if number of partitions and excludeNeighbors flag are > > >equal > > >for two functions, these functions are also equal. > > > > > >With filters it's a bit more complicated as these are custom > > >implementations and in general case we don't know how to compare them. > > >Although, to solve this problem, we can enforce user to implement > > >equals() > > >method for these implementation if Fair AF is used. > > > > > >I propose to make changes described above and bring Fair AF back. > > > > > >Thoughts? > > > > > >-Val > > >
Re: Resurrect FairAffinityFunction
Valentin, As far as I know in 2.0 some changes were made in rendezvous function so now it can provide better result. Do you have some numbers for 2.0 so that we can compare rendezvous and fair affinity functions? Thanks On Tue, Jul 25, 2017 at 5:13 AM, wrote: > Agree with Val, we should bring it back. > > D. > > On Jul 24, 2017, 8:14 PM, at 8:14 PM, Valentin Kulichenko < > valentin.kuliche...@gmail.com> wrote: > >Guys, > > > >Some time ago we removed FairAffinityFunction from the project. > >However, my > >communication with users clearly shows that is was a rush decision. > >Distribution showed by Fair AF is much better than default and for some > >users it's extremely important. Basically, there are cases when > >rendezvous > >function is no-go. > > > >The reason for removal was that it was possible to get inconsistent > >results > >in case multiple caches were created on different topologies. However, > >I > >think this is fixable. As far as I understand, the only thing we need > >to do > >is to maintain a single AffinityFunctionContext for all the caches with > >same affinity function. Currently for each cache we have separate > >context > >which holds the state used by Fair AF. If the state is different, we > >have > >an issue. > > > >The only question is how to check whether two functions are the same or > >not. In case both cache node filter and backup filter are not > >configured, > >this is easy - if number of partitions and excludeNeighbors flag are > >equal > >for two functions, these functions are also equal. > > > >With filters it's a bit more complicated as these are custom > >implementations and in general case we don't know how to compare them. > >Although, to solve this problem, we can enforce user to implement > >equals() > >method for these implementation if Fair AF is used. > > > >I propose to make changes described above and bring Fair AF back. > > > >Thoughts? > > > >-Val >
Re: Resurrect FairAffinityFunction
Agree with Val, we should bring it back. D. On Jul 24, 2017, 8:14 PM, at 8:14 PM, Valentin Kulichenko wrote: >Guys, > >Some time ago we removed FairAffinityFunction from the project. >However, my >communication with users clearly shows that is was a rush decision. >Distribution showed by Fair AF is much better than default and for some >users it's extremely important. Basically, there are cases when >rendezvous >function is no-go. > >The reason for removal was that it was possible to get inconsistent >results >in case multiple caches were created on different topologies. However, >I >think this is fixable. As far as I understand, the only thing we need >to do >is to maintain a single AffinityFunctionContext for all the caches with >same affinity function. Currently for each cache we have separate >context >which holds the state used by Fair AF. If the state is different, we >have >an issue. > >The only question is how to check whether two functions are the same or >not. In case both cache node filter and backup filter are not >configured, >this is easy - if number of partitions and excludeNeighbors flag are >equal >for two functions, these functions are also equal. > >With filters it's a bit more complicated as these are custom >implementations and in general case we don't know how to compare them. >Although, to solve this problem, we can enforce user to implement >equals() >method for these implementation if Fair AF is used. > >I propose to make changes described above and bring Fair AF back. > >Thoughts? > >-Val
Resurrect FairAffinityFunction
Guys, Some time ago we removed FairAffinityFunction from the project. However, my communication with users clearly shows that is was a rush decision. Distribution showed by Fair AF is much better than default and for some users it's extremely important. Basically, there are cases when rendezvous function is no-go. The reason for removal was that it was possible to get inconsistent results in case multiple caches were created on different topologies. However, I think this is fixable. As far as I understand, the only thing we need to do is to maintain a single AffinityFunctionContext for all the caches with same affinity function. Currently for each cache we have separate context which holds the state used by Fair AF. If the state is different, we have an issue. The only question is how to check whether two functions are the same or not. In case both cache node filter and backup filter are not configured, this is easy - if number of partitions and excludeNeighbors flag are equal for two functions, these functions are also equal. With filters it's a bit more complicated as these are custom implementations and in general case we don't know how to compare them. Although, to solve this problem, we can enforce user to implement equals() method for these implementation if Fair AF is used. I propose to make changes described above and bring Fair AF back. Thoughts? -Val