Re: [DISCUSSION] Customers Migraton from Thick to Thin Clients
Folks, why should we construct a fully functional affinity function on the
client side by custom client code? We need only a key to partition mapping,
so only this simple mapper factory will be enough.
>From my point of view there are to options possible:
- provide ability to set partition to key mapper factory by the client code
or
- serialize custom affinity functions and custom affinity key mappers to
binary objects and send it together with ClientCachePartitionResponse
message. In this case, explicit client configuration or some kind of client
code is not required, affinity function (with type id and its internals)
will be deserialized if classes are present on thin-client side and will be
used implicitly (for java thin client). For other thin clients (except
java) type id can be retrieved from the binary object and some key to
partition mapper can be provided based on this type id (for example by
configured factory).
пн, 11 июл. 2022 г. в 16:22, Pavel Tupitsyn :
> No objections to the AffinityFunctionFactory approach from my side.
> I think it should be based on cacheName, not groupId.
>
> On Mon, Jul 11, 2022 at 3:52 PM Maxim Muzafarov wrote:
>
> > Folks,
> >
> > I've done research about the proposed solution and I'd like to discuss
> > with you a possible options we may have. After we agreed on a solution
> > design I'll create a new IEP with everything we've discussed above.
> >
> > So, let's consider the most completed case:
> > the RendezvousAffinityFunction with a custom affinity mapper function is
> > used.
> >
> >
> > =
> >
> > The solution with sending AffintyFunction typeId.
> >
> > a. The deprecated AffinityKeyMapper used prior to the AffinityFunction
> > for calculation a key to partition mapping.
> > See the link [1] - affFunction.partition(affinityKey(key))
> >
> > b. We are able to map typeId of the RendezvousAffinityFunction to
> > another AffinityFunction on the client side to solve the mapping
> > problem with any combination of AffinityFunction and
> > AffinityKeyMappers ware used.
> >
> > c. When the cache partitions mapping request [2] is executed we are
> > able to send the typeId of the RendezvousAffinityFunction back to the
> > client, however without sending the typeId's of custom affinity
> > mappers we are not able to identify the case when a substitution of
> > the AffinityFunction is required on the client side.
> >
> > For instance,
> >
> > cacheGroup_1 has RendezvousAffinityFunction + FirstAffinityKeyMapper
> > cacheGroup_2 has RendezvousAffinityFunction + SecondAffinityKeyMapper
> >
> > Adding a deprecated classes and their corresponding typeId's to the
> > exchange client-server protocol sound not a good idea. However, this
> > will cover all the cases.
> >
> > =
> >
> > The solution with factory method for AffintyFunction on the client side.
> >
> > We can also improve the solution that was proposed by me a few
> > messages ago. Instead of the withPartitionAwarenessKeyMapper() [4]
> > method which looks a bit weird we can add a factory method to the
> > IgniteClientConfiguration that will provide a custom AffinityFunction
> > for caches wift `non-default` mappings.
> >
> > The factory may looks like:
> >
> > class AffinityFunctionFactory implements BiFunction > AffinityFunction> {
> > @Override public AffinityFunction apply(Integer groupId, Integer
> > parts) {
> > return new RendezvousAffinityFunction(false, parts);
> > }
> > }
> >
> > This cases will have a bit straightforward implementation and explicit
> > AffinityFunction initialization by a user.
> >
> >
> > WDYT?
> >
> >
> >
> > [1]
> >
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheAffinityManager.java#L185
> > [2]
> >
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/platform/client/ClientMessageParser.java#L535
> > [3]
> >
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/cache/affinity/AffinityFunction.java#L53
> > [4]
> >
> https://github.com/apache/ignite/pull/10140/files#diff-fb60155466fa2c31d6d1270c0e08f15f1ad9c1a2272fc9bb137c07d9ae5c1b98R47
> >
> > On Mon, 11 Jul 2022 at 13:14, Ivan Daschinsky
> wrote:
> > >
> > > Still don't understand how type id can help me to obtain valid mapper
> > from
> > > key to int32. Especially when we are talking about non-jvm languages.
> > >
> > > пн, 11 июл. 2022 г., 12:33 Николай Ижиков :
> > >
> > > > Pavel.
> > > >
> > > > > It is not cryptic, it is very straightforward and builds on
> existing
> > > > binary type mechanism.
> > > >
> > > > I see the Ivan’s point here.
> > > > As I said earlier - It must be absolutely clear for user - PA works
> or
> > not.
> > > >
> > > > With the logic you propose it will be hiding inside Ignite machinery
> > and
> > > > logs.
> > > >
> > > > > Unfortunately, this is a breaking change. Curre
[MTCGA]: new failures in builds [6511383] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. If your changes can lead to this failure(s): We're grateful that you were a volunteer to make the contribution to this project, but things change and you may no longer be able to finalize your contribution. Could you respond to this email and indicate if you wish to continue and fix test failures or step down and some committer may revert you commit. *Test with high flaky rate in master ZookeeperDiscoveryTopologyChangeAndReconnectTest.testRandomTopologyChanges_CloseClients https://ci2.ignite.apache.org/test/5664736877531567951?currentProjectId=IgniteTests24Java8&branch=%3Cdefault%3E Changes may lead to failure were done by - nikita amelchev https://ci2.ignite.apache.org/viewModification.html?modId=976717 - nikolay https://ci2.ignite.apache.org/viewModification.html?modId=976712 - sergey korotkov https://ci2.ignite.apache.org/viewModification.html?modId=976757 - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 04:38:23 12-07-2022
Re: OSGi metadata fixes for Ignite 2.x
Hello Maxim, My replies inline On 11.07.2022 15:57, Maxim Muzafarov wrote: Hello Łukasz, One of major problems which are hard to solve without cooperation is mentioned "split package" which is currently found in the sources. Can you clarify details of what is the issue of having such a case? I my understanding it will be better to consider ignite-core + ignite-indexing + ignite-calcite as a single feature. Thus having a "split package" issue should not be a problem. It will be a quite challenging to keep these modules thruly without package duplication in future. Split package happens when same package is defined by more than one module. In such situation OSGi framework will stick with first provider of a given package (ie. ignite-core), simply ignoring subsequent code chunks provided by other modules (think of ignite-indexing). Because OSGi validate wiring at a package level, it has some impact on how code should be structured. Simply speaking module can not import a foreign package and export it with additional contents. I attempted to duplicate some of split package contents within ignite-indexing but it should be seen as a temporary workaround. I've also briefly looked at the integration with the Apache Karaf. Will it be possible to remove a module dependencies from the feature-file [4]? It seems it's a bit complicated to maintain these dependencies and corresponding versions each time some module changed. These dependencies are recommended as it will force proper build order of modules. Without it you may end up with a situation that feature validation involving ignite-core will be made before before ignite-core is being built locally. Relying on sequence order of elements in reactor pom is often insufficient to assure that. The issue IGNITE-17046 [1] you mentioned above was merged. I've also moved the OSGi sub-modules to the Apache Ignite Extensions project [2], so I hope we can proceed here. Please, note that I've removed the ignite-paxloggin module due to it is completely out of date. The ignite-log4j will be removed very soon [3], so we have to migrate everything to the ignite-log4j2. Fair enough, I can take it forward from that place and try to come with a clear Karaf feature set also for some of ignite extensions. I noticed some of them were refereed. I've also take a look at the packages you mentioned above. Here is the list of packages with my comments: org.apache.ignite.internal.managers.systemview.walker (will be moved soon from ignite-indexing to the ignite-core) org.apache.ignite.internal.mxbean (ignite-indexing - classes deprecated and will be removed here [5]) org.apache.ignite.internal.processors.cache.query (ignite-indexing - I doubt it can be simply removed currently, refactoring is required) org.apache.ignite.internal.processors.query.stat (ignite-indexing - I doubt we can do anything here) org.apache.ignite.internal.visor.cache.index (ignite-indexing - we can simply move everything to the ignite-core) org.apache.ignite.internal.visor.verify (ignite-indexing - we can simply move everything to the ignite-core) org.apache.ignite.internal.managers.systemview (will be moved soon from ignite-indexing to the ignite-core) I understand that moving code is difficult and causes a friction to dependent projects, so easiest path is moving dependencies up (to ignite-core), making it even bigger. We do not need a full functinality re-factoring to satisfy "singularity" of a package. It is sufficient to have unique name for package structure. For example: org.apache.ignite.internal.processors.cache.query in indexing could be: - org.apache.ignite.internal.processors.cache.indexing.query - org.apache.ignite.indexing.internal.processors.cache.query or similar. Best, Łukasz [1] https://issues.apache.org/jira/browse/IGNITE-17046 [2] https://issues.apache.org/jira/browse/IGNITE-13570 [3] https://issues.apache.org/jira/browse/IGNITE-16650 [4] https://github.com/apache/ignite-extensions/blob/master/modules/osgi-ext/osgi-karaf/src/main/resources/features.xml#L131 [5] https://issues.apache.org/jira/browse/IGNITE-15761 On Mon, 4 Jul 2022 at 21:42, Łukasz Dywicki wrote: Hello Maxim, Thank you for your answer. I haven't expected such quick feedback. I am here to help you with OSGi, I believe that together we will be able to get Ignite free of troubles it gives. I abstained from creating new issues as there are two which already outline end user problem: IGNITE-17216, IGNITE-17216. I do not see a problem with moving ignite-osgi module to other repository as long as we avoid dependency cycles. Since igite-osgi functionality seem to be limited to bootstrapping ignite and its classloading it does not depend per say on where code itself is located. I suppose that osgi-karaf and osgi-paxlogging should follow that move to keep all osgi related things in one place. Major concern I can raise is how to keep it up so it does not sink over time. Integration tests I
Re: OSGi metadata fixes for Ignite 2.x
Hello Łukasz, > One of major problems which are hard to solve without cooperation is > mentioned "split package" which is currently found in the sources. Can you clarify details of what is the issue of having such a case? I my understanding it will be better to consider ignite-core + ignite-indexing + ignite-calcite as a single feature. Thus having a "split package" issue should not be a problem. It will be a quite challenging to keep these modules thruly without package duplication in future. I've also briefly looked at the integration with the Apache Karaf. Will it be possible to remove a module dependencies from the feature-file [4]? It seems it's a bit complicated to maintain these dependencies and corresponding versions each time some module changed. The issue IGNITE-17046 [1] you mentioned above was merged. I've also moved the OSGi sub-modules to the Apache Ignite Extensions project [2], so I hope we can proceed here. Please, note that I've removed the ignite-paxloggin module due to it is completely out of date. The ignite-log4j will be removed very soon [3], so we have to migrate everything to the ignite-log4j2. I've also take a look at the packages you mentioned above. Here is the list of packages with my comments: org.apache.ignite.internal.managers.systemview.walker (will be moved soon from ignite-indexing to the ignite-core) org.apache.ignite.internal.mxbean (ignite-indexing - classes deprecated and will be removed here [5]) org.apache.ignite.internal.processors.cache.query (ignite-indexing - I doubt it can be simply removed currently, refactoring is required) org.apache.ignite.internal.processors.query.stat (ignite-indexing - I doubt we can do anything here) org.apache.ignite.internal.visor.cache.index (ignite-indexing - we can simply move everything to the ignite-core) org.apache.ignite.internal.visor.verify (ignite-indexing - we can simply move everything to the ignite-core) org.apache.ignite.internal.managers.systemview (will be moved soon from ignite-indexing to the ignite-core) [1] https://issues.apache.org/jira/browse/IGNITE-17046 [2] https://issues.apache.org/jira/browse/IGNITE-13570 [3] https://issues.apache.org/jira/browse/IGNITE-16650 [4] https://github.com/apache/ignite-extensions/blob/master/modules/osgi-ext/osgi-karaf/src/main/resources/features.xml#L131 [5] https://issues.apache.org/jira/browse/IGNITE-15761 On Mon, 4 Jul 2022 at 21:42, Łukasz Dywicki wrote: > > Hello Maxim, > Thank you for your answer. I haven't expected such quick feedback. I am > here to help you with OSGi, I believe that together we will be able to > get Ignite free of troubles it gives. > > I abstained from creating new issues as there are two which already > outline end user problem: IGNITE-17216, IGNITE-17216. > > I do not see a problem with moving ignite-osgi module to other > repository as long as we avoid dependency cycles. > Since igite-osgi functionality seem to be limited to bootstrapping > ignite and its classloading it does not depend per say on where code > itself is located. I suppose that osgi-karaf and osgi-paxlogging should > follow that move to keep all osgi related things in one place. > > Major concern I can raise is how to keep it up so it does not sink over > time. Integration tests I mentioned will form a safety net, but they > need to be part of build pipeline to follow changes in main repository. > Some of recent changes I found about cleaning out some inner-dependency > paths (ie. indexing-h2 relation: IGNITE-17046), are very helpful to > solidify OSGi metadata. > One of major problems which are hard to solve without cooperation is > mentioned "split package" which is currently found in the sources. This > happens when same package is defined (and exported) from two places. > Currently both ignite-core and indexing bring contents of > org.apache.ignite.internal and spi packages: > - org.apache.ignite.internal.managers.systemview.walker > - org.apache.ignite.internal.mxbean > - org.apache.ignite.internal.processors.cache.query > - org.apache.ignite.internal.processors.query.stat > - org.apache.ignite.internal.visor.cache.index > - org.apache.ignite.internal.visor.verify > - org.apache.ignite.spi.systemview.view > To be fair, I don't know yet what is scope of these and that's where I > need most of your guidance. I don't think we will be able to stabilize > osgi integration to guarantee it will work reliably if these stay. > > Kind regards, > Łukasz > > On 4.07.2022 19:43, Maxim Muzafarov wrote: > > Hello Łukasz, > > > > Nice to meet you! > > > > Currently there are some known issues with the Apache Ignite OSGi > > integration. We have a lack of time and knowledg to support it, so any > > help are very appreciated. I also have a a plan to move this > > integration to the Apache Ignite Extensions project [1] and release it > > when all the issues are resolved. Here is an umbrella ticket for it > > [2]. > > > > What do you think if I move the OSGi to the Apache Ignite
Re: [DISCUSSION] Customers Migraton from Thick to Thin Clients
No objections to the AffinityFunctionFactory approach from my side.
I think it should be based on cacheName, not groupId.
On Mon, Jul 11, 2022 at 3:52 PM Maxim Muzafarov wrote:
> Folks,
>
> I've done research about the proposed solution and I'd like to discuss
> with you a possible options we may have. After we agreed on a solution
> design I'll create a new IEP with everything we've discussed above.
>
> So, let's consider the most completed case:
> the RendezvousAffinityFunction with a custom affinity mapper function is
> used.
>
>
> =
>
> The solution with sending AffintyFunction typeId.
>
> a. The deprecated AffinityKeyMapper used prior to the AffinityFunction
> for calculation a key to partition mapping.
> See the link [1] - affFunction.partition(affinityKey(key))
>
> b. We are able to map typeId of the RendezvousAffinityFunction to
> another AffinityFunction on the client side to solve the mapping
> problem with any combination of AffinityFunction and
> AffinityKeyMappers ware used.
>
> c. When the cache partitions mapping request [2] is executed we are
> able to send the typeId of the RendezvousAffinityFunction back to the
> client, however without sending the typeId's of custom affinity
> mappers we are not able to identify the case when a substitution of
> the AffinityFunction is required on the client side.
>
> For instance,
>
> cacheGroup_1 has RendezvousAffinityFunction + FirstAffinityKeyMapper
> cacheGroup_2 has RendezvousAffinityFunction + SecondAffinityKeyMapper
>
> Adding a deprecated classes and their corresponding typeId's to the
> exchange client-server protocol sound not a good idea. However, this
> will cover all the cases.
>
> =
>
> The solution with factory method for AffintyFunction on the client side.
>
> We can also improve the solution that was proposed by me a few
> messages ago. Instead of the withPartitionAwarenessKeyMapper() [4]
> method which looks a bit weird we can add a factory method to the
> IgniteClientConfiguration that will provide a custom AffinityFunction
> for caches wift `non-default` mappings.
>
> The factory may looks like:
>
> class AffinityFunctionFactory implements BiFunction AffinityFunction> {
> @Override public AffinityFunction apply(Integer groupId, Integer
> parts) {
> return new RendezvousAffinityFunction(false, parts);
> }
> }
>
> This cases will have a bit straightforward implementation and explicit
> AffinityFunction initialization by a user.
>
>
> WDYT?
>
>
>
> [1]
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheAffinityManager.java#L185
> [2]
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/platform/client/ClientMessageParser.java#L535
> [3]
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/cache/affinity/AffinityFunction.java#L53
> [4]
> https://github.com/apache/ignite/pull/10140/files#diff-fb60155466fa2c31d6d1270c0e08f15f1ad9c1a2272fc9bb137c07d9ae5c1b98R47
>
> On Mon, 11 Jul 2022 at 13:14, Ivan Daschinsky wrote:
> >
> > Still don't understand how type id can help me to obtain valid mapper
> from
> > key to int32. Especially when we are talking about non-jvm languages.
> >
> > пн, 11 июл. 2022 г., 12:33 Николай Ижиков :
> >
> > > Pavel.
> > >
> > > > It is not cryptic, it is very straightforward and builds on existing
> > > binary type mechanism.
> > >
> > > I see the Ivan’s point here.
> > > As I said earlier - It must be absolutely clear for user - PA works or
> not.
> > >
> > > With the logic you propose it will be hiding inside Ignite machinery
> and
> > > logs.
> > >
> > > > Unfortunately, this is a breaking change. Currently this scenario
> works
> > > without errors - uses default socket for custom affinity caches.
> > >
> > > It’s OK from my point of view to introduce such a change.
> > >
> > >
> > > > 10 июля 2022 г., в 22:05, Pavel Tupitsyn
> > > написал(а):
> > > >
> > > >> providing simple function Object -> int32 in cache configuration is
> > > wrong
> > > > decision
> > > >
> > > > Because it is error-prone and does not work for all cases:
> > > > - You have to set it explicitly on the client for every cache.
> > > > - No way to do that if you get an existing cache by name.
> > > >
> > > >> still being unable to solve the problem
> > > >
> > > > I believe the proposed solution covers all use cases.
> > > >
> > > >> cryptic logic
> > > >
> > > > It is not cryptic, it is very straightforward and builds on existing
> > > binary
> > > > type mechanism.
> > > >
> > > >> feature flags
> > > >
> > > > What's bad about feature flags?
> > > >
> > > >> how an ability to obtain a classname of java class can help my
> python or
> > > > C++ client map key to partition
> > > >
> > > > Not a java class name, but type id. You can set up type id mapping
> > > properly
> > > > and use this in any thin client
Re: [DISCUSSION] Customers Migraton from Thick to Thin Clients
Folks,
I've done research about the proposed solution and I'd like to discuss
with you a possible options we may have. After we agreed on a solution
design I'll create a new IEP with everything we've discussed above.
So, let's consider the most completed case:
the RendezvousAffinityFunction with a custom affinity mapper function is used.
=
The solution with sending AffintyFunction typeId.
a. The deprecated AffinityKeyMapper used prior to the AffinityFunction
for calculation a key to partition mapping.
See the link [1] - affFunction.partition(affinityKey(key))
b. We are able to map typeId of the RendezvousAffinityFunction to
another AffinityFunction on the client side to solve the mapping
problem with any combination of AffinityFunction and
AffinityKeyMappers ware used.
c. When the cache partitions mapping request [2] is executed we are
able to send the typeId of the RendezvousAffinityFunction back to the
client, however without sending the typeId's of custom affinity
mappers we are not able to identify the case when a substitution of
the AffinityFunction is required on the client side.
For instance,
cacheGroup_1 has RendezvousAffinityFunction + FirstAffinityKeyMapper
cacheGroup_2 has RendezvousAffinityFunction + SecondAffinityKeyMapper
Adding a deprecated classes and their corresponding typeId's to the
exchange client-server protocol sound not a good idea. However, this
will cover all the cases.
=
The solution with factory method for AffintyFunction on the client side.
We can also improve the solution that was proposed by me a few
messages ago. Instead of the withPartitionAwarenessKeyMapper() [4]
method which looks a bit weird we can add a factory method to the
IgniteClientConfiguration that will provide a custom AffinityFunction
for caches wift `non-default` mappings.
The factory may looks like:
class AffinityFunctionFactory implements BiFunction {
@Override public AffinityFunction apply(Integer groupId, Integer parts) {
return new RendezvousAffinityFunction(false, parts);
}
}
This cases will have a bit straightforward implementation and explicit
AffinityFunction initialization by a user.
WDYT?
[1]
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheAffinityManager.java#L185
[2]
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/platform/client/ClientMessageParser.java#L535
[3]
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/cache/affinity/AffinityFunction.java#L53
[4]
https://github.com/apache/ignite/pull/10140/files#diff-fb60155466fa2c31d6d1270c0e08f15f1ad9c1a2272fc9bb137c07d9ae5c1b98R47
On Mon, 11 Jul 2022 at 13:14, Ivan Daschinsky wrote:
>
> Still don't understand how type id can help me to obtain valid mapper from
> key to int32. Especially when we are talking about non-jvm languages.
>
> пн, 11 июл. 2022 г., 12:33 Николай Ижиков :
>
> > Pavel.
> >
> > > It is not cryptic, it is very straightforward and builds on existing
> > binary type mechanism.
> >
> > I see the Ivan’s point here.
> > As I said earlier - It must be absolutely clear for user - PA works or not.
> >
> > With the logic you propose it will be hiding inside Ignite machinery and
> > logs.
> >
> > > Unfortunately, this is a breaking change. Currently this scenario works
> > without errors - uses default socket for custom affinity caches.
> >
> > It’s OK from my point of view to introduce such a change.
> >
> >
> > > 10 июля 2022 г., в 22:05, Pavel Tupitsyn
> > написал(а):
> > >
> > >> providing simple function Object -> int32 in cache configuration is
> > wrong
> > > decision
> > >
> > > Because it is error-prone and does not work for all cases:
> > > - You have to set it explicitly on the client for every cache.
> > > - No way to do that if you get an existing cache by name.
> > >
> > >> still being unable to solve the problem
> > >
> > > I believe the proposed solution covers all use cases.
> > >
> > >> cryptic logic
> > >
> > > It is not cryptic, it is very straightforward and builds on existing
> > binary
> > > type mechanism.
> > >
> > >> feature flags
> > >
> > > What's bad about feature flags?
> > >
> > >> how an ability to obtain a classname of java class can help my python or
> > > C++ client map key to partition
> > >
> > > Not a java class name, but type id. You can set up type id mapping
> > properly
> > > and use this in any thin client.
> > > This will work for .NET client, not sure about Python and C++.
> > >
> > > On Sun, Jul 10, 2022 at 9:33 PM Ivan Daschinsky
> > wrote:
> > >
> > >> Guys, I simply cant understand why providing simple function Object ->
> > >> int32 in cache configuration is wrong decision, but implementing cryptic
> > >> logic with class names, feature flags and still being unable to solve
> > the
> > >> probem is ok. I don't understand how an ab
Re: [DISCUSSION] Customers Migraton from Thick to Thin Clients
Still don't understand how type id can help me to obtain valid mapper from key to int32. Especially when we are talking about non-jvm languages. пн, 11 июл. 2022 г., 12:33 Николай Ижиков : > Pavel. > > > It is not cryptic, it is very straightforward and builds on existing > binary type mechanism. > > I see the Ivan’s point here. > As I said earlier - It must be absolutely clear for user - PA works or not. > > With the logic you propose it will be hiding inside Ignite machinery and > logs. > > > Unfortunately, this is a breaking change. Currently this scenario works > without errors - uses default socket for custom affinity caches. > > It’s OK from my point of view to introduce such a change. > > > > 10 июля 2022 г., в 22:05, Pavel Tupitsyn > написал(а): > > > >> providing simple function Object -> int32 in cache configuration is > wrong > > decision > > > > Because it is error-prone and does not work for all cases: > > - You have to set it explicitly on the client for every cache. > > - No way to do that if you get an existing cache by name. > > > >> still being unable to solve the problem > > > > I believe the proposed solution covers all use cases. > > > >> cryptic logic > > > > It is not cryptic, it is very straightforward and builds on existing > binary > > type mechanism. > > > >> feature flags > > > > What's bad about feature flags? > > > >> how an ability to obtain a classname of java class can help my python or > > C++ client map key to partition > > > > Not a java class name, but type id. You can set up type id mapping > properly > > and use this in any thin client. > > This will work for .NET client, not sure about Python and C++. > > > > On Sun, Jul 10, 2022 at 9:33 PM Ivan Daschinsky > wrote: > > > >> Guys, I simply cant understand why providing simple function Object -> > >> int32 in cache configuration is wrong decision, but implementing cryptic > >> logic with class names, feature flags and still being unable to solve > the > >> probem is ok. I don't understand how an ability to obtain a classname of > >> java class can help my python or C++ client map key to partition. > >> > >> Max's proposal seems to be a good variant, just change naming a little > bit, > >> maybe > >> > >> пт, 8 июл. 2022 г., 15:35 Pavel Tupitsyn : > >> > >>> Nikolay, > >>> > Add ability to sent custom affinity class name. > >>> Can you please elaborate? What is sent where? > >>> > If `partitionAwarenessEnabled=true` but custom affinity can’t be > >>> instantiated on the client - throw an exception > >>> Unfortunately, this is a breaking change. Currently this scenario works > >>> without errors - uses default socket for custom affinity caches. > >>> > >>> > >>> On Fri, Jul 8, 2022 at 3:23 PM Николай Ижиков > >> wrote: > >>> > Hello, Pavel. > > I support your proposal to transparently create custom AffinityMapper > based on server node classname, in general. > Partition Awareness crucial point for a thin client performance so It > should be absolutely clear for a user - PA works correctly or not. > > So, I think, we should implement the following: > > 0. Add ability to sent custom affinity class name. > 1. If `partitionAwarenessEnabled=true` but custom affinity can’t be > instantiated on the client - throw an exception. > > WDYT? > > > 8 июля 2022 г., в 08:37, Pavel Tupitsyn > написал(а): > > > > To clarify: you can use a different AffinityFunction implementation > >> on > the > > client side. It just has to map to the same binary typeId. > > Even if there is a legacy setup with AffinityKeyMapper on servers, > >> you > can > > craft an AffinityFunction for the client that achieves correct > >>> behavior. > > So I believe this should cover any use case. > > > > On Thu, Jul 7, 2022 at 10:36 PM Pavel Tupitsyn >>> > wrote: > > > >> AffinityKeyMapper is just a subset of AffinityFunction, isn't it? > >> So by supporting AffinityFunction we cover all AffinityKeyMapper use > cases. > >> > >> > >>> managing the classpath, a user should always keep in mind it > >> I don't consider this a problem. This is how Java works, what can we > >>> do? > >> You can also stick the class into BinaryConfiguration on the client > >> to > >> make it explicit and make sure it is loaded. > >> > >>> it still possible having user bad implementations of > >> AffinityFunction > >> that can't be easily instantiated; > >> It is also possible to provide a bad implementation of > >> ToIntFunction, > >>> no > >> difference. > >> > >>> a question - what we should do in case of class instantiating > >> loading > >> fails? > >> Currently we don't fail on custom affinity, right? So we should keep > this > >> behavior. Print a warning. > >> > >> On Thu, Jul 7, 2022 at 10:27 PM Maxim Muzafarov > wrote: > >> > >>
Re: [DISCUSSION] Customers Migraton from Thick to Thin Clients
Pavel. > It is not cryptic, it is very straightforward and builds on existing binary > type mechanism. I see the Ivan’s point here. As I said earlier - It must be absolutely clear for user - PA works or not. With the logic you propose it will be hiding inside Ignite machinery and logs. > Unfortunately, this is a breaking change. Currently this scenario works > without errors - uses default socket for custom affinity caches. It’s OK from my point of view to introduce such a change. > 10 июля 2022 г., в 22:05, Pavel Tupitsyn написал(а): > >> providing simple function Object -> int32 in cache configuration is wrong > decision > > Because it is error-prone and does not work for all cases: > - You have to set it explicitly on the client for every cache. > - No way to do that if you get an existing cache by name. > >> still being unable to solve the problem > > I believe the proposed solution covers all use cases. > >> cryptic logic > > It is not cryptic, it is very straightforward and builds on existing binary > type mechanism. > >> feature flags > > What's bad about feature flags? > >> how an ability to obtain a classname of java class can help my python or > C++ client map key to partition > > Not a java class name, but type id. You can set up type id mapping properly > and use this in any thin client. > This will work for .NET client, not sure about Python and C++. > > On Sun, Jul 10, 2022 at 9:33 PM Ivan Daschinsky wrote: > >> Guys, I simply cant understand why providing simple function Object -> >> int32 in cache configuration is wrong decision, but implementing cryptic >> logic with class names, feature flags and still being unable to solve the >> probem is ok. I don't understand how an ability to obtain a classname of >> java class can help my python or C++ client map key to partition. >> >> Max's proposal seems to be a good variant, just change naming a little bit, >> maybe >> >> пт, 8 июл. 2022 г., 15:35 Pavel Tupitsyn : >> >>> Nikolay, >>> Add ability to sent custom affinity class name. >>> Can you please elaborate? What is sent where? >>> If `partitionAwarenessEnabled=true` but custom affinity can’t be >>> instantiated on the client - throw an exception >>> Unfortunately, this is a breaking change. Currently this scenario works >>> without errors - uses default socket for custom affinity caches. >>> >>> >>> On Fri, Jul 8, 2022 at 3:23 PM Николай Ижиков >> wrote: >>> Hello, Pavel. I support your proposal to transparently create custom AffinityMapper based on server node classname, in general. Partition Awareness crucial point for a thin client performance so It should be absolutely clear for a user - PA works correctly or not. So, I think, we should implement the following: 0. Add ability to sent custom affinity class name. 1. If `partitionAwarenessEnabled=true` but custom affinity can’t be instantiated on the client - throw an exception. WDYT? > 8 июля 2022 г., в 08:37, Pavel Tupitsyn написал(а): > > To clarify: you can use a different AffinityFunction implementation >> on the > client side. It just has to map to the same binary typeId. > Even if there is a legacy setup with AffinityKeyMapper on servers, >> you can > craft an AffinityFunction for the client that achieves correct >>> behavior. > So I believe this should cover any use case. > > On Thu, Jul 7, 2022 at 10:36 PM Pavel Tupitsyn >> wrote: > >> AffinityKeyMapper is just a subset of AffinityFunction, isn't it? >> So by supporting AffinityFunction we cover all AffinityKeyMapper use cases. >> >> >>> managing the classpath, a user should always keep in mind it >> I don't consider this a problem. This is how Java works, what can we >>> do? >> You can also stick the class into BinaryConfiguration on the client >> to >> make it explicit and make sure it is loaded. >> >>> it still possible having user bad implementations of >> AffinityFunction >> that can't be easily instantiated; >> It is also possible to provide a bad implementation of >> ToIntFunction, >>> no >> difference. >> >>> a question - what we should do in case of class instantiating >> loading >> fails? >> Currently we don't fail on custom affinity, right? So we should keep this >> behavior. Print a warning. >> >> On Thu, Jul 7, 2022 at 10:27 PM Maxim Muzafarov wrote: >> >>> Pavel, >>> >>> >>> Yes, you're right that AffinityKeyMapper is deprecated, this is sad >>> to >>> say but it still used. >>> >>> Let me describe the cases in more detail. In general, we have >>> customers which are using: >>> >>> 1. an own custom AffinityFunction; >>> 2. the default RendezvousAffinityFunction + an own deprecated >>> AffinityKeyMapper; >>> 3. an own custom Af
