Re: [DISCUSSION] Merge APIs of IgniteAuthenticationProcessor and IgniteSecurity
Hello, Mikhail! Proposed realization provides a security plugin when isAuthenticationEnabled is true and, in this way, makes IgniteSecurity enabled. But current implementation of IgniteAuthenticationProcessor (security plugin) does not allow: - apply a security policy, so authorization does not make sense - authenticate nodes - get the actual security context by a security subject id. Another hand, security-enabled mode adds to every communication message a security subject id, makes a security context switch, authorizes an operation, so these all look like a waste of resources. IMHO a better solution would be to implement a full-functional default security plugin. пн, 22 мар. 2021 г. в 15:52, Mikhail Petrov : > Maxim, this issue should be fixed as part of [1]. Note, that the current > PR [2] is draft and its description just shows the current state of > work. Of course the task i'm working on can't be resolved without fixing > this issue. > > [1] - https://issues.apache.org/jira/browse/IGNITE-13112 > [2] - https://github.com/apache/ignite/pull/8892 > > On 22.03.2021 15:38, Maxim Muzafarov wrote: > > Mikhail, > > > >> Note, that the current PR breaks management of the users via REST > because the SecurityContext is not propagated properly during REST requests > execution. > > Do we have a JIRA taks to fix this behaviour or do we need such > > behaviour at all? > > > > On Mon, 22 Mar 2021 at 13:34, Nikolay Izhikov > wrote: > >> Hello, Mikhail. > >> > >> I'm +1 to follow your suggestion. > >> > >> чт, 18 мар. 2021 г. в 17:53, Mikhail Petrov : > >> > >>> Hello, Igniters. > >>> > >>> As of now, there are two independent APIs related to security: > >>> 1. IgniteSecurity - handle node/client authentication and authorize all > >>> operations. > >>> 2. IgniteAuthenticationProcessor - handle authentication of thin > clients > >>> only. > >>> > >>> The main purpose of creating the IgniteAuthenticationProcessor was to > >>> bring default security implementation in Ignite (see [1]) because > >>> IgniteSecurity has always had a single implementation that delegates > >>> authorization and authentication operation to an external security > plugin. > >>> > >>> But two different APIs that are related to the same leads to security > >>> checks duplication in code. As of now, it's possible to configure both > >>> security approaches at the same time, and that is confusing for the > >>> user. E.g., the user can provide a security plugin and execute ALTER / > >>> DROP / CREATE commands successfully. In this case, mentioned commands > >>> will do nothing because they only work with the authentication > processor > >>> > >>> I propose to merge the two mentioned above security APIs into one based > >>> on the IgniteSecurity interface. > >>> > >>> For this it is proposed: > >>> 1. Remove an IgniteAuthenticationProcessor that is now treated as an > >>> independent processor. > >>> 2. Move the logic of IgniteAuthenticationProcessor into the > >>> implementation of the security plugin that will be used if > >>> authentication is enabled via configuration. > >>> 3. Remove duplication of security checks and leave IgniteSecurity as a > >>> single security API. As of now, authentication operations are not > >>> delegated to IgniteAuthenticationProcessor if a security plugin is > >>> specified. So the overall security behavior from the user side will > >>> remain intact. > >>> 4. Remove the AuthorizationContext completely as IgniteSecurity > provides > >>> an API for storing and managing the security contexts. > >>> 5. Extend GridSecurityPlugin interface with methods that provide the > >>> ability to manage security users to support existing commands available > >>> for authentication processor - alter/drop/create through SQL and REST. > >>> > >>> As a result, we will make the security-related code more consistent and > >>> simpler. > >>> > >>> Proposed signatures of GridSecurityPlugin methods that provide the > >>> ability to manage security users: > >>> > >>> public void createUser(String login, UserOptions opts) throws > >>> IgniteCheckedException  > >>>  > >>> public void alterUser(String login, UserOptions opts) throws > >>> IgniteCheckedException > >>>  > >>> public void dropUser(String login) throws IgniteCheckedException > >>> > >>> The UserOptions class is a wrapper over EnumMap that maps option values > >>> to their aliases. This allows the class to be used for both create > >>> and alter user operations and doesn't break backward compatibility in > >>> case new options are declared. > >>> > >>>  > >>> The proposed changes lead to the following compatibility issues: > >>> > >>> 1. When a user provides a security plugin and enables authentication - > >>> in this case, the user will face exceptions during the node start while > >>> now node starts smoothly. This case makes a little sense because now > >>> authentication operations are not delegated to > >>>
Re: Azure Cloud IP Finder
Hello,
Thank you for taking a look.
I am not sure what the confusion is. On the current iteration of PR, I
am able to build and release and looking in the location you
mentioned, I see:
Atris-MacBook-Pro-15:libs atrisharma$ cd optional/
Atris-MacBook-Pro-15:optional atrisharma$ cd ignite-azure/
Atris-MacBook-Pro-15:ignite-azure atrisharma$ ls
README.txt azure-storage-blob-12.0.0.jar ignite-azure-2.11.0-SNAPSHOT.jar
Attached is the screenshot.
I have fixed the comments on the PR, please see and let me know.
Atri
On Thu, Apr 1, 2021 at 8:21 PM Ilya Kasnacheev
wrote:
>
> Hello!
>
> Were you successful with using it from the deliverable of mvn initialize
> -Prelease?
>
> Please refer for example to aws module dependencies section:
>
>
> com.amazonaws
> aws-java-sdk-core
> ${aws.sdk.version}
>
>
>
> com.amazonaws
> aws-java-sdk-s3
> ${aws.sdk.version}
>
>
>
> com.amazonaws
> aws-java-sdk-ec2
> ${aws.sdk.version}
>
>
>
> com.amazonaws
> aws-java-sdk-elasticloadbalancing
> ${aws.sdk.version}
>
>
>
> com.amazonaws
> aws-java-sdk-elasticloadbalancingv2
> ${aws.sdk.version}
>
>
>
> com.amazonaws
> aws-java-sdk-kms
> ${aws.sdk.version}
>
>
>
>
> com.fasterxml.jackson.core
> jackson-core
> ${jackson.version}
>
>
>
>
> com.fasterxml.jackson.core
> jackson-annotations
> ${jackson.version}
>
>
>
>
> com.fasterxml.jackson.core
> jackson-databind
> ${jackson.version}
>
>
>
> com.amazonaws
> aws-encryption-sdk-java
> ${aws.encryption.sdk.version}
>
>
> org.bouncycastle
> bcprov-ext-jdk15on
>
>
>
>
>
> org.bouncycastle
> bcprov-ext-jdk15on
> ${bouncycastle.version}
>
>
>
> joda-time
> joda-time
> 2.8.1
>
>
>
> org.apache.httpcomponents
> httpclient
> ${httpclient.version}
>
>
>
> org.apache.httpcomponents
> httpcore
> ${httpcore.version}
>
>
> azure module should likewise list all of its actual dependencies. Right now
> the module directory is empty so I'm not even going to test to run it in
> stand-alone mode:
> ~/Downloads/apache-ignite-2.11.0-SNAPSHOT-bin% ls libs/optional/ignite-azure
> azure-storage-blob-12.0.0.jar ignite-azure-2.11.0-SNAPSHOT.jar README.txt
>
> Please address this issue along with other things which I have also
> commented on the PR.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 31 мар. 2021 г. в 08:17, Atri Sharma :
>
> > Thank you for the review.
> >
> > I have updated the PR. Please see.
> >
> > On Mon, Mar 29, 2021 at 8:27 PM Ilya Kasnacheev
> > wrote:
> > >
> > > Hello!
> > >
> > > I have left some more comments after trying this change on a real Azure
> > > cluster.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > вт, 23 мар. 2021 г. в 18:32, Atri Sharma :
> > >
> > > > Thank you!
> > > >
> > > > I have updated the PR. Please see and let me know.
> > > >
> > > > On Tue, Mar 23, 2021 at 4:27 PM Ilya Kasnacheev
> > > > wrote:
> > > > >
> > > > > Hello!
> > > > >
> > > > > I am going to check this change out when I have time, using my Azure
> > > > > account.
> > > > >
> > > > > Regards,
> > > > > --
> > > > > Ilya Kasnacheev
> > > > >
> > > > >
> > > > > вт, 23 мар. 2021 г. в 07:20, Atri Sharma :
> > > > >
> > > > > > Gentle reminder on this -- please help in reviewing this.
> > > > > >
> > > > > > On Fri, Mar 19, 2021 at 10:23 AM Atri Sharma
> > wrote:
> > > > > > >
> > > > > > > Thanks Denis.
> > > > > > >
> > > > > > > I have raised a PR for the same:
> > > > > > >
> > > > > > > https://github.com/apache/ignite/pull/8897
> > > > > > >
> > > > > > > Regards,
> > > > > > >
> > > > > > > Atri
> > > > > > >
> > > > > > > On Wed, Mar 10, 2021 at 1:21 AM Denis Magda
> > > > wrote:
> > > > > > > >
> > > > > > > > Atri,
> > > > > > > >
> > > > > > > > Let's discuss the subj together with the community. Ignite
> > already
> > > > > > supports
> > > > > > > > AWS [1] and GCE [2] IP Finders out of the box, but the Azure
> > one is
> > > > > > still
> > > > > > > > missing. I can confirm that the demand exists, and rather
> > > > frequently,
> > > > > > I see
> > > > > > > > developers asking for an Azure-native IP finder for Ignite.
> > > > > > > >
> > > > > > > > Atri, could you please research how to implement the IP finder
> > and
> > > > > > suggest
> > > > > > > > a solution in this discussion thread? See how it was done for
> > AWS
> > > > and
> > > > > > GCE,
> > > > > > > > we might go the same route or use a more contemporary and
> > > > > > easy-to-configure
> > > > > > > > approach for Azure.
> > > > > > > >
> > > > > > > > [1]
> > > > > > > >
> > > > > >
> > > >
> > https://ignite.apache.org/docs/latest/clustering/discovery-in-the-cloud#amazon-s3-ip-finder
> > > > > > > > [2]
> > > > > > > >
> > > > > >
> > > >
> >
[MTCGA]: new failures in builds [5947187] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. If your changes can lead to this failure(s): We're grateful that you were a volunteer to make the contribution to this project, but things change and you may no longer be able to finalize your contribution. Could you respond to this email and indicate if you wish to continue and fix test failures or step down and some committer may revert you commit. *New Critical Failure in master Control Utility https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_ControlUtility?branch=%3Cdefault%3E Changes may lead to failure were done by - ibessonov https://ci.ignite.apache.org/viewModification.html?modId=921247 - maksim timonin https://ci.ignite.apache.org/viewModification.html?modId=921324 - ilya kasnacheev https://ci.ignite.apache.org/viewModification.html?modId=921306 - slava koptilin https://ci.ignite.apache.org/viewModification.html?modId=921290 - semyon danilov https://ci.ignite.apache.org/viewModification.html?modId=921267 - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 05:25:22 02-04-2021
[jira] [Created] (IGNITE-14465) Add the ability to activate the cluster via the Python thin client
Igor Sapego created IGNITE-14465: Summary: Add the ability to activate the cluster via the Python thin client Key: IGNITE-14465 URL: https://issues.apache.org/jira/browse/IGNITE-14465 Project: Ignite Issue Type: Bug Components: python, thin client Affects Versions: python-0.3.4 Reporter: Igor Sapego Assignee: Igor Sapego Fix For: python-0.4.0 This feature will be extremely useful when working with clusters that have the "persistenceEnabled" option. Since they require activation to get started. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Spark 3 support plans
Hi Nikolay, As a main spark integration maintainer, do you have any plans for upgrading spark dependencies to version 3+? I see that some users are asking for it: https://issues.apache.org/jira/browse/IGNITE-13181. Thanks, Evgenii
[jira] [Created] (IGNITE-14464) Remove released version of Ignite from regular ducktests
Nikolay Izhikov created IGNITE-14464: Summary: Remove released version of Ignite from regular ducktests Key: IGNITE-14464 URL: https://issues.apache.org/jira/browse/IGNITE-14464 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov No need to tests the released ignite version regularly. We can remove the 2.10.0 version from the source test matrix. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Azure Cloud IP Finder
Hello!
Were you successful with using it from the deliverable of mvn initialize
-Prelease?
Please refer for example to aws module dependencies section:
com.amazonaws
aws-java-sdk-core
${aws.sdk.version}
com.amazonaws
aws-java-sdk-s3
${aws.sdk.version}
com.amazonaws
aws-java-sdk-ec2
${aws.sdk.version}
com.amazonaws
aws-java-sdk-elasticloadbalancing
${aws.sdk.version}
com.amazonaws
aws-java-sdk-elasticloadbalancingv2
${aws.sdk.version}
com.amazonaws
aws-java-sdk-kms
${aws.sdk.version}
com.fasterxml.jackson.core
jackson-core
${jackson.version}
com.fasterxml.jackson.core
jackson-annotations
${jackson.version}
com.fasterxml.jackson.core
jackson-databind
${jackson.version}
com.amazonaws
aws-encryption-sdk-java
${aws.encryption.sdk.version}
org.bouncycastle
bcprov-ext-jdk15on
org.bouncycastle
bcprov-ext-jdk15on
${bouncycastle.version}
joda-time
joda-time
2.8.1
org.apache.httpcomponents
httpclient
${httpclient.version}
org.apache.httpcomponents
httpcore
${httpcore.version}
azure module should likewise list all of its actual dependencies. Right now
the module directory is empty so I'm not even going to test to run it in
stand-alone mode:
~/Downloads/apache-ignite-2.11.0-SNAPSHOT-bin% ls libs/optional/ignite-azure
azure-storage-blob-12.0.0.jar ignite-azure-2.11.0-SNAPSHOT.jar README.txt
Please address this issue along with other things which I have also
commented on the PR.
Regards,
--
Ilya Kasnacheev
ср, 31 мар. 2021 г. в 08:17, Atri Sharma :
> Thank you for the review.
>
> I have updated the PR. Please see.
>
> On Mon, Mar 29, 2021 at 8:27 PM Ilya Kasnacheev
> wrote:
> >
> > Hello!
> >
> > I have left some more comments after trying this change on a real Azure
> > cluster.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > вт, 23 мар. 2021 г. в 18:32, Atri Sharma :
> >
> > > Thank you!
> > >
> > > I have updated the PR. Please see and let me know.
> > >
> > > On Tue, Mar 23, 2021 at 4:27 PM Ilya Kasnacheev
> > > wrote:
> > > >
> > > > Hello!
> > > >
> > > > I am going to check this change out when I have time, using my Azure
> > > > account.
> > > >
> > > > Regards,
> > > > --
> > > > Ilya Kasnacheev
> > > >
> > > >
> > > > вт, 23 мар. 2021 г. в 07:20, Atri Sharma :
> > > >
> > > > > Gentle reminder on this -- please help in reviewing this.
> > > > >
> > > > > On Fri, Mar 19, 2021 at 10:23 AM Atri Sharma
> wrote:
> > > > > >
> > > > > > Thanks Denis.
> > > > > >
> > > > > > I have raised a PR for the same:
> > > > > >
> > > > > > https://github.com/apache/ignite/pull/8897
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > Atri
> > > > > >
> > > > > > On Wed, Mar 10, 2021 at 1:21 AM Denis Magda
> > > wrote:
> > > > > > >
> > > > > > > Atri,
> > > > > > >
> > > > > > > Let's discuss the subj together with the community. Ignite
> already
> > > > > supports
> > > > > > > AWS [1] and GCE [2] IP Finders out of the box, but the Azure
> one is
> > > > > still
> > > > > > > missing. I can confirm that the demand exists, and rather
> > > frequently,
> > > > > I see
> > > > > > > developers asking for an Azure-native IP finder for Ignite.
> > > > > > >
> > > > > > > Atri, could you please research how to implement the IP finder
> and
> > > > > suggest
> > > > > > > a solution in this discussion thread? See how it was done for
> AWS
> > > and
> > > > > GCE,
> > > > > > > we might go the same route or use a more contemporary and
> > > > > easy-to-configure
> > > > > > > approach for Azure.
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > >
> > >
> https://ignite.apache.org/docs/latest/clustering/discovery-in-the-cloud#amazon-s3-ip-finder
> > > > > > > [2]
> > > > > > >
> > > > >
> > >
> https://ignite.apache.org/docs/latest/clustering/discovery-in-the-cloud#google-compute-discovery
> > > > > > >
> > > > > > > -
> > > > > > > Denis
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Regards,
> > > > > >
> > > > > > Atri
> > > > > > Apache Concerted
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Regards,
> > > > >
> > > > > Atri
> > > > > Apache Concerted
> > > > >
> > >
> > > --
> > > Regards,
> > >
> > > Atri
> > > Apache Concerted
> > >
>
> --
> Regards,
>
> Atri
> Apache Concerted
>
Re: Joining in
Hello! I have added you to contributors role. Please make sure to read https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute Regards, -- Ilya Kasnacheev чт, 1 апр. 2021 г. в 16:27, Игорь Гусев : > > Hi all! > I am Igor Gusev, and I am an experienced technical writer. I am learning > in-memory processing technologies and would be glad to contribute to the > community. Please give me access to Apache Ignite Jira. > My JIRA ID is igusev. > > Thanks in advance! > Igor
Joining in
Hi all! I am Igor Gusev, and I am an experienced technical writer. I am learning in-memory processing technologies and would be glad to contribute to the community. Please give me access to Apache Ignite Jira. My JIRA ID is igusev. Thanks in advance! Igor
[jira] [Created] (IGNITE-14463) Support check Ignite-based products with ducktests
Nikolay Izhikov created IGNITE-14463:
Summary: Support check Ignite-based products with ducktests
Key: IGNITE-14463
URL: https://issues.apache.org/jira/browse/IGNITE-14463
Project: Ignite
Issue Type: Improvement
Reporter: Nikolay Izhikov
We need to provide the way to run ducktape tests on the products that based on
Ignite codebase.
For that 2 edits required:
1. Ability to specify "project" variable in globals.
2. Ability to specify custom docker options(mount point) from the outside of
{{run_tests.sh}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-14462) Enable EmptyCatchBlock checkstyle check
Maxim Muzafarov created IGNITE-14462:
Summary: Enable EmptyCatchBlock checkstyle check
Key: IGNITE-14462
URL: https://issues.apache.org/jira/browse/IGNITE-14462
Project: Ignite
Issue Type: Task
Reporter: Maxim Muzafarov
Assignee: Maxim Muzafarov
Fix For: 2.11
Checks for empty catch blocks. By default, check allows an empty catch block
with any comment inside.
{code}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-14461) Track down those who initiated a query
Taras Ledkov created IGNITE-14461:
-
Summary: Track down those who initiated a query
Key: IGNITE-14461
URL: https://issues.apache.org/jira/browse/IGNITE-14461
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Taras Ledkov
Assignee: Taras Ledkov
Fix For: 2.11
We need to know and track who initiated a query. It could be useful for an IT
guy to be able to find an application and ask the developers to change the
code. It could be as part of {{LOCAL_SQL_RUNNING_QUERIES}} system view or may
be better to have separate view which could be joined with running queries view.
For first glance it should be :
# ip, port, host name, user for thin clients.
# for thick client - nodeId, clientId, ip, host name,
# task name for compute
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [DISCUSSION] IgniteFuture class future in Ignite-3.0.
Andrey,
> Reactive abstractions look more suitable for Queries rather than
> cache/table async operations and don't offer the power of chaining like
> CompletableStage.
Could you please elaborate what capabilities do you mean? AFAIK there
are quite powerful APIs for singular reactive abstractions as well.
E.g. [1].
[1]
https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Mono.html
2021-04-01 12:33 GMT+03:00, Andrey Mashenkov :
> Val,
> I just complain JDK CompletableFuture itself is not ideal, but already
> implemented, well-known and tested.
> It still a good alternative compared to custom future implementation to me.
>
> Ok, I feel most of us agree with CompletableFuture as a replacement for
> custom IgniteFuture.
> Let's try to use it, but keep in mind that we MUST return a defective copy
> (via copy() or minimalStage()) to user to prevent misusage on the user
> side.
> It would be nice if we'll follow the same requirement in our internal code,
> e.g. if a component returns a future that further can be used in other
> components, especially in custom plugins.
>
> Ivan, thanks for the example.
> Reactive abstractions look more suitable for Queries rather than
> cache/table async operations and don't offer the power of chaining like
> CompletableStage.
> AFAIK, guys who involved in SQL engine development based on Calcite already
> use reactive patterns.
>
>
> On Thu, Apr 1, 2021 at 3:15 AM Ivan Pavlukhin wrote:
>
>> Folks,
>>
>> Regarding Reactive abstractions. While it might look too complex for
>> simple KV cases it can be quite powerful for queries. Also there are
>> known solutions for cancellation, backpressure and flow control. It
>> can greatly simplify things for users familiar with Reactive
>> programming rather than learning Ignite-specific Query/QueryCursor
>> API.
>>
>> Also it looks like there are well-defined semantics [1]. E.g.
>> cancellation seems to be defined much better than for
>> CompletableFuture.
>>
>> [1]
>> https://github.com/reactive-streams/reactive-streams-jvm#specification
>>
>> 2021-03-31 21:30 GMT+03:00, Valentin Kulichenko <
>> valentin.kuliche...@gmail.com>:
>> > Hi Andrey,
>> >
>> > Please see below.
>> >
>> > -Val
>> >
>> > On Wed, Mar 31, 2021 at 7:55 AM Andrey Mashenkov
>> >
>> > wrote:
>> >
>> >> CompletableFuture cancellation will not work as many users expected.
>> >> Javadoc says:
>> >> /* Since (unlike {@link FutureTask}) this class has no direct
>> >> * control over the computation that causes it to be completed,
>> >> * cancellation is treated as just another form of exceptional
>> >> * completion.
>> >> */
>> >>
>> >
>> > Let's not make assumptions about the expectations of the users. That's
>> > exactly why I initially leaned towards a custom interface as well, but
>> > that's a mistake.
>> > Indeed, this contract might look weird to us, because the current
>> > version
>> > of Ignite behaves differently. However, there are much more developers
>> > using CompletableFuture and other standard async frameworks, than
>> > developers using the async functionality of Ignite. Therefore, our
>> > intuitions can easily be wrong.
>> >
>> >
>> >> Completion of a child future doesn't trigger the completion of a
>> >> parent.
>> >> So, we will need to extend the future anyway.
>> >>
>> >
>> > First of all, as Pavel mentioned, you can attach your own listener
>> > before
>> > returning a CompletableFuture to the user. You don't need to extend.
>> > Second of all, there is still a discussion to be had on whether the
>> parent
>> > needs to be completed. I don't actually think it's obvious, and most
>> likely
>> > it's case by case. With CompletableFuture you have enough flexibility
>> > to
>> > control the behavior.
>> >
>> >
>> >>
>> >> Also, cancel() method contract differs from IgniteFuture in 2.0,
>> >> Completable method return true if the future was cancelled,
>> >> but IgniteFuture return true only if it wasn't cancel prior the call.
>> >> Thus, if cancel() was called twice we will have different results for
>> >> CompletableFuture and IgniteFuture,
>> >> that makes CompletableFuture barely usable for our internal purposes.
>> >>
>> >
>> > It doesn't really matter how IgniteFuture in 2.0 behaves. It was
>> > created
>> > long before continuations and other async concepts became mainstream
>> > (in
>> > Java world at least).
>> > Also, we don't have to use it for internal purposes, of course. I'm
>> > only
>> > talking about public APIs.
>> >
>> >
>> >>
>> >> BTW, CompletableFuture.anyOf() still can be used, see
>> >> CompletionStage.toCompletableFuture() contract.
>> >>
>> >
>> > In my view, this actually kills the idea of a custom future. Basically,
>> > the proposal is to introduce a custom entity to restrict access to some
>> of
>> > the CompletableFuture methods, but then allow to convert this custom
>> entity
>> > to a CompletableFuture that has all the methods. This makes zero sense
>> > to
>> > me.
>> >
>> >
>> >>
>> >> The more I
[jira] [Created] (IGNITE-14460) Implement RAFT server stub
Alexey Scherbakov created IGNITE-14460: -- Summary: Implement RAFT server stub Key: IGNITE-14460 URL: https://issues.apache.org/jira/browse/IGNITE-14460 Project: Ignite Issue Type: Task Reporter: Alexey Scherbakov Assignee: Alexey Scherbakov We need a raft group server implementation stub to unlock further development of dependent services. In the simplest approach we will have a single node group, so no consensus is required. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: IGNITE-13399 Fix access right issues in computation of system metrics
Hi Denis! Thank you for the PR! I've left a comment. Best regards, Mirza Aliev чт, 1 апр. 2021 г. в 11:48, Denis Mekhanikov : > Hi everyone! > > I've prepared a PR for the following issue: > https://issues.apache.org/jira/browse/IGNITE-13399 > Currently on Java 8 CpuLoad metric is reported to be equal to -1 when > running in embedded mode. > > Could anyone take a look? > Thanks! > > Denis >
Re: [DISCUSSION] IgniteFuture class future in Ignite-3.0.
Val,
I just complain JDK CompletableFuture itself is not ideal, but already
implemented, well-known and tested.
It still a good alternative compared to custom future implementation to me.
Ok, I feel most of us agree with CompletableFuture as a replacement for
custom IgniteFuture.
Let's try to use it, but keep in mind that we MUST return a defective copy
(via copy() or minimalStage()) to user to prevent misusage on the user side.
It would be nice if we'll follow the same requirement in our internal code,
e.g. if a component returns a future that further can be used in other
components, especially in custom plugins.
Ivan, thanks for the example.
Reactive abstractions look more suitable for Queries rather than
cache/table async operations and don't offer the power of chaining like
CompletableStage.
AFAIK, guys who involved in SQL engine development based on Calcite already
use reactive patterns.
On Thu, Apr 1, 2021 at 3:15 AM Ivan Pavlukhin wrote:
> Folks,
>
> Regarding Reactive abstractions. While it might look too complex for
> simple KV cases it can be quite powerful for queries. Also there are
> known solutions for cancellation, backpressure and flow control. It
> can greatly simplify things for users familiar with Reactive
> programming rather than learning Ignite-specific Query/QueryCursor
> API.
>
> Also it looks like there are well-defined semantics [1]. E.g.
> cancellation seems to be defined much better than for
> CompletableFuture.
>
> [1] https://github.com/reactive-streams/reactive-streams-jvm#specification
>
> 2021-03-31 21:30 GMT+03:00, Valentin Kulichenko <
> valentin.kuliche...@gmail.com>:
> > Hi Andrey,
> >
> > Please see below.
> >
> > -Val
> >
> > On Wed, Mar 31, 2021 at 7:55 AM Andrey Mashenkov
> >
> > wrote:
> >
> >> CompletableFuture cancellation will not work as many users expected.
> >> Javadoc says:
> >> /* Since (unlike {@link FutureTask}) this class has no direct
> >> * control over the computation that causes it to be completed,
> >> * cancellation is treated as just another form of exceptional
> >> * completion.
> >> */
> >>
> >
> > Let's not make assumptions about the expectations of the users. That's
> > exactly why I initially leaned towards a custom interface as well, but
> > that's a mistake.
> > Indeed, this contract might look weird to us, because the current version
> > of Ignite behaves differently. However, there are much more developers
> > using CompletableFuture and other standard async frameworks, than
> > developers using the async functionality of Ignite. Therefore, our
> > intuitions can easily be wrong.
> >
> >
> >> Completion of a child future doesn't trigger the completion of a parent.
> >> So, we will need to extend the future anyway.
> >>
> >
> > First of all, as Pavel mentioned, you can attach your own listener before
> > returning a CompletableFuture to the user. You don't need to extend.
> > Second of all, there is still a discussion to be had on whether the
> parent
> > needs to be completed. I don't actually think it's obvious, and most
> likely
> > it's case by case. With CompletableFuture you have enough flexibility to
> > control the behavior.
> >
> >
> >>
> >> Also, cancel() method contract differs from IgniteFuture in 2.0,
> >> Completable method return true if the future was cancelled,
> >> but IgniteFuture return true only if it wasn't cancel prior the call.
> >> Thus, if cancel() was called twice we will have different results for
> >> CompletableFuture and IgniteFuture,
> >> that makes CompletableFuture barely usable for our internal purposes.
> >>
> >
> > It doesn't really matter how IgniteFuture in 2.0 behaves. It was created
> > long before continuations and other async concepts became mainstream (in
> > Java world at least).
> > Also, we don't have to use it for internal purposes, of course. I'm only
> > talking about public APIs.
> >
> >
> >>
> >> BTW, CompletableFuture.anyOf() still can be used, see
> >> CompletionStage.toCompletableFuture() contract.
> >>
> >
> > In my view, this actually kills the idea of a custom future. Basically,
> > the proposal is to introduce a custom entity to restrict access to some
> of
> > the CompletableFuture methods, but then allow to convert this custom
> entity
> > to a CompletableFuture that has all the methods. This makes zero sense to
> > me.
> >
> >
> >>
> >> The more I learn about CompletableFuture the stronger my opinion about
> >> overriding it.
> >>
> >>
> >> On Wed, Mar 31, 2021 at 9:31 AM Denis Garus
> wrote:
> >>
> >> > > Stripping them from such functionality, which they are used to, is
> >> > > most
> >> > likely a bad idea.
> >> >
> >> > Completely agree with this point of view.
> >> > Moreover, a user can pass CompletableFuture to another library to do
> >> > any
> >> > manipulations.
> >> > So if we want to introduce our class instead of the java class, we
> >> > should
> >> > have solid arguments;
> >> > otherwise, it can be a reason for irritation.
> >> >
> >> > ср, 31 мар. 2021
[jira] [Created] (IGNITE-14459) Affinity call may fail if called upon merged exchanges
Alexey Goncharuk created IGNITE-14459:
-
Summary: Affinity call may fail if called upon merged exchanges
Key: IGNITE-14459
URL: https://issues.apache.org/jira/browse/IGNITE-14459
Project: Ignite
Issue Type: Improvement
Components: compute
Reporter: Alexey Goncharuk
When exchanges are merged, intermediate affinity assignments are not filled. At
the same time, when a client chooses topology to run affinity call on, it may
take a non-completed exchange version. As a result, when the affinity fetch
task arrives on a node, it will look up a non-existing assignment, resulting in
"Getting affinity for topology version earlier than affinity is calculated"
exception.
{{CacheAffinityCallSelfTest.testAffinityCallNoServerNode}} is flaky because of
this bug.
The following test case for {{CacheAffinityCallSelfTest}} demonstrates the
issue:
{code}
/**
* @throws Exception if failed.
*/
@Test
public void testAffinityCallMergedExchanges() throws Exception {
startGrids(SRVS);
final Integer key = 1;
final IgniteEx client = startClientGrid(SRVS);
assertTrue(client.configuration().isClientMode());
assertNull(client.context().cache().cache(CACHE_NAME));
try {
grid(0).context().cache().context().exchange().mergeExchangesTestWaitVersion(
new AffinityTopologyVersion(SRVS + 3, 0),
null
);
IgniteInternalFuture fut1 = GridTestUtils.runAsync(() ->
startGrid(SRVS + 1));
assertTrue(GridTestUtils.waitForCondition(() ->
client.context().cache().context()
.exchange().lastTopologyFuture()
.initialVersion().equals(new AffinityTopologyVersion(SRVS + 2,
0)), 5_000));
assertFalse(fut1.isDone());
// The future should not complete until second node is started.
IgniteInternalFuture fut2 = GridTestUtils.runAsync(() ->
client.compute().affinityCall(CACHE_NAME, key, new
CheckCallable(key, null)));
startGrid(SRVS + 2);
fut1.get();
fut2.get();
}
finally {
stopAllGrids();
}
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
IGNITE-13399 Fix access right issues in computation of system metrics
Hi everyone! I've prepared a PR for the following issue: https://issues.apache.org/jira/browse/IGNITE-13399 Currently on Java 8 CpuLoad metric is reported to be equal to -1 when running in embedded mode. Could anyone take a look? Thanks! Denis
[jira] [Created] (IGNITE-14458) Fix flaky IgniteLocalWalSizeTest
Kirill Tkalenko created IGNITE-14458: Summary: Fix flaky IgniteLocalWalSizeTest Key: IGNITE-14458 URL: https://issues.apache.org/jira/browse/IGNITE-14458 Project: Ignite Issue Type: Improvement Components: persistence Reporter: Kirill Tkalenko Assignee: Kirill Tkalenko Fix For: 2.11 Fix the flaky *IgniteLocalWalSizeTest* tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
