Re: IP Filtering in IPFinders

2021-04-21 Thread Atri Sharma
Another thing is IP addresses blocked by firewalls -- such IPs will
cause the cluster bootstrap to slow down.

On Thu, Apr 22, 2021 at 10:20 AM Atri Sharma  wrote:
>
> Hi Val,
>
> Consider a scenario where multiple Ignite clusters are running and for
> operational ease (and also compliance, in some cases, e.g. to make
> auditing easier), people can configure cloud based IP finders to share
> the same container (blob container in Azure, S3 container in AWS etc).
>
> In such a case, IPs for all clusters will be in the same container.
> IPFinders of both the clusters will read the entire list. In this
> case, address filtering will help ignore the irrelevant IP addresses.
>
> Thank you for pointing me to the alternate direction. Let me research
> that and revert.
>
> Atri
>
> On Wed, Apr 21, 2021 at 10:46 PM Valentin Kulichenko
>  wrote:
> >
> > Hi Atri,
> >
> > Can you describe the scenario in a little more detail? What exactly do you
> > mean by a container shared by multiple clusters? What are the consequences
> > of this? How does the proposed solution solve the problem?
> >
> > Also, I would suggest revisiting the design - I'm not sure such filtering
> > should be done on the IP finder level. Why not do this on the SPI level
> > instead? I would simply add something like "addressFilter" to the
> > TcpDiscoverySpi. The filter can be a generic IgnitePredicate, so you will
> > be able to provide any implementations, including regex or anything else.
> >
> > -Val
> >
> > On Wed, Apr 21, 2021 at 9:04 AM Atri Sharma  wrote:
> >
> > > Hi,
> > >
> > > When a container is shared by multiple clusters, then this can be useful
> > > for filtering IPs.
> > >
> > > Also, things like VPC based barriers can be circumvented using this
> > > technique.
> > >
> > > On Wed, 21 Apr 2021, 15:49 Ilya Kasnacheev, 
> > > wrote:
> > >
> > > > Hello!
> > > >
> > > > What are the expected use cases for this feature? Can you please
> > > elaborate?
> > > >
> > > > Thanks,
> > > > --
> > > > Ilya Kasnacheev
> > > >
> > > >
> > > > ср, 21 апр. 2021 г. в 08:23, Atri Sharma :
> > > >
> > > > > Hi All,
> > > > >
> > > > > I have opened the following JIRA for the said topic:
> > > > >
> > > > > https://issues.apache.org/jira/browse/IGNITE-14606
> > > > >
> > > > > The concept is to filter IPs based on a pattern or a blocklist in
> > > > > IPFinders while consuming IPs. This is more pertinent for cloud based
> > > > > IPFinders since they can have shared containers.
> > > > >
> > > > > For the moment, I have implemented regex based filtering:
> > > > >
> > > > > https://issues.apache.org/jira/browse/IGNITE-14607
> > > > >
> > > > > for Azure Blob Storage IP Finder. Over time, we can extend the same to
> > > > > other IP finders.
> > > > >
> > > > > Please see the PR:
> > > > >
> > > > > https://github.com/apache/ignite/pull/9024
> > > > >
> > > > > Regards,
> > > > >
> > > > > Atri
> > > > >
> > > > > --
> > > > > Regards,
> > > > >
> > > > > Atri
> > > > > Apache Concerted
> > > > >
> > > >
> > >
>
> --
> Regards,
>
> Atri
> Apache Concerted



-- 
Regards,

Atri
Apache Concerted


Re: IP Filtering in IPFinders

2021-04-21 Thread Atri Sharma
Hi Andrey,

Thanks for the message. Yes, that is the case since I wanted to show
the functionality to the group first.

Val has made a point about an alternate approach which seems cleaner
to me, so let me explore that. If we go that route, then we will not
need to change at IPFinder level which will make the change cleaner.

Atri

On Wed, Apr 21, 2021 at 10:50 PM Andrey Mashenkov
 wrote:
>
> Hi Atri,
>
> You've added a new property to a base TcpDiscoveryIpFinder interface.
> Actually, the only Azure IpFinder uses this setting, but the others.
> This behavior may confuse the users.
>
> Would you mind either making regexp filter setting a part of Azure IpFinder
> only or fix other IpFinders as well?
>
>
> On Wed, Apr 21, 2021 at 7:04 PM Atri Sharma  wrote:
>
> > Hi,
> >
> > When a container is shared by multiple clusters, then this can be useful
> > for filtering IPs.
> >
> > Also, things like VPC based barriers can be circumvented using this
> > technique.
> >
> > On Wed, 21 Apr 2021, 15:49 Ilya Kasnacheev, 
> > wrote:
> >
> > > Hello!
> > >
> > > What are the expected use cases for this feature? Can you please
> > elaborate?
> > >
> > > Thanks,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > ср, 21 апр. 2021 г. в 08:23, Atri Sharma :
> > >
> > > > Hi All,
> > > >
> > > > I have opened the following JIRA for the said topic:
> > > >
> > > > https://issues.apache.org/jira/browse/IGNITE-14606
> > > >
> > > > The concept is to filter IPs based on a pattern or a blocklist in
> > > > IPFinders while consuming IPs. This is more pertinent for cloud based
> > > > IPFinders since they can have shared containers.
> > > >
> > > > For the moment, I have implemented regex based filtering:
> > > >
> > > > https://issues.apache.org/jira/browse/IGNITE-14607
> > > >
> > > > for Azure Blob Storage IP Finder. Over time, we can extend the same to
> > > > other IP finders.
> > > >
> > > > Please see the PR:
> > > >
> > > > https://github.com/apache/ignite/pull/9024
> > > >
> > > > Regards,
> > > >
> > > > Atri
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Atri
> > > > Apache Concerted
> > > >
> > >
> >
>
>
> --
> Best regards,
> Andrey V. Mashenkov

-- 
Regards,

Atri
Apache Concerted


Re: IP Filtering in IPFinders

2021-04-21 Thread Atri Sharma
Hi Val,

Consider a scenario where multiple Ignite clusters are running and for
operational ease (and also compliance, in some cases, e.g. to make
auditing easier), people can configure cloud based IP finders to share
the same container (blob container in Azure, S3 container in AWS etc).

In such a case, IPs for all clusters will be in the same container.
IPFinders of both the clusters will read the entire list. In this
case, address filtering will help ignore the irrelevant IP addresses.

Thank you for pointing me to the alternate direction. Let me research
that and revert.

Atri

On Wed, Apr 21, 2021 at 10:46 PM Valentin Kulichenko
 wrote:
>
> Hi Atri,
>
> Can you describe the scenario in a little more detail? What exactly do you
> mean by a container shared by multiple clusters? What are the consequences
> of this? How does the proposed solution solve the problem?
>
> Also, I would suggest revisiting the design - I'm not sure such filtering
> should be done on the IP finder level. Why not do this on the SPI level
> instead? I would simply add something like "addressFilter" to the
> TcpDiscoverySpi. The filter can be a generic IgnitePredicate, so you will
> be able to provide any implementations, including regex or anything else.
>
> -Val
>
> On Wed, Apr 21, 2021 at 9:04 AM Atri Sharma  wrote:
>
> > Hi,
> >
> > When a container is shared by multiple clusters, then this can be useful
> > for filtering IPs.
> >
> > Also, things like VPC based barriers can be circumvented using this
> > technique.
> >
> > On Wed, 21 Apr 2021, 15:49 Ilya Kasnacheev, 
> > wrote:
> >
> > > Hello!
> > >
> > > What are the expected use cases for this feature? Can you please
> > elaborate?
> > >
> > > Thanks,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > ср, 21 апр. 2021 г. в 08:23, Atri Sharma :
> > >
> > > > Hi All,
> > > >
> > > > I have opened the following JIRA for the said topic:
> > > >
> > > > https://issues.apache.org/jira/browse/IGNITE-14606
> > > >
> > > > The concept is to filter IPs based on a pattern or a blocklist in
> > > > IPFinders while consuming IPs. This is more pertinent for cloud based
> > > > IPFinders since they can have shared containers.
> > > >
> > > > For the moment, I have implemented regex based filtering:
> > > >
> > > > https://issues.apache.org/jira/browse/IGNITE-14607
> > > >
> > > > for Azure Blob Storage IP Finder. Over time, we can extend the same to
> > > > other IP finders.
> > > >
> > > > Please see the PR:
> > > >
> > > > https://github.com/apache/ignite/pull/9024
> > > >
> > > > Regards,
> > > >
> > > > Atri
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Atri
> > > > Apache Concerted
> > > >
> > >
> >

-- 
Regards,

Atri
Apache Concerted


Re: [VOTE][EXTENSION] Release Apache Ignite performance-statistics-ext, spring-data-all-ext and spring-tx-ext extensions 1.0.0 RC1

2021-04-21 Thread Nikita Amelchev
I have created the issue to fix modules names:

https://issues.apache.org/jira/browse/IGNITE-14621

ср, 21 апр. 2021 г. в 11:46, Nikita Amelchev :
>
> +1 to formalize extension modules names:
> ignite-{directory-name}
>
> The release script has this issue too. It will work fine with that name.
>
> ср, 21 апр. 2021 г. в 10:37, Petr Ivanov :
> >
> > I checked the modules and there is misnaming issue which I think is 
> > critical to test integration automation on TC.
> > Can we change maven module names sping-data-2.x-ext to align with directory 
> > name? Currently there is underscore in maven module name, which is hyphen 
> > in directory name.
> >
> >
> > > On 21 Apr 2021, at 10:22, Nikita Amelchev  wrote:
> > >
> > > +1 to postpone the spring-tx-ext extension release.
> > >
> > > So, the following extensions will be released now:
> > >
> > > spring-data-ext
> > > spring-data-2.0-ext
> > > spring-data-2.2-ext
> > > spring-data-commons
> > > performance-statistics-ext
> > >
> > > вт, 20 апр. 2021 г. в 14:49, Mikhail Petrov :
> > >>
> > >> Igniters,
> > >>
> > >> Changing the scope of Spring dependencies to "provided" in Ignite Spring
> > >> extensions does not currently work as expected:
> > >> some versions of Spring that a user can specify via maven configuration
> > >> for Spring extensions may conflict with the hard-coded version of Spring
> > >> dependencies that the ignite-spring module relies on.
> > >>
> > >> The issue mentioned above affects the Ignite Spring Transactions
> > >> integration. And since this integration is included in the Ignite 2.10
> > >> release, I suggest postponing the release of the Ignite Spring
> > >> Transactions integration until the above issue is properly fixed.
> > >>
> > >> Any objections?
> > >>
> > >> On 16.04.2021 09:15, Ivan Daschinsky wrote:
> > >>> -1 From me. There is an absence of NOTICE and LICENSE files in binary
> > >>> package. Also, there is no source package. These is a violation of 
> > >>> apache
> > >>> release policy [1]
> > >>> [1] https://www.apache.org/legal/release-policy.html
> > >>>
> > >>> чт, 15 апр. 2021 г. в 23:23, Nikita Amelchev :
> > >>>
> >  According to ASF release policy [1] non-PMC committers can sign 
> >  artifacts:
> > 
> > > all artifacts placed in their directory must be signed by a committer,
> >  preferably by a PMC member.
> > 
> >  [1] https://www.apache.org/legal/release-policy.html
> > 
> >  чт, 15 апр. 2021 г. в 23:05, Dmitriy Pavlov :
> > > My best guess that since PMCs have a binding vote in voting in 
> > > release, a
> > > PMC member should sign binaries as well. And I suppose that in the 
> > > past
> > > only PMC members were signing the release.
> > >
> > > Meanwhile, https://infra.apache.org/release-signing.html does not
> >  contain
> > > any mention of PMC role and only committers are mentioned there. It
> > > might be not an answer, since a lot of projects have PMC=Committer and
> >  just
> > > one election.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > чт, 15 апр. 2021 г. в 22:05, Ivan Daschinsky :
> > >
> > >> I'm sorry, but is it OK, that artifacts are signed with signature of
> > >> non-PMC?
> > >>
> > >> чт, 15 апр. 2021 г. в 19:26, Nikita Amelchev :
> > >>
> > >>> Dear Ignite Community,
> > >>>
> > >>> I have uploaded a release candidate of the following extension
> >  modules:
> > >>> performance-statistics-ext
> > >>> spring-data-ext
> > >>> spring-data-2.0-ext
> > >>> spring-data-2.2-ext
> > >>> spring-data-commons
> > >>> spring-tx-ext
> > >>>
> > >>> The release candidate of the performance-statistics-ext extension:
> > >>>
> > >>>
> >  https://dist.apache.org/repos/dist/dev/ignite/ignite-performance-statistics-ext/1.0.0-rc1/
> > >>> The following staging can be used for testing:
> > >>>
> >  https://repository.apache.org/content/repositories/orgapacheignite-1509
> > >>> Tags were created:
> > >>>
> > >>> ignite-performance-statistics-ext-1.0.0-rc1
> > >>>
> > >>>
> >  https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=b992d9758278c38e93b375b08e4052954744a436
> > >>> ignite-spring-data-ext-1.0.0-rc1
> > >>> ignite-spring-data-2.2-ext-1.0.0-rc1
> > >>> ignite-spring-data-2.0-ext-1.0.0-rc1
> > >>> ignite-spring-data-commons-1.0.0-rc1
> > >>> ignite-spring-data-all-ext-1.0.0-rc1
> > >>>
> > >>>
> >  https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=277ca95f6472d8bd46d906e70ba1a312d36dadb0
> > >>> ignite-spring-tx-ext-1.0.0-rc1
> > >>>
> > >>>
> >  https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=55a3ae9e011ba48796847a33481842f154f0febb
> > >>> RELEASE NOTES:
> > >>>
> > >>>
> >  

[jira] [Created] (IGNITE-14621) Ignite Extensions: change spring-data-*-ext modules names to align with a directory name

2021-04-21 Thread Amelchev Nikita (Jira)
Amelchev Nikita created IGNITE-14621:


 Summary: Ignite Extensions: change spring-data-*-ext modules names 
to align with a directory name
 Key: IGNITE-14621
 URL: https://issues.apache.org/jira/browse/IGNITE-14621
 Project: Ignite
  Issue Type: Task
Reporter: Amelchev Nikita
Assignee: Amelchev Nikita


Change the following modules names:
{{ignite-spring-data_2.2-ext}}
{{ignite-spring-data_2.0-ext}}
on
{{ignite-spring-data-2.2-ext}}
{{ignite-spring-data-2.0-ext}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-14620) GridCacheAsyncOperationsLimitSelfTest#testAsyncOps is flaky

2021-04-21 Thread Pavel Tupitsyn (Jira)
Pavel Tupitsyn created IGNITE-14620:
---

 Summary: GridCacheAsyncOperationsLimitSelfTest#testAsyncOps is 
flaky
 Key: IGNITE-14620
 URL: https://issues.apache.org/jira/browse/IGNITE-14620
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.11
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 2.11


GridCacheAsyncOperationsLimitSelfTest#testAsyncOps became flaky because of 
changes from IGNITE-12033



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [DISCUSSION] Error handling in Ignite 3

2021-04-21 Thread Alexei Scherbakov
Alexei,

I think it's ok to do error conversion if it makes sense, but better to
preserve the root cause whenever possible.
Another way to solve the described scenario is to introduce something like
checked IgniteRetryAgainException, which forces the user to retry or ignore
it.
It's difficult to foresee exactly at this point what is the best solution
without knowing the exact scenario.

Andrey,

I've just realized your proposal to add IGN to each string code. This is ok
to me, for example IGN-TBL-0001

ср, 21 апр. 2021 г. в 17:49, Alexey Goncharuk :

> Aleksei,
>
> > The method should always report root cause, in your example it will be
> > B-, no matter which module API is called
>
> I may be wrong, but I doubt this will be usable for an end-user. Let's
> imagine that the same root exception was raised in different contexts
> resulting in two outcomes. The first one is safe to retry (say, the root
> cause led to a transaction prepare failure), but the second outcome may be
> a state in which no matter how many retries will be attempted, the
> operation will never succeed. Only the upper-level layer can tell the
> difference and return a proper message to the user, so I would say that
> some error conversion/wrapping will be required no matter what.
>
> --AG
>
> пт, 16 апр. 2021 г. в 16:31, Alexei Scherbakov <
> alexey.scherbak...@gmail.com
> >:
>
> > чт, 15 апр. 2021 г. в 18:21, Andrey Mashenkov <
> andrey.mashen...@gmail.com
> > >:
> >
> > > Hi Alexey,
> > > I like the idea.
> > >
> > > 1.
> > >
> > > >   TBL-0001 is a *string representation* of the error. It is built
> from
> > 2
> > > > byte scope id (mapped to name TBL) and 2 byte number (0001). Both
> > > > internally packed in int. No any kind of parsing will be necessary to
> > > read
> > > > scope/category.
> > >
> > > I think Alexey mean if it will be possible to make smth like that
> > >
> > > catch (IgniteException e) {
> > > if (e.getScope() == "TBL" && e.getCode() == 1234)
> > > continue; // E.g. retry my TX
> > > }
> > >
> > > It looks useful to me.
> > >
> >
> > I have in mind something like this:
> >
> > public class IgniteException extends RuntimeException {
> > private int errorCode;
> >
> > public IgniteException(ErrorScope scope, int code, String message,
> > Throwable cause) {
> > super(message, cause);
> > this.errorCode = make(scope, code);
> > }
> >
> > public boolean matches(ErrorScope scope, int code) {
> > return errorCode == make(scope, code);
> > }
> >
> > private int make(ErrorScope scope, int code) {
> > return ((scope.ordinal() << 16) | code);
> > }
> >
> > public ErrorScope scope() {
> > return ErrorScope.values()[errorCode >> 16];
> > }
> >
> > public int code() {
> > return 0x & errorCode;
> > }
> >
> > public static void main(String[] args) {
> > IgniteException e = new IgniteException(ErrorScope.RAFT, 1,
> "test",
> > null);
> >
> > System.out.println(e.matches(ErrorScope.RAFT, 2));
> > System.out.println(e.scope());
> > System.out.println(e.code());
> >
> > try {
> > throw e;
> > }
> > catch (IgniteException ee) {
> > System.out.println(ee.matches(ErrorScope.RAFT, 1));
> > }
> > }
> > }
> >
> >
> > >
> > > 2. How you see a cross-module exception throwing?
> > > Assume, user call -> module A, which recursively call -> module B,
> which
> > > fails.
> > > So, module A component calls a module B component and got an Exception
> > with
> > > "B-1234" exception.
> > > Module A do not expect any exception here and doesn't take care of
> > "B-xxx"
> > > error codes, but only "A-.
> > > Should it rethrow exception with "A-unknown" (maybe "UNK-0001") code
> > > or reuse "B-" code with the own message, pointing original
> exception
> > as
> > > a cause for both cases?
> > >
> > > The first approach may looks confusing, while the second one produces
> too
> > > many "UNK-" codes.
> > > What code should get the user in that case?
> > >
> > >
> > >
> > The method should always report root cause, in your example it will be
> > B-, no matter which module API is called.
> >
> >
> > >
> > >
> > >
> > > On Thu, Apr 15, 2021 at 5:36 PM Alexei Scherbakov <
> > > alexey.scherbak...@gmail.com> wrote:
> > >
> > > > чт, 15 апр. 2021 г. в 14:26, Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com
> > > >:
> > > >
> > > > > Hello!
> > > > >
> > > > > > All public methods throw only unchecked
> > > > > org.apache.ignite.lang.IgniteException containing aforementioned
> > > fields.
> > > > > > Each public method must have a section in the javadoc with a list
> > of
> > > > all
> > > > > possible error codes for this method.
> > > > >
> > > > > I don't think this is feasible at all.
> > > > > Imagine javadoc for cache.get() method featuring three pages of
> > > possible
> > > > > error codes thrown by this method.
> > > > >
> > > >
> 

Re: IP Filtering in IPFinders

2021-04-21 Thread Andrey Mashenkov
Hi Atri,

You've added a new property to a base TcpDiscoveryIpFinder interface.
Actually, the only Azure IpFinder uses this setting, but the others.
This behavior may confuse the users.

Would you mind either making regexp filter setting a part of Azure IpFinder
only or fix other IpFinders as well?


On Wed, Apr 21, 2021 at 7:04 PM Atri Sharma  wrote:

> Hi,
>
> When a container is shared by multiple clusters, then this can be useful
> for filtering IPs.
>
> Also, things like VPC based barriers can be circumvented using this
> technique.
>
> On Wed, 21 Apr 2021, 15:49 Ilya Kasnacheev, 
> wrote:
>
> > Hello!
> >
> > What are the expected use cases for this feature? Can you please
> elaborate?
> >
> > Thanks,
> > --
> > Ilya Kasnacheev
> >
> >
> > ср, 21 апр. 2021 г. в 08:23, Atri Sharma :
> >
> > > Hi All,
> > >
> > > I have opened the following JIRA for the said topic:
> > >
> > > https://issues.apache.org/jira/browse/IGNITE-14606
> > >
> > > The concept is to filter IPs based on a pattern or a blocklist in
> > > IPFinders while consuming IPs. This is more pertinent for cloud based
> > > IPFinders since they can have shared containers.
> > >
> > > For the moment, I have implemented regex based filtering:
> > >
> > > https://issues.apache.org/jira/browse/IGNITE-14607
> > >
> > > for Azure Blob Storage IP Finder. Over time, we can extend the same to
> > > other IP finders.
> > >
> > > Please see the PR:
> > >
> > > https://github.com/apache/ignite/pull/9024
> > >
> > > Regards,
> > >
> > > Atri
> > >
> > > --
> > > Regards,
> > >
> > > Atri
> > > Apache Concerted
> > >
> >
>


-- 
Best regards,
Andrey V. Mashenkov


Re: IP Filtering in IPFinders

2021-04-21 Thread Valentin Kulichenko
Hi Atri,

Can you describe the scenario in a little more detail? What exactly do you
mean by a container shared by multiple clusters? What are the consequences
of this? How does the proposed solution solve the problem?

Also, I would suggest revisiting the design - I'm not sure such filtering
should be done on the IP finder level. Why not do this on the SPI level
instead? I would simply add something like "addressFilter" to the
TcpDiscoverySpi. The filter can be a generic IgnitePredicate, so you will
be able to provide any implementations, including regex or anything else.

-Val

On Wed, Apr 21, 2021 at 9:04 AM Atri Sharma  wrote:

> Hi,
>
> When a container is shared by multiple clusters, then this can be useful
> for filtering IPs.
>
> Also, things like VPC based barriers can be circumvented using this
> technique.
>
> On Wed, 21 Apr 2021, 15:49 Ilya Kasnacheev, 
> wrote:
>
> > Hello!
> >
> > What are the expected use cases for this feature? Can you please
> elaborate?
> >
> > Thanks,
> > --
> > Ilya Kasnacheev
> >
> >
> > ср, 21 апр. 2021 г. в 08:23, Atri Sharma :
> >
> > > Hi All,
> > >
> > > I have opened the following JIRA for the said topic:
> > >
> > > https://issues.apache.org/jira/browse/IGNITE-14606
> > >
> > > The concept is to filter IPs based on a pattern or a blocklist in
> > > IPFinders while consuming IPs. This is more pertinent for cloud based
> > > IPFinders since they can have shared containers.
> > >
> > > For the moment, I have implemented regex based filtering:
> > >
> > > https://issues.apache.org/jira/browse/IGNITE-14607
> > >
> > > for Azure Blob Storage IP Finder. Over time, we can extend the same to
> > > other IP finders.
> > >
> > > Please see the PR:
> > >
> > > https://github.com/apache/ignite/pull/9024
> > >
> > > Regards,
> > >
> > > Atri
> > >
> > > --
> > > Regards,
> > >
> > > Atri
> > > Apache Concerted
> > >
> >
>


[RESULT][VOTE] Release pyignite-0.4.0-rc1

2021-04-21 Thread Ivan Daschinsky
Hello, Igniters!

Release pyignite-0.4.0-rc1 has been accepted.

The votes received:
3 "+1" binding votes
2 "+1" votes

There are no "+0" or "-1"

Here the votes received:
- Denis Magda (binding)
- Igor Sapego (binding)
- Anton Vinogradov (binding)
- Ivan Daschinsky
- Alexander Surkov

Link to the thread:
http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Release-pyignite-0-4-0-rc1-td52364.html

Thank you!


Re: IP Filtering in IPFinders

2021-04-21 Thread Atri Sharma
Hi,

When a container is shared by multiple clusters, then this can be useful
for filtering IPs.

Also, things like VPC based barriers can be circumvented using this
technique.

On Wed, 21 Apr 2021, 15:49 Ilya Kasnacheev, 
wrote:

> Hello!
>
> What are the expected use cases for this feature? Can you please elaborate?
>
> Thanks,
> --
> Ilya Kasnacheev
>
>
> ср, 21 апр. 2021 г. в 08:23, Atri Sharma :
>
> > Hi All,
> >
> > I have opened the following JIRA for the said topic:
> >
> > https://issues.apache.org/jira/browse/IGNITE-14606
> >
> > The concept is to filter IPs based on a pattern or a blocklist in
> > IPFinders while consuming IPs. This is more pertinent for cloud based
> > IPFinders since they can have shared containers.
> >
> > For the moment, I have implemented regex based filtering:
> >
> > https://issues.apache.org/jira/browse/IGNITE-14607
> >
> > for Azure Blob Storage IP Finder. Over time, we can extend the same to
> > other IP finders.
> >
> > Please see the PR:
> >
> > https://github.com/apache/ignite/pull/9024
> >
> > Regards,
> >
> > Atri
> >
> > --
> > Regards,
> >
> > Atri
> > Apache Concerted
> >
>


Re: Stop sending IGNITE Created e-mails to dev@

2021-04-21 Thread Ilya Kasnacheev
Hello!

INFRA ticket created: https://issues.apache.org/jira/browse/INFRA-21762

I have asked to keep sending the created issue notifications for
"Contributors 1" role, which is empty at present. So if you wish to keep
getting those e-mails, please add yourself to this role or tell me to do so
for you.

Regards,
-- 
Ilya Kasnacheev


ср, 21 апр. 2021 г. в 17:59, Alexey Goncharuk :

> I support the idea. All issues notifications are also sent to
> iss...@ignite.apache.org so one can subscribe to this list in order to
> track the created tickets. The notifications trash the devlist archive UI
> and make it extremely difficult to navigate.
>
> вт, 20 апр. 2021 г. в 18:35, Ilya Kasnacheev :
>
> > Hello, Maxim!
> >
> > You are free to revert any commit which has led to any new stable test
> > failure, or new flaky test that was non-flaky before.
> >
> > Just revert the change and reopen the ticket.
> >
> > The problem here is that it's very hard to detect on the spot, most of
> > MTCGA e-mails are false positives and even if they are not, it is not
> > relevant for most of developers.
> >
> > WDYT? I'm also still waiting for more input.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > ср, 14 апр. 2021 г. в 21:26, Maxim Muzafarov :
> >
> > > +1 for new JIRA issues
> > > -1 for MTCGA notifications
> > >
> > > Why we should hide errors from the dev-list? Who should take care of
> > > issues reported by MTCGA.Bot in this case?
> > > We must apply stricter rules for such issues: a commit leading to an
> > > error must be reverted.
> > >
> > > On Wed, 14 Apr 2021 at 20:00, Denis Mekhanikov 
> > > wrote:
> > > >
> > > > Huge +1 to this.
> > > >
> > > > I've already brought up this topic in the past:
> > > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Bots-on-dev-list-td34406.html
> > > > I hope some day newcomers won't need to set up their email filters
> when
> > > > they come to the developers list.
> > > >
> > > > Denis
> > > >
> > > > ср, 14 апр. 2021 г. в 18:07, Atri Sharma :
> > > >
> > > > > +1 to move issues to the issues list.
> > > > >
> > > > > For MTCGA, maybe build@?
> > > > >
> > > > > On Wed, Apr 14, 2021 at 8:35 PM Ilya Kasnacheev 
> > > wrote:
> > > > > >
> > > > > > Hello!
> > > > > >
> > > > > > We have a discussion on how to ensure best engagement in dev@
> > list,
> > > and
> > > > > it
> > > > > > seems that Issue Created emails from IGNITE project consume a lot
> > of
> > > > > screen
> > > > > > space, it's hard to spot genuine discussions in
> > > > > > https://lists.apache.org/list.html?dev@ignite.apache.org for
> > > example.
> > > > > >
> > > > > > We already have issues@ mailing list. I propose that we stop
> > > sending any
> > > > > > JIRA emails to dev@. If anyone wishes to get just Created
> emails,
> > > they
> > > > > can
> > > > > > subscribe to these messages in their JIRA account settings. I
> > imagine
> > > > > most
> > > > > > of you already filter these messages out, so you may need to
> adjust
> > > your
> > > > > > filters slightly.
> > > > > >
> > > > > > A distant second is MTCGA messages, which are also autogenerated
> > and
> > > not
> > > > > > informative for most readers of the channel, since they are at
> best
> > > > > > targeted at a single committer and at worst flaky.
> > > > > >
> > > > > > Where could we move those? What is your opinion here, on both
> > issues?
> > > > > >
> > > > > > Regards,
> > > > >
> > > > > --
> > > > > Regards,
> > > > >
> > > > > Atri
> > > > > Apache Concerted
> > > > >
> > >
> >
>


Re: Stop sending IGNITE Created e-mails to dev@

2021-04-21 Thread Alexey Goncharuk
I support the idea. All issues notifications are also sent to
iss...@ignite.apache.org so one can subscribe to this list in order to
track the created tickets. The notifications trash the devlist archive UI
and make it extremely difficult to navigate.

вт, 20 апр. 2021 г. в 18:35, Ilya Kasnacheev :

> Hello, Maxim!
>
> You are free to revert any commit which has led to any new stable test
> failure, or new flaky test that was non-flaky before.
>
> Just revert the change and reopen the ticket.
>
> The problem here is that it's very hard to detect on the spot, most of
> MTCGA e-mails are false positives and even if they are not, it is not
> relevant for most of developers.
>
> WDYT? I'm also still waiting for more input.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 14 апр. 2021 г. в 21:26, Maxim Muzafarov :
>
> > +1 for new JIRA issues
> > -1 for MTCGA notifications
> >
> > Why we should hide errors from the dev-list? Who should take care of
> > issues reported by MTCGA.Bot in this case?
> > We must apply stricter rules for such issues: a commit leading to an
> > error must be reverted.
> >
> > On Wed, 14 Apr 2021 at 20:00, Denis Mekhanikov 
> > wrote:
> > >
> > > Huge +1 to this.
> > >
> > > I've already brought up this topic in the past:
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Bots-on-dev-list-td34406.html
> > > I hope some day newcomers won't need to set up their email filters when
> > > they come to the developers list.
> > >
> > > Denis
> > >
> > > ср, 14 апр. 2021 г. в 18:07, Atri Sharma :
> > >
> > > > +1 to move issues to the issues list.
> > > >
> > > > For MTCGA, maybe build@?
> > > >
> > > > On Wed, Apr 14, 2021 at 8:35 PM Ilya Kasnacheev 
> > wrote:
> > > > >
> > > > > Hello!
> > > > >
> > > > > We have a discussion on how to ensure best engagement in dev@
> list,
> > and
> > > > it
> > > > > seems that Issue Created emails from IGNITE project consume a lot
> of
> > > > screen
> > > > > space, it's hard to spot genuine discussions in
> > > > > https://lists.apache.org/list.html?dev@ignite.apache.org for
> > example.
> > > > >
> > > > > We already have issues@ mailing list. I propose that we stop
> > sending any
> > > > > JIRA emails to dev@. If anyone wishes to get just Created emails,
> > they
> > > > can
> > > > > subscribe to these messages in their JIRA account settings. I
> imagine
> > > > most
> > > > > of you already filter these messages out, so you may need to adjust
> > your
> > > > > filters slightly.
> > > > >
> > > > > A distant second is MTCGA messages, which are also autogenerated
> and
> > not
> > > > > informative for most readers of the channel, since they are at best
> > > > > targeted at a single committer and at worst flaky.
> > > > >
> > > > > Where could we move those? What is your opinion here, on both
> issues?
> > > > >
> > > > > Regards,
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Atri
> > > > Apache Concerted
> > > >
> >
>


Re: Review for IGNITE-13112 The current security context should be obtained using the IgniteSecurity interface only.

2021-04-21 Thread Maksim Stepachev
Please wait. I'm watching your review.

вт, 6 апр. 2021 г. в 20:14, Denis Garus :

> Hello, Igniters!
>
> I've raised the PR [1] for the issue [2].
> Could somebody review it?
>
> Suggested implementation
>
> If Ignite Security (IS) is enabled, then executors, accessed through the
> PoolProcessor,
> are wrapped to a security-aware implementation. Security-aware
> implementation sets proper
> security context for tasks that the executor performs.
>
> The field subject id was deleted from communication requests for cache and
> compute operations;
> a remote node gets the subject id that initiates the ignite operation from
> GridIoSecurityAwareMessage.
> IgniteSecurity uses this id to set a proper security context during the
> execution of the request.
>
> Remove GridTaskThreadContextKey#TC_SUBJ_ID,
> GridCacheContext#subjectIdPerCall;
> a consumer has to obtain a current security subject id through
> IgniteSecurity
> or the set of SecurityUtils methods.
>
> For all events that include the subject id field, are set the following
> rule.
> If IS is enabled, this field must contain a subject id that initiates
> an ignite operation, otherwise null.
>
> Implement SecurityAwareCustomMessageWrapper for discovery requests that act
> as
> GridIoSecurityAwareMessage for communication requests. It allows setting
> proper
> context during the discovery message execution.
>
> Implement SecurityAwareGridRestCommandHandler to allow GridRestProcessor
> to execute all client requests with the proper security context.
>
> 1. https://github.com/apache/ignite/pull/8038
> 2. https://issues.apache.org/jira/browse/IGNITE-13112
>


Re: [DISCUSSION] Error handling in Ignite 3

2021-04-21 Thread Alexey Goncharuk
Aleksei,

> The method should always report root cause, in your example it will be
> B-, no matter which module API is called

I may be wrong, but I doubt this will be usable for an end-user. Let's
imagine that the same root exception was raised in different contexts
resulting in two outcomes. The first one is safe to retry (say, the root
cause led to a transaction prepare failure), but the second outcome may be
a state in which no matter how many retries will be attempted, the
operation will never succeed. Only the upper-level layer can tell the
difference and return a proper message to the user, so I would say that
some error conversion/wrapping will be required no matter what.

--AG

пт, 16 апр. 2021 г. в 16:31, Alexei Scherbakov :

> чт, 15 апр. 2021 г. в 18:21, Andrey Mashenkov  >:
>
> > Hi Alexey,
> > I like the idea.
> >
> > 1.
> >
> > >   TBL-0001 is a *string representation* of the error. It is built from
> 2
> > > byte scope id (mapped to name TBL) and 2 byte number (0001). Both
> > > internally packed in int. No any kind of parsing will be necessary to
> > read
> > > scope/category.
> >
> > I think Alexey mean if it will be possible to make smth like that
> >
> > catch (IgniteException e) {
> > if (e.getScope() == "TBL" && e.getCode() == 1234)
> > continue; // E.g. retry my TX
> > }
> >
> > It looks useful to me.
> >
>
> I have in mind something like this:
>
> public class IgniteException extends RuntimeException {
> private int errorCode;
>
> public IgniteException(ErrorScope scope, int code, String message,
> Throwable cause) {
> super(message, cause);
> this.errorCode = make(scope, code);
> }
>
> public boolean matches(ErrorScope scope, int code) {
> return errorCode == make(scope, code);
> }
>
> private int make(ErrorScope scope, int code) {
> return ((scope.ordinal() << 16) | code);
> }
>
> public ErrorScope scope() {
> return ErrorScope.values()[errorCode >> 16];
> }
>
> public int code() {
> return 0x & errorCode;
> }
>
> public static void main(String[] args) {
> IgniteException e = new IgniteException(ErrorScope.RAFT, 1, "test",
> null);
>
> System.out.println(e.matches(ErrorScope.RAFT, 2));
> System.out.println(e.scope());
> System.out.println(e.code());
>
> try {
> throw e;
> }
> catch (IgniteException ee) {
> System.out.println(ee.matches(ErrorScope.RAFT, 1));
> }
> }
> }
>
>
> >
> > 2. How you see a cross-module exception throwing?
> > Assume, user call -> module A, which recursively call -> module B, which
> > fails.
> > So, module A component calls a module B component and got an Exception
> with
> > "B-1234" exception.
> > Module A do not expect any exception here and doesn't take care of
> "B-xxx"
> > error codes, but only "A-.
> > Should it rethrow exception with "A-unknown" (maybe "UNK-0001") code
> > or reuse "B-" code with the own message, pointing original exception
> as
> > a cause for both cases?
> >
> > The first approach may looks confusing, while the second one produces too
> > many "UNK-" codes.
> > What code should get the user in that case?
> >
> >
> >
> The method should always report root cause, in your example it will be
> B-, no matter which module API is called.
>
>
> >
> >
> >
> > On Thu, Apr 15, 2021 at 5:36 PM Alexei Scherbakov <
> > alexey.scherbak...@gmail.com> wrote:
> >
> > > чт, 15 апр. 2021 г. в 14:26, Ilya Kasnacheev <
> ilya.kasnach...@gmail.com
> > >:
> > >
> > > > Hello!
> > > >
> > > > > All public methods throw only unchecked
> > > > org.apache.ignite.lang.IgniteException containing aforementioned
> > fields.
> > > > > Each public method must have a section in the javadoc with a list
> of
> > > all
> > > > possible error codes for this method.
> > > >
> > > > I don't think this is feasible at all.
> > > > Imagine javadoc for cache.get() method featuring three pages of
> > possible
> > > > error codes thrown by this method.
> > > >
> > >
> > > Of course there is no need to write 3 pages of error codes, this makes
> no
> > > sense.
> > > I think we can use error ranges here, or, probably, document most
> > important
> > > error scenarios.
> > > The point here is to try to document errors as much as possible.
> > >
> > >
> > > > Also, updated every two weeks to account for changes in
> > > > underlying mechanisms.
> > > >
> > > > There is still a confusion between "error code for any error in logs"
> > and
> > > > "error code for any pair of method & exception". Which one are we
> > > > discussing really?
> > > >
> > > > This is impossible to track or test, but is susceptible for common
> > > > error-hiding antipattern where all exceptions are caught in
> cache.get()
> > > and
> > > > discarded, and instead a brand new CH-0001 "error in cache.get()" is
> > > thrown
> > > > to the user.
> > > >
> > > > Which is certainly not 

Re: Apache Ignite 3 / RPM

2021-04-21 Thread Ilya Kasnacheev
Hello!

Were you able to get any traction here?

Apache Ignite 3.0 seems to be in too early stage for packaging, IMHO.
We see very few interest in improving Apache Ignite 2.x packages, anyway.

Regards,
-- 
Ilya Kasnacheev


ср, 24 февр. 2021 г. в 19:01, Petr Ivanov :

> Hi, Igniters.
>
>
> Sincerely asking you to help me with testing new RPM package for apache
> ignite 3.x [1]
> Currently what is needed to be tested — successful installation and binary
> run under most of RHEL based distributions: Red Hat, CentOS, Oracle Linux,
> Fedora, etc.
>
> Any other thoughts and suggestions are more than welcome too.
> RPM assembly code can be found here [2]
>
>
> Thanks in advance!
>
>
>
> [1]
> https://issues.apache.org/jira/secure/attachment/13021121/apache-ignite-3.0.0-1.noarch.rpm
> [2] https://github.com/apache/ignite-3/pull/29/files
>
>


[jira] [Created] (IGNITE-14619) Refactoring of GridDeploymentCommunication

2021-04-21 Thread Denis Chudov (Jira)
Denis Chudov created IGNITE-14619:
-

 Summary: Refactoring of GridDeploymentCommunication
 Key: IGNITE-14619
 URL: https://issues.apache.org/jira/browse/IGNITE-14619
 Project: Ignite
  Issue Type: Improvement
Reporter: Denis Chudov


org.apache.ignite.internal.managers.deployment.GridDeploymentCommunication#sendResourceRequest
 uses "while" loop with mutex instead of future, and creates listeners for 
discovery events and communication messages for each request. This complicates 
the code and may affect class loading performance.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [DISCUSSION] Error handling in Ignite 3

2021-04-21 Thread Alexei Scherbakov
Andrey,

I've already proposed the similar string representation in the very first
message of the topic.

ср, 21 апр. 2021 г. в 12:31, Andrey Mashenkov :

> Alexey,
>
> As I understand, you suggest ErrorScope enum for easier analysis and it
> will be a part of a error code.
> But what about String representation?
>
> I think we should use a common prefix for error codes in error messages.
> Such codes will be more searchable and as a bonus, vendor-specific.
> String representation might look like IGN-001042 or IGN-TBL042.
>
>
>
>
> On Wed, Apr 21, 2021 at 11:43 AM Alexei Scherbakov <
> alexey.scherbak...@gmail.com> wrote:
>
> > I've create the ticket for implementing this [1]
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-14611
> >
> > пт, 16 апр. 2021 г. в 16:30, Alexei Scherbakov <
> > alexey.scherbak...@gmail.com
> > >:
> >
> > >
> > >
> > > чт, 15 апр. 2021 г. в 18:21, Andrey Mashenkov <
> > andrey.mashen...@gmail.com
> > > >:
> > >
> > >> Hi Alexey,
> > >> I like the idea.
> > >>
> > >> 1.
> > >>
> > >> >   TBL-0001 is a *string representation* of the error. It is built
> > from 2
> > >> > byte scope id (mapped to name TBL) and 2 byte number (0001). Both
> > >> > internally packed in int. No any kind of parsing will be necessary
> to
> > >> read
> > >> > scope/category.
> > >>
> > >> I think Alexey mean if it will be possible to make smth like that
> > >>
> > >> catch (IgniteException e) {
> > >> if (e.getScope() == "TBL" && e.getCode() == 1234)
> > >> continue; // E.g. retry my TX
> > >> }
> > >>
> > >> It looks useful to me.
> > >>
> > >
> > > I have in mind something like this:
> > >
> > > public class IgniteException extends RuntimeException {
> > > private int errorCode;
> > >
> > > public IgniteException(ErrorScope scope, int code, String message,
> > > Throwable cause) {
> > > super(message, cause);
> > > this.errorCode = make(scope, code);
> > > }
> > >
> > > public boolean matches(ErrorScope scope, int code) {
> > > return errorCode == make(scope, code);
> > > }
> > >
> > > private int make(ErrorScope scope, int code) {
> > > return ((scope.ordinal() << 16) | code);
> > > }
> > >
> > > public ErrorScope scope() {
> > > return ErrorScope.values()[errorCode >> 16];
> > > }
> > >
> > > public int code() {
> > > return 0x & errorCode;
> > > }
> > >
> > > public static void main(String[] args) {
> > > IgniteException e = new IgniteException(ErrorScope.RAFT, 1,
> > > "test", null);
> > >
> > > System.out.println(e.matches(ErrorScope.RAFT, 2));
> > > System.out.println(e.scope());
> > > System.out.println(e.code());
> > >
> > > try {
> > > throw e;
> > > }
> > > catch (IgniteException ee) {
> > > System.out.println(ee.matches(ErrorScope.RAFT, 1));
> > > }
> > > }
> > > }
> > >
> > >
> > >>
> > >> 2. How you see a cross-module exception throwing?
> > >> Assume, user call -> module A, which recursively call -> module B,
> which
> > >> fails.
> > >> So, module A component calls a module B component and got an Exception
> > >> with
> > >> "B-1234" exception.
> > >> Module A do not expect any exception here and doesn't take care of
> > "B-xxx"
> > >> error codes, but only "A-.
> > >> Should it rethrow exception with "A-unknown" (maybe "UNK-0001") code
> > >> or reuse "B-" code with the own message, pointing original
> exception
> > >> as
> > >> a cause for both cases?
> > >>
> > >> The first approach may looks confusing, while the second one produces
> > too
> > >> many "UNK-" codes.
> > >> What code should get the user in that case?
> > >>
> > >>
> > >>
> > > The method should always report root cause, in your example it will be
> > > B-, no matter which module API is called.
> > >
> > >
> > >>
> > >>
> > >>
> > >> On Thu, Apr 15, 2021 at 5:36 PM Alexei Scherbakov <
> > >> alexey.scherbak...@gmail.com> wrote:
> > >>
> > >> > чт, 15 апр. 2021 г. в 14:26, Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com
> > >> >:
> > >> >
> > >> > > Hello!
> > >> > >
> > >> > > > All public methods throw only unchecked
> > >> > > org.apache.ignite.lang.IgniteException containing aforementioned
> > >> fields.
> > >> > > > Each public method must have a section in the javadoc with a
> list
> > of
> > >> > all
> > >> > > possible error codes for this method.
> > >> > >
> > >> > > I don't think this is feasible at all.
> > >> > > Imagine javadoc for cache.get() method featuring three pages of
> > >> possible
> > >> > > error codes thrown by this method.
> > >> > >
> > >> >
> > >> > Of course there is no need to write 3 pages of error codes, this
> makes
> > >> no
> > >> > sense.
> > >> > I think we can use error ranges here, or, probably, document most
> > >> important
> > >> > error scenarios.
> > >> > The point here is to try to document errors as much as possible.
> > >> >
> > >> >
> > >> > > Also, 

[jira] [Created] (IGNITE-14618) Increase stop timeout to 60 seconds

2021-04-21 Thread Anton Vinogradov (Jira)
Anton Vinogradov created IGNITE-14618:
-

 Summary: Increase stop timeout to 60 seconds
 Key: IGNITE-14618
 URL: https://issues.apache.org/jira/browse/IGNITE-14618
 Project: Ignite
  Issue Type: Sub-task
Reporter: Anton Vinogradov
Assignee: Anton Vinogradov


Graceful stops may take 10+ seconds, so we should increase the timeout to have 
no false-negative results.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-14617) Calcite. Add opportunity to use NUMERIC in WHEN syntax.

2021-04-21 Thread Stanilovsky Evgeny (Jira)
Stanilovsky Evgeny created IGNITE-14617:
---

 Summary: Calcite. Add opportunity to use NUMERIC in WHEN syntax.
 Key: IGNITE-14617
 URL: https://issues.apache.org/jira/browse/IGNITE-14617
 Project: Ignite
  Issue Type: Improvement
  Components: sql
Reporter: Stanilovsky Evgeny


Probably we need to support such kind of expressions:

{noformat}
SELECT CASE WHEN 1 THEN 13 ELSE 12 END;
{noformat}

_WHEN 1=1_ - works correctly in such case.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-14616) eplace the string variables that store the security password with array of char.

2021-04-21 Thread Mikhail Petrov (Jira)
Mikhail Petrov created IGNITE-14616:
---

 Summary: eplace the string variables that store the security 
password with array of char.
 Key: IGNITE-14616
 URL: https://issues.apache.org/jira/browse/IGNITE-14616
 Project: Ignite
  Issue Type: Improvement
Reporter: Mikhail Petrov


It is needed to replace the string variables that store the security password 
with an array of characters.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-14615) Add support of grant/revoke commands for Ignite security plugin

2021-04-21 Thread Mikhail Petrov (Jira)
Mikhail Petrov created IGNITE-14615:
---

 Summary: Add support of grant/revoke commands for Ignite security 
plugin
 Key: IGNITE-14615
 URL: https://issues.apache.org/jira/browse/IGNITE-14615
 Project: Ignite
  Issue Type: Improvement
Reporter: Mikhail Petrov


It's needed to add support of grant/revoke commands for Ignite security plugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-14614) Add the ability to create security user with custom options

2021-04-21 Thread Mikhail Petrov (Jira)
Mikhail Petrov created IGNITE-14614:
---

 Summary: Add the ability to create security user with custom 
options
 Key: IGNITE-14614
 URL: https://issues.apache.org/jira/browse/IGNITE-14614
 Project: Ignite
  Issue Type: Improvement
Reporter: Mikhail Petrov


It is needed to add the ability to create security user with custom options 
that can be specific for security plugin implementation.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-14613) Mark CacheConfiguration.rebalanceDelay as deprecated.

2021-04-21 Thread Ilya Kasnacheev (Jira)
Ilya Kasnacheev created IGNITE-14613:


 Summary: Mark CacheConfiguration.rebalanceDelay as deprecated.
 Key: IGNITE-14613
 URL: https://issues.apache.org/jira/browse/IGNITE-14613
 Project: Ignite
  Issue Type: Improvement
Reporter: Ilya Kasnacheev
Assignee: Ilya Kasnacheev


Now that we have baseline topology and baseline auto-adjust, rebalance delay is 
rarely needed and does not see enough use/testing. See for example the mailing 
list thread where it became apparent it will disable historical rebalancing.

Let's mark it as deprecated, target its removal in 3.0.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: IP Filtering in IPFinders

2021-04-21 Thread Ilya Kasnacheev
Hello!

What are the expected use cases for this feature? Can you please elaborate?

Thanks,
-- 
Ilya Kasnacheev


ср, 21 апр. 2021 г. в 08:23, Atri Sharma :

> Hi All,
>
> I have opened the following JIRA for the said topic:
>
> https://issues.apache.org/jira/browse/IGNITE-14606
>
> The concept is to filter IPs based on a pattern or a blocklist in
> IPFinders while consuming IPs. This is more pertinent for cloud based
> IPFinders since they can have shared containers.
>
> For the moment, I have implemented regex based filtering:
>
> https://issues.apache.org/jira/browse/IGNITE-14607
>
> for Azure Blob Storage IP Finder. Over time, we can extend the same to
> other IP finders.
>
> Please see the PR:
>
> https://github.com/apache/ignite/pull/9024
>
> Regards,
>
> Atri
>
> --
> Regards,
>
> Atri
> Apache Concerted
>


[jira] [Created] (IGNITE-14612) Calcite. SELECT with CASE without boolean expression - fails.

2021-04-21 Thread Stanilovsky Evgeny (Jira)
Stanilovsky Evgeny created IGNITE-14612:
---

 Summary: Calcite. SELECT with CASE without boolean expression - 
fails.
 Key: IGNITE-14612
 URL: https://issues.apache.org/jira/browse/IGNITE-14612
 Project: Ignite
  Issue Type: Bug
  Components: sql
Reporter: Stanilovsky Evgeny
Assignee: Stanilovsky Evgeny


For example such expression - fails.
{noformat}
SELECT CASE WHEN 1 THEN 13 ELSE 12 END;{noformat}

failed with:

{code:java}
class org.apache.ignite.IgniteException: Error at: 
test_constant_comparisons.test:91. sql: SELECT CASE WHEN 1 THEN 13 ELSE 12 END;

at 
org.apache.ignite.internal.processors.query.calcite.logical.SqlScriptRunner$Query.execute(SqlScriptRunner.java:518)
at 
org.apache.ignite.internal.processors.query.calcite.logical.SqlScriptRunner.run(SqlScriptRunner.java:93)
at 
org.apache.ignite.internal.processors.query.calcite.logical.ScriptTestRunner$1.run(ScriptTestRunner.java:214)
at java.lang.Thread.run(Thread.java:748)
Caused by: class 
org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to plan 
query.
at 
org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.prepareQuery(ExecutionServiceImpl.java:541)
at 
org.apache.ignite.internal.processors.query.calcite.prepare.QueryPlanCacheImpl.queryPlan(QueryPlanCacheImpl.java:84)
at 
org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.executeQuery(ExecutionServiceImpl.java:398)
at 
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.query(CalciteQueryProcessor.java:246)
at 
org.apache.ignite.internal.processors.query.calcite.logical.SqlScriptRunner.sql(SqlScriptRunner.java:111)
at 
org.apache.ignite.internal.processors.query.calcite.logical.SqlScriptRunner.access$600(SqlScriptRunner.java:51)
at 
org.apache.ignite.internal.processors.query.calcite.logical.SqlScriptRunner$Query.execute(SqlScriptRunner.java:513)
... 3 more
Caused by: org.apache.calcite.runtime.CalciteContextException: From line 1, 
column 8 to line 1, column 38: Expected a boolean type
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at 
org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:467)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:883)
{code}

and this work perfectly well:
{noformat}
SELECT CASE WHEN 1=1 THEN 13 ELSE 12 END;{noformat}

a little reserch shows that calcite also fails in non boolean expression, root 
cause in 
SqlCaseOperator#checkOperandTypes and further check : 
{code:java}
if (!SqlTypeUtil.inBooleanFamily(type){code} if we change this place, 
appropriate test

{noformat}
  @Test void testCaseWhen2() {
checkPlanEquals("SELECT CASE WHEN 1=1 THEN 13 ELSE 12 END");
  }{noformat}
will work fine in calcite but will fail in ignite with janino error:

{noformat}
class org.apache.ignite.IgniteException: Line 3, Column 7: Not a boolean 
expression

at 
org.apache.ignite.internal.processors.query.calcite.util.Commons.compile(Commons.java:299)
at 
org.apache.ignite.internal.processors.query.calcite.exec.exp.ExpressionFactoryImpl.compile(ExpressionFactoryImpl.java:290)
at 
org.apache.ignite.internal.processors.query.calcite.exec.exp.ExpressionFactoryImpl.lambda$scalar$4(ExpressionFactoryImpl.java:241)
at 
java.util.concurrent.ConcurrentMap.computeIfAbsent(ConcurrentMap.java:324)
at 
org.apache.ignite.internal.processors.query.calcite.exec.exp.ExpressionFactoryImpl.scalar(ExpressionFactoryImpl.java:241)
at 
org.apache.ignite.internal.processors.query.calcite.exec.exp.ExpressionFactoryImpl.project(ExpressionFactoryImpl.java:198)
at 
org.apache.ignite.internal.processors.query.calcite.exec.LogicalRelImplementor.visit(LogicalRelImplementor.java:195)
at 
org.apache.ignite.internal.processors.query.calcite.exec.LogicalRelImplementor.visit(LogicalRelImplementor.java:102)
at 
org.apache.ignite.internal.processors.query.calcite.rel.IgniteProject.accept(IgniteProject.java:89)
at 
org.apache.ignite.internal.processors.query.calcite.exec.LogicalRelImplementor.visit(LogicalRelImplementor.java:615)
at 
org.apache.ignite.internal.processors.query.calcite.exec.LogicalRelImplementor.go(LogicalRelImplementor.java:630)
at 
org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.executeQuery(ExecutionServiceImpl.java:752)
at 

Re: [DISCUSSION] Error handling in Ignite 3

2021-04-21 Thread Andrey Mashenkov
Alexey,

As I understand, you suggest ErrorScope enum for easier analysis and it
will be a part of a error code.
But what about String representation?

I think we should use a common prefix for error codes in error messages.
Such codes will be more searchable and as a bonus, vendor-specific.
String representation might look like IGN-001042 or IGN-TBL042.




On Wed, Apr 21, 2021 at 11:43 AM Alexei Scherbakov <
alexey.scherbak...@gmail.com> wrote:

> I've create the ticket for implementing this [1]
>
> [1] https://issues.apache.org/jira/browse/IGNITE-14611
>
> пт, 16 апр. 2021 г. в 16:30, Alexei Scherbakov <
> alexey.scherbak...@gmail.com
> >:
>
> >
> >
> > чт, 15 апр. 2021 г. в 18:21, Andrey Mashenkov <
> andrey.mashen...@gmail.com
> > >:
> >
> >> Hi Alexey,
> >> I like the idea.
> >>
> >> 1.
> >>
> >> >   TBL-0001 is a *string representation* of the error. It is built
> from 2
> >> > byte scope id (mapped to name TBL) and 2 byte number (0001). Both
> >> > internally packed in int. No any kind of parsing will be necessary to
> >> read
> >> > scope/category.
> >>
> >> I think Alexey mean if it will be possible to make smth like that
> >>
> >> catch (IgniteException e) {
> >> if (e.getScope() == "TBL" && e.getCode() == 1234)
> >> continue; // E.g. retry my TX
> >> }
> >>
> >> It looks useful to me.
> >>
> >
> > I have in mind something like this:
> >
> > public class IgniteException extends RuntimeException {
> > private int errorCode;
> >
> > public IgniteException(ErrorScope scope, int code, String message,
> > Throwable cause) {
> > super(message, cause);
> > this.errorCode = make(scope, code);
> > }
> >
> > public boolean matches(ErrorScope scope, int code) {
> > return errorCode == make(scope, code);
> > }
> >
> > private int make(ErrorScope scope, int code) {
> > return ((scope.ordinal() << 16) | code);
> > }
> >
> > public ErrorScope scope() {
> > return ErrorScope.values()[errorCode >> 16];
> > }
> >
> > public int code() {
> > return 0x & errorCode;
> > }
> >
> > public static void main(String[] args) {
> > IgniteException e = new IgniteException(ErrorScope.RAFT, 1,
> > "test", null);
> >
> > System.out.println(e.matches(ErrorScope.RAFT, 2));
> > System.out.println(e.scope());
> > System.out.println(e.code());
> >
> > try {
> > throw e;
> > }
> > catch (IgniteException ee) {
> > System.out.println(ee.matches(ErrorScope.RAFT, 1));
> > }
> > }
> > }
> >
> >
> >>
> >> 2. How you see a cross-module exception throwing?
> >> Assume, user call -> module A, which recursively call -> module B, which
> >> fails.
> >> So, module A component calls a module B component and got an Exception
> >> with
> >> "B-1234" exception.
> >> Module A do not expect any exception here and doesn't take care of
> "B-xxx"
> >> error codes, but only "A-.
> >> Should it rethrow exception with "A-unknown" (maybe "UNK-0001") code
> >> or reuse "B-" code with the own message, pointing original exception
> >> as
> >> a cause for both cases?
> >>
> >> The first approach may looks confusing, while the second one produces
> too
> >> many "UNK-" codes.
> >> What code should get the user in that case?
> >>
> >>
> >>
> > The method should always report root cause, in your example it will be
> > B-, no matter which module API is called.
> >
> >
> >>
> >>
> >>
> >> On Thu, Apr 15, 2021 at 5:36 PM Alexei Scherbakov <
> >> alexey.scherbak...@gmail.com> wrote:
> >>
> >> > чт, 15 апр. 2021 г. в 14:26, Ilya Kasnacheev <
> ilya.kasnach...@gmail.com
> >> >:
> >> >
> >> > > Hello!
> >> > >
> >> > > > All public methods throw only unchecked
> >> > > org.apache.ignite.lang.IgniteException containing aforementioned
> >> fields.
> >> > > > Each public method must have a section in the javadoc with a list
> of
> >> > all
> >> > > possible error codes for this method.
> >> > >
> >> > > I don't think this is feasible at all.
> >> > > Imagine javadoc for cache.get() method featuring three pages of
> >> possible
> >> > > error codes thrown by this method.
> >> > >
> >> >
> >> > Of course there is no need to write 3 pages of error codes, this makes
> >> no
> >> > sense.
> >> > I think we can use error ranges here, or, probably, document most
> >> important
> >> > error scenarios.
> >> > The point here is to try to document errors as much as possible.
> >> >
> >> >
> >> > > Also, updated every two weeks to account for changes in
> >> > > underlying mechanisms.
> >> > >
> >> > > There is still a confusion between "error code for any error in
> logs"
> >> and
> >> > > "error code for any pair of method & exception". Which one are we
> >> > > discussing really?
> >> > >
> >> > > This is impossible to track or test, but is susceptible for common
> >> > > error-hiding antipattern where all exceptions are caught in
> >> cache.get()
> >> > and
> >> > > 

Re: [VOTE][EXTENSION] Release Apache Ignite performance-statistics-ext, spring-data-all-ext and spring-tx-ext extensions 1.0.0 RC1

2021-04-21 Thread Nikita Amelchev
+1 to formalize extension modules names:
ignite-{directory-name}

The release script has this issue too. It will work fine with that name.

ср, 21 апр. 2021 г. в 10:37, Petr Ivanov :
>
> I checked the modules and there is misnaming issue which I think is critical 
> to test integration automation on TC.
> Can we change maven module names sping-data-2.x-ext to align with directory 
> name? Currently there is underscore in maven module name, which is hyphen in 
> directory name.
>
>
> > On 21 Apr 2021, at 10:22, Nikita Amelchev  wrote:
> >
> > +1 to postpone the spring-tx-ext extension release.
> >
> > So, the following extensions will be released now:
> >
> > spring-data-ext
> > spring-data-2.0-ext
> > spring-data-2.2-ext
> > spring-data-commons
> > performance-statistics-ext
> >
> > вт, 20 апр. 2021 г. в 14:49, Mikhail Petrov :
> >>
> >> Igniters,
> >>
> >> Changing the scope of Spring dependencies to "provided" in Ignite Spring
> >> extensions does not currently work as expected:
> >> some versions of Spring that a user can specify via maven configuration
> >> for Spring extensions may conflict with the hard-coded version of Spring
> >> dependencies that the ignite-spring module relies on.
> >>
> >> The issue mentioned above affects the Ignite Spring Transactions
> >> integration. And since this integration is included in the Ignite 2.10
> >> release, I suggest postponing the release of the Ignite Spring
> >> Transactions integration until the above issue is properly fixed.
> >>
> >> Any objections?
> >>
> >> On 16.04.2021 09:15, Ivan Daschinsky wrote:
> >>> -1 From me. There is an absence of NOTICE and LICENSE files in binary
> >>> package. Also, there is no source package. These is a violation of apache
> >>> release policy [1]
> >>> [1] https://www.apache.org/legal/release-policy.html
> >>>
> >>> чт, 15 апр. 2021 г. в 23:23, Nikita Amelchev :
> >>>
>  According to ASF release policy [1] non-PMC committers can sign 
>  artifacts:
> 
> > all artifacts placed in their directory must be signed by a committer,
>  preferably by a PMC member.
> 
>  [1] https://www.apache.org/legal/release-policy.html
> 
>  чт, 15 апр. 2021 г. в 23:05, Dmitriy Pavlov :
> > My best guess that since PMCs have a binding vote in voting in release, 
> > a
> > PMC member should sign binaries as well. And I suppose that in the past
> > only PMC members were signing the release.
> >
> > Meanwhile, https://infra.apache.org/release-signing.html does not
>  contain
> > any mention of PMC role and only committers are mentioned there. It
> > might be not an answer, since a lot of projects have PMC=Committer and
>  just
> > one election.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > чт, 15 апр. 2021 г. в 22:05, Ivan Daschinsky :
> >
> >> I'm sorry, but is it OK, that artifacts are signed with signature of
> >> non-PMC?
> >>
> >> чт, 15 апр. 2021 г. в 19:26, Nikita Amelchev :
> >>
> >>> Dear Ignite Community,
> >>>
> >>> I have uploaded a release candidate of the following extension
>  modules:
> >>> performance-statistics-ext
> >>> spring-data-ext
> >>> spring-data-2.0-ext
> >>> spring-data-2.2-ext
> >>> spring-data-commons
> >>> spring-tx-ext
> >>>
> >>> The release candidate of the performance-statistics-ext extension:
> >>>
> >>>
>  https://dist.apache.org/repos/dist/dev/ignite/ignite-performance-statistics-ext/1.0.0-rc1/
> >>> The following staging can be used for testing:
> >>>
>  https://repository.apache.org/content/repositories/orgapacheignite-1509
> >>> Tags were created:
> >>>
> >>> ignite-performance-statistics-ext-1.0.0-rc1
> >>>
> >>>
>  https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=b992d9758278c38e93b375b08e4052954744a436
> >>> ignite-spring-data-ext-1.0.0-rc1
> >>> ignite-spring-data-2.2-ext-1.0.0-rc1
> >>> ignite-spring-data-2.0-ext-1.0.0-rc1
> >>> ignite-spring-data-commons-1.0.0-rc1
> >>> ignite-spring-data-all-ext-1.0.0-rc1
> >>>
> >>>
>  https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=277ca95f6472d8bd46d906e70ba1a312d36dadb0
> >>> ignite-spring-tx-ext-1.0.0-rc1
> >>>
> >>>
>  https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=55a3ae9e011ba48796847a33481842f154f0febb
> >>> RELEASE NOTES:
> >>>
> >>>
>  https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=RELEASE_NOTES.txt;h=22f8d665c194ba2dfa6167354d97ad53d5ef206f;hb=c8de80ee14d1fb76d6cbb0b18513bb70b499c3cb
> >>> DOCUMENTATION:
> >>> Documentation of listed extensions was updated in the following
>  issues:
> >>>
>  https://issues.apache.org/jira/issues/?filter=-1=key%20in%20(IGNITE-14417%2CIGNITE-14398%2CIGNITE-14397%2CIGNITE-14493)
> >>> The vote is formal, see voting guidelines
> 

Re: [DISCUSSION] Error handling in Ignite 3

2021-04-21 Thread Alexei Scherbakov
I've create the ticket for implementing this [1]

[1] https://issues.apache.org/jira/browse/IGNITE-14611

пт, 16 апр. 2021 г. в 16:30, Alexei Scherbakov :

>
>
> чт, 15 апр. 2021 г. в 18:21, Andrey Mashenkov  >:
>
>> Hi Alexey,
>> I like the idea.
>>
>> 1.
>>
>> >   TBL-0001 is a *string representation* of the error. It is built from 2
>> > byte scope id (mapped to name TBL) and 2 byte number (0001). Both
>> > internally packed in int. No any kind of parsing will be necessary to
>> read
>> > scope/category.
>>
>> I think Alexey mean if it will be possible to make smth like that
>>
>> catch (IgniteException e) {
>> if (e.getScope() == "TBL" && e.getCode() == 1234)
>> continue; // E.g. retry my TX
>> }
>>
>> It looks useful to me.
>>
>
> I have in mind something like this:
>
> public class IgniteException extends RuntimeException {
> private int errorCode;
>
> public IgniteException(ErrorScope scope, int code, String message,
> Throwable cause) {
> super(message, cause);
> this.errorCode = make(scope, code);
> }
>
> public boolean matches(ErrorScope scope, int code) {
> return errorCode == make(scope, code);
> }
>
> private int make(ErrorScope scope, int code) {
> return ((scope.ordinal() << 16) | code);
> }
>
> public ErrorScope scope() {
> return ErrorScope.values()[errorCode >> 16];
> }
>
> public int code() {
> return 0x & errorCode;
> }
>
> public static void main(String[] args) {
> IgniteException e = new IgniteException(ErrorScope.RAFT, 1,
> "test", null);
>
> System.out.println(e.matches(ErrorScope.RAFT, 2));
> System.out.println(e.scope());
> System.out.println(e.code());
>
> try {
> throw e;
> }
> catch (IgniteException ee) {
> System.out.println(ee.matches(ErrorScope.RAFT, 1));
> }
> }
> }
>
>
>>
>> 2. How you see a cross-module exception throwing?
>> Assume, user call -> module A, which recursively call -> module B, which
>> fails.
>> So, module A component calls a module B component and got an Exception
>> with
>> "B-1234" exception.
>> Module A do not expect any exception here and doesn't take care of "B-xxx"
>> error codes, but only "A-.
>> Should it rethrow exception with "A-unknown" (maybe "UNK-0001") code
>> or reuse "B-" code with the own message, pointing original exception
>> as
>> a cause for both cases?
>>
>> The first approach may looks confusing, while the second one produces too
>> many "UNK-" codes.
>> What code should get the user in that case?
>>
>>
>>
> The method should always report root cause, in your example it will be
> B-, no matter which module API is called.
>
>
>>
>>
>>
>> On Thu, Apr 15, 2021 at 5:36 PM Alexei Scherbakov <
>> alexey.scherbak...@gmail.com> wrote:
>>
>> > чт, 15 апр. 2021 г. в 14:26, Ilya Kasnacheev > >:
>> >
>> > > Hello!
>> > >
>> > > > All public methods throw only unchecked
>> > > org.apache.ignite.lang.IgniteException containing aforementioned
>> fields.
>> > > > Each public method must have a section in the javadoc with a list of
>> > all
>> > > possible error codes for this method.
>> > >
>> > > I don't think this is feasible at all.
>> > > Imagine javadoc for cache.get() method featuring three pages of
>> possible
>> > > error codes thrown by this method.
>> > >
>> >
>> > Of course there is no need to write 3 pages of error codes, this makes
>> no
>> > sense.
>> > I think we can use error ranges here, or, probably, document most
>> important
>> > error scenarios.
>> > The point here is to try to document errors as much as possible.
>> >
>> >
>> > > Also, updated every two weeks to account for changes in
>> > > underlying mechanisms.
>> > >
>> > > There is still a confusion between "error code for any error in logs"
>> and
>> > > "error code for any pair of method & exception". Which one are we
>> > > discussing really?
>> > >
>> > > This is impossible to track or test, but is susceptible for common
>> > > error-hiding antipattern where all exceptions are caught in
>> cache.get()
>> > and
>> > > discarded, and instead a brand new CH-0001 "error in cache.get()" is
>> > thrown
>> > > to the user.
>> > >
>> > > Which is certainly not something that anybody wants.
>> > >
>> >
>> > Certainly not. We are talking here about root cause - what is exactly
>> the
>> > reason for method call failure.
>> >
>> >
>> > >
>> > > Regards,
>> > > --
>> > > Ilya Kasnacheev
>> > >
>> > >
>> > > чт, 15 апр. 2021 г. в 13:03, Vladislav Pyatkov > >:
>> > >
>> > > > Hi Alexei,
>> > > >
>> > > > > Each public method *must *have a section in the javadoc with a
>> list
>> > of
>> > > > all possible error codes for this method.
>> > > >
>> > > > I consider it is redundant, because any public exception can be
>> thrown
>> > > from
>> > > > public API.
>> > > > If it not happens today, it may change tomorrow: one will be
>> removed,
>> > > > another one will be 

[jira] [Created] (IGNITE-14611) Implement error handling for public API based on error codes

2021-04-21 Thread Alexey Scherbakov (Jira)
Alexey Scherbakov created IGNITE-14611:
--

 Summary: Implement error handling for public API based on error 
codes
 Key: IGNITE-14611
 URL: https://issues.apache.org/jira/browse/IGNITE-14611
 Project: Ignite
  Issue Type: Task
Reporter: Alexey Scherbakov
 Fix For: 3.0


Dev list discusstion [1]

[1] 
http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Error-handling-in-Ignite-3-td52269.html



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-14610) BinaryBuilderReader doesn't supports reference (HANDLE) to collection

2021-04-21 Thread Taras Ledkov (Jira)
Taras Ledkov created IGNITE-14610:
-

 Summary: BinaryBuilderReader doesn't supports reference (HANDLE) 
to collection
 Key: IGNITE-14610
 URL: https://issues.apache.org/jira/browse/IGNITE-14610
 Project: Ignite
  Issue Type: Bug
  Components: sql
Affects Versions: 2.10
 Environment: Steps to reproduce:
1. Create object with two objects collection fields, e.g. List
2. Assign the reference of the one field to the second.
3. Serialize object to BinaryObject
4. Create BinaryObjectBuiler from the object.
5. Add/change any field except collections.
6. Try to build new BinaryObject form the builer

Exception is thrown:
{code:java}
class org.apache.ignite.binary.BinaryObjectException: Unsupported protocol 
version: 2
at 
org.apache.ignite.internal.binary.BinaryUtils.checkProtocolVersion(BinaryUtils.java:795)
at 
org.apache.ignite.internal.binary.builder.BinaryObjectBuilderImpl.(BinaryObjectBuilderImpl.java:139)
at 
org.apache.ignite.internal.binary.builder.BinaryBuilderReader.parseValue(BinaryBuilderReader.java:522)
at 
org.apache.ignite.internal.binary.builder.BinaryObjectBuilderImpl.serializeTo(BinaryObjectBuilderImpl.java:286)
at 
org.apache.ignite.internal.binary.builder.BinaryBuilderSerializer.writeValue(BinaryBuilderSerializer.java:100)
at 
org.apache.ignite.internal.binary.builder.BinaryBuilderSerializer.writeValue(BinaryBuilderSerializer.java:53)
at 
org.apache.ignite.internal.binary.builder.BinaryObjectBuilderImpl.serializeTo(BinaryObjectBuilderImpl.java:293)
at 
org.apache.ignite.internal.binary.builder.BinaryBuilderSerializer.writeValue(BinaryBuilderSerializer.java:100)
at 
org.apache.ignite.internal.binary.builder.BinaryBuilderSerializer.writeValue(BinaryBuilderSerializer.java:53)
at 
org.apache.ignite.internal.binary.builder.BinaryObjectBuilderImpl.serializeTo(BinaryObjectBuilderImpl.java:293)
at 
org.apache.ignite.internal.binary.builder.BinaryObjectBuilderImpl.build(BinaryObjectBuilderImpl.java:188)
{code}

Root cause:
BinaryBuilderReader#parseValue doesn’t support handle to collection. Only 
handles to object are expected.

Reporter: Taras Ledkov
Assignee: Taras Ledkov
 Fix For: 2.11






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [VOTE][EXTENSION] Release Apache Ignite performance-statistics-ext, spring-data-all-ext and spring-tx-ext extensions 1.0.0 RC1

2021-04-21 Thread Petr Ivanov
I checked the modules and there is misnaming issue which I think is critical to 
test integration automation on TC.
Can we change maven module names sping-data-2.x-ext to align with directory 
name? Currently there is underscore in maven module name, which is hyphen in 
directory name.


> On 21 Apr 2021, at 10:22, Nikita Amelchev  wrote:
> 
> +1 to postpone the spring-tx-ext extension release.
> 
> So, the following extensions will be released now:
> 
> spring-data-ext
> spring-data-2.0-ext
> spring-data-2.2-ext
> spring-data-commons
> performance-statistics-ext
> 
> вт, 20 апр. 2021 г. в 14:49, Mikhail Petrov :
>> 
>> Igniters,
>> 
>> Changing the scope of Spring dependencies to "provided" in Ignite Spring
>> extensions does not currently work as expected:
>> some versions of Spring that a user can specify via maven configuration
>> for Spring extensions may conflict with the hard-coded version of Spring
>> dependencies that the ignite-spring module relies on.
>> 
>> The issue mentioned above affects the Ignite Spring Transactions
>> integration. And since this integration is included in the Ignite 2.10
>> release, I suggest postponing the release of the Ignite Spring
>> Transactions integration until the above issue is properly fixed.
>> 
>> Any objections?
>> 
>> On 16.04.2021 09:15, Ivan Daschinsky wrote:
>>> -1 From me. There is an absence of NOTICE and LICENSE files in binary
>>> package. Also, there is no source package. These is a violation of apache
>>> release policy [1]
>>> [1] https://www.apache.org/legal/release-policy.html
>>> 
>>> чт, 15 апр. 2021 г. в 23:23, Nikita Amelchev :
>>> 
 According to ASF release policy [1] non-PMC committers can sign artifacts:
 
> all artifacts placed in their directory must be signed by a committer,
 preferably by a PMC member.
 
 [1] https://www.apache.org/legal/release-policy.html
 
 чт, 15 апр. 2021 г. в 23:05, Dmitriy Pavlov :
> My best guess that since PMCs have a binding vote in voting in release, a
> PMC member should sign binaries as well. And I suppose that in the past
> only PMC members were signing the release.
> 
> Meanwhile, https://infra.apache.org/release-signing.html does not
 contain
> any mention of PMC role and only committers are mentioned there. It
> might be not an answer, since a lot of projects have PMC=Committer and
 just
> one election.
> 
> Sincerely,
> Dmitriy Pavlov
> 
> чт, 15 апр. 2021 г. в 22:05, Ivan Daschinsky :
> 
>> I'm sorry, but is it OK, that artifacts are signed with signature of
>> non-PMC?
>> 
>> чт, 15 апр. 2021 г. в 19:26, Nikita Amelchev :
>> 
>>> Dear Ignite Community,
>>> 
>>> I have uploaded a release candidate of the following extension
 modules:
>>> performance-statistics-ext
>>> spring-data-ext
>>> spring-data-2.0-ext
>>> spring-data-2.2-ext
>>> spring-data-commons
>>> spring-tx-ext
>>> 
>>> The release candidate of the performance-statistics-ext extension:
>>> 
>>> 
 https://dist.apache.org/repos/dist/dev/ignite/ignite-performance-statistics-ext/1.0.0-rc1/
>>> The following staging can be used for testing:
>>> 
 https://repository.apache.org/content/repositories/orgapacheignite-1509
>>> Tags were created:
>>> 
>>> ignite-performance-statistics-ext-1.0.0-rc1
>>> 
>>> 
 https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=b992d9758278c38e93b375b08e4052954744a436
>>> ignite-spring-data-ext-1.0.0-rc1
>>> ignite-spring-data-2.2-ext-1.0.0-rc1
>>> ignite-spring-data-2.0-ext-1.0.0-rc1
>>> ignite-spring-data-commons-1.0.0-rc1
>>> ignite-spring-data-all-ext-1.0.0-rc1
>>> 
>>> 
 https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=277ca95f6472d8bd46d906e70ba1a312d36dadb0
>>> ignite-spring-tx-ext-1.0.0-rc1
>>> 
>>> 
 https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=55a3ae9e011ba48796847a33481842f154f0febb
>>> RELEASE NOTES:
>>> 
>>> 
 https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=RELEASE_NOTES.txt;h=22f8d665c194ba2dfa6167354d97ad53d5ef206f;hb=c8de80ee14d1fb76d6cbb0b18513bb70b499c3cb
>>> DOCUMENTATION:
>>> Documentation of listed extensions was updated in the following
 issues:
>>> 
 https://issues.apache.org/jira/issues/?filter=-1=key%20in%20(IGNITE-14417%2CIGNITE-14398%2CIGNITE-14397%2CIGNITE-14493)
>>> The vote is formal, see voting guidelines
>>> https://www.apache.org/foundation/voting.html
>>> 
>>> +1 - to accept all the Apache Ignite performance-statistics-ext,
>>> spring-data-all-ext and spring-tx-ext extensions 1.0.0-rc1 listed in
>>> the thread
>>> 0 - don't care either way
>>> -1 - DO NOT accept either of the Apache Ignite
>>> performance-statistics-ext, spring-data-all-ext and spring-tx-ext
>>> 

Re: [VOTE][EXTENSION] Release Apache Ignite performance-statistics-ext, spring-data-all-ext and spring-tx-ext extensions 1.0.0 RC1

2021-04-21 Thread Nikita Amelchev
+1 to postpone the spring-tx-ext extension release.

So, the following extensions will be released now:

spring-data-ext
spring-data-2.0-ext
spring-data-2.2-ext
spring-data-commons
performance-statistics-ext

вт, 20 апр. 2021 г. в 14:49, Mikhail Petrov :
>
> Igniters,
>
> Changing the scope of Spring dependencies to "provided" in Ignite Spring
> extensions does not currently work as expected:
> some versions of Spring that a user can specify via maven configuration
> for Spring extensions may conflict with the hard-coded version of Spring
> dependencies that the ignite-spring module relies on.
>
> The issue mentioned above affects the Ignite Spring Transactions
> integration. And since this integration is included in the Ignite 2.10
> release, I suggest postponing the release of the Ignite Spring
> Transactions integration until the above issue is properly fixed.
>
> Any objections?
>
> On 16.04.2021 09:15, Ivan Daschinsky wrote:
> > -1 From me. There is an absence of NOTICE and LICENSE files in binary
> > package. Also, there is no source package. These is a violation of apache
> > release policy [1]
> > [1] https://www.apache.org/legal/release-policy.html
> >
> > чт, 15 апр. 2021 г. в 23:23, Nikita Amelchev :
> >
> >> According to ASF release policy [1] non-PMC committers can sign artifacts:
> >>
> >>> all artifacts placed in their directory must be signed by a committer,
> >> preferably by a PMC member.
> >>
> >> [1] https://www.apache.org/legal/release-policy.html
> >>
> >> чт, 15 апр. 2021 г. в 23:05, Dmitriy Pavlov :
> >>> My best guess that since PMCs have a binding vote in voting in release, a
> >>> PMC member should sign binaries as well. And I suppose that in the past
> >>> only PMC members were signing the release.
> >>>
> >>> Meanwhile, https://infra.apache.org/release-signing.html does not
> >> contain
> >>> any mention of PMC role and only committers are mentioned there. It
> >>> might be not an answer, since a lot of projects have PMC=Committer and
> >> just
> >>> one election.
> >>>
> >>> Sincerely,
> >>> Dmitriy Pavlov
> >>>
> >>> чт, 15 апр. 2021 г. в 22:05, Ivan Daschinsky :
> >>>
>  I'm sorry, but is it OK, that artifacts are signed with signature of
>  non-PMC?
> 
>  чт, 15 апр. 2021 г. в 19:26, Nikita Amelchev :
> 
> > Dear Ignite Community,
> >
> > I have uploaded a release candidate of the following extension
> >> modules:
> > performance-statistics-ext
> > spring-data-ext
> > spring-data-2.0-ext
> > spring-data-2.2-ext
> > spring-data-commons
> > spring-tx-ext
> >
> > The release candidate of the performance-statistics-ext extension:
> >
> >
> >> https://dist.apache.org/repos/dist/dev/ignite/ignite-performance-statistics-ext/1.0.0-rc1/
> > The following staging can be used for testing:
> >
> >> https://repository.apache.org/content/repositories/orgapacheignite-1509
> > Tags were created:
> >
> > ignite-performance-statistics-ext-1.0.0-rc1
> >
> >
> >> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=b992d9758278c38e93b375b08e4052954744a436
> > ignite-spring-data-ext-1.0.0-rc1
> > ignite-spring-data-2.2-ext-1.0.0-rc1
> > ignite-spring-data-2.0-ext-1.0.0-rc1
> > ignite-spring-data-commons-1.0.0-rc1
> > ignite-spring-data-all-ext-1.0.0-rc1
> >
> >
> >> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=277ca95f6472d8bd46d906e70ba1a312d36dadb0
> > ignite-spring-tx-ext-1.0.0-rc1
> >
> >
> >> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=55a3ae9e011ba48796847a33481842f154f0febb
> > RELEASE NOTES:
> >
> >
> >> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=RELEASE_NOTES.txt;h=22f8d665c194ba2dfa6167354d97ad53d5ef206f;hb=c8de80ee14d1fb76d6cbb0b18513bb70b499c3cb
> > DOCUMENTATION:
> > Documentation of listed extensions was updated in the following
> >> issues:
> >
> >> https://issues.apache.org/jira/issues/?filter=-1=key%20in%20(IGNITE-14417%2CIGNITE-14398%2CIGNITE-14397%2CIGNITE-14493)
> > The vote is formal, see voting guidelines
> > https://www.apache.org/foundation/voting.html
> >
> > +1 - to accept all the Apache Ignite performance-statistics-ext,
> > spring-data-all-ext and spring-tx-ext extensions 1.0.0-rc1 listed in
> > the thread
> > 0 - don't care either way
> > -1 - DO NOT accept either of the Apache Ignite
> > performance-statistics-ext, spring-data-all-ext and spring-tx-ext
> > extensions 1.0.0-rc1 (explain why)
> >
> > The vote will hold for 72 hours and will end on April 18 2021 17:00
> >> UTC
> >
> >> https://www.timeanddate.com/countdown/generic?iso=20210418T17=1440=cursive
> > --
> > Best wishes,
> > Amelchev Nikita
> >
> 
>  --
>  Sincerely yours, Ivan Daschinskiy
> 
> >>
> >>
> >> --
> >> Best wishes,
> >> Amelchev Nikita
> >>
> >



-- 

[jira] [Created] (IGNITE-14609) Document old and new async continuation behavior

2021-04-21 Thread Pavel Tupitsyn (Jira)
Pavel Tupitsyn created IGNITE-14609:
---

 Summary: Document old and new async continuation behavior
 Key: IGNITE-14609
 URL: https://issues.apache.org/jira/browse/IGNITE-14609
 Project: Ignite
  Issue Type: Improvement
  Components: documentation
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 2.11


IGNITE-12033 changed the default async behavior for cache operations in Java 
and .NET, plus Compute operations in .NET.
Document old and new async continuation behavior for Java and .NET.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)