[jira] [Created] (IGNITE-11512) Add counter left partition for index rebuild in CacheGroupMetricsMXBean
Alexand Polyakov created IGNITE-11512: - Summary: Add counter left partition for index rebuild in CacheGroupMetricsMXBean Key: IGNITE-11512 URL: https://issues.apache.org/jira/browse/IGNITE-11512 Project: Ignite Issue Type: Improvement Components: general Affects Versions: 2.7 Reporter: Alexand Polyakov Assignee: Alexand Polyakov Now, if ran rebuild indexes, this can determined only on load CPU and thread dump. The presence of the "how many partitions left to index" metric will help determine whether the rebuilding is going on and on which cache, as well as the percentage of completion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11511) SQL: Possible bug with parameters passing for complex DML queries
Vladimir Ozerov created IGNITE-11511:
Summary: SQL: Possible bug with parameters passing for complex DML
queries
Key: IGNITE-11511
URL: https://issues.apache.org/jira/browse/IGNITE-11511
Project: Ignite
Issue Type: Bug
Components: sql
Reporter: Vladimir Ozerov
Assignee: Pavel Kuznetsov
Fix For: 2.8
See methods {{IgniteH2Indexing.executeSelectLocal}} and
{{IgniteH2Indexing.executeSelectForDml}}. They both could be invoked for
{{SELECT}} statements extracted from DML.
But notice how parameters are passed: it seems that we may pass parameters from
DML statement unchanged, which is illegal. E.g. consider the following DML:
{code}
UPDATE table SET x=? WHERE x=?
{code}
In this case SELECT statement should get only the second parameter.
Need to create tests to confirm that this is the case and make necessary fixes
if needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-11510) SQL: Rework running queries tests to make them stable to internal code changes
Vladimir Ozerov created IGNITE-11510:
Summary: SQL: Rework running queries tests to make them stable to
internal code changes
Key: IGNITE-11510
URL: https://issues.apache.org/jira/browse/IGNITE-11510
Project: Ignite
Issue Type: Task
Components: sql
Reporter: Vladimir Ozerov
See {{RunningQueriesTest}}. It hacks into {{IgniteH2Indexing.querySqlFields}}.
This is not resilent to internal code changes. We need to make sure that the
whole test use as less hacks as possible. E.g. we can hack into running queries
manager instead of indexing.
Several DML tests are muted due to changes introduced in IGNITE-11227.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Ignite 2.8 Release: Time & Scope & Release manager
Dear Denis, I agree, that mentioned fixes should be accessible for developers, maybe we could release them as an emergency release 2.7.xxx or another minor version of 2.7, nor 2.8. And 2.8 could be planned in next 1-2 months with a lot of cool features пт, 8 марта 2019 г., 13:29 Alexey Zinoviev zaleslaw@gmail.com: > I suggest to make a release from master but at 15 April-1 May to prepare > all release features. It's bad practice to announce release and cut master > in one week. And in future put the approximate date of release for feature > planning. > > In ML we are in progress of changing API and it takes 1 month to stabilize > that and fix and we hope to deliver that functionality in 2.8. > > чт, 7 марта 2019 г., 21:03 Denis Magda dma...@apache.org: > >> Dmitriy, >> >> Please find a copy-paste from the first conversation when impactful >> usability problems were reported more than a month ago: >> >> *I played with the latest Oracle JDK 11 on Mac OS Mojave. Results are >> sad:* >> >>- *Starting a node from cmd (ignite.sh) - FAILED* >>- *Opening Ignite examples - BAD EXPERIENCE* >> - *pom.xml wasn't detected automatically, had to select it manually* >> - *was hard to start the code samples (same issue as with cmd). As a >> committer, I know how to fix it >> ( >> https://apacheignite.readme.io/docs/getting-started#section-running-ignite-with-java-9-10-11 >> < >> https://apacheignite.readme.io/docs/getting-started#section-running-ignite-with-java-9-10-11 >> >), >> but most of the developers have no glue and will give up* >> - *The step above have to be repeated for every single sample* >> >> Now, imagine that dozens of users new to Ignite go through this. Most of >> them will quit after the failures above and switch to an alternate >> solution >> - there are many choices depending on a use case. >> >> Give me a call if it still doesn't sound convincing to you. What I would >> do, considering Vladimirs's feedback, if the master is really in a bad >> shape then I would release 2.8 from 2.7 and 2.8.1, 2.8.2, etc. will be >> released from the master. >> >> - >> Denis >> >> >> On Thu, Mar 7, 2019 at 9:52 AM Dmitriy Pavlov wrote: >> >> > Denis, there is not so much difference in Java 9 vs Java 11, so previous >> > Java 9-efforts done by Igniters should be applicable for 11. >> > >> > So I don't understand why we can go through the normal release process >> and >> > pilot minor releases afterward. Please share a particular case when the >> > absence of `emergency 2.8` is a problem for the user. >> > >> > Is it still our rush and 'highway or no way'? I was in the hope it is >> gone. >> > >> > чт, 7 мар. 2019 г. в 20:43, Denis Magda : >> > >> > > Vova, >> > > >> > > Thanks for the inputs. If it takes weeks to stabilize the master then >> > let's >> > > release from 2.7 cherry-picking Java 11 improvements. We can't wait >> for >> > > months holding these improvements - the world is switching to Java 11 >> and >> > > Ignite fails during the first runs presently. >> > > >> > > - >> > > Denis >> > > >> > > >> > > On Thu, Mar 7, 2019 at 9:28 AM Vladimir Ozerov >> > > wrote: >> > > >> > > > Igniters, >> > > > >> > > > Making release from master is not an option. We have a lot of >> > > not-yet-ready >> > > > and not-yet-tested features. From SQL side this is partition pruning >> > and >> > > > SQL views with KILL command. >> > > > >> > > > So if we do not want to release a mess, then there are only two >> > options: >> > > > release Java 11 fixes on top of 2.7, or make normal release in about >> > > 1.5-2 >> > > > month with proper feature freeze process and testing. >> > > > >> > > > Vladimir. >> > > > >> > > > чт, 7 марта 2019 г. в 20:10, Ilya Kasnacheev < >> > ilya.kasnach...@gmail.com >> > > >: >> > > > >> > > > > Hello! >> > > > > >> > > > > Then please fast-forward review and merge >> > > > > https://issues.apache.org/jira/browse/IGNITE-11299 because it >> breaks >> > > SSL >> > > > > on >> > > > > Windows under Java 11. >> > > > > >> > > > > Anything else that needs to be merged before release is branched? >> > > > > >> > > > > Regards, >> > > > > -- >> > > > > Ilya Kasnacheev >> > > > > >> > > > > >> > > > > чт, 7 мар. 2019 г. в 20:07, Nikolay Izhikov > >: >> > > > > >> > > > > > +1 >> > > > > > >> > > > > > чт, 7 марта 2019 г., 20:00 Denis Magda : >> > > > > > >> > > > > > > Igniters, >> > > > > > > >> > > > > > > How about releasing Ignite 2.8 from the master - creating the >> > > release >> > > > > > > branch on Monday-Tuesday, as fast as we can? Don't want us to >> > delay >> > > > > with >> > > > > > > Java 11 improvements, they are really helpful from the >> usability >> > > > > > > standpoint. >> > > > > > > >> > > > > > > After this release, let's introduce a practice of maintenance >> > > > releases >> > > > > > > 2.8.x. Those who are working on any improvements and won't >> merge >> > > them >> > > > > to >> > > > > > > the release branch on
Re: Ignite 2.8 Release: Time & Scope & Release manager
I suggest to make a release from master but at 15 April-1 May to prepare all release features. It's bad practice to announce release and cut master in one week. And in future put the approximate date of release for feature planning. In ML we are in progress of changing API and it takes 1 month to stabilize that and fix and we hope to deliver that functionality in 2.8. чт, 7 марта 2019 г., 21:03 Denis Magda dma...@apache.org: > Dmitriy, > > Please find a copy-paste from the first conversation when impactful > usability problems were reported more than a month ago: > > *I played with the latest Oracle JDK 11 on Mac OS Mojave. Results are sad:* > >- *Starting a node from cmd (ignite.sh) - FAILED* >- *Opening Ignite examples - BAD EXPERIENCE* > - *pom.xml wasn't detected automatically, had to select it manually* > - *was hard to start the code samples (same issue as with cmd). As a > committer, I know how to fix it > ( > https://apacheignite.readme.io/docs/getting-started#section-running-ignite-with-java-9-10-11 > < > https://apacheignite.readme.io/docs/getting-started#section-running-ignite-with-java-9-10-11 > >), > but most of the developers have no glue and will give up* > - *The step above have to be repeated for every single sample* > > Now, imagine that dozens of users new to Ignite go through this. Most of > them will quit after the failures above and switch to an alternate solution > - there are many choices depending on a use case. > > Give me a call if it still doesn't sound convincing to you. What I would > do, considering Vladimirs's feedback, if the master is really in a bad > shape then I would release 2.8 from 2.7 and 2.8.1, 2.8.2, etc. will be > released from the master. > > - > Denis > > > On Thu, Mar 7, 2019 at 9:52 AM Dmitriy Pavlov wrote: > > > Denis, there is not so much difference in Java 9 vs Java 11, so previous > > Java 9-efforts done by Igniters should be applicable for 11. > > > > So I don't understand why we can go through the normal release process > and > > pilot minor releases afterward. Please share a particular case when the > > absence of `emergency 2.8` is a problem for the user. > > > > Is it still our rush and 'highway or no way'? I was in the hope it is > gone. > > > > чт, 7 мар. 2019 г. в 20:43, Denis Magda : > > > > > Vova, > > > > > > Thanks for the inputs. If it takes weeks to stabilize the master then > > let's > > > release from 2.7 cherry-picking Java 11 improvements. We can't wait for > > > months holding these improvements - the world is switching to Java 11 > and > > > Ignite fails during the first runs presently. > > > > > > - > > > Denis > > > > > > > > > On Thu, Mar 7, 2019 at 9:28 AM Vladimir Ozerov > > > wrote: > > > > > > > Igniters, > > > > > > > > Making release from master is not an option. We have a lot of > > > not-yet-ready > > > > and not-yet-tested features. From SQL side this is partition pruning > > and > > > > SQL views with KILL command. > > > > > > > > So if we do not want to release a mess, then there are only two > > options: > > > > release Java 11 fixes on top of 2.7, or make normal release in about > > > 1.5-2 > > > > month with proper feature freeze process and testing. > > > > > > > > Vladimir. > > > > > > > > чт, 7 марта 2019 г. в 20:10, Ilya Kasnacheev < > > ilya.kasnach...@gmail.com > > > >: > > > > > > > > > Hello! > > > > > > > > > > Then please fast-forward review and merge > > > > > https://issues.apache.org/jira/browse/IGNITE-11299 because it > breaks > > > SSL > > > > > on > > > > > Windows under Java 11. > > > > > > > > > > Anything else that needs to be merged before release is branched? > > > > > > > > > > Regards, > > > > > -- > > > > > Ilya Kasnacheev > > > > > > > > > > > > > > > чт, 7 мар. 2019 г. в 20:07, Nikolay Izhikov : > > > > > > > > > > > +1 > > > > > > > > > > > > чт, 7 марта 2019 г., 20:00 Denis Magda : > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > How about releasing Ignite 2.8 from the master - creating the > > > release > > > > > > > branch on Monday-Tuesday, as fast as we can? Don't want us to > > delay > > > > > with > > > > > > > Java 11 improvements, they are really helpful from the > usability > > > > > > > standpoint. > > > > > > > > > > > > > > After this release, let's introduce a practice of maintenance > > > > releases > > > > > > > 2.8.x. Those who are working on any improvements and won't > merge > > > them > > > > > to > > > > > > > the release branch on Monday-Tuesday will be able to roll out > in > > a > > > > > point > > > > > > > release like 2.8.1 slightly later. > > > > > > > > > > > > > > - > > > > > > > Denis > > > > > > > > > > > > > > > > > > > > > On Thu, Mar 7, 2019 at 6:22 AM Dmitriy Pavlov < > > dpav...@apache.org> > > > > > > wrote: > > > > > > > > > > > > > > > Hi Ignite Developers, > > > > > > > > > > > > > > > > In the separate topic, we've touched the question of next > > release > > > > of > > > > > > >
[jira] [Created] (IGNITE-11509) Remove DistributedBaselineConfiguration and replace to methods on IgniteCluster
Dmitriy Govorukhin created IGNITE-11509: --- Summary: Remove DistributedBaselineConfiguration and replace to methods on IgniteCluster Key: IGNITE-11509 URL: https://issues.apache.org/jira/browse/IGNITE-11509 Project: Ignite Issue Type: Bug Reporter: Dmitriy Govorukhin -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Best Effort Affinity for thin clients
> maxConnectionNumber parameter What's the idea? Follow the Best Effor Affinity logic, but establish up to N connections? On Thu, Mar 7, 2019 at 1:23 PM Igor Sapego wrote: > I can propose two improvements here: > > 1. A simple one. Lets introduce maxConnectionNumber parameter > in ClientConfiguration. As it is easy to implement it may be introduced > together with the new feature to give user an additional control. > > 2. Asynchronous connection establishment. In this case startup method > of a client returns control to user once it have established at least one > connection. Other connections established in background by a separate > thread. This one is harder to implement and maybe it makes sense to add > it as a separate feature. > > Best Regards, > Igor > > > On Wed, Mar 6, 2019 at 9:43 PM Pavel Tupitsyn > wrote: > > > Hi, > > > > I'm in progress of implementing this IEP for Ignite.NET, and I have > > concerns about the following: > > > > > On thin client startup it connects to all nodes provided by client > > configuration > > > > Should we, at least, make this behavior optional? > > > > One of the benefits of thin client is quick startup/connect time and low > > resource usage. > > Adding "connect all" behavior can negate those benefits, especially on > > large clusters. > > > > Thoughts? > > > > On Thu, Feb 14, 2019 at 5:39 PM Igor Sapego wrote: > > > > > Guys, I've updated the IEP page [1] once again. > > > > > > Please, pay attention to sections Cache affinity mapping acquiring > > > (4.a, format of Cache Partitions Request) and Changes to cache > > > operations with single key (3 and 4, algorithm). > > > > > > Long story short, I've decided to add some additional data to Cache > > > Partitions Response, so that client can determine how to calculate > > > partition for a given key properly. > > > > > > [1] - > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients > > > > > > Best Regards, > > > Igor > > > > > > > > > On Mon, Feb 4, 2019 at 8:24 PM Pavel Tupitsyn > > > wrote: > > > > > > > Looks good to me. > > > > > > > > On Mon, Feb 4, 2019 at 6:30 PM Igor Sapego > wrote: > > > > > > > > > I've updated IEP page: [1] > > > > > > > > > > What do you think now? To me it looks cleaner. > > > > > > > > > > [1] - > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients > > > > > > > > > > Best Regards, > > > > > Igor > > > > > > > > > > > > > > > On Mon, Feb 4, 2019 at 4:44 PM Igor Sapego > > wrote: > > > > > > > > > > > Ok, I understand now. I'll try updating IEP according to this > > > proposal > > > > > and > > > > > > notify you guys. > > > > > > > > > > > > Best Regards, > > > > > > Igor > > > > > > > > > > > > > > > > > > On Mon, Feb 4, 2019 at 4:27 PM Vladimir Ozerov < > > voze...@gridgain.com > > > > > > > > > > wrote: > > > > > > > > > > > >> Igor, > > > > > >> > > > > > >> My idea is simply to add the list of caches with the same > > > distribution > > > > > to > > > > > >> the end of partition response. Client can use this information > to > > > > > populate > > > > > >> partition info for more caches in a single request. > > > > > >> > > > > > >> On Mon, Feb 4, 2019 at 3:06 PM Igor Sapego > > > > wrote: > > > > > >> > > > > > >> > Vladimir, > > > > > >> > > > > > > >> > So correct me if I'm wrong, what you propose is to avoid > > > mentioning > > > > > >> > of cache groups, and use instead of "cache group" term > something > > > > like > > > > > >> > "distribution"? Or do you propose some changes in protocol? If > > so, > > > > can > > > > > >> > you briefly explain, what kind of changes they are? > > > > > >> > > > > > > >> > Best Regards, > > > > > >> > Igor > > > > > >> > > > > > > >> > > > > > > >> > On Mon, Feb 4, 2019 at 1:13 PM Vladimir Ozerov < > > > > voze...@gridgain.com> > > > > > >> > wrote: > > > > > >> > > > > > > >> > > Igor, > > > > > >> > > > > > > > >> > > Yes, cache groups are public API. However, we try to avoid > new > > > > APIs > > > > > >> > > depending on them. > > > > > >> > > The main point from my side is that “similar cache group” > can > > be > > > > > >> easily > > > > > >> > > generalized to “similar distribution”. This way we avoid > cache > > > > > groups > > > > > >> on > > > > > >> > > protocol level at virtually no cost. > > > > > >> > > > > > > > >> > > Vladimir. > > > > > >> > > > > > > > >> > > пн, 4 февр. 2019 г. в 12:48, Igor Sapego < > isap...@apache.org > > >: > > > > > >> > > > > > > > >> > > > Guys, > > > > > >> > > > > > > > > >> > > > Can you explain why do we want to avoid Cache groups in > > > > protocol? > > > > > >> > > > > > > > > >> > > > If it's about simplicity of the protocol, then removing > > cache > > > > > groups > > > > > >> > will > > > > > >> > > > not help much with it - we will still need to include > > > > > >> "knownCacheIds" > > > > > >> > > > field in request and
