[GitHub] ignite pull request #5466: Ignite 1793 test mass run.
Github user xtern closed the pull request at: https://github.com/apache/ignite/pull/5466 ---
Re: [IMPORTANT] Future of Binary Objects
Sergi, I think we should not guess for users what is right or wrong for them. It is up to user to decide what is valid. For example, consider a user who operates on a list of Integers, and to optimize memory consumption he decide to save in the same field either List, or plain Integer in case only single element exists. Another example - a kind of data lake or data cleansing application, which may receive the same field in different forms. E.g. age in the form of Integer or String. Does it work for user or not? We do not know. Will he need to migrate the whole data set? We do not know either. The only place in the product where we case is SQL. But in this case instead of adding checks on binary level, we should validate data on cache level. In fact, Ignite already works this way. E.g. nullability checks are performed on cache level rather than binary. All we need is to move all checks to cache level from binary level. On Thu, Nov 22, 2018 at 9:41 AM Sergi Vladykin wrote: > It may be OK to extend compatible field types (like from Int to Long). > > In Protobuf for example this is allowed just because there is no difference > between Int and Long in binary format: they all are equally varlen encoded > and Longs just will occupy up to 9 bytes, while Ints up to 5. > > But for every other case, where binary representation is type dependent, I > would be against. This will either require to migrate the whole dataset to > a new model (which is always risky, since you may need to rollback to > previous version of your code) or it will require type checks/conversions > for each field access, which is a hard to reason complication and possible > performance penalty. > > Sergi > > > > чт, 22 нояб. 2018 г. в 09:23, Vladimir Ozerov : > > > Denis, > > > > Several examples: > > 1) DEFAULT values - in SQL you may avoid storing default value in the > table > > and store it in metadata instead. Not applicable for BinaryObject because > > the same binary object may be saved to two SQL tables with different > > defaults > > 2) DATE and other temporal types - in SQL you want to store it in special > > format to be able to extract date parts quickly (typically - 11 bytes). > But > > in Java and some other languages the best format is plain long. this is > why > > we use it BinaryObject > > 3) String charset - in SQL you may choose different charsets for > different > > tables. E.g. UTF-8 for one, ASCII for another. In BinaryObject we store > > everything in UTF-8, and this is fine for most cases, well ... except of > > SQL :-) > > > > The key thing here is that you cannot define a format which will be good > > for both SQL, and native API. They are very different. This is why I > > propose to define additional interface on cache level defining how to > store > > values, which will be very different from binary objects. > > > > Vladimir. > > > > On Thu, Nov 22, 2018 at 3:32 AM Denis Magda wrote: > > > > > Vladimir, > > > > > > Could you educate me a little bit, why the current format is bad for > SQL > > > and why another one is more suitable? > > > > > > Also, if we introduce the new format then why would we keep the binary > > one? > > > Is the new format just a next version of the binary one. > > > > > > 2.3) Remove restrictions on changing field type > > > > I do not know why we did that in the first place. This restriction > > > prevents > > > > type evolution and confuses users. > > > > > > > > > That is a hot requirement shared by those who use Ignite SQL in > > production. > > > +1. > > > > > > -- > > > Denis > > > > > > On Mon, Nov 19, 2018 at 11:05 PM Vladimir Ozerov > > > > wrote: > > > > > > > Igniters, > > > > > > > > It is very likely that Apache Ignite 3.0 will be released next year. > So > > > we > > > > need to start thinking about major product improvements. I'd like to > > > start > > > > with binary objects. > > > > > > > > Currently they are one of the main limiting factors for the product. > > They > > > > are fat - 30+ bytes overhead on average, high TCO of Apache Ignite > > > > comparing to other vendors. They are slow - not suitable for SQL at > > all. > > > > > > > > I would like to ask all of you who worked with binary objects to > share > > > your > > > > feedback and ideas, so that we understand how they should look like > in > > AI > > > > 3.0. This is a brain storm - let's accumulate ideas first and > minimize > > > > critics. Then we will work on ideas in separate topics. > > > > > > > > 1) Historical background > > > > > > > > BO were implemented around 2014 (Apache Ignite 1.5) when we started > > > working > > > > on .NET and CPP clients. During design we had several ideas in mind: > > > > - ability to read object fields in O(1) without deserialization > > > > - interoperabillty between Java, .NET and CPP. > > > > > > > > Since then a number of other concepts were mixed to the cocktail: > > > > - Affinity key fields > > > > - Strict typing for existing fields (aka metadata) > > > > - Bina
[jira] [Created] (IGNITE-10375) Fix inspection failures
Ryabov Dmitrii created IGNITE-10375: --- Summary: Fix inspection failures Key: IGNITE-10375 URL: https://issues.apache.org/jira/browse/IGNITE-10375 Project: Ignite Issue Type: Bug Reporter: Ryabov Dmitrii Assignee: Ryabov Dmitrii 4 failures in the master https://ci.ignite.apache.org/viewLog.html?buildId=2376109&buildTypeId=IgniteTests24Java8_InspectionsCore&tab=Inspection -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [MTCGA]: new failures in builds [2371886] needs to be handled
Hi Igor Seliverstov, I can see some test is failed with issue link https://issues.apache.org/jira/browse/IGNITE-10104 Could you please mute test on TC as well? Sincerely, Dmitriy Pavlov чт, 22 нояб. 2018 г. в 07:21, : > Hi Igniters, > > I've detected some new issue on TeamCity to be handled. You are more than > welcomed to help. > > If your changes can lead to this failure(s): We're grateful that you were > a volunteer to make the contribution to this project, but things change and > you may no longer be able to finalize your contribution. > Could you respond to this email and indicate if you wish to continue and > fix test failures or step down and some committer may revert you commit. > > *New test failure in master > CacheMvccPartitionedBackupsTest.testBackupsCoherenceWithLargeOperations > https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-7557959062889759280&branch=%3Cdefault%3E&tab=testDetails > > *New test failure in master > CacheMvccReplicatedBackupsTest.testBackupsCoherenceWithLargeOperations > https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=6825219049988632083&branch=%3Cdefault%3E&tab=testDetails > Changes may lead to failure were done by > - gvvinblade > https://ci.ignite.apache.org/viewModification.html?modId=839708 > - andrey.mashenkov > https://ci.ignite.apache.org/viewModification.html?modId=839694 > - pudov.max > https://ci.ignite.apache.org/viewModification.html?modId=839688 > - ivandasch > https://ci.ignite.apache.org/viewModification.html?modId=839748 > - dmitrievanthony > https://ci.ignite.apache.org/viewModification.html?modId=839684 > - jokserfn > https://ci.ignite.apache.org/viewModification.html?modId=839734 > > - Here's a reminder of what contributors were agreed to do > https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute > - Should you have any questions please contact > dev@ignite.apache.org > > Best Regards, > Apache Ignite TeamCity Bot > https://github.com/apache/ignite-teamcity-bot > Notification generated at 07:21:40 22-11-2018 >
[GitHub] ignite pull request #5469: IGNITE-10375 Fix inspection failures
GitHub user SomeFire opened a pull request: https://github.com/apache/ignite/pull/5469 IGNITE-10375 Fix inspection failures You can merge this pull request into a Git repository by running: $ git pull https://github.com/SomeFire/ignite IGNITE-10375 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5469.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5469 commit c8453b48126bbae8d8a0b2fe66f7bbf65fe4eae7 Author: Dmitrii Ryabov Date: 2018-11-22T08:12:37Z IGNITE-10375 Fix inspection failures ---
[GitHub] ignite pull request #5470: IGNITE-10358
GitHub user dmelnichuk opened a pull request: https://github.com/apache/ignite/pull/5470 IGNITE-10358 Fixes a Collection-related bug in Python thin client. You can merge this pull request into a Git repository by running: $ git pull https://github.com/nobitlost/ignite ignite-10358 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5470.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5470 commit 243d83438d078c94e05b4a0f9e7e9ea3e8e33be8 Author: Dmitry Melnichuk Date: 2018-11-22T00:12:07Z IGNITE-10358 Advance version number. Fix reading README in setup for non-unicode OSes. commit c1adb4ca555c9756215d7c43d508990738e43a37 Author: Dmitry Melnichuk Date: 2018-11-22T04:26:21Z IGNITE-10358 Make `is_hinted` more specific to avoid potential collision with Complex objects. commit 05f57fe98316d9e862e2973a15488ba10ada3adc Author: Dmitry Melnichuk Date: 2018-11-22T06:23:24Z IGNITE-10358 Fix using type hints in Collection and CollectionObject. commit 2d5c3b4d15f8cc2638d60491d195a3249291bdee Author: Dmitry Melnichuk Date: 2018-11-22T07:47:33Z IGNITE-10358 Extend tests. commit e15660914a983c7a449cdc285de53324306b0fec Author: Dmitry Melnichuk Date: 2018-11-22T08:06:19Z IGNITE-10358 Update documentation. ---
Re: New API for changing configuration of persistent caches
I don't see how you variant handles user-defined objects (factories, affinity-functions, interceptors, etc.). Could you describe? On Thu, Nov 22, 2018 at 10:47 AM Vladimir Ozerov wrote: > My variant of API avoids cache configuration. > > One more thing to note - as we found out control.sh cannot dump XML > configuration. Currently it returns only subset of properties. And in > general case it is impossible to convert CacheConfiguration to Spring XML, > because Spring XMLis not serialization protocol. So API with > CacheConfiguration doesn’t seem to work for control.sh as well. > > чт, 22 нояб. 2018 г. в 10:05, Eduard Shangareev < > eduard.shangar...@gmail.com > >: > > > Vovan, > > > > We couldn't avoid API with cache configuration. > > Almost all of ~70 properties could be changed, some of them are instances > > of objects or could be user-defined class. > > Could you come up with alternatives for user-defined affinity function? > > > > Also, the race would have a place in other scenarios. > > > > > > > > On Thu, Nov 22, 2018 at 8:50 AM Vladimir Ozerov > > wrote: > > > > > Ed, > > > > > > We may have API similar to “cache” and “getOrCreateCache”, or may not. > It > > > is up to us to decide. Similarity on it’s own is weak argument. > > > Functionality and simplicity - this is what matters. > > > > > > Approach with cache configuration has three major issues > > > 1) It exposes properties which user will not be able to change, so > > typical > > > user actions would be: try to change property, fail as it is > unsupported, > > > go reading documentation. Approach with separate POJO is intuitive and > > > self-documenting. > > > 2) It has race condition between config read and config apply, so user > do > > > not know what exactly he changes, unless you change API to something > like > > > “restartCaches(Tuple...)”, > which > > > user will need to call in a loop. > > > 3) And it is not suitable for non-Java platform, which is a > showstopper - > > > all API should be available from all platforms unless it is proven to > be > > > impossible to implement. > > > > > > Vladimir. > > > > > > чт, 22 нояб. 2018 г. в 1:06, Eduard Shangareev < > > > eduard.shangar...@gmail.com > > > >: > > > > > > > Vovan, > > > > > > > > Would you argue that we should have the similar API in Java as > > > > Ignite.cache(CacheConfiguration) or > > > > Ignite.getOrCreateCache(CacheConfiguration)? > > > > > > > > With a proposed solution, every other API call would rely on it > > finally. > > > > > > > > I am interested in having such feature not arguing about API > > > alternatives. > > > > > > > > We definitely should have the ability to change it via control.sh and > > > Java > > > > API. Everything else is optional from my point of view (at least on > the > > > > current stage). > > > > > > > > Moreover, your arguments are more about our format of > > CacheConfiguration > > > > which couldn't be defined in other languages and clients. So, maybe > we > > > > should start a discussion about how we should change it in 3.0? > > > > > > > > > > > > > > > > > > > > On Wed, Nov 21, 2018 at 7:45 PM Vladimir Ozerov < > voze...@gridgain.com> > > > > wrote: > > > > > > > > > Ed, > > > > > > > > > > Why do we want to operate on CacheConfiguration so desperately? > Your > > > > > example raises even more questions: > > > > > 1) What to do with thin clients? > > > > > 2) What to do with aforementioned race conditions, when cache could > > be > > > > > changed concurrently? > > > > > 3) Why such trivial operation from user perspective is only > supported > > > > from > > > > > control.sh and not from the rest API (even Java client nodes will > be > > > > > affected - remember our plans to remove requirement to have cache > > > classes > > > > > on client nodes, which is yet to be implemented). > > > > > > > > > > Compare it to alternative API: > > > > > > > > > > 1) Native call from any language without limitations: > > > > > > > > > > > > > > > > > > > > Ignite.changeCache(CacheConfigurationChange.create().setCacheMode(REPLICATED).setBackups(2)); > > > > > > > > > > 2) Call from control.sh in one line without race conditions with > > > > concurrent > > > > > cache changes: > > > > > control.sh --cache --change cacheMode=REPLICATED backups=2 > > > > > > > > > > On Wed, Nov 21, 2018 at 7:32 PM Eduard Shangareev < > > > > > eduard.shangar...@gmail.com> wrote: > > > > > > > > > > > Vovan, > > > > > > > > > > > > user already is able to get cache configuration as xml. > > > > > > control.sh --cache list '.' --config > > > > > > > > > > > > So, user could update it and run: > > > > > > control.sh --cache --restart -cfg=xml.path > > > > > > > > > > > > > > > > > > On Wed, Nov 21, 2018 at 7:06 PM Vladimir Ozerov < > > > voze...@gridgain.com> > > > > > > wrote: > > > > > > > > > > > > > Ed, > > > > > > > > > > > > > > He can do that programmatically. But I meant another case - > Java > > > node > > > > > > > creates a cache. Then .NET node wants to ch
Re: New API for changing configuration of persistent caches
Guys, I like the idea with the configuration builder more. We could limit the set of properties by providing only modifiable ones in the builder interface. Otherwise only runtime will show whether you tried to modify a proper setting. And if we decide to make another property modifiable, then we will just add more methods to the builder interface, so users won't need to remember, which properties in which versions are available for change. We should think about possibility to change a data region for a cache. This is a valid use-case, when you realize, that data in your cache occupies too mush space, so you want it to become persistent. Allowing to change data region configuration on nodes in run-time would be ideal, but it's outside of the scope of the proposed change. Denis чт, 22 нояб. 2018 г. в 11:30, Eduard Shangareev : > I don't see how you variant handles user-defined objects (factories, > affinity-functions, interceptors, etc.). Could you describe? > > On Thu, Nov 22, 2018 at 10:47 AM Vladimir Ozerov > wrote: > > > My variant of API avoids cache configuration. > > > > One more thing to note - as we found out control.sh cannot dump XML > > configuration. Currently it returns only subset of properties. And in > > general case it is impossible to convert CacheConfiguration to Spring > XML, > > because Spring XMLis not serialization protocol. So API with > > CacheConfiguration doesn’t seem to work for control.sh as well. > > > > чт, 22 нояб. 2018 г. в 10:05, Eduard Shangareev < > > eduard.shangar...@gmail.com > > >: > > > > > Vovan, > > > > > > We couldn't avoid API with cache configuration. > > > Almost all of ~70 properties could be changed, some of them are > instances > > > of objects or could be user-defined class. > > > Could you come up with alternatives for user-defined affinity function? > > > > > > Also, the race would have a place in other scenarios. > > > > > > > > > > > > On Thu, Nov 22, 2018 at 8:50 AM Vladimir Ozerov > > > wrote: > > > > > > > Ed, > > > > > > > > We may have API similar to “cache” and “getOrCreateCache”, or may > not. > > It > > > > is up to us to decide. Similarity on it’s own is weak argument. > > > > Functionality and simplicity - this is what matters. > > > > > > > > Approach with cache configuration has three major issues > > > > 1) It exposes properties which user will not be able to change, so > > > typical > > > > user actions would be: try to change property, fail as it is > > unsupported, > > > > go reading documentation. Approach with separate POJO is intuitive > and > > > > self-documenting. > > > > 2) It has race condition between config read and config apply, so > user > > do > > > > not know what exactly he changes, unless you change API to something > > like > > > > “restartCaches(Tuple...)”, > > which > > > > user will need to call in a loop. > > > > 3) And it is not suitable for non-Java platform, which is a > > showstopper - > > > > all API should be available from all platforms unless it is proven to > > be > > > > impossible to implement. > > > > > > > > Vladimir. > > > > > > > > чт, 22 нояб. 2018 г. в 1:06, Eduard Shangareev < > > > > eduard.shangar...@gmail.com > > > > >: > > > > > > > > > Vovan, > > > > > > > > > > Would you argue that we should have the similar API in Java as > > > > > Ignite.cache(CacheConfiguration) or > > > > > Ignite.getOrCreateCache(CacheConfiguration)? > > > > > > > > > > With a proposed solution, every other API call would rely on it > > > finally. > > > > > > > > > > I am interested in having such feature not arguing about API > > > > alternatives. > > > > > > > > > > We definitely should have the ability to change it via control.sh > and > > > > Java > > > > > API. Everything else is optional from my point of view (at least on > > the > > > > > current stage). > > > > > > > > > > Moreover, your arguments are more about our format of > > > CacheConfiguration > > > > > which couldn't be defined in other languages and clients. So, maybe > > we > > > > > should start a discussion about how we should change it in 3.0? > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 21, 2018 at 7:45 PM Vladimir Ozerov < > > voze...@gridgain.com> > > > > > wrote: > > > > > > > > > > > Ed, > > > > > > > > > > > > Why do we want to operate on CacheConfiguration so desperately? > > Your > > > > > > example raises even more questions: > > > > > > 1) What to do with thin clients? > > > > > > 2) What to do with aforementioned race conditions, when cache > could > > > be > > > > > > changed concurrently? > > > > > > 3) Why such trivial operation from user perspective is only > > supported > > > > > from > > > > > > control.sh and not from the rest API (even Java client nodes will > > be > > > > > > affected - remember our plans to remove requirement to have cache > > > > classes > > > > > > on client nodes, which is yet to be implemented). > > > > > > > > > > > > Compare it to alternative API: > > > > > > > > > > > > 1) Native cal
Re: New API for changing configuration of persistent caches
Ed, We have ~70 properties in CacheConfiguration. ~50 of them are plain, ~20 are custom classes. My variant allows to change plain properties from any platform, and the rest 20 from any platform when user has relevant BinaryType. On Thu, Nov 22, 2018 at 11:30 AM Eduard Shangareev < eduard.shangar...@gmail.com> wrote: > I don't see how you variant handles user-defined objects (factories, > affinity-functions, interceptors, etc.). Could you describe? > > On Thu, Nov 22, 2018 at 10:47 AM Vladimir Ozerov > wrote: > > > My variant of API avoids cache configuration. > > > > One more thing to note - as we found out control.sh cannot dump XML > > configuration. Currently it returns only subset of properties. And in > > general case it is impossible to convert CacheConfiguration to Spring > XML, > > because Spring XMLis not serialization protocol. So API with > > CacheConfiguration doesn’t seem to work for control.sh as well. > > > > чт, 22 нояб. 2018 г. в 10:05, Eduard Shangareev < > > eduard.shangar...@gmail.com > > >: > > > > > Vovan, > > > > > > We couldn't avoid API with cache configuration. > > > Almost all of ~70 properties could be changed, some of them are > instances > > > of objects or could be user-defined class. > > > Could you come up with alternatives for user-defined affinity function? > > > > > > Also, the race would have a place in other scenarios. > > > > > > > > > > > > On Thu, Nov 22, 2018 at 8:50 AM Vladimir Ozerov > > > wrote: > > > > > > > Ed, > > > > > > > > We may have API similar to “cache” and “getOrCreateCache”, or may > not. > > It > > > > is up to us to decide. Similarity on it’s own is weak argument. > > > > Functionality and simplicity - this is what matters. > > > > > > > > Approach with cache configuration has three major issues > > > > 1) It exposes properties which user will not be able to change, so > > > typical > > > > user actions would be: try to change property, fail as it is > > unsupported, > > > > go reading documentation. Approach with separate POJO is intuitive > and > > > > self-documenting. > > > > 2) It has race condition between config read and config apply, so > user > > do > > > > not know what exactly he changes, unless you change API to something > > like > > > > “restartCaches(Tuple...)”, > > which > > > > user will need to call in a loop. > > > > 3) And it is not suitable for non-Java platform, which is a > > showstopper - > > > > all API should be available from all platforms unless it is proven to > > be > > > > impossible to implement. > > > > > > > > Vladimir. > > > > > > > > чт, 22 нояб. 2018 г. в 1:06, Eduard Shangareev < > > > > eduard.shangar...@gmail.com > > > > >: > > > > > > > > > Vovan, > > > > > > > > > > Would you argue that we should have the similar API in Java as > > > > > Ignite.cache(CacheConfiguration) or > > > > > Ignite.getOrCreateCache(CacheConfiguration)? > > > > > > > > > > With a proposed solution, every other API call would rely on it > > > finally. > > > > > > > > > > I am interested in having such feature not arguing about API > > > > alternatives. > > > > > > > > > > We definitely should have the ability to change it via control.sh > and > > > > Java > > > > > API. Everything else is optional from my point of view (at least on > > the > > > > > current stage). > > > > > > > > > > Moreover, your arguments are more about our format of > > > CacheConfiguration > > > > > which couldn't be defined in other languages and clients. So, maybe > > we > > > > > should start a discussion about how we should change it in 3.0? > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Nov 21, 2018 at 7:45 PM Vladimir Ozerov < > > voze...@gridgain.com> > > > > > wrote: > > > > > > > > > > > Ed, > > > > > > > > > > > > Why do we want to operate on CacheConfiguration so desperately? > > Your > > > > > > example raises even more questions: > > > > > > 1) What to do with thin clients? > > > > > > 2) What to do with aforementioned race conditions, when cache > could > > > be > > > > > > changed concurrently? > > > > > > 3) Why such trivial operation from user perspective is only > > supported > > > > > from > > > > > > control.sh and not from the rest API (even Java client nodes will > > be > > > > > > affected - remember our plans to remove requirement to have cache > > > > classes > > > > > > on client nodes, which is yet to be implemented). > > > > > > > > > > > > Compare it to alternative API: > > > > > > > > > > > > 1) Native call from any language without limitations: > > > > > > > > > > > > > > > > > > > > > > > > > > > Ignite.changeCache(CacheConfigurationChange.create().setCacheMode(REPLICATED).setBackups(2)); > > > > > > > > > > > > 2) Call from control.sh in one line without race conditions with > > > > > concurrent > > > > > > cache changes: > > > > > > control.sh --cache --change cacheMode=REPLICATED backups=2 > > > > > > > > > > > > On Wed, Nov 21, 2018 at 7:32 PM Eduard Shangareev < > > > > > > eduard.shangar...@gmail.com> wrote:
Re: [IMPORTANT] Future of Binary Objects
If we are developing a product for users, we already guessing what is right and what is wrong for them. So let's avoid these sophistic statements. In the end it is always our responsibility to provide a balanced set of trade-offs between usability, performance and safety. Let me repeat, I'm not against any possible type conversions, but I'm strongly against binary incompatible ones. If we always store List.of(1) as 1 and make them binary interchangeable, I'm OK with that. And still for good practices I'd suggest to look at what Protobuf allows and what not: https://developers.google.com/protocol-buffers/docs/proto3#updating Sergi чт, 22 нояб. 2018 г. в 11:04, Vladimir Ozerov : > Sergi, > > I think we should not guess for users what is right or wrong for them. It > is up to user to decide what is valid. For example, consider a user who > operates on a list of Integers, and to optimize memory consumption he > decide to save in the same field either List, or plain Integer in > case only single element exists. Another example - a kind of data lake or > data cleansing application, which may receive the same field in different > forms. E.g. age in the form of Integer or String. Does it work for user or > not? We do not know. Will he need to migrate the whole data set? We do not > know either. > > The only place in the product where we case is SQL. But in this case > instead of adding checks on binary level, we should validate data on cache > level. In fact, Ignite already works this way. E.g. nullability checks are > performed on cache level rather than binary. All we need is to move all > checks to cache level from binary level. > > > On Thu, Nov 22, 2018 at 9:41 AM Sergi Vladykin > wrote: > > > It may be OK to extend compatible field types (like from Int to Long). > > > > In Protobuf for example this is allowed just because there is no > difference > > between Int and Long in binary format: they all are equally varlen > encoded > > and Longs just will occupy up to 9 bytes, while Ints up to 5. > > > > But for every other case, where binary representation is type dependent, > I > > would be against. This will either require to migrate the whole dataset > to > > a new model (which is always risky, since you may need to rollback to > > previous version of your code) or it will require type checks/conversions > > for each field access, which is a hard to reason complication and > possible > > performance penalty. > > > > Sergi > > > > > > > > чт, 22 нояб. 2018 г. в 09:23, Vladimir Ozerov : > > > > > Denis, > > > > > > Several examples: > > > 1) DEFAULT values - in SQL you may avoid storing default value in the > > table > > > and store it in metadata instead. Not applicable for BinaryObject > because > > > the same binary object may be saved to two SQL tables with different > > > defaults > > > 2) DATE and other temporal types - in SQL you want to store it in > special > > > format to be able to extract date parts quickly (typically - 11 bytes). > > But > > > in Java and some other languages the best format is plain long. this is > > why > > > we use it BinaryObject > > > 3) String charset - in SQL you may choose different charsets for > > different > > > tables. E.g. UTF-8 for one, ASCII for another. In BinaryObject we store > > > everything in UTF-8, and this is fine for most cases, well ... except > of > > > SQL :-) > > > > > > The key thing here is that you cannot define a format which will be > good > > > for both SQL, and native API. They are very different. This is why I > > > propose to define additional interface on cache level defining how to > > store > > > values, which will be very different from binary objects. > > > > > > Vladimir. > > > > > > On Thu, Nov 22, 2018 at 3:32 AM Denis Magda wrote: > > > > > > > Vladimir, > > > > > > > > Could you educate me a little bit, why the current format is bad for > > SQL > > > > and why another one is more suitable? > > > > > > > > Also, if we introduce the new format then why would we keep the > binary > > > one? > > > > Is the new format just a next version of the binary one. > > > > > > > > 2.3) Remove restrictions on changing field type > > > > > I do not know why we did that in the first place. This restriction > > > > prevents > > > > > type evolution and confuses users. > > > > > > > > > > > > That is a hot requirement shared by those who use Ignite SQL in > > > production. > > > > +1. > > > > > > > > -- > > > > Denis > > > > > > > > On Mon, Nov 19, 2018 at 11:05 PM Vladimir Ozerov < > voze...@gridgain.com > > > > > > > wrote: > > > > > > > > > Igniters, > > > > > > > > > > It is very likely that Apache Ignite 3.0 will be released next > year. > > So > > > > we > > > > > need to start thinking about major product improvements. I'd like > to > > > > start > > > > > with binary objects. > > > > > > > > > > Currently they are one of the main limiting factors for the > product. > > > They > > > > > are fat - 30+ bytes overhead on average, high TCO of A
[GitHub] ignite pull request #5471: IGNITE-10216: disable sort annotations in inspect...
GitHub user Mmuzaf opened a pull request: https://github.com/apache/ignite/pull/5471 IGNITE-10216: disable sort annotations in inspection config You can merge this pull request into a Git repository by running: $ git pull https://github.com/Mmuzaf/ignite ignite-10216 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5471.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5471 commit ea6512bc23a6ac2b4eda7081e5af886574b91bdb Author: Maxim Muzafarov Date: 2018-11-22T09:29:16Z IGNITE-10216: disable sort annotations in inspection config ---
Re: proposed realization KILL QUERY command
Actually, option with separate parameters was mentioned in another thread http://apache-ignite-developers.2346864.n4.nabble.com/proposed-design-for-thin-client-SQL-management-and-monitoring-view-running-queries-and-kill-it-tp37713p38056.html Denis чт, 22 нояб. 2018 г. в 08:51, Vladimir Ozerov : > Denis, > > Problems with separate parameters are explained above. > > чт, 22 нояб. 2018 г. в 3:23, Denis Magda : > > > Vladimir, > > > > All of the alternatives are reminiscent of mathematical operations. Don't > > look like a SQL command. What if we use a SQL approach introducing named > > parameters: > > > > KILL QUERY query_id=10 [AND node_id=5] > > > > -- > > Denis > > > > On Wed, Nov 21, 2018 at 4:11 AM Vladimir Ozerov > > wrote: > > > > > Denis, > > > > > > Space is bad candidate because it is a whitespace. Without whitespaces > we > > > can have syntax without quotes at all. Any non-whitespace delimiter > will > > > work, though: > > > > > > KILL QUERY 45.1 > > > KILL QUERY 45-1 > > > KILL QUERY 45:1 > > > > > > On Wed, Nov 21, 2018 at 3:06 PM Юрий > > wrote: > > > > > > > Denis, > > > > > > > > Let's consider parameter of KILL QUERY just a string with some query > > id, > > > > without any meaning for user. User just need to get the id and pass > as > > > > parameter to KILL QUERY command. > > > > > > > > Even if query is distributed it have single query id from user > > > perspective > > > > and will killed on all nodes. User just need to known one global > query > > > id. > > > > > > > > How it can works. > > > > 1)SELECT * from running_queries > > > > result is > > > > query_id | node_id > > > > | sql | schema_name | connection_id | duration > > > > 123.33 | e0a69cb8-a1a8-45f6-b84d-ead367a0 | SELECT ... | > ... > > > > | 22 | 23456 > > > > 333.31 | aaa6acb8-a4a5-42f6-f842-ead111b00020 | UPDATE... | > > ... > > > > | 321| 346 > > > > 2) KILL QUERY '123.33' > > > > > > > > So, user need select query_id from running_queries view and use it > for > > > KILL > > > > QUERY command. > > > > > > > > I hope it became clearer. > > > > > > > > > > > > > > > > ср, 21 нояб. 2018 г. в 02:11, Denis Magda : > > > > > > > > > Folks, > > > > > > > > > > The decimal syntax is really odd - KILL QUERY > > > > > '[node_order].[query_counter]' > > > > > > > > > > Confusing, let's use a space to separate parameters. > > > > > > > > > > Also, what if I want to halt a specific query with certain ID? > Don't > > > know > > > > > the node number, just know that the query is distributed and runs > > > across > > > > > several machines. Sounds like the syntax still should consider > > > > > [node_order/id] as an optional parameter. > > > > > > > > > > Probably, if you explain to me how an end user will use this > command > > > from > > > > > the very beginning (how do I look for a query id and node id, etc) > > then > > > > the > > > > > things get clearer. > > > > > > > > > > -- > > > > > Denis > > > > > > > > > > On Tue, Nov 20, 2018 at 1:03 AM Юрий > > > > wrote: > > > > > > > > > > > Hi Vladimir, > > > > > > > > > > > > Thanks for your suggestion to use MANAGEMENT_POOL for processing > > > > > > cancellation requests. > > > > > > > > > > > > About your questions. > > > > > > 1) I'm going to implements SQL view to provide list of running > > > queries. > > > > > The > > > > > > SQL VIEW has been a little bit discussed earlier. Proposed name > is > > > > > > *running_queries* with following columns: query_id, node_id, sql, > > > > > > schema_name, connection_id, duration. Currently most of the > > > information > > > > > can > > > > > > be retrieved through cache API, however it doesn't matter, any > > case > > > we > > > > > > need to expose SQL VIEW. Seem's you are right - the part should > be > > > > > > implemented firstly. > > > > > > 2) Fully agree that we need to support all kind of SQL queries > > > > > > (SLECT/DML/DDL, transactional, non transnational, local, > > > distributed). > > > > I > > > > > > definitely sure that it will possible for all of above, however > I'm > > > not > > > > > > sure about DDL - need to investigate it deeper. Also need to > > > understand > > > > > > that canceled DML operation can lead to partially updated data > for > > > non > > > > > > transational caches. > > > > > > > > > > > > > > > > > > > > > > > > пн, 19 нояб. 2018 г. в 19:17, Vladimir Ozerov < > > voze...@gridgain.com > > > >: > > > > > > > > > > > > > Hi Yuriy, > > > > > > > > > > > > > > I think we can use MANAGEMENT_POOL for this. It is already used > > for > > > > > some > > > > > > > internal Ignite tasks, and it appears to be a good candidate to > > > > process > > > > > > > cancel requests. > > > > > > > > > > > > > > But there are several things which are not clear enough for me > at > > > the > > > > > > > moment: > > > > > > > 1) How user is going to get the list of running queries in the > > > first > >
[GitHub] ignite pull request #5472: -
GitHub user daradurvs opened a pull request: https://github.com/apache/ignite/pull/5472 - You can merge this pull request into a Git repository by running: $ git pull https://github.com/daradurvs/ignite ignite-9023 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5472.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5472 commit fa1bee70d1ba6357a2e6b985f3391e5ab8ecf99e Author: Vyacheslav Daradur Date: 2018-11-22T10:29:52Z Fixed. ---
[GitHub] ignite pull request #5076: IGNITE-9171 Use lazy mode with results pre-fetch
Github user tledkov-gridgain closed the pull request at: https://github.com/apache/ignite/pull/5076 ---
[jira] [Created] (IGNITE-10376) NPE during invocation from cache
Ivan Fedotov created IGNITE-10376: - Summary: NPE during invocation from cache Key: IGNITE-10376 URL: https://issues.apache.org/jira/browse/IGNITE-10376 Project: Ignite Issue Type: Test Reporter: Ivan Fedotov Assignee: Ivan Fedotov Null pointer exception sometimes appears in [ testAtomicOnheapTwoBackupAsyncFullSync|[https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=3300126853696550025&tab=testDetails]] at the [moment|https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryOrderingEventTest.java#L371] of CacheEntryProcessor invocation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ignite pull request #5473: IGNITE-9171 Use lazy mode for default
GitHub user tledkov-gridgain opened a pull request: https://github.com/apache/ignite/pull/5473 IGNITE-9171 Use lazy mode for default You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-9171 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5473.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5473 commit 044e1e51e6609e92ff46b0650ec8d69afb2b424d Author: tledkov-gridgain Date: 2018-10-23T09:10:02Z IGNITE-9171: save the progress commit 05ee7dba340395208ab040c7e13710433e44fe1e Author: tledkov-gridgain Date: 2018-10-24T13:22:02Z IGNITE-9171: lazy mode performance optimization for one-page results commit e2ffb4a9cc83242e3e6c4283319582c1933eb6b2 Author: tledkov-gridgain Date: 2018-10-24T14:25:58Z Merge branch '_master' into ignite-9171 commit 7ddcacbfcf484ed359fd1f9e2cbcc6aff414249d Author: tledkov-gridgain Date: 2018-10-25T12:01:25Z IGNITE-9171: lazy mode performance optimization commit 70be3700b47a1e367f365b049e7872f4ab84efa8 Author: tledkov-gridgain Date: 2018-10-25T12:06:49Z Merge master into ignite-9171 commit e2832dcb363aa74579175e957d7a3dc88b8263e1 Author: tledkov-gridgain Date: 2018-10-25T15:23:33Z IGNITE-9171: revert debug commit 85efbcd748b7e322a9d9bd8681c10bfa69853d2d Author: tledkov-gridgain Date: 2018-10-26T10:33:14Z IGNITE-9171: fix lazy query cancel after performance fix commit 1777fd393644e4549391afa13fee0d2a322438ef Author: tledkov-gridgain Date: 2018-10-26T12:59:02Z IGNITE-9171: fix lazy query cancel on map phase after performance fix commit 08dec581cbe81452ce8d11dd3fcaf7b54b7318a5 Author: tledkov-gridgain Date: 2018-10-29T08:50:17Z Merge branch '_master' into ignite-9171 commit f330268e70a12930e9f8e28b0966191719d1b39c Author: tledkov-gridgain Date: 2018-10-30T11:52:09Z IGNITE-9171: add OOM test commit 392e1e09550c7f60c25b54cd077e61d58d407dad Author: tledkov-gridgain Date: 2018-10-31T11:46:35Z IGNITE-9171: add OOM tests and suite commit 4be3e9d96a3e82615ef15860b5b298eaf3d5f0ff Author: tledkov-gridgain Date: 2018-10-31T12:04:43Z Merge master into ignite-9171 commit 877204f0beac1a0ab0b8553b449fd3ed03cccb57 Author: tledkov-gridgain Date: 2018-10-31T12:24:32Z IGNITE-9171: fix tests suite after merge commit ece54bb35afd85b9195811657ffadef42ef6e5d5 Author: tledkov-gridgain Date: 2018-10-31T12:28:28Z IGNITE-9171: add lazy paged SQL test to thin client commit 06dd2d33db7a4dbe8029a9e3206d17ab47d9c966 Author: tledkov Date: 2018-11-01T10:00:16Z IGNITE-9171: enhance lazy paged SQL test to thin client commit 7c46ae6f9f2134f4beee4fec65de10e201ef7da5 Author: tledkov Date: 2018-11-01T13:18:01Z IGNITE-9171: fix disabled lazy mode, add tests commit 275f205f4140d939403533291569cf79b3c9cc3f Author: tledkov Date: 2018-11-01T13:24:43Z IGNITE-9171: switch default lazy mode to true for php and nodejs thin clients commit be469480ca698699c2e54019ca19885a8c27619e Author: tledkov Date: 2018-11-01T13:28:54Z IGNITE-9171: add lazy=false tests for java thin client commit fbc2ee7516aa1c9e75837628565f2b166ef2de5b Author: tledkov Date: 2018-11-01T13:32:46Z IGNITE-9171: switch default lazy mode to true for python thin clients commit 4b0c74c4c642d835429308a20183a39539cebad7 Author: tledkov Date: 2018-11-01T14:05:51Z Merge branch '_master' into ignite-9171 commit 366f0582d5d8aa7e7d05b60162cd2da8b89f3db3 Author: tledkov Date: 2018-11-02T09:58:09Z IGNITE-9171: add collocated group by to OOM test commit c8d9cc0faba28437a37c0b7cb6c27685b64ab0b7 Author: tledkov Date: 2018-11-02T10:11:55Z IGNITE-9171: add collocated group by to OOM test commit 708820a51ea11c8a0642f7303a2a2dec9f957389 Author: tledkov Date: 2018-11-02T10:27:50Z Merge master into ignite-9171 commit a78264a783370937e42675366701e0c5322b396f Author: tledkov Date: 2018-11-02T12:46:04Z IGNITE-9171: save the progress commit 4d0a1eb89f06473796578de5695564a7ec525da5 Author: tledkov Date: 2018-11-02T12:55:44Z Merge branch '_master' into ignite-9171 commit 1e7742867c2c95fe8aca39c17d77c48f9e608972 Author: tledkov Date: 2018-11-02T14:20:20Z IGNITE-9171: graceful shutdown on restart commit c879d6c25add7e53ed7640091703829a9843861b Author: tledkov Date: 2018-11-02T15:56:14Z IGNITE-9171: add test suites for lazy = false commit b5b10d4baed4da40356c83f9f4b6c672f34d28d0 Author: tledkov Date: 2018-11-06T08:54:38Z Merge branch '_master' into ignite-9171 commit 3089cfe4976fa8c546d24d3e90317124af1ce68f Author: tledkov Date: 2018-11-07T13:10:52Z Merge branch '_master' into ignite-9171 commit cd758ed0e6b1c07ef83d2b391d298b58fb4d49e3 Author: tledkov Date: 2018-11-07T13:35:56Z
[GitHub] ignite pull request #5353: IGNITE-10047: Wrong MVCC coordinator assignment w...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/5353 ---
[GitHub] ignite pull request #5461: IGNITE-10370: Fix TensorFlowLocalInferenceExample...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/5461 ---
[GitHub] ignite pull request #5474: IGNITE-10050: MVCC: Create "Cache 5" test suite f...
GitHub user AMashenkov opened a pull request: https://github.com/apache/ignite/pull/5474 IGNITE-10050: MVCC: Create "Cache 5" test suite for MVCC mode. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-10050 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5474.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5474 commit 50b9e9dcf5875727b42abcf707fdb50f03f898ce Author: Andrey V. Mashenkov Date: 2018-10-25T14:55:01Z IGNITE-10002: Add MvccCacheTestSuite2. Add flag to force mvcc mode. commit b307bfb1a73955ff416d8c5b9dd74c4e2990f91d Author: Andrey V. Mashenkov Date: 2018-10-25T16:21:24Z IGNITE-10002: Ignore AtomicCache tests. Mute NearCache tests. commit 3bb55091b240fce4888fc1e1c44b9132532dba1a Author: Andrey V. Mashenkov Date: 2018-10-26T18:02:55Z IGNITE-10004: Mute NearCache and CacheStore tests. commit 3c0071d2a6494dae2881a3c19849e26390bfd5d9 Author: Andrey V. Mashenkov Date: 2018-10-26T18:17:39Z IGNITE-10004: WIP. commit 915fc162783dfa05f14ebd44918eec9452cf5ae9 Author: Andrey V. Mashenkov Date: 2018-10-26T18:32:33Z IGNITE-10004: Mute unsupported cases. commit 6257df961693b15433568d1223976151ac37458e Author: Andrey V. Mashenkov Date: 2018-10-29T12:26:39Z IGNITE-10004: WIP. commit 3732a8b049200022e0af85e2441e6c1921461b4d Author: Andrey V. Mashenkov Date: 2018-10-29T13:40:37Z IGNITE-10004: WIP. commit 185baf7ce1ccc414067640fdf2f13b00e052ed23 Author: Andrey V. Mashenkov Date: 2018-10-29T13:41:27Z IGNITE-10004: WIP. commit 4c65f29b7b11556a0d9c38f107f1e3f5eb0cebe9 Author: Andrey V. Mashenkov Date: 2018-10-29T13:53:40Z IGNITE-10004: WIP. commit 3d48430a957459cd0819e1b1f676b251b7b86438 Author: Andrey V. Mashenkov Date: 2018-10-29T14:20:37Z IGNITE-10004: WIP. commit a44f9be8bac14c31ebd1b84351b5a659156b4466 Author: Andrey V. Mashenkov Date: 2018-10-29T16:13:22Z IGNITE-10004: WIP. commit 7d242f4b6b6b13a2876fc8068e2d8f37aa9b2a57 Author: Andrey V. Mashenkov Date: 2018-10-29T16:15:26Z IGNITE-10004: WIP. commit 0ac066f7a10938d2309b56fefb8735fdc1675100 Author: Andrey V. Mashenkov Date: 2018-10-30T08:54:58Z IGNITE-10002: WIP. commit 28f105989b8270e96c5bf59f24639a92c74aca97 Author: Andrey V. Mashenkov Date: 2018-10-30T09:04:44Z IGNITE-10002: Minor. commit 2a61ec244c9a02e200beb211a3c7b0c73be1ec2a Author: Andrey V. Mashenkov Date: 2018-10-30T10:49:35Z IGNITE-10002: Disable non-supported Tx modes. commit 160b39753c41de3d0e3041b07db893fe327c891b Author: Andrey V. Mashenkov Date: 2018-10-30T12:19:42Z IGNITE-10002: Disable non-supported Tx modes. commit 58f00edd49e2a45c650a9c0089ffc9b13217ca74 Author: Andrey V. Mashenkov Date: 2018-10-30T12:49:42Z IGNITE-10002: Minor. commit e5f5838fd1e1655283ba5c56bc8a6cc4ae69412c Author: Andrey V. Mashenkov Date: 2018-11-13T08:15:53Z IGNITE-10002: Fix test. commit 982b4574d201c055ce52005e61308536cb532547 Author: Andrey V. Mashenkov Date: 2018-11-14T09:34:10Z IGNITE-10002: Fix FORCE_MVCC flag. commit 4c3ccee618180a7282bedc49752c9c75f0711e68 Author: Andrey V. Mashenkov Date: 2018-11-14T09:34:20Z IGNITE-10002: Fix tests. commit 9d65e2614c6182c4fcaf6fe74bf299c73af2bc07 Author: Andrey V. Mashenkov Date: 2018-11-14T11:03:26Z IGNITE-10002: Fix tx tests. commit ddb59fcea609a62aa77532db5bcde83c96e0f382 Author: Andrey V. Mashenkov Date: 2018-11-14T12:10:07Z IGNITE-10002: Fix hanged test. commit 712e6d8ddcdaa5aecd7762d25cb062f94ef06ce5 Author: Andrey V. Mashenkov Date: 2018-11-14T13:34:41Z IGNITE-10002: Fix node stop. commit db9aca4792753e6b0ca2eddb8c8795b41498e682 Author: Andrey V. Mashenkov Date: 2018-11-14T15:08:51Z IGNITE-10002: Mute hanged test. commit 30d24b1c0a961d3cb9e64231867cc87982f84332 Author: user Date: 2018-11-14T19:38:52Z IGNITE-10052: Fix and mute mvcc tests. commit 191500222e87256f3417019bd2df3dc886918806 Author: user Date: 2018-11-14T19:48:10Z IGNITE-10002: Mute mvcc tests. commit 0293f36c7621843a8cf84d848df37f6996a21d71 Author: user Date: 2018-11-14T20:10:59Z IGNITE-10002: Mute mvcc tests. commit 760972570935ad59a118267ce3589ccad77f7ad0 Author: Andrey V. Mashenkov Date: 2018-11-15T09:05:46Z IGNITE-10002: Fix FORCE_MVCC flag. commit 219c402f6c95ca12015bf8ba314b9c20cc1b6683 Author: Andrey V. Mashenkov Date: 2018-11-15T09:07:42Z IGNITE-10002: Unmute mvcc localPeek tests. commit c0fc6c9c113c23be058d56b63fbe336ffd165491 Author: Andrey V. Mashenkov Date: 2018-11-15T09:35:00Z IGNITE-10002: Mute tests. ---
[jira] [Created] (IGNITE-10377) MVCC: Incorrect exception is thrown if no data nodes found for a partition.
Andrew Mashenkov created IGNITE-10377: - Summary: MVCC: Incorrect exception is thrown if no data nodes found for a partition. Key: IGNITE-10377 URL: https://issues.apache.org/jira/browse/IGNITE-10377 Project: Ignite Issue Type: Bug Components: mvcc Reporter: Andrew Mashenkov See NotMappedPartitionInTxTest. ClusterTopologyServerNotFoundException is expected, instead of "ClusterTopologyCheckedException: Failed to get primary node" -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ignite pull request #5475: IGNIE-10339 Skip index partition file integrity c...
GitHub user ivandasch opened a pull request: https://github.com/apache/ignite/pull/5475 IGNIE-10339 Skip index partition file integrity check for in-memory caches. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-10339-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5475.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5475 commit 53ac9d18309ec68f6754e74fd16926342c818cc0 Author: Ivan Daschinskiy Date: 2018-11-22T12:25:45Z IGNIE-10339 Skip index partition file integrity check for in-memory caches. ---
[GitHub] dspavlov opened a new pull request #79: IGNITE-10372: Optimize master trends page, fast fix
dspavlov opened a new pull request #79: IGNITE-10372: Optimize master trends page, fast fix URL: https://github.com/apache/ignite-teamcity-bot/pull/79 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (IGNITE-10378) MVCC TX: Possible race on MVCC coordinator reassignment
Igor Seliverstov created IGNITE-10378: - Summary: MVCC TX: Possible race on MVCC coordinator reassignment Key: IGNITE-10378 URL: https://issues.apache.org/jira/browse/IGNITE-10378 Project: Ignite Issue Type: Bug Components: mvcc Environment: We need to add an additional check to finish MVCC coordinator init future on particular exchange done only. Reporter: Igor Seliverstov Fix For: 2.8 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] asfgit closed pull request #79: IGNITE-10372: Optimize master trends page, fast fix
asfgit closed pull request #79: IGNITE-10372: Optimize master trends page, fast fix URL: https://github.com/apache/ignite-teamcity-bot/pull/79 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
How to deprecate unused Ignite's system property properly? (IGNITE-7441 Drop IGNITE_SERVICES_COMPATIBILITY_MODE system property)
Hi, Igniters! Here is Jira issue [1] to drop one of Ignite's system property "IGNITE_SERVICES_COMPATIBILITY_MODE" because it is not used. I looked through git history and related Jira issues, the common conclusions: - the property was introduced in Ignite 1.7 within the task [2] to use LazyServiceConfiguration with premarshaled services instance; - Ignite 2.* was released which is incompatible with 1.*; - since Ignite 2.3 the property is completely ignored after introduced batch deployment mode [3]; Looks like we can just remove the property without introducing any compatibility issues. Also, related node attribute 'ATTR_SERVICES_COMPATIBILITY_MODE' can be safely removed. So, my question is: can I just remove following properties or I should deprecate them and remove all usages? IgniteSystemProperties#IGNITE_SERVICES_COMPATIBILITY_MODE IgniteNodeAttributes#ATTR_SERVICES_COMPATIBILITY_MODE [1] https://issues.apache.org/jira/browse/IGNITE-7441 [2] https://issues.apache.org/jira/browse/IGNITE-3056 [3] https://issues.apache.org/jira/browse/IGNITE-5145 -- Best Regards, Vyacheslav D.
[GitHub] ignite pull request #5476: IGNITE-10323 Contol utility --deactivate on non-a...
GitHub user vldpyatkov opened a pull request: https://github.com/apache/ignite/pull/5476 IGNITE-10323 Contol utility --deactivate on non-activate cluster prod⦠â¦use NPE and handler stop nodes You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-10323 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5476.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5476 commit 4bb188428168b90c3c047faa7b868bd996ffc18a Author: vd-pyatkov Date: 2018-11-22T12:59:01Z IGNITE-10323 Contol utility --deactivate on non-activate cluster produse NPE and handler stop nodes ---
[GitHub] ignite pull request #5477: IGNITE-8558 Provide an opportunity to stop grids ...
GitHub user Rodion4ik opened a pull request: https://github.com/apache/ignite/pull/5477 IGNITE-8558 Provide an opportunity to stop grids and cancel tasks after execution all tests Add method to class GridAbstractTest Add new boolean method "isWillJobsCancelled()" which can be overriden You can merge this pull request into a Git repository by running: $ git pull https://github.com/Rodion4ik/ignite ignite-8558 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5477.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5477 commit 74689ab65faaa43e45c6176eab996d2297771bc7 Author: Rodion4ik Date: 2018-11-22T13:00:59Z IGNITE-8558 Add new boolean method "isWillJobsCancelled()" which can be overriden ---
[GitHub] ignite pull request #5478: IGNITE-10365: MVCC: Create "Cache 6" test suite f...
GitHub user AMashenkov opened a pull request: https://github.com/apache/ignite/pull/5478 IGNITE-10365: MVCC: Create "Cache 6" test suite for MVCC mode. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-10365 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5478.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5478 commit 50b9e9dcf5875727b42abcf707fdb50f03f898ce Author: Andrey V. Mashenkov Date: 2018-10-25T14:55:01Z IGNITE-10002: Add MvccCacheTestSuite2. Add flag to force mvcc mode. commit b307bfb1a73955ff416d8c5b9dd74c4e2990f91d Author: Andrey V. Mashenkov Date: 2018-10-25T16:21:24Z IGNITE-10002: Ignore AtomicCache tests. Mute NearCache tests. commit 3bb55091b240fce4888fc1e1c44b9132532dba1a Author: Andrey V. Mashenkov Date: 2018-10-26T18:02:55Z IGNITE-10004: Mute NearCache and CacheStore tests. commit 3c0071d2a6494dae2881a3c19849e26390bfd5d9 Author: Andrey V. Mashenkov Date: 2018-10-26T18:17:39Z IGNITE-10004: WIP. commit 915fc162783dfa05f14ebd44918eec9452cf5ae9 Author: Andrey V. Mashenkov Date: 2018-10-26T18:32:33Z IGNITE-10004: Mute unsupported cases. commit 6257df961693b15433568d1223976151ac37458e Author: Andrey V. Mashenkov Date: 2018-10-29T12:26:39Z IGNITE-10004: WIP. commit 3732a8b049200022e0af85e2441e6c1921461b4d Author: Andrey V. Mashenkov Date: 2018-10-29T13:40:37Z IGNITE-10004: WIP. commit 185baf7ce1ccc414067640fdf2f13b00e052ed23 Author: Andrey V. Mashenkov Date: 2018-10-29T13:41:27Z IGNITE-10004: WIP. commit 4c65f29b7b11556a0d9c38f107f1e3f5eb0cebe9 Author: Andrey V. Mashenkov Date: 2018-10-29T13:53:40Z IGNITE-10004: WIP. commit 3d48430a957459cd0819e1b1f676b251b7b86438 Author: Andrey V. Mashenkov Date: 2018-10-29T14:20:37Z IGNITE-10004: WIP. commit a44f9be8bac14c31ebd1b84351b5a659156b4466 Author: Andrey V. Mashenkov Date: 2018-10-29T16:13:22Z IGNITE-10004: WIP. commit 7d242f4b6b6b13a2876fc8068e2d8f37aa9b2a57 Author: Andrey V. Mashenkov Date: 2018-10-29T16:15:26Z IGNITE-10004: WIP. commit 0ac066f7a10938d2309b56fefb8735fdc1675100 Author: Andrey V. Mashenkov Date: 2018-10-30T08:54:58Z IGNITE-10002: WIP. commit 28f105989b8270e96c5bf59f24639a92c74aca97 Author: Andrey V. Mashenkov Date: 2018-10-30T09:04:44Z IGNITE-10002: Minor. commit 2a61ec244c9a02e200beb211a3c7b0c73be1ec2a Author: Andrey V. Mashenkov Date: 2018-10-30T10:49:35Z IGNITE-10002: Disable non-supported Tx modes. commit 160b39753c41de3d0e3041b07db893fe327c891b Author: Andrey V. Mashenkov Date: 2018-10-30T12:19:42Z IGNITE-10002: Disable non-supported Tx modes. commit 58f00edd49e2a45c650a9c0089ffc9b13217ca74 Author: Andrey V. Mashenkov Date: 2018-10-30T12:49:42Z IGNITE-10002: Minor. commit e5f5838fd1e1655283ba5c56bc8a6cc4ae69412c Author: Andrey V. Mashenkov Date: 2018-11-13T08:15:53Z IGNITE-10002: Fix test. commit 982b4574d201c055ce52005e61308536cb532547 Author: Andrey V. Mashenkov Date: 2018-11-14T09:34:10Z IGNITE-10002: Fix FORCE_MVCC flag. commit 4c3ccee618180a7282bedc49752c9c75f0711e68 Author: Andrey V. Mashenkov Date: 2018-11-14T09:34:20Z IGNITE-10002: Fix tests. commit 9d65e2614c6182c4fcaf6fe74bf299c73af2bc07 Author: Andrey V. Mashenkov Date: 2018-11-14T11:03:26Z IGNITE-10002: Fix tx tests. commit ddb59fcea609a62aa77532db5bcde83c96e0f382 Author: Andrey V. Mashenkov Date: 2018-11-14T12:10:07Z IGNITE-10002: Fix hanged test. commit 712e6d8ddcdaa5aecd7762d25cb062f94ef06ce5 Author: Andrey V. Mashenkov Date: 2018-11-14T13:34:41Z IGNITE-10002: Fix node stop. commit db9aca4792753e6b0ca2eddb8c8795b41498e682 Author: Andrey V. Mashenkov Date: 2018-11-14T15:08:51Z IGNITE-10002: Mute hanged test. commit 30d24b1c0a961d3cb9e64231867cc87982f84332 Author: user Date: 2018-11-14T19:38:52Z IGNITE-10052: Fix and mute mvcc tests. commit 191500222e87256f3417019bd2df3dc886918806 Author: user Date: 2018-11-14T19:48:10Z IGNITE-10002: Mute mvcc tests. commit 0293f36c7621843a8cf84d848df37f6996a21d71 Author: user Date: 2018-11-14T20:10:59Z IGNITE-10002: Mute mvcc tests. commit 760972570935ad59a118267ce3589ccad77f7ad0 Author: Andrey V. Mashenkov Date: 2018-11-15T09:05:46Z IGNITE-10002: Fix FORCE_MVCC flag. commit 219c402f6c95ca12015bf8ba314b9c20cc1b6683 Author: Andrey V. Mashenkov Date: 2018-11-15T09:07:42Z IGNITE-10002: Unmute mvcc localPeek tests. commit c0fc6c9c113c23be058d56b63fbe336ffd165491 Author: Andrey V. Mashenkov Date: 2018-11-15T09:35:00Z IGNITE-10002: Mute tests. ---
[GitHub] ignite pull request #5479: IGNITE-10329: query and query-join benchmarks for...
GitHub user pavel-kuznetsov opened a pull request: https://github.com/apache/ignite/pull/5479 IGNITE-10329: query and query-join benchmarks for Ignite, Postgresql and Mysql You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-10329 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5479.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5479 commit 6a1144bc95b45f7d048675442ac91839255ad167 Author: Pavel Kuznetsov Date: 2018-11-20T18:33:14Z ignite-10329: Draft of simple select benchmark. commit 58b05e696323781f0696b6e814aab16eadf52fe5 Author: Pavel Kuznetsov Date: 2018-11-20T19:40:41Z ignite-10329: Draft for benchmarks. Added 2 benchmarks that perform select and select with join operations. Added configuration file only for Ignite. commit 38647a095fe05f74792706827fc732057f773ad3 Author: Pavel Kuznetsov Date: 2018-11-20T19:53:23Z ignite-10329: fixed comment. commit 7ca39c87a86969465c53b10f94530ec550473b80 Author: Pavel Kuznetsov Date: 2018-11-20T20:00:48Z ignite-10329: Fixed parameter in test(). commit 512a8c41f12768922d4cd02a1ca3794ea8b233de Author: Pavel Kuznetsov Date: 2018-11-20T20:06:34Z ignite-10329: Fixed parameters in test(). commit ea93a3f61e7272e169ef7156a04fe985c4a841fd Author: Pavel Kuznetsov Date: 2018-11-21T09:17:07Z ignite-10329: Fixed typo in the query. Join benchmark have been fixed. commit fdf62dfb159f105b06cef69461c6d36b5358d921 Author: Pavel Kuznetsov Date: 2018-11-21T14:00:35Z ignite-10329: Fixed queries for postgresql. Changed LONG -> BIGINT, used PUBLIC schema explicitly.Added properties file for postgresql run. commit a1a9cb02166fa6aee84d1417225fb2bc02574c6d Author: Pavel Kuznetsov Date: 2018-11-22T11:25:04Z ignite-10329: Added mysql configuration. commit 14932ff449af4ddfcf0c28e327e8b48229974eba Author: Pavel Kuznetsov Date: 2018-11-22T14:17:25Z ignite-10329: Made loading faster. Logging. Added batching, disabled autocommit (made mysql loading super-fast) and improved logging of database population process. ---
Re: Improve lazy mode SQL query execution (IGNITE-9171)
Hi, Lets discuss changes of Ignite public interface relates to retry SQL queries. Should we add new public exception (.e.g: `QueryRetryException`) for cache SQL API and special vendor error code to SQLException for JDBC? 21.11.2018 16:32, Vladimir Ozerov пишет: Hi Taras, Unfortunately, this will not be easy to implement. We already have AffinityTopologyVersion which is updated on certain schema change operations, such as CREATE/DROP TABLE. Moreover, it is already passed in query request messages (GridH2QueryRequest.topVer). Unfortunately, it is not updated for other DDL scenarios, such as ALTER TABLE. We can introduce our own global counter. We can try to integrate with AffinityTopologyVersion. Both solutions will require tight integration with PME mechanism to function properly. But good news is that this is not a new problem. Currently it is already possible for two map requests to be executed on different schemas: client_1: SEND_QUERY client_2: SEND_ALTER node_1 :EXEC_QUERY EXEC_ALTER node_2 : EXEC_ALTER EXEC_QUERY So I would say that nothing changes for us even after proposed changes, and way may leave this problem aside for now. Makes sense? On Wed, Nov 21, 2018 at 4:15 PM Taras Ledkov wrote: Vladimir, The query protocol will be changed in both solution. The tables versions must be added to the 'GridQueryNextPageResponse' message and must be compared on the reduce node too. Because the DLL / DML race may be happens on different nodes. 21.11.2018 15:24, Vladimir Ozerov пишет: Taras, Thank you for analysis. I'd like to add that there is another solution - PME :-) In this case DDL will be blocked until current SELECTs and updates are finished, and do not allow new operations to be executed. This approach solves all issues - it avoids deadlocks between query threads and DDL threads when they are reordered on different nodes, it avoids starvation of DDL operations, and it doesn't cancel any queries. But there is serious drawback - performance. The drawback is that it is more complex to implement (query protocol changes might be required), it blocks the cluster even when it is needed, and it may destabilize PME mechanism, which is already on his last legs. For this reason killing queries which interleave with DDL looks like a balanced solution for now - it is reasonably simple, allows us to we avoid OOME in many cases, and do not introduce any additional complexity for users, as they are already prepared for re-tries. But I would like to stress one thing - we will need integration with PME at some point in time anyway. Some DDL operations are blocking in their nature (e.g. DROP COLUMN). Other DDL operations may be non-blocking, but blocking implementation may give them serious performance benefits (e.g. CREATE INDEX). So I propose to go with your solution for now, and start thinking about SQL -> PME integration in the background. Vladimir. On Wed, Nov 21, 2018 at 2:23 PM Taras Ledkov wrote: Hi community, We will enhance lazy mode for SQL query execution. Lazy mode overview: Lazy mode is related to H2 lazy mode when the all query results are not copied to the RAM in some cases. The mode it is applicable for SELECTs that doesn't not require materialize all results in memory, e.g. simple scan plans, IDX lookup, merge join etc. And not applicable for SORT by not indexed fields, aggregates, nested loops joins etc. When mode is applicable it produces result with iterator-like behavior on server side and not consume RAM. So the huge result set may be selected without OOME. The current implementation. The current implementation is start separate thread for each query with 'lazy=true'. This is caused by the our implementation of 'GridH2Table'. In details: the table's locks. The table must be locked while result set is completed. When lazy is disabled a complete result is generated on the first step of a query execution (then tables unlock) and result is stored on the node and sent to other node (or client) page by page. When lazy is enabled tables are locked until result set delivery to client. The start new thread causes overhead for requests that returns small result set. But current table lock is used `ReentrantReadWriteLock` and we cannot lock tables from one thread of QUERY thread pool and unlock in the other thread (when query is complete or cancel). The trivial solve is using the 'StampedLock' it solve the lock behavior, but not solve the table DDL starvation / deadlock. Example: Lets the QUERY thread pool contains only one thread. The case is scaled for any thread pool size. Write operation that require to exclusive table lock is DDL operation. 1. The query Q0 acquires the shared lock for the table T, send first page result and leave thread 'threadQP0' control. 2. DDL0 blocks on write lock the table T at the 'threadWP0 ' 3. The query Q1 blocks on read lock the 'threadQP0' (because the writer in the queue). The de
[jira] [Created] (IGNITE-10379) SQL: Extract partition info from BETWEEN and range conditions for integer types
Vladimir Ozerov created IGNITE-10379:
Summary: SQL: Extract partition info from BETWEEN and range
conditions for integer types
Key: IGNITE-10379
URL: https://issues.apache.org/jira/browse/IGNITE-10379
Project: Ignite
Issue Type: Task
Components: sql
Reporter: Vladimir Ozerov
If there is a range condition on affinity column of integer type, we may try to
extract partition info from it in a way similar to IN clause [1]:
{{x BETWEEN 1 and 5}} -> {{x IN (1, 2, 3, 4, 5)}}
{{x > 1 and x <= 5}} -> {{x IN (2, 3, 4, 5)}}
[1] IGNITE-9632
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: How to deprecate unused Ignite's system property properly? (IGNITE-7441 Drop IGNITE_SERVICES_COMPATIBILITY_MODE system property)
Vyacheslav, You are right. This property is not used anywhere, so you can safely remove it. I don't think, there is any need in deprecation. You can just go ahead and drop it, since it doesn't have any effect. Denis чт, 22 нояб. 2018 г. в 15:47, Vyacheslav Daradur : > Hi, Igniters! > > Here is Jira issue [1] to drop one of Ignite's system property > "IGNITE_SERVICES_COMPATIBILITY_MODE" because it is not used. > > I looked through git history and related Jira issues, the common > conclusions: > - the property was introduced in Ignite 1.7 within the task [2] to use > LazyServiceConfiguration with premarshaled services instance; > - Ignite 2.* was released which is incompatible with 1.*; > - since Ignite 2.3 the property is completely ignored after introduced > batch deployment mode [3]; > > Looks like we can just remove the property without introducing any > compatibility issues. > Also, related node attribute 'ATTR_SERVICES_COMPATIBILITY_MODE' can be > safely removed. > > So, my question is: can I just remove following properties or I should > deprecate them and remove all usages? > IgniteSystemProperties#IGNITE_SERVICES_COMPATIBILITY_MODE > IgniteNodeAttributes#ATTR_SERVICES_COMPATIBILITY_MODE > > [1] https://issues.apache.org/jira/browse/IGNITE-7441 > [2] https://issues.apache.org/jira/browse/IGNITE-3056 > [3] https://issues.apache.org/jira/browse/IGNITE-5145 > > -- > Best Regards, Vyacheslav D. >
[GitHub] dspavlov opened a new pull request #80: IGNITE-10372 Step 2 In memory caching of build statistics, don't loading tests for composite
dspavlov opened a new pull request #80: IGNITE-10372 Step 2 In memory caching of build statistics, don't loading tests for composite URL: https://github.com/apache/ignite-teamcity-bot/pull/80 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ignite pull request #5480: Ignite 2.7 master
GitHub user alamar opened a pull request: https://github.com/apache/ignite/pull/5480 Ignite 2.7 master You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-2.7-master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5480.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5480 commit 762e2f4f745974e5742cd7bdd71f7ab4ec3bfc51 Author: Ivan Daschinskiy Date: 2018-09-28T15:03:45Z IGNITE-9693 Scale up wal compression workers to increase performance - Fixes #4831. Signed-off-by: Ivan Rakov commit 58dfe061cf8b4c18ac57fa762a559d711cfbf274 Author: Anton Kalashnikov Date: 2018-10-02T15:26:20Z IGNITE-9761 Fixed deadlock in WAL manager - Fixes #4890. Signed-off-by: Alexey Goncharuk commit 0bae33501e12ed27322fcba65736676ba06bb4b8 Author: Anton Kalashnikov Date: 2018-10-02T16:07:36Z IGNITE-9760 Fixed NPE in WAL manager for FSYNC mode - Fixes #4888. Signed-off-by: Alexey Goncharuk commit 60a8c8c8d0c983ebd62df48d59fdf4b0b40d8336 Author: Ilya Kasnacheev Date: 2018-10-02T16:13:30Z GG-14266 Make IGNITE-9084 available from 2.5.3+ commit a35dac25c996e3fd6ddbe159217d50d5218c30a7 Author: devozerov Date: 2018-10-04T08:33:10Z Merge branch 'ignite-2.7' into ignite-2.7-master commit ad6416ca4905d0d0bd032c636dca55eee497fbdb Author: Pavel Kovalenko Date: 2018-10-04T09:46:54Z IGNITE-9661 Improved performance of partition state validation during PME - Fixes #4850. commit e8176634df933f0936999fc5c4e0383cd5754bf6 Author: Ivan Rakov Date: 2018-10-04T17:16:04Z Merge branch 'ignite-2.7' into ignite-2.7-master commit ebea1bc90e6619816826f0bdc4be8f7f98d9eefc Author: Alexey Kuznetsov Date: 2018-10-05T09:35:05Z IGNITE-9792 Fixed assert in case if IGNITE_MBEANS_DISABLED is true. Fixed tests. (cherry picked from commit 78c2d3bbbd620bb7795d9f362785e073d2dec0a2) commit a543be2cebfba4dc5352a9b5a4995fd244eae333 Author: devozerov Date: 2018-10-08T11:08:43Z Merge branch 'ignite-2.7' into ignite-2.7-master commit b18ea0055906363ca1d3577b0d21788c3099d562 Author: devozerov Date: 2018-10-08T11:14:15Z Merge branch 'ignite-2.7' into ignite-2.7-master commit dfd6fcbd36510b8cdbc89162d0d7371a10f7f73a Author: devozerov Date: 2018-10-09T06:06:00Z Merge branch 'ignite-2.7' into ignite-2.7-master commit 7c9bd23c345bb4217af5c7ae1a4afb1798072039 Author: Aleksey Plekhanov Date: 2018-10-09T11:33:25Z IGNITE-9785 Introduce read-only state in local node context - Fixes #4907. Signed-off-by: Ivan Rakov (cherry picked from commit 179b09b) commit 89bfa814267827e40e01c492cea7391fdd218279 Author: Igor Sapego Date: 2018-10-09T15:01:20Z Merge branch 'ignite-2.7' into ignite-2.7-master commit 097f2c437e1dd43d02dbbbe31d354435f11bed15 Author: Igor Sapego Date: 2018-10-09T15:26:18Z Merge branch 'ignite-2.7' into ignite-2.7-master commit 25e94c30b1de41ac6d3e459a5a466a0cb803a0a0 Author: Aleksey Plekhanov Date: 2018-10-10T08:57:11Z IGNITE-9500: SQL: implemented system view for list of caches (IGNITE.CACHES). This closes #4716. commit 41ac0540451a2c9c349f18f9388ebe440d391f89 Author: devozerov Date: 2018-10-10T09:42:06Z Merge branch 'ignite-2.7' into ignite-2.7-master commit 2a03b23c5840923377549d86693cdb74fac25e6a Author: Denis Mekhanikov Date: 2018-10-05T13:13:45Z IGNITE-9794 Handle UnregisteredBinaryTypeException on metadata registration under topology lock. - Fixes #4916. Signed-off-by: Dmitriy Govorukhin (cherry picked from commit b1206121e7a87f2d84414ab03b86b8614c0bc3c0) commit f9cbd98498f70a8315316b44185fe7853a588145 Author: Igor Sapego Date: 2018-10-10T16:50:58Z Merge branch 'ignite-2.7' into ignite-2.7-master commit 91134337ea2a3816be9ce486998f132d0b9fd109 Author: Vasiliy Sisko Date: 2018-10-11T03:03:34Z IGNITE-7926 Web Agent: Support launching with Java 8+. (cherry picked from commit f313d650448207942357a88bcdeab5833c8bd963) commit 44c0545051302378693a56d538ea4411b36a14a9 Author: Igor Sapego Date: 2018-10-11T10:32:43Z Merge branch 'ignite-2.7' into ignite-2.7-master commit 6ab987ec66929b035245c7ab14ee8e155753 Author: devozerov Date: 2018-10-11T11:38:53Z Merge branch 'ignite-2.7' into ignite-2.7-master commit 408943770ab3ee55f88812e41d30e2943daeb90d Author: Igor Sapego Date: 2018-10-11T12:09:58Z Merge branch 'ignite-2.7' into ignite-2.7-master commit 8a48441e209f88bb428795a6459ad02689a0e04d Author: Eduard Shangareev Date: 2018-10-11T07:58:15Z IGNITE-9796 NPE if you call array() method on empty GridLongList - Fixes #4917. Signed-off-by: Ivan Rakov (cherry picked from commit 447ce47) commit 66c309424f3846f18c32
[GitHub] asfgit closed pull request #80: IGNITE-10372 Step 2 In memory caching of build statistics, don't loading tests for composite
asfgit closed pull request #80: IGNITE-10372 Step 2 In memory caching of build statistics, don't loading tests for composite URL: https://github.com/apache/ignite-teamcity-bot/pull/80 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ignite pull request #5352: IGNITE-10194: ZookeeperDiscoverySpiTest fails on ...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/5352 ---
[jira] [Created] (IGNITE-10381) U.doInParallel can terminate early due to an error in batch processing
Alexey Goncharuk created IGNITE-10381: - Summary: U.doInParallel can terminate early due to an error in batch processing Key: IGNITE-10381 URL: https://issues.apache.org/jira/browse/IGNITE-10381 Project: Ignite Issue Type: Improvement Reporter: Alexey Goncharuk -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10380) [ML] Drop Multi-label Classification for Logistic Regression and SVM
Aleksey Zinoviev created IGNITE-10380: - Summary: [ML] Drop Multi-label Classification for Logistic Regression and SVM Key: IGNITE-10380 URL: https://issues.apache.org/jira/browse/IGNITE-10380 Project: Ignite Issue Type: Improvement Components: ml Affects Versions: 2.8 Reporter: Aleksey Zinoviev Assignee: Aleksey Zinoviev Fix For: 2.8 After One-vs-Rest implementation these separate algorithms could be dropped both. Also, rename BinaryClassification LogReg -> LogReg BinarySVM -> SVM NOTE: Appropriate Docs should be dropped in release 2.8 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ignite pull request #5481: IGNITE-9145: Added EncodingSortingStrategy
GitHub user zaleslaw opened a pull request: https://github.com/apache/ignite/pull/5481 IGNITE-9145: Added EncodingSortingStrategy You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-9145 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5481.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5481 commit b90b1de2129ffec9741386cc9a1e9c23e199461e Author: Zinoviev Alexey Date: 2018-11-21T17:22:08Z IGNITE-9145: Added EncodingSortingStrategy ---
[GitHub] ignite pull request #5369: IGNITE-10174 migrate examples tests from Junit 3 ...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/5369 ---
[GitHub] ignite pull request #5482: IGNITE-7441 Drop IGNITE_SERVICES_COMPATIBILITY_MO...
GitHub user daradurvs opened a pull request: https://github.com/apache/ignite/pull/5482 IGNITE-7441 Drop IGNITE_SERVICES_COMPATIBILITY_MODE system property You can merge this pull request into a Git repository by running: $ git pull https://github.com/daradurvs/ignite ignite-7441 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5482.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5482 commit 3cf7c582f405d1acf06b73fad576948823e8c453 Author: Vyacheslav Daradur Date: 2018-11-22T15:39:49Z Removed IGNITE_SERVICES_COMPATIBILITY_MODE ---
[GitHub] ignite pull request #5483: IGNITE-10381 Fixed U.doInParallel early terminati...
GitHub user agoncharuk opened a pull request: https://github.com/apache/ignite/pull/5483 IGNITE-10381 Fixed U.doInParallel early termination You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-10381 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5483.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5483 commit b8d13437f7aeb4b9ec6aaec83ee451fcaf24e281 Author: Alexey Goncharuk Date: 2018-11-22T16:07:52Z IGNITE-10381 Fixed U.doInParallel early termination ---
[GitHub] ignite pull request #5484: IGNITE-10272: Inject learning environment into sc...
GitHub user artemmalykh opened a pull request: https://github.com/apache/ignite/pull/5484 IGNITE-10272: Inject learning environment into scope of dataset compute task. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-10272 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5484.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5484 commit 765c4400588398695c954a183887149dfd72cb6e Author: Artem Malykh Date: 2018-11-15T20:00:06Z First version commit 78857ec45c527ef07517f3a66d85843554851433 Author: Artem Malykh Date: 2018-11-16T12:49:21Z LearningEnvironmentBuilder is now interface commit 762568c28eb9f7d0ca044d7c4a3a4e4173dfdb34 Author: Artem Malykh Date: 2018-11-19T14:38:17Z Started integrating learning environment into transformers commit 74f9b09602a435c04127218f05e601e1876dee8a Author: Artem Malykh Date: 2018-11-20T11:13:25Z Learning environment included in bagging commit eeefda2107e60dc47a9229c01f50303fbb0d877e Author: Artem Malykh Date: 2018-11-22T15:22:43Z Insterted learning environment into compute commit 2ce75e07bc531092a9388688f67125463bdd7ea6 Author: Artem Malykh Date: 2018-11-22T15:48:58Z Fixed javadocs commit 13bfc70d4b9c39eaec8f4c37ba6b341ba1f7401a Author: Artem Malykh Date: 2018-11-22T16:03:03Z Javadocs commit 8454f2dbfe105fedb28025359a9414a2d7666090 Author: Artem Malykh Date: 2018-11-22T16:06:22Z Merge branch 'master-apache' into ignite-10272 ---
[GitHub] ignite pull request #5460: IGNITE-10191 Incorrect comparison of lists in Ren...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/5460 ---
[GitHub] ignite pull request #4882: IGNITE-9754 Increase timeout waiting for commitLa...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4882 ---
[GitHub] ignite pull request #5469: IGNITE-10375 Fix inspection failures
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/5469 ---
[jira] [Created] (IGNITE-10382) [ML] Move datasets from example to ml module
Yury Babak created IGNITE-10382: --- Summary: [ML] Move datasets from example to ml module Key: IGNITE-10382 URL: https://issues.apache.org/jira/browse/IGNITE-10382 Project: Ignite Issue Type: Improvement Components: ml Reporter: Yury Babak Fix For: 2.8 We use mostly same datasets for ml tests and for ml examples. So we could move those datasets to ml module -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ignite pull request #5450: IGNITE-10298 Cover possible deadlock in case of c...
Github user Jokser closed the pull request at: https://github.com/apache/ignite/pull/5450 ---
[jira] [Created] (IGNITE-10383) Partition Full message finished info printed out in logs after full message sent info
Max Shonichev created IGNITE-10383:
--
Summary: Partition Full message finished info printed out in logs
after full message sent info
Key: IGNITE-10383
URL: https://issues.apache.org/jira/browse/IGNITE-10383
Project: Ignite
Issue Type: Bug
Reporter: Max Shonichev
see excerpt from logs:
{noformat}
==[crd]==
2018-11-19 18:57:55.845 [INFO
][sys-#3636%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture]
Finish exchange future [startVer=AffinityTopologyVersion [topVer=282,
minorTopVer=0], resVer=AffinityTopologyVersion [topVer=282, minorTopVer=0],
err=null]
2018-11-19 18:57:55.845 [INFO
][sys-#3636%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture]
Sending Full Message performed in 3 ms.
2018-11-19 18:57:55.845 [INFO
][sys-#3636%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture]
Sending Full Message to all nodes performed in 5 ms.
2018-11-19 18:59:01.289 [INFO
][sys-#3610%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.GridCachePartitionExchangeManager]
Finished full message creation [msgTopVer=AffinityTopologyVersion [topVer=282,
minorTopVer=0], groups=[CacheGroupContext
[grp=CACHEGROUP_PARTICLE_union-module_com.sbt.cdm.model.uss27.d
2018-11-19 18:59:01.898 [INFO
][sys-#3610%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.GridCachePartitionExchangeManager]
Finished sending full message [msgTopVer=AffinityTopologyVersion [topVer=282,
minorTopVer=0], groups=[CacheGroupContext
[grp=CACHEGROUP_PARTICLE_union-module_com.sbt.cdm.model.uss27.di
==[non-crd]==
2018-11-19 18:57:56.693 [INFO
][sys-#8210%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture]
Received full message, will finish exchange
[node=5f56aea6-3ecf-4a55-b2e1-cdafdb534e32, resVer=AffinityTopologyVersion
[topVer=282, minorTopVer=0]]
2018-11-19 18:57:57.466 [INFO
][sys-#8210%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.CacheAffinitySharedManager]
Affinity recalculation (on server join) performed in 773 ms.
2018-11-19 18:57:58.077 [INFO
][sys-#8210%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture]
Affinity changes applied in 1384 ms.
2018-11-19 18:57:58.210 [INFO
][sys-#8210%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture]
Finish exchange future [startVer=AffinityTopologyVersion [topVer=282,
minorTopVer=0], resVer=AffinityTopologyVersion [topVer=282, minorTopVer=0],
err=null]
{noformat}
'Finished sending full message' is printed out way too after message was sent
and seemingly received by other node.
That confuses PME analyse, please fix it
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-10384) Stop caches in parallel
Sergey Antonov created IGNITE-10384: --- Summary: Stop caches in parallel Key: IGNITE-10384 URL: https://issues.apache.org/jira/browse/IGNITE-10384 Project: Ignite Issue Type: Improvement Reporter: Sergey Antonov Assignee: Sergey Antonov Fix For: 2.8 Stopping 6000+ caches sequentially go on more 80 seconds. We must parallel this process. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10385) NPE in CachePartitionPartialCountersMap.toString
Anton Kurbanov created IGNITE-10385: --- Summary: NPE in CachePartitionPartialCountersMap.toString Key: IGNITE-10385 URL: https://issues.apache.org/jira/browse/IGNITE-10385 Project: Ignite Issue Type: Bug Affects Versions: 2.4 Reporter: Anton Kurbanov [exchange-worker-#44%gg%] ERROR org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture - Failed to reinitialize local partitions (preloading will be stopped): GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=411, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=0a755ac5-2bf2-4cd8-b73a-c2bd5f0e66ed, addrs=[10.131.0.164], sockAddrs=[/10.131.0.164:47500], discPort=47500, order=2, intOrder=2, lastExchangeTime=1542811944907, loc=false, ver=2.4.10#20180926-sha1:82656e70, isClient=false], topVer=411, nodeId8=bd022daa, msg=Node failed: TcpDiscoveryNode [id=0a755ac5-2bf2-4cd8-b73a-c2bd5f0e66ed, addrs=[10.131.0.164], sockAddrs=[/10.131.0.164:47500], discPort=47500, order=2, intOrder=2, lastExchangeTime=1542811944907, loc=false, ver=2.4.10#20180926-sha1:82656e70, isClient=false], type=NODE_FAILED, tstamp=1542812954959], nodeId=0a755ac5, evt=NODE_FAILED] org.apache.ignite.IgniteException: null at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toStringImpl(GridToStringBuilder.java:1032) ~[ignite-core-2.4.10.jar:2.4.10] at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:868) ~[ignite-core-2.4.10.jar:2.4.10] at org.apache.ignite.internal.managers.communication.GridIoMessage.toString(GridIoMessage.java:358) ~[ignite-core-2.4.10.jar:2.4.10] at java.lang.String.valueOf(String.java:2994) ~[?:1.8.0_171] at java.lang.StringBuilder.append(StringBuilder.java:131) ~[?:1.8.0_171] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2653) ~[ignite-core-2.4.10.jar:2.4.10] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:2586) ~[ignite-core-2.4.10.jar:2.4.10] at org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1642) ~[ignite-core-2.4.10.jar:2.4.10] at org.apache.ignite.internal.managers.communication.GridIoManager.sendToGridTopic(GridIoManager.java:1714) ~[ignite-core-2.4.10.jar:2.4.10] at org.apache.ignite.internal.processors.cache.GridCacheIoManager.send(GridCacheIoManager.java:1160) ~[ignite-core-2.4.10.jar:2.4.10] at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.sendLocalPartitions(GridDhtPartitionsExchangeFuture.java:1399) ~[ignite-core-2.4.10.jar:2.4.10] at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.sendPartitions(GridDhtPartitionsExchangeFuture.java:1506) ~[ignite-core-2.4.10.jar:2.4.10] at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.distributedExchange(GridDhtPartitionsExchangeFuture.java:1139) ~[ignite-core-2.4.10.jar:2.4.10] at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:703) [ignite-core-2.4.10.jar:2.4.10] at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2379) [ignite-core-2.4.10.jar:2.4.10] at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) [ignite-core-2.4.10.jar:2.4.10] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171] Caused by: org.apache.ignite.IgniteException at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toStringImpl(GridToStringBuilder.java:1032) ~[ignite-core-2.4.10.jar:2.4.10] at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:830) ~[ignite-core-2.4.10.jar:2.4.10] at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:787) ~[ignite-core-2.4.10.jar:2.4.10] at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:889) ~[ignite-core-2.4.10.jar:2.4.10] at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsSingleMessage.toString(GridDhtPartitionsSingleMessage.java:551) ~[ignite-core-2.4.10.jar:2.4.10] at java.lang.String.valueOf(String.java:2994) ~[?:1.8.0_171] at org.apache.ignite.internal.util.GridStringBuilder.a(GridStringBuilder.java:101) ~[ignite-core-2.4.10.jar:2.4.10] at org.apache.ignite.internal.util.tostring.SBLimitedLength.a(SBLimitedLength.java:88) ~[ignite-core-2.4.10.jar:2.4.10] at org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:943) ~[ignite-core-2.4.10.jar:
[GitHub] dspavlov opened a new pull request #81: IGNITE-10372 Optimize master trends page step 3: Tests migrated to new concept
dspavlov opened a new pull request #81: IGNITE-10372 Optimize master trends page step 3: Tests migrated to new concept URL: https://github.com/apache/ignite-teamcity-bot/pull/81 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ignite pull request #4768: IGNITE-9610 IgniteStandByClusterTest.testSimple i...
Github user ibessonov closed the pull request at: https://github.com/apache/ignite/pull/4768 ---
[jira] [Created] (IGNITE-10386) Add mode when WAL won't be disabled during rebalancing caused by BLT change
Ivan Rakov created IGNITE-10386: --- Summary: Add mode when WAL won't be disabled during rebalancing caused by BLT change Key: IGNITE-10386 URL: https://issues.apache.org/jira/browse/IGNITE-10386 Project: Ignite Issue Type: Improvement Reporter: Ivan Rakov Fix For: 2.8 Enabling IgniteSystemProperties#IGNITE_DISABLE_WAL_DURING_REBALANCING disables WAL for cache group during rebalancing in case local node has no OWNING partitions for this group. We should add mode when in specific case (after BaselineTopology change) WAL won't be disabled even if this property is switched on. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10387) Enable IgniteNodeAttributes#ATTR_MACS_OVERRIDE attribute in basic TC suites
Ivan Rakov created IGNITE-10387: --- Summary: Enable IgniteNodeAttributes#ATTR_MACS_OVERRIDE attribute in basic TC suites Key: IGNITE-10387 URL: https://issues.apache.org/jira/browse/IGNITE-10387 Project: Ignite Issue Type: Improvement Reporter: Ivan Rakov Fix For: 2.8 On TeamCity all node instances start on the same physical machine, which implies that ATTR_MACS attribute will be the same over the whole cluster. Some logic in Ignite depends on ATTR_MACS. Example is load balancing: GridCacheContext#selectAffinityNodeBalanced tries to pick node that has same MACs. This makes features like node balancing not fully covered by our tests. In order to improve tests coverage we should use ATTR_MACS_OVERRIDE attribute at least in basic tests suites. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Thin clients all in one
Stepan,
Thank you for your great job in evaluating Python thin client, as well
as other thin clients.
There was indeed a bug in Python client regarding the handling of type
hints in Collection type. I created a fix and did a PR under
IGNITE-10358 task, but the same PR is also fixes the problem in
IGNITE-10230 task.
As of handling the type mapping in gists you provided, I left comments
on both tasks.
Dmitry
On 11/21/18 6:37 PM, Stepan Pilschikov wrote:
Dmitry, Alexey
Thank you for help, this answers help me a lot with understanding how
clients are work
Not so long time ago i met problem which is have expected behavior, but its
may broke some workflows in future for some users
Its all about not specified data types in collections and map's
All description and examples in
https://issues.apache.org/jira/browse/IGNITE-10358
Dmitry, can you have a quick look at it and maybe in future somehow fix it?
пт, 26 окт. 2018 г. в 19:05, Dmitry Melnichuk <
dmitry.melnic...@nobitlost.com>:
Stepan!
TL/DR: what you got with Python client in your gist is an intended
behavior.
Explanation: As per docs, Object array contains of type ID (which is
defaults to -1) and an array of objects.
https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-object-array
Your gist might be fixed accordingly:
```
from pyignite import Client
from pyignite.datatypes import *
OBJECT_ARRAY_TYPE_ID = -1
OBJECT_ARRAY_CONTENTS = [1, 2]
client = Client()
client.connect('127.0.0.1', 10800)
cache = client.get_or_create_cache("PY_OBJECT_ARRAY")
cache.put(
1,
(OBJECT_ARRAY_TYPE_ID, OBJECT_ARRAY_CONTENTS),
key_hint=IntObject,
value_hint=ObjectArrayObject,
)
# Python output: print(cache.get(1))
# (-1, [1, 2])
```
The situation is similar with Map and Collection, they have types. Types
and type IDs are mostly useless in Python, but I left them for
interoperability reasons. If you think I should kick them out, just let
me know.
The usage of these 3 data types is documented and tested. You can refer
to the table in “Data types” section:
https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/datatypes/parsers.html
The tests are here:
https://github.com/apache/ignite/blob/master/modules/platforms/python/tests/test_datatypes.py#L116-L124
On 10/26/18 11:57 PM, Stepan Pilschikov wrote:
Hi, everyone
Create new thread to centralize cross compatibility and others common
problems between thin clients
Tying to use Object array to exchange different data between JS, PHP and
Python thin clients
JS and PHP simply can't put any type of arrays
Python can put data, but if you take it, data would be completely
different, maybe during this put process data is changed
Code and output:
PHP -
https://gist.github.com/pilshchikov/e887d31d4f2f36923470fead14c7801a
JS -
https://gist.github.com/pilshchikov/ba49067fd8924ebdf4414ec63838b86d
Python -
https://gist.github.com/pilshchikov/f4bbf76e31547e2dca7d4cc5d55bd24f
I'm tried different data types (string, double, float, complex objects,
just random objects, char, byte, Date), still
How, from my perspective, it should works:
put array of any type and then get this array.
Example: put [1,2,3] -> get [1,2,3] or put [new Date(), new Date()] ->
get [[Date object], [Date object]] (like in java thin client)
Looks like bug in all clients, what you think?
I wanted to try Collections, but i think this type have same problem
[jira] [Created] (IGNITE-10388) Prevent Ignite from "dying" if first attempt to join the cluster failed
Sergey Karpushin created IGNITE-10388:
-
Summary: Prevent Ignite from "dying" if first attempt to join the
cluster failed
Key: IGNITE-10388
URL: https://issues.apache.org/jira/browse/IGNITE-10388
Project: Ignite
Issue Type: Bug
Components: general
Affects Versions: 2.6
Environment: * Linux OS (I'm using Manjaro 1.18)
* Jdk 1.8
* Docker-compose 3.7
Reporter: Sergey Karpushin
I'm creating this issue as suggested on
[stackoverflow|https://stackoverflow.com/questions/53433749/how-to-prevent-ignite-from-dying-if-first-attempt-to-join-the-cluster-failed]
I've also created a simple [github
project|https://github.com/skarpushin/issue20181122] to help reproduce the
issue (please check README file before running the project).
I'm using {{docker-compose}} to start several containers with java web
application with embedded Ignite 2.6.
I configured Ignite so it will use SSL.
If I start node1 and wait it to fully start, and then start node2, they will
join cluster no problem.
But if I start them simultaneously (that's how it works by default with
docker-compose) they both will fail with exception and will never retry or get
back.
class org.apache.ignite.IgniteException: Unable to establish secure
connection. Was remote cluster configured with SSL? [rmtAddr=/10.9.9.1:47501,
errMsg="Remote host closed connection during handshake"] at
org.apache.ignite.spi.discovery.tcp.ServerImpl.sendMessageDirectly(ServerImpl.java:1293)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.sendJoinRequestMessage(ServerImpl.java:1046)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:890)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:373) at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:1948)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:915)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1721) at
org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1028) at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2014)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1723)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1151) at
org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:649) at
org.apache.ignite.IgniteSpring.start(IgniteSpring.java:66) at
org.apache.ignite.IgniteSpringBean.afterSingletonsInstantiated(IgniteSpringBean.java:172)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:777)
at
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:869)
at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550)
at
org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:409)
at
org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:291)
at
org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4853)
at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5314)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:753)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729) at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) at
org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:976) at
org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1853) at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
java.util.concurrent.FutureTask.run(FutureTask.java:266) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) Caused by:
javax.net.ssl.SSLHandshakeException: Remote host closed connection during
handshake at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1002)
at
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:757) at
sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123) at
java.io.OutputStream.write(OutputStream.java:75) at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.w
[MTCGA]: new failures in builds [2381922] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. If your changes can lead to this failure(s): We're grateful that you were a volunteer to make the contribution to this project, but things change and you may no longer be able to finalize your contribution. Could you respond to this email and indicate if you wish to continue and fix test failures or step down and some committer may revert you commit. *Recently contributed test failed in master TaskExamplesMultiNodeSelfTest.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-5917517976576215554&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master ClusterGroupExampleSelfTest.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-878488338790165&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master MonteCarloExamplesMultiNodeSelfTest.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-924688690363212384&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master ContinuationExamplesMultiNodeSelfTest.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=6047678023796560683&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master TaskExamplesSelfTest.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-4133967790505792702&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master DeploymentExamplesMultiNodeSelfTest.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-4750693290259760962&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master MonteCarloExamplesSelfTest.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-1617259290883754314&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master MemcacheRestExamplesSelfTest.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-4711279198236915182&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master ContinuationExamplesSelfTest.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-6026321494486034735&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master ContinuousMapperExamplesSelfTest.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=4470984311852304089&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master LifecycleExamplesSelfTest.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-4445737465195407541&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master IgfsExamplesSelfTest.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=8427106324084470669&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master DeploymentExamplesSelfTest.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=7397385867219657348&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master SpringBeanExamplesSelfTest.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-3682540866846716810&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master CheckpointExamplesSelfTest.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=7331710564801133366&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master MemcacheRestExamplesMultiNodeSelfTest.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-3180197102757276402&branch=%3Cdefault%3E&tab=testDetails *Recently contributed test failed in master ContinuousMapperExamplesMultiNodeSelfTest.initializationError https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-592872012226723313&branch=%3Cdefault%3E&tab=testDetails Changes may lead to failure were done by - oignatenko https://ci.ignite.apache.org/viewModification.html?modId=840076 - oignatenko https://ci.ignite.apache.org/viewModification.html?modId=840109 - aplatonovv https://ci.ignite.apache.org/viewModification.html?modId=840074 - dmitrievanthony https://ci.ignite.apache.org/viewModification.html?modId=839867 - somefireone
[MTCGA]: new failures in builds [2381990] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. If your changes can lead to this failure(s): We're grateful that you were a volunteer to make the contribution to this project, but things change and you may no longer be able to finalize your contribution. Could you respond to this email and indicate if you wish to continue and fix test failures or step down and some committer may revert you commit. *New test failure in master IgnitePdsDataRegionMetricsTest.testMemoryUsageMultipleNodes https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=1305663291495840872&branch=%3Cdefault%3E&tab=testDetails *New test failure in master IgnitePdsDynamicCacheTest.testMultipleDynamicCaches https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-4243164587486042948&branch=%3Cdefault%3E&tab=testDetails *New test failure in master IgniteDbDynamicCacheSelfTest.testMultipleDynamicCaches https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=564440028244951927&branch=%3Cdefault%3E&tab=testDetails *New test failure in master IgnitePdsCheckpointSimulationWithRealCpDisabledTest.testCheckpointSimulationMultiThreaded https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=1651231497806015150&branch=%3Cdefault%3E&tab=testDetails Changes may lead to failure were done by - oignatenko https://ci.ignite.apache.org/viewModification.html?modId=840076 - oignatenko https://ci.ignite.apache.org/viewModification.html?modId=840109 - aplatonovv https://ci.ignite.apache.org/viewModification.html?modId=840074 - dmitrievanthony https://ci.ignite.apache.org/viewModification.html?modId=839867 - somefireone https://ci.ignite.apache.org/viewModification.html?modId=840132 - kondakov87 https://ci.ignite.apache.org/viewModification.html?modId=839857 - oignatenko https://ci.ignite.apache.org/viewModification.html?modId=840115 - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 05:10:08 23-11-2018
[MTCGA]: new failures in builds [2381995] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. If your changes can lead to this failure(s): We're grateful that you were a volunteer to make the contribution to this project, but things change and you may no longer be able to finalize your contribution. Could you respond to this email and indicate if you wish to continue and fix test failures or step down and some committer may revert you commit. *New test failure in master IgnitePdsCheckpointSimulationWithRealCpDisabledTest.testCheckpointSimulationMultiThreaded https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-1615751728537497724&branch=%3Cdefault%3E&tab=testDetails *New test failure in master IgniteDbDynamicCacheSelfTest.testMultipleDynamicCaches https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-7398186543260636032&branch=%3Cdefault%3E&tab=testDetails *New test failure in master IgnitePdsDataRegionMetricsTest.testMemoryUsageMultipleNodes https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=4423108220031025514&branch=%3Cdefault%3E&tab=testDetails *New test failure in master CpTriggeredWalDeltaConsistencyTest.testPutRemoveCacheDestroy https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=8178121524675905397&branch=%3Cdefault%3E&tab=testDetails *New test failure in master IgnitePdsDynamicCacheTest.testMultipleDynamicCaches https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=7302780156883196251&branch=%3Cdefault%3E&tab=testDetails Changes may lead to failure were done by - oignatenko https://ci.ignite.apache.org/viewModification.html?modId=840076 - oignatenko https://ci.ignite.apache.org/viewModification.html?modId=840109 - aplatonovv https://ci.ignite.apache.org/viewModification.html?modId=840074 - dmitrievanthony https://ci.ignite.apache.org/viewModification.html?modId=839867 - somefireone https://ci.ignite.apache.org/viewModification.html?modId=840132 - kondakov87 https://ci.ignite.apache.org/viewModification.html?modId=839857 - oignatenko https://ci.ignite.apache.org/viewModification.html?modId=840115 - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 05:40:08 23-11-2018
[MTCGA]: new failures in builds [2381993] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. If your changes can lead to this failure(s): We're grateful that you were a volunteer to make the contribution to this project, but things change and you may no longer be able to finalize your contribution. Could you respond to this email and indicate if you wish to continue and fix test failures or step down and some committer may revert you commit. *New test failure in master IgnitePdsCacheDestroyDuringCheckpointTest.testCacheCreatePutCheckpointDestroy https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-7203383466441328541&branch=%3Cdefault%3E&tab=testDetails *New test failure in master IgniteLogicalRecoveryTest.testRecoveryOnJoinToDifferentBlt https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-6056172351868338567&branch=%3Cdefault%3E&tab=testDetails Changes may lead to failure were done by - oignatenko https://ci.ignite.apache.org/viewModification.html?modId=840076 - oignatenko https://ci.ignite.apache.org/viewModification.html?modId=840109 - aplatonovv https://ci.ignite.apache.org/viewModification.html?modId=840074 - dmitrievanthony https://ci.ignite.apache.org/viewModification.html?modId=839867 - somefireone https://ci.ignite.apache.org/viewModification.html?modId=840132 - kondakov87 https://ci.ignite.apache.org/viewModification.html?modId=839857 - oignatenko https://ci.ignite.apache.org/viewModification.html?modId=840115 - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 06:10:09 23-11-2018
[MTCGA]: new failures in builds [2381936] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. If your changes can lead to this failure(s): We're grateful that you were a volunteer to make the contribution to this project, but things change and you may no longer be able to finalize your contribution. Could you respond to this email and indicate if you wish to continue and fix test failures or step down and some committer may revert you commit. *New Critical Failure in master Queries 2 https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Queries2&branch=%3Cdefault%3E&tab=buildTypeStatusDiv Changes may lead to failure were done by - oignatenko https://ci.ignite.apache.org/viewModification.html?modId=840076 - oignatenko https://ci.ignite.apache.org/viewModification.html?modId=840109 - aplatonovv https://ci.ignite.apache.org/viewModification.html?modId=840074 - dmitrievanthony https://ci.ignite.apache.org/viewModification.html?modId=839867 - somefireone https://ci.ignite.apache.org/viewModification.html?modId=840132 - kondakov87 https://ci.ignite.apache.org/viewModification.html?modId=839857 - oignatenko https://ci.ignite.apache.org/viewModification.html?modId=840115 - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 06:55:10 23-11-2018
[MTCGA]: new failures in builds [2381981] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. If your changes can lead to this failure(s): We're grateful that you were a volunteer to make the contribution to this project, but things change and you may no longer be able to finalize your contribution. Could you respond to this email and indicate if you wish to continue and fix test failures or step down and some committer may revert you commit. *New test failure in master CheckpointBufferDeadlockTest.testOneCheckpointThread https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=2823482355827194921&branch=%3Cdefault%3E&tab=testDetails *New test failure in master CheckpointBufferDeadlockTest.testFourCheckpointThreads https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-3817933280317769007&branch=%3Cdefault%3E&tab=testDetails Changes may lead to failure were done by - oignatenko https://ci.ignite.apache.org/viewModification.html?modId=840076 - oignatenko https://ci.ignite.apache.org/viewModification.html?modId=840109 - aplatonovv https://ci.ignite.apache.org/viewModification.html?modId=840074 - dmitrievanthony https://ci.ignite.apache.org/viewModification.html?modId=839867 - somefireone https://ci.ignite.apache.org/viewModification.html?modId=840132 - kondakov87 https://ci.ignite.apache.org/viewModification.html?modId=839857 - oignatenko https://ci.ignite.apache.org/viewModification.html?modId=840115 - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 07:10:20 23-11-2018
Re: proposed realization KILL QUERY command
Hi, May be I am a little bit late with my thoughts about a command syntax. How do I see it is going to be used: 1. A user is able to kill a query by unique id belonging only to this query. 2. A user is able to kill all queries started by a specific node. For killing a single query we must identify it by unique id which is going to be received directly from Ignite (e.g. running queries view) and not calculated by user. Internally the id is compound but why cannot we convert it to opaque integer or string which hides real structure? E.g. base16String(concat(nodeOrder.toString(), ".", queryIdOnNode.toString())) The syntax could be KILL QUERY '123' or KILL QUERY WHERE queryId = '123' For killing all queries started by some node we need to use only node order (or id). It could be like KILL QUERY WHERE nodeOrder = 34. чт, 22 нояб. 2018 г. в 12:56, Denis Mekhanikov : > > Actually, option with separate parameters was mentioned in another thread > http://apache-ignite-developers.2346864.n4.nabble.com/proposed-design-for-thin-client-SQL-management-and-monitoring-view-running-queries-and-kill-it-tp37713p38056.html > > Denis > > чт, 22 нояб. 2018 г. в 08:51, Vladimir Ozerov : > > > Denis, > > > > Problems with separate parameters are explained above. > > > > чт, 22 нояб. 2018 г. в 3:23, Denis Magda : > > > > > Vladimir, > > > > > > All of the alternatives are reminiscent of mathematical operations. Don't > > > look like a SQL command. What if we use a SQL approach introducing named > > > parameters: > > > > > > KILL QUERY query_id=10 [AND node_id=5] > > > > > > -- > > > Denis > > > > > > On Wed, Nov 21, 2018 at 4:11 AM Vladimir Ozerov > > > wrote: > > > > > > > Denis, > > > > > > > > Space is bad candidate because it is a whitespace. Without whitespaces > > we > > > > can have syntax without quotes at all. Any non-whitespace delimiter > > will > > > > work, though: > > > > > > > > KILL QUERY 45.1 > > > > KILL QUERY 45-1 > > > > KILL QUERY 45:1 > > > > > > > > On Wed, Nov 21, 2018 at 3:06 PM Юрий > > > wrote: > > > > > > > > > Denis, > > > > > > > > > > Let's consider parameter of KILL QUERY just a string with some query > > > id, > > > > > without any meaning for user. User just need to get the id and pass > > as > > > > > parameter to KILL QUERY command. > > > > > > > > > > Even if query is distributed it have single query id from user > > > > perspective > > > > > and will killed on all nodes. User just need to known one global > > query > > > > id. > > > > > > > > > > How it can works. > > > > > 1)SELECT * from running_queries > > > > > result is > > > > > query_id | node_id > > > > > | sql | schema_name | connection_id | duration > > > > > 123.33 | e0a69cb8-a1a8-45f6-b84d-ead367a0 | SELECT ... | > > ... > > > > > | 22 | 23456 > > > > > 333.31 | aaa6acb8-a4a5-42f6-f842-ead111b00020 | UPDATE... | > > > ... > > > > > | 321| 346 > > > > > 2) KILL QUERY '123.33' > > > > > > > > > > So, user need select query_id from running_queries view and use it > > for > > > > KILL > > > > > QUERY command. > > > > > > > > > > I hope it became clearer. > > > > > > > > > > > > > > > > > > > > ср, 21 нояб. 2018 г. в 02:11, Denis Magda : > > > > > > > > > > > Folks, > > > > > > > > > > > > The decimal syntax is really odd - KILL QUERY > > > > > > '[node_order].[query_counter]' > > > > > > > > > > > > Confusing, let's use a space to separate parameters. > > > > > > > > > > > > Also, what if I want to halt a specific query with certain ID? > > Don't > > > > know > > > > > > the node number, just know that the query is distributed and runs > > > > across > > > > > > several machines. Sounds like the syntax still should consider > > > > > > [node_order/id] as an optional parameter. > > > > > > > > > > > > Probably, if you explain to me how an end user will use this > > command > > > > from > > > > > > the very beginning (how do I look for a query id and node id, etc) > > > then > > > > > the > > > > > > things get clearer. > > > > > > > > > > > > -- > > > > > > Denis > > > > > > > > > > > > On Tue, Nov 20, 2018 at 1:03 AM Юрий > > > > > wrote: > > > > > > > > > > > > > Hi Vladimir, > > > > > > > > > > > > > > Thanks for your suggestion to use MANAGEMENT_POOL for processing > > > > > > > cancellation requests. > > > > > > > > > > > > > > About your questions. > > > > > > > 1) I'm going to implements SQL view to provide list of running > > > > queries. > > > > > > The > > > > > > > SQL VIEW has been a little bit discussed earlier. Proposed name > > is > > > > > > > *running_queries* with following columns: query_id, node_id, sql, > > > > > > > schema_name, connection_id, duration. Currently most of the > > > > information > > > > > > can > > > > > > > be retrieved through cache API, however it doesn't matter, any > > > case > > > > we > > > > > > > need to expose SQL VIEW. Seem's you are right - the part should > > be > > > > >
Re: Rename "cache group" to "tablespace"?
Hi Vladimir,
How do you see a default tablespace configuration in the "bright"
future? Is it going to be a single file for an Ignite instance? In
what cases will be there benefits to explicitly configure additional
tablespaces?
By the way was the name "tablespace" hand-crafted or it was it
borrowed somewhere?
Also if it is TABLEspace should it be aligned with renaming
IgniteCache to IgniteTable discussed in [1]?
[1]
http://apache-ignite-developers.2346864.n4.nabble.com/Applicability-of-term-cache-to-Apache-Ignite-td36541.html
вт, 20 нояб. 2018 г. в 10:57, Dmitriy Setrakyan :
>
> I really like the idea.
>
> On Mon, Nov 19, 2018, 23:45 Vladimir Ozerov
> > Igniters,
> >
> > We had several discussion about cache groups [1]. Our current consensus is
> > that this is not that good design decision - bad scan performance, no API.
> > A kind of shortcut to solve memory consumption problems. For this reason we
> > try to avoid exposing "cache group" to API when possible. Also we
> > understand that current file-per-partition approach is not ideal either -
> > when there are too many partitions we have to fsync a lot of files. And as
> > an idea for "bright future" we consider tablespaces - one or several files
> > where all database objects are stored in dedicated "segments", which are
> > allocated in smaller units called "extents".
> >
> > I was thinking on how to "legalize" cache groups on API on the one hand
> > (product needs them currently), and to let real tablespaces to appear in
> > future. Idea is simple - treat cache group as special kind of tablespace.
> > We may say that current storage organization is one type of tablespace, and
> > proposed "file-segment-extent" storage organization is another type of
> > tablespace.
> >
> > With this in mind we can configure tablespaces in IgniteConfiguration, then
> > assign each cache a tablespace. As advanced tasks we may allow dynamic
> > table space create/alter/drop, I'll show SQL examples below.
> >
> > // Interface
> > interface Tablespace {
> > String name();
> > }
> >
> > // Current non-shared tablespace (aka "physical cache")
> > class FilePerPartitionTablespace implements Tablespace {
> > // ...
> > }
> >
> > // Shared tablespace (aka "cache group") - note that now we have a legal
> > place for cache group configuration:
> > class FilePerPartitionSharedTablespace implements Tablespace {
> > CacheMode cacheMode;
> > CacheAtomicityMode cacheAtomicityMode;
> > AffinityFunction affinityFunction;
> > // + Other parameters from
> > ClusterCachesInfo.validateCacheGroupConfiguration
> > // + Some parameters from "DataRegionConfiguration" (e.g. paths)
> > }
> >
> > // Future "real" tablespace
> > class SegmentedTablespace implements Tablespace {
> > // Whatever is needed.
> > }
> >
> > // Cache configuration
> > class CacheConfiguration {
> > ...
> > String tablespace;
> >...
> > }
> >
> > // Managing tablespaces from SQL:
> > CREATE TABLESPACE my_tbs ENGINE=FILE_PER_PARTITION_SHARED
> > CACHE_MODE=PARTITIONED BACKUPS=1
> > ALTER TABLESPACE my_tbs ENCRYPTED=1
> > CREATE TABLE my_table (...) TABLESPACE my_tbs
> >
> > What do you think?
> >
> > Vladimir.
> >
> > [1]
> >
> > http://apache-ignite-developers.2346864.n4.nabble.com/Remove-cache-groups-in-AI-3-0-td29184.html
> >
--
Best regards,
Ivan Pavlukhin
