[jira] [Created] (IGNITE-11724) IgniteSpark integration forget to close the IgniteContext and stops the client node in case if error during PairFunction logic
Andrey Aleksandrov created IGNITE-11724:
---
Summary: IgniteSpark integration forget to close the IgniteContext
and stops the client node in case if error during PairFunction logic
Key: IGNITE-11724
URL: https://issues.apache.org/jira/browse/IGNITE-11724
Project: Ignite
Issue Type: Bug
Components: spark
Affects Versions: 2.7
Reporter: Andrey Aleksandrov
Fix For: 2.8
Next code could hang in case if PairFunction logic will throw the exception:
JavaPairRDD rdd_records = records.mapToPair(new MapFunction());
JavaIgniteContext igniteContext = new JavaIgniteContext<>(sparkCtx,
configUrl);
JavaIgniteRDD igniteRdd = igniteContext.fromCache(cacheName);
igniteRdd.savePairs(rdd_records);
Looks like next internal code (saveValues method)should also close the
IgniteContext in case of an unexpected exception, not only data streamer:
try {
it.foreach(value ⇒ {
val key = affinityKeyFunc(value, node.orNull)
streamer.addData(key, value)
})
}
finally {
streamer.close()
}
})
}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-11723) IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame)
Andrey Aleksandrov created IGNITE-11723: --- Summary: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame) Key: IGNITE-11723 URL: https://issues.apache.org/jira/browse/IGNITE-11723 Project: Ignite Issue Type: Improvement Components: spark Affects Versions: 2.7 Reporter: Andrey Aleksandrov Assignee: Andrey Aleksandrov Fix For: 2.8 At the moment this option can't be set. But this integrations could be used for initial data loading also for the caches with cache stores implementation. With skipStore option, we could avoid write-through behavior during this initial data loading. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11722) Batch BPlustTree updates for cache batch operations
Semen Boikov created IGNITE-11722: - Summary: Batch BPlustTree updates for cache batch operations Key: IGNITE-11722 URL: https://issues.apache.org/jira/browse/IGNITE-11722 Project: Ignite Issue Type: Improvement Components: cache Reporter: Semen Boikov Fix For: 2.8 Now cache batch operations (putAll, invokeAll, removeAll) update BPlustTree entries one by one. It makes sense to support batch updates in BPlustTree and use it for cache batch operations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11721) IgniteJdbcDriver with streaming enabled doesn't use IgniteDataStreamer on executeBatch
Anton Kurbanov created IGNITE-11721: --- Summary: IgniteJdbcDriver with streaming enabled doesn't use IgniteDataStreamer on executeBatch Key: IGNITE-11721 URL: https://issues.apache.org/jira/browse/IGNITE-11721 Project: Ignite Issue Type: Bug Components: jdbc, streaming Affects Versions: 2.7 Reporter: Anton Kurbanov -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Data regions on client nodes
Hello, Ignite. Finally, I've implemented this ticket! :) Jira - https://issues.apache.org/jira/browse/IGNITE-9113 PR - https://github.com/apache/ignite/pull/6388 Tests seems to be OK. Anyone wants to take a look? В Чт, 09/08/2018 в 16:41 +0300, Alexey Goncharuk пишет: > Once the OS gave us the pointer, no exceptions can be raised in the user > code. If the OS overcommitted the memory, then the process will be halted > when OOME occurs, not much we can do here. > > My point was that dynamic data region allocation can lead to another > dynamic exception that should be properly handled during cache start. When > the data region is allocated statically, this exception is handled on node > start, which is much easier. > > ср, 8 авг. 2018 г. в 18:18, Dmitriy Pavlov : > > > I used to think that OS allocates pages not immediately after call to > > allocate(), but only during first touch of each page. > > > > I'm not sure every OS provides guaranee that 'allocated' memory will never > > cause OOME. Please correct me if I'm wrong. > > > > ср, 8 авг. 2018 г. в 17:38, Dmitriy Setrakyan : > > > > > Alexey, > > > > > > I am not sure I understand. If cache creation proceeds, but memory region > > > was not allocated, how can the cache operate? > > > > > > D. > > > > > > On Wed, Aug 8, 2018 at 8:05 AM, Alexey Goncharuk < > > > alexey.goncha...@gmail.com > > > > wrote: > > > > I do not mind making this change, but note that the reason for non-lazy > > > > region allocation is the need to gracefully handle OOME errors during > > > > > > cache > > > > start. When a region is pre-allocated, no OOME can happen. > > > > > > > > If a region is allocated dynamically, then all errors that happened > > > > > > during > > > > the node start before should be properly handled (a client node must be > > > > stopped, but cache creation should proceed). > > > > > > > > пт, 27 июл. 2018 г. в 20:04, Valentin Kulichenko < > > > > valentin.kuliche...@gmail.com>: > > > > > > > > > Ticket created: https://issues.apache.org/jira/browse/IGNITE-9113 > > > > > > > > > > -Val > > > > > > > > > > On Fri, Jul 27, 2018 at 5:59 AM Dmitry Pavlov > > > > wrote: > > > > > > > > > > > Maxim, thank you. > > > > > > > > > > > > If it seems it is technically possible, we can file ticket for this > > > > > > > > > > change. > > > > > > > > > > > > I find this proposal reasonable, change makes perfectly sense to > > > > me. > > > > > > > > > > > > We can wait Alex G. feedback on this change before starting actual > > > > > > implementation. It can take for a while, because he is travelling > > > > > > now. > > > > > > > > > > > > пт, 27 июл. 2018 г. в 14:35, Maxim Muzafarov : > > > > > > > > > > > > > Guys, > > > > > > > > > > > > > > I can miss some details, but at the first glance we have > > > > everething > > > > we > > > > > > need > > > > > > > to defer > > > > > > > region memory allocation if it has no cache groups assignments. > > > > And > > > > it > > > > > > > doesn't matter > > > > > > > where it happens on client or server nodes. > > > > > > > > > > > > > > Currently region memory allocation happens at exchange future > > > > init > > > > > > method. > > > > > > > At the > > > > > > > node startup method initCachesOnLocalJoin executes. This method > > > > > > > > > > > > resposnible > > > > > > > for > > > > > > > memory allocation (through initiating cache managers) and it also > > > > > > > > > > starts > > > > > > > caches. > > > > > > > So, at this point we have all existing caches descriptors and can > > > > > > > > find > > > > > > out > > > > > > > which > > > > > > > cache matches which region to defer some regions initialization > > > > to > > > > the > > > > > > > moment when > > > > > > > newly cache assings to this region (happend at > > > > > > onCacheChangeRequest). > > > > > > > > > > > > > > Please, сorrect me if I'm wrong and missing something. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, 25 Jul 2018 at 19:32 Dmitry Pavlov < > > > > dpavlov@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > Hi Maxim, > > > > > > > > > > > > > > > > thank you for stepping in. How do you think, is it possible to > > > > > > > > check > > > > > > > cache > > > > > > > > assignment to region at stage of memory allocation? > > > > > > > > > > > > > > > > Sincerely, > > > > > > > > Dmitriy Pavlov > > > > > > > > > > > > > > > > ср, 25 июл. 2018 г. в 18:22, Maxim Muzafarov < > > > > maxmu...@gmail.com > > > > : > > > > > > > > > > > > > > > > > Folks, > > > > > > > > > > > > > > > > > > I've checked memory allocation. It looks like we are > > > > allocating > > > > > > memory > > > > > > > > only > > > > > > > > > on the first exchange future init on local join occurs on > > > > node. > > > > > Also, > > > > > > > > seems > > > > > > > > > like we are allocating only the first chunk of memory (not > > > > the > > > > > whole > > > > > > > > bunch) > > > > > > > > >
[jira] [Created] (IGNITE-11720) Document new SQL system view "COLUMNS"
Taras Ledkov created IGNITE-11720: - Summary: Document new SQL system view "COLUMNS" Key: IGNITE-11720 URL: https://issues.apache.org/jira/browse/IGNITE-11720 Project: Ignite Issue Type: Task Components: documentation Affects Versions: 2.7 Reporter: Taras Ledkov Fix For: 2.8 See the ticket IGNITE-11434 for list of view's columns. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11719) Document cluster read-only mode
Sergey Antonov created IGNITE-11719: --- Summary: Document cluster read-only mode Key: IGNITE-11719 URL: https://issues.apache.org/jira/browse/IGNITE-11719 Project: Ignite Issue Type: New Feature Components: documentation Reporter: Sergey Antonov Assignee: Artem Budnikov Fix For: 2.8 We should document cluster wide read-only mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11718) Authorizing of SERVICE_INVOKE permission
Denis Garus created IGNITE-11718: Summary: Authorizing of SERVICE_INVOKE permission Key: IGNITE-11718 URL: https://issues.apache.org/jira/browse/IGNITE-11718 Project: Ignite Issue Type: Improvement Affects Versions: 2.7 Reporter: Denis Garus We authorize SERVICE_INVOKE only once on creation of service instance however, we have to authorize SERVICE_INVOKE whenever an operation is invoked on the service instance. If we change some permissions in runtime right after a client got instance, it doesn't see those changes. The scenario: * Client A creates a service instance B. * Administrator forbids invocation of service B to client A. * As long as client A keeps service instance, it can invoke any methods of service B. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Request for contributors permissions
Hi. Fix Version is time-to-time set before reviewing a patch, but for open tickets fix version does not mean too much. Open tickets migrated from version being released to the next one automatically. Currently, all fixed issues go to 2.8 by default, and some of them were assigned to 2.7.5 (scope is closed now). ср, 10 апр. 2019 г. в 12:04, Oleksii Mohylin : > Nope, I meant whether fixVersion was set before or as part of or after > patch review. From your answer I understand that correct answer is: after > patch review. > > - > Oleksii > > -Original Message- > From: Ilya Kasnacheev > Sent: Wednesday, April 10, 2019 12:00 PM > To: dev@ignite.apache.org > Subject: Re: Request for contributors permissions > > Hello! > > Usually you set fixVersion when your ticket is committed to master and it > becomes obvious. > > It's not like you can force community to do a ticket by setting urgent > fixVersion, if that's what you are asking. > > Regards, > -- > Ilya Kasnacheev > > > ср, 10 апр. 2019 г. в 11:34, Oleksii Mohylin : > > > Thank you! Just a follow-up question, who and when can decide on > > fixVersion of a ticket? Is that something contributors can do, or > reviewers > > or someone else? I couldn't find this in wiki. > > > > - > > Oleksii > > > > -Original Message- > > From: Denis Magda > > Sent: Wednesday, April 10, 2019 2:58 AM > > To: dev > > Subject: Re: Request for contributors permissions > > > > Done, you're in. Feel free to take over open tickets/issues. > > > > - > > Denis > > > > > > On Tue, Apr 9, 2019 at 1:33 PM wrote: > > > > > Sure. > > > > > > Username: olemogyl > > > Email is same: olek...@reseguiden.se > > > > > > - > > > Oleksii > > > > > > -Original Message- > > > From: Denis Magda > > > Sent: Tuesday, April 9, 2019 8:22 PM > > > To: dev > > > Subject: Re: Request for contributors permissions > > > > > > Hello Oleksii, > > > > > > Sure, please tell your JIRA ID. > > > > > > - > > > Denis > > > > > > > > > On Tue, Apr 9, 2019 at 10:18 AM Oleksii Mohylin > > > > > > wrote: > > > > > > > Hello > > > > > > > > > > > > > > > > I would like to become a contributor of Apache Ignite project. I'm > > > > working on an OSGI application project with cache implementation by > > > > Apache Ignite and have already discovered some issues. Could you > > > > please assign my account to proper user group in Jira? > > > > > > > > > > > > > > > > Thanks! > > > > > > > > Oleksii > > > > > > > > > > > > > > > > > > > >
[jira] [Created] (IGNITE-11717) Web console: project generation runtime error caused by IGFS
Ilya Borisov created IGNITE-11717: - Summary: Web console: project generation runtime error caused by IGFS Key: IGNITE-11717 URL: https://issues.apache.org/jira/browse/IGNITE-11717 Project: Ignite Issue Type: Bug Components: wizards Reporter: Ilya Borisov Assignee: Vasiliy Sisko Attachments: image-2019-04-10-17-52-48-599.png *What happens:* Runtime error in console. !image-2019-04-10-17-52-48-599.png! How to reproduce: 1. Go to cluster configuration/create igfs. 2. Choose "Caching" FS factory. 3. Choose "Chained" user name mapper. 4. Open new mapper sub form, choose custom. 5. Change FS factory a couple of time, change it back to "Caching". 6. Do the same with with "Name mapper" select, change it back to "Custom". 7. If the error does not happen, play around by changing other select values. *Expected behavior:* Run time error should not happen. This happens in master. [~vsisko] I think you should look into why `bean` is `null` or handle this case gracefully. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11716) Web console: can't select cluster memory eviction mode
Ilya Borisov created IGNITE-11716: - Summary: Web console: can't select cluster memory eviction mode Key: IGNITE-11716 URL: https://issues.apache.org/jira/browse/IGNITE-11716 Project: Ignite Issue Type: Bug Components: wizards Reporter: Ilya Borisov Assignee: Ilya Borisov Attachments: image-2019-04-10-17-02-17-310.png What happens: Memory eviction mode select has no options to choose from and there's an error in console. !image-2019-04-10-17-02-17-310.png! Expected behavior: The error should not happen, user should be able to select eviction mode. Note: select configuration version 2.1 to see "memory" section in advanced cluster configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11715) Add support for nojmx flag when running Ignite in docker
Kresimir Horvat created IGNITE-11715: Summary: Add support for nojmx flag when running Ignite in docker Key: IGNITE-11715 URL: https://issues.apache.org/jira/browse/IGNITE-11715 Project: Ignite Issue Type: Wish Reporter: Kresimir Horvat Add option to run ignite with -nojmx when running Ignite in docker container. Currently run.sh script doesn't support passing -nojmx option. Desired result is to run ignite with -nojmx flag and possibility of passing JMX_MON as env variable over docker run. More details about problem are described in thread [http://apache-ignite-users.70518.x6.nabble.com/Running-control-sh-in-docker-container-when-JXM-is-started-td27812.html] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
RE: Request for contributors permissions
Nope, I meant whether fixVersion was set before or as part of or after patch review. From your answer I understand that correct answer is: after patch review. - Oleksii -Original Message- From: Ilya Kasnacheev Sent: Wednesday, April 10, 2019 12:00 PM To: dev@ignite.apache.org Subject: Re: Request for contributors permissions Hello! Usually you set fixVersion when your ticket is committed to master and it becomes obvious. It's not like you can force community to do a ticket by setting urgent fixVersion, if that's what you are asking. Regards, -- Ilya Kasnacheev ср, 10 апр. 2019 г. в 11:34, Oleksii Mohylin : > Thank you! Just a follow-up question, who and when can decide on > fixVersion of a ticket? Is that something contributors can do, or reviewers > or someone else? I couldn't find this in wiki. > > - > Oleksii > > -Original Message- > From: Denis Magda > Sent: Wednesday, April 10, 2019 2:58 AM > To: dev > Subject: Re: Request for contributors permissions > > Done, you're in. Feel free to take over open tickets/issues. > > - > Denis > > > On Tue, Apr 9, 2019 at 1:33 PM wrote: > > > Sure. > > > > Username: olemogyl > > Email is same: olek...@reseguiden.se > > > > - > > Oleksii > > > > -Original Message- > > From: Denis Magda > > Sent: Tuesday, April 9, 2019 8:22 PM > > To: dev > > Subject: Re: Request for contributors permissions > > > > Hello Oleksii, > > > > Sure, please tell your JIRA ID. > > > > - > > Denis > > > > > > On Tue, Apr 9, 2019 at 10:18 AM Oleksii Mohylin > > > > wrote: > > > > > Hello > > > > > > > > > > > > I would like to become a contributor of Apache Ignite project. I'm > > > working on an OSGI application project with cache implementation by > > > Apache Ignite and have already discovered some issues. Could you > > > please assign my account to proper user group in Jira? > > > > > > > > > > > > Thanks! > > > > > > Oleksii > > > > > > > > > > > >
[jira] [Created] (IGNITE-11714) Web console: multiple select items overflow
Ilya Borisov created IGNITE-11714: - Summary: Web console: multiple select items overflow Key: IGNITE-11714 URL: https://issues.apache.org/jira/browse/IGNITE-11714 Project: Ignite Issue Type: Bug Components: wizards Reporter: Ilya Borisov Assignee: Ilya Borisov Attachments: image-2019-04-10-16-00-55-734.png *What happens:* !image-2019-04-10-16-00-55-734.png! *Expected behavior:* Overflow is handled gracefully, value is contained in control. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Request for contributors permissions
Hello! Usually you set fixVersion when your ticket is committed to master and it becomes obvious. It's not like you can force community to do a ticket by setting urgent fixVersion, if that's what you are asking. Regards, -- Ilya Kasnacheev ср, 10 апр. 2019 г. в 11:34, Oleksii Mohylin : > Thank you! Just a follow-up question, who and when can decide on > fixVersion of a ticket? Is that something contributors can do, or reviewers > or someone else? I couldn't find this in wiki. > > - > Oleksii > > -Original Message- > From: Denis Magda > Sent: Wednesday, April 10, 2019 2:58 AM > To: dev > Subject: Re: Request for contributors permissions > > Done, you're in. Feel free to take over open tickets/issues. > > - > Denis > > > On Tue, Apr 9, 2019 at 1:33 PM wrote: > > > Sure. > > > > Username: olemogyl > > Email is same: olek...@reseguiden.se > > > > - > > Oleksii > > > > -Original Message- > > From: Denis Magda > > Sent: Tuesday, April 9, 2019 8:22 PM > > To: dev > > Subject: Re: Request for contributors permissions > > > > Hello Oleksii, > > > > Sure, please tell your JIRA ID. > > > > - > > Denis > > > > > > On Tue, Apr 9, 2019 at 10:18 AM Oleksii Mohylin > > > > wrote: > > > > > Hello > > > > > > > > > > > > I would like to become a contributor of Apache Ignite project. I'm > > > working on an OSGI application project with cache implementation by > > > Apache Ignite and have already discovered some issues. Could you > > > please assign my account to proper user group in Jira? > > > > > > > > > > > > Thanks! > > > > > > Oleksii > > > > > > > > > > > >
[jira] [Created] (IGNITE-11713) Web console: cluster configuration events content fit issue
Ilya Borisov created IGNITE-11713: - Summary: Web console: cluster configuration events content fit issue Key: IGNITE-11713 URL: https://issues.apache.org/jira/browse/IGNITE-11713 Project: Ignite Issue Type: Bug Components: wizards Reporter: Ilya Borisov Assignee: Vasiliy Sisko Attachments: image-2019-04-10-15-46-39-859.png *What happens:* In master, cluster configuration (2.8) / events / local events listeners individual event listnered item sub form controls don't fit well when viewport width is under 1420px: !image-2019-04-10-15-46-39-859.png! Expected behavior: Form controls should be laid out such they look don't overflow/wrap. As a solution, I propose to put each control into separate row. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
RE: Request for contributors permissions
Thank you! Just a follow-up question, who and when can decide on fixVersion of a ticket? Is that something contributors can do, or reviewers or someone else? I couldn't find this in wiki. - Oleksii -Original Message- From: Denis Magda Sent: Wednesday, April 10, 2019 2:58 AM To: dev Subject: Re: Request for contributors permissions Done, you're in. Feel free to take over open tickets/issues. - Denis On Tue, Apr 9, 2019 at 1:33 PM wrote: > Sure. > > Username: olemogyl > Email is same: olek...@reseguiden.se > > - > Oleksii > > -Original Message- > From: Denis Magda > Sent: Tuesday, April 9, 2019 8:22 PM > To: dev > Subject: Re: Request for contributors permissions > > Hello Oleksii, > > Sure, please tell your JIRA ID. > > - > Denis > > > On Tue, Apr 9, 2019 at 10:18 AM Oleksii Mohylin > > wrote: > > > Hello > > > > > > > > I would like to become a contributor of Apache Ignite project. I'm > > working on an OSGI application project with cache implementation by > > Apache Ignite and have already discovered some issues. Could you > > please assign my account to proper user group in Jira? > > > > > > > > Thanks! > > > > Oleksii > > > > > >
[jira] [Created] (IGNITE-11712) SQL: review security check for SQL/DML queries
Roman Kondakov created IGNITE-11712: --- Summary: SQL: review security check for SQL/DML queries Key: IGNITE-11712 URL: https://issues.apache.org/jira/browse/IGNITE-11712 Project: Ignite Issue Type: Improvement Components: sql Reporter: Roman Kondakov Currently the security check (read/write permissions) is carried out during the query execution. It involves some extra actions (like a query registration) which can be avoided if the security check is conducted on the earlier stage of the query execution, for example right away after the parsing. For SELECT queries only read permission should be checked. For INSERT queries without SELECT only write permission should be checked. For UPDATE queries or INSERT queries with SELECT both read and write permissions should be checked. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
