Review needed: IGNITE-13420, Add assertion message to assert in CachedDeploymentInfo private constructor

2020-09-09 Thread Konstantin Sirotkin 
Hi!

Please review the changes.

https://issues.apache.org/jira/browse/IGNITE-13420 


Thanks!

---
Kind Regards,
Konstantin Sirotkin

Re: Proposal of new event QUERY_EXECUTION_EVENT

2020-09-09 Thread Denis Magda 
Ignite SQL experts, could you please step in?

-
Denis


On Mon, Sep 7, 2020 at 1:53 PM Dmitrii Ryabov  wrote:

> Any objections to create a separate event, which will be fired before
> executing a query?
>
> ср, 2 сент. 2020 г. в 22:33, Dmitrii Ryabov :
> >
> > I agree with Max, we need to add a separate event for starting query
> > execution, and EVT_CACHE_QUERY_EXECUTED shouldn't be deprecated,
> > because it is another case - it is fired when cache query was
> > successfully finished.
> >
> > > Would the event notification be synchronous? In which thread?
> > As Max said, synchronicity depends on implementation. As I see, we
> > don't use a separate thread for any record calls.
> >
> > > What happens in case the event listener fails?
> > Exceptions are logged by `U.error(...)` call. Errors are thrown out.
> >
> > > Should we discuss this within this topic?
> > I suggest to separate adding a new event and configuring existing events.
> >
> > пн, 20 июл. 2020 г. в 14:37, Max Timonin :
> > >
> > > Looks like EVT_CACHE_QUERY_EXECUTED just works for different use cases:
> > > 1. it relates to a specific cache (Event for SQL queries looks
> different as
> > > it could contain multiple caches or none of them);
> > > 2. Also the EVT_CACHE_QUERY_EXECUTED event fires multiple times for
> > > distributed queries, see GridMapQueryExecutor class (For SQL query it's
> > > required to fire once independently on how many nodes are affected).
> > >
> > > So there are different patterns for events. I think
> > > EVT_CACHE_QUERY_EXECUTED should not be deprecated or changed.
> > >
> > > > What happens in case the event listener fails?
> > > > Would the event notification be synchronous?
> > > It depends on how other events are implemented. As I see for the
> > > EVT_CACHE_QUERY_EXECUTED event - it's synchronous, and listener errors
> > > aren't handled.
> > >
> > > I think these questions are related to GridEventStorageManager as it
> just
> > > provides an API for recording events. Internal implementations (sync
> > > / async, error handling) is not related to an event, is it?
> > >
> > > > I have some doubts about provide text of a query even with
> > > hidden arguments, probably it should be configured due to it could lead
> > > to security leak
> > > Currently event EVT_CACHE_QUERY_EXECUTED provides a sql clause without
> > > limitations. If we're going to provide some restrictions it will
> require
> > > additional investigation. I see at least 2 configurations here:
> > > 1. Ignite can be configured to hide clase, params only or nothing for
> all
> > > listeners;
> > > 2. Only authorized listeners can subscribe to the event.
> > >
> > > Should we discuss this within this topic?
> > >
> > > On Mon, Jul 20, 2020 at 1:55 PM Юрий 
> wrote:
> > >
> > > > In my opinion existing events EVT_CACHE_QUERY_EXECUTION_STARTED
> should be
> > > > deprecated and added two new for start and for end of queries which
> should
> > > > cover all SQL query types.
> > > > I have some doubts about provide text of a query even with hidden
> > > > arguments, probably it should be configured due to it could lead to
> > > > security leak
> > > >
> > > > пн, 20 июл. 2020 г. в 12:49, Stanislav Lukyanov <
> stanlukya...@gmail.com>:
> > > >
> > > > > Maksim,
> > > > >
> > > > > Can we change the EVT_CACHE_QUERY_EXECUTED to fire earlier? Or
> should
> > > > > there be an EVT_CACHE_QUERY_EXECUTION_STARTED for the query start,
> while
> > > > > the old event would continue to work for query finish?
> > > > > I think the new event needs to either reuse the old one, or be its
> mirror
> > > > > for the query start. It also means that we probably should resolve
> the
> > > > > issues you've listed.
> > > > >
> > > > > Would the event notification be synchronous? In which thread?
> > > > Asynchronous
> > > > > is generally preferred - would it work?
> > > > >
> > > > > What happens in case the event listener fails?
> > > > >
> > > > > Thanks,
> > > > > Stan
> > > > >
> > > > > > On 16 Jul 2020, at 18:49, Denis Magda  wrote:
> > > > > >
> > > > > > Taras, Yury, Ivan,
> > > > > >
> > > > > > Could you please join this thread and share your thoughts? Do we
> > > > already
> > > > > > have any plans on tracking of the DDL and DML queries?
> > > > > >
> > > > > > -
> > > > > > Denis
> > > > > >
> > > > > >
> > > > > > On Wed, Jul 15, 2020 at 12:09 AM Max Timonin <
> timonin.ma...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > >> Hi Denis, thanks for the answer!
> > > > > >>
> > > > > >> We already checked EVT_CACHE_QUERY_EXECUTED and found that it
> works
> > > > > only in
> > > > > >> cases:
> > > > > >> 1. Scan queries and Select queries (common pattern is access to
> cache
> > > > > >> data);
> > > > > >> 2. This event triggers only if query execution succeeds, in
> case of
> > > > > failure
> > > > > >> while execution this event won't fire.
> > > > > >>
> > > > > >> Our additional requirements are to protocol queries:
> > > > > >> 1. that aren't cac

Re: IEP-54: Schema-first approach for 3.0

2020-09-09 Thread Denis Magda 
Val, makes sense, thanks for explaining.

Agree that we need to have a separate discussion thread for the "table" and
"cache" terms substitution. I'll appreciate it if you start the thread
sharing pointers to any relevant IEPs and reasoning behind the suggested
change.

-
Denis


On Tue, Sep 8, 2020 at 6:01 PM Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Hi Denis,
>
> I guess the wording in the IEP is a little bit confusing. All it means is
> that you should not create nested POJOs, but rather inline fields into a
> single POJO that is mapped to a particular schema. In other words, nested
> POJOs are not supported.
>
> Alex, is this correct? Please let me know if I'm missing something.
>
> As for the "cache" term, I agree that it is outdated, but I'm not sure
> what we can replace it with. "Table" is tightly associated with SQL, but
> SQL is optional in our case. Do you want to create a separate discussion
> about this?
>
> -Val
>
> On Tue, Sep 8, 2020 at 4:37 PM Denis Magda  wrote:
>
>> Val,
>>
>> I've checked the IEP again and have a few questions.
>>
>> Arbitrary nested objects and collections are not allowed as column values.
>> > Nested POJOs should either be inlined into schema, or stored as BLOBs
>>
>>
>> Could you provide a DDL code snippet showing how the inlining of POJOs is
>> supposed to work?
>>
>> Also, we keep using the terms "cache" and "table" throughout the IEP. Is
>> it
>> the right time to discuss an alternate name that would replace those too?
>> Personally, the "table" should stay and the "cache" should go considering
>> that SQL is one of the primary APIs in Ignite and that DDL is supported
>> out-of-the-box.
>>
>>
>> -
>> Denis
>>
>>
>> On Mon, Sep 7, 2020 at 12:26 PM Valentin Kulichenko <
>> valentin.kuliche...@gmail.com> wrote:
>>
>> > Ivan,
>> >
>> > I see your point. I agree that with the automatic updates we step into
>> the
>> > schema-last territory.
>> >
>> > Actually, if we support automatic evolution, we can as well support
>> > creating a cache without schema and inferring it from the first insert.
>> In
>> > other words, we can have both "schema-first" and "schema-last" modes.
>> >
>> > Alexey, what do you think?
>> >
>> > -Val
>> >
>> > On Mon, Sep 7, 2020 at 5:59 AM Alexey Goncharuk <
>> > alexey.goncha...@gmail.com>
>> > wrote:
>> >
>> > > Ivan,
>> > >
>> > > Thank you, I got your concern now. As it is mostly regarding the
>> > > terminology, I am absolutely fine with changing the name to whatever
>> fits
>> > > the approach best. Dynamic or evolving schema sounds great. I will
>> make
>> > > corresponding changes to the IEP once we settle on the name.
>> > >
>> > > пн, 7 сент. 2020 г. в 11:33, Ivan Pavlukhin :
>> > >
>> > > > Hi Val,
>> > > >
>> > > > Thank you for your answer!
>> > > >
>> > > > My understanding is a little bit different. Yes, schema evolution
>> > > > definitely should be possible. But I see a main difference in "how
>> > > > schema is updated". I treat a common SQL approach schema-first.
>> Schema
>> > > > and data manipulation operations are clearly separated and it
>> enables
>> > > > interesting capabilities, e.g. preventing untended schema changes by
>> > > > mistaken data operations, restricting user permissions to change
>> > > > schema.
>> > > >
>> > > > > Schema-first means that schema exists in advance and all the
>> stored
>> > > data
>> > > > is compliant with it - that's exactly what is proposed.
>> > > >
>> > > > A schema-last approach mentioned in [1] also assumes that schema
>> > > > exists, but it is inferred from data. Is not it more similar to the
>> > > > proposing approach?
>> > > >
>> > > > And I would like to say, that my main concern so far is mostly about
>> > > > terminology. And I suppose if it confuses me then others might be
>> > > > confused as well. My feeling is closer to "dynamic or liquid or may
>> be
>> > > > evolving schema".
>> > > >
>> > > > [1]
>> > > >
>> > https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf
>> > > >
>> > > > 2020-09-07 0:47 GMT+03:00, Valentin Kulichenko <
>> > > > valentin.kuliche...@gmail.com>:
>> > > > > Hi Ivan,
>> > > > >
>> > > > > I don't see an issue with that. Schema-first means that schema
>> exists
>> > > in
>> > > > > advance and all the stored data is compliant with it - that's
>> exactly
>> > > > what
>> > > > > is proposed. There are no restrictions prohibiting changes to the
>> > > schema.
>> > > > >
>> > > > > -Val
>> > > > >
>> > > > > On Sat, Sep 5, 2020 at 9:52 PM Ivan Pavlukhin <
>> vololo...@gmail.com>
>> > > > wrote:
>> > > > >
>> > > > >> Alexey,
>> > > > >>
>> > > > >> I am a little bit confused with terminology. My understanding
>> > conforms
>> > > > >> to a survey [1] (see part X Semi Structured Data). Can we really
>> > treat
>> > > > >> a "dynamic schema" approach as a kind of "schema-first"?
>> > > > >>
>> > > > >> [1]
>> > > > >>
>> > >
>> https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf
>> > > > >>
>

[jira] [Created] (IGNITE-13423) Add JMX SQLGlobalMemoryQuotaBytes and SQLQueryMemoryQuotaBytes to return corresponding attributes as long

2020-09-09 Thread Alexander Korenshteyn (Jira) 
Alexander Korenshteyn created IGNITE-13423:
--

 Summary: Add JMX  SQLGlobalMemoryQuotaBytes and 
SQLQueryMemoryQuotaBytes to return corresponding attributes as long
 Key: IGNITE-13423
 URL: https://issues.apache.org/jira/browse/IGNITE-13423
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexander Korenshteyn
Assignee: Alexander Korenshteyn


currently the prometheus JMX exporter can't parse strings: 

and therefore SQLGlobalMemoryQuota and SQLQueryMemoryQuota are not shown by 
default

Add SQLGlobalMemoryQuotaBytes and SQLQueryMemoryQuotaBytes that would return 
the corresponding attributes as bytes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSSION] control.sh argument to enable experimental commands

2020-09-09 Thread Nikolay Izhikov 
Hello, Igniters.

Ticket [1] to implement explicit option to enable experimental commands created.

[1] https://issues.apache.org/jira/browse/IGNITE-13422

> 7 сент. 2020 г., в 14:08, Philipp Masharov  написал(а):
> 
> Nikolay, thanks for mentioning it. I fetched the last changes and got this:
> 
> Print metadata command help:
>control.(sh|bat) --meta help
> 
>  Print list of binary metadata types:
>control.(sh|bat) --meta list
> 
>  Print detailed info about specified binary type (the type must be
> specified by type name or by type identifier):
>control.(sh|bat) --meta details [--typeId ] [--typeName
> ]
> 
>  Remove the metadata of the specified type (the type must be specified by
> type name or by type identifier) from cluster and saves the removed
> metadata to the specified file.
> If the file name isn't specified the output file name is: '.bin'
>control.(sh|bat) --meta remove [--typeId ] [--typeName
> ] [--out ]
> 
>  Update cluster metadata from specified file (file name is required)
>control.(sh|bat) --meta update --in 
> 
> Print tracing configuration:
>control.(sh|bat) --tracing-configuration
> 
>  Print tracing configuration:
>control.(sh|bat) --tracing-configuration get_all [--scope
> DISCOVERY|EXCHANGE|COMMUNICATION|TX]
> 
>  Print specific tracing configuration based on specified --scope and
> --label:
>control.(sh|bat) --tracing-configuration get (--scope
> DISCOVERY|EXCHANGE|COMMUNICATION|TX) [--label]
> 
>  Reset all specific tracing configuration the to default. If --scope is
> specified, then remove all label specific configuration for the given scope
> and reset given scope specific configuration to the default, if --scope is
> skipped then reset all tracing configurations to the default. Print tracing
> configuration.
>control.(sh|bat) --tracing-configuration reset_all [--scope
> DISCOVERY|EXCHANGE|COMMUNICATION|TX]
> 
>  Reset specific tracing configuration to the default. If both --scope and
> --label are specified then remove given configuration, if only --scope is
> specified then reset given configuration to the default. Print reseted
> configuration.
>control.(sh|bat) --tracing-configuration reset (--scope
> DISCOVERY|EXCHANGE|COMMUNICATION|TX) [--label]
> 
>  Set new tracing configuration. If both --scope and --label are specified
> then add or override label specific configuration, if only --scope is
> specified, then override scope specific configuration. Print applied
> configuration.
>control.(sh|bat) --tracing-configuration set (--scope
> DISCOVERY|EXCHANGE|COMMUNICATION|TX [--label] [--sampling-rate Decimal
> value between 0 and 1, where 0 means never and 1 means always. More or less
> reflects the probability of sampling specific trace.] [--included-scopes
> Set of scopes with comma as separator  DISCOVERY|EXCHANGE|COMMUNICATION|TX])
> 
> 
> On Mon, Sep 7, 2020 at 1:44 PM Nikolay Izhikov  wrote:
> 
>> Philipp, at least, we have tracing and binary meta commands in the master.
>> 
>> Please, take a look at current Ignite sources.
>> 
>>> 7 сент. 2020 г., в 13:34, Nikolay Izhikov 
>> написал(а):
>>> 
>>> What is wrong with that?
>>> 
>>> We just improve the way user enables experimental commands.
>>> JVM uses similar approach, please, take a look at
>> '-XX:+UnlockExperimentalVMOptions’.
>>> 
 7 сент. 2020 г., в 13:22, ткаленко кирилл 
>> написал(а):
 
 Hi, Nikolay!
 
 It seems that you shouldn't just open experimental commands, that's why
>> they are experimental.
 
 
 07.09.2020, 13:03, "Philipp Masharov" :
> I will try it. Including information about experimental commands into
> documentation sounds like a good idea. Am I need to create a Jira
>> ticket?
> Definitely I need to follow some process to enhance the product
> documentation.
> 
> On Mon, Sep 7, 2020 at 12:42 PM Nikolay Izhikov 
>> wrote:
> 
>> Hello, Philipp.
>> 
>> Indeed, lack of the information about experimental control.sh command
>> is a
>> gap in Ignite documentation.
>> Do you want to contribute corresponding page to close this gap?
>> 
>> To see all experimental commands you can build Ignite from sources and
>> execute the following commands
>> 
 $ export IGNITE_ENABLE_EXPERIMENTAL_COMMAND="true"
 $ ./bin/control.sh
>> 
>>> 7 сент. 2020 г., в 12:26, Philipp Masharov 
>> написал(а):
>>> 
>>> Hello Nikolay!
>>> 
>>> It's more convenient to use an option instead of an environmental
>> variable.
>>> But what is experiment commands? Are these commands not
>> production-ready? I
>>> never heard about it and I can't find information about experiment
>> commands
>>> in the product documentation.
>>> 
>>> On Mon, Sep 7, 2020 at 12:08 PM Nikolay Izhikov >> 
>> wrote:
>>> 
 Hello, Igniters.
 
 For now, experimental commands in control.sh can be enabled only via

[jira] [Created] (IGNITE-13422) Explicit command.sh option to enable experimental commands

2020-09-09 Thread Nikolay Izhikov (Jira) 
Nikolay Izhikov created IGNITE-13422:


 Summary: Explicit command.sh option to enable experimental commands
 Key: IGNITE-13422
 URL: https://issues.apache.org/jira/browse/IGNITE-13422
 Project: Ignite
  Issue Type: Improvement
Reporter: Nikolay Izhikov


Right now experimental commands can be enabled only via an environment 
variable(IGNITE_ENABLE_EXPERIMENTAL_COMMAND).
We need to add an explicit option to command.sh "--enable-experimental" which 
will turn on experimental commands.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13421) Automatically check Ignite system properties are documented

2020-09-09 Thread Nikolay Izhikov (Jira) 
Nikolay Izhikov created IGNITE-13421:


 Summary: Automatically check Ignite system properties are 
documented
 Key: IGNITE-13421
 URL: https://issues.apache.org/jira/browse/IGNITE-13421
 Project: Ignite
  Issue Type: Improvement
Reporter: Nikolay Izhikov


We should have some automatic check that will ensure all system properties that 
is accessed by Ignite properly documented with @SystemProperty annotation.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13420) Add assertion message to assert in CachedDeploymentInfo private constructor

2020-09-09 Thread Konstantin Sirotkin (Jira) 
Konstantin Sirotkin created IGNITE-13420:


 Summary: Add assertion message to assert in CachedDeploymentInfo 
private constructor
 Key: IGNITE-13420
 URL: https://issues.apache.org/jira/browse/IGNITE-13420
 Project: Ignite
  Issue Type: Improvement
Reporter: Konstantin Sirotkin
Assignee: Konstantin Sirotkin


We have an assertion in 
org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager.CachedDeploymentInfo#CachedDeploymentInfo:

 
{code:java}
private CachedDeploymentInfo(UUID sndId, IgniteUuid ldrId, String userVer, 
DeploymentMode depMode,
Map participants) {
assert sndId.equals(ldrId.globalId()) || participants != null;

this.sndId = sndId;
this.ldrId = ldrId;
this.userVer = userVer;
this.depMode = depMode;
this.participants = participants == null || participants.isEmpty() 
? null :
new ConcurrentLinkedHashMap<>(participants);
}
{code}
Now we can't define assertion reason. We should add assertion message to that 
place.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]

2020-09-09 Thread Alex Plehanov 
Hello guys,

I've tried to optimize tracing implementation (ticket [1]), it reduced the
drop, but not completely removed it.
Ivan Rakov, Alexander Lapin, can you please review the patch?
Ivan Artiukhov, can you please benchmark the patch [2] against 2.8.1
release on your environment?
With this patch on our environment, it's about a 3% drop left, it's close
to measurement error and I think such a drop is not a showstopper. Guys,
WDYT?

Also, I found that compatibility is broken for JDBC thin driver between 2.8
and 2.9 versions (ticket [3]). I think it's a blocker and should be
fixed in 2.9. I've prepared the patch.
Taras Ledkov, can you please review this patch?

And one more ticket I propose to include into 2.9 [4] (NIO message
send problem in some circumstances). I will cherry-pick it if there is no
objection.

[1] https://issues.apache.org/jira/browse/IGNITE-13411
[2] https://github.com/apache/ignite/pull/8223
[3] https://issues.apache.org/jira/browse/IGNITE-13414
[4] https://issues.apache.org/jira/browse/IGNITE-13361

пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov :

> Alexey,
>
> I propose to include [1] issue to the 2.9 release. Since this issue is
> related to the new master key change functionality which haven't been
> released yet I think it will be safe to cherry-pick commit to the
> release branch.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-13390
>
> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov  wrote:
> >
> > Hello, Igniters.
> >
> > Alexey, please, include one more Python thin client fix [1] into the 2.9
> release
> > It fixes kinda major issue - "Python client returns fields in wrong
> order since the 2 row when fields_count>10"
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-12809
> > [2]
> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
> >
> > > 31 авг. 2020 г., в 19:23, Alexey Goncharuk 
> написал(а):
> > >
> > > Alexey, thanks, got it. I am not sure we can optimize anything out of
> the
> > > message factory with suppliers (at least I have no ideas right now), so
> > > most likely the only move here is to switch back to the switch approach
> > > somehow preserving the metrics part. Probably, inlining the Ignite
> messages
> > > to the IgniteMessageFactoryImpl should do the trick. Let me explore the
> > > code a bit.
> > >
> > > P.S. I am surprised by the impact this part makes for the performance.
> > > Message creation is indeed on the hot path, but a single virtual call
> > > should not make that much of a difference given the amount of other
> work
> > > happening during the message processing.
> > >
> > > пн, 31 авг. 2020 г. в 18:33, Alex Plehanov :
> > >
> > >> Alexey, sorry, I wrongly interpreted our benchmark results. Actually,
> we
> > >> were looking for a drop using bi-sect in the range between e6a7f93
> (first
> > >> commit in the 2.9 branch after 2.8 branch cut) and 6592dfa5 (last
> commit in
> > >> the 2.9 branch). And we found these two problematic commits.
> > >>
> > >> Perhaps only IGNITE-13060 (Tracing) is responsible for a drop between
> > >> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with reverted
> IGNITE-13060
> > >> now and performance looks the same)
> > >>
> > >> Ticket IGNITE-12568 (MessageFactory refactoring) is not related to
> drop
> > >> between 2.8.1 and 2.9, but still has some performance problem, and we
> can
> > >> win back IGNITE-13060 drop by this ticket.
> > >>
> > >> Do we need more investigation on IGNITE-13060 or we leave it as is?
> > >>
> > >> What should we do with IGNITE-12568 (MessageFactory refactoring)?
> > >>
> > >> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
> alexey.goncha...@gmail.com
> > >>> :
> > >>
> > >>> Alexey,
> > >>>
> > >>> While investigating, I found that IGNITE-12568 has an incorrect fix
> > >>> version and is actually present in ignite-2.8.1 branch [1], so it
> cannot be
> > >>> the source of the drop against 2.8.1.
> > >>>
> > >>> P.S. Looks like we need to enforce a more accurate work with fix
> versions
> > >>> or develop some sort of tooling to verify the fix versions.
> > >>>
> > >>> --AG
> > >>>
> > >>> [1]
> > >>>
> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
> > >>>
> > >>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
> alexey.goncha...@gmail.com
> >  :
> > >>>
> >  пт, 28 авг. 2020 г. в 11:16, Alex Plehanov  >:
> > 
> > > Guys,
> > >
> > > We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568
> (reverted
> > > it
> > > locally) and got the same performance as on 2.8.1
> > >
> > > IGNITE-13060 (Tracing) - some code was added to hot paths, to trace
> > > these
> > > hot paths, it's clear why we have performance drop here.
> > >
> > > IGNITE-12568 (MessageFactory refactoring) - switch/case block was
> > > refactored to an array of message suppliers. The message factory
> is on
> > > the
> > > hot path, which explains why this commit has an impact on

[jira] [Created] (IGNITE-13419) New permission to grant access to SYS schema

2020-09-09 Thread Nikolay Izhikov (Jira) 
Nikolay Izhikov created IGNITE-13419:


 Summary: New permission to grant access to SYS schema
 Key: IGNITE-13419
 URL: https://issues.apache.org/jira/browse/IGNITE-13419
 Project: Ignite
  Issue Type: Improvement
Reporter: Nikolay Izhikov


We need to create permission to restrict user access to SYS schema.
Because SYS contains a system view that can contain some sensitive information



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-13418) Deadlock on multiple cache delete

2020-09-09 Thread Denis Chudov (Jira) 
Denis Chudov created IGNITE-13418:
-

 Summary: Deadlock on multiple cache delete
 Key: IGNITE-13418
 URL: https://issues.apache.org/jira/browse/IGNITE-13418
 Project: Ignite
  Issue Type: Bug
Reporter: Denis Chudov
Assignee: Denis Chudov


Thread 1:
 - acquires checkpoint read lock in 
GridCacheProcessor#processCacheStopRequestOnExchangeDone
 - acquires GridQueryProcessor#stateMux in GridQueryProcessor.onCacheStop
 - enters H2TreeIndex.destroy
 - releases checkpoint read lock in H2Tree.temporaryReleaseLock, can't take it 
again because of db-checkpoint-thread

Thread 2:
 - acquires checkpoint read lock in 
GridCacheProcessor#processCacheStopRequestOnExchangeDone
 - trying to acquire GridQueryProcessor#stateMux in 
GridQueryProcessor.onCacheStop which is held by thread 1

db-checkpoint-thread:
 - trying to acquire checkpoint write lock, can't do it because of Thread 2

Decision: H2Tree.temporaryReleaseLock should release lock only in case when 
tree deletion is asynchronous (H2TreeIndex.destroy is called with async=true), 
i.e. it happens inside of DurableBackgroundTask. Such tasks are executed in 
separate threads, which don't hold any other locks.

Thread dump:
{code:java}
Thread [name="sys-#1220%DPL_GRID%DplGridNodeName%", id=3200, state=BLOCKED, 
blockCnt=1, waitCnt=0]
 Lock [object=java.lang.Object@6a9a92ba, 
ownerName=sys-#1215%DPL_GRID%DplGridNodeName%, ownerId=3195]
 at 
o.a.i.i.processors.query.GridQueryProcessor.onCacheStop0(GridQueryProcessor.java:1695)
 at 
o.a.i.i.processors.query.GridQueryProcessor.onCacheStop(GridQueryProcessor.java:902)
 at 
o.a.i.i.processors.cache.GridCacheProcessor.stopCache(GridCacheProcessor.java:1169)
 at 
o.a.i.i.processors.cache.GridCacheProcessor.prepareCacheStop(GridCacheProcessor.java:2644)
 at 
o.a.i.i.processors.cache.GridCacheProcessor.lambda$processCacheStopRequestOnExchangeDone$629e8679$1(GridCacheProcessor.java:2803)
 at 
o.a.i.i.processors.cache.GridCacheProcessor$$Lambda$620/1418386924.apply(Unknown
 Source)
 at o.a.i.i.util.IgniteUtils.lambda$null$1(IgniteUtils.java:10879)
 at o.a.i.i.util.IgniteUtils$$Lambda$436/321848940.call(Unknown Source)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)
 Locked synchronizers:
 java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@2582f93c
 java.util.concurrent.ThreadPoolExecutor$Worker@4da1cafe
Thread [name="sys-#1215%DPL_GRID%DplGridNodeName%", id=3195, state=BLOCKED, 
blockCnt=4, waitCnt=437520]
 Lock [object=o.a.i.i.processors.failure.FailureProcessor@78edb1e9, 
ownerName=async-durable-background-task-executor-1-#1222%DPL_GRID%DplGridNodeName%,
 ownerId=3202]
 at 
o.a.i.i.processors.failure.FailureProcessor.process(FailureProcessor.java:162)
 at 
o.a.i.i.processors.failure.FailureProcessor.process(FailureProcessor.java:151)
 at 
o.a.i.i.processors.cache.persistence.GridCacheDatabaseSharedManager.failCheckpointReadLock(GridCacheDatabaseSharedManager.java:1787)
 at 
o.a.i.i.processors.cache.persistence.GridCacheDatabaseSharedManager.checkpointReadLock(GridCacheDatabaseSharedManager.java:1722)
 at 
o.a.i.i.processors.query.h2.database.H2Tree.temporaryReleaseLock(H2Tree.java:690)
 at 
o.a.i.i.processors.cache.persistence.tree.BPlusTree.temporaryReleaseLock(BPlusTree.java:2367)
 at 
o.a.i.i.processors.cache.persistence.tree.BPlusTree.destroyDownPages(BPlusTree.java:2548)
 at 
o.a.i.i.processors.cache.persistence.tree.BPlusTree.destroyDownPages(BPlusTree.java:2522)
 at 
o.a.i.i.processors.cache.persistence.tree.BPlusTree.destroyDownPages(BPlusTree.java:2522)
 at 
o.a.i.i.processors.cache.persistence.tree.BPlusTree.destroyDownPages(BPlusTree.java:2522)
 at 
o.a.i.i.processors.cache.persistence.tree.BPlusTree.destroy(BPlusTree.java:2441)
 at 
o.a.i.i.processors.cache.persistence.tree.BPlusTree.destroy(BPlusTree.java:2392)
 at 
o.a.i.i.processors.query.h2.database.H2TreeIndex.destroy0(H2TreeIndex.java:671)
 at 
o.a.i.i.processors.query.h2.database.H2TreeIndex.destroy(H2TreeIndex.java:639)
 at o.a.i.i.processors.query.h2.opt.GridH2Table.destroy(GridH2Table.java:567)
 at 
o.a.i.i.processors.query.h2.H2TableDescriptor.onDrop(H2TableDescriptor.java:347)
 at o.a.i.i.processors.query.h2.H2Schema.drop(H2Schema.java:127)
 at 
o.a.i.i.processors.query.h2.IgniteH2Indexing.unregisterCache(IgniteH2Indexing.java:2595)
 at 
o.a.i.i.processors.query.GridQueryProcessor.onCacheStop0(GridQueryProcessor.java:1727)
 - locked java.lang.Object@6a9a92ba
 at 
o.a.i.i.processors.query.GridQueryProcessor.onCacheStop(GridQueryProcessor.java:902)
 at 
o.a.i.i.processors.cache.GridCacheProcessor.stopCache(GridCacheProcessor.java:1169)
 at 
o.a.i.i.processors.cache.GridCacheProcessor.prepareCacheStop(G

[jira] [Created] (IGNITE-13417) Cache Interceptors deserialization on client nodes

2020-09-09 Thread Vladislav Pyatkov (Jira) 
Vladislav Pyatkov created IGNITE-13417:
--

 Summary: Cache Interceptors deserialization on client nodes
 Key: IGNITE-13417
 URL: https://issues.apache.org/jira/browse/IGNITE-13417
 Project: Ignite
  Issue Type: Bug
Reporter: Vladislav Pyatkov


After fix https://issues.apache.org/jira/browse/IGNITE-1903, Cache Interceptors 
still don't work

Looks like we need to add @SerializeSeparately to this field too



--
This message was sent by Atlassian Jira
(v8.3.4#803005)