[DISCUSSION] IEP-130: Ignite 3. KILL queries
Hi Igniters! Please review the proposal for Ignite-3 [1] and let me know what you think. [1] IEP-130: Ignite 3. KILL queries <https://cwiki.apache.org/confluence/display/IGNITE/IEP-130%3A+Ignite+3.+KILL+queries> -- Regards, Konstantin Orlov
Re: [DISCUSSION] IEP-127 Catalog Service
Forgot to mention, that this proposal is for Ignite-3. On Fri, Aug 30, 2024 at 5:57 PM Konstantin Orlov wrote: > Hi Igniters! > > Please review the proposal [1] and let me know what you think. > > [1] IEP-127: Catalog Service > <https://cwiki.apache.org/confluence/display/IGNITE/IEP-127%3A+Catalog+Service> > > -- > Regards, > Konstantin Orlov >
[DISCUSSION] IEP-127 Catalog Service
Hi Igniters! Please review the proposal [1] and let me know what you think. [1] IEP-127: Catalog Service <https://cwiki.apache.org/confluence/display/IGNITE/IEP-127%3A+Catalog+Service> -- Regards, Konstantin Orlov
[DISCUSSION] IEP-111: Ignite 3. Basic Multi-Statement Support
Hi all, Please review "IEP-111: Ignite 3. Basic Multi-Statement Support” proposal. The main purpose of this document is to state behavior of script processing from the user's point of view. https://cwiki.apache.org/confluence/display/IGNITE/IEP-111%3A+Ignite+3.+Basic+Multi-Statement+Support -- Regards, Konstantin Orlov
Re: [DISCUSSION] IEP-107 Logging in Ignite 3
Hi, Ilya The presence of thick clients in Ignite 2 slipped my mind, so I was trying to justify running two data nodes in the same JVM. Agree, this definitely makes sense for Ignite 2. Thanks for clarification! -- Regards, Konstantin Orlov > On 13 Jun 2023, at 18:06, Ilya Kasnacheev wrote: > > Hello! > > Even apart from testing, it is not uncommon to have JVM connect two > different Ignite clusters (two heterogenous ones or one main cluster and > the second "management" small cluster, for example) > > Regards, > -- > Ilya Kasnacheev > > > вт, 13 июн. 2023 г. в 12:46, Konstantin Orlov <mailto:kor...@gridgain.com>>: > >> Hi Ilya, >> >> I wonder why do you try to run several Ignite instances in the same JVM in >> the first place? Is there any value to do so, apart for testing? >> >> Anyway, with current proposal you could try to configure message routing >> based on a thread name: every thread in Ignite 3 should have name which >> starts with "%%” prefix. >> >> -- >> Regards, >> Konstantin Orlov >> >> >> >> >>> On 8 Jun 2023, at 18:14, Ilya Korol wrote: >>> >>> Hi Konstantin. Thanks for you work. >>> >>> Despite you proposal is more about public API of logging Ignite >> facility, I believe that this thread is the best place to highlight some >> topic about internal details of IgniteLogger based on my experience with >> Ignite 2 >>> >>> I already tried to initiate a discussion at the end of previous year, >> unfortunately there were not so many community members eager to discuss it. >>> >>> Here is the link >> https://lists.apache.org/thread/xcllwbz12cpb5437hrt5n1xfdfp0wqct >>> >>> The feature, that from my perspective, is missing in current >> implementation (AI2) and that should be kept in mind when it comes to >> implement logging IEP, >>> is ability to separate messages produced by different Ignite instances >> running in same JVM, >>> so at appenders level (no matter of the actual logging implementation) >> there is no way to properly route log messages to separate locations based >> on the message source node. >>> >>> Maybe community will find this feature useful. >>> >>> >>> >>> On 08.06.2023 17:45, Konstantin Orlov wrote: >>>> Hi, >>>> >>>> Please review "IEP-107 Logging in Ignite 3” proposal >>>> >>>> >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-107+Logging+in+Ignite+3 >>>> >>>> >>>> -- >>>> Regards, >>>> Konstantin Orlov
Re: [DISCUSSION] IEP-107 Logging in Ignite 3
Hi Ilya, I wonder why do you try to run several Ignite instances in the same JVM in the first place? Is there any value to do so, apart for testing? Anyway, with current proposal you could try to configure message routing based on a thread name: every thread in Ignite 3 should have name which starts with "%%” prefix. -- Regards, Konstantin Orlov > On 8 Jun 2023, at 18:14, Ilya Korol wrote: > > Hi Konstantin. Thanks for you work. > > Despite you proposal is more about public API of logging Ignite facility, I > believe that this thread is the best place to highlight some topic about > internal details of IgniteLogger based on my experience with Ignite 2 > > I already tried to initiate a discussion at the end of previous year, > unfortunately there were not so many community members eager to discuss it. > > Here is the link > https://lists.apache.org/thread/xcllwbz12cpb5437hrt5n1xfdfp0wqct > > The feature, that from my perspective, is missing in current implementation > (AI2) and that should be kept in mind when it comes to implement logging IEP, > is ability to separate messages produced by different Ignite instances > running in same JVM, > so at appenders level (no matter of the actual logging implementation) there > is no way to properly route log messages to separate locations based on the > message source node. > > Maybe community will find this feature useful. > > > > On 08.06.2023 17:45, Konstantin Orlov wrote: >> Hi, >> >> Please review "IEP-107 Logging in Ignite 3” proposal >> >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-107+Logging+in+Ignite+3 >> >> >> -- >> Regards, >> Konstantin Orlov >> >> >> >> >>
[DISCUSSION] IEP-107 Logging in Ignite 3
Hi, Please review "IEP-107 Logging in Ignite 3” proposal https://cwiki.apache.org/confluence/display/IGNITE/IEP-107+Logging+in+Ignite+3 -- Regards, Konstantin Orlov
Re: [ANNOUNCE] New PMC member: Vyacheslav Koptilin
Congratulation, Vyacheslav! Well deserved! -- Regards, Konstantin Orlov
Re: [PROPOSAL] Release Calcite-based SQL engine as an experimental feature
This is great news! Alex, thanks for making it to the end! -- Regards, Konstantin Orlov > On 24 Mar 2022, at 18:07, Alex Plehanov wrote: > > Hello Igniters, > > I've merged the pull request. The Calcite-based SQL engine is in the master > branch now. > If you desire to try it, you can find configuration instructions in > modules/calcite/README.txt file. > > вс, 6 мар. 2022 г. в 13:03, Alex Plehanov : > >> Hello Igniters, >> >> I've prepared the pull request [1] and have plans to merge it to the >> master branch in about two weeks, if there is no objection. >> >> [1]: https://github.com/apache/ignite/pull/9855 >> >> чт, 30 дек. 2021 г. в 13:43, Anton Vinogradov : >> >>>> it would be great to release a new SQL engine in 2.13 as an >>> experimental feature. >>> ++1 >>> >>> On Thu, Dec 30, 2021 at 12:55 PM Alex Plehanov >>> wrote: >>> >>>> Andrey, >>>> >>>>> Is this [1] a full scope of the tickets that MUST be resolved before >>> the >>>> engine could be merged? >>>> Yes, we must resolve at least these tickets before merging. If you see >>> any >>>> other release blockers fill free to attach them to this ticket. >>>> >>>>> I think we have to add instructions to the readme file on how to turn >>> a >>>> new SQL engine on. >>>> Sure, I think it should be the part of documentation ticket. >>>> >>>>> Also, I don't like the module name "ignite-calcite", because Calcite >>> is >>>> an independent project. >>>> Personally, I see no problems here (but it's discussable). We have a >>> lot of >>>> modules where the name is an independent project: "ignite-kafka", >>>> "ignite-spring", "ignite-kubernetes", "ignite-log4j", >>> "ignite-zookeeper", >>>> etc. >>>> >>>>> So, would you mind renaming the module to e.g. "ignite-sql-engine" or >>>> "ignite-sql"? >>>> Module "ignite-indexing" also contains SQL engine, so names like >>>> "ignite-sql-engine" or "ignite-sql" will be ambiguous. >>>> >>>> чт, 30 дек. 2021 г. в 13:54, Andrey Mashenkov < >>> andrey.mashen...@gmail.com >>>>> : >>>> >>>>> Alex, >>>>> it would be great to release a new SQL engine in 2.13 as an >>>>> experimental feature. >>>>> >>>>> Is this [1] a full scope of the tickets that MUST be resolved before >>> the >>>>> engine could be merged? >>>>> I think we have to add instructions to the readme file on how to turn >>> a >>>> new >>>>> SQL engine on. >>>>> >>>>> Also, I don't like the module name "ignite-calcite", because Calcite >>> is >>>> an >>>>> independent project. >>>>> and Ignite just uses it. >>>>> So, would you mind renaming the module to e.g. "ignite-sql-engine" or >>>>> "ignite-sql"? >>>>> >>>>> [1] https://issues.apache.org/jira/browse/IGNITE-15436 >>>>> >>>>> On Thu, Dec 30, 2021 at 11:10 AM Zhenya Stanilovsky >>>>> wrote: >>>>> >>>>>> >>>>>> Alex, great ! >>>>>> If someone wants to touch codebase somehow plz use this branch [1] >>>>>> Test passed can be found here [2] [3] >>>>>> >>>>>> [1] >>> https://github.com/apache/ignite/tree/sql-calcite/modules/calcite >>>>>> [2] >>>>>> >>>>> >>>> >>> https://github.com/apache/ignite/tree/sql-calcite/modules/calcite/src/test/java/org/apache/ignite/internal/processors/query/calcite >>>>>> [3] >>>>>> >>>>> >>>> >>> https://github.com/apache/ignite/tree/sql-calcite/modules/calcite/src/test/sql >>>>>> >>>>>>> >>>>>>>> >>>>>>>>> Hello, Igniters! >>>>>>>>> >>>>>>>>> As you may already know there is the new Ignite SQL engine based >>> on >>>>>> Apache >>>>>>>>> Calcite curre
Re: Ban Java Streams usage in Ignite 3 code
Hi! Agree with Ivan that it’s an overkill. Code readability and maintainability should have the same priority as the performance (with some exceptions of course). BTW the result of your benchmark looks quite strange. The performance penalty on my laptop (Core i7 9750H, 6 cores up to 4.50 GHz) is 25%, not 8 times: Benchmark Mode Cnt Score Error Units JmhIncrementBenchmark.loopSumthrpt 10 32347.819 ± 676.548 ops/ms JmhIncrementBenchmark.streamSum thrpt 10 24459.196 ± 610.152 ops/ms -- Regards, Konstantin Orlov > On 8 Sep 2021, at 12:23, Ivan Bessonov wrote: > > Hello Igniters, > > I object, banning streams is an overkill. I would argue that most of the > code > is not on hot paths and that allocations in TLAB don't create much pressure > on GC. > > Streams must be used cautiously, developers should know whether they > write hot methods or not. And if methods are not hot, code simplicity must > be > the first priority. I don't want Ignite 3 code to look like Ignite 2 code, > where > people would iterate over Lists using explicit access by indexes, because it > saves them a single Iterator allocation. That's absurd. > > ср, 8 сент. 2021 г. в 11:43, Pavel Tupitsyn : > >> Igniters, >> >> Java streams are known to be slower and cause more GC pressure than an >> equivalent loop. >> Below is a simple filter/map/reduce scenario (code [1]): >> >> * Benchmark Mode Cnt >>Score Error Units >> >> * StreamVsLoopBenchmark.loopSum thrpt3 >> 7987.016 ± 293.013 ops/ms >> * StreamVsLoopBenchmark.loopSum:·gc.alloc.rate thrpt3 >> ≈ 10⁻⁴MB/sec >> * StreamVsLoopBenchmark.loopSum:·gc.count thrpt3 >> ≈ 0counts >> >> * StreamVsLoopBenchmark.streamSum thrpt3 >> 1060.244 ± 36.485 ops/ms >> * StreamVsLoopBenchmark.streamSum:·gc.alloc.ratethrpt3 >> 315.819 ± 10.844 MB/sec >> * StreamVsLoopBenchmark.streamSum:·gc.count thrpt3 >> 55.000counts >> >> Loop is several times faster and does not allocate at all. >> >> 1. Performance is one of the most important features of our product. >> 2. Most of our APIs will be on the hot path. >> >> One can argue about performance differences in real-world scenarios, >> but increasing GC pressure just to make the code a little bit nicer is >> unacceptable. >> >> I propose to ban streams usage in the codebase (except for the tests). >> >> Thoughts, objections? >> >> [1] https://gist.github.com/ptupitsyn/5934bbbf8f92ac4937e534af9386da97 >> > > > -- > Sincerely yours, > Ivan Bessonov
Re: [ANNOUNCE] New committer: Petr Ivanov
Well deserved! -- Regards, Konstantin Orlov > On 19 Aug 2021, at 19:11, Kevin Corbett wrote: > > Congratulations Petr!! :) > >> On Aug 19, 2021, at 5:58 AM, Dmitry Pavlov wrote: >> >> Hello Ignite community, >> >> The Project Management Committee (PMC) for Apache Ignite >> has invited Petr Ivanov (vveider) to become a committer and we are pleased >> to announce that he has accepted. >> >> Petr is community veteran, who supports TeamCity instance, as well as TC >> Bot, and other services. He contributes also to supporting our >> checkstyle/PMD/maven scripts, docker images, DEB, and RPM packages, Ignite >> start scripts, and many more things, which are crucial for Apache Ignite >> development process and running in producition. >> >> Please join me in congratulating Petr with his new role in the community. >> >> Being a committer enables easier contribution to the >> project since there is no need to go via the patch >> submission process. This should enable better productivity. >> >> Best Regards, >> Dmitriy Pavlov >> on behalf of Apache Ignite PMC >
Re: Google Guava in Ignite 3
Hi, Andrey! But what should we do with Calcite then? It already brings Guava to the project. Should we considered exclusion of the main and only query engine from the Ignite-3.0? -- Regards, Konstantin Orlov > On 5 Aug 2021, at 17:23, Andrey Mashenkov wrote: > > -1 > It is sad to say -1, as Guava has very useful stuff and it looks easier to > add it as a dependency rather than copy-paste a code. My concerns are: 1. > Original Bytecode module depends on 26.0-jre Calcite depends on 29.0-jre We > maybe will use some other version. A user might want to use one more > version. So, I'd disagree legalizing Guava will help with maintainability > anyhow. 2. Guava supports JDK-8. Is it possible to handle different > versions of Guava in dependencies with JigSaw? What impact will have > potential future CVEs (and the current one) with the JigSaw? 3. Guava has > an unresolved CVE [1]. They just mark a vulnerable method as Deprecated and > didn't actually fix it [2]. [1] https://github.com/google/guava/issues/4011 > [2] https://github.com/google/guava/issues/4011 > > On Thu, Aug 5, 2021 at 4:54 PM Konstantin Orlov wrote: > >> +1, I considered it a necessary evil >> >> -- >> Regards, >> Konstantin Orlov >> >> >> >>> On 5 Aug 2021, at 16:37, Alexei Scherbakov >> wrote: >>> >>> +1 >>> >>> чт, 5 авг. 2021 г. в 16:12, Alexander Polovtcev >> : >>> >>>> Hello, dear Igniters! >>>> >>>> I would like to discuss the possibility of using Guava >>>> <https://github.com/google/guava> in Ignite 3. I know about the >>>> restrictive >>>> policy of using it in Ignite 2, but I have the following reasons: >>>> >>>> 1. We are de-facto using it already as an implicit dependency, since the >>>> Calcite module depends on it, and Calcite is going to stay for a while >> =) >>>> 2. AFAIK, the "bytecode" module is copied into the codebase only to >> strip >>>> Guava away from it. We can remove this module, which will improve the >>>> maintainability of the project. >>>> 3. We have some copy-paste of Guava code in the project. For example, >> see >>>> this >>>> < >>>> >> https://github.com/apache/ignite-3/blob/main/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L136 >>>>> >>>> and this >>>> < >>>> >> https://github.com/apache/ignite-3/blob/main/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L428 >>>>> >>>> . >>>> 4. Regarding security concerns, this report >>>> < >> https://www.cvedetails.com/product/52274/Google-Guava.html?vendor_id=1224 >>>>> >>>> shows no major vulnerability issues for the last three years. >>>> >>>> Taking these points into account, I propose to allow using Guava both in >>>> production and test code and to add it as an explicit dependency. >>>> >>>> What do you think? >>>> >>>> -- >>>> With regards, >>>> Aleksandr Polovtcev >>>> >>> >>> >>> -- >>> >>> Best regards, >>> Alexei Scherbakov >> >> > > -- > Best regards, > Andrey V. Mashenkov
Re: Google Guava in Ignite 3
+1, I considered it a necessary evil -- Regards, Konstantin Orlov > On 5 Aug 2021, at 16:37, Alexei Scherbakov > wrote: > > +1 > > чт, 5 авг. 2021 г. в 16:12, Alexander Polovtcev : > >> Hello, dear Igniters! >> >> I would like to discuss the possibility of using Guava >> <https://github.com/google/guava> in Ignite 3. I know about the >> restrictive >> policy of using it in Ignite 2, but I have the following reasons: >> >> 1. We are de-facto using it already as an implicit dependency, since the >> Calcite module depends on it, and Calcite is going to stay for a while =) >> 2. AFAIK, the "bytecode" module is copied into the codebase only to strip >> Guava away from it. We can remove this module, which will improve the >> maintainability of the project. >> 3. We have some copy-paste of Guava code in the project. For example, see >> this >> < >> https://github.com/apache/ignite-3/blob/main/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L136 >>> >> and this >> < >> https://github.com/apache/ignite-3/blob/main/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L428 >>> >> . >> 4. Regarding security concerns, this report >> <https://www.cvedetails.com/product/52274/Google-Guava.html?vendor_id=1224 >>> >> shows no major vulnerability issues for the last three years. >> >> Taking these points into account, I propose to allow using Guava both in >> production and test code and to add it as an explicit dependency. >> >> What do you think? >> >> -- >> With regards, >> Aleksandr Polovtcev >> > > > -- > > Best regards, > Alexei Scherbakov
Re: [ANNOUNCE] Welcome Zhenya Stanilovsky as a new committer
Zhenya, Congrats! -- Regards, Konstantin Orlov > On 30 Jul 2021, at 12:20, Вячеслав Коптилин wrote: > > Hooray! > > Congrats! May the Force be with you! > > Thanks, > S. > > пт, 30 июл. 2021 г. в 11:17, Anton Vinogradov : > >> Congrats! >> >> On Fri, Jul 30, 2021 at 10:19 AM ткаленко кирилл >> wrote: >> >>> Zhenya, congratulations! >>> >>> Пересылаемое сообщение >>> 30.07.2021, 09:50, "Ivan Daschinsky" : >>> >>> >>> Zhenya, congrats, well deserved! >>> >>> пт, 30 июл. 2021 г. в 00:44, Andrey Mashenkov < >> andrey.mashen...@gmail.com >>>> : >>> >>>> Congratulations Zhenya! >>>> >>>> On Fri, Jul 30, 2021 at 12:06 AM Maxim Muzafarov >>>> wrote: >>>> >>>>> The Project Management Committee (PMC) for Apache Ignite has invited >>>>> Zhenya Stanilovsky to become a committer and we are pleased to >>> announce >>>>> that >>>>> he has accepted. >>>>> >>>>> For the last few years, Zhenya made a lot of performance fixes >>>>> especially for the core modules and important contributions to the >>>>> Apache Ignite codebase. He is actively involved in integrating the >>>>> Calcite framework with Apache Ignite. Moreover, he has been a great >>>>> help in the preparation of several Apache Ignite major releases by >>>>> carrying out stress-load tests. Besides the code contributions, >> Zhenya >>>>> is also an active community member and help users on dev and users >>>>> lists. >>>>> >>>>> Being a committer enables easier contribution to the project since >>> there >>>> is >>>>> no need to go via the patch submission process. This should enable >>> better >>>>> productivity. >>>>> >>>>> Please join me in welcoming Ivan, and congratulating him on the new >>> role >>>> in >>>>> the Apache Ignite Community. >>>>> >>>>> Best Regards, >>>>> Maxim Muzafarov >>>>> >>>> >>>> -- >>>> Best regards, >>>> Andrey V. Mashenkov >>> >>> Конец пересылаемого сообщения >>> >>
Re: Checking generated sources with PMD
Hi, Petr!
The idea sounds good in theory, but reality is bit complicated.
When the code is generated from scratch, it's seems possible to comply
with all requirements and checks. But in case of templating there is no
chance to avoid all violations.
Here is an example:
public AstNode createCustomNode(ParserPos pos, String token) {
return
}
The code of creation of a custom node may use the pos variable or may not,
but template should consider both cases.
Besides, the template of SQL parser is out of our control now. Of course we
could copy this to our codebase, but then we need to update this every time
we update Calcite version, that I would prefer to avoid.
To sum up: personally I consider the check for generated code is good thing,
but some exceptions should be possible.
--
Regards,
Konstantin Orlov
> On 26 Jul 2021, at 16:09, Petr Ivanov wrote:
>
> Hi, Igniters!
>
>
> While working with Semyon on IGNITE-15182 [1] we've stumbled upon the
> following dilemma, that I would like to discuss.
>
> Currently, `mvn compile pmd:check` command (as mentioned in Devnotes) also
> adds to check generated sources and fails on them, while mvn install && mvn
> pmd:check (as it is done in TC) does not (the reason is in absence of
> execution of maven-compiler-plugin which is not called on `pmd:check` goal).
> PR from [1] introduces solution to add generated and other sources to check,
> but now there is a question: should we really add additional sources for
> examination?
> From one side generated sources are not in our direct control and it can be
> hard to impossible to fix them right, from another — generated sources still
> a part of project and some flaws and worst practices in it can influent
> project's stability in an unpredictable way.
>
> Please, share your thoughts on the matter!
>
>
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-15182
>
Re: [DISCUSS] Confuse default inspections.
+ for both
--
Regards,
Konstantin Orlov
> On 20 Jul 2021, at 16:32, Pavel Tupitsyn wrote:
>
> Agree with for both points
>
> On Tue, Jul 20, 2021 at 3:14 PM Alexander Polovtcev
> wrote:
>
>> this is a very welcome change for me
>>
>> On Tue, Jul 20, 2021 at 10:13 AM Ivan Pavlukhin
>> wrote:
>>
>>> + for both points.
>>>
>>> 2021-07-20 9:56 GMT+03:00, Ivan Daschinsky :
>>>> Hi!
>>>>
>>>> Firstly, lets talk about interfaces.
>>>>
>>>> 1. First of all, do we have an automatic inspection for it? AFAIK we
>>> don't
>>>> 2. I am for consistency. At least for production code. Nothing worse is
>>>> when someone mixes both approaches.
>>>>
>>>> About a prohibition of curly brackets around one line -- I am strongly
>>>> against this rule, it should be removed. There were a few bugs that
>> this
>>>> rule caused.
>>>>
>>>> вт, 20 июл. 2021 г. в 09:23, Zhenya Stanilovsky
>>> >>>> :
>>>>
>>>>>
>>>>> Igniters, i understand that this is very long and fundamental story.
>> but
>>>>> … still want to rise up this discussion, we have 2 very strange
>>>>> inspections:
>>>>> * «public» modifier in interface methods.
>>>>> * Illegal ‘{}’ for one line statement. — i found it harmful.
>>>>> I don`t want to link additional discussion about pos. 2 i think that
>>> harm
>>>>> is obvious.
>>>>> I suggest to get rid of them.
>>>>>
>>>>> what do you think ?
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Sincerely yours, Ivan Daschinskiy
>>>>
>>>
>>>
>>> --
>>>
>>> Best regards,
>>> Ivan Pavlukhin
>>>
>>
>>
>> --
>> With regards,
>> Aleksandr Polovtcev
>>
Re: Setting IGNITE_TO_STRING_INCLUDE_SENSITIVE=false prevents VM Arguments output
+1 to Ivan’s idea about adding extra param to IgniteSystemProperties. > Ok, and what's about other VM arguments, not included to > IgniteSystemProperties? IMO all VM arguments should be considered sensitive except those annotated with @IgniteSystemProperties(sensitive=false). -- Regards, Konstantin Orlov > On 2 Jul 2021, at 20:52, Shishkov Ilya wrote: > > Ok, and what's about other VM arguments, not included to > IgniteSystemProperties? > > пт, 2 июл. 2021 г. в 09:27, Zhenya Stanilovsky : > >> >> -1 for extra arg, +1 for Ivan`s upper proposal : @IgniteSystemProperty >> annotation. >> Look, someone will set some of IGNITE_* option and after possibly cluster >> problems will give this logs into analysis and engineer can`t reproduce >> such a case, cause no param is applied. >> >>> An extra argument for IgniteSystemProperty sounds reasonable. >>> >>> -Val >>> >>> On Thu, Jul 1, 2021 at 10:04 AM Ivan Daschinsky < ivanda...@gmail.com > >> wrote: >>> >>>> Ok, this can be excluded using blocklist-jvm-params.properties or just >> by >>>> providing and extra arg to annotation, as I have just suggested >>>> >>>> чт, 1 июл. 2021 г., 19:51 Valentin Kulichenko < >>>> valentin.kuliche...@gmail.com >>>>> : >>>> >>>>> Ivan, >>>>> >>>>> IP addresses (e.g. IGNITE_TCP_DISCOVERY_ADDRESSES) and file paths >>>>> (e.g. IGNITE_CONFIG_URL) are often considered sensitive information. >> Data >>>>> related to authentication (e.g. IGNITE_SSH_USER_NAME) is very likely >> to >>>> be >>>>> sensitive. >>>>> >>>>> Once again - I would exclude any property that can contain >> user-specific >>>>> information. Only our internal settings (stuff >>>>> like IGNITE_SQL_MERGE_TABLE_MAX_SIZE) are OK to appear in the logs. >>>>> >>>>> -Val >>>>> >>>>> On Thu, Jul 1, 2021 at 9:47 AM Ivan Daschinsky < ivanda...@gmail.com >>> >>>>> wrote: >>>>> >>>>>> We can add add an extra param in annotation, that blocks param to be >>>>>> printed, just set it to false by default and block it wheb set to >> true >>>>>> >>>>>> чт, 1 июл. 2021 г., 19:45 Atri Sharma < a...@apache.org >: >>>>>> >>>>>>> What if we allowed a blocklist of parameters that are never >> printed? >>>>>>> >>>>>>> On Thu, 1 Jul 2021, 22:06 Valentin Kulichenko, < >>>>>>> valentin.kuliche...@gmail.com > wrote: >>>>>>> >>>>>>>> Not all of them are OK to be printed out. At the very least, we >>>>> should >>>>>>> have >>>>>>>> a mechanism to exclude some of them. I would still go with >> opt-in >>>>>> rather >>>>>>>> than opt-out though, but I guess that is up to a discussion. >>>>>>>> >>>>>>>> -Val >>>>>>>> >>>>>>>> On Thu, Jul 1, 2021 at 9:29 AM Ivan Daschinsky < >>>> ivanda...@gmail.com > >>>>>>>> wrote: >>>>>>>> >>>>>>>>> This is security through obscurity, an obvious and a >> well-known >>>>> anti >>>>>>>>> pattern. I suppose that printing jvm options, that is >> registered >>>> by >>>>>>>>> @IgniteSystemProperty annotation is an ideal variant >>>>>>>>> >>>>>>>>> чт, 1 июл. 2021 г., 19:25 Valentin Kulichenko < >>>>>>>>> valentin.kuliche...@gmail.com >>>>>>>>>> : >>>>>>>>> >>>>>>>>>> Folks, >>>>>>>>>> >>>>>>>>>> *Anything* that a user provides to the system can >> potentially >>>> be >>>>>>>>> considered >>>>>>>>>> sensitive information. This includes the VM arguments. We >> can't >>>>>>> predict >>>>>>>>>> what exactly one can put there, so let's not make >> assumptions. >>>>>>>>>> >>>>>>>&g
Re: [DISCUSSION] Code style. Variable abbrevations
Enforced validation doesn't guarantee code consistency. No validation in the world prevent me from typing manually "mess" instead of "msg"/"message" or "svc" instead of "srvc"/"service" (btw "svc" is more common imo). And the fact that someone name a variable "count" instead of "cnt" doesn't make the whole project immature. -- Regards, Konstantin Orlov > On 17 Jun 2021, at 08:34, Ivan Pavlukhin wrote: > > Hi Nikolay, > > Thanks, it is rather interesting. > > Do we all agree that using different conventions for different code > packages does not break "consistency"? Or did I get something wrong? > > 2021-06-17 7:12 GMT+03:00, Николай Ижиков : >> Hello, Ivan. >> >> We can create checkstyle rule to enforce usage of abbreviations. >> Internal/public code differs by package. >> >> PoC of rule [1] >> >> [1] https://github.com/apache/ignite/pull/9153 >> >>> 17 июня 2021 г., в 07:01, Ivan Pavlukhin >>> написал(а): >>> >>> Nikita, >>> >>> Do you have a plan in your mind how to make Ignite codebase consistent? >>> >>> AFAIR, we had it intentionally inconsistent for a long time at least >>> for one sake: for internal code we used special conventions (e.g. >>> abbreviations, shorten getters) and common Java conventions for public >>> API and examples (e.g. no abbreviations and usual getters). >>> >>> 2021-06-16 23:37 GMT+03:00, Nikita Ivanov : >>>> Consistency is what makes it easier to contribute to the project and >>>> attract new members. Consistency implies strong, well defined and >>>> universally enforced rules. Just because we have some individuals who >>>> are lazy or inexperienced - it does not mean that the entire project >>>> should relax the basic-level engineering discipline. >>>> >>>> On a personal node - nothing screams "immaturity" louder that a code >>>> that uses inconsistent naming, commenting, code style & organization, >>>> etc. >>>> -- >>>> Nikita Ivanov >>>> >>>>> On Wed, Jun 16, 2021 at 5:56 AM Andrey Gura wrote: >>>>> >>>>> Hi, >>>>> >>>>> I understand that this rule seems too strict or may be weird. But >>>>> removing the rule could lead to review comments like "use future >>>>> instead of fut". So my proposal is to change rule from "required" to >>>>> "recommended". >>>>> >>>>> On Wed, Jun 16, 2021 at 2:49 PM Valentin Kulichenko >>>>> wrote: >>>>>> >>>>>> Konstantin, thanks for chiming in. >>>>>> >>>>>> That's exactly my concern. Overformalization is generally not a good >>>>>> thing. >>>>>> Someone used "mess" to abbreviate "message"? That is surely not a good >>>>>> coding style, but that's what code reviews are for. I believe that our >>>>>> committers are more than capable of identifying such issues. >>>>>> >>>>>> At the same time, with the current rules, we are *forced* to use >>>>>> abbreviations like "locAddrGrpMgr", whether we like it or not. All it >>>>>> does >>>>>> is makes it harder to contribute to Ignite, without providing any >>>>>> clear >>>>>> value. >>>>>> >>>>>> -Val >>>>>> >>>>>> On Thu, Jun 10, 2021 at 9:46 AM Konstantin Orlov >>>>>> wrote: >>>>>> >>>>>>> +1 for making this optional >>>>>>> >>>>>>> Common abbreviation rules is good for sure, but sometimes I getting >>>>>>> sick >>>>>>> of all those locAddrGrpMgr. >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Regards, >>>>>>> Konstantin Orlov >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On 7 Jun 2021, at 14:31, Nikolay Izhikov >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hello, Anton, Alexei. >>>>>>>> >>&
Re: [DISCUSSION] Code style. Variable abbrevations
+1 for making this optional Common abbreviation rules is good for sure, but sometimes I getting sick of all those locAddrGrpMgr. -- Regards, Konstantin Orlov > On 7 Jun 2021, at 14:31, Nikolay Izhikov wrote: > > Hello, Anton, Alexei. > > Thanks for the feedback. > > Personally, I’m pretty happy current abbreviation rules too. > Let see what we can do to make our codebase even more consistent. > >> 7 июня 2021 г., в 13:23, Alexei Scherbakov >> написал(а): >> >> -1 >> Common abbrevs add quality to the code. >> >> пн, 7 июн. 2021 г. в 12:38, Anton Vinogradov : >> >>> -1 here. >>> We can fix the code and set up the rule. >>> >>> This will help to prevent having a weird abbreviation like "mess" (from >>> "message") or "ign" (from "Ignite"). >>> Also, the abbreviations list (hardcoded at IDEA plugin) allows to have same >>> names across the whole code, this should simplify the reading. >>> >>> On Sat, Jun 5, 2021 at 10:49 PM Valentin Kulichenko < >>> valentin.kuliche...@gmail.com> wrote: >>> >>>> I also support removing this requirement. It’s not the first time someone >>>> brings this up, and so far we haven’t been able to fix it. Not worth it >>> in >>>> my view. >>>> >>>> -Val >>>> >>>> On Sat, Jun 5, 2021 at 11:54 AM Nikolay Izhikov >>>> wrote: >>>> >>>>> Hello, guys. >>>>> >>>>> Thanks for the feedback. >>>>> >>>>> Dmitry, >>>>> >>>>>> Manual rule enforcement saves a couple of symbols in code >>>>> >>>>> I’m talking about automatic check. >>>>> We can implement it easily. >>>>> So, if we decide to keep this rule all we need is to fix current >>>>> violations (several thousand!). >>>>> After it, checkstyle will automatically enforce this rule. >>>>> As you may know, currently, any PR checked according to our checkstyle >>>>> rules. >>>>> Please, take a look at little green check sign after PR name. >>>>> >>>>> >>>>>> 5 июня 2021 г., в 00:57, Dmitry Pavlov >>>> написал(а): >>>>>> >>>>>> +1 for removal. Cnt and count both seem to be fine. >>>>>> >>>>>> Manual rule enforcement saves a couple of symbols in code, but >>> requires >>>>> some time from both contributor and reviewer. >>>>>> >>>>>> Sincerely, >>>>>> Dmitriy Pavlov >>>>>> >>>>>> On 2021/06/04 19:18:37, Pavel Tupitsyn wrote: >>>>>>> In my opinion, we should remove this rule. >>>>>>> Looks like a waste of time. freq or frequency, cnt or count, it is >>>> fine >>>>>>> either way. >>>>>>> >>>>>>> On Fri, Jun 4, 2021 at 7:39 PM Nikolay Izhikov >>> >>>>> wrote: >>>>>>> >>>>>>>> Hello, Igniters. >>>>>>>> >>>>>>>> Right now, we have the rule to use some predefined list of >>>> abbrevation >>>>> for >>>>>>>> variable names [1]. >>>>>>>> Some of the reviewers ask to follow this rule strictly. >>>>>>>> >>>>>>>>> It is required to use abbreviated form for code consistency. >>>>>>>> >>>>>>>> I tried to implement this rule in form of checkstyle check [2] and >>> it >>>>>>>> seems like many of use doesn’t follow this requirement. >>>>>>>> My check found 4124 violation in core module. >>>>>>>> >>>>>>>> Should we make this rule optional in documentation or should we >>>> remove >>>>> it >>>>>>>> completely? >>>>>>>> Or should someone proceed and fix all the violations? >>>>>>>> >>>>>>>> WDYT? >>>>>>>> >>>>>>>> >>>>>>>> Example of output: >>>>>>>> ``` >>>>>>>> [ERROR] >>>>>>>> >>>>> >>>> >>
[jira] [Created] (IGNITE-14602) Calcite engine. Wrong string representation of dates outside standard ISO range
Konstantin Orlov created IGNITE-14602:
-
Summary: Calcite engine. Wrong string representation of dates
outside standard ISO range
Key: IGNITE-14602
URL: https://issues.apache.org/jira/browse/IGNITE-14602
Project: Ignite
Issue Type: Bug
Components: sql
Reporter: Konstantin Orlov
Subj.
Affected tests:
{{src/test/sql/types/date/date_parsing.test_ignored}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-14597) Calcite engine. Exception when using FIRST
Konstantin Orlov created IGNITE-14597:
-
Summary: Calcite engine. Exception when using FIRST
Key: IGNITE-14597
URL: https://issues.apache.org/jira/browse/IGNITE-14597
Project: Ignite
Issue Type: Bug
Components: sql
Reporter: Konstantin Orlov
Exception is thrown when running a query {{SELECT FIRST(NULL::DECIMAL)}}
{code:java}
class org.apache.ignite.IgniteException: Error at: decimal_aggregates.test:10.
sql: SELECT FIRST(NULL::DECIMAL)
at
org.apache.ignite.internal.processors.query.calcite.logical.SqlScriptRunner$Query.execute(SqlScriptRunner.java:518)
at
org.apache.ignite.internal.processors.query.calcite.logical.SqlScriptRunner.run(SqlScriptRunner.java:93)
at
org.apache.ignite.internal.processors.query.calcite.logical.ScriptTestRunner$1.run(ScriptTestRunner.java:219)
at java.lang.Thread.run(Thread.java:748)
Caused by: class
org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to plan
query.
at
org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.prepareQuery(ExecutionServiceImpl.java:541)
at
org.apache.ignite.internal.processors.query.calcite.prepare.QueryPlanCacheImpl.queryPlan(QueryPlanCacheImpl.java:84)
at
org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.executeQuery(ExecutionServiceImpl.java:398)
at
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.query(CalciteQueryProcessor.java:246)
at
org.apache.ignite.internal.processors.query.calcite.logical.SqlScriptRunner.sql(SqlScriptRunner.java:111)
at
org.apache.ignite.internal.processors.query.calcite.logical.SqlScriptRunner.access$600(SqlScriptRunner.java:51)
at
org.apache.ignite.internal.processors.query.calcite.logical.SqlScriptRunner$Query.execute(SqlScriptRunner.java:513)
... 3 more
Caused by: org.apache.calcite.runtime.CalciteContextException: From line 1,
column 8 to line 1, column 27: Function 'FIRST(NULL :: DECIMAL, 0)' can only be
used in MATCH_RECOGNIZE
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:467)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:883)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:868)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:5043)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateCall(SqlValidatorImpl.java:5574)
at
org.apache.ignite.internal.processors.query.calcite.prepare.IgniteSqlValidator.validateCall(IgniteSqlValidator.java:230)
at org.apache.calcite.sql.SqlCall.validate(SqlCall.java:116)
at org.apache.calcite.sql.SqlNode.validateExpr(SqlNode.java:273)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateExpr(SqlValidatorImpl.java:4259)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4234)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3474)
at
org.apache.ignite.internal.processors.query.calcite.prepare.IgniteSqlValidator.validateSelect(IgniteSqlValidator.java:176)
at
org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
at
org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1067)
at
org.apache.ignite.internal.processors.query.calcite.prepare.IgniteSqlValidator.validateNamespace(IgniteSqlValidator.java:190)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1041)
at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:232)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1016)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:724)
at
org.apache.ignite.internal.processors.query.calcite.prepare.IgnitePlanner.validateAndGetTypeMetadata(IgnitePlanner.java:202)
at
org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.prepareQuery(ExecutionServiceImpl.java:587)
at
org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.prepareSingle(ExecutionServiceImpl.java:562)
[jira] [Created] (IGNITE-14596) Calcite engine. Support for function TYPEOF
Konstantin Orlov created IGNITE-14596:
-
Summary: Calcite engine. Support for function TYPEOF
Key: IGNITE-14596
URL: https://issues.apache.org/jira/browse/IGNITE-14596
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Konstantin Orlov
Currently a function TYPEOF is not supported.
Affected tests:
{{src/test/sql/types/decimal/decimal_aggregates.test_ignored}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-14590) Calcite engine. Sort out the types/decimal and types/string directories
Konstantin Orlov created IGNITE-14590:
-
Summary: Calcite engine. Sort out the types/decimal and
types/string directories
Key: IGNITE-14590
URL: https://issues.apache.org/jira/browse/IGNITE-14590
Project: Ignite
Issue Type: Task
Components: sql
Reporter: Konstantin Orlov
Assignee: Konstantin Orlov
We need to sort out results of the {{ScriptRunnerTestSuite}} to mute an every
red test and file a ticket for related problems.
This ticket covers only two directories: {{types/decimal}} and {{types/string}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-14589) Calcite engine. Numerous problem with type Decimal
Konstantin Orlov created IGNITE-14589:
-
Summary: Calcite engine. Numerous problem with type Decimal
Key: IGNITE-14589
URL: https://issues.apache.org/jira/browse/IGNITE-14589
Project: Ignite
Issue Type: Bug
Components: sql
Reporter: Konstantin Orlov
There are numerous problem with a decimal types:
* very big numbers printed in scientific notation
* leading and trailing zeros are truncated when converting to string
* casting to precise scale is not working( {{select cast('0.01' as decimal(10,
1)) * 10}} returns 0.1 instead of 0)
Affected tests:
{{src/test/sql/types/decimal}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-14555) Calcite engine. Create table from query result
Konstantin Orlov created IGNITE-14555:
-
Summary: Calcite engine. Create table from query result
Key: IGNITE-14555
URL: https://issues.apache.org/jira/browse/IGNITE-14555
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Konstantin Orlov
Provide ability to create table from the result of the query: {{CREATE TABLE
my_tbl AS }}.
Affected tests:
{{modules/calcite/src/test/sql/types/decimal/decimal_aggregates.test_ignore}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-14554) Calcite engine. Provide ability to parse huge numeric literals
Konstantin Orlov created IGNITE-14554:
-
Summary: Calcite engine. Provide ability to parse huge numeric
literals
Key: IGNITE-14554
URL: https://issues.apache.org/jira/browse/IGNITE-14554
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Konstantin Orlov
Currently literals like this {{17014118346046923173168730371588410572}} can't
pass validation because they overflow range of long values.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-14553) Calcite engine. Duplicated result on insert
Konstantin Orlov created IGNITE-14553:
-
Summary: Calcite engine. Duplicated result on insert
Key: IGNITE-14553
URL: https://issues.apache.org/jira/browse/IGNITE-14553
Project: Ignite
Issue Type: Bug
Components: sql
Reporter: Konstantin Orlov
Please see the test
{{modules/calcite/src/test/sql/types/string/test_scan_big_varchar.test_ignore}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-14552) Calcite engine. Alias for function CHARACTER_LENGTH
Konstantin Orlov created IGNITE-14552:
-
Summary: Calcite engine. Alias for function CHARACTER_LENGTH
Key: IGNITE-14552
URL: https://issues.apache.org/jira/browse/IGNITE-14552
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Konstantin Orlov
Currently the function to get the length of the string is named
{{CHARACTER_LENGTH}}. It would be nice to have an alias {{LENGTH}} for this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-14549) Calcite. ORDER BY column not from SELECT LIST
Konstantin Orlov created IGNITE-14549:
-
Summary: Calcite. ORDER BY column not from SELECT LIST
Key: IGNITE-14549
URL: https://issues.apache.org/jira/browse/IGNITE-14549
Project: Ignite
Issue Type: Bug
Reporter: Konstantin Orlov
Assignee: Konstantin Orlov
Currently queries with sorting by a column not presented in select list are
failed during planning.
{{select col1 from my_table order by col2}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [DISCUSSION] IgniteFuture class future in Ignite-3.0.
CompletableFuture seems a better option to me.
--
Regards,
Konstantin Orlov
> On 26 Mar 2021, at 11:07, Pavel Tupitsyn wrote:
>
> On the one hand, I agree with Alexey.
> CompletableFuture has complete* methods which should not be available to
> the user code.
> This can be solved with a simple interface like we do in Thin Client:
> IgniteClientFuture extends Future, CompletionStage
>
>
> On the other hand:
> - CompletionStage has toCompletableFuture anyway (rather weird)
> - Other libraries use CompletableFuture and it seems to be fine
> - Using CompletableFuture is the simplest approach
>
>
> So I lean towards CompletableFuture too.
>
> On Fri, Mar 26, 2021 at 10:46 AM Alexey Kukushkin
> wrote:
>
>> I do not like Java's CompletableFuture and prefer our own Future (revised
>> IgniteFuture).
>>
>> My understanding of the Future (or Promise) pattern in general is having
>> two separate APIs:
>>
>> 1. Server-side: create, set result, raise error, cancel from server.
>> 2. Client-side: get result, handle error, cancel from client
>>
>> Java's CompletableFuture looks like both the client-side and
>> server-side API. The "Completeable" prefix in the name is already confusing
>> for a client since it cannot "complete" an operation, only a server can.
>>
>> I would create our own IgniteFuture adding client-side functionality we
>> currently miss (like client-side cancellation).
>>
>>
>> пт, 26 мар. 2021 г. в 01:08, Valentin Kulichenko <
>> valentin.kuliche...@gmail.com>:
>>
>>> Andrey,
>>>
>>> Can you compile a full list of these risky methods, and elaborate on what
>>> the risks are?
>>>
>>> Generally, CompletableFuture is a much better option, because it's
>>> standard. But we need to make sure it actually fits our needs and doesn't
>>> do more harm than good.
>>>
>>> -Val
>>>
>>> On Thu, Mar 25, 2021 at 12:23 PM Alexei Scherbakov <
>>> alexey.scherbak...@gmail.com> wrote:
>>>
>>>> I think both options are fine, but personally lean toward
>>>> CompletableFuture.
>>>>
>>>> чт, 25 мар. 2021 г. в 17:56, Atri Sharma :
>>>>
>>>>> I would suggest using CompletableFuture -- I don't see a need for a
>>>> custom
>>>>> interface that is unique to us.
>>>>>
>>>>> It also allows a lower barrier for new contributors for understanding
>>>>> existing code
>>>>>
>>>>> On Thu, 25 Mar 2021, 20:18 Andrey Mashenkov, <
>>> andrey.mashen...@gmail.com
>>>>>
>>>>> wrote:
>>>>>
>>>>>> Hi Igniters,
>>>>>>
>>>>>> I'd like to start a discussion about replacing our custom
>>> IgniteFuture
>>>>>> class with CompletableFuture - existed JDK class
>>>>>> or rework it's implementation (like some other products done) to a
>>>>>> composition of CompletionStage and Future interfaces.
>>>>>> or maybe other option if you have any ideas. Do you?
>>>>>>
>>>>>> 1. The first approach pros and cons are
>>>>>> + Well-known JDK class
>>>>>> + Already implemented
>>>>>> - It is a class, not an interface.
>>>>>> - Expose some potentially harmful methods like "complete()".
>>>>>>
>>>>>> On the other side, it has copy() method to create defensive copy
>> and
>>>>>> minimalCompletionStage() to restrict harmful method usage.
>>>>>> Thus, this look like an applicable solution, but we should be
>> careful
>>>>>> exposing internal future to the outside.
>>>>>>
>>>>>> 2. The second approach is to implement our own interface like the
>>> next
>>>>> one:
>>>>>>
>>>>>> interface IgniteFuture extends CompletableStage, Future {
>>>>>> }
>>>>>>
>>>>>> Pros and cons are
>>>>>> + Our interfaces/classes contracts will expose an interface rather
>>> than
>>>>>> concrete implementation.
>>>>>> + All methods are safe.
>>>>>> - Some implementation is required.
>>>>>> - CompletableStage has a method toCompletableFuture() and can be
>>&g
[jira] [Created] (IGNITE-14281) Calcite. Colocated tables considered non colocated
Konstantin Orlov created IGNITE-14281: - Summary: Calcite. Colocated tables considered non colocated Key: IGNITE-14281 URL: https://issues.apache.org/jira/browse/IGNITE-14281 Project: Ignite Issue Type: Bug Components: sql Reporter: Konstantin Orlov Assignee: Konstantin Orlov Currently distribution trait for replicated cache is built over 0th column which is _KEY column. Because of this a colocated join is considered non-colocated until _KEY columns is not used explicitly as join predicate. Need to fix this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14277) Calcite.
Konstantin Orlov created IGNITE-14277:
-
Summary: Calcite.
Key: IGNITE-14277
URL: https://issues.apache.org/jira/browse/IGNITE-14277
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Konstantin Orlov
Currently SqlToRelConverter rewrites queries with IN predicate into semi-join
but without actually using semi-join. The resulting plan includes two joins and
several aggregates over IN argument list to calculate some sort of indicators.
This plan is quite cumbersome, it contains a lot of nodes, thus boost a search
space.
As workaround this optimization was disabled by increasing inSubQueryThreshold
to a MAX_INT value.
But a safer solution would be to rewrite the IN predicate as a true semi-join,
or better yet, an inner join. To achieve this, we need to convert the list of
values to an inline table with only single values as the left shoulder of the
inner join, and place the original table as the right shoulder. Thus, we could
take advantage of the Indexed Nested Loop in case there is an index on a column
that is part of the IN predicate.
Starting point for this ticket is
{{org.apache.calcite.sql2rel.SqlToRelConverter#substituteSubQuery}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-14034) Calcite integration. indexCondition refactoring
Konstantin Orlov created IGNITE-14034: - Summary: Calcite integration. indexCondition refactoring Key: IGNITE-14034 URL: https://issues.apache.org/jira/browse/IGNITE-14034 Project: Ignite Issue Type: Improvement Components: sql Reporter: Konstantin Orlov Currently IndexCondition is quite cumbersome and hard to understand. The difference between bounds and conditions is unclear as well as unclear what should be used to estimate a selectivity and what should be used to estimate a self cost. Thus I suggest to change it in a follow way: * remove [lower|upper]Cond * bounds remains as is * self cost estimation of an AbstractIndex should be calculated with regard to bounds * selectivity should be calculated with regards to whole condition that is member of ProjectableFilterableTableScan -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13973) Calcite integration. Query get frozen if assertion error was thrown
Konstantin Orlov created IGNITE-13973:
-
Summary: Calcite integration. Query get frozen if assertion error
was thrown
Key: IGNITE-13973
URL: https://issues.apache.org/jira/browse/IGNITE-13973
Project: Ignite
Issue Type: Bug
Components: sql
Reporter: Konstantin Orlov
Need to fix the issue and verify that assertion error on any stage of query
planning/execution (this includes fragment splitting phase) doesn't lead to
freezes.
Mentioned problem could be reproduced for example as follow:
1) place fake {{assert false}} inside {{IgniteMergeJoin(RelInput input)}}
2) run {{CalciteQueryProcessorTest.query2()}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13964) Query cancellation freezes on local lazy queries
Konstantin Orlov created IGNITE-13964: - Summary: Query cancellation freezes on local lazy queries Key: IGNITE-13964 URL: https://issues.apache.org/jira/browse/IGNITE-13964 Project: Ignite Issue Type: Bug Components: sql Affects Versions: 2.9.1 Reporter: Konstantin Orlov Assignee: Konstantin Orlov The reason is a query cancellation that acquires an iterator's lock to check the availability of a next row and thus update a state of the cursor to prevent unnecessary request cancelation. But the lock is already held by a query thread to fetch a next page, so the cancellation is forced to wait till the page will be completely prefetched. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Removing MVCC public API
+1 -- Regards, Konstantin Orlov Software Engineer, St. Petersburg +7 (921) 445-65-75 https://www.gridgain.com Powered by Apache® Ignite™
Re: [VOTE][EXTENSION] Release Apache Ignite Streaming extensions 1.0.0 RC1
+1 -- Regards, Konstantin Orlov
[jira] [Created] (IGNITE-13776) BPlus tree lock retries limit reached with sqlOnHeapCacheEnabled
Konstantin Orlov created IGNITE-13776:
-
Summary: BPlus tree lock retries limit reached with
sqlOnHeapCacheEnabled
Key: IGNITE-13776
URL: https://issues.apache.org/jira/browse/IGNITE-13776
Project: Ignite
Issue Type: Bug
Components: sql
Affects Versions: 2.9
Reporter: Konstantin Orlov
Assignee: Konstantin Orlov
Issue might be reproduced with sqlOnHeapCacheEnabled=true.
The following exception is thrown while streaming some data into caches with
data streamer:
{code:java}
[2020-07-08 20:22:16,766][ERROR][data-streamer-stripe-0-#9][root] Critical
system error detected. Will be handled accordingly to configured handler
[hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet
[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]],
failureCtx=FailureContext [type=CRITICAL_ERROR, err=class
o.a.i.IgniteCheckedException: Maximum number of retries 1000 reached for Put
operation (the tree may be corrupted). Increase IGNITE_BPLUS_TREE_LOCK_RETRIES
system property if you regularly see this message (current value is 1000).]]
class org.apache.ignite.IgniteCheckedException: Maximum number of retries 1000
reached for Put operation (the tree may be corrupted). Increase
IGNITE_BPLUS_TREE_LOCK_RETRIES system property if you regularly see this
message (current value is 1000).
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Get.checkLockRetry(BPlusTree.java:3090)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Put.checkLockRetry(BPlusTree.java:3887)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.putDown(BPlusTree.java:2771)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doPut(BPlusTree.java:2413)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.putx(BPlusTree.java:2393)
at
org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex.putx(H2TreeIndex.java:437)
at
org.apache.ignite.internal.processors.query.h2.opt.GridH2Table.update(GridH2Table.java:757)
at
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.store(IgniteH2Indexing.java:379)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.store(GridQueryProcessor.java:2187)
at
org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.store(GridCacheQueryManager.java:406)
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.finishUpdate(IgniteCacheOffheapManagerImpl.java:2561)
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke0(IgniteCacheOffheapManagerImpl.java:1670)
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1645)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.invoke(GridCacheOffheapManager.java:2473)
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:436)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:2328)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2601)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:2062)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1879)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1686)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:299)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:486)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:446)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:248)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1172)
at
Re: please advise. I'm not sure why TeamCity build fails
Hi Aleksey! Apache Ignite is pretty complex thing with huge amount of different tests in several languages. So some of them will fail time to time and it considered normal (at least right now). To verify your PR you could take an advantage of Apache Ignite Teamcity Bot [1]. It will check result of RunAll (<— it’s a TeamCity RunAll task [2]) of your changes agains result of master and build a report. The most interest part of this report is a "Possible blockers" section. There will be tests that pass on master branch and fail with your changes. All those blockers require investigation and almost often has to be fixed (however, it is possible that these tests were failed in master branch too, so it's a good idea to check it first). So the possible steps to verify your changes is following: 1) push the changes and create PR 2) go to the Apache Ignite Teamcity Bot [1] page and press the "Inspect contribution" button. 3) select the "apache" tab if it is not selected 4) find your PR by it's number and hit the "More" button and then hit the "Trigger build" button 5) now you have to wait until TC build is finished 6) finally the "Show pull//head report" button will appear Some information about Apache Ignite Teamcity Bot you could find here [3]. It obsolete a little but still may be useful. [1] https://mtcga.gridgain.com <https://mtcga.gridgain.com/> [2] https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_RunAll?mode=builds#all-projects <https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_RunAll?mode=builds#all-projects> [3] https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot <https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot> -- Regards, Konstantin Orlov > On 31 Jul 2020, at 14:43, Aleksey Kurinov wrote: > > Hi guys > > Please help me to understand the TeamCity test failing problem. > I created a pull request and TeanCity failed. > As part of the problem investigation I've created a new pull request with > minor code change to check how TeamCity would test this minor change. I've > just added a comment line. > Team city test has failed again and I'm not sure why it happens. > pull request is https://github.com/apache/ignite/pull/8105 > <https://github.com/apache/ignite/pull/8105> > > https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_RunBasicTests/5502960?buildTab=overview > > <https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_RunBasicTests/5502960?buildTab=overview> > > errors are following: > #9445: > Unexpected error during build messages processing in TeamCity > Unexpected error occurred during build message processing in TeamCity, > please contact your system administrator > Intermediate data was not removed and can be found at > /opt/buildagent/temp/buildTmp/inspection689365797700297143result > > #24031 > Test(s) failed. System.Collections.Generic.KeyNotFoundException : The given > key was not present in the cache: 0 > at > Apache.Ignite.Core.Impl.Cache.CacheImpl`2.<>c__DisplayClass30.b__2f(IBinaryStream > stream, Int64 res) in > c:\BuildAgent\work\7bc1c54bc719b67c\modules\platforms\dotnet\Apache.Ignite.Core\Impl\Cache\CacheImpl.cs:line > 549 > > ... > > btw my original pull request is https://github.com/apache/ignite/pull/7977 > <https://github.com/apache/ignite/pull/7977> > > thank you, > Aleksei
[jira] [Created] (IGNITE-13166) Flaky H2RowCachePageEvictionTest and IgniteCacheQueryH2IndexingLeakTest tests
Konstantin Orlov created IGNITE-13166: - Summary: Flaky H2RowCachePageEvictionTest and IgniteCacheQueryH2IndexingLeakTest tests Key: IGNITE-13166 URL: https://issues.apache.org/jira/browse/IGNITE-13166 Project: Ignite Issue Type: Task Reporter: Konstantin Orlov Assignee: Konstantin Orlov Currently we have several tests on TC which considered flaky. Need to investigate and fix them. [https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-4422957309644246466&tab=testDetails] [https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=8791646325227962211&tab=testDetails] [https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-7378154946319099847&tab=testDetails] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13075) NPE on request JDBC metadata
Konstantin Orlov created IGNITE-13075:
-
Summary: NPE on request JDBC metadata
Key: IGNITE-13075
URL: https://issues.apache.org/jira/browse/IGNITE-13075
Project: Ignite
Issue Type: Bug
Components: sql
Reporter: Konstantin Orlov
Assignee: Konstantin Orlov
NPE occure when metadata is requested and there is a table created through java
API like a
{code:java}
ignite.createCache(
new CacheConfiguration<>(DEFAULT_CACHE_NAME)
.setIndexedTypes(Integer.class, String.class)
);{code}
Error here:
{code:java}
[2020-05-26
15:52:24,502][ERROR][client-connector-#264%thin.JdbcThinMetadataSelfTest0%][JdbcRequestHandler]
Failed to get columns metadata [reqId=15, req=JdbcMetaColumnsRequest
[schemaName=testCache, tblName=null, colName=null]][2020-05-26
15:52:24,502][ERROR][client-connector-#264%thin.JdbcThinMetadataSelfTest0%][JdbcRequestHandler]
Failed to get columns metadata [reqId=15, req=JdbcMetaColumnsRequest
[schemaName=testCache, tblName=null,
colName=null]]java.lang.NullPointerException at
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.lambda$null$10(IgniteH2Indexing.java:1902)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at
java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at
java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:270) at
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at
java.util.concurrent.ConcurrentHashMap$ValueSpliterator.forEachRemaining(ConcurrentHashMap.java:3566)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.columnsInformation(IgniteH2Indexing.java:1910)
at
org.apache.ignite.internal.processors.odbc.jdbc.JdbcMetadataInfo.getColumnsMeta(JdbcMetadataInfo.java:180)
at
org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.getColumnsMeta(JdbcRequestHandler.java:1128)
at
org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.doHandle(JdbcRequestHandler.java:348)
at
org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.handle(JdbcRequestHandler.java:257)
at
org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:200)
at
org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:54)
at
org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279)
at
org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
at
org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:97)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at
org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:70)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [DISCUSSION] Separate code sanity check and build task
Hi Ivan, Thanks for your reply! It’s better to get an answer late than never :) -- Regards, Konstantin Orlov > On 18 May 2020, at 09:04, Ivan Pavlukhin wrote: > > Hi Konstantin, > > Surprisingly, I found your message in a Spam folder (gmail). > > We had discussions about the subject before. The most recent one and > reflecting a current state is [1]. You can find many thoughts and > arguments in another discussion [2] (it might be better to start > reading from a bottom). > > [1] > https://lists.apache.org/thread.html/6995a4e789117ba3f5577651866cfa99a6ffcc208cf60330d17d5a48%40%3Cdev.ignite.apache.org%3E > [2] > http://apache-ignite-developers.2346864.n4.nabble.com/Code-inspection-td27709i80.html#a41297 > > 2020-04-20 11:00 GMT+03:00, Konstantin Orlov : >> Igniters, >> >> Currently we have code sanity checks [1][2] integrated within a build task >> [3]. Do we really need to fail the build (and therefore the other tasks) if >> there is a minor flaw like a missing line at the end of a file or an unused >> import? As for me it could be separated from the build task. >> >> What do you think? >> >> [1] >> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_CheckCodeStyle >> <https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_CheckCodeStyle> >> [2] >> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_LicensesHeaders >> <https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_LicensesHeaders> >> [3] >> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_BuildApacheIgnite >> <https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_BuildApacheIgnite> >> >> -- >> Regards, >> Konstantin Orlov >> >> >> > > > -- > > Best regards, > Ivan Pavlukhin
[jira] [Created] (IGNITE-13007) JDBC: Introduce feature flags for JDBC thin
Konstantin Orlov created IGNITE-13007: - Summary: JDBC: Introduce feature flags for JDBC thin Key: IGNITE-13007 URL: https://issues.apache.org/jira/browse/IGNITE-13007 Project: Ignite Issue Type: Improvement Components: sql Reporter: Konstantin Orlov Assignee: Konstantin Orlov Motivation the same as for https://issues.apache.org/jira/browse/IGNITE-12853 The thin client & JDBC, ODBC have different protocol specific and may require implement different features. Each client (thin cli, thin JDBC, ODBC) should have its own feature flags set. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13002) Supports user's java object by JDBC thin client
Konstantin Orlov created IGNITE-13002:
-
Summary: Supports user's java object by JDBC thin client
Key: IGNITE-13002
URL: https://issues.apache.org/jira/browse/IGNITE-13002
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Konstantin Orlov
Assignee: Konstantin Orlov
Now JDBC thin client doesn't support no-SQL types.
e.g.:
{{CREATE TABLE TEST (ID INT PRIMARY KEY, VAL OTHER)}}
{code}
PreparedStatement stmt = conn.prepareStatement("INSERT INTO TEST VALUES(?, ?)")
stmt.setInteger(1, 0);
stmt.setObject(2, new MyClass());
{code}
We have to support {{GridBinaryMarshaller}} for the JDBC thin client.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Why OPTIMIZE_REUSE_RESULT is set to 0 explicitly in o.a.i.i.p.query.h2.ConnectionManager#DEFAULT_DB_OPTIONS
Ivan, Nikolay, guys Sorry if I didn’t put it very clearly. The only caveat I would like to emphasize is that we must invalidate the results. H2 does this by comparing table's maxDataModificationId with the databases' current modificationDataId. We have a H2StatementCache, where Query object with lastResult is cached. If the result is not invalidated when update is happen (even not concurrent one), the query will return an incorrect result. Consider the following scenario: 0) OPTIMIZE_REUSE_RESULT is set to 1, GridH2Table#getMaxDataModificationId still returns 0 1) execute some query with non-correlated sub-query several times (10 should be enough), for example "select * from outerTbl where inner_id in (select id from innerTbl)" 2) clear innerTbl table 3) execute query one more time. Empty result set is expected This behavior was not obvious to me when I once looked at disabled 'OPTIMIZE_REUSE_RESULT', so I decided to share with you just in case =) -- Regards, Konstantin Orlov Software Engineer, St. Petersburg +7 (921) 445-65-75 https://www.gridgain.com Powered by Apache® Ignite™ > On 27 Apr 2020, at 13:36, Nikolay Izhikov wrote: > > Hello, Konstantin. > >> I think we cannot just turn this optimization on > > Why is that? > > If we turn on `OPTIMIZE_REUSE_RESULT` consequences would be the following: > > * concurrent updated of the subquery table will not be used(we have no > guarantee on that, for now) > * performance of the select increase due to the subquery cache. > >> And incrementing org.h2.engine.Database#modificationDataId on each data >> update may lead to some performance issues. It should be benchmarked first. > > For now, Ignite doesn’t guarantee the visibility of concurrent update into > select results [1] [2] > >> It should be benchmarked first. > > +1 > > What have I missed? > > [1] > https://apacheignite-sql.readme.io/docs/how-ignite-sql-works#subqueries-in-where-clause > [2] > https://apacheignite-sql.readme.io/docs/how-ignite-sql-works#concurrent-modifications > > >> 22 апр. 2020 г., в 10:39, Ivan Daschinsky написал(а): >> >> Thank you for your response, Konstantin >> >> I think we cannot just turn this optimization >>> >> Of course not, that is the reason why I started this thread. >> >> org.apache.ignite.internal.processors.query.h2.opt.GridH2Table don't change >>> it neither on remove or update >> >> >> Yes, but we can increment this counter and find possible workarounds. >> >> It should be benchmarked first. >>> >> Absolutelly agree. >> >> I started this thread to discuss possible caveats and traps and decide >> whether to implement this optimization or not. >> If there is too many traps or moreover this is impossible or SQL >> maintainers are against this, then it's probably not worth to do it. >> >> >> >> ср, 22 апр. 2020 г. в 09:52, Konstantin Orlov : >> >>> Hello, Ivan >>> >>> I think we cannot just turn this optimization on because a result >>> invalidation counts on org.h2.engine.Database#modificationDataId (see >>> org.h2.command.dml.Query#sameResultAsLast(Session, Value[], Value[], >>> long)), but org.apache.ignite.internal.processors.query.h2.opt.GridH2Table >>> don't change it neither on remove or update. And incrementing >>> org.h2.engine.Database#modificationDataId on each data update may lead to >>> some performance issues. It should be benchmarked first. >>> >>> -- >>> Regards, >>> Konstantin Orlov >>> >>> >>>> On 21 Apr 2020, at 14:51, Ivan Daschinskiy wrote: >>>> >>>> Hello folks. >>>> >>>> Recently I trying to investigate why when the query, i.e. «SELECT * FROM >>> T1 WHERE ID IN (SELECT T1_ID FROM T2), is executed locally, >>>> subquery evaluated, even if result is the same. I noticed, that in >>> mentioned in subj (.ConnectionManager#DEFAULT_DB_OPTIONS), >>>> parameter OPTIMIZE_REUSE_RESULT explicitly set to 0. This value disable >>> internal H2 optimization (see details here >>> org.h2.command.dml.Query#query(int, org.h2.result.ResultTarget) and lead >>> the mentioned above query to be ineffective. >>>> >>>> I cannot understand why this decision was made. Also I don’t find any >>> evidence that setting OPTIMIZE_REUSE_RESULT to 1 could break something. >>> Unfortunatelly, it is impossible to change this behavior per Session >>> without forking H2, because this flag is set in org.h2.engine.Database. >>>> >>>> Let’s discuss possible caveats in enabling this optimization by default >>> in DEFAULT_DB_OPTIONS. >>>> >>> >>> >> >> -- >> Sincerely yours, Ivan Daschinskiy >
Re: Why OPTIMIZE_REUSE_RESULT is set to 0 explicitly in o.a.i.i.p.query.h2.ConnectionManager#DEFAULT_DB_OPTIONS
Hello, Ivan I think we cannot just turn this optimization on because a result invalidation counts on org.h2.engine.Database#modificationDataId (see org.h2.command.dml.Query#sameResultAsLast(Session, Value[], Value[], long)), but org.apache.ignite.internal.processors.query.h2.opt.GridH2Table don't change it neither on remove or update. And incrementing org.h2.engine.Database#modificationDataId on each data update may lead to some performance issues. It should be benchmarked first. -- Regards, Konstantin Orlov > On 21 Apr 2020, at 14:51, Ivan Daschinskiy wrote: > > Hello folks. > > Recently I trying to investigate why when the query, i.e. «SELECT * FROM T1 > WHERE ID IN (SELECT T1_ID FROM T2), is executed locally, > subquery evaluated, even if result is the same. I noticed, that in mentioned > in subj (.ConnectionManager#DEFAULT_DB_OPTIONS), > parameter OPTIMIZE_REUSE_RESULT explicitly set to 0. This value disable > internal H2 optimization (see details here > org.h2.command.dml.Query#query(int, org.h2.result.ResultTarget) and lead the > mentioned above query to be ineffective. > > I cannot understand why this decision was made. Also I don’t find any > evidence that setting OPTIMIZE_REUSE_RESULT to 1 could break something. > Unfortunatelly, it is impossible to change this behavior per Session without > forking H2, because this flag is set in org.h2.engine.Database. > > Let’s discuss possible caveats in enabling this optimization by default in > DEFAULT_DB_OPTIONS. >
[DISCUSSION] Separate code sanity check and build task
Igniters, Currently we have code sanity checks [1][2] integrated within a build task [3]. Do we really need to fail the build (and therefore the other tasks) if there is a minor flaw like a missing line at the end of a file or an unused import? As for me it could be separated from the build task. What do you think? [1] https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_CheckCodeStyle <https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_CheckCodeStyle> [2] https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_LicensesHeaders <https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_LicensesHeaders> [3] https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_BuildApacheIgnite <https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_BuildApacheIgnite> -- Regards, Konstantin Orlov
[jira] [Created] (IGNITE-12688) Improve performance of index inline JAVA_OBJECT fields
Konstantin Orlov created IGNITE-12688: - Summary: Improve performance of index inline JAVA_OBJECT fields Key: IGNITE-12688 URL: https://issues.apache.org/jira/browse/IGNITE-12688 Project: Ignite Issue Type: Bug Reporter: Konstantin Orlov Assignee: Konstantin Orlov Inline JAVA_OBJECT may be reason of performance drop on index creation. The *root cause* is the low selectivity of current part of JAVA_OBJECT that is inlined. Now first N bytes of binary view of object is placed into inline space of the index. But the first offset where the two objects with the same type may be different is 8 (HASH_CODE_POS). With default inline = 10 we NEVER inline it: Inline format: 1 byte - type, 2 bytes - size >> 7 bytes - data; *My proposal:* - For metapage v4 (BPlusMetaIO) add flag *inlineObjectHash*; - use this flag to work in compatibility mode. - inline ONLY hash for JAVA_OBJECT fields for new indexes; Also this approach solves the inconsistent between comparison JAVA_OBJECT by inline and full value, because value comparator uses hash to compare object before compare byte arrays. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12687) Sqline throws unexpected exceptions on few specific arguments to !describe and !primarykeys
Konstantin Orlov created IGNITE-12687:
-
Summary: Sqline throws unexpected exceptions on few specific
arguments to !describe and !primarykeys
Key: IGNITE-12687
URL: https://issues.apache.org/jira/browse/IGNITE-12687
Project: Ignite
Issue Type: Bug
Reporter: Konstantin Orlov
Assignee: Konstantin Orlov
%subj
{noformat}
0: jdbc:ignite:thin://127.0.0.1/> !describe '
java.lang.StringIndexOutOfBoundsException: String index out of range: -1
at java.lang.String.substring(String.java:1967)
at sqlline.SqlLine.dequote(SqlLine.java:1322)
at sqlline.SqlLine.split(SqlLine.java:1335)
at sqlline.SqlLine.split(SqlLine.java:1154)
at
sqlline.AbstractCommandHandler.matches(AbstractCommandHandler.java:65)
at sqlline.SqlLine.dispatch(SqlLine.java:774)
at sqlline.SqlLine.begin(SqlLine.java:668)
at sqlline.SqlLine.start(SqlLine.java:373)
at sqlline.SqlLine.main(SqlLine.java:265)
{noformat}
when at least one table was created, following command also fails:
{noformat}
0: jdbc:ignite:thin://127.0.0.1/> !describe ?
[17:19:31,763][SEVERE][client-connector-#110][JdbcRequestHandler] Failed to get
columns metadata [reqId=108, req=JdbcMetaColumnsRequest [schemaName=null,
tblName=?, colName=%]]
java.util.regex.PatternSyntaxException: Dangling meta character '?' near index 0
?
^
at java.util.regex.Pattern.error(Pattern.java:1955)
at java.util.regex.Pattern.sequence(Pattern.java:2123)
at java.util.regex.Pattern.expr(Pattern.java:1996)
at java.util.regex.Pattern.compile(Pattern.java:1696)
at java.util.regex.Pattern.(Pattern.java:1351)
at java.util.regex.Pattern.compile(Pattern.java:1028)
at java.util.regex.Pattern.matches(Pattern.java:1133)
at java.lang.String.matches(String.java:2121)
at
org.apache.ignite.internal.processors.query.QueryUtils.matches(QueryUtils.java:1547)
at
org.apache.ignite.internal.processors.odbc.jdbc.JdbcMetadataInfo.getColumnsMeta(JdbcMetadataInfo.java:170)
at
org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.getColumnsMeta(JdbcRequestHandler.java:1089)
at
org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.doHandle(JdbcRequestHandler.java:332)
at
org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.handle(JdbcRequestHandler.java:241)
at
org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:180)
at
org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:47)
at
org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:278)
at
org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:108)
at
org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:96)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at
org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:69)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Error: Dangling meta character '?' near index 0
?
^ (state=5,code=1)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12678) GridLongList fails on add when created with explicit zero size
Konstantin Orlov created IGNITE-12678: - Summary: GridLongList fails on add when created with explicit zero size Key: IGNITE-12678 URL: https://issues.apache.org/jira/browse/IGNITE-12678 Project: Ignite Issue Type: Bug Components: data structures Reporter: Konstantin Orlov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12677) Extend test coverage for GridLongList serialization
Konstantin Orlov created IGNITE-12677: - Summary: Extend test coverage for GridLongList serialization Key: IGNITE-12677 URL: https://issues.apache.org/jira/browse/IGNITE-12677 Project: Ignite Issue Type: Task Components: binary Reporter: Konstantin Orlov Assignee: Konstantin Orlov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12675) Binary types registered in two stages when entry processor is used
Konstantin Orlov created IGNITE-12675: - Summary: Binary types registered in two stages when entry processor is used Key: IGNITE-12675 URL: https://issues.apache.org/jira/browse/IGNITE-12675 Project: Ignite Issue Type: Bug Components: binary Reporter: Konstantin Orlov When you try to add new value with entry processor, binary type registration proceeded in two stages: - class registration - binary metadata registration Perhaps it could be achieved by only one stage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12674) Extend test coverage for binary types registration
Konstantin Orlov created IGNITE-12674: - Summary: Extend test coverage for binary types registration Key: IGNITE-12674 URL: https://issues.apache.org/jira/browse/IGNITE-12674 Project: Ignite Issue Type: Task Components: binary Reporter: Konstantin Orlov Assignee: Konstantin Orlov -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [VOTE] Allow or prohibit a joint use of @deprecated and @IgniteExperimental
-1 Prohibit We should not deprecate the old API if the new API could change in the near future.
[jira] [Created] (IGNITE-12571) Statistics of query cache statements
Konstantin Orlov created IGNITE-12571: - Summary: Statistics of query cache statements Key: IGNITE-12571 URL: https://issues.apache.org/jira/browse/IGNITE-12571 Project: Ignite Issue Type: Bug Components: sql Reporter: Konstantin Orlov Assignee: Konstantin Orlov We need to collect hit/miss statistics for the query cache. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12569) Can't set serialized enum to a BinaryObject's field
Konstantin Orlov created IGNITE-12569:
-
Summary: Can't set serialized enum to a BinaryObject's field
Key: IGNITE-12569
URL: https://issues.apache.org/jira/browse/IGNITE-12569
Project: Ignite
Issue Type: Bug
Components: binary
Reporter: Konstantin Orlov
Assignee: Konstantin Orlov
The deserialization of the BinaryObject fails since a serialized representation
of the enum was set to a field instead of the enum itself. Because of this
issue there is no way to work with enums without a corresponding class.
Error message thrown during the deserialization:
{noformat}[18:03:06,159][ERROR][main][BinaryContext] Failed to deserialize
object [typeName=binary.BinaryEnumExample$TestClass]
org.apache.ignite.binary.BinaryObjectException: Failed to read field
[name=theEnum]
at
org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:191)
~[classes/:?]
at
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:887)
[classes/:?]
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1762)
[classes/:?]
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1714)
[classes/:?]
at
org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:794)
[classes/:?]
at
org.apache.ignite.internal.binary.BinaryObjectImpl.deserialize(BinaryObjectImpl.java:636)
[classes/:?]
at binary.BinaryEnumExample.main(BinaryEnumExample.java:20) [classes/:?]
Caused by: org.apache.ignite.binary.BinaryObjectException: Unexpected field
type [pos=24, expected=Enum, actual=Enum]
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.checkFlagNoHandles(BinaryReaderExImpl.java:1677)
~[classes/:?]
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.readEnum0(BinaryReaderExImpl.java:1403)
~[classes/:?]
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.readEnum(BinaryReaderExImpl.java:1387)
~[classes/:?]
at
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:885)
~[classes/:?]
at
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:702)
~[classes/:?]
at
org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:187)
~[classes/:?]
... 6 more
Exception in thread "main" class
org.apache.ignite.binary.BinaryObjectException: Failed to deserialize object
[typeName=binary.BinaryEnumExample$TestClass]
at
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:926)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1762)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1714)
at
org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:794)
at
org.apache.ignite.internal.binary.BinaryObjectImpl.deserialize(BinaryObjectImpl.java:636)
at binary.BinaryEnumExample.main(BinaryEnumExample.java:20)
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to read
field [name=theEnum]
at
org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:191)
at
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:887)
... 5 more
Caused by: class org.apache.ignite.binary.BinaryObjectException: Unexpected
field type [pos=24, expected=Enum, actual=Enum]
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.checkFlagNoHandles(BinaryReaderExImpl.java:1677)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.readEnum0(BinaryReaderExImpl.java:1403)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.readEnum(BinaryReaderExImpl.java:1387)
at
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryFieldAccessor.java:885)
at
org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(BinaryFieldAccessor.java:702)
at
org.apache.ignite.internal.binary.BinaryFieldAccessor.read(BinaryFieldAccessor.java:187)
... 6 more{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12567) H2Tree goes into illegal state when non-indexed columns are dropped
Konstantin Orlov created IGNITE-12567:
-
Summary: H2Tree goes into illegal state when non-indexed columns
are dropped
Key: IGNITE-12567
URL: https://issues.apache.org/jira/browse/IGNITE-12567
Project: Ignite
Issue Type: Bug
Components: sql
Reporter: Konstantin Orlov
Assignee: Konstantin Orlov
*How to reproduce:*
* {{CREATE TABLE tbl (f0 INT, f1 INT, f2 INT, f3 INT, f4 INT, PRIMARY KEY (f3,
f4))}}
* populate table with some data
* {{ALTER TABLE tbl DROP COLUMN f1, f2}}
* try to execute query which will use pk index: {{SELECT * FROM tbl WHERE f3 =
1 AND f4 = 1}}
*Expected*:
* Query returns result set
*Actual*:
* Query fails with {{General error: "java.lang.ArrayIndexOutOfBoundsException:
5";}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12369) JdbcThinClient and Server compatibility broken
Konstantin Orlov created IGNITE-12369:
-
Summary: JdbcThinClient and Server compatibility broken
Key: IGNITE-12369
URL: https://issues.apache.org/jira/browse/IGNITE-12369
Project: Ignite
Issue Type: Bug
Components: sql
Affects Versions: mas
Reporter: Konstantin Orlov
Fix For: mas
Query cancellation support was introduced in IGNITE-5439. From this time server
expects that ALL request have id. It breaks compatibility with clients whose
version prior to 2.8.0.
Furthermore backport of this feature to 8.7 breaks compatibility newer build of
server with older build of client.
See {{ClientListenerProcessor#makeFilters}}. New filter with overrided
_onMessageReceived_ is created here. It tries to read request id and if there
is not enough data to read long value, it crashes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Welcome letter
Hello Ignite Community! My name is Konstantin and I want to contribute to Apache Ignite. -- Regards, Konstantin Orlov Software Engineer, St. Petersburg +7 (921) 445-65-75 https://www.gridgain.com Powered by Apache® Ignite™
