Re: IEP-44 Thin Client Discovery

[Denis Magda]

Pavel,

That would be a tremendous improvement for the recently release best effort
affinity feature. Without this capability, we force application developers
to reopen thin client connections every type a cluster is scaled out. I
believe that once the folks start using the best effort affinity, we'll be
hearing more of a feature request for what you're proposing in this thread.
So, thanks for taking care of this proactively!

As for the public API changes, do we really need any extra flag? I would
enable the capability if the best effort affinity is on. For me, it's just
a natural improvement of the latter and it sounds reasonable to reuse the
best effort affinity's flag.

-
Denis


On Mon, Apr 27, 2020 at 2:58 AM Pavel Tupitsyn  wrote:

> Igniters,
>
> I've prepared an IEP [1] and a POC [2] for Thin Client Discovery feature.
> Let's discuss it here.
>
> In particular, I'd like to address the following points:
>
> 1. Value: do you think this would be a good feature to have?
> 2. Public API changes: is a boolean property enough? Should we have
> something more complex, so users can plug in custom logic to filter and/or
> translate IPs and host names?
> 3. Server-side implementation details: should we use Compute, Node
> Attributes, or something else to retrieve client endpoints from all nodes
> in cluster?
>
> [1]
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
> [2] https://github.com/apache/ignite/pull/7744
> [3] https://issues.apache.org/jira/browse/IGNITE-12932
>

Re: Check indexes inline size tool

[Denis Magda]

Hi Sergey,

Your changes look useful from the application developer perspective.
However, I'm curious why would the one change some low-level
IGNITE_MAX_INDEX_PAYLOAD_SIZE parameter when it's advised to pass
INLINE_SIZE to CREATE TABLE to change the index size cluster-wide.

-
Denis


On Mon, Apr 27, 2020 at 5:38 AM Sergey Antonov 
wrote:

> Hi, Igniters!
>
> I'd like to share a new small feature in AI [1].
>
> For different reasons, the cluster could have a different SQL index inline
> size [2] on cluster nodes. For example due to
> different IGNITE_MAX_INDEX_PAYLOAD_SIZE [3] value on cluster nodes.
>
> The difference in index inline size may lead to performance degradation.
> I think we must compare inline sizes on node join and warn if difference
> found. Also, We should have the ability to check inline sizes on demand.
>
> I've implemented this check on node join and new command in control.sh
>
> Look at warning message and utility command output:
>
> Warn message on a node in the cluster during new node join:
>
> [2020-04-27 15:36:21,185][WARN ][tcp-disco-msg-worker-[6ba0b823
> 127.0.0.1:47502
> crd]-#17%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root] Inline
> sizes on local node and node 5bf6ca48-34a0-4aff-8db2-c0b6df303d3f are
> different. Please drop and create again these indexes to avoid performance
> problems with SQL queries. Problem indexes:
>
> PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
>
> Warn messages on a joining node, if difference found:
> [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
> 127.0.0.1:47501
> ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> Inline sizes on local node and node a86e9cea-63e8-42af-a897-cec4be51
> are different. Please drop and create again these indexes to avoid
> performance problems with SQL queries. Problem indexes:
>
> PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
> [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
> 127.0.0.1:47501
> ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> Inline sizes on local node and node a08de16d-df05-48af-a0b9-5596d9c2
> are different. Please drop and create again these indexes to avoid
> performance problems with SQL queries. Problem indexes:
>
> PUBLIC#TEST_TABLE#L_IDX(1,3),PUBLIC#TEST_TABLE#S1_IDX(1,3),PUBLIC#TEST_TABLE#I_IDX(1,3)
>
>
> Utility output, if a difference in inline sizes was found:
>
> Control utility [ver. 2.9.0-SNAPSHOT#20200427-sha1:DEV]
> 2020 Copyright(C) Apache Software Foundation
> User: santonov
> Time: 2020-04-27T15:32:25.759
> Command [CACHE] started
> Arguments: --cache check_index_inline_sizes --yes
>
> 
> Found 4 secondary indexes.
> 3 index(es) have different effective inline size on nodes. It can lead to
> performance degradation in SQL queries.
> Index(es):
>   Full index name: PUBLIC#TEST_TABLE#L_IDX nodes:
> [ca1d2bae-89d4-4e8d-ae11-6c68f390] inline size: 1, nodes:
> [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
>   Full index name: PUBLIC#TEST_TABLE#S1_IDX nodes:
> [ca1d2bae-89d4-4e8d-ae11-6c68f390] inline size: 1, nodes:
> [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
>   Full index name: PUBLIC#TEST_TABLE#I_IDX nodes:
> [ca1d2bae-89d4-4e8d-ae11-6c68f390] inline size: 1, nodes:
> [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
>
> Recommendations:
>   Check that value of property IGNITE_MAX_INDEX_PAYLOAD_SIZE are the same
> on all nodes.
>   Recreate indexes (execute DROP INDEX, CREATE INDEX commands) with
> different inline size.
> Command [CACHE] finished with code: 0
> Control utility has completed execution at: 2020-04-27T15:32:28.025
> Execution time: 2266 ms
>
> Utility output, if all indexes have the same inline size:
>
> Control utility [ver. 2.9.0-SNAPSHOT#20200427-sha1:DEV]
> 2020 Copyright(C) Apache Software Foundation
> User: santonov
> Time: 2020-04-27T15:30:20.950
> Command [CACHE] started
> Arguments: --cache check_index_inline_sizes --yes
>
> 
> Found 2 secondary indexes.
> All secondary indexes have the same effective inline size on all cluster
> nodes.
> Command [CACHE] finished with code: 0
> Control utility has completed execution at: 2020-04-27T15:30:23.428
> Execution time: 2478 ms
>
> Any objections?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-12942
> [2] https://apacheignite-sql.readme.io/docs/create-index
> [3]
>
> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteSystemProperties.html#IGNITE_MAX_INDEX_PAYLOAD_SIZE
>
> --
> BR, Sergey Antonov
>

Re: [VOTE][EXTENSION] Release Apache Ignite Spring Boot extensions 1.0.0 RC1

[Saikat Maitra]

Hi Nikolay,

1. Can we update the   DEVNOTES.txt to contain build command as below

"mvn clean install" instead of "mvn clean install
-Pall-java,all-scala,licenses -DskipTests"

I have verified that the build passes with tests and also we do not have
tasks such as all-java, all-scala or licenses.

2. Also this voting is for Apache Ignite Spring Boot extensions 1.0.0 RC1
instead of Apache Ignite Ignite 2.8.0-rc1. Is this correct?

Regards,
Saikat

On Mon, Apr 27, 2020 at 6:05 PM Denis Magda  wrote:

> Hi Nikolay,
>
> Sorry for being picky about the documentation but I would encourage us as
> the community to prepare this release artifact before sending a release for
> a vote. As you might remember, we agreed to update the release process [1]
> to ensure this practice is followed. Just in case that's a reference to the
> discussion [2].
>
> Overall, I believe the extensions need to be placed under the
> "Integrations" tab [3] of our docs. Also, our website still misses generic
> info about the extensions (repo, purpose, etc.). Should we start a separate
> discussion and push this quickly to the finish line? Ready to facilitate
> from my end.
>
>
> [1]
>
> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-4.1EnsureDocumentationReadinessandAccouncementBlogPostActivity
> <
> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-4.1EnsureDocumentationReadinessandAccouncementBlogPostActivity
> >
> [2]
>
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Changes-in-Ignite-release-process-related-to-documentation-td46471.html
> [3] https://apacheignite-mix.readme.io/docs
>
> -
> Denis
>
>
> On Mon, Apr 27, 2020 at 1:24 PM Nikolay Izhikov 
> wrote:
>
> > Dear Community,
> >
> > I have uploaded a release candidate of the two extension modules
> > `ignite-spring-boot-autoconfigure` and
> > `ignite-spring-boot-client-autoconfigure`.
> >
> > The following staging can be used for testing:
> > https://repository.apache.org/content/repositories/orgapacheignite-1478/
> >
> > Tag with name `ignite-spring-boot-1.0.0-rc1` created:
> >
> >
> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=8b5f2d9143d281b90aceb6f11dabde50db48f6b3
> >
> > Release 1.0 contains an initial implementation of the modules, please
> > refer to the RELEASE_NOTES:
> >
> >
> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=RELEASE_NOTES.txt;h=e69173583068467b44f5e148151654ade3fe2452;hb=HEAD
> >
> > Complete list of resolved issues:
> >
> >
> >
> https://issues.apache.org/jira/issues/?jql=labels%20%3D%20spring-boot-autoconfigure
> >
> >
> https://issues.apache.org/jira/issues/?jql=labels%20%3D%20spring-boot-client-autoconfigure
> >
> > DEVNOTES
> >
> >
> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=DEVNOTES.txt;h=7f25887adef738db956ae0f70c59839f73965d5f;hb=HEAD
> >
> > The vote is formal, see voting guidelines
> > https://www.apache.org/foundation/voting.html
> >
> > +1 - to accept Apache Ignite 2.8.0-rc1
> > 0 - don't care either way
> > -1 - DO NOT accept Apache Ignite Ignite 2.8.0-rc1 (explain why)
> >
> > See notes on how to verify release here
> > https://www.apache.org/info/verification.html
> > and
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification
> >
> > The vote will hold for 72 hours and will end on April 30 2020 20:20 UTC
> >
> >
> https://www.timeanddate.com/countdown/generic?iso=20200430T2320=166=cursive
>

Re: [DISCUSSION] Major changes in Ignite in 2020

[Denis Magda]

Nikolay, thanks for responding,

Would you prefer adding this item to the roadmap page after finishing the
idea testing? If there are still many unknowns it can make to the roadmap
after you share the results and propose a final solution.

-
Denis


On Mon, Apr 27, 2020 at 2:06 PM Nikolay Izhikov  wrote:

> Hello, Igniters.
>
> Right now I working on POC for some «framework» for integration testing of
> Ignite.
>
> Feature I want to see in this framework:
>
> * To manage several Ignite instances inside docker or bare metal
> servers.
> * To manage Ignite cluster
> * Start/stop arbitrary application to interact with Ignite.
> * Kill nodes
> * collect logs.
> * implement complex real-life based scenarios
> * etc.
>
> My POC are based on duck tape library [1]
>
> [1] https://github.com/confluentinc/ducktape
>
> > 25 апр. 2020 г., в 00:51, Denis Magda  написал(а):
> >
> > Dmitry, highlighted such a disclaimer in italic. Thanks.
> >
> > -
> > Denis
> >
> >
> > On Thu, Apr 23, 2020 at 3:53 AM Dmitriy Pavlov 
> wrote:
> >
> >> Hi Denis,
> >>
> >> Thank you for driving this.
> >>
> >> Igniters,
> >>
> >> I would suggest to stress that Apache Ignite community does not
> guarantee
> >> these features to be available.
> >>
> >> Can we add some kind of disclaimer that says Ignite Roadmap does not
> imply
> >> any obligations regarding availability and timeline. A number of
> >> contributions can be done on best efforts principle, it is always
> tricky to
> >> make a promise.
> >>
> >> Sincerely,
> >> Dmitriy Pavlov
> >>
> >> чт, 23 апр. 2020 г. в 00:06, Denis Magda :
> >>
> >>> Igniters,
> >>>
> >>> Here is a draft of our very first roadmap. I decided to make it damp
> >> simple
> >>> but descriptive:
> >>>
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Roadmap
> >>>
> >>> What we need to do next is to:
> >>>
> >>>   - Fill in the "Readiness Estimated Time" column with your best guess
> >> of
> >>>   when an improvement is to be ready for a release.
> >>>   - Add references to JIRAs or IEPs to the first column.
> >>>   - Add the names of those contributors who will be cooperating with
> you
> >>>   during development. Goes to the "Contributors" column.
> >>>
> >>> Once this step is complete, we'll see how many features converge around
> >> the
> >>> same date/months and we'll plan through 2.9, 2.10, etc. releases
> >>> accordingly.
> >>>
> >>> Please don't put this aside for a while, let's move on quicker. If the
> >>> roadmap misses any contributions that you are going to add, then edit
> the
> >>> wiki page.
> >>>
> >>> -
> >>> Denis
> >>>
> >>>
> >>> On Wed, Apr 15, 2020 at 3:35 AM Nikita Amelchev 
> >>> wrote:
> >>>
>  Hello, Igniters.
> 
>  I am going to contribute a new feature - profiling tool and
>  performance report. This is part of IEP-35. [1]
> 
>  The tool will be able to collect performance statistics and create a
>  human-readable report. It will help to analyze workload and to tune
>  configuration and applications.
> 
>  Example of report [2, 3, 4].
> 
>  [1]
> 
> >>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
>  [2]
> 
> >>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png
>  [3]
> 
> >>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png
>  [4]
> 
> >>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png
> 
>  сб, 11 апр. 2020 г. в 13:54, Alexei Scherbakov <
>  alexey.scherbak...@gmail.com>:
> >
> > Folks,
> >
> > I keep working on tasks related to data consistency.
> >
> > This includes:
> >
> > Lost partitions handling overhaul (almost done) and tombstones
> >> support
> > aforementioned by Ivan Rakov
> > Atomic protocol overhaul (see [1])
> >
> > The ultimate goal of the year is to prepare Ignite for passing Jepsen
>  tests.
> >
> > [1]
> >
> 
> >>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-12+Make+ATOMIC+Caches+Consistent+Again
> >
> >
> > пт, 10 апр. 2020 г. в 18:49, Denis Magda :
> >
> >> Steven,
> >>
> >> Please start a dedicated discussion for the Golang support. At the
>  moment,
> >> I'm not aware if anybody from the community planned to provide
> >>> support
> >> out-of-the-box. However, that's not a tricky task thanks to
> >> Ignite's
>  binary
> >> protocol that lets enable any programming language support easily.
> >>
> >> -
> >> Denis
> >>
> >>
> >> On Fri, Apr 10, 2020 at 8:43 AM smeadows-abb <
>  steven.mead...@us.abb.com>
> >> wrote:
> >>
> >>> First thanks for your 

Re: IGNITE-12357 Migrate Twitter module to ignite-extensions

[Saikat Maitra]

Hi Petr,

Yes, I have created an issue to update TC configuration once changes are
merged.

https://issues.apache.org/jira/browse/IGNITE-12960

Please let me know if the PR changes looks good.

Regards,
Saikat


On Mon, Apr 27, 2020 at 1:39 AM Petr Ivanov  wrote:

> Please, remember to update TC configuration or create issue to do that.
>
>
> > On 26 Apr 2020, at 20:40, Saikat Maitra  wrote:
> >
> > Hi,
> >
> > I have raised PRs for migration of twitter module in ignite-extensions
> > repository.
> >
> > Jira
> > https://issues.apache.org/jira/browse/IGNITE-12357
> >
> > PRs
> > https://github.com/apache/ignite-extensions/pull/12
> > https://github.com/apache/ignite/pull/7733
> >
> > Please review and share your feedback.
> >
> > Regards,
> > Saikat
>
>

[jira] [Created] (IGNITE-12960) Remove twitter module tests from Teamcity Streamers TestSuite

[Saikat Maitra (Jira)]

Saikat Maitra created IGNITE-12960:
--

 Summary: Remove twitter module tests from Teamcity Streamers 
TestSuite
 Key: IGNITE-12960
 URL: https://issues.apache.org/jira/browse/IGNITE-12960
 Project: Ignite
  Issue Type: Sub-task
  Components: streaming
Reporter: Saikat Maitra


Remove twitter module tests from Teamcity Streamers TestSuite

This issue is related to https://issues.apache.org/jira/browse/IGNITE-12357



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE][EXTENSION] Release Apache Ignite Spring Boot extensions 1.0.0 RC1

[Denis Magda]

Hi Nikolay,

Sorry for being picky about the documentation but I would encourage us as
the community to prepare this release artifact before sending a release for
a vote. As you might remember, we agreed to update the release process [1]
to ensure this practice is followed. Just in case that's a reference to the
discussion [2].

Overall, I believe the extensions need to be placed under the
"Integrations" tab [3] of our docs. Also, our website still misses generic
info about the extensions (repo, purpose, etc.). Should we start a separate
discussion and push this quickly to the finish line? Ready to facilitate
from my end.


[1]
https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-4.1EnsureDocumentationReadinessandAccouncementBlogPostActivity

[2]
http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Changes-in-Ignite-release-process-related-to-documentation-td46471.html
[3] https://apacheignite-mix.readme.io/docs

-
Denis


On Mon, Apr 27, 2020 at 1:24 PM Nikolay Izhikov  wrote:

> Dear Community,
>
> I have uploaded a release candidate of the two extension modules
> `ignite-spring-boot-autoconfigure` and
> `ignite-spring-boot-client-autoconfigure`.
>
> The following staging can be used for testing:
> https://repository.apache.org/content/repositories/orgapacheignite-1478/
>
> Tag with name `ignite-spring-boot-1.0.0-rc1` created:
>
> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=8b5f2d9143d281b90aceb6f11dabde50db48f6b3
>
> Release 1.0 contains an initial implementation of the modules, please
> refer to the RELEASE_NOTES:
>
> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=RELEASE_NOTES.txt;h=e69173583068467b44f5e148151654ade3fe2452;hb=HEAD
>
> Complete list of resolved issues:
>
>
> https://issues.apache.org/jira/issues/?jql=labels%20%3D%20spring-boot-autoconfigure
>
> https://issues.apache.org/jira/issues/?jql=labels%20%3D%20spring-boot-client-autoconfigure
>
> DEVNOTES
>
> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=DEVNOTES.txt;h=7f25887adef738db956ae0f70c59839f73965d5f;hb=HEAD
>
> The vote is formal, see voting guidelines
> https://www.apache.org/foundation/voting.html
>
> +1 - to accept Apache Ignite 2.8.0-rc1
> 0 - don't care either way
> -1 - DO NOT accept Apache Ignite Ignite 2.8.0-rc1 (explain why)
>
> See notes on how to verify release here
> https://www.apache.org/info/verification.html
> and
> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification
>
> The vote will hold for 72 hours and will end on April 30 2020 20:20 UTC
>
> https://www.timeanddate.com/countdown/generic?iso=20200430T2320=166=cursive

Re: [VOTE][EXTENSION] Release Apache Ignite Spring Boot extensions 1.0.0 RC1

[Sergei Ryzhov]

+1

пн, 27 апр. 2020 г. в 23:26, Nikolay Izhikov :

> +1 (binding)
>
> I’ve checked uploaded artifacts with the test application [1]
>
> [1] https://github.com/nizhikov/ignite-spring-boot-autoconfigure-example
>
> > 27 апр. 2020 г., в 23:24, Nikolay Izhikov 
> написал(а):
> >
> > Dear Community,
> >
> > I have uploaded a release candidate of the two extension modules
> `ignite-spring-boot-autoconfigure` and
> `ignite-spring-boot-client-autoconfigure`.
> >
> > The following staging can be used for testing:
> > https://repository.apache.org/content/repositories/orgapacheignite-1478/
> >
> > Tag with name `ignite-spring-boot-1.0.0-rc1` created:
> >
> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=8b5f2d9143d281b90aceb6f11dabde50db48f6b3
> >
> > Release 1.0 contains an initial implementation of the modules, please
> refer to the RELEASE_NOTES:
> >
> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=RELEASE_NOTES.txt;h=e69173583068467b44f5e148151654ade3fe2452;hb=HEAD
> >
> > Complete list of resolved issues:
> >
> >
> https://issues.apache.org/jira/issues/?jql=labels%20%3D%20spring-boot-autoconfigure
> >
> https://issues.apache.org/jira/issues/?jql=labels%20%3D%20spring-boot-client-autoconfigure
> >
> > DEVNOTES
> >
> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=DEVNOTES.txt;h=7f25887adef738db956ae0f70c59839f73965d5f;hb=HEAD
> >
> > The vote is formal, see voting guidelines
> > https://www.apache.org/foundation/voting.html
> >
> > +1 - to accept Apache Ignite 2.8.0-rc1
> > 0 - don't care either way
> > -1 - DO NOT accept Apache Ignite Ignite 2.8.0-rc1 (explain why)
> >
> > See notes on how to verify release here
> > https://www.apache.org/info/verification.html
> > and
> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification
> >
> > The vote will hold for 72 hours and will end on April 30 2020 20:20 UTC
> >
> https://www.timeanddate.com/countdown/generic?iso=20200430T2320=166=cursive
>
>

-- 
Best regards,
Sergei Ryzhov

Re: [DISCUSSION] Major changes in Ignite in 2020

[Nikolay Izhikov]

Hello, Igniters.

Right now I working on POC for some «framework» for integration testing of 
Ignite.

Feature I want to see in this framework:

* To manage several Ignite instances inside docker or bare metal 
servers.
* To manage Ignite cluster
* Start/stop arbitrary application to interact with Ignite.
* Kill nodes
* collect logs.
* implement complex real-life based scenarios
* etc.

My POC are based on duck tape library [1]

[1] https://github.com/confluentinc/ducktape

> 25 апр. 2020 г., в 00:51, Denis Magda  написал(а):
> 
> Dmitry, highlighted such a disclaimer in italic. Thanks.
> 
> -
> Denis
> 
> 
> On Thu, Apr 23, 2020 at 3:53 AM Dmitriy Pavlov  wrote:
> 
>> Hi Denis,
>> 
>> Thank you for driving this.
>> 
>> Igniters,
>> 
>> I would suggest to stress that Apache Ignite community does not guarantee
>> these features to be available.
>> 
>> Can we add some kind of disclaimer that says Ignite Roadmap does not imply
>> any obligations regarding availability and timeline. A number of
>> contributions can be done on best efforts principle, it is always tricky to
>> make a promise.
>> 
>> Sincerely,
>> Dmitriy Pavlov
>> 
>> чт, 23 апр. 2020 г. в 00:06, Denis Magda :
>> 
>>> Igniters,
>>> 
>>> Here is a draft of our very first roadmap. I decided to make it damp
>> simple
>>> but descriptive:
>>> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Roadmap
>>> 
>>> What we need to do next is to:
>>> 
>>>   - Fill in the "Readiness Estimated Time" column with your best guess
>> of
>>>   when an improvement is to be ready for a release.
>>>   - Add references to JIRAs or IEPs to the first column.
>>>   - Add the names of those contributors who will be cooperating with you
>>>   during development. Goes to the "Contributors" column.
>>> 
>>> Once this step is complete, we'll see how many features converge around
>> the
>>> same date/months and we'll plan through 2.9, 2.10, etc. releases
>>> accordingly.
>>> 
>>> Please don't put this aside for a while, let's move on quicker. If the
>>> roadmap misses any contributions that you are going to add, then edit the
>>> wiki page.
>>> 
>>> -
>>> Denis
>>> 
>>> 
>>> On Wed, Apr 15, 2020 at 3:35 AM Nikita Amelchev 
>>> wrote:
>>> 
 Hello, Igniters.
 
 I am going to contribute a new feature - profiling tool and
 performance report. This is part of IEP-35. [1]
 
 The tool will be able to collect performance statistics and create a
 human-readable report. It will help to analyze workload and to tune
 configuration and applications.
 
 Example of report [2, 3, 4].
 
 [1]
 
>>> 
>> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
 [2]
 
>>> 
>> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png
 [3]
 
>>> 
>> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png
 [4]
 
>>> 
>> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png
 
 сб, 11 апр. 2020 г. в 13:54, Alexei Scherbakov <
 alexey.scherbak...@gmail.com>:
> 
> Folks,
> 
> I keep working on tasks related to data consistency.
> 
> This includes:
> 
> Lost partitions handling overhaul (almost done) and tombstones
>> support
> aforementioned by Ivan Rakov
> Atomic protocol overhaul (see [1])
> 
> The ultimate goal of the year is to prepare Ignite for passing Jepsen
 tests.
> 
> [1]
> 
 
>>> 
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-12+Make+ATOMIC+Caches+Consistent+Again
> 
> 
> пт, 10 апр. 2020 г. в 18:49, Denis Magda :
> 
>> Steven,
>> 
>> Please start a dedicated discussion for the Golang support. At the
 moment,
>> I'm not aware if anybody from the community planned to provide
>>> support
>> out-of-the-box. However, that's not a tricky task thanks to
>> Ignite's
 binary
>> protocol that lets enable any programming language support easily.
>> 
>> -
>> Denis
>> 
>> 
>> On Fri, Apr 10, 2020 at 8:43 AM smeadows-abb <
 steven.mead...@us.abb.com>
>> wrote:
>> 
>>> First thanks for your quick response.
>>> 
>>> I looked at  https://github.com/amsokol/ignite-go-client and its
>>> NOT
>>> completed and nothing has been for last 16 months. Initial test
>>> with
>>> package
>>> failed, so trying to determine your project roadmap with regards
>> to
>> Golang
>>> and maybe Rust support.
>>> 
>>> I'm NOT sure of 'AFAIK' ?
>>> 
>>> We may need to implement your Restful API to provide support for
 Golang
>> and
>>> Rust, provided it's complete?
>>> 
>>> Thanks,
>>>  Steve
>>> 
>>> 
>>> 

Re: IEP-44 Thin Client Discovery

[Igor Sapego]

Yeah, I remember that because I used this approach when I implemented
first experimental version of partition awareness for the C++ thin client.
Only thing we need to add is the "ThinClientPort" node attribute, or
something like that.

Best Regards,
Igor


On Mon, Apr 27, 2020 at 8:41 PM Pavel Tupitsyn  wrote:

> Igor, actually you are right.
>
> We already have org.apache.ignite.ips node attribute, which contains all
> the IPs.
> No need to add anything else, just filter the value according to
> IgniteConfiguration.LocalHost.
> I'll update the IEP and POC.
>
> Thanks
>
>
>
> On Mon, Apr 27, 2020 at 8:17 PM Pavel Tupitsyn 
> wrote:
>
> > > There is already a list of addresses of every node kept by every node,
> > isn't it?
> >
> > Can you please elaborate, what exactly do you mean here?
> > Sure, we can get addresses of other nodes from Discovery/Communication
> > SPIs,
> > but those addresses may be not what we want. Nodes can belong to multiple
> > subnets.
> >
> > For example, server nodes may communicate over 192.168.0.0/24, but
> expose
> > client endpoints over 10.0.0.0/24 as well.
> >
> > We have to conform with IgniteConfiguration.LocalHost setting.
> > If it is not set, or set to 0.0.0.0, we should gather IPs from all
> > interfaces.
> > But if it is set to something, we should gather only matching IPs.
> >
> > On Mon, Apr 27, 2020 at 5:43 PM Igor Sapego  wrote:
> >
> >> Great, this feature is long awaited.
> >>
> >> 1. I believe so. Since I've proposed Partition Awareness feature I was
> >> thinking
> >> about a way for clients to discover cluster nodes.
> >>
> >> 2. In my opinion a simple boolean flag is enough for the beginning. In
> >> future
> >> maybe we can add a node filter. This can be useful in cases when part
> of a
> >> cluster is not available from client network or if user wants to avoid
> >> connection
> >> to a node for some other reason.
> >>
> >> 3. There is already a list of addresses of every node kept by every
> node,
> >> isn't it?
> >>
> >> Best Regards,
> >> Igor
> >>
> >>
> >> On Mon, Apr 27, 2020 at 12:58 PM Pavel Tupitsyn 
> >> wrote:
> >>
> >> > Igniters,
> >> >
> >> > I've prepared an IEP [1] and a POC [2] for Thin Client Discovery
> >> feature.
> >> > Let's discuss it here.
> >> >
> >> > In particular, I'd like to address the following points:
> >> >
> >> > 1. Value: do you think this would be a good feature to have?
> >> > 2. Public API changes: is a boolean property enough? Should we have
> >> > something more complex, so users can plug in custom logic to filter
> >> and/or
> >> > translate IPs and host names?
> >> > 3. Server-side implementation details: should we use Compute, Node
> >> > Attributes, or something else to retrieve client endpoints from all
> >> nodes
> >> > in cluster?
> >> >
> >> > [1]
> >> >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
> >> > [2] https://github.com/apache/ignite/pull/7744
> >> > [3] https://issues.apache.org/jira/browse/IGNITE-12932
> >> >
> >>
> >
>

Re: [VOTE][EXTENSION] Release Apache Ignite Spring Boot extensions 1.0.0 RC1

[Nikolay Izhikov]

+1 (binding)

I’ve checked uploaded artifacts with the test application [1]

[1] https://github.com/nizhikov/ignite-spring-boot-autoconfigure-example

> 27 апр. 2020 г., в 23:24, Nikolay Izhikov  написал(а):
> 
> Dear Community,
> 
> I have uploaded a release candidate of the two extension modules 
> `ignite-spring-boot-autoconfigure` and 
> `ignite-spring-boot-client-autoconfigure`.
> 
> The following staging can be used for testing:
> https://repository.apache.org/content/repositories/orgapacheignite-1478/
> 
> Tag with name `ignite-spring-boot-1.0.0-rc1` created:
> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=8b5f2d9143d281b90aceb6f11dabde50db48f6b3
> 
> Release 1.0 contains an initial implementation of the modules, please refer 
> to the RELEASE_NOTES:
> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=RELEASE_NOTES.txt;h=e69173583068467b44f5e148151654ade3fe2452;hb=HEAD
> 
> Complete list of resolved issues:
> 
> https://issues.apache.org/jira/issues/?jql=labels%20%3D%20spring-boot-autoconfigure
> https://issues.apache.org/jira/issues/?jql=labels%20%3D%20spring-boot-client-autoconfigure
> 
> DEVNOTES
> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=DEVNOTES.txt;h=7f25887adef738db956ae0f70c59839f73965d5f;hb=HEAD
> 
> The vote is formal, see voting guidelines
> https://www.apache.org/foundation/voting.html
> 
> +1 - to accept Apache Ignite 2.8.0-rc1
> 0 - don't care either way
> -1 - DO NOT accept Apache Ignite Ignite 2.8.0-rc1 (explain why)
> 
> See notes on how to verify release here
> https://www.apache.org/info/verification.html
> and 
> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification
> 
> The vote will hold for 72 hours and will end on April 30 2020 20:20 UTC
> https://www.timeanddate.com/countdown/generic?iso=20200430T2320=166=cursive

[VOTE][EXTENSION] Release Apache Ignite Spring Boot extensions 1.0.0 RC1

[Nikolay Izhikov]

Dear Community,

I have uploaded a release candidate of the two extension modules 
`ignite-spring-boot-autoconfigure` and 
`ignite-spring-boot-client-autoconfigure`.

The following staging can be used for testing:
https://repository.apache.org/content/repositories/orgapacheignite-1478/

Tag with name `ignite-spring-boot-1.0.0-rc1` created:
https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=8b5f2d9143d281b90aceb6f11dabde50db48f6b3

Release 1.0 contains an initial implementation of the modules, please refer to 
the RELEASE_NOTES:
https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=RELEASE_NOTES.txt;h=e69173583068467b44f5e148151654ade3fe2452;hb=HEAD

Complete list of resolved issues:

https://issues.apache.org/jira/issues/?jql=labels%20%3D%20spring-boot-autoconfigure
https://issues.apache.org/jira/issues/?jql=labels%20%3D%20spring-boot-client-autoconfigure

DEVNOTES
https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=DEVNOTES.txt;h=7f25887adef738db956ae0f70c59839f73965d5f;hb=HEAD

The vote is formal, see voting guidelines
https://www.apache.org/foundation/voting.html

+1 - to accept Apache Ignite 2.8.0-rc1
0 - don't care either way
-1 - DO NOT accept Apache Ignite Ignite 2.8.0-rc1 (explain why)

See notes on how to verify release here
https://www.apache.org/info/verification.html
and 
https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification

The vote will hold for 72 hours and will end on April 30 2020 20:20 UTC
https://www.timeanddate.com/countdown/generic?iso=20200430T2320=166=cursive

[jira] [Created] (IGNITE-12959) Add support for GenericWhitespace to checkstyle rules

[Amelchev Nikita (Jira)]

Amelchev Nikita created IGNITE-12959:


 Summary: Add support for GenericWhitespace to checkstyle rules
 Key: IGNITE-12959
 URL: https://issues.apache.org/jira/browse/IGNITE-12959
 Project: Ignite
  Issue Type: Task
Reporter: Amelchev Nikita
Assignee: Amelchev Nikita


Add a new rule to checkstyle according to Apache Ignite Whitespaces and empty 
lines conventions.



https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines#CodingGuidelines-Whitespacesandemptylines




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-12958) Add licenses check to Travis configuration

[Amelchev Nikita (Jira)]

Amelchev Nikita created IGNITE-12958:


 Summary: Add licenses check to Travis configuration
 Key: IGNITE-12958
 URL: https://issues.apache.org/jira/browse/IGNITE-12958
 Project: Ignite
  Issue Type: Task
Reporter: Amelchev Nikita
Assignee: Amelchev Nikita


1. Add licenses check to Travis configuration
2. Make checks on the compile lifecycle



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: IEP-44 Thin Client Discovery

[Pavel Tupitsyn]

Igor, actually you are right.

We already have org.apache.ignite.ips node attribute, which contains all
the IPs.
No need to add anything else, just filter the value according to
IgniteConfiguration.LocalHost.
I'll update the IEP and POC.

Thanks



On Mon, Apr 27, 2020 at 8:17 PM Pavel Tupitsyn  wrote:

> > There is already a list of addresses of every node kept by every node,
> isn't it?
>
> Can you please elaborate, what exactly do you mean here?
> Sure, we can get addresses of other nodes from Discovery/Communication
> SPIs,
> but those addresses may be not what we want. Nodes can belong to multiple
> subnets.
>
> For example, server nodes may communicate over 192.168.0.0/24, but expose
> client endpoints over 10.0.0.0/24 as well.
>
> We have to conform with IgniteConfiguration.LocalHost setting.
> If it is not set, or set to 0.0.0.0, we should gather IPs from all
> interfaces.
> But if it is set to something, we should gather only matching IPs.
>
> On Mon, Apr 27, 2020 at 5:43 PM Igor Sapego  wrote:
>
>> Great, this feature is long awaited.
>>
>> 1. I believe so. Since I've proposed Partition Awareness feature I was
>> thinking
>> about a way for clients to discover cluster nodes.
>>
>> 2. In my opinion a simple boolean flag is enough for the beginning. In
>> future
>> maybe we can add a node filter. This can be useful in cases when part of a
>> cluster is not available from client network or if user wants to avoid
>> connection
>> to a node for some other reason.
>>
>> 3. There is already a list of addresses of every node kept by every node,
>> isn't it?
>>
>> Best Regards,
>> Igor
>>
>>
>> On Mon, Apr 27, 2020 at 12:58 PM Pavel Tupitsyn 
>> wrote:
>>
>> > Igniters,
>> >
>> > I've prepared an IEP [1] and a POC [2] for Thin Client Discovery
>> feature.
>> > Let's discuss it here.
>> >
>> > In particular, I'd like to address the following points:
>> >
>> > 1. Value: do you think this would be a good feature to have?
>> > 2. Public API changes: is a boolean property enough? Should we have
>> > something more complex, so users can plug in custom logic to filter
>> and/or
>> > translate IPs and host names?
>> > 3. Server-side implementation details: should we use Compute, Node
>> > Attributes, or something else to retrieve client endpoints from all
>> nodes
>> > in cluster?
>> >
>> > [1]
>> >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
>> > [2] https://github.com/apache/ignite/pull/7744
>> > [3] https://issues.apache.org/jira/browse/IGNITE-12932
>> >
>>
>

Re: IEP-44 Thin Client Discovery

[Pavel Tupitsyn]

> There is already a list of addresses of every node kept by every node,
isn't it?

Can you please elaborate, what exactly do you mean here?
Sure, we can get addresses of other nodes from Discovery/Communication SPIs,
but those addresses may be not what we want. Nodes can belong to multiple
subnets.

For example, server nodes may communicate over 192.168.0.0/24, but expose
client endpoints over 10.0.0.0/24 as well.

We have to conform with IgniteConfiguration.LocalHost setting.
If it is not set, or set to 0.0.0.0, we should gather IPs from all
interfaces.
But if it is set to something, we should gather only matching IPs.

On Mon, Apr 27, 2020 at 5:43 PM Igor Sapego  wrote:

> Great, this feature is long awaited.
>
> 1. I believe so. Since I've proposed Partition Awareness feature I was
> thinking
> about a way for clients to discover cluster nodes.
>
> 2. In my opinion a simple boolean flag is enough for the beginning. In
> future
> maybe we can add a node filter. This can be useful in cases when part of a
> cluster is not available from client network or if user wants to avoid
> connection
> to a node for some other reason.
>
> 3. There is already a list of addresses of every node kept by every node,
> isn't it?
>
> Best Regards,
> Igor
>
>
> On Mon, Apr 27, 2020 at 12:58 PM Pavel Tupitsyn 
> wrote:
>
> > Igniters,
> >
> > I've prepared an IEP [1] and a POC [2] for Thin Client Discovery feature.
> > Let's discuss it here.
> >
> > In particular, I'd like to address the following points:
> >
> > 1. Value: do you think this would be a good feature to have?
> > 2. Public API changes: is a boolean property enough? Should we have
> > something more complex, so users can plug in custom logic to filter
> and/or
> > translate IPs and host names?
> > 3. Server-side implementation details: should we use Compute, Node
> > Attributes, or something else to retrieve client endpoints from all nodes
> > in cluster?
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
> > [2] https://github.com/apache/ignite/pull/7744
> > [3] https://issues.apache.org/jira/browse/IGNITE-12932
> >
>

[jira] [Created] (IGNITE-12957) .NET: Docs have unavailable resource logo3.png

[Pavel Tupitsyn (Jira)]

Pavel Tupitsyn created IGNITE-12957:
---

 Summary: .NET: Docs have unavailable resource logo3.png
 Key: IGNITE-12957
 URL: https://issues.apache.org/jira/browse/IGNITE-12957
 Project: Ignite
  Issue Type: New Feature
  Components: documentation, platforms
Affects Versions: 2.8
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 2.9






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-12956) Fully prepared TX rolled back on recovery if TX coordinator failed and some primary has only reads

[Anton Vinogradov (Jira)]

Anton Vinogradov created IGNITE-12956:
-

 Summary: Fully prepared TX rolled back on recovery if TX 
coordinator failed and some primary has only reads
 Key: IGNITE-12956
 URL: https://issues.apache.org/jira/browse/IGNITE-12956
 Project: Ignite
  Issue Type: Task
Reporter: Anton Vinogradov


We have 3 nodes with partitioned cache with 2 backups.
 A - tx coordinator.
 B and C - other nodes.

Let's start tx from A and perform
{noformat}
cache.put(primaryKeyA, someVal);
cache.get(primaryKeyB;
tx.prepare();
{noformat}
then kill A.

Expected: tx recovered and
{noformat}
cache.get(primaryKeyA)!=null
{noformat}
Actual: tx rolled back and
 and
{noformat}
cache.get(primaryKeyA)==null
{noformat}
Reason: Node C has only 1 active transaction (because reads not propagated to 
backup), but expect to have 2.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSSION] Hot cache backup

[Alexey Goncharuk]

Maxim,

I saw the list of the tickets you want to work on in the PR, it looks nice.
I was wondering, what part of that list are you planning to implement
before the feature is released to end users? For example, I agree with
Slava that we should implement a command-line utility part for snapshots
before the release, however I think it's better to do it in a separate
ticket.

I know we do not have a strict policy regarding big features development in
the community, so perhaps it's a good time to discuss this? If we are ok
with merging separate tickets to master, how we ensure a complete feature
is released to public? If not, should we create a feature branch and wait
for all related tickets to be merged there? Will be glad to discuss this in
a separate thread if needed.

пн, 27 апр. 2020 г. в 14:38, Maxim Muzafarov :

> Folks,
>
>
> Are there any cases left which we need to discuss?
>
> Do you have any questions?
> I'm ready to provide all the details you need for the review.
>
> Who else what to take a look at my changes [1] [2]?
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-11073
> [2] https://github.com/apache/ignite/pull/7607
>
> On Fri, 24 Apr 2020 at 15:01, Maxim Muzafarov  wrote:
> >
> > Alexey,
> >
> >
> > I've addressed all your comments, please, take a look at the PR [1].
> > Additional tests were added.
> > Additional comments with further steps were added.
> >
> >
> > [1] https://github.com/apache/ignite/pull/7607
> > [2] https://issues.apache.org/jira/browse/IGNITE-11073
> >
> > On Tue, 21 Apr 2020 at 09:53, Alexey Goncharuk
> >  wrote:
> > >
> > > Maxim,
> > >
> > > I've left my comments in the PR.
> > >
> > > пн, 20 апр. 2020 г. в 12:52, Maxim Muzafarov :
> > >
> > > > Alex P,
> > > > Thank you for the great sophisticated review.
> > > >
> > > >
> > > > Alexey G,
> > > > Will you take a look at my changes[1]?
> > > > The fresh TC.Bot visa attached.
> > > >
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-11073
> > > >
> > > > On Mon, 20 Apr 2020 at 11:54, Alex Plehanov  >
> > > > wrote:
> > > > >
> > > > > Maxim, I've reviewed your PR and it looks good to me. Good job!
> > > > >
> > > > > пт, 10 апр. 2020 г. в 19:43, Alexey Goncharuk <
> > > > alexey.goncha...@gmail.com>:
> > > > >
> > > > > > Maxim,
> > > > > >
> > > > > > Thanks for raising this PR. I will do a review during next week.
> > > > > >
> > > > > > --AG
> > > > > >
> > > >
>

Re: IEP-44 Thin Client Discovery

[Igor Sapego]

Great, this feature is long awaited.

1. I believe so. Since I've proposed Partition Awareness feature I was
thinking
about a way for clients to discover cluster nodes.

2. In my opinion a simple boolean flag is enough for the beginning. In
future
maybe we can add a node filter. This can be useful in cases when part of a
cluster is not available from client network or if user wants to avoid
connection
to a node for some other reason.

3. There is already a list of addresses of every node kept by every node,
isn't it?

Best Regards,
Igor


On Mon, Apr 27, 2020 at 12:58 PM Pavel Tupitsyn 
wrote:

> Igniters,
>
> I've prepared an IEP [1] and a POC [2] for Thin Client Discovery feature.
> Let's discuss it here.
>
> In particular, I'd like to address the following points:
>
> 1. Value: do you think this would be a good feature to have?
> 2. Public API changes: is a boolean property enough? Should we have
> something more complex, so users can plug in custom logic to filter and/or
> translate IPs and host names?
> 3. Server-side implementation details: should we use Compute, Node
> Attributes, or something else to retrieve client endpoints from all nodes
> in cluster?
>
> [1]
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
> [2] https://github.com/apache/ignite/pull/7744
> [3] https://issues.apache.org/jira/browse/IGNITE-12932
>

Re: [DISCUSSION] `static final` code-style constant naming rules

[Ivan Pavlukhin]

Maxim,

> 1. The Apache Ignite may start multiple instances...
Ignite codebase contains something more than Ignite instances code:
control utility, tests. And it does not sound good to me to use some
project-specific naming rule if it is not really critical. I wish
Ignite coding guidelines are as common as possible in order simplify
things for new contributors. I briefly examined code and found some
static final ThreadLocals and Atomic variables. Both are not
constants.

Currently my opinion is that the proposed change can make more harm than profit.

Best regards,
Ivan Pavlukhin

пн, 27 апр. 2020 г. в 14:57, Maxim Muzafarov :
>
> Pavel, Ivan,
>
>
> Let me clarify some details you missed which may lead you to the light side.
>
> 1. The Apache Ignite may start multiple instances on the same VM, so
> keeping a class field `static` for mutable objects have no sense in
> most of the cases. In case of keeping, the immutable structure will be
> shared between Ignite instances which happens in very rare and special
> cases. According to this assumption, most of `final static` fields are
> really constants.
> Even thread-local objects probably better to keep `non-static` as
> Alexey mentioned before [1].
>
> 2. The common `final static` fields like - `log`, `logger`,
> `serialVersionUID` added to the exclusion list. If it requires we can
> extend it more.
>
> Yes, I know the java code naming convention mentioned above by myself,
> but exactly for Ignite IMHO it's better to apply for all `static
> final` fields naming as `constant naming`.
>
> [1] 
> http://apache-ignite-developers.2346864.n4.nabble.com/Class-field-ThreadLocal-Why-not-static-tp34878p34881.html
>
> On Mon, 27 Apr 2020 at 14:13, Anton Vinogradov  wrote:
> >
> > >> +1 to force upper case for `static final` variables.
> > +1 too
> >
> > On Mon, Apr 27, 2020 at 12:08 PM Nikolay Izhikov 
> > wrote:
> >
> > > +1 to force upper case for `static final` variables.
> > >
> > > > 25 апр. 2020 г., в 07:39, Ivan Pavlukhin 
> > > написал(а):
> > > >
> > > > Maxim,
> > > >
> > > > Thank you for efforts in a code quality improvement!
> > > >
> > > > Unfortunately I do not agree with the proposal. Usually I like to
> > > > refer a following guide [1]. While a question "what is a constant" is
> > > > not trivial but I suppose that every static final field capitalization
> > > > can cause some problems. For example, a developer can make false
> > > > assumptions about immutability and thread safety basing on a
> > > > capitalized field name.
> > > >
> > > > [1]
> > > https://google.github.io/styleguide/javaguide.html#s5.2.4-constant-names
> > > >
> > > > Best regards,
> > > > Ivan Pavlukhin
> > > >
> > > > пт, 24 апр. 2020 г. в 21:58, Pavel Tupitsyn :
> > > >>
> > > >>> All static final object reference types that are never followed by "."
> > > >> (dot)
> > > >> With this way of thinking we can say that everything is a constant if 
> > > >> we
> > > >> don't change it - no need for static final.
> > > >> "Constant" is usually something that you can't change, even you want -
> > > >> compiler won't let you.
> > > >> In Java that would be static final primitives and read-only objects 
> > > >> like
> > > >> String.
> > > >>
> > > >> On Fri, Apr 24, 2020 at 8:25 PM Pavel Pereslegin 
> > > wrote:
> > > >>
> > > >>> Maxim,
> > > >>>
> > >  But what exactly is `constant` means?! I'd suggest applying this rule
> > >  to all class fields with `static final` modifiers without any 
> > >  clauses.
> > >  (check Java Naming convention [2] paragraph 3.3).
> > > >>>
> > > >>> I disagree with this and want to clarify what exactly the Java naming
> > > >>> convention says:
> > > >>>
> > > > The following are considered to be constants:
> > > > 1. All static final primitive types (Remember that all interface
> > > fields
> > > >>> are inherently static final).
> > > > 2. All static final object reference types that are never followed 
> > > > by
> > > >>> "." (dot).
> > > > 3. All static final arrays that are never followed by"[" (dot)
> > > >>>
> > > >>> I don't see that convention says "any static final field".
> > > >>>
> > > >>> пт, 24 апр. 2020 г. в 20:11, Sergey Antonov  > > >:
> > > 
> > >  Maxim, It's a good idea!
> > > 
> > >  Please don't forget to update out code style guidelines too.
> > > 
> > >  Thank you for keeping the code cleaner!
> > > 
> > >  пт, 24 апр. 2020 г. в 19:49, Maxim Muzafarov :
> > > 
> > > > Igniters,
> > > >
> > > >
> > > > It is not directly mentioned through the Apache Ignite Coding
> > > > Guidelines [1] about naming the `static final` class fields using
> > > only
> > > > upper-case letters. I'd like to suggest to fill this gap.
> > > >
> > > >> Constants should all be upper-case.
> > > > But what exactly is `constant` means?! I'd suggest applying this 
> > > > rule
> > > > to all class fields with `static final` modifiers without 

[jira] [Created] (IGNITE-12955) Document a new command control.sh --cache check_index_inline_sizes

[Sergey Antonov (Jira)]

Sergey Antonov created IGNITE-12955:
---

 Summary: Document a new command control.sh --cache 
check_index_inline_sizes
 Key: IGNITE-12955
 URL: https://issues.apache.org/jira/browse/IGNITE-12955
 Project: Ignite
  Issue Type: New Feature
  Components: documentation
Reporter: Sergey Antonov
 Fix For: 2.9


Please look at IGNITE-12942 and devlist discussion linked to IGNITE-12942 for 
all information.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Check indexes inline size tool

[Sergey Antonov]

Hi, Igniters!

I'd like to share a new small feature in AI [1].

For different reasons, the cluster could have a different SQL index inline
size [2] on cluster nodes. For example due to
different IGNITE_MAX_INDEX_PAYLOAD_SIZE [3] value on cluster nodes.

The difference in index inline size may lead to performance degradation.
I think we must compare inline sizes on node join and warn if difference
found. Also, We should have the ability to check inline sizes on demand.

I've implemented this check on node join and new command in control.sh

Look at warning message and utility command output:

Warn message on a node in the cluster during new node join:

[2020-04-27 15:36:21,185][WARN ][tcp-disco-msg-worker-[6ba0b823
127.0.0.1:47502
crd]-#17%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root] Inline
sizes on local node and node 5bf6ca48-34a0-4aff-8db2-c0b6df303d3f are
different. Please drop and create again these indexes to avoid performance
problems with SQL queries. Problem indexes:
PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)

Warn messages on a joining node, if difference found:
[2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
127.0.0.1:47501]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
Inline sizes on local node and node a86e9cea-63e8-42af-a897-cec4be51
are different. Please drop and create again these indexes to avoid
performance problems with SQL queries. Problem indexes:
PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
[2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
127.0.0.1:47501]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
Inline sizes on local node and node a08de16d-df05-48af-a0b9-5596d9c2
are different. Please drop and create again these indexes to avoid
performance problems with SQL queries. Problem indexes:
PUBLIC#TEST_TABLE#L_IDX(1,3),PUBLIC#TEST_TABLE#S1_IDX(1,3),PUBLIC#TEST_TABLE#I_IDX(1,3)


Utility output, if a difference in inline sizes was found:

Control utility [ver. 2.9.0-SNAPSHOT#20200427-sha1:DEV]
2020 Copyright(C) Apache Software Foundation
User: santonov
Time: 2020-04-27T15:32:25.759
Command [CACHE] started
Arguments: --cache check_index_inline_sizes --yes

Found 4 secondary indexes.
3 index(es) have different effective inline size on nodes. It can lead to
performance degradation in SQL queries.
Index(es):
  Full index name: PUBLIC#TEST_TABLE#L_IDX nodes:
[ca1d2bae-89d4-4e8d-ae11-6c68f390] inline size: 1, nodes:
[8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
  Full index name: PUBLIC#TEST_TABLE#S1_IDX nodes:
[ca1d2bae-89d4-4e8d-ae11-6c68f390] inline size: 1, nodes:
[8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
  Full index name: PUBLIC#TEST_TABLE#I_IDX nodes:
[ca1d2bae-89d4-4e8d-ae11-6c68f390] inline size: 1, nodes:
[8327abd1-df08-4b97-8720-de95e363e745] inline size: 2

Recommendations:
  Check that value of property IGNITE_MAX_INDEX_PAYLOAD_SIZE are the same
on all nodes.
  Recreate indexes (execute DROP INDEX, CREATE INDEX commands) with
different inline size.
Command [CACHE] finished with code: 0
Control utility has completed execution at: 2020-04-27T15:32:28.025
Execution time: 2266 ms

Utility output, if all indexes have the same inline size:

Control utility [ver. 2.9.0-SNAPSHOT#20200427-sha1:DEV]
2020 Copyright(C) Apache Software Foundation
User: santonov
Time: 2020-04-27T15:30:20.950
Command [CACHE] started
Arguments: --cache check_index_inline_sizes --yes

Found 2 secondary indexes.
All secondary indexes have the same effective inline size on all cluster
nodes.
Command [CACHE] finished with code: 0
Control utility has completed execution at: 2020-04-27T15:30:23.428
Execution time: 2478 ms

Any objections?

[1] https://issues.apache.org/jira/browse/IGNITE-12942
[2] https://apacheignite-sql.readme.io/docs/create-index
[3]
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteSystemProperties.html#IGNITE_MAX_INDEX_PAYLOAD_SIZE

-- 
BR, Sergey Antonov

Re: [DISCUSSION] `static final` code-style constant naming rules

[Maxim Muzafarov]

Pavel, Ivan,


Let me clarify some details you missed which may lead you to the light side.

1. The Apache Ignite may start multiple instances on the same VM, so
keeping a class field `static` for mutable objects have no sense in
most of the cases. In case of keeping, the immutable structure will be
shared between Ignite instances which happens in very rare and special
cases. According to this assumption, most of `final static` fields are
really constants.
Even thread-local objects probably better to keep `non-static` as
Alexey mentioned before [1].

2. The common `final static` fields like - `log`, `logger`,
`serialVersionUID` added to the exclusion list. If it requires we can
extend it more.

Yes, I know the java code naming convention mentioned above by myself,
but exactly for Ignite IMHO it's better to apply for all `static
final` fields naming as `constant naming`.

[1] 
http://apache-ignite-developers.2346864.n4.nabble.com/Class-field-ThreadLocal-Why-not-static-tp34878p34881.html

On Mon, 27 Apr 2020 at 14:13, Anton Vinogradov  wrote:
>
> >> +1 to force upper case for `static final` variables.
> +1 too
>
> On Mon, Apr 27, 2020 at 12:08 PM Nikolay Izhikov 
> wrote:
>
> > +1 to force upper case for `static final` variables.
> >
> > > 25 апр. 2020 г., в 07:39, Ivan Pavlukhin 
> > написал(а):
> > >
> > > Maxim,
> > >
> > > Thank you for efforts in a code quality improvement!
> > >
> > > Unfortunately I do not agree with the proposal. Usually I like to
> > > refer a following guide [1]. While a question "what is a constant" is
> > > not trivial but I suppose that every static final field capitalization
> > > can cause some problems. For example, a developer can make false
> > > assumptions about immutability and thread safety basing on a
> > > capitalized field name.
> > >
> > > [1]
> > https://google.github.io/styleguide/javaguide.html#s5.2.4-constant-names
> > >
> > > Best regards,
> > > Ivan Pavlukhin
> > >
> > > пт, 24 апр. 2020 г. в 21:58, Pavel Tupitsyn :
> > >>
> > >>> All static final object reference types that are never followed by "."
> > >> (dot)
> > >> With this way of thinking we can say that everything is a constant if we
> > >> don't change it - no need for static final.
> > >> "Constant" is usually something that you can't change, even you want -
> > >> compiler won't let you.
> > >> In Java that would be static final primitives and read-only objects like
> > >> String.
> > >>
> > >> On Fri, Apr 24, 2020 at 8:25 PM Pavel Pereslegin 
> > wrote:
> > >>
> > >>> Maxim,
> > >>>
> >  But what exactly is `constant` means?! I'd suggest applying this rule
> >  to all class fields with `static final` modifiers without any clauses.
> >  (check Java Naming convention [2] paragraph 3.3).
> > >>>
> > >>> I disagree with this and want to clarify what exactly the Java naming
> > >>> convention says:
> > >>>
> > > The following are considered to be constants:
> > > 1. All static final primitive types (Remember that all interface
> > fields
> > >>> are inherently static final).
> > > 2. All static final object reference types that are never followed by
> > >>> "." (dot).
> > > 3. All static final arrays that are never followed by"[" (dot)
> > >>>
> > >>> I don't see that convention says "any static final field".
> > >>>
> > >>> пт, 24 апр. 2020 г. в 20:11, Sergey Antonov  > >:
> > 
> >  Maxim, It's a good idea!
> > 
> >  Please don't forget to update out code style guidelines too.
> > 
> >  Thank you for keeping the code cleaner!
> > 
> >  пт, 24 апр. 2020 г. в 19:49, Maxim Muzafarov :
> > 
> > > Igniters,
> > >
> > >
> > > It is not directly mentioned through the Apache Ignite Coding
> > > Guidelines [1] about naming the `static final` class fields using
> > only
> > > upper-case letters. I'd like to suggest to fill this gap.
> > >
> > >> Constants should all be upper-case.
> > > But what exactly is `constant` means?! I'd suggest applying this rule
> > > to all class fields with `static final` modifiers without any
> > clauses.
> > > (check Java Naming convention [2] paragraph 3.3).
> > >
> > >
> > > I've prepared PR [3] with capitalizing letters on all of the constant
> > > names simultaneously with supporting the standard ConstantName
> > > checkstyle [4] rule.
> > >
> > > Can we proceed with this change?
> > > WDYT?
> > >
> > >
> > > [1]
> > >
> > >>>
> > https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines#CodingGuidelines-Naming
> > > [2]
> > >
> > >>>
> > https://web.archive.org/web/20120911192801/developers.sun.com/sunstudio/products/archive/whitepapers/java-style.pdf
> > > [3] https://github.com/apache/ignite/pull/7662
> > > [4] https://issues.apache.org/jira/browse/IGNITE-12888
> > >
> > 
> > 
> >  --
> >  BR, Sergey Antonov
> > >>>
> >
> >

Re: [DISCUSSION] `static final` code-style constant naming rules

[Nikita Amelchev]

Hi, Maxim

> But what exactly is `constant` means?! I'd suggest applying this rule
> to all class fields with `static final` modifiers without any clauses.
> (check Java Naming convention [2] paragraph 3.3).

I read this convention [1] and found clarifying exclusion (3.3.2):
All static final object reference types that are never followed by "." (dot).

Suppose we have a static final map. Will this map be in the upper-case
naming if remove operations present?

[1] 
https://web.archive.org/web/20120911192801/developers.sun.com/sunstudio/products/archive/whitepapers/java-style.pdf

пн, 27 апр. 2020 г. в 14:13, Anton Vinogradov :
>
> >> +1 to force upper case for `static final` variables.
> +1 too
>
> On Mon, Apr 27, 2020 at 12:08 PM Nikolay Izhikov 
> wrote:
>
> > +1 to force upper case for `static final` variables.
> >
> > > 25 апр. 2020 г., в 07:39, Ivan Pavlukhin 
> > написал(а):
> > >
> > > Maxim,
> > >
> > > Thank you for efforts in a code quality improvement!
> > >
> > > Unfortunately I do not agree with the proposal. Usually I like to
> > > refer a following guide [1]. While a question "what is a constant" is
> > > not trivial but I suppose that every static final field capitalization
> > > can cause some problems. For example, a developer can make false
> > > assumptions about immutability and thread safety basing on a
> > > capitalized field name.
> > >
> > > [1]
> > https://google.github.io/styleguide/javaguide.html#s5.2.4-constant-names
> > >
> > > Best regards,
> > > Ivan Pavlukhin
> > >
> > > пт, 24 апр. 2020 г. в 21:58, Pavel Tupitsyn :
> > >>
> > >>> All static final object reference types that are never followed by "."
> > >> (dot)
> > >> With this way of thinking we can say that everything is a constant if we
> > >> don't change it - no need for static final.
> > >> "Constant" is usually something that you can't change, even you want -
> > >> compiler won't let you.
> > >> In Java that would be static final primitives and read-only objects like
> > >> String.
> > >>
> > >> On Fri, Apr 24, 2020 at 8:25 PM Pavel Pereslegin 
> > wrote:
> > >>
> > >>> Maxim,
> > >>>
> >  But what exactly is `constant` means?! I'd suggest applying this rule
> >  to all class fields with `static final` modifiers without any clauses.
> >  (check Java Naming convention [2] paragraph 3.3).
> > >>>
> > >>> I disagree with this and want to clarify what exactly the Java naming
> > >>> convention says:
> > >>>
> > > The following are considered to be constants:
> > > 1. All static final primitive types (Remember that all interface
> > fields
> > >>> are inherently static final).
> > > 2. All static final object reference types that are never followed by
> > >>> "." (dot).
> > > 3. All static final arrays that are never followed by"[" (dot)
> > >>>
> > >>> I don't see that convention says "any static final field".
> > >>>
> > >>> пт, 24 апр. 2020 г. в 20:11, Sergey Antonov  > >:
> > 
> >  Maxim, It's a good idea!
> > 
> >  Please don't forget to update out code style guidelines too.
> > 
> >  Thank you for keeping the code cleaner!
> > 
> >  пт, 24 апр. 2020 г. в 19:49, Maxim Muzafarov :
> > 
> > > Igniters,
> > >
> > >
> > > It is not directly mentioned through the Apache Ignite Coding
> > > Guidelines [1] about naming the `static final` class fields using
> > only
> > > upper-case letters. I'd like to suggest to fill this gap.
> > >
> > >> Constants should all be upper-case.
> > > But what exactly is `constant` means?! I'd suggest applying this rule
> > > to all class fields with `static final` modifiers without any
> > clauses.
> > > (check Java Naming convention [2] paragraph 3.3).
> > >
> > >
> > > I've prepared PR [3] with capitalizing letters on all of the constant
> > > names simultaneously with supporting the standard ConstantName
> > > checkstyle [4] rule.
> > >
> > > Can we proceed with this change?
> > > WDYT?
> > >
> > >
> > > [1]
> > >
> > >>>
> > https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines#CodingGuidelines-Naming
> > > [2]
> > >
> > >>>
> > https://web.archive.org/web/20120911192801/developers.sun.com/sunstudio/products/archive/whitepapers/java-style.pdf
> > > [3] https://github.com/apache/ignite/pull/7662
> > > [4] https://issues.apache.org/jira/browse/IGNITE-12888
> > >
> > 
> > 
> >  --
> >  BR, Sergey Antonov
> > >>>
> >
> >



-- 
Best wishes,
Amelchev Nikita

Re: [DISCUSSION] Hot cache backup

[Maxim Muzafarov]

Folks,


Are there any cases left which we need to discuss?

Do you have any questions?
I'm ready to provide all the details you need for the review.

Who else what to take a look at my changes [1] [2]?


[1] https://issues.apache.org/jira/browse/IGNITE-11073
[2] https://github.com/apache/ignite/pull/7607

On Fri, 24 Apr 2020 at 15:01, Maxim Muzafarov  wrote:
>
> Alexey,
>
>
> I've addressed all your comments, please, take a look at the PR [1].
> Additional tests were added.
> Additional comments with further steps were added.
>
>
> [1] https://github.com/apache/ignite/pull/7607
> [2] https://issues.apache.org/jira/browse/IGNITE-11073
>
> On Tue, 21 Apr 2020 at 09:53, Alexey Goncharuk
>  wrote:
> >
> > Maxim,
> >
> > I've left my comments in the PR.
> >
> > пн, 20 апр. 2020 г. в 12:52, Maxim Muzafarov :
> >
> > > Alex P,
> > > Thank you for the great sophisticated review.
> > >
> > >
> > > Alexey G,
> > > Will you take a look at my changes[1]?
> > > The fresh TC.Bot visa attached.
> > >
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-11073
> > >
> > > On Mon, 20 Apr 2020 at 11:54, Alex Plehanov 
> > > wrote:
> > > >
> > > > Maxim, I've reviewed your PR and it looks good to me. Good job!
> > > >
> > > > пт, 10 апр. 2020 г. в 19:43, Alexey Goncharuk <
> > > alexey.goncha...@gmail.com>:
> > > >
> > > > > Maxim,
> > > > >
> > > > > Thanks for raising this PR. I will do a review during next week.
> > > > >
> > > > > --AG
> > > > >
> > >

Re: [DISCUSSION] `static final` code-style constant naming rules

[Anton Vinogradov]

>> +1 to force upper case for `static final` variables.
+1 too

On Mon, Apr 27, 2020 at 12:08 PM Nikolay Izhikov 
wrote:

> +1 to force upper case for `static final` variables.
>
> > 25 апр. 2020 г., в 07:39, Ivan Pavlukhin 
> написал(а):
> >
> > Maxim,
> >
> > Thank you for efforts in a code quality improvement!
> >
> > Unfortunately I do not agree with the proposal. Usually I like to
> > refer a following guide [1]. While a question "what is a constant" is
> > not trivial but I suppose that every static final field capitalization
> > can cause some problems. For example, a developer can make false
> > assumptions about immutability and thread safety basing on a
> > capitalized field name.
> >
> > [1]
> https://google.github.io/styleguide/javaguide.html#s5.2.4-constant-names
> >
> > Best regards,
> > Ivan Pavlukhin
> >
> > пт, 24 апр. 2020 г. в 21:58, Pavel Tupitsyn :
> >>
> >>> All static final object reference types that are never followed by "."
> >> (dot)
> >> With this way of thinking we can say that everything is a constant if we
> >> don't change it - no need for static final.
> >> "Constant" is usually something that you can't change, even you want -
> >> compiler won't let you.
> >> In Java that would be static final primitives and read-only objects like
> >> String.
> >>
> >> On Fri, Apr 24, 2020 at 8:25 PM Pavel Pereslegin 
> wrote:
> >>
> >>> Maxim,
> >>>
>  But what exactly is `constant` means?! I'd suggest applying this rule
>  to all class fields with `static final` modifiers without any clauses.
>  (check Java Naming convention [2] paragraph 3.3).
> >>>
> >>> I disagree with this and want to clarify what exactly the Java naming
> >>> convention says:
> >>>
> > The following are considered to be constants:
> > 1. All static final primitive types (Remember that all interface
> fields
> >>> are inherently static final).
> > 2. All static final object reference types that are never followed by
> >>> "." (dot).
> > 3. All static final arrays that are never followed by"[" (dot)
> >>>
> >>> I don't see that convention says "any static final field".
> >>>
> >>> пт, 24 апр. 2020 г. в 20:11, Sergey Antonov  >:
> 
>  Maxim, It's a good idea!
> 
>  Please don't forget to update out code style guidelines too.
> 
>  Thank you for keeping the code cleaner!
> 
>  пт, 24 апр. 2020 г. в 19:49, Maxim Muzafarov :
> 
> > Igniters,
> >
> >
> > It is not directly mentioned through the Apache Ignite Coding
> > Guidelines [1] about naming the `static final` class fields using
> only
> > upper-case letters. I'd like to suggest to fill this gap.
> >
> >> Constants should all be upper-case.
> > But what exactly is `constant` means?! I'd suggest applying this rule
> > to all class fields with `static final` modifiers without any
> clauses.
> > (check Java Naming convention [2] paragraph 3.3).
> >
> >
> > I've prepared PR [3] with capitalizing letters on all of the constant
> > names simultaneously with supporting the standard ConstantName
> > checkstyle [4] rule.
> >
> > Can we proceed with this change?
> > WDYT?
> >
> >
> > [1]
> >
> >>>
> https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines#CodingGuidelines-Naming
> > [2]
> >
> >>>
> https://web.archive.org/web/20120911192801/developers.sun.com/sunstudio/products/archive/whitepapers/java-style.pdf
> > [3] https://github.com/apache/ignite/pull/7662
> > [4] https://issues.apache.org/jira/browse/IGNITE-12888
> >
> 
> 
>  --
>  BR, Sergey Antonov
> >>>
>
>

Re: Why OPTIMIZE_REUSE_RESULT is set to 0 explicitly in o.a.i.i.p.query.h2.ConnectionManager#DEFAULT_DB_OPTIONS

[Nikolay Izhikov]

Hello, Konstantin.

> I think we cannot just turn this optimization on

Why is that?

If we turn on `OPTIMIZE_REUSE_RESULT` consequences would be the following:

* concurrent updated of the subquery table will not be used(we have no 
guarantee on that, for now)
* performance of the select increase due to the subquery cache.

> And incrementing org.h2.engine.Database#modificationDataId on each data 
> update may lead to some performance issues. It should be benchmarked first.

For now, Ignite doesn’t guarantee the visibility of concurrent update into 
select results [1] [2]

> It should be benchmarked first.

+1

What have I missed?

[1] 
https://apacheignite-sql.readme.io/docs/how-ignite-sql-works#subqueries-in-where-clause
[2] 
https://apacheignite-sql.readme.io/docs/how-ignite-sql-works#concurrent-modifications


> 22 апр. 2020 г., в 10:39, Ivan Daschinsky  написал(а):
> 
> Thank you for your response, Konstantin
> 
> I think we cannot just turn this optimization
>> 
> Of course not, that is the reason why I started this thread.
> 
> org.apache.ignite.internal.processors.query.h2.opt.GridH2Table don't change
>> it neither on remove or update
> 
> 
> Yes, but we can increment this counter and find possible workarounds.
> 
> It should be benchmarked first.
>> 
> Absolutelly agree.
> 
> I started this thread to discuss possible caveats and traps and decide
> whether to implement this optimization or not.
> If there is too many traps or moreover this is impossible or SQL
> maintainers are against this, then it's probably not worth to do it.
> 
> 
> 
> ср, 22 апр. 2020 г. в 09:52, Konstantin Orlov :
> 
>> Hello, Ivan
>> 
>> I think we cannot just turn this optimization on because a result
>> invalidation counts on org.h2.engine.Database#modificationDataId (see
>> org.h2.command.dml.Query#sameResultAsLast(Session, Value[], Value[],
>> long)), but org.apache.ignite.internal.processors.query.h2.opt.GridH2Table
>> don't change it neither on remove or update. And incrementing
>> org.h2.engine.Database#modificationDataId on each data update may lead to
>> some performance issues. It should be benchmarked first.
>> 
>> --
>> Regards,
>> Konstantin Orlov
>> 
>> 
>>> On 21 Apr 2020, at 14:51, Ivan Daschinskiy  wrote:
>>> 
>>> Hello folks.
>>> 
>>> Recently I trying to investigate why when the query, i.e. «SELECT * FROM
>> T1 WHERE ID IN (SELECT T1_ID FROM T2), is executed locally,
>>> subquery evaluated, even if result is the same. I noticed, that in
>> mentioned in subj (.ConnectionManager#DEFAULT_DB_OPTIONS),
>>> parameter OPTIMIZE_REUSE_RESULT explicitly set to 0. This value disable
>> internal H2 optimization (see details here
>> org.h2.command.dml.Query#query(int, org.h2.result.ResultTarget) and lead
>> the mentioned above query to be ineffective.
>>> 
>>> I cannot understand why this decision was made. Also I don’t find any
>> evidence that setting OPTIMIZE_REUSE_RESULT to 1 could break something.
>> Unfortunatelly, it is impossible to change this behavior per Session
>> without forking H2, because this flag is set in org.h2.engine.Database.
>>> 
>>> Let’s discuss possible caveats in enabling this optimization by default
>> in DEFAULT_DB_OPTIONS.
>>> 
>> 
>> 
> 
> -- 
> Sincerely yours, Ivan Daschinskiy

IEP-44 Thin Client Discovery

[Pavel Tupitsyn]

Igniters,

I've prepared an IEP [1] and a POC [2] for Thin Client Discovery feature.
Let's discuss it here.

In particular, I'd like to address the following points:

1. Value: do you think this would be a good feature to have?
2. Public API changes: is a boolean property enough? Should we have
something more complex, so users can plug in custom logic to filter and/or
translate IPs and host names?
3. Server-side implementation details: should we use Compute, Node
Attributes, or something else to retrieve client endpoints from all nodes
in cluster?

[1]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
[2] https://github.com/apache/ignite/pull/7744
[3] https://issues.apache.org/jira/browse/IGNITE-12932

Re: [DISCUSSION] `static final` code-style constant naming rules

[Nikolay Izhikov]

+1 to force upper case for `static final` variables.

> 25 апр. 2020 г., в 07:39, Ivan Pavlukhin  написал(а):
> 
> Maxim,
> 
> Thank you for efforts in a code quality improvement!
> 
> Unfortunately I do not agree with the proposal. Usually I like to
> refer a following guide [1]. While a question "what is a constant" is
> not trivial but I suppose that every static final field capitalization
> can cause some problems. For example, a developer can make false
> assumptions about immutability and thread safety basing on a
> capitalized field name.
> 
> [1] https://google.github.io/styleguide/javaguide.html#s5.2.4-constant-names
> 
> Best regards,
> Ivan Pavlukhin
> 
> пт, 24 апр. 2020 г. в 21:58, Pavel Tupitsyn :
>> 
>>> All static final object reference types that are never followed by "."
>> (dot)
>> With this way of thinking we can say that everything is a constant if we
>> don't change it - no need for static final.
>> "Constant" is usually something that you can't change, even you want -
>> compiler won't let you.
>> In Java that would be static final primitives and read-only objects like
>> String.
>> 
>> On Fri, Apr 24, 2020 at 8:25 PM Pavel Pereslegin  wrote:
>> 
>>> Maxim,
>>> 
 But what exactly is `constant` means?! I'd suggest applying this rule
 to all class fields with `static final` modifiers without any clauses.
 (check Java Naming convention [2] paragraph 3.3).
>>> 
>>> I disagree with this and want to clarify what exactly the Java naming
>>> convention says:
>>> 
> The following are considered to be constants:
> 1. All static final primitive types (Remember that all interface fields
>>> are inherently static final).
> 2. All static final object reference types that are never followed by
>>> "." (dot).
> 3. All static final arrays that are never followed by"[" (dot)
>>> 
>>> I don't see that convention says "any static final field".
>>> 
>>> пт, 24 апр. 2020 г. в 20:11, Sergey Antonov :
 
 Maxim, It's a good idea!
 
 Please don't forget to update out code style guidelines too.
 
 Thank you for keeping the code cleaner!
 
 пт, 24 апр. 2020 г. в 19:49, Maxim Muzafarov :
 
> Igniters,
> 
> 
> It is not directly mentioned through the Apache Ignite Coding
> Guidelines [1] about naming the `static final` class fields using only
> upper-case letters. I'd like to suggest to fill this gap.
> 
>> Constants should all be upper-case.
> But what exactly is `constant` means?! I'd suggest applying this rule
> to all class fields with `static final` modifiers without any clauses.
> (check Java Naming convention [2] paragraph 3.3).
> 
> 
> I've prepared PR [3] with capitalizing letters on all of the constant
> names simultaneously with supporting the standard ConstantName
> checkstyle [4] rule.
> 
> Can we proceed with this change?
> WDYT?
> 
> 
> [1]
> 
>>> https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines#CodingGuidelines-Naming
> [2]
> 
>>> https://web.archive.org/web/20120911192801/developers.sun.com/sunstudio/products/archive/whitepapers/java-style.pdf
> [3] https://github.com/apache/ignite/pull/7662
> [4] https://issues.apache.org/jira/browse/IGNITE-12888
> 
 
 
 --
 BR, Sergey Antonov
>>> 

Re: Apache Ignite 2.8.1 RELEASE [Time, Scope, Manager]

[Nikolay Izhikov]

Taras.

Thank you, very much!
You changes merged to 2.8.1 branch.

Igniters,

We have 10 tickets scheduled for 2.8.1 release:

OPEN:

IGNITE-11687Concurrent WAL replay & log may fail with CRC error on read - 
Dmitriy Govorukhin
IGNITE-12346.NET: Platform error:System.NullReferenceException -Pavel 
Tupitsyn

IN PROGRESS:

IGNITE-12637IgniteSparkSession doesn't start the clients on really 
distributed cluster - Yaroslav Molochkov
IGNITE-12788Cluster achieved fully rebalanced (PME-free ready) state metric 
- Nikolay Izhikov

PATCH AVAILABLE:

IGNITE-10417notifyDiscoveryListener() call can be lost. - Pavel Voronkin
IGNITE-12852Comma in field is not supported by COPY command - YuJue Li
IGNITE-12252Unchecked exceptions during rebalancing should be handled - 
Nikolai Kulagin
IGNITE-12905QueryKeyValueIterable missing custom spliterator() 
implementation - Johnny Galatikitis
IGNITE-12801Possible extra page release when throttling and checkpoint 
thread store its concurrently - Anton Kalashnikov
IGNITE-12794Scan query fails with an assertion error: Unexpected row key - 
Denis Mekhanikov


> 27 апр. 2020 г., в 11:08, Taras Ledkov  написал(а):
> 
> Hi,
> 
> Nikolay, i've created PR [1] that contains the SQL-related tickets to port 
> into 2.8.1:
> 
> IGNITE-12790 Introduce distributed SQL configuration and ability to disable 
> SQL functions.
> IGNITE-12887 Fix handle type mismatch exception on compare values while 
> traversing index tree.
> IGNITE-12848 fix H2Connection leaks on INSERT
> IGNITE-12800  SQL: local queries cursors must be closed or full read to 
> unlock the GridH2Table.
> 
> TC test are OK. Please take a look at the TC bot report [2].
> Please review & merge the patch into ignite-2.8.1.
> 
> [1]. https://github.com/apache/ignite/pull/7703
> [2]. 
> https://mtcga.gridgain.com/pr.html?serverId=apache=IgniteTests24Java8_RunAll=pull%2F7703%2Fhead=Latest=ignite-2.8.1
> 
> 
> On 23.04.2020 13:25, Mikhail Petrov wrote:
>> Hello, Igniters.
>> 
>> I propose to cherry-pick to 2.8.1 ticket [1].
>> 
>> 
>> In addition to adding a new metric, it fixes a bug when, after deactivation, 
>> GridDhtPartitionsExchangeFuture#rebalanced flag was not reset. And 
>> therefore, it can be different on nodes that are already in the cluster from 
>> newly joined ones.
>> 
>> [1] - https://issues.apache.org/jira/browse/IGNITE-12788
>> 
>> Regards,
>> Mikhail.
>> 
>> On 22.04.2020 14:03, Nikolay Izhikov wrote:
>>> Hello, Ivan.
>>> 
>>> I think we can include this improvements.
>>> Please, go ahead.
>>> 
 22 апр. 2020 г., в 10:21, Ivan Bessonov  написал(а):
 
 Hi Igniters,
 
 I'm continuing with IGNITE-12756 PR creation. It turned out that we need 3
 more cherry-picks
 to avoid massive code changes. Tests started failing after my initial
 attempt to create that PR.
 
 So, in total I have PR with 6 commits in it. Some of them fix components
 and tests for them,
 others are required for new tests to compile and run without changes in
 2.8.1 branch.
 RunAll is in progress now and I'll reply with news if I have any.
 
 Andrey Gura, Nikolay Izhikov, are you OK with 3 more commits in the scope?
 If you don't,
 then I won't be able to port IGNITE-12756 to 2.8.1 properly by myself, I
 guess it would be
 easier to reimplement it from the scratch.
 
 [1] https://issues.apache.org/jira/browse/IGNITE-12735
 [2] https://issues.apache.org/jira/browse/IGNITE-12568
 [3] https://issues.apache.org/jira/browse/IGNITE-12756
 
 [4] https://issues.apache.org/jira/browse/IGNITE-12285
 [5] https://issues.apache.org/jira/browse/IGNITE-12668
 [6] https://issues.apache.org/jira/browse/IGNITE-12682
 [7] https://github.com/apache/ignite/pull/7708
 
 
 вт, 21 апр. 2020 г. в 19:36, Pavel Pereslegin :
 
> Hello, Nikolay.
> 
> This has been fixed by increasing the test timeout in IGNITE-12683 [1][2].
> 
> [1] https://issues.apache.org/jira/browse/IGNITE-12683
> [2]
> https://github.com/apache/ignite/commit/bf394a77e1de6432e493eb2818243a82b577f11e
>  
> 
> вт, 21 апр. 2020 г. в 18:28, Nikolay Izhikov :
>> Hello, Igniters.
>> 
>> While cherry-picking to 2.8.1 I found flaky test regarding BPlusTree
> [1], [2]
>> All failures of this tests relates to PR’s based on 2.8.
>> It seems we hav fix in master but doesn’t have it in 2.8 and 2.8.1
>> 
>> Can someone suggest, what commit fixes this tests?
>> 
>> [1]
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8==testDetails=-7895536196794411367=TEST_STATUS_DESC_IgniteTests24Java8=__all_branches__=50
>  
>> [2]
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8==testDetails=-3485591522651012009=TEST_STATUS_DESC_IgniteTests24Java8=__all_branches__=50
>  

Re: Apache Ignite 2.8.1 RELEASE [Time, Scope, Manager]

[Taras Ledkov]

Hi,

Nikolay, i've created PR [1] that contains the SQL-related tickets to 
port into 2.8.1:


IGNITE-12790 Introduce distributed SQL configuration and ability to 
disable SQL functions.
IGNITE-12887 Fix handle type mismatch exception on compare values while 
traversing index tree.

IGNITE-12848 fix H2Connection leaks on INSERT
IGNITE-12800  SQL: local queries cursors must be closed or full read to 
unlock the GridH2Table.


TC test are OK. Please take a look at the TC bot report [2].
Please review & merge the patch into ignite-2.8.1.

[1]. https://github.com/apache/ignite/pull/7703
[2]. 
https://mtcga.gridgain.com/pr.html?serverId=apache=IgniteTests24Java8_RunAll=pull%2F7703%2Fhead=Latest=ignite-2.8.1



On 23.04.2020 13:25, Mikhail Petrov wrote:

Hello, Igniters.

I propose to cherry-pick to 2.8.1 ticket [1].


In addition to adding a new metric, it fixes a bug when, after 
deactivation, GridDhtPartitionsExchangeFuture#rebalanced flag was not 
reset. And therefore, it can be different on nodes that are already in 
the cluster from newly joined ones.


[1] - https://issues.apache.org/jira/browse/IGNITE-12788

Regards,
Mikhail.

On 22.04.2020 14:03, Nikolay Izhikov wrote:

Hello, Ivan.

I think we can include this improvements.
Please, go ahead.

22 апр. 2020 г., в 10:21, Ivan Bessonov  
написал(а):


Hi Igniters,

I'm continuing with IGNITE-12756 PR creation. It turned out that we 
need 3

more cherry-picks
to avoid massive code changes. Tests started failing after my initial
attempt to create that PR.

So, in total I have PR with 6 commits in it. Some of them fix 
components

and tests for them,
others are required for new tests to compile and run without changes in
2.8.1 branch.
RunAll is in progress now and I'll reply with news if I have any.

Andrey Gura, Nikolay Izhikov, are you OK with 3 more commits in the 
scope?

If you don't,
then I won't be able to port IGNITE-12756 to 2.8.1 properly by 
myself, I

guess it would be
easier to reimplement it from the scratch.

[1] https://issues.apache.org/jira/browse/IGNITE-12735
[2] https://issues.apache.org/jira/browse/IGNITE-12568
[3] https://issues.apache.org/jira/browse/IGNITE-12756

[4] https://issues.apache.org/jira/browse/IGNITE-12285
[5] https://issues.apache.org/jira/browse/IGNITE-12668
[6] https://issues.apache.org/jira/browse/IGNITE-12682
[7] https://github.com/apache/ignite/pull/7708


вт, 21 апр. 2020 г. в 19:36, Pavel Pereslegin :


Hello, Nikolay.

This has been fixed by increasing the test timeout in IGNITE-12683 
[1][2].


[1] https://issues.apache.org/jira/browse/IGNITE-12683
[2]
https://github.com/apache/ignite/commit/bf394a77e1de6432e493eb2818243a82b577f11e 



вт, 21 апр. 2020 г. в 18:28, Nikolay Izhikov :

Hello, Igniters.

While cherry-picking to 2.8.1 I found flaky test regarding BPlusTree

[1], [2]

All failures of this tests relates to PR’s based on 2.8.
It seems we hav fix in master but doesn’t have it in 2.8 and 2.8.1

Can someone suggest, what commit fixes this tests?

[1]
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8==testDetails=-7895536196794411367=TEST_STATUS_DESC_IgniteTests24Java8=__all_branches__=50 


[2]
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8==testDetails=-3485591522651012009=TEST_STATUS_DESC_IgniteTests24Java8=__all_branches__=50 




21 апр. 2020 г., в 17:49, Ivan Bessonov 

написал(а):

Sure, here's PR with 3 cherry-picked commits that I mentioned:
https://github.com/apache/ignite/pull/7708

вт, 21 апр. 2020 г. в 17:17, Nikolay Izhikov :


OK then, let’s include it in the 2.8.1

Ivan, can you, please, prepare PR in the ignite-2.8.1 branch that

contain

cherry-pick for all required commits?

21 апр. 2020 г., в 17:01, Andrey Gura  
написал(а):


Hi


IGNITE-12735 - Metric exporter implementation could lead to

NullPointerException from gauge which invoke communication

IGNITE-12568 - MessageFactory implementations refactoring
Personally, I’m against any refactoring improvements in bug fix

release.

So, I propose to exclude IGNITE-12756 from 2.8.1
Andrey, what do you think as a committer of this improvements?

Mainly IGNITE-12756 brings some improvements related with TCP
communication metrics (performance, memory footprint,

IgniteSpiContext

improved in order to provide ability to implement metrics related
SPI's without using internal API's, code improvements)

But! It also fixes potential NPE's which can be thrown on node 
start.

So it would be great to include this fix to 2.8.1 release.

On Tue, Apr 21, 2020 at 11:12 AM Nikolay Izhikov <

nizhi...@apache.org>

wrote:

I've cherry-picked IGNITE-12734 to 2.8.1 branch.


Thanks!


considering commit "683f22e64f IGNITE-12756 TcpCommunication SPI

metrics

improvement" - it depends
on https://issues.apache.org/jira/browse/IGNITE-12735 and
https://issues.apache.org/jira/browse/IGNITE-12568,
both were targeted to 2.9, but this has to be changed probably.

Re: IGNITE-12357 Migrate Twitter module to ignite-extensions

[Petr Ivanov]

Please, remember to update TC configuration or create issue to do that.


> On 26 Apr 2020, at 20:40, Saikat Maitra  wrote:
> 
> Hi,
> 
> I have raised PRs for migration of twitter module in ignite-extensions
> repository.
> 
> Jira
> https://issues.apache.org/jira/browse/IGNITE-12357
> 
> PRs
> https://github.com/apache/ignite-extensions/pull/12
> https://github.com/apache/ignite/pull/7733
> 
> Please review and share your feedback.
> 
> Regards,
> Saikat