[MTCGA]: new failures in builds [5257877] needs to be handled

[dpavlov . tasks]

Hi Igniters,

 I've detected some new issue on TeamCity to be handled. You are more than 
welcomed to help.

 *Test with high flaky rate in master 
ZookeeperDiscoverySegmentationAndConnectionRestoreTest.testStopNodeOnSegmentaion
 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=5167715885538994183=%3Cdefault%3E=testDetails
 No changes in the build

 - Here's a reminder of what contributors were agreed to do 
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute 
 - Should you have any questions please contact dev@ignite.apache.org 

Best Regards,
Apache Ignite TeamCity Bot 
https://github.com/apache/ignite-teamcity-bot
Notification generated at 05:46:31 29-04-2020 

Re: [VOTE][EXTENSION] Release Apache Ignite Spring Boot extensions 1.0.0 RC1

[Saikat Maitra]

+1

Yes, we can update the DEVNOTES.txt in the next release.

Regards,
Saikat


On Tue, Apr 28, 2020 at 4:55 AM Maksim Stepachev 
wrote:

> +1
>
> вт, 28 апр. 2020 г. в 11:27, Nikolay Izhikov :
>
> > *** Formal vote description fixed ***
> >
> > Dear Community,
> >
> > I have uploaded a release candidate of the two extension modules
> > `ignite-spring-boot-autoconfigure` and
> > `ignite-spring-boot-client-autoconfigure`.
> >
> > The following staging can be used for testing:
> > https://repository.apache.org/content/repositories/orgapacheignite-1478/
> >
> > Tag with name `ignite-spring-boot-1.0.0-rc1` created:
> >
> >
> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=8b5f2d9143d281b90aceb6f11dabde50db48f6b3
> >
> > Release 1.0 contains an initial implementation of the modules, please
> > refer to the RELEASE_NOTES:
> >
> >
> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=RELEASE_NOTES.txt;h=e69173583068467b44f5e148151654ade3fe2452;hb=HEAD
> >
> > Complete list of resolved issues:
> >
> >
> >
> https://issues.apache.org/jira/issues/?jql=labels%20%3D%20spring-boot-autoconfigure
> >
> >
> https://issues.apache.org/jira/issues/?jql=labels%20%3D%20spring-boot-client-autoconfigure
> >
> > DEVNOTES
> >
> >
> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=DEVNOTES.txt;h=7f25887adef738db956ae0f70c59839f73965d5f;hb=HEAD
> >
> > DOCUMENTATION
> > https://apacheignite.readme.io/docs/spring-boot-autoconfigure
> >
> https://apacheignite.readme.io/docs/ignite-spring-boot-client-autoconfigure
> >
> > The vote is formal, see voting guidelines
> > https://www.apache.org/foundation/voting.html
> >
> > +1 - to accept Apache Ignite spring-boot auto configure extensions
> > 1.0.0-rc1
> > 0 - don't care either way
> > -1 - DO NOT accept Apache Ignite spring-boot auto configure extensions
> > 1.0.0-rc1 (explain why)
> >
> > See notes on how to verify release here
> > https://www.apache.org/info/verification.html
> > and
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification
> >
> > The vote will hold for 72 hours and will end on April 30 2020 20:20 UTC
> >
> >
> https://www.timeanddate.com/countdown/generic?iso=20200430T2320=166=cursive
>

[MTCGA]: new failures in builds [5257245] needs to be handled

[dpavlov . tasks]

Hi Igniters,

 I've detected some new issue on TeamCity to be handled. You are more than 
welcomed to help.

 *Test with high flaky rate in master 
PageMemoryImplTest.testCheckpointBufferCantOverflowMixedLoadRatioBased 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-432402548321959382=%3Cdefault%3E=testDetails
 No changes in the build

 - Here's a reminder of what contributors were agreed to do 
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute 
 - Should you have any questions please contact dev@ignite.apache.org 

Best Regards,
Apache Ignite TeamCity Bot 
https://github.com/apache/ignite-teamcity-bot
Notification generated at 02:16:31 29-04-2020 

[MTCGA]: new failures in builds [5254051] needs to be handled

[dpavlov . tasks]

Hi Igniters,

 I've detected some new issue on TeamCity to be handled. You are more than 
welcomed to help.

 *Test with high flaky rate in master 
AtomicPartitionCounterStateConsistencyTest.testPartitionConsistencyWithBackupsRestart
 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-7950380606755761695=%3Cdefault%3E=testDetails
 No changes in the build

 - Here's a reminder of what contributors were agreed to do 
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute 
 - Should you have any questions please contact dev@ignite.apache.org 

Best Regards,
Apache Ignite TeamCity Bot 
https://github.com/apache/ignite-teamcity-bot
Notification generated at 02:01:31 29-04-2020 

[GitHub] [ignite-website] mstekl opened a new pull request #10: Seo updates April 28

[GitBox]

mstekl opened a new pull request #10:
URL: https://github.com/apache/ignite-website/pull/10


   I'm submitting a few set of improvements:
   
   - Ran a fresh crawl and rebuilt sitemap.xml
   - Fixed accessibility issues when using  on header and homepage
   - Changed the text for links btns underuse cases section on homapage to a 
more descriptive text
   - Replaced old 4.7 fontawesome with latest 5.11 which avoids font-display 
audit failure



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[MTCGA]: new failures in builds [5255463, 5253395] needs to be handled

[dpavlov . tasks]

Hi Igniters,

 I've detected some new issue on TeamCity to be handled. You are more than 
welcomed to help.

 If your changes can lead to this failure(s): We're grateful that you were a 
volunteer to make the contribution to this project, but things change and you 
may no longer be able to finalize your contribution.
 Could you respond to this email and indicate if you wish to continue and fix 
test failures or step down and some committer may revert you commit. 

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testObjectQueryWithSwap 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-2568454516644195915=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testUserDefinedFunction 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=1185802565888476284=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testScanQueryEvents 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-7181959255382788501=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testIntegerType 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=2854480535417540416=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testComplexType 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=4322044466324358808=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testTextQueryEvents 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-6068481763100452268=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testOrderByOnly 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=4470497289618319103=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testPaginationGetNamedCache 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-4112405730107884639=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testObjectWithString 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-1057187676187630808=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testLocalSqlQueryFromClient 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-5240247212714884007=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testStringType 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-398256761422014123=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testScanFilters 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=3757219458023244362=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testSimpleCustomTableName 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=1653225369888578899=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testPaginationIteratorDefaultCache 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=5010517106299466536=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testFieldsQueryEvents 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-3760605754349657199=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testDifferentValueTypes 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-1062632785854547492=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testPaginationGetDefaultCache 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-615322202308010968=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testFieldsQueryMetadata 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-4867510320614841211=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testSelectQuery 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-8104788462129583522=%3Cdefault%3E=testDetails
 Changes may lead to failure were done by 
 - alexey scherbakov  
https://ci.ignite.apache.org/viewModification.html?modId=901024

 *Test with high flaky rate in master 
AtomicPartitionCounterStateConsistencyTest.testPartitionConsistencyWithBackupsRestart
 

[GitHub] [ignite-website] mstekl opened a new pull request #9: Added couple of new events

[GitBox]

mstekl opened a new pull request #9:
URL: https://github.com/apache/ignite-website/pull/9


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: IEP-44 Thin Client Discovery

[Pavel Tupitsyn]

Ok, I've updated IEP and POC accordingly:
* Config flag removed
* IPs and host names retrieval simplified - use existing node properties
and attributes instead of Compute

On Tue, Apr 28, 2020 at 7:57 PM Igor Sapego  wrote:

> I guess it makes sense. If anyone needs more control over connection
> we would need to implement a new feature anyway (like node filter we
> discussed earlier)
>
> Best Regards,
> Igor
>
>
> On Tue, Apr 28, 2020 at 12:29 PM Pavel Tupitsyn 
> wrote:
>
> > > enable the capability if the best effort affinity is on
> > I agree, makes sense.
> >
> > Igor, what do you think?
> >
> > On Tue, Apr 28, 2020 at 8:25 AM Denis Magda  wrote:
> >
> > > Pavel,
> > >
> > > That would be a tremendous improvement for the recently release best
> > effort
> > > affinity feature. Without this capability, we force application
> > developers
> > > to reopen thin client connections every type a cluster is scaled out. I
> > > believe that once the folks start using the best effort affinity, we'll
> > be
> > > hearing more of a feature request for what you're proposing in this
> > thread.
> > > So, thanks for taking care of this proactively!
> > >
> > > As for the public API changes, do we really need any extra flag? I
> would
> > > enable the capability if the best effort affinity is on. For me, it's
> > just
> > > a natural improvement of the latter and it sounds reasonable to reuse
> the
> > > best effort affinity's flag.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Mon, Apr 27, 2020 at 2:58 AM Pavel Tupitsyn 
> > > wrote:
> > >
> > > > Igniters,
> > > >
> > > > I've prepared an IEP [1] and a POC [2] for Thin Client Discovery
> > feature.
> > > > Let's discuss it here.
> > > >
> > > > In particular, I'd like to address the following points:
> > > >
> > > > 1. Value: do you think this would be a good feature to have?
> > > > 2. Public API changes: is a boolean property enough? Should we have
> > > > something more complex, so users can plug in custom logic to filter
> > > and/or
> > > > translate IPs and host names?
> > > > 3. Server-side implementation details: should we use Compute, Node
> > > > Attributes, or something else to retrieve client endpoints from all
> > nodes
> > > > in cluster?
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
> > > > [2] https://github.com/apache/ignite/pull/7744
> > > > [3] https://issues.apache.org/jira/browse/IGNITE-12932
> > > >
> > >
> >
>

Re: IEP-44 Thin Client Discovery

[Igor Sapego]

I guess it makes sense. If anyone needs more control over connection
we would need to implement a new feature anyway (like node filter we
discussed earlier)

Best Regards,
Igor


On Tue, Apr 28, 2020 at 12:29 PM Pavel Tupitsyn 
wrote:

> > enable the capability if the best effort affinity is on
> I agree, makes sense.
>
> Igor, what do you think?
>
> On Tue, Apr 28, 2020 at 8:25 AM Denis Magda  wrote:
>
> > Pavel,
> >
> > That would be a tremendous improvement for the recently release best
> effort
> > affinity feature. Without this capability, we force application
> developers
> > to reopen thin client connections every type a cluster is scaled out. I
> > believe that once the folks start using the best effort affinity, we'll
> be
> > hearing more of a feature request for what you're proposing in this
> thread.
> > So, thanks for taking care of this proactively!
> >
> > As for the public API changes, do we really need any extra flag? I would
> > enable the capability if the best effort affinity is on. For me, it's
> just
> > a natural improvement of the latter and it sounds reasonable to reuse the
> > best effort affinity's flag.
> >
> > -
> > Denis
> >
> >
> > On Mon, Apr 27, 2020 at 2:58 AM Pavel Tupitsyn 
> > wrote:
> >
> > > Igniters,
> > >
> > > I've prepared an IEP [1] and a POC [2] for Thin Client Discovery
> feature.
> > > Let's discuss it here.
> > >
> > > In particular, I'd like to address the following points:
> > >
> > > 1. Value: do you think this would be a good feature to have?
> > > 2. Public API changes: is a boolean property enough? Should we have
> > > something more complex, so users can plug in custom logic to filter
> > and/or
> > > translate IPs and host names?
> > > 3. Server-side implementation details: should we use Compute, Node
> > > Attributes, or something else to retrieve client endpoints from all
> nodes
> > > in cluster?
> > >
> > > [1]
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
> > > [2] https://github.com/apache/ignite/pull/7744
> > > [3] https://issues.apache.org/jira/browse/IGNITE-12932
> > >
> >
>

[jira] [Created] (IGNITE-12963) Request snapshot from remote node

[Maxim Muzafarov (Jira)]

Maxim Muzafarov created IGNITE-12963:


 Summary: Request snapshot from remote node
 Key: IGNITE-12963
 URL: https://issues.apache.org/jira/browse/IGNITE-12963
 Project: Ignite
  Issue Type: Sub-task
Reporter: Maxim Muzafarov
Assignee: Maxim Muzafarov






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

question about apache configuration

[Marciniak Fryderyk IS - Partner Detal]

Hello. The calculator suggested a configuration based on 4 nodes with a 
specific configuration each (2x5118N, 12x64GB of RAM, 2x480SSD system, 2x 1.6TB 
NVMe base). The question is, can we limit the configuration to 3 nodes by 
increasing the memory of each of them to 24x64GB and whether by increasing the 
amount of frame we need to increase the disk capacity for the base from 2x1.6 
to 2x3.2TB ??
[cid:image002.png@01D61D48.2E50B3E0]
[https://mdm.corpnet.pl/images/IntegratedSolution/left_stripes.png]

Fryderyk Marciniak
Główny Inżynier Projektant Rozwiązań
fmarcin...@i-s.com.pl
Tel: Kom.: +48 505 591 421

[Logo Integrated Solution]


Integrated Solutions Sp. z o.o.
ul.: Józefa Piłsudskiego 20, 61-246 Poznań
KRS 385043 | REGON 142924560 | NIP 525-25-06-859
www.integratedsolutions.pl | 
LinkedIn

[https://mdm.corpnet.pl/images/IntegratedSolution/right_stripes.png]

Re: Check indexes inline size tool

[Sergey Antonov]

Hi, the ticket is ready for review.

[1] https://github.com/apache/ignite/pull/7728

вт, 28 апр. 2020 г. в 14:39, Sergey Antonov :

> Maxim, I'm talking about cluster upgrade through cluster stop -> binary
> update -> cluster start.
>
> вт, 28 апр. 2020 г. в 14:37, Maxim Muzafarov :
>
>> Sergey,
>>
>> Are you talking about a cluster rolling upgrade feature? AFAIK, Apache
>> Ignite doesn't support this feature, so why we should care about it?
>>
>> On Tue, 28 Apr 2020 at 14:32, Sergey Antonov 
>> wrote:
>> >
>> > Maxim,
>> >
>> > > should we _reject_ joining nodes which have different
>> > From my point of view, it's a breaking change on cluster update.
>> >
>> > We can get a different inline size in other scenarios too: as I know we
>> did
>> > some improvements in calculation effective (actual) index inline size.
>> > Let's imagine, we have PDS cluster created on the "old" apache ignite
>> > version. We decided to upgrade Ignite version and after that, join a new
>> > node to the cluster. On a new node, effective inline sizes will be
>> > calculated by the optimized algorithm. On the old nodes, the inline size
>> > will not be recalculated. It can lead to a difference in inline sizes
>> > without IGNITE_MAX_INDEX_PAYLOAD_SIZE option.
>> >
>> >
>> >
>> > вт, 28 апр. 2020 г. в 14:22, Maxim Muzafarov :
>> >
>> > > Sergey, Ilya,
>> > >
>> > >
>> > > Since inline size for the `create table` clause not supported yet and
>> > > the IGNITE_MAX_INDEX_PAYLOAD_SIZE is the only option, should we
>> > > _reject_ joining nodes which have different
>> > > IGNITE_MAX_INDEX_PAYLOAD_SIZE value instead for allowing and printing
>> > > warning message? Thus we will force users to have the same values of
>> > > IGNITE_MAX_INDEX_PAYLOAD_SIZE property.
>> > >
>> > >
>> > > On Tue, 28 Apr 2020 at 14:01, Ilya Kasnacheev <
>> ilya.kasnach...@gmail.com>
>> > > wrote:
>> > > >
>> > > > Hello!
>> > > >
>> > > > Unfortunately and embarrassingly, we still do not support passing
>> > > > INLINE_SIZE to CREATE TABLE, at least in 2.8.0.
>> > > > This means IGNITE_MAX_INDEX_PAYLOAD_SIZE is the only option to
>> create an
>> > > > implicit primary key index with specified inline size.
>> > > >
>> > > > Regards,
>> > > > --
>> > > > Ilya Kasnacheev
>> > > >
>> > > >
>> > > > вт, 28 апр. 2020 г. в 02:31, Denis Magda :
>> > > >
>> > > > > Hi Sergey,
>> > > > >
>> > > > > Your changes look useful from the application developer
>> perspective.
>> > > > > However, I'm curious why would the one change some low-level
>> > > > > IGNITE_MAX_INDEX_PAYLOAD_SIZE parameter when it's advised to pass
>> > > > > INLINE_SIZE to CREATE TABLE to change the index size cluster-wide.
>> > > > >
>> > > > > -
>> > > > > Denis
>> > > > >
>> > > > >
>> > > > > On Mon, Apr 27, 2020 at 5:38 AM Sergey Antonov <
>> > > antonovserge...@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > Hi, Igniters!
>> > > > > >
>> > > > > > I'd like to share a new small feature in AI [1].
>> > > > > >
>> > > > > > For different reasons, the cluster could have a different SQL
>> index
>> > > > > inline
>> > > > > > size [2] on cluster nodes. For example due to
>> > > > > > different IGNITE_MAX_INDEX_PAYLOAD_SIZE [3] value on cluster
>> nodes.
>> > > > > >
>> > > > > > The difference in index inline size may lead to performance
>> > > degradation.
>> > > > > > I think we must compare inline sizes on node join and warn if
>> > > difference
>> > > > > > found. Also, We should have the ability to check inline sizes on
>> > > demand.
>> > > > > >
>> > > > > > I've implemented this check on node join and new command in
>> > > control.sh
>> > > > > >
>> > > > > > Look at warning message and utility command output:
>> > > > > >
>> > > > > > Warn message on a node in the cluster during new node join:
>> > > > > >
>> > > > > > [2020-04-27 15:36:21,185][WARN ][tcp-disco-msg-worker-[6ba0b823
>> > > > > > 127.0.0.1:47502
>> > > > > >
>> crd]-#17%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
>> > > > > Inline
>> > > > > > sizes on local node and node
>> 5bf6ca48-34a0-4aff-8db2-c0b6df303d3f are
>> > > > > > different. Please drop and create again these indexes to avoid
>> > > > > performance
>> > > > > > problems with SQL queries. Problem indexes:
>> > > > > >
>> > > > > >
>> > > > >
>> > >
>> PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
>> > > > > >
>> > > > > > Warn messages on a joining node, if difference found:
>> > > > > > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
>> > > > > > 127.0.0.1:47501
>> > > > > >
>> ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
>> > > > > > Inline sizes on local node and node
>> > > a86e9cea-63e8-42af-a897-cec4be51
>> > > > > > are different. Please drop and create again these indexes to
>> avoid
>> > > > > > performance problems with SQL queries. Problem indexes:
>> > > > > >
>> > > > > >
>> > > > >
>> > >
>> 

[MTCGA]: new failures in builds [5254099, 5254075] needs to be handled

[dpavlov . tasks]

Hi Igniters,

 I've detected some new issue on TeamCity to be handled. You are more than 
welcomed to help.

 *Test with high flaky rate in master-nightly 
IgnitePdsWithTtlTest.testRebalancingWithTtlExpirable 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-5948415390916609782=%3Cdefault%3E=testDetails
 No changes in the build

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testDifferentValueTypes 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-1062632785854547492=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testStringType 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-398256761422014123=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testFieldsQueryEvents 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-3760605754349657199=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testScanQueryEvents 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-7181959255382788501=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testPaginationGetNamedCache 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-4112405730107884639=%3Cdefault%3E=testDetails

 *Test with high flaky rate in master 
IgniteCacheReplicatedQuerySelfTest.testPaginationIteratorDefaultCache 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=5010517106299466536=%3Cdefault%3E=testDetails
 No changes in the build

 - Here's a reminder of what contributors were agreed to do 
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute 
 - Should you have any questions please contact dev@ignite.apache.org 

Best Regards,
Apache Ignite TeamCity Bot 
https://github.com/apache/ignite-teamcity-bot
Notification generated at 16:30:01 28-04-2020 

[jira] [Created] (IGNITE-12962) Blacklist and whitelist of classes allowed to deserialize via HTTP-REST should be supported

[Aleksey Plekhanov (Jira)]

Aleksey Plekhanov created IGNITE-12962:
--

 Summary: Blacklist and whitelist of classes allowed to deserialize 
via HTTP-REST should be supported
 Key: IGNITE-12962
 URL: https://issues.apache.org/jira/browse/IGNITE-12962
 Project: Ignite
  Issue Type: Improvement
  Components: rest
Reporter: Aleksey Plekhanov


Since we have the ability to deserialize custom objects (implemented by 
IGNITE-12857) we should have the ability to limit the scope of classes allowed 
to safe deserialization.

There are already two system properties used for such purpose in Ignite:
{code:java}
/** Defines path to the file that contains list of classes allowed to safe 
deserialization.*/
public static final String IGNITE_MARSHALLER_WHITELIST = 
"IGNITE_MARSHALLER_WHITELIST";

/** Defines path to the file that contains list of classes disallowed to safe 
deserialization.*/
public static final String IGNITE_MARSHALLER_BLACKLIST = 
"IGNITE_MARSHALLER_BLACKLIST";{code}
HTTP-REST should support these properties too.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Apache Ignite 2.8.1 RELEASE [Time, Scope, Manager]

[Nikolay Izhikov]

Hello, Alex.

+1 from me.

> 28 апр. 2020 г., в 15:03, Alex Plehanov  написал(а):
> 
> Hello guys,
> 
> While we are still waiting for some tickets to resolve I propose to
> cherry-pick to 2.8.1 two more bugfixes:
> IGNITE-12933 Fixed node failure after put incorrect key class for cache
> with indexed types
> IGNITE-12855 Fixed node failure with concurrent get operation and entry
> expiration when persistent is enabled
> Both fixes prevent node failure in some circumstances, both fixed already
> merged to master.
> 
> WDYT?
> 
> пн, 27 апр. 2020 г. в 11:53, Nikolay Izhikov :
> 
>> Taras.
>> 
>> Thank you, very much!
>> You changes merged to 2.8.1 branch.
>> 
>> Igniters,
>> 
>> We have 10 tickets scheduled for 2.8.1 release:
>> 
>> OPEN:
>> 
>> IGNITE-11687Concurrent WAL replay & log may fail with CRC error on
>> read - Dmitriy Govorukhin
>> IGNITE-12346.NET: Platform error:System.NullReferenceException -
>> Pavel Tupitsyn
>> 
>> IN PROGRESS:
>> 
>> IGNITE-12637IgniteSparkSession doesn't start the clients on really
>> distributed cluster - Yaroslav Molochkov
>> IGNITE-12788Cluster achieved fully rebalanced (PME-free ready) state
>> metric - Nikolay Izhikov
>> 
>> PATCH AVAILABLE:
>> 
>> IGNITE-10417notifyDiscoveryListener() call can be lost. - Pavel
>> Voronkin
>> IGNITE-12852Comma in field is not supported by COPY command - YuJue Li
>> IGNITE-12252Unchecked exceptions during rebalancing should be handled
>> - Nikolai Kulagin
>> IGNITE-12905QueryKeyValueIterable missing custom spliterator()
>> implementation - Johnny Galatikitis
>> IGNITE-12801Possible extra page release when throttling and checkpoint
>> thread store its concurrently - Anton Kalashnikov
>> IGNITE-12794Scan query fails with an assertion error: Unexpected row
>> key - Denis Mekhanikov
>> 
>> 
>>> 27 апр. 2020 г., в 11:08, Taras Ledkov 
>> написал(а):
>>> 
>>> Hi,
>>> 
>>> Nikolay, i've created PR [1] that contains the SQL-related tickets to
>> port into 2.8.1:
>>> 
>>> IGNITE-12790 Introduce distributed SQL configuration and ability to
>> disable SQL functions.
>>> IGNITE-12887 Fix handle type mismatch exception on compare values while
>> traversing index tree.
>>> IGNITE-12848 fix H2Connection leaks on INSERT
>>> IGNITE-12800  SQL: local queries cursors must be closed or full read to
>> unlock the GridH2Table.
>>> 
>>> TC test are OK. Please take a look at the TC bot report [2].
>>> Please review & merge the patch into ignite-2.8.1.
>>> 
>>> [1]. https://github.com/apache/ignite/pull/7703
>>> [2].
>> https://mtcga.gridgain.com/pr.html?serverId=apache=IgniteTests24Java8_RunAll=pull%2F7703%2Fhead=Latest=ignite-2.8.1
>>> 
>>> 
>>> On 23.04.2020 13:25, Mikhail Petrov wrote:
 Hello, Igniters.
 
 I propose to cherry-pick to 2.8.1 ticket [1].
 
 
 In addition to adding a new metric, it fixes a bug when, after
>> deactivation, GridDhtPartitionsExchangeFuture#rebalanced flag was not
>> reset. And therefore, it can be different on nodes that are already in the
>> cluster from newly joined ones.
 
 [1] - https://issues.apache.org/jira/browse/IGNITE-12788
 
 Regards,
 Mikhail.
 
 On 22.04.2020 14:03, Nikolay Izhikov wrote:
> Hello, Ivan.
> 
> I think we can include this improvements.
> Please, go ahead.
> 
>> 22 апр. 2020 г., в 10:21, Ivan Bessonov 
>> написал(а):
>> 
>> Hi Igniters,
>> 
>> I'm continuing with IGNITE-12756 PR creation. It turned out that we
>> need 3
>> more cherry-picks
>> to avoid massive code changes. Tests started failing after my initial
>> attempt to create that PR.
>> 
>> So, in total I have PR with 6 commits in it. Some of them fix
>> components
>> and tests for them,
>> others are required for new tests to compile and run without changes
>> in
>> 2.8.1 branch.
>> RunAll is in progress now and I'll reply with news if I have any.
>> 
>> Andrey Gura, Nikolay Izhikov, are you OK with 3 more commits in the
>> scope?
>> If you don't,
>> then I won't be able to port IGNITE-12756 to 2.8.1 properly by
>> myself, I
>> guess it would be
>> easier to reimplement it from the scratch.
>> 
>> [1] https://issues.apache.org/jira/browse/IGNITE-12735
>> [2] https://issues.apache.org/jira/browse/IGNITE-12568
>> [3] https://issues.apache.org/jira/browse/IGNITE-12756
>> 
>> [4] https://issues.apache.org/jira/browse/IGNITE-12285
>> [5] https://issues.apache.org/jira/browse/IGNITE-12668
>> [6] https://issues.apache.org/jira/browse/IGNITE-12682
>> [7] https://github.com/apache/ignite/pull/7708
>> 
>> 
>> вт, 21 апр. 2020 г. в 19:36, Pavel Pereslegin :
>> 
>>> Hello, Nikolay.
>>> 
>>> This has been fixed by increasing the test timeout in IGNITE-12683
>> [1][2].
>>> 
>>> [1] https://issues.apache.org/jira/browse/IGNITE-12683
>>> [2]

Re: Apache Ignite 2.8.1 RELEASE [Time, Scope, Manager]

[Alex Plehanov]

Hello guys,

While we are still waiting for some tickets to resolve I propose to
cherry-pick to 2.8.1 two more bugfixes:
IGNITE-12933 Fixed node failure after put incorrect key class for cache
with indexed types
IGNITE-12855 Fixed node failure with concurrent get operation and entry
expiration when persistent is enabled
Both fixes prevent node failure in some circumstances, both fixed already
merged to master.

WDYT?

пн, 27 апр. 2020 г. в 11:53, Nikolay Izhikov :

> Taras.
>
> Thank you, very much!
> You changes merged to 2.8.1 branch.
>
> Igniters,
>
> We have 10 tickets scheduled for 2.8.1 release:
>
> OPEN:
>
> IGNITE-11687Concurrent WAL replay & log may fail with CRC error on
> read - Dmitriy Govorukhin
> IGNITE-12346.NET: Platform error:System.NullReferenceException -
> Pavel Tupitsyn
>
> IN PROGRESS:
>
> IGNITE-12637IgniteSparkSession doesn't start the clients on really
> distributed cluster - Yaroslav Molochkov
> IGNITE-12788Cluster achieved fully rebalanced (PME-free ready) state
> metric - Nikolay Izhikov
>
> PATCH AVAILABLE:
>
> IGNITE-10417notifyDiscoveryListener() call can be lost. - Pavel
> Voronkin
> IGNITE-12852Comma in field is not supported by COPY command - YuJue Li
> IGNITE-12252Unchecked exceptions during rebalancing should be handled
> - Nikolai Kulagin
> IGNITE-12905QueryKeyValueIterable missing custom spliterator()
> implementation - Johnny Galatikitis
> IGNITE-12801Possible extra page release when throttling and checkpoint
> thread store its concurrently - Anton Kalashnikov
> IGNITE-12794Scan query fails with an assertion error: Unexpected row
> key - Denis Mekhanikov
>
>
> > 27 апр. 2020 г., в 11:08, Taras Ledkov 
> написал(а):
> >
> > Hi,
> >
> > Nikolay, i've created PR [1] that contains the SQL-related tickets to
> port into 2.8.1:
> >
> > IGNITE-12790 Introduce distributed SQL configuration and ability to
> disable SQL functions.
> > IGNITE-12887 Fix handle type mismatch exception on compare values while
> traversing index tree.
> > IGNITE-12848 fix H2Connection leaks on INSERT
> > IGNITE-12800  SQL: local queries cursors must be closed or full read to
> unlock the GridH2Table.
> >
> > TC test are OK. Please take a look at the TC bot report [2].
> > Please review & merge the patch into ignite-2.8.1.
> >
> > [1]. https://github.com/apache/ignite/pull/7703
> > [2].
> https://mtcga.gridgain.com/pr.html?serverId=apache=IgniteTests24Java8_RunAll=pull%2F7703%2Fhead=Latest=ignite-2.8.1
> >
> >
> > On 23.04.2020 13:25, Mikhail Petrov wrote:
> >> Hello, Igniters.
> >>
> >> I propose to cherry-pick to 2.8.1 ticket [1].
> >>
> >>
> >> In addition to adding a new metric, it fixes a bug when, after
> deactivation, GridDhtPartitionsExchangeFuture#rebalanced flag was not
> reset. And therefore, it can be different on nodes that are already in the
> cluster from newly joined ones.
> >>
> >> [1] - https://issues.apache.org/jira/browse/IGNITE-12788
> >>
> >> Regards,
> >> Mikhail.
> >>
> >> On 22.04.2020 14:03, Nikolay Izhikov wrote:
> >>> Hello, Ivan.
> >>>
> >>> I think we can include this improvements.
> >>> Please, go ahead.
> >>>
>  22 апр. 2020 г., в 10:21, Ivan Bessonov 
> написал(а):
> 
>  Hi Igniters,
> 
>  I'm continuing with IGNITE-12756 PR creation. It turned out that we
> need 3
>  more cherry-picks
>  to avoid massive code changes. Tests started failing after my initial
>  attempt to create that PR.
> 
>  So, in total I have PR with 6 commits in it. Some of them fix
> components
>  and tests for them,
>  others are required for new tests to compile and run without changes
> in
>  2.8.1 branch.
>  RunAll is in progress now and I'll reply with news if I have any.
> 
>  Andrey Gura, Nikolay Izhikov, are you OK with 3 more commits in the
> scope?
>  If you don't,
>  then I won't be able to port IGNITE-12756 to 2.8.1 properly by
> myself, I
>  guess it would be
>  easier to reimplement it from the scratch.
> 
>  [1] https://issues.apache.org/jira/browse/IGNITE-12735
>  [2] https://issues.apache.org/jira/browse/IGNITE-12568
>  [3] https://issues.apache.org/jira/browse/IGNITE-12756
>  
>  [4] https://issues.apache.org/jira/browse/IGNITE-12285
>  [5] https://issues.apache.org/jira/browse/IGNITE-12668
>  [6] https://issues.apache.org/jira/browse/IGNITE-12682
>  [7] https://github.com/apache/ignite/pull/7708
> 
> 
>  вт, 21 апр. 2020 г. в 19:36, Pavel Pereslegin :
> 
> > Hello, Nikolay.
> >
> > This has been fixed by increasing the test timeout in IGNITE-12683
> [1][2].
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-12683
> > [2]
> >
> https://github.com/apache/ignite/commit/bf394a77e1de6432e493eb2818243a82b577f11e
> >
> > вт, 21 апр. 2020 г. в 18:28, Nikolay Izhikov :
> >> Hello, Igniters.
> >>
> >> While 

Re: Check indexes inline size tool

[Sergey Antonov]

Maxim, I'm talking about cluster upgrade through cluster stop -> binary
update -> cluster start.

вт, 28 апр. 2020 г. в 14:37, Maxim Muzafarov :

> Sergey,
>
> Are you talking about a cluster rolling upgrade feature? AFAIK, Apache
> Ignite doesn't support this feature, so why we should care about it?
>
> On Tue, 28 Apr 2020 at 14:32, Sergey Antonov 
> wrote:
> >
> > Maxim,
> >
> > > should we _reject_ joining nodes which have different
> > From my point of view, it's a breaking change on cluster update.
> >
> > We can get a different inline size in other scenarios too: as I know we
> did
> > some improvements in calculation effective (actual) index inline size.
> > Let's imagine, we have PDS cluster created on the "old" apache ignite
> > version. We decided to upgrade Ignite version and after that, join a new
> > node to the cluster. On a new node, effective inline sizes will be
> > calculated by the optimized algorithm. On the old nodes, the inline size
> > will not be recalculated. It can lead to a difference in inline sizes
> > without IGNITE_MAX_INDEX_PAYLOAD_SIZE option.
> >
> >
> >
> > вт, 28 апр. 2020 г. в 14:22, Maxim Muzafarov :
> >
> > > Sergey, Ilya,
> > >
> > >
> > > Since inline size for the `create table` clause not supported yet and
> > > the IGNITE_MAX_INDEX_PAYLOAD_SIZE is the only option, should we
> > > _reject_ joining nodes which have different
> > > IGNITE_MAX_INDEX_PAYLOAD_SIZE value instead for allowing and printing
> > > warning message? Thus we will force users to have the same values of
> > > IGNITE_MAX_INDEX_PAYLOAD_SIZE property.
> > >
> > >
> > > On Tue, 28 Apr 2020 at 14:01, Ilya Kasnacheev <
> ilya.kasnach...@gmail.com>
> > > wrote:
> > > >
> > > > Hello!
> > > >
> > > > Unfortunately and embarrassingly, we still do not support passing
> > > > INLINE_SIZE to CREATE TABLE, at least in 2.8.0.
> > > > This means IGNITE_MAX_INDEX_PAYLOAD_SIZE is the only option to
> create an
> > > > implicit primary key index with specified inline size.
> > > >
> > > > Regards,
> > > > --
> > > > Ilya Kasnacheev
> > > >
> > > >
> > > > вт, 28 апр. 2020 г. в 02:31, Denis Magda :
> > > >
> > > > > Hi Sergey,
> > > > >
> > > > > Your changes look useful from the application developer
> perspective.
> > > > > However, I'm curious why would the one change some low-level
> > > > > IGNITE_MAX_INDEX_PAYLOAD_SIZE parameter when it's advised to pass
> > > > > INLINE_SIZE to CREATE TABLE to change the index size cluster-wide.
> > > > >
> > > > > -
> > > > > Denis
> > > > >
> > > > >
> > > > > On Mon, Apr 27, 2020 at 5:38 AM Sergey Antonov <
> > > antonovserge...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi, Igniters!
> > > > > >
> > > > > > I'd like to share a new small feature in AI [1].
> > > > > >
> > > > > > For different reasons, the cluster could have a different SQL
> index
> > > > > inline
> > > > > > size [2] on cluster nodes. For example due to
> > > > > > different IGNITE_MAX_INDEX_PAYLOAD_SIZE [3] value on cluster
> nodes.
> > > > > >
> > > > > > The difference in index inline size may lead to performance
> > > degradation.
> > > > > > I think we must compare inline sizes on node join and warn if
> > > difference
> > > > > > found. Also, We should have the ability to check inline sizes on
> > > demand.
> > > > > >
> > > > > > I've implemented this check on node join and new command in
> > > control.sh
> > > > > >
> > > > > > Look at warning message and utility command output:
> > > > > >
> > > > > > Warn message on a node in the cluster during new node join:
> > > > > >
> > > > > > [2020-04-27 15:36:21,185][WARN ][tcp-disco-msg-worker-[6ba0b823
> > > > > > 127.0.0.1:47502
> > > > > >
> crd]-#17%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > > > > Inline
> > > > > > sizes on local node and node
> 5bf6ca48-34a0-4aff-8db2-c0b6df303d3f are
> > > > > > different. Please drop and create again these indexes to avoid
> > > > > performance
> > > > > > problems with SQL queries. Problem indexes:
> > > > > >
> > > > > >
> > > > >
> > >
> PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
> > > > > >
> > > > > > Warn messages on a joining node, if difference found:
> > > > > > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
> > > > > > 127.0.0.1:47501
> > > > > > ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > > > > > Inline sizes on local node and node
> > > a86e9cea-63e8-42af-a897-cec4be51
> > > > > > are different. Please drop and create again these indexes to
> avoid
> > > > > > performance problems with SQL queries. Problem indexes:
> > > > > >
> > > > > >
> > > > >
> > >
> PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
> > > > > > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
> > > > > > 127.0.0.1:47501
> > > > > > ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > > > > > Inline sizes on local node and node
> > > 

Re: Check indexes inline size tool

[Maxim Muzafarov]

Sergey,

Are you talking about a cluster rolling upgrade feature? AFAIK, Apache
Ignite doesn't support this feature, so why we should care about it?

On Tue, 28 Apr 2020 at 14:32, Sergey Antonov  wrote:
>
> Maxim,
>
> > should we _reject_ joining nodes which have different
> From my point of view, it's a breaking change on cluster update.
>
> We can get a different inline size in other scenarios too: as I know we did
> some improvements in calculation effective (actual) index inline size.
> Let's imagine, we have PDS cluster created on the "old" apache ignite
> version. We decided to upgrade Ignite version and after that, join a new
> node to the cluster. On a new node, effective inline sizes will be
> calculated by the optimized algorithm. On the old nodes, the inline size
> will not be recalculated. It can lead to a difference in inline sizes
> without IGNITE_MAX_INDEX_PAYLOAD_SIZE option.
>
>
>
> вт, 28 апр. 2020 г. в 14:22, Maxim Muzafarov :
>
> > Sergey, Ilya,
> >
> >
> > Since inline size for the `create table` clause not supported yet and
> > the IGNITE_MAX_INDEX_PAYLOAD_SIZE is the only option, should we
> > _reject_ joining nodes which have different
> > IGNITE_MAX_INDEX_PAYLOAD_SIZE value instead for allowing and printing
> > warning message? Thus we will force users to have the same values of
> > IGNITE_MAX_INDEX_PAYLOAD_SIZE property.
> >
> >
> > On Tue, 28 Apr 2020 at 14:01, Ilya Kasnacheev 
> > wrote:
> > >
> > > Hello!
> > >
> > > Unfortunately and embarrassingly, we still do not support passing
> > > INLINE_SIZE to CREATE TABLE, at least in 2.8.0.
> > > This means IGNITE_MAX_INDEX_PAYLOAD_SIZE is the only option to create an
> > > implicit primary key index with specified inline size.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > вт, 28 апр. 2020 г. в 02:31, Denis Magda :
> > >
> > > > Hi Sergey,
> > > >
> > > > Your changes look useful from the application developer perspective.
> > > > However, I'm curious why would the one change some low-level
> > > > IGNITE_MAX_INDEX_PAYLOAD_SIZE parameter when it's advised to pass
> > > > INLINE_SIZE to CREATE TABLE to change the index size cluster-wide.
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Mon, Apr 27, 2020 at 5:38 AM Sergey Antonov <
> > antonovserge...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi, Igniters!
> > > > >
> > > > > I'd like to share a new small feature in AI [1].
> > > > >
> > > > > For different reasons, the cluster could have a different SQL index
> > > > inline
> > > > > size [2] on cluster nodes. For example due to
> > > > > different IGNITE_MAX_INDEX_PAYLOAD_SIZE [3] value on cluster nodes.
> > > > >
> > > > > The difference in index inline size may lead to performance
> > degradation.
> > > > > I think we must compare inline sizes on node join and warn if
> > difference
> > > > > found. Also, We should have the ability to check inline sizes on
> > demand.
> > > > >
> > > > > I've implemented this check on node join and new command in
> > control.sh
> > > > >
> > > > > Look at warning message and utility command output:
> > > > >
> > > > > Warn message on a node in the cluster during new node join:
> > > > >
> > > > > [2020-04-27 15:36:21,185][WARN ][tcp-disco-msg-worker-[6ba0b823
> > > > > 127.0.0.1:47502
> > > > > crd]-#17%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > > > Inline
> > > > > sizes on local node and node 5bf6ca48-34a0-4aff-8db2-c0b6df303d3f are
> > > > > different. Please drop and create again these indexes to avoid
> > > > performance
> > > > > problems with SQL queries. Problem indexes:
> > > > >
> > > > >
> > > >
> > PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
> > > > >
> > > > > Warn messages on a joining node, if difference found:
> > > > > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
> > > > > 127.0.0.1:47501
> > > > > ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > > > > Inline sizes on local node and node
> > a86e9cea-63e8-42af-a897-cec4be51
> > > > > are different. Please drop and create again these indexes to avoid
> > > > > performance problems with SQL queries. Problem indexes:
> > > > >
> > > > >
> > > >
> > PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
> > > > > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
> > > > > 127.0.0.1:47501
> > > > > ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > > > > Inline sizes on local node and node
> > a08de16d-df05-48af-a0b9-5596d9c2
> > > > > are different. Please drop and create again these indexes to avoid
> > > > > performance problems with SQL queries. Problem indexes:
> > > > >
> > > > >
> > > >
> > PUBLIC#TEST_TABLE#L_IDX(1,3),PUBLIC#TEST_TABLE#S1_IDX(1,3),PUBLIC#TEST_TABLE#I_IDX(1,3)
> > > > >
> > > > >
> > > > > Utility output, if a difference in inline sizes was found:
> > > > >
> > > > > Control utility [ver. 

Re: Check indexes inline size tool

[Sergey Antonov]

Maxim,

> should we _reject_ joining nodes which have different
>From my point of view, it's a breaking change on cluster update.

We can get a different inline size in other scenarios too: as I know we did
some improvements in calculation effective (actual) index inline size.
Let's imagine, we have PDS cluster created on the "old" apache ignite
version. We decided to upgrade Ignite version and after that, join a new
node to the cluster. On a new node, effective inline sizes will be
calculated by the optimized algorithm. On the old nodes, the inline size
will not be recalculated. It can lead to a difference in inline sizes
without IGNITE_MAX_INDEX_PAYLOAD_SIZE option.



вт, 28 апр. 2020 г. в 14:22, Maxim Muzafarov :

> Sergey, Ilya,
>
>
> Since inline size for the `create table` clause not supported yet and
> the IGNITE_MAX_INDEX_PAYLOAD_SIZE is the only option, should we
> _reject_ joining nodes which have different
> IGNITE_MAX_INDEX_PAYLOAD_SIZE value instead for allowing and printing
> warning message? Thus we will force users to have the same values of
> IGNITE_MAX_INDEX_PAYLOAD_SIZE property.
>
>
> On Tue, 28 Apr 2020 at 14:01, Ilya Kasnacheev 
> wrote:
> >
> > Hello!
> >
> > Unfortunately and embarrassingly, we still do not support passing
> > INLINE_SIZE to CREATE TABLE, at least in 2.8.0.
> > This means IGNITE_MAX_INDEX_PAYLOAD_SIZE is the only option to create an
> > implicit primary key index with specified inline size.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > вт, 28 апр. 2020 г. в 02:31, Denis Magda :
> >
> > > Hi Sergey,
> > >
> > > Your changes look useful from the application developer perspective.
> > > However, I'm curious why would the one change some low-level
> > > IGNITE_MAX_INDEX_PAYLOAD_SIZE parameter when it's advised to pass
> > > INLINE_SIZE to CREATE TABLE to change the index size cluster-wide.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Mon, Apr 27, 2020 at 5:38 AM Sergey Antonov <
> antonovserge...@gmail.com>
> > > wrote:
> > >
> > > > Hi, Igniters!
> > > >
> > > > I'd like to share a new small feature in AI [1].
> > > >
> > > > For different reasons, the cluster could have a different SQL index
> > > inline
> > > > size [2] on cluster nodes. For example due to
> > > > different IGNITE_MAX_INDEX_PAYLOAD_SIZE [3] value on cluster nodes.
> > > >
> > > > The difference in index inline size may lead to performance
> degradation.
> > > > I think we must compare inline sizes on node join and warn if
> difference
> > > > found. Also, We should have the ability to check inline sizes on
> demand.
> > > >
> > > > I've implemented this check on node join and new command in
> control.sh
> > > >
> > > > Look at warning message and utility command output:
> > > >
> > > > Warn message on a node in the cluster during new node join:
> > > >
> > > > [2020-04-27 15:36:21,185][WARN ][tcp-disco-msg-worker-[6ba0b823
> > > > 127.0.0.1:47502
> > > > crd]-#17%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > > Inline
> > > > sizes on local node and node 5bf6ca48-34a0-4aff-8db2-c0b6df303d3f are
> > > > different. Please drop and create again these indexes to avoid
> > > performance
> > > > problems with SQL queries. Problem indexes:
> > > >
> > > >
> > >
> PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
> > > >
> > > > Warn messages on a joining node, if difference found:
> > > > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
> > > > 127.0.0.1:47501
> > > > ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > > > Inline sizes on local node and node
> a86e9cea-63e8-42af-a897-cec4be51
> > > > are different. Please drop and create again these indexes to avoid
> > > > performance problems with SQL queries. Problem indexes:
> > > >
> > > >
> > >
> PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
> > > > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
> > > > 127.0.0.1:47501
> > > > ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > > > Inline sizes on local node and node
> a08de16d-df05-48af-a0b9-5596d9c2
> > > > are different. Please drop and create again these indexes to avoid
> > > > performance problems with SQL queries. Problem indexes:
> > > >
> > > >
> > >
> PUBLIC#TEST_TABLE#L_IDX(1,3),PUBLIC#TEST_TABLE#S1_IDX(1,3),PUBLIC#TEST_TABLE#I_IDX(1,3)
> > > >
> > > >
> > > > Utility output, if a difference in inline sizes was found:
> > > >
> > > > Control utility [ver. 2.9.0-SNAPSHOT#20200427-sha1:DEV]
> > > > 2020 Copyright(C) Apache Software Foundation
> > > > User: santonov
> > > > Time: 2020-04-27T15:32:25.759
> > > > Command [CACHE] started
> > > > Arguments: --cache check_index_inline_sizes --yes
> > > >
> > > >
> > >
> 
> > > > Found 4 secondary indexes.
> > > > 3 index(es) have different effective inline size on nodes. It can
> lead 

Re: Check indexes inline size tool

[Sergey Antonov]

Hello!

Unfortunately, that's true. But the user can restart cluster after tables
creation and create secondary indexes (CREATE INDEX) after restart. My
workaround has a lot of limitations: it doesn't work with in-memory
clusters, it's unuseful.

вт, 28 апр. 2020 г. в 14:01, Ilya Kasnacheev :

> Hello!
>
> Unfortunately and embarrassingly, we still do not support passing
> INLINE_SIZE to CREATE TABLE, at least in 2.8.0.
> This means IGNITE_MAX_INDEX_PAYLOAD_SIZE is the only option to create an
> implicit primary key index with specified inline size.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 28 апр. 2020 г. в 02:31, Denis Magda :
>
> > Hi Sergey,
> >
> > Your changes look useful from the application developer perspective.
> > However, I'm curious why would the one change some low-level
> > IGNITE_MAX_INDEX_PAYLOAD_SIZE parameter when it's advised to pass
> > INLINE_SIZE to CREATE TABLE to change the index size cluster-wide.
> >
> > -
> > Denis
> >
> >
> > On Mon, Apr 27, 2020 at 5:38 AM Sergey Antonov <
> antonovserge...@gmail.com>
> > wrote:
> >
> > > Hi, Igniters!
> > >
> > > I'd like to share a new small feature in AI [1].
> > >
> > > For different reasons, the cluster could have a different SQL index
> > inline
> > > size [2] on cluster nodes. For example due to
> > > different IGNITE_MAX_INDEX_PAYLOAD_SIZE [3] value on cluster nodes.
> > >
> > > The difference in index inline size may lead to performance
> degradation.
> > > I think we must compare inline sizes on node join and warn if
> difference
> > > found. Also, We should have the ability to check inline sizes on
> demand.
> > >
> > > I've implemented this check on node join and new command in control.sh
> > >
> > > Look at warning message and utility command output:
> > >
> > > Warn message on a node in the cluster during new node join:
> > >
> > > [2020-04-27 15:36:21,185][WARN ][tcp-disco-msg-worker-[6ba0b823
> > > 127.0.0.1:47502
> > > crd]-#17%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > Inline
> > > sizes on local node and node 5bf6ca48-34a0-4aff-8db2-c0b6df303d3f are
> > > different. Please drop and create again these indexes to avoid
> > performance
> > > problems with SQL queries. Problem indexes:
> > >
> > >
> >
> PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
> > >
> > > Warn messages on a joining node, if difference found:
> > > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
> > > 127.0.0.1:47501
> > > ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > > Inline sizes on local node and node
> a86e9cea-63e8-42af-a897-cec4be51
> > > are different. Please drop and create again these indexes to avoid
> > > performance problems with SQL queries. Problem indexes:
> > >
> > >
> >
> PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
> > > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
> > > 127.0.0.1:47501
> > > ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > > Inline sizes on local node and node
> a08de16d-df05-48af-a0b9-5596d9c2
> > > are different. Please drop and create again these indexes to avoid
> > > performance problems with SQL queries. Problem indexes:
> > >
> > >
> >
> PUBLIC#TEST_TABLE#L_IDX(1,3),PUBLIC#TEST_TABLE#S1_IDX(1,3),PUBLIC#TEST_TABLE#I_IDX(1,3)
> > >
> > >
> > > Utility output, if a difference in inline sizes was found:
> > >
> > > Control utility [ver. 2.9.0-SNAPSHOT#20200427-sha1:DEV]
> > > 2020 Copyright(C) Apache Software Foundation
> > > User: santonov
> > > Time: 2020-04-27T15:32:25.759
> > > Command [CACHE] started
> > > Arguments: --cache check_index_inline_sizes --yes
> > >
> > >
> >
> 
> > > Found 4 secondary indexes.
> > > 3 index(es) have different effective inline size on nodes. It can lead
> to
> > > performance degradation in SQL queries.
> > > Index(es):
> > >   Full index name: PUBLIC#TEST_TABLE#L_IDX nodes:
> > > [ca1d2bae-89d4-4e8d-ae11-6c68f390] inline size: 1, nodes:
> > > [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
> > >   Full index name: PUBLIC#TEST_TABLE#S1_IDX nodes:
> > > [ca1d2bae-89d4-4e8d-ae11-6c68f390] inline size: 1, nodes:
> > > [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
> > >   Full index name: PUBLIC#TEST_TABLE#I_IDX nodes:
> > > [ca1d2bae-89d4-4e8d-ae11-6c68f390] inline size: 1, nodes:
> > > [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
> > >
> > > Recommendations:
> > >   Check that value of property IGNITE_MAX_INDEX_PAYLOAD_SIZE are the
> same
> > > on all nodes.
> > >   Recreate indexes (execute DROP INDEX, CREATE INDEX commands) with
> > > different inline size.
> > > Command [CACHE] finished with code: 0
> > > Control utility has completed execution at: 2020-04-27T15:32:28.025
> > > Execution time: 2266 ms
> > >
> > > Utility output, if all indexes have the same inline size:
> > >

Re: Check indexes inline size tool

[Maxim Muzafarov]

Sergey, Ilya,


Since inline size for the `create table` clause not supported yet and
the IGNITE_MAX_INDEX_PAYLOAD_SIZE is the only option, should we
_reject_ joining nodes which have different
IGNITE_MAX_INDEX_PAYLOAD_SIZE value instead for allowing and printing
warning message? Thus we will force users to have the same values of
IGNITE_MAX_INDEX_PAYLOAD_SIZE property.


On Tue, 28 Apr 2020 at 14:01, Ilya Kasnacheev  wrote:
>
> Hello!
>
> Unfortunately and embarrassingly, we still do not support passing
> INLINE_SIZE to CREATE TABLE, at least in 2.8.0.
> This means IGNITE_MAX_INDEX_PAYLOAD_SIZE is the only option to create an
> implicit primary key index with specified inline size.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 28 апр. 2020 г. в 02:31, Denis Magda :
>
> > Hi Sergey,
> >
> > Your changes look useful from the application developer perspective.
> > However, I'm curious why would the one change some low-level
> > IGNITE_MAX_INDEX_PAYLOAD_SIZE parameter when it's advised to pass
> > INLINE_SIZE to CREATE TABLE to change the index size cluster-wide.
> >
> > -
> > Denis
> >
> >
> > On Mon, Apr 27, 2020 at 5:38 AM Sergey Antonov 
> > wrote:
> >
> > > Hi, Igniters!
> > >
> > > I'd like to share a new small feature in AI [1].
> > >
> > > For different reasons, the cluster could have a different SQL index
> > inline
> > > size [2] on cluster nodes. For example due to
> > > different IGNITE_MAX_INDEX_PAYLOAD_SIZE [3] value on cluster nodes.
> > >
> > > The difference in index inline size may lead to performance degradation.
> > > I think we must compare inline sizes on node join and warn if difference
> > > found. Also, We should have the ability to check inline sizes on demand.
> > >
> > > I've implemented this check on node join and new command in control.sh
> > >
> > > Look at warning message and utility command output:
> > >
> > > Warn message on a node in the cluster during new node join:
> > >
> > > [2020-04-27 15:36:21,185][WARN ][tcp-disco-msg-worker-[6ba0b823
> > > 127.0.0.1:47502
> > > crd]-#17%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > Inline
> > > sizes on local node and node 5bf6ca48-34a0-4aff-8db2-c0b6df303d3f are
> > > different. Please drop and create again these indexes to avoid
> > performance
> > > problems with SQL queries. Problem indexes:
> > >
> > >
> > PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
> > >
> > > Warn messages on a joining node, if difference found:
> > > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
> > > 127.0.0.1:47501
> > > ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > > Inline sizes on local node and node a86e9cea-63e8-42af-a897-cec4be51
> > > are different. Please drop and create again these indexes to avoid
> > > performance problems with SQL queries. Problem indexes:
> > >
> > >
> > PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
> > > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
> > > 127.0.0.1:47501
> > > ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > > Inline sizes on local node and node a08de16d-df05-48af-a0b9-5596d9c2
> > > are different. Please drop and create again these indexes to avoid
> > > performance problems with SQL queries. Problem indexes:
> > >
> > >
> > PUBLIC#TEST_TABLE#L_IDX(1,3),PUBLIC#TEST_TABLE#S1_IDX(1,3),PUBLIC#TEST_TABLE#I_IDX(1,3)
> > >
> > >
> > > Utility output, if a difference in inline sizes was found:
> > >
> > > Control utility [ver. 2.9.0-SNAPSHOT#20200427-sha1:DEV]
> > > 2020 Copyright(C) Apache Software Foundation
> > > User: santonov
> > > Time: 2020-04-27T15:32:25.759
> > > Command [CACHE] started
> > > Arguments: --cache check_index_inline_sizes --yes
> > >
> > >
> > 
> > > Found 4 secondary indexes.
> > > 3 index(es) have different effective inline size on nodes. It can lead to
> > > performance degradation in SQL queries.
> > > Index(es):
> > >   Full index name: PUBLIC#TEST_TABLE#L_IDX nodes:
> > > [ca1d2bae-89d4-4e8d-ae11-6c68f390] inline size: 1, nodes:
> > > [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
> > >   Full index name: PUBLIC#TEST_TABLE#S1_IDX nodes:
> > > [ca1d2bae-89d4-4e8d-ae11-6c68f390] inline size: 1, nodes:
> > > [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
> > >   Full index name: PUBLIC#TEST_TABLE#I_IDX nodes:
> > > [ca1d2bae-89d4-4e8d-ae11-6c68f390] inline size: 1, nodes:
> > > [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
> > >
> > > Recommendations:
> > >   Check that value of property IGNITE_MAX_INDEX_PAYLOAD_SIZE are the same
> > > on all nodes.
> > >   Recreate indexes (execute DROP INDEX, CREATE INDEX commands) with
> > > different inline size.
> > > Command [CACHE] finished with code: 0
> > > Control utility has completed execution at: 2020-04-27T15:32:28.025
> > > Execution time: 

Re: Check indexes inline size tool

[Ilya Kasnacheev]

Hello!

Unfortunately and embarrassingly, we still do not support passing
INLINE_SIZE to CREATE TABLE, at least in 2.8.0.
This means IGNITE_MAX_INDEX_PAYLOAD_SIZE is the only option to create an
implicit primary key index with specified inline size.

Regards,
-- 
Ilya Kasnacheev


вт, 28 апр. 2020 г. в 02:31, Denis Magda :

> Hi Sergey,
>
> Your changes look useful from the application developer perspective.
> However, I'm curious why would the one change some low-level
> IGNITE_MAX_INDEX_PAYLOAD_SIZE parameter when it's advised to pass
> INLINE_SIZE to CREATE TABLE to change the index size cluster-wide.
>
> -
> Denis
>
>
> On Mon, Apr 27, 2020 at 5:38 AM Sergey Antonov 
> wrote:
>
> > Hi, Igniters!
> >
> > I'd like to share a new small feature in AI [1].
> >
> > For different reasons, the cluster could have a different SQL index
> inline
> > size [2] on cluster nodes. For example due to
> > different IGNITE_MAX_INDEX_PAYLOAD_SIZE [3] value on cluster nodes.
> >
> > The difference in index inline size may lead to performance degradation.
> > I think we must compare inline sizes on node join and warn if difference
> > found. Also, We should have the ability to check inline sizes on demand.
> >
> > I've implemented this check on node join and new command in control.sh
> >
> > Look at warning message and utility command output:
> >
> > Warn message on a node in the cluster during new node join:
> >
> > [2020-04-27 15:36:21,185][WARN ][tcp-disco-msg-worker-[6ba0b823
> > 127.0.0.1:47502
> > crd]-#17%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> Inline
> > sizes on local node and node 5bf6ca48-34a0-4aff-8db2-c0b6df303d3f are
> > different. Please drop and create again these indexes to avoid
> performance
> > problems with SQL queries. Problem indexes:
> >
> >
> PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
> >
> > Warn messages on a joining node, if difference found:
> > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
> > 127.0.0.1:47501
> > ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > Inline sizes on local node and node a86e9cea-63e8-42af-a897-cec4be51
> > are different. Please drop and create again these indexes to avoid
> > performance problems with SQL queries. Problem indexes:
> >
> >
> PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
> > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
> > 127.0.0.1:47501
> > ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > Inline sizes on local node and node a08de16d-df05-48af-a0b9-5596d9c2
> > are different. Please drop and create again these indexes to avoid
> > performance problems with SQL queries. Problem indexes:
> >
> >
> PUBLIC#TEST_TABLE#L_IDX(1,3),PUBLIC#TEST_TABLE#S1_IDX(1,3),PUBLIC#TEST_TABLE#I_IDX(1,3)
> >
> >
> > Utility output, if a difference in inline sizes was found:
> >
> > Control utility [ver. 2.9.0-SNAPSHOT#20200427-sha1:DEV]
> > 2020 Copyright(C) Apache Software Foundation
> > User: santonov
> > Time: 2020-04-27T15:32:25.759
> > Command [CACHE] started
> > Arguments: --cache check_index_inline_sizes --yes
> >
> >
> 
> > Found 4 secondary indexes.
> > 3 index(es) have different effective inline size on nodes. It can lead to
> > performance degradation in SQL queries.
> > Index(es):
> >   Full index name: PUBLIC#TEST_TABLE#L_IDX nodes:
> > [ca1d2bae-89d4-4e8d-ae11-6c68f390] inline size: 1, nodes:
> > [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
> >   Full index name: PUBLIC#TEST_TABLE#S1_IDX nodes:
> > [ca1d2bae-89d4-4e8d-ae11-6c68f390] inline size: 1, nodes:
> > [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
> >   Full index name: PUBLIC#TEST_TABLE#I_IDX nodes:
> > [ca1d2bae-89d4-4e8d-ae11-6c68f390] inline size: 1, nodes:
> > [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
> >
> > Recommendations:
> >   Check that value of property IGNITE_MAX_INDEX_PAYLOAD_SIZE are the same
> > on all nodes.
> >   Recreate indexes (execute DROP INDEX, CREATE INDEX commands) with
> > different inline size.
> > Command [CACHE] finished with code: 0
> > Control utility has completed execution at: 2020-04-27T15:32:28.025
> > Execution time: 2266 ms
> >
> > Utility output, if all indexes have the same inline size:
> >
> > Control utility [ver. 2.9.0-SNAPSHOT#20200427-sha1:DEV]
> > 2020 Copyright(C) Apache Software Foundation
> > User: santonov
> > Time: 2020-04-27T15:30:20.950
> > Command [CACHE] started
> > Arguments: --cache check_index_inline_sizes --yes
> >
> >
> 
> > Found 2 secondary indexes.
> > All secondary indexes have the same effective inline size on all cluster
> > nodes.
> > Command [CACHE] finished with code: 0
> > Control utility has completed execution at: 2020-04-27T15:30:23.428
> > 

[jira] [Created] (IGNITE-12961) Start snapshot operation via control.sh

[Maxim Muzafarov (Jira)]

Maxim Muzafarov created IGNITE-12961:


 Summary: Start snapshot operation via control.sh
 Key: IGNITE-12961
 URL: https://issues.apache.org/jira/browse/IGNITE-12961
 Project: Ignite
  Issue Type: Task
Reporter: Maxim Muzafarov
Assignee: Maxim Muzafarov
 Fix For: 2.9






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE][EXTENSION] Release Apache Ignite Spring Boot extensions 1.0.0 RC1

[Maksim Stepachev]

+1

вт, 28 апр. 2020 г. в 11:27, Nikolay Izhikov :

> *** Formal vote description fixed ***
>
> Dear Community,
>
> I have uploaded a release candidate of the two extension modules
> `ignite-spring-boot-autoconfigure` and
> `ignite-spring-boot-client-autoconfigure`.
>
> The following staging can be used for testing:
> https://repository.apache.org/content/repositories/orgapacheignite-1478/
>
> Tag with name `ignite-spring-boot-1.0.0-rc1` created:
>
> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=8b5f2d9143d281b90aceb6f11dabde50db48f6b3
>
> Release 1.0 contains an initial implementation of the modules, please
> refer to the RELEASE_NOTES:
>
> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=RELEASE_NOTES.txt;h=e69173583068467b44f5e148151654ade3fe2452;hb=HEAD
>
> Complete list of resolved issues:
>
>
> https://issues.apache.org/jira/issues/?jql=labels%20%3D%20spring-boot-autoconfigure
>
> https://issues.apache.org/jira/issues/?jql=labels%20%3D%20spring-boot-client-autoconfigure
>
> DEVNOTES
>
> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=DEVNOTES.txt;h=7f25887adef738db956ae0f70c59839f73965d5f;hb=HEAD
>
> DOCUMENTATION
> https://apacheignite.readme.io/docs/spring-boot-autoconfigure
> https://apacheignite.readme.io/docs/ignite-spring-boot-client-autoconfigure
>
> The vote is formal, see voting guidelines
> https://www.apache.org/foundation/voting.html
>
> +1 - to accept Apache Ignite spring-boot auto configure extensions
> 1.0.0-rc1
> 0 - don't care either way
> -1 - DO NOT accept Apache Ignite spring-boot auto configure extensions
> 1.0.0-rc1 (explain why)
>
> See notes on how to verify release here
> https://www.apache.org/info/verification.html
> and
> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification
>
> The vote will hold for 72 hours and will end on April 30 2020 20:20 UTC
>
> https://www.timeanddate.com/countdown/generic?iso=20200430T2320=166=cursive

Re: IEP-44 Thin Client Discovery

[Pavel Tupitsyn]

> enable the capability if the best effort affinity is on
I agree, makes sense.

Igor, what do you think?

On Tue, Apr 28, 2020 at 8:25 AM Denis Magda  wrote:

> Pavel,
>
> That would be a tremendous improvement for the recently release best effort
> affinity feature. Without this capability, we force application developers
> to reopen thin client connections every type a cluster is scaled out. I
> believe that once the folks start using the best effort affinity, we'll be
> hearing more of a feature request for what you're proposing in this thread.
> So, thanks for taking care of this proactively!
>
> As for the public API changes, do we really need any extra flag? I would
> enable the capability if the best effort affinity is on. For me, it's just
> a natural improvement of the latter and it sounds reasonable to reuse the
> best effort affinity's flag.
>
> -
> Denis
>
>
> On Mon, Apr 27, 2020 at 2:58 AM Pavel Tupitsyn 
> wrote:
>
> > Igniters,
> >
> > I've prepared an IEP [1] and a POC [2] for Thin Client Discovery feature.
> > Let's discuss it here.
> >
> > In particular, I'd like to address the following points:
> >
> > 1. Value: do you think this would be a good feature to have?
> > 2. Public API changes: is a boolean property enough? Should we have
> > something more complex, so users can plug in custom logic to filter
> and/or
> > translate IPs and host names?
> > 3. Server-side implementation details: should we use Compute, Node
> > Attributes, or something else to retrieve client endpoints from all nodes
> > in cluster?
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-44%3A+Thin+client+cluster+discovery
> > [2] https://github.com/apache/ignite/pull/7744
> > [3] https://issues.apache.org/jira/browse/IGNITE-12932
> >
>

Re: Check indexes inline size tool

[Sergey Antonov]

Hi Denis,

The problem could be shown when you invoke CREATE INDEX without the
INLINE_SIZE parameter. You don't face with described problem If index
creates by CREATE_INDEX with explicit INLINE SIZE value.

вт, 28 апр. 2020 г. в 02:31, Denis Magda :

> Hi Sergey,
>
> Your changes look useful from the application developer perspective.
> However, I'm curious why would the one change some low-level
> IGNITE_MAX_INDEX_PAYLOAD_SIZE parameter when it's advised to pass
> INLINE_SIZE to CREATE TABLE to change the index size cluster-wide.
>
> -
> Denis
>
>
> On Mon, Apr 27, 2020 at 5:38 AM Sergey Antonov 
> wrote:
>
> > Hi, Igniters!
> >
> > I'd like to share a new small feature in AI [1].
> >
> > For different reasons, the cluster could have a different SQL index
> inline
> > size [2] on cluster nodes. For example due to
> > different IGNITE_MAX_INDEX_PAYLOAD_SIZE [3] value on cluster nodes.
> >
> > The difference in index inline size may lead to performance degradation.
> > I think we must compare inline sizes on node join and warn if difference
> > found. Also, We should have the ability to check inline sizes on demand.
> >
> > I've implemented this check on node join and new command in control.sh
> >
> > Look at warning message and utility command output:
> >
> > Warn message on a node in the cluster during new node join:
> >
> > [2020-04-27 15:36:21,185][WARN ][tcp-disco-msg-worker-[6ba0b823
> > 127.0.0.1:47502
> > crd]-#17%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> Inline
> > sizes on local node and node 5bf6ca48-34a0-4aff-8db2-c0b6df303d3f are
> > different. Please drop and create again these indexes to avoid
> performance
> > problems with SQL queries. Problem indexes:
> >
> >
> PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
> >
> > Warn messages on a joining node, if difference found:
> > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
> > 127.0.0.1:47501
> > ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > Inline sizes on local node and node a86e9cea-63e8-42af-a897-cec4be51
> > are different. Please drop and create again these indexes to avoid
> > performance problems with SQL queries. Problem indexes:
> >
> >
> PUBLIC#TEST_TABLE#L_IDX(1,2),PUBLIC#TEST_TABLE#S1_IDX(1,2),PUBLIC#TEST_TABLE#I_IDX(1,2)
> > [2020-04-27 15:35:17,326][WARN ][tcp-disco-msg-worker-[a86e9cea
> > 127.0.0.1:47501
> > ]-#11%cache.CheckIndexesInlineSizeOnNodeJoinMultiJvmTest0%][root]
> > Inline sizes on local node and node a08de16d-df05-48af-a0b9-5596d9c2
> > are different. Please drop and create again these indexes to avoid
> > performance problems with SQL queries. Problem indexes:
> >
> >
> PUBLIC#TEST_TABLE#L_IDX(1,3),PUBLIC#TEST_TABLE#S1_IDX(1,3),PUBLIC#TEST_TABLE#I_IDX(1,3)
> >
> >
> > Utility output, if a difference in inline sizes was found:
> >
> > Control utility [ver. 2.9.0-SNAPSHOT#20200427-sha1:DEV]
> > 2020 Copyright(C) Apache Software Foundation
> > User: santonov
> > Time: 2020-04-27T15:32:25.759
> > Command [CACHE] started
> > Arguments: --cache check_index_inline_sizes --yes
> >
> >
> 
> > Found 4 secondary indexes.
> > 3 index(es) have different effective inline size on nodes. It can lead to
> > performance degradation in SQL queries.
> > Index(es):
> >   Full index name: PUBLIC#TEST_TABLE#L_IDX nodes:
> > [ca1d2bae-89d4-4e8d-ae11-6c68f390] inline size: 1, nodes:
> > [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
> >   Full index name: PUBLIC#TEST_TABLE#S1_IDX nodes:
> > [ca1d2bae-89d4-4e8d-ae11-6c68f390] inline size: 1, nodes:
> > [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
> >   Full index name: PUBLIC#TEST_TABLE#I_IDX nodes:
> > [ca1d2bae-89d4-4e8d-ae11-6c68f390] inline size: 1, nodes:
> > [8327abd1-df08-4b97-8720-de95e363e745] inline size: 2
> >
> > Recommendations:
> >   Check that value of property IGNITE_MAX_INDEX_PAYLOAD_SIZE are the same
> > on all nodes.
> >   Recreate indexes (execute DROP INDEX, CREATE INDEX commands) with
> > different inline size.
> > Command [CACHE] finished with code: 0
> > Control utility has completed execution at: 2020-04-27T15:32:28.025
> > Execution time: 2266 ms
> >
> > Utility output, if all indexes have the same inline size:
> >
> > Control utility [ver. 2.9.0-SNAPSHOT#20200427-sha1:DEV]
> > 2020 Copyright(C) Apache Software Foundation
> > User: santonov
> > Time: 2020-04-27T15:30:20.950
> > Command [CACHE] started
> > Arguments: --cache check_index_inline_sizes --yes
> >
> >
> 
> > Found 2 secondary indexes.
> > All secondary indexes have the same effective inline size on all cluster
> > nodes.
> > Command [CACHE] finished with code: 0
> > Control utility has completed execution at: 2020-04-27T15:30:23.428
> > Execution time: 2478 ms
> >
> > Any objections?
> >
> > [1] 

Re: [VOTE][EXTENSION] Release Apache Ignite Spring Boot extensions 1.0.0 RC1

[Nikolay Izhikov]

*** Formal vote description fixed ***

Dear Community,

I have uploaded a release candidate of the two extension modules 
`ignite-spring-boot-autoconfigure` and 
`ignite-spring-boot-client-autoconfigure`.

The following staging can be used for testing:
https://repository.apache.org/content/repositories/orgapacheignite-1478/

Tag with name `ignite-spring-boot-1.0.0-rc1` created:
https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=8b5f2d9143d281b90aceb6f11dabde50db48f6b3

Release 1.0 contains an initial implementation of the modules, please refer to 
the RELEASE_NOTES:
https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=RELEASE_NOTES.txt;h=e69173583068467b44f5e148151654ade3fe2452;hb=HEAD

Complete list of resolved issues:

https://issues.apache.org/jira/issues/?jql=labels%20%3D%20spring-boot-autoconfigure
https://issues.apache.org/jira/issues/?jql=labels%20%3D%20spring-boot-client-autoconfigure

DEVNOTES
https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=DEVNOTES.txt;h=7f25887adef738db956ae0f70c59839f73965d5f;hb=HEAD

DOCUMENTATION
https://apacheignite.readme.io/docs/spring-boot-autoconfigure
https://apacheignite.readme.io/docs/ignite-spring-boot-client-autoconfigure

The vote is formal, see voting guidelines
https://www.apache.org/foundation/voting.html

+1 - to accept Apache Ignite spring-boot auto configure extensions 1.0.0-rc1
0 - don't care either way
-1 - DO NOT accept Apache Ignite spring-boot auto configure extensions 
1.0.0-rc1 (explain why)

See notes on how to verify release here
https://www.apache.org/info/verification.html
and 
https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification

The vote will hold for 72 hours and will end on April 30 2020 20:20 UTC
https://www.timeanddate.com/countdown/generic?iso=20200430T2320=166=cursive

Re: [VOTE][EXTENSION] Release Apache Ignite Spring Boot extensions 1.0.0 RC1

[Nikolay Izhikov]

Hello, Saikat, Denis.

Thanks for feedback.

> 2. Also this voting is for Apache Ignite Spring Boot extensions 1.0.0 RC1 
> instead of Apache Ignite Ignite 2.8.0-rc1. Is this correct?

Good catch, thank you.

> As you might remember, we agreed to update the release process [1]

I missed that we agreed on this.
Do we have a vote or something similar for it?
It seems for me, that such requirement can significantly reduce number of 
extension releases.

Anyway, here is documentation for spring-boot modules:

* https://apacheignite.readme.io/docs/spring-boot-autoconfigure
* 
https://apacheignite.readme.io/docs/ignite-spring-boot-client-autoconfigure


> 28 апр. 2020 г., в 02:29, Saikat Maitra  написал(а):
> 
> Hi Nikolay,
> 
> 1. Can we update the   DEVNOTES.txt to contain build command as below
> 
> "mvn clean install" instead of "mvn clean install
> -Pall-java,all-scala,licenses -DskipTests"
> 
> I have verified that the build passes with tests and also we do not have
> tasks such as all-java, all-scala or licenses.
> 
> 2. Also this voting is for Apache Ignite Spring Boot extensions 1.0.0 RC1
> instead of Apache Ignite Ignite 2.8.0-rc1. Is this correct?
> 
> Regards,
> Saikat
> 
> On Mon, Apr 27, 2020 at 6:05 PM Denis Magda  wrote:
> 
>> Hi Nikolay,
>> 
>> Sorry for being picky about the documentation but I would encourage us as
>> the community to prepare this release artifact before sending a release for
>> a vote. As you might remember, we agreed to update the release process [1]
>> to ensure this practice is followed. Just in case that's a reference to the
>> discussion [2].
>> 
>> Overall, I believe the extensions need to be placed under the
>> "Integrations" tab [3] of our docs. Also, our website still misses generic
>> info about the extensions (repo, purpose, etc.). Should we start a separate
>> discussion and push this quickly to the finish line? Ready to facilitate
>> from my end.
>> 
>> 
>> [1]
>> 
>> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-4.1EnsureDocumentationReadinessandAccouncementBlogPostActivity
>> <
>> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-4.1EnsureDocumentationReadinessandAccouncementBlogPostActivity
>>> 
>> [2]
>> 
>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Changes-in-Ignite-release-process-related-to-documentation-td46471.html
>> [3] https://apacheignite-mix.readme.io/docs
>> 
>> -
>> Denis
>> 
>> 
>> On Mon, Apr 27, 2020 at 1:24 PM Nikolay Izhikov 
>> wrote:
>> 
>>> Dear Community,
>>> 
>>> I have uploaded a release candidate of the two extension modules
>>> `ignite-spring-boot-autoconfigure` and
>>> `ignite-spring-boot-client-autoconfigure`.
>>> 
>>> The following staging can be used for testing:
>>> https://repository.apache.org/content/repositories/orgapacheignite-1478/
>>> 
>>> Tag with name `ignite-spring-boot-1.0.0-rc1` created:
>>> 
>>> 
>> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=commit;h=8b5f2d9143d281b90aceb6f11dabde50db48f6b3
>>> 
>>> Release 1.0 contains an initial implementation of the modules, please
>>> refer to the RELEASE_NOTES:
>>> 
>>> 
>> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=RELEASE_NOTES.txt;h=e69173583068467b44f5e148151654ade3fe2452;hb=HEAD
>>> 
>>> Complete list of resolved issues:
>>> 
>>> 
>>> 
>> https://issues.apache.org/jira/issues/?jql=labels%20%3D%20spring-boot-autoconfigure
>>> 
>>> 
>> https://issues.apache.org/jira/issues/?jql=labels%20%3D%20spring-boot-client-autoconfigure
>>> 
>>> DEVNOTES
>>> 
>>> 
>> https://gitbox.apache.org/repos/asf?p=ignite-extensions.git;a=blob;f=DEVNOTES.txt;h=7f25887adef738db956ae0f70c59839f73965d5f;hb=HEAD
>>> 
>>> The vote is formal, see voting guidelines
>>> https://www.apache.org/foundation/voting.html
>>> 
>>> +1 - to accept Apache Ignite 2.8.0-rc1
>>> 0 - don't care either way
>>> -1 - DO NOT accept Apache Ignite Ignite 2.8.0-rc1 (explain why)
>>> 
>>> See notes on how to verify release here
>>> https://www.apache.org/info/verification.html
>>> and
>>> 
>> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification
>>> 
>>> The vote will hold for 72 hours and will end on April 30 2020 20:20 UTC
>>> 
>>> 
>> https://www.timeanddate.com/countdown/generic?iso=20200430T2320=166=cursive
>> 

Re: [DISCUSSION] Major changes in Ignite in 2020

[Anton Vinogradov]

Folks,

I keep working on crash recovery speed-up.

The main goal is to have put/get operations latency less than 500 ms on
node fail/left.
Currently, latency can be increased to seconds or even dozens of seconds.

The task is split to 2 threads

- Switch and tx recovery speed-up.
Speed-up can be gained by code refactoring, simplification, and tuning as
well as by special tricks like pme-free switch or cellular affinity usage.

- Failure detection.
Already found that some constants used at failure detection are hardcoded
and large.
Also, code responsible for this feature performs a lot of re-checks and
re-waits and you may have detection time close to failureDetectionTimeout
x2 or even x3.
Another problem is GC, and it may increase failure detection dramatically,
so, watchdog started from another JVM or from native code can help here.

On Tue, Apr 28, 2020 at 2:13 AM Denis Magda  wrote:

> Nikolay, thanks for responding,
>
> Would you prefer adding this item to the roadmap page after finishing the
> idea testing? If there are still many unknowns it can make to the roadmap
> after you share the results and propose a final solution.
>
> -
> Denis
>
>
> On Mon, Apr 27, 2020 at 2:06 PM Nikolay Izhikov 
> wrote:
>
> > Hello, Igniters.
> >
> > Right now I working on POC for some «framework» for integration testing
> of
> > Ignite.
> >
> > Feature I want to see in this framework:
> >
> > * To manage several Ignite instances inside docker or bare metal
> > servers.
> > * To manage Ignite cluster
> > * Start/stop arbitrary application to interact with Ignite.
> > * Kill nodes
> > * collect logs.
> > * implement complex real-life based scenarios
> > * etc.
> >
> > My POC are based on duck tape library [1]
> >
> > [1] https://github.com/confluentinc/ducktape
> >
> > > 25 апр. 2020 г., в 00:51, Denis Magda  написал(а):
> > >
> > > Dmitry, highlighted such a disclaimer in italic. Thanks.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Thu, Apr 23, 2020 at 3:53 AM Dmitriy Pavlov 
> > wrote:
> > >
> > >> Hi Denis,
> > >>
> > >> Thank you for driving this.
> > >>
> > >> Igniters,
> > >>
> > >> I would suggest to stress that Apache Ignite community does not
> > guarantee
> > >> these features to be available.
> > >>
> > >> Can we add some kind of disclaimer that says Ignite Roadmap does not
> > imply
> > >> any obligations regarding availability and timeline. A number of
> > >> contributions can be done on best efforts principle, it is always
> > tricky to
> > >> make a promise.
> > >>
> > >> Sincerely,
> > >> Dmitriy Pavlov
> > >>
> > >> чт, 23 апр. 2020 г. в 00:06, Denis Magda :
> > >>
> > >>> Igniters,
> > >>>
> > >>> Here is a draft of our very first roadmap. I decided to make it damp
> > >> simple
> > >>> but descriptive:
> > >>>
> > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Roadmap
> > >>>
> > >>> What we need to do next is to:
> > >>>
> > >>>   - Fill in the "Readiness Estimated Time" column with your best
> guess
> > >> of
> > >>>   when an improvement is to be ready for a release.
> > >>>   - Add references to JIRAs or IEPs to the first column.
> > >>>   - Add the names of those contributors who will be cooperating with
> > you
> > >>>   during development. Goes to the "Contributors" column.
> > >>>
> > >>> Once this step is complete, we'll see how many features converge
> around
> > >> the
> > >>> same date/months and we'll plan through 2.9, 2.10, etc. releases
> > >>> accordingly.
> > >>>
> > >>> Please don't put this aside for a while, let's move on quicker. If
> the
> > >>> roadmap misses any contributions that you are going to add, then edit
> > the
> > >>> wiki page.
> > >>>
> > >>> -
> > >>> Denis
> > >>>
> > >>>
> > >>> On Wed, Apr 15, 2020 at 3:35 AM Nikita Amelchev <
> nsamelc...@gmail.com>
> > >>> wrote:
> > >>>
> >  Hello, Igniters.
> > 
> >  I am going to contribute a new feature - profiling tool and
> >  performance report. This is part of IEP-35. [1]
> > 
> >  The tool will be able to collect performance statistics and create a
> >  human-readable report. It will help to analyze workload and to tune
> >  configuration and applications.
> > 
> >  Example of report [2, 3, 4].
> > 
> >  [1]
> > 
> > >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> >  [2]
> > 
> > >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png
> >  [3]
> > 
> > >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png
> >  [4]
> > 
> > >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png
> > 
> >  сб, 11 апр. 2020 г. в 13:54, Alexei Scherbakov <
> >