Re: New Monitoring System: items to complete before GA

[Denis Magda]

Nikolay, thanks for a detailed response.

(I've created a handful of tickets linking to IEP-35 with the label named
similarly. I didn't assign any tickets on you letting you decide what you'd
like to take over personally)

With filter user can exclude arbitrary registry from arbitrary exporter.
> Example of filtering out all cache metrics from JMX:


Gotcha, now it's clear. I would add this to the documentation if you don't
mind:
https://issues.apache.org/jira/browse/IGNITE-12943

> 2. Extension of IgniteSpiContext in order to allow Ignite developers to
> implement their own exporters
> Please, clarify, your idea.
> Exporters can be implemented without usage of the internal API.
> You can look into JmxMetricExporterSpi as an example.


@Andrey Gura , could you please join and clarify this
requirement?

> 4. After registering an exporter with a node only a subset of metrics
> gets updated. This is extremely confusing. Any way we can remove this
> ambiguity?
> Yes, it a bad thing.
> And it’s known restriction of the current implementation that was
> introduced to preserve backward compatibility.
> It seems we can remove this restriction after  IGNITE-11927 would be
> implemented.


Agree, created a ticket we can jump on after IGNITE-11927 is ready:
https://issues.apache.org/jira/browse/IGNITE-12944

In the meantime, I've added callout "Enabling Collection of Legacy Metrics"
to the first section of the documentation page:
https://apacheignite.readme.io/docs/new-metrics

> 5. The system view that should track compute tasks (views.tasks) is
> always empty
> TASKS reflects the running task that was started on the local node.
> You can find an example of using this view in tests [2]
> It seems we should add examples for the views.


I've updated the documentation putting an example. Please check if it
sounds correct:
https://apacheignite.readme.io/docs/tasks


> > 6. Ignite MBeans are listed under "org.apache.{node_id}" but should be
> placed under "org.apache.ignite.{node_id}".
> Please, create a ticket and I will try to fix it.


https://issues.apache.org/jira/browse/IGNITE-12945

> we can also add descriptions to MXBeans explaining what a metric is used
> for.
> Great idea! Let’s do it, for sure!
> Please, create a ticket for it


Cool! https://issues.apache.org/jira/browse/IGNITE-12946
-
Denis


On Sat, Apr 18, 2020 at 6:10 AM Nikolay Izhikov  wrote:

> > 1. Add ability to enable\disable subset of the metrics for collection:
> > https://issues.apache.org/jira/browse/IGNITE-11927
> > Am I right that the task will let us perform a desired configuration
> MetricExporterSPI and its specific implementation?
>
> This improvement should allow Ignite users to disable or enable
> `MetricsRegistry` globally on the node.
> When some `MetricRegistry` disabled then registry:
> * Do not consume resources(CPU, RAM) to keep metrics values up to
> date(don’t perform increment, decrement operations,  etc)
> * Can’t be viewed in any exporter.
>
> > Frankly, I couldn't figure out how to do that with
> "setExportFilter(Predicate filter);"
>
> With filter user can exclude arbitrary registry from arbitrary exporter.
> Example of filtering out all cache metrics from JMX:
>
> ```
> JmxMetricExporterSpi jmxSpi = new JmxMetricExporterSpi();
> jmxSpi.setExportFilter(mreg -> !mreg.name().startsWith(«cache»));
>
> cfg.setMetricExporterSpi(jmxSpi);
> ```
>
> > 2. Extension of IgniteSpiContext in order to allow Ignite developers to
> implement their own exporters
>
> Please, clarify, your idea.
> Exporters can be implemented without usage of the internal API.
> You can look into JmxMetricExporterSpi as an example.
>
> > 3. Adding/extending/removing public metrics related to the API facade.
>
> I think we can start with a discussion and finishing Java API to read
> metrics.
> It implemented in my PR [1].
>
> > 4. After registering an exporter with a node only a subset of metrics
> gets updated. This is extremely confusing. Any way we can remove this
> ambiguity?
>
> Yes, it a bad thing.
> And it’s known restriction of the current implementation that was
> introduced to preserve backward compatibility.
> It seems we can remove this restriction after  IGNITE-11927 would be
> implemented.
>
> > 5. The system view that should track compute tasks (views.tasks) is
> always empty
>
> TASKS reflects the running task that was started on the local node.
> You can find an example of using this view in tests [2]
> It seems we should add examples for the views.
>
> The view to observe compute JOBS(parts of the task that are executed on the
> remote node) was added by me in [3]
> It’s not a part of 2.8.0 release but should be available in 2.8.1
>
> > 6. Ignite MBeans are listed under "org.apache.{node_id}" but should be
> placed under "org.apache.ignite.{node_id}".
>
> Please, create a ticket and I will try to fix it.
>
> > we can also add descriptions to MXBeans explaining what a metric is used
> for.
>
> Great idea! Let’s do it, for sure!
> Please, 

[jira] [Created] (IGNITE-12946) Add description to MBeans

[Denis A. Magda (Jira)]

Denis A. Magda created IGNITE-12946:
---

 Summary: Add description to MBeans
 Key: IGNITE-12946
 URL: https://issues.apache.org/jira/browse/IGNITE-12946
 Project: Ignite
  Issue Type: Improvement
Affects Versions: 2.8
Reporter: Denis A. Magda


Presently, application developers will see hundreds of metrics once connect 
through JMX and it will be hard for them to sort out the valuable ones.

The documentation can do a great job outlining the right ones but we can also 
add descriptions to MXBeans explaining what each metric reports.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-12945) Put Ignite MBeans under a proper path

[Denis A. Magda (Jira)]

Denis A. Magda created IGNITE-12945:
---

 Summary: Put Ignite MBeans under a proper path
 Key: IGNITE-12945
 URL: https://issues.apache.org/jira/browse/IGNITE-12945
 Project: Ignite
  Issue Type: Task
Affects Versions: 2.8
Reporter: Denis A. Magda


Presently, Ignite MBeans are listed under {{org.apache.{node_id}} path but 
should be rather placed under {{org.apache.ignite.{node_id}}}.

Most likely we can do it only for Ignite 3.0 to preserve backward compatibility.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-12944) Remove additional parameters required for collection of some metrics

[Denis A. Magda (Jira)]

Denis A. Magda created IGNITE-12944:
---

 Summary: Remove additional parameters required for collection of 
some metrics
 Key: IGNITE-12944
 URL: https://issues.apache.org/jira/browse/IGNITE-12944
 Project: Ignite
  Issue Type: Task
Affects Versions: 2.8
Reporter: Denis A. Magda


For some of the metrics, it won't be enough to create an exporter to enable 
their collection. For instance, data region and cache APIs require to turn on 
additional parameters:
* 
https://apacheignite.readme.io/docs/memory-metrics#enabling-data-region-metrics
* https://apacheignite.readme.io/docs/cache-metrics#enabling-cache-metrics

We should remove such requirements after 
https://issues.apache.org/jira/browse/IGNITE-11927 is completed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSSION] `static final` code-style constant naming rules

[Ivan Pavlukhin]

Maxim,

Thank you for efforts in a code quality improvement!

Unfortunately I do not agree with the proposal. Usually I like to
refer a following guide [1]. While a question "what is a constant" is
not trivial but I suppose that every static final field capitalization
can cause some problems. For example, a developer can make false
assumptions about immutability and thread safety basing on a
capitalized field name.

[1] https://google.github.io/styleguide/javaguide.html#s5.2.4-constant-names

Best regards,
Ivan Pavlukhin

пт, 24 апр. 2020 г. в 21:58, Pavel Tupitsyn :
>
> > All static final object reference types that are never followed by "."
> (dot)
> With this way of thinking we can say that everything is a constant if we
> don't change it - no need for static final.
> "Constant" is usually something that you can't change, even you want -
> compiler won't let you.
> In Java that would be static final primitives and read-only objects like
> String.
>
> On Fri, Apr 24, 2020 at 8:25 PM Pavel Pereslegin  wrote:
>
> > Maxim,
> >
> > > But what exactly is `constant` means?! I'd suggest applying this rule
> > > to all class fields with `static final` modifiers without any clauses.
> > > (check Java Naming convention [2] paragraph 3.3).
> >
> > I disagree with this and want to clarify what exactly the Java naming
> > convention says:
> >
> > >> The following are considered to be constants:
> > >> 1. All static final primitive types (Remember that all interface fields
> > are inherently static final).
> > >> 2. All static final object reference types that are never followed by
> > "." (dot).
> > >> 3. All static final arrays that are never followed by"[" (dot)
> >
> > I don't see that convention says "any static final field".
> >
> > пт, 24 апр. 2020 г. в 20:11, Sergey Antonov :
> > >
> > > Maxim, It's a good idea!
> > >
> > > Please don't forget to update out code style guidelines too.
> > >
> > > Thank you for keeping the code cleaner!
> > >
> > > пт, 24 апр. 2020 г. в 19:49, Maxim Muzafarov :
> > >
> > > > Igniters,
> > > >
> > > >
> > > > It is not directly mentioned through the Apache Ignite Coding
> > > > Guidelines [1] about naming the `static final` class fields using only
> > > > upper-case letters. I'd like to suggest to fill this gap.
> > > >
> > > > > Constants should all be upper-case.
> > > > But what exactly is `constant` means?! I'd suggest applying this rule
> > > > to all class fields with `static final` modifiers without any clauses.
> > > > (check Java Naming convention [2] paragraph 3.3).
> > > >
> > > >
> > > > I've prepared PR [3] with capitalizing letters on all of the constant
> > > > names simultaneously with supporting the standard ConstantName
> > > > checkstyle [4] rule.
> > > >
> > > > Can we proceed with this change?
> > > > WDYT?
> > > >
> > > >
> > > > [1]
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines#CodingGuidelines-Naming
> > > > [2]
> > > >
> > https://web.archive.org/web/20120911192801/developers.sun.com/sunstudio/products/archive/whitepapers/java-style.pdf
> > > > [3] https://github.com/apache/ignite/pull/7662
> > > > [4] https://issues.apache.org/jira/browse/IGNITE-12888
> > > >
> > >
> > >
> > > --
> > > BR, Sergey Antonov
> >

[jira] [Created] (IGNITE-12943) Document how to filter out metrics from registries

[Denis A. Magda (Jira)]

Denis A. Magda created IGNITE-12943:
---

 Summary: Document how to filter out metrics from registries
 Key: IGNITE-12943
 URL: https://issues.apache.org/jira/browse/IGNITE-12943
 Project: Ignite
  Issue Type: Task
Reporter: Denis A. Magda
 Fix For: 2.8.1


As per `MetricExporterSpi.setExportFilter` contract, the user can filter out 
metrics for a specific exporter instance. For instance, this is how we can ask 
a JMX exporter instance to ignore the cache metrics:

{code}
JmxMetricExporterSpi jmxSpi = new JmxMetricExporterSpi();
jmxSpi.setExportFilter(mreg -> !mreg.name().startsWith(«cache»));

cfg.setMetricExporterSpi(jmxSpi);
{code}

We should add `Metrics Filtering` section to this documentation page [1] 
explaining how to use the filtering. Also, I would clarify in the 
`MetricExporterSpi.setExportFilter` JavaDocs that the method filters out 
certain metrics from a specific exporter.

Also, should we possibly rename the method to 
`MetricExporterSpi.setMetricsFilter` to make things crystal clear?

[1] https://apacheignite.readme.io/docs/new-metrics



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSSION] Major changes in Ignite in 2020

[Denis Magda]

Dmitry, highlighted such a disclaimer in italic. Thanks.

-
Denis


On Thu, Apr 23, 2020 at 3:53 AM Dmitriy Pavlov  wrote:

> Hi Denis,
>
> Thank you for driving this.
>
> Igniters,
>
> I would suggest to stress that Apache Ignite community does not guarantee
> these features to be available.
>
> Can we add some kind of disclaimer that says Ignite Roadmap does not imply
> any obligations regarding availability and timeline. A number of
> contributions can be done on best efforts principle, it is always tricky to
> make a promise.
>
> Sincerely,
> Dmitriy Pavlov
>
> чт, 23 апр. 2020 г. в 00:06, Denis Magda :
>
> > Igniters,
> >
> > Here is a draft of our very first roadmap. I decided to make it damp
> simple
> > but descriptive:
> > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Roadmap
> >
> > What we need to do next is to:
> >
> >- Fill in the "Readiness Estimated Time" column with your best guess
> of
> >when an improvement is to be ready for a release.
> >- Add references to JIRAs or IEPs to the first column.
> >- Add the names of those contributors who will be cooperating with you
> >during development. Goes to the "Contributors" column.
> >
> > Once this step is complete, we'll see how many features converge around
> the
> > same date/months and we'll plan through 2.9, 2.10, etc. releases
> > accordingly.
> >
> > Please don't put this aside for a while, let's move on quicker. If the
> > roadmap misses any contributions that you are going to add, then edit the
> > wiki page.
> >
> > -
> > Denis
> >
> >
> > On Wed, Apr 15, 2020 at 3:35 AM Nikita Amelchev 
> > wrote:
> >
> > > Hello, Igniters.
> > >
> > > I am going to contribute a new feature - profiling tool and
> > > performance report. This is part of IEP-35. [1]
> > >
> > > The tool will be able to collect performance statistics and create a
> > > human-readable report. It will help to analyze workload and to tune
> > > configuration and applications.
> > >
> > > Example of report [2, 3, 4].
> > >
> > > [1]
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> > > [2]
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png
> > > [3]
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png
> > > [4]
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png
> > >
> > > сб, 11 апр. 2020 г. в 13:54, Alexei Scherbakov <
> > > alexey.scherbak...@gmail.com>:
> > > >
> > > > Folks,
> > > >
> > > > I keep working on tasks related to data consistency.
> > > >
> > > > This includes:
> > > >
> > > > Lost partitions handling overhaul (almost done) and tombstones
> support
> > > > aforementioned by Ivan Rakov
> > > > Atomic protocol overhaul (see [1])
> > > >
> > > > The ultimate goal of the year is to prepare Ignite for passing Jepsen
> > > tests.
> > > >
> > > > [1]
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-12+Make+ATOMIC+Caches+Consistent+Again
> > > >
> > > >
> > > > пт, 10 апр. 2020 г. в 18:49, Denis Magda :
> > > >
> > > > > Steven,
> > > > >
> > > > > Please start a dedicated discussion for the Golang support. At the
> > > moment,
> > > > > I'm not aware if anybody from the community planned to provide
> > support
> > > > > out-of-the-box. However, that's not a tricky task thanks to
> Ignite's
> > > binary
> > > > > protocol that lets enable any programming language support easily.
> > > > >
> > > > > -
> > > > > Denis
> > > > >
> > > > >
> > > > > On Fri, Apr 10, 2020 at 8:43 AM smeadows-abb <
> > > steven.mead...@us.abb.com>
> > > > > wrote:
> > > > >
> > > > > > First thanks for your quick response.
> > > > > >
> > > > > > I looked at  https://github.com/amsokol/ignite-go-client and its
> > NOT
> > > > > > completed and nothing has been for last 16 months. Initial test
> > with
> > > > > > package
> > > > > > failed, so trying to determine your project roadmap with regards
> to
> > > > > Golang
> > > > > > and maybe Rust support.
> > > > > >
> > > > > > I'm NOT sure of 'AFAIK' ?
> > > > > >
> > > > > > We may need to implement your Restful API to provide support for
> > > Golang
> > > > > and
> > > > > > Rust, provided it's complete?
> > > > > >
> > > > > > Thanks,
> > > > > >   Steve
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Sent from:
> http://apache-ignite-developers.2346864.n4.nabble.com/
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Best regards,
> > > > Alexei Scherbakov
> > >
> > >
> > >
> > > --
> > > Best wishes,
> > > Amelchev Nikita
> > >
> >
>

[jira] [Created] (IGNITE-12942) control.sh add command for checking that inline size is same on all cluster nodes

[Sergey Antonov (Jira)]

Sergey Antonov created IGNITE-12942:
---

 Summary: control.sh add command for checking that inline size is 
same on all cluster nodes
 Key: IGNITE-12942
 URL: https://issues.apache.org/jira/browse/IGNITE-12942
 Project: Ignite
  Issue Type: New Feature
Reporter: Sergey Antonov
Assignee: Sergey Antonov
 Fix For: 2.9


We should check that the inline size is the same for the index on all nodes in 
the cluster.

1. Check inline_size of secondary indexes on node join. Warn to log if they 
differ and propose a recreate problem index.
2. Introduce a new command to control.sh (control.sh --cache 
check_index_inline_sizes). The command will check inline sizes of secondary 
indexes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-12941) .NET: Support .NET 5

[Pavel Tupitsyn (Jira)]

Pavel Tupitsyn created IGNITE-12941:
---

 Summary: .NET: Support .NET 5
 Key: IGNITE-12941
 URL: https://issues.apache.org/jira/browse/IGNITE-12941
 Project: Ignite
  Issue Type: New Feature
  Components: platforms
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 2.9


.NET 5 is in preview. Ignite.NET does not seem to work there. Tested on Ubuntu:
* Install .NET 5 SDK from Snap
* Create new console app, add Apache.Ignite nuget package
* Add Ignition.Start
* dotnet run

{code}
Unhandled exception. Apache.Ignite.Core.Common.IgniteException: Failed to load 
libjvm.so:
[option=/usr/bin/java, 
path=/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so, 
error=Unknown error]
[option=/usr/bin/java, 
path=/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so, 
error=/snap/core18/current/lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' 
not found (required by 
/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so)]
   at Apache.Ignite.Core.Impl.Unmanaged.Jni.JvmDll.Load(String 
configJvmDllPath, ILogger log)
   at Apache.Ignite.Core.Ignition.Start(IgniteConfiguration cfg)
   at Apache.Ignite.Core.Ignition.Start()
   at dotnet5.Program.Main(String[] args) in 
/home/pavel/w/tests/dotnet5/Program.cs:line 10

{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSSION] `static final` code-style constant naming rules

[Pavel Tupitsyn]

> All static final object reference types that are never followed by "."
(dot)
With this way of thinking we can say that everything is a constant if we
don't change it - no need for static final.
"Constant" is usually something that you can't change, even you want -
compiler won't let you.
In Java that would be static final primitives and read-only objects like
String.

On Fri, Apr 24, 2020 at 8:25 PM Pavel Pereslegin  wrote:

> Maxim,
>
> > But what exactly is `constant` means?! I'd suggest applying this rule
> > to all class fields with `static final` modifiers without any clauses.
> > (check Java Naming convention [2] paragraph 3.3).
>
> I disagree with this and want to clarify what exactly the Java naming
> convention says:
>
> >> The following are considered to be constants:
> >> 1. All static final primitive types (Remember that all interface fields
> are inherently static final).
> >> 2. All static final object reference types that are never followed by
> "." (dot).
> >> 3. All static final arrays that are never followed by"[" (dot)
>
> I don't see that convention says "any static final field".
>
> пт, 24 апр. 2020 г. в 20:11, Sergey Antonov :
> >
> > Maxim, It's a good idea!
> >
> > Please don't forget to update out code style guidelines too.
> >
> > Thank you for keeping the code cleaner!
> >
> > пт, 24 апр. 2020 г. в 19:49, Maxim Muzafarov :
> >
> > > Igniters,
> > >
> > >
> > > It is not directly mentioned through the Apache Ignite Coding
> > > Guidelines [1] about naming the `static final` class fields using only
> > > upper-case letters. I'd like to suggest to fill this gap.
> > >
> > > > Constants should all be upper-case.
> > > But what exactly is `constant` means?! I'd suggest applying this rule
> > > to all class fields with `static final` modifiers without any clauses.
> > > (check Java Naming convention [2] paragraph 3.3).
> > >
> > >
> > > I've prepared PR [3] with capitalizing letters on all of the constant
> > > names simultaneously with supporting the standard ConstantName
> > > checkstyle [4] rule.
> > >
> > > Can we proceed with this change?
> > > WDYT?
> > >
> > >
> > > [1]
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines#CodingGuidelines-Naming
> > > [2]
> > >
> https://web.archive.org/web/20120911192801/developers.sun.com/sunstudio/products/archive/whitepapers/java-style.pdf
> > > [3] https://github.com/apache/ignite/pull/7662
> > > [4] https://issues.apache.org/jira/browse/IGNITE-12888
> > >
> >
> >
> > --
> > BR, Sergey Antonov
>

[jira] [Created] (IGNITE-12940) .NET: NuGet tests fail with compilation error

[Pavel Tupitsyn (Jira)]

Pavel Tupitsyn created IGNITE-12940:
---

 Summary: .NET: NuGet tests fail with compilation error
 Key: IGNITE-12940
 URL: https://issues.apache.org/jira/browse/IGNITE-12940
 Project: Ignite
  Issue Type: New Feature
  Components: platforms
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-12939) Add WhitespaceAround support checkstyle rule

[Maxim Muzafarov (Jira)]

Maxim Muzafarov created IGNITE-12939:


 Summary: Add WhitespaceAround support checkstyle rule
 Key: IGNITE-12939
 URL: https://issues.apache.org/jira/browse/IGNITE-12939
 Project: Ignite
  Issue Type: Task
Reporter: Maxim Muzafarov
Assignee: Maxim Muzafarov
 Fix For: 2.9


White spaces must be checked according to java naming conventions.
Add the appropriate checkstyle rule.

{code}

 
 
 
 
 
 

 {code}
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Tool for performance statistics reports

[Denis Magda]

>
> Tracing is more deeply takes statistics. If it will be possible, I'm for
> reuse.


Looks like we need to sync up on these activities/initiatives to ensure we
don't do a duplicate job. If you think a separate discussion is necessary
let's kick it off.

-
Denis


On Fri, Apr 24, 2020 at 9:18 AM Nikita Amelchev 
wrote:

> Denis, Ilya,
>
> I will try to integrate profiling functionality into control.sh utility.
>
> > Speaking of the probes/statistics collection approach, is it supposed to
> > reuse tracing capabilities that are to be added as part of IEP-35?
> Tracing is more deeply takes statistics. If it will be possible, I'm for
> reuse.
>
> пт, 24 апр. 2020 г. в 18:59, Ilya Kasnacheev :
> >
> > Hello!
> >
> > I suggest that it's one of the places where it could be put instead of
> > adding a new tool.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > пт, 24 апр. 2020 г. в 18:56, Nikita Amelchev :
> >
> > > Ilya,
> > >
> > > You suggest using control.sh to build the report?
> > >
> > > пт, 24 апр. 2020 г. в 18:20, Ilya Kasnacheev <
> ilya.kasnach...@gmail.com>:
> > > >
> > > > Hello!
> > > >
> > > > You need to throw in control.sh also, which does some kind of
> statistics
> > > > too, such as idle_verify.
> > > >
> > > > Regards,
> > > > --
> > > > Ilya Kasnacheev
> > > >
> > > >
> > > > пт, 24 апр. 2020 г. в 18:06, Вячеслав Коптилин <
> slava.kopti...@gmail.com
> > > >:
> > > >
> > > > > Hello Nikita,
> > > > >
> > > > > Perhaps, I am missing something...
> > > > > Apache Ignite already has a web-console tool. Do we want to
> improve the
> > > > > existing tool instead of creating a new one?
> > > > > It seems to me, this can be confusing for users.
> > > > >  - visor (which is deprecated)
> > > > >  - web-console (to be honest, I don't quite understand the status
> of
> > > this
> > > > > tool)
> > > > >  - new ignite-profiling (which is a monitoring tool as well,
> judging
> > > by the
> > > > > provided link [1] )
> > > > >
> > > > > I am not against the new tool, I just want to understand the
> > > motivation to
> > > > > not improve the existing sub-projects.
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> > > > >
> > > > > Thanks,
> > > > > S.
> > > > >
> > > > > пт, 24 апр. 2020 г. в 14:58, Nikita Amelchev  >:
> > > > >
> > > > > > Hi, Igniters.
> > > > > >
> > > > > > I'm working on cluster profiling and the tool for creating a
> > > > > > performance report. [1] I have prepared PoC based on performance
> > > > > > logging to a separate category of Ignite log. The report
> contains:
> > > > > >
> > > > > > - Cache operations and its distribution by types [2]
> > > > > > - Transactions and histogram of durations [3]
> > > > > > - SQL and Scan query statistics, top of slowest queries, logical
> and
> > > > > > physical reads by query [4]
> > > > > > - Compute statistics, top of slowest tasks and their jobs [5]
> > > > > > Soon I will add:
> > > > > > - Topology and Ignite versions info
> > > > > > - Client ID in case of operations from clients
> > > > > >
> > > > > > For now, I'm developing a binary logging format to reduce the
> effect
> > > > > > on performance. I'll try to reuse Ignite mechanisms.
> > > > > >
> > > > > > I would like to hear suggestions for the profiling and the
> > > performance
> > > > > > report.
> > > > > >
> > > > > > [1]
> > > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> > > > > > [2]
> > > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png
> > > > > > [3]
> > > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png
> > > > > > [4]
> > > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png
> > > > > > [5]
> > > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/152112279/p5.png
> > > > > >
> > > > > > --
> > > > > > Best wishes,
> > > > > > Amelchev Nikita
> > > > > >
> > > > >
> > >
> > >
> > >
> > > --
> > > Best wishes,
> > > Amelchev Nikita
> > >
>
>
>
> --
> Best wishes,
> Amelchev Nikita
>

[jira] [Created] (IGNITE-12938) controls.sh utility commands: IdleVerify and ValidateIndexes use eventual payload check.

[Stanilovsky Evgeny (Jira)]

Stanilovsky Evgeny created IGNITE-12938:
---

 Summary: controls.sh utility commands: IdleVerify and 
ValidateIndexes use eventual payload check.
 Key: IGNITE-12938
 URL: https://issues.apache.org/jira/browse/IGNITE-12938
 Project: Ignite
  Issue Type: Improvement
  Components: general
Affects Versions: 2.8
Reporter: Stanilovsky Evgeny
Assignee: Stanilovsky Evgeny
 Fix For: 2.9


"Checkpoint with dirty pages started! Cluster not idle" - not idle detection 
command, but mechanism for check payload during commands execution. 
Additionally current functional miss check on caches without persistence.  
Remove old functionality from PageMemory and move it into update counters usage.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSSION] `static final` code-style constant naming rules

[Pavel Pereslegin]

Maxim,

> But what exactly is `constant` means?! I'd suggest applying this rule
> to all class fields with `static final` modifiers without any clauses.
> (check Java Naming convention [2] paragraph 3.3).

I disagree with this and want to clarify what exactly the Java naming
convention says:

>> The following are considered to be constants:
>> 1. All static final primitive types (Remember that all interface fields are 
>> inherently static final).
>> 2. All static final object reference types that are never followed by "." 
>> (dot).
>> 3. All static final arrays that are never followed by"[" (dot)

I don't see that convention says "any static final field".

пт, 24 апр. 2020 г. в 20:11, Sergey Antonov :
>
> Maxim, It's a good idea!
>
> Please don't forget to update out code style guidelines too.
>
> Thank you for keeping the code cleaner!
>
> пт, 24 апр. 2020 г. в 19:49, Maxim Muzafarov :
>
> > Igniters,
> >
> >
> > It is not directly mentioned through the Apache Ignite Coding
> > Guidelines [1] about naming the `static final` class fields using only
> > upper-case letters. I'd like to suggest to fill this gap.
> >
> > > Constants should all be upper-case.
> > But what exactly is `constant` means?! I'd suggest applying this rule
> > to all class fields with `static final` modifiers without any clauses.
> > (check Java Naming convention [2] paragraph 3.3).
> >
> >
> > I've prepared PR [3] with capitalizing letters on all of the constant
> > names simultaneously with supporting the standard ConstantName
> > checkstyle [4] rule.
> >
> > Can we proceed with this change?
> > WDYT?
> >
> >
> > [1]
> > https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines#CodingGuidelines-Naming
> > [2]
> > https://web.archive.org/web/20120911192801/developers.sun.com/sunstudio/products/archive/whitepapers/java-style.pdf
> > [3] https://github.com/apache/ignite/pull/7662
> > [4] https://issues.apache.org/jira/browse/IGNITE-12888
> >
>
>
> --
> BR, Sergey Antonov

Re: [DISCUSSION] `static final` code-style constant naming rules

[Sergey Antonov]

Maxim, It's a good idea!

Please don't forget to update out code style guidelines too.

Thank you for keeping the code cleaner!

пт, 24 апр. 2020 г. в 19:49, Maxim Muzafarov :

> Igniters,
>
>
> It is not directly mentioned through the Apache Ignite Coding
> Guidelines [1] about naming the `static final` class fields using only
> upper-case letters. I'd like to suggest to fill this gap.
>
> > Constants should all be upper-case.
> But what exactly is `constant` means?! I'd suggest applying this rule
> to all class fields with `static final` modifiers without any clauses.
> (check Java Naming convention [2] paragraph 3.3).
>
>
> I've prepared PR [3] with capitalizing letters on all of the constant
> names simultaneously with supporting the standard ConstantName
> checkstyle [4] rule.
>
> Can we proceed with this change?
> WDYT?
>
>
> [1]
> https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines#CodingGuidelines-Naming
> [2]
> https://web.archive.org/web/20120911192801/developers.sun.com/sunstudio/products/archive/whitepapers/java-style.pdf
> [3] https://github.com/apache/ignite/pull/7662
> [4] https://issues.apache.org/jira/browse/IGNITE-12888
>


-- 
BR, Sergey Antonov

[DISCUSSION] `static final` code-style constant naming rules

[Maxim Muzafarov]

Igniters,


It is not directly mentioned through the Apache Ignite Coding
Guidelines [1] about naming the `static final` class fields using only
upper-case letters. I'd like to suggest to fill this gap.

> Constants should all be upper-case.
But what exactly is `constant` means?! I'd suggest applying this rule
to all class fields with `static final` modifiers without any clauses.
(check Java Naming convention [2] paragraph 3.3).


I've prepared PR [3] with capitalizing letters on all of the constant
names simultaneously with supporting the standard ConstantName
checkstyle [4] rule.

Can we proceed with this change?
WDYT?


[1] 
https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines#CodingGuidelines-Naming
[2] 
https://web.archive.org/web/20120911192801/developers.sun.com/sunstudio/products/archive/whitepapers/java-style.pdf
[3] https://github.com/apache/ignite/pull/7662
[4] https://issues.apache.org/jira/browse/IGNITE-12888

[jira] [Created] (IGNITE-12937) Create pull-request template for the github repo

[Maxim Muzafarov (Jira)]

Maxim Muzafarov created IGNITE-12937:


 Summary: Create pull-request template for the github repo
 Key: IGNITE-12937
 URL: https://issues.apache.org/jira/browse/IGNITE-12937
 Project: Ignite
  Issue Type: Task
Reporter: Maxim Muzafarov


Create {{.github/pull_request_template.md}} to allow users to have a 
pull-request all usefull information while contributing to the Apache Ignite.

Example:

[ ] Coding Guidelines are followed
[ ] TeamCity build passes
[ ] JIRA ticked is in Patch Available state, a review has been requested in
comments



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Tool for performance statistics reports

[Nikita Amelchev]

Denis, Ilya,

I will try to integrate profiling functionality into control.sh utility.

> Speaking of the probes/statistics collection approach, is it supposed to
> reuse tracing capabilities that are to be added as part of IEP-35?
Tracing is more deeply takes statistics. If it will be possible, I'm for reuse.

пт, 24 апр. 2020 г. в 18:59, Ilya Kasnacheev :
>
> Hello!
>
> I suggest that it's one of the places where it could be put instead of
> adding a new tool.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 24 апр. 2020 г. в 18:56, Nikita Amelchev :
>
> > Ilya,
> >
> > You suggest using control.sh to build the report?
> >
> > пт, 24 апр. 2020 г. в 18:20, Ilya Kasnacheev :
> > >
> > > Hello!
> > >
> > > You need to throw in control.sh also, which does some kind of statistics
> > > too, such as idle_verify.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > пт, 24 апр. 2020 г. в 18:06, Вячеслав Коптилин  > >:
> > >
> > > > Hello Nikita,
> > > >
> > > > Perhaps, I am missing something...
> > > > Apache Ignite already has a web-console tool. Do we want to improve the
> > > > existing tool instead of creating a new one?
> > > > It seems to me, this can be confusing for users.
> > > >  - visor (which is deprecated)
> > > >  - web-console (to be honest, I don't quite understand the status of
> > this
> > > > tool)
> > > >  - new ignite-profiling (which is a monitoring tool as well, judging
> > by the
> > > > provided link [1] )
> > > >
> > > > I am not against the new tool, I just want to understand the
> > motivation to
> > > > not improve the existing sub-projects.
> > > >
> > > > [1]
> > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> > > >
> > > > Thanks,
> > > > S.
> > > >
> > > > пт, 24 апр. 2020 г. в 14:58, Nikita Amelchev :
> > > >
> > > > > Hi, Igniters.
> > > > >
> > > > > I'm working on cluster profiling and the tool for creating a
> > > > > performance report. [1] I have prepared PoC based on performance
> > > > > logging to a separate category of Ignite log. The report contains:
> > > > >
> > > > > - Cache operations and its distribution by types [2]
> > > > > - Transactions and histogram of durations [3]
> > > > > - SQL and Scan query statistics, top of slowest queries, logical and
> > > > > physical reads by query [4]
> > > > > - Compute statistics, top of slowest tasks and their jobs [5]
> > > > > Soon I will add:
> > > > > - Topology and Ignite versions info
> > > > > - Client ID in case of operations from clients
> > > > >
> > > > > For now, I'm developing a binary logging format to reduce the effect
> > > > > on performance. I'll try to reuse Ignite mechanisms.
> > > > >
> > > > > I would like to hear suggestions for the profiling and the
> > performance
> > > > > report.
> > > > >
> > > > > [1]
> > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> > > > > [2]
> > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png
> > > > > [3]
> > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png
> > > > > [4]
> > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png
> > > > > [5]
> > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/152112279/p5.png
> > > > >
> > > > > --
> > > > > Best wishes,
> > > > > Amelchev Nikita
> > > > >
> > > >
> >
> >
> >
> > --
> > Best wishes,
> > Amelchev Nikita
> >



-- 
Best wishes,
Amelchev Nikita

Re: Tool for performance statistics reports

[Ilya Kasnacheev]

Hello!

I suggest that it's one of the places where it could be put instead of
adding a new tool.

Regards,
-- 
Ilya Kasnacheev


пт, 24 апр. 2020 г. в 18:56, Nikita Amelchev :

> Ilya,
>
> You suggest using control.sh to build the report?
>
> пт, 24 апр. 2020 г. в 18:20, Ilya Kasnacheev :
> >
> > Hello!
> >
> > You need to throw in control.sh also, which does some kind of statistics
> > too, such as idle_verify.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > пт, 24 апр. 2020 г. в 18:06, Вячеслав Коптилин  >:
> >
> > > Hello Nikita,
> > >
> > > Perhaps, I am missing something...
> > > Apache Ignite already has a web-console tool. Do we want to improve the
> > > existing tool instead of creating a new one?
> > > It seems to me, this can be confusing for users.
> > >  - visor (which is deprecated)
> > >  - web-console (to be honest, I don't quite understand the status of
> this
> > > tool)
> > >  - new ignite-profiling (which is a monitoring tool as well, judging
> by the
> > > provided link [1] )
> > >
> > > I am not against the new tool, I just want to understand the
> motivation to
> > > not improve the existing sub-projects.
> > >
> > > [1]
> > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> > >
> > > Thanks,
> > > S.
> > >
> > > пт, 24 апр. 2020 г. в 14:58, Nikita Amelchev :
> > >
> > > > Hi, Igniters.
> > > >
> > > > I'm working on cluster profiling and the tool for creating a
> > > > performance report. [1] I have prepared PoC based on performance
> > > > logging to a separate category of Ignite log. The report contains:
> > > >
> > > > - Cache operations and its distribution by types [2]
> > > > - Transactions and histogram of durations [3]
> > > > - SQL and Scan query statistics, top of slowest queries, logical and
> > > > physical reads by query [4]
> > > > - Compute statistics, top of slowest tasks and their jobs [5]
> > > > Soon I will add:
> > > > - Topology and Ignite versions info
> > > > - Client ID in case of operations from clients
> > > >
> > > > For now, I'm developing a binary logging format to reduce the effect
> > > > on performance. I'll try to reuse Ignite mechanisms.
> > > >
> > > > I would like to hear suggestions for the profiling and the
> performance
> > > > report.
> > > >
> > > > [1]
> > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> > > > [2]
> > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png
> > > > [3]
> > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png
> > > > [4]
> > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png
> > > > [5]
> > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/152112279/p5.png
> > > >
> > > > --
> > > > Best wishes,
> > > > Amelchev Nikita
> > > >
> > >
>
>
>
> --
> Best wishes,
> Amelchev Nikita
>

Re: Tool for performance statistics reports

[Denis Magda]

Nikita,

Thanks for preparing the IEP and breaking the initiative down into pieces.
I studied the IEP and it feels like the goal is to built a tool similar to
Oracle AWR. If it's so then shouldn't we add this as an extra function to
either visorcmd or control.sh. Hopefully, we can start moving in the
direction of the tooling unification.

Speaking of the probes/statistics collection approach, is it supposed to
reuse tracing capabilities that are to be added as part of IEP-35? [1].
According to the draft of our roadmap [2], @Ivan Rakov  is
planning to contribute to the feature this year.

[1]
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=112820392
[2] https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Roadmap

-
Denis


On Fri, Apr 24, 2020 at 8:48 AM Nikita Amelchev 
wrote:

> Slava,
>
> The web-console is used to interactive configuration wizard,
> management and monitoring tool [1].
>
> I want to implement a profiling tool. It's one of the monitoring parts
> and helps to answer the question: What will be running slow? See
> IEP-35 [2].
>
> [1] https://apacheignite-tools.readme.io/docs/ignite-web-console
> [2]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=112820392
>
> пт, 24 апр. 2020 г. в 18:46, Nikita Amelchev :
> >
> > Maxim,
> >
> > > Can you provide more information about the binary logging format?
> > I started to investigate. It should be with minimal effect on
> > performance. I haven't any details for now.
> >
> > > What the additional log category are you talking about?
> > PoC is based on logging performance statistics to the "profiling"
> > category to split with other logging and be able to log to a separate
> > file. See IgniteLogger#getLogger.
> >
> > пт, 24 апр. 2020 г. в 18:20, Ilya Kasnacheev  >:
> > >
> > > Hello!
> > >
> > > You need to throw in control.sh also, which does some kind of
> statistics
> > > too, such as idle_verify.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > пт, 24 апр. 2020 г. в 18:06, Вячеслав Коптилин <
> slava.kopti...@gmail.com>:
> > >
> > > > Hello Nikita,
> > > >
> > > > Perhaps, I am missing something...
> > > > Apache Ignite already has a web-console tool. Do we want to improve
> the
> > > > existing tool instead of creating a new one?
> > > > It seems to me, this can be confusing for users.
> > > >  - visor (which is deprecated)
> > > >  - web-console (to be honest, I don't quite understand the status of
> this
> > > > tool)
> > > >  - new ignite-profiling (which is a monitoring tool as well, judging
> by the
> > > > provided link [1] )
> > > >
> > > > I am not against the new tool, I just want to understand the
> motivation to
> > > > not improve the existing sub-projects.
> > > >
> > > > [1]
> > > >
> > > >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> > > >
> > > > Thanks,
> > > > S.
> > > >
> > > > пт, 24 апр. 2020 г. в 14:58, Nikita Amelchev :
> > > >
> > > > > Hi, Igniters.
> > > > >
> > > > > I'm working on cluster profiling and the tool for creating a
> > > > > performance report. [1] I have prepared PoC based on performance
> > > > > logging to a separate category of Ignite log. The report contains:
> > > > >
> > > > > - Cache operations and its distribution by types [2]
> > > > > - Transactions and histogram of durations [3]
> > > > > - SQL and Scan query statistics, top of slowest queries, logical
> and
> > > > > physical reads by query [4]
> > > > > - Compute statistics, top of slowest tasks and their jobs [5]
> > > > > Soon I will add:
> > > > > - Topology and Ignite versions info
> > > > > - Client ID in case of operations from clients
> > > > >
> > > > > For now, I'm developing a binary logging format to reduce the
> effect
> > > > > on performance. I'll try to reuse Ignite mechanisms.
> > > > >
> > > > > I would like to hear suggestions for the profiling and the
> performance
> > > > > report.
> > > > >
> > > > > [1]
> > > > >
> > > >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> > > > > [2]
> > > > >
> > > >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png
> > > > > [3]
> > > > >
> > > >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png
> > > > > [4]
> > > > >
> > > >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png
> > > > > [5]
> > > > >
> > > >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/152112279/p5.png
> > > > >
> > > > > --
> > > > > Best wishes,
> > > > > Amelchev Nikita
> > > > >
> > > >
> >
> >
> >
> > --
> > Best wishes,
> > Amelchev Nikita
>
>
>
> --
> Best wishes,
> Amelchev Nikita
>

Re: Tool for performance statistics reports

[Nikita Amelchev]

Ilya,

You suggest using control.sh to build the report?

пт, 24 апр. 2020 г. в 18:20, Ilya Kasnacheev :
>
> Hello!
>
> You need to throw in control.sh also, which does some kind of statistics
> too, such as idle_verify.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 24 апр. 2020 г. в 18:06, Вячеслав Коптилин :
>
> > Hello Nikita,
> >
> > Perhaps, I am missing something...
> > Apache Ignite already has a web-console tool. Do we want to improve the
> > existing tool instead of creating a new one?
> > It seems to me, this can be confusing for users.
> >  - visor (which is deprecated)
> >  - web-console (to be honest, I don't quite understand the status of this
> > tool)
> >  - new ignite-profiling (which is a monitoring tool as well, judging by the
> > provided link [1] )
> >
> > I am not against the new tool, I just want to understand the motivation to
> > not improve the existing sub-projects.
> >
> > [1]
> >
> > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> >
> > Thanks,
> > S.
> >
> > пт, 24 апр. 2020 г. в 14:58, Nikita Amelchev :
> >
> > > Hi, Igniters.
> > >
> > > I'm working on cluster profiling and the tool for creating a
> > > performance report. [1] I have prepared PoC based on performance
> > > logging to a separate category of Ignite log. The report contains:
> > >
> > > - Cache operations and its distribution by types [2]
> > > - Transactions and histogram of durations [3]
> > > - SQL and Scan query statistics, top of slowest queries, logical and
> > > physical reads by query [4]
> > > - Compute statistics, top of slowest tasks and their jobs [5]
> > > Soon I will add:
> > > - Topology and Ignite versions info
> > > - Client ID in case of operations from clients
> > >
> > > For now, I'm developing a binary logging format to reduce the effect
> > > on performance. I'll try to reuse Ignite mechanisms.
> > >
> > > I would like to hear suggestions for the profiling and the performance
> > > report.
> > >
> > > [1]
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> > > [2]
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png
> > > [3]
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png
> > > [4]
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png
> > > [5]
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/152112279/p5.png
> > >
> > > --
> > > Best wishes,
> > > Amelchev Nikita
> > >
> >



-- 
Best wishes,
Amelchev Nikita

Re: Tool for performance statistics reports

[Nikita Amelchev]

Slava,

The web-console is used to interactive configuration wizard,
management and monitoring tool [1].

I want to implement a profiling tool. It's one of the monitoring parts
and helps to answer the question: What will be running slow? See
IEP-35 [2].

[1] https://apacheignite-tools.readme.io/docs/ignite-web-console
[2] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=112820392

пт, 24 апр. 2020 г. в 18:46, Nikita Amelchev :
>
> Maxim,
>
> > Can you provide more information about the binary logging format?
> I started to investigate. It should be with minimal effect on
> performance. I haven't any details for now.
>
> > What the additional log category are you talking about?
> PoC is based on logging performance statistics to the "profiling"
> category to split with other logging and be able to log to a separate
> file. See IgniteLogger#getLogger.
>
> пт, 24 апр. 2020 г. в 18:20, Ilya Kasnacheev :
> >
> > Hello!
> >
> > You need to throw in control.sh also, which does some kind of statistics
> > too, such as idle_verify.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > пт, 24 апр. 2020 г. в 18:06, Вячеслав Коптилин :
> >
> > > Hello Nikita,
> > >
> > > Perhaps, I am missing something...
> > > Apache Ignite already has a web-console tool. Do we want to improve the
> > > existing tool instead of creating a new one?
> > > It seems to me, this can be confusing for users.
> > >  - visor (which is deprecated)
> > >  - web-console (to be honest, I don't quite understand the status of this
> > > tool)
> > >  - new ignite-profiling (which is a monitoring tool as well, judging by 
> > > the
> > > provided link [1] )
> > >
> > > I am not against the new tool, I just want to understand the motivation to
> > > not improve the existing sub-projects.
> > >
> > > [1]
> > >
> > > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> > >
> > > Thanks,
> > > S.
> > >
> > > пт, 24 апр. 2020 г. в 14:58, Nikita Amelchev :
> > >
> > > > Hi, Igniters.
> > > >
> > > > I'm working on cluster profiling and the tool for creating a
> > > > performance report. [1] I have prepared PoC based on performance
> > > > logging to a separate category of Ignite log. The report contains:
> > > >
> > > > - Cache operations and its distribution by types [2]
> > > > - Transactions and histogram of durations [3]
> > > > - SQL and Scan query statistics, top of slowest queries, logical and
> > > > physical reads by query [4]
> > > > - Compute statistics, top of slowest tasks and their jobs [5]
> > > > Soon I will add:
> > > > - Topology and Ignite versions info
> > > > - Client ID in case of operations from clients
> > > >
> > > > For now, I'm developing a binary logging format to reduce the effect
> > > > on performance. I'll try to reuse Ignite mechanisms.
> > > >
> > > > I would like to hear suggestions for the profiling and the performance
> > > > report.
> > > >
> > > > [1]
> > > >
> > > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> > > > [2]
> > > >
> > > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png
> > > > [3]
> > > >
> > > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png
> > > > [4]
> > > >
> > > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png
> > > > [5]
> > > >
> > > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/152112279/p5.png
> > > >
> > > > --
> > > > Best wishes,
> > > > Amelchev Nikita
> > > >
> > >
>
>
>
> --
> Best wishes,
> Amelchev Nikita



-- 
Best wishes,
Amelchev Nikita

Re: Tool for performance statistics reports

[Nikita Amelchev]

Maxim,

> Can you provide more information about the binary logging format?
I started to investigate. It should be with minimal effect on
performance. I haven't any details for now.

> What the additional log category are you talking about?
PoC is based on logging performance statistics to the "profiling"
category to split with other logging and be able to log to a separate
file. See IgniteLogger#getLogger.

пт, 24 апр. 2020 г. в 18:20, Ilya Kasnacheev :
>
> Hello!
>
> You need to throw in control.sh also, which does some kind of statistics
> too, such as idle_verify.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 24 апр. 2020 г. в 18:06, Вячеслав Коптилин :
>
> > Hello Nikita,
> >
> > Perhaps, I am missing something...
> > Apache Ignite already has a web-console tool. Do we want to improve the
> > existing tool instead of creating a new one?
> > It seems to me, this can be confusing for users.
> >  - visor (which is deprecated)
> >  - web-console (to be honest, I don't quite understand the status of this
> > tool)
> >  - new ignite-profiling (which is a monitoring tool as well, judging by the
> > provided link [1] )
> >
> > I am not against the new tool, I just want to understand the motivation to
> > not improve the existing sub-projects.
> >
> > [1]
> >
> > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> >
> > Thanks,
> > S.
> >
> > пт, 24 апр. 2020 г. в 14:58, Nikita Amelchev :
> >
> > > Hi, Igniters.
> > >
> > > I'm working on cluster profiling and the tool for creating a
> > > performance report. [1] I have prepared PoC based on performance
> > > logging to a separate category of Ignite log. The report contains:
> > >
> > > - Cache operations and its distribution by types [2]
> > > - Transactions and histogram of durations [3]
> > > - SQL and Scan query statistics, top of slowest queries, logical and
> > > physical reads by query [4]
> > > - Compute statistics, top of slowest tasks and their jobs [5]
> > > Soon I will add:
> > > - Topology and Ignite versions info
> > > - Client ID in case of operations from clients
> > >
> > > For now, I'm developing a binary logging format to reduce the effect
> > > on performance. I'll try to reuse Ignite mechanisms.
> > >
> > > I would like to hear suggestions for the profiling and the performance
> > > report.
> > >
> > > [1]
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> > > [2]
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png
> > > [3]
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png
> > > [4]
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png
> > > [5]
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/152112279/p5.png
> > >
> > > --
> > > Best wishes,
> > > Amelchev Nikita
> > >
> >



-- 
Best wishes,
Amelchev Nikita

Re: Tool for performance statistics reports

[Ilya Kasnacheev]

Hello!

You need to throw in control.sh also, which does some kind of statistics
too, such as idle_verify.

Regards,
-- 
Ilya Kasnacheev


пт, 24 апр. 2020 г. в 18:06, Вячеслав Коптилин :

> Hello Nikita,
>
> Perhaps, I am missing something...
> Apache Ignite already has a web-console tool. Do we want to improve the
> existing tool instead of creating a new one?
> It seems to me, this can be confusing for users.
>  - visor (which is deprecated)
>  - web-console (to be honest, I don't quite understand the status of this
> tool)
>  - new ignite-profiling (which is a monitoring tool as well, judging by the
> provided link [1] )
>
> I am not against the new tool, I just want to understand the motivation to
> not improve the existing sub-projects.
>
> [1]
>
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
>
> Thanks,
> S.
>
> пт, 24 апр. 2020 г. в 14:58, Nikita Amelchev :
>
> > Hi, Igniters.
> >
> > I'm working on cluster profiling and the tool for creating a
> > performance report. [1] I have prepared PoC based on performance
> > logging to a separate category of Ignite log. The report contains:
> >
> > - Cache operations and its distribution by types [2]
> > - Transactions and histogram of durations [3]
> > - SQL and Scan query statistics, top of slowest queries, logical and
> > physical reads by query [4]
> > - Compute statistics, top of slowest tasks and their jobs [5]
> > Soon I will add:
> > - Topology and Ignite versions info
> > - Client ID in case of operations from clients
> >
> > For now, I'm developing a binary logging format to reduce the effect
> > on performance. I'll try to reuse Ignite mechanisms.
> >
> > I would like to hear suggestions for the profiling and the performance
> > report.
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> > [2]
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png
> > [3]
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png
> > [4]
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png
> > [5]
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/152112279/p5.png
> >
> > --
> > Best wishes,
> > Amelchev Nikita
> >
>

Re: Tool for performance statistics reports

[Вячеслав Коптилин]

Hello Nikita,

Perhaps, I am missing something...
Apache Ignite already has a web-console tool. Do we want to improve the
existing tool instead of creating a new one?
It seems to me, this can be confusing for users.
 - visor (which is deprecated)
 - web-console (to be honest, I don't quite understand the status of this
tool)
 - new ignite-profiling (which is a monitoring tool as well, judging by the
provided link [1] )

I am not against the new tool, I just want to understand the motivation to
not improve the existing sub-projects.

[1]
https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool

Thanks,
S.

пт, 24 апр. 2020 г. в 14:58, Nikita Amelchev :

> Hi, Igniters.
>
> I'm working on cluster profiling and the tool for creating a
> performance report. [1] I have prepared PoC based on performance
> logging to a separate category of Ignite log. The report contains:
>
> - Cache operations and its distribution by types [2]
> - Transactions and histogram of durations [3]
> - SQL and Scan query statistics, top of slowest queries, logical and
> physical reads by query [4]
> - Compute statistics, top of slowest tasks and their jobs [5]
> Soon I will add:
> - Topology and Ignite versions info
> - Client ID in case of operations from clients
>
> For now, I'm developing a binary logging format to reduce the effect
> on performance. I'll try to reuse Ignite mechanisms.
>
> I would like to hear suggestions for the profiling and the performance
> report.
>
> [1]
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> [2]
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png
> [3]
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png
> [4]
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png
> [5]
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/152112279/p5.png
>
> --
> Best wishes,
> Amelchev Nikita
>

Re: Tool for performance statistics reports

[Maxim Muzafarov]

Hello, Nikita

Nice to have such a tool in Apache Ignite.

Can you provide more information about the binary logging format?
Do you have any examples?
Is there are any problems you faced?

>The Ignite will log some additional internal statistics using separate log 
>category of IgniteLogger.
What the additional log category are you talking about?

On Fri, 24 Apr 2020 at 14:58, Nikita Amelchev  wrote:
>
> Hi, Igniters.
>
> I'm working on cluster profiling and the tool for creating a
> performance report. [1] I have prepared PoC based on performance
> logging to a separate category of Ignite log. The report contains:
>
> - Cache operations and its distribution by types [2]
> - Transactions and histogram of durations [3]
> - SQL and Scan query statistics, top of slowest queries, logical and
> physical reads by query [4]
> - Compute statistics, top of slowest tasks and their jobs [5]
> Soon I will add:
> - Topology and Ignite versions info
> - Client ID in case of operations from clients
>
> For now, I'm developing a binary logging format to reduce the effect
> on performance. I'll try to reuse Ignite mechanisms.
>
> I would like to hear suggestions for the profiling and the performance report.
>
> [1] 
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> [2] 
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png
> [3] 
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png
> [4] 
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png
> [5] 
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/152112279/p5.png
>
> --
> Best wishes,
> Amelchev Nikita

[jira] [Created] (IGNITE-12936) .NET inspections fail on master

[Igor Sapego (Jira)]

Igor Sapego created IGNITE-12936:


 Summary: .NET inspections fail on master
 Key: IGNITE-12936
 URL: https://issues.apache.org/jira/browse/IGNITE-12936
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Reporter: Igor Sapego
Assignee: Igor Sapego


The following suites are failing:
[Platform .NET (NuGet) 
|https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_PlatformNetNuGet?branch=%3Cdefault%3E]
[Platform .NET (Inspections) 
|https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_PlatformNetInspections?branch=%3Cdefault%3E]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSSION] Hot cache backup

[Maxim Muzafarov]

Alexey,


I've addressed all your comments, please, take a look at the PR [1].
Additional tests were added.
Additional comments with further steps were added.


[1] https://github.com/apache/ignite/pull/7607
[2] https://issues.apache.org/jira/browse/IGNITE-11073

On Tue, 21 Apr 2020 at 09:53, Alexey Goncharuk
 wrote:
>
> Maxim,
>
> I've left my comments in the PR.
>
> пн, 20 апр. 2020 г. в 12:52, Maxim Muzafarov :
>
> > Alex P,
> > Thank you for the great sophisticated review.
> >
> >
> > Alexey G,
> > Will you take a look at my changes[1]?
> > The fresh TC.Bot visa attached.
> >
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-11073
> >
> > On Mon, 20 Apr 2020 at 11:54, Alex Plehanov 
> > wrote:
> > >
> > > Maxim, I've reviewed your PR and it looks good to me. Good job!
> > >
> > > пт, 10 апр. 2020 г. в 19:43, Alexey Goncharuk <
> > alexey.goncha...@gmail.com>:
> > >
> > > > Maxim,
> > > >
> > > > Thanks for raising this PR. I will do a review during next week.
> > > >
> > > > --AG
> > > >
> >

Tool for performance statistics reports

[Nikita Amelchev]

Hi, Igniters.

I'm working on cluster profiling and the tool for creating a
performance report. [1] I have prepared PoC based on performance
logging to a separate category of Ignite log. The report contains:

- Cache operations and its distribution by types [2]
- Transactions and histogram of durations [3]
- SQL and Scan query statistics, top of slowest queries, logical and
physical reads by query [4]
- Compute statistics, top of slowest tasks and their jobs [5]
Soon I will add:
- Topology and Ignite versions info
- Client ID in case of operations from clients

For now, I'm developing a binary logging format to reduce the effect
on performance. I'll try to reuse Ignite mechanisms.

I would like to hear suggestions for the profiling and the performance report.

[1] 
https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
[2] 
https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png
[3] 
https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png
[4] 
https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png
[5] 
https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/152112279/p5.png

-- 
Best wishes,
Amelchev Nikita

Re: [MTCGA]: new failures in builds [5251854, 5251851] needs to be handled

[Pavel Tupitsyn]

This is correct. I've talked to Igor earlier today and he is working on it.

On Fri, Apr 24, 2020 at 1:40 PM Dmitriy Pavlov  wrote:

> My best guess it is related to  IGNITE-12853
>  : Introduced features
> for the thin client protocol
>
> Build details
>
> https://ci.ignite.apache.org/viewLog.html?buildId=5254068=Inspection=IgniteTests24Java8_PlatformNetInspections
>
>
> says that srvVer is missed in the configuration. And this field is
> mentioned in the PR for IGNITE-12853
> 
>
> пт, 24 апр. 2020 г. в 06:08, :
>
> > Hi Igniters,
> >
> >  I've detected some new issue on TeamCity to be handled. You are more
> than
> > welcomed to help.
> >
> >  If your changes can lead to this failure(s): We're grateful that you
> were
> > a volunteer to make the contribution to this project, but things change
> and
> > you may no longer be able to finalize your contribution.
> >  Could you respond to this email and indicate if you wish to continue and
> > fix test failures or step down and some committer may revert you commit.
> >
> >  *New Critical Failure in master Platform .NET (NuGet)*
> >
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_PlatformNetNuGet?branch=%3Cdefault%3E
> >  Changes may lead to failure were done by
> >  - igor sapego 
> > https://ci.ignite.apache.org/viewModification.html?modId=900989
> >  - pavel pereslegin 
> > https://ci.ignite.apache.org/viewModification.html?modId=900984
> >  - nikita amelchev 
> > https://ci.ignite.apache.org/viewModification.html?modId=900986
> >
> >  *New Critical Failure in master Platform .NET (Inspections)*
> >
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_PlatformNetInspections?branch=%3Cdefault%3E
> >  Changes may lead to failure were done by
> >  - igor sapego 
> > https://ci.ignite.apache.org/viewModification.html?modId=900989
> >  - pavel pereslegin 
> > https://ci.ignite.apache.org/viewModification.html?modId=900984
> >  - nikita amelchev 
> > https://ci.ignite.apache.org/viewModification.html?modId=900986
> >
> >  - Here's a reminder of what contributors were agreed to do
> > https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
> >  - Should you have any questions please contact
> > dev@ignite.apache.org
> >
> > Best Regards,
> > Apache Ignite TeamCity Bot
> > https://github.com/apache/ignite-teamcity-bot
> > Notification generated at 06:07:28 24-04-2020
> >
>

Re: [MTCGA]: new failures in builds [5251854, 5251851] needs to be handled

[Dmitriy Pavlov]

My best guess it is related to  IGNITE-12853
 : Introduced features
for the thin client protocol

Build details
https://ci.ignite.apache.org/viewLog.html?buildId=5254068=Inspection=IgniteTests24Java8_PlatformNetInspections


says that srvVer is missed in the configuration. And this field is
mentioned in the PR for IGNITE-12853


пт, 24 апр. 2020 г. в 06:08, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *New Critical Failure in master Platform .NET (NuGet)*
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_PlatformNetNuGet?branch=%3Cdefault%3E
>  Changes may lead to failure were done by
>  - igor sapego 
> https://ci.ignite.apache.org/viewModification.html?modId=900989
>  - pavel pereslegin 
> https://ci.ignite.apache.org/viewModification.html?modId=900984
>  - nikita amelchev 
> https://ci.ignite.apache.org/viewModification.html?modId=900986
>
>  *New Critical Failure in master Platform .NET (Inspections)*
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_PlatformNetInspections?branch=%3Cdefault%3E
>  Changes may lead to failure were done by
>  - igor sapego 
> https://ci.ignite.apache.org/viewModification.html?modId=900989
>  - pavel pereslegin 
> https://ci.ignite.apache.org/viewModification.html?modId=900984
>  - nikita amelchev 
> https://ci.ignite.apache.org/viewModification.html?modId=900986
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 06:07:28 24-04-2020
>

[jira] [Created] (IGNITE-12935) Disadvantages in log of historical rebalance

[Vladislav Pyatkov (Jira)]

Vladislav Pyatkov created IGNITE-12935:
--

 Summary: Disadvantages in log of historical rebalance
 Key: IGNITE-12935
 URL: https://issues.apache.org/jira/browse/IGNITE-12935
 Project: Ignite
  Issue Type: Improvement
Reporter: Vladislav Pyatkov


# Mention in the log only partitions for which there are no nodes that suit as 
historical supplier
 For these partitions, print minimal counter (since which we should perform 
historical rebalancing) with corresponding node and maximum reserved counter 
(since which cluster can perform historical rebalancing) with corresponding 
node.
This will let us know:
 ## Whether history was reserved at all
 ## How much reserved history we lack to perform a historical rebalancing
 ## I see resulting output like this:
Historical rebalancing wasn't scheduled for some partitions:
 History wasn't reserved for: [list of partitions and groups]
 History was reserved, but minimum present counter is less than maximum 
reserved: [[grp=GRP, part=ID, minCntr=cntr, minNodeId=ID, maxReserved=cntr, 
maxReservedNodeId=ID], ...]
 ## We can also aggregate previous message by (minNodeId) to easily find the 
exact node (or nodes) which were the reason of full rebalance.
 # Log results of reserveHistoryForExchange(). They can be compactly 
represented as mappings: (grpId -> checkpoint (id, timestamp)). For every 
group, also log message about why the previous checkpoint wasn't successfully 
reserved.
There can be three reasons:
 ## Previous checkpoint simply isn't present in the history (the oldest is 
reserved)
 ## WAL reservation failure (call below returned false)

{code:java}
chpEntry = entry(cpTs);boolean reserved =  
cctx.wal().reserve(chpEntry.checkpointMark());// If checkpoint WAL history 
can't be reserved, stop searching.
if (!reserved)
  break;
{code}

 ## Checkpoint was marked as inapplicable for historical rebalancing

{code:java}
for (Integer grpId : new HashSet<>(groupsAndPartitions.keySet()))
  if (!isCheckpointApplicableForGroup(grpId, chpEntry))
groupsAndPartitions.remove(grpId);
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)