[jira] [Created] (IGNITE-12755) Not enough information about error

[Surkov Aleksandr (Jira)]

Surkov Aleksandr created IGNITE-12755:
-

 Summary: Not enough information about error
 Key: IGNITE-12755
 URL: https://issues.apache.org/jira/browse/IGNITE-12755
 Project: Ignite
  Issue Type: Improvement
Reporter: Surkov Aleksandr


One of the nodes fell out, but not enough information in the log file 
console.log.
{code:java}
JVM will be halted immediately due to the failure: [failureCtx=FailureContext 
[type=SYSTEM_WORKER_TERMINATION, err=java.lang.NullPointerException]] 
vmop    [threads: total initially_running wait_to_block]
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Apache Ignite ML & Python

[Alexey Zinoviev]

Ken, thanks for the feedback, part of the ideas looks like the good
candidates for the next release for the Java API.

We should understand, that Python API could only wrap Java API.

We approach with wrapping via P4j library as in mentioned repository could
be used, it is a common approach, it is used in PySparj, for example.

Currently I m not ready to make Python wrapper a part of Ignite for many
reasons: part of ML API is released firstly, this is a big work for many
Committer, we couldnt guarantee the release cycle for such component.


пт, 6 мар. 2020 г., 2:49 Denis Magda :

> Folks,
>
> Does it make sense to take an approach of Python ML implementation
> available for GridGain in a beta mode? (where Python APIs wrap around Java
> ML library)
>
> https://www.gridgain.com/docs/latest/developers-guide/python-ml/using-python-ml
>
> -
> Denis
>
>
> On Thu, Mar 5, 2020 at 6:50 AM Alexey Zinoviev 
> wrote:
>
> > Agree with simple case, I think we could start from the simple poc for
> the
> > Python for ML in the next release
> >
> > чт, 5 мар. 2020 г., 17:05 AG :
> >
> > >
> > > Thanks, for the reply!
> > >
> > > It looks like a high-level API similar to Sklearn pipelines.
> > > In my opinion, for the first steps easier to add simple assess to gain
> > the
> > > ability to run a simple model or simple preprocessor from python.
> > >
> > > According to your example:
> > > Here is raw dataset, already inside this cluster cache "myName", with
> > > Label column "MyLable".
> > >
> > > I want to run from notebook UI imputer and knn using python API. Export
> > > results to file storage as an example.
> > >
> > > In my opinion, the ability to create such a simple workflow should be
> our
> > > goal for the first time.
> > >
> > > Thank You!
> > >
> > > Best regards,
> > > Andrei Gavrilov.
> > >
> > > Sent with ProtonMail Secure Email.
> > >
> > > ‐‐‐ Original Message ‐‐‐
> > > On Wednesday, March 4, 2020 10:49 PM, kencottrell <
> > > ken.cottr...@gridgain.com> wrote:
> > >
> > > > Andrei,
> > > >
> > > > I am also working with Apache Ignite ML and am interested in
> providing
> > > > wrappers for Ignite ML API, but am wondering if instead of simply
> > > recreating
> > > > the low level Java API for ML inside Python, how about creating some
> > > higher
> > > > level services "Auto ML" workflow ? For example:
> > > >
> > > > 1.  here is raw dataset, already inside this cluster cache "myName",
> > with
> > > > Label column "MyLable" , take N samples tell me which appear to
> be
> > > numeric,
> > > > unique id, and categorical values?
> > > >
> > > > 2.  based on N samples, , please run some analysis and tell me the
> top
> > 5
> > > > feature columns in terms of predictive value using algorithm =
> > > RandonForest
> > > >
> > > > 3.  do a batch run, sample size = N, using these preprocessing steps
> > list
> > > > {impute, scale, etc} and algorithms (knn, Decision Tree, etc} and
> > > give me a
> > > > report of accuracies obtain with each.
> > > >
> > > > In other words, we have a simple sample in the Tutorial demo
> where
> > > these
> > > > all run and then we compare outputs - why not automate these
> with a
> > > Python
> > > > Notebook UI of some sort?
> > > >
> > > > --
> > > > Sent from:
> http://apache-ignite-developers.2346864.n4.nabble.com/
> > > >
> > >
> > >
> > >
> >
>

Re: Ignite Website: New Look

[Nikita Ivanov]

Denis - looks very nice! I do indeed think we need to work on better
content (home page specifically).
Thanks!
--
Nikita Ivanov



On Thu, Mar 5, 2020 at 3:18 PM Denis Magda  wrote:

> Folks,
>
> As many of you know, these days I mostly contribute by optimizing our
> website, preparing different content including documentation/articles, and
> presenting the project at various events. One of the continuous website
> activities we undertake together with Mauricio and Ignacio is search engine
> optimization (SEO). It helps our website to be ranked higher by search
> engines for user searches falling in categories of in-memory caches, grids,
> databases, etc. (check this simple guide [1] if you'd like to learn the
> internals of SEO).
>
> After checking the results of our recent user questionnaire [2] and
> confirming key capabilities with use cases Ignite is selected for, we
> decided to put more effort into the SEO. And, in addition to the keywords
> optimizations, we invested some time into the structural and UI changes of
> the website trying to make the experience better and, as a result, be
> ranked higher.
>
> So, today we'd like to share our first results and check your thoughts. In
> particular, pay attention to:
>
>- The new UI - instead of a dark and bloody theme, we decided to
>experiment with a more lightweight and contemporary design.
>- A new main page structure with the following blocks - banner, use
>cases, features, quicks links + tweeter feed.
>- Updated navigation menu - kept essential or highly-ranked pages.
>
>
> Do NOT pay attention to the following yet (we're ready to carry on with the
> items below once hear your feedback):
>
>- Text on the front page will be massaged and tweaked going forward to
>get better SEO. Presently, something might sound off to you.
>- Secondary pages are broken from the UI and structure standpoint
>- Some of the pages of the navigation menu, such as Videos, are not
>created yet.
>
>
> Alright, we deployed our branch to this test server. Go and check:
> http://157.245.190.104 (user: ignite, pass: apache2020)
>
>
>
> [1] https://moz.com/beginners-guide-to-seo
> [2]
>
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Evolution-Direction-short-questionary-td44577.html
>
> -
> Denis
>

Re: Out of memory with eviction failure on persisted cache

[Raymond Wilson]

The reproducer is my development system, which is hard to share.

I have increased the size of the buffer to 256Mb, and it copes with the
example data load, though I have not tried larger data sets.

>From an analytical perspective, is this an error that is possible or
expected to occur when using a cache with a persistent data region defined?

I'll see if I can make a small reproducer.

On Fri, Mar 6, 2020 at 11:34 AM Evgenii Zhuravlev 
wrote:

> Hi Raymond,
>
> I tried to reproduce it, but without success. Can you share the reproducer?
>
> Also, have you tried to load much more data with 256mb data region? I
> think it should work without issues.
>
> Thanks,
> Evgenii
>
> ср, 4 мар. 2020 г. в 16:14, Raymond Wilson :
>
>> Hi Evgenii,
>>
>> I am individually Put()ing the elements using PutIfAbsent(). Each element
>> can range 2kb-35Kb in size.
>>
>> Actually, the process that writes the data does not write the data
>> directly to the cache, it uses a compute function to send the payload to
>> the process that is doing the reading. The compute function applies
>> validation logic and uses PutIfAbsent() to write the data into the cache.
>>
>> Sorry for the confusion.
>>
>> Raymond.
>>
>>
>> On Thu, Mar 5, 2020 at 1:09 PM Evgenii Zhuravlev <
>> e.zhuravlev...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> How are you loading the data? Do you use putAll or DataStreamer?
>>>
>>> Evgenii
>>>
>>> ср, 4 мар. 2020 г. в 15:37, Raymond Wilson :
>>>
 To add some further detail:

 There are two processes interacting with the cache. One process is
 writing
 data into the cache, while the second process is extracting data from
 the
 cache using a continuous query. The process that is the reader of the
 data
 is throwing the exception.

 Increasing the cache size further to 256 Mb resolves the problem for
 this
 data set, however we have data sets more than 100 times this size which
 we
 will be processing.

 Thanks,
 Raymond.


 On Thu, Mar 5, 2020 at 12:10 PM Raymond Wilson <
 raymond_wil...@trimble.com>
 wrote:

 > I've been having a sporadic issue with the Ignite 2.7.5 JVM halting
 due to
 > out of memory error related to a cache with persistence enabled
 >
 > I just upgraded to the C#.Net, Ignite 2.7.6 client to pick up support
 for
 > C# affinity functions and now have this issue appearing regularly
 while
 > adding around 400Mb of data into the cache which is configured to have
 > 128Mb of memory (this was 64Mb but I increased it to see if the
 failure
 > would resolve.
 >
 > The error I get is:
 >
 > 2020-03-05 11:58:57,568 [542] ERR [MutableCacheComputeServer] JVM
 will be
 > halted immediately due to the failure: [failureCtx=FailureContext
 > [type=CRITICAL_ERROR, err=class
 o.a.i.i.mem.IgniteOutOfMemoryException:
 > Failed to find a page for eviction [segmentCapacity=1700, loaded=676,
 > maxDirtyPages=507, dirtyPages=675, cpPages=0, pinnedInSegment=2,
 > failedToPrepare=675]
 > Out of memory in data region [name=TAGFileBufferQueue, initSize=128.0
 MiB,
 > maxSize=128.0 MiB, persistenceEnabled=true] Try the following:
 >   ^-- Increase maximum off-heap memory size
 > (DataRegionConfiguration.maxSize)
 >   ^-- Enable Ignite persistence
 > (DataRegionConfiguration.persistenceEnabled)
 >   ^-- Enable eviction or expiration policies]]
 >
 > I'm not running an eviction policy as I thought this was not required
 for
 > caches with persistence enabled.
 >
 > I'm surprised by this behaviour as I expected the persistence
 mechanism to
 > handle it. The error relating to failure to find a page for eviction
 > suggest the persistence mechanism has fallen behind. If this is the
 case,
 > this seems like an unfriendly failure mode.
 >
 > Thanks,
 > Raymond.
 >
 >
 >

>>>

Re: Apache Ignite ML & Python

[Denis Magda]

Folks,

Does it make sense to take an approach of Python ML implementation
available for GridGain in a beta mode? (where Python APIs wrap around Java
ML library)
https://www.gridgain.com/docs/latest/developers-guide/python-ml/using-python-ml

-
Denis


On Thu, Mar 5, 2020 at 6:50 AM Alexey Zinoviev 
wrote:

> Agree with simple case, I think we could start from the simple poc for the
> Python for ML in the next release
>
> чт, 5 мар. 2020 г., 17:05 AG :
>
> >
> > Thanks, for the reply!
> >
> > It looks like a high-level API similar to Sklearn pipelines.
> > In my opinion, for the first steps easier to add simple assess to gain
> the
> > ability to run a simple model or simple preprocessor from python.
> >
> > According to your example:
> > Here is raw dataset, already inside this cluster cache "myName", with
> > Label column "MyLable".
> >
> > I want to run from notebook UI imputer and knn using python API. Export
> > results to file storage as an example.
> >
> > In my opinion, the ability to create such a simple workflow should be our
> > goal for the first time.
> >
> > Thank You!
> >
> > Best regards,
> > Andrei Gavrilov.
> >
> > Sent with ProtonMail Secure Email.
> >
> > ‐‐‐ Original Message ‐‐‐
> > On Wednesday, March 4, 2020 10:49 PM, kencottrell <
> > ken.cottr...@gridgain.com> wrote:
> >
> > > Andrei,
> > >
> > > I am also working with Apache Ignite ML and am interested in providing
> > > wrappers for Ignite ML API, but am wondering if instead of simply
> > recreating
> > > the low level Java API for ML inside Python, how about creating some
> > higher
> > > level services "Auto ML" workflow ? For example:
> > >
> > > 1.  here is raw dataset, already inside this cluster cache "myName",
> with
> > > Label column "MyLable" , take N samples tell me which appear to be
> > numeric,
> > > unique id, and categorical values?
> > >
> > > 2.  based on N samples, , please run some analysis and tell me the top
> 5
> > > feature columns in terms of predictive value using algorithm =
> > RandonForest
> > >
> > > 3.  do a batch run, sample size = N, using these preprocessing steps
> list
> > > {impute, scale, etc} and algorithms (knn, Decision Tree, etc} and
> > give me a
> > > report of accuracies obtain with each.
> > >
> > > In other words, we have a simple sample in the Tutorial demo where
> > these
> > > all run and then we compare outputs - why not automate these with a
> > Python
> > > Notebook UI of some sort?
> > >
> > > --
> > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> > >
> >
> >
> >
>

Re: Ignite 2.8 announcement plan

[Denis Magda]

Pavel, thanks,

I enjoyed reading the blog, crystal clear and straight to the point! Please
consider these several items that might strengthen the article a bit:

   - I would use "thick client" as a term instead of the "classic client"
   (and mention that Ignite.NET client is a thick one). The thick client is
   already a coined term that based on my observations are used a lot by dev
   and user communities. Also, you might add some differences of thick vs.
   thin taking from this page -
   
https://www.gridgain.com/docs/latest/installation-guide/deployment-modes#thick-vs-thin-clients
   - Should we mention that presently partition-awareness doesn't handle
   topology changes automatically (partition map won't be updated on the
   client-side)? This might be a blocker for some users.
   - Excited to read about the cross-platform support, that's huge! Now I
   can run Ignite.NET easily on my Mac OS machine. I would insert a reference
   to updated documentation pages that explain how to start with Ignite.NET on
   various platforms.

Hope, you will find this helpful, thanks for helping with project promotion!

-
Denis


On Thu, Mar 5, 2020 at 7:32 AM Pavel Tupitsyn  wrote:

> Denis,
>
> The first post is going to be ready soon, probably by tomorrow.
> Here is a draft, feedback welcome:
>
> https://github.com/ptupitsyn/ptupitsyn.github.io/blob/ignite-2.8/_posts/2020-03-05-Whats-New-In-Ignite-Net-2.8.md
>
> On Wed, Mar 4, 2020 at 5:15 PM Denis Magda  wrote:
>
> > Hi Pavel,
> >
> > Excellent! It will be good to publish the first article (what's new in
> > Ignite.NET 2.8) prior to a generic blog on blogs.apache.org so that we
> can
> > link your post in for those who are looking for more details. Do you have
> > any timeline in mind for this article?
> >
> > @Alexey Zinoviev , how about you preparing
> several
> > paragraphs for the blog highlighting the biggest changes in ML? The same
> > highlighted content will be elaborated during the webinar.
> >
> > -
> > Denis
> >
> >
> > On Wed, Mar 4, 2020 at 2:29 AM Pavel Tupitsyn 
> > wrote:
> >
> > > Denis, I have a few blog posts on Ignite.NET planned:
> > >
> > > * What's new in Ignite.NET 2.8 (Thin Client Partition Awareness,
> logging,
> > > expiry policy, cluster API, .NET Core 3.x, Dockerfile)
> > > * Ignite.NET performance improvements on .NET Core 3.x
> > > * Ignite.NET Partition Awareness performance
> > > * Fixing JNI thread leak in Ignite.NET
> > >
> > > What do you think? Should we have a schedule of some sorts?
> > >
> > > On Tue, Mar 3, 2020 at 9:58 PM Alexey Zinoviev  >
> > > wrote:
> > >
> > > > Sounds ineresting, will help with the post, please share the
> template,
> > > not
> > > > sure about webinar
> > > >
> > > > вт, 3 мар. 2020 г., 20:59 Denis Magda :
> > > >
> > > > > Igniters,
> > > > >
> > > > > Let's discuss approaches for a global announcement/promotion of the
> > > > > release. I would suggest focusing on a blog post and a community
> > > webinar.
> > > > >
> > > > > The blog post will introduce significant improvements (service
> grid,
> > > thin
> > > > > clients, new metrics system, ML, etc.) sharing references to
> > > > documentation
> > > > > pages with more details.  It will be published on blogs.apache.org
> > in
> > > a
> > > > > format similar to this one -
> > > > > https://blogs.apache.org/ignite/entry/apache-ignite-2-7-deep. I
> can
> > > work
> > > > > on
> > > > > it unless anybody else is willing to share the news on behalf of
> the
> > > > > community.
> > > > >
> > > > > Next, the blog post will be featuring a community webinar that is
> > > > breaking
> > > > > down a subset of the improvements in more detail. Please see an
> > > abstract
> > > > > below with suggested topics for a detailed overview. @Alexey
> Zinoviev
> > > > > , would you be able to present the ML
> part?
> > > > > @Nikolay
> > > > > Izhikov  or @Andrey Gura 
> > > would
> > > > > you like to take over the metrics section? I'll work the attendees
> > > > through
> > > > > the items listed in "Sustainable production under high load". We
> > should
> > > > > target the webinar for the April timeframe.
> > > > >
> > > > >
> > > > >
> > > > > *Topmost changes in Apache Ignite 2.8 for production maintenance
> and
> > > > > machine learning*
> > > > >
> > > > >
> > > > > *Apache Ignite community rolled out more than 1900 changes in
> Ignite
> > > 2.8
> > > > > that enhanced almost all the components of the platform. The
> release
> > > > notes
> > > > > go with hundreds of lines trying to catalog the improvements. Join
> > this
> > > > > webinar led by Ignite community members demonstrating and
> dissecting
> > > new
> > > > > capabilities related to production maintenance, monitoring, and
> > machine
> > > > > learning that you do not want to lose sight of:*
> > > > >
> > > > >- *Sustainable production under high load: Ignite persistence
> > > > compaction
> > > > >and consistent crash recovery, baseline topology
> auto-adjustment,
> > no
> > 

Ignite Website: New Look

[Denis Magda]

Folks,

As many of you know, these days I mostly contribute by optimizing our
website, preparing different content including documentation/articles, and
presenting the project at various events. One of the continuous website
activities we undertake together with Mauricio and Ignacio is search engine
optimization (SEO). It helps our website to be ranked higher by search
engines for user searches falling in categories of in-memory caches, grids,
databases, etc. (check this simple guide [1] if you'd like to learn the
internals of SEO).

After checking the results of our recent user questionnaire [2] and
confirming key capabilities with use cases Ignite is selected for, we
decided to put more effort into the SEO. And, in addition to the keywords
optimizations, we invested some time into the structural and UI changes of
the website trying to make the experience better and, as a result, be
ranked higher.

So, today we'd like to share our first results and check your thoughts. In
particular, pay attention to:

   - The new UI - instead of a dark and bloody theme, we decided to
   experiment with a more lightweight and contemporary design.
   - A new main page structure with the following blocks - banner, use
   cases, features, quicks links + tweeter feed.
   - Updated navigation menu - kept essential or highly-ranked pages.


Do NOT pay attention to the following yet (we're ready to carry on with the
items below once hear your feedback):

   - Text on the front page will be massaged and tweaked going forward to
   get better SEO. Presently, something might sound off to you.
   - Secondary pages are broken from the UI and structure standpoint
   - Some of the pages of the navigation menu, such as Videos, are not
   created yet.


Alright, we deployed our branch to this test server. Go and check:
http://157.245.190.104 (user: ignite, pass: apache2020)



[1] https://moz.com/beginners-guide-to-seo
[2]
http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Evolution-Direction-short-questionary-td44577.html

-
Denis

Re: Apache Ignite ML & Python

[Ken Cottrell]

Alexey, Andrei,

Here are some thoughts on what would be good to have in Python-Ignite ML
notebook:

   - some way to pick an optional sample size (out of a very big cache
   size) that gets communicated and set aside to all partitions
   - some way to count number of unique values for a category (for example
   should we do a one-hot or String encoding) - this might need to be entire
   dataset if you want to do one-hot
   - some way to do a quick assessment and simple listing (similar to
   SKlearn pretty bar chart) about contributions for each feature to the label.
   - some way to allow vector to choose its own indexes based on:
   predictive weight, data type (for example automatically encodes category)
   - some way to report simple cluster metrics in  Python notebook - but
   focus on ML stuff like raw cache, sample cache, vector info, preprocessing
   / training stats etc
   - some way to input lists of things to do on a data set in parallel
   (list of algorithms for example) and then let Ignite ML run them all and
   report comparisons back
   - some way to explain the steps that were run by ML in the background
   and a report on all the steps

We might even just create some sort of demo Python wrapper that sits in
front of the  org.apache.ignite.examples.ml.tutorial code, but pass in a
different cache handle instead of Titanic and also run all of the Java
classes (DT, impute, categorial encoding, scaling, etc etc) in parallel
instead of serially.




*Ken Cottrell*

*mobile: +1 (214) 546-5100*
*ken.cottr...@gridgain.com *

*https://www.linkedin.com/in/kennethcottrell
*



On Thu, Mar 5, 2020 at 8:53 AM Alexey Zinoviev 
wrote:

> Agree with simple case, I think we could start from the simple poc for the
> Python for ML in the next release
>
> чт, 5 мар. 2020 г., 17:05 AG :
>
> >
> > Thanks, for the reply!
> >
> > It looks like a high-level API similar to Sklearn pipelines.
> > In my opinion, for the first steps easier to add simple assess to gain
> the
> > ability to run a simple model or simple preprocessor from python.
> >
> > According to your example:
> > Here is raw dataset, already inside this cluster cache "myName", with
> > Label column "MyLable".
> >
> > I want to run from notebook UI imputer and knn using python API. Export
> > results to file storage as an example.
> >
> > In my opinion, the ability to create such a simple workflow should be our
> > goal for the first time.
> >
> > Thank You!
> >
> > Best regards,
> > Andrei Gavrilov.
> >
> > Sent with ProtonMail Secure Email.
> >
> > ‐‐‐ Original Message ‐‐‐
> > On Wednesday, March 4, 2020 10:49 PM, kencottrell <
> > ken.cottr...@gridgain.com> wrote:
> >
> > > Andrei,
> > >
> > > I am also working with Apache Ignite ML and am interested in providing
> > > wrappers for Ignite ML API, but am wondering if instead of simply
> > recreating
> > > the low level Java API for ML inside Python, how about creating some
> > higher
> > > level services "Auto ML" workflow ? For example:
> > >
> > > 1.  here is raw dataset, already inside this cluster cache "myName",
> with
> > > Label column "MyLable" , take N samples tell me which appear to be
> > numeric,
> > > unique id, and categorical values?
> > >
> > > 2.  based on N samples, , please run some analysis and tell me the top
> 5
> > > feature columns in terms of predictive value using algorithm =
> > RandonForest
> > >
> > > 3.  do a batch run, sample size = N, using these preprocessing steps
> list
> > > {impute, scale, etc} and algorithms (knn, Decision Tree, etc} and
> > give me a
> > > report of accuracies obtain with each.
> > >
> > > In other words, we have a simple sample in the Tutorial demo where
> > these
> > > all run and then we compare outputs - why not automate these with a
> > Python
> > > Notebook UI of some sort?
> > >
> > > --
> > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> > >
> >
> >
> >
>

Re: Out of memory with eviction failure on persisted cache

[Evgenii Zhuravlev]

Hi Raymond,

I tried to reproduce it, but without success. Can you share the reproducer?

Also, have you tried to load much more data with 256mb data region? I think
it should work without issues.

Thanks,
Evgenii

ср, 4 мар. 2020 г. в 16:14, Raymond Wilson :

> Hi Evgenii,
>
> I am individually Put()ing the elements using PutIfAbsent(). Each element
> can range 2kb-35Kb in size.
>
> Actually, the process that writes the data does not write the data
> directly to the cache, it uses a compute function to send the payload to
> the process that is doing the reading. The compute function applies
> validation logic and uses PutIfAbsent() to write the data into the cache.
>
> Sorry for the confusion.
>
> Raymond.
>
>
> On Thu, Mar 5, 2020 at 1:09 PM Evgenii Zhuravlev 
> wrote:
>
>> Hi,
>>
>> How are you loading the data? Do you use putAll or DataStreamer?
>>
>> Evgenii
>>
>> ср, 4 мар. 2020 г. в 15:37, Raymond Wilson :
>>
>>> To add some further detail:
>>>
>>> There are two processes interacting with the cache. One process is
>>> writing
>>> data into the cache, while the second process is extracting data from the
>>> cache using a continuous query. The process that is the reader of the
>>> data
>>> is throwing the exception.
>>>
>>> Increasing the cache size further to 256 Mb resolves the problem for this
>>> data set, however we have data sets more than 100 times this size which
>>> we
>>> will be processing.
>>>
>>> Thanks,
>>> Raymond.
>>>
>>>
>>> On Thu, Mar 5, 2020 at 12:10 PM Raymond Wilson <
>>> raymond_wil...@trimble.com>
>>> wrote:
>>>
>>> > I've been having a sporadic issue with the Ignite 2.7.5 JVM halting
>>> due to
>>> > out of memory error related to a cache with persistence enabled
>>> >
>>> > I just upgraded to the C#.Net, Ignite 2.7.6 client to pick up support
>>> for
>>> > C# affinity functions and now have this issue appearing regularly while
>>> > adding around 400Mb of data into the cache which is configured to have
>>> > 128Mb of memory (this was 64Mb but I increased it to see if the failure
>>> > would resolve.
>>> >
>>> > The error I get is:
>>> >
>>> > 2020-03-05 11:58:57,568 [542] ERR [MutableCacheComputeServer] JVM will
>>> be
>>> > halted immediately due to the failure: [failureCtx=FailureContext
>>> > [type=CRITICAL_ERROR, err=class o.a.i.i.mem.IgniteOutOfMemoryException:
>>> > Failed to find a page for eviction [segmentCapacity=1700, loaded=676,
>>> > maxDirtyPages=507, dirtyPages=675, cpPages=0, pinnedInSegment=2,
>>> > failedToPrepare=675]
>>> > Out of memory in data region [name=TAGFileBufferQueue, initSize=128.0
>>> MiB,
>>> > maxSize=128.0 MiB, persistenceEnabled=true] Try the following:
>>> >   ^-- Increase maximum off-heap memory size
>>> > (DataRegionConfiguration.maxSize)
>>> >   ^-- Enable Ignite persistence
>>> > (DataRegionConfiguration.persistenceEnabled)
>>> >   ^-- Enable eviction or expiration policies]]
>>> >
>>> > I'm not running an eviction policy as I thought this was not required
>>> for
>>> > caches with persistence enabled.
>>> >
>>> > I'm surprised by this behaviour as I expected the persistence
>>> mechanism to
>>> > handle it. The error relating to failure to find a page for eviction
>>> > suggest the persistence mechanism has fallen behind. If this is the
>>> case,
>>> > this seems like an unfriendly failure mode.
>>> >
>>> > Thanks,
>>> > Raymond.
>>> >
>>> >
>>> >
>>>
>>

Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1

[Denis Magda]

Actually, we already have a related discussion:
http://apache-ignite-developers.2346864.n4.nabble.com/Slim-binary-release-and-docker-image-for-Apache-Ignite-td45110.html

Ilya, would you mind restarting it highlighting that it's suggested to
release a slim package for Ignite 2.8.

-
Denis


On Thu, Mar 5, 2020 at 12:27 PM Denis Magda  wrote:

> Ilya,
>
> Thanks for driving this activity. It would be nice to shift this
> conversation to another thread, agree on the slim package structure and
> release it for 2.8. This discussion is not the most visible one for the
> topic you're raising.
>
> -
> Denis
>
>
> On Thu, Mar 5, 2020 at 10:49 AM Ilya Kasnacheev 
> wrote:
>
>> Hello!
>>
>> Unfortunately, ignite-zookeeper has a lot of dependencies (8M) which we do
>> not update often enough (such as guava, curator, jackson), and which may
>> form an attack surface.
>>
>> Not a pressing problem for 'integrated' ignite-zookeeper users, since they
>> can re-import these dependencies with more recent versions using maven or
>> gradle.
>> But for our users who rely on binary package for all JARs, outdated
>> dependencies may pose a problem.
>>
>> Therefore my opinion is to exclude this dependency and not put our faith
>> on
>> zookeeper dependency version.
>>
>> The same can be put for ignite-compress, and indeed, I'm not sure if we
>> should keep it.
>>
>> We can have an ad-hoc vote here.
>>
>> I would like to hear arguments for both inclusion and exclusion of
>> ignite-zookeeper and ignite-compress into slim package (in any
>> combination).
>>
>> I would also like to know if you want a formal vote on the issue.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> чт, 5 мар. 2020 г. в 21:08, Maxim Muzafarov :
>>
>> > Ilya,
>> >
>> >
>> > Thank you for preparing the slim package. Can we discuss and fix the
>> > list of modules which will be included to "slim" package in the
>> > appropriate thread [1]? From my understanding, we should include
>> > `ignite-zookeeper` to the package and update the TeamCity release
>> > suite also.
>> >
>> >
>> > I see no reasons to rush with it since, as you mentioned, it doesn't
>> > affect the release build and we can upload a bit later for 2.8.0 too.
>> >
>> >
>> > [1]
>> >
>> http://apache-ignite-developers.2346864.n4.nabble.com/Slim-binary-release-and-docker-image-for-Apache-Ignite-td45110.html
>> >
>> > On Thu, 5 Mar 2020 at 20:27, Ilya Kasnacheev > >
>> > wrote:
>> > >
>> > > Hello!
>> > >
>> > > libs:
>> > > core/shmem/jcache
>> > > ignite-indexing
>> > > ignite-spring
>> > >
>> > > libs/optional:
>> > > ignite-compress  ignite-kubernetes  ignite-log4j2
>> ignite-rest-http
>> > >  ignite-spring-data_2.2
>> > > ignite-jta   ignite-log4j   ignite-opencensus  ignite-slf4j
>> > >  ignite-urideploy
>> > >
>> > > I have kept examples, but removed benchmarks. sqlline still present,
>> of
>> > > course.
>> > >
>> > > Regards,
>> > > --
>> > > Ilya Kasnacheev
>> > >
>> > >
>> > > чт, 5 мар. 2020 г. в 20:19, Nikolay Izhikov :
>> > >
>> > > > Hello, Ilya!
>> > > >
>> > > > Thanks for putting your effort into this task.
>> > > >
>> > > > Can you, please, list modules that you include in the slim package?
>> > > >
>> > > > > 5 марта 2020 г., в 20:17, Ilya Kasnacheev <
>> ilya.kasnach...@gmail.com
>> > >
>> > > > написал(а):
>> > > > >
>> > > > > Hello!
>> > > > >
>> > > > > I have prepared assemblies for Apache Ignite slim packaging:
>> > > > > https://github.com/apache/ignite/tree/ignite-slim
>> > > > >
>> > > > > It is based on ignite-2.8
>> > > > >
>> > > > > You can build it with mvn initialize -Prelease,lgpl
>> > > > > -Dignite.edition=apache-ignite-slim
>> > > > > after a normal release build.
>> > > > >
>> > > > > Please consider the contents of resulting
>> > > > > target/bin/apache-ignite-slim-2.8.0-bin.zip
>> > > > > It will be a 65M download as opposed to main 455M
>> apache-ignite-2.8.0
>> > > > > distribution.
>> > > > >
>> > > > > My suggestion is that we can publish it as a post-release step
>> since
>> > it
>> > > > > does not affect the release in any way.
>> > > > > What do you think?
>> > > > >
>> > > > > If we do, we should probably indicate size for every kind of
>> > artifact in
>> > > > > our download section, so our users can choose based on that
>> > information.
>> > > > >
>> > > > > NB: I have kept ignite-compress in this slim package. If removed,
>> > it's
>> > > > > extra 9M shaved.
>> > > > >
>> > > > > Regards,
>> > > > > --
>> > > > > Ilya Kasnacheev
>> > > > >
>> > > > >
>> > > > > чт, 5 мар. 2020 г. в 13:26, Sergey-A Kosarev <
>> > sergey-a.kosa...@db.com>:
>> > > > >
>> > > > >> Classification: Public
>> > > > >>
>> > > > >> There are no RC anymore, I believe. It's now just 2.8.0 (as in
>> Maven
>> > > > >> Central)
>> > > > >>
>> > > > >> Thank you to all involved in the release guys, especially Maxim!
>> > > > >>
>> > > > >> Kind regards,
>> > > > >> Sergey Kosarev
>> > > > >>
>> > > > >> -Original Message-
>> > > > >> From: Ivan Pavlukhin 

Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1

[Denis Magda]

Ilya,

Thanks for driving this activity. It would be nice to shift this
conversation to another thread, agree on the slim package structure and
release it for 2.8. This discussion is not the most visible one for the
topic you're raising.

-
Denis


On Thu, Mar 5, 2020 at 10:49 AM Ilya Kasnacheev 
wrote:

> Hello!
>
> Unfortunately, ignite-zookeeper has a lot of dependencies (8M) which we do
> not update often enough (such as guava, curator, jackson), and which may
> form an attack surface.
>
> Not a pressing problem for 'integrated' ignite-zookeeper users, since they
> can re-import these dependencies with more recent versions using maven or
> gradle.
> But for our users who rely on binary package for all JARs, outdated
> dependencies may pose a problem.
>
> Therefore my opinion is to exclude this dependency and not put our faith on
> zookeeper dependency version.
>
> The same can be put for ignite-compress, and indeed, I'm not sure if we
> should keep it.
>
> We can have an ad-hoc vote here.
>
> I would like to hear arguments for both inclusion and exclusion of
> ignite-zookeeper and ignite-compress into slim package (in any
> combination).
>
> I would also like to know if you want a formal vote on the issue.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> чт, 5 мар. 2020 г. в 21:08, Maxim Muzafarov :
>
> > Ilya,
> >
> >
> > Thank you for preparing the slim package. Can we discuss and fix the
> > list of modules which will be included to "slim" package in the
> > appropriate thread [1]? From my understanding, we should include
> > `ignite-zookeeper` to the package and update the TeamCity release
> > suite also.
> >
> >
> > I see no reasons to rush with it since, as you mentioned, it doesn't
> > affect the release build and we can upload a bit later for 2.8.0 too.
> >
> >
> > [1]
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Slim-binary-release-and-docker-image-for-Apache-Ignite-td45110.html
> >
> > On Thu, 5 Mar 2020 at 20:27, Ilya Kasnacheev 
> > wrote:
> > >
> > > Hello!
> > >
> > > libs:
> > > core/shmem/jcache
> > > ignite-indexing
> > > ignite-spring
> > >
> > > libs/optional:
> > > ignite-compress  ignite-kubernetes  ignite-log4j2  ignite-rest-http
> > >  ignite-spring-data_2.2
> > > ignite-jta   ignite-log4j   ignite-opencensus  ignite-slf4j
> > >  ignite-urideploy
> > >
> > > I have kept examples, but removed benchmarks. sqlline still present, of
> > > course.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > чт, 5 мар. 2020 г. в 20:19, Nikolay Izhikov :
> > >
> > > > Hello, Ilya!
> > > >
> > > > Thanks for putting your effort into this task.
> > > >
> > > > Can you, please, list modules that you include in the slim package?
> > > >
> > > > > 5 марта 2020 г., в 20:17, Ilya Kasnacheev <
> ilya.kasnach...@gmail.com
> > >
> > > > написал(а):
> > > > >
> > > > > Hello!
> > > > >
> > > > > I have prepared assemblies for Apache Ignite slim packaging:
> > > > > https://github.com/apache/ignite/tree/ignite-slim
> > > > >
> > > > > It is based on ignite-2.8
> > > > >
> > > > > You can build it with mvn initialize -Prelease,lgpl
> > > > > -Dignite.edition=apache-ignite-slim
> > > > > after a normal release build.
> > > > >
> > > > > Please consider the contents of resulting
> > > > > target/bin/apache-ignite-slim-2.8.0-bin.zip
> > > > > It will be a 65M download as opposed to main 455M
> apache-ignite-2.8.0
> > > > > distribution.
> > > > >
> > > > > My suggestion is that we can publish it as a post-release step
> since
> > it
> > > > > does not affect the release in any way.
> > > > > What do you think?
> > > > >
> > > > > If we do, we should probably indicate size for every kind of
> > artifact in
> > > > > our download section, so our users can choose based on that
> > information.
> > > > >
> > > > > NB: I have kept ignite-compress in this slim package. If removed,
> > it's
> > > > > extra 9M shaved.
> > > > >
> > > > > Regards,
> > > > > --
> > > > > Ilya Kasnacheev
> > > > >
> > > > >
> > > > > чт, 5 мар. 2020 г. в 13:26, Sergey-A Kosarev <
> > sergey-a.kosa...@db.com>:
> > > > >
> > > > >> Classification: Public
> > > > >>
> > > > >> There are no RC anymore, I believe. It's now just 2.8.0 (as in
> Maven
> > > > >> Central)
> > > > >>
> > > > >> Thank you to all involved in the release guys, especially Maxim!
> > > > >>
> > > > >> Kind regards,
> > > > >> Sergey Kosarev
> > > > >>
> > > > >> -Original Message-
> > > > >> From: Ivan Pavlukhin [mailto:vololo...@gmail.com]
> > > > >> Sent: 05 March 2020 12:49
> > > > >> To: dev 
> > > > >> Subject: Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1
> > > > >>
> > > > >> Wow. I impressed that users try out even RC.
> > > > >>
> > > > >> Best regards,
> > > > >> Ivan Pavlukhin
> > > > >>
> > > > >> чт, 5 мар. 2020 г. в 12:01, Maxim Muzafarov :
> > > > >>>
> > > > >>> Ivan,
> > > > >>>
> > > > >>> I think `final steps` is mostly related not to binary\sources
> > itself
> > > > >>> but for the announce message. 

[jira] [Created] (IGNITE-12754) .NET: Thin Client: Service invocation

[Pavel Tupitsyn (Jira)]

Pavel Tupitsyn created IGNITE-12754:
---

 Summary: .NET: Thin Client: Service invocation
 Key: IGNITE-12754
 URL: https://issues.apache.org/jira/browse/IGNITE-12754
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 2.9


Provide an API to invoke Ignite Services from Thin Clients.

.NET API:
{code}
IIgniteClient.GetServices().GetServiceProxy("name").Bar();
{code}

Thin Client protocol: 
* One operation, OP_SERVICE_INVOKE
* Takes service name, method name, optionally node ids (cluster projection), 
0..n args

See PlatformServices, we just have to combine OP_SERVICE_PROXY with OP_INVOKE 
from there in one call.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1

[Ilya Kasnacheev]

Hello!

Unfortunately, ignite-zookeeper has a lot of dependencies (8M) which we do
not update often enough (such as guava, curator, jackson), and which may
form an attack surface.

Not a pressing problem for 'integrated' ignite-zookeeper users, since they
can re-import these dependencies with more recent versions using maven or
gradle.
But for our users who rely on binary package for all JARs, outdated
dependencies may pose a problem.

Therefore my opinion is to exclude this dependency and not put our faith on
zookeeper dependency version.

The same can be put for ignite-compress, and indeed, I'm not sure if we
should keep it.

We can have an ad-hoc vote here.

I would like to hear arguments for both inclusion and exclusion of
ignite-zookeeper and ignite-compress into slim package (in any combination).

I would also like to know if you want a formal vote on the issue.

Regards,
-- 
Ilya Kasnacheev


чт, 5 мар. 2020 г. в 21:08, Maxim Muzafarov :

> Ilya,
>
>
> Thank you for preparing the slim package. Can we discuss and fix the
> list of modules which will be included to "slim" package in the
> appropriate thread [1]? From my understanding, we should include
> `ignite-zookeeper` to the package and update the TeamCity release
> suite also.
>
>
> I see no reasons to rush with it since, as you mentioned, it doesn't
> affect the release build and we can upload a bit later for 2.8.0 too.
>
>
> [1]
> http://apache-ignite-developers.2346864.n4.nabble.com/Slim-binary-release-and-docker-image-for-Apache-Ignite-td45110.html
>
> On Thu, 5 Mar 2020 at 20:27, Ilya Kasnacheev 
> wrote:
> >
> > Hello!
> >
> > libs:
> > core/shmem/jcache
> > ignite-indexing
> > ignite-spring
> >
> > libs/optional:
> > ignite-compress  ignite-kubernetes  ignite-log4j2  ignite-rest-http
> >  ignite-spring-data_2.2
> > ignite-jta   ignite-log4j   ignite-opencensus  ignite-slf4j
> >  ignite-urideploy
> >
> > I have kept examples, but removed benchmarks. sqlline still present, of
> > course.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > чт, 5 мар. 2020 г. в 20:19, Nikolay Izhikov :
> >
> > > Hello, Ilya!
> > >
> > > Thanks for putting your effort into this task.
> > >
> > > Can you, please, list modules that you include in the slim package?
> > >
> > > > 5 марта 2020 г., в 20:17, Ilya Kasnacheev  >
> > > написал(а):
> > > >
> > > > Hello!
> > > >
> > > > I have prepared assemblies for Apache Ignite slim packaging:
> > > > https://github.com/apache/ignite/tree/ignite-slim
> > > >
> > > > It is based on ignite-2.8
> > > >
> > > > You can build it with mvn initialize -Prelease,lgpl
> > > > -Dignite.edition=apache-ignite-slim
> > > > after a normal release build.
> > > >
> > > > Please consider the contents of resulting
> > > > target/bin/apache-ignite-slim-2.8.0-bin.zip
> > > > It will be a 65M download as opposed to main 455M apache-ignite-2.8.0
> > > > distribution.
> > > >
> > > > My suggestion is that we can publish it as a post-release step since
> it
> > > > does not affect the release in any way.
> > > > What do you think?
> > > >
> > > > If we do, we should probably indicate size for every kind of
> artifact in
> > > > our download section, so our users can choose based on that
> information.
> > > >
> > > > NB: I have kept ignite-compress in this slim package. If removed,
> it's
> > > > extra 9M shaved.
> > > >
> > > > Regards,
> > > > --
> > > > Ilya Kasnacheev
> > > >
> > > >
> > > > чт, 5 мар. 2020 г. в 13:26, Sergey-A Kosarev <
> sergey-a.kosa...@db.com>:
> > > >
> > > >> Classification: Public
> > > >>
> > > >> There are no RC anymore, I believe. It's now just 2.8.0 (as in Maven
> > > >> Central)
> > > >>
> > > >> Thank you to all involved in the release guys, especially Maxim!
> > > >>
> > > >> Kind regards,
> > > >> Sergey Kosarev
> > > >>
> > > >> -Original Message-
> > > >> From: Ivan Pavlukhin [mailto:vololo...@gmail.com]
> > > >> Sent: 05 March 2020 12:49
> > > >> To: dev 
> > > >> Subject: Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1
> > > >>
> > > >> Wow. I impressed that users try out even RC.
> > > >>
> > > >> Best regards,
> > > >> Ivan Pavlukhin
> > > >>
> > > >> чт, 5 мар. 2020 г. в 12:01, Maxim Muzafarov :
> > > >>>
> > > >>> Ivan,
> > > >>>
> > > >>> I think `final steps` is mostly related not to binary\sources
> itself
> > > >>> but for the announce message. All documentation must be up-to-date
> > > >>> when it occurs.
> > > >>>
> > > >>> But I think you're right that users already using a new version of
> > > >>> Apache Ignite without announcing. We've got some questions\bugs on
> > > >>> user-list already [1] [2].
> > > >>>
> > > >>>
> > > >>> [1]
> > > >>>
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-2-8-get-exceptio
> > > >>> n-while-use-batch-insert-in-streaming-mode-td31614.html
> > > >>> [2]
> > > >>>
> http://apache-ignite-users.70518.x6.nabble.com/Zookeeper-discovery-wit
> > > >>> h-Ignite-2-8-0-class-NoClassDefFound-td31615.html
> > > >>>
> > > >>> On Thu, 5 Mar 

Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1

[Maxim Muzafarov]

Ilya,


Thank you for preparing the slim package. Can we discuss and fix the
list of modules which will be included to "slim" package in the
appropriate thread [1]? From my understanding, we should include
`ignite-zookeeper` to the package and update the TeamCity release
suite also.


I see no reasons to rush with it since, as you mentioned, it doesn't
affect the release build and we can upload a bit later for 2.8.0 too.


[1] 
http://apache-ignite-developers.2346864.n4.nabble.com/Slim-binary-release-and-docker-image-for-Apache-Ignite-td45110.html

On Thu, 5 Mar 2020 at 20:27, Ilya Kasnacheev  wrote:
>
> Hello!
>
> libs:
> core/shmem/jcache
> ignite-indexing
> ignite-spring
>
> libs/optional:
> ignite-compress  ignite-kubernetes  ignite-log4j2  ignite-rest-http
>  ignite-spring-data_2.2
> ignite-jta   ignite-log4j   ignite-opencensus  ignite-slf4j
>  ignite-urideploy
>
> I have kept examples, but removed benchmarks. sqlline still present, of
> course.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> чт, 5 мар. 2020 г. в 20:19, Nikolay Izhikov :
>
> > Hello, Ilya!
> >
> > Thanks for putting your effort into this task.
> >
> > Can you, please, list modules that you include in the slim package?
> >
> > > 5 марта 2020 г., в 20:17, Ilya Kasnacheev 
> > написал(а):
> > >
> > > Hello!
> > >
> > > I have prepared assemblies for Apache Ignite slim packaging:
> > > https://github.com/apache/ignite/tree/ignite-slim
> > >
> > > It is based on ignite-2.8
> > >
> > > You can build it with mvn initialize -Prelease,lgpl
> > > -Dignite.edition=apache-ignite-slim
> > > after a normal release build.
> > >
> > > Please consider the contents of resulting
> > > target/bin/apache-ignite-slim-2.8.0-bin.zip
> > > It will be a 65M download as opposed to main 455M apache-ignite-2.8.0
> > > distribution.
> > >
> > > My suggestion is that we can publish it as a post-release step since it
> > > does not affect the release in any way.
> > > What do you think?
> > >
> > > If we do, we should probably indicate size for every kind of artifact in
> > > our download section, so our users can choose based on that information.
> > >
> > > NB: I have kept ignite-compress in this slim package. If removed, it's
> > > extra 9M shaved.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > чт, 5 мар. 2020 г. в 13:26, Sergey-A Kosarev :
> > >
> > >> Classification: Public
> > >>
> > >> There are no RC anymore, I believe. It's now just 2.8.0 (as in Maven
> > >> Central)
> > >>
> > >> Thank you to all involved in the release guys, especially Maxim!
> > >>
> > >> Kind regards,
> > >> Sergey Kosarev
> > >>
> > >> -Original Message-
> > >> From: Ivan Pavlukhin [mailto:vololo...@gmail.com]
> > >> Sent: 05 March 2020 12:49
> > >> To: dev 
> > >> Subject: Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1
> > >>
> > >> Wow. I impressed that users try out even RC.
> > >>
> > >> Best regards,
> > >> Ivan Pavlukhin
> > >>
> > >> чт, 5 мар. 2020 г. в 12:01, Maxim Muzafarov :
> > >>>
> > >>> Ivan,
> > >>>
> > >>> I think `final steps` is mostly related not to binary\sources itself
> > >>> but for the announce message. All documentation must be up-to-date
> > >>> when it occurs.
> > >>>
> > >>> But I think you're right that users already using a new version of
> > >>> Apache Ignite without announcing. We've got some questions\bugs on
> > >>> user-list already [1] [2].
> > >>>
> > >>>
> > >>> [1]
> > >>> http://apache-ignite-users.70518.x6.nabble.com/Ignite-2-8-get-exceptio
> > >>> n-while-use-batch-insert-in-streaming-mode-td31614.html
> > >>> [2]
> > >>> http://apache-ignite-users.70518.x6.nabble.com/Zookeeper-discovery-wit
> > >>> h-Ignite-2-8-0-class-NoClassDefFound-td31615.html
> > >>>
> > >>> On Thu, 5 Mar 2020 at 11:51, Ivan Pavlukhin 
> > wrote:
> > 
> >  Maxim,
> > 
> >  You did the great job! Many thanks for that!
> > 
> >  I suppose there is some confusion with terminology. I understand
> >  that Ignite release process has several final steps. And for example
> >  binaries can be published before documentation is ready and it is ok
> >  (I suppose opposite will make no harm as well). My personal
> >  understanding is that a product version "is considered released"
> >  once a user can receive and use it tagged with the target version.
> >  In other words, once it is generally available (GA). And we should
> >  not force users to check product web sites and other announcement
> >  channels to check that a package available in the Maven repository
> >  is valid. For someone it might be just "keeping all dependencies up to
> > >> date".
> > 
> >  Best regards,
> >  Ivan Pavlukhin
> > 
> >  чт, 5 мар. 2020 г. в 09:33, Maxim Muzafarov :
> > >
> > > Ivan,
> > >
> > > I did something wrong?
> > > What should I do to release all effectively?
> > >
> > > On Thu, 5 Mar 2020 at 08:23, Denis Magda  wrote:
> > >>
> > >> Ivan,

Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1

[Ilya Kasnacheev]

Hello!

libs:
core/shmem/jcache
ignite-indexing
ignite-spring

libs/optional:
ignite-compress  ignite-kubernetes  ignite-log4j2  ignite-rest-http
 ignite-spring-data_2.2
ignite-jta   ignite-log4j   ignite-opencensus  ignite-slf4j
 ignite-urideploy

I have kept examples, but removed benchmarks. sqlline still present, of
course.

Regards,
-- 
Ilya Kasnacheev


чт, 5 мар. 2020 г. в 20:19, Nikolay Izhikov :

> Hello, Ilya!
>
> Thanks for putting your effort into this task.
>
> Can you, please, list modules that you include in the slim package?
>
> > 5 марта 2020 г., в 20:17, Ilya Kasnacheev 
> написал(а):
> >
> > Hello!
> >
> > I have prepared assemblies for Apache Ignite slim packaging:
> > https://github.com/apache/ignite/tree/ignite-slim
> >
> > It is based on ignite-2.8
> >
> > You can build it with mvn initialize -Prelease,lgpl
> > -Dignite.edition=apache-ignite-slim
> > after a normal release build.
> >
> > Please consider the contents of resulting
> > target/bin/apache-ignite-slim-2.8.0-bin.zip
> > It will be a 65M download as opposed to main 455M apache-ignite-2.8.0
> > distribution.
> >
> > My suggestion is that we can publish it as a post-release step since it
> > does not affect the release in any way.
> > What do you think?
> >
> > If we do, we should probably indicate size for every kind of artifact in
> > our download section, so our users can choose based on that information.
> >
> > NB: I have kept ignite-compress in this slim package. If removed, it's
> > extra 9M shaved.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > чт, 5 мар. 2020 г. в 13:26, Sergey-A Kosarev :
> >
> >> Classification: Public
> >>
> >> There are no RC anymore, I believe. It's now just 2.8.0 (as in Maven
> >> Central)
> >>
> >> Thank you to all involved in the release guys, especially Maxim!
> >>
> >> Kind regards,
> >> Sergey Kosarev
> >>
> >> -Original Message-
> >> From: Ivan Pavlukhin [mailto:vololo...@gmail.com]
> >> Sent: 05 March 2020 12:49
> >> To: dev 
> >> Subject: Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1
> >>
> >> Wow. I impressed that users try out even RC.
> >>
> >> Best regards,
> >> Ivan Pavlukhin
> >>
> >> чт, 5 мар. 2020 г. в 12:01, Maxim Muzafarov :
> >>>
> >>> Ivan,
> >>>
> >>> I think `final steps` is mostly related not to binary\sources itself
> >>> but for the announce message. All documentation must be up-to-date
> >>> when it occurs.
> >>>
> >>> But I think you're right that users already using a new version of
> >>> Apache Ignite without announcing. We've got some questions\bugs on
> >>> user-list already [1] [2].
> >>>
> >>>
> >>> [1]
> >>> http://apache-ignite-users.70518.x6.nabble.com/Ignite-2-8-get-exceptio
> >>> n-while-use-batch-insert-in-streaming-mode-td31614.html
> >>> [2]
> >>> http://apache-ignite-users.70518.x6.nabble.com/Zookeeper-discovery-wit
> >>> h-Ignite-2-8-0-class-NoClassDefFound-td31615.html
> >>>
> >>> On Thu, 5 Mar 2020 at 11:51, Ivan Pavlukhin 
> wrote:
> 
>  Maxim,
> 
>  You did the great job! Many thanks for that!
> 
>  I suppose there is some confusion with terminology. I understand
>  that Ignite release process has several final steps. And for example
>  binaries can be published before documentation is ready and it is ok
>  (I suppose opposite will make no harm as well). My personal
>  understanding is that a product version "is considered released"
>  once a user can receive and use it tagged with the target version.
>  In other words, once it is generally available (GA). And we should
>  not force users to check product web sites and other announcement
>  channels to check that a package available in the Maven repository
>  is valid. For someone it might be just "keeping all dependencies up to
> >> date".
> 
>  Best regards,
>  Ivan Pavlukhin
> 
>  чт, 5 мар. 2020 г. в 09:33, Maxim Muzafarov :
> >
> > Ivan,
> >
> > I did something wrong?
> > What should I do to release all effectively?
> >
> > On Thu, 5 Mar 2020 at 08:23, Denis Magda  wrote:
> >>
> >> Ivan,
> >>
> >> The release of binaries is one of the final steps. There are
> >> still several steps left before we can consider all the 2.8
> >> release activities completed:
> >> https://cwiki.apache.org/confluence/display/IGNITE/Release+Proce
> >> ss#ReleaseProcess-6.3.4.Releasedocumentaiononreadme.io
> >> https://cwiki.apache.org/confluence/display/IGNITE/Release+Proce
> >> ss#ReleaseProcess-6.4.Announcerelease
> >>
> >> -
> >> Denis
> >>
> >>
> >> On Wed, Mar 4, 2020 at 7:57 PM Ivan Pavlukhin 
> >> wrote:
> >>
> >>> Denis, folks,
> >>>
> >>> As it was already mentioned 2.8 is already in maven [1].
> >>> Consequently "effectively" released. In my opinion it is not
> >>> correct to consider release in progress.
> >>>
> >>> Agree that 2.8.1 sounds as a proper option here.
> >>>
> 

Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1

[Nikolay Izhikov]

Hello, Ilya!

Thanks for putting your effort into this task.

Can you, please, list modules that you include in the slim package?

> 5 марта 2020 г., в 20:17, Ilya Kasnacheev  
> написал(а):
> 
> Hello!
> 
> I have prepared assemblies for Apache Ignite slim packaging:
> https://github.com/apache/ignite/tree/ignite-slim
> 
> It is based on ignite-2.8
> 
> You can build it with mvn initialize -Prelease,lgpl
> -Dignite.edition=apache-ignite-slim
> after a normal release build.
> 
> Please consider the contents of resulting
> target/bin/apache-ignite-slim-2.8.0-bin.zip
> It will be a 65M download as opposed to main 455M apache-ignite-2.8.0
> distribution.
> 
> My suggestion is that we can publish it as a post-release step since it
> does not affect the release in any way.
> What do you think?
> 
> If we do, we should probably indicate size for every kind of artifact in
> our download section, so our users can choose based on that information.
> 
> NB: I have kept ignite-compress in this slim package. If removed, it's
> extra 9M shaved.
> 
> Regards,
> -- 
> Ilya Kasnacheev
> 
> 
> чт, 5 мар. 2020 г. в 13:26, Sergey-A Kosarev :
> 
>> Classification: Public
>> 
>> There are no RC anymore, I believe. It's now just 2.8.0 (as in Maven
>> Central)
>> 
>> Thank you to all involved in the release guys, especially Maxim!
>> 
>> Kind regards,
>> Sergey Kosarev
>> 
>> -Original Message-
>> From: Ivan Pavlukhin [mailto:vololo...@gmail.com]
>> Sent: 05 March 2020 12:49
>> To: dev 
>> Subject: Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1
>> 
>> Wow. I impressed that users try out even RC.
>> 
>> Best regards,
>> Ivan Pavlukhin
>> 
>> чт, 5 мар. 2020 г. в 12:01, Maxim Muzafarov :
>>> 
>>> Ivan,
>>> 
>>> I think `final steps` is mostly related not to binary\sources itself
>>> but for the announce message. All documentation must be up-to-date
>>> when it occurs.
>>> 
>>> But I think you're right that users already using a new version of
>>> Apache Ignite without announcing. We've got some questions\bugs on
>>> user-list already [1] [2].
>>> 
>>> 
>>> [1]
>>> http://apache-ignite-users.70518.x6.nabble.com/Ignite-2-8-get-exceptio
>>> n-while-use-batch-insert-in-streaming-mode-td31614.html
>>> [2]
>>> http://apache-ignite-users.70518.x6.nabble.com/Zookeeper-discovery-wit
>>> h-Ignite-2-8-0-class-NoClassDefFound-td31615.html
>>> 
>>> On Thu, 5 Mar 2020 at 11:51, Ivan Pavlukhin  wrote:
 
 Maxim,
 
 You did the great job! Many thanks for that!
 
 I suppose there is some confusion with terminology. I understand
 that Ignite release process has several final steps. And for example
 binaries can be published before documentation is ready and it is ok
 (I suppose opposite will make no harm as well). My personal
 understanding is that a product version "is considered released"
 once a user can receive and use it tagged with the target version.
 In other words, once it is generally available (GA). And we should
 not force users to check product web sites and other announcement
 channels to check that a package available in the Maven repository
 is valid. For someone it might be just "keeping all dependencies up to
>> date".
 
 Best regards,
 Ivan Pavlukhin
 
 чт, 5 мар. 2020 г. в 09:33, Maxim Muzafarov :
> 
> Ivan,
> 
> I did something wrong?
> What should I do to release all effectively?
> 
> On Thu, 5 Mar 2020 at 08:23, Denis Magda  wrote:
>> 
>> Ivan,
>> 
>> The release of binaries is one of the final steps. There are
>> still several steps left before we can consider all the 2.8
>> release activities completed:
>> https://cwiki.apache.org/confluence/display/IGNITE/Release+Proce
>> ss#ReleaseProcess-6.3.4.Releasedocumentaiononreadme.io
>> https://cwiki.apache.org/confluence/display/IGNITE/Release+Proce
>> ss#ReleaseProcess-6.4.Announcerelease
>> 
>> -
>> Denis
>> 
>> 
>> On Wed, Mar 4, 2020 at 7:57 PM Ivan Pavlukhin 
>> wrote:
>> 
>>> Denis, folks,
>>> 
>>> As it was already mentioned 2.8 is already in maven [1].
>>> Consequently "effectively" released. In my opinion it is not
>>> correct to consider release in progress.
>>> 
>>> Agree that 2.8.1 sounds as a proper option here.
>>> 
>>> [1]
>>> https://mvnrepository.com/artifact/org.apache.ignite/ignite-co
>>> re/2.8.0
>>> 
>>> Best regards,
>>> Ivan Pavlukhin
>>> 
>>> ср, 4 мар. 2020 г. в 17:08, Denis Magda :
 
 Dmitriy,
 
 Ignite 2.8 will be announced as soon as the documentation is
 prepared and released. Until then, the release is still
>> considered under progress.
>>> Here
 is a dedicated discussion for documentation activities. We
 should be
>>> ready
 soon:
 
>>> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-2
>>> 

Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1

[Ilya Kasnacheev]

Hello!

I have prepared assemblies for Apache Ignite slim packaging:
https://github.com/apache/ignite/tree/ignite-slim

It is based on ignite-2.8

You can build it with mvn initialize -Prelease,lgpl
-Dignite.edition=apache-ignite-slim
after a normal release build.

Please consider the contents of resulting
target/bin/apache-ignite-slim-2.8.0-bin.zip
It will be a 65M download as opposed to main 455M apache-ignite-2.8.0
distribution.

My suggestion is that we can publish it as a post-release step since it
does not affect the release in any way.
What do you think?

If we do, we should probably indicate size for every kind of artifact in
our download section, so our users can choose based on that information.

NB: I have kept ignite-compress in this slim package. If removed, it's
extra 9M shaved.

Regards,
-- 
Ilya Kasnacheev


чт, 5 мар. 2020 г. в 13:26, Sergey-A Kosarev :

> Classification: Public
>
> There are no RC anymore, I believe. It's now just 2.8.0 (as in Maven
> Central)
>
> Thank you to all involved in the release guys, especially Maxim!
>
> Kind regards,
> Sergey Kosarev
>
> -Original Message-
> From: Ivan Pavlukhin [mailto:vololo...@gmail.com]
> Sent: 05 March 2020 12:49
> To: dev 
> Subject: Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1
>
> Wow. I impressed that users try out even RC.
>
> Best regards,
> Ivan Pavlukhin
>
> чт, 5 мар. 2020 г. в 12:01, Maxim Muzafarov :
> >
> > Ivan,
> >
> > I think `final steps` is mostly related not to binary\sources itself
> > but for the announce message. All documentation must be up-to-date
> > when it occurs.
> >
> > But I think you're right that users already using a new version of
> > Apache Ignite without announcing. We've got some questions\bugs on
> > user-list already [1] [2].
> >
> >
> > [1]
> > http://apache-ignite-users.70518.x6.nabble.com/Ignite-2-8-get-exceptio
> > n-while-use-batch-insert-in-streaming-mode-td31614.html
> > [2]
> > http://apache-ignite-users.70518.x6.nabble.com/Zookeeper-discovery-wit
> > h-Ignite-2-8-0-class-NoClassDefFound-td31615.html
> >
> > On Thu, 5 Mar 2020 at 11:51, Ivan Pavlukhin  wrote:
> > >
> > > Maxim,
> > >
> > > You did the great job! Many thanks for that!
> > >
> > > I suppose there is some confusion with terminology. I understand
> > > that Ignite release process has several final steps. And for example
> > > binaries can be published before documentation is ready and it is ok
> > > (I suppose opposite will make no harm as well). My personal
> > > understanding is that a product version "is considered released"
> > > once a user can receive and use it tagged with the target version.
> > > In other words, once it is generally available (GA). And we should
> > > not force users to check product web sites and other announcement
> > > channels to check that a package available in the Maven repository
> > > is valid. For someone it might be just "keeping all dependencies up to
> date".
> > >
> > > Best regards,
> > > Ivan Pavlukhin
> > >
> > > чт, 5 мар. 2020 г. в 09:33, Maxim Muzafarov :
> > > >
> > > > Ivan,
> > > >
> > > > I did something wrong?
> > > > What should I do to release all effectively?
> > > >
> > > > On Thu, 5 Mar 2020 at 08:23, Denis Magda  wrote:
> > > > >
> > > > > Ivan,
> > > > >
> > > > > The release of binaries is one of the final steps. There are
> > > > > still several steps left before we can consider all the 2.8
> release activities completed:
> > > > > https://cwiki.apache.org/confluence/display/IGNITE/Release+Proce
> > > > > ss#ReleaseProcess-6.3.4.Releasedocumentaiononreadme.io
> > > > > https://cwiki.apache.org/confluence/display/IGNITE/Release+Proce
> > > > > ss#ReleaseProcess-6.4.Announcerelease
> > > > >
> > > > > -
> > > > > Denis
> > > > >
> > > > >
> > > > > On Wed, Mar 4, 2020 at 7:57 PM Ivan Pavlukhin 
> wrote:
> > > > >
> > > > > > Denis, folks,
> > > > > >
> > > > > > As it was already mentioned 2.8 is already in maven [1].
> > > > > > Consequently "effectively" released. In my opinion it is not
> > > > > > correct to consider release in progress.
> > > > > >
> > > > > > Agree that 2.8.1 sounds as a proper option here.
> > > > > >
> > > > > > [1]
> > > > > > https://mvnrepository.com/artifact/org.apache.ignite/ignite-co
> > > > > > re/2.8.0
> > > > > >
> > > > > > Best regards,
> > > > > > Ivan Pavlukhin
> > > > > >
> > > > > > ср, 4 мар. 2020 г. в 17:08, Denis Magda :
> > > > > > >
> > > > > > > Dmitriy,
> > > > > > >
> > > > > > > Ignite 2.8 will be announced as soon as the documentation is
> > > > > > > prepared and released. Until then, the release is still
> considered under progress.
> > > > > > Here
> > > > > > > is a dedicated discussion for documentation activities. We
> > > > > > > should be
> > > > > > ready
> > > > > > > soon:
> > > > > > >
> > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-2
> > > > > > -8-documentation-td46008.html
> > > > > > >
> > > > > > > Also, I've started a discussion related to our announcement
> > > > > > > 

Re: Ignite 2.8 announcement plan

[Pavel Tupitsyn]

Denis,

The first post is going to be ready soon, probably by tomorrow.
Here is a draft, feedback welcome:
https://github.com/ptupitsyn/ptupitsyn.github.io/blob/ignite-2.8/_posts/2020-03-05-Whats-New-In-Ignite-Net-2.8.md

On Wed, Mar 4, 2020 at 5:15 PM Denis Magda  wrote:

> Hi Pavel,
>
> Excellent! It will be good to publish the first article (what's new in
> Ignite.NET 2.8) prior to a generic blog on blogs.apache.org so that we can
> link your post in for those who are looking for more details. Do you have
> any timeline in mind for this article?
>
> @Alexey Zinoviev , how about you preparing several
> paragraphs for the blog highlighting the biggest changes in ML? The same
> highlighted content will be elaborated during the webinar.
>
> -
> Denis
>
>
> On Wed, Mar 4, 2020 at 2:29 AM Pavel Tupitsyn 
> wrote:
>
> > Denis, I have a few blog posts on Ignite.NET planned:
> >
> > * What's new in Ignite.NET 2.8 (Thin Client Partition Awareness, logging,
> > expiry policy, cluster API, .NET Core 3.x, Dockerfile)
> > * Ignite.NET performance improvements on .NET Core 3.x
> > * Ignite.NET Partition Awareness performance
> > * Fixing JNI thread leak in Ignite.NET
> >
> > What do you think? Should we have a schedule of some sorts?
> >
> > On Tue, Mar 3, 2020 at 9:58 PM Alexey Zinoviev 
> > wrote:
> >
> > > Sounds ineresting, will help with the post, please share the template,
> > not
> > > sure about webinar
> > >
> > > вт, 3 мар. 2020 г., 20:59 Denis Magda :
> > >
> > > > Igniters,
> > > >
> > > > Let's discuss approaches for a global announcement/promotion of the
> > > > release. I would suggest focusing on a blog post and a community
> > webinar.
> > > >
> > > > The blog post will introduce significant improvements (service grid,
> > thin
> > > > clients, new metrics system, ML, etc.) sharing references to
> > > documentation
> > > > pages with more details.  It will be published on blogs.apache.org
> in
> > a
> > > > format similar to this one -
> > > > https://blogs.apache.org/ignite/entry/apache-ignite-2-7-deep. I can
> > work
> > > > on
> > > > it unless anybody else is willing to share the news on behalf of the
> > > > community.
> > > >
> > > > Next, the blog post will be featuring a community webinar that is
> > > breaking
> > > > down a subset of the improvements in more detail. Please see an
> > abstract
> > > > below with suggested topics for a detailed overview. @Alexey Zinoviev
> > > > , would you be able to present the ML part?
> > > > @Nikolay
> > > > Izhikov  or @Andrey Gura 
> > would
> > > > you like to take over the metrics section? I'll work the attendees
> > > through
> > > > the items listed in "Sustainable production under high load". We
> should
> > > > target the webinar for the April timeframe.
> > > >
> > > >
> > > >
> > > > *Topmost changes in Apache Ignite 2.8 for production maintenance and
> > > > machine learning*
> > > >
> > > >
> > > > *Apache Ignite community rolled out more than 1900 changes in Ignite
> > 2.8
> > > > that enhanced almost all the components of the platform. The release
> > > notes
> > > > go with hundreds of lines trying to catalog the improvements. Join
> this
> > > > webinar led by Ignite community members demonstrating and dissecting
> > new
> > > > capabilities related to production maintenance, monitoring, and
> machine
> > > > learning that you do not want to lose sight of:*
> > > >
> > > >- *Sustainable production under high load: Ignite persistence
> > > compaction
> > > >and consistent crash recovery, baseline topology auto-adjustment,
> no
> > > >interruption of operations for some cluster topology change
> events.*
> > > >- *Next-generation system for monitoring and code tracing: design
> > and
> > > >usage, exporters configuration (JMX, SQL, OpenCensus) *
> > > >- *Ignite Machine Learning major upgrade: a revised approach for
> > > models
> > > >training/evaluation, models importing from Spark ML, XGBoost and
> > much
> > > > more
> > > >  *
> > > >
> > > >
> > > > -
> > > > Denis
> > > >
> > >
> >
>

Re: Apache Ignite ML & Python

[Alexey Zinoviev]

Agree with simple case, I think we could start from the simple poc for the
Python for ML in the next release

чт, 5 мар. 2020 г., 17:05 AG :

>
> Thanks, for the reply!
>
> It looks like a high-level API similar to Sklearn pipelines.
> In my opinion, for the first steps easier to add simple assess to gain the
> ability to run a simple model or simple preprocessor from python.
>
> According to your example:
> Here is raw dataset, already inside this cluster cache "myName", with
> Label column "MyLable".
>
> I want to run from notebook UI imputer and knn using python API. Export
> results to file storage as an example.
>
> In my opinion, the ability to create such a simple workflow should be our
> goal for the first time.
>
> Thank You!
>
> Best regards,
> Andrei Gavrilov.
>
> Sent with ProtonMail Secure Email.
>
> ‐‐‐ Original Message ‐‐‐
> On Wednesday, March 4, 2020 10:49 PM, kencottrell <
> ken.cottr...@gridgain.com> wrote:
>
> > Andrei,
> >
> > I am also working with Apache Ignite ML and am interested in providing
> > wrappers for Ignite ML API, but am wondering if instead of simply
> recreating
> > the low level Java API for ML inside Python, how about creating some
> higher
> > level services "Auto ML" workflow ? For example:
> >
> > 1.  here is raw dataset, already inside this cluster cache "myName", with
> > Label column "MyLable" , take N samples tell me which appear to be
> numeric,
> > unique id, and categorical values?
> >
> > 2.  based on N samples, , please run some analysis and tell me the top 5
> > feature columns in terms of predictive value using algorithm =
> RandonForest
> >
> > 3.  do a batch run, sample size = N, using these preprocessing steps list
> > {impute, scale, etc} and algorithms (knn, Decision Tree, etc} and
> give me a
> > report of accuracies obtain with each.
> >
> > In other words, we have a simple sample in the Tutorial demo where
> these
> > all run and then we compare outputs - why not automate these with a
> Python
> > Notebook UI of some sort?
> >
> > --
> > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >
>
>
>

Re: Apache Ignite ML & Python

[Alexey Zinoviev]

Agree with simple case, I think we could start from the simple poc for the
Python for ML in the next release

чт, 5 мар. 2020 г., 17:05 AG :

>
> Thanks, for the reply!
>
> It looks like a high-level API similar to Sklearn pipelines.
> In my opinion, for the first steps easier to add simple assess to gain the
> ability to run a simple model or simple preprocessor from python.
>
> According to your example:
> Here is raw dataset, already inside this cluster cache "myName", with
> Label column "MyLable".
>
> I want to run from notebook UI imputer and knn using python API. Export
> results to file storage as an example.
>
> In my opinion, the ability to create such a simple workflow should be our
> goal for the first time.
>
> Thank You!
>
> Best regards,
> Andrei Gavrilov.
>
> Sent with ProtonMail Secure Email.
>
> ‐‐‐ Original Message ‐‐‐
> On Wednesday, March 4, 2020 10:49 PM, kencottrell <
> ken.cottr...@gridgain.com> wrote:
>
> > Andrei,
> >
> > I am also working with Apache Ignite ML and am interested in providing
> > wrappers for Ignite ML API, but am wondering if instead of simply
> recreating
> > the low level Java API for ML inside Python, how about creating some
> higher
> > level services "Auto ML" workflow ? For example:
> >
> > 1.  here is raw dataset, already inside this cluster cache "myName", with
> > Label column "MyLable" , take N samples tell me which appear to be
> numeric,
> > unique id, and categorical values?
> >
> > 2.  based on N samples, , please run some analysis and tell me the top 5
> > feature columns in terms of predictive value using algorithm =
> RandonForest
> >
> > 3.  do a batch run, sample size = N, using these preprocessing steps list
> > {impute, scale, etc} and algorithms (knn, Decision Tree, etc} and
> give me a
> > report of accuracies obtain with each.
> >
> > In other words, we have a simple sample in the Tutorial demo where
> these
> > all run and then we compare outputs - why not automate these with a
> Python
> > Notebook UI of some sort?
> >
> > --
> > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >
>
>
>

[jira] [Created] (IGNITE-12753) Cache SSL contexts in SslContextFactory

[Denis Mekhanikov (Jira)]

Denis Mekhanikov created IGNITE-12753:
-

 Summary: Cache SSL contexts in SslContextFactory
 Key: IGNITE-12753
 URL: https://issues.apache.org/jira/browse/IGNITE-12753
 Project: Ignite
  Issue Type: Improvement
Reporter: Denis Mekhanikov
Assignee: Denis Mekhanikov


When SSL is enabled in a cluster, SslContextFactory#createSslContext is created 
every time when connections between nodes are created. It involves accessing 
key storage on disk. It may slow down creation of new communication connections 
and block striped pool threads if disks are slow.

SSL contexts are stateless and can be shared between threads, so it's safe to 
create an SSL context once and use the same instance every time.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Apache Ignite ML & Python

[AG]

Thanks, for the reply!

It looks like a high-level API similar to Sklearn pipelines.
In my opinion, for the first steps easier to add simple assess to gain the 
ability to run a simple model or simple preprocessor from python.

According to your example:
Here is raw dataset, already inside this cluster cache "myName", with Label 
column "MyLable".

I want to run from notebook UI imputer and knn using python API. Export results 
to file storage as an example.

In my opinion, the ability to create such a simple workflow should be our goal 
for the first time.

Thank You!

Best regards,
Andrei Gavrilov.

Sent with ProtonMail Secure Email.

‐‐‐ Original Message ‐‐‐
On Wednesday, March 4, 2020 10:49 PM, kencottrell  
wrote:

> Andrei,
>
> I am also working with Apache Ignite ML and am interested in providing
> wrappers for Ignite ML API, but am wondering if instead of simply recreating
> the low level Java API for ML inside Python, how about creating some higher
> level services "Auto ML" workflow ? For example:
>
> 1.  here is raw dataset, already inside this cluster cache "myName", with
> Label column "MyLable" , take N samples tell me which appear to be 
> numeric,
> unique id, and categorical values?
>
> 2.  based on N samples, , please run some analysis and tell me the top 5
> feature columns in terms of predictive value using algorithm = 
> RandonForest
>
> 3.  do a batch run, sample size = N, using these preprocessing steps list
> {impute, scale, etc} and algorithms (knn, Decision Tree, etc} and give me 
> a
> report of accuracies obtain with each.
>
> In other words, we have a simple sample in the Tutorial demo where these
> all run and then we compare outputs - why not automate these with a Python
> Notebook UI of some sort?
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

Re: Adding experimental support for Intel Optane DC Persistent Memory

[steve.hostett...@gmail.com]

Dear all,

while I probably cannot help a lot with the implementation, I have access to
a machine  with Optane memory. So if I can help in running benchmarks, let
me know.

Steve



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: Reference of local service.

[Andrey Gura]

Hi there,

what if we will generate proxy that collects service's metrics only if
service will implement some special interface? In such case:

- we won't affect existing services at all.
- we can impose additional requirement for services that want use
metrics out of box (i.e. service that implements our special interface
*must* also have own interface and only invocations of methods of this
interface will be taken into account for metrics collection).
- user always can use own metrics framework instead of our (just do
not implement this new special interface).

About metrics enabling/disabling. At present IGNITE-11927 doesn't
solve this problem. Just because there is no metrics implementation
for services :)
Anyway we should provide a way for configuring service metrics (in
sense of enabled/disabled) during service deploy. It's easy for cases
where deploy() methods have ServiceConfiguration as parameter. But
there are "short cut" methods like deployXxxSingleton(). I have ideas
how to solve this problem. For example we can introduce "short cut"
factory methods like nodeSingletonConfiguration(String name, Service
service) and clusterSingletonConfiguration(String name, Service
service). This methods will return configuration which has parameters
for given type of deployment and could be modified, e.g. metrics could
be enabled.

WDYT?

On Wed, Mar 4, 2020 at 8:42 PM Vladimir Steshin  wrote:
>
> Vyacheslav, Denis, hi again.
>
>
>
> >>> I agree with the proposal to introduce a new method which returns proxy
> include the case of locally deployed services.
>
>
>
> I see one is restricted to use an interface for service with
> IgniteServiceProcessor#serviceProxy(…):
>
>
>
> A.ensure(svcItf.isInterface(), "Service class must be an interface: " +
> svcItf);
>
>
>
> What if we change IgniteService#serviceProxy(...) so that it will return
> proxy everytime? That looks safe for user code. Doing so we might only
> deprecate IgniteService#service(...).
>
>
>
> вт, 3 мар. 2020 г., 11:03 Vyacheslav Daradur :
>
> > Denis, finally I understood your arguments about interfaces check, thank
> > you for the explanation.
> >
> > I agree with the proposal to introduce a new method which returns proxy
> > include the case of locally deployed services.
> >
> > Also, such a method should be able to work in mode "local services
> > preferred", perhaps with load-balancing (in case of multiple locally
> > deployed instances). This allows our end-users to reach better performance.
> >
> >
> >
> > On Mon, Mar 2, 2020 at 7:51 PM Denis Mekhanikov 
> > wrote:
> >
> > > Vyacheslav,
> > >
> > > You can't make service interfaces extend
> > > *org.apache.ignite.services.Service*. Currently it works perfectly if
> > > *org.apache.ignite.services.Service* and a user-defined interface are
> > > independent. This is actually the case in our current examples:
> > >
> > >
> > https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/servicegrid/SimpleMapService.java
> > > I mentioned the *Serializable* interface just as an example of an
> > interface
> > > that can be present, but it's not the one that is going to be called by a
> > > user.
> > >
> > > What I'm trying to say is that there is no way to say whether the service
> > > is going to be used through a proxy only, or usage of a local instance is
> > > also possible.
> > >
> > > Vladimir,
> > >
> > > I don't like the idea, that enabling or disabling of metrics will change
> > > the behaviour of the component you collect the metrics for. Such
> > behaviour
> > > is far from obvious.
> > >
> > > Nikolay,
> > >
> > > I agree, that such approach is valid and makes total sense. But making
> > the
> > > *IgniteServices#serviceProxy()* method always return a proxy instead of a
> > > local instance will change the public contract. The javadoc currently
> > says
> > > the following:
> > >
> > > > If service is available locally, then local instance is returned,
> > > > otherwise, a remote proxy is dynamically created and provided for the
> > > > specified service.
> > >
> > >
> > > I propose introducing a new method that will always return a service
> > proxy
> > > regardless of local availability, and deprecating *serviceProxy()* and
> > > *service()
> > > *methods. What do you think?
> > >
> > > Denis
> > >
> > > пн, 2 мар. 2020 г. в 16:08, Nikolay Izhikov :
> > >
> > > > Hello, Vladimir.
> > > >
> > > > > What if we just provide an option to disable service metrics at all?
> > > >
> > > > I don't think we should create an explicit property for service
> > metrics.
> > > > We will implement the way to disable any metrics in the scope of
> > > > IGNITE-11927 [1].
> > > >
> > > > > Usage of a proxy instead of service instances can lead to performance
> > > > > degradation for local instances, which is another argument against
> > such
> > > > change.
> > > >
> > > > As far as I know, many and many modern frameworks use a proxy approach.
> > > > Just to name one - 

Re: Ignite 2.8 documentation

[Anton Vinogradov]

Artem, great!

Some minors:

>> read operations become more costly for caches with backup copies.
Since it makes sense only for cache with backups, can we say something like
"read operations become at least 2 times costly since backups checked as
well"

>> for atomic caches, a consistency violation exception is thrown.
... after N checks, where N is 3 by default and can be set by
"IGNITE_NEAR_GET_MAX_REMAPS" system property.

Also, need to mention that each found violation triggers event with type ==
org.apache.ignite.events.EventType#EVT_CONSISTENCY_VIOLATION includes
org.apache.ignite.events.CacheConsistencyViolationEvent which should be
used for rechecking of repait results.

On Thu, Mar 5, 2020 at 3:51 PM Artem Budnikov 
wrote:

> Anton,
>
> Could you please review the page about Read Rapair?
>
> https://apacheignite.readme.io/docs/read-repair
>
> -Artem
>
> On 05.03.2020 12:20, Artem Budnikov wrote:
> > OK, I'll recreate it.
> >
> > Nikolay, please let me know when you are finished with the Metrics and
> > system views documentation. I'm done with the list of docs we
> > identified in this thread and want to publish v. 2.8.
> >
> > -Artem
> >
> > On 05.03.2020 11:55, Maxim Muzafarov wrote:
> >> Artem,
> >>
> >> I've created that. It is not public and can be safely removed since
> >> it's a full copy of 2.7.6 (at that moment)
> >>
> >> On Thu, 5 Mar 2020 at 11:53, Artem Budnikov
> >>  wrote:
> >>> Guys,
> >>>
> >>> I see that there is already version 2.8 for Ignite docs on readme.io.
> >>> Who created it and when? I've changed some pages under 2.7.6 version
> >>> without knowing that there is a newer version.
> >>>
> >>> -Artem
> >>>
> >>> On 05.03.2020 11:45, Artem Budnikov wrote:
>  I'm confused. Igor, could you please double check?
> 
>  -Artem
> 
>  On 05.03.2020 04:15, Ivan Pavlukhin wrote:
> >> That's right, only C++ and .NET clients have partition awareness
> > Are your sure? Was not the feature implemented for java thin client
> > in [1]?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-11898
> >
> > Best regards,
> > Ivan Pavlukhin
> >
> > ср, 4 мар. 2020 г. в 18:18, Denis Magda :
> >> Maxim,
> >>
> >> Yes, it's preferable to have metrics pages fully completed even
> >> though the
> >> feature is an experimental state. We want to encourage to try it out
> >> and
> >> switch to the new APIs eventually. Without technical instructions
> >> available
> >> our users will have a hard time checking the new capabilities.
> >>
> >> Nikolay, thanks a lot!
> >>
> >> -
> >> Denis
> >>
> >>
> >> On Wed, Mar 4, 2020 at 8:52 AM Nikolay Izhikov  >
> >> wrote:
> >>
> >>> I think yes.
> >>>
> >>> I will prepare documentation shortly.
> >>>
>  4 марта 2020 г., в 17:50, Maxim Muzafarov 
> >>> написал(а):
>  Folks,
> 
> 
>  Do we need a fully complete public documentation for metrics
>  marked
>  with @ExperimentalFeature? The API can significantly be changed.
> 
>  On Wed, 4 Mar 2020 at 17:10, Artem Budnikov
>  
> >>> wrote:
> > The only feature that is left is "WAL page compression"
> >
> > The three other features are  limitations that were present in
> > Ignite
> > 2.7 and now they are removed. Since they were never mentioned
> > in the
> > docs, there is nothing to do.
> >
> > -Artem
> >
> > On 04.03.2020 17:02, Denis Magda wrote:
> >>> I'll work on the following items today and tomorrow:
> >>>
> >>>  - JDBC: Support for query cancellation
> >>>
> >>>
> >>>  - JDBC: Support for query timeout
> >>>
> >>>
> >>>  - suspend/resume for pessimistic transactions
> >>>
> >>>
> >>>  - WAL page compression
> >>>
> >>>
> >> Artem, are these the only tickets left apart from the metrics &
> >> monitoring? @Nikolay
> >> Izhikov  how soon will you be able to
> >> finish the
> >> metrics documentation pages?
> >>
> >> -
> >> Denis
> >>
> >>
> >> On Wed, Mar 4, 2020 at 2:55 AM Artem Budnikov
> >> 
> >> wrote:
> >>
> >>> Hi everyone,
> >>>
> >>> I have created the docs for the following items so far:
> >>>
> >>>  -Default Ignite work dir location
> >>>
> >>>
> >>>
> https://apacheignite.readme.io/docs/getting-started-28#section-setting-up-work-directory
> >>>
> >>>
> >>>  - Baseline auto-adjust feature
> >>>
> >>>
> >>>
> https://apacheignite.readme.io/docs/baseline-topology-28#section-baseline-topology-autoadjustment
> >>>
> 

[jira] [Created] (IGNITE-12752) Pass thin clients SSL certificates to security

[Ilya Kasnacheev (Jira)]

Ilya Kasnacheev created IGNITE-12752:


 Summary: Pass thin clients SSL certificates to security
 Key: IGNITE-12752
 URL: https://issues.apache.org/jira/browse/IGNITE-12752
 Project: Ignite
  Issue Type: Bug
  Components: security
Reporter: Ilya Kasnacheev


Add SSL certificates to AuthenticationContext, so, authenticators can make 
additional checks based on SSL certificates.

Please note that this is *Phase I* which only concerns all flavors of thin 
clients, but not server/client nodes! It includes JDBC, old and new flavor of 
Ignite thin client, control.sh and REST.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Ignite 2.8 documentation

[Artem Budnikov]

Anton,

Could you please review the page about Read Rapair?

https://apacheignite.readme.io/docs/read-repair

-Artem

On 05.03.2020 12:20, Artem Budnikov wrote:

OK, I'll recreate it.

Nikolay, please let me know when you are finished with the Metrics and 
system views documentation. I'm done with the list of docs we 
identified in this thread and want to publish v. 2.8.


-Artem

On 05.03.2020 11:55, Maxim Muzafarov wrote:

Artem,

I've created that. It is not public and can be safely removed since
it's a full copy of 2.7.6 (at that moment)

On Thu, 5 Mar 2020 at 11:53, Artem Budnikov 
 wrote:

Guys,

I see that there is already version 2.8 for Ignite docs on readme.io.
Who created it and when? I've changed some pages under 2.7.6 version
without knowing that there is a newer version.

-Artem

On 05.03.2020 11:45, Artem Budnikov wrote:

I'm confused. Igor, could you please double check?

-Artem

On 05.03.2020 04:15, Ivan Pavlukhin wrote:

That's right, only C++ and .NET clients have partition awareness

Are your sure? Was not the feature implemented for java thin client
in [1]?

[1] https://issues.apache.org/jira/browse/IGNITE-11898

Best regards,
Ivan Pavlukhin

ср, 4 мар. 2020 г. в 18:18, Denis Magda :

Maxim,

Yes, it's preferable to have metrics pages fully completed even
though the
feature is an experimental state. We want to encourage to try it out
and
switch to the new APIs eventually. Without technical instructions
available
our users will have a hard time checking the new capabilities.

Nikolay, thanks a lot!

-
Denis


On Wed, Mar 4, 2020 at 8:52 AM Nikolay Izhikov 
wrote:


I think yes.

I will prepare documentation shortly.


4 марта 2020 г., в 17:50, Maxim Muzafarov 

написал(а):

Folks,


Do we need a fully complete public documentation for metrics 
marked

with @ExperimentalFeature? The API can significantly be changed.

On Wed, 4 Mar 2020 at 17:10, Artem Budnikov


wrote:

The only feature that is left is "WAL page compression"

The three other features are  limitations that were present in
Ignite
2.7 and now they are removed. Since they were never mentioned 
in the

docs, there is nothing to do.

-Artem

On 04.03.2020 17:02, Denis Magda wrote:

I'll work on the following items today and tomorrow:

 - JDBC: Support for query cancellation


 - JDBC: Support for query timeout


 - suspend/resume for pessimistic transactions


 - WAL page compression



Artem, are these the only tickets left apart from the metrics &
monitoring? @Nikolay
Izhikov  how soon will you be able to
finish the
metrics documentation pages?

-
Denis


On Wed, Mar 4, 2020 at 2:55 AM Artem Budnikov

wrote:


Hi everyone,

I have created the docs for the following items so far:

 -    Default Ignite work dir location


https://apacheignite.readme.io/docs/getting-started-28#section-setting-up-work-directory 




 - Baseline auto-adjust feature


https://apacheignite.readme.io/docs/baseline-topology-28#section-baseline-topology-autoadjustment 




 - Cluster (de)activation events documentation


https://apacheignite.readme.io/docs/baseline-topology-28#section-cluster-activationdeactivation-events 




 - Remove SqlQuery documentation
 done

 - Partition awareness for thin clients


https://apacheignite-net.readme.io/docs/thin-client-28#section-partition-awareness 



https://apacheignite-cpp.readme.io/docs/thin-client-28#section-partition-awareness 




 - Transactions support in java thin client


https://apacheignite.readme.io/docs/key-value-28#section-client-transactions 




 - KILL QUERY command


https://dash.readme.io/project/apacheignite-sql/v2.7.6/docs/kill-query 


 - Move rebalance configuration properties to the
 IgniteConfiguration level


https://apacheignite.readme.io/docs/data-rebalancing-28#section-configuration 




 - Renamed IGNITE schema to SYS
 done

I'll work on the following items today and tomorrow:

 - JDBC: Support for query cancellation
 - JDBC: Support for query timeout
 - suspend/resume for pessimistic transactions
 - WAL page compression


Nikolay,

How is your progress with the Metrics and System Views
documentation?


Artem Budnikov
Technical Writergridgain.com

On 03.03.2020 19:06, Maxim Muzafarov wrote:

Folks,

This is in the `Ignite Core` section, so I think mostly 
related to

java functionality and thin client protocol extension [1].
Sorry for
not being too accurate with this statement
.
The same change is mentioned for C++ and .NET sections of the
release
notes (but not for other thin clients).


[1] https://issues.apache.org/jira/browse/IGNITE-11898

On Tue, 3 Mar 2020 at 17:33, Igor Sapego  <

isap...@apache.org> wrote:
That's right, only C++ and .NET clients have partition 
awareness


Best Regards,
Igor


On Tue, Mar 3, 2020 at 5:02 PM Artem Budnikov




wrote:


Hi everyone,

Looks like the following line from the Ignite 2.8 

Re: Node.JS, PHP, Python API references for Ignite 2.8 release

[Igor Sapego]

Guys, I'm going to take this task and try to fix mentioned issues by the
next release.

Best Regards,
Igor


On Thu, Feb 27, 2020 at 9:40 PM Maxim Muzafarov  wrote:

> Folks,
>
> It seems that we have a lack of knowledge at the platforms README file [1].
> Should we also update it?
>
> [1]
> https://github.com/apache/ignite/blob/master/modules/platforms/README.txt
>
> On Wed, 26 Feb 2020 at 21:42, Denis Magda  wrote:
> >
> > Igor Sapego, Igniters,
> >
> > I've been working on Ignite website improvements that will be introduced
> > and contributed later and found that we still don't generate API
> references
> > for Node.JS, PHP, Python. The APIs are not even shipped withing binary
> > packages.
> >
> > Do our scripts for build/release generate those APIs? Can we start
> > uploading them to the website under this path?
> > https://svn.apache.org/repos/asf/ignite/site/trunk/releases
> >
> > -
> > Denis
>

[jira] [Created] (IGNITE-12751) callAsync(jobs, rdc) performance degrades quadratically as jobs.size() grows

[Ilya Kasnacheev (Jira)]

Ilya Kasnacheev created IGNITE-12751:


 Summary: callAsync(jobs, rdc) performance degrades quadratically 
as jobs.size() grows
 Key: IGNITE-12751
 URL: https://issues.apache.org/jira/browse/IGNITE-12751
 Project: Ignite
  Issue Type: Bug
  Components: compute
Affects Versions: 2.7.6
Reporter: Ilya Kasnacheev
 Attachments: word-count-reproducer.zip

Please consider attached reproducer and linked report.

{{compute.callAsync(jobs, reducer);
Result [res=33, tookMs=81, jobs=5] //warm up
Result [res=99, tookMs=21, jobs=15]
Result [res=330, tookMs=22, jobs=50]
Result [res=990, tookMs=57, jobs=150]
Result [res=3300, tookMs=146, jobs=500]
Result [res=9900, tookMs=231, jobs=1500]
Result [res=33000, tookMs=840, jobs=5000]
Result [res=99000, tookMs=6965, jobs=15000]
Result [res=33, tookMs=118394, jobs=5]}}

As soon jobs.size() grows past 5000, performance begins to degrade 
quadratically.

I don't expect that it will be completely linear, but I would assume that it 
should stay linear-ish until size() hits at least 10, given that we see 
clusters of 100 nodes and it's not unthinkable to expect 1000 jobs to be run on 
each node. 5000 jobs (which still give OK performance) / 100 nodes is just 50 
jobs per node, which becomes limiting factor.

Linked question also mentions OOM event, which may be caused of intermediate 
storage of (N^2) data.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

RE: [DISCUSSION] Release Apache Ignite 2.8.0 RC1

[Sergey-A Kosarev]

Classification: Public

There are no RC anymore, I believe. It's now just 2.8.0 (as in Maven Central)

Thank you to all involved in the release guys, especially Maxim!

Kind regards,
Sergey Kosarev

-Original Message-
From: Ivan Pavlukhin [mailto:vololo...@gmail.com]
Sent: 05 March 2020 12:49
To: dev 
Subject: Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1

Wow. I impressed that users try out even RC.

Best regards,
Ivan Pavlukhin

чт, 5 мар. 2020 г. в 12:01, Maxim Muzafarov :
>
> Ivan,
>
> I think `final steps` is mostly related not to binary\sources itself
> but for the announce message. All documentation must be up-to-date
> when it occurs.
>
> But I think you're right that users already using a new version of
> Apache Ignite without announcing. We've got some questions\bugs on
> user-list already [1] [2].
>
>
> [1]
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-2-8-get-exceptio
> n-while-use-batch-insert-in-streaming-mode-td31614.html
> [2]
> http://apache-ignite-users.70518.x6.nabble.com/Zookeeper-discovery-wit
> h-Ignite-2-8-0-class-NoClassDefFound-td31615.html
>
> On Thu, 5 Mar 2020 at 11:51, Ivan Pavlukhin  wrote:
> >
> > Maxim,
> >
> > You did the great job! Many thanks for that!
> >
> > I suppose there is some confusion with terminology. I understand
> > that Ignite release process has several final steps. And for example
> > binaries can be published before documentation is ready and it is ok
> > (I suppose opposite will make no harm as well). My personal
> > understanding is that a product version "is considered released"
> > once a user can receive and use it tagged with the target version.
> > In other words, once it is generally available (GA). And we should
> > not force users to check product web sites and other announcement
> > channels to check that a package available in the Maven repository
> > is valid. For someone it might be just "keeping all dependencies up to 
> > date".
> >
> > Best regards,
> > Ivan Pavlukhin
> >
> > чт, 5 мар. 2020 г. в 09:33, Maxim Muzafarov :
> > >
> > > Ivan,
> > >
> > > I did something wrong?
> > > What should I do to release all effectively?
> > >
> > > On Thu, 5 Mar 2020 at 08:23, Denis Magda  wrote:
> > > >
> > > > Ivan,
> > > >
> > > > The release of binaries is one of the final steps. There are
> > > > still several steps left before we can consider all the 2.8 release 
> > > > activities completed:
> > > > https://cwiki.apache.org/confluence/display/IGNITE/Release+Proce
> > > > ss#ReleaseProcess-6.3.4.Releasedocumentaiononreadme.io
> > > > https://cwiki.apache.org/confluence/display/IGNITE/Release+Proce
> > > > ss#ReleaseProcess-6.4.Announcerelease
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Wed, Mar 4, 2020 at 7:57 PM Ivan Pavlukhin  
> > > > wrote:
> > > >
> > > > > Denis, folks,
> > > > >
> > > > > As it was already mentioned 2.8 is already in maven [1].
> > > > > Consequently "effectively" released. In my opinion it is not
> > > > > correct to consider release in progress.
> > > > >
> > > > > Agree that 2.8.1 sounds as a proper option here.
> > > > >
> > > > > [1]
> > > > > https://mvnrepository.com/artifact/org.apache.ignite/ignite-co
> > > > > re/2.8.0
> > > > >
> > > > > Best regards,
> > > > > Ivan Pavlukhin
> > > > >
> > > > > ср, 4 мар. 2020 г. в 17:08, Denis Magda :
> > > > > >
> > > > > > Dmitriy,
> > > > > >
> > > > > > Ignite 2.8 will be announced as soon as the documentation is
> > > > > > prepared and released. Until then, the release is still considered 
> > > > > > under progress.
> > > > > Here
> > > > > > is a dedicated discussion for documentation activities. We
> > > > > > should be
> > > > > ready
> > > > > > soon:
> > > > > >
> > > > > http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-2
> > > > > -8-documentation-td46008.html
> > > > > >
> > > > > > Also, I've started a discussion related to our announcement
> > > > > > steps. Please check it out and get involved:
> > > > > >
> > > > > http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-2
> > > > > -8-announcement-plan-td46238.html
> > > > > >
> > > > > > -
> > > > > > Denis
> > > > > >
> > > > > >
> > > > > > On Wed, Mar 4, 2020 at 6:29 AM Dmitriy Pavlov
> > > > > > 
> > > > > wrote:
> > > > > >
> > > > > > > Hi Folks,
> > > > > > >
> > > > > > > Unfortunately
> > > > > > > https://dist.apache.org/repos/dist/release/ignite/2.8.0/
> > > > > is
> > > > > > > already there, so release was happened. I'm not sure it is
> > > > > > > a good idea
> > > > > to
> > > > > > > substitute binaries later.  Was it announced to announce@ ?
> > > > > > >
> > > > > > > IMO, 2.8.1 is ok for that case.
> > > > > > >
> > > > > > > Sincerely,
> > > > > > > Dmitriy Pavlov
> > > > > > >
> > > > > > > ср, 4 мар. 2020 г. в 11:29, Nikolay Izhikov :
> > > > > > >
> > > > > > > > Pavel, ++1.
> > > > > > > >
> > > > > > > > > 4 марта 2020 г., в 11:20, Pavel Tupitsyn
> > > > > > > > > 
> > > > > > > > написал(а):
> > > > > > > > >
> > > > > > > > > 

Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1

[Nikolay Izhikov]

It shows huge interest in the Ignite from users.
So I think we should significantly shorten release cycle in the first place.

> 5 марта 2020 г., в 13:08, Alexey Goncharuk  
> написал(а):
> 
> Hm, actually this is something we've never seen before. I thing we should
> definitely reconsider release vote timings as Ilya suggested to collect
> users feedback for a longer period of time.
> 
> чт, 5 мар. 2020 г. в 12:49, Ivan Pavlukhin :
> 
>> Wow. I impressed that users try out even RC.
>> 
>> Best regards,
>> Ivan Pavlukhin
>> 
>> чт, 5 мар. 2020 г. в 12:01, Maxim Muzafarov :
>>> 
>>> Ivan,
>>> 
>>> I think `final steps` is mostly related not to binary\sources itself
>>> but for the announce message. All documentation must be up-to-date
>>> when it occurs.
>>> 
>>> But I think you're right that users already using a new version of
>>> Apache Ignite without announcing. We've got some questions\bugs on
>>> user-list already [1] [2].
>>> 
>>> 
>>> [1]
>> http://apache-ignite-users.70518.x6.nabble.com/Ignite-2-8-get-exception-while-use-batch-insert-in-streaming-mode-td31614.html
>>> [2]
>> http://apache-ignite-users.70518.x6.nabble.com/Zookeeper-discovery-with-Ignite-2-8-0-class-NoClassDefFound-td31615.html
>>> 
>>> On Thu, 5 Mar 2020 at 11:51, Ivan Pavlukhin  wrote:
 
 Maxim,
 
 You did the great job! Many thanks for that!
 
 I suppose there is some confusion with terminology. I understand that
 Ignite release process has several final steps. And for example
 binaries can be published before documentation is ready and it is ok
 (I suppose opposite will make no harm as well). My personal
 understanding is that a product version "is considered released" once
 a user can receive and use it tagged with the target version. In other
 words, once it is generally available (GA). And we should not force
 users to check product web sites and other announcement channels to
 check that a package available in the Maven repository is valid. For
 someone it might be just "keeping all dependencies up to date".
 
 Best regards,
 Ivan Pavlukhin
 
 чт, 5 мар. 2020 г. в 09:33, Maxim Muzafarov :
> 
> Ivan,
> 
> I did something wrong?
> What should I do to release all effectively?
> 
> On Thu, 5 Mar 2020 at 08:23, Denis Magda  wrote:
>> 
>> Ivan,
>> 
>> The release of binaries is one of the final steps. There are still
>> several
>> steps left before we can consider all the 2.8 release activities
>> completed:
>> 
>> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-6.3.4.Releasedocumentaiononreadme.io
>> 
>> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-6.4.Announcerelease
>> 
>> -
>> Denis
>> 
>> 
>> On Wed, Mar 4, 2020 at 7:57 PM Ivan Pavlukhin 
>> wrote:
>> 
>>> Denis, folks,
>>> 
>>> As it was already mentioned 2.8 is already in maven [1].
>> Consequently
>>> "effectively" released. In my opinion it is not correct to
>> consider
>>> release in progress.
>>> 
>>> Agree that 2.8.1 sounds as a proper option here.
>>> 
>>> [1]
>> https://mvnrepository.com/artifact/org.apache.ignite/ignite-core/2.8.0
>>> 
>>> Best regards,
>>> Ivan Pavlukhin
>>> 
>>> ср, 4 мар. 2020 г. в 17:08, Denis Magda :
 
 Dmitriy,
 
 Ignite 2.8 will be announced as soon as the documentation is
>> prepared and
 released. Until then, the release is still considered under
>> progress.
>>> Here
 is a dedicated discussion for documentation activities. We
>> should be
>>> ready
 soon:
 
>>> 
>> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-2-8-documentation-td46008.html
 
 Also, I've started a discussion related to our announcement
>> steps. Please
 check it out and get involved:
 
>>> 
>> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-2-8-announcement-plan-td46238.html
 
 -
 Denis
 
 
 On Wed, Mar 4, 2020 at 6:29 AM Dmitriy Pavlov <
>> dpav...@apache.org>
>>> wrote:
 
> Hi Folks,
> 
> Unfortunately
>> https://dist.apache.org/repos/dist/release/ignite/2.8.0/
>>> is
> already there, so release was happened. I'm not sure it is a
>> good idea
>>> to
> substitute binaries later.  Was it announced to announce@ ?
> 
> IMO, 2.8.1 is ok for that case.
> 
> Sincerely,
> Dmitriy Pavlov
> 
> ср, 4 мар. 2020 г. в 11:29, Nikolay Izhikov <
>> nizhi...@apache.org>:
> 
>> Pavel, ++1.
>> 
>>> 4 марта 2020 г., в 11:20, Pavel Tupitsyn <
>> ptupit...@apache.org>
>> написал(а):
>>> 
>>> Igniters,
>>> 
>>> The release has already 

Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1

[Alexey Goncharuk]

Hm, actually this is something we've never seen before. I thing we should
definitely reconsider release vote timings as Ilya suggested to collect
users feedback for a longer period of time.

чт, 5 мар. 2020 г. в 12:49, Ivan Pavlukhin :

> Wow. I impressed that users try out even RC.
>
> Best regards,
> Ivan Pavlukhin
>
> чт, 5 мар. 2020 г. в 12:01, Maxim Muzafarov :
> >
> > Ivan,
> >
> > I think `final steps` is mostly related not to binary\sources itself
> > but for the announce message. All documentation must be up-to-date
> > when it occurs.
> >
> > But I think you're right that users already using a new version of
> > Apache Ignite without announcing. We've got some questions\bugs on
> > user-list already [1] [2].
> >
> >
> > [1]
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-2-8-get-exception-while-use-batch-insert-in-streaming-mode-td31614.html
> > [2]
> http://apache-ignite-users.70518.x6.nabble.com/Zookeeper-discovery-with-Ignite-2-8-0-class-NoClassDefFound-td31615.html
> >
> > On Thu, 5 Mar 2020 at 11:51, Ivan Pavlukhin  wrote:
> > >
> > > Maxim,
> > >
> > > You did the great job! Many thanks for that!
> > >
> > > I suppose there is some confusion with terminology. I understand that
> > > Ignite release process has several final steps. And for example
> > > binaries can be published before documentation is ready and it is ok
> > > (I suppose opposite will make no harm as well). My personal
> > > understanding is that a product version "is considered released" once
> > > a user can receive and use it tagged with the target version. In other
> > > words, once it is generally available (GA). And we should not force
> > > users to check product web sites and other announcement channels to
> > > check that a package available in the Maven repository is valid. For
> > > someone it might be just "keeping all dependencies up to date".
> > >
> > > Best regards,
> > > Ivan Pavlukhin
> > >
> > > чт, 5 мар. 2020 г. в 09:33, Maxim Muzafarov :
> > > >
> > > > Ivan,
> > > >
> > > > I did something wrong?
> > > > What should I do to release all effectively?
> > > >
> > > > On Thu, 5 Mar 2020 at 08:23, Denis Magda  wrote:
> > > > >
> > > > > Ivan,
> > > > >
> > > > > The release of binaries is one of the final steps. There are still
> several
> > > > > steps left before we can consider all the 2.8 release activities
> completed:
> > > > >
> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-6.3.4.Releasedocumentaiononreadme.io
> > > > >
> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-6.4.Announcerelease
> > > > >
> > > > > -
> > > > > Denis
> > > > >
> > > > >
> > > > > On Wed, Mar 4, 2020 at 7:57 PM Ivan Pavlukhin 
> wrote:
> > > > >
> > > > > > Denis, folks,
> > > > > >
> > > > > > As it was already mentioned 2.8 is already in maven [1].
> Consequently
> > > > > > "effectively" released. In my opinion it is not correct to
> consider
> > > > > > release in progress.
> > > > > >
> > > > > > Agree that 2.8.1 sounds as a proper option here.
> > > > > >
> > > > > > [1]
> https://mvnrepository.com/artifact/org.apache.ignite/ignite-core/2.8.0
> > > > > >
> > > > > > Best regards,
> > > > > > Ivan Pavlukhin
> > > > > >
> > > > > > ср, 4 мар. 2020 г. в 17:08, Denis Magda :
> > > > > > >
> > > > > > > Dmitriy,
> > > > > > >
> > > > > > > Ignite 2.8 will be announced as soon as the documentation is
> prepared and
> > > > > > > released. Until then, the release is still considered under
> progress.
> > > > > > Here
> > > > > > > is a dedicated discussion for documentation activities. We
> should be
> > > > > > ready
> > > > > > > soon:
> > > > > > >
> > > > > >
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-2-8-documentation-td46008.html
> > > > > > >
> > > > > > > Also, I've started a discussion related to our announcement
> steps. Please
> > > > > > > check it out and get involved:
> > > > > > >
> > > > > >
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-2-8-announcement-plan-td46238.html
> > > > > > >
> > > > > > > -
> > > > > > > Denis
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Mar 4, 2020 at 6:29 AM Dmitriy Pavlov <
> dpav...@apache.org>
> > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Folks,
> > > > > > > >
> > > > > > > > Unfortunately
> https://dist.apache.org/repos/dist/release/ignite/2.8.0/
> > > > > > is
> > > > > > > > already there, so release was happened. I'm not sure it is a
> good idea
> > > > > > to
> > > > > > > > substitute binaries later.  Was it announced to announce@ ?
> > > > > > > >
> > > > > > > > IMO, 2.8.1 is ok for that case.
> > > > > > > >
> > > > > > > > Sincerely,
> > > > > > > > Dmitriy Pavlov
> > > > > > > >
> > > > > > > > ср, 4 мар. 2020 г. в 11:29, Nikolay Izhikov <
> nizhi...@apache.org>:
> > > > > > > >
> > > > > > > > > Pavel, ++1.
> > > > > > > > >
> > > > > > > > > > 4 марта 2020 г., в 11:20, Pavel Tupitsyn <
> ptupit...@apache.org>
> > > > > > > > > написал(а):
> > 

Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1

[Ivan Pavlukhin]

Wow. I impressed that users try out even RC.

Best regards,
Ivan Pavlukhin

чт, 5 мар. 2020 г. в 12:01, Maxim Muzafarov :
>
> Ivan,
>
> I think `final steps` is mostly related not to binary\sources itself
> but for the announce message. All documentation must be up-to-date
> when it occurs.
>
> But I think you're right that users already using a new version of
> Apache Ignite without announcing. We've got some questions\bugs on
> user-list already [1] [2].
>
>
> [1] 
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-2-8-get-exception-while-use-batch-insert-in-streaming-mode-td31614.html
> [2] 
> http://apache-ignite-users.70518.x6.nabble.com/Zookeeper-discovery-with-Ignite-2-8-0-class-NoClassDefFound-td31615.html
>
> On Thu, 5 Mar 2020 at 11:51, Ivan Pavlukhin  wrote:
> >
> > Maxim,
> >
> > You did the great job! Many thanks for that!
> >
> > I suppose there is some confusion with terminology. I understand that
> > Ignite release process has several final steps. And for example
> > binaries can be published before documentation is ready and it is ok
> > (I suppose opposite will make no harm as well). My personal
> > understanding is that a product version "is considered released" once
> > a user can receive and use it tagged with the target version. In other
> > words, once it is generally available (GA). And we should not force
> > users to check product web sites and other announcement channels to
> > check that a package available in the Maven repository is valid. For
> > someone it might be just "keeping all dependencies up to date".
> >
> > Best regards,
> > Ivan Pavlukhin
> >
> > чт, 5 мар. 2020 г. в 09:33, Maxim Muzafarov :
> > >
> > > Ivan,
> > >
> > > I did something wrong?
> > > What should I do to release all effectively?
> > >
> > > On Thu, 5 Mar 2020 at 08:23, Denis Magda  wrote:
> > > >
> > > > Ivan,
> > > >
> > > > The release of binaries is one of the final steps. There are still 
> > > > several
> > > > steps left before we can consider all the 2.8 release activities 
> > > > completed:
> > > > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-6.3.4.Releasedocumentaiononreadme.io
> > > > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-6.4.Announcerelease
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Wed, Mar 4, 2020 at 7:57 PM Ivan Pavlukhin  
> > > > wrote:
> > > >
> > > > > Denis, folks,
> > > > >
> > > > > As it was already mentioned 2.8 is already in maven [1]. Consequently
> > > > > "effectively" released. In my opinion it is not correct to consider
> > > > > release in progress.
> > > > >
> > > > > Agree that 2.8.1 sounds as a proper option here.
> > > > >
> > > > > [1] 
> > > > > https://mvnrepository.com/artifact/org.apache.ignite/ignite-core/2.8.0
> > > > >
> > > > > Best regards,
> > > > > Ivan Pavlukhin
> > > > >
> > > > > ср, 4 мар. 2020 г. в 17:08, Denis Magda :
> > > > > >
> > > > > > Dmitriy,
> > > > > >
> > > > > > Ignite 2.8 will be announced as soon as the documentation is 
> > > > > > prepared and
> > > > > > released. Until then, the release is still considered under 
> > > > > > progress.
> > > > > Here
> > > > > > is a dedicated discussion for documentation activities. We should be
> > > > > ready
> > > > > > soon:
> > > > > >
> > > > > http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-2-8-documentation-td46008.html
> > > > > >
> > > > > > Also, I've started a discussion related to our announcement steps. 
> > > > > > Please
> > > > > > check it out and get involved:
> > > > > >
> > > > > http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-2-8-announcement-plan-td46238.html
> > > > > >
> > > > > > -
> > > > > > Denis
> > > > > >
> > > > > >
> > > > > > On Wed, Mar 4, 2020 at 6:29 AM Dmitriy Pavlov 
> > > > > wrote:
> > > > > >
> > > > > > > Hi Folks,
> > > > > > >
> > > > > > > Unfortunately 
> > > > > > > https://dist.apache.org/repos/dist/release/ignite/2.8.0/
> > > > > is
> > > > > > > already there, so release was happened. I'm not sure it is a good 
> > > > > > > idea
> > > > > to
> > > > > > > substitute binaries later.  Was it announced to announce@ ?
> > > > > > >
> > > > > > > IMO, 2.8.1 is ok for that case.
> > > > > > >
> > > > > > > Sincerely,
> > > > > > > Dmitriy Pavlov
> > > > > > >
> > > > > > > ср, 4 мар. 2020 г. в 11:29, Nikolay Izhikov :
> > > > > > >
> > > > > > > > Pavel, ++1.
> > > > > > > >
> > > > > > > > > 4 марта 2020 г., в 11:20, Pavel Tupitsyn 
> > > > > > > > > 
> > > > > > > > написал(а):
> > > > > > > > >
> > > > > > > > > Igniters,
> > > > > > > > >
> > > > > > > > > The release has already happened.
> > > > > > > > > It is on our website [1], it is on Maven [2].
> > > > > > > > > People are using it, so I don't think we can do anything at 
> > > > > > > > > this
> > > > > point.
> > > > > > > > >
> > > > > > > > > Bugs happen, we can't delay the release forever because of 
> > > > > > > > > them.
> > > > > Let's
> > > > > > > > fix

Read load balancing, read-though, ttl and optimistic serializable transactions

[Alexey Goncharuk]

Igniters,

I have recently discovered [1] that Ignite can arrive in a state when an
optimistic serializable transaction can never be successfully committed
from a backup node [2].

In short, the root cause of this issue is that there are configurations
that allow a key to be stored on primary and backup nodes with different
versions. This is a fundamental design choice that made a while ago,
however, I am not sure if this is a right way to go. When primary and
backup versions differ and read load balancing is enabled, the read version
will always mismatch with primary version and optimistic serializable
transaction will always fail.

Here I wanted to discuss both short-term mitigation plan for the issue [2]
as well as a longer-term changes to replication protocol.

As a short-term solution for [2] I suggest to force reads from a primary
node inside optimistic serializable transactions. The question is whether
to enforce this behavior only if the cache has a 3-rd party persistence
storage or this behavior should be always enforced. Note that the version
mismatch may appear even without a 3-rd party persistence storage when an
expiry policy is used. However, in this case, the version mismatch is
time-bound to the TTL cleanup lag. Personally, I would go with always
enforcing primary-node reads inside an optimistic serializable transaction.

As a long-term solution which would eliminate the possibility of versions
desync on primary and backup nodes, I would suggest to revisit the
read-through and TTL expiry semantics. It looks like quite a lot of users
are actually struggling with the current implementation of read-through
because a miss does not load the value to all partition nodes [3]. As for
TTL, I remember it clearing up entries locally was a big issue for a proper
MVCC rebalance implementation (we ended up prohibiting TTL for MVCC caches).
I think it may be better to make read-through and entry expiry a
partition-wide operation with the underlying cache guarantees. For
read-through it is justified because a partition-wide operation penalty is
comparable with the cache store load anyway (otherwise, a 3rd party storage
makes little sense). For entries expiration it should not make any
difference because it happens in background anyways.

Any thoughts on the subject are very much appreciated.

--AG

[1]
http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-NodeOrder-in-GridCacheVersion-td46108.html
[2] https://issues.apache.org/jira/browse/IGNITE-12739
[3]
http://apache-ignite-developers.2346864.n4.nabble.com/Re-Read-through-not-working-as-expected-in-case-of-Replicated-cache-td46083.html

Re: Ignite 2.8 documentation

[Artem Budnikov]

OK, I'll recreate it.

Nikolay, please let me know when you are finished with the Metrics and 
system views documentation. I'm done with the list of docs we identified 
in this thread and want to publish v. 2.8.


-Artem

On 05.03.2020 11:55, Maxim Muzafarov wrote:

Artem,

I've created that. It is not public and can be safely removed since
it's a full copy of 2.7.6 (at that moment)

On Thu, 5 Mar 2020 at 11:53, Artem Budnikov  wrote:

Guys,

I see that there is already version 2.8 for Ignite docs on readme.io.
Who created it and when? I've changed some pages under 2.7.6 version
without knowing that there is a newer version.

-Artem

On 05.03.2020 11:45, Artem Budnikov wrote:

I'm confused. Igor, could you please double check?

-Artem

On 05.03.2020 04:15, Ivan Pavlukhin wrote:

That's right, only C++ and .NET clients have partition awareness

Are your sure? Was not the feature implemented for java thin client
in [1]?

[1] https://issues.apache.org/jira/browse/IGNITE-11898

Best regards,
Ivan Pavlukhin

ср, 4 мар. 2020 г. в 18:18, Denis Magda :

Maxim,

Yes, it's preferable to have metrics pages fully completed even
though the
feature is an experimental state. We want to encourage to try it out
and
switch to the new APIs eventually. Without technical instructions
available
our users will have a hard time checking the new capabilities.

Nikolay, thanks a lot!

-
Denis


On Wed, Mar 4, 2020 at 8:52 AM Nikolay Izhikov 
wrote:


I think yes.

I will prepare documentation shortly.


4 марта 2020 г., в 17:50, Maxim Muzafarov 

написал(а):

Folks,


Do we need a fully complete public documentation for metrics marked
with @ExperimentalFeature? The API can significantly be changed.

On Wed, 4 Mar 2020 at 17:10, Artem Budnikov


wrote:

The only feature that is left is "WAL page compression"

The three other features are  limitations that were present in
Ignite
2.7 and now they are removed. Since they were never mentioned in the
docs, there is nothing to do.

-Artem

On 04.03.2020 17:02, Denis Magda wrote:

I'll work on the following items today and tomorrow:

 - JDBC: Support for query cancellation


 - JDBC: Support for query timeout


 - suspend/resume for pessimistic transactions


 - WAL page compression



Artem, are these the only tickets left apart from the metrics &
monitoring? @Nikolay
Izhikov  how soon will you be able to
finish the
metrics documentation pages?

-
Denis


On Wed, Mar 4, 2020 at 2:55 AM Artem Budnikov

wrote:


Hi everyone,

I have created the docs for the following items so far:

 -Default Ignite work dir location



https://apacheignite.readme.io/docs/getting-started-28#section-setting-up-work-directory


 - Baseline auto-adjust feature



https://apacheignite.readme.io/docs/baseline-topology-28#section-baseline-topology-autoadjustment


 - Cluster (de)activation events documentation



https://apacheignite.readme.io/docs/baseline-topology-28#section-cluster-activationdeactivation-events


 - Remove SqlQuery documentation
 done

 - Partition awareness for thin clients



https://apacheignite-net.readme.io/docs/thin-client-28#section-partition-awareness

https://apacheignite-cpp.readme.io/docs/thin-client-28#section-partition-awareness


 - Transactions support in java thin client



https://apacheignite.readme.io/docs/key-value-28#section-client-transactions


 - KILL QUERY command



https://dash.readme.io/project/apacheignite-sql/v2.7.6/docs/kill-query

 - Move rebalance configuration properties to the
 IgniteConfiguration level



https://apacheignite.readme.io/docs/data-rebalancing-28#section-configuration


 - Renamed IGNITE schema to SYS
 done

I'll work on the following items today and tomorrow:

 - JDBC: Support for query cancellation
 - JDBC: Support for query timeout
 - suspend/resume for pessimistic transactions
 - WAL page compression


Nikolay,

How is your progress with the Metrics and System Views
documentation?


Artem Budnikov
Technical Writergridgain.com

On 03.03.2020 19:06, Maxim Muzafarov wrote:

Folks,

This is in the `Ignite Core` section, so I think mostly related to
java functionality and thin client protocol extension [1].
Sorry for
not being too accurate with this statement
.
The same change is mentioned for C++ and .NET sections of the
release
notes (but not for other thin clients).


[1] https://issues.apache.org/jira/browse/IGNITE-11898

On Tue, 3 Mar 2020 at 17:33, Igor Sapego  <

isap...@apache.org> wrote:

That's right, only C++ and .NET clients have partition awareness

Best Regards,
Igor


On Tue, Mar 3, 2020 at 5:02 PM Artem Budnikov




wrote:


Hi everyone,

Looks like the following line from the Ignite 2.8 release notes
is a

bit

of an overstatement and should be removed:


Added support best effort affinity for thin clients (partition

awareness)

Java thin client does not support partition 

Re: Security Subject of thin client on remote nodes

[Denis Garus]

Hi, guys!


I've prepared the IEP-41: Security Context of a thin client on remote nodes
[1]; take a look, please.

If there aren't any questions, I could create an issue and start work.


Ivan, could you be an IEP sponsor?


Thx!



   1.
   
https://cwiki.apache.org/confluence/display/IGNITE/IEP-41%3A+Security+Context+of+a+thin+client+on+remote+nodes


ср, 26 февр. 2020 г. в 12:42, Mikhail Petrov :

> Hi, Alexei.
>
> The ticket [1] describes the general problem for all thin client
> authorizations. Its particular case is described in the mentioned in [1]
> ticket [2] on the example of a JDBC client  with the reproducer attached.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-12589
> [2] https://issues.apache.org/jira/browse/IGNITE-12579
>
> On 26.02.2020 11:47, Alexei Scherbakov wrote:
> > Denis Garus,
> >
> > It is forbidden to remove any public IGNITE attribute without proper
> > deprecation steps.
> >
> > I have read the thread and still do not clearly see the problem. The
> subject id
> > is not required to be a node id.
> >
> > The referenced in the thread ticket [1] do not provide any reproducers.
> >
> > Can you properly demonstrate a broken scenario ?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-12589
> >
> > пт, 21 февр. 2020 г. в 13:35, Andrey Kuznetsov :
> >
> >> Hi, guys!
> >>
> >> The change suggested by Denis looks robust to me: it covers security
> >> subject handling by all kinds of clients/nodes at once. As for
> >> ATTR_SECURITY_SUBJECT_V2 attribute, it is really better to move it to
> >> plugin implementations to support backward compatibility with peer
> nodes of
> >> older versions. Obviously, cluster with security disabled will not
> suffer
> >> from attribute removal. Ignite core should know nothing about the
> specific
> >> way of security context propagation.
> >>
> >> Denis, could you please create Jira issue for your change?
> >>
> >> чт, 20 февр. 2020 г. в 17:01, Denis Garus :
> >>
>  I just transmitted security subjects for rest requests.
> >>> SecurityContext has an unlimited size so we can get significant
> overhead.
> >>> And we do not solve problems with other thin clients.
> >>>
>  If you remove ATTR_SECURITY_SUBJECT_V2, it breaks compatibility
> between
> >>> old
> >>> versions and new.
> >>>
> >>> I suggest removing ATTR_SECURITY_SUBJECT_V2 from Ignite's codebase, but
> >> for
> >>> compatibility, it can be used by a security plugin like in PoC.
> >>>
> >>> чт, 20 февр. 2020 г. в 16:47, Maksim Stepachev <
> >> maksim.stepac...@gmail.com
>  :
>  Yes, I said about it at 07.19.
> 
> 
> >>
> http://apache-ignite-developers.2346864.n4.nabble.com/Improvements-for-new-security-approach-td42698.html#a42708
>  And in my solution, I just transmitted security subjects for rest
> >>> requests.
>  If you remove ATTR_SECURITY_SUBJECT_V2, it breaks compatibility
> between
> >>> old
>  versions and new.
> 
>  чт, 20 февр. 2020 г. в 15:56, Denis Garus :
> 
> > Hi, Igniters!
> >
> >
> > At present, a security subject id is assumed to be node id.
> >
> > But when we are dealing with thin client, JDBC or REST subject id is
>  random
> > UUID. In this case, we cannot get the subject information on a remote
>  node,
> > and we get problems like these [1], [2].
> >
> > To fix the problem, we should spread the client session to the whole
> > cluster.
> >
> >
> > I want to suggest a solution to the problem.
> >
> >
> > First, we should get subject information using GridSecurityProcessor.
> >
> > How GridSecurityProcessor will retrieve a subject data, it is up to
>  plugin
> > developers.
> >
> >
> > Second, we should get rid of the assumption that a subject id is a
> >> node
>  id
> > and remove the ATTR_SECURITY_SUBJECT_V2 attribute.
> >
> >
> > I have prepared PoC PR [3] that:
> >
> > - places the existing logic of spreading security context to
> > GridSecurityProcessor;
> >
> > - uses GridSecurityProcessor to get SecurityContext.
> >
> >
> >
> > 1.
> >
> >
> >>
> http://apache-ignite-developers.2346864.n4.nabble.com/JDBC-thin-client-incorrect-security-context-td45929.html
> > 2. https://issues.apache.org/jira/browse/IGNITE-12589
> > 3. https://github.com/apache/ignite/pull/7375
> >
> >>
> >> --
> >> Best regards,
> >>Andrey Kuznetsov.
> >>
> >
>

Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1

[Maxim Muzafarov]

Ivan,

I think `final steps` is mostly related not to binary\sources itself
but for the announce message. All documentation must be up-to-date
when it occurs.

But I think you're right that users already using a new version of
Apache Ignite without announcing. We've got some questions\bugs on
user-list already [1] [2].


[1] 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-2-8-get-exception-while-use-batch-insert-in-streaming-mode-td31614.html
[2] 
http://apache-ignite-users.70518.x6.nabble.com/Zookeeper-discovery-with-Ignite-2-8-0-class-NoClassDefFound-td31615.html

On Thu, 5 Mar 2020 at 11:51, Ivan Pavlukhin  wrote:
>
> Maxim,
>
> You did the great job! Many thanks for that!
>
> I suppose there is some confusion with terminology. I understand that
> Ignite release process has several final steps. And for example
> binaries can be published before documentation is ready and it is ok
> (I suppose opposite will make no harm as well). My personal
> understanding is that a product version "is considered released" once
> a user can receive and use it tagged with the target version. In other
> words, once it is generally available (GA). And we should not force
> users to check product web sites and other announcement channels to
> check that a package available in the Maven repository is valid. For
> someone it might be just "keeping all dependencies up to date".
>
> Best regards,
> Ivan Pavlukhin
>
> чт, 5 мар. 2020 г. в 09:33, Maxim Muzafarov :
> >
> > Ivan,
> >
> > I did something wrong?
> > What should I do to release all effectively?
> >
> > On Thu, 5 Mar 2020 at 08:23, Denis Magda  wrote:
> > >
> > > Ivan,
> > >
> > > The release of binaries is one of the final steps. There are still several
> > > steps left before we can consider all the 2.8 release activities 
> > > completed:
> > > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-6.3.4.Releasedocumentaiononreadme.io
> > > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-6.4.Announcerelease
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Wed, Mar 4, 2020 at 7:57 PM Ivan Pavlukhin  wrote:
> > >
> > > > Denis, folks,
> > > >
> > > > As it was already mentioned 2.8 is already in maven [1]. Consequently
> > > > "effectively" released. In my opinion it is not correct to consider
> > > > release in progress.
> > > >
> > > > Agree that 2.8.1 sounds as a proper option here.
> > > >
> > > > [1] 
> > > > https://mvnrepository.com/artifact/org.apache.ignite/ignite-core/2.8.0
> > > >
> > > > Best regards,
> > > > Ivan Pavlukhin
> > > >
> > > > ср, 4 мар. 2020 г. в 17:08, Denis Magda :
> > > > >
> > > > > Dmitriy,
> > > > >
> > > > > Ignite 2.8 will be announced as soon as the documentation is prepared 
> > > > > and
> > > > > released. Until then, the release is still considered under progress.
> > > > Here
> > > > > is a dedicated discussion for documentation activities. We should be
> > > > ready
> > > > > soon:
> > > > >
> > > > http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-2-8-documentation-td46008.html
> > > > >
> > > > > Also, I've started a discussion related to our announcement steps. 
> > > > > Please
> > > > > check it out and get involved:
> > > > >
> > > > http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-2-8-announcement-plan-td46238.html
> > > > >
> > > > > -
> > > > > Denis
> > > > >
> > > > >
> > > > > On Wed, Mar 4, 2020 at 6:29 AM Dmitriy Pavlov 
> > > > wrote:
> > > > >
> > > > > > Hi Folks,
> > > > > >
> > > > > > Unfortunately 
> > > > > > https://dist.apache.org/repos/dist/release/ignite/2.8.0/
> > > > is
> > > > > > already there, so release was happened. I'm not sure it is a good 
> > > > > > idea
> > > > to
> > > > > > substitute binaries later.  Was it announced to announce@ ?
> > > > > >
> > > > > > IMO, 2.8.1 is ok for that case.
> > > > > >
> > > > > > Sincerely,
> > > > > > Dmitriy Pavlov
> > > > > >
> > > > > > ср, 4 мар. 2020 г. в 11:29, Nikolay Izhikov :
> > > > > >
> > > > > > > Pavel, ++1.
> > > > > > >
> > > > > > > > 4 марта 2020 г., в 11:20, Pavel Tupitsyn 
> > > > > > > написал(а):
> > > > > > > >
> > > > > > > > Igniters,
> > > > > > > >
> > > > > > > > The release has already happened.
> > > > > > > > It is on our website [1], it is on Maven [2].
> > > > > > > > People are using it, so I don't think we can do anything at this
> > > > point.
> > > > > > > >
> > > > > > > > Bugs happen, we can't delay the release forever because of them.
> > > > Let's
> > > > > > > fix
> > > > > > > > things in 2.8.1.
> > > > > > > >
> > > > > > > > [1] https://ignite.apache.org/download.cgi
> > > > > > > > [2] https://mvnrepository.com/artifact/org.apache.ignite
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Mar 4, 2020 at 10:06 AM Nikolay Izhikov <
> > > > nizhi...@apache.org>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > >> Hello, Igniters.
> > > > > > > >>
> > > > > > > >> Do we have a person who can fix this issue in the nearest 

Re: Ignite 2.8 documentation

[Maxim Muzafarov]

Artem,

I've created that. It is not public and can be safely removed since
it's a full copy of 2.7.6 (at that moment)

On Thu, 5 Mar 2020 at 11:53, Artem Budnikov  wrote:
>
> Guys,
>
> I see that there is already version 2.8 for Ignite docs on readme.io.
> Who created it and when? I've changed some pages under 2.7.6 version
> without knowing that there is a newer version.
>
> -Artem
>
> On 05.03.2020 11:45, Artem Budnikov wrote:
> > I'm confused. Igor, could you please double check?
> >
> > -Artem
> >
> > On 05.03.2020 04:15, Ivan Pavlukhin wrote:
> >>> That's right, only C++ and .NET clients have partition awareness
> >> Are your sure? Was not the feature implemented for java thin client
> >> in [1]?
> >>
> >> [1] https://issues.apache.org/jira/browse/IGNITE-11898
> >>
> >> Best regards,
> >> Ivan Pavlukhin
> >>
> >> ср, 4 мар. 2020 г. в 18:18, Denis Magda :
> >>> Maxim,
> >>>
> >>> Yes, it's preferable to have metrics pages fully completed even
> >>> though the
> >>> feature is an experimental state. We want to encourage to try it out
> >>> and
> >>> switch to the new APIs eventually. Without technical instructions
> >>> available
> >>> our users will have a hard time checking the new capabilities.
> >>>
> >>> Nikolay, thanks a lot!
> >>>
> >>> -
> >>> Denis
> >>>
> >>>
> >>> On Wed, Mar 4, 2020 at 8:52 AM Nikolay Izhikov 
> >>> wrote:
> >>>
>  I think yes.
> 
>  I will prepare documentation shortly.
> 
> > 4 марта 2020 г., в 17:50, Maxim Muzafarov 
>  написал(а):
> > Folks,
> >
> >
> > Do we need a fully complete public documentation for metrics marked
> > with @ExperimentalFeature? The API can significantly be changed.
> >
> > On Wed, 4 Mar 2020 at 17:10, Artem Budnikov
> > 
>  wrote:
> >> The only feature that is left is "WAL page compression"
> >>
> >> The three other features are  limitations that were present in
> >> Ignite
> >> 2.7 and now they are removed. Since they were never mentioned in the
> >> docs, there is nothing to do.
> >>
> >> -Artem
> >>
> >> On 04.03.2020 17:02, Denis Magda wrote:
>  I'll work on the following items today and tomorrow:
> 
>  - JDBC: Support for query cancellation
> 
> 
>  - JDBC: Support for query timeout
> 
> 
>  - suspend/resume for pessimistic transactions
> 
> 
>  - WAL page compression
> 
> 
> >>> Artem, are these the only tickets left apart from the metrics &
> >>> monitoring? @Nikolay
> >>> Izhikov  how soon will you be able to
> >>> finish the
> >>> metrics documentation pages?
> >>>
> >>> -
> >>> Denis
> >>>
> >>>
> >>> On Wed, Mar 4, 2020 at 2:55 AM Artem Budnikov
> >>> 
> >>> wrote:
> >>>
>  Hi everyone,
> 
>  I have created the docs for the following items so far:
> 
>  -Default Ignite work dir location
> 
> 
>  https://apacheignite.readme.io/docs/getting-started-28#section-setting-up-work-directory
> 
>  - Baseline auto-adjust feature
> 
> 
>  https://apacheignite.readme.io/docs/baseline-topology-28#section-baseline-topology-autoadjustment
> 
>  - Cluster (de)activation events documentation
> 
> 
>  https://apacheignite.readme.io/docs/baseline-topology-28#section-cluster-activationdeactivation-events
> 
>  - Remove SqlQuery documentation
>  done
> 
>  - Partition awareness for thin clients
> 
> 
>  https://apacheignite-net.readme.io/docs/thin-client-28#section-partition-awareness
> 
> 
>  https://apacheignite-cpp.readme.io/docs/thin-client-28#section-partition-awareness
> 
>  - Transactions support in java thin client
> 
> 
>  https://apacheignite.readme.io/docs/key-value-28#section-client-transactions
> 
>  - KILL QUERY command
> 
> 
>  https://dash.readme.io/project/apacheignite-sql/v2.7.6/docs/kill-query
>  - Move rebalance configuration properties to the
>  IgniteConfiguration level
> 
> 
>  https://apacheignite.readme.io/docs/data-rebalancing-28#section-configuration
> 
>  - Renamed IGNITE schema to SYS
>  done
> 
>  I'll work on the following items today and tomorrow:
> 
>  - JDBC: Support for query cancellation
>  - JDBC: Support for query timeout
>  - suspend/resume for pessimistic transactions
>  - WAL page compression
> 
> 
>  Nikolay,
> 
>  How is your progress with the Metrics and System Views
>  documentation?

Re: Ignite 2.8 documentation

[Artem Budnikov]

Guys,

I see that there is already version 2.8 for Ignite docs on readme.io. 
Who created it and when? I've changed some pages under 2.7.6 version 
without knowing that there is a newer version.


-Artem

On 05.03.2020 11:45, Artem Budnikov wrote:

I'm confused. Igor, could you please double check?

-Artem

On 05.03.2020 04:15, Ivan Pavlukhin wrote:

That's right, only C++ and .NET clients have partition awareness
Are your sure? Was not the feature implemented for java thin client 
in [1]?


[1] https://issues.apache.org/jira/browse/IGNITE-11898

Best regards,
Ivan Pavlukhin

ср, 4 мар. 2020 г. в 18:18, Denis Magda :

Maxim,

Yes, it's preferable to have metrics pages fully completed even 
though the
feature is an experimental state. We want to encourage to try it out 
and
switch to the new APIs eventually. Without technical instructions 
available

our users will have a hard time checking the new capabilities.

Nikolay, thanks a lot!

-
Denis


On Wed, Mar 4, 2020 at 8:52 AM Nikolay Izhikov  
wrote:



I think yes.

I will prepare documentation shortly.


4 марта 2020 г., в 17:50, Maxim Muzafarov 

написал(а):

Folks,


Do we need a fully complete public documentation for metrics marked
with @ExperimentalFeature? The API can significantly be changed.

On Wed, 4 Mar 2020 at 17:10, Artem Budnikov 


wrote:

The only feature that is left is "WAL page compression"

The three other features are  limitations that were present in 
Ignite

2.7 and now they are removed. Since they were never mentioned in the
docs, there is nothing to do.

-Artem

On 04.03.2020 17:02, Denis Magda wrote:

I'll work on the following items today and tomorrow:

    - JDBC: Support for query cancellation


    - JDBC: Support for query timeout


    - suspend/resume for pessimistic transactions


    - WAL page compression



Artem, are these the only tickets left apart from the metrics &
monitoring? @Nikolay
Izhikov  how soon will you be able to 
finish the

metrics documentation pages?

-
Denis


On Wed, Mar 4, 2020 at 2:55 AM Artem Budnikov 


wrote:


Hi everyone,

I have created the docs for the following items so far:

    -    Default Ignite work dir location


https://apacheignite.readme.io/docs/getting-started-28#section-setting-up-work-directory 


    - Baseline auto-adjust feature


https://apacheignite.readme.io/docs/baseline-topology-28#section-baseline-topology-autoadjustment 


    - Cluster (de)activation events documentation


https://apacheignite.readme.io/docs/baseline-topology-28#section-cluster-activationdeactivation-events 


    - Remove SqlQuery documentation
    done

    - Partition awareness for thin clients


https://apacheignite-net.readme.io/docs/thin-client-28#section-partition-awareness 



https://apacheignite-cpp.readme.io/docs/thin-client-28#section-partition-awareness 


    - Transactions support in java thin client


https://apacheignite.readme.io/docs/key-value-28#section-client-transactions 


    - KILL QUERY command



https://dash.readme.io/project/apacheignite-sql/v2.7.6/docs/kill-query

    - Move rebalance configuration properties to the
    IgniteConfiguration level


https://apacheignite.readme.io/docs/data-rebalancing-28#section-configuration 


    - Renamed IGNITE schema to SYS
    done

I'll work on the following items today and tomorrow:

    - JDBC: Support for query cancellation
    - JDBC: Support for query timeout
    - suspend/resume for pessimistic transactions
    - WAL page compression


Nikolay,

How is your progress with the Metrics and System Views 
documentation?



Artem Budnikov
Technical Writergridgain.com

On 03.03.2020 19:06, Maxim Muzafarov wrote:

Folks,

This is in the `Ignite Core` section, so I think mostly related to
java functionality and thin client protocol extension [1]. 
Sorry for

not being too accurate with this statement
.
The same change is mentioned for C++ and .NET sections of the 
release

notes (but not for other thin clients).


[1] https://issues.apache.org/jira/browse/IGNITE-11898

On Tue, 3 Mar 2020 at 17:33, Igor Sapego  <

isap...@apache.org> wrote:

That's right, only C++ and .NET clients have partition awareness

Best Regards,
Igor


On Tue, Mar 3, 2020 at 5:02 PM Artem Budnikov 




wrote:


Hi everyone,

Looks like the following line from the Ignite 2.8 release notes 
is a

bit

of an overstatement and should be removed:


Added support best effort affinity for thin clients (partition

awareness)

Java thin client does not support partition awareness. Nodejs and

python
thin clients from the distribution package do not have signs of 
this

feature as well.

Igor,

Could you please confirm or deny this?

-Artem

Artem Budnikov
Technical Writergridgain.com

On 03.03.2020 01:47, Denis Magda wrote:

Hi Alexey,

Thanks for updating the documentation. The update process is

cumbersome as
of now. What will happen is that we will be replacing the 
content of

the
current 

Re: [DISCUSSION] Release Apache Ignite 2.8.0 RC1

[Ivan Pavlukhin]

Maxim,

You did the great job! Many thanks for that!

I suppose there is some confusion with terminology. I understand that
Ignite release process has several final steps. And for example
binaries can be published before documentation is ready and it is ok
(I suppose opposite will make no harm as well). My personal
understanding is that a product version "is considered released" once
a user can receive and use it tagged with the target version. In other
words, once it is generally available (GA). And we should not force
users to check product web sites and other announcement channels to
check that a package available in the Maven repository is valid. For
someone it might be just "keeping all dependencies up to date".

Best regards,
Ivan Pavlukhin

чт, 5 мар. 2020 г. в 09:33, Maxim Muzafarov :
>
> Ivan,
>
> I did something wrong?
> What should I do to release all effectively?
>
> On Thu, 5 Mar 2020 at 08:23, Denis Magda  wrote:
> >
> > Ivan,
> >
> > The release of binaries is one of the final steps. There are still several
> > steps left before we can consider all the 2.8 release activities completed:
> > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-6.3.4.Releasedocumentaiononreadme.io
> > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-6.4.Announcerelease
> >
> > -
> > Denis
> >
> >
> > On Wed, Mar 4, 2020 at 7:57 PM Ivan Pavlukhin  wrote:
> >
> > > Denis, folks,
> > >
> > > As it was already mentioned 2.8 is already in maven [1]. Consequently
> > > "effectively" released. In my opinion it is not correct to consider
> > > release in progress.
> > >
> > > Agree that 2.8.1 sounds as a proper option here.
> > >
> > > [1] https://mvnrepository.com/artifact/org.apache.ignite/ignite-core/2.8.0
> > >
> > > Best regards,
> > > Ivan Pavlukhin
> > >
> > > ср, 4 мар. 2020 г. в 17:08, Denis Magda :
> > > >
> > > > Dmitriy,
> > > >
> > > > Ignite 2.8 will be announced as soon as the documentation is prepared 
> > > > and
> > > > released. Until then, the release is still considered under progress.
> > > Here
> > > > is a dedicated discussion for documentation activities. We should be
> > > ready
> > > > soon:
> > > >
> > > http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-2-8-documentation-td46008.html
> > > >
> > > > Also, I've started a discussion related to our announcement steps. 
> > > > Please
> > > > check it out and get involved:
> > > >
> > > http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-2-8-announcement-plan-td46238.html
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Wed, Mar 4, 2020 at 6:29 AM Dmitriy Pavlov 
> > > wrote:
> > > >
> > > > > Hi Folks,
> > > > >
> > > > > Unfortunately https://dist.apache.org/repos/dist/release/ignite/2.8.0/
> > > is
> > > > > already there, so release was happened. I'm not sure it is a good idea
> > > to
> > > > > substitute binaries later.  Was it announced to announce@ ?
> > > > >
> > > > > IMO, 2.8.1 is ok for that case.
> > > > >
> > > > > Sincerely,
> > > > > Dmitriy Pavlov
> > > > >
> > > > > ср, 4 мар. 2020 г. в 11:29, Nikolay Izhikov :
> > > > >
> > > > > > Pavel, ++1.
> > > > > >
> > > > > > > 4 марта 2020 г., в 11:20, Pavel Tupitsyn 
> > > > > > написал(а):
> > > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > The release has already happened.
> > > > > > > It is on our website [1], it is on Maven [2].
> > > > > > > People are using it, so I don't think we can do anything at this
> > > point.
> > > > > > >
> > > > > > > Bugs happen, we can't delay the release forever because of them.
> > > Let's
> > > > > > fix
> > > > > > > things in 2.8.1.
> > > > > > >
> > > > > > > [1] https://ignite.apache.org/download.cgi
> > > > > > > [2] https://mvnrepository.com/artifact/org.apache.ignite
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Mar 4, 2020 at 10:06 AM Nikolay Izhikov <
> > > nizhi...@apache.org>
> > > > > > wrote:
> > > > > > >
> > > > > > >> Hello, Igniters.
> > > > > > >>
> > > > > > >> Do we have a person who can fix this issue in the nearest time?
> > > > > > >> Right now IGNITE-12746 is Open and Unassigned.
> > > > > > >>
> > > > > > >>> 3 марта 2020 г., в 23:54, Denis Magda 
> > > > > написал(а):
> > > > > > >>>
> > > > > > >>> Ilya,
> > > > > > >>>
> > > > > > >>> Thanks for catching that and sharing. I wonder if there are any
> > > other
> > > > > > >>> blockers that were pushed to 2.8.x version. If there are a few
> > > then
> > > > > we
> > > > > > >> can
> > > > > > >>> produce 2.8.1 shortly, otherwise, it sounds reasonable to me to
> > > fix
> > > > > the
> > > > > > >>> issue with putAll, update the binaries and finish the process.
> > > > > > >>>
> > > > > > >>> -
> > > > > > >>> Denis
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> On Tue, Mar 3, 2020 at 2:34 PM Ilya Kasnacheev <
> > > > > > >> ilya.kasnach...@gmail.com>
> > > > > > >>> wrote:
> > > > > > >>>
> > > > > >  Hello!
> > > > > > 
> > > > > >  I have filed 

Re: Ignite 2.8 documentation

[Artem Budnikov]

I'm confused. Igor, could you please double check?

-Artem

On 05.03.2020 04:15, Ivan Pavlukhin wrote:

That's right, only C++ and .NET clients have partition awareness

Are your sure? Was not the feature implemented for java thin client in [1]?

[1] https://issues.apache.org/jira/browse/IGNITE-11898

Best regards,
Ivan Pavlukhin

ср, 4 мар. 2020 г. в 18:18, Denis Magda :

Maxim,

Yes, it's preferable to have metrics pages fully completed even though the
feature is an experimental state. We want to encourage to try it out and
switch to the new APIs eventually. Without technical instructions available
our users will have a hard time checking the new capabilities.

Nikolay, thanks a lot!

-
Denis


On Wed, Mar 4, 2020 at 8:52 AM Nikolay Izhikov  wrote:


I think yes.

I will prepare documentation shortly.


4 марта 2020 г., в 17:50, Maxim Muzafarov 

написал(а):

Folks,


Do we need a fully complete public documentation for metrics marked
with @ExperimentalFeature? The API can significantly be changed.

On Wed, 4 Mar 2020 at 17:10, Artem Budnikov 

wrote:

The only feature that is left is "WAL page compression"

The three other features are  limitations that were present in Ignite
2.7 and now they are removed. Since they were never mentioned in the
docs, there is nothing to do.

-Artem

On 04.03.2020 17:02, Denis Magda wrote:

I'll work on the following items today and tomorrow:

- JDBC: Support for query cancellation


- JDBC: Support for query timeout


- suspend/resume for pessimistic transactions


- WAL page compression



Artem, are these the only tickets left apart from the metrics &
monitoring? @Nikolay
Izhikov  how soon will you be able to finish the
metrics documentation pages?

-
Denis


On Wed, Mar 4, 2020 at 2:55 AM Artem Budnikov 
wrote:


Hi everyone,

I have created the docs for the following items so far:

-Default Ignite work dir location



https://apacheignite.readme.io/docs/getting-started-28#section-setting-up-work-directory

- Baseline auto-adjust feature



https://apacheignite.readme.io/docs/baseline-topology-28#section-baseline-topology-autoadjustment

- Cluster (de)activation events documentation



https://apacheignite.readme.io/docs/baseline-topology-28#section-cluster-activationdeactivation-events

- Remove SqlQuery documentation
done

- Partition awareness for thin clients



https://apacheignite-net.readme.io/docs/thin-client-28#section-partition-awareness



https://apacheignite-cpp.readme.io/docs/thin-client-28#section-partition-awareness

- Transactions support in java thin client



https://apacheignite.readme.io/docs/key-value-28#section-client-transactions

- KILL QUERY command



https://dash.readme.io/project/apacheignite-sql/v2.7.6/docs/kill-query

- Move rebalance configuration properties to the
IgniteConfiguration level



https://apacheignite.readme.io/docs/data-rebalancing-28#section-configuration

- Renamed IGNITE schema to SYS
done

I'll work on the following items today and tomorrow:

- JDBC: Support for query cancellation
- JDBC: Support for query timeout
- suspend/resume for pessimistic transactions
- WAL page compression


Nikolay,

How is your progress with the Metrics and System Views documentation?


Artem Budnikov
Technical Writergridgain.com

On 03.03.2020 19:06, Maxim Muzafarov wrote:

Folks,

This is in the `Ignite Core` section, so I think mostly related to
java functionality and thin client protocol extension [1]. Sorry for
not being too accurate with this statement
.
The same change is mentioned for C++ and .NET sections of the release
notes (but not for other thin clients).


[1] https://issues.apache.org/jira/browse/IGNITE-11898

On Tue, 3 Mar 2020 at 17:33, Igor Sapego  <

isap...@apache.org> wrote:

That's right, only C++ and .NET clients have partition awareness

Best Regards,
Igor


On Tue, Mar 3, 2020 at 5:02 PM Artem Budnikov 



wrote:


Hi everyone,

Looks like the following line from the Ignite 2.8 release notes is a

bit

of an overstatement and should be removed:


Added support best effort affinity for thin clients (partition

awareness)

Java thin client does not support partition awareness. Nodejs and

python

thin clients from the distribution package do not have signs of this
feature as well.

Igor,

Could you please confirm or deny this?

-Artem

Artem Budnikov
Technical Writergridgain.com

On 03.03.2020 01:47, Denis Magda wrote:

Hi Alexey,

Thanks for updating the documentation. The update process is

cumbersome as

of now. What will happen is that we will be replacing the content of

the

current pages (pre 2.8 pages) with the content from the 2.8 versions.

Once

the text is copied manually, a 2.8 version of the page will be

deleted. I

would advise Artem to do that this time and update the wiki page with

more

details:

Re: Distributed DMX Lighting Control With Apache Ignite.NET

[Pavel Tupitsyn]

Hi Mike, Denis,

The post series is great, so exciting to know that Ignite.NET is used in
such a cool way,
and across multiple APIs (Cache, Compute, Services, Messaging).

Any feedback will be greatly appreciated, please feel free to reach me over
this email address or through the user list.

Thanks,
Pavel


On Thu, Mar 5, 2020 at 12:58 AM Denis Magda  wrote:

> Hi Igniters,
>
> Let me introduce you to Mike James, who has been working on a gripping
> project that uses Apache Ignite as a backbone for a distributed DMX Control
> system. That's a use case of the entertainment industry and, hopefully,
> many rockstars will switch to the solution to produce light effects and
> shows during their performances ;)
>
> Check this two-part series by Mike:
>
>- Intro and use case:
>https://lightconsole.dev/2020/02/26/intro-to-dmx-control-systems/
>- Architecture:
>
> https://lightconsole.dev/2020/03/04/distributing-dmx-compute-with-apache-ignite/
>
> You might be interested to see how the system relies on various
> capabilities of Ignite to make the things working at scale - compute,
> storage, service grid, etc.
>
> @Pavel Tupitsyn, I would especially want to connect you with Mike, who can
> share his feedback in regards to our .NET component. Mike, stay in touch
> with Pavel, who is our main maintainer and contributor to .NET.
>
> -
> Denis
>

[jira] [Created] (IGNITE-12750) Correct the setting of the pool to build the index

[Kirill Tkalenko (Jira)]

Kirill Tkalenko created IGNITE-12750:


 Summary: Correct the setting of the pool to build the index
 Key: IGNITE-12750
 URL: https://issues.apache.org/jira/browse/IGNITE-12750
 Project: Ignite
  Issue Type: Bug
Reporter: Kirill Tkalenko
Assignee: Kirill Tkalenko
 Fix For: 2.9


There was a bug in setting up a ThreadPool for building an index, so it turns 
out that only 1 thread always works. We need more threads.
{code:java}
org.apache.ignite.internal.IgnitionEx.IgniteNamedInstance#buildIdxExecSvc{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)