Re: Class field ThreadLocal. Why not static?

2018-09-12 Thread Alexey Goncharuk
Maxim,

If multiple instances of Ignite is started in the same JVM and a user
thread will access first one instance of Ignite, then another, you will end
up with the static thread local holding the last WAL pointer from the
second grid. This is possible, for example, when a user thread commits a
transaction or runs an atomic update on a data node. Any access of the
first Ignite instance will have an invalid thread-local value.

вт, 11 сент. 2018 г. в 13:29, Maxim Muzafarov :

> Alexey, Ivan,
>
> Agree. Keeping strong references to the Thread object is the source of
> memory leak with ThreadLocals variables
> and the values that it stores. ThreadLocalMap is bound to the Thread
> lifespan [1], so I think when we are using
> everything right all will be GC'ed correctly.
> Is this memory leaks with ThreadLocal's you mean, Alexey? If not, please,
> share your example.
>
> Also, agree that these usages should be bound to the component lifespan.
> But for `FileWriteAheadLogManager`
> I think this variable used not semantically right. I've dumped all threads
> (total ~49 threads)
> that are using `lastWalPtr` in `FileWriteAheadLogManager`. For instance,
>  * exchange-worker-#40%wal.IgniteWalRecoveryTest0%
>  * sys-#148%wal.IgniteWalRecoveryTest1%
>  * db-checkpoint-thread-#129%wal.IgniteWalRecoveryTest2%
> Suppose everything would be OK here for `static` and `non-static` case of
> ThreadLocal.
>
> [1]
>
> http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/tip/src/share/classes/java/lang/Thread.java#l760
>
> On Tue, 11 Sep 2018 at 13:05 Павлухин Иван  wrote:
>
> > Dmitriy,
> >
> > Could you point to some piece of code implementing described pattern?
> >
> > 2018-09-11 13:02 GMT+03:00 Павлухин Иван :
> >
> > > Alex,
> > >
> > > ThreadLocal subclass is used in IgniteH2Indexing for simple access to
> H2
> > > Connection from current thread. Such subclass has a capability to
> create
> > > connection if one does not exist, so obtaining connection is merely
> > > ThreadLocal.get. Also there are scheduled routines to cleanup
> connections
> > > and associated with them statement cache after some expiration time.
> For
> > > that reason Map is maintained. As query
> can
> > > run on user thread we need to cleanup mentioned map to avoid a leak
> when
> > > Thread is terminated. So we need to check thread status in cleanup
> > routines
> > > and remove entries for terminated Threads. And historically there was
> no
> > > cleanup for terminated threads and leak was possible. And also great
> care
> > > must be taken in order to avoid cyclic reference between ThreadLocal
> > > instance and a stored value. Which easily could occur if the stored
> value
> > > is covered by multiple layers of abstraction.
> > >
> > > And I am describing some historical state. Now machinery in
> > IgniteH2Indexing
> > > is even more complex (I hope we will have a chance to improve it).
> > >
> > > 2018-09-11 11:00 GMT+03:00 Alexey Goncharuk <
> alexey.goncha...@gmail.com
> > >:
> > >
> > >> Ivan,
> > >>
> > >> Can you elaborate on the issue with the thread local cleanup you've
> > faced?
> > >>
> > >> вт, 11 сент. 2018 г. в 9:13, Павлухин Иван :
> > >>
> > >> > Guys,
> > >> >
> > >> > As we know ThreadLocal is an instrument which should be used with
> > great
> > >> > care. And I recently faced with problems related to proper cleanup
> of
> > >> > ThreadLocal which is not needed anymore. In my opinion the best
> thing
> > >> (in
> > >> > ideal world) is to get rid of ThreadLocal where possible, but I
> guess
> > >> that
> > >> > it is quite hard (in real world).
> > >> >
> > >> > Also, a question comes to my mind. As ThreadLocal is so common in
> our
> > >> code,
> > >> > could you suggest some guidance or code fragments which address
> proper
> > >> > ThreadLocal
> > >> >  lifecycle control and especially cleanup?
> > >> >
> > >> > 2018-09-10 12:46 GMT+03:00 Alexey Goncharuk <
> > alexey.goncha...@gmail.com
> > >> >:
> > >> >
> > >> > > Maxim,
> > >> > >
> > >> > > Ignite supports starting multiple instances of Ignite in the same
> > VM,
> > >> so
> > >> > > having static thread locals for the fields you mentioned does not
> > >> work.
> > >> > >
> > >> > > Generally, I think thread-local should be bound to the lifespan of
> > the
> > >> > > component it describes. Static thread-locals are hard to clean-up
> > and
> > >> > they
> > >> > > often lead to leaks, so I would rather changed existing static
> > >> > > thread-locals to be non-static.
> > >> > >
> > >> > > --AG
> > >> > >
> > >> > > пн, 10 сент. 2018 г. в 11:54, Maxim Muzafarov  >:
> > >> > >
> > >> > > > Igniters,
> > >> > > >
> > >> > > > According to javadoc [1] class ThreadLocal:
> > >> > > > `ThreadLocal instances are typically private *static* fields in
> > >> classes
> > >> > > > that wish to associate state with a thread (e.g., a user ID or
> > >> > > Transaction
> > >> > > > ID).`
> > >> > > >
> > >> > > > So, AFAIK non-static ThreadLocal usage means as `per thread -
> per
> > >> cla

[jira] [Created] (IGNITE-9567) Updated VisorPersistenceMetrics with new metrics from DataStorageMetrics

2018-09-12 Thread Alexey Kuznetsov (JIRA)
Alexey Kuznetsov created IGNITE-9567:


 Summary: Updated VisorPersistenceMetrics with new metrics from 
DataStorageMetrics
 Key: IGNITE-9567
 URL: https://issues.apache.org/jira/browse/IGNITE-9567
 Project: Ignite
  Issue Type: Bug
  Components: wizards
Reporter: Alexey Kuznetsov
Assignee: Alexey Kuznetsov
 Fix For: 2.7


A lot of new metrics were added in ignite-2.5 (see IGNITE-8087).

We should update VisorPersistenceMetrics class to allow collect that metrics by 
tools like Visor CMD / Web Console.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-9566) Make possible execute Explain by selected part of the query

2018-09-12 Thread Pavel Konstantinov (JIRA)
Pavel Konstantinov created IGNITE-9566:
--

 Summary: Make possible execute Explain by selected part of the 
query
 Key: IGNITE-9566
 URL: https://issues.apache.org/jira/browse/IGNITE-9566
 Project: Ignite
  Issue Type: Improvement
Reporter: Pavel Konstantinov


Currently, it is possible to execute a query by a selected part, but not 
possible make explain.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-9565) Update chartjs streaming plugin

2018-09-12 Thread Alexander Kalinin (JIRA)
Alexander Kalinin created IGNITE-9565:
-

 Summary: Update chartjs streaming plugin
 Key: IGNITE-9565
 URL: https://issues.apache.org/jira/browse/IGNITE-9565
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexander Kalinin
Assignee: Alexander Kalinin


[https://github.com/nagix/chartjs-plugin-streaming/releases]

Chartjs-streaming plugin now has a ttl property which allows to set maximum 
points. 

We should update version of plugin and remove our code for stripping extra 
points.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] ignite pull request #4741: IGNITE-9026 fix random class loading failures

2018-09-12 Thread DaveWHarvey
GitHub user DaveWHarvey opened a pull request:

https://github.com/apache/ignite/pull/4741

IGNITE-9026 fix random class loading failures

Skip recursive resource requests to orginating nodes, rather than failing 
the entire request.   Continue to search other nodes on errors, because 
assumption that all nodes have the same view is incorrect.
Restrict the recursive searches that a node should do when looking for 
resources by avoiding the nodes that the sender has or will search.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/percipiomedia/ignite p2p_two_hops

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4741.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4741


commit 218469c6157f2aada33acb69adac60e25112a73a
Author: Dave Harvey 
Date:   2018-07-18T20:51:50Z

IGNITE-9026 fix random class loading failures

Skip recursive resource requests to orginating nodes, rather than failing 
the entire request.   Continue to search other nodes on errors, because 
assumption that all nodes have the same view is incorrect.
Restrict the recursive searches that a node should do when looking for 
resources by avoiding the nodes that the sender has or will search.




---


Re: Python thin client

2018-09-12 Thread Dmitry Melnichuk

Igor,

Yes, it's my bad, sorry. Just merged the Ignite master with my branch.

On 9/12/18 8:47 PM, Igor Sapego wrote:

Pavel,
Yes, I did. I tried completely clean environment, followed the same
steps and got the same error. Then I removed attr, and out of sudden
everything started working.

Dmitry,
Thanks, now it's more clear:
Handshake error: Unsupported version. Server expects binary protocol
version 1.1.0. Client provides 1.2.0.

Why do you use version 1.2.0, but have outdated server code? I
propose you to merge with or rebase onto Ignite's master branch.

Best Regards,
Igor


On Tue, Sep 11, 2018 at 11:21 PM Dmitry Melnichuk <
dmitry.melnic...@nobitlost.com> wrote:


Igor,

I have just commited an improvment to the HandshakeError message
generation algorithm. I hope it is now easier to understand what expects
what in case of binary protocol version mismatch.

Thank you for pointing this out.

On 9/12/18 2:13 AM, Igor Sapego wrote:

I managed to start tests, and now I'm getting the following message:

pyignite.exceptions.HandshakeError: Handshake error: Unsupported
version. Expected protocol version: 1.1.0.

It would be useful to print "Unexpected version" itself, because I can
not understand what is the issue.

Best Regards,
Igor








Re: Wrong off-heap size is reported for a node

2018-09-12 Thread Alex Plehanov
Hi Dmitriy,

Yes, I will finish review tomorrow.

ср, 12 сент. 2018 г. в 20:37, Dmitriy Pavlov :

> Hi Igniters,
>
> It seems the issue is a blocker of 2.7.
>
> We need a volunteer here to run review.
>
> Alexey P. will you step in?
>
> Sincerely,
> Dmitriy Pavlov
>
> ср, 29 авг. 2018 г. в 16:16, Nikita Amelchev :
>
> > Pavel,
> >
> > I think that point 1 is the correct way to calculate the committed
> > size for a log. It is already calculated regardless of the
> > metricsEnabled flag.
> >
> > In addition, I suggest more readable log format for data regions in
> > issue comments [1].
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-9305
> >
> > вт, 21 авг. 2018 г. в 12:15, Pavel Pereslegin :
> > >
> > > Hello, Igniters.
> > >
> > > I assigned ticket [1] created by Denis and want to clarify how to log
> > > committed size.
> > > The metric offHeapSize (in DataRegionMetricsImpl) is always
> > > calculated, but getOffHeapSize returns zero if memory metrics are
> > > disabled for this data region.
> > >
> > > So I see the following options:
> > > 1. Modify method getOffHeapSize so that it always returns actual value
> > > offHeapSize.
> > > 2. Add another offHeapSize() method.
> > > 3. Output to log max size instead of committed (change "comm" to "max"
> > > in log output).
> > > 4. Don't bother about disabling metrics and output to log value
> > > returned by getOffHeapSize.
> > >
> > > Any thoughts?
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-9305
> > > сб, 18 авг. 2018 г. в 3:17, Denis Magda :
> > > >
> > > > Vova, the things are even simpler - we have this
> > > >
> > > > ignite.dataRegionMetrics().getPhysicalMemorySize() that returns the
> > > > number equal/comparabel to pageNumber X pageSize.
> > > >
> > > >
> > > > Igniters, if you believe that we need to do more work here then let's
> > > > do it iteratively. Let's fix the off-heap occupied size the way above
> > > > (just print out getPhysicalMemorySize() for every data region). Then
> > > > do the rest. This needs to be fixed in 2.7.
> > > >
> > > >
> > > > --
> > > >
> > > > Denis
> > > >
> > > >
> > > > On Fri, Aug 17, 2018 at 10:20 AM Vladimir Ozerov <
> voze...@gridgain.com
> > >
> > > > wrote:
> > > >
> > > > > Folks,
> > > > >
> > > > > We already have this:
> > > > > >>> PageMemory [pages=6997377]
> > > > >
> > > > > Then we can multiply it by page size and get occupied memory. Am I
> > wrong?
> > > > >
> > > > > On Fri, Aug 17, 2018 at 12:56 PM Dmitriy Pavlov <
> > dpavlov@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Maxim,
> > > > > >
> > > > > > thank you for stepping in and for finding these issues. Yes,
> these
> > > > > tickets
> > > > > > are correct.
> > > > > >
> > > > > > I can move https://issues.apache.org/jira/browse/IGNITE-5583 to
> > > > > unassigned
> > > > > > if someone would like to implement this change. I will not have
> > enough
> > > > > time
> > > > > > to complete it in 1 month (before 2.7 release).
> > > > > >
> > > > > > Sincerely,
> > > > > > Dmitriy Pavlov
> > > > > >
> > > > > > пт, 17 авг. 2018 г. в 11:04, Maxim Muzafarov  >:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > Suppose, Dmitry is talking about IGNITE-5583 [1] - `Switch
> > non-heap
> > > > > > memory
> > > > > > > metrics
> > > > > > > to new page memory semantics` and related previous disscustions
> > to it
> > > > > > [4].
> > > > > > >
> > > > > > > Also we have some additional improvements to CacheMetrics:
> > > > > > > IGNITE-5490 [2] - `Implement replacement for obsolete
> > > > > > > CacheMetrics#getOffHeapAllocatedSize`
> > > > > > > IGNITE-5765 [3] - `CacheMetrics interface cleanup,
> documentation
> > and
> > > > > > fixes`
> > > > > > >
> > > > > > >
> > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-5583
> > > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-5490
> > > > > > > [3] https://issues.apache.org/jira/browse/IGNITE-5765
> > > > > > > [4]
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Negative-non-heap-memory-maximum-td17990.html
> > > > > > >
> > > > > > > On Fri, 17 Aug 2018 at 10:14 Dmitriy Pavlov <
> > dpavlov@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Igniters,
> > > > > > > >
> > > > > > > > It is not an easy fix, so I'm not sure it is possible to do
> in
> > 2.7.
> > > > > > > >
> > > > > > > > Offheap size is not reported by VM (it returns -1). To
> > implement it
> > > > > we
> > > > > > > need
> > > > > > > > totally migrate off-heap memory metrics to durable memory
> data.
> > > > > > > >
> > > > > > > > I think this issue was reported and I'll find the duplicate.
> > > > > > > >
> > > > > > > > Sincerely,
> > > > > > > > Dmitriy Pavlov
> > > > > > > >
> > > > > > > > пт, 17 авг. 2018 г. в 6:10, Denis Magda :
> > > > > > > >
> > > > > > > > > Yes, it was at the end of my wordy email :)
> > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > > 

Re: Data streaming using Apache Ignite and Flink

2018-09-12 Thread Denis Magda
Prachi,

Did you forget to add Saikat's blog to the list?

--
Denis

On Wed, Sep 12, 2018 at 1:44 PM Dmitriy Pavlov 
wrote:

> Hi Denis,
>
> I didn't find the blog post there.
>
> Could you please add it or advice me how can I do it?
>
> Sincerely,
> Dmitriy Pavlov
>
> вт, 28 авг. 2018 г. в 4:19, Saikat Maitra :
>
> > Thank you so much Denis.
> >
> > Regards,
> > Saikat
> >
> > On Mon, Aug 27, 2018 at 5:00 PM, Denis Magda  wrote:
> >
> > > Hello Saikat,
> > >
> > > Thanks for the article and contribution. We'll get it added to:
> > > https://ignite.apache.org/blogs.html
> > >
> > > --
> > > Denis
> > >
> > > On Sun, Aug 26, 2018 at 2:59 PM Saikat Maitra  >
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > I recently published blog on how we can stream data using Apache
> Ignite
> > > and
> > > > Flink. This uses IgniteSink with recent changes merged (release due
> in
> > > > 2.7.0) which will allow us to run IgniteSink using Apache Flink in
> > > cluster
> > > > mode.
> > > >
> > > >
> > > >
> > > > https://samaitra.blogspot.com/2018/08/data-streaming-using-
> > > apache-flink-and.html
> > > >
> > > > Please review and let me know if you have feedback.
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > >
> >
>


Re: Data streaming using Apache Ignite and Flink

2018-09-12 Thread Dmitriy Pavlov
Hi Denis,

I didn't find the blog post there.

Could you please add it or advice me how can I do it?

Sincerely,
Dmitriy Pavlov

вт, 28 авг. 2018 г. в 4:19, Saikat Maitra :

> Thank you so much Denis.
>
> Regards,
> Saikat
>
> On Mon, Aug 27, 2018 at 5:00 PM, Denis Magda  wrote:
>
> > Hello Saikat,
> >
> > Thanks for the article and contribution. We'll get it added to:
> > https://ignite.apache.org/blogs.html
> >
> > --
> > Denis
> >
> > On Sun, Aug 26, 2018 at 2:59 PM Saikat Maitra 
> > wrote:
> >
> > > Hello,
> > >
> > > I recently published blog on how we can stream data using Apache Ignite
> > and
> > > Flink. This uses IgniteSink with recent changes merged (release due in
> > > 2.7.0) which will allow us to run IgniteSink using Apache Flink in
> > cluster
> > > mode.
> > >
> > >
> > >
> > > https://samaitra.blogspot.com/2018/08/data-streaming-using-
> > apache-flink-and.html
> > >
> > > Please review and let me know if you have feedback.
> > >
> > > Regards,
> > > Saikat
> > >
> >
>


[jira] [Created] (IGNITE-9564) Ignite K8 service and sticky session with AWS K8 deployments

2018-09-12 Thread Denis Magda (JIRA)
Denis Magda created IGNITE-9564:
---

 Summary: Ignite K8 service and sticky session with AWS K8 
deployments
 Key: IGNITE-9564
 URL: https://issues.apache.org/jira/browse/IGNITE-9564
 Project: Ignite
  Issue Type: Task
Reporter: Denis Magda
Assignee: Ilya Murchenko
 Fix For: 2.8


_sessionAffinity_ parameter is not supported by AWS Kubernetes which causes the 
failure of the standard Ignite service deployment:
http://apache-ignite-users.70518.x6.nabble.com/Error-installing-Ignite-on-K8s-td23999.html

See how Ignite service configuration have to be updated considering sticky 
session capabilities of AWS:
https://aws.amazon.com/elasticloadbalancing/details/



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: [MTCGA]: new failures in builds [1588345] needs to be handled

2018-09-12 Thread Dmitriy Pavlov
I suggest upgrading, and we can agree by lazy consensus after 72hrs.

пн, 27 авг. 2018 г. в 13:03, Alexey Goncharuk :

> Folks, this is a new failure in IGFS/Hadoop domain. Looks like the
> http://www.us.apache.org/dist/hadoop/core/hadoop-2.5.2/hadoop-2.5.2.tar.gz
> was moved (the URL gives 404).
>
> Vladimir, can you take a look or point community how to take care of this
> failure? Should we simply upgrade the Hadoop version?
>
> пн, 27 авг. 2018 г. в 9:43, :
>
> > Hi Ignite Developer,
> >
> > I am MTCGA.Bot, and I've detected some issue on TeamCity to be addressed.
> > I hope you can help.
> >
> >  *Recently contributed test failed in master
> >
> org.apache.ignite.testsuites.IgniteIgfsLinuxAndMacOSTestSuite.initializationError
> >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&testNameId=-4034437294452110435&branch=%3Cdefault%3E&tab=testDetails
> >  No changes in build
> >
> > - If your changes can led to this failure(s), please create issue
> > with label MakeTeamCityGreenAgain and assign it to you.
> > -- If you have fix, please set ticket to PA state and write to
> dev
> > list fix is ready
> > -- For case fix will require some time please mute test and set
> > label Muted_Test to issue
> > - If you know which change caused failure please contact change
> > author directly
> > - If you don't know which change caused failure please send
> > message to dev list to find out
> > Should you have any questions please contact dev@ignite.apache.org
> > Best Regards,
> > MTCGA.Bot
> > Notification generated at Mon Aug 27 09:43:20 MSK 2018
> >
>


Re: Wrong off-heap size is reported for a node

2018-09-12 Thread Dmitriy Pavlov
Hi Igniters,

It seems the issue is a blocker of 2.7.

We need a volunteer here to run review.

Alexey P. will you step in?

Sincerely,
Dmitriy Pavlov

ср, 29 авг. 2018 г. в 16:16, Nikita Amelchev :

> Pavel,
>
> I think that point 1 is the correct way to calculate the committed
> size for a log. It is already calculated regardless of the
> metricsEnabled flag.
>
> In addition, I suggest more readable log format for data regions in
> issue comments [1].
>
> [1] https://issues.apache.org/jira/browse/IGNITE-9305
>
> вт, 21 авг. 2018 г. в 12:15, Pavel Pereslegin :
> >
> > Hello, Igniters.
> >
> > I assigned ticket [1] created by Denis and want to clarify how to log
> > committed size.
> > The metric offHeapSize (in DataRegionMetricsImpl) is always
> > calculated, but getOffHeapSize returns zero if memory metrics are
> > disabled for this data region.
> >
> > So I see the following options:
> > 1. Modify method getOffHeapSize so that it always returns actual value
> > offHeapSize.
> > 2. Add another offHeapSize() method.
> > 3. Output to log max size instead of committed (change "comm" to "max"
> > in log output).
> > 4. Don't bother about disabling metrics and output to log value
> > returned by getOffHeapSize.
> >
> > Any thoughts?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-9305
> > сб, 18 авг. 2018 г. в 3:17, Denis Magda :
> > >
> > > Vova, the things are even simpler - we have this
> > >
> > > ignite.dataRegionMetrics().getPhysicalMemorySize() that returns the
> > > number equal/comparabel to pageNumber X pageSize.
> > >
> > >
> > > Igniters, if you believe that we need to do more work here then let's
> > > do it iteratively. Let's fix the off-heap occupied size the way above
> > > (just print out getPhysicalMemorySize() for every data region). Then
> > > do the rest. This needs to be fixed in 2.7.
> > >
> > >
> > > --
> > >
> > > Denis
> > >
> > >
> > > On Fri, Aug 17, 2018 at 10:20 AM Vladimir Ozerov  >
> > > wrote:
> > >
> > > > Folks,
> > > >
> > > > We already have this:
> > > > >>> PageMemory [pages=6997377]
> > > >
> > > > Then we can multiply it by page size and get occupied memory. Am I
> wrong?
> > > >
> > > > On Fri, Aug 17, 2018 at 12:56 PM Dmitriy Pavlov <
> dpavlov@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Maxim,
> > > > >
> > > > > thank you for stepping in and for finding these issues. Yes, these
> > > > tickets
> > > > > are correct.
> > > > >
> > > > > I can move https://issues.apache.org/jira/browse/IGNITE-5583 to
> > > > unassigned
> > > > > if someone would like to implement this change. I will not have
> enough
> > > > time
> > > > > to complete it in 1 month (before 2.7 release).
> > > > >
> > > > > Sincerely,
> > > > > Dmitriy Pavlov
> > > > >
> > > > > пт, 17 авг. 2018 г. в 11:04, Maxim Muzafarov :
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > Suppose, Dmitry is talking about IGNITE-5583 [1] - `Switch
> non-heap
> > > > > memory
> > > > > > metrics
> > > > > > to new page memory semantics` and related previous disscustions
> to it
> > > > > [4].
> > > > > >
> > > > > > Also we have some additional improvements to CacheMetrics:
> > > > > > IGNITE-5490 [2] - `Implement replacement for obsolete
> > > > > > CacheMetrics#getOffHeapAllocatedSize`
> > > > > > IGNITE-5765 [3] - `CacheMetrics interface cleanup, documentation
> and
> > > > > fixes`
> > > > > >
> > > > > >
> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-5583
> > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-5490
> > > > > > [3] https://issues.apache.org/jira/browse/IGNITE-5765
> > > > > > [4]
> > > > > >
> > > > > >
> > > > >
> > > >
> http://apache-ignite-developers.2346864.n4.nabble.com/Negative-non-heap-memory-maximum-td17990.html
> > > > > >
> > > > > > On Fri, 17 Aug 2018 at 10:14 Dmitriy Pavlov <
> dpavlov@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > Hi Igniters,
> > > > > > >
> > > > > > > It is not an easy fix, so I'm not sure it is possible to do in
> 2.7.
> > > > > > >
> > > > > > > Offheap size is not reported by VM (it returns -1). To
> implement it
> > > > we
> > > > > > need
> > > > > > > totally migrate off-heap memory metrics to durable memory data.
> > > > > > >
> > > > > > > I think this issue was reported and I'll find the duplicate.
> > > > > > >
> > > > > > > Sincerely,
> > > > > > > Dmitriy Pavlov
> > > > > > >
> > > > > > > пт, 17 авг. 2018 г. в 6:10, Denis Magda :
> > > > > > >
> > > > > > > > Yes, it was at the end of my wordy email :)
> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-9305
> > > > > > > >
> > > > > > > > --
> > > > > > > > Denis
> > > > > > > >
> > > > > > > > On Thu, Aug 16, 2018 at 11:03 PM Dmitriy Setrakyan <
> > > > > > > dsetrak...@apache.org>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Is there a blocker ticket for 2.7?
> > > > > > > > >
> > > > > > > > > On Thu, Aug 16, 2018, 19:59 Denis Magda  >
> > > > wrote:
> > > > > > > > >
> > > > > > > > > > Igniters,
> > > > > > > > > >
> > > >

Re: affinityBackupFilter for AWS Availability Zones

2018-09-12 Thread Valentin Kulichenko
Yes, will try to review this week.

-Val

On Wed, Sep 12, 2018 at 10:24 AM Dmitriy Pavlov 
wrote:

> Hi Val,
>
> I'm not an expert in AWS, so could you please pick up the review?
>
> Thank you in advance!
>
> Sincerely,
> Dmitriy Pavlov
>
> вт, 11 сент. 2018 г. в 1:28, Dave Harvey :
>
> > Submitted a patch for this
> > https://issues.apache.org/jira/browse/IGNITE-9365
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >
>


Re: New committer: Dmitriy Govorukhin

2018-09-12 Thread Dmitriy Pavlov
Hi Dmitriy,

It seems I little bit late, but anyway please accept my sincere
congratulations.

With Regards,
Dmitriy Pavlov

чт, 30 авг. 2018 г. в 21:18, Pavel Kovalenko :

> Dmitriy,
>
> Congratulations, you finally got it :)
>
> чт, 30 авг. 2018 г. в 20:25, Ivan Rakov :
>
> > Dmitriy,
> >
> > Congratulations! Your new role is well deserved, keep going!
> >
> > Best Regards,
> > Ivan Rakov
> >
> > On 30.08.2018 1:20, Dmitriy Setrakyan wrote:
> > > Dmitriy, congrats! Looking forward to many contributions to come.
> > >
> > > On Wed, Aug 29, 2018 at 12:35 PM, Denis Magda 
> wrote:
> > >
> > >> The Project Management Committee (PMC) for Apache Ignite
> > >> has invited Dmitriy Govorukhin to become a committer and we are
> pleased
> > >> to announce that he has accepted.
> > >>
> > >> Being a committer enables easier contribution to the
> > >> project since there is no need to go via the patch
> > >> submission process. This should enable better productivity.
> > >>
> > >> Congrats, Dmitriy! Look forward to your contributions.
> > >>
> > >> --
> > >> Denis
> > >>
> >
> >
>


Re: TDE Implementation details.

2018-09-12 Thread Denis Magda
Hello Nikolay,

Excellent progress, look forward to seeing the TDE released in 2.7!

--
Denis

On Wed, Sep 12, 2018 at 2:47 AM Nikolay Izhikov  wrote:

> Hello, Denis.
>
> > Could you please list the limitations of Phase 1?
> > I'm curious what won't be supported in 2.7.
>
> 1. We will have ability to encrypt data stored on the disk for specific
> cache.
>
> - There is no limitation on API usage or something for an
> encrypted cache.
> - If some cache in a cache group is encrypted, all other caches in
> this group must be encrypted.
> - Setting up master key(standard jdk key storage) is prerequisite
> and should be done by an administrator.
> - `encryptionEnabled` flag setting up on cache creation and can't
> be changed in runtime.
>
> 2. We won't be able to change encryption keys for existing cache(key
> rotation procedure).
> This will be implemented in Phase 2.
>
> В Вт, 11/09/2018 в 23:41 -0400, Denis Magda пишет:
> > Nikolay,
> >
> > Could you please list the limitations of Phase 1? I'm curious what won't
> be
> > supported in 2.7.
> >
> > --
> > Denis
> >
> > On Tue, Sep 11, 2018 at 4:29 PM Nikolay Izhikov 
> wrote:
> >
> > > > As I understand the plan is to get TDE Phase 1 released in 2.7,
> right?
> > >
> > > Yes. We will release TDE in 2.7
> > >
> > > > Could you also list what needs to be done at Phase 2 and how much
> time
> > >
> > > it might take.
> > >
> > > Yes, I think Dmitry Ryabov will send Phase 2 design
> > >
> > >
> > > В Вт, 11/09/2018 в 23:27 +0300, Nikolay Izhikov пишет:
> > > > Hello, Denis.
> > > >
> > > > Yes, Vladimir made 2 rounds of review.
> > > > I planning to fix all issues with implementation in a few days.
> > > >
> > > >
> > > > В Вт, 11/09/2018 в 10:40 -0400, Denis Magda пишет:
> > > > > Hi Nikolay,
> > > > >
> > > > > Has anybody started looking into your request? As I understand the
> > >
> > > plan is
> > > > > to get TDE Phase 1 released in 2.7, right?
> > > > >
> > >
> > >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=89067473
> > > > >
> > > > > Could you also list what needs to be done at Phase 2 and how much
> time
> > >
> > > it
> > > > > might take.
> > > > >
> > > > > --
> > > > > Denis
> > > > >
> > > > > On Thu, Aug 9, 2018 at 8:48 AM Nikolay Izhikov <
> nizhi...@apache.org>
> > >
> > > wrote:
> > > > >
> > > > > > Hello, Igniters.
> > > > > >
> > > > > > I want to share with you TDE implementation details.
> > > > > > I think it can simplify review and acception of TDE
> implementation.
> > > > > > This mail is copy of wiki page [1].
> > > > > >
> > > > > > Please, share your thoughts.
> > > > > >
> > > > > > TDE is ready for review [2].
> > > > > > Please, let me know, who is able to make final review.
> > > > > >
> > > > > > This document describes some internal details of TDE.Phase 1
> > > > > > implementation [3].
> > > > > > I suggest that reader familiar with the general design described
> in
> > >
> > > IEP-18
> > > > > > [4].
> > > > > >
> > > > > >
> > > > > > Cache group key management and node join enhancements:
> > > > > >
> > > > > > 1. GridEncryptionManager encapsulates all logic related
> to
> > >
> > > key
> > > > > > management:
> > > > > > a. All group encryption keys are stored in
> MetaStore.
> > > > > >
> > > > > > b. Joining node sends to cluster:
> > > > > > * Master key digest.
> > > > > > * All group keys saved locally
> (Map > > > > > byte[]>). Keys send over a network in encrypted form.
> > > > > >
> > > > > > c. Coordinator on new node join:
> > > > > > * Compares new node master key digest
> with
> > >
> > > the
> > > > > > local master key digest.
> > > > > > If differs then new node join is
> rejected.
> > > > > >
> > > > > > * Compares local and received group keys.
> > > > > > If some key differs then new node join is
> > > > > > rejected.
> > > > > >
> > > > > > d. All server nodes:
> > > > > > * If some of received keys are new then
> they
> > >
> > > save
> > > > > > locally.
> > > > > >
> > > > > > 2. Dynamic cache creation:
> > > > > > a. On server node - Encryption key is generated
> and
> > >
> > > added
> > > > > > to DynamicCacheChangeRequest.
> > > > > >
> > > > > > b. On client node:
> > > > > > * Prior to creation of
> > >
> > > DynamicCacheChangeRequest
> > > > > > we have to get new encryption key from some server node.
> > > > > > * New request added to solve this -
> > > > > > GenerateEncryptionKeyRequest.
> > > > > >
> > > > > >
> > > > > > WAL Record encryption:
> > > > > >
> > > > > >
> > > > > > 1. New type of WAL record created – EncryptedRecord.
> > > > > >
> > > > > > 2. EncryptedRecord is a container th

Re: affinityBackupFilter for AWS Availability Zones

2018-09-12 Thread Dmitriy Pavlov
Hi Val,

I'm not an expert in AWS, so could you please pick up the review?

Thank you in advance!

Sincerely,
Dmitriy Pavlov

вт, 11 сент. 2018 г. в 1:28, Dave Harvey :

> Submitted a patch for this
> https://issues.apache.org/jira/browse/IGNITE-9365
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>


Re: Class field ThreadLocal. Why not static?

2018-09-12 Thread Dmitriy Pavlov
Hi Ivan,

For example, I remember thread local buffer in our old WAL implementation:
org.apache.ignite.internal.processors.cache.persistence.wal.FsyncModeFileWriteAheadLogManager#tlb

Sincerely,
Dmitriy Pavlov

вт, 11 сент. 2018 г. в 13:05, Павлухин Иван :

> Dmitriy,
>
> Could you point to some piece of code implementing described pattern?
>
> 2018-09-11 13:02 GMT+03:00 Павлухин Иван :
>
> > Alex,
> >
> > ThreadLocal subclass is used in IgniteH2Indexing for simple access to H2
> > Connection from current thread. Such subclass has a capability to create
> > connection if one does not exist, so obtaining connection is merely
> > ThreadLocal.get. Also there are scheduled routines to cleanup connections
> > and associated with them statement cache after some expiration time. For
> > that reason Map is maintained. As query can
> > run on user thread we need to cleanup mentioned map to avoid a leak when
> > Thread is terminated. So we need to check thread status in cleanup
> routines
> > and remove entries for terminated Threads. And historically there was no
> > cleanup for terminated threads and leak was possible. And also great care
> > must be taken in order to avoid cyclic reference between ThreadLocal
> > instance and a stored value. Which easily could occur if the stored value
> > is covered by multiple layers of abstraction.
> >
> > And I am describing some historical state. Now machinery in
> IgniteH2Indexing
> > is even more complex (I hope we will have a chance to improve it).
> >
> > 2018-09-11 11:00 GMT+03:00 Alexey Goncharuk  >:
> >
> >> Ivan,
> >>
> >> Can you elaborate on the issue with the thread local cleanup you've
> faced?
> >>
> >> вт, 11 сент. 2018 г. в 9:13, Павлухин Иван :
> >>
> >> > Guys,
> >> >
> >> > As we know ThreadLocal is an instrument which should be used with
> great
> >> > care. And I recently faced with problems related to proper cleanup of
> >> > ThreadLocal which is not needed anymore. In my opinion the best thing
> >> (in
> >> > ideal world) is to get rid of ThreadLocal where possible, but I guess
> >> that
> >> > it is quite hard (in real world).
> >> >
> >> > Also, a question comes to my mind. As ThreadLocal is so common in our
> >> code,
> >> > could you suggest some guidance or code fragments which address proper
> >> > ThreadLocal
> >> >  lifecycle control and especially cleanup?
> >> >
> >> > 2018-09-10 12:46 GMT+03:00 Alexey Goncharuk <
> alexey.goncha...@gmail.com
> >> >:
> >> >
> >> > > Maxim,
> >> > >
> >> > > Ignite supports starting multiple instances of Ignite in the same
> VM,
> >> so
> >> > > having static thread locals for the fields you mentioned does not
> >> work.
> >> > >
> >> > > Generally, I think thread-local should be bound to the lifespan of
> the
> >> > > component it describes. Static thread-locals are hard to clean-up
> and
> >> > they
> >> > > often lead to leaks, so I would rather changed existing static
> >> > > thread-locals to be non-static.
> >> > >
> >> > > --AG
> >> > >
> >> > > пн, 10 сент. 2018 г. в 11:54, Maxim Muzafarov :
> >> > >
> >> > > > Igniters,
> >> > > >
> >> > > > According to javadoc [1] class ThreadLocal:
> >> > > > `ThreadLocal instances are typically private *static* fields in
> >> classes
> >> > > > that wish to associate state with a thread (e.g., a user ID or
> >> > > Transaction
> >> > > > ID).`
> >> > > >
> >> > > > So, AFAIK non-static ThreadLocal usage means as `per thread - per
> >> class
> >> > > > instance`. What the real cases of using non-static ThreadLocal
> class
> >> > > fields
> >> > > > in Ignite code project? When we need it?
> >> > > >
> >> > > > In Ignite code project I've found ThreadLocal usage as:
> >> > > >  - non-static - 67
> >> > > >  - static  - 68
> >> > > >
> >> > > > Back to my example, I've checked FileWriteAheadLogManager. It has:
> >> > > > 1) private final ThreadLocal interrupted [2]
> >> > > > 2) private final ThreadLocal lastWALPtr [3]
> >> > > > I think both of these fields should be set and used as `static`.
> Can
> >> > > anyone
> >> > > > confirm it?
> >> > > >
> >> > > >
> >> > > > [1]
> >> > https://docs.oracle.com/javase/8/docs/api/java/lang/ThreadLocal.html
> >> > > > [2]
> >> > > >
> >> > > > https://github.com/apache/ignite/blob/master/modules/
> >> > > core/src/main/java/org/apache/ignite/internal/processors/
> >> > > cache/persistence/wal/FileWriteAheadLogManager.java#L253
> >> > > > [3]
> >> > > >
> >> > > > https://github.com/apache/ignite/blob/master/modules/
> >> > > core/src/main/java/org/apache/ignite/internal/processors/
> >> > > cache/persistence/wal/FileWriteAheadLogManager.java#L340
> >> > > > --
> >> > > > --
> >> > > > Maxim Muzafarov
> >> > > >
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > Best regards,
> >> > Ivan Pavlukhin
> >> >
> >>
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>


[GitHub] ignite pull request #4740: IGNITE-9563 Try to fix processor count / thread p...

2018-09-12 Thread alamar
GitHub user alamar opened a pull request:

https://github.com/apache/ignite/pull/4740

IGNITE-9563 Try to fix processor count / thread pool size test.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9563

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4740.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4740


commit 109d99c73f92fb96fb5064e12df377e6b92c6407
Author: Ilya Kasnacheev 
Date:   2018-09-12T16:01:51Z

IGNITE-9563 Try to fix processor count / thread pool size test.




---


[jira] [Created] (IGNITE-9563) .NET: TestSpringXml is flaky on Core Linux

2018-09-12 Thread Ilya Kasnacheev (JIRA)
Ilya Kasnacheev created IGNITE-9563:
---

 Summary: .NET: TestSpringXml is flaky on Core Linux
 Key: IGNITE-9563
 URL: https://issues.apache.org/jira/browse/IGNITE-9563
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Affects Versions: 2.6
Reporter: Ilya Kasnacheev
Assignee: Ilya Kasnacheev


Maybe it's 
https://stackoverflow.com/questions/27965962/c-sharp-environment-processorcount-does-not-always-return-the-full-number-of-log



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] asfgit closed pull request #6: IGNITE-9377 Handle print crit failures from RunAll to the JIRA ticket

2018-09-12 Thread GitBox
asfgit closed pull request #6: IGNITE-9377 Handle print crit failures from 
RunAll to the JIRA ticket
URL: https://github.com/apache/ignite-teamcity-bot/pull/6
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/HelperConfig.java 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/HelperConfig.java
index b3b51c7..37880c1 100644
--- a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/HelperConfig.java
+++ b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/HelperConfig.java
@@ -48,7 +48,14 @@
 @Deprecated
 private static final String PASSWORD = "password";
 public static final String ENCODED_PASSWORD = "encoded_password";
+
+/** GitHub authorization token property name. */
 public static final String GITHUB_AUTH_TOKEN = "github.auth_token";
+
+/** JIRA authorization token property name. */
+public static final String JIRA_AUTH_TOKEN = "jira.auth_token";
+
+/** Slack authorization token property name. */
 public static final String SLACK_AUTH_TOKEN = "slack.auth_token";
 public static final String SLACK_CHANNEL = "slack.channel";
 public static final String LOGS = "logs";
@@ -135,6 +142,21 @@ public static File resolveWorkDir() {
 return pwd;
 }
 
+/**
+ * Extract JIRA basic authorization token from properties.
+ *
+ * @param props Properties, where token is placed.
+ * @return Null or decoded auth token for Github.
+ */
+@Nullable static String prepareJiraHttpAuthToken(Properties props) {
+String pwd = props.getProperty(JIRA_AUTH_TOKEN);
+
+if (isNullOrEmpty(pwd))
+return null;
+
+return pwd;
+}
+
 /**
  * Extract TeamCity authorization token from properties.
  *
diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITcHelper.java 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITcHelper.java
index b2beb51..69df43d 100644
--- a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITcHelper.java
+++ b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITcHelper.java
@@ -18,6 +18,7 @@
 package org.apache.ignite.ci;
 
 import java.util.List;
+import org.apache.ignite.ci.observer.BuildObserver;
 import org.apache.ignite.ci.issue.IssueDetector;
 import org.apache.ignite.ci.issue.IssuesStorage;
 import org.apache.ignite.ci.user.ICredentialsProv;
@@ -42,6 +43,11 @@
 
 IssueDetector issueDetector();
 
+/**
+ * @return Build observer.
+ */
+BuildObserver buildObserver();
+
 IAnalyticsEnabledTeamcity server(String srvId, @Nullable ICredentialsProv 
prov);
 
 ITcAnalytics tcAnalytics(String serverId);
diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITeamcity.java 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITeamcity.java
index 7e105c7..8201f5c 100644
--- a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITeamcity.java
+++ b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITeamcity.java
@@ -239,7 +239,12 @@ default SingleBuildRunCtx loadTestsAndProblems(@Nonnull 
Build build, @Deprecated
  * @param cleanRebuild Rebuild all dependencies.
  * @param queueAtTop Put at the top of the build queue.
  */
-void triggerBuild(String id, String name, boolean cleanRebuild, boolean 
queueAtTop);
+Build triggerBuild(String id, String name, boolean cleanRebuild, boolean 
queueAtTop);
+
+/**
+ * @param tok TeamCity authorization token.
+ */
+void setAuthToken(String tok);
 
 /**
  * @return {@code True} if TeamCity authorization token is available.
@@ -247,9 +252,9 @@ default SingleBuildRunCtx loadTestsAndProblems(@Nonnull 
Build build, @Deprecated
 boolean isTeamCityTokenAvailable();
 
 /**
- * @param token TeamCity authorization token.
+ * @param token GitHub authorization token.
  */
-void setAuthToken(String token);
+void setGitToken(String token);
 
 /**
  * @return {@code True} if GitHub authorization token is available.
@@ -257,9 +262,14 @@ default SingleBuildRunCtx loadTestsAndProblems(@Nonnull 
Build build, @Deprecated
 boolean isGitTokenAvailable();
 
 /**
- * @param token GitHub authorization token.
+ * @param tok Jira authorization token.
  */
-void setGitToken(String token);
+void setJiraToken(String tok);
+
+/**
+ * @return {@code True} if JIRA authorization token is available.
+ */
+boolean isJiraTokenAvailable();
 
 /**
  * Send POST request with given body.
@@ -276,6 +286,14 @@ default SingleBuildRunCtx loadTestsAndProblems(@Nonnull 
Build build, @Deprecated
  */
 PullRequest getPullRequest(String branch);
 
+/**
+ * 

[jira] [Created] (IGNITE-9562) Destroyed cache that resurrected on a old offline node breaks PME

2018-09-12 Thread Pavel Kovalenko (JIRA)
Pavel Kovalenko created IGNITE-9562:
---

 Summary: Destroyed cache that resurrected on a old offline node 
breaks PME
 Key: IGNITE-9562
 URL: https://issues.apache.org/jira/browse/IGNITE-9562
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.5
Reporter: Pavel Kovalenko
 Fix For: 2.8


Given:
2 nodes, persistence enabled.
1) Stop 1 node
2) Destroy cache through client
3) Start stopped node

When the stopped node joins to cluster it starts all caches that it has seen 
before stopping.
If that cache was cluster-widely destroyed it leads to breaking the crash 
recovery process or PME.

Root cause - we don't start/collect caches from the stopped node on another 
part of a cluster.

In case of PARTITIONED cache mode that scenario breaks crash recovery:
{noformat}
java.lang.AssertionError: AffinityTopologyVersion [topVer=-1, minorTopVer=0]

at 
org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.cachedAffinity(GridAffinityAssignmentCache.java:696)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.updateLocal(GridDhtPartitionTopologyImpl.java:2449)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.afterStateRestored(GridDhtPartitionTopologyImpl.java:679)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restorePartitionStates(GridCacheDatabaseSharedManager.java:2445)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.applyLastUpdates(GridCacheDatabaseSharedManager.java:2321)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreState(GridCacheDatabaseSharedManager.java:1568)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.beforeExchange(GridCacheDatabaseSharedManager.java:1308)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.distributedExchange(GridDhtPartitionsExchangeFuture.java:1255)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:766)
at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2577)
at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2457)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:748)
{noformat}

In case of REPLICATED cache mode that scenario breaks PME coordinator process:
{noformat}
[2018-09-12 
18:50:36,407][ERROR][sys-#148%distributed.CacheStopAndRessurectOnOldNodeTest0%][GridCacheIoManager]
 Failed to process message [senderId=4b6fd0d4-b756-4a9f-90ca-f0ee2511, 
messageType=class 
o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionsSingleMessage]
java.lang.AssertionError: 3080586
at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.clientTopology(GridCachePartitionExchangeManager.java:815)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.updatePartitionSingleMap(GridDhtPartitionsExchangeFuture.java:3621)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.processSingleMessage(GridDhtPartitionsExchangeFuture.java:2439)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.access$100(GridDhtPartitionsExchangeFuture.java:137)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply(GridDhtPartitionsExchangeFuture.java:2261)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply(GridDhtPartitionsExchangeFuture.java:2249)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:383)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:353)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onReceiveSingleMessage(GridDhtPartitionsExchangeFuture.java:2249)
at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.processSinglePartitionUpdate(GridCachePartitionExchangeManager.java:1628)
at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.access$1100(GridCachePartitionExchangeM

[GitHub] ignite pull request #4739: IGNITE-8855 Client nodes make a lot of attempts t...

2018-09-12 Thread ibessonov
GitHub user ibessonov opened a pull request:

https://github.com/apache/ignite/pull/4739

IGNITE-8855 Client nodes make a lot of attempts to join topology if 
EXCHANGE_HISTORY is significantly smaller than number of clients



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8855

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4739.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4739


commit 74226e273730c2264ae93b9196399626fe040e52
Author: ibessonov 
Date:   2018-09-12T15:49:17Z

IGNITE-8855 Added exponential delay before reconnection that was caused by 
small EXCHANGE_HISTORY




---


[jira] [Created] (IGNITE-9561) Optimize affinity initialization for started cache groups

2018-09-12 Thread Pavel Kovalenko (JIRA)
Pavel Kovalenko created IGNITE-9561:
---

 Summary: Optimize affinity initialization for started cache groups
 Key: IGNITE-9561
 URL: https://issues.apache.org/jira/browse/IGNITE-9561
 Project: Ignite
  Issue Type: Improvement
  Components: cache
Affects Versions: 2.5
Reporter: Pavel Kovalenko
Assignee: Pavel Kovalenko
 Fix For: 2.7


At the end of
{noformat}
org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager#processCacheStartRequests
 
{noformat}
method we're initializing affinity for cache groups starting at current 
exchange.
We do it one-by-one and synchronously wait for AffinityFetchResponse for each 
of the starting groups. This is inefficient. We may parallelize this process 
and speed up caches starting process.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] ignite pull request #4729: IGNITE-9545 IgniteProjectionStartStopRestartSelfT...

2018-09-12 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4729


---


[jira] [Created] (IGNITE-9560) Security permissions to restrict arbitrary code exectution

2018-09-12 Thread Anton Vinogradov (JIRA)
Anton Vinogradov created IGNITE-9560:


 Summary: Security permissions to restrict arbitrary code exectution
 Key: IGNITE-9560
 URL: https://issues.apache.org/jira/browse/IGNITE-9560
 Project: Ignite
  Issue Type: Task
  Components: security
Affects Versions: 2.6
Reporter: Anton Vinogradov


{{SecurityPermission}} class should be extended to cover all cases able to 
cause arbitrary code execution.

1) Restriction on listener registration 
- EventStorageSpi listener
- CQ listener 
2) Restriction on closure (able to be executed on the remote node) execution
- Compute API (seems to be covered, should be rechecked)
- Services
- Entry processor
3) We have to make sure that cases listed at #1 and #2 are the all possible 
cases.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] ignite pull request #4738: IGNITE-9330

2018-09-12 Thread NSAmelchev
GitHub user NSAmelchev opened a pull request:

https://github.com/apache/ignite/pull/4738

IGNITE-9330

Fix flaky CacheMetricsManageTest tests

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/NSAmelchev/ignite ignite-9330

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4738.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4738


commit 46328dc86ddf12891550e40d02e628ee11ae0ad2
Author: NSAmelchev 
Date:   2018-09-12T14:01:57Z

Fix SynchronizationMode




---


[GitHub] ignite pull request #4737: IGNITE-9544 BinaryOutputStream#writeByte excessiv...

2018-09-12 Thread a-polyakov
GitHub user a-polyakov opened a pull request:

https://github.com/apache/ignite/pull/4737

IGNITE-9544 BinaryOutputStream#writeByte excessive copying after reaching 
1Gb



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/a-polyakov/ignite IGNITE-9544

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4737.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4737


commit 0b73c53f1a11698fbfb73dee64725f24391cba5f
Author: a-polyakov 
Date:   2018-09-12T13:45:27Z

IGNITE-9544 BinaryOutputStream#writeByte excessive copying after reaching 
1Gb

Signed-off-by: a-polyakov 




---


[GitHub] ignite pull request #4710: IGNITE-9495 Update version for org.apache.lucene

2018-09-12 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4710


---


[GitHub] asfgit closed pull request #5: IGNITE-9377

2018-09-12 Thread GitBox
asfgit closed pull request #5: IGNITE-9377
URL: https://github.com/apache/ignite-teamcity-bot/pull/5
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/HelperConfig.java 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/HelperConfig.java
index 3f2e9b8..b3b51c7 100644
--- a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/HelperConfig.java
+++ b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/HelperConfig.java
@@ -48,6 +48,7 @@
 @Deprecated
 private static final String PASSWORD = "password";
 public static final String ENCODED_PASSWORD = "encoded_password";
+public static final String GITHUB_AUTH_TOKEN = "github.auth_token";
 public static final String SLACK_AUTH_TOKEN = "slack.auth_token";
 public static final String SLACK_CHANNEL = "slack.channel";
 public static final String LOGS = "logs";
@@ -117,19 +118,45 @@ public static File resolveWorkDir() {
 return ensureDirExist(workDir);
 }
 
+/**
+ * Extract GitHub authorization token from properties.
+ *
+ * @param props Properties, where token is placed.
+ * @return Null or decoded auth token for Github.
+ */
+@Nullable static String prepareGithubHttpAuthToken(Properties props) {
+String pwd = props.getProperty(GITHUB_AUTH_TOKEN);
+
+if (isNullOrEmpty(pwd))
+return null;
+
+pwd = PasswordEncoder.decode(pwd);
+
+return pwd;
+}
+
+/**
+ * Extract TeamCity authorization token from properties.
+ *
+ * @param props Properties, where token is placed.
+ * @param configName Configuration name.
+ * @return Null or decoded auth token for Github.
+ */
 @Nullable static String prepareBasicHttpAuthToken(Properties props, String 
configName) {
 final String user = getMandatoryProperty(props, USERNAME, configName);
 String pwd = props.getProperty(PASSWORD);
 boolean filled = !isNullOrEmpty(pwd);
-if(!filled) {
+
+if (!filled) {
 String enc = props.getProperty(ENCODED_PASSWORD);
+
 if(!isNullOrEmpty(enc)) {
 pwd = PasswordEncoder.decode(enc);
 filled = true;
 }
 }
 
-if(!filled)
+if (!filled)
 return null;
 
 return Base64Util.encodeUtf8String(user + ":" + pwd);
diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITeamcity.java 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITeamcity.java
index d9d8d1d..6fa8e63 100644
--- a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITeamcity.java
+++ b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITeamcity.java
@@ -18,6 +18,7 @@
 package org.apache.ignite.ci;
 
 import java.io.File;
+import java.io.IOException;
 import java.util.List;
 import java.util.concurrent.CompletableFuture;
 import java.util.concurrent.ExecutorService;
@@ -26,6 +27,7 @@
 import org.apache.ignite.ci.analysis.LogCheckResult;
 import org.apache.ignite.ci.analysis.MultBuildRunCtx;
 import org.apache.ignite.ci.analysis.SingleBuildRunCtx;
+import org.apache.ignite.ci.github.PullRequest;
 import org.apache.ignite.ci.tcmodel.agent.Agent;
 import org.apache.ignite.ci.tcmodel.changes.Change;
 import org.apache.ignite.ci.tcmodel.changes.ChangeRef;
@@ -240,8 +242,41 @@ default SingleBuildRunCtx loadTestsAndProblems(@Nonnull 
Build build, @Deprecated
  */
 void triggerBuild(String id, String name, boolean cleanRebuild, boolean 
queueAtTop);
 
+/**
+ * @return {@code True} if TeamCity authorization token is available.
+ */
+boolean isTeamCityTokenAvailable();
+
+/**
+ * @param token TeamCity authorization token.
+ */
 void setAuthToken(String token);
 
+/**
+ * @return {@code True} if GitHub authorization token is available.
+ */
+boolean isGitTokenAvailable();
+
+/**
+ * @param token GitHub authorization token.
+ */
+void setGitToken(String token);
+
+/**
+ * Send POST request with given body.
+ *
+ * @param url Url.
+ * @param body Request body.
+ * @return {@code True} - if GitHub was notified. {@code False} - 
otherwise.
+ */
+boolean notifyGit(String url, String body);
+
+/**
+ * @param branch TeamCity's branch name. Looks like "pull/123/head".
+ * @return Pull Request.
+ */
+PullRequest getPullRequest(String branch);
+
 default void setAuthData(String user, String password) {
 setAuthToken(
 Base64Util.encodeUtf8String(user + ":" + password));
diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/IgnitePersistentTeamcity.java
 
b/ignite-tc-helper-

[jira] [Created] (IGNITE-9559) Node.js thin: nodejs directory does appear in platforms directory after maven install command

2018-09-12 Thread Igor Sapego (JIRA)
Igor Sapego created IGNITE-9559:
---

 Summary: Node.js thin: nodejs directory does appear in platforms 
directory after maven install command
 Key: IGNITE-9559
 URL: https://issues.apache.org/jira/browse/IGNITE-9559
 Project: Ignite
  Issue Type: Bug
  Components: build, thin client
Reporter: Igor Sapego
 Fix For: 2.7


Currently, nodejs directory is not generated by the \{{maven install}} command. 
Need to add appropriate copy steps to \{{assembly/release-fabric-base.xml}} 
file.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] ignite pull request #4722: IGNITE-9531: ZookeeperDiscovery testClientReconne...

2018-09-12 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4722


---


[jira] [Created] (IGNITE-9558) Avoid changing AffinityTopologyVersion on client connect when possible

2018-09-12 Thread Alexey Goncharuk (JIRA)
Alexey Goncharuk created IGNITE-9558:


 Summary: Avoid changing AffinityTopologyVersion on client connect 
when possible
 Key: IGNITE-9558
 URL: https://issues.apache.org/jira/browse/IGNITE-9558
 Project: Ignite
  Issue Type: Improvement
Affects Versions: 2.0
Reporter: Alexey Goncharuk


Currently a client join event changes discovery topology version which, in 
turn, changes AffinityTopologyVersion.
When a client maps transaction on new AffinityTopologyVersion, corresponding 
message is not processed on remote node until remote node receives the 
corresponding discovery event. If discovery event delivery is delayed for some 
reason, this will result in transaction stalls on client joins.

Since the client node does not change partition affinity, we can safely map 
transactions on the previous topology version and do not change the affinity 
topology version at all.
Some cases need special care and probably do not qualify for this optimization, 
such as when client has near cache or client hosts partition for REPLICATED 
cache.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] ignite pull request #4736: Ignite 6587 debug

2018-09-12 Thread andrey-kuznetsov
GitHub user andrey-kuznetsov opened a pull request:

https://github.com/apache/ignite/pull/4736

Ignite 6587 debug

For debug purposes.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/andrey-kuznetsov/ignite ignite-6587-debug

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4736.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4736


commit 3b25cb934838feb3300c7088db4b3840ec7f9ab5
Author: Andrey Kuznetsov 
Date:   2018-05-07T12:43:26Z

IGNITE-6587: Refactored ServerImpl.TcpServer to use GridWorker.

commit 665ba863ceb95b643a805d6920f3f384275dae20
Author: Andrey Kuznetsov 
Date:   2018-05-07T13:06:22Z

IGNITE-6587: Refactored ServerImpl.RingMessageWorker to use GridWorker.

commit b10bcd350efb55de95792b369996ba266b3612a9
Author: Andrey Kuznetsov 
Date:   2018-05-07T15:59:56Z

IGNITE-6587: Refactored WAL manager critical threads to use GridWorker.

commit 3154112c8bab70d1ff4fe20639648e8a9c7da373
Author: Andrey Kuznetsov 
Date:   2018-05-07T18:55:25Z

IGNITE-6587: Minor improvements.

commit a44fc8064a9ba8f42e4faacde018ac9bc1193f01
Author: Andrey Kuznetsov 
Date:   2018-05-08T10:52:01Z

IGNITE-6587: Refactored CommunicationWorker to use GridWorker.

commit 0efc5354f33666ac705c738d75955258084091de
Author: Andrey Kuznetsov 
Date:   2018-05-08T15:25:56Z

IGNITE-6587: (WIP) switching to timed waits/polls, updating heartbeats.

commit 48c889441a119ef2192c1ff61485c20f80659e82
Author: Andrey Kuznetsov 
Date:   2018-05-14T13:38:47Z

IGNITE-6587: Heartbeat updates in critical WAL manager workers.

commit 69059bda2071f7d6af0cef30cf8d77b160fc56be
Author: Andrey Kuznetsov 
Date:   2018-05-14T15:55:51Z

IGNITE-6587: (WiP) More heartbeat updates in critical workers.

commit 6fb2edf1fef15bb187481f89eddb3710d5daadba
Author: Andrey Kuznetsov 
Date:   2018-05-15T13:09:16Z

IGNITE-6587: Heartbeat updates in checkpointer.

commit 9cd8cba1fdc8c21d999df4c334daaef7e226aa0f
Author: Andrey Kuznetsov 
Date:   2018-05-15T13:23:17Z

IGNITE-6587: More heartbeat updates in critical workers.

commit 5245abc0c72812c4746127b25638a28dac7af11a
Author: Andrey Kuznetsov 
Date:   2018-05-16T20:14:31Z

IGNITE-6587: Extended StripedExecutor.Stripe from GridWorker.

commit 2eb448f6a7ff7a6be423742336480a7b98793c2d
Author: Andrey Kuznetsov 
Date:   2018-05-17T11:37:08Z

IGNITE-6587: Heartbeat updates in StripedExecutor workers.

commit 8304fa92879fe293f112d1c7ba515b5bd30cc271
Author: Andrey Kuznetsov 
Date:   2018-05-17T12:33:23Z

Merge branch 'master' into ignite-6587

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/FileWriteAheadLogManager.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/FsyncModeFileWriteAheadLogManager.java

commit 5051aa4d44e78b5362dd0094bc5a57566eea4855
Author: Andrey Kuznetsov 
Date:   2018-05-17T12:36:23Z

IGNITE-6587: Addition to merge commit.

commit 732d2c25013d94d67188f42b2f96c786eeaf6346
Author: Andrey Kuznetsov 
Date:   2018-05-17T12:42:05Z

IGNITE-6587: Updated WAL FileDecompressors to conform uniform poll style.

commit 0c6f7762f1bc709c1444e545ff6475edc3fd6152
Author: Andrey Kuznetsov 
Date:   2018-05-18T09:54:05Z

IGNITE-6587: Added Checkpointer to worker registry.

commit d64e2f8e97e9d8a339695087ceea1443f8f75141
Author: Andrey Kuznetsov 
Date:   2018-05-18T14:43:22Z

IGNITE-6587: Added Stripe workers to worker registry.

commit fd142e80f80e73d420c0a03923ba3a4783dc7f34
Author: Andrey Kuznetsov 
Date:   2018-05-21T11:40:46Z

IGNITE-6587: Optional reference to worker registry in GridNioServer.

commit de98f8782f1728cc7e4a2852fbd0931cb21d3af9
Author: Andrey Kuznetsov 
Date:   2018-05-21T11:42:11Z

IGNITE-6587: Added worker registry to communication SPI GridNioServer.

commit f24f93abe8b8a3833f556b204c9995d28cd9cf44
Author: Andrey Kuznetsov 
Date:   2018-05-21T13:47:11Z

Merge branch 'master' into ignite-6587

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/FileWriteAheadLogManager.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/FsyncModeFileWriteAheadLogManager.java

commit e3a7b9aaed79d45b205715b6b281db702e7be95d
Author: Andrey Kuznetsov 
Date:   2018-05-22T12:30:30Z

IGNITE-6587: Recovering after incorrect merge commit.

commit 45c64a1ebaf5dabcfdf64e35a14542c0bddaa4c2
Author: Andrey Kuznetsov 
Date:   2018-05-22T12:56:27Z

IGNITE-6587: Moved WAL timeout constants to proper place.

commit 95f1ce0a9c3002a72fa47c616f928521570d2d40
Author: Andrey Kuznetsov 
Date:   2018-05-22T13:12:36Z

IGNITE-6587: Made WAL FileDecompressors not critical again.

commit 538fa40

[GitHub] ignite pull request #4735: Cache 2 stab experiment

2018-09-12 Thread SpiderRus
GitHub user SpiderRus opened a pull request:

https://github.com/apache/ignite/pull/4735

Cache 2 stab experiment



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite 
ignite-stabilization-cache2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4735.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4735


commit c45f2aa1e50e4920ae35acb784a243a45e3e4e7d
Author: Alexey Stelmak 
Date:   2018-09-11T13:42:09Z

Test




---


[GitHub] ignite pull request #4634: IGNITE-8485: force encrypted mode for all caches

2018-09-12 Thread nizhikov
Github user nizhikov closed the pull request at:

https://github.com/apache/ignite/pull/4634


---


Re: Python thin client

2018-09-12 Thread Igor Sapego
Pavel,
Yes, I did. I tried completely clean environment, followed the same
steps and got the same error. Then I removed attr, and out of sudden
everything started working.

Dmitry,
Thanks, now it's more clear:
Handshake error: Unsupported version. Server expects binary protocol
version 1.1.0. Client provides 1.2.0.

Why do you use version 1.2.0, but have outdated server code? I
propose you to merge with or rebase onto Ignite's master branch.

Best Regards,
Igor


On Tue, Sep 11, 2018 at 11:21 PM Dmitry Melnichuk <
dmitry.melnic...@nobitlost.com> wrote:

> Igor,
>
> I have just commited an improvment to the HandshakeError message
> generation algorithm. I hope it is now easier to understand what expects
> what in case of binary protocol version mismatch.
>
> Thank you for pointing this out.
>
> On 9/12/18 2:13 AM, Igor Sapego wrote:
> > I managed to start tests, and now I'm getting the following message:
> >
> > pyignite.exceptions.HandshakeError: Handshake error: Unsupported
> > version. Expected protocol version: 1.1.0.
> >
> > It would be useful to print "Unexpected version" itself, because I can
> > not understand what is the issue.
> >
> > Best Regards,
> > Igor
>


[GitHub] ignite pull request #4734: IGNITE-9549 control.sh add command to collect inf...

2018-09-12 Thread a-polyakov
GitHub user a-polyakov opened a pull request:

https://github.com/apache/ignite/pull/4734

IGNITE-9549 control.sh add command to collect information on the dist…

…ribution of partitions and reset lost partitions

Signed-off-by: a-polyakov 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/a-polyakov/ignite IGNITE-9549

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4734.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4734


commit 17bc1534e4b8bf9c13cb072ac424a7592d310fba
Author: a-polyakov 
Date:   2018-09-12T10:16:34Z

IGNITE-9549 control.sh add command to collect information on the 
distribution of partitions and reset lost partitions

Signed-off-by: a-polyakov 




---


[GitHub] ignite pull request #4733: IGNITE-7618 validateCache synchronously waits for...

2018-09-12 Thread ibessonov
GitHub user ibessonov opened a pull request:

https://github.com/apache/ignite/pull/4733

IGNITE-7618 validateCache synchronously waits for state change



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7618

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4733.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4733


commit e73fc53dae414bb7990a67814387fad7ac143fb3
Author: ibessonov 
Date:   2018-09-12T09:43:43Z

IGNITE-7618 Avoid "publicApiActiveState" call in 
GridDhtTopologyFutureAdapter#validateCache




---


[jira] [Created] (IGNITE-9557) Assert exception on explain of DELETE query.

2018-09-12 Thread Vasiliy Sisko (JIRA)
Vasiliy Sisko created IGNITE-9557:
-

 Summary: Assert exception on explain of DELETE query.
 Key: IGNITE-9557
 URL: https://issues.apache.org/jira/browse/IGNITE-9557
 Project: Ignite
  Issue Type: Bug
  Components: sql
Reporter: Vasiliy Sisko


On execution of an query:
{code:java}
explain delete from "schema".type {code}
received exception:
{code:java}
java.lang.AssertionError
    at 
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.parseAndSplit(IgniteH2Indexing.java:2309)
    at 
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:2105)
    at 
org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2139)
    at 
org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2134)
    at 
org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
    at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2711)
    at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2148)
    at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2083)
    at 
org.apache.ignite.internal.visor.query.VisorQueryTask$VisorQueryJob.run(VisorQueryTask.java:94)
    at 
org.apache.ignite.internal.visor.query.VisorQueryTask$VisorQueryJob.run(VisorQueryTask.java:59)
    at org.apache.ignite.internal.visor.VisorJob.execute(VisorJob.java:69)
    at 
org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:568)
    at 
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6765)
    at 
org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:562)
    at 
org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:491)
    at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
    at 
org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1125)
    at 
org.apache.ignite.internal.processors.task.GridTaskWorker.sendRequest(GridTaskWorker.java:1420)
    at 
org.apache.ignite.internal.processors.task.GridTaskWorker.processMappedJobs(GridTaskWorker.java:666)
    at 
org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:538)
    at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
    at 
org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:764)
    at 
org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:509)
    at 
org.apache.ignite.internal.IgniteComputeImpl.executeAsync0(IgniteComputeImpl.java:488)
    at 
org.apache.ignite.internal.IgniteComputeImpl.executeAsync(IgniteComputeImpl.java:468)
    at 
org.apache.ignite.internal.visor.compute.VisorGatewayTask$VisorGatewayJob.execute(VisorGatewayTask.java:462)
    at 
org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:568)
    at 
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6765)
    at 
org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:562)
    at 
org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:491)
    at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
    at 
org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1125)
    at 
org.apache.ignite.internal.processors.task.GridTaskWorker.sendRequest(GridTaskWorker.java:1420)
    at 
org.apache.ignite.internal.processors.task.GridTaskWorker.processMappedJobs(GridTaskWorker.java:666)
    at 
org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:538)
    at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
    at 
org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:764)
    at 
org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:509)
    at 
org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:489)
    at 
org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler.handleAsyncUnsafe(GridTaskCommandHandler.java:227)
    at 
org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler.handleAsync(GridTaskCommandHandler.java:163)
    at 
org.apache.ignite.internal.processors.rest.GridRestProcessor.handleRequest(GridRestProcessor.java:318)
    at 
org.apache.ignite.internal.processors.rest.GridRestProcessor.access$100(GridRestProcessor.java:99)
    at 
org.apache.ignite.internal.processors.rest.Gri

[jira] [Created] (IGNITE-9556) Web console: Babel 7 rc.2 compatibility issues

2018-09-12 Thread Ilya Borisov (JIRA)
Ilya Borisov created IGNITE-9556:


 Summary: Web console: Babel 7 rc.2 compatibility issues
 Key: IGNITE-9556
 URL: https://issues.apache.org/jira/browse/IGNITE-9556
 Project: Ignite
  Issue Type: Bug
  Components: wizards
Reporter: Ilya Borisov
Assignee: Ilya Borisov


Bebel 7 rc.2 causes at least one breaking issue: AngularJS-module-related run 
time error happens when built for production target.
!screenshot-1.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)