Platform .NET (Core Linux) Exit code 1 (new) On TeamCity

[kcheng.mvp]

I have triggered my tests many times on TeamCity. every time I get the same
result.

Platform .NET (Core Linux)  pull/4300/head  #1109   Exit code 1 (new) 

https://ci.ignite.apache.org/viewLog.html?buildId=1460429&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_RunBasicTests.

I checked the history of 'Platform .NET (Core Linux) ' and found there are
many other build also runs into the same result.





--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[GitHub] ignite pull request #4319: IGNITE-8945: Add additional checks with informati...

[ivandasch]

GitHub user ivandasch opened a pull request:

https://github.com/apache/ignite/pull/4319

IGNITE-8945: Add additional checks with informative error message

and implement more robust storedcachedata save.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ivandasch/ignite ignite-8945

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4319.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4319


commit c8e80325c8862d76289516a5053a4b5b66738044
Author: Ivan Daschinskiy 
Date:   2018-07-05T22:38:21Z

IGNITE-8945: Add additional checks with informative error message
and implement more robust storedcachedata save.




---

[jira] [Created] (IGNITE-8946) AssertionError can occur during reservation of WAL history for historical rebalance

[Ivan Rakov (JIRA)]

Ivan Rakov created IGNITE-8946:
--

 Summary: AssertionError can occur during reservation of WAL 
history for historical rebalance
 Key: IGNITE-8946
 URL: https://issues.apache.org/jira/browse/IGNITE-8946
 Project: Ignite
  Issue Type: Bug
Reporter: Ivan Rakov


Attempt to release WAL after exchange may fail with AssertionError. Seems like 
we have a bug and may try to release more WAL segments than we have reserved:
{noformat}
java.lang.AssertionError: null
at 
org.apache.ignite.internal.processors.cache.persistence.wal.SegmentReservationStorage.release(SegmentReservationStorage.java:54)
  - locked <0x1c12> (a 
org.apache.ignite.internal.processors.cache.persistence.wal.SegmentReservationStorage)
  at 
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.release(FileWriteAheadLogManager.java:862)
  at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.releaseHistoryForExchange(GridCacheDatabaseSharedManager.java:1691)
  - locked <0x1c17> (a 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager)
  at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:1751)
  at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.finishExchangeOnCoordinator(GridDhtPartitionsExchangeFuture.java:2858)
  at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onAllReceived(GridDhtPartitionsExchangeFuture.java:2591)
  at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.processSingleMessage(GridDhtPartitionsExchangeFuture.java:2283)
  at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.access$100(GridDhtPartitionsExchangeFuture.java:129)
  at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply(GridDhtPartitionsExchangeFuture.java:2140)
  at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$2.apply(GridDhtPartitionsExchangeFuture.java:2128)
  at 
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:383)
  at 
org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:353)
  at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onReceiveSingleMessage(GridDhtPartitionsExchangeFuture.java:2128)
  at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.processSinglePartitionUpdate(GridCachePartitionExchangeManager.java:1580)
  at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.access$1000(GridCachePartitionExchangeManager.java:138)
  at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$2.onMessage(GridCachePartitionExchangeManager.java:345)
  at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$2.onMessage(GridCachePartitionExchangeManager.java:325)
  at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCachePartitionExchangeManager.java:2848)
  at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCachePartitionExchangeManager.java:2827)
  at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056)
  at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581)
  at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380)
  at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306)
  at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101)
  at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295)
  at 
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
  at 
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
  at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125)
  at 
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091)
  at 
java.util.con

Re: Ignite as distributed file storage

[Pavel Kovalenko]

Vladimir,

I just want to add to my words, that we can implement BLOB storage and
then, if community really wants it, we can adapt this storage to use as
underlying file system in IGFS. But IGFS shouldn't be entry point for BLOB
storage. I think this conclusion can satisfy both of us.

2018-07-06 0:47 GMT+03:00 Pavel Kovalenko :

> Vladimir,
>
> I didn't say that it stores data in on-heap, I said that it performs a lot
> of operations with byte[] arrays in on-heap as I see in , which will lead
> to frequent GCs and unnecessary data copying.
> "But the whole idea around mmap sounds like premature optimisation to me"
> - this is not premature optimisation, this is on of the key performance
> features. E.g. Apache Kafka wouldn't be so fast and extremely performant
> without zero-copy.
> If we can do better, why not just do it? Especially if it costs nothing
> for us (This is OS level).
> As I said in my first message, our end target is handling video and
> streaming, copying every chunk of it to on-heap userspace then to offheap
> and then to disk is unacceptable.
> You ask me to implement almost anything using just IGFS interface, why we
> need to do that? Proxy mode looks like crutch, to support replication and
> possibility to have some data in-memory I need to write again a lot of
> stuff.
> Let's finally leave IGFS alone and wait for IEP.
>
>
> 2018-07-06 0:01 GMT+03:00 Vladimir Ozerov :
>
>> Pavel,
>>
>> IGFS doesn't enforce you to have block in heap. What you suggest can be
>> achieved with IGFS as follows:
>> 1) Disable caching, so data cache is not used ("PROXY" mode)
>> 2) Implement IgniteFileSystem interface which operates on abstract streams
>>
>> But the whole idea around mmap sounds like premature optimisation to me. I
>> conducted a number of experiments with IGFS on large Hadoop workload. Even
>> with old AI 1.x architecture, where everything was stored onheap, I never
>> had an issue with GC. The key point is that IGFS operates on large (64Kb)
>> data blocks, so even with 100Gb full of these blocks you will have
>> relatively small number of objects and normal GC pauses. Additional memory
>> copying is not an issue either in most workloads in distributed systems,
>> because most of the time is spent on IO and internal synchronization
>> anyway.
>>
>> Do you have specific scenario when you observed long GC pauses with GC or
>> serious performance degradation with IGFS?
>>
>> Even if we agree that mmap usage is a critical piece, all we need is to
>> implement a single IGFS interface.
>>
>> On Thu, Jul 5, 2018 at 10:44 PM Pavel Kovalenko 
>> wrote:
>>
>> > Vladimir,
>> >
>> > The key difference between BLOB storage and IGFS is that BLOB storage
>> will
>> > have persistent-based architecture with possibility to cache blocks in
>> > offheap (using mmap, which is more simple, because we delegate it to OS
>> > level)
>> > , while IGFS has in-memory based architecture with possibility to
>> persist
>> > blocks.
>> > BLOB storage will have possibility to work with small amount of RAM
>> without
>> > signficant performance drop (Using zero-copy from socket to disk) and in
>> > opposite case it can keep all available blocks in offheap if it's
>> possible
>> > (Using mmap again).
>> > IGFS perform a lot of operations with blocks in on-heap which leads to
>> > unnecessary data copies, long GC pauses and performance drop. All IGFS
>> > architecture tightly bound with in-memory features, so it's too hard to
>> > rewrite IGFS in persistent-based manner. But, cool IGFS features such as
>> > intelligent affinity routing, chunk colocation will be reused in BLOB
>> > storage.
>> > Does it make sense?
>> >
>> >
>> >
>> > 2018-07-05 19:01 GMT+03:00 Vladimir Ozerov :
>> >
>> > > Pavel,
>> > > Design you described is almost precisely what IGFS does. It has a
>> cache
>> > for
>> > > metadata, split binary data in chunks with intelligent affinity
>> routing.
>> > In
>> > > addition we have map-reduce feature on top of it and integration with
>> > > underlying file system with optional caching. Data can be accessed in
>> > > blocks or streams. IGFS is not in active development, but it is not
>> > > outdated either.
>> > > Can you shortly explain why do you think that we need to drop IGFS and
>> > > re-implement almost the same thing from scratch?
>> > >
>> > > Dima, Sergey,
>> > > Yes, we need BLOB support you described. Unfortunately it is not that
>> > easy
>> > > to implement from SQL perspective. To support it we would need either
>> > MVCC
>> > > (with it's own drawbacks) or read-locks for SELECT.
>> > >
>> > > Vladimir.
>> > >
>> > > On Tue, Jul 3, 2018 at 10:40 AM Sergey Kozlov 
>> > > wrote:
>> > >
>> > > > Dmitriy
>> > > >
>> > > > You're right that that large objects storing should be optmized.
>> > > >
>> > > > Let's assume the large object means the regular object having large
>> > > fields
>> > > > and such fileds won't be used for comparison thus we can do not
>> restore
>> > > the
>> > > > BLOB fields 

Re: Add cluster (de)activation events IGNITE-8376

[Dmitriy Setrakyan]

On Thu, Jul 5, 2018 at 1:55 AM, Evgenii Zhuravlev 
wrote:

> Guys,
>
> Do we really need events for activation/deactivation? We already have a
> ticket for implementation lifecycle events for it:
> https://issues.apache.org/jira/browse/IGNITE-5427, won't it be enough?
>

Hm... I think these two tickets are duplicates of one another, no?

Re: Ignite as distributed file storage

[Pavel Kovalenko]

Vladimir,

I didn't say that it stores data in on-heap, I said that it performs a lot
of operations with byte[] arrays in on-heap as I see in , which will lead
to frequent GCs and unnecessary data copying.
"But the whole idea around mmap sounds like premature optimisation to me" -
this is not premature optimisation, this is on of the key performance
features. E.g. Apache Kafka wouldn't be so fast and extremely performant
without zero-copy.
If we can do better, why not just do it? Especially if it costs nothing for
us (This is OS level).
As I said in my first message, our end target is handling video and
streaming, copying every chunk of it to on-heap userspace then to offheap
and then to disk is unacceptable.
You ask me to implement almost anything using just IGFS interface, why we
need to do that? Proxy mode looks like crutch, to support replication and
possibility to have some data in-memory I need to write again a lot of
stuff.
Let's finally leave IGFS alone and wait for IEP.


2018-07-06 0:01 GMT+03:00 Vladimir Ozerov :

> Pavel,
>
> IGFS doesn't enforce you to have block in heap. What you suggest can be
> achieved with IGFS as follows:
> 1) Disable caching, so data cache is not used ("PROXY" mode)
> 2) Implement IgniteFileSystem interface which operates on abstract streams
>
> But the whole idea around mmap sounds like premature optimisation to me. I
> conducted a number of experiments with IGFS on large Hadoop workload. Even
> with old AI 1.x architecture, where everything was stored onheap, I never
> had an issue with GC. The key point is that IGFS operates on large (64Kb)
> data blocks, so even with 100Gb full of these blocks you will have
> relatively small number of objects and normal GC pauses. Additional memory
> copying is not an issue either in most workloads in distributed systems,
> because most of the time is spent on IO and internal synchronization
> anyway.
>
> Do you have specific scenario when you observed long GC pauses with GC or
> serious performance degradation with IGFS?
>
> Even if we agree that mmap usage is a critical piece, all we need is to
> implement a single IGFS interface.
>
> On Thu, Jul 5, 2018 at 10:44 PM Pavel Kovalenko 
> wrote:
>
> > Vladimir,
> >
> > The key difference between BLOB storage and IGFS is that BLOB storage
> will
> > have persistent-based architecture with possibility to cache blocks in
> > offheap (using mmap, which is more simple, because we delegate it to OS
> > level)
> > , while IGFS has in-memory based architecture with possibility to persist
> > blocks.
> > BLOB storage will have possibility to work with small amount of RAM
> without
> > signficant performance drop (Using zero-copy from socket to disk) and in
> > opposite case it can keep all available blocks in offheap if it's
> possible
> > (Using mmap again).
> > IGFS perform a lot of operations with blocks in on-heap which leads to
> > unnecessary data copies, long GC pauses and performance drop. All IGFS
> > architecture tightly bound with in-memory features, so it's too hard to
> > rewrite IGFS in persistent-based manner. But, cool IGFS features such as
> > intelligent affinity routing, chunk colocation will be reused in BLOB
> > storage.
> > Does it make sense?
> >
> >
> >
> > 2018-07-05 19:01 GMT+03:00 Vladimir Ozerov :
> >
> > > Pavel,
> > > Design you described is almost precisely what IGFS does. It has a cache
> > for
> > > metadata, split binary data in chunks with intelligent affinity
> routing.
> > In
> > > addition we have map-reduce feature on top of it and integration with
> > > underlying file system with optional caching. Data can be accessed in
> > > blocks or streams. IGFS is not in active development, but it is not
> > > outdated either.
> > > Can you shortly explain why do you think that we need to drop IGFS and
> > > re-implement almost the same thing from scratch?
> > >
> > > Dima, Sergey,
> > > Yes, we need BLOB support you described. Unfortunately it is not that
> > easy
> > > to implement from SQL perspective. To support it we would need either
> > MVCC
> > > (with it's own drawbacks) or read-locks for SELECT.
> > >
> > > Vladimir.
> > >
> > > On Tue, Jul 3, 2018 at 10:40 AM Sergey Kozlov 
> > > wrote:
> > >
> > > > Dmitriy
> > > >
> > > > You're right that that large objects storing should be optmized.
> > > >
> > > > Let's assume the large object means the regular object having large
> > > fields
> > > > and such fileds won't be used for comparison thus we can do not
> restore
> > > the
> > > > BLOB fields in offheap page memory e.g for sql queries if select
> > doesn't
> > > > include them explicitly. It can reduce page eviction and speed up the
> > > > perfomance and make less chance to get OOM.
> > > >
> > > >
> > > >
> > > > On Tue, Jul 3, 2018 at 1:06 AM, Dmitriy Setrakyan <
> > dsetrak...@apache.org
> > > >
> > > > wrote:
> > > >
> > > > > To be honest, I am not sure if we need to kick off another file
> > system
> > > > > storage discussion in Ignite. It soun

Re: Ignite as distributed file storage

[Vladimir Ozerov]

Pavel,

IGFS doesn't enforce you to have block in heap. What you suggest can be
achieved with IGFS as follows:
1) Disable caching, so data cache is not used ("PROXY" mode)
2) Implement IgniteFileSystem interface which operates on abstract streams

But the whole idea around mmap sounds like premature optimisation to me. I
conducted a number of experiments with IGFS on large Hadoop workload. Even
with old AI 1.x architecture, where everything was stored onheap, I never
had an issue with GC. The key point is that IGFS operates on large (64Kb)
data blocks, so even with 100Gb full of these blocks you will have
relatively small number of objects and normal GC pauses. Additional memory
copying is not an issue either in most workloads in distributed systems,
because most of the time is spent on IO and internal synchronization anyway.

Do you have specific scenario when you observed long GC pauses with GC or
serious performance degradation with IGFS?

Even if we agree that mmap usage is a critical piece, all we need is to
implement a single IGFS interface.

On Thu, Jul 5, 2018 at 10:44 PM Pavel Kovalenko  wrote:

> Vladimir,
>
> The key difference between BLOB storage and IGFS is that BLOB storage will
> have persistent-based architecture with possibility to cache blocks in
> offheap (using mmap, which is more simple, because we delegate it to OS
> level)
> , while IGFS has in-memory based architecture with possibility to persist
> blocks.
> BLOB storage will have possibility to work with small amount of RAM without
> signficant performance drop (Using zero-copy from socket to disk) and in
> opposite case it can keep all available blocks in offheap if it's possible
> (Using mmap again).
> IGFS perform a lot of operations with blocks in on-heap which leads to
> unnecessary data copies, long GC pauses and performance drop. All IGFS
> architecture tightly bound with in-memory features, so it's too hard to
> rewrite IGFS in persistent-based manner. But, cool IGFS features such as
> intelligent affinity routing, chunk colocation will be reused in BLOB
> storage.
> Does it make sense?
>
>
>
> 2018-07-05 19:01 GMT+03:00 Vladimir Ozerov :
>
> > Pavel,
> > Design you described is almost precisely what IGFS does. It has a cache
> for
> > metadata, split binary data in chunks with intelligent affinity routing.
> In
> > addition we have map-reduce feature on top of it and integration with
> > underlying file system with optional caching. Data can be accessed in
> > blocks or streams. IGFS is not in active development, but it is not
> > outdated either.
> > Can you shortly explain why do you think that we need to drop IGFS and
> > re-implement almost the same thing from scratch?
> >
> > Dima, Sergey,
> > Yes, we need BLOB support you described. Unfortunately it is not that
> easy
> > to implement from SQL perspective. To support it we would need either
> MVCC
> > (with it's own drawbacks) or read-locks for SELECT.
> >
> > Vladimir.
> >
> > On Tue, Jul 3, 2018 at 10:40 AM Sergey Kozlov 
> > wrote:
> >
> > > Dmitriy
> > >
> > > You're right that that large objects storing should be optmized.
> > >
> > > Let's assume the large object means the regular object having large
> > fields
> > > and such fileds won't be used for comparison thus we can do not restore
> > the
> > > BLOB fields in offheap page memory e.g for sql queries if select
> doesn't
> > > include them explicitly. It can reduce page eviction and speed up the
> > > perfomance and make less chance to get OOM.
> > >
> > >
> > >
> > > On Tue, Jul 3, 2018 at 1:06 AM, Dmitriy Setrakyan <
> dsetrak...@apache.org
> > >
> > > wrote:
> > >
> > > > To be honest, I am not sure if we need to kick off another file
> system
> > > > storage discussion in Ignite. It sounds like a huge effort and likely
> > > will
> > > > not be productive.
> > > >
> > > > However, I think an ability to store large objects will make sense.
> For
> > > > example, how do I store a 10GB blob in Ignite cache? Most likely we
> > have
> > > to
> > > > have a separate memory or disk space, allocated for blobs only. We
> also
> > > > need to be able to efficiently transfer a 10GB Blob object over the
> > > network
> > > > and store it off-heap right away, without bringing it into main heap
> > > memory
> > > > (otherwise we would run out of memory).
> > > >
> > > > I suggest that we create an IEP about this use case alone and leave
> the
> > > > file system for the future discussions.
> > > >
> > > > D.
> > > >
> > > > On Mon, Jul 2, 2018 at 6:50 AM, Vladimir Ozerov <
> voze...@gridgain.com>
> > > > wrote:
> > > >
> > > > > Pavel,
> > > > >
> > > > > Thank you. I'll wait for feature comparison and concrete use cases,
> > > > because
> > > > > for me this feature still sounds too abstract to judge whether
> > product
> > > > > would benefit from it.
> > > > >
> > > > > On Mon, Jul 2, 2018 at 3:15 PM Pavel Kovalenko  >
> > > > wrote:
> > > > >
> > > > > > Dmitriy,
> > > > > >
> > > > > > I think we have a little 

[jira] [Created] (IGNITE-8945) Stored cache data files corruption when node stops abruptly.

[Ivan Daschinskiy (JIRA)]

Ivan Daschinskiy created IGNITE-8945:


 Summary: Stored cache data files corruption when node stops 
abruptly.
 Key: IGNITE-8945
 URL: https://issues.apache.org/jira/browse/IGNITE-8945
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.5
Reporter: Ivan Daschinskiy
Assignee: Ivan Daschinskiy
 Fix For: 2.7


When node is halted during saving stored cache data, content of this file can 
be corrupted. 
1. Additional check should be implemented in FilePageStoreManager.readCacheData 
 
(print the name of corrupted file)
2. In storeCacheData we need to serialize StoredCacheData to temp file then 
swap.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Ignite as distributed file storage

[Pavel Kovalenko]

Vladimir,

The key difference between BLOB storage and IGFS is that BLOB storage will
have persistent-based architecture with possibility to cache blocks in
offheap (using mmap, which is more simple, because we delegate it to OS
level)
, while IGFS has in-memory based architecture with possibility to persist
blocks.
BLOB storage will have possibility to work with small amount of RAM without
signficant performance drop (Using zero-copy from socket to disk) and in
opposite case it can keep all available blocks in offheap if it's possible
(Using mmap again).
IGFS perform a lot of operations with blocks in on-heap which leads to
unnecessary data copies, long GC pauses and performance drop. All IGFS
architecture tightly bound with in-memory features, so it's too hard to
rewrite IGFS in persistent-based manner. But, cool IGFS features such as
intelligent affinity routing, chunk colocation will be reused in BLOB
storage.
Does it make sense?



2018-07-05 19:01 GMT+03:00 Vladimir Ozerov :

> Pavel,
> Design you described is almost precisely what IGFS does. It has a cache for
> metadata, split binary data in chunks with intelligent affinity routing. In
> addition we have map-reduce feature on top of it and integration with
> underlying file system with optional caching. Data can be accessed in
> blocks or streams. IGFS is not in active development, but it is not
> outdated either.
> Can you shortly explain why do you think that we need to drop IGFS and
> re-implement almost the same thing from scratch?
>
> Dima, Sergey,
> Yes, we need BLOB support you described. Unfortunately it is not that easy
> to implement from SQL perspective. To support it we would need either MVCC
> (with it's own drawbacks) or read-locks for SELECT.
>
> Vladimir.
>
> On Tue, Jul 3, 2018 at 10:40 AM Sergey Kozlov 
> wrote:
>
> > Dmitriy
> >
> > You're right that that large objects storing should be optmized.
> >
> > Let's assume the large object means the regular object having large
> fields
> > and such fileds won't be used for comparison thus we can do not restore
> the
> > BLOB fields in offheap page memory e.g for sql queries if select doesn't
> > include them explicitly. It can reduce page eviction and speed up the
> > perfomance and make less chance to get OOM.
> >
> >
> >
> > On Tue, Jul 3, 2018 at 1:06 AM, Dmitriy Setrakyan  >
> > wrote:
> >
> > > To be honest, I am not sure if we need to kick off another file system
> > > storage discussion in Ignite. It sounds like a huge effort and likely
> > will
> > > not be productive.
> > >
> > > However, I think an ability to store large objects will make sense. For
> > > example, how do I store a 10GB blob in Ignite cache? Most likely we
> have
> > to
> > > have a separate memory or disk space, allocated for blobs only. We also
> > > need to be able to efficiently transfer a 10GB Blob object over the
> > network
> > > and store it off-heap right away, without bringing it into main heap
> > memory
> > > (otherwise we would run out of memory).
> > >
> > > I suggest that we create an IEP about this use case alone and leave the
> > > file system for the future discussions.
> > >
> > > D.
> > >
> > > On Mon, Jul 2, 2018 at 6:50 AM, Vladimir Ozerov 
> > > wrote:
> > >
> > > > Pavel,
> > > >
> > > > Thank you. I'll wait for feature comparison and concrete use cases,
> > > because
> > > > for me this feature still sounds too abstract to judge whether
> product
> > > > would benefit from it.
> > > >
> > > > On Mon, Jul 2, 2018 at 3:15 PM Pavel Kovalenko 
> > > wrote:
> > > >
> > > > > Dmitriy,
> > > > >
> > > > > I think we have a little miscommunication here. Of course, I meant
> > > > > supporting large entries / chunks of binary data. Internally it
> will
> > be
> > > > > BLOB storage, which can be accessed through various interfaces.
> > > > > "File" is just an abstraction for an end user for convenience, a
> > > wrapper
> > > > > layer to have user-friendly API to directly store BLOBs. We
> shouldn't
> > > > > support full file protocol support with file system capabilities.
> It
> > > can
> > > > be
> > > > > added later, but now it's absolutely unnecessary and introduces
> extra
> > > > > complexity.
> > > > >
> > > > > We can implement our BLOB storage step by step. The first thing is
> > > > > core functionality and support to save large parts of binary
> objects
> > to
> > > > it.
> > > > > "File" layer, Web layer, etc. can be added later.
> > > > >
> > > > > The initial IGFS design doesn't have good capabilities to have a
> > > > > persistence layer. I think we shouldn't do any changes to it, this
> > > > project
> > > > > as for me is almost outdated. We will drop IGFS after implementing
> > File
> > > > > System layer over our BLOB storage.
> > > > >
> > > > > Vladimir,
> > > > >
> > > > > I will prepare a comparison with other existing distributed file
> > > storages
> > > > > and file systems in a few days.
> > > > >
> > > > > About usage data grid, I never said, that we need transacti

Re: Erroneous Baseline Topology documentation

[Prachi Garg]

Thanks Ivan. I will fix the doc accordingly.

On Thu, Jul 5, 2018 at 5:31 AM, Ivan Rakov  wrote:

> I guess just activating the cluster would add all the existing nodes to
> the baseline topology?
>
> Exactly. Persistent cluster can't exist in active state without baseline
> topology. First activation will establish BLT from current set of server
> nodes.
>
> Best Regards,
> Ivan Rakov
>
> On 04.07.2018 1:55, Prachi Garg wrote:
>
> Hi Ivan,
>
> I have fixed and rephrased the section - https://apacheignite.readme.
> io/v2.5/docs/baseline-topology#section-cluster-activation
>
> However, I have a question regarding setting the baseline topology when
> activating the cluster for the first time. In the web console, when we
> activate the cluster, using the toggle switch, all server nodes in the
> cluster are automatically added to the baseline topology. Does this mean
> that when we activate the cluster for the first time, via code, we do not
> need the following piece code?
>
> // Get all server nodes that are already up and running.
> Collection nodes = ignite.cluster().forServers().nodes();
>
> // Set the baseline topology that is represented by these nodes.
> ignite.cluster().setBaselineTopology(nodes);
>
> I guess just activating the cluster would add all the existing nodes to
> the baseline topology?
>
>
>
> On Tue, Jul 3, 2018 at 12:48 PM, Ivan Rakov  wrote:
>
>> I've tried to execute exactly the same code, it resulted with
>>
>> class org.apache.ignite.IgniteException: Changing BaselineTopology on
>>> inactive cluster is not allowed.
>>>
>> Basically, the code snippet is in "Setting the Topology From Code"
>> section, so we can make it correct by just removing "activation" and "first
>> baseline topology" parts.
>>
>> Best Regards,
>> Ivan Rakov
>>
>>
>>
>> On 03.07.2018 22:30, Denis Magda wrote:
>>
>>> Prachi,
>>>
>>> I do remember that that code, Ivan is referring to, worked fine for you.
>>> Please double check. Probably you need to add "ignite.cluster.activate()"
>>> to the code snippet.
>>>
>>> --
>>> Denis
>>>
>>> On Tue, Jul 3, 2018 at 12:19 PM Ivan Rakov 
>>> wrote:
>>>
>>> Igniters,

 Seems like we have an inconsistency in our Baseline Topology
 documentation:

 https://apacheignite.readme.io/docs/baseline-topology#sectio
 n-setting-the-topology-from-code

 Java developers can use the IgniteCluster interface to initialize the
> very first baseline topology or to adjust an existing one. The sample
> below shows how to add all the existing server nodes to the baseline
> topology:​
> // Connecting to the cluster.
> Ignite ignite = Ignition.start();
>
> // Setting the baseline topology to a specific Ignite cluster topology
> version.
> ignite.cluster().setBaselineTopology(2);
>
 This is not true; baseline topology can't be changed on inactive
 cluster. The only viable way to initialize the very first baseline
 topology is manual cluster activation. This is correctly explained in
 this section:

 https://apacheignite.readme.io/docs/baseline-topology#sectio
 n-first-cluster-startup

 --
 Best Regards,
 Ivan Rakov



>>
>
>

[GitHub] ignite pull request #4204: IGNITE-8789 Invoke failure processor in case of m...

[agura]

Github user agura closed the pull request at:

https://github.com/apache/ignite/pull/4204


---

[GitHub] ignite pull request #4306: IGNITE-8931 Spring Framework versions upgraded

[agura]

Github user agura closed the pull request at:

https://github.com/apache/ignite/pull/4306


---

[jira] [Created] (IGNITE-8944) TcpDiscoverySpi: set connection check frequency to fixed value

[Dmitry Karachentsev (JIRA)]

Dmitry Karachentsev created IGNITE-8944:
---

 Summary: TcpDiscoverySpi: set connection check frequency to fixed 
value
 Key: IGNITE-8944
 URL: https://issues.apache.org/jira/browse/IGNITE-8944
 Project: Ignite
  Issue Type: Improvement
Reporter: Dmitry Karachentsev


Now connCheckFreq value calculated by some not obvious code:
{code:java}
private void initConnectionCheckFrequency() {
if (spi.failureDetectionTimeoutEnabled())
connCheckThreshold = spi.failureDetectionTimeout();
else
connCheckThreshold = Math.min(spi.getSocketTimeout(), 
spi.metricsUpdateFreq);

for (int i = 3; i > 0; i--) {
connCheckFreq = connCheckThreshold / i;

if (connCheckFreq > 10)
break;
}

assert connCheckFreq > 0;

if (log.isInfoEnabled())
log.info("Connection check frequency is calculated: " + 
connCheckFreq);
}
{code}

It should be replaced with constant.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4318: IGNITE-8935 toString() or exclusion for most clas...

[alamar]

GitHub user alamar opened a pull request:

https://github.com/apache/ignite/pull/4318

IGNITE-8935 toString() or exclusion for most classes accessible from 
IgniteConfiguration



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8935

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4318.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4318


commit 2c664b7be5cce57f5e5e1ecb97705f372632c36a
Author: Ilya Kasnacheev 
Date:   2018-07-05T17:52:51Z

IGNITE-8935 toString() or exclusion for most classes accessible from 
IgniteConfiguration.




---

[GitHub] ignite pull request #4299: Ignite 2.4.6.b1

[dkarachentsev]

Github user dkarachentsev closed the pull request at:

https://github.com/apache/ignite/pull/4299


---

[jira] [Created] (IGNITE-8943) Deactivation in large cluster hangs during rebalance

[Andrew Medvedev (JIRA)]

Andrew Medvedev created IGNITE-8943:
---

 Summary: Deactivation in large cluster hangs during rebalance
 Key: IGNITE-8943
 URL: https://issues.apache.org/jira/browse/IGNITE-8943
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.5
Reporter: Andrew Medvedev


In large cluster (> 100 nodes) deactivation during rebalance does not finish 
(we have waited for at least an hour).

All nodes log "Start deactivation process" message, but on all nodes 
exchange-worker threads in cluster are WAITING with this stack:

"exchange-worker-#152%DPL_GRID%DplGridNodeName%" #520 daemon prio=5 os_prio=0 
tid=0x7f5cfb2b6000 nid=0x2dc7bc waiting on condition [0x7f59a05b2000]
   java.lang.Thread.State: WAITING (parking)
  at sun.misc.Unsafe.park(Native Method)
  at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
  at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
  at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
  at 
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.cancelInternalQuery(CacheContinuousQueryManager.java:575)
  at 
org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.onKernalStop(DataStructuresProcessor.java:253)
  at 
org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.onDeActivate(DataStructuresProcessor.java:299)
  at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onClusterStateChangeRequest(GridDhtPartitionsExchangeFuture.java:954)
  at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:650)
  at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2452)
  at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2332)
  at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
  at java.lang.Thread.run(Thread.java:745)

This is reproducible



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4317: IGNITE-8766 - TcpDiscoverySpi: discovery threads ...

[dkarachentsev]

GitHub user dkarachentsev opened a pull request:

https://github.com/apache/ignite/pull/4317

IGNITE-8766 - TcpDiscoverySpi: discovery threads naming



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8766

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4317.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4317


commit 8971f29cefe49b1b30bd885919dedef71811fe99
Author: dkarachentsev 
Date:   2018-07-05T17:25:18Z

IGNITE-8766 - TcpDiscoverySpi: discovery threads naming




---

[GitHub] ignite pull request #4316: IGNITE-8938 Failure handling for file-decompresso...

[agura]

GitHub user agura opened a pull request:

https://github.com/apache/ignite/pull/4316

IGNITE-8938 Failure handling for file-decompressor thread added



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/agura/incubator-ignite ignite-8938

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4316.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4316


commit cd2b1562fe01cc4a1172aabdab74c3c1308b3237
Author: Andrey Gura 
Date:   2018-07-05T16:40:26Z

IGNITE-8938 Failure handling for file-decompressor thread added




---

[GitHub] ignite pull request #4315: ignite-8905 with merged master

[sergey-chugunov-1985]

GitHub user sergey-chugunov-1985 opened a pull request:

https://github.com/apache/ignite/pull/4315

ignite-8905 with merged master



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8905-mm

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4315.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4315


commit f2194169e9eda7232bc642bf3b1b5a1ad4967af0
Author: Sergey Chugunov 
Date:   2018-07-02T10:54:55Z

IGNITE-8905 incorrect assertion was replaced with if-check

commit d516fe70d377a5989b2f5324bc6c981f6917b776
Author: Sergey Chugunov 
Date:   2018-07-03T12:49:23Z

IGNITE-8905 additional check that doesn't work for clients in 'forceServer' 
mode was simplified

commit 7349f50139821eae51bceda69f79926c08acdfc3
Author: Sergey Chugunov 
Date:   2018-07-05T16:33:20Z

Merge branch 'master' into ignite-8905-mm




---

[jira] [Created] (IGNITE-8942) In some cases grid cannot be deactivated because of hanging CQ internal cleanup.

[Alexei Scherbakov (JIRA)]

Alexei Scherbakov created IGNITE-8942:
-

 Summary: In some cases grid cannot be deactivated because of 
hanging CQ internal cleanup.
 Key: IGNITE-8942
 URL: https://issues.apache.org/jira/browse/IGNITE-8942
 Project: Ignite
  Issue Type: Bug
Reporter: Alexei Scherbakov
Assignee: Alexei Scherbakov
 Attachments: thread_dump_eip-server_2018-07-05-18-02.log

See the attachment for thread dump.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Ignite as distributed file storage

[Vladimir Ozerov]

Pavel,
Design you described is almost precisely what IGFS does. It has a cache for
metadata, split binary data in chunks with intelligent affinity routing. In
addition we have map-reduce feature on top of it and integration with
underlying file system with optional caching. Data can be accessed in
blocks or streams. IGFS is not in active development, but it is not
outdated either.
Can you shortly explain why do you think that we need to drop IGFS and
re-implement almost the same thing from scratch?

Dima, Sergey,
Yes, we need BLOB support you described. Unfortunately it is not that easy
to implement from SQL perspective. To support it we would need either MVCC
(with it's own drawbacks) or read-locks for SELECT.

Vladimir.

On Tue, Jul 3, 2018 at 10:40 AM Sergey Kozlov  wrote:

> Dmitriy
>
> You're right that that large objects storing should be optmized.
>
> Let's assume the large object means the regular object having large fields
> and such fileds won't be used for comparison thus we can do not restore the
> BLOB fields in offheap page memory e.g for sql queries if select doesn't
> include them explicitly. It can reduce page eviction and speed up the
> perfomance and make less chance to get OOM.
>
>
>
> On Tue, Jul 3, 2018 at 1:06 AM, Dmitriy Setrakyan 
> wrote:
>
> > To be honest, I am not sure if we need to kick off another file system
> > storage discussion in Ignite. It sounds like a huge effort and likely
> will
> > not be productive.
> >
> > However, I think an ability to store large objects will make sense. For
> > example, how do I store a 10GB blob in Ignite cache? Most likely we have
> to
> > have a separate memory or disk space, allocated for blobs only. We also
> > need to be able to efficiently transfer a 10GB Blob object over the
> network
> > and store it off-heap right away, without bringing it into main heap
> memory
> > (otherwise we would run out of memory).
> >
> > I suggest that we create an IEP about this use case alone and leave the
> > file system for the future discussions.
> >
> > D.
> >
> > On Mon, Jul 2, 2018 at 6:50 AM, Vladimir Ozerov 
> > wrote:
> >
> > > Pavel,
> > >
> > > Thank you. I'll wait for feature comparison and concrete use cases,
> > because
> > > for me this feature still sounds too abstract to judge whether product
> > > would benefit from it.
> > >
> > > On Mon, Jul 2, 2018 at 3:15 PM Pavel Kovalenko 
> > wrote:
> > >
> > > > Dmitriy,
> > > >
> > > > I think we have a little miscommunication here. Of course, I meant
> > > > supporting large entries / chunks of binary data. Internally it will
> be
> > > > BLOB storage, which can be accessed through various interfaces.
> > > > "File" is just an abstraction for an end user for convenience, a
> > wrapper
> > > > layer to have user-friendly API to directly store BLOBs. We shouldn't
> > > > support full file protocol support with file system capabilities. It
> > can
> > > be
> > > > added later, but now it's absolutely unnecessary and introduces extra
> > > > complexity.
> > > >
> > > > We can implement our BLOB storage step by step. The first thing is
> > > > core functionality and support to save large parts of binary objects
> to
> > > it.
> > > > "File" layer, Web layer, etc. can be added later.
> > > >
> > > > The initial IGFS design doesn't have good capabilities to have a
> > > > persistence layer. I think we shouldn't do any changes to it, this
> > > project
> > > > as for me is almost outdated. We will drop IGFS after implementing
> File
> > > > System layer over our BLOB storage.
> > > >
> > > > Vladimir,
> > > >
> > > > I will prepare a comparison with other existing distributed file
> > storages
> > > > and file systems in a few days.
> > > >
> > > > About usage data grid, I never said, that we need transactions, sync
> > > backup
> > > > and etc. We need just a few core things - Atomic cache with
> > persistence,
> > > > Discovery, Baseline, Affinity, and Communication.
> > > > Other things we can implement by ourselves. So this feature can
> develop
> > > > independently of other non-core features.
> > > > For me Ignite way is providing to our users a fast and convenient way
> > to
> > > > solve their problems with good performance and durability. We have
> the
> > > > problem with storing large data, we should solve it.
> > > > About other things see my message to Dmitriy above.
> > > >
> > > > вс, 1 июл. 2018 г. в 9:48, Dmitriy Setrakyan  >:
> > > >
> > > > > Pavel,
> > > > >
> > > > > I have actually misunderstood the use case. To be honest, I thought
> > > that
> > > > > you were talking about the support of large values in Ignite
> caches,
> > > e.g.
> > > > > objects that are several megabytes in cache.
> > > > >
> > > > > If we are tackling the distributed file system, then in my view, we
> > > > should
> > > > > be talking about IGFS and adding persistence support to IGFS (which
> > is
> > > > > based on HDFS API). It is not clear to me that you are talking
> about
> > > > IGFS.
> > > > > Can y

Re: IgniteConfiguration, TcpDiscoverySpi, TcpCommunicationSpitimeouts

[Valentin Kulichenko]

Stan,

What is the purpose of clientFailureDetectionTimeout? Why can't we just
always use failureDetectionTimeout? Is there any difference between these
two timeouts?

-Val



On Wed, Jul 4, 2018 at 7:00 AM Stanislav Lukyanov 
wrote:

> Hi,
>
> I’ve updated the proposed documentation update with a description of
> metricsUpdateFrequency and a detailed description of
> failureDetectionTimeout and clientFailureDetectionTimeout relations. The
> draft is attached to https://issues.apache.org/jira/browse/IGNITE-7704.
>
> It seems that relation between failureDetectionTimeout and
> clientFailureDetectionTimeout is currently too tricky and should also be
> changed in future.
> The problem is that in a server-client connection the server will use
> clientFailureDetectionTimeout but client will use failureDetectionTimeout.
> In other words, clients ignore clientFailureDetectionTimeout and just use
> failureDetectionTimeout. Because of that, one has to provide different
> values of failureDetectionTimeout in server and client configs which seems
> confusing and inconvenient.
> So I’d like to add one more point to my earlier proposal:
>
> 5. Always use clientFailureDetectionTimeout on clients instead of
> failureDetectionTimeout
> *What*: change code to use clientFailureDetectionTimeout on clients
> *When*: update code and readme.io docs in 2.7
>
> Thanks,
> Stan
>
> From: Valentin Kulichenko
> Sent: 30 мая 2018 г. 19:09
> To: dev@ignite.apache.org
> Subject: Re: IgniteConfiguration, TcpDiscoverySpi,
> TcpCommunicationSpitimeouts
>
> Stan,
>
> Looks like you suggest to only change the default. If so, it's OK. But
> let's not change the behavior of these timeouts for the case they are
> explicitly set in config.
>
> Thanks,
> Val
>
> On Wed, May 30, 2018 at 1:06 AM, Stanislav Lukyanov <
> stanlukya...@gmail.com>
> wrote:
>
> > On networkTimeout: no, we don’t have anything like that in
> > TcpCommunicationSpi.
> >
> > On socketWriteTimeout:
> > First, its semantic is very close to TcpDicsoverySpi.socketTimeout (with
> > the exception that communication uses NIO), and the latter defaults to
> > failureDetectionTimeout,
> > so I think it would help to avoid confusion.
> > Second, I think we can’t deprecate something without an alternative that
> > would work for most users.
> > On the other hand, if we do default socketWriteTimeout to
> > failureDetectionTimeout then we reach a pretty decent API state
> > where one only needs two properties in IgniteConfiguration neither of
> > which we’re considering for deprecation and removal in 3.0.
> >
> > Stan
> >
> > From: Valentin Kulichenko
> > Sent: 29 мая 2018 г. 22:17
> > To: dev@ignite.apache.org
> > Subject: Re: IgniteConfiguration, TcpDiscoverySpi,
> > TcpCommunicationSpitimeouts
> >
> > Stan,
> >
> > OK, I got confused a little :)
> >
> > I do agree that TcpDiscoverySpi.networkTimeout should inherit from
> > IgniteConfiguration.networkTImeout if not set explicitly. Do we have the
> > same setting for TcpCommunicationSpi, BTW? If yes, behavior should be
> > consistent.
> >
> > As for TcpCommunicationSpi.socketWriteTimeout, I'm not sure why you want
> > to
> > change its behavior. Can we just deprecate it and eventually remove, just
> > as we plan to do for all timeouts from #2?
> >
> > -Val
> >
> > On Tue, May 29, 2018 at 3:50 AM, Stanislav Lukyanov <
> > stanlukya...@gmail.com>
> > wrote:
> >
> > > Val,
> > >
> > > Which timeouts do you mean?
> > >
> > > In #2 I don’t propose to change behavior.
> > >
> > > I propose to change behavior for a couple of settings in #3 though.
> > > I believe the correct approach here would be to target the behavior
> > change
> > > for 2.6,
> > > but keep in mind that we’ll need to carefully analyze the impact before
> > > actually making the changes.
> > >
> > > Thanks,
> > > Stan
> > >
> > > From: Valentin Kulichenko
> > > Sent: 29 мая 2018 г. 0:57
> > > To: dev@ignite.apache.org
> > > Subject: Re: IgniteConfiguration, TcpDiscoverySpi,
> > > TcpCommunicationSpitimeouts
> > >
> > > Hi Stan,
> > >
> > > I'm 100% for this activity, however I don't think we should change the
> > > behavior of timeouts you listed in #2 - this can lead to unexpected
> > > behavior for users who already use them. I would just deprecate them
> and
> > > eventually remove.
> > >
> > > -Val
> > >
> > > On Mon, May 28, 2018 at 1:29 PM, Stanislav Lukyanov <
> > > stanlukya...@gmail.com>
> > > wrote:
> > >
> > > > Hi folks,
> > > >
> > > > It looks like we stopped half-way with this activity. I’d like to
> pick
> > it
> > > > up.
> > > >
> > > > All seem to agree that we should simplify the timeout settings.
> > > > Here are the specific actions I’d like to propose:
> > > >
> > > > 1. Promote the use of global timeouts as the best practice
> > > > *What*: update the docs to encourage users to rely on the following
> > > > timeouts for their “network stability” settings
> > > > IgniteConfiguration.failureDetectionTimeout
> > > > IgniteConfiguration.clientFailureDetecti

Re: Stable/experimental releases

[Vladimir Ozerov]

Hi Dmitriy,

AFAIK we have an idea to introduce maintenance releases for Ignite. E.g.
2.6.0 - features, 2.6.1+ - stabilization.

This seems to be more standard and flexible approach.

чт, 5 июля 2018 г. в 17:39, Dmitry Karachentsev :

> Hi igniters!
>
> Following our discussions about emergency releases I see that here might
> be applied new way for doing releases. Like it was for Linux or like it
> is for Ubuntu. I mean do interleaving releases: first is experimental
> with newest features and second - with bug fixes ONLY.
>
> For example, odd version number is unstable and even is stable: 2.5
> introduces a lot of new features, when 2.6 brings more stability to
> product.
>
> Pros:
>
> 1. User always has a choice what to choose: cutting edge technology or
> release that has less problems.
>
> 2. It will be much easier to add more effort to make TC green again, as
> fixes are not mixed with features.
>
> 3. We may spend more time on prepare stable release and do more rigorous
> testing.
>
> 4. Stable release may keep 100% compatibility to previous release (not
> always, of course) to make it easier to migrate and take important bug
> fixes without introducing a new ones.
>
> 5. Not all users will fall in critical issues, in other words, only some
> group of users will try to use unstable release with experimental features.
>
> Cons:
>
> 1. Necessity of keeping two branches simultaneous: master and stable
> release. Migrate fixes between branches.
>
> 2. Less users could report about found issues, as consequence of item #5
> from pros.
>
> 3. A bit more complex release procedure???
>
> I think it's common and right way to create a less buggy product.
>
> What do you think?
>
> Thanks!
>
>
>

Stable/experimental releases

[Dmitry Karachentsev]

Hi igniters!

Following our discussions about emergency releases I see that here might 
be applied new way for doing releases. Like it was for Linux or like it 
is for Ubuntu. I mean do interleaving releases: first is experimental 
with newest features and second - with bug fixes ONLY.


For example, odd version number is unstable and even is stable: 2.5 
introduces a lot of new features, when 2.6 brings more stability to product.


Pros:

1. User always has a choice what to choose: cutting edge technology or 
release that has less problems.


2. It will be much easier to add more effort to make TC green again, as 
fixes are not mixed with features.


3. We may spend more time on prepare stable release and do more rigorous 
testing.


4. Stable release may keep 100% compatibility to previous release (not 
always, of course) to make it easier to migrate and take important bug 
fixes without introducing a new ones.


5. Not all users will fall in critical issues, in other words, only some 
group of users will try to use unstable release with experimental features.


Cons:

1. Necessity of keeping two branches simultaneous: master and stable 
release. Migrate fixes between branches.


2. Less users could report about found issues, as consequence of item #5 
from pros.


3. A bit more complex release procedure???

I think it's common and right way to create a less buggy product.

What do you think?

Thanks!

[jira] [Created] (IGNITE-8941) BinaryInvalidTypeException is thrown on invoke

[Mikhail Cherkasov (JIRA)]

Mikhail Cherkasov created IGNITE-8941:
-

 Summary: BinaryInvalidTypeException is thrown on invoke
 Key: IGNITE-8941
 URL: https://issues.apache.org/jira/browse/IGNITE-8941
 Project: Ignite
  Issue Type: Bug
  Components: general
Reporter: Mikhail Cherkasov
 Attachments: MyPocTest.java

Reproducer is attached.

The following exception is thrown:

[2018-07-05 16:31:44,554][ERROR][Thread-6][GridDhtAtomicCache]  
Unexpected exception during cache update
class org.apache.ignite.binary.BinaryInvalidTypeException: invoke0
 at 
org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:707)
 at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1757)
 at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
 at 
org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:798)
 at 
org.apache.ignite.internal.binary.BinaryObjectImpl.value(BinaryObjectImpl.java:143)
 at 
org.apache.ignite.internal.processors.cache.GridCacheUtils.value(GridCacheUtils.java:1312)
 at 
org.apache.ignite.internal.processors.cache.GridCacheReturn.addEntryProcessResult(GridCacheReturn.java:253)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2553)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:1898)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1740)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1630)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:299)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:483)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:443)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:248)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1119)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.invoke0(GridDhtAtomicCache.java:827)
 at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.invoke(GridDhtAtomicCache.java:787)
 at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.invoke(IgniteCacheProxyImpl.java:1417)
 at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.invoke(IgniteCacheProxyImpl.java:1461)
 at 
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.invoke(GatewayProtectedCacheProxy.java:1220)
 at my_poc_test.MyPocTest$InvokeTask.run(MyPocTest.java:172)
 at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: invoke0
 at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
 at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 at java.lang.Class.forName0(Native Method)
 at java.lang.Class.forName(Class.java:348)
 at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8640)
 at 
org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:349)
 at 
org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:698)
 ... 22 more



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4314: IGNITE-8619 Remote node could not start in ssh co...

[1vanan]

GitHub user 1vanan opened a pull request:

https://github.com/apache/ignite/pull/4314

IGNITE-8619 Remote node could not start in ssh connection



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/1vanan/ignite IGNITE-8619

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4314.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4314


commit e9e9e4ae5c1b31d9218865981cc3a936c6e922b8
Author: Fedotov 
Date:   2018-07-05T09:36:31Z

change shell command

commit b90c9cbde31eb8cfd522c7159d1d8c855e4f8904
Author: Fedotov 
Date:   2018-07-05T11:58:13Z

fix shell command




---

[GitHub] ignite pull request #4313: IGNITE-8714

[NSAmelchev]

GitHub user NSAmelchev opened a pull request:

https://github.com/apache/ignite/pull/4313

IGNITE-8714

For IGNITE-8714.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/NSAmelchev/ignite ignite-8714

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4313.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4313


commit 9ab86440476be4b499c618d00a8729db39528dd6
Author: Alexander Menshikov 
Date:   2018-06-04T09:11:31Z

ignite-8687 Add JCache TCK 1.1.0 to TC

Signed-off-by: NSAmelchev 

commit 187a644b99ae8c0a29f44b2ab46a084cd30af370
Author: Alexander Menshikov 
Date:   2018-06-07T09:37:51Z

ignite-8714 CacheEntryEvent.getOldValue should be available

Signed-off-by: NSAmelchev 

commit f626675f622d50b86fb3d5d9643f365ed67b4713
Author: Alexander Menshikov 
Date:   2018-06-14T10:05:40Z

Revert "ignite-8687 Add JCache TCK 1.1.0 to TC"

Signed-off-by: NSAmelchev 

commit 525da4854829855efa29b7a737b1a561180afa52
Author: Alexander Menshikov 
Date:   2018-06-19T10:35:59Z

ignite-8714 fix problems with ignite's tests

Signed-off-by: NSAmelchev 

commit 6f701a090003dd69b28d46559d22665e1ebe042b
Author: Alexander Menshikov 
Date:   2018-06-21T08:55:33Z

temp commit

Signed-off-by: NSAmelchev 

commit 75ff37961ea72e782deaf6b73b71a57bc5009de8
Author: Alexander Menshikov 
Date:   2018-06-21T09:55:46Z

fix problems with tests

Signed-off-by: NSAmelchev 

commit f59f9ee66b5afd4848d83d020dd0b69746ea10f6
Author: Alexander Menshikov 
Date:   2018-06-21T12:25:32Z

fix our tests

Signed-off-by: NSAmelchev 

commit 4d9ae8b680e1d27e8455b92854f3b2b1fbd89099
Author: Alexander Menshikov 
Date:   2018-06-21T16:09:07Z

fix prblms with tests

Signed-off-by: NSAmelchev 

commit 6d2960c00d07170618fb15edd6dde8d42b38aff9
Author: Alexander Menshikov 
Date:   2018-06-22T11:31:12Z

fx pr t

Signed-off-by: NSAmelchev 

commit c791f9a4ee010519c8deb6ede793a02f4b34aa31
Author: Alexander Menshikov 
Date:   2018-06-22T12:50:05Z

tempcmt1

Signed-off-by: NSAmelchev 

commit da17a41b10f66a82fd680873a7c5240db861fa5a
Author: Alexander Menshikov 
Date:   2018-06-22T15:35:50Z

tmp cmt

Signed-off-by: NSAmelchev 

commit 79a0d4e59ecc7d35427f5724b1afb225c4004a9d
Author: Alexander Menshikov 
Date:   2018-06-27T11:24:00Z

fix hang

Signed-off-by: NSAmelchev 

commit cc37a46c8032694448556135e0e3e8033c615235
Author: Alexander Menshikov 
Date:   2018-06-27T16:49:53Z

remove temp files

Signed-off-by: NSAmelchev 

commit b123a276801060cab722faba1ba5d0db08e204d8
Author: NSAmelchev 
Date:   2018-07-05T13:05:16Z

revert optimization




---

[GitHub] ignite pull request #4312: Ignite 2.4.6.b2

[aealeksandrov]

GitHub user aealeksandrov opened a pull request:

https://github.com/apache/ignite/pull/4312

Ignite 2.4.6.b2

PR created for team city test runs. Please don't merge.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.4.6.b2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4312.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4312


commit d53504adb1d4276f79ede2401e2d1512fe0287ec
Author: devozerov 
Date:   2018-02-22T07:07:19Z

IGNITE-7253: JDBC thin streaming: fixed default local buffer size, improved 
error messages in case of unsupported SQL statements.

commit debc906a25d3e2d65db58e16307fae6f08460eeb
Author: devozerov 
Date:   2018-02-27T09:13:52Z

Revert "IGNITE-7253: JDBC thin streaming: fixed default local buffer size, 
improved error messages in case of unsupported SQL statements."

This reverts commit d53504adb1d4276f79ede2401e2d1512fe0287ec.

commit d8cf9e042c0e9afe65508c006d9d1c39779d78b9
Author: devozerov 
Date:   2018-02-27T09:14:21Z

Revert "IGNITE-7253: Streaming mode for JDBC thin driver. This closes 
#3499."

This reverts commit bc331f9de716c30d6f733e28821ab44da7ed0cf7.

commit 975a7cdb68cb89da74ec78c3cf23f644050918ff
Author: devozerov 
Date:   2018-02-27T09:49:44Z

Merge branch 'ignite-2.4' into ignite-2.4.3

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/FsyncModeFileWriteAheadLogManager.java
#   parent/pom.xml

commit 74a54d23913bd7195c525d8e222b4e4047515843
Author: Ilya Lantukh 
Date:   2018-02-28T07:08:54Z

IGNITE-7836 Fixed handling of state change message when forceReassignment 
is false

commit 2a70ede048f59753061973495f83927f47452d66
Author: Andrey Novikov 
Date:   2018-01-19T03:05:03Z

IGNITE-6920 Web Console: Create default account for direct-install package.

(cherry picked from commit e5005d9)

commit 2f1ea2cdb76863008d4514f26845457bdeb7d6ed
Author: Andrey Novikov 
Date:   2018-01-19T04:35:30Z

IGNITE-6920 Minor fix.

(cherry picked from commit 9cc7cbf)

commit 62652f3fb1563ba149dcbccb80928d50b822ff36
Author: alexdel 
Date:   2018-01-25T08:49:28Z

IGNITE-7064 Web Console: Implemented basic E2E tests.

(cherry picked from commit ce96e4f)

commit 06e891f1161af598e0aa4665f7a6047637d1c476
Author: Dmitriy Shabalin 
Date:   2018-01-25T09:51:44Z

IGNITE-7522 Web Console: Fixed cluster selector state after cluster restart.

(cherry picked from commit ac3cfb8)

commit 291cb2c88118ccffebcf3383db629647faec1eee
Author: Dmitriy Shabalin 
Date:   2018-01-25T10:33:13Z

IGNITE-7529 Web Console: Refactor UIGrid column filters.

(cherry picked from commit 08658ea)

commit 443eafc718685e66c9a60058bd0ab56d88f9f0a6
Author: alexdel 
Date:   2018-01-25T14:38:36Z

IGNITE-7064 Web Console: Minor test fix.

(cherry picked from commit 4b6d9ad)

commit 6f1df5c40100363b5922734223a774ff1d6a008e
Author: Ilya Borisov 
Date:   2018-01-26T09:07:47Z

IGNITE-7031 Web Console: Refactored confirmation cancellation logic.

(cherry picked from commit 92ae3fe)

commit 16982825fb06ff2724ba4583781cc15443c4f46d
Author: Alexey Kuznetsov 
Date:   2018-02-02T10:07:57Z

IGNITE-7610 Web Console: Profile page refactored to component.

(cherry picked from commit 958975e)

commit 9c6a52250e058c4546aef0d0ecee977c07076a1a
Author: Alexey Kuznetsov 
Date:   2018-02-02T10:09:37Z

IGNITE-7612 Web Console: Refactored mongoose schemas to separate module.

(cherry picked from commit 71db707)

commit 89e15830dedcb46f24d9cc9b922ba3b013331a18
Author: Vasiliy Sisko 
Date:   2018-02-12T10:22:10Z

Web Agent: Fixed wrong config of IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE in 
demo startup.

(cherry picked from commit 1a6e544)

commit 18966673570425192e1b89fbb2c63d164b47eaca
Author: Vasiliy Sisko 
Date:   2018-02-12T13:24:30Z

IGNITE-7578 Actualized client connector configuration.

(cherry picked from commit 819d746)

commit 237063efa35c54bb9e9800ecf63ea223ec20a9ef
Author: alexdel 
Date:   2018-02-19T04:25:24Z

IGNITE-7650 Extracted signin/signup form to separate page, improved landing 
page.

(cherry picked from commit 1925674)

commit 679aeca7a3ff60a9dd478966d3949107d302d5db
Author: Andrey Novikov 
Date:   2018-02-19T07:56:07Z

IGNITE-7650 Fixed headers.

(cherry picked from commit 67922b3)

commit 5d5a6a05ec49f895e8a5edd505e496dcfe58a208
Author: Vasiliy Sisko 
Date:   2018-02-21T04:21:02Z

IGNITE-6094 Web Agent: Enabled persistent in demo mode.

(cherry picked from commit 3c35900)

commit e35d8cfb06f52765959fc2e1883bf70fe0259f45
Author: Alexander Kalinin 
Date:   2018-02-21T07:03:20Z

IGNITE-7320 Web Cons

[GitHub] ignite pull request #4311: IGNITE-8754

[vldpyatkov]

GitHub user vldpyatkov opened a pull request:

https://github.com/apache/ignite/pull/4311

IGNITE-8754

Node outside of baseline does not start when service configured

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4022

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4311.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4311


commit 33c2e656d6fa64283b4e41ae6a15411039f4e9d9
Author: vd-pyatkov 
Date:   2018-07-05T12:51:10Z

IGNITE-8754
Node outside of baseline does not start when service configured




---

[GitHub] ignite pull request #4310: IGNITE-8570 Create lighter version of GridStringL...

[voipp]

GitHub user voipp opened a pull request:

https://github.com/apache/ignite/pull/4310

IGNITE-8570 Create lighter version of GridStringLogger



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/voipp/ignite IGNITE-8570

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4310.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4310


commit c5f1277edb5ffa2eed337304a0d5ed8e66d76f9b
Author: voipp 
Date:   2018-06-28T20:14:54Z

logger implemented




---

Re: Erroneous Baseline Topology documentation

[Ivan Rakov]

I guess just activating the cluster would add all the existing nodes 
to the baseline topology?
Exactly. Persistent cluster can't exist in active state without baseline 
topology. First activation will establish BLT from current set of server 
nodes.


Best Regards,
Ivan Rakov

On 04.07.2018 1:55, Prachi Garg wrote:

Hi Ivan,

I have fixed and rephrased the section - 
https://apacheignite.readme.io/v2.5/docs/baseline-topology#section-cluster-activation


However, I have a question regarding setting the baseline topology 
when activating the cluster for the first time. In the web console, 
when we activate the cluster, using the toggle switch, all server 
nodes in the cluster are automatically added to the baseline topology. 
Does this mean that when we activate the cluster for the first time, 
via code, we do not need the following piece code?


// Get all server nodes that are already up and running.
Collection nodes = ignite.cluster().forServers().nodes();

// Set the baseline topology that is represented by these nodes.
ignite.cluster().setBaselineTopology(nodes);

I guess just activating the cluster would add all the existing nodes 
to the baseline topology?




On Tue, Jul 3, 2018 at 12:48 PM, Ivan Rakov > wrote:


I've tried to execute exactly the same code, it resulted with

class org.apache.ignite.IgniteException: Changing
BaselineTopology on inactive cluster is not allowed.

Basically, the code snippet is in "Setting the Topology From Code"
section, so we can make it correct by just removing "activation"
and "first baseline topology" parts.

Best Regards,
Ivan Rakov



On 03.07.2018 22:30, Denis Magda wrote:

Prachi,

I do remember that that code, Ivan is referring to, worked
fine for you.
Please double check. Probably you need to add
"ignite.cluster.activate()"
to the code snippet.

--
Denis

On Tue, Jul 3, 2018 at 12:19 PM Ivan Rakov
mailto:ivan.glu...@gmail.com>> wrote:

Igniters,

Seems like we have an inconsistency in our Baseline Topology
documentation:


https://apacheignite.readme.io/docs/baseline-topology#section-setting-the-topology-from-code



Java developers can use the IgniteCluster interface to
initialize the
very first baseline topology or to adjust an existing
one. The sample
below shows how to add all the existing server nodes
to the baseline
topology:​
// Connecting to the cluster.
Ignite ignite = Ignition.start();

// Setting the baseline topology to a specific Ignite
cluster topology
version.
ignite.cluster().setBaselineTopology(2);

This is not true; baseline topology can't be changed on
inactive
cluster. The only viable way to initialize the very first
baseline
topology is manual cluster activation. This is correctly
explained in
this section:


https://apacheignite.readme.io/docs/baseline-topology#section-first-cluster-startup



--
Best Regards,
Ivan Rakov

[GitHub] ignite pull request #4309: IGNITE-8940 force fail of testFailGetLock

[akalash]

GitHub user akalash opened a pull request:

https://github.com/apache/ignite/pull/4309

IGNITE-8940 force fail of testFailGetLock



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8940

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4309.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4309


commit a6ec608d41c1f51fd26c91572309ac42434c80a7
Author: Anton Kalashnikov 
Date:   2018-07-05T12:30:00Z

IGNITE-8940 force fail of testFailGetLock




---

[jira] [Created] (IGNITE-8940) Activation job hangs(IgniteChangeGlobalStateTest#testFailGetLock)

[Anton Kalashnikov (JIRA)]

Anton Kalashnikov created IGNITE-8940:
-

 Summary: Activation job 
hangs(IgniteChangeGlobalStateTest#testFailGetLock)
 Key: IGNITE-8940
 URL: https://issues.apache.org/jira/browse/IGNITE-8940
 Project: Ignite
  Issue Type: Test
Reporter: Anton Kalashnikov


given:
 # Cluster consisted from 3 nodes which should fail activation cause "can't get 
lock" 
 # 3 clients for cluster

when:
 # Try to activate cluster from one of client
 # Activation job start execution on one of 3 server nodes
 # Activation triggered exchange
 # Exchange is finished with error cause lock can not be acquired

then:

If job is executed on coordinator, its future successfuly finished otherwise 
job is hang

 

expected:

Job is finished on any nodes.

 

why it is happen:

During handling of GridDhtPartitionsFullMessage on non-coordinator node, 
execution finished before job's future finished cause if 
clause(GridDhtPartitionsExchangeFuture#3230)

 

Reason of this maybe task of https://issues.apache.org/jira/browse/IGNITE-8657 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8939) Error in binary meta data after RESTORE with wal_compaction

[Evgenii Zagumennov (JIRA)]

Evgenii Zagumennov created IGNITE-8939:
--

 Summary: Error in binary meta data after RESTORE with 
wal_compaction
 Key: IGNITE-8939
 URL: https://issues.apache.org/jira/browse/IGNITE-8939
 Project: Ignite
  Issue Type: Bug
  Components: binary
Reporter: Evgenii Zagumennov


Steps to reproduce:
 # CREATE snapshot
 # RESTORE from snapshot
 # CHECK snapshot
 # Restart client nodes
 # Run some jobs on grid

Node fails with exception:

 
org.apache.ignite.binary.BinaryObjectException: Cannot find schema for object 
with compact footer [typeName=...]
    at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.getOrCreateSchema(BinaryReaderExImpl.java:2033)
    at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.(BinaryReaderExImpl.java:286)
    at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.(BinaryReaderExImpl.java:185)
    at 
org.apache.ignite.internal.binary.BinaryObjectImpl.reader(BinaryObjectImpl.java:832)
    at 
org.apache.ignite.internal.binary.BinaryObjectImpl.reader(BinaryObjectImpl.java:846)
    at 
org.apache.ignite.internal.binary.BinaryObjectImpl.field(BinaryObjectImpl.java:626)
    at 
org.apache.ignite.internal.binary.BinaryObjectExImpl.toString(BinaryObjectExImpl.java:225)
    at 
org.apache.ignite.internal.binary.BinaryObjectExImpl.appendValue(BinaryObjectExImpl.java:280)
    at 
org.apache.ignite.internal.binary.BinaryObjectExImpl.toString(BinaryObjectExImpl.java:229)
    at 
org.apache.ignite.internal.binary.BinaryObjectExImpl.toString(BinaryObjectExImpl.java:186)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8938) file-decompressor thread termination should be handled with Failure handler

[Andrey Gura (JIRA)]

Andrey Gura created IGNITE-8938:
---

 Summary: file-decompressor thread termination should be handled 
with Failure handler
 Key: IGNITE-8938
 URL: https://issues.apache.org/jira/browse/IGNITE-8938
 Project: Ignite
  Issue Type: Bug
Reporter: Andrey Gura
Assignee: Andrey Gura
 Fix For: 2.7


file-decompressor thread termination should be handled with Failure Handler 
(IEP-14).
It doesn't work right now. Also exception handling in {{body()}} method should 
be revised.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4244: IGNITE-8737: Improve checkpoint logging informati...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4244


---

[jira] [Created] (IGNITE-8937) JVM Crash in C++ suites on Windows/Java 8 (in PageIO)

[Ilya Kasnacheev (JIRA)]

Ilya Kasnacheev created IGNITE-8937:
---

 Summary: JVM Crash in C++ suites on Windows/Java 8 (in PageIO)
 Key: IGNITE-8937
 URL: https://issues.apache.org/jira/browse/IGNITE-8937
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.7
Reporter: Ilya Kasnacheev
Assignee: Eduard Shangareev


https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_PlatformCWindowsX64

Builds of Platform C++ (Windows x64) and Platform C++ (Windows x86) fail almost 
every time with JVM Crash:
{code}
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc005) at pc=0x02537c64, pid=44768, 
tid=0x56c8
#
# JRE version: Java(TM) SE Runtime Environment (8.0_161-b12) (build 
1.8.0_161-b12)
# Java VM: Java HotSpot(TM) Server VM (25.161-b12 mixed mode windows-x86 )
# Problematic frame:
# J 2991 C2 
org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.initNewPage(JJI)V
 (48 bytes) @ 0x02537c64 [0x02537c40+0x24]
{code}

Looks like incorrect usage of Unsafe.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Cases of using AffinityAssignment.clientEventChange method

[Maxim Muzafarov]

Hello everyone,

Suppose clientEventChange method is useless and have confusing name.

I'm suggesting to remove it as not used. I've filed issue for it:

https://issues.apache.org/jira/browse/IGNITE-8936


чт, 28 июн. 2018 г. в 16:53, Maxim Muzafarov :

> Hi Igniters,
>
> Recently I've faced with AffinityAssignment.clientEventChange() method and
> not completly sure about the range of its applicability. The javadoc says
> "return {@code True} if related discovery event did not cause affinity
> assignment change and this assignment is just reference to the previous
> one."
>
> Three facts about it:
> 1) Method is the part of internal Ignite API
> 2) It​ is​ not used anywhere in Ignite project code
> 3) "clientEventChage" confusing name for this method. "true" value can be
> set not only by client-related events (e.g. cacheChangeRequest,
> affinityChangeRequest etc.).
>
> I've prepared diagram when it has "TRUE" value [1].
>
> * Question #1 * When and for what cases we can use returned value of this
> method?
> * Question #2 * Can it be removed? My suggestion is to keep internal API
> as simple as possible.
>
>
> [1] https://image.ibb.co/cW6Mx8/Client_Event_Change_1.png
>

[jira] [Created] (IGNITE-8936) Remove AffinityAssignment#clientEventChange as not used

[Maxim Muzafarov (JIRA)]

Maxim Muzafarov created IGNITE-8936:
---

 Summary: Remove AffinityAssignment#clientEventChange as not used
 Key: IGNITE-8936
 URL: https://issues.apache.org/jira/browse/IGNITE-8936
 Project: Ignite
  Issue Type: Task
Affects Versions: 2.7
Reporter: Maxim Muzafarov


We should try to keep Ignite project code as simple as possible.


Motivation: 
http://apache-ignite-developers.2346864.n4.nabble.com/Cases-of-using-AffinityAssignment-clientEventChange-method-td32068.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8935) IgniteConfiguration dependents should have toString

[Ilya Kasnacheev (JIRA)]

Ilya Kasnacheev created IGNITE-8935:
---

 Summary: IgniteConfiguration dependents should have toString
 Key: IGNITE-8935
 URL: https://issues.apache.org/jira/browse/IGNITE-8935
 Project: Ignite
  Issue Type: Improvement
Affects Versions: 2.5
Reporter: Ilya Kasnacheev
Assignee: Ilya Kasnacheev


Ignite configuration is printed on startup, but some classes which are usually 
referred do not have toString() implemented, leading to gems such as 
connectorCfg=org.apache.ignite.configuration.ConnectorConfiguration@7d8704ef

Those classes should have toString implemented which will conform to 
https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines#CodingGuidelines-StringOutput



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Add cluster (de)activation events IGNITE-8376

[Evgenii Zhuravlev]

Guys,

Do we really need events for activation/deactivation? We already have a
ticket for implementation lifecycle events for it:
https://issues.apache.org/jira/browse/IGNITE-5427, won't it be enough?

Evgenii

2018-07-03 16:06 GMT+03:00 Ken Cheng :

> Hi dsetrakyan,
>
> I checked the source again and found there is a customized Event sent out
> with below code
>
> org.apache.ignite.internal.processors.cluster.GridClusterSta
> teProcessor#changeGlobalState0
>
> I am not sure what you are referencing is this part? or still we need a
> dedicated event for cluster status changes?
>
> Thank you very much,
> kcmvp
>
> ==
> ChangeGlobalStateMessage msg = new ChangeGlobalStateMessage(start
> edFut.requestId,
> ctx.localNodeId(),
> storedCfgs,
> activate,
> blt,
> forceChangeBaselineTopology,
> System.currentTimeMillis());
>
> try {
> if (log.isInfoEnabled())
> U.log(log, "Sending " + prettyStr(activate) + " request
> with BaselineTopology " + blt);
>
> ctx.discovery().sendCustomEvent(msg);
> 
> Thanks,
> Ken Cheng
>
>
> On Tue, Jul 3, 2018 at 7:41 AM Dmitriy Setrakyan 
> wrote:
>
> > Do we really not have events in Ignite for cluster activation?
> >
> > Alexey Goncharuk, can you please comment?
> >
> > D.
> >
> > On Mon, Jul 2, 2018 at 1:34 AM, kcheng.mvp  wrote:
> >
> > > Dear igniters,I am going to pick up
> > > https://issues.apache.org/jira/browse/IGNITE-8376, and did some
> research
> > > based on the comments.based on my understanding, if we want to know
> this
> > > action is triggered by which way(rest, mbean, auto or visocmd)  then we
> > > need
> > > to change the core method's signature.I am not sure my understanding is
> > > correct or not. Can anybody help to clarify this? Thank you very
> much.By
> > > the
> > > way, I commented this in jira as well.Thanks youkcvmp
> > >
> > >
> > >
> > > --
> > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >
>

Re: .NET tests fail on Linux - need help with TeamCity

[Petr Ivanov]

I can.


Can you prepare reproduce steps so that I’ll be able to pinpoint the problem 
faster, please?


> On 5 Jul 2018, at 11:25, Pavel Tupitsyn  wrote:
> 
> Igniters,
> 
> I need help with TeamCity.
> .NET Linux Tests [1] fail for a very weird reason:
> Newtonsoft.Json.dll seems to be corrupted or empty " *Image is too small.*
> ".
> I tried adding a step to clean NuGet caches, but it does not help.
> 
> On my Ubuntu box tests pass. And there were no changes to .NET lately, so
> this failure seems to be TC-related only.
> 
> Can someone with TeamCity agent access help me with this and check where
> that file comes from and what does it look like?
> 
> Thanks,
> Pavel
> 
> [1]
> https://ci.ignite.apache.org/viewLog.html?buildId=1454493&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_PlatformNetCoreLinux

[GitHub] ignite pull request #4308: IGNITE-8934 remove error message when LongJVMPaus...

[ezhuravl]

GitHub user ezhuravl opened a pull request:

https://github.com/apache/ignite/pull/4308

IGNITE-8934 remove error message when LongJVMPauseDetector stops



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8934

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4308.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4308


commit 111dc822630490ee937f336941301af3cd091738
Author: ezhuravl 
Date:   2018-07-05T08:28:07Z

IGNITE-8934 remove error message when LongJVMPauseDetector stops




---

.NET tests fail on Linux - need help with TeamCity

[Pavel Tupitsyn]

Igniters,

I need help with TeamCity.
.NET Linux Tests [1] fail for a very weird reason:
Newtonsoft.Json.dll seems to be corrupted or empty " *Image is too small.*
".
I tried adding a step to clean NuGet caches, but it does not help.

On my Ubuntu box tests pass. And there were no changes to .NET lately, so
this failure seems to be TC-related only.

Can someone with TeamCity agent access help me with this and check where
that file comes from and what does it look like?

Thanks,
Pavel

[1]
https://ci.ignite.apache.org/viewLog.html?buildId=1454493&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_PlatformNetCoreLinux

[jira] [Created] (IGNITE-8934) LongJVMPauseDetector prints error on thread interruption when node stopping

[Evgenii Zhuravlev (JIRA)]

Evgenii Zhuravlev created IGNITE-8934:
-

 Summary: LongJVMPauseDetector prints error on thread interruption 
when node stopping
 Key: IGNITE-8934
 URL: https://issues.apache.org/jira/browse/IGNITE-8934
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.5
Reporter: Evgenii Zhuravlev
Assignee: Evgenii Zhuravlev






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4307: IGNITE-8361 Use discovery messages for service de...

[daradurvs]

GitHub user daradurvs opened a pull request:

https://github.com/apache/ignite/pull/4307

IGNITE-8361 Use discovery messages for service deployment



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/daradurvs/ignite ignite-8361-rc

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4307.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4307






---

[jira] [Created] (IGNITE-8933) Fix web-console tests CI because of package-lock

[Alexander Kalinin (JIRA)]

Alexander Kalinin created IGNITE-8933:
-

 Summary: Fix web-console tests CI because of package-lock
 Key: IGNITE-8933
 URL: https://issues.apache.org/jira/browse/IGNITE-8933
 Project: Ignite
  Issue Type: Bug
Reporter: Alexander Kalinin


Current CI builds fail because of issues with npm and package-lock.json. To 
overcome it we need to
 * remove package-lock.json from VCS control.
 * set dependencies versions to exact values by removing carets ^. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)