[jira] [Created] (IGNITE-8119) NPE on clear DB and unclear WAL/WAL_ARCHIVE

[Alexander Belyak (JIRA)]

Alexander Belyak created IGNITE-8119:


 Summary: NPE on clear DB and unclear WAL/WAL_ARCHIVE
 Key: IGNITE-8119
 URL: https://issues.apache.org/jira/browse/IGNITE-8119
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 2.4
Reporter: Alexander Belyak
 Attachments: ClearTestP.java

1) Start grid (1 node will be enought), activate it and populate some data
2) Stop node and clear db folder
3) Start grid and activate it
Expected result:
Error about inconsistent storage configuration with/without start node with 
such store
Actual result:
Exchange-worker on node stop with NPE, this can hang whole cluster from 
complete any PME operations.
{noformat}
Failed to reinitialize local partitions (preloading will be stopped): 
GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1, 
minorTopVer=1], ...
java.lang.NullPointerException
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.applyUpdate(GridCacheDatabaseSharedManager.java:2354)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.applyLastUpdates(GridCacheDatabaseSharedManager.java:2099)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreState(GridCacheDatabaseSharedManager.java:1325)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.beforeExchange(GridCacheDatabaseSharedManager.java:1113)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.distributedExchange(GridDhtPartitionsExchangeFuture.java:1063)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:661)
at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2329)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:748)
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Timeline for support of compute functions by thin clients

[Dmitriy Setrakyan]

Val, my preference would be not to have compute functionality on thin clients, 
as it would introduce extra security risk.

Any particular reason why you are asking for this feature?

⁣D.​

On Apr 2, 2018, 8:47 PM, at 8:47 PM, Valentin Kulichenko 
 wrote:
>Folks,
>
>Any other thoughts on this? Should we create tickets for compute
>support if
>there are no objections?
>
>-Val
>
>On Thu, Mar 22, 2018 at 4:27 PM, Valentin Kulichenko <
>valentin.kuliche...@gmail.com> wrote:
>
>> I agree that compute and services functionality is important for thin
>> client. It doesn't seem to be very hard to implement, but would
>provide
>> much better flexibility, as users would be able to do remote
>invocation of
>> arbitrary code, use collocated processing, etc. Having an ability to
>do
>> this from a thin client without joining the topology is a huge
>advantage.
>>
>> -Val
>>
>> On Fri, Mar 16, 2018 at 4:00 AM, Vladimir Ozerov
>
>> wrote:
>>
>>> Denis,
>>>
>>> From client perspective any compute task is also request - response.
>This
>>> doesn't distinguish compute from any other API anyhow. There are no
>>> problem
>>> to add closures, tasks, services, etc.. What is really difficult is
>>> components requiring non-trivial thread interaction and complex
>request
>>> workflows. E.g. streaming, COPY command, continuous queries, events.
>>>
>>> On Thu, Mar 15, 2018 at 10:25 PM, Denis Magda 
>wrote:
>>>
>>> > Pavel,
>>> >
>>> > I just don't see a substantial reason why we need to support the
>>> > compute APIs.
>>> >
>>> > As you properly mentioned, it's not easy to copy all the APIs and,
>>> again,
>>> > for what. It's right that the thin client allows decoupling .NET
>from
>>> JVM,
>>> > but its implementation won't be more performant than the regular
>>> client's
>>> > one.
>>> >
>>> > So, personally, a thin client (.NET, Node.JS, Java, Python, etc.)
>is a
>>> > lightweight connection to the cluster that supports classic
>>> client-server
>>> > request-response operations. If someone needs more (compute,
>services,
>>> > streaming, ML), then go for the regular client which is
>battle-tested
>>> and
>>> > available for usage.
>>> >
>>> > --
>>> > Denis
>>> >
>>> >
>>> >
>>> > On Wed, Mar 14, 2018 at 1:33 PM, Pavel Tupitsyn
>
>>> > wrote:
>>> >
>>> > > Hi Denis,
>>> > >
>>> > > > There are no any plans for that level of support
>>> > > Why do you think so?
>>> > > We already have ScanQuery with filter in .NET Thin Client, which
>>> involves
>>> > > remote code execution on server nodes.
>>> > > It is quite similar to Compute.Broadcast and such.
>>> > >
>>> > > Thanks,
>>> > > Pavel
>>> > >
>>> > >
>>> > > On Wed, Mar 14, 2018 at 11:32 PM, Denis Magda
>
>>> wrote:
>>> > >
>>> > > > Raymond,
>>> > > >
>>> > > > Then I would suggest you keep using the regular .NET client
>that
>>> > supports
>>> > > > and optimized for computations. Is there any reason why you
>can't
>>> use
>>> > the
>>> > > > regular one?
>>> > > >
>>> > > > --
>>> > > > Denis
>>> > > >
>>> > > > On Wed, Mar 14, 2018 at 12:53 PM, Raymond Wilson <
>>> > > > raymond_wil...@trimble.com
>>> > > > > wrote:
>>> > > >
>>> > > > > Hi Denis,
>>> > > > >
>>> > > > > We are using Ignite.Net and are planning to use 2.4 + .Net
>Core +
>>> > thin
>>> > > > > client support to enable lightweight containerisable
>services that
>>> > > > interact
>>> > > > > with the main Ignite compute grid.
>>> > > > >
>>> > > > > These work flows are less about Get/Put style semantics, and
>more
>>> > about
>>> > > > > using grid compute.
>>> > > > >
>>> > > > > Eg: Here's an example where a client context asks a remote
>>> context to
>>> > > > > render
>>> > > > > a bitmap tile in an ICompute:
>>> > > > >
>>> > > > > public Bitmap Execute(TileRenderRequestArgument arg)
>>> > > > > {
>>> > > > > IComputeFunc
>func
>>> =
>>> > new
>>> > > > > TileRenderRequestComputeFunc();
>>> > > > >
>>> > > > > return
>>> > > > > _ignite.GetCluster().ForRemotes().GetCompute().Apply(func,
>arg);
>>> > > > > }
>>> > > > >
>>> > > > > In this example, the calling context here could be a
>lightweight
>>> > > Kestrel
>>> > > > > web
>>> > > > > service end point delegating rendering to a remote service.
>>> > > > >
>>> > > > > Thanks,
>>> > > > > Raymond.
>>> > > > >
>>> > > > > -Original Message-
>>> > > > > From: Denis Magda [mailto:dma...@apache.org]
>>> > > > > Sent: Thursday, March 15, 2018 8:31 AM
>>> > > > > To: dev@ignite.apache.org
>>> > > > > Subject: Re: Timeline for support of compute functions by
>thin
>>> > clients
>>> > > > >
>>> > > > > Hi Raymond,
>>> > > > >
>>> > > > > There are no any plans for that level of support. The thin
>clients
>>> > are
>>> > > > > targeted for classic client-server processing use cases when
>a
>>> client
>>> > > > > request data from a server, does something with 

[jira] [Created] (IGNITE-8118) sqlline.bat throws NPE under PowerShell

[Pavel Konstantinov (JIRA)]

Pavel Konstantinov created IGNITE-8118:
--

 Summary: sqlline.bat throws NPE under PowerShell
 Key: IGNITE-8118
 URL: https://issues.apache.org/jira/browse/IGNITE-8118
 Project: Ignite
  Issue Type: Bug
Reporter: Pavel Konstantinov


{code}
.\sqlline.bat -u 
"'jdbc:ignite:thin://18.17.12.22:9652?user=ignite=111'"
No known driver to handle "'jdbc:ignite:thin://18.17.12.22:9652?user=ignite". 
Searching for known drivers...
java.lang.NullPointerException
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Timeline for support of compute functions by thin clients

[Valentin Kulichenko]

Folks,

Any other thoughts on this? Should we create tickets for compute support if
there are no objections?

-Val

On Thu, Mar 22, 2018 at 4:27 PM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> I agree that compute and services functionality is important for thin
> client. It doesn't seem to be very hard to implement, but would provide
> much better flexibility, as users would be able to do remote invocation of
> arbitrary code, use collocated processing, etc. Having an ability to do
> this from a thin client without joining the topology is a huge advantage.
>
> -Val
>
> On Fri, Mar 16, 2018 at 4:00 AM, Vladimir Ozerov 
> wrote:
>
>> Denis,
>>
>> From client perspective any compute task is also request - response. This
>> doesn't distinguish compute from any other API anyhow. There are no
>> problem
>> to add closures, tasks, services, etc.. What is really difficult is
>> components requiring non-trivial thread interaction and complex request
>> workflows. E.g. streaming, COPY command, continuous queries, events.
>>
>> On Thu, Mar 15, 2018 at 10:25 PM, Denis Magda  wrote:
>>
>> > Pavel,
>> >
>> > I just don't see a substantial reason why we need to support the
>> > compute APIs.
>> >
>> > As you properly mentioned, it's not easy to copy all the APIs and,
>> again,
>> > for what. It's right that the thin client allows decoupling .NET from
>> JVM,
>> > but its implementation won't be more performant than the regular
>> client's
>> > one.
>> >
>> > So, personally, a thin client (.NET, Node.JS, Java, Python, etc.) is a
>> > lightweight connection to the cluster that supports classic
>> client-server
>> > request-response operations. If someone needs more (compute, services,
>> > streaming, ML), then go for the regular client which is battle-tested
>> and
>> > available for usage.
>> >
>> > --
>> > Denis
>> >
>> >
>> >
>> > On Wed, Mar 14, 2018 at 1:33 PM, Pavel Tupitsyn 
>> > wrote:
>> >
>> > > Hi Denis,
>> > >
>> > > > There are no any plans for that level of support
>> > > Why do you think so?
>> > > We already have ScanQuery with filter in .NET Thin Client, which
>> involves
>> > > remote code execution on server nodes.
>> > > It is quite similar to Compute.Broadcast and such.
>> > >
>> > > Thanks,
>> > > Pavel
>> > >
>> > >
>> > > On Wed, Mar 14, 2018 at 11:32 PM, Denis Magda 
>> wrote:
>> > >
>> > > > Raymond,
>> > > >
>> > > > Then I would suggest you keep using the regular .NET client that
>> > supports
>> > > > and optimized for computations. Is there any reason why you can't
>> use
>> > the
>> > > > regular one?
>> > > >
>> > > > --
>> > > > Denis
>> > > >
>> > > > On Wed, Mar 14, 2018 at 12:53 PM, Raymond Wilson <
>> > > > raymond_wil...@trimble.com
>> > > > > wrote:
>> > > >
>> > > > > Hi Denis,
>> > > > >
>> > > > > We are using Ignite.Net and are planning to use 2.4 + .Net Core +
>> > thin
>> > > > > client support to enable lightweight containerisable services that
>> > > > interact
>> > > > > with the main Ignite compute grid.
>> > > > >
>> > > > > These work flows are less about Get/Put style semantics, and more
>> > about
>> > > > > using grid compute.
>> > > > >
>> > > > > Eg: Here's an example where a client context asks a remote
>> context to
>> > > > > render
>> > > > > a bitmap tile in an ICompute:
>> > > > >
>> > > > > public Bitmap Execute(TileRenderRequestArgument arg)
>> > > > > {
>> > > > > IComputeFunc func
>> =
>> > new
>> > > > > TileRenderRequestComputeFunc();
>> > > > >
>> > > > > return
>> > > > > _ignite.GetCluster().ForRemotes().GetCompute().Apply(func, arg);
>> > > > > }
>> > > > >
>> > > > > In this example, the calling context here could be a lightweight
>> > > Kestrel
>> > > > > web
>> > > > > service end point delegating rendering to a remote service.
>> > > > >
>> > > > > Thanks,
>> > > > > Raymond.
>> > > > >
>> > > > > -Original Message-
>> > > > > From: Denis Magda [mailto:dma...@apache.org]
>> > > > > Sent: Thursday, March 15, 2018 8:31 AM
>> > > > > To: dev@ignite.apache.org
>> > > > > Subject: Re: Timeline for support of compute functions by thin
>> > clients
>> > > > >
>> > > > > Hi Raymond,
>> > > > >
>> > > > > There are no any plans for that level of support. The thin clients
>> > are
>> > > > > targeted for classic client-server processing use cases when a
>> client
>> > > > > request data from a server, does something with it locally and
>> > > > potentially
>> > > > > writes changes back to the server. ICache, SQL fall under this
>> > > category.
>> > > > >
>> > > > > Are you intended to use .NET thin client or anyone else?
>> > > > >
>> > > > > --
>> > > > > Denis
>> > > > >
>> > > > > On Wed, Mar 14, 2018 at 12:25 PM, Raymond Wilson <
>> > > > > raymond_wil...@trimble.com
>> > > > > > wrote:
>> > > > >
>> > > > > > Hi,
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > 

[GitHub] ignite pull request #3737: IGNITE-7944 Disconnected client node tries to sen...

[gromtech]

GitHub user gromtech opened a pull request:

https://github.com/apache/ignite/pull/3737

IGNITE-7944 Disconnected client node tries to send JOB_CANCEL message

* Skip sending messages if client disconnected.
* Throw IgniteCheckedException if a client node is disconnected and 
communication client is null.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7944

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3737.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3737


commit e043f87b34178d7e010c5e5396382aa557e5486f
Author: Roman Guseinov 
Date:   2018-03-16T02:55:18Z

IGNITE-7944 Disconnected client node tries to send JOB_CANCEL message

* Skip sending message if client disconnected.
* Throw IgniteCheckedException if a client node is disconnected and 
communication client is null.




---

[jira] [Created] (IGNITE-8117) Slow Ignite Cache with multi-nodes

[Rick Lin (JIRA)]

Rick Lin created IGNITE-8117:


 Summary: Slow Ignite Cache with multi-nodes
 Key: IGNITE-8117
 URL: https://issues.apache.org/jira/browse/IGNITE-8117
 Project: Ignite
  Issue Type: Improvement
  Components: cache
 Environment: OS: Ubuntn 14.04.3 LTS

JAVA: JDK 1.7

Ignite: 2.4

 

 

 
Reporter: Rick Lin
 Fix For: 2.4
 Attachments: WriteKV.java

Dear sir,

I am running Ignite in my project about  with three nodes:

{color:#14892c}Three nodes: "ubuntu7","ubuntu8","ubuntu9"{color}

{color:#14892c}cacheConf.setIndexedTypes(String.class, String.class){color}

{color:#14892c}cacheConf.setCacheMode(CacheMode.PARTITIONED);{color}

{color:#14892c}cacheConf.setAtomicityMode(CacheAtomicityMode.ATOMIC);{color}

When using above setting for
 * Put,
 * putAll,
 * SqlFieldsQuery,
 * IgniteJdbcThinDriver,
 * IgniteDataStreamer add
 * IgniteDataStreamer addAll

, there are a situation about data latency, and the experiment results are in 
the following.

One node(ubuntu7)
|*milliseconds*|*1,000*|*10,000*|*100,000*|*500,000*|*1,000,000*|
|*Put*|401|1,108|3,874|12,636|23,561|
|*putAll*|146|428|2,092|8,940|17,854|
|*SqlFieldsQuery*|282|943|4,522|20,821|41,223|
|*IgniteJdbcThinDriver*|491|1,581|8,938|41,429|80,599|
|*IgniteDataStreamer*
*Add*|152|439|2,161|7,374|12,296|
|*IgniteDataStreamer*
*addAll*|101|234|2,028|5,037|9,181|

 

Two nodes(ubuntu7 write and ubuntu8)
|*milliseconds*|*1,000*|*10,000*|*100,000*|*500,000*|*1,000,000*|
|*Put*|1,185|5,481|26,142|-|-|
|*putAll*|107|292|1,785|-|-|
|*SqlFieldsQuery*|786|3,195|30,528|-|-|
|*IgniteJdbcThinDriver*|859|3,569|34,080|-|-|
|*IgniteDataStreamer*
*add*|169|410|1389|-|-|
|*IgniteDataStreamer*
*addAll*|99|206|988|-|-|

 

Three nodes(ubuntu7 write, ubuntu8, and ubuntu9)
|*milliseconds*|*1,000*|*10,000*|*100,000*|*500,000*|*1,000,000*|
|*Put*|1,664|7,794|41,116|-|-|
|*putAll*|101|294|1,911|-|-|
|*SqlFieldsQuery*|1,086|5,718|41,997|-|-|
|*IgniteJdbcThinDriver*|1,247|4,899|48,566|-|-|
|*IgniteDataStreamer*
*add*|168|385|1,364|-|-|
|*IgniteDataStreamer*
*addAll*|86|249|905|-|-|

>From above results, these tables show that the three ways: put, 
>SqlFieldsQuery, IgniteJdbcThinDriver have gradually bad performance on data 
>latency  with increasing ignite server nodes, and the other ways: putAll, 
>IgniteDataStreameradd, and IgniteDataStreameraddAll have better performances.

I don't know if the testing via my java codes are suitable and correct to show 
these results or not.

Here, i provide my codes to reproduce my experimental situation.

{color:#14892c}*Attached is my codes for your reference.*{color}

if any further information is needed, I am glad to be informed and will provide 
to you as soon as possible.

If you have any idea about this issue, I am looking forward to hearing from you.

Yours sincerely,

Rick

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: IGNITE-6827 - Review needed.

[Dmitriy Setrakyan]

I looked into the ticket and do not understand this property
name: RollbackOnTopologyChangeTimeout

I think what we need to configure is a forceful kill of transactions in
case of partition map exchange, right? In that case, the timeout should be
configured for PME, not transactions. The property name should be
PartitionMapExchangePreparationTimeout or PartitionMapExchangeTxTimeout.

Am I missing something?

D.

On Mon, Apr 2, 2018 at 2:48 AM, Alexey Goncharuk  wrote:

> Guys, especially Vladimir Ozerov, Yakov Zhanov, Dmitriy Setrakyan,
>
> While the technical side of the change is being reviewed, I would also ask
> you to pay attention to the public API changes here.
>
> --AG
>
> 2018-03-21 23:43 GMT+03:00 Pavel Tupitsyn :
>
> > Hi Alexei,
> >
> > >  Pavel Tupitsyn, could you lend a hand to help completing .NET part
> (two
> > new
> > public API methods are added as a part of working on [3], java code in
> > branch [2])
> >
> > Please file a separate ticket for .NET changes.
> > I would like to help, but maybe some time later. No need to delay your
> > changes because of that.
> >
> > Thanks,
> > Pavel
> >
> > On Wed, Mar 21, 2018 at 1:08 PM, Dmitry Pavlov 
> > wrote:
> >
> > > Hi Alexei ,
> > >
> > > Thank you for contribution, unfortunately there is too much tests
> failed
> > > there:
> > > .NET is broken
> > >
> > >   Ignite Platform .NET Long Running [ tests 3 ]
> > > CachePartitionedNearEnabledTest.TestTransactionScopeMultiCache
> (True)
> > > CachePartitionedTest.TestTransactionScopeMultiCache(True)
> > > CacheReplicatedTest.TestTransactionScopeMultiCache(True)
> > >
> > >Ignite Platform .NET Core Linux [ tests 2 TC_EXIT_CODE ]
> > >  TransactionsParityTest.TestTransactions (fail rate 0,0%)
> > >  CachePartitionedTest.TestTransactionScopeMultiCache(True) (fail
> > rate
> > > 0,0%)
> > >
> > >Ignite Platform .NET [ tests 1 ] agoncharuk More info >>
> > > Test fail rate less than 1%, probably new failure  exe:
> > > TransactionsParityTest.TestTransactions (fail rate 0,0%)
> > >
> > > Too much timeouts:
> > >Ignite Cache Failover [2]
> > >  with CacheAsyncOperationsFailoverTxTest.testPutAllAsyncFailover (last
> > > started)
> > >
> > >~[Obsolete] Ignite Cache Expiry Policy [ tests 1 ]
> > > with  IgniteCacheExpiryPolicyTestSuite:
> > > IgniteCacheTtlCleanupSelfTest.testDeferredDeleteTtl
> > >
> > >  Ignite Cache Restarts [2]
> > >   with
> > > GridCachePutAllFailoverSelfTest.testPutAllFailoverNearDisabled
> > ThreeBackups
> > > (last started)
> > >
> > >   Ignite Cache Restarts [1]
> > >with
> > > GridCachePartitionedNearDisabledOptimisticTxNodeRestartTest.
> > > testRestartWithTxTenNodesTwoBackups
> > > (last started)
> > >
> > >  Cache [6]
> > >with  WalModeChangeAdvancedSelfTest.testServerRestartNonCoordinator
> > > (last started)
> > > it is not all.
> > >
> > > Please fix tests before review. Also you could change ticket status to
> > 'In
> > > Progress'.
> > >
> > > As easy option, you can merge current master to your branch, it is
> > probably
> > > that some test were fixed already.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > ср, 21 мар. 2018 г. в 9:58, Alexei Scherbakov <
> > > alexey.scherbak...@gmail.com
> > > >:
> > >
> > > > Igniters,
> > > >
> > > > I prepared rather important patch [1] related to grid stability and
> > need
> > > a
> > > > review and some help to finish it.
> > > >
> > > > Semyon Boikov, could you please take a look on patch code ? [2]
> > > >
> > > > Pavel Tupitsyn, could you lend a hand to help completing .NET part
> (two
> > > new
> > > > public API methods are added as a part of working on [3], java code
> in
> > > > branch [2])
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-6827
> > > >
> > > > [2] https://github.com/gridgain/apache-ignite/tree/ignite-6827-2
> > > >
> > > > [3] https://issues.apache.org/jira/browse/IGNITE-7910
> > > >
> > > > --
> > > >
> > > > Best regards,
> > > > Alexei Scherbakov
> > > >
> > >
> >
>

[GitHub] ignite pull request #3736: Ignite 2.4.2 master

[EdShangGG]

GitHub user EdShangGG opened a pull request:

https://github.com/apache/ignite/pull/3736

Ignite 2.4.2 master



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.4.2-master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3736.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3736


commit e7ca9b65a68de7752195c8f4d2b5180f3c77d19f
Author: Dmitriy Govorukhin 
Date:   2017-11-13T18:52:47Z

ignite-blt-merge -> ignite-2.4.1

commit cc8168fc184bb7f5e3cc3bbb0743397097f78bfb
Author: Dmitriy Govorukhin 
Date:   2017-11-13T19:13:01Z

merge ignite-pitr-rc1 -> ignite-2.4.1

commit 87e6d74cf6a251c7984f9e68c391f790feccc281
Author: Dmitriy Govorukhin 
Date:   2017-11-14T12:49:33Z

ignite-gg-12877 Compact consistent ID in WAL

commit 9f5a22711baea05bd37ab07c8f928a4837dd83a4
Author: Ilya Lantukh 
Date:   2017-11-14T14:12:28Z

Fixed javadoc.

commit d5af2d78dd8eef8eca8ac5391d31d8c779649bb0
Author: Alexey Kuznetsov 
Date:   2017-11-15T08:09:00Z

IGNITE-6913 Baseline: Added new options to controls.sh for baseline 
manipulations.

commit 713924ce865752b6e99b03bd624136541cea5f9f
Author: Sergey Chugunov 
Date:   2017-11-15T09:03:12Z

IGNITE-5850 failover tests for cache operations during BaselineTopology 
changes

commit b65fd134e748d496f732ec2aa0953a0531f544b8
Author: Ilya Lantukh 
Date:   2017-11-15T12:54:35Z

TX read logging if PITR is enabled.

commit 9b2a567c0e04dc33116b51f88bee75f76e9107d1
Author: Ilya Lantukh 
Date:   2017-11-15T13:45:16Z

TX read logging if PITR is enabled.

commit 993058ccf0b2b8d9e80750c3e45a9ffa31d85dfa
Author: Dmitriy Govorukhin 
Date:   2017-11-15T13:51:54Z

ignite-2.4.1 optimization for store full set node more compacted

commit 1eba521f608d39967aec376b397b7fc800234e54
Author: Dmitriy Govorukhin 
Date:   2017-11-15T13:52:22Z

Merge remote-tracking branch 'professional/ignite-2.4.1' into ignite-2.4.1

commit 564b3fd51f8a7d1d81cb6874df66d0270623049c
Author: Sergey Chugunov 
Date:   2017-11-15T14:00:51Z

IGNITE-5850 fixed issue with initialization of data regions on node 
activation, fixed issue with auto-activation when random node joins inactive 
cluster with existing BLT

commit c6d1fa4da7adfadc80abdc7eaf6452b86a4f6aa4
Author: Sergey Chugunov 
Date:   2017-11-15T16:23:08Z

IGNITE-5850 transitionResult is set earlier when request for changing 
BaselineTopology is sent

commit d65674363163e38a4c5fdd73d1c8d8e1c7610797
Author: Sergey Chugunov 
Date:   2017-11-16T11:59:07Z

IGNITE-5850 new failover tests for changing BaselineTopology up (new node 
added to topology)

commit 20552f3851fe8825191b144179be032965e0b5c6
Author: Sergey Chugunov 
Date:   2017-11-16T12:53:43Z

IGNITE-5850 improved error message when online node is removed from baseline

commit 108bbcae4505ac904a6db774643ad600bfb42c21
Author: Sergey Chugunov 
Date:   2017-11-16T13:45:52Z

IGNITE-5850 BaselineTopology should not change on cluster deactivation

commit deb641ad3bdbf260fa60ad6bf607629652e324bd
Author: Dmitriy Govorukhin 
Date:   2017-11-17T09:45:44Z

ignite-2.4.1 truncate wal and checkpoint history on move/delete snapshot

commit 3c8b06f3659af30d1fd148ccc0f40e216a56c998
Author: Alexey Goncharuk 
Date:   2017-11-17T12:48:12Z

IGNITE-6947 Abandon remap after single map if future is done (fixes NPE)

commit ba2047e5ae7d271a677e0c418375d82d78c4023e
Author: devozerov 
Date:   2017-11-14T12:26:31Z

IGNITE-6901: Fixed assertion during 
IgniteH2Indexing.rebuildIndexesFromHash. This closes #3027.

commit abfc0466d6d61d87255d0fe38cbdf11ad46d4f89
Author: Sergey Chugunov 
Date:   2017-11-17T13:40:57Z

IGNITE-5850 tests for queries in presence of BaselineTopology

commit f4eabaf2a905abacc4c60c01d3ca04f6ca9ec188
Author: Sergey Chugunov 
Date:   2017-11-17T17:23:02Z

IGNITE-5850 implementation for setBaselineTopology(long topVer) migrated 
from wc-251

commit 4edeccd3e0b671aa277f58995df9ff9935baa95a
Author: EdShangGG 
Date:   2017-11-17T18:21:17Z

GG-13074 Multiple snapshot test failures after baseline topology is 
introduced
-adding baseline test to suite
-fixing issues with baseline

commit edae228c8f55990c15ef3044be987dcb00d6c81a
Author: EdShangGG 
Date:   2017-11-18T10:36:41Z

hack with sleep

commit b5bffc7580a4a8ffbcc06f60c282e73979179578
Author: Ilya 

[GitHub] ignite pull request #3646: IGNITE-7931 Wrong arguments for ConcurrentHashMap

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3646


---

[jira] [Created] (IGNITE-8116) Historic WAL rebalance fails

[Eduard Shangareev (JIRA)]

Eduard Shangareev created IGNITE-8116:
-

 Summary: Historic WAL rebalance fails
 Key: IGNITE-8116
 URL: https://issues.apache.org/jira/browse/IGNITE-8116
 Project: Ignite
  Issue Type: Bug
  Components: persistence
Reporter: Eduard Shangareev
 Fix For: 2.5


So, my reproducer fails because rebalance is never completed. 

Rebalance fails with next error:
{code}
Exception in thread "sys-#95%wal.IgniteWalRebalanceTest0%" 
java.lang.AssertionError
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplier.handleDemandMessage(GridDhtPartitionSupplier.java:390)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.handleDemandMessage(GridDhtPreloader.java:364)
at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:372)
at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:357)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1054)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$700(GridCacheIoManager.java:99)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager$OrderedMessageListener.onMessage(GridCacheIoManager.java:1603)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$4100(GridIoManager.java:125)
at 
org.apache.ignite.internal.managers.communication.GridIoManager$GridCommunicationMessageSet.unwind(GridIoManager.java:2752)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.unwindMessageSet(GridIoManager.java:1516)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$4400(GridIoManager.java:125)
at 
org.apache.ignite.internal.managers.communication.GridIoManager$10.run(GridIoManager.java:1485)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}

Reproducer:

{code}
/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *  http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.apache.ignite.internal.processors.cache.persistence.db.wal;

import java.util.concurrent.TimeUnit;
import org.apache.ignite.IgniteCache;
import org.apache.ignite.cache.CacheAtomicityMode;
import org.apache.ignite.cache.CacheMode;
import org.apache.ignite.cache.CacheRebalanceMode;
import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
import org.apache.ignite.cache.query.annotations.QuerySqlField;
import org.apache.ignite.configuration.CacheConfiguration;
import org.apache.ignite.configuration.DataRegionConfiguration;
import org.apache.ignite.configuration.DataStorageConfiguration;
import org.apache.ignite.configuration.IgniteConfiguration;
import org.apache.ignite.configuration.WALMode;
import org.apache.ignite.internal.IgniteEx;
import org.apache.ignite.internal.util.typedef.internal.S;
import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;

import static 
org.apache.ignite.IgniteSystemProperties.IGNITE_PDS_WAL_REBALANCE_THRESHOLD;

/**
 * Historic WAL rebalance test
 */
public class IgniteWalRebalanceTest extends GridCommonAbstractTest {
/** Cache name. */
private static final String CACHE_NAME = "cache";

/** {@inheritDoc} */
@Override protected IgniteConfiguration getConfiguration(String gridName) 
throws Exception {
System.setProperty(IGNITE_PDS_WAL_REBALANCE_THRESHOLD, "0"); //to make 
all rebalance wal-based

IgniteConfiguration cfg = super.getConfiguration(gridName);

CacheConfiguration ccfg = new 

[GitHub] ignite pull request #3735: IGNITE-8106 Collect suppressed exceptions from ca...

[alamar]

GitHub user alamar opened a pull request:

https://github.com/apache/ignite/pull/3735

IGNITE-8106 Collect suppressed exceptions from causes.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8106

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3735.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3735


commit 1421c53bb2fff8a64afdb9719d20794f4890feb1
Author: Ilya Kasnacheev 
Date:   2018-04-02T17:06:50Z

IGNITE-8106 Collect suppressed exceptions from causes.




---

Re: IGNITE-8111: Add extra validation for WAL segment size

[Lucas Beeler]

+1 for this from an Ignite professional services consultant. The more we can do 
to protect users from non-sensical configuration settings, the better. I’ve 
seen a lot of them in the field (and not just related to WAL segment size). As 
Scott McNealy of Sun fame once said, “there’s nothing dumber than an end user.”

Take care,
Lucas

--
Lucas BEELER
Technical Consultant, Professional Services
GridGain Systems
www.gridgain.com

> On Apr 2, 2018, at 9:51 AM, Denis Magda  wrote:
> 
> Ivan,
> 
> Will his value be used as a default one as well or only is a possible
> minimum? As for the defaults, I see that Postgres and Cassandra set 16mb as
> a default segment size:
> https://www.postgresql.org/docs/9.1/static/wal-configuration.html
> https://www.enterprisedb.com/docs/en/10.0/EPAS_Guide_v10/EDB_Postgres_Advanced_Server_Guide.1.45.html
> 
> Should we have the same defaults?
> 
> --
> Denis
> 
> 
> On Mon, Apr 2, 2018 at 9:16 AM, Ivan Rakov  wrote:
> 
>> Igniters,
>> 
>> We've bumped into situation when using extremely small (<10 pages) WAL
>> segment size caused unobvious errors and assertions:
>> https://issues.apache.org/jira/browse/IGNITE-8111
>> Let's reach a consensus on which value of WAL segment size should be
>> considered as minimum. I tried tests with 128kb WAL segment and they didn't
>> fail.
>> Keeping a little reserve, I propose 512kb as minimum allowable WAL segment
>> size. I don't know why anyone may need a lesser value.
>> 
>> Please share your thoughts and let know if you have objections on 512kb.
>> 
>> --
>> Best Regards,
>> Ivan Rakov
>> 
>> 

Re: IGNITE-8111: Add extra validation for WAL segment size

[Ivan Rakov]

Denis,

The topic is about a possible minimum only.
Our default for WAL segment size is 64mb and we are not going to change 
it (at least under this ticket).


Best Regards,
Ivan Rakov

On 02.04.2018 19:51, Denis Magda wrote:

Ivan,

Will his value be used as a default one as well or only is a possible
minimum? As for the defaults, I see that Postgres and Cassandra set 16mb as
a default segment size:
https://www.postgresql.org/docs/9.1/static/wal-configuration.html
https://www.enterprisedb.com/docs/en/10.0/EPAS_Guide_v10/EDB_Postgres_Advanced_Server_Guide.1.45.html

Should we have the same defaults?

--
Denis


On Mon, Apr 2, 2018 at 9:16 AM, Ivan Rakov  wrote:


Igniters,

We've bumped into situation when using extremely small (<10 pages) WAL
segment size caused unobvious errors and assertions:
https://issues.apache.org/jira/browse/IGNITE-8111
Let's reach a consensus on which value of WAL segment size should be
considered as minimum. I tried tests with 128kb WAL segment and they didn't
fail.
Keeping a little reserve, I propose 512kb as minimum allowable WAL segment
size. I don't know why anyone may need a lesser value.

Please share your thoughts and let know if you have objections on 512kb.

--
Best Regards,
Ivan Rakov

Re: IGNITE-8111: Add extra validation for WAL segment size

[Denis Magda]

Ivan,

Will his value be used as a default one as well or only is a possible
minimum? As for the defaults, I see that Postgres and Cassandra set 16mb as
a default segment size:
https://www.postgresql.org/docs/9.1/static/wal-configuration.html
https://www.enterprisedb.com/docs/en/10.0/EPAS_Guide_v10/EDB_Postgres_Advanced_Server_Guide.1.45.html

Should we have the same defaults?

--
Denis


On Mon, Apr 2, 2018 at 9:16 AM, Ivan Rakov  wrote:

> Igniters,
>
> We've bumped into situation when using extremely small (<10 pages) WAL
> segment size caused unobvious errors and assertions:
> https://issues.apache.org/jira/browse/IGNITE-8111
> Let's reach a consensus on which value of WAL segment size should be
> considered as minimum. I tried tests with 128kb WAL segment and they didn't
> fail.
> Keeping a little reserve, I propose 512kb as minimum allowable WAL segment
> size. I don't know why anyone may need a lesser value.
>
> Please share your thoughts and let know if you have objections on 512kb.
>
> --
> Best Regards,
> Ivan Rakov
>
>

[GitHub] ignite pull request #3021: IGNITE-6802

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3021


---

Re: IGNITE-8111: Add extra validation for WAL segment size

[Dmitry Pavlov]

I have no objections on 512kb

пн, 2 апр. 2018 г. в 19:17, Ivan Rakov :

> Igniters,
>
> We've bumped into situation when using extremely small (<10 pages) WAL
> segment size caused unobvious errors and assertions:
> https://issues.apache.org/jira/browse/IGNITE-8111
> Let's reach a consensus on which value of WAL segment size should be
> considered as minimum. I tried tests with 128kb WAL segment and they
> didn't fail.
> Keeping a little reserve, I propose 512kb as minimum allowable WAL
> segment size. I don't know why anyone may need a lesser value.
>
> Please share your thoughts and let know if you have objections on 512kb.
>
> --
> Best Regards,
> Ivan Rakov
>
>

IGNITE-8111: Add extra validation for WAL segment size

[Ivan Rakov]

Igniters,

We've bumped into situation when using extremely small (<10 pages) WAL 
segment size caused unobvious errors and assertions: 
https://issues.apache.org/jira/browse/IGNITE-8111
Let's reach a consensus on which value of WAL segment size should be 
considered as minimum. I tried tests with 128kb WAL segment and they 
didn't fail.
Keeping a little reserve, I propose 512kb as minimum allowable WAL 
segment size. I don't know why anyone may need a lesser value.


Please share your thoughts and let know if you have objections on 512kb.

--
Best Regards,
Ivan Rakov

[jira] [Created] (IGNITE-8115) Add a warning on local node startup if the node is not in Baseline

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-8115:


 Summary: Add a warning on local node startup if the node is not in 
Baseline
 Key: IGNITE-8115
 URL: https://issues.apache.org/jira/browse/IGNITE-8115
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexey Goncharuk






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3734: IGNITE-8114 Add fail recovery mechanism to tracki...

[EdShangGG]

GitHub user EdShangGG opened a pull request:

https://github.com/apache/ignite/pull/3734

IGNITE-8114 Add fail recovery mechanism to tracking pages



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8114

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3734.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3734


commit e34a82625d7ade969848cbb9fa1073adb66c80d9
Author: Eduard Shangareev 
Date:   2018-03-14T13:48:23Z

IGNITE-8114 Add fail recovery mechanism to tracking pages




---

[GitHub] ignite pull request #3733: ignite-8112: updated build configuration (direct-...

[sk0x50]

GitHub user sk0x50 opened a pull request:

https://github.com/apache/ignite/pull/3733

ignite-8112: updated build configuration (direct-io artifact should be 
published in maven repository)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8112

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3733.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3733


commit 291300a4b63419ba68dbb3e3f3f2adeec33118f0
Author: Slava Koptilin 
Date:   2018-04-02T15:50:01Z

ignite-8112: updated build configuration (direct-io artifact should be 
published in maven repo)




---

[jira] [Created] (IGNITE-8114) Add fail recovery mechanism to tracking pages

[Eduard Shangareev (JIRA)]

Eduard Shangareev created IGNITE-8114:
-

 Summary: Add fail recovery mechanism to tracking pages
 Key: IGNITE-8114
 URL: https://issues.apache.org/jira/browse/IGNITE-8114
 Project: Ignite
  Issue Type: Improvement
Reporter: Eduard Shangareev
Assignee: Eduard Shangareev
 Fix For: 2.5


Now we just assert that last saved tag in tracking page should be not greater 
than passed one.
But because of different issues, it could happen.

So, I suggest to handle it by adding new "corruption" state to tracking state. 
If tracking page is in a corrupted state than we would throw an exception on 
any querying of the tracking page. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8113) ZookeeperClientTest#testReconnect3, testReconnect4 fail on TC

[Sergey Chugunov (JIRA)]

Sergey Chugunov created IGNITE-8113:
---

 Summary: ZookeeperClientTest#testReconnect3, testReconnect4 fail 
on TC
 Key: IGNITE-8113
 URL: https://issues.apache.org/jira/browse/IGNITE-8113
 Project: Ignite
  Issue Type: Bug
  Components: zookeeper
Reporter: Sergey Chugunov


Tests always fail on TC, testReconnect3 failure is reproducible locally as well.

A message in logs says that ZooKeeper cluster shuts itself down during the test:
{noformat}
[Step 4/5] java.lang.Exception: shutdown Leader! reason: Not sufficient 
followers synced, only synced with sids: [ 11 ]
{noformat}

Most likely there is a bug in test itself, no real functionality is affected.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8112) DirectIO artifact is not published in the maven repository

[Vyacheslav Koptilin (JIRA)]

Vyacheslav Koptilin created IGNITE-8112:
---

 Summary: DirectIO artifact is not published in the maven repository
 Key: IGNITE-8112
 URL: https://issues.apache.org/jira/browse/IGNITE-8112
 Project: Ignite
  Issue Type: Bug
  Components: build
Affects Versions: 2.4
Reporter: Vyacheslav Koptilin
Assignee: Vyacheslav Koptilin






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3705: ignite-8044: IgniteQueryGenerator.getOptions() me...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3705


---

Re: Apache Ignite RPM packaging (Stage II)

[Ilya Kasnacheev]

Hello!

Let me share my idea of how this shoud work. Splitting package into
sub-packages should be dependency-driven.

It means that all Ignite modules without dependencies or with small
dependencies (such as ignite-log4j) should be included in ignite-core. It
doesn't make sense to make a zillion RPM packages.

Critical things like ignite-spring and ignite-indexing should be in
ignite-core of course, even if they have dependencies. Ignite-core should
be fully self-sufficient and feature-complete.

However, e.g. .net API should probably be in a separate package, because it
should depend on mono | net-core. We may also have ignite-devel package
which should include all modules which only make sense for developers who
write code. Such as hibernate integration.

I'm not sure about MR modules. The main question should be, does it have
dependencies? Can it run stand-alone without writing code?

Hope this helps,

-- 
Ilya Kasnacheev

2018-03-27 15:10 GMT+03:00 Petr Ivanov :

> Hi, Igniters!
>
>
> Here are some news on our RPM packages initiative.
>
> 1. I’ve finished preliminary developing of Stage II version of RPM
> packages [1]. Main “new feature” is — split design. Also I’ve added
> package.sh script for automating package building process which will help
> organise corresponding builds in TC as well as simplify process for
> developers who wishes to have custom packages.
> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache
> Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer.
> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
> repositories on Apache Bintray. Though they are already prepared for
> hosting RPM and DEB packages respectively, and there is a way of linking
> them to apache.org/dist/ignite page, there is possible alternative in
> storing there only plain directory layout corresponding to each repository
> type (RPM and DEB) and manage this layout (repodata, distributions,
> versions, etc.) by ourselves, having more control over repositories but
> lacking some simplicity of deploying new releases. WDYT? Should we try
> Cassandra approach? They are storing their DEB packages as I described
> above [5].
>
> Also — a question arose while I was working on this issue: which OSes (and
> which versions of each) are we going to support (if we are going) in terms
> of step-by-step list? Currently RPM packages are tested only with latest
> CentOS (and, respectively — RHEL), but there are a lot more RPM-based
> distributives [6] some of which are more o less popular among OS community
> (ALT, Fedora, openSUSE, etc.).
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-7647
> [2] https://github.com/apache/ignite/pull/3703
> [3] https://bintray.com/apache/ignite-rpm
> [4] https://bintray.com/apache/ignite-deb
> [5] https://bintray.com/apache/cassandra/debian#files/
> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_distributions
>
>
>
>
> > On 15 Mar 2018, at 22:15, Petr Ivanov  wrote:
> >
> > I suppose that most everything if not all from libs/options will go to
> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
> > More precise lib selection (if something from optional would better to
> have in core package) will be discussed right after preliminary split
> architecture agreement.
> >
> >
> >
> >> On 15 Mar 2018, at 22:11, Dmitry Pavlov  wrote:
> >>
> >> I like idea of keeping simple system of modules, so +1 from me.
> >>
> >> Where optional libs (e.g Direct IO plugin) would be included, would it
> be
> >> core or optional?
> >>
> >> чт, 15 мар. 2018 г. в 22:09, Denis Magda :
> >>
> 
> >
> > How big would be a final core module?
>  Around 30M. Can be shrinked to ~15M if separate Visor and create it’s
> own
>  package.
> >>>
> >>>
> >>> Guys, 30 vs 280M is a hge difference.  I would agree with Petr and
> >>> propose the simplest modular system:
> >>>
> >>>  - core module that includes basic Ignite capabilities including SQL,
> >>>  compute grid, service grid, k/v
> >>>  - optional module hosts the rest - ML, streamers integration (kafka,
> >>>  flink), kubernetes, etc.
> >>>
> >>> What do you think?
> >>>
> >>> --
> >>> Denis
> >>>
> >>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov 
> wrote:
> >>>
>  *DEB package
> 
> 
> > On 15 Mar 2018, at 10:35, Petr Ivanov  wrote:
> >
> > Considering that DEV package for now is almost platform independent
> >>> (its
>  a java application more or less), that package will work almost on any
>  DEB-based linux, including but not limited to Ubuntu, Debian, etc.
> > The only restriction is existence of systemctl (systemd) service
> >>> manager
>  — we are dependent on it.
> >
> > Thats why, for instance, our RPM repository is called simply ‘rpm’
> and
>  package has no arch or dist suffix — it will 

Re: IGNITE-6827 - Review needed.

[Andrey Gura]

Guys,

I've looked at IGNITE-6827 implementation and have comment. I think
that rollbackOnTopologyChangeTimeout parameter isn't related with
implemented functionality. It's responsibility of PME to rollback
transactions that blocks exchange, so rollbackOnTopologyChangeTimeout
should be configured for PME not for transactions.

On Mon, Apr 2, 2018 at 2:53 PM, Dmitry Pavlov  wrote:
> Hi Igniters,
>
> A number of .NET tests failed, it seems it requied to be updated/fixed first
>Ignite Platform .NET Core Linux [ tests 2 ]
>   dll: TransactionsParityTest.TestTransactions (master fail rate 0,0%)
>   dll: CachePartitionedTest.TestTransactionScopeMultiCache(True)
> (master fail rate 0,0%)
>
>Ignite Queries [1] [ tests 1 ]
>   IgniteBinaryCacheQueryTestSuite:
> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
> (master fail rate 0,0%)
>
>Ignite Platform .NET
>   exe: TransactionsParityTest.TestTransactions (master fail rate 0,0%)
>
>Ignite Platform .NET Long Running [ tests 4 TIMEOUT ]
>   exe:
> CachePartitionedNearEnabledTest.TestTransactionScopeMultiCache(True)
> (master fail rate 0,0%)
>   exe: CachePartitionedTest.TestTransactionScopeMultiCache(True)
> (master fail rate 0,0%)
>   exe: CacheReplicatedTest.TestTransactionScopeMultiCache(True) (master
> fail rate 0,0%)
>
>
> And I don't have statitsics on test timeouts, but several timeouts still
> ocurred:
>Cache (Restarts) [1] [ tests 0 TIMEOUT , Exit Code ]
>  
> GridCachePartitionedNearDisabledOptimisticTxNodeRestartTest.testRestartWithTxTenNodesTwoBackups
> (last started) More >>
>
>Client Nodes [ tests 0 TIMEOUT , Exit Code warn 1 ]
>  IgniteClientReconnectFailoverTest.testReconnectStreamerApi (last started)
>
>Ignite PDS Indexing [ tests 0 TIMEOUT , Exit Code warn 9 ]
> IgnitePdsDiskErrorsRecoveringTest.testRecoveringOnWALErrorWithoutMmap (last
> started)
>
> Sincerely,
> Dmitriy Pavlov
>
> пн, 2 апр. 2018 г. в 12:48, Alexey Goncharuk :
>
>> Guys, especially Vladimir Ozerov, Yakov Zhanov, Dmitriy Setrakyan,
>>
>> While the technical side of the change is being reviewed, I would also ask
>> you to pay attention to the public API changes here.
>>
>> --AG
>>
>> 2018-03-21 23:43 GMT+03:00 Pavel Tupitsyn :
>>
>> > Hi Alexei,
>> >
>> > >  Pavel Tupitsyn, could you lend a hand to help completing .NET part
>> (two
>> > new
>> > public API methods are added as a part of working on [3], java code in
>> > branch [2])
>> >
>> > Please file a separate ticket for .NET changes.
>> > I would like to help, but maybe some time later. No need to delay your
>> > changes because of that.
>> >
>> > Thanks,
>> > Pavel
>> >
>> > On Wed, Mar 21, 2018 at 1:08 PM, Dmitry Pavlov 
>> > wrote:
>> >
>> > > Hi Alexei ,
>> > >
>> > > Thank you for contribution, unfortunately there is too much tests
>> failed
>> > > there:
>> > > .NET is broken
>> > >
>> > >   Ignite Platform .NET Long Running [ tests 3 ]
>> > >
>>  CachePartitionedNearEnabledTest.TestTransactionScopeMultiCache(True)
>> > > CachePartitionedTest.TestTransactionScopeMultiCache(True)
>> > > CacheReplicatedTest.TestTransactionScopeMultiCache(True)
>> > >
>> > >Ignite Platform .NET Core Linux [ tests 2 TC_EXIT_CODE ]
>> > >  TransactionsParityTest.TestTransactions (fail rate 0,0%)
>> > >  CachePartitionedTest.TestTransactionScopeMultiCache(True) (fail
>> > rate
>> > > 0,0%)
>> > >
>> > >Ignite Platform .NET [ tests 1 ] agoncharuk More info >>
>> > > Test fail rate less than 1%, probably new failure  exe:
>> > > TransactionsParityTest.TestTransactions (fail rate 0,0%)
>> > >
>> > > Too much timeouts:
>> > >Ignite Cache Failover [2]
>> > >  with CacheAsyncOperationsFailoverTxTest.testPutAllAsyncFailover (last
>> > > started)
>> > >
>> > >~[Obsolete] Ignite Cache Expiry Policy [ tests 1 ]
>> > > with  IgniteCacheExpiryPolicyTestSuite:
>> > > IgniteCacheTtlCleanupSelfTest.testDeferredDeleteTtl
>> > >
>> > >  Ignite Cache Restarts [2]
>> > >   with
>> > > GridCachePutAllFailoverSelfTest.testPutAllFailoverNearDisabled
>> > ThreeBackups
>> > > (last started)
>> > >
>> > >   Ignite Cache Restarts [1]
>> > >with
>> > > GridCachePartitionedNearDisabledOptimisticTxNodeRestartTest.
>> > > testRestartWithTxTenNodesTwoBackups
>> > > (last started)
>> > >
>> > >  Cache [6]
>> > >with  WalModeChangeAdvancedSelfTest.testServerRestartNonCoordinator
>> > > (last started)
>> > > it is not all.
>> > >
>> > > Please fix tests before review. Also you could change ticket status to
>> > 'In
>> > > Progress'.
>> > >
>> > > As easy option, you can merge current master to your branch, it is
>> > probably
>> > > that some test were fixed already.
>> > >
>> > > Sincerely,
>> > > Dmitriy Pavlov
>> > >
>> > > ср, 21 мар. 2018 г. в 9:58, Alexei Scherbakov <
>> > > alexey.scherbak...@gmail.com
>> > > >:
>> > >
>> > > > Igniters,
>> > 

[GitHub] ignite pull request #3732: Ignite 8071

[x-kreator]

GitHub user x-kreator opened a pull request:

https://github.com/apache/ignite/pull/3732

Ignite 8071



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/x-kreator/ignite ignite-8071

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3732.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3732


commit ca0a3a91d024bcd36fe554731f80f762cf205543
Author: Dmitriy Sorokin 
Date:   2018-03-30T14:17:52Z

IGNITE-8071: Add tests for failure handlers

commit 80a072c654ba73883bb6b59eb1b9210aa7a45245
Author: Dmitriy Sorokin 
Date:   2018-04-02T15:07:26Z

IGNITE-8071: + StopNodeOrHaltFailureHandlerTest.




---

Re: Static code analysis for Java

[Dmitry Pavlov]

According recent conversation 'Code Inspection' I would like to bump up
this thread too.

Igniters, who can advice if coverity has integration/ability to be run on
TeamCity? Who can help to proof, that it is possible to be executed using
maven.

вт, 19 сент. 2017 г. в 14:27, Yakov Zhdanov :

> Bas, thanks for joining!
>
> Can you please point me  to the page listing all types of issue LGTM can
> find (like this one -
> https://scan.coverity.com/faq#what-types-of-issues-tool-find)?
>
> LGTM really helped to find some bugs like incorrect key type when working
> with hash map instance, unnecessary boxing, unused collections, possible
> resource leaks and some more.
>
> Do you users integrate with CI servers somehow? esp TeamCity? It would be
> cool to have project state (at least from code standpoint) in one place -
> i.e. at CI.
>
> --Yakov
>

[jira] [Created] (IGNITE-8111) Add extra validation for WAL segment size

[Ivan Rakov (JIRA)]

Ivan Rakov created IGNITE-8111:
--

 Summary: Add extra validation for WAL segment size
 Key: IGNITE-8111
 URL: https://issues.apache.org/jira/browse/IGNITE-8111
 Project: Ignite
  Issue Type: Improvement
Affects Versions: 2.4
Reporter: Ivan Rakov


Currently we can set extra small DataStorageConfiguration#walSegmentSize (10 
pages or even less than one page), which will trigger multiple assertion errors 
in code.
We have to implement validation on node start that WAL segment size has 
reasonable value (e.g. more than 512KB).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8110) GridCacheWriteBehindStore.Flusher thread uses the wrong transformation from milliseconds to nanoseconds.

[Vyacheslav Koptilin (JIRA)]

Vyacheslav Koptilin created IGNITE-8110:
---

 Summary: GridCacheWriteBehindStore.Flusher thread uses the wrong 
transformation from milliseconds to nanoseconds.
 Key: IGNITE-8110
 URL: https://issues.apache.org/jira/browse/IGNITE-8110
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 2.4
Reporter: Vyacheslav Koptilin


The initial value of a cache flushing frequency is defined as follows:
{code}
/** Cache flushing frequence in nanos. */
protected long cacheFlushFreqNanos = cacheFlushFreq * 1000;
{code}

where is {{cacheFlushFreq}} is equal to
{code}
/** Default flush frequency for write-behind cache store in milliseconds. */
public static final long DFLT_WRITE_BEHIND_FLUSH_FREQUENCY = 5000;
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8109) IgniteCacheEntryListenerAtomicTest#testConcurrentRegisterDeregister timeouts when ZooKeeper Discovery is used

[Sergey Chugunov (JIRA)]

Sergey Chugunov created IGNITE-8109:
---

 Summary: 
IgniteCacheEntryListenerAtomicTest#testConcurrentRegisterDeregister timeouts 
when ZooKeeper Discovery is used
 Key: IGNITE-8109
 URL: https://issues.apache.org/jira/browse/IGNITE-8109
 Project: Ignite
  Issue Type: Bug
  Components: zookeeper
Reporter: Sergey Chugunov


IgniteCacheEntryListenerAtomicTest#testConcurrentRegisterDeregister test works 
fast when TCP-based Discovery is used but takes a long time to finish on 
ZooKeeper-based Discovery. 

Reproducible locally; usually takes from 3 to 4 minutes for one test, on TC 
fails with 5 minutes timeout.

Investigation of slow down root cause is needed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3731: IGNITE-8108: .NET: Fix flaky EntityFrameworkCache...

[apopovgg]

GitHub user apopovgg opened a pull request:

https://github.com/apache/ignite/pull/3731

IGNITE-8108: .NET: Fix flaky EntityFrameworkCacheInitializationTest.T…

…estConfigurationAndStartup

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8108

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3731.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3731


commit 096db41e45d21c386e7e1ff2c661e42492f55dae
Author: apopov 
Date:   2018-04-02T14:12:59Z

IGNITE-8108: .NET: Fix flaky 
EntityFrameworkCacheInitializationTest.TestConfigurationAndStartup




---

Re: Apache Ignite RPM packaging (Stage II)

[Petr Ivanov]

> On 29 Mar 2018, at 12:32, Max Shonichev  wrote:
> 
> 
> 
> 1. About packaging/package.sh
> 
> 1.1. I personally dislike the use of bashism, e.g. like reserved keywords,
> double '[[', double '((', but that's ok
> for targeted distro (RHEL/Centos). The only requirement is please add more
> comments in tough places
> to improve code readability and maintanbility.

Will try to get rid of them as much as possible, but SH is really outdated 
scripting language and most all of the systems now support BASH, even Windows :)


> 
> 
> 1.2. few code style comments:
> 
>>> name=$(cat /etc/*release | grep ^NAME | sed -r 's|.*"(.*)".*|\1|')
> 
> add
> 
>   | head -n 1
> 
> or else it will fail in case more than one file with *release mask in /etc

I disagree. Script will not fail, just cat every file matched by mask and then 
pass everything as one single stdout.


> 
> 
> 1.3. rework prepEnv
> 
>>> if [ ! -z installCmd ]; then
> 
> this will fail miserably, replace with
> 
>   if [ ! -z $installCmd ]; then

Fixed


> 
> 1.3.1. however running package.sh under Ubuntu still would fail due to
> $installCmd is unpacked to 'apt --no-install-recommends'
> and -z 'apt --no-install-recommends' would always return false.

The main purpose of checking that variable ${installCmd} is not empty — because 
it is instantiated empty and will stay empty if OS detection won’t be able to 
define OS and pre-build software installation and environment preparation will 
be skipped.


> 
> 1.3.2. and still, when apt/yum is installed and rpm/rpm-build is not, the
> whole prepEnv would fail to ensure all
> build-time requirements are installed, because it only checks for apt/yum.

I suppose that apt / yum is always installed by default because it is default 
package manager for corresponding OSes.
I did not get where is error here.


> 
> 1.3.3. installing requires root privileges. However, no checks for $EUID nor
> running apt/yum with 'sudo' present.
> Please, add this check, otherwise running package.sh under unprivileged user
> with no installed build-time dependencies
> will fail with very undescriptive errors.

Check for ${EUID} will mean that whole build package process will run under 
root, which is undesirable.
Added INFO message that explains necessity for super user privileges and added 
sudo command to root-required commands.


> 
> 
> 1.4. getBin
> 
> 1.4.1. I don't like detecting target apache ignite package version by
> parsing embedded changelog. This might fail
> when any package text description would contain '*' in the first char of the
> string. Either move changelog to separate
> Changelog file and %include it in .spec or parse for, e.g. '^Version:
> [0-9\.]+'
> 
>>> igniteVersion=$(cat rpm/SPECS/apache-ignite.spec | grep '^*' | head -1 |
>>> sed -r 's|.*\ -\ (.*)-.*|\1|’)

Replaced with parsing of main version


> 
> 1.4.2.
> 
>>> ls ${binName} && binPreparedFlag=true || true
> 
> why not use
> 
>  if [ ! -f $binName ];

Indeed this variant is much better, fixed.


> 
> 1.4.3. 'curl' is used, but not checked as build-time dependency.

Added to build dependencies (via prepEnv function)


> 
> 1.4.4. we try to find and download 'apache-ignite-fabric-$igniteVersion',
> that's ok for <2.4 releases, but in future
> we would need to find and download 'apache-ignite-$igniteVersion' too.
> Better add this check as a fallback option, e.g.
> try to use curl with both urls in correct order.
> 
>  if curl --output /dev/null --silent --head --fail "$url"; then
>echo "URL exists: $url"
>  else
>echo "URL does not exist: $url"
>  fi

1. Packages did not exist before 2.4.0.
2. Build script does not provide opportunity to build packages of different 
from .spec versions and is considered to be part of packaging build process 
(specifications and packaging script are modified together) — a shortcut, thus 
relying on current (same commit) state for building.


> 
> 1.4.5. also strange, but may be ok that we download
> apache-ignite-fabric-2.4.0

Fabric removal task is not yet merged to master.
Will be supported in its reimplementation


> 
> 
> 2. about apache-ignite.spec
> 
> 2.1. fix typo
> 
>>> Apache Ignite's optinal libs and integrations

Fixed


> 
> 2.2. fix description
> 
>>> C++ files necessary for using Apache Ignite
> 
> platform files are not necessary for Ignite, only for integration with .NET,
> afaik?

Introduced alternative description


> 
> 2.3. Tough place with 'Obsoletes:apache-ignite < 2.5.0'
> 
> I understand the current logic behind splitting apache-ignite to several
> .rpm's and setting obsolete for the old
> apache-ignite.
> 
> However, for future releases, e.g. if/when we handover this task to another
> engineer, it is not quite obvious that one
> must (or must not?) update version in 5 different places, including
> changelog. Please add a few lines about
> 'How to package next (new) version of Apache Ignite RPMs' to Confluence.

Everything that relies on package version is 

[jira] [Created] (IGNITE-8108) .NET: Fix flaky EntityFrameworkCacheInitializationTest.TestConfigurationAndStartup

[Alexey Popov (JIRA)]

Alexey Popov created IGNITE-8108:


 Summary: .NET: Fix flaky 
EntityFrameworkCacheInitializationTest.TestConfigurationAndStartup
 Key: IGNITE-8108
 URL: https://issues.apache.org/jira/browse/IGNITE-8108
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Affects Versions: 2.5
Reporter: Alexey Popov
Assignee: Alexey Popov


https://ci.ignite.apache.org/viewLog.html?buildId=1171195=IgniteTests24Java8_IgnitePlatformNetIntegrations=buildLog&_focus=6035

Test uses multicast ipFinder.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #2749: IGNITE-6186: Removed redundant parameter.

[andrey-kuznetsov]

Github user andrey-kuznetsov closed the pull request at:

https://github.com/apache/ignite/pull/2749


---

[jira] [Created] (IGNITE-8107) Drop cache with SQL table, recreate table on a different cache - cluster fails to activate

[Ilya Kasnacheev (JIRA)]

Ilya Kasnacheev created IGNITE-8107:
---

 Summary: Drop cache with SQL table, recreate table on a different 
cache - cluster fails to activate
 Key: IGNITE-8107
 URL: https://issues.apache.org/jira/browse/IGNITE-8107
 Project: Ignite
  Issue Type: Bug
  Components: sql
Reporter: Ilya Kasnacheev


This is a bizzare twist on IGNITE-8021

Imagine you have a cache "tbl" created by public API with corresponding table 
"tbl".TBL. You also have PDS. Drop this cache, CREATE TABLE TBL with 
corresponding SQL_PUBLIC_TBL cache. It would work, but if you shut down cluster 
and try to bring it online, it will fail to activate:

{code}SchemaOperationException [code=3, msg=Table already exists: TBL]{code}






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8106) In control.sh -active, exception will be eaten and not displayed

[Ilya Kasnacheev (JIRA)]

Ilya Kasnacheev created IGNITE-8106:
---

 Summary: In control.sh -active, exception will be eaten and not 
displayed
 Key: IGNITE-8106
 URL: https://issues.apache.org/jira/browse/IGNITE-8106
 Project: Ignite
  Issue Type: Bug
  Components: clients
Affects Versions: 2.5
Reporter: Ilya Kasnacheev


Here is the relevant code from GridChangeStateCommandHandler:
{code}
sb.a(e.getMessage()).a("\n").a("suppressed: \n");

for (Throwable t:e.getSuppressed())
sb.a(t.getMessage()).a("\n");
{code}

However, before that code the exception will pass through 
IgniteUtils.convertException(), which will wrap the old IgniteCheckedException 
with suppressed parts with a new IgniteException with 0 suppressed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Fwd: Data Loss while upgrading custom jar from old jar in server and client nodes

[Andrey Mashenkov]

Crossposting to dev.

Guys,
User use Ignite native persistence with CacheStore configured.
Seems, changing CacheStore requires cache recreation in his case to changes
can be applied.

Does anybody has idea how it can be fixed?

Do we allow to rewrite cache config in native store if it has minor changes
that will not cause issues? E.g. changing cacheStore factory or changing
number of backups?
Otherwise, I'd suggest to deprecate such configuration.


On Tue, Mar 27, 2018 at 4:18 PM, siva  wrote:

> Hi,
>
> Thanks for checking out issue. I think its not about IDE issue.
> Again same thing i have reproduce like bellow steps
>
> 1.run the program from IDE as client
> 2.copy jar to ignite libs folder and start the server
> 3.Then check previous message steps so that can able to reproduce the
> issue.
> https://www.youtube.com/watch?v=COQiu2gi8ag=43s
> 
> I have attached the video link.Can you go through that video and check the
> issue.
>
>
> Thanks
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov



-- 
Best regards,
Andrey V. Mashenkov

[GitHub] ignite pull request #2854: IGNITE-6630 Time units fix.

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2854


---

[jira] [Created] (IGNITE-8105) Close() while auto activate

[Alexander Belyak (JIRA)]

Alexander Belyak created IGNITE-8105:


 Summary: Close() while auto activate
 Key: IGNITE-8105
 URL: https://issues.apache.org/jira/browse/IGNITE-8105
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 2.4
Reporter: Alexander Belyak


1) Start one node, activate, fill some data
2) Close node
3) Start node and right after start call close()
Expected result:
Node start and close correctly (maybe with auto activate/deactivate, maybe 
without)
Actual result:
Node start and throw java.nio.channels.ClosedByInterruptException in activation 
process cause close() process close checkpoint file channel
Expection is:
{noformat}
[2018-04-02 
19:57:27,831][ERROR][exchange-worker-#94%srv1%][GridCachePartitionExchangeManager]
 Failed to wait for completion of partition map exchange (preloading will not 
start): GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryCustomEvent 
[customMsg=null, affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], 
super=DiscoveryEvent [evtNode=TcpDiscoveryNode 
[id=a9fa729f-613f-496b-8e7c-e53142817226, addrs=[0:0:0:0:0:0:0:1%lo, 10.0.3.1, 
10.38.184.66, 10.42.1.107, 127.0.0.1, 172.17.0.1], 
sockAddrs=[/10.38.184.66:47500, /172.17.0.1:47500, /0:0:0:0:0:0:0:1%lo:47500, 
/10.0.3.1:47500, /10.42.1.107:47500, /127.0.0.1:47500], discPort=47500, 
order=1, intOrder=1, lastExchangeTime=1522673846477, loc=true, 
ver=2.4.0#19700101-sha1:, isClient=false], topVer=1, nodeId8=a9fa729f, 
msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1522673847732]], 
crd=TcpDiscoveryNode [id=a9fa729f-613f-496b-8e7c-e53142817226, 
addrs=[0:0:0:0:0:0:0:1%lo, 10.0.3.1, 10.38.184.66, 10.42.1.107, 127.0.0.1, 
172.17.0.1], sockAddrs=[/10.38.184.66:47500, /172.17.0.1:47500, 
/0:0:0:0:0:0:0:1%lo:47500, /10.0.3.1:47500, /10.42.1.107:47500, 
/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, 
lastExchangeTime=1522673846477, loc=true, ver=2.4.0#19700101-sha1:, 
isClient=false], exchId=GridDhtPartitionExchangeId 
[topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], 
discoEvt=DiscoveryCustomEvent [customMsg=null, 
affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], 
super=DiscoveryEvent [evtNode=TcpDiscoveryNode 
[id=a9fa729f-613f-496b-8e7c-e53142817226, addrs=[0:0:0:0:0:0:0:1%lo, 10.0.3.1, 
10.38.184.66, 10.42.1.107, 127.0.0.1, 172.17.0.1], 
sockAddrs=[/10.38.184.66:47500, /172.17.0.1:47500, /0:0:0:0:0:0:0:1%lo:47500, 
/10.0.3.1:47500, /10.42.1.107:47500, /127.0.0.1:47500], discPort=47500, 
order=1, intOrder=1, lastExchangeTime=1522673846477, loc=true, 
ver=2.4.0#19700101-sha1:, isClient=false], topVer=1, nodeId8=a9fa729f, 
msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1522673847732]], nodeId=a9fa729f, 
evt=DISCOVERY_CUSTOM_EVT], added=true, initFut=GridFutureAdapter 
[ignoreInterrupts=false, state=DONE, res=false, hash=791289709], init=false, 
lastVer=null, partReleaseFut=PartitionReleaseFuture 
[topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], 
futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=1, 
minorTopVer=1], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion 
[topVer=1, minorTopVer=1], futures=[]], AtomicUpdateReleaseFuture 
[topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], futures=[]], 
DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=1, 
minorTopVer=1], futures=[, exchActions=null, affChangeMsg=null, 
initTs=1522673847742, centralizedAff=false, changeGlobalStateE=class 
o.a.i.IgniteCheckedException: Failed to read checkpoint pointer from marker 
file: 
/tmp/test/srv1/db/cons_srv1/cp/1522673842894-84776dc9-6fac-4aa0-804c-f56cbee68c12-START.bin,
 done=true, state=CRD, evtLatch=0, remaining=[], super=GridFutureAdapter 
[ignoreInterrupts=false, state=DONE, res=class o.a.i.IgniteCheckedException: 
Failed to read checkpoint pointer from marker file: 
/tmp/test/srv1/db/cons_srv1/cp/1522673842894-84776dc9-6fac-4aa0-804c-f56cbee68c12-START.bin,
 hash=1311860231]]
class org.apache.ignite.IgniteCheckedException: Failed to read checkpoint 
pointer from marker file: 
/tmp/test/srv1/db/cons_srv1/cp/1522673842894-84776dc9-6fac-4aa0-804c-f56cbee68c12-START.bin
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readPointer(GridCacheDatabaseSharedManager.java:1794)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readCheckpointStatus(GridCacheDatabaseSharedManager.java:1764)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreState(GridCacheDatabaseSharedManager.java:1321)
at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.beforeExchange(GridCacheDatabaseSharedManager.java:1114)
at 

[jira] [Created] (IGNITE-8104) Proper toString() implementation for IgniteConfiguration properties

[Ivan Fedotov (JIRA)]

Ivan Fedotov created IGNITE-8104:


 Summary: Proper toString() implementation for IgniteConfiguration 
properties
 Key: IGNITE-8104
 URL: https://issues.apache.org/jira/browse/IGNITE-8104
 Project: Ignite
  Issue Type: Improvement
 Environment: In continuing of ticket Ignite-5798 'Logging Ignite 
configuration at startup' [1] it's necessary to make proper {{toString()}} 
implementation for every every logged configuration component, such the every 
meaningful property should be properly logged.

[1]https://issues.apache.org/jira/browse/IGNITE-5798
Reporter: Ivan Fedotov






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3660: IGNITE-7987 Use all discovery events for affinity...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3660


---

[GitHub] ignite pull request #3721: IGNITE-8080 Avoid updating tx entries twice

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3721


---

[jira] [Created] (IGNITE-8103) Node with BLT is not allowed to join cluster without one

[Alexander Belyak (JIRA)]

Alexander Belyak created IGNITE-8103:


 Summary: Node with BLT is not allowed to join cluster without one
 Key: IGNITE-8103
 URL: https://issues.apache.org/jira/browse/IGNITE-8103
 Project: Ignite
  Issue Type: Improvement
  Components: general
Affects Versions: 2.4
Reporter: Alexander Belyak


1) Start cluster of 2-3 nodes and activate it, fill some data
2) Stop cluster, clear LFS on first node
3) Start cluster from first node (or start all nodes synchronously)
Expected result: ?
Actual result: "Node with set up BaselineTopology is not allowed to join 
cluster without one: cons_srv2"
In the technical point of view it's expected behaviour, because first node with 
cleared storage became grid coordinator and reject any connection attempts from 
nodes with different baseline. But it's bad for usability: if we always start 
all nodes together and wanna clear storage on one node by some reason - we need 
to define start sequence.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Service grid redesign

[Denis Mekhanikov]

Vyacheslav,

Service deployment design, based on replicated utility cache has proven to
be unstable and deadlock-prone.
You can find a list of JIRA issues, connected to it, in my previous letter.

The intention behind it is similar to the binary metadata redesign, that
happened in the following ticket: IGNITE-4157

This change in service deployment procedure will eliminate need for another
internal replicated cache
and make service deployment more reliable on unstable topology.

Denis

вт, 27 мар. 2018 г. в 23:21, Vyacheslav Daradur :

> Hi, Denis Mekhanikov!
>
> As far as I know, Ignite services are based on IgniteCache and we have
> all its features. We can use listeners or continuous queries for
> deployment synchronizations.
>
> Why do you want using the discovery layer for that?
>
> One more thing: we can use baseline approach for services, that means
> *IgniteService.deploy()* returns ready to work service after
> deployment on baseline nodes and deploy to other nodes on demand, for
> example when deployed service's loading will be hight.
>
> About versioning, maybe there is sense to extend public API:
> IgniteServices.service(name, *version*)?
>
> At first deployment, we can compute service's hashcode (just for an
> example) and store it, after new deployment request for services with
> an existing name we will compute new service's hashcode and compare
> them if they have different hashcodes that we will deploy new service
> as service with a different version.
>
>
> On Fri, Mar 23, 2018 at 10:03 PM, Denis Magda  wrote:
> > Denis,
> >
> > Thanks for the extensive analysis. There is a vast room for optimizations
> > on the service grid side.
> >
> > Yakov, Sam, Alex G.,
> >
> > How do you like the idea of the usage of discovery protocol for the
> service
> > grid system messages exchange? Any pitfalls?
> >
> >
> > --
> > Denis
> >
> >
> > On Fri, Mar 23, 2018 at 8:01 AM, Denis Mekhanikov  >
> > wrote:
> >
> >> Igniters,
> >>
> >> I'd like to start a discussion on Ignite service grid redesign.
> >> We have a number of problems in our current architecture, that have to
> be
> >> addressed.
> >>
> >> Here are the most severe ones:
> >>
> >> One of them is lack of guarantee, that service is successfully deployed
> and
> >> ready for work by the time, when *IgniteService.deploy*()* methods
> return.
> >> Furthermore, if an exception is thrown from *Service.init() *method,
> then
> >> the deploying side is not able to receive it, or even understand, that
> >> service is in unusable state.
> >> So, you may end up in such situation, when you deployed a service
> without
> >> receiving any errors, then called a service's method, and hung
> indefinitely
> >> on this invocation.
> >> JIRA ticket: https://issues.apache.org/jira/browse/IGNITE-3392
> >>
> >> Another problem is locking during service deployment on unstable
> topology.
> >> This issue is caused by missing updates in continuous query listeners on
> >> the internal cache.
> >> It is hard to reproduce, but it happens sometimes. We shouldn't allow
> such
> >> possibility, that deployment methods hang without saying anything.
> >> JIRA ticket: https://issues.apache.org/jira/browse/IGNITE-6259
> >>
> >> I think, we should change the deployment procedure to make it more
> >> reliable.
> >> Moving from operating over internal replicated service cache to sending
> >> custom discovery events seems to be a good idea.
> >> Service deployment may trigger a discovery event, that will make chosen
> >> nodes deploy the service, and the same event will notify other nodes
> about
> >> the deployed service instances.
> >> It will eliminate the need for distributed transactions on the internal
> >> replicated system cache, and make the service deployment protocol more
> >> transparent.
> >>
> >> There are a few points, that should be taken into account though.
> >>
> >> First of all, we can't wait for services to be deployed and initialised
> in
> >> the discovery thread.
> >> So, we need to make notification about service deployment result
> >> asynchronous, presumably over communication protocol.
> >> I can think of a procedure similar to the current exchange protocol,
> when
> >> service deployment is initialised with an initial discovery message,
> >> followed by asynchronous notifications from the hosting servers over
> >> communication. And finally, one more discovery message will notify all
> >> nodes about the service deployment result and location of the deployed
> >> service instances. Coordinator will be responsible for collecting of the
> >> deployment results in this scheme.
> >>
> >> Another problem is failover in case, when some nodes fail during
> deployment
> >> or further work.
> >> The following cases should be handled:
> >>
> >>1. coordinator failure during deployment;
> >>2. failure of nodes, that were chosen to host the 

Re: IGNITE-6827 - Review needed.

[Dmitry Pavlov]

Hi Igniters,

A number of .NET tests failed, it seems it requied to be updated/fixed first
   Ignite Platform .NET Core Linux [ tests 2 ]
  dll: TransactionsParityTest.TestTransactions (master fail rate 0,0%)
  dll: CachePartitionedTest.TestTransactionScopeMultiCache(True)
(master fail rate 0,0%)

   Ignite Queries [1] [ tests 1 ]
  IgniteBinaryCacheQueryTestSuite:
IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
(master fail rate 0,0%)

   Ignite Platform .NET
  exe: TransactionsParityTest.TestTransactions (master fail rate 0,0%)

   Ignite Platform .NET Long Running [ tests 4 TIMEOUT ]
  exe:
CachePartitionedNearEnabledTest.TestTransactionScopeMultiCache(True)
(master fail rate 0,0%)
  exe: CachePartitionedTest.TestTransactionScopeMultiCache(True)
(master fail rate 0,0%)
  exe: CacheReplicatedTest.TestTransactionScopeMultiCache(True) (master
fail rate 0,0%)


And I don't have statitsics on test timeouts, but several timeouts still
ocurred:
   Cache (Restarts) [1] [ tests 0 TIMEOUT , Exit Code ]
 
GridCachePartitionedNearDisabledOptimisticTxNodeRestartTest.testRestartWithTxTenNodesTwoBackups
(last started) More >>

   Client Nodes [ tests 0 TIMEOUT , Exit Code warn 1 ]
 IgniteClientReconnectFailoverTest.testReconnectStreamerApi (last started)

   Ignite PDS Indexing [ tests 0 TIMEOUT , Exit Code warn 9 ]
IgnitePdsDiskErrorsRecoveringTest.testRecoveringOnWALErrorWithoutMmap (last
started)

Sincerely,
Dmitriy Pavlov

пн, 2 апр. 2018 г. в 12:48, Alexey Goncharuk :

> Guys, especially Vladimir Ozerov, Yakov Zhanov, Dmitriy Setrakyan,
>
> While the technical side of the change is being reviewed, I would also ask
> you to pay attention to the public API changes here.
>
> --AG
>
> 2018-03-21 23:43 GMT+03:00 Pavel Tupitsyn :
>
> > Hi Alexei,
> >
> > >  Pavel Tupitsyn, could you lend a hand to help completing .NET part
> (two
> > new
> > public API methods are added as a part of working on [3], java code in
> > branch [2])
> >
> > Please file a separate ticket for .NET changes.
> > I would like to help, but maybe some time later. No need to delay your
> > changes because of that.
> >
> > Thanks,
> > Pavel
> >
> > On Wed, Mar 21, 2018 at 1:08 PM, Dmitry Pavlov 
> > wrote:
> >
> > > Hi Alexei ,
> > >
> > > Thank you for contribution, unfortunately there is too much tests
> failed
> > > there:
> > > .NET is broken
> > >
> > >   Ignite Platform .NET Long Running [ tests 3 ]
> > >
>  CachePartitionedNearEnabledTest.TestTransactionScopeMultiCache(True)
> > > CachePartitionedTest.TestTransactionScopeMultiCache(True)
> > > CacheReplicatedTest.TestTransactionScopeMultiCache(True)
> > >
> > >Ignite Platform .NET Core Linux [ tests 2 TC_EXIT_CODE ]
> > >  TransactionsParityTest.TestTransactions (fail rate 0,0%)
> > >  CachePartitionedTest.TestTransactionScopeMultiCache(True) (fail
> > rate
> > > 0,0%)
> > >
> > >Ignite Platform .NET [ tests 1 ] agoncharuk More info >>
> > > Test fail rate less than 1%, probably new failure  exe:
> > > TransactionsParityTest.TestTransactions (fail rate 0,0%)
> > >
> > > Too much timeouts:
> > >Ignite Cache Failover [2]
> > >  with CacheAsyncOperationsFailoverTxTest.testPutAllAsyncFailover (last
> > > started)
> > >
> > >~[Obsolete] Ignite Cache Expiry Policy [ tests 1 ]
> > > with  IgniteCacheExpiryPolicyTestSuite:
> > > IgniteCacheTtlCleanupSelfTest.testDeferredDeleteTtl
> > >
> > >  Ignite Cache Restarts [2]
> > >   with
> > > GridCachePutAllFailoverSelfTest.testPutAllFailoverNearDisabled
> > ThreeBackups
> > > (last started)
> > >
> > >   Ignite Cache Restarts [1]
> > >with
> > > GridCachePartitionedNearDisabledOptimisticTxNodeRestartTest.
> > > testRestartWithTxTenNodesTwoBackups
> > > (last started)
> > >
> > >  Cache [6]
> > >with  WalModeChangeAdvancedSelfTest.testServerRestartNonCoordinator
> > > (last started)
> > > it is not all.
> > >
> > > Please fix tests before review. Also you could change ticket status to
> > 'In
> > > Progress'.
> > >
> > > As easy option, you can merge current master to your branch, it is
> > probably
> > > that some test were fixed already.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > ср, 21 мар. 2018 г. в 9:58, Alexei Scherbakov <
> > > alexey.scherbak...@gmail.com
> > > >:
> > >
> > > > Igniters,
> > > >
> > > > I prepared rather important patch [1] related to grid stability and
> > need
> > > a
> > > > review and some help to finish it.
> > > >
> > > > Semyon Boikov, could you please take a look on patch code ? [2]
> > > >
> > > > Pavel Tupitsyn, could you lend a hand to help completing .NET part
> (two
> > > new
> > > > public API methods are added as a part of working on [3], java code
> in
> > > > branch [2])
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-6827
> > > >
> > > > [2] https://github.com/gridgain/apache-ignite/tree/ignite-6827-2
> > > 

[jira] [Created] (IGNITE-8102) IgnitePdsCorruptedStoreTest.testNodeInvalidatedWhenPersistenceIsCorrupted failed in Direct IO 2 suite

[Dmitriy Pavlov (JIRA)]

Dmitriy Pavlov created IGNITE-8102:
--

 Summary: 
IgnitePdsCorruptedStoreTest.testNodeInvalidatedWhenPersistenceIsCorrupted 
failed in Direct IO 2 suite
 Key: IGNITE-8102
 URL: https://issues.apache.org/jira/browse/IGNITE-8102
 Project: Ignite
  Issue Type: Test
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov


   PDS (Direct IO) [2] [ tests 1 ] 
  IgnitePdsNativeIoTestSuite2: 
IgnitePdsCorruptedStoreTest.testNodeInvalidatedWhenPersistenceIsCorrupted 
(master fail rate 6,8%)  

https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=7830903222674504151=pull%2F3624%2Fhead=testDetails_IgniteTests24Java8=%3Cdefault%3E

Failure contains suspicious error:
{noformat}
[2018-04-01 
15:35:51,704][ERROR][exchange-worker-#25589%persistence.IgnitePdsCorruptedStoreTest0%][IgniteTestResources]
 Critical failure. Will be handled accordingly to configured handler [hnd=class 
o.a.i.failure.NoOpFailureHandler, failureCtx=FailureContext 
[type=CRITICAL_ERROR, err=java.lang.IllegalStateException: Page is broken. 
Can't restore it from WAL. (grpId = -1368047377, pageId = 100020008).]] 
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8101) Ability to terminate system workers by JMX for test purposes

[Dmitriy Sorokin (JIRA)]

Dmitriy Sorokin created IGNITE-8101:
---

 Summary: Ability to terminate system workers by JMX for test 
purposes
 Key: IGNITE-8101
 URL: https://issues.apache.org/jira/browse/IGNITE-8101
 Project: Ignite
  Issue Type: Improvement
Reporter: Dmitriy Sorokin
Assignee: Dmitriy Sorokin
 Fix For: 2.5






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8100) jdbc getSchemas method could miss schemas for not started remote caches

[Pavel Kuznetsov (JIRA)]

Pavel Kuznetsov created IGNITE-8100:
---

 Summary: jdbc getSchemas method could miss schemas for not started 
remote caches
 Key: IGNITE-8100
 URL: https://issues.apache.org/jira/browse/IGNITE-8100
 Project: Ignite
  Issue Type: Bug
Reporter: Pavel Kuznetsov


On jdbc side we have 
org.apache.ignite.internal.jdbc.thin.JdbcThinDatabaseMetadata#getSchemas(java.lang.String,
 java.lang.String)

on the server side result is constructed by this:
{noformat}
for (String cacheName : ctx.cache().publicCacheNames()) {
for (GridQueryTypeDescriptor table : ctx.query().types(cacheName)) {
if (matches(table.schemaName(), schemaPtrn))
   schemas.add(table.schemaName());
}
}
{noformat}

see 
org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler#getSchemas

This if we have not started cache(with a table) on some remote node, we will 
miss that scheme.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3702: IGNITE-7884

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3702


---

[GitHub] ignite pull request #3730: ignite-zk latest master merged with ZooKeeper dis...

[sergey-chugunov-1985]

GitHub user sergey-chugunov-1985 opened a pull request:

https://github.com/apache/ignite/pull/3730

ignite-zk latest master merged with ZooKeeper discovery impl branch



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-zk-sch-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3730.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3730


commit 54211bfaee22bc77714f6c821b0765815f11e386
Author: sboikov 
Date:   2017-11-14T13:05:49Z

zk

commit d2f5a76cc725052d76a216603b2579f17ba92d60
Author: sboikov 
Date:   2017-11-14T13:34:42Z

zk

commit 775a60f79904686b2e443789ae8c62df74f4d6fa
Author: sboikov 
Date:   2017-11-14T14:36:24Z

zk

commit bedc4e99e14bd597616b134d99ea75cb4d22ea08
Author: sboikov 
Date:   2017-11-14T19:37:24Z

zk

commit 1ee551c856e988287ed6beeb054d53d89cb8800d
Author: sboikov 
Date:   2017-11-14T20:41:53Z

zk

commit ac07cbee75f3f230a12075ac91e4dad0f1a89b0b
Author: sboikov 
Date:   2017-11-15T07:10:24Z

Merge remote-tracking branch 'origin/ignite-zk' into ignite-zk

commit aa0ca90cbaec809715190c1036654a6aad0fb0a3
Author: sboikov 
Date:   2017-11-15T09:50:14Z

zk

commit 4ec75fa2900e3f4624d6516e0c50fc1877d7b5cc
Author: sboikov 
Date:   2017-11-15T10:12:26Z

zk

commit c55d5c2d039f58e3bd0d3ac089f2dfc09d6f90b9
Author: sboikov 
Date:   2017-11-15T11:20:08Z

zk

commit 11e2567fffa724e6b4af6021cda1bfbcf775370b
Author: sboikov 
Date:   2017-11-16T13:10:04Z

zk

commit 98a171c68a1f5610e5f5830144306ee73df866d6
Author: sboikov 
Date:   2017-11-16T14:42:05Z

zk

commit b389f38cbc59f41dc1c95854684059f15b225b8c
Author: sboikov 
Date:   2017-11-17T06:33:30Z

zk

commit 5793ffbe23a095127cc88c1962785cad7cf2432d
Author: sboikov 
Date:   2017-11-17T09:05:55Z

zk

commit 45e7e40603fa2b496b94f00fc287221d30073c98
Author: sboikov 
Date:   2017-11-20T08:14:36Z

zk

commit a4be5afd03fef3b3fa8ce8c017d24636859c82f3
Author: sboikov 
Date:   2017-11-20T10:27:23Z

zk

commit 97e85179418bc369066c26ec086edd138419c406
Author: sboikov 
Date:   2017-11-20T14:21:33Z

zk

commit fcee8c846274890f8eee8cc8f3644cdda912dedd
Author: sboikov 
Date:   2017-11-21T09:24:58Z

zk

commit fb6bd0ac39f97db0d7e347aff6fa26edda10f940
Author: sboikov 
Date:   2017-11-21T10:43:15Z

zk

commit 1f82a5311d099ad31c22deec391ad91d568f9705
Author: sboikov 
Date:   2017-11-21T12:09:27Z

Merge remote-tracking branch 'remotes/origin/master' into ignite-zk

commit 18527db9ba5d13b0964ec9c87c8b155295921c9a
Author: sboikov 
Date:   2017-11-21T13:54:38Z

zk

commit 7ba0feb55d79794146ea4deb41be2560eec9e42d
Author: sboikov 
Date:   2017-11-21T15:36:29Z

zk

commit 4090eb717f25331c2967645a802714830ac0d4f7
Author: sboikov 
Date:   2017-11-21T15:37:35Z

Merge remote-tracking branch 'origin/ignite-zk' into ignite-zk

commit e0aba812643c0d773359a95b514daead9730ee6e
Author: sboikov 
Date:   2017-11-22T08:47:55Z

zk

commit a6b452823422290e19238238e119f5aaad87b6af
Author: sboikov 
Date:   2017-11-22T10:22:14Z

Merge remote-tracking branch 'remotes/origin/master' into ignite-zk

commit 6cb8c06f73ace3030f47ccfe21e2f46d6b054e5f
Author: sboikov 
Date:   2017-11-22T10:28:51Z

zk

commit 9761a0294e3978120c471dc7bdaa5ad94bbaf1b3
Author: sboikov 
Date:   2017-11-22T10:41:16Z

zk

commit bc297aa4b47cae1e5cb87e5d7863900c82fa90ce
Author: sboikov 
Date:   2017-11-22T10:48:27Z

zk

commit 4749d332c7ec1b6d24f8a8d0f6d5abf50de5d71d
Author: sboikov 
Date:   2017-11-22T11:27:47Z

zk

commit f5f2060aa6978367d4bf160fd96dc4efa57a7a8c
Author: sboikov 
Date:   2017-11-22T13:13:18Z

zk

commit 93dd7ab79c6ed38c29a87ca7a676fbfcefd57273
Author: sboikov 
Date:   2017-11-22T13:22:17Z

zk




---

Re: IGNITE-6827 - Review needed.

[Alexey Goncharuk]

Guys, especially Vladimir Ozerov, Yakov Zhanov, Dmitriy Setrakyan,

While the technical side of the change is being reviewed, I would also ask
you to pay attention to the public API changes here.

--AG

2018-03-21 23:43 GMT+03:00 Pavel Tupitsyn :

> Hi Alexei,
>
> >  Pavel Tupitsyn, could you lend a hand to help completing .NET part (two
> new
> public API methods are added as a part of working on [3], java code in
> branch [2])
>
> Please file a separate ticket for .NET changes.
> I would like to help, but maybe some time later. No need to delay your
> changes because of that.
>
> Thanks,
> Pavel
>
> On Wed, Mar 21, 2018 at 1:08 PM, Dmitry Pavlov 
> wrote:
>
> > Hi Alexei ,
> >
> > Thank you for contribution, unfortunately there is too much tests failed
> > there:
> > .NET is broken
> >
> >   Ignite Platform .NET Long Running [ tests 3 ]
> > CachePartitionedNearEnabledTest.TestTransactionScopeMultiCache(True)
> > CachePartitionedTest.TestTransactionScopeMultiCache(True)
> > CacheReplicatedTest.TestTransactionScopeMultiCache(True)
> >
> >Ignite Platform .NET Core Linux [ tests 2 TC_EXIT_CODE ]
> >  TransactionsParityTest.TestTransactions (fail rate 0,0%)
> >  CachePartitionedTest.TestTransactionScopeMultiCache(True) (fail
> rate
> > 0,0%)
> >
> >Ignite Platform .NET [ tests 1 ] agoncharuk More info >>
> > Test fail rate less than 1%, probably new failure  exe:
> > TransactionsParityTest.TestTransactions (fail rate 0,0%)
> >
> > Too much timeouts:
> >Ignite Cache Failover [2]
> >  with CacheAsyncOperationsFailoverTxTest.testPutAllAsyncFailover (last
> > started)
> >
> >~[Obsolete] Ignite Cache Expiry Policy [ tests 1 ]
> > with  IgniteCacheExpiryPolicyTestSuite:
> > IgniteCacheTtlCleanupSelfTest.testDeferredDeleteTtl
> >
> >  Ignite Cache Restarts [2]
> >   with
> > GridCachePutAllFailoverSelfTest.testPutAllFailoverNearDisabled
> ThreeBackups
> > (last started)
> >
> >   Ignite Cache Restarts [1]
> >with
> > GridCachePartitionedNearDisabledOptimisticTxNodeRestartTest.
> > testRestartWithTxTenNodesTwoBackups
> > (last started)
> >
> >  Cache [6]
> >with  WalModeChangeAdvancedSelfTest.testServerRestartNonCoordinator
> > (last started)
> > it is not all.
> >
> > Please fix tests before review. Also you could change ticket status to
> 'In
> > Progress'.
> >
> > As easy option, you can merge current master to your branch, it is
> probably
> > that some test were fixed already.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > ср, 21 мар. 2018 г. в 9:58, Alexei Scherbakov <
> > alexey.scherbak...@gmail.com
> > >:
> >
> > > Igniters,
> > >
> > > I prepared rather important patch [1] related to grid stability and
> need
> > a
> > > review and some help to finish it.
> > >
> > > Semyon Boikov, could you please take a look on patch code ? [2]
> > >
> > > Pavel Tupitsyn, could you lend a hand to help completing .NET part (two
> > new
> > > public API methods are added as a part of working on [3], java code in
> > > branch [2])
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-6827
> > >
> > > [2] https://github.com/gridgain/apache-ignite/tree/ignite-6827-2
> > >
> > > [3] https://issues.apache.org/jira/browse/IGNITE-7910
> > >
> > > --
> > >
> > > Best regards,
> > > Alexei Scherbakov
> > >
> >
>

[GitHub] ignite pull request #3565: IGNITE-7702: Adopt kNN classifcation to the new d...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3565


---

Re: Upgrade from 2.1.0 to 2.4.0 resulting in error within transaction block

[Yakov Zhdanov]

Cross posting to dev.

Vladimir Ozerov, can you please take a look at NPE from query processor
(see below - GridQueryProcessor.typeByValue(GridQueryProcessor.java:1901))?

--Yakov

2018-03-29 0:19 GMT+03:00 smurphy :

> Code works in Ignite 2.1.0. Upgrading to 2.4.0 produces the stack trace
> below. The delete statement that is causing the error is:
>
> SqlFieldsQuery sqlQuery = new SqlFieldsQuery("delete from EngineFragment
> where " + criteria());
> fragmentCache.query(sqlQuery.setArgs(criteria.getArgs()));
>
> The code above is called from within a transactional block managed by a
> PlatformTransactionManager which is in turn managed by Spring's
> ChainedTransactionManager. If the @Transactional annotation is removed from
> the surrounding code, then the code works ok...
>
> 2018-03-28 15:50:05,748 WARN  [engine 127.0.0.1] progress_monitor_2 unknown
> unknown {ProgressMonitorImpl.java:112} - Scan
> [ec7af5e8-a773-40fd-9722-f81103de73dc] is unable to process!
> javax.cache.CacheException: Failed to process key '247002'
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(
> IgniteCacheProxyImpl.java:618)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(
> IgniteCacheProxyImpl.java:557)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.
> query(GatewayProtectedCacheProxy.java:382)
> at
> com.company.core.dao.ignite.IgniteFragmentDao.delete(
> IgniteFragmentDao.java:143)
> at
> com.company.core.dao.ignite.IgniteFragmentDao$$FastClassBySpringCGLIB$$
> c520aa1b.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(
> MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.
> invokeJoinpoint(CglibAopProxy.java:720)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(
> ReflectiveMethodInvocation.java:157)
> at
> org.springframework.dao.support.PersistenceExceptionTranslatio
> nInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(
> ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.
> intercept(CglibAopProxy.java:655)
> at
> com.company.core.dao.ignite.IgniteFragmentDao$$EnhancerBySpringCGLIB$$
> ce60f71c.delete()
> at
> com.company.core.core.service.impl.InternalScanServiceImpl.
> purgeScanFromGrid(InternalScanServiceImpl.java:455)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> 62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection
> (AopUtils.java:302)
> at
> org.springframework.aop.framework.JdkDynamicAopProxy.
> invoke(JdkDynamicAopProxy.java:202)
> at com.sun.proxy.$Proxy417.purgeScanFromGrid(Unknown Source)
> at com.company.core.core.async.tasks.PurgeTask.process(
> PurgeTask.java:85)
> at sun.reflect.GeneratedMethodAccessor197.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection
> (AopUtils.java:302)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.
> invokeJoinpoint(ReflectiveMethodInvocation.java:190)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(
> ReflectiveMethodInvocation.java:157)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor$1.
> proceedWithInvocation(TransactionInterceptor.java:99)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.
> invokeWithinTransaction(TransactionAspectSupport.java:281)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(
> TransactionInterceptor.java:96)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(
> ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.JdkDynamicAopProxy.
> invoke(JdkDynamicAopProxy.java:208)
> at com.sun.proxy.$Proxy418.process(Unknown Source)
> at
> com.company.core.core.async.impl.ProgressMonitorImpl._
> runTasks(ProgressMonitorImpl.java:128)
> at
> com.company.core.core.async.impl.ProgressMonitorImpl.lambda$null$0(
> ProgressMonitorImpl.java:98)
> at java.util.concurrent.Executors$RunnableAdapter.
> call(Executors.java:511)
> at
> 

[jira] [Created] (IGNITE-8099) Web console: wrong editing behavior in case of incorrect field value

[Pavel Konstantinov (JIRA)]

Pavel Konstantinov created IGNITE-8099:
--

 Summary: Web console: wrong editing behavior in case of incorrect 
field value
 Key: IGNITE-8099
 URL: https://issues.apache.org/jira/browse/IGNITE-8099
 Project: Ignite
  Issue Type: Bug
Reporter: Pavel Konstantinov


For reproduce just try to edit 'Key Type' field of Model



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8098) Getting affinity for topology version earlier than affinity is calculated because of data race

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-8098:
--

 Summary: Getting affinity for topology version earlier than 
affinity is calculated because of data race
 Key: IGNITE-8098
 URL: https://issues.apache.org/jira/browse/IGNITE-8098
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.3
Reporter: Andrey Aleksandrov


>From time to time the Ignite cluster with services throws next exception 
>during restarting of  some nodes:

java.lang.IllegalStateException: Getting affinity for topology version earlier 
than affinity is calculated [locNode=TcpDiscoveryNode 
[id=c770dbcf-2908-442d-8aa0-bf26a2aecfef, addrs=[10.44.162.169, 127.0.0.1], 
sockAddrs=[clrv041279.ic.ing.net/10.44.162.169:56500, /127.0.0.1:56500], 
discPort=56500, order=11, intOrder=8, lastExchangeTime=1520931375337, loc=true, 
ver=2.3.3#20180213-sha1:f446df34, isClient=false], grp=ignite-sys-cache, 
topVer=AffinityTopologyVersion [topVer=13, minorTopVer=0], 
head=AffinityTopologyVersion [topVer=15, minorTopVer=0], 
history=[AffinityTopologyVersion [topVer=11, minorTopVer=0], 
AffinityTopologyVersion [topVer=11, minorTopVer=1], AffinityTopologyVersion 
[topVer=12, minorTopVer=0], AffinityTopologyVersion [topVer=15, minorTopVer=0]]]

Looks like the reason of this issue is the data race in GridServiceProcessor 
class.


How to reproduce:

1)To simulate data race you should update next place in source code:



Class: GridServiceProcessor
Method: @Override public void onEvent(final DiscoveryEvent evt, final 
DiscoCache discoCache) {
Place:



try {
 svcName.set(dep.configuration().getName());

 ctx.cache().internalCache(UTILITY_CACHE_NAME).context().affinity().
 affinityReadyFuture(topVer).get();

//HERE (between GET and REASSIGN) you should add Thread.sleep(100) for example.

//try {
//Thread.sleep(100);
//}
//catch (InterruptedException e1) {
//e1.printStackTrace();
//}
 
 reassign(dep, topVer);
}
catch (IgniteCheckedException ex) {
 if (!(e instanceof ClusterTopologyCheckedException))
 LT.error(log, ex, "Failed to do service reassignment (will retry): " +
 dep.configuration().getName());

 retries.add(dep);
}

...

2)After that you should imitate start/shutdown iterations. For reproducing I 
used GridServiceProcessorBatchDeploySelfTest (but timeout on future.get should 
be increased to avoid timeout error)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-8097) Java thin client: throw handshake exception on connect phase

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-8097:
---

 Summary: Java thin client: throw handshake exception on connect 
phase
 Key: IGNITE-8097
 URL: https://issues.apache.org/jira/browse/IGNITE-8097
 Project: Ignite
  Issue Type: Task
  Components: thin client
Reporter: Vladimir Ozerov
 Fix For: 2.5


Currently a call to {{Ignition.startClient}} return client instance even if we 
know for sure that connection is not usable. Real exception (e.g. protocol 
mismatch, auth error, etc.) is thrown on attempt to execute first operation on 
the client. This is bad UX - use may think that everything is OK for a long 
time.

Instead, connection should be established eagerly in {{startClient}}, any 
exception should be propagated to the user immediately.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3720: IGNITE-8076 Java Thin Client Authentication

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3720


---

[GitHub] ignite pull request #3685: IGNITE-8019 : Optimized checkpointReadLock acquis...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3685


---

[GitHub] ignite pull request #3729: IGNITE-8069 IgniteOutOfMemoryException should be ...

[alex-plekhanov]

GitHub user alex-plekhanov opened a pull request:

https://github.com/apache/ignite/pull/3729

IGNITE-8069 IgniteOutOfMemoryException should be handled accordingly …

…to provided failure handler

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/alex-plekhanov/ignite ignite-8069

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3729.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3729


commit fddeea5c509939e29dc126197e29cdcbc14bc33d
Author: Aleksey Plekhanov 
Date:   2018-04-02T07:35:25Z

IGNITE-8069 IgniteOutOfMemoryException should be handled accordingly to 
provided failure handler




---

[jira] [Created] (IGNITE-8096) Web console: add ability to clone cluster configuration entities

[Ilya Borisov (JIRA)]

Ilya Borisov created IGNITE-8096:


 Summary: Web console: add ability to clone cluster configuration 
entities
 Key: IGNITE-8096
 URL: https://issues.apache.org/jira/browse/IGNITE-8096
 Project: Ignite
  Issue Type: New Feature
  Components: wizards
Reporter: Ilya Borisov
Assignee: Ilya Borisov


New configuration introduced by IGNITE-5466 lacks a way to clone cluster 
configuration child entities (caches, igfss and domain models).

*What to do:*
Implement cluster config entity cloning mechanism. It might be a single (or 
multiple) items table context action, which creates new entities under the hood 
using a default unique name generator function, sends one or multiple requests 
to the backend and shows an error notification if something goes wrong.

*Potential issues:*
1. Nested reference IDs and what to do with them.
2. If user decides to clone current edited item, can unsaved changes trigger 
any unwanted behavior?
3. One vs multiple items to clone at the same time.

*Additional ideas:*
1. Clone the item specified amount of times. Useful when user needs 10 caches 
with similar configuration.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)