Re: BinaryObjectBuilder: policy of reusing

[Dmitriy Setrakyan]

Sergey, why not prohibit any reuse of the builder and throw exception right
away?

On Wed, Jul 26, 2017 at 6:20 AM, Sergey Chugunov 
wrote:

> Hello folks,
>
> Recently I filed a ticket [1] with very simple test where correct usage of
> BinaryObjectBuilder is vague.
>
> The issue boils down to the fact that it is unclear from documentation and
> behavior whether reusing of BinaryObjectBuilder is allowed.
>
> Let's discuss here what is the correct behavior and how to fix the issue.
>
> For sure documentation on BinaryObjectBuilder must be updated as well.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-5839
>

Re: BinaryObjectBuilder: policy of reusing

[Pavel Tupitsyn]

BinaryObjectBuilder reuse should be allowed, performance is an important
point of this mechanism.
Creating a new builder just to modify a single field is wasteful.

Well-known builders, such as StringBuilder, are reusable.

Pavel

On Thu, Jul 27, 2017 at 10:51 AM, Dmitriy Setrakyan 
wrote:

> Sergey, why not prohibit any reuse of the builder and throw exception right
> away?
>
> On Wed, Jul 26, 2017 at 6:20 AM, Sergey Chugunov <
> sergey.chugu...@gmail.com>
> wrote:
>
> > Hello folks,
> >
> > Recently I filed a ticket [1] with very simple test where correct usage
> of
> > BinaryObjectBuilder is vague.
> >
> > The issue boils down to the fact that it is unclear from documentation
> and
> > behavior whether reusing of BinaryObjectBuilder is allowed.
> >
> > Let's discuss here what is the correct behavior and how to fix the issue.
> >
> > For sure documentation on BinaryObjectBuilder must be updated as well.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-5839
> >
>

Re: BinaryObjectBuilder: policy of reusing

[Dmitriy Setrakyan]

On Thu, Jul 27, 2017 at 3:14 AM, Pavel Tupitsyn 
wrote:

> BinaryObjectBuilder reuse should be allowed, performance is an important
> point of this mechanism.
> Creating a new builder just to modify a single field is wasteful.
>
> Well-known builders, such as StringBuilder, are reusable.
>

In this case, what is preventing the binary builder from being reusable?

The ticket filed by Sergey states that there is a bug in the builder, but
also suggests that we should create a new builder all the time. It sounds
that it is not a good approach then.


>
> Pavel
>
> On Thu, Jul 27, 2017 at 10:51 AM, Dmitriy Setrakyan  >
> wrote:
>
> > Sergey, why not prohibit any reuse of the builder and throw exception
> right
> > away?
> >
> > On Wed, Jul 26, 2017 at 6:20 AM, Sergey Chugunov <
> > sergey.chugu...@gmail.com>
> > wrote:
> >
> > > Hello folks,
> > >
> > > Recently I filed a ticket [1] with very simple test where correct usage
> > of
> > > BinaryObjectBuilder is vague.
> > >
> > > The issue boils down to the fact that it is unclear from documentation
> > and
> > > behavior whether reusing of BinaryObjectBuilder is allowed.
> > >
> > > Let's discuss here what is the correct behavior and how to fix the
> issue.
> > >
> > > For sure documentation on BinaryObjectBuilder must be updated as well.
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-5839
> > >
> >
>

[jira] [Created] (IGNITE-5849) Introduce ignite node persistent meta-store

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-5849:


 Summary: Introduce ignite node persistent meta-store
 Key: IGNITE-5849
 URL: https://issues.apache.org/jira/browse/IGNITE-5849
 Project: Ignite
  Issue Type: New Feature
  Components: persistence
Affects Versions: 2.1
Reporter: Alexey Goncharuk
 Fix For: 2.2


As persistence feature is being developed, we will have a need for a component, 
which reliably stores arbitrary metadata about node and cluster.

We already have reserved partition IDs for this purpose, all we need to do is 
to extend the partition store abstraction to store non-cache objects and make 
sure this new store participates in all common recovery procedures.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-5850) Introduce an API for baseline topology for persistence-enabled clusters

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-5850:


 Summary: Introduce an API for baseline topology for 
persistence-enabled clusters
 Key: IGNITE-5850
 URL: https://issues.apache.org/jira/browse/IGNITE-5850
 Project: Ignite
  Issue Type: New Feature
  Components: persistence
Affects Versions: 2.1
Reporter: Alexey Goncharuk
 Fix For: 2.2


Since the persistence was introduced, we require that cluster be started in an 
inactive mode and activation happens only manually.

We need to add a concept of baseline topology which is fixed on the first 
cluster activation and may be changed later by a user. We need to develop a 
consistent API facade for this purpose.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-5851) Automatically activate cluster if baseline topology is reached

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-5851:


 Summary: Automatically activate cluster if baseline topology is 
reached
 Key: IGNITE-5851
 URL: https://issues.apache.org/jira/browse/IGNITE-5851
 Project: Ignite
  Issue Type: New Feature
  Components: persistence
Affects Versions: 2.1
Reporter: Alexey Goncharuk
 Fix For: 2.2


When we have an API for baseline topology management, we can automatically 
activate the cluster once all the nodes from the baseline topology are started.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-5852) Cache affinity should be calculated wrt baseline topology

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-5852:


 Summary: Cache affinity should be calculated wrt baseline topology
 Key: IGNITE-5852
 URL: https://issues.apache.org/jira/browse/IGNITE-5852
 Project: Ignite
  Issue Type: New Feature
  Components: persistence
Affects Versions: 2.1
Reporter: Alexey Goncharuk
 Fix For: 2.2


After we have an API for baseline topology management, we need to provide a way 
to operate when real topology differs from the baseline. To facilitate this, we 
need to calculate the affinity on the set of nodes which is an intersection of 
the baseline topology and current topology



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-5853) Provide an interface to mark user attributes used in affinity calculation

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-5853:


 Summary: Provide an interface to mark user attributes used in 
affinity calculation
 Key: IGNITE-5853
 URL: https://issues.apache.org/jira/browse/IGNITE-5853
 Project: Ignite
  Issue Type: New Feature
  Components: persistence
Affects Versions: 2.1
Reporter: Alexey Goncharuk
 Fix For: 2.2


Since an affinity function may use user attributes to calculate affinity 
distribution, we need to save these attributes to the metastore. However, 
storing all the attributes is not very effective, so we need to have a way to 
determine which attributes should be stored.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-5854) Validate baseline topology change history during node joins

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-5854:


 Summary: Validate baseline topology change history during node 
joins
 Key: IGNITE-5854
 URL: https://issues.apache.org/jira/browse/IGNITE-5854
 Project: Ignite
  Issue Type: New Feature
  Components: persistence
Affects Versions: 2.1
Reporter: Alexey Goncharuk
 Fix For: 2.2


It is possible to have a cluster-split restarts when persistence enabled:
Start 1, 2, 3, 4, update data. Stop
Start 1, 2. Update data. Stop
Start 3, 4. Update data. Stop.
Start 1, 2, 3, 4. Stop

We need to make sure that the second start is not allowed. This can be done by 
introducing baseline topology history and topology hash generation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-5855) SQL: BigInteger is supported in SQL queries.

[Andrew Mashenkov (JIRA)]

Andrew Mashenkov created IGNITE-5855:


 Summary: SQL: BigInteger is supported in SQL queries.
 Key: IGNITE-5855
 URL: https://issues.apache.org/jira/browse/IGNITE-5855
 Project: Ignite
  Issue Type: Bug
  Components: sql
Reporter: Andrew Mashenkov
 Fix For: 2.2


Looks like BigInteger is not supported in SQL queries at all.
PFA reproducer.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2351: 8.0.3.ea13

[agoncharuk]

GitHub user agoncharuk opened a pull request:

https://github.com/apache/ignite/pull/2351

8.0.3.ea13



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-8.0.3.ea13

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2351.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2351


commit 52556f4bf6d544a44bfd49b02d84aa32f741813f
Author: Ilya Lantukh 
Date:   2017-04-28T16:40:31Z

Multiple optimizations from ignite-gg-8.0.3.ea5-atomicbench.

commit 58b6e05e82978c68ba9e2e53a3b9b866e2c474ca
Author: Ilya Lantukh 
Date:   2017-04-28T16:57:32Z

Optimizations from ignite-5068.

commit 925370c3c6c587870e19f31f8660139884847e06
Author: sboikov 
Date:   2017-05-06T09:15:21Z

client join race

(cherry picked from commit 682ca8e)

commit 643e6d49d0e0fb1d66b18dcfe74b570c15924036
Author: Alexey Goncharuk 
Date:   2017-05-10T09:14:42Z

GG-12170 - Added assertions for tag value, skip failed-to-write-lock-pages 
on checkpoint begin

commit f00785611e1d785fe8c2247770bf07e07ddb1ebe
Author: Ivan Rakov 
Date:   2017-05-12T16:39:52Z

GG-12194: Backport GG-12184 into 8.0.3.ea8

commit b39f7c8d4f486f25e48a812e4a9d5aa4bbb5932f
Author: Alexey Kuznetsov 
Date:   2017-05-13T15:37:25Z

GG-12190 Backported to 803ea8.

commit 8f1398f43038700cdf66a4538d24c4dfd227cb0e
Author: Alexey Goncharuk 
Date:   2017-05-15T09:47:38Z

Additional debug

commit c35dbf4ec3ba6b01932c76f14ad4c45fed402391
Author: Yakov Zhdanov 
Date:   2017-05-15T15:03:07Z

Added IO latency test + made it available from MBean

(cherry picked from commit 8195ae0)

commit b1116069549be224d59983b93d2ee22cab8402b8
Author: sboikov 
Date:   2017-05-16T08:24:11Z

Improved exchange timeout debug logging.

commit 560ef60bf90643dfc4a329a760714d652c73e9a8
Author: sboikov 
Date:   2017-05-16T08:30:29Z

DirectByteBufferStreamImpl: converted asserts into exceptions.

commit 250b4e04606d95821ee9e87c40225c45f3432d3c
Author: Yakov Zhdanov 
Date:   2017-05-17T13:36:27Z

Results printout for IO latency test

(cherry picked from commit a0fc6ee)

commit 46cba2a46966759e6d658f3ba991f15722a6634f
Author: Alexey Goncharuk 
Date:   2017-05-17T16:04:10Z

Do not re-create node2part map on every singleMap message

commit d4c999795917b7695cc13f2fea3e69cd1a3d5078
Author: Alexey Goncharuk 
Date:   2017-05-17T17:58:21Z

MetaPageInitRecord fix

commit 7b545fa9029ba9f3d90828cd38611f6a2988cb25
Author: EdShangGG 
Date:   2017-05-16T16:07:03Z

GG-12140 We will lose data if we cancel snapshot restore
(cherry picked from commit 8e3ad6d)

commit e590a8110b1ce7f8140f06ae4a504f60777847e6
Author: Eduard Shangareev 
Date:   2017-05-17T23:12:44Z

GG-12140 We will lose data if we cancel snapshot restore
(cherry picked from commit 7721838)

commit db84a921920ff6a4ebe55af4cf8047e37e0addb1
Author: EdShangGG 
Date:   2017-05-18T14:37:30Z

fixing compilation after cherry-picking

commit afbade50e151d2aa793e9762637853ec6d6f2f93
Author: EdShangGG 
Date:   2017-05-18T16:30:00Z

GG-12192 Concurrent node join and snapshot restore cause to coordinator fail
-fixing tests and compilation

commit b29b918aece6a99a12c8bf3f21c5419a9d97de25
Author: Alexey Kuznetsov 
Date:   2017-05-19T07:18:23Z

Added Affinity topology version and Pending exchanges to Visor data 
collector task.
(cherry picked from commit 7402ea1)

commit 096404d36c1cc3a1d9da1db3a2b0a2b7fcdd702d
Author: Yakov Zhdanov 
Date:   2017-05-19T17:33:36Z

Results printout for IO latency test

commit 33f1c3376a05e728a63df5cf8802d5df6f9e02f5
Author: Alexey Goncharuk 
Date:   2017-05-22T16:46:39Z

Fixed assertion in checkpointer on cache stop

commit e7edb38fdd42008fd358e320dfe003a47b22cbe6
Author: EdShangGG 
Date:   2017-05-23T12:31:55Z

GG-12192 Concurrent node join and snapshot restore cause to coordinator fail
-adding test
-fixing problem with coordinator left

commit 34a0d5f5f7c97c4f401c14b4211af35eaa34e850
Author: Vladislav Pyatkov 
Date:   2017-05-23T12:33:39Z

ignite-5212 Allow custom affinity function for data structures cache
(cherry picked from commit f353faf)

commit f169a1898aacdd08584210fe3377767d12be563d
Author: Yakov Zhdanov 
Date:   2017-05-23T14:39:37Z

Results printout for IO latency test and new metrics

commit 767fe9abd578302159caaa59502f5ffb6f53336c
Author: Ilya Lantukh 
Date:   2017-05-23T15:56:07Z

Renting primary node - fix.

commit 2256fe195da7f82006bc68de1d38cd5e2c4bbbe2
Author: Ilya Lantukh 
Date:   2017-05-23T15:58:25Z

Merge branch 'ignite-gg-8.0.3.ea9' of 
https://github.com/gridgain/apache-ignite into ignite-gg-8.0.3.ea9

commit 6cf5d59c07a86c9bb0c78d2447970ba11977621d
Author: EdShangGG 
Date:   2017-05-23T15:02:29Z

GG-12210 Don't allow joi

[GitHub] ignite pull request #2352: IGNITE-5769 Abstract away .NET->Java calls

[ptupitsyn]

GitHub user ptupitsyn opened a pull request:

https://github.com/apache/ignite/pull/2352

IGNITE-5769 Abstract away .NET->Java calls



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ptupitsyn/ignite ignite-5769

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2352.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2352


commit 6eaa88049b951383b10084aa4e86318250c2d2ef
Author: Pavel Tupitsyn 
Date:   2017-07-25T14:36:44Z

IGNITE-5769 Abstract away .NET->Java calls

commit 151007ae869f07ee453a6c7cc534f230b6de55c4
Author: Pavel Tupitsyn 
Date:   2017-07-25T14:48:46Z

Refactor writer extensions

commit 55f3df9573c12849eecaa91fdbd556a138d80239
Author: Pavel Tupitsyn 
Date:   2017-07-25T15:25:36Z

wip

commit b809bd4fee69abd2c9110751942175a63075b6eb
Author: Pavel Tupitsyn 
Date:   2017-07-25T15:31:51Z

wip

commit 5738df5808dd3e4b479d1704322bd7d3c66bf862
Author: Pavel Tupitsyn 
Date:   2017-07-25T15:33:30Z

wip

commit 45e2330219cff3f9aa850e9e4dcbf27de39e0ab6
Author: Pavel Tupitsyn 
Date:   2017-07-26T08:59:24Z

Merge branch 'master' into ignite-5769

commit 699ee441b071a527937aef4d1d717baddb25b41e
Author: Pavel Tupitsyn 
Date:   2017-07-26T10:10:33Z

inStreamOutObjectAsync

commit c17daafc599d63005b05c01991a14cd2da5e3966
Author: Pavel Tupitsyn 
Date:   2017-07-26T10:20:09Z

wip

commit 8b88d3982a413e5d59ac922ec763124c5142f282
Author: Pavel Tupitsyn 
Date:   2017-07-26T10:32:02Z

wip

commit bfaa5b695af9a2c7f235103395a84585d3ed0900
Author: Pavel Tupitsyn 
Date:   2017-07-26T10:52:43Z

wip async cancellation

commit 51b853ad64286937e5cca1863b7ebe4a62f7ddf6
Author: Pavel Tupitsyn 
Date:   2017-07-26T10:54:30Z

cancellation test

commit 452f4aed44b485dacb0f8b9f11fecd85d824b4b4
Author: Pavel Tupitsyn 
Date:   2017-07-26T10:55:38Z

fix exports

commit df9afba880035b68e00f40ad999781f1aec9f8e1
Author: Pavel Tupitsyn 
Date:   2017-07-26T11:13:59Z

fix listenable proxy

commit 35c0fc63c3013fd09b64c00e242a11968c039d23
Author: Pavel Tupitsyn 
Date:   2017-07-26T11:25:43Z

Cancellation test added

commit c6f1e022294706d1f2e2024be8e207751fc27f0b
Author: Pavel Tupitsyn 
Date:   2017-07-26T11:28:40Z

wip

commit 1ba3bab7e4bed0de8abf7bb62454d074032e4bf8
Author: Pavel Tupitsyn 
Date:   2017-07-26T12:25:30Z

Add PlatformJniTarget

commit cb4412d7cf7640d7923b7e03cc14c18545224629
Author: Pavel Tupitsyn 
Date:   2017-07-26T12:26:55Z

wip

commit e990d2bc9eed844a552d418606f41143f998917e
Author: Pavel Tupitsyn 
Date:   2017-07-26T14:54:09Z

wip

commit ecff85fcfc2e5a6286ce4a5c30e292b00e14affe
Author: Pavel Tupitsyn 
Date:   2017-07-26T15:13:41Z

wip

commit 1b5e41c64861f5e3187a852d12516701bfeed1bd
Author: Pavel Tupitsyn 
Date:   2017-07-26T15:38:42Z

wip

commit 6cb0f28dd06933c04253f5952470ae65e641ad42
Author: Pavel Tupitsyn 
Date:   2017-07-26T15:42:21Z

wip

commit 8293570116b279db6b6c248b83d2bc5deab5ce09
Author: Pavel Tupitsyn 
Date:   2017-07-26T15:43:04Z

wip

commit 6e19de4f8a940617a1d5d96bcee4f13473b1df1a
Author: Pavel Tupitsyn 
Date:   2017-07-26T15:47:14Z

wip

commit 37dd6b3f2e0bbc8ec75fb2960cd400740f0b1b65
Author: Pavel Tupitsyn 
Date:   2017-07-27T09:38:46Z

wip refactoring

commit b61f1891bca9055bc85e719db89aecc16d0f792f
Author: Pavel Tupitsyn 
Date:   2017-07-27T09:40:36Z

wip

commit a2a949e3c8383d648fb4450bd06c26ef1f8ec0ad
Author: Pavel Tupitsyn 
Date:   2017-07-27T09:48:27Z

wip

commit a4567adc31f1de543c1dde1d4064fd9ba41a26f3
Author: Pavel Tupitsyn 
Date:   2017-07-27T09:53:54Z

wip refactoring

commit 3f7e2ed2f976851580d2f5dc88c0b96e7728b519
Author: Pavel Tupitsyn 
Date:   2017-07-27T10:06:43Z

wip

commit 58a66d916f96383d362bbc74ae27a7d5b45fac76
Author: Pavel Tupitsyn 
Date:   2017-07-27T10:11:15Z

Compilation fixed! Yay!




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-5856) BLAS integration phase 1

[Yury Babak (JIRA)]

Yury Babak created IGNITE-5856:
--

 Summary: BLAS integration phase 1
 Key: IGNITE-5856
 URL: https://issues.apache.org/jira/browse/IGNITE-5856
 Project: Ignite
  Issue Type: Sub-task
  Components: ml
Reporter: Yury Babak


(i) BLAS multiplication for dense and sparse local matrices.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[RESULT] [VOTE] Apache Ignite 2.1.0 Release (RC4)

[Anton Vinogradov]

Igniters,

Apache Ignite 2.1.0 release (RC4) has been accepted.

7 "+1" votes received:

- Valentin Kulichenko (binding)
- Nikolai Tikhonov (binding)
- Alexey Kuznetsov (binding)
- Yakov Zhdanov (binding)
- Konstantin Boudnik (binding)
- Andrey Gura (binding)
- Denis Magda (binding)

Vote thread:

http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Apache-Ignite-2-1-0-RC4-td19969.html

Ignite 2.1.0 will be released soon. Woohoo!

Re: сhecksum algorythm

[Anton Vinogradov]

Oleg,

Let's check what's used at another popular Apache Projects.

On Wed, Jul 26, 2017 at 11:10 PM, Dmitry Pavlov 
wrote:

> Hi Oleg,
>
> Both MD5 and SHA1 are deprecated and can't be considered as trustfull.
>
> I think at-least-256 bit member of the SHA-2 family (e. g. sha512) should
> be used.
>
> Sincerely,
> Dmitriy Pavlov
>
> ср, 26 июл. 2017 г. в 22:27, Oleg Ostanin :
>
> > Hi,
> >
> > We need to decide what сhecksum algorythm we should use for signing
> release
> > artifacts. Currently we use md5 and sha-1. sha-1 will be replaced by
> > sha-256 soon. Should we keep md5 or use only sha-256?
> >
>

Re: Non-UTF-8 string encoding support in BinaryMarshaller (IGNITE-5655)

[Igor Sapego]

Just a note from the platforms guy:

Solution with table-level configuration is going to be significantly
harder to implement for platforms and ODBC then field-level one.

Also, what about binary objects, which are not stored in cache,
but being marshalled?


Best Regards,
Igor

On Wed, Jul 26, 2017 at 7:22 PM, Dmitriy Setrakyan 
wrote:

> On Wed, Jul 26, 2017 at 3:40 AM, Vyacheslav Daradur 
> wrote:
>
> >
> > > Encoding must be set on per field basis. This will give us as most
> > flexible
> > > solution at the cost of 1-byte overhead.
> >
> > > Vova, I agree that the encoding should be set on per-field basis, but
> at
> > > the table level, not at a cell level.
> >
> > Dmitriy, Vladimir,
> > Let's use both approaches :-)
> > We can add parameter to CacheConfiguration.
> > If parameter specifie to use cache level encoding then marshaller will
> use
> > encoding in a cache,
> > otherwise marshaller will use per-field encoding.
> > Of course only if it doesn't complicate the solution.
> >
> >
> I think that it will complicate the solution and will complicate the
> marshalling protocol. The advantage of specifying the encoding at
> table/cache level is that we don't need to add extra encoding bytes to the
> marshalling protocol.
>
> I think Vova was suggesting encoding at the cell level, not at the field
> level, which seems to be redundant to me.
>
> Vova, do you agree?
>

[jira] [Created] (IGNITE-5857) Implement cache group destroy

[Semen Boikov (JIRA)]

Semen Boikov created IGNITE-5857:


 Summary: Implement cache group destroy
 Key: IGNITE-5857
 URL: https://issues.apache.org/jira/browse/IGNITE-5857
 Project: Ignite
  Issue Type: Improvement
  Components: cache
Reporter: Semen Boikov
 Fix For: 2.2


It can be useful to have possibility to destroy cache group (this will destroy 
all caches in this group).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Non-UTF-8 string encoding support in BinaryMarshaller (IGNITE-5655)

[Dmitriy Setrakyan]

On Thu, Jul 27, 2017 at 9:04 AM, Igor Sapego  wrote:

> Just a note from the platforms guy:
>
> Solution with table-level configuration is going to be significantly
> harder to implement for platforms and ODBC then field-level one.
>

Igor, it seems like you are advocating the per-cell configuration, not
per-field one. The per-field configuration can be defined at the
table/cache level.

I see your point about C++ and .NET integrations however. Can't we provide
this info at node-join time or table-creation time? This way all nodes will
receive it and you will be able to grab it on different platforms.


>
> Also, what about binary objects, which are not stored in cache,
> but being marshalled?
>

I think the default system encoding should be used here. If we don't have
configuration for default encoding, we should add it.


>
>
> Best Regards,
> Igor
>
> On Wed, Jul 26, 2017 at 7:22 PM, Dmitriy Setrakyan 
> wrote:
>
> > On Wed, Jul 26, 2017 at 3:40 AM, Vyacheslav Daradur  >
> > wrote:
> >
> > >
> > > > Encoding must be set on per field basis. This will give us as most
> > > flexible
> > > > solution at the cost of 1-byte overhead.
> > >
> > > > Vova, I agree that the encoding should be set on per-field basis, but
> > at
> > > > the table level, not at a cell level.
> > >
> > > Dmitriy, Vladimir,
> > > Let's use both approaches :-)
> > > We can add parameter to CacheConfiguration.
> > > If parameter specifie to use cache level encoding then marshaller will
> > use
> > > encoding in a cache,
> > > otherwise marshaller will use per-field encoding.
> > > Of course only if it doesn't complicate the solution.
> > >
> > >
> > I think that it will complicate the solution and will complicate the
> > marshalling protocol. The advantage of specifying the encoding at
> > table/cache level is that we don't need to add extra encoding bytes to
> the
> > marshalling protocol.
> >
> > I think Vova was suggesting encoding at the cell level, not at the field
> > level, which seems to be redundant to me.
> >
> > Vova, do you agree?
> >
>

[jira] [Created] (IGNITE-5858) Ignite Cache 5 Timeout:

[Dmitriy Pavlov (JIRA)]

Dmitriy Pavlov created IGNITE-5858:
--

 Summary: Ignite Cache 5 Timeout: 
 Key: IGNITE-5858
 URL: https://issues.apache.org/jira/browse/IGNITE-5858
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.2
Reporter: Dmitriy Pavlov
Priority: Blocker
 Fix For: 2.2


Reproduced for master:
{noformat}
Invalid affinity version [last=AffinityTopologyVersion [topVer=-1, 
minorTopVer=0], 
{noformat}

Failed tests
CacheLateAffinityAssignmentTest.testRandomOperations
CacheLateAffinityAssignmentTest.testAffinitySimpleNoCacheOnCoordinator1


Failed builds
http://ci.ignite.apache.org/viewType.html?buildTypeId=Ignite20Tests_IgniteCache5&tab=buildTypeHistoryList&branch_Ignite20Tests=%3Cdefault%3E

Following builds were failed
742325,742326,742327,742328,742329,742330,742331,742332,742333,742334,742335,742336,742337,742338,742339,742340,742341,742342,742343,742344



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Non-UTF-8 string encoding support in BinaryMarshaller (IGNITE-5655)

[Igor Sapego]

> Igor, it seems like you are advocating the per-cell configuration, not
> per-field one.

True, some terms mismatch here.

> I see your point about C++ and .NET integrations however. Can't we provide
> this info at node-join time or table-creation time? This way all nodes
will
> receive it and you will be able to grab it on different platforms.

This issue can be solved in different ways, I just say that it will be
significantly
more complicated. Just something we may want to consider when we choose
a solution here.

Best Regards,
Igor

On Thu, Jul 27, 2017 at 5:10 PM, Dmitriy Setrakyan 
wrote:

> On Thu, Jul 27, 2017 at 9:04 AM, Igor Sapego  wrote:
>
> > Just a note from the platforms guy:
> >
> > Solution with table-level configuration is going to be significantly
> > harder to implement for platforms and ODBC then field-level one.
> >
>
> Igor, it seems like you are advocating the per-cell configuration, not
> per-field one. The per-field configuration can be defined at the
> table/cache level.
>
> I see your point about C++ and .NET integrations however. Can't we provide
> this info at node-join time or table-creation time? This way all nodes will
> receive it and you will be able to grab it on different platforms.
>
>
> >
> > Also, what about binary objects, which are not stored in cache,
> > but being marshalled?
> >
>
> I think the default system encoding should be used here. If we don't have
> configuration for default encoding, we should add it.
>
>
> >
> >
> > Best Regards,
> > Igor
> >
> > On Wed, Jul 26, 2017 at 7:22 PM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> > wrote:
> >
> > > On Wed, Jul 26, 2017 at 3:40 AM, Vyacheslav Daradur <
> daradu...@gmail.com
> > >
> > > wrote:
> > >
> > > >
> > > > > Encoding must be set on per field basis. This will give us as most
> > > > flexible
> > > > > solution at the cost of 1-byte overhead.
> > > >
> > > > > Vova, I agree that the encoding should be set on per-field basis,
> but
> > > at
> > > > > the table level, not at a cell level.
> > > >
> > > > Dmitriy, Vladimir,
> > > > Let's use both approaches :-)
> > > > We can add parameter to CacheConfiguration.
> > > > If parameter specifie to use cache level encoding then marshaller
> will
> > > use
> > > > encoding in a cache,
> > > > otherwise marshaller will use per-field encoding.
> > > > Of course only if it doesn't complicate the solution.
> > > >
> > > >
> > > I think that it will complicate the solution and will complicate the
> > > marshalling protocol. The advantage of specifying the encoding at
> > > table/cache level is that we don't need to add extra encoding bytes to
> > the
> > > marshalling protocol.
> > >
> > > I think Vova was suggesting encoding at the cell level, not at the
> field
> > > level, which seems to be redundant to me.
> > >
> > > Vova, do you agree?
> > >
> >
>

Re: Non-UTF-8 string encoding support in BinaryMarshaller (IGNITE-5655)

[Pavel Tupitsyn]

I'm not sure I uderstand how this "per field" configuration is supposed to
be implemented.
* Marshaller is not tied to a cache. It serializes all kinds of things,
like compute job parameters and results.
* Raw mode does not involve field names.

Also it seems like a complicated and expensive solution - looking up string
format somewhere in the metadata will be slow.

"encoded string" data type suggestion from Vladimir looks better to me from
performance and implementation standpoint.

Thanks,
Pavel



On Thu, Jul 27, 2017 at 5:10 PM, Dmitriy Setrakyan 
wrote:

> On Thu, Jul 27, 2017 at 9:04 AM, Igor Sapego  wrote:
>
> > Just a note from the platforms guy:
> >
> > Solution with table-level configuration is going to be significantly
> > harder to implement for platforms and ODBC then field-level one.
> >
>
> Igor, it seems like you are advocating the per-cell configuration, not
> per-field one. The per-field configuration can be defined at the
> table/cache level.
>
> I see your point about C++ and .NET integrations however. Can't we provide
> this info at node-join time or table-creation time? This way all nodes will
> receive it and you will be able to grab it on different platforms.
>
>
> >
> > Also, what about binary objects, which are not stored in cache,
> > but being marshalled?
> >
>
> I think the default system encoding should be used here. If we don't have
> configuration for default encoding, we should add it.
>
>
> >
> >
> > Best Regards,
> > Igor
> >
> > On Wed, Jul 26, 2017 at 7:22 PM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> > wrote:
> >
> > > On Wed, Jul 26, 2017 at 3:40 AM, Vyacheslav Daradur <
> daradu...@gmail.com
> > >
> > > wrote:
> > >
> > > >
> > > > > Encoding must be set on per field basis. This will give us as most
> > > > flexible
> > > > > solution at the cost of 1-byte overhead.
> > > >
> > > > > Vova, I agree that the encoding should be set on per-field basis,
> but
> > > at
> > > > > the table level, not at a cell level.
> > > >
> > > > Dmitriy, Vladimir,
> > > > Let's use both approaches :-)
> > > > We can add parameter to CacheConfiguration.
> > > > If parameter specifie to use cache level encoding then marshaller
> will
> > > use
> > > > encoding in a cache,
> > > > otherwise marshaller will use per-field encoding.
> > > > Of course only if it doesn't complicate the solution.
> > > >
> > > >
> > > I think that it will complicate the solution and will complicate the
> > > marshalling protocol. The advantage of specifying the encoding at
> > > table/cache level is that we don't need to add extra encoding bytes to
> > the
> > > marshalling protocol.
> > >
> > > I think Vova was suggesting encoding at the cell level, not at the
> field
> > > level, which seems to be redundant to me.
> > >
> > > Vova, do you agree?
> > >
> >
>

Bug in method

[Vitaliy Biryukov]

Hi, igniters!

I noticed that the method "IgniteUtils.ceilPow2" can overflow for values
greater than 2^30 and return Integer.MIN_VALUE for them.

Maybe this check was skipped  for method speed. In this case, we need to
add information about this into javaDoc.

Current implementation:

public static int ceilPow2(int v) {
return Integer.highestOneBit(v - 1) << 1;
}


It can be fixed like this:
bit impl:

int i = v - 1;

return Integer.highestOneBit(i) << 1 - (i >>> 30 ^ v >> 31);

more readable impl:

if (v == Integer.MIN_VALUE)
return 0;

int pow30 = 1 << 30;

return v >= pow30 ? pow30 : Integer.highestOneBit(v - 1) << 1;


What do you think? Sould I create  ticket with PR?

-- 
Best Regards,
Vitaliy Biryukov

Re: Non-UTF-8 string encoding support in BinaryMarshaller (IGNITE-5655)

[Dmitriy Setrakyan]

Pavel, what would be the size overhead? Are we adding 1 byte for every
field just for this? If you would like to have this info in the binary
object directly, can we in this case have some bitmap of field-to-encoding?

D.

On Thu, Jul 27, 2017 at 9:22 AM, Pavel Tupitsyn 
wrote:

> I'm not sure I uderstand how this "per field" configuration is supposed to
> be implemented.
> * Marshaller is not tied to a cache. It serializes all kinds of things,
> like compute job parameters and results.
> * Raw mode does not involve field names.
>
> Also it seems like a complicated and expensive solution - looking up string
> format somewhere in the metadata will be slow.
>
> "encoded string" data type suggestion from Vladimir looks better to me from
> performance and implementation standpoint.
>
> Thanks,
> Pavel
>
>
>
> On Thu, Jul 27, 2017 at 5:10 PM, Dmitriy Setrakyan 
> wrote:
>
> > On Thu, Jul 27, 2017 at 9:04 AM, Igor Sapego  wrote:
> >
> > > Just a note from the platforms guy:
> > >
> > > Solution with table-level configuration is going to be significantly
> > > harder to implement for platforms and ODBC then field-level one.
> > >
> >
> > Igor, it seems like you are advocating the per-cell configuration, not
> > per-field one. The per-field configuration can be defined at the
> > table/cache level.
> >
> > I see your point about C++ and .NET integrations however. Can't we
> provide
> > this info at node-join time or table-creation time? This way all nodes
> will
> > receive it and you will be able to grab it on different platforms.
> >
> >
> > >
> > > Also, what about binary objects, which are not stored in cache,
> > > but being marshalled?
> > >
> >
> > I think the default system encoding should be used here. If we don't have
> > configuration for default encoding, we should add it.
> >
> >
> > >
> > >
> > > Best Regards,
> > > Igor
> > >
> > > On Wed, Jul 26, 2017 at 7:22 PM, Dmitriy Setrakyan <
> > dsetrak...@apache.org>
> > > wrote:
> > >
> > > > On Wed, Jul 26, 2017 at 3:40 AM, Vyacheslav Daradur <
> > daradu...@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > >
> > > > > > Encoding must be set on per field basis. This will give us as
> most
> > > > > flexible
> > > > > > solution at the cost of 1-byte overhead.
> > > > >
> > > > > > Vova, I agree that the encoding should be set on per-field basis,
> > but
> > > > at
> > > > > > the table level, not at a cell level.
> > > > >
> > > > > Dmitriy, Vladimir,
> > > > > Let's use both approaches :-)
> > > > > We can add parameter to CacheConfiguration.
> > > > > If parameter specifie to use cache level encoding then marshaller
> > will
> > > > use
> > > > > encoding in a cache,
> > > > > otherwise marshaller will use per-field encoding.
> > > > > Of course only if it doesn't complicate the solution.
> > > > >
> > > > >
> > > > I think that it will complicate the solution and will complicate the
> > > > marshalling protocol. The advantage of specifying the encoding at
> > > > table/cache level is that we don't need to add extra encoding bytes
> to
> > > the
> > > > marshalling protocol.
> > > >
> > > > I think Vova was suggesting encoding at the cell level, not at the
> > field
> > > > level, which seems to be redundant to me.
> > > >
> > > > Vova, do you agree?
> > > >
> > >
> >
>

Re: Non-UTF-8 string encoding support in BinaryMarshaller (IGNITE-5655)

[Pavel Tupitsyn]

> 1 byte for every field just for this
GridBinaryMarshaller.STRING data type remains untouched.
We add GridBinaryMarshaller.STRING_ENCODED, which has additional byte for
encoding type.

This means no overhead for existing code.
I think the most common use case is English, which uses 1 byte per char in
UTF-8.
This is already as fast and compact as possible, and we don't want to
introduce any lookup overhead here.

And when user knows that their data will be more compact in some specific
encoding,
they use some BinaryWriter.writeString overload, which writes a different
type code.

Yes, it also writes an extra byte, but you save a byte per char of the
actual string
(for example, when using Windows-1251 for Russian text), so this does not
matter.

On Thu, Jul 27, 2017 at 5:35 PM, Dmitriy Setrakyan 
wrote:

> Pavel, what would be the size overhead? Are we adding 1 byte for every
> field just for this? If you would like to have this info in the binary
> object directly, can we in this case have some bitmap of field-to-encoding?
>
> D.
>
> On Thu, Jul 27, 2017 at 9:22 AM, Pavel Tupitsyn 
> wrote:
>
> > I'm not sure I uderstand how this "per field" configuration is supposed
> to
> > be implemented.
> > * Marshaller is not tied to a cache. It serializes all kinds of things,
> > like compute job parameters and results.
> > * Raw mode does not involve field names.
> >
> > Also it seems like a complicated and expensive solution - looking up
> string
> > format somewhere in the metadata will be slow.
> >
> > "encoded string" data type suggestion from Vladimir looks better to me
> from
> > performance and implementation standpoint.
> >
> > Thanks,
> > Pavel
> >
> >
> >
> > On Thu, Jul 27, 2017 at 5:10 PM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> > wrote:
> >
> > > On Thu, Jul 27, 2017 at 9:04 AM, Igor Sapego 
> wrote:
> > >
> > > > Just a note from the platforms guy:
> > > >
> > > > Solution with table-level configuration is going to be significantly
> > > > harder to implement for platforms and ODBC then field-level one.
> > > >
> > >
> > > Igor, it seems like you are advocating the per-cell configuration, not
> > > per-field one. The per-field configuration can be defined at the
> > > table/cache level.
> > >
> > > I see your point about C++ and .NET integrations however. Can't we
> > provide
> > > this info at node-join time or table-creation time? This way all nodes
> > will
> > > receive it and you will be able to grab it on different platforms.
> > >
> > >
> > > >
> > > > Also, what about binary objects, which are not stored in cache,
> > > > but being marshalled?
> > > >
> > >
> > > I think the default system encoding should be used here. If we don't
> have
> > > configuration for default encoding, we should add it.
> > >
> > >
> > > >
> > > >
> > > > Best Regards,
> > > > Igor
> > > >
> > > > On Wed, Jul 26, 2017 at 7:22 PM, Dmitriy Setrakyan <
> > > dsetrak...@apache.org>
> > > > wrote:
> > > >
> > > > > On Wed, Jul 26, 2017 at 3:40 AM, Vyacheslav Daradur <
> > > daradu...@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > > Encoding must be set on per field basis. This will give us as
> > most
> > > > > > flexible
> > > > > > > solution at the cost of 1-byte overhead.
> > > > > >
> > > > > > > Vova, I agree that the encoding should be set on per-field
> basis,
> > > but
> > > > > at
> > > > > > > the table level, not at a cell level.
> > > > > >
> > > > > > Dmitriy, Vladimir,
> > > > > > Let's use both approaches :-)
> > > > > > We can add parameter to CacheConfiguration.
> > > > > > If parameter specifie to use cache level encoding then marshaller
> > > will
> > > > > use
> > > > > > encoding in a cache,
> > > > > > otherwise marshaller will use per-field encoding.
> > > > > > Of course only if it doesn't complicate the solution.
> > > > > >
> > > > > >
> > > > > I think that it will complicate the solution and will complicate
> the
> > > > > marshalling protocol. The advantage of specifying the encoding at
> > > > > table/cache level is that we don't need to add extra encoding bytes
> > to
> > > > the
> > > > > marshalling protocol.
> > > > >
> > > > > I think Vova was suggesting encoding at the cell level, not at the
> > > field
> > > > > level, which seems to be redundant to me.
> > > > >
> > > > > Vova, do you agree?
> > > > >
> > > >
> > >
> >
>

Re: Bug in method

[Anton Vinogradov]

Vitaliy,

Let's file an issue and solve it :)
Don't forget to add tests.

On Thu, Jul 27, 2017 at 5:24 PM, Vitaliy Biryukov <
biryukovvitali...@gmail.com> wrote:

> Hi, igniters!
>
> I noticed that the method "IgniteUtils.ceilPow2" can overflow for values
> greater than 2^30 and return Integer.MIN_VALUE for them.
>
> Maybe this check was skipped  for method speed. In this case, we need to
> add information about this into javaDoc.
>
> Current implementation:
>
> public static int ceilPow2(int v) {
> return Integer.highestOneBit(v - 1) << 1;
> }
>
>
> It can be fixed like this:
> bit impl:
>
> int i = v - 1;
>
> return Integer.highestOneBit(i) << 1 - (i >>> 30 ^ v >> 31);
>
> more readable impl:
>
> if (v == Integer.MIN_VALUE)
> return 0;
>
> int pow30 = 1 << 30;
>
> return v >= pow30 ? pow30 : Integer.highestOneBit(v - 1) << 1;
>
>
> What do you think? Sould I create  ticket with PR?
>
> --
> Best Regards,
> Vitaliy Biryukov
>

Assertions as binary data validation checks in deserialization

[Andrey Kuznetsov]

Hi Igniters,

While examining BinaryObjectImpl code I found this curious line in typeId()
method:

  assert arr[off] == GridBinaryMarshaller.STRING : arr[off];

Is it OK to check external binary data with assertions?
I think it can lead to undefined behaviour on corrupt data from the wire.

--
Best regards,
  Andrey Kuznetsov.

[GitHub] ignite pull request #2256: Ignite 2190

[nizhikov]

Github user nizhikov closed the pull request at:

https://github.com/apache/ignite/pull/2256


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

My first patch into Ignite

[Николай Ижиков]

Hello, guys.



Got my very first patch merged into master.



I resolved IGNITE-2190 [1].

I add some warnings about inconsistent Ignite usage: BinaryMarshaller
without ignite.withBinary().

Now users can resolve some issues just by carefully reading log.



Thanks all the Ignite community and especially Anton Vinogradov.

[1] https://issues.apache.org/jira/browse/IGNITE-2190

-- 
Nikolay Izhikov
nizhikov@gmail.com

[GitHub] ignite pull request #2353: IGNITE-5813: Inconsistent cache store state when ...

[AMashenkov]

GitHub user AMashenkov opened a pull request:

https://github.com/apache/ignite/pull/2353

IGNITE-5813: Inconsistent cache store state when originating node fails on 
commit.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5813

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2353.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2353


commit c411722afe960260027a9081a38e3800ca3c28de
Author: Andrey V. Mashenkov 
Date:   2017-07-27T16:27:03Z

WIP.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #2354: IGNITE-5758: CPP: Added pointer semantics for pri...

[isapego]

GitHub user isapego opened a pull request:

https://github.com/apache/ignite/pull/2354

IGNITE-5758: CPP: Added pointer semantics for primitive types



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5758

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2354.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2354


commit 14bc8e263244760e3e1057ce2fd645183c926e0e
Author: Igor Sapego 
Date:   2017-07-26T17:15:08Z

IGNITE-5758: Added test

commit de3b0e51425df81d9abaf67deb0c4817c090d901
Author: Igor Sapego 
Date:   2017-07-26T17:15:29Z

IGNITE-5758: Implemented writing.

commit 65cd23facb66a119ba44c442a3482620079f0a70
Author: Igor Sapego 
Date:   2017-07-26T18:10:51Z

IGNITE-5758: Implemented reading.

commit b60db26a52d590171b498a92eb9287684845eb4c
Author: Igor Sapego 
Date:   2017-07-27T16:05:56Z

IGNITE-5758: Fixed tests and reader

commit fc7f70c6f66479ee6080a2b42ae2bd25ee2c95d3
Author: Igor Sapego 
Date:   2017-07-27T16:12:08Z

IGNITE-5758: Added raw tests

commit 30c0d07876db35127ceabdef0d905a86c2086e93
Author: Igor Sapego 
Date:   2017-07-27T16:32:31Z

IGNITE-5758: Added null fields to tests




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-5859) IgniteUtils.ceilPow2 overflow for values greater than 2^30

[Vitaliy Biryukov (JIRA)]

Vitaliy Biryukov  created IGNITE-5859:
-

 Summary: IgniteUtils.ceilPow2 overflow for values greater than 2^30
 Key: IGNITE-5859
 URL: https://issues.apache.org/jira/browse/IGNITE-5859
 Project: Ignite
  Issue Type: Bug
Reporter: Vitaliy Biryukov 


Method "IgniteUtils.ceilPow2" can overflow for values greater than 2^30 and 
return Integer.MIN_VALUE for them.
Maybe this check was skipped for method speed. In this case, we need to add 
information about this into javaDoc.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-5860) Client disconnects if server it is connected to goes unresponsive

[Evgenii Zhuravlev (JIRA)]

Evgenii Zhuravlev created IGNITE-5860:
-

 Summary: Client disconnects if server it is connected to goes 
unresponsive 
 Key: IGNITE-5860
 URL: https://issues.apache.org/jira/browse/IGNITE-5860
 Project: Ignite
  Issue Type: Bug
Affects Versions: 1.7
Reporter: Evgenii Zhuravlev
 Fix For: 2.2


Scenario is the following:

# Start at least two server nodes.
# Start a client node.
# Detect server node client is connected to in discovery SPI.
# Suspend that server (^Z in terminal or 'kill -SIGSTOP ').
# It's expected that client will drop connection with this server and 
connect to another one.
# However, a client gets dropped from topology and disconnects.

A client should reconnect cluster before the timeout and without 
EVT_CLIENT_NODE_RECONNECTED event.

In ClientImpl.Reconnector in joinTopology method it gets all addresses from 
discoverySpi and addresses of the server that was suspended on top of this list.

Suggested solution:
Move addresses of the suspended server to the end of the list for the join.






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2355: IGNITE-5859: Utils.ceilPow2 method fixed.

[BiryukovVA]

GitHub user BiryukovVA opened a pull request:

https://github.com/apache/ignite/pull/2355

IGNITE-5859: Utils.ceilPow2 method fixed.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/BiryukovVA/ignite IGNITE-5859

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/2355.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2355


commit b2c4c01d8bccc55532de276cdca24ab23680fcc6
Author: Vitaliy Biryukov 
Date:   2017-07-27T17:38:08Z

IGNITE-5859: Utils.ceilPow2 method fixed.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [RESULT] [VOTE] Apache Ignite 2.1.0 Release (RC4)

[Anton Vinogradov]

Done.

- Maven artifacts released
- Sources released
- Site updated
- Git tag added

On Thu, Jul 27, 2017 at 4:24 PM, Anton Vinogradov  wrote:

> Igniters,
>
> Apache Ignite 2.1.0 release (RC4) has been accepted.
>
> 7 "+1" votes received:
>
> - Valentin Kulichenko (binding)
> - Nikolai Tikhonov (binding)
> - Alexey Kuznetsov (binding)
> - Yakov Zhdanov (binding)
> - Konstantin Boudnik (binding)
> - Andrey Gura (binding)
> - Denis Magda (binding)
>
> Vote thread:
>
> http://apache-ignite-developers.2346864.n4.nabble.
> com/VOTE-Apache-Ignite-2-1-0-RC4-td19969.html
>
> Ignite 2.1.0 will be released soon. Woohoo!
>

Re: Non-UTF-8 string encoding support in BinaryMarshaller (IGNITE-5655)

[Valentin Kulichenko]

Pavel,

This forces user to implement Binarylizable for whole type in case they
want to change encoding for one-two fields, right? I really don't like it,
why not add default encoding to BinaryTypeConfiguration?

-Val

On Thu, Jul 27, 2017 at 7:54 AM, Pavel Tupitsyn 
wrote:

> > 1 byte for every field just for this
> GridBinaryMarshaller.STRING data type remains untouched.
> We add GridBinaryMarshaller.STRING_ENCODED, which has additional byte for
> encoding type.
>
> This means no overhead for existing code.
> I think the most common use case is English, which uses 1 byte per char in
> UTF-8.
> This is already as fast and compact as possible, and we don't want to
> introduce any lookup overhead here.
>
> And when user knows that their data will be more compact in some specific
> encoding,
> they use some BinaryWriter.writeString overload, which writes a different
> type code.
>
> Yes, it also writes an extra byte, but you save a byte per char of the
> actual string
> (for example, when using Windows-1251 for Russian text), so this does not
> matter.
>
> On Thu, Jul 27, 2017 at 5:35 PM, Dmitriy Setrakyan 
> wrote:
>
> > Pavel, what would be the size overhead? Are we adding 1 byte for every
> > field just for this? If you would like to have this info in the binary
> > object directly, can we in this case have some bitmap of
> field-to-encoding?
> >
> > D.
> >
> > On Thu, Jul 27, 2017 at 9:22 AM, Pavel Tupitsyn 
> > wrote:
> >
> > > I'm not sure I uderstand how this "per field" configuration is supposed
> > to
> > > be implemented.
> > > * Marshaller is not tied to a cache. It serializes all kinds of things,
> > > like compute job parameters and results.
> > > * Raw mode does not involve field names.
> > >
> > > Also it seems like a complicated and expensive solution - looking up
> > string
> > > format somewhere in the metadata will be slow.
> > >
> > > "encoded string" data type suggestion from Vladimir looks better to me
> > from
> > > performance and implementation standpoint.
> > >
> > > Thanks,
> > > Pavel
> > >
> > >
> > >
> > > On Thu, Jul 27, 2017 at 5:10 PM, Dmitriy Setrakyan <
> > dsetrak...@apache.org>
> > > wrote:
> > >
> > > > On Thu, Jul 27, 2017 at 9:04 AM, Igor Sapego 
> > wrote:
> > > >
> > > > > Just a note from the platforms guy:
> > > > >
> > > > > Solution with table-level configuration is going to be
> significantly
> > > > > harder to implement for platforms and ODBC then field-level one.
> > > > >
> > > >
> > > > Igor, it seems like you are advocating the per-cell configuration,
> not
> > > > per-field one. The per-field configuration can be defined at the
> > > > table/cache level.
> > > >
> > > > I see your point about C++ and .NET integrations however. Can't we
> > > provide
> > > > this info at node-join time or table-creation time? This way all
> nodes
> > > will
> > > > receive it and you will be able to grab it on different platforms.
> > > >
> > > >
> > > > >
> > > > > Also, what about binary objects, which are not stored in cache,
> > > > > but being marshalled?
> > > > >
> > > >
> > > > I think the default system encoding should be used here. If we don't
> > have
> > > > configuration for default encoding, we should add it.
> > > >
> > > >
> > > > >
> > > > >
> > > > > Best Regards,
> > > > > Igor
> > > > >
> > > > > On Wed, Jul 26, 2017 at 7:22 PM, Dmitriy Setrakyan <
> > > > dsetrak...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > On Wed, Jul 26, 2017 at 3:40 AM, Vyacheslav Daradur <
> > > > daradu...@gmail.com
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > >
> > > > > > > > Encoding must be set on per field basis. This will give us as
> > > most
> > > > > > > flexible
> > > > > > > > solution at the cost of 1-byte overhead.
> > > > > > >
> > > > > > > > Vova, I agree that the encoding should be set on per-field
> > basis,
> > > > but
> > > > > > at
> > > > > > > > the table level, not at a cell level.
> > > > > > >
> > > > > > > Dmitriy, Vladimir,
> > > > > > > Let's use both approaches :-)
> > > > > > > We can add parameter to CacheConfiguration.
> > > > > > > If parameter specifie to use cache level encoding then
> marshaller
> > > > will
> > > > > > use
> > > > > > > encoding in a cache,
> > > > > > > otherwise marshaller will use per-field encoding.
> > > > > > > Of course only if it doesn't complicate the solution.
> > > > > > >
> > > > > > >
> > > > > > I think that it will complicate the solution and will complicate
> > the
> > > > > > marshalling protocol. The advantage of specifying the encoding at
> > > > > > table/cache level is that we don't need to add extra encoding
> bytes
> > > to
> > > > > the
> > > > > > marshalling protocol.
> > > > > >
> > > > > > I think Vova was suggesting encoding at the cell level, not at
> the
> > > > field
> > > > > > level, which seems to be redundant to me.
> > > > > >
> > > > > > Vova, do you agree?
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Assertions as binary data validation checks in deserialization

[Valentin Kulichenko]

Andrey,

How will it corrupt the data? Assertions only reads the array, not updates
it, right?

-Val

On Thu, Jul 27, 2017 at 8:54 AM, Andrey Kuznetsov  wrote:

> Hi Igniters,
>
> While examining BinaryObjectImpl code I found this curious line in typeId()
> method:
>
>   assert arr[off] == GridBinaryMarshaller.STRING : arr[off];
>
> Is it OK to check external binary data with assertions?
> I think it can lead to undefined behaviour on corrupt data from the wire.
>
> --
> Best regards,
>   Andrey Kuznetsov.
>

Re: Assertions as binary data validation checks in deserialization

[Andrey Kuznetsov]

Valentin,

I meant the behaviour of this code when corrupted data from the network are
being deserialized. Assertion is no-op in production, and we silently
ignore binary format violation.

27 июля 2017 г. 21:09 пользователь "Valentin Kulichenko" <
valentin.kuliche...@gmail.com> написал:

> Andrey,
>
> How will it corrupt the data? Assertions only reads the array, not updates
> it, right?
>
> -Val
>
> On Thu, Jul 27, 2017 at 8:54 AM, Andrey Kuznetsov 
> wrote:
>
> > Hi Igniters,
> >
> > While examining BinaryObjectImpl code I found this curious line in
> typeId()
> > method:
> >
> >   assert arr[off] == GridBinaryMarshaller.STRING : arr[off];
> >
> > Is it OK to check external binary data with assertions?
> > I think it can lead to undefined behaviour on corrupt data from the wire.
> >
> > --
> > Best regards,
> >   Andrey Kuznetsov.
> >
>

Re: [RESULT] [VOTE] Apache Ignite 2.1.0 Release (RC4)

[Denis Magda]

I’m preparing the docs and site for the release. You’ll not recall the front 
page after it’s out.

Mauricio, could you update the “latest” docs to refer to version 2.1.0
For instance this JavaDoc still points to version 2.0.0: 
https://ignite.apache.org/releases/latest/javadoc/index.html 


—
Denis

> On Jul 27, 2017, at 11:06 AM, Anton Vinogradov  wrote:
> 
> Done.
> 
> - Maven artifacts released
> - Sources released
> - Site updated
> - Git tag added
> 
> On Thu, Jul 27, 2017 at 4:24 PM, Anton Vinogradov  wrote:
> 
>> Igniters,
>> 
>> Apache Ignite 2.1.0 release (RC4) has been accepted.
>> 
>> 7 "+1" votes received:
>> 
>> - Valentin Kulichenko (binding)
>> - Nikolai Tikhonov (binding)
>> - Alexey Kuznetsov (binding)
>> - Yakov Zhdanov (binding)
>> - Konstantin Boudnik (binding)
>> - Andrey Gura (binding)
>> - Denis Magda (binding)
>> 
>> Vote thread:
>> 
>> http://apache-ignite-developers.2346864.n4.nabble.
>> com/VOTE-Apache-Ignite-2-1-0-RC4-td19969.html
>> 
>> Ignite 2.1.0 will be released soon. Woohoo!
>> 

Re: Assertions as binary data validation checks in deserialization

[Valentin Kulichenko]

Do you suggest to throw an exception instead of assertions?

-Val

On Thu, Jul 27, 2017 at 11:52 AM, Andrey Kuznetsov 
wrote:

> Valentin,
>
> I meant the behaviour of this code when corrupted data from the network are
> being deserialized. Assertion is no-op in production, and we silently
> ignore binary format violation.
>
> 27 июля 2017 г. 21:09 пользователь "Valentin Kulichenko" <
> valentin.kuliche...@gmail.com> написал:
>
> > Andrey,
> >
> > How will it corrupt the data? Assertions only reads the array, not
> updates
> > it, right?
> >
> > -Val
> >
> > On Thu, Jul 27, 2017 at 8:54 AM, Andrey Kuznetsov 
> > wrote:
> >
> > > Hi Igniters,
> > >
> > > While examining BinaryObjectImpl code I found this curious line in
> > typeId()
> > > method:
> > >
> > >   assert arr[off] == GridBinaryMarshaller.STRING : arr[off];
> > >
> > > Is it OK to check external binary data with assertions?
> > > I think it can lead to undefined behaviour on corrupt data from the
> wire.
> > >
> > > --
> > > Best regards,
> > >   Andrey Kuznetsov.
> > >
> >
>

Re: Assertions as binary data validation checks in deserialization

[Andrey Kuznetsov]

Indeed, "let it crash" approach is better than unclear error in some
indeterminate place later. Here we depend on data from "inpredictable"
source, so assertions are not suitable.

27 июля 2017 г. 22:35 пользователь "Valentin Kulichenko" <
valentin.kuliche...@gmail.com> написал:

Do you suggest to throw an exception instead of assertions?

-Val

Re: Assertions as binary data validation checks in deserialization

[Valentin Kulichenko]

Makes sense to me. Feel free to create a ticket unless someone else has any
objection. However, I think that we should revise other code for such
places then. Fixing only this one line doesn't change a lot.

-Val

On Thu, Jul 27, 2017 at 12:55 PM, Andrey Kuznetsov 
wrote:

> Indeed, "let it crash" approach is better than unclear error in some
> indeterminate place later. Here we depend on data from "inpredictable"
> source, so assertions are not suitable.
>
> 27 июля 2017 г. 22:35 пользователь "Valentin Kulichenko" <
> valentin.kuliche...@gmail.com> написал:
>
> Do you suggest to throw an exception instead of assertions?
>
> -Val
>

Re: Assertions as binary data validation checks in deserialization

[Andrey Kuznetsov]

May I change this line while working on existing ticket, unless someone
else objects?

27 июля 2017 г. 23:37 пользователь "Valentin Kulichenko" <
valentin.kuliche...@gmail.com> написал:

Makes sense to me. Feel free to create a ticket unless someone else has any
objection. However, I think that we should revise other code for such
places then. Fixing only this one line doesn't change a lot.

-Val

Re: [RESULT] [VOTE] Apache Ignite 2.1.0 Release (RC4)

[Denis Magda]

Well, folks, I’m done with the documentation and site update.

Mauricio, will update the references soon.

So, as far as I see all the main post-release procedures are completed. I’ll 
send an announcement shortly.

*Anton*, just in case, please double check that we hasn’t missed anything 
significant:
https://cwiki.apache.org/confluence/display/IGNITE/Release+Process

—
Denis

> On Jul 27, 2017, at 12:27 PM, Denis Magda  wrote:
> 
> I’m preparing the docs and site for the release. You’ll not recall the front 
> page after it’s out.
> 
> Mauricio, could you update the “latest” docs to refer to version 2.1.0
> For instance this JavaDoc still points to version 2.0.0: 
> https://ignite.apache.org/releases/latest/javadoc/index.html 
> 
> 
> —
> Denis
> 
>> On Jul 27, 2017, at 11:06 AM, Anton Vinogradov > > wrote:
>> 
>> Done.
>> 
>> - Maven artifacts released
>> - Sources released
>> - Site updated
>> - Git tag added
>> 
>> On Thu, Jul 27, 2017 at 4:24 PM, Anton Vinogradov > > wrote:
>> 
>>> Igniters,
>>> 
>>> Apache Ignite 2.1.0 release (RC4) has been accepted.
>>> 
>>> 7 "+1" votes received:
>>> 
>>> - Valentin Kulichenko (binding)
>>> - Nikolai Tikhonov (binding)
>>> - Alexey Kuznetsov (binding)
>>> - Yakov Zhdanov (binding)
>>> - Konstantin Boudnik (binding)
>>> - Andrey Gura (binding)
>>> - Denis Magda (binding)
>>> 
>>> Vote thread:
>>> 
>>> http://apache-ignite-developers.2346864.n4.nabble 
>>> .
>>> com/VOTE-Apache-Ignite-2-1-0-RC4-td19969.html
>>> 
>>> Ignite 2.1.0 will be released soon. Woohoo!
>>> 
> 

Apache Ignite Site Changed its Face

[Denis Magda]

Igniters,

In favor of the 2.1 release we’ve reworked and renewed the front page and some 
of the existing pages of the site:
https://ignite.apache.org

The front page reflects the main additions of 2.1, durable memory + persistent 
store, that opened new opportunities for Ignite. Please welcome new Ignite that 
“provides in-memory performance with the durability of disk”:
https://twitter.com/denismagda/status/890683287852601345

—
Denis

[ANNOUNCE] Apache Ignite 2.1.0 Released

[Denis Magda]

The Apache Ignite Community is pleased to announce the release of Apache Ignite 
2.1.0 [1].
https://blogs.apache.org/ignite/entry/apache-ignite-2-1-a

This release incorporates a tremendous feature that was donated to the project 
- Ignite Persistent Store that altogether with the Durable Memory architecture 
empowers your applications with in-memory performance and durability of the 
disk.

The Ignite Persistent Store is a distributed ACID and SQL-compliant disk store 
that transparently integrates with Ignite as an optional disk tier (SSD, Flash, 
3D XPoint). Having the store enabled, you no longer need to keep all the data 
in memory or warm RAM up after the whole cluster restart. The persistent store 
will keep the superset of data and all the SQL indexes on disk making Ignite 
fully operational from disk. 

Considering these and many other changes we redefined the definition of Ignite 
a bit that know sounds:

Ignite is a memory-centric platform
• combining a distributed SQL database
• with a key-value data grid
• that is ACID-compliant
• and horizontally scalable

Learn more about the transition from the in-memory to memory-centric 
architecture from 2.1 announcement blog post: 
https://blogs.apache.org/ignite/entry/apache-ignite-2-1-a

Attend the meetup today in Bay Area, CA to get insights on the new additions: 
https://www.meetup.com/Bay-Area-In-Memory-Computing/events/241381155/

In addition the release includes support for CREATE and DROP table commands, 
adds new Machine Learning algorithms such as K-means clustering and 
regressions, enables peer-class-loading for .NET and provides Compute Grid APIs 
for C++.

The full list of the changes can be found here: 
https://ignite.apache.org/releases/2.1.0/release_notes.html

Please visit this page and check the release out:
https://ignite.apache.org/download.cgi


Regards,
Denis

[1] https://ignite.apache.org