Re: SELECT subqueries in DML statements

[Sergi Vladykin]

1. Actually this doc looks misleading and wrong to me. What does it mean "it
will not be distributed"? It only means that we will not execute any
distributed joins inside of this subquery. In all other means this
operation will be correctly distributed.

We will generate query like this :

SELECT _key, _val FROM Person WHERE _key IN (subquery)

And execute it on all the nodes and if the subquery is correctly colocated
with Person everything will work fine.

2. This is not something we are going to change because subquery will be
executed for each row in Person. If it will have a distributed join in it,
this operation will never return and most probably will kill the cluster by
message flooding.

Sergi



2016-12-23 3:05 GMT+03:00 Denis Magda :

> Sergi,
>
> Presently in the DML statement doc we mention the following limitation
> related to SELECT subqueries
> http://apacheignite.gridgain.org/docs/dml#section-
> subqueries-in-where-clause
>
> Does it mean that this limitation will be no longer true for DML
> statements as well once the improvement below gets released?
> https://issues.apache.org/jira/browse/IGNITE-3860
>
> —
> Denis
>

Re: DML Documentation Readiness

[Prachi Garg]

Sure, will look into it.

-P

On Thu, Dec 22, 2016 at 4:00 PM, Denis Magda  wrote:

> I’ve finished the review and released all the parts of DML documentation
> [1].
>
> Prachi, could you review whole SQL Grid documentation [2] including [1]?
> Also it will be great if you check this newly added section [3].
>
> [1] http://apacheignite.gridgain.org/docs/dml
> [2] http://apacheignite.gridgain.org/docs/sql-grid
> [3] http://apacheignite.gridgain.org/docs/binary-
> marshaller#handling-hash-code-generation-and-equals-execution
>
> —
> Denis
>
> On Dec 9, 2016, at 12:34 AM, Alexander Paschenko <
> alexander.a.pasche...@gmail.com> wrote:
>
> Denis,
>
> About your question: 'putIfAbsent' is used only when we have single
> key-value pair to process as long as we don't have method like 'put
> all if absent'. When there's a bunch of pairs, an entry processor is
> created for each pair to make its processing atomic (effectively CAS
> style processing), and all those processors are then executed via
> 'invokeAll'.
>
> - Alex
>
> 2016-12-09 8:17 GMT+03:00 Denis Magda :
>
> Alexander, thanks for all the material you’ve prepared so far.
>
> Actually, I’ve reviewed, restructured and released only a part of it [1].
>
> The rest, that is missing, requires more time from my side and I’ll be
> releasing additional blocks throughout the next week. The full
> documentation
> with the unreviewed parts is invisible and located here [2].
>
> Please don’t modify neither of the docs ([1] and [2]) until I’ve finished
> with the overall review. Send all your suggestions over the email.
>
> At the moment I have the following question. What cache method(s) do we
> convert INSERT statements to [3]? Is it ‘putIfAbsent’?
>
> [1] https://apacheignite.readme.io/docs/dml
> [2] https://apacheignite.readme.io/docs/distributed-dml-to-be-reviewe
> [3] https://apacheignite.readme.io/docs/dml#section-insert
>
> —
> Denis
>
> On Dec 8, 2016, at 1:35 AM, Alexander Paschenko
>  wrote:
>
> Denis,
>
> it's all ready, JDBC driver page is also updated (I've created a
> hidden clone page, please find it on the left) as well as one on
> binary marshaller. Also I've renamed 'distributed DML' page to just
> 'DML' as long as there's much less to distribution of DML operations
> themselves than to queries and there's clearly no need to make another
> doc page to distinguish. Removed batches mention from limitations.
>
> - Alex
>
> 2016-12-08 5:26 GMT+03:00 Denis Magda :
>
> Alexander,
>
> How close are you to the finalization of the DML doc?
> https://apacheignite.readme.io/docs/distributed-dml
>
> Since we’re approaching 1.8 release I would like to do a final review of it
> polishing whatever is needed tomorrow.
>
> As I see that only the limitations section is left. The first note is that
> I
> wouldn’t mention “JDBC Batching mode” for now because this is something
> that
> doesn’t affect the usability and SQL API's scope of support.
>
> —
> Denis
>
>
>

EXPLAIN for DML statements

[Denis Magda]

Alex P., Sergi,

All we know that EXPLAIN is not supported for DML statements at the moment.
http://apacheignite.gridgain.org/docs/dml#section-explain-support-for-dml-statements
 


However, I can’t get how I can get over this limitation for UPDATE and DELETE 
statements following this recommendation 

"One possible approach is to execute EXPLAIN for the SELECT automatically 
generated (UPDATE, DELETE) or used (INSERT, MERGE) by DML statements. This will 
give an insight on the indexes that are used while a DML operation is executed.”

How can I get SELECT queries that are generated by these DML operations 
automatically?

—
Denis

SELECT subqueries in DML statements

[Denis Magda]

Sergi,

Presently in the DML statement doc we mention the following limitation related 
to SELECT subqueries
http://apacheignite.gridgain.org/docs/dml#section-subqueries-in-where-clause 


Does it mean that this limitation will be no longer true for DML statements as 
well once the improvement below gets released?
https://issues.apache.org/jira/browse/IGNITE-3860 


—
Denis

Re: DML Documentation Readiness

[Denis Magda]

I’ve finished the review and released all the parts of DML documentation [1].

Prachi, could you review whole SQL Grid documentation [2] including [1]? Also 
it will be great if you check this newly added section [3].

[1] http://apacheignite.gridgain.org/docs/dml 

[2] http://apacheignite.gridgain.org/docs/sql-grid 

[3] 
http://apacheignite.gridgain.org/docs/binary-marshaller#handling-hash-code-generation-and-equals-execution
 


—
Denis

> On Dec 9, 2016, at 12:34 AM, Alexander Paschenko 
>  wrote:
> 
> Denis,
> 
> About your question: 'putIfAbsent' is used only when we have single
> key-value pair to process as long as we don't have method like 'put
> all if absent'. When there's a bunch of pairs, an entry processor is
> created for each pair to make its processing atomic (effectively CAS
> style processing), and all those processors are then executed via
> 'invokeAll'.
> 
> - Alex
> 
> 2016-12-09 8:17 GMT+03:00 Denis Magda  >:
>> Alexander, thanks for all the material you’ve prepared so far.
>> 
>> Actually, I’ve reviewed, restructured and released only a part of it [1].
>> 
>> The rest, that is missing, requires more time from my side and I’ll be
>> releasing additional blocks throughout the next week. The full documentation
>> with the unreviewed parts is invisible and located here [2].
>> 
>> Please don’t modify neither of the docs ([1] and [2]) until I’ve finished
>> with the overall review. Send all your suggestions over the email.
>> 
>> At the moment I have the following question. What cache method(s) do we
>> convert INSERT statements to [3]? Is it ‘putIfAbsent’?
>> 
>> [1] https://apacheignite.readme.io/docs/dml
>> [2] https://apacheignite.readme.io/docs/distributed-dml-to-be-reviewe 
>> 
>> [3] https://apacheignite.readme.io/docs/dml#section-insert 
>> 
>> 
>> —
>> Denis
>> 
>> On Dec 8, 2016, at 1:35 AM, Alexander Paschenko
>>  wrote:
>> 
>> Denis,
>> 
>> it's all ready, JDBC driver page is also updated (I've created a
>> hidden clone page, please find it on the left) as well as one on
>> binary marshaller. Also I've renamed 'distributed DML' page to just
>> 'DML' as long as there's much less to distribution of DML operations
>> themselves than to queries and there's clearly no need to make another
>> doc page to distinguish. Removed batches mention from limitations.
>> 
>> - Alex
>> 
>> 2016-12-08 5:26 GMT+03:00 Denis Magda :
>> 
>> Alexander,
>> 
>> How close are you to the finalization of the DML doc?
>> https://apacheignite.readme.io/docs/distributed-dml
>> 
>> Since we’re approaching 1.8 release I would like to do a final review of it
>> polishing whatever is needed tomorrow.
>> 
>> As I see that only the limitations section is left. The first note is that I
>> wouldn’t mention “JDBC Batching mode” for now because this is something that
>> doesn’t affect the usability and SQL API's scope of support.
>> 
>> —
>> Denis

[GitHub] ignite pull request #1369: IGNITE-4439 - Attribute based node filter

[vkulichenko]

Github user vkulichenko closed the pull request at:

https://github.com/apache/ignite/pull/1369


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Node attribute filter

[Valentin Kulichenko]

Thanks, Denis! Will merge shortly.

On Thu, Dec 22, 2016 at 11:55 AM, Denis Magda  wrote:

> Val, thanks. Reviewed. Looks good to merge.
>
> —
> Denis
>
> > On Dec 20, 2016, at 3:27 PM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
> >
> > Guys,
> >
> > I created a simple implementation of IgnitePredicate based
> on
> > user attributes. This is the most commonly used approach for node
> filters,
> > but currently users are forced to create their own implementations each
> > time. Would be useful to have it out of the box.
> >
> > Can someone take a look at the implementation and tell me if there is
> > anything wrong/missed?
> >
> > Ticket: https://issues.apache.org/jira/browse/IGNITE-4439
> > PR: https://github.com/apache/ignite/pull/1369
> >
> > -Val
>
>

Re: Node attribute filter

[Denis Magda]

Val, thanks. Reviewed. Looks good to merge.

—
Denis

> On Dec 20, 2016, at 3:27 PM, Valentin Kulichenko 
>  wrote:
> 
> Guys,
> 
> I created a simple implementation of IgnitePredicate based on
> user attributes. This is the most commonly used approach for node filters,
> but currently users are forced to create their own implementations each
> time. Would be useful to have it out of the box.
> 
> Can someone take a look at the implementation and tell me if there is
> anything wrong/missed?
> 
> Ticket: https://issues.apache.org/jira/browse/IGNITE-4439
> PR: https://github.com/apache/ignite/pull/1369
> 
> -Val

Re: Capacity Planning - Calculating Memory Usage

[Valentin Kulichenko]

Alexandr,

See my comments below.

On Wed, Dec 21, 2016 at 7:01 PM, Alexandr Kuramshin 
wrote:

> Hi Val,
>
> the understanding is simple.
>
> When you enables the single index on entry class you get "First index
> overhead" per entry.
>
> When you enables two indices on entry class you get "First index overhead"
> + "Next index overhead" per entry.
>
> With three indices you get "First index overhead" + 2 * "Next index
> overhead", and so on...
>

This should be explained in more detail, probably with some trivial
example. Currently it's very unclear.


>
> Each annotated field with @QuerySqlField is an index, except multiple
> fields annotated with @QuerySqlField.Group.
>

This actually confuses me a lot, because a field can be created with or
without index? Can you please clarify? How much overhead is introduced by a
field without index? With index? What about group indexes?


>
> Another way to defining indices is to use property "queryEntities" and it's
> subproperty "indexes". See the article [1]
>
> [1] https://apacheignite.readme.io/docs/indexes
>
> 2016-12-20 8:38 GMT+07:00 Valentin Kulichenko <
> valentin.kuliche...@gmail.com
> >:
>
> > Alexandr,
> >
> > Can you please clarify what is "First index overhead" and "Next index
> > overhead"? Generally, I think overhead provided by indexes should be
> > described in more details, now it's not very clear what happens when
> > indexes are added.
> >
> > Also the calculation example should be a separate section.
> >
> > -Val
> >
> > On Wed, Dec 14, 2016 at 1:07 AM, Alexandr Kuramshin <
> ein.nsk...@gmail.com>
> > wrote:
> >
> > > Thank you, Andrey,
> > >
> > > I'll do additional tests with expire policy and update the article.
> > >
> > > 2016-12-13 22:10 GMT+07:00 Andrey Mashenkov <
> andrey.mashen...@gmail.com
> > >:
> > >
> > > > Alexandr,
> > > >
> > > > In addition. If expire policy is configured, there is additional
> > overhead
> > > > to entries can be tracked by TtlManager.
> > > > This overhead is OnHeap and does not depend on cache MemoryMode
> (until
> > > > Ignite-3840 will be in master).
> > > >
> > > > For now overhead is about 32-40 bytes (EntryWrapper itself) + (40-48)
> > > bytes
> > > > (ConcurrentSkipList node) per entry.
> > > >
> > > >
> > > >
> > > > On Tue, Dec 13, 2016 at 10:37 AM, Alexandr Kuramshin <
> > > ein.nsk...@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hello, Igniters,
> > > > >
> > > > > I'd like to represent updated article [1] about the subject.
> > > > >
> > > > > And I'll very appreciate your comments and questions about it.
> > > > >
> > > > > Please review.
> > > > >
> > > > > [1] http://apacheignite.gridgain.org/docs/capacity-planning
> > > > >
> > > > > --
> > > > > Thanks,
> > > > > Alexandr Kuramshin
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > С уважением,
> > > > Машенков Андрей Владимирович
> > > > Тел. +7-921-932-61-82
> > > >
> > > > Best regards,
> > > > Andrey V. Mashenkov
> > > > Cerr: +7-921-932-61-82
> > > >
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > Alexandr Kuramshin
> > >
> >
>
>
>
> --
> Thanks,
> Alexandr Kuramshin
>

[jira] [Created] (IGNITE-4485) CacheJdbcPojoStore returns unexpected BinaryObject upon loadCache()

[Alexandr Kuramshin (JIRA)]

Alexandr Kuramshin created IGNITE-4485:
--

 Summary: CacheJdbcPojoStore returns unexpected BinaryObject upon 
loadCache()
 Key: IGNITE-4485
 URL: https://issues.apache.org/jira/browse/IGNITE-4485
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 1.8, 1.7
Reporter: Alexandr Kuramshin


When calling loadCache(IgniteBiInClosure clo, Object... args) sometimes we get 
unexpected values of type BinaryObject in IgniteBiInClosure.apply(), whereas 
POJO value kind was registered previously for well known key type.

It's so because getOrCreateCacheMappings returns HashMap which resorts entity 
mappings for the same key but with different value kind. When BinaryMarshaller 
is used, then this map contains two mappings for the same key - POJO and BINARY.

Possible fix is to use LinkedHashMap, then POJO mapping will be picked first.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Google Summer Of Code 2017

[Denis Magda]

This is great!

So, we already have three mentors in total (Val, Dmitriy, Denis). Anyone else?

Any other tasks you think are suitable for this summer program?

—
Denis

> On Dec 21, 2016, at 4:26 PM, Valentin Kulichenko 
>  wrote:
> 
> I can take over DataFrames integration as I already did some investigation
> in this area. Spark also has GraphX and GraphFrames for graph processing,
> we can think about integration with them as well.
> 
> Machine learning also sounds very interesting.
> 
> -Val
> 
> On Wed, Dec 21, 2016 at 4:22 PM, Dmitriy Setrakyan 
> wrote:
> 
>> I can also do mentoring for any of the listed features.
>> 
>> On Wed, Dec 21, 2016 at 2:50 PM, Denis Magda  wrote:
>> 
>>> Alex, thanks for bringing this out!
>>> 
>>> This is a great opportunity for our community to find talented students
>>> who are capable of adding new features to Ignite. I can take care of all
>>> the paperwork. The features list won’t be a big deal as well. But we need
>>> mentors who will voluntarily dedicate their time helping the
>> participants.
>>> 
>>> Well, here is a list of features that pop up immediately out of my head
>>> and that I would like to offer to GSoC (Google Summer Of Code).
>>> Native client for Python (data grid, compute grid, continuous queries).
>>> Native client for Node.JS (https://issues.apache.org/
>>> jira/browse/IGNITE-961 >> )
>>> Kubernetis integration (IGNITE-4159 >> jira/browse/IGNITE-4159>).
>>> Spark Data Frames API implementation.
>>> Machine Learning Grid implementation or integration with Spark MLib.
>>> 
>>> Any other features?
>>> 
>>> As for the mentoring I can take over one or two of the following -
>>> Kubernetis, Machine Learning or Node.JS.
>>> 
>>> Is there anyone else who can try a role of a mentor?
>>> 
>>> —
>>> Denis
>>> 
 On Dec 19, 2016, at 8:15 PM, Alexey Kuznetsov 
>>> wrote:
 
 Igniters!
 
 What do you think about participating in Google Summer Of Code 2017?
 See: https://developers.google.com/open-source/gsoc
 
 The Google Summer of Code, often abbreviated to GSoC, is an
>> international
 annual program, first held from May to August 2005, in which Google
>>> awards
 stipends (of US$5,500, as of 2016 ) to all students who successfully
 complete a requested free and open-source software coding project
>> during
 the summer.
 
 Students will work on some issues in Ignite.
 Ignite community will provide mentors for those students and answer
 students questions on dev list.
 Google will pay students. :)
 
 We need to prepare list of tasks that will be useful for Ignite and
>> could
 be finished in three months by students.
 
 What do you think?
 --
 Alexey Kuznetsov
>>> 
>>> 
>> 

Re: Document issue

[Denis Magda]

Hi,

Do be honest I don’t see any issue there. Apache Ignite images for AWS and GCE 
run inside of a docker container. So the latest docker related commands look 
valid for me. Nick, please confirm.

—
Denis

> On Dec 22, 2016, at 8:10 AM, 李玉珏@163 <18624049...@163.com> wrote:
> 
> Hi，
> 
> The following two pages of documents, more errors, I hope the community can 
> be a comprehensive review, and then fix it.
> 
> https://apacheignite.readme.io/docs/aws-deployment
> 
> https://apacheignite.readme.io/docs/google-compute-deployment
> 
> 
> For example, under the two page, are all about docker information.
> 
> 

[jira] [Created] (IGNITE-4484) Call commit for read-only transactions.

[Alexei Scherbakov (JIRA)]

Alexei Scherbakov created IGNITE-4484:
-

 Summary: Call commit for read-only transactions.
 Key: IGNITE-4484
 URL: https://issues.apache.org/jira/browse/IGNITE-4484
 Project: Ignite
  Issue Type: Improvement
  Components: cache
Affects Versions: 1.8
Reporter: Alexei Scherbakov
 Fix For: 1.9


IGNITE-3601 introduces a behavior, which is not compliant with tx isolation 
semantics.

Read-only transaction doesn't throw optimisitc exception in case if entry 
version were changes by other writing tx.

With the appearance of IGNITE-4285 this is no longer needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] ignite pull request #1378: IGNITE-1443: Minor fix.

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1378


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Document issue

[李玉珏]

Hi，

The following two pages of documents, more errors, I hope the community 
can be a comprehensive review, and then fix it.


https://apacheignite.readme.io/docs/aws-deployment

https://apacheignite.readme.io/docs/google-compute-deployment


For example, under the two page, are all about docker information.

[GitHub] ignite pull request #1270: IGNITE-4063 BinaryType.fieldNames() should return...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1270


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1323: IGNITE-4379: SQL: Local SQL field query broken in...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1323


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1336: IGNITE-4264: Offheap cache entries statistics is ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1336


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1379: gg 8.0.2.ea1

[ilantukh]

GitHub user ilantukh opened a pull request:

https://github.com/apache/ignite/pull/1379

gg 8.0.2.ea1



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-8.0.2.ea1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1379.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1379


commit 169c4d2053701532ce2ad1256033942ef1bd9693
Author: Alexey Goncharuk 
Date:   2016-10-14T12:29:24Z

Merge ignite-gg-11510 into 8.0.0.ea4 - Removed unused method

commit eb8bafb98a88e410575c184acf33bf6106ffd70d
Author: Alexey Goncharuk 
Date:   2016-10-14T12:33:49Z

Merge gg-11606 into ignite-gg-8.0.0.ea4

commit 3a3ef8f23928bffde6ab9ef78a7f88a0f5d7b161
Author: Alexey Goncharuk 
Date:   2016-10-14T12:33:53Z

Merge gg-11606 into ignite-gg-8.0.0.ea4

commit 0cb1f10b0b85d63697a333d89c195c70aed5868d
Author: Alexey Goncharuk 
Date:   2016-10-17T08:59:55Z

Merge ignite-gg-11611 into ignite-gg-8.0.0.ea4

commit 63fa77aa138f87b2d2cc900cde29ae276459d202
Author: Alexey Goncharuk 
Date:   2016-10-17T09:00:25Z

Merge ignite-gg-11611 into ignite-gg-8.0.0.ea4

commit 2fb0910fae25ce8a44de03f6b3519d8ec2979af4
Author: EdShangGG 
Date:   2016-10-17T13:47:59Z

GG-11615 Incremental backups: Track changed pages for incremental backups
-fixes in tracking logic;

commit f8b507937aa8533cd4dd8a73efea413ed7fb4657
Author: Alexey Goncharuk 
Date:   2016-10-18T09:55:41Z

Merge ignite-gg-11533 into ignite-gg-8.0.0.ea4

commit a64004c368cdfaf6972981575b472846dab1e174
Author: Alexey Goncharuk 
Date:   2016-10-18T10:00:45Z

GG-11533 - Code review.

commit 258ac73a51ba6f7a758d6bb3a2734fecd9d428f4
Author: Ilya Lantukh 
Date:   2016-10-18T14:20:45Z

GG-11616 : WIP - backup ID generation and tracking.

commit 772909cc0690db0327997dcb86a6d7bd2bec9481
Author: Ilya Lantukh 
Date:   2016-10-18T15:27:18Z

GG-11616 : WIP - backup ID generation and tracking.

commit 6c1ee81a89ccaf336ef948cfd28bcffabeb7fc20
Author: EdShangGG 
Date:   2016-10-18T18:36:27Z

GG-11615 Incremental backups: Track changed pages for incremental backups
-pageId iterator
-test fixes

commit 4d8d78f06514129d3d38c9dca9456baa3b05b104
Author: EdShangGG 
Date:   2016-10-18T18:48:57Z

Merge remote-tracking branch 'ignite-gg/ignite-db-x' into 
ignite-db-x-backups

commit dc7df2e2ce4680b10c093e1ceb8b80a8ead3399e
Author: EdShangGG 
Date:   2016-10-18T18:56:23Z

GG-11615 Incremental backups: Track changed pages for incremental backups
-skip marking page as changed while restoration
-fix npe

commit 6bfa38370a5dff58baad7a47a0c43f769e858072
Author: EdShangGG 
Date:   2016-10-19T10:25:53Z

fix NPE

commit 1652559ef02f55fd98db60ad8e8700f5ec3aa324
Author: Alexey Goncharuk 
Date:   2016-10-19T10:58:47Z

GG-11625 - Fixed invalid page ID in pages list record

commit 5fa4402af7de644eeadfc1308940ef3317ba22cb
Author: Alexey Goncharuk 
Date:   2016-10-19T10:59:37Z

Merge branch 'ignite-gg-8.0.0.ea4' of 
https://github.com/gridgain/apache-ignite into ignite-db-x

commit b58cf3210a89ee22bca906f798f8aafa4a6629a7
Author: EdShangGG 
Date:   2016-10-19T13:33:15Z

GG-11615 Incremental backups: Track changed pages for incremental backups
-fixed issue with assert in wasChanged

commit 3592236e666d842e4ff346466458c8fc9394ffb9
Author: EdShangGG 
Date:   2016-10-19T16:34:25Z

GG-11615 Incremental backups: Track changed pages for incremental backups
-fixed issue with assert

commit 07e1df6bea30d33c4f0e9b3bc6a78385eb32d53b
Author: Ilya Lantukh 
Date:   2016-10-19T16:46:11Z

GG-11616 : Minors.

commit bab8e3efc9eb4ddc5e9b69bde5674e1182ce332a
Author: Ilya Lantukh 
Date:   2016-10-19T16:47:09Z

Merge remote-tracking branch 'community/ignite-db-x-backups' into 
ignite-db-x-backups

commit 81e6cb4f3331aaa05dd30c75f9de6e5ed28632f9
Author: EdShangGG 
Date:   2016-10-20T15:08:47Z

GG-11615 Incremental backups: Track changed pages for incremental backups
-fixed tag erasing

commit fbd5529460e924b48b5d93bf4e6d825443036f64
Author: Ilya Lantukh 
Date:   2016-10-20T15:31:46Z

GG-11616 : WIP - fault handling.

commit 6ed7521af5fa1a99ef6dc384e86cfa4821484c4e
Author: Ilya Lantukh 
Date:  

[jira] [Created] (IGNITE-4483) REST API - AtomicLong get/set/remove

[Roman Bielik (JIRA)]

Roman Bielik created IGNITE-4483:


 Summary: REST API - AtomicLong get/set/remove
 Key: IGNITE-4483
 URL: https://issues.apache.org/jira/browse/IGNITE-4483
 Project: Ignite
  Issue Type: Improvement
  Components: rest
Affects Versions: 1.8
Reporter: Roman Bielik


The Ignite REST API lacks operations for setting and deleting a counter (atomic 
long). It would be also nice to have a pure get operation, but there is at 
least a workaround in using increment/decrement with delta = 0.

IgniteAtomicLong.get()
IgniteAtomicLong.getAndSet()
IgniteAtomicLong.close()




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Sort nodes in the ring in order to minimize the number of reconnections

[Vyacheslav Daradur]

Hello, Alex!

I think it is a great idea.

I suggest to build communications between nodes on weight (or priority).

For example, ordering on latency:
- nodes on one host = 1
- nodes in one rack-blade = 2
- nodes in one server-rack = 3
- nodes in one physical cluster = 4
- nodes in one subnet = 5
- etc.

Maybe it'll be better to use some metrics from ClusterMetrics interface.

The algorithm of ordering can be implemented in a class such as Comparator
and use it when we build a cluster or we select a place for a new node.

--
With best regards,
Vyacheslav Daradur

2016-12-22 13:59 GMT+03:00 Александр Меньшиков :

> Hello everyone,
>
> As far as I know nodes are connected in a ring. For example if i have 6
> nodes, with names A, B, C, D, E, and F they can connect in ring any
> possible way: A-B-C-D-E-F-A, or A-F-B-E-C-D-A, etc. And if some node falls
> out of topology neighboring nodes must reconnect. If nodes A,B and C
> located in the same physical location, and D, E and F in another, and in
> some time one physical location is not available in another, we can get
> different number of reconnections. Best case scenario if we have ring like
> A-B-CxD-E-FxA ('x' mean disconnect) -- then we get only one reconnect (C
> reconnect to A or F reconnect to D -- depending on what part of the cluster
> we leave alive). But now possible that case AxFxBxExCxDxA -- then we get a
> lot of reconnections (A to B, B to C, C to A -- in general n/2
> reconnections, where n -- number of nodes). And i think to add something to
> ensure that we always have good sorting of nodes connections
> (A-B-C-...-Z-A).
>
> Of course in real world we can have multiple levels of physical closeness.
>
> In my opinion enough to add one parameter of 'int' to configuration (with
> name like 'ExtraNodeOrder') and to change the method of comparison nodes so
> that it first compared the 'ExtraNodeOrder', and then according to the old
> criterion (as far as I know Ignite use topology version). So if some users
> have multiple levels of physical closeness, he can use different bits. For
> example use 16 high bits for DC number, and low 16 bits for racks.
>
> Alternatively, we can add array of ‘int’ to configuration and compare nodes
> in sequence from the zero element to the last.
>

[GitHub] ignite pull request #1378: IGNITE-1443: Minor fix.

[isapego]

GitHub user isapego opened a pull request:

https://github.com/apache/ignite/pull/1378

IGNITE-1443: Minor fix.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-1443

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1378.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1378


commit 7fe666b2942ffb04ad8554de2ad3268b405d82d9
Author: Igor Sapego 
Date:   2016-12-22T14:19:45Z

IGNITE-1443: Minor fix.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1377: IGNITE-4471: Fixed ODBC string-reading utility fu...

[isapego]

GitHub user isapego opened a pull request:

https://github.com/apache/ignite/pull/1377

IGNITE-4471: Fixed ODBC string-reading utility function.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4471

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1377.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1377


commit 95d417d570af82d05119ee6be193b10106e99614
Author: Igor Sapego 
Date:   2016-12-22T13:34:10Z

IGNITE-4471: Fixed ODBC string-reading utility function.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1376: Ignite gg 8.0.1.ea7 opt

[devozerov]

GitHub user devozerov opened a pull request:

https://github.com/apache/ignite/pull/1376

Ignite gg 8.0.1.ea7 opt



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-8.0.1.ea7-opt

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1376.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1376


commit d574038a4694d3a7273797696af4e1b712f8cfbc
Author: Alexey Goncharuk 
Date:   2016-09-15T07:12:17Z

Merge branch 'ignite-db-x' of https://github.com/gridgain/apache-ignite 
into ignite-gg-8.0.0.ea1

commit 36f4c862c905dfbd814eeab0a7182c50cc2970a0
Author: Alexey Goncharuk 
Date:   2016-09-15T07:15:58Z

Merge branch 'ignite-gg-8.0.0.ea1' of 
https://github.com/gridgain/apache-ignite into ignite-gg-11427

commit a1af356acab214f6bf10ce27228dfb2d339f1ce7
Author: Alexey Goncharuk 
Date:   2016-09-15T08:04:47Z

Code review WIP

commit aa7d28222563adbac0ddd246745e5e14e62d886e
Author: Alexey Goncharuk 
Date:   2016-09-15T08:13:38Z

EA1 - Fixed NPE

commit e9ca1474ba23457b4fc7cd0951632b64c3037987
Author: sboikov 
Date:   2016-09-15T09:40:00Z

GG-11386 Change data structure in ReuseList (merge with new FreeList)

commit 031edc84fed21d9262f9997121f75fbaabc50db8
Author: Alexey Goncharuk 
Date:   2016-09-15T12:06:06Z

EA1 - Fixes and verbose exceptions.

commit b747e9444d6e8bf43c0e9419c7f938280413f0e0
Author: Alexey Goncharuk 
Date:   2016-09-15T12:31:43Z

Merge ignite-db-x into ignite-gg-8.0.0.ea1

commit 49b72a60be52c3d89e9653bbb160560b00674f33
Author: Alexey Goncharuk 
Date:   2016-09-15T12:31:49Z

Merge ignite-db-x into ignite-gg-8.0.0.ea1

commit be990b13c8dfa05f873b6a6a408017baae38f24c
Author: EdShangGG 
Date:   2016-09-15T12:47:52Z

GG-11421 evictPage in PageMemoryImpl.Segment can fall in infinitive loop

commit 4f6da47402ecd8c53a87539126a55a5552877cc5
Author: Ilya Lantukh 
Date:   2016-09-15T14:16:22Z

Fixed GridCacheEntryMemorySizeSelfTest.

commit 8a97480637ecb76d6a03d7adce0f3069b8f69433
Author: Alexey Goncharuk 
Date:   2016-09-15T14:47:06Z

Merge ignite-gg-8.0.0.ea1 into ignite-gg-11427

commit fab881145246a85a18210a2418f17fdc22ce9fd3
Author: Ilya Lantukh 
Date:   2016-09-15T15:36:09Z

Muted test.

commit 6c8b647b76ece036350f45c505a08b9b162b0a25
Author: Ilya Lantukh 
Date:   2016-09-15T15:46:08Z

Fixed local cache size calculation.

commit a07ac3a8a447a420a32fc8c5a1d4d4fdc71313d9
Author: Alexey Goncharuk 
Date:   2016-09-15T16:00:36Z

Merge ignite-gg-8.0.0.ea1 into ignite-gg-11427

commit 00c9d150c9d80d936d5463f05a68d028c87a6632
Author: Ilya Lantukh 
Date:   2016-09-15T16:08:53Z

Removed test that is no longer relevant.

commit 346d1f2b91011e0348f119f20409590e22d36ddc
Author: Ilya Lantukh 
Date:   2016-09-15T16:17:28Z

Fixed GridCacheAtomicNearCacheSelfTest.

commit 7c24c87aa14cc53df134a24a9368fb5bb6509a17
Author: Ilya Lantukh 
Date:   2016-09-15T17:20:30Z

Added IndexingSpi#remove(...) call on removing non-existing entry to fix 
IgniteTxExceptionAbstractSelfTest.

commit eed389fea4d846ceb390e3ad43fb860034be41da
Author: Sergi Vladykin 
Date:   2016-09-15T18:36:04Z

ignite-db-x - validateTail

commit 491068f0dd47ef1545b27224ba516f61554fe1d6
Author: Sergi Vladykin 
Date:   2016-09-16T08:24:24Z

ignite-db-x - fixed concurrent remove

commit 43bf3b7ae64f031522b8d6997d4e3d0e8970906f
Author: Sergi Vladykin 
Date:   2016-09-16T08:25:50Z

Merge branch 'ignite-db-x' of https://github.com/gridgain/apache-ignite 
into ignite-db-x

commit a47283149b2ad824a2d10562002626526b83d1a0
Author: Alexey Goncharuk 
Date:   2016-09-16T08:45:29Z

GG-11427 - Fixing WIP

commit 04cb5377bfe1cbc50b08cfcbaf35b3efe67b7414
Author: Alexey Goncharuk 
Date:   2016-09-16T08:58:55Z

DB-X - Fixed Java 1.7 compilation

commit 3094cb88fbc28857090cc8863b1260f8f58c31ef
Author: Sergi Vladykin 
Date:   2016-09-16T09:09:25Z

Merge branch 'ignite-db-x' of https://github.com/gridgain/apache-ignite 
into ignite-db-x

commit dce51244c1d6bcb4516fe780769b6922c9ca3ea7
Author: Sergi Vladykin 
Date:   2016-09-16T09:11:02Z

ignite-db-x - more keys for remove

commit 

[jira] [Created] (IGNITE-4481) Part of IGNITE-4212

[Oleg Ostanin (JIRA)]

Oleg Ostanin created IGNITE-4481:


 Summary: Part of IGNITE-4212
 Key: IGNITE-4481
 URL: https://issues.apache.org/jira/browse/IGNITE-4481
 Project: Ignite
  Issue Type: Sub-task
Reporter: Oleg Ostanin
Assignee: Oleg Ostanin


The goal of this subtask is to resolve this part of original task:

Every deliverable must contain an executable (bat or sh file) with a clear 
instruction on how to start a specific benchmark from the console.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Make async API great again

[Vladimir Ozerov]

And several more improvements to future API:
1) Remove startTime() and duration() methods:
https://issues.apache.org/jira/browse/IGNITE-4479
2) Fix broken cancellation semantics:
https://issues.apache.org/jira/browse/IGNITE-4480

On Thu, Dec 22, 2016 at 1:40 PM, Vladimir Ozerov 
wrote:

> Gents,
>
> I created tickets for all proposed improvements:
> 1) Nice async API: https://issues.apache.org/jira/browse/IGNITE-4475
> 2) Do not process IO messages synchronously for local node:
> https://issues.apache.org/jira/browse/IGNITE-4476
> 3) Better IgniteFuture API and callback semantics: https://issues.
> apache.org/jira/browse/IGNITE-4477
>
> Please review it and let me know if you have any comments.
>
> Vladimir.
>
> On Wed, Dec 21, 2016 at 4:32 AM, Dmitriy Setrakyan 
> wrote:
>
>> Would be nice if someone would prototype a new cache API and post the
>> generated javadoc here. I think we all will benefit from reviewing it.
>>
>> On Tue, Dec 20, 2016 at 12:17 PM, Vladimir Ozerov 
>> wrote:
>>
>> > Async API rework is mechanical addition of ~100 methods through
>> copy-paste.
>> > Should not take more than a day to implement and more than another day
>> to
>> > rework tests.
>> >
>> > On Tue, Dec 20, 2016 at 10:00 PM, Dmitriy Setrakyan <
>> dsetrak...@apache.org
>> > >
>> > wrote:
>> >
>> > > How difficult is this change? Does not look like it can be done
>> > overnight.
>> > >
>> > > On Tue, Dec 20, 2016 at 10:46 AM, Vladimir Ozerov <
>> voze...@gridgain.com>
>> > > wrote:
>> > >
>> > > > We already discussed this several months ago in other thread.
>> > > >
>> > > > "Async" methods is the most simple and straight API possible. .NET
>> > world
>> > > > goes this way all over their frameworks and nobody died. Hazelcast
>> also
>> > > > goes this way. Java goes this way (see CompletableFuture). This is
>> > common
>> > > > and well-known practice. The most impacted part of our API will be
>> > cache,
>> > > > +33 new methods. Though, I do not see how it can affect learning
>> curve.
>> > > >
>> > > > Agree that we should deprecate AsyncSupport gradually and remove it
>> no
>> > > > earlier than in Apache Ignite 3.0.
>> > > >
>> > > > On Tue, Dec 20, 2016 at 9:31 PM, Dmitriy Setrakyan <
>> > > dsetrak...@apache.org>
>> > > > wrote:
>> > > >
>> > > > > On Tue, Dec 20, 2016 at 10:28 AM, Sergi Vladykin <
>> > > > sergi.vlady...@gmail.com
>> > > > > >
>> > > > > wrote:
>> > > > >
>> > > > > > +1 For removing withAsync. It is a broken design.
>> > > > > >
>> > > > >
>> > > > > Sergi, do you also want to add all the async methods to the main
>> API
>> > or
>> > > > do
>> > > > > you have some other design in mind?
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

[jira] [Created] (IGNITE-4480) Incorrect cancellation semantics in IgniteFuture.

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-4480:
---

 Summary: Incorrect cancellation semantics in IgniteFuture.
 Key: IGNITE-4480
 URL: https://issues.apache.org/jira/browse/IGNITE-4480
 Project: Ignite
  Issue Type: Task
  Components: general
Affects Versions: 1.8
Reporter: Vladimir Ozerov
 Fix For: 2.0


*Problem*
Normally, if user invoke "cancel()" on future, he expects that it will be 
completed immediately. E.g. this is how JDK {{CompletableFuture}} works. 
However, this is not the case for Ignite - we inform user about cancellation 
through flag, which is also not set properly in most cases.

*Solution*
1) {{cancel()}} must complete future with special "CancellationException" 
immediately.
2) {{isCancelled()}} method should be either removed or deprecated.
3{ We should add {{isCompletedExceptionally()}} method which will return 
{{true}} in case future was completed with any kind of exception. This will 
work similar to JDK {{CompletableFuture}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4479) Remove startTime() and duration() methods from IgniteFuture

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-4479:
---

 Summary: Remove startTime() and duration() methods from 
IgniteFuture
 Key: IGNITE-4479
 URL: https://issues.apache.org/jira/browse/IGNITE-4479
 Project: Ignite
  Issue Type: Task
  Components: general
Affects Versions: 1.8
Reporter: Vladimir Ozerov
 Fix For: 2.0


These methods are of no use for users and should not be in future API. THe only 
place where they are used is cache exchange manager. 

We should remove these methods form all futures and leave them only for cache 
exchange logic.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Sort nodes in the ring in order to minimize the number of reconnections

[Александр Меньшиков]

Hello everyone,

As far as I know nodes are connected in a ring. For example if i have 6
nodes, with names A, B, C, D, E, and F they can connect in ring any
possible way: A-B-C-D-E-F-A, or A-F-B-E-C-D-A, etc. And if some node falls
out of topology neighboring nodes must reconnect. If nodes A,B and C
located in the same physical location, and D, E and F in another, and in
some time one physical location is not available in another, we can get
different number of reconnections. Best case scenario if we have ring like
A-B-CxD-E-FxA ('x' mean disconnect) -- then we get only one reconnect (C
reconnect to A or F reconnect to D -- depending on what part of the cluster
we leave alive). But now possible that case AxFxBxExCxDxA -- then we get a
lot of reconnections (A to B, B to C, C to A -- in general n/2
reconnections, where n -- number of nodes). And i think to add something to
ensure that we always have good sorting of nodes connections
(A-B-C-...-Z-A).

Of course in real world we can have multiple levels of physical closeness.

In my opinion enough to add one parameter of 'int' to configuration (with
name like 'ExtraNodeOrder') and to change the method of comparison nodes so
that it first compared the 'ExtraNodeOrder', and then according to the old
criterion (as far as I know Ignite use topology version). So if some users
have multiple levels of physical closeness, he can use different bits. For
example use 16 high bits for DC number, and low 16 bits for racks.

Alternatively, we can add array of ‘int’ to configuration and compare nodes
in sequence from the zero element to the last.

[jira] [Created] (IGNITE-4478) Part of IGNITE-4212

[Oleg Ostanin (JIRA)]

Oleg Ostanin created IGNITE-4478:


 Summary: Part of IGNITE-4212
 Key: IGNITE-4478
 URL: https://issues.apache.org/jira/browse/IGNITE-4478
 Project: Ignite
  Issue Type: Sub-task
Reporter: Oleg Ostanin
Assignee: Oleg Ostanin


The goal of this subtask is to resolve the first part of original task:

Ignite-yardstick benchmarks have to be pre-compiled and available with every 
Ignite deliverable.

We need to create "benchmarks" folder and put it under the root of an Apache 
Ignite distribution. The structure of this folder is shown below.
apache_ignite_root_folder
– bin
– examples
...
– benchmarks
— bin (contains scripts needed to run the benchmarks)
— src (contains benchmarks sources and a pom.xml in order to create a project 
in an IDE instantaneously)
— config (configs for amazon and well as for local benchmarks execution)
— libs (compiled and ready-to-run benchmarks)
— README.txt (instructions how to run the benchmarks)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Make async API great again

[Vladimir Ozerov]

Gents,

I created tickets for all proposed improvements:
1) Nice async API: https://issues.apache.org/jira/browse/IGNITE-4475
2) Do not process IO messages synchronously for local node:
https://issues.apache.org/jira/browse/IGNITE-4476
3) Better IgniteFuture API and callback semantics:
https://issues.apache.org/jira/browse/IGNITE-4477

Please review it and let me know if you have any comments.

Vladimir.

On Wed, Dec 21, 2016 at 4:32 AM, Dmitriy Setrakyan 
wrote:

> Would be nice if someone would prototype a new cache API and post the
> generated javadoc here. I think we all will benefit from reviewing it.
>
> On Tue, Dec 20, 2016 at 12:17 PM, Vladimir Ozerov 
> wrote:
>
> > Async API rework is mechanical addition of ~100 methods through
> copy-paste.
> > Should not take more than a day to implement and more than another day to
> > rework tests.
> >
> > On Tue, Dec 20, 2016 at 10:00 PM, Dmitriy Setrakyan <
> dsetrak...@apache.org
> > >
> > wrote:
> >
> > > How difficult is this change? Does not look like it can be done
> > overnight.
> > >
> > > On Tue, Dec 20, 2016 at 10:46 AM, Vladimir Ozerov <
> voze...@gridgain.com>
> > > wrote:
> > >
> > > > We already discussed this several months ago in other thread.
> > > >
> > > > "Async" methods is the most simple and straight API possible. .NET
> > world
> > > > goes this way all over their frameworks and nobody died. Hazelcast
> also
> > > > goes this way. Java goes this way (see CompletableFuture). This is
> > common
> > > > and well-known practice. The most impacted part of our API will be
> > cache,
> > > > +33 new methods. Though, I do not see how it can affect learning
> curve.
> > > >
> > > > Agree that we should deprecate AsyncSupport gradually and remove it
> no
> > > > earlier than in Apache Ignite 3.0.
> > > >
> > > > On Tue, Dec 20, 2016 at 9:31 PM, Dmitriy Setrakyan <
> > > dsetrak...@apache.org>
> > > > wrote:
> > > >
> > > > > On Tue, Dec 20, 2016 at 10:28 AM, Sergi Vladykin <
> > > > sergi.vlady...@gmail.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > +1 For removing withAsync. It is a broken design.
> > > > > >
> > > > >
> > > > > Sergi, do you also want to add all the async methods to the main
> API
> > or
> > > > do
> > > > > you have some other design in mind?
> > > > >
> > > >
> > >
> >
>

[jira] [Created] (IGNITE-4477) Fix IgniteFuture.listen() and IgniteFuture.chain() semantics

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-4477:
---

 Summary: Fix IgniteFuture.listen() and IgniteFuture.chain() 
semantics
 Key: IGNITE-4477
 URL: https://issues.apache.org/jira/browse/IGNITE-4477
 Project: Ignite
  Issue Type: Task
  Components: general
Affects Versions: 1.8
Reporter: Vladimir Ozerov
 Fix For: 2.0


*Problem*
We allow users to pass continuations to {{IgniteFuture}} which will be executed 
on future completion. This can be done through {{listen}} or {{chain}} methods.
However, continuation semantics is broken intrinsically:
1) If future is already completed, user code executed in the same thread;
2) If future is not completed yet, it will be executed in completion thread.
Neither of this options are valid because it easily leads to starvation. E.g.:
{code}
IgniteFuture fut = cache.getAsync(key2);

fut.listen(fut0 -> {
 cache.put(key2, val2); // Possible deadlock, because invoked in sys pool;
});
{code}

*Solution*
1) By default callbacks must be executed asynchronously in some common pool 
(public pool? new "callback pool"? FJP?)
2) It should be possible to specify where to execute a callback explicitly:
{code}
IgniteFuture.listen(IgniteClosure, ExecutorService);
{code}
3) We may want to expose our public pool on API for convenience.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4476) GridIoManger must always process messages asynchronously

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-4476:
---

 Summary: GridIoManger must always process messages asynchronously
 Key: IGNITE-4476
 URL: https://issues.apache.org/jira/browse/IGNITE-4476
 Project: Ignite
  Issue Type: Task
  Components: general
Affects Versions: 1.8
Reporter: Vladimir Ozerov
 Fix For: 2.0


*Problem*
If message is to be sent to remote node, we just send it (surprise :-)). But if 
message is to be sent to local node, we have a strange "optimization" - we 
process it synchronously in the same thread. 
This is wrong as it easily leads to all kind of weird starvations and deadlocks.

*Solution*
Never ever process messages synchronously. For local node we should put message 
runnable into appropriate thread pool and exit.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4475) Simplify async API

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-4475:
---

 Summary: Simplify async API
 Key: IGNITE-4475
 URL: https://issues.apache.org/jira/browse/IGNITE-4475
 Project: Ignite
  Issue Type: Task
  Components: general
Affects Versions: 1.8
Reporter: Vladimir Ozerov
 Fix For: 2.0


*Problem*
We need to simplify our async API. It is to complex and verbose at the moment:
{code}
IgniteCache asyncCache = cache.withAsync();
asyncCache.get(key);
IgniteFuture fut = asyncCache.future();
{code}

*Proposed solution*
1) Deprecate {{IgniteAsyncSupport}} interface.
2) Make async operations more straightforward:
{code}
IgniteFuture fut = cache.getAsync(key);
{code}

*Scope*
~80 async methods in all public interfaces.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4474) Ignite.createCache hangs on exception in CachePluginConfiguration.createProvider

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-4474:
--

 Summary: Ignite.createCache hangs on exception in 
CachePluginConfiguration.createProvider
 Key: IGNITE-4474
 URL: https://issues.apache.org/jira/browse/IGNITE-4474
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 1.8
Reporter: Pavel Tupitsyn
Priority: Minor
 Fix For: 2.0


When there is an exception during provider initialization, {{createCache}} 
hangs.

{code}
public class CacheCfg implements CachePluginConfiguration {
@Override public CachePluginProvider createProvider(CachePluginContext ctx) 
{
throw new IgniteException("foo");
}
}
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4473) Client should re-try connection attempt in case of concurrent network failure

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-4473:
---

 Summary: Client should re-try connection attempt in case of 
concurrent network failure
 Key: IGNITE-4473
 URL: https://issues.apache.org/jira/browse/IGNITE-4473
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 1.8
Reporter: Vladimir Ozerov
 Fix For: 2.0


*Problem*
Consider the following scenario:
1) Client started, but there are no servers, so it hangs somewhere inside start 
routine.
2) Server appears, and discovery finishes successfully.
3) Nodes start talking to each other through communication SPI to finish start 
process (e.g. to complete exchange).
4) But network glitch occurs and server becomes unreachable.

*Expected behavior*
Client disconnects and hangs waiting for reconnect.

*Actual behavior*
Client throws an exception and never tries to reconnect.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] ignite pull request #1374: Ignite 1.7.5

[devozerov]

GitHub user devozerov opened a pull request:

https://github.com/apache/ignite/pull/1374

Ignite 1.7.5



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-1.7.5

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1374.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1374


commit 671a77a2d81cac401765dddf25f30fba4e4ab17f
Author: sboikov 
Date:   2016-12-08T09:56:46Z

ignite-4154 Fixed node version check for compression feature usage

commit 391f4be4c687a7f325aeec8b727c9c85ca003454
Author: agura 
Date:   2016-12-07T17:11:50Z

ignite-2358 toString() method for cache store implementations

commit bc977d3211906ef94e1f7d3f0f988efbed65034f
Author: Alexey Kuznetsov 
Date:   2016-12-09T09:11:31Z

IGNITE-4350 Reworked JdbcTypesDefaultTransformed logic. Improved improved 
error messages in CacheJdbcPojoStore.

commit b83ec8e57c7c48f2baa4780cf3b2e46df075f3df
Author: sboikov 
Date:   2016-12-09T11:32:42Z

IGNITE-500 CacheLoadingConcurrentGridStartSelfTest fails: prevent 
'localUpdate' execution while top read lock is held.

commit 6e485637e2738a7e809eac1a802f0964dc12383d
Author: Andrey V. Mashenkov 
Date:   2016-12-09T12:42:40Z

IGNITE-4379: Fixed local query execution. This closes #1323.

commit 6fd8bf6338470275e687a686044c7d935d3714ff
Author: Andrey V. Mashenkov 
Date:   2016-12-07T15:49:06Z

Fixed tests for IGNITE-4379 commit.

commit c143bc1a77baa13f61d6ba00509fa1fcb33757b1
Author: tledkov-gridgain 
Date:   2016-12-09T13:05:03Z

IGNITE-4063:  Preserved order of fields in the metadata according with 
schema. This closes #1270.

commit 6616a878598b8ed74b71558290a22b266a8ac03e
Author: devozerov 
Date:   2016-12-09T13:05:24Z

Merge remote-tracking branch 'upstream/ignite-1.7.4' into ignite-1.7.4

commit f087b661c466c9fd27d049efa8e55a2d7bdf9834
Author: Andrey V. Mashenkov 
Date:   2016-12-09T14:45:33Z

IGNITE-4264: Fixed cluster wide cache metrics. This closes #1336.

commit ceb60d27d38782d82ec5275a56a6f77851b6
Author: Andrey V. Mashenkov 
Date:   2016-12-09T14:46:21Z

Merge remote-tracking branch 'origin/ignite-1.7.4' into ignite-1.7.4

commit 6d744db27111c68e13b06ec99428a4c4148d97b6
Author: dkarachentsev 
Date:   2016-12-12T08:44:57Z

IGNITE-4231 - Hangs on compute result serialization error. Fix

commit eeb2f2a38fb792b711eb665e380d14bc00f6e078
Author: dkarachentsev 
Date:   2016-12-12T09:14:01Z

IGNITE-3862 - GridServiceProxy invocation never times out. Fix

commit 8dd4ada6287af49a243aba1e58daffa7e44f39a9
Author: sboikov 
Date:   2016-12-12T11:56:44Z

GridClosureProcessor: fixed javadoc.

commit 83d961cff88cf2ead0bbc4ded3285f4faf9157fc
Author: Pavel Tupitsyn 
Date:   2016-12-12T14:52:22Z

IGNITE-4413 .NET: Fix DateTime argument handling in SqlQuery

This closes #1341

# Conflicts:
#   
modules/platforms/dotnet/Apache.Ignite.Core.Tests/Cache/Query/CacheQueriesCodeConfigurationTest.cs

commit c103ac33d590492577bb687b29969550cdb532cf
Author: Andrey V. Mashenkov 
Date:   2016-12-21T13:53:18Z

IGNITE-4248: Fixed: Some loggers ignore IGNITE_QUIET system property. This 
closes #1367.

commit 708cc8c6849b21063a555895671f6f820d92184a
Author: Andrey V. Mashenkov 
Date:   2016-12-22T09:48:58Z

IGNITE-4408: Allow BinaryObjects pass to IndexingSpi. This closes #1353.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1367: IGNITE-4248: Slf4jLogger ignores IGNITE_QUIET sys...

[AMashenkov]

Github user AMashenkov closed the pull request at:

https://github.com/apache/ignite/pull/1367


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1373: Ignite 4109

[dkarachentsev]

GitHub user dkarachentsev opened a pull request:

https://github.com/apache/ignite/pull/1373

Ignite 4109



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4109

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1373.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1373


commit 2bfa06dd75fa55ad01438fa59d14b864ec95834e
Author: Alexey Kuznetsov 
Date:   2016-09-28T01:22:40Z

Fixed typo.

commit c188c3c4a96eacb85ea8e08f0634288332432c1c
Author: Alexey Kuznetsov 
Date:   2016-09-28T01:46:23Z

IGNITE-3983 Fixed wrong cache load optimization. Test added.

commit 89c30c8b0be6915d2399be508ddcd9eb439a9aaa
Author: Alexey Kuznetsov 
Date:   2016-09-28T01:57:45Z

IGNITE-3965 @GridInternal tasks should run via standart LoadBalancingSpi. 
Added test.

commit a53c399e10926120106379d1c764edd7d3854e6a
Author: Alexey Kuznetsov 
Date:   2016-09-28T02:07:55Z

Merge ignite-1.6.9 into ignite-1.7.2.

commit ec9ddcd3d99d19403bf19e1172ede2afdab6c86f
Author: sboikov 
Date:   2016-09-28T09:05:28Z

Code style fixes.

commit 17c2fc0b69abd023b2a1e5da344e67951fd49408
Author: sboikov 
Date:   2016-09-28T09:56:17Z

ignite-2833 Need call 'touch' for cache entry if it was obtained using 
'entryEx'.

commit daf974d261efa525678d5fabc6191642c07f9ad4
Author: AKuznetsov 
Date:   2016-09-28T10:22:10Z

IGNITE-3965 Fixed issues found on review.

commit 53fbad7ddafdae7b368b0f207d06d16574978d62
Author: AKuznetsov 
Date:   2016-09-28T10:24:56Z

Merge branch ignite-1.6.9 into ignite-1.7.2.

commit 4ff19c20b169e0373eafc8025a838db8bfc61f27
Author: sboikov 
Date:   2016-09-28T10:47:51Z

ignite-3621 Fixed 'testEvictExpired'.

commit bfe4458448a59542713830f57713b3ac2af08e2b
Author: sboikov 
Date:   2016-09-28T11:31:24Z

ignite-3621 Fixed 'testEvictExpired'.

commit d643dcf2dd2caac4c3ff04cb72f31bbfbf97339a
Author: Pavel Tupitsyn 
Date:   2016-09-28T11:34:23Z

IGNITE-3970 .NET: Fix Cyrillic 'C' letters in code - add test

commit 474ade276c4ae3e8f93cce37473d37270b4e7ad9
Author: vozerov-gridgain 
Date:   2016-09-28T11:38:04Z

IGNITE-3988: Moved failing cloud tests to ignore module.

commit ccfaaf8d060ef984678d2376b16b5a17e7c17e9d
Author: vozerov-gridgain 
Date:   2016-09-28T11:38:17Z

Merge remote-tracking branch 'upstream/ignite-1.6.9' into ignite-1.6.9

commit c7fa918c10d771efa91cde1017662c26d0a61085
Author: Pavel Tupitsyn 
Date:   2016-09-28T11:47:17Z

Merge remote-tracking branch 'remotes/community/ignite-1.6.9' into 
ignite-1.7.3

commit b9105df77cc70b532f94899c754fba47e3e05e9a
Author: vozerov-gridgain 
Date:   2016-09-28T11:53:20Z

IGNITE-3989: Moved failing JTA tests to ignore module.

commit d595345765db2151ff432beecd478ce056393593
Author: vozerov-gridgain 
Date:   2016-09-28T12:08:38Z

IGNITE-3990: Moved failing Spring tests to "ignore" module.

commit e3f13455d4273e615727d0410783e3719db98f76
Author: sboikov 
Date:   2016-09-28T09:56:17Z

ignite-2833 Need call 'touch' for cache entry if it was obtained using 
'entryEx'.

(cherry picked from commit 17c2fc0)

commit b2faa339acb2eea24e6dd5e0c21fc3d3d0592ff6
Author: sboikov 
Date:   2016-09-28T10:47:51Z

ignite-3621 Fixed 'testEvictExpired'.

(cherry picked from commit 4ff19c2)

commit 74d2fc2416b8e6bc0598152552021f984a013061
Author: sboikov 
Date:   2016-09-28T11:31:24Z

ignite-3621 Fixed 'testEvictExpired'.

(cherry picked from commit bfe4458)

commit d2563dacceea61b19bb6e083e29ebacc28fdd323
Author: vozerov-gridgain 
Date:   2016-09-28T12:51:55Z

IGNITE-3993: Added failing client tests to "ignored" test suite.

commit 33d34941390c00e8d6a2488e8f2e11e6abba8a01
Author: sboikov 
Date:   2016-09-28T12:54:52Z

Merge remote-tracking branch 'community/ignite-1.6.9' into ignite-1.6.9

commit 78144c4c9d6200ceef8b666a186039685f053381
Author: vozerov-gridgain 
Date:   2016-09-28T13:52:13Z

Fixed incorrect test count calculation leading to afterTestsStopped() not 
being called.

commit e3dfdecc3607b5f3183bfcb1ce36c57543a8965f
Author: Alexander Paschenko 
Date:   2016-09-28T13:46:46Z

Validate hash code presence in BinaryObject. Fixes #928

commit 53229e290f7d6aab9b504693bd2b93155ecd2bad
Author: Pavel Tupitsyn