Re: moving to geronimo JCache jar

[Alexander Fedotov]

Hi all,

https://issues.apache.org/jira/browse/IGNITE-3793 is completed.
Kindly take a look at the corresponding PR https://github.com/apache/
ignite/pull/1475 .

On Wed, Jan 25, 2017 at 8:04 PM, Denis Magda  wrote:

> We need to replace content of ignite-core-licenses.txt file which is the
> following at the moment
>
> // --
> // List of ignite-core module's dependencies provided as a part of this
> distribution
> // which licenses differ from Apache Software License.
> // --
>
> 
> ==
> For JSR107 API and SPI (https://github.com/jsr107/jsr107spec)
> javax.cache:cache-api:jar:1.0.0
> 
> ==
> This product bundles JSR107 API and SPI which is available under a:
> JSR-000107 JCACHE 2.9 Public Review - Updated Specification License. For
> details, see https://raw.github.com/jsr107/jsr107spec/master/LICENSE.txt.
>
>
> Updated this ticket description: https://issues.apache.org/
> jira/browse/IGNITE-3793
>
> —
> Denis
> > On Jan 24, 2017, at 8:24 PM, Dmitriy Setrakyan 
> wrote:
> >
> > Awesome, you are right. I just checked and the license is indeed Apache
> > 2.0. Is there anything we need to do at all right now?
> >
> > On Tue, Jan 24, 2017 at 8:17 PM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> >> This change was incorporated in this ticket: https://issues.apache.
> >> org/jira/browse/IGNITE-3793. We can't do it before 2.0 for compatibility
> >> reasons.
> >>
> >> However, my point is that they changed the license to Apache 2.0, so I'm
> >> not sure that licensing issue still exists.
> >>
> >> -Val
> >>
> >> On Tue, Jan 24, 2017 at 7:04 PM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> >> wrote:
> >>
> >>> Any reason why we need to wait for 2.0? Sorry if this has already been
> >>> discussed.
> >>>
> >>> On Tue, Jan 24, 2017 at 7:02 PM, Denis Magda 
> wrote:
> >>>
>  Yes, we planned to do that in 2.0. Val, the ticket is closed
>  https://issues.apache.org/jira/browse/IGNITE-2949 <
>  https://issues.apache.org/jira/browse/IGNITE-2949>
> 
>  Do we need to reopen it making sure that geronimo jar is added to 2.0?
> 
>  —
>  Denis
> 
> > On Jan 24, 2017, at 6:36 PM, Dmitriy Setrakyan <
> >> dsetrak...@apache.org>
>  wrote:
> >
> > We absolutely need to upgrade to the geronimo jcache library in the
> >>> next
> > release.
> >
> > On Tue, Jan 24, 2017 at 3:45 PM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> >> Guys,
> >>
> >> I noticed that the JCache license was updated to Apache 2.0 several
>  months
> >> ago [1]. However, there was no release with the new license and
> >> 1.0.0
>  still
> >> has the old license name in the POM file [2] (the link is pointing
> >> to
>  the
> >> new one though).
> >>
> >> Is this enough from legal standpoint? Do we still need to move to
>  Geronimo?
> >>
> >> [1] https://github.com/jsr107/jsr107spec/blob/master/LICENSE.txt
> >> [2] http://mvnrepository.com/artifact/javax.cache/cache-api/1.0.0
> >>
> >> -Val
> >>
> >>
> >> On Mon, Apr 11, 2016 at 5:43 PM, Dmitriy Setrakyan <
>  dsetrak...@apache.org>
> >> wrote:
> >>
> >>> I would say that we are OK with alpha for now, as there is no real
> >>> difference between 1.0-alpha and 1.0. We can switch to 1.0 whenever
> >>> geronimo project updates the JAR.
> >>>
> >>> D.
> >>>
> >>> On Mon, Apr 11, 2016 at 5:10 PM, Valentin Kulichenko <
> >>> valentin.kuliche...@gmail.com> wrote:
> >>>
>  Folks,
> 
>  I tried to switch to Geronimo and it works fine for me. Are we
> >> going
>  to
>  wait for version 1.0, or we're OK with alpha?
> 
>  -Val
> 
>  On Mon, Mar 28, 2016 at 7:37 AM, Dmitriy Setrakyan <
> >>> dsetrak...@apache.org>
>  wrote:
> 
> > Igniters,
> >
> > Can someone check if the Geronimo JCache jar is the same as the
> >> JSR107?
> >
> >
> >
>  http://mvnrepository.com/artifact/org.apache.geronimo.
> >>> specs/geronimo-jcache_1.0_spec
> >
> > We should try switching to the Geronimo JAR starting next
> >> release,
> >>> as
> >>> it
>  is
> > licensed under Apache 2.0.
> >
> > D.
> >
> 
> >>>
> >>
> 
> 
> >>>
> >>
>
>


-- 
Kind regards,
Alexander.

Re: newbie ticket issue

[ALEKSEY KUZNETSOV]

Hi! My JIRA ID :
https://issues.apache.org/jira/secure/ViewProfile.jspa?name=Alexey+Kuznetsov

пт, 27 янв. 2017 г. в 20:52, Denis Magda :

> + dev list
>
> > On Jan 27, 2017, at 9:52 AM, Denis Magda  wrote:
> >
> > Hello Aleksey,
> >
> > Welcome to the Ignite community!
> >
> > Share your JIRA ID with us so that we can add you to the contributors
> list. After that you will be able to assign tickets on yourself.
> >
> > As for this ticket, it’s no longer relevant and I’m going to close it.
> Feel free to pick any other of your interest. Here is a good selection:
> > https://ignite.apache.org/community/contribute.html#pick-ticket <
> https://ignite.apache.org/community/contribute.html#pick-ticket>
> >
> > Additional information that will be useful.
> >
> > Subscribe to both dev and user lists:
> > https://ignite.apache.org/community/resources.html#mail-lists <
> https://ignite.apache.org/community/resources.html#mail-lists>
> >
> > Get familiar with Ignite development process described here:
> > https://cwiki.apache.org/confluence/display/IGNITE/Development+Process <
> https://cwiki.apache.org/confluence/display/IGNITE/Development+Process>
> >
> > Instructions on how to contribute can be found here:
> > https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute <
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute>
> >
> > Project setup in Intellij IDEAL
> > https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup <
> https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup>
> >
> > Looking forward to your contributions!
> >
> > Regards,
> > Denis
> >
> >> On Jan 27, 2017, at 2:21 AM, ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com > wrote:
> >>
> >> Добрый день! Я подключился проекту недавно и у меня возникли проблемы с
> джирой. Нет прав на стар тикета и присвоения его себе. Вот тикет -
> https://issues.apache.org/jira/browse/IGNITE-3385 <
> https://issues.apache.org/jira/browse/IGNITE-3385>
> >> --
> >> Best Regards,
> >>
> >> Kuznetsov Aleksey
> >>
> >
>
> --

*Best Regards,*

*Kuznetsov Aleksey*

Re: IGNITE-4492

[ALEKSEY KUZNETSOV]

thanx

пт, 27 янв. 2017 г. в 17:15, Alexey Kuznetsov :

> Added to contributors.
>
> On Fri, Jan 27, 2017 at 9:04 PM, ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> > wrote:
>
> > https://issues.apache.org/jira/secure/ViewProfile.jspa?
> > name=Alexey+Kuznetsov
> >
> > пт, 27 янв. 2017 г. в 16:48, Alexey Kuznetsov :
> >
> > > What is your JIRA ID?
> > > I will add you to Apache Ignite contributors.
> > >
> > > On Fri, Jan 27, 2017 at 7:34 PM, ALEKSEY KUZNETSOV <
> > > alkuznetsov...@gmail.com
> > > > wrote:
> > >
> > > > I know! But i don't have sufficient privileges yet
> > > >
> > > > пт, 27 янв. 2017 г. в 14:41, Alexey Kuznetsov  >:
> > > >
> > > > > Aleksey,
> > > > >
> > > > > FYI: You should assign issue in JIRA to your self. (click "Assign
> to
> > > me"
> > > > > link).
> > > > >
> > > > > On Fri, Jan 27, 2017 at 6:28 PM, ALEKSEY KUZNETSOV <
> > > > > alkuznetsov...@gmail.com
> > > > > > wrote:
> > > > >
> > > > > > will take IGNITE-4492  > > > jira/browse/IGNITE-4492
> > > > > >
> > > > > > if
> > > > > > you don't mind
> > > > > > --
> > > > > >
> > > > > > *Best Regards,*
> > > > > >
> > > > > > *Kuznetsov Aleksey*
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Alexey Kuznetsov
> > > > >
> > > > --
> > > >
> > > > *Best Regards,*
> > > >
> > > > *Kuznetsov Aleksey*
> > > >
> > >
> > >
> > >
> > > --
> > > Alexey Kuznetsov
> > >
> > --
> >
> > *Best Regards,*
> >
> > *Kuznetsov Aleksey*
> >
>
>
>
> --
> Alexey Kuznetsov
>
-- 

*Best Regards,*

*Kuznetsov Aleksey*

Re: IGNITE-1948 ClusterTopologyCheckedException can return null for retryReadyFuture()

[Alexey Goncharuk]

Alexander,

Do you have a use-case in mind which results in IgniteTopologyException
thrown from public API with retryFuture unset?

retryFuture makes sense only for certain cache operations and may be set
only in certain places in the code where we already know what to wait for.
I do not see how you can initialize this future, for example, in
 GridCacheIoManager#sendNoRetry(ClusterNode node, GridCacheMessage msg, byte
plc)

--AG

2017-01-27 15:45 GMT+03:00 Александр Меньшиков :

> I found a lot of using "ClusterTopologyCheckedException" exception. In
> most
> use-case these exceptions were just ignored. It's easier to see if
> issue-4612 will be applied (I'm waiting for code review by my team leader)
> where I renamed all unused exceptions in the "ignored".
>
> It means that in some case "retryReadyFuture" can left unset. But there are
> code where exception is being getting from "get()" method in some "future"
> class and don't ignored (exception is sending to method
> "GridFutureAdapter#onDone()" for example).
>
> So I decided not to do a deep global analysis of code and make some simple
> rules.
> 1. If some method "X" throws ClusterTopologyCheckedException and ignores
> it
> (or converts to String, there are some variants to lose exception like
> object) in all methods that call the method "X", then we can don't set
> "retryReadyFuture" for optimization. But this should be clarified in
> javadoc.
> 2. But if exception escapes at least once like object: we must set
> "retryReadyFuture" in that method.
>
> A few times when call tree was simple, I went deeper.
>
> So I found only three methods which can throw
> "ClusterTopologyCheckedException" without setting "retryReadyFuture":
>
> 1. IgfsFragmentizerManager#sendWithRetries(UUID nodeId,
> IgfsCommunicationMessage msg)
> 2. GridCacheIoManager#sendNoRetry(ClusterNode node, GridCacheMessage msg,
> byte plc)
> 3. GridCacheIoManager#sendOrderedMessage(ClusterNode node, Object topic,
> GridCacheMessage msg, byte plc, long timeout)
>
> In all other methods "ClusterTopologyCheckedException" escapes away too
> far.
>
> What do you think about this approach to make compromise between
> optimization and correctness?
>

Re: Do not use H2 parser for DDL

[Alexander Paschenko]

Dima,

H2's grammar for CREATE INDEX currently does not allow expressing all
that Ignite has in this field - say, I can't specify FULLTEXT index
types. (But, for clarity, I should also note that I don't see anything
else missing right away).

On the other hand, if we want to enhance indexes someday, H2 may not
be there to support our custom syntax.

All in all, I believe that the moment for messing the custom syntax
will no doubt come, sooner or later, and we can't avoid it forever. So
all our attempts to avoid messing with it now just delay the moment
when we will simply have no choice and will have to come up with
something quick and dirty in a rush and hurry.

Yes, Ignite mode for H2 probably could be a solution for parsing, but
we have to keep in mind that compatibility mode in H2 is more than
syntax. And H2 folks will not likely be happy with just bunch of
random syntax changes that are otherwise irrelevant to RDBMS and will
not be used by or implemented in H2 itself anyway.

- Alex

2017-01-30 4:18 GMT+03:00 Dmitriy Setrakyan :
> On Fri, Jan 27, 2017 at 8:51 PM, Alexander Paschenko <
> alexander.a.pasche...@gmail.com> wrote:
>
>> Guys,
>>
>> And what would you say if I suggested that we implement custom grammar
>> support with ANTLR? It allows you to describe pretty much any grammar
>> in a declarative way, generates lexer and parser and then allows to
>> easily process parsed commands by implementing few (generated)
>> interfaces. Yesterday I gave it a try and it's really simple.
>> Downsides are use of generated code itself (I'm pretty sure someone is
>> strongly against it) and relative wordiness of resulting code written
>> manually. But this approach will no doubt save time and allow any
>> extensions or changes in syntax in the future w/o worrying about H2 or
>> anyone 3rd party. Thoughts?
>>
>>
> Alex,
>
> My preference would be to keep it simple, without introducing any custom
> grammar, at least for the "CREATE INDEX" command. I understand the need for
> decoupling cache from schema, but it will take much longer to implement,
> and I would leave it for phase 2. In phase 1 we can focus on delivering
> this much needed feature to the community as soon as possible.
>
> Do you agree?
>
>
>> - Alex
>>
>> 2017-01-27 21:56 GMT+03:00 Vladimir Ozerov :
>> > My point was that we can avoid dependency on 3rd party developers for
>> this
>> > relatively simple logic.
>> >
>> > On Fri, Jan 27, 2017 at 8:07 PM, Dmitriy Setrakyan <
>> dsetrak...@apache.org>
>> > wrote:
>> >
>> >> On Fri, Jan 27, 2017 at 5:45 AM, Sergi Vladykin <
>> sergi.vlady...@gmail.com>
>> >> wrote:
>> >>
>> >> > H2 to some extent supports syntax (and quirks) from other databases.
>> For
>> >> > example you can start it with MODE=MySQL and it will allow some MySQL
>> >> > specific syntax to be handled.
>> >> >
>> >> > Having said that, IMO the most correct way to handle non-standard
>> syntax
>> >> is
>> >> > to introduce H2 MODE=ApacheIgnite and put the needed switches there.
>> >> >
>> >> > Though this needs to be negotiated with H2 folks.
>> >> >
>> >>
>> >> Sergi, can you ask the H2 folks about this? I agree, this sounds like
>> the
>> >> most correct way.
>> >>
>>

IGNITE-3196 - ready for review

[Vyacheslav Daradur]

Hello. I fixed it. Please, review.

https://issues.apache.org/jira/browse/IGNITE-3196 - Marshaling works wrong
for the BigDecimals that have negative scale

[GitHub] ignite pull request #1477: IGNITE-4492 Add MBean for StripedExecutor

[voipp]

GitHub user voipp opened a pull request:

https://github.com/apache/ignite/pull/1477

IGNITE-4492 Add MBean for StripedExecutor

Added MBean for StripedExecutor

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/voipp/ignite IGNITE-4492

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1477.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1477


commit cda50f5ba812e443b85713f581f072cbeff70f7e
Author: voipp 
Date:   2017-01-30T09:51:37Z

IGNITE-4492 Add MBean for StripedExecutor




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1478: ignite-4626

[kdudkov]

GitHub user kdudkov opened a pull request:

https://github.com/apache/ignite/pull/1478

ignite-4626



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4626

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1478.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1478


commit 61e9b654a34474d2b88faaaf9378b94a515be86c
Author: Konstantin Dudkov 
Date:   2017-01-30T10:37:58Z

ignite-4626




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1479: IGNITE-4533 GridDhtPartitionsExchangeFuture store...

[ezhuravl]

GitHub user ezhuravl opened a pull request:

https://github.com/apache/ignite/pull/1479

IGNITE-4533 GridDhtPartitionsExchangeFuture stores unnecessary messag…

…es after processing done

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ezhuravl/ignite ignite-4533

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1479.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1479


commit 6be5c89dcf05c395ca7a2fe1c85f4e634b5eef87
Author: Evgenii Zhuravlev 
Date:   2017-01-30T10:48:35Z

IGNITE-4533 GridDhtPartitionsExchangeFuture stores unnecessary messages 
after processing done




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

How to improve SQL testing

[Sergey Kozlov]

Hi

Hi reviewed the links below (thanks for Vova Ozerov to pointing me) and
found that the provided solution is really excellent!
Unfortunately we can't use it as is (from my standpoint) due lacks of
Python support in Ignite but we can re-use that idea for Ignite and make
SQL combinations coverage significantly better.

I filed the ticket IGNITE-4627 Tester for SQL functionality


Please share your thoughts and ideas.

[1] https://www.voltdb.com/blog/testing-voltdb-sqlcoverage
[2] https://github.com/VoltDB/voltdb/tree/master/tests/sqlcoverage

-- 
Sergey Kozlov
www.gridgain.com

Re: IGNITE-1948 ClusterTopologyCheckedException can return null for retryReadyFuture()

[Александр Меньшиков]

Alexey,

For GridCacheIoManager#sendNoRetry it's not necessary because exception
will be ignored (or will cast to String). Perhaps my message was unclear.
I try to say that three methods can throw "ClusterTopologyCheckedException"
without any problem.

But you are right, in some methods I can't set "retryFuture". We must
ensure that exception doesn't escape away. The best option is to ignore it
(or cast to String).

But any way, look here:

IgniteTxHandler#sendReply(UUID nodeId, GridDhtTxFinishRequest req, boolean
committed, GridCacheVersion nearTxId)

Inside you can found next lines:
__
ClusterTopologyCheckedException *cause* =
new ClusterTopologyCheckedException("Primary node left grid.");

res.checkCommittedError(new IgniteTxRollbackCheckedException("Failed to
commit transaction " +
"(transaction has been rolled back on backup node): " + req.version(),
*cause*));
__

How do you unsure that *"cause"* can't come to
GridCacheUtils#retryTopologySafe(final Callable c) ?


Perhaps I don't know some rules which you use to maintain it now?


2017-01-30 12:24 GMT+03:00 Alexey Goncharuk :

> Alexander,
>
> Do you have a use-case in mind which results in IgniteTopologyException
> thrown from public API with retryFuture unset?
>
> retryFuture makes sense only for certain cache operations and may be set
> only in certain places in the code where we already know what to wait for.
> I do not see how you can initialize this future, for example, in
>  GridCacheIoManager#sendNoRetry(ClusterNode node, GridCacheMessage msg,
> byte
> plc)
>
> --AG
>
> 2017-01-27 15:45 GMT+03:00 Александр Меньшиков :
>
> > I found a lot of using "ClusterTopologyCheckedException" exception. In
> > most
> > use-case these exceptions were just ignored. It's easier to see if
> > issue-4612 will be applied (I'm waiting for code review by my team
> leader)
> > where I renamed all unused exceptions in the "ignored".
> >
> > It means that in some case "retryReadyFuture" can left unset. But there
> are
> > code where exception is being getting from "get()" method in some
> "future"
> > class and don't ignored (exception is sending to method
> > "GridFutureAdapter#onDone()" for example).
> >
> > So I decided not to do a deep global analysis of code and make some
> simple
> > rules.
> > 1. If some method "X" throws ClusterTopologyCheckedException and ignores
> > it
> > (or converts to String, there are some variants to lose exception like
> > object) in all methods that call the method "X", then we can don't set
> > "retryReadyFuture" for optimization. But this should be clarified in
> > javadoc.
> > 2. But if exception escapes at least once like object: we must set
> > "retryReadyFuture" in that method.
> >
> > A few times when call tree was simple, I went deeper.
> >
> > So I found only three methods which can throw
> > "ClusterTopologyCheckedException" without setting "retryReadyFuture":
> >
> > 1. IgfsFragmentizerManager#sendWithRetries(UUID nodeId,
> > IgfsCommunicationMessage msg)
> > 2. GridCacheIoManager#sendNoRetry(ClusterNode node, GridCacheMessage
> msg,
> > byte plc)
> > 3. GridCacheIoManager#sendOrderedMessage(ClusterNode node, Object topic,
> > GridCacheMessage msg, byte plc, long timeout)
> >
> > In all other methods "ClusterTopologyCheckedException" escapes away too
> > far.
> >
> > What do you think about this approach to make compromise between
> > optimization and correctness?
> >
>

Re: How to improve SQL testing

[Alexey Kuznetsov]

Sergey,

Could we use http://www.jython.org/ to make it works as-is?

On Mon, Jan 30, 2017 at 6:06 PM, Sergey Kozlov  wrote:

> Hi
>
> Hi reviewed the links below (thanks for Vova Ozerov to pointing me) and
> found that the provided solution is really excellent!
> Unfortunately we can't use it as is (from my standpoint) due lacks of
> Python support in Ignite but we can re-use that idea for Ignite and make
> SQL combinations coverage significantly better.
>
> I filed the ticket IGNITE-4627 Tester for SQL functionality
> 
>
> Please share your thoughts and ideas.
>
> [1] https://www.voltdb.com/blog/testing-voltdb-sqlcoverage
> [2] https://github.com/VoltDB/voltdb/tree/master/tests/sqlcoverage
>
> --
> Sergey Kozlov
> www.gridgain.com
>



-- 
Alexey Kuznetsov

Ignite-2.0 - Make near readers update async by default

[Yakov Zhdanov]

Guys,

Currently when we do cache updates in FULL_SYNC mode we update near readers
(near cache entries) synchronously. This is quite big drawback in design, I
think. I get each near reader update at cost of 1 extra backup update. I
think everyone understands that partitioned cache easily turns to
replicated once near readers number increases. In TX cache cost of such
updates doubles.

I do not see any benefit on updating near entries in sync way. Outdated
reads can still be possible if I don't read from primary or in TX context.

So, what I suggest:
1. introduce flag for cahce - withSyncNearUpdates() or extra
CacheWriteSynchronizationMode.DHT_SYNC and make it default mode.
2. Near entries are updated in async way
2.1 in atomic mode together with backup updates
2.2 in TX mode after tx is committed on primary. I would also suggest to
exclude near readers from lock acquisition/release steps. Only force
updates. Updates order will be ensured by single primary node and
per-partition striping.
3. Near readers do not respond to primary. Once primary fails near readers
get invalidated, if primary is alive then communication recovery ensures
that message will be delivered to near.

Please share your thoughts.

--Yakov

Re: How to improve SQL testing

[Sergey Kozlov]

Alexey, I suppose that it's a hard way to do that and at least see a few
disadvantages below:

1. The VoltDB has no multithreaded mode. It means huge execution time
consumption (say we talk about thousands statement multiplied on caches
count, it's the millions of queries).
2. We should add support of Ignite.
3. We should add support of H2 as base engine for us (MySQL/Postgress have
less priority)
4. They run it under Jenkins so we need to integrate it with TC somehow.

>From my standpoint better to make the utility in Ignite than spending time
and efforts to adopt an existing tool. At least we have good (or even best)
Java experts/developers in community and the task doesn't look a complex
one.


On Mon, Jan 30, 2017 at 2:58 PM, Alexey Kuznetsov 
wrote:

> Sergey,
>
> Could we use http://www.jython.org/ to make it works as-is?
>
> On Mon, Jan 30, 2017 at 6:06 PM, Sergey Kozlov 
> wrote:
>
> > Hi
> >
> > Hi reviewed the links below (thanks for Vova Ozerov to pointing me) and
> > found that the provided solution is really excellent!
> > Unfortunately we can't use it as is (from my standpoint) due lacks of
> > Python support in Ignite but we can re-use that idea for Ignite and make
> > SQL combinations coverage significantly better.
> >
> > I filed the ticket IGNITE-4627 Tester for SQL functionality
> > 
> >
> > Please share your thoughts and ideas.
> >
> > [1] https://www.voltdb.com/blog/testing-voltdb-sqlcoverage
> > [2] https://github.com/VoltDB/voltdb/tree/master/tests/sqlcoverage
> >
> > --
> > Sergey Kozlov
> > www.gridgain.com
> >
>
>
>
> --
> Alexey Kuznetsov
>



-- 
Sergey Kozlov
GridGain Systems
www.gridgain.com

Re: IgniteConfiguration.gridName is very confusing

[Alexander Fedotov]

Hi,

Created Upsource review for the subject:
http://reviews.ignite.apache.org/ignite/review/IGNT-CR-81

On Thu, Jan 19, 2017 at 7:59 PM, Alexander Fedotov <
alexander.fedot...@gmail.com> wrote:

> Hi,
>
> I've completed working on IGNITE-3207
> https://issues.apache.org/jira/browse/IGNITE-3207
>
> Looks like TC test results don't have problems related to my changes
> http://ci.ignite.apache.org/viewLog.html?buildId=423955&;
> tab=buildResultsDiv&buildTypeId=IgniteTests_RunAll
>
> Kindly take a look at PR https://github.com/apache/ignite/pull/1435/
>
> On Thu, Jan 12, 2017 at 1:16 AM, Denis Magda  wrote:
>
>> Support Pavel’s point of view.
>>
>> Also Alexander please make sure that your changes are merged into
>> ignite-2.0 branch rather than to the master. I think this functionality
>> has to be available in 2.0 first. Finally, please update 2.0 Migration
>> Guide once you’ve finished with this task:
>> https://cwiki.apache.org/confluence/display/IGNITE/Apache+
>> Ignite+2.0+Migration+Guide > luence/display/IGNITE/Apache+Ignite+2.0+Migration+Guide>
>>
>> —
>> Denis
>>
>> > On Jan 10, 2017, at 1:58 AM, Pavel Tupitsyn 
>> wrote:
>> >
>> > I think we should fix log output as well and replace all "grid"
>> occurences
>> > with "instance".
>> >
>> > On Tue, Jan 10, 2017 at 12:55 PM, Alexander Fedotov <
>> > alexander.fedot...@gmail.com> wrote:
>> >
>> >> Hi,
>> >>
>> >> I think we should leave null as a default value for unnamed Ignite
>> >> instances. At least that change should be considered out of the current
>> >> scope.
>> >>
>> >> What about naming, I'm also renaming log occurrences of "grid" and
>> "grid
>> >> name" where it stands reasonable.
>> >> Are there places in the logging logic where we should prefer name
>> "grid" or
>> >> "grid name" instead of "Ignite instance name" or "Ignite instance
>> name" can
>> >> be used without any semantic impact?
>> >>
>> >> On Sat, Dec 31, 2016 at 11:23 AM, Alexander Fedotov <
>> >> alexander.fedot...@gmail.com> wrote:
>> >>
>> >>> Okay. From the all said above I suppose "instanceName" should work for
>> >>> IgniteConfiguration and "igniteInstanceName" in all other places.
>> >>>
>> >>> Regards,
>> >>> Alexander
>> >>>
>> >>> 31 дек. 2016 г. 3:43 AM пользователь "Dmitriy Setrakyan" <
>> >>> dsetrak...@apache.org> написал:
>> >>>
>> >>> It sounds like it must be unique then. I would propose the following:
>> >>>
>> >>>   1. If user defines the instanceName, then we assign it to the node.
>> >>>   2. If user does not define the instance name, then we have to give
>> it
>> >>>   some unique value, like node ID or PID.
>> >>>
>> >>> Will this change be backward compatible, or should we leave it as
>> null if
>> >>> user does not define it?
>> >>>
>> >>> D.
>> >>>
>> >>> On Fri, Dec 30, 2016 at 4:19 PM, Denis Magda 
>> >> wrote:
>> >>>
>>  Sounds reasonable. Agree that 'instanceName' suits better considering
>> >>> your
>>  explanation.
>> 
>>  --
>>  Denis
>> 
>>  On Friday, December 30, 2016, Valentin Kulichenko <
>>  valentin.kuliche...@gmail.com> wrote:
>> > This name identifies instance of Ignite, in case there are more than
>> >>> one
>> > within an application. Here are our API methods around this:
>> >
>> > // We provide a name and get newly started *Ignite* instance.
>> > Ignite ignite = Ignition.start(new
>>  IgniteConfiguration().setGridName(name));
>> >
>> > // We provide a name and get existing *Ignite* instance.
>> > Ignite ignite = Ignition.ignite(name);
>> >
>> > This has nothing to do with nodes. For node representation we have
>> > ClusterNode API, which already has nodeId() method for
>> >> identification.
>> >
>> > In other words, if we choose nodeName, we will have both nodeName
>> and
>> > nodeId in the product, but with absolutely different meaning and
>> used
>> >>> in
>> > different parts of API. How user is going to understand the
>> >> difference
>> > between them? In my view, this is even more confusing than current
>>  gridName.
>> >
>> > -Val
>> >
>> > On Fri, Dec 30, 2016 at 2:42 PM, Denis Magda 
>>  wrote:
>> >
>> >> Alexander, frankly speaking I'm still for your original proposal -
>> >> nodeName. The uniqueness specificities can be set in the doc.
>> >>
>> >> --
>> >> Denis
>> >>
>> >> On Friday, December 30, 2016, Alexander Fedotov <
>> >> alexander.fedot...@gmail.com> wrote:
>> >>> Well, then may be we should go with one of the below names:
>> >>>
>> >>> processNodeName
>> >>> jvmNodeName
>> >>> runtimeNodeName
>> >>> processScopedNodeName
>> >>> jvmScopedNodeName
>> >>> runtimeScopedNodeName
>> >>> processWideNodeName
>> >>> jvmWideNodeName
>> >>> runtimeWideNodeName
>> >>>
>> >>> Regards,
>> >>> Alexander
>> >>>
>> >>> 31 дек. 2016 г. 12:37 AM пользователь "Denis Magda" <
>> 

Re: moving to geronimo JCache jar

[Alexander Fedotov]

Hi,

Created Upsource review for the subject:
http://reviews.ignite.apache.org/ignite/review/IGNT-CR-82

On Mon, Jan 30, 2017 at 11:52 AM, Alexander Fedotov <
alexander.fedot...@gmail.com> wrote:

> Hi all,
>
> https://issues.apache.org/jira/browse/IGNITE-3793 is completed.
> Kindly take a look at the corresponding PR https://github.com/apache/i
> gnite/pull/1475 .
>
> On Wed, Jan 25, 2017 at 8:04 PM, Denis Magda  wrote:
>
>> We need to replace content of ignite-core-licenses.txt file which is the
>> following at the moment
>>
>> // --
>> // List of ignite-core module's dependencies provided as a part of this
>> distribution
>> // which licenses differ from Apache Software License.
>> // --
>>
>> 
>> ==
>> For JSR107 API and SPI (https://github.com/jsr107/jsr107spec)
>> javax.cache:cache-api:jar:1.0.0
>> 
>> ==
>> This product bundles JSR107 API and SPI which is available under a:
>> JSR-000107 JCACHE 2.9 Public Review - Updated Specification License. For
>> details, see https://raw.github.com/jsr107/jsr107spec/master/LICENSE.txt.
>>
>>
>> Updated this ticket description: https://issues.apache.org/jira
>> /browse/IGNITE-3793
>>
>> —
>> Denis
>> > On Jan 24, 2017, at 8:24 PM, Dmitriy Setrakyan 
>> wrote:
>> >
>> > Awesome, you are right. I just checked and the license is indeed Apache
>> > 2.0. Is there anything we need to do at all right now?
>> >
>> > On Tue, Jan 24, 2017 at 8:17 PM, Valentin Kulichenko <
>> > valentin.kuliche...@gmail.com> wrote:
>> >
>> >> This change was incorporated in this ticket: https://issues.apache.
>> >> org/jira/browse/IGNITE-3793. We can't do it before 2.0 for
>> compatibility
>> >> reasons.
>> >>
>> >> However, my point is that they changed the license to Apache 2.0, so
>> I'm
>> >> not sure that licensing issue still exists.
>> >>
>> >> -Val
>> >>
>> >> On Tue, Jan 24, 2017 at 7:04 PM, Dmitriy Setrakyan <
>> dsetrak...@apache.org>
>> >> wrote:
>> >>
>> >>> Any reason why we need to wait for 2.0? Sorry if this has already been
>> >>> discussed.
>> >>>
>> >>> On Tue, Jan 24, 2017 at 7:02 PM, Denis Magda 
>> wrote:
>> >>>
>>  Yes, we planned to do that in 2.0. Val, the ticket is closed
>>  https://issues.apache.org/jira/browse/IGNITE-2949 <
>>  https://issues.apache.org/jira/browse/IGNITE-2949>
>> 
>>  Do we need to reopen it making sure that geronimo jar is added to
>> 2.0?
>> 
>>  —
>>  Denis
>> 
>> > On Jan 24, 2017, at 6:36 PM, Dmitriy Setrakyan <
>> >> dsetrak...@apache.org>
>>  wrote:
>> >
>> > We absolutely need to upgrade to the geronimo jcache library in the
>> >>> next
>> > release.
>> >
>> > On Tue, Jan 24, 2017 at 3:45 PM, Valentin Kulichenko <
>> > valentin.kuliche...@gmail.com> wrote:
>> >
>> >> Guys,
>> >>
>> >> I noticed that the JCache license was updated to Apache 2.0 several
>>  months
>> >> ago [1]. However, there was no release with the new license and
>> >> 1.0.0
>>  still
>> >> has the old license name in the POM file [2] (the link is pointing
>> >> to
>>  the
>> >> new one though).
>> >>
>> >> Is this enough from legal standpoint? Do we still need to move to
>>  Geronimo?
>> >>
>> >> [1] https://github.com/jsr107/jsr107spec/blob/master/LICENSE.txt
>> >> [2] http://mvnrepository.com/artifact/javax.cache/cache-api/1.0.0
>> >>
>> >> -Val
>> >>
>> >>
>> >> On Mon, Apr 11, 2016 at 5:43 PM, Dmitriy Setrakyan <
>>  dsetrak...@apache.org>
>> >> wrote:
>> >>
>> >>> I would say that we are OK with alpha for now, as there is no real
>> >>> difference between 1.0-alpha and 1.0. We can switch to 1.0
>> whenever
>> >>> geronimo project updates the JAR.
>> >>>
>> >>> D.
>> >>>
>> >>> On Mon, Apr 11, 2016 at 5:10 PM, Valentin Kulichenko <
>> >>> valentin.kuliche...@gmail.com> wrote:
>> >>>
>>  Folks,
>> 
>>  I tried to switch to Geronimo and it works fine for me. Are we
>> >> going
>>  to
>>  wait for version 1.0, or we're OK with alpha?
>> 
>>  -Val
>> 
>>  On Mon, Mar 28, 2016 at 7:37 AM, Dmitriy Setrakyan <
>> >>> dsetrak...@apache.org>
>>  wrote:
>> 
>> > Igniters,
>> >
>> > Can someone check if the Geronimo JCache jar is the same as the
>> >> JSR107?
>> >
>> >
>> >
>>  http://mvnrepository.com/artifact/org.apache.geronimo.
>> >>> specs/geronimo-jcache_1.0_spec
>> >
>> > We should try switching to the Geronimo JAR starting next
>> >> release,
>> >>> as
>> >>> it
>>  is
>> > licensed under Apache 2.0.
>> >>>

Re: IgniteConfiguration.gridName is very confusing

[Pavel Tupitsyn]

Alexander,

Please name the review appropriately and link it in the ticket as described:
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewWithUpsource

Thanks,
Pavel

On Mon, Jan 30, 2017 at 4:00 PM, Alexander Fedotov <
alexander.fedot...@gmail.com> wrote:

> Hi,
>
> Created Upsource review for the subject:
> http://reviews.ignite.apache.org/ignite/review/IGNT-CR-81
>
> On Thu, Jan 19, 2017 at 7:59 PM, Alexander Fedotov <
> alexander.fedot...@gmail.com> wrote:
>
> > Hi,
> >
> > I've completed working on IGNITE-3207
> > https://issues.apache.org/jira/browse/IGNITE-3207
> >
> > Looks like TC test results don't have problems related to my changes
> > http://ci.ignite.apache.org/viewLog.html?buildId=423955&;
> > tab=buildResultsDiv&buildTypeId=IgniteTests_RunAll
> >
> > Kindly take a look at PR https://github.com/apache/ignite/pull/1435/
> >
> > On Thu, Jan 12, 2017 at 1:16 AM, Denis Magda  wrote:
> >
> >> Support Pavel’s point of view.
> >>
> >> Also Alexander please make sure that your changes are merged into
> >> ignite-2.0 branch rather than to the master. I think this functionality
> >> has to be available in 2.0 first. Finally, please update 2.0 Migration
> >> Guide once you’ve finished with this task:
> >> https://cwiki.apache.org/confluence/display/IGNITE/Apache+
> >> Ignite+2.0+Migration+Guide  >> luence/display/IGNITE/Apache+Ignite+2.0+Migration+Guide>
> >>
> >> —
> >> Denis
> >>
> >> > On Jan 10, 2017, at 1:58 AM, Pavel Tupitsyn 
> >> wrote:
> >> >
> >> > I think we should fix log output as well and replace all "grid"
> >> occurences
> >> > with "instance".
> >> >
> >> > On Tue, Jan 10, 2017 at 12:55 PM, Alexander Fedotov <
> >> > alexander.fedot...@gmail.com> wrote:
> >> >
> >> >> Hi,
> >> >>
> >> >> I think we should leave null as a default value for unnamed Ignite
> >> >> instances. At least that change should be considered out of the
> current
> >> >> scope.
> >> >>
> >> >> What about naming, I'm also renaming log occurrences of "grid" and
> >> "grid
> >> >> name" where it stands reasonable.
> >> >> Are there places in the logging logic where we should prefer name
> >> "grid" or
> >> >> "grid name" instead of "Ignite instance name" or "Ignite instance
> >> name" can
> >> >> be used without any semantic impact?
> >> >>
> >> >> On Sat, Dec 31, 2016 at 11:23 AM, Alexander Fedotov <
> >> >> alexander.fedot...@gmail.com> wrote:
> >> >>
> >> >>> Okay. From the all said above I suppose "instanceName" should work
> for
> >> >>> IgniteConfiguration and "igniteInstanceName" in all other places.
> >> >>>
> >> >>> Regards,
> >> >>> Alexander
> >> >>>
> >> >>> 31 дек. 2016 г. 3:43 AM пользователь "Dmitriy Setrakyan" <
> >> >>> dsetrak...@apache.org> написал:
> >> >>>
> >> >>> It sounds like it must be unique then. I would propose the
> following:
> >> >>>
> >> >>>   1. If user defines the instanceName, then we assign it to the
> node.
> >> >>>   2. If user does not define the instance name, then we have to give
> >> it
> >> >>>   some unique value, like node ID or PID.
> >> >>>
> >> >>> Will this change be backward compatible, or should we leave it as
> >> null if
> >> >>> user does not define it?
> >> >>>
> >> >>> D.
> >> >>>
> >> >>> On Fri, Dec 30, 2016 at 4:19 PM, Denis Magda 
> >> >> wrote:
> >> >>>
> >>  Sounds reasonable. Agree that 'instanceName' suits better
> considering
> >> >>> your
> >>  explanation.
> >> 
> >>  --
> >>  Denis
> >> 
> >>  On Friday, December 30, 2016, Valentin Kulichenko <
> >>  valentin.kuliche...@gmail.com> wrote:
> >> > This name identifies instance of Ignite, in case there are more
> than
> >> >>> one
> >> > within an application. Here are our API methods around this:
> >> >
> >> > // We provide a name and get newly started *Ignite* instance.
> >> > Ignite ignite = Ignition.start(new
> >>  IgniteConfiguration().setGridName(name));
> >> >
> >> > // We provide a name and get existing *Ignite* instance.
> >> > Ignite ignite = Ignition.ignite(name);
> >> >
> >> > This has nothing to do with nodes. For node representation we have
> >> > ClusterNode API, which already has nodeId() method for
> >> >> identification.
> >> >
> >> > In other words, if we choose nodeName, we will have both nodeName
> >> and
> >> > nodeId in the product, but with absolutely different meaning and
> >> used
> >> >>> in
> >> > different parts of API. How user is going to understand the
> >> >> difference
> >> > between them? In my view, this is even more confusing than current
> >>  gridName.
> >> >
> >> > -Val
> >> >
> >> > On Fri, Dec 30, 2016 at 2:42 PM, Denis Magda  >
> >>  wrote:
> >> >
> >> >> Alexander, frankly speaking I'm still for your original proposal
> -
> >> >> nodeName. The uniqueness specificities can be set in the doc.
> >> >>
> >> >> --
> >> >> Denis
> >> >>
> >> >>>

[GitHub] ignite pull request #1480: IGNITE-4444 Extend .NET plugin API to interact wi...

[ptupitsyn]

GitHub user ptupitsyn opened a pull request:

https://github.com/apache/ignite/pull/1480

IGNITE- Extend .NET plugin API to interact with Java



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1480.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1480


commit dda42481977326d4213d3ccbc966785bad542ad3
Author: Pavel Tupitsyn 
Date:   2017-01-30T13:33:27Z

IGNITE- Extend .NET plugin API to interact with Java




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: IgniteConfiguration.gridName is very confusing

[Alexander Fedotov]

Done. But it looks like something went wrong since Upsource reports:
"Review has too many files (1244), aborting".

Also guys, I believe we need to merge this change in short time because
it's targeted for 2.0 and chances for a conflict are high.



On Mon, Jan 30, 2017 at 4:16 PM, Pavel Tupitsyn 
wrote:

> Alexander,
>
> Please name the review appropriately and link it in the ticket as
> described:
> https://cwiki.apache.org/confluence/display/IGNITE/How+
> to+Contribute#HowtoContribute-ReviewWithUpsource
>
> Thanks,
> Pavel
>
> On Mon, Jan 30, 2017 at 4:00 PM, Alexander Fedotov <
> alexander.fedot...@gmail.com> wrote:
>
> > Hi,
> >
> > Created Upsource review for the subject:
> > http://reviews.ignite.apache.org/ignite/review/IGNT-CR-81
> >
> > On Thu, Jan 19, 2017 at 7:59 PM, Alexander Fedotov <
> > alexander.fedot...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I've completed working on IGNITE-3207
> > > https://issues.apache.org/jira/browse/IGNITE-3207
> > >
> > > Looks like TC test results don't have problems related to my changes
> > > http://ci.ignite.apache.org/viewLog.html?buildId=423955&;
> > > tab=buildResultsDiv&buildTypeId=IgniteTests_RunAll
> > >
> > > Kindly take a look at PR https://github.com/apache/ignite/pull/1435/
> > >
> > > On Thu, Jan 12, 2017 at 1:16 AM, Denis Magda 
> wrote:
> > >
> > >> Support Pavel’s point of view.
> > >>
> > >> Also Alexander please make sure that your changes are merged into
> > >> ignite-2.0 branch rather than to the master. I think this
> functionality
> > >> has to be available in 2.0 first. Finally, please update 2.0 Migration
> > >> Guide once you’ve finished with this task:
> > >> https://cwiki.apache.org/confluence/display/IGNITE/Apache+
> > >> Ignite+2.0+Migration+Guide  > >> luence/display/IGNITE/Apache+Ignite+2.0+Migration+Guide>
> > >>
> > >> —
> > >> Denis
> > >>
> > >> > On Jan 10, 2017, at 1:58 AM, Pavel Tupitsyn 
> > >> wrote:
> > >> >
> > >> > I think we should fix log output as well and replace all "grid"
> > >> occurences
> > >> > with "instance".
> > >> >
> > >> > On Tue, Jan 10, 2017 at 12:55 PM, Alexander Fedotov <
> > >> > alexander.fedot...@gmail.com> wrote:
> > >> >
> > >> >> Hi,
> > >> >>
> > >> >> I think we should leave null as a default value for unnamed Ignite
> > >> >> instances. At least that change should be considered out of the
> > current
> > >> >> scope.
> > >> >>
> > >> >> What about naming, I'm also renaming log occurrences of "grid" and
> > >> "grid
> > >> >> name" where it stands reasonable.
> > >> >> Are there places in the logging logic where we should prefer name
> > >> "grid" or
> > >> >> "grid name" instead of "Ignite instance name" or "Ignite instance
> > >> name" can
> > >> >> be used without any semantic impact?
> > >> >>
> > >> >> On Sat, Dec 31, 2016 at 11:23 AM, Alexander Fedotov <
> > >> >> alexander.fedot...@gmail.com> wrote:
> > >> >>
> > >> >>> Okay. From the all said above I suppose "instanceName" should work
> > for
> > >> >>> IgniteConfiguration and "igniteInstanceName" in all other places.
> > >> >>>
> > >> >>> Regards,
> > >> >>> Alexander
> > >> >>>
> > >> >>> 31 дек. 2016 г. 3:43 AM пользователь "Dmitriy Setrakyan" <
> > >> >>> dsetrak...@apache.org> написал:
> > >> >>>
> > >> >>> It sounds like it must be unique then. I would propose the
> > following:
> > >> >>>
> > >> >>>   1. If user defines the instanceName, then we assign it to the
> > node.
> > >> >>>   2. If user does not define the instance name, then we have to
> give
> > >> it
> > >> >>>   some unique value, like node ID or PID.
> > >> >>>
> > >> >>> Will this change be backward compatible, or should we leave it as
> > >> null if
> > >> >>> user does not define it?
> > >> >>>
> > >> >>> D.
> > >> >>>
> > >> >>> On Fri, Dec 30, 2016 at 4:19 PM, Denis Magda  >
> > >> >> wrote:
> > >> >>>
> > >>  Sounds reasonable. Agree that 'instanceName' suits better
> > considering
> > >> >>> your
> > >>  explanation.
> > >> 
> > >>  --
> > >>  Denis
> > >> 
> > >>  On Friday, December 30, 2016, Valentin Kulichenko <
> > >>  valentin.kuliche...@gmail.com> wrote:
> > >> > This name identifies instance of Ignite, in case there are more
> > than
> > >> >>> one
> > >> > within an application. Here are our API methods around this:
> > >> >
> > >> > // We provide a name and get newly started *Ignite* instance.
> > >> > Ignite ignite = Ignition.start(new
> > >>  IgniteConfiguration().setGridName(name));
> > >> >
> > >> > // We provide a name and get existing *Ignite* instance.
> > >> > Ignite ignite = Ignition.ignite(name);
> > >> >
> > >> > This has nothing to do with nodes. For node representation we
> have
> > >> > ClusterNode API, which already has nodeId() method for
> > >> >> identification.
> > >> >
> > >> > In other words, if we choose nodeName, we will have both
> nodeName
> > >> and
> > >> > nodeId in the product, but with absolu

Re: IGNITE-1948 ClusterTopologyCheckedException can return null for retryReadyFuture()

[Alexey Goncharuk]

In the example above there is no point of setting the retry future in the
exception because it will be serialized and sent to a remote node.

I see the only possible way to ensure this: make this be verified at
compile time. This looks like a big change, but the draft may look like so:
1) Introduce new exception type, like CacheTopology*Checked*Exception which
*must* take the minimum topology version to wait for
2) In all cases when a cache operation fails because of topology change,
provide the appropriate exception
3) When CacheTopologyException is being constructed, create a corresponding
local future based on wait version and throw the exception.

Even though this still does not give us 100% guarantee that we did not miss
anything, it should cover a lot more cases than now.

2017-01-30 14:20 GMT+03:00 Александр Меньшиков :

> Alexey,
>
> For GridCacheIoManager#sendNoRetry it's not necessary because exception
> will be ignored (or will cast to String). Perhaps my message was unclear.
> I try to say that three methods can throw "ClusterTopologyCheckedException"
> without any problem.
>
> But you are right, in some methods I can't set "retryFuture". We must
> ensure that exception doesn't escape away. The best option is to ignore it
> (or cast to String).
>
> But any way, look here:
>
> IgniteTxHandler#sendReply(UUID nodeId, GridDhtTxFinishRequest req, boolean
> committed, GridCacheVersion nearTxId)
>
> Inside you can found next lines:
> __
> ClusterTopologyCheckedException *cause* =
> new ClusterTopologyCheckedException("Primary node left grid.");
>
> res.checkCommittedError(new IgniteTxRollbackCheckedException("Failed to
> commit transaction " +
> "(transaction has been rolled back on backup node): " + req.version(),
> *cause*));
> __
>
> How do you unsure that *"cause"* can't come to 
> GridCacheUtils#retryTopologySafe(final
> Callable c) ?
>
>
> Perhaps I don't know some rules which you use to maintain it now?
>
>
> 2017-01-30 12:24 GMT+03:00 Alexey Goncharuk :
>
>> Alexander,
>>
>> Do you have a use-case in mind which results in IgniteTopologyException
>> thrown from public API with retryFuture unset?
>>
>> retryFuture makes sense only for certain cache operations and may be set
>> only in certain places in the code where we already know what to wait for.
>> I do not see how you can initialize this future, for example, in
>>  GridCacheIoManager#sendNoRetry(ClusterNode node, GridCacheMessage msg,
>> byte
>> plc)
>>
>> --AG
>>
>> 2017-01-27 15:45 GMT+03:00 Александр Меньшиков :
>>
>> > I found a lot of using "ClusterTopologyCheckedException" exception. In
>> > most
>> > use-case these exceptions were just ignored. It's easier to see if
>> > issue-4612 will be applied (I'm waiting for code review by my team
>> leader)
>> > where I renamed all unused exceptions in the "ignored".
>> >
>> > It means that in some case "retryReadyFuture" can left unset. But there
>> are
>> > code where exception is being getting from "get()" method in some
>> "future"
>> > class and don't ignored (exception is sending to method
>> > "GridFutureAdapter#onDone()" for example).
>> >
>> > So I decided not to do a deep global analysis of code and make some
>> simple
>> > rules.
>> > 1. If some method "X" throws ClusterTopologyCheckedException and
>> ignores
>> > it
>> > (or converts to String, there are some variants to lose exception like
>> > object) in all methods that call the method "X", then we can don't set
>> > "retryReadyFuture" for optimization. But this should be clarified in
>> > javadoc.
>> > 2. But if exception escapes at least once like object: we must set
>> > "retryReadyFuture" in that method.
>> >
>> > A few times when call tree was simple, I went deeper.
>> >
>> > So I found only three methods which can throw
>> > "ClusterTopologyCheckedException" without setting "retryReadyFuture":
>> >
>> > 1. IgfsFragmentizerManager#sendWithRetries(UUID nodeId,
>> > IgfsCommunicationMessage msg)
>> > 2. GridCacheIoManager#sendNoRetry(ClusterNode node, GridCacheMessage
>> msg,
>> > byte plc)
>> > 3. GridCacheIoManager#sendOrderedMessage(ClusterNode node, Object
>> topic,
>> > GridCacheMessage msg, byte plc, long timeout)
>> >
>> > In all other methods "ClusterTopologyCheckedException" escapes away too
>> > far.
>> >
>> > What do you think about this approach to make compromise between
>> > optimization and correctness?
>> >
>>
>
>

Re: IGNITE-1948 ClusterTopologyCheckedException can return null for retryReadyFuture()

[Александр Меньшиков]

Okay, it seems good. So you want to remove field *readyFut* from
*ClusterTopologyCheckedException*  and add *CacheTopologyCheckedException*
like *ClusterTopologyCheckedException* but initialize *readyFut* in
constructor, yes?

2017-01-30 16:54 GMT+03:00 Alexey Goncharuk :

> In the example above there is no point of setting the retry future in the
> exception because it will be serialized and sent to a remote node.
>
> I see the only possible way to ensure this: make this be verified at
> compile time. This looks like a big change, but the draft may look like so:
> 1) Introduce new exception type, like CacheTopology*Checked*Exception
> which
> *must* take the minimum topology version to wait for
> 2) In all cases when a cache operation fails because of topology change,
> provide the appropriate exception
> 3) When CacheTopologyException is being constructed, create a corresponding
> local future based on wait version and throw the exception.
>
> Even though this still does not give us 100% guarantee that we did not miss
> anything, it should cover a lot more cases than now.
>
> 2017-01-30 14:20 GMT+03:00 Александр Меньшиков :
>
> > Alexey,
> >
> > For GridCacheIoManager#sendNoRetry it's not necessary because exception
> > will be ignored (or will cast to String). Perhaps my message was unclear.
> > I try to say that three methods can throw "
> ClusterTopologyCheckedException"
> > without any problem.
> >
> > But you are right, in some methods I can't set "retryFuture". We must
> > ensure that exception doesn't escape away. The best option is to ignore
> it
> > (or cast to String).
> >
> > But any way, look here:
> >
> > IgniteTxHandler#sendReply(UUID nodeId, GridDhtTxFinishRequest req,
> boolean
> > committed, GridCacheVersion nearTxId)
> >
> > Inside you can found next lines:
> > __
> > ClusterTopologyCheckedException *cause* =
> > new ClusterTopologyCheckedException("Primary node left grid.");
> >
> > res.checkCommittedError(new IgniteTxRollbackCheckedException("Failed to
> > commit transaction " +
> > "(transaction has been rolled back on backup node): " +
> req.version(),
> > *cause*));
> > __
> >
> > How do you unsure that *"cause"* can't come to GridCacheUtils#
> retryTopologySafe(final
> > Callable c) ?
> >
> >
> > Perhaps I don't know some rules which you use to maintain it now?
> >
> >
> > 2017-01-30 12:24 GMT+03:00 Alexey Goncharuk  >:
> >
> >> Alexander,
> >>
> >> Do you have a use-case in mind which results in IgniteTopologyException
> >> thrown from public API with retryFuture unset?
> >>
> >> retryFuture makes sense only for certain cache operations and may be set
> >> only in certain places in the code where we already know what to wait
> for.
> >> I do not see how you can initialize this future, for example, in
> >>  GridCacheIoManager#sendNoRetry(ClusterNode node, GridCacheMessage msg,
> >> byte
> >> plc)
> >>
> >> --AG
> >>
> >> 2017-01-27 15:45 GMT+03:00 Александр Меньшиков :
> >>
> >> > I found a lot of using "ClusterTopologyCheckedException" exception.
> In
> >> > most
> >> > use-case these exceptions were just ignored. It's easier to see if
> >> > issue-4612 will be applied (I'm waiting for code review by my team
> >> leader)
> >> > where I renamed all unused exceptions in the "ignored".
> >> >
> >> > It means that in some case "retryReadyFuture" can left unset. But
> there
> >> are
> >> > code where exception is being getting from "get()" method in some
> >> "future"
> >> > class and don't ignored (exception is sending to method
> >> > "GridFutureAdapter#onDone()" for example).
> >> >
> >> > So I decided not to do a deep global analysis of code and make some
> >> simple
> >> > rules.
> >> > 1. If some method "X" throws ClusterTopologyCheckedException and
> >> ignores
> >> > it
> >> > (or converts to String, there are some variants to lose exception like
> >> > object) in all methods that call the method "X", then we can don't set
> >> > "retryReadyFuture" for optimization. But this should be clarified in
> >> > javadoc.
> >> > 2. But if exception escapes at least once like object: we must set
> >> > "retryReadyFuture" in that method.
> >> >
> >> > A few times when call tree was simple, I went deeper.
> >> >
> >> > So I found only three methods which can throw
> >> > "ClusterTopologyCheckedException" without setting "retryReadyFuture":
> >> >
> >> > 1. IgfsFragmentizerManager#sendWithRetries(UUID nodeId,
> >> > IgfsCommunicationMessage msg)
> >> > 2. GridCacheIoManager#sendNoRetry(ClusterNode node, GridCacheMessage
> >> msg,
> >> > byte plc)
> >> > 3. GridCacheIoManager#sendOrderedMessage(ClusterNode node, Object
> >> topic,
> >> > GridCacheMessage msg, byte plc, long timeout)
> >> >
> >> > In all other methods "ClusterTopologyCheckedException" escapes away
> too
> >> > far.
> >> >
> >> > What do you think about this approach to make compromise between
> >> > optimization and correctness?
> >> >
> >>
> >
> >
>

Review for IGNITE-4436 API to collect list of running SQL queries

[Alexey Kuznetsov]

Hi All!

I prepared first version https://issues.apache.org/jira/browse/IGNITE-4436
(SQL: create API to collect list of running SQL queries and cancel them).

Please review it and give a feedback:
http://reviews.ignite.apache.org/ignite/review/IGNT-CR-84

But also I have following questions:
1) Current implementation will collect info and cancel only "TwoStep"
queries.
Should we also support LOCAL queries?

2) Should we also support SqlQuery / ScanQuery ?

-- 
Alexey Kuznetsov

Re: Unused variables in code

[Александр Меньшиков]

Done.

My PR:
https://github.com/apache/ignite/pull/1466/files




2017-01-25 13:11 GMT+03:00 Александр Меньшиков :

> Okay, I created issue:
> https://issues.apache.org/jira/browse/IGNITE-4612
>
>
> 2017-01-24 18:19 GMT+03:00 Alexey Kuznetsov :
>
>> I think it is better to have such issue in JIRA "Minor code cleanup":
>>
>> 1) Cleanup unused imports.
>> 2) Rename e -> ignored
>> 3) Remove unused variables
>> 4)...
>> 5) PROFIT :)
>>
>> On Tue, Jan 24, 2017 at 9:46 PM, Александр Меньшиков <
>> sharple...@gmail.com>
>> wrote:
>>
>> > Hello, everyone!
>> >
>> > I found some unused variables in code. For example:
>> >
>> > Unnecessary argument in method
>> > MiniFuture#onResult(ClusterTopologyCheckedException e) in inner class
>> of
>> > GridDhtLockFuture.
>> >
>> > There isn't another onResult() method (in super class including) with
>> > another behavior. So argument can be safe delete.
>> >
>> > Also I found some ignored exceptions wasn't named "ignored". For example
>> > unused exception was named "e0" in method:
>> >
>> > GridDhtTransactionalCacheAdapter#onForceKeysError(UUID,
>> > GridDhtLockRequest,
>> > IgniteCheckedException)
>> >
>> >
>> > So I think i can go through code and cleanup thing like that.
>> >
>> > Need I create new issue in JIRA for cleanup or some issue for cleanup
>> > already exists? If that please give me a link.
>> >
>>
>>
>>
>> --
>> Alexey Kuznetsov
>>
>
>

[GitHub] ignite pull request #1454: Ignite 4003 1.7 no drop

[agura]

Github user agura closed the pull request at:

https://github.com/apache/ignite/pull/1454


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Python Platform for Ignite

[Denis Magda]

Hi Shanshank,

I’ve prepared tickets related to this activity:
https://issues.apache.org/jira/browse/IGNITE-4600 


I would agree that if we want to be as generic as possible then we need to 
write our own implementation rather relying on SWIG.
So, let’s try to base Ignite.Python on existing Ignite.C++ shared libs that 
Vladimir mentioned. Could you investigate this part? The community will assist 
if something is unclear or required more details.

—
Denis

> On Jan 28, 2017, at 10:22 AM, Shashank Gandham  
> wrote:
> 
> *As asked by Denis (Correction from Dmitry)
> 
>> On 28-Jan-2017, at 11:17 PM, Shashank Gandham  
>> wrote:
>> 
>> As I mentioned earlier I would like to work on developing the Python 
>> platform [1]
>> 
>>  Vladimir made things clear that we will have to develop a Python platform 
>> from the ground up if we have to have all the functionalities of Ignite. I 
>> think it would be worth the effort to build things from the ground up, 
>> rather than using the REST API or any other workarounds [2]
>> 
>> As asked by Dmitry, I am starting a new thread to have a broader discussion 
>> on how should we work on this. This will be my first contribution to a 
>> project as big as Ignite so I may take some time to adjust to all the 
>> standards and procedures, I assure I will pick up the pace as fast as 
>> possible.
>> 
>> 
>> [1] 
>> http://apache-ignite-developers.2346864.n4.nabble.com/GSOC-2017-td13970.html
>> [2] 
>> http://apache-ignite-developers.2346864.n4.nabble.com/Read-this-if-you-want-to-integrate-Ignite-with-other-platforms-Python-R-etc-td14006.html

Re: newbie ticket issue

[Denis Magda]

Yes, according to other discussion you were successfully added.

—
Dens

> On Jan 30, 2017, at 1:03 AM, ALEKSEY KUZNETSOV  
> wrote:
> 
> Hi! My JIRA ID :
> https://issues.apache.org/jira/secure/ViewProfile.jspa?name=Alexey+Kuznetsov
> 
> пт, 27 янв. 2017 г. в 20:52, Denis Magda :
> 
>> + dev list
>> 
>>> On Jan 27, 2017, at 9:52 AM, Denis Magda  wrote:
>>> 
>>> Hello Aleksey,
>>> 
>>> Welcome to the Ignite community!
>>> 
>>> Share your JIRA ID with us so that we can add you to the contributors
>> list. After that you will be able to assign tickets on yourself.
>>> 
>>> As for this ticket, it’s no longer relevant and I’m going to close it.
>> Feel free to pick any other of your interest. Here is a good selection:
>>> https://ignite.apache.org/community/contribute.html#pick-ticket <
>> https://ignite.apache.org/community/contribute.html#pick-ticket>
>>> 
>>> Additional information that will be useful.
>>> 
>>> Subscribe to both dev and user lists:
>>> https://ignite.apache.org/community/resources.html#mail-lists <
>> https://ignite.apache.org/community/resources.html#mail-lists>
>>> 
>>> Get familiar with Ignite development process described here:
>>> https://cwiki.apache.org/confluence/display/IGNITE/Development+Process <
>> https://cwiki.apache.org/confluence/display/IGNITE/Development+Process>
>>> 
>>> Instructions on how to contribute can be found here:
>>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute <
>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute>
>>> 
>>> Project setup in Intellij IDEAL
>>> https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup <
>> https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup>
>>> 
>>> Looking forward to your contributions!
>>> 
>>> Regards,
>>> Denis
>>> 
 On Jan 27, 2017, at 2:21 AM, ALEKSEY KUZNETSOV <
>> alkuznetsov...@gmail.com > wrote:
 
 Добрый день! Я подключился проекту недавно и у меня возникли проблемы с
>> джирой. Нет прав на стар тикета и присвоения его себе. Вот тикет -
>> https://issues.apache.org/jira/browse/IGNITE-3385 <
>> https://issues.apache.org/jira/browse/IGNITE-3385>
 --
 Best Regards,
 
 Kuznetsov Aleksey
 
>>> 
>> 
>> --
> 
> *Best Regards,*
> 
> *Kuznetsov Aleksey*

Re: moving to geronimo JCache jar

[Denis Magda]

Alexander, thanks!

I’ll review it in the nearest couple of days.

—
Denis

> On Jan 30, 2017, at 5:10 AM, Alexander Fedotov  
> wrote:
> 
> Hi,
> 
> Created Upsource review for the subject:
> http://reviews.ignite.apache.org/ignite/review/IGNT-CR-82
> 
> On Mon, Jan 30, 2017 at 11:52 AM, Alexander Fedotov <
> alexander.fedot...@gmail.com> wrote:
> 
>> Hi all,
>> 
>> https://issues.apache.org/jira/browse/IGNITE-3793 is completed.
>> Kindly take a look at the corresponding PR https://github.com/apache/i
>> gnite/pull/1475 .
>> 
>> On Wed, Jan 25, 2017 at 8:04 PM, Denis Magda  wrote:
>> 
>>> We need to replace content of ignite-core-licenses.txt file which is the
>>> following at the moment
>>> 
>>> // --
>>> // List of ignite-core module's dependencies provided as a part of this
>>> distribution
>>> // which licenses differ from Apache Software License.
>>> // --
>>> 
>>> 
>>> ==
>>> For JSR107 API and SPI (https://github.com/jsr107/jsr107spec)
>>> javax.cache:cache-api:jar:1.0.0
>>> 
>>> ==
>>> This product bundles JSR107 API and SPI which is available under a:
>>> JSR-000107 JCACHE 2.9 Public Review - Updated Specification License. For
>>> details, see https://raw.github.com/jsr107/jsr107spec/master/LICENSE.txt.
>>> 
>>> 
>>> Updated this ticket description: https://issues.apache.org/jira
>>> /browse/IGNITE-3793
>>> 
>>> —
>>> Denis
 On Jan 24, 2017, at 8:24 PM, Dmitriy Setrakyan 
>>> wrote:
 
 Awesome, you are right. I just checked and the license is indeed Apache
 2.0. Is there anything we need to do at all right now?
 
 On Tue, Jan 24, 2017 at 8:17 PM, Valentin Kulichenko <
 valentin.kuliche...@gmail.com> wrote:
 
> This change was incorporated in this ticket: https://issues.apache.
> org/jira/browse/IGNITE-3793. We can't do it before 2.0 for
>>> compatibility
> reasons.
> 
> However, my point is that they changed the license to Apache 2.0, so
>>> I'm
> not sure that licensing issue still exists.
> 
> -Val
> 
> On Tue, Jan 24, 2017 at 7:04 PM, Dmitriy Setrakyan <
>>> dsetrak...@apache.org>
> wrote:
> 
>> Any reason why we need to wait for 2.0? Sorry if this has already been
>> discussed.
>> 
>> On Tue, Jan 24, 2017 at 7:02 PM, Denis Magda 
>>> wrote:
>> 
>>> Yes, we planned to do that in 2.0. Val, the ticket is closed
>>> https://issues.apache.org/jira/browse/IGNITE-2949 <
>>> https://issues.apache.org/jira/browse/IGNITE-2949>
>>> 
>>> Do we need to reopen it making sure that geronimo jar is added to
>>> 2.0?
>>> 
>>> —
>>> Denis
>>> 
 On Jan 24, 2017, at 6:36 PM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
>>> wrote:
 
 We absolutely need to upgrade to the geronimo jcache library in the
>> next
 release.
 
 On Tue, Jan 24, 2017 at 3:45 PM, Valentin Kulichenko <
 valentin.kuliche...@gmail.com> wrote:
 
> Guys,
> 
> I noticed that the JCache license was updated to Apache 2.0 several
>>> months
> ago [1]. However, there was no release with the new license and
> 1.0.0
>>> still
> has the old license name in the POM file [2] (the link is pointing
> to
>>> the
> new one though).
> 
> Is this enough from legal standpoint? Do we still need to move to
>>> Geronimo?
> 
> [1] https://github.com/jsr107/jsr107spec/blob/master/LICENSE.txt
> [2] http://mvnrepository.com/artifact/javax.cache/cache-api/1.0.0
> 
> -Val
> 
> 
> On Mon, Apr 11, 2016 at 5:43 PM, Dmitriy Setrakyan <
>>> dsetrak...@apache.org>
> wrote:
> 
>> I would say that we are OK with alpha for now, as there is no real
>> difference between 1.0-alpha and 1.0. We can switch to 1.0
>>> whenever
>> geronimo project updates the JAR.
>> 
>> D.
>> 
>> On Mon, Apr 11, 2016 at 5:10 PM, Valentin Kulichenko <
>> valentin.kuliche...@gmail.com> wrote:
>> 
>>> Folks,
>>> 
>>> I tried to switch to Geronimo and it works fine for me. Are we
> going
>>> to
>>> wait for version 1.0, or we're OK with alpha?
>>> 
>>> -Val
>>> 
>>> On Mon, Mar 28, 2016 at 7:37 AM, Dmitriy Setrakyan <
>> dsetrak...@apache.org>
>>> wrote:
>>> 
 Igniters,
 
 Can someone check if the Geronimo JCache jar is the same as the
> JSR107?
 
 
 
>>> http://mvnrepository.com/artifact/org.apache.gero

Re: Transactional tasks

[Alexei Scherbakov]

Alexey,

Every job is executed in transaction having same semantics as any other
transaction, but transaction is managed by Ignite transparently.

Job's transaction is rolled back on throwing any unchecked exception or
special TransactionRollbackException.

Concurrency issues between jobs are solved using entry locking or currency
primitives.

I think the closest similar thing is JTA transaction enlisting. Maybe I
even'll implement such logic using current Ignite JTA module, haven't tried
yet. But I don't like JTA overhead and a requirement to have JTA
coordinator.

2017-01-27 13:40 GMT+03:00 Alexey Goncharuk :

> Alexei,
>
> Can you describe the semantics of the task in greater detail?
>  * What if two jobs on different nodes access and try to put the same key?
> (this can be resolved by allowing a job only access local primary keys)
>

This is prefectly valid. Second transaction'll wait for completion of first.


>  * How do you define the lock acquisition order and prevent deadlocks? I
> assume that running such a compute task will involve a lot of keys, so it
> is quite unlikely that OPTIMISTIC SERIALIZABLE transactions are applicable
> here
>

Same as any other transaction - user is responsible for avoiding deadlock.


>  * Currently a started transaction locks the cache topology, so job
> failover in the case of node crash will be very hard to implement
>

Could you provide more details? Current job transactions must be rolled
back all together and task must be routed to other node.


>
> 2017-01-26 23:36 GMT+03:00 Alexei Scherbakov  >:
>
> > Guys,
> >
> > How difficult would be to support transactional tasks?
> >
> > What means every job in task executed in it's own transaction.
> >
> > In case of single job failure or reduce phase failure all transaction
> > started by jobs are rolled back.
> >
> > Only if all jobs are successfully executed, corresponding transactions
> are
> > commited.
> >
> > Also it would be very desirable to implement tasks failover in the
> similar
> > way how jobs failover is implemented.
> >
> > In case of master's failure jobs are rolled back, and task is restarted
> on
> > another node.
> >
> > This should greatly simplify implementing complex business processes.
> >
> > --
> >
> > Best regards,
> > Alexei Scherbakov
> >
>



-- 

Best regards,
Alexei Scherbakov

Re: How to improve SQL testing

[Denis Magda]

Hi,

That’s true that the utility/testing framework needs to be as flexible as 
possible. If the community can’t reuse an existing one let’s make it up.

As for VoltDB SQL Coverage Suite, if it’s Python based then can we implement 
those Python APIs directly calling Ignite.C++?

—
Denis

> On Jan 30, 2017, at 4:39 AM, Sergey Kozlov  wrote:
> 
> Alexey, I suppose that it's a hard way to do that and at least see a few
> disadvantages below:
> 
> 1. The VoltDB has no multithreaded mode. It means huge execution time
> consumption (say we talk about thousands statement multiplied on caches
> count, it's the millions of queries).
> 2. We should add support of Ignite.
> 3. We should add support of H2 as base engine for us (MySQL/Postgress have
> less priority)
> 4. They run it under Jenkins so we need to integrate it with TC somehow.
> 
> From my standpoint better to make the utility in Ignite than spending time
> and efforts to adopt an existing tool. At least we have good (or even best)
> Java experts/developers in community and the task doesn't look a complex
> one.
> 
> 
> On Mon, Jan 30, 2017 at 2:58 PM, Alexey Kuznetsov 
> wrote:
> 
>> Sergey,
>> 
>> Could we use http://www.jython.org/ to make it works as-is?
>> 
>> On Mon, Jan 30, 2017 at 6:06 PM, Sergey Kozlov 
>> wrote:
>> 
>>> Hi
>>> 
>>> Hi reviewed the links below (thanks for Vova Ozerov to pointing me) and
>>> found that the provided solution is really excellent!
>>> Unfortunately we can't use it as is (from my standpoint) due lacks of
>>> Python support in Ignite but we can re-use that idea for Ignite and make
>>> SQL combinations coverage significantly better.
>>> 
>>> I filed the ticket IGNITE-4627 Tester for SQL functionality
>>> 
>>> 
>>> Please share your thoughts and ideas.
>>> 
>>> [1] https://www.voltdb.com/blog/testing-voltdb-sqlcoverage
>>> [2] https://github.com/VoltDB/voltdb/tree/master/tests/sqlcoverage
>>> 
>>> --
>>> Sergey Kozlov
>>> www.gridgain.com
>>> 
>> 
>> 
>> 
>> --
>> Alexey Kuznetsov
>> 
> 
> 
> 
> -- 
> Sergey Kozlov
> GridGain Systems
> www.gridgain.com

Re: IGNITE-4212 (Ignite Benchmarking Simplification and Automation)

[Denis Magda]

Hi Oleg,

Great progress, thanks for keep driving this!

I’ve left some minor notes in GitHub’s pull-request. I have the following 
questions aside:

- What is the difference between "Building from standalone sources" and 
"Building from Ignite Sources"? In my understanding, a user downloads Apache 
Ignite release that has all the sources locally.

- I do remember we planned to add the benchmarks sources in a form of a ready 
to be used project with its own pom.xml (similar to examples). Did you put this 
task off?

—
Denis

> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin  wrote:
> 
> Hi!
> 
> I've changed the README.txt and DEVNOTES.txt files. Also added a simple
> config file for quick and easy start. Please take a look at them and tell
> me what you think.
> 
> https://github.com/apache/ignite/pull/1471
> 
> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov  wrote:
> 
>> Denis,
>> 
>> I think we can remove all configs except:
>> 
>> benchmark-multicast.properties
>> 
>> benchmark.properties
>> 
>> ignite-base-config.xml
>> 
>> ignite-localhost-config.xml
>> 
>> ignite-multicast-config.xml
>> 
>> 2016-12-28 2:49 GMT+03:00 Denis Magda :
>> 
>>> I would have only those configs that are useful. Ilya Suntsov, basing on
>>> your experience, please suggest which configs makes sense to include into
>>> every Ignite release.
>>> 
>>> Oleg, also please note that community decided to include not only the
>>> benchmarking binaries but the sources as well into every Apache Ignite
>>> release. I’ve update the ticket before. Hope you followed the discussion
>> ;)
>>> https://issues.apache.org/jira/browse/IGNITE-4212?
>>> focusedCommentId=15765151&page=com.atlassian.jira.
>>> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151
>>> 
>>> —
>>> Denis
>>> 
 On Dec 27, 2016, at 5:35 AM, Oleg Ostanin 
>> wrote:
 
 I mean removing those configs from binary assembly, not from
>> repository.
 
 On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin 
>>> wrote:
 
> Hello Igniters.
> I think it would be better to remove some configuration files from
> benchmarks/config:
> 
> ignite-base-load-config.xml
> ignite-cache-load-config.xml
> ignite-failover-base-config.xml
> ignite-failover-localhost-config.xml
> benchmark-cache-load.properties
> benchmark-cache-load-win.properties
> benchmark-failover.properties
> 
> because those configs do not relate to any of performance tests.
> 
> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda 
>>> wrote:
> 
>> Summarized the discussion updating the ticket
>> https://issues.apache.org/jira/browse/IGNITE-4212# <
>> https://issues.apache.org/jira/browse/IGNITE-4212#>
>> 
>> —
>> Denis
>> 
>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan <
>>> dsetrak...@apache.org>
>> wrote:
>>> 
>>> Sergey,
>>> 
>>> I am not sure I like "extras". I am voting for "benchmarks" folder
>>> right
>>> under the root folder.
>>> 
>>> D.
>>> 
>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov <
>> skoz...@gridgain.com
 
>>> wrote:
>>> 
 Formatting has cut lines:
 
 — apache_ignite_root_folder
 — bin
 — examples
 — extras
 — benchmarks
   — bin
   — src (benchmarks sources with pom.xml)
   — config
   — libs (compiled benchmarks)
 
 
 
 On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov <
>>> skoz...@gridgain.com>
 wrote:
 
> Denis,
> 
> Mostly yes. But I look ahead and think that we may include more
>> things in
> future than yardstick only. It's why I suggest something like
>> that:
> — apache_ignite_root_folder
>  — bin
>  — examples
>  — extras
>  — benchmarks
>  — bin
>  — src (benchmarks sources with pom.xml)
>  — config
>  — libs (compiled benchmarks)
> 
> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda 
>> wrote:
> 
>> Well, if to refer to Dmitriy suggestion we can have the following
>> structure
>> 
>> — apache_ignite_root_folder
>>  — examples
>>  — bin
>>  — benchmarks
>>  — bin
>>  — src (benchmarks sources with pom.xml)
>>  — config
>>  — libs (compiled benchmarks)
>> 
>> Sergey, will it cover all the use case you’ve met previously?
>> 
>> —
>> Denis
>> 
>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov <
>> skoz...@gridgain.com>
>> wrote:
>>> 
>>> Yardstick requires own scripts/configurations (/bin, /config,
>>> /libs)
 and
>>> creates work/logs directory under yardstick root.
>> "libs/optional"
>>> is
 for
>>> optional modules but in gen

Re: Transactional tasks

[Yakov Zhdanov]

>> This is prefectly valid. Second transaction'll wait for completion of
first.

Alex S., I think you did not get the point. It seems AG meant two jobs of
the same task, so it is one TX. It is unclear to me how to resolve this
automatically. I would agree with AG and permit only primary keys local to
the node job is running on.

>> Could you provide more details? Current job transactions must be rolled back
all together and task must be routed to other node.
AG, this sounds like a solution? Can we put hard requirement here - task TX
does not survive node failure before prepare completes?


--Yakov

2017-01-30 23:55 GMT+03:00 Alexei Scherbakov :

> Alexey,
>
> Every job is executed in transaction having same semantics as any other
> transaction, but transaction is managed by Ignite transparently.
>
> Job's transaction is rolled back on throwing any unchecked exception or
> special TransactionRollbackException.
>
> Concurrency issues between jobs are solved using entry locking or currency
> primitives.
>
> I think the closest similar thing is JTA transaction enlisting. Maybe I
> even'll implement such logic using current Ignite JTA module, haven't tried
> yet. But I don't like JTA overhead and a requirement to have JTA
> coordinator.
>
> 2017-01-27 13:40 GMT+03:00 Alexey Goncharuk :
>
> > Alexei,
> >
> > Can you describe the semantics of the task in greater detail?
> >  * What if two jobs on different nodes access and try to put the same
> key?
> > (this can be resolved by allowing a job only access local primary keys)
> >
>
> This is prefectly valid. Second transaction'll wait for completion of
> first.
>
>
> >  * How do you define the lock acquisition order and prevent deadlocks? I
> > assume that running such a compute task will involve a lot of keys, so it
> > is quite unlikely that OPTIMISTIC SERIALIZABLE transactions are
> applicable
> > here
> >
>
> Same as any other transaction - user is responsible for avoiding deadlock.
>
>
> >  * Currently a started transaction locks the cache topology, so job
> > failover in the case of node crash will be very hard to implement
> >
>
> Could you provide more details? Current job transactions must be rolled
> back all together and task must be routed to other node.
>
>
> >
> > 2017-01-26 23:36 GMT+03:00 Alexei Scherbakov <
> alexey.scherbak...@gmail.com
> > >:
> >
> > > Guys,
> > >
> > > How difficult would be to support transactional tasks?
> > >
> > > What means every job in task executed in it's own transaction.
> > >
> > > In case of single job failure or reduce phase failure all transaction
> > > started by jobs are rolled back.
> > >
> > > Only if all jobs are successfully executed, corresponding transactions
> > are
> > > commited.
> > >
> > > Also it would be very desirable to implement tasks failover in the
> > similar
> > > way how jobs failover is implemented.
> > >
> > > In case of master's failure jobs are rolled back, and task is restarted
> > on
> > > another node.
> > >
> > > This should greatly simplify implementing complex business processes.
> > >
> > > --
> > >
> > > Best regards,
> > > Alexei Scherbakov
> > >
> >
>
>
>
> --
>
> Best regards,
> Alexei Scherbakov
>