RE: CacheAbstractJdbcStore batch deleteAll bug.

2017-05-10 Thread Gordon Reid (Nine Mile)
Sorry, I don’t have time to produce a sample right now, but I can see that this bug still exists in ignite 2.0 Just look in CacheAbstractJdbcStore.deleteAll You can see that delStmt never gets updated after it is first initialized Thanks, Gordon. From: Alexey Kuznetsov

Re: vertx-ignite

2017-05-10 Thread Anil
HI Andrey, i am using vertx-ignite 3.4.1 and ignite 1.9 version. i will check the default-ignite.xml. Thanks On 10 May 2017 at 21:31, Andrey Gura wrote: > Anil, > > What version of vertx-ignite or Ignite itself do you use? > > In provided ignite.xml there is no minimal

Re: Write Behind with delete performance

2017-05-10 Thread Sasha Belyak
Yes, if key wasn't flushed (end event wasn't collected by flusher thread to flush) - it will be just overwritten in memory with new value (doesn't matter what operation, i.e. insert can be overwritten to delete or visa versa). So in your example if WB cache get insert k1 =0, k2=2, k3 =3 and delete

Re: [ANNOUNCE] Apache Ignite 2.0.0 Released

2017-05-10 Thread Denis Magda
Pavel, excellent blog post! Truly profound with cool source code examples: https://twitter.com/denismagda/status/862460325211197441 — Denis > On May 10, 2017, at 8:26 AM, Pavel Tupitsyn wrote: > > Ignite.NET 2.0 blog post: >

Re: Ignite2.0 and Memory Policies

2017-05-10 Thread Denis Magda
Hi, > 1. non-heap memory used increased for both nodes. Does this confirm that > data went to off-heap memory? The data gets to off-heap via Java heap. This is why you see heap metrics varying over the time. > 2. I am just doing a cacheStreamer.addData(). If data is going into off-heap >

Re: Sizing in Ignite

2017-05-10 Thread Denis Magda
Hi, Honestly, it’s unclear why you use multipliers like 2.5 and 1.3 in the formula. Please refer to this capacity guide to make up a rough estimation: https://apacheignite.readme.io/docs/capacity-planning — Denis > On May 10, 2017, at

Re: Pessimistic TXN did not release lock on a key, all subsequent txns failed

2017-05-10 Thread bintisepaha
Hey guys, we had a key lock issue again on 1.7.0. here is a suspicious thread dump. Is this helpful for tracking down our issue further? we did not see any topology changes or any other exceptions. Attaching the entire thread dump too tdump.zip

Re: How to monitor and alert for server counts

2017-05-10 Thread ignite_user2016
we did not made any tweak to memory configuration and our configuration is a bare bone configuration. Please note that our need with ignite is not big data. On Wed, May 10, 2017 at 2:08 AM, tysli2016 [via Apache Ignite Users] < ml+s70518n12582...@n6.nabble.com> wrote: > we run the visor in 10

Re: Closures stuck in 2.0 when try to add an element into the queue.

2017-05-10 Thread fatality
Hi I have just subscribed to the user group also same question is posted at http://stackoverflow.com/questions/43891757/closures-stuck-in-2-0-when-try-to-add-an-element-into-the-queue -- View this message in context:

Re: GroupBy with index is really slow.

2017-05-10 Thread Guillermo Ortiz
Yes, **Result SELECT distinct(age) FROM PERSONWITHINDEX 97,98,99,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19, 20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39, 40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,

Re: GroupBy with index is really slow.

2017-05-10 Thread Andrey Mashenkov
Hi, I've just think you can achive same result with query without groupBy. Select distinct age from .. Does it workable for you? 10 мая 2017 г. 17:56 пользователь "Guillermo Ortiz" написал: > It's my laptop. 16gb, i7.. The collection size it's 5Mill objects (1,2GB) > and

Re: Docker cluster set up

2017-05-10 Thread afedotov
Also, setting --net=host when running both of containers may probably help too. Kind regards, Alex. On Wed, May 10, 2017 at 8:28 PM, waterg [via Apache Ignite Users] < ml+s70518n12605...@n6.nabble.com> wrote: > Thank you Alex! > > On Wed, May 10, 2017 at 1:54 AM, afedotov <[hidden email] >

Re: Docker cluster set up

2017-05-10 Thread Jessie Lin
Thank you Alex! On Wed, May 10, 2017 at 1:54 AM, afedotov wrote: > Hi. > Did I get it right that containers reside on different hosts? If it's so > then containers won't see each other out of box. You probably need to start > them via docker overlay network or use

Re: Write Behind with delete performance

2017-05-10 Thread waterg
Thank you for the quick explaining and creating jira tickets. Just a thought: If insert k1 =0, k2=2, k3 =3 and delete k1=0 and k1 is not being flushed to store yet, would it be possible to just delete in memory, w/o flushing to store? and deleteAll for all k1 like records that have been flushed

Re: Closures stuck in 2.0 when try to add an element into the queue.

2017-05-10 Thread afedotov
Hi, Please properly subscribe to the mailing list so that the community can receive email notifications for your messages. To subscribe, send empty email to user-subscr...@ignite.apache.org and follow simple instructions in the reply. Kind regards, Alex. On Wed, May 10, 2017 at 2:30 PM,

Re: GroupBy with index is really slow.

2017-05-10 Thread afedotov
Hi. I think the problem is that age and month have bad selectivity. For example, in the case of month, each month index will select in average 200 / 12 ~ 167 rows. In this case, it's probably faster scan all the rows in storage/disk order then using the index order. Kind regards, Alex.

Re: vertx-ignite

2017-05-10 Thread Andrey Gura
Anil, What version of vertx-ignite or Ignite itself do you use? In provided ignite.xml there is no minimal configuration that is mandatory for Ignite cluster manager for vert.x (see default-ignite.xml for example). On Tue, May 2, 2017 at 9:18 AM, Anil wrote: > > Hi Andrey,

Re: GroupBy with index is really slow.

2017-05-10 Thread Guillermo Ortiz
Are indices used always that it'spossible or it depends how many records data you have to read as Oracle? So, if You use and index but the result it's to read 25% of the data, usually indices are not used because sequencial read are faster. As this is in memory I understand that it doesn't apply

Re: [ANNOUNCE] Apache Ignite 2.0.0 Released

2017-05-10 Thread Pavel Tupitsyn
Ignite.NET 2.0 blog post: https://ptupitsyn.github.io/Whats-New-In-Ignite-Net-2.0/ On Wed, May 10, 2017 at 12:44 PM, Pavel Tupitsyn wrote: > Great news! > > Thanks Denis, I've done some minor corrections to .NET part of release > notes page (IGNITE-4495 does not belong

Re: GroupBy with index is really slow.

2017-05-10 Thread Guillermo Ortiz
If I execute many times, time is similar in both. 2017-05-10 17:25 GMT+02:00 Guillermo Ortiz : > I'm using 2.0.0. > Right now in my laptop with 2Mill objects. > > > *WITH INDICES* > **Result SELECT distinct(MONTH) FROM PERSONWITHINDEX >

Re: GroupBy with index is really slow.

2017-05-10 Thread Guillermo Ortiz
I'm using 2.0.0. Right now in my laptop with 2Mill objects. *WITH INDICES* **Result SELECT distinct(MONTH) FROM PERSONWITHINDEX September,February,March,August,June,April,July,January,May,November,October, Elapsed time SELECT distinct(MONTH) FROM PERSONWITHINDEX:1990ms [SELECT DISTINCT

Re: Group indices and group by query

2017-05-10 Thread Guillermo Ortiz
Is there any limitation like databases which if it has to read more than x% of data it do a full scan? I tried this: Result *SELECT MONTH, MAX(ID) FROM PERSONWITHINDEX GROUP BY MONTH *

Re: HDP, Hive + Ignite

2017-05-10 Thread Ivan Veselovsky
Alena, regarding NPEs in Ignite node logs, this seems to be https://issues.apache.org/jira/browse/IGNITE-4862 , fixed, but not yet merged. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/HDP-Hive-Ignite-tp12195p12594.html Sent from the Apache Ignite Users

Re: GroupBy with index is really slow.

2017-05-10 Thread afedotov
Hi, Which Ignite version do you use? How many Ignite nodes do you run? What is your cache configuration? As well, could you please try the following query and see the performance? SELECT DISTINCT age FROM PersonWithindex; Kind regards, Alex. On Wed, May 10, 2017 at 5:57 PM, Guillermo Ortiz

Re: GroupBy with index is really slow.

2017-05-10 Thread Guillermo Ortiz
It's my laptop. 16gb, i7.. The collection size it's 5Mill objects (1,2GB) and it takes about 30sec. I have tried to execute the query with the same collection with and without indices with the same time results. This is a test before to run in the real cluster with 6 nodes of 512Gb and 48cores

Using Native Infiniband for node interconnect?

2017-05-10 Thread connie.woodward
I see posts about people using IP over IB for inter-node communication, but is it possible to use native IB protocols? (IB RDMA, etc.). I googled around and saw that this was possible with Apache Spark, but what about for Ignite? instructions? thanks -- View this message in context:

RE: How to correctly shut down Ignite Application

2017-05-10 Thread Juan Barani
Thanks Andrey, For the moment it works correctly with preferIPv4Stack. The issue first appeared on 2.0, we run the same test suite since 1.6, and we have never seen it. Thanks again, Juan From: Andrey Mashenkov [mailto:andrey.mashen...@gmail.com] Sent: Wednesday, May 10, 2017 12:32 PM To:

Re: HDP, Hive + Ignite

2017-05-10 Thread Alena Melnikova
Hi Ivan, 1. I tried to run analytical query on table that created in IGFS. Here couple of examples of errors. beeline_output_1.log ignite-node-dn1_1.log

Re: How to correctly shut down Ignite Application

2017-05-10 Thread Andrey Mashenkov
Hi Juan, Looks like there is a bug, sometimes ignite choose ipv6 protocol to communicate with other nodes that causes strange communication errors. On Wed, May 10, 2017 at 1:01 PM, Juan Barani wrote: > Hi, > > > > Not necessarily is the case that the client

RE: How to correctly shut down Ignite Application

2017-05-10 Thread Juan Barani
Hi, Not necessarily is the case that the client cannot connect to the server. I experienced yesterday with ignite 2.0, that when a client is forced with: l_discoverySpi.setForceServerMode(true); it comes as isClient=false in the exception message, even if setClientMode=true. Since updating to

Re: [ANNOUNCE] Apache Ignite 2.0.0 Released

2017-05-10 Thread Pavel Tupitsyn
Great news! Thanks Denis, I've done some minor corrections to .NET part of release notes page (IGNITE-4495 does not belong there). On Fri, May 5, 2017 at 11:13 PM, Denis Magda wrote: > The Apache Ignite Community is pleased to announce the release of Apache > Ignite 2.0.0. >

Re: Docker cluster set up

2017-05-10 Thread afedotov
Hi. Did I get it right that containers reside on different hosts? If it's so then containers won't see each other out of box. You probably need to start them via docker overlay network or use Swarm. Kind regards, Alex 9 мая 2017 г. 12:29 AM пользователь "waterg [via Apache Ignite Users]" <

Re: How to monitor and alert for server counts

2017-05-10 Thread tysli2016
we run the visor in 10 every mins, the Ignite servers would OOME in a couple weeks. Ignite server allocated 1g memory. would you mind to share you server's config.xml? any JVM parameters changes? thanks Tom ignite_user2016 wrote > yes you are correct, we run visor in the batch mode. > > we