Re: Help needed

2017-02-10 Thread Ranjit Sahu
Hi Jorn, The use case what we are working on is something like this. We have three functions, one to extractEntity, one to resolve and to resolve we need an accesor. So first we build a RDD calling extract function in Step 1 below. Next we prepare the look up RDD and lets call it as Accesor. Thir

Re: Help needed

2017-02-10 Thread Ranjit Sahu
Hi Jorn, The use case what we are working on is something like this. We have three functions, one to extractEntity, one to resolve and to resolve we need an accesor. So first we build a RDD calling extract function in Step 1 below. Next we prepare the look up RDD and lets call it as Accesor. Thir

Re: Can continuousAsyncQuery guarantee event to be processed in order?

2017-02-10 Thread ght230
Does what you said "notification guarantees" mean "exactly once delivery of an event"? Does it also mean "keep the order of the events"? -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Can-continuousAsyncQuery-guarantee-event-to-be-processed-in-order-tp10521p105

Re: EntryProcessor for cache

2017-02-10 Thread Anil
and Does compute task execute in parallel on number of partitions per node ? (like entry processor) thanks. On 10 February 2017 at 10:52, Anil wrote: > Hi Val, > > i have attached the code. please let me know if you see any issues with > approach. thanks. > > Thanks. > > On 10 February 2017 at 0

Bug with using Map as a cache key in Ignite

2017-02-10 Thread Ramzinator
Hi GridGain/Ignite guys, I've recently encountered a bug while using Ignite. The scenario is as follows:I have a class *A* that just has a *java.util.Map* field that I am using as an ignite cache key to map to some value. I have 2 instances of the same service: *S1* & *S2* , each running an igni

How to submit job to Ignite Yarn application

2017-02-10 Thread dpolacza
Hi, I have submitted ignite as yarn application like it is described in chapter "yarn-deployment". In this case Ignite should schedule resources in resourcemanager and install ignite nodes automatically But how to send job to this application? How "Configuration" object should be defined? Regards

ignite as Hadoop's Map Reduce engine - how it works?

2017-02-10 Thread dpolacza
Hi, Can someone describe how MapReduce is implemented when hadoop's job is processed? How does shuffling work? How is Reduce node chosen? Is it possible to speedup job scheduling by changing standard settings? I saw in Debug log that some transactions are initiated is it possible to turn it off?

Continuous reduce in Ignite

2017-02-10 Thread Raymond Wilson
I’m looking at the very nice MapReduce implementation in Ignite and have a question about the reduce stage. It appears that the reduce stage only occurs once all mapped results have been accumulated. I have a use case where the size of that data set is very large and problematic to hold in memor

Re: Continuous reduce in Ignite

2017-02-10 Thread vkulichenko
Hi Raymond, Are you implementing tasks in Java or .NET? For Java there is a special @ComputeTaskNoResultCache annotation that prevents from caching all job results in a single collection. Not sure there is such an option for .NET though. -Val -- View this message in context: http://apache-ign

Re: Bug with using Map as a cache key in Ignite

2017-02-10 Thread vkulichenko
Responded on StackOverflow: http://stackoverflow.com/questions/42161278/bug-using-map-as-a-cache-key-in-ignite -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Bug-with-using-Map-as-a-cache-key-in-Ignite-tp10549p10554.html Sent from the Apache Ignite Users m

Re: Can continuousAsyncQuery guarantee event to be processed in order?

2017-02-10 Thread vkulichenko
Yes, it does. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Can-continuousAsyncQuery-guarantee-event-to-be-processed-in-order-tp10521p10555.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: restore Java Object from BinaryObject

2017-02-10 Thread vkulichenko
Shawn, What's the actual issue after you disabled compact footer? Exception? Incorrect behavior? -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/restore-Java-Object-from-BinaryObject-tp10525p10556.html Sent from the Apache Ignite Users mailing list archive

Re: loadCache with ExpiryPolicy

2017-02-10 Thread vkulichenko
Hi, When done right, expiry policy provided in CacheConfiguration must be applied to entries loaded from store. However there was an issue that was fixed recently (not released yet): https://issues.apache.org/jira/browse/IGNITE-3699 As for NPE, not sure why this happens. What is null there? -Val

Re: Help needed

2017-02-10 Thread vkulichenko
Ranjit, I would recommend you to start a standalone Ignite cluster, separate from Spark. You may also need to specify clientMode=true in the configuration provided to IgniteContext, to make sure that nodes on executors are clients, not servers. -Val -- View this message in context: http://apa

Re: EntryProcessor for cache

2017-02-10 Thread vkulichenko
Looks ok except that the first query should also be local I guess. Also note that you used split adapter, so didn't actually map the jobs to nodes, leaving this to Ignite. This means that there is a chance some nodes will get more than one job, and some none of the jobs. Round robin balancing is us

Re: Continuous reduce in Ignite

2017-02-10 Thread Raymond Wilson
Hi Val, This would be from .Net. Is there an option for extending the .Net client to support this? Thanks, Raymond. Sent from my iPhone > On 11/02/2017, at 11:19 AM, vkulichenko wrote: > > Hi Raymond, > > Are you implementing tasks in Java or .NET? For Java there is a special > @ComputeTas

Ignite client connections hanged

2017-02-10 Thread Raja
IgniteCluster.jpg Hi, I have Ignite and Web applications deployment as shown in the attached image. I added couple of new functions to th

Re: NOT IN in ignite

2017-02-10 Thread Anil
Hi Val, Do you guys accept this behavior as bug ? correct it so that group by with non affinity key returns same results with/without collocated true ? Thanks. On 9 February 2017 at 09:40, Anil wrote: > Hi Val, > > You are right. i collocated the data and set the collocated = true. and it > im

Ignite for fine-grained policy/quota control

2017-02-10 Thread Evans Ye
Hi Igniters, We recently are using Ignite building a real-time notification system. Ignite plays a crucial part on policy control for notifications. For example, we would like a user who has a quota of 100 notifications per day gets 100 messages per day precisely. Once quota exceeded, Further noti