Re: Ignite Compute - Rebalance Scenario

[Kamal C]

Understood. Thanks!

Regards
Kamal C


On Tue, Aug 16, 2016 at 3:47 PM, Taras Ledkov  wrote:

> Hi,
>
> Affinity job doesn't block rebalance. We reserve partition to prevent
> remove data from the node while job is running.
> So, topology is changed, partitions are moved to another nodes but data is
> available locally during affinity job. Magic!
> Please modify your test like below and be sure that data is available for
> local query.
>
> @IgniteInstanceResource
> private Ignite ig;
> ...
> public void run() {
> ...
> while (true) {
> assert ignite.cache(CACHE_NAME).localPeek('a', null).equals('a');
>
> On 16.08.2016 11:32, Kamal C wrote:
>
> Vladislav,
>
> Partitions are moved to another node while executing the job.
>
> Val,
>
> I've tried with the latest nighty build. Still, it has the same behavior.
> To reproduce the issue, I've used `while(true)` inside the IgniteRunnable
> task. You can reproduce it with the below gist[1].
>
> [1]: https://gist.github.com/Kamal15/0a4066de152b8ebc856fc264f7b4037d
>
> Regards,
> Kamal C
>
> On Sat, Aug 13, 2016 at 12:15 AM, vkulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
>> Note that this was changed recently [1] and the change was not released in
>> 1.7. Kamal, can you try the nightly build [2] and check if it works as you
>> expect there?
>>
>> [1] https://issues.apache.org/jira/browse/IGNITE-2310
>> [2]
>> https://builds.apache.org/view/H-L/view/Ignite/job/Ignite-
>> nightly/lastSuccessfulBuild/
>>
>>
>>
>> --
>> View this message in context: http://apache-ignite-users.705
>> 18.x6.nabble.com/Ignite-Compute-Rebalance-Scenario-tp7004p7021.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>
> --
> Taras Ledkov
> Mail-To: tled...@gridgain.com
>
>

Re: what are the differences between put and replace method

[vdpyatkov]

Hello,

Replace method is equivalent to the code snippet:

/if (cache.containsKey(key)) {
   cache.put(key, value);
   return true;
 } else {
   return false;
 }/

Can you lock at the jcache Javadoc:
http://ignite.apache.org/jcache/1.0.0/javadoc/javax/cache/Cache.html#replace(K,
V)
http://ignite.apache.org/jcache/1.0.0/javadoc/javax/cache/Cache.html#put(K,
V)


timothy wrote
> what are the differences between put and replace API? in the document,
> replace method is described as Atomic method





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/what-are-the-differences-between-put-and-replace-method-tp7117p7128.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: LOOK THORUGH THIS ERROR

[Vladislav Pyatkov]

Ravi,

Can you provide example project, where throwing the exception?

On Wed, Aug 17, 2016 at 9:58 AM, Ravi Puri 
wrote:

> No i'm not using maven . I am adding this jars to project and using it..
> Everything was working fine but this error doesnt allow me to go forward..?
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/LOOK-THORUGH-THIS-ERROR-tp6977p7125.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>



-- 
Vladislav Pyatkov

Re: Cannot get the data in Tableau

[Igor Sapego]

Austin,

Can you run some simple custom SQL query like "select * from Person" and
share your results?
There should be an error message.

Best Regards,
Igor

On Tue, Aug 16, 2016 at 5:59 PM, austin solomon  wrote:

> Hi Igor,
>
> I have specified "Cache" connection argument while connecting to Ignite's
> ODBC, also I can see the column names of the created schema. (Check my
> previous post for screen shots)
>
> I can't  able to preview or query the cache.
>
> Below is my full xml configuration.
>
> http://www.springframework.org/schema/beans";
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>xmlns:util="http://www.springframework.org/schema/util";
>xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd
> http://www.springframework.org/schema/util
> http://www.springframework.org/schema/util/spring-util.xsd";>
>
>
>  class="org.springframework.beans.factory.config.
> PropertyPlaceholderConfigurer">
>  value="SYSTEM_PROPERTIES_MODE_FALLBACK"/>
> 
> 
>
>  class="org.apache.ignite.configuration.IgniteConfiguration">
>
>
> 
>  class="org.apache.ignite.configuration.OdbcConfiguration">
> 
>
> 
> 
>  class="org.apache.ignite.configuration.CacheConfiguration">
> 
> 
> 
>  value="FULL_SYNC"/>
>
>
> 
> 
>  class="org.apache.ignite.cache.CacheTypeMetadata">
>  value="java.lang.Long"/>
> 
> 
> 
>  value="java.lang.Long"/>
> 
> 
> 
> 
>  value="java.lang.Integer"/>
>  value="java.lang.String"/>
>  value="java.lang.String"/>
>  value="java.lang.String"/>
>  value="java.lang.Integer"/>
> 
> 
> 
> 
> 
> 
>
>  class="org.apache.ignite.configuration.CacheConfiguration">
> 
> 
> 
>  value="FULL_SYNC"/>
>
>
> 
> 
>  class="org.apache.ignite.cache.CacheTypeMetadata">
>  value="java.lang.Long"/>
>  value="Organization"/>
> 
> 
>  value="java.lang.String"/>
> 
> 
> 
> 
> 
> 
> 
> 
>
> 
>  class="org.apache.ignite.configuration.ConnectorConfiguration">
> 
> 
> 
>
>
>
> 
>  class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
> 
>
>
>
>  class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.
> TcpDiscoveryMulticastIpFinder">
> 
> 
>
> 10.10.1.33:47500..47509
> 
> 
> 
> 
> 
> 
> 
> 
>
> Thanks,
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Cannot-get-the-data-in-Tableau-tp6876p7104.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Cannot get the data in Tableau

[austin solomon]

Hi Igor, 

I was able to fix the issue, what I did is I removed the  from the xml.

Now I can query the cache, but the problem is only the string columns data
are showng (ie. firstName, lastName & resume ). The columns with Double,
Long Integer are not coming.

Following is the java code am using: 

public class Person implements Serializable {
/**
 * 
 */
private static final long serialVersionUID = 1167624889662198398L;

/** */
private static final AtomicLong ID_GEN = new AtomicLong();

/** Person ID (indexed). */
@QuerySqlField(index = true)
public Long id;

/** Organization ID (indexed). */
@QuerySqlField(index = true)
public Long orgId;

/** First name (not-indexed). */
@QuerySqlField
public String firstName;

/** Last name (not indexed). */
@QuerySqlField
public String lastName;

/** Resume text (create LUCENE-based TEXT index for this field). */
@QueryTextField
public String resume;

/** Salary (indexed). */
@QuerySqlField(index = true)
public double salary;

/** Custom cache key to guarantee that person is always collocated with
its organization. */
private transient AffinityKey key;

public Person(Organization org, String firstName, String lastName,
double salary, String resume) {
// Generate unique ID for this person.
id = ID_GEN.incrementAndGet();

orgId = org.id();

this.firstName = firstName;
this.lastName = lastName;
this.salary = salary;
this.resume = resume;
}

public Person(Long id, Long orgId, String firstName, String lastName,
double salary, String resume) {
this.id = id;
this.orgId = orgId;
this.firstName = firstName;
this.lastName = lastName;
this.salary = salary;
this.resume = resume;
}

public Person(Long id, String firstName, String lastName) {
this.id = id;

this.firstName = firstName;
this.lastName = lastName;
}

public static void main(String[] args) throws IgniteException {
try (Ignite ignite = Ignition.start("config/test-odbc.xml")) {
CacheConfiguration orgCacheCfg = new
CacheConfiguration<>("Organization");

  orgCacheCfg.setCacheMode(CacheMode.PARTITIONED); // Default.
  orgCacheCfg.setIndexedTypes(Long.class, Organization.class);
  
  CacheConfiguration, Person> 
personCacheCfg =
new CacheConfiguration<>("Person");

  personCacheCfg.setCacheMode(CacheMode.PARTITIONED); // 
Default.
  personCacheCfg.setIndexedTypes(AffinityKey.class, 
Person.class);

  // Populate cache.
  try (
IgniteCache orgCache =
ignite.getOrCreateCache(orgCacheCfg);
IgniteCache, Person> 
personCache =
ignite.getOrCreateCache(personCacheCfg)
  ) {
orgCache.clear();

// Organizations.
Organization org1 = new Organization("ApacheIgnite");
Organization org2 = new Organization("Other");

orgCache.put(org1.id(), org1);
orgCache.put(org2.id(), org2);

personCache.clear();

// People.
Person p1 = new Person(org1, "John","Doe", 2000,"Fresher");
Person p2 = new Person(org1,"Jane", "Wilson", 3000,
"Experienced");
Person p3 = new Person(org2, "Edward", "Smith", 2000,
"Intermediate");
Person p4 = new Person(org2, "David", "Warn", 5000, 
"Expert");
Person p5 = new Person(org2, "Tim", "Cook", 5000, "Expert");


personCache.put(p1.key(), p1);
personCache.put(p2.key(), p2);
personCache.put(p3.key(), p3);
personCache.put(p4.key(), p4);
personCache.put(p5.key(), p5);

  }
}

}


public AffinityKey key() {
if (key == null)
key = new AffinityKey<>(id, orgId);
System.out.println("Key==="+key);

return key;
}

}  


Thanks



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Cannot-get-the-data-in-Tableau-tp6876p7131.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Fail to join topology and repeat join process

[Vladislav Pyatkov]

Hi,

Communication unlikely influence to Discovery, because they use difference
thread pools.

Follow to steps:
1) Exclude Communication from DEBUG log level, and left in place DEBUG
level for package org.apache.ignite.spi.discovery.
2) Provide log file from a node, where a message like  "Node FAILED:
TcpDiscoveryNode..." appear first by time.
3) Provide GC log file from the node, who was joined.

After all logs will be received, we will move forward.

On Tue, Aug 16, 2016 at 12:45 PM, Jason  wrote:

> hi Vladislav,
>
> By scanning through the log, found that the serialization error may be the
> root cause. When the previous node of the new added node tried to send the
> messages (NodeAddedMessage/NodeAddFinishedMessage) to the new node, it
> failed, so it marked the new node as "failed" and sent an NodeFailedMessage
> to coordinator and when the coordinator received that, removed the new node
> and notified all the other nodes.
>
> I've attached the serialization error, would you like to help take a look
> at?
>
> Thanks,
> -Jason
>
> serialization_error.txt
>  n7092/serialization_error.txt>
>
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Fail-to-join-topology-and-repeat-join-
> process-tp6987p7092.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>



-- 
Vladislav Pyatkov

Re: Failed to deserialize the job (.net) using code from examples

[serrnovik]

I'm back with the same issue. I've just changed to 1.7 version (both nuget
and binary execution). Same test. As before if it is just nuget nodes that
installed then everything is perfect. If I add at least one node from binary
distribution - execution fails when I try to compute with not very
descriptive null reference: 


/
> System.NullReferenceException : Object reference not set to an instance of
> an object.
>at Apache.Ignite.Core.Impl.Common.Future`1.Get()
/

Full error from the console attached bellow. 

Other than nuget vs project I run binary as following: 
C:\apache-ignite\apache-ignite-fabric-1.7.0-bin\platforms\dotnet\bin\Apache.Ignite.exe
-assembly="MY ASSEMBLY PATH"


/
> [15:48:04,700][SEVERE][pub-#15%null%][GridJobWorker] Failed to execute job
> [jobId=ba932c89651-5dd85491-0bce-4872-83c5-49d4943cc246,
> ses=GridJobSessionImpl [ses=GridTaskSessionImpl
> [taskName=o.a.i.i.processors.platform.compute.PlatformBalancingMultiClosureTask,
> dep=LocalDeployment [super=GridDeployment [ts=1471441453448,
> depMode=SHARED, clsLdr=sun.misc.Launcher$AppClassLoader@c387f44,
> clsLdrId=3dabeb89651-469415f2-3eb4-4221-a5db-d5d3e6f39441, userVer=0,
> loc=true, sampleClsName=java.lang.String, pendingUndeploy=false,
> undeployed=false, usage=0]],
> taskClsName=o.a.i.i.processors.platform.compute.PlatformBalancingMultiClosureTask,
> sesId=aa932c89651-5dd85491-0bce-4872-83c5-49d4943cc246,
> startTime=1471441684674, endTime=9223372036854775807,
> taskNodeId=5dd85491-0bce-4872-83c5-49d4943cc246,
> clsLdr=sun.misc.Launcher$AppClassLoader@c387f44, closed=false, cpSpi=null,
> failSpi=null, loadSpi=null, usage=1, fullSup=false,
> subjId=5dd85491-0bce-4872-83c5-49d4943cc246, mapFut=IgniteFuture
> [orig=GridFutureAdapter [resFlag=0, res=null, startTime=1471441684699,
> endTime=0, ignoreInterrupts=false, state=INIT]]],
> jobId=ba932c89651-5dd85491-0bce-4872-83c5-49d4943cc246]]
> class org.apache.ignite.IgniteException: Native platform exception
> occurred.
> at
> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:908)
> at
> org.apache.ignite.internal.processors.platform.compute.PlatformAbstractJob.execute(PlatformAbstractJob.java:83)
> at
> org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:509)
> at
> org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6521)
> at
> org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:503)
> at
> org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:456)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at
> org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1161)
> at
> org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1766)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1238)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:866)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:106)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:829)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: PlatformNativeException [cause=BinaryObject [idHash=1737396909,
> hash=28379535, typeId=87]]
> at
> org.apache.ignite.internal.processors.platform.PlatformContextImpl.createNativeException(PlatformContextImpl.java:652)
> at
> org.apache.ignite.internal.processors.platform.utils.PlatformUtils.readInvocationResult(PlatformUtils.java:770)
> at
> org.apache.ignite.internal.processors.platform.compute.PlatformClosureJob.execute0(PlatformClosureJob.java:74)
> at
> org.apache.ignite.internal.processors.platform.compute.PlatformAbstractJob.execute(PlatformAbstractJob.java:80)
> ... 14 more
/




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Failed-to-deserialize-the-job-net-using-code-from-examples-tp6708p7134.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Deploying service programmatically hangs the app

[juanma.cvega]

Hi,

sorry I haven't replied before but I'm on holidays at the moment. Someone
in my team may have tried it already but I won't know until next Tuesday.
I'll reply back once I get back to work.

Thanks.


2016-08-08 11:50 GMT+02:00 vdpyatkov [via Apache Ignite Users] <
ml-node+s70518n6848...@n6.nabble.com>:

> Hello,
>
> I try to reproduce your issue, but my code not hangs.
> See attachment example.
>
> On Fri, Aug 5, 2016 at 7:03 PM, juanma.cvega <[hidden email]
> > wrote:
>
>> Hi,
>>
>> I already replied to the confirmation email twice. Is there any other
>> thing
>> I have to do?
>>
>> The place were it hangs is when calling
>> ignite.services().deploy(serviceConfiguration()) from inside the bean
>> IgniteClusterSingletonStarter. The full code is in the first post. If I
>> remove that bean and simply provide ServiceConfiguration in the
>> IgniteConfiguration bean (code is also in the first post), I can see it's
>> registered by the logs from inside the init() method in the TestService
>> implementation. So basically, registering the cluster singleton through
>> xml
>> configuration before create the Ignite instance works. If I create the
>> Ignite instance and then try to create the cluster singleton it hangs.
>> I don't know exactly what you mean with reusing IgniteConfiguration,
>> CommunicationSPI and discoverySPI, sorry. The whole Ignite configuration
>> is
>> attached in the first post. From there, what I do in some beans is inject
>> the Ignite instance to create a queue, a topic and listeners to node
>> events.
>> This is the code for that. The only part missing in the code is the
>> cluster
>> singleton that should produce messages to the distributed queue:
>>
>> public TopicSubscriber(Ignite ignite,
>> String topicName,
>> Service service){
>>   this.service = service;
>>   this.ignite = ignite;
>>   this.topicName = topicName;
>>}
>>
>>@PostConstruct
>>private void initSubscriber() {
>>   ignite.message(ignite.cluster().forRemotes()).localListen(to
>> picName,
>> (uuid, deleteRequest) -> {
>>  service.remove((Request) deleteRequest);
>>  return true;
>>   });
>>}
>>
>> public class EventListener {
>>
>>private final Service service;
>>private final IgniteEvents igniteEvents;
>>
>>public PriceChangeAlertClusterEventListener(IgniteEvents igniteEvents,
>>Service service) {
>>   this.service = service;
>>   this.igniteEvents = igniteEvents;
>>}
>>
>>@PostConstruct
>>private void initializeEventsListener() {
>>   igniteEvents.localListen(this::clearSubscriptions,
>> EventType.EVT_NODE_FAILED, EventType.EVT_NODE_JOINED,
>> EventType.EVT_NODE_LEFT);
>>}
>>
>>private boolean clearSubscriptions(Event event) {
>>   service.unsubscribeAll();
>>   return true;
>>}
>>
>> public QueueConsumer(Service service
>> IgniteQueue queue) {
>>   this.service = service;
>>   this.queue = queue;
>>}
>>
>>@PostConstruct
>>private void startConsuming() {
>>   Executors.newFixedThreadPool(1).execute(() ->
>> service.update(queue.take()));
>>}
>>
>>
>> The implementation of TestService is basically this
>>
>> public class TestService implements Service {
>>@Override
>>public void cancel(ServiceContext ctx) {
>>   log.info("Cancle");
>>}
>>
>>@Override
>>public void init(ServiceContext ctx) throws Exception {
>>   log.info("Init");
>>}
>>
>>@Override
>>public void execute(ServiceContext ctx) throws Exception {
>>   log.info("execute");
>>}
>> }
>>
>> Thanks.
>>
>>
>>
>> --
>> View this message in context: http://apache-ignite-users.705
>> 18.x6.nabble.com/Deploying-service-programmatically-hangs-
>> the-app-tp6791p6807.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>
>
> --
> Vladislav Pyatkov
>
> *DeployingService.zip* (6K) Download Attachment
> 
>
>
> --
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/Deploying-
> service-programmatically-hangs-the-app-tp6791p6848.html
> To unsubscribe from Deploying service programmatically hangs the app, click
> here
> 
> .
> NAML
> 

Re: Failed to wait for initial partition map exchange

[Jason]

hi Val,

I reduce the server nodes to 5 with big cache in off_heap and can definitely
reproduce this issue when the new node tries to join the topology.
For the new joining node, it takes hundreds of seconds for syncing the cache
partitions, and it says it has finished with the log "Completed (final)
rebalancing [cache=cache_raw_gbievent", but still "Failed to wait for
partition map exchange".

>From the log, seems that there're two waiting partition future: one is the
partition exchange map and the other one is the cache eviction.

I've attached the full logs for 5 server nodes and the config files for
them. 
Would you like to help take a look at and provide some suggestion? If any
further info, don't hesitate to ask for and I can easily reproduce it to
provide.

FYI, CO3SCH050520537 is the new added node and you can use its time as a
reference.

Any advice or suggestion should be appreciated.

Apache.config
  
default-config.xml
  
logs.zip
  

Thanks,
-Jason





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Failed-to-wait-for-initial-partition-map-exchange-tp6252p7135.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Failed to deserialize the job (.net) using code from examples

[Pavel Tupitsyn]

I tried to reproduce your scenario:

* Create a new project in Visual Studio, add Apache.Ignite 1.7 NuGet
package, paste your code from the first message, build the project
* Download binary distribution 1.7 from ignite.apache.org, run
apache.ignite.exe -assembly=ConsoleApplication13.exe
* Run the project in Visual Studio

This works for me. Both nodes print "Hello, World!".

Please make sure:
1) You have 1.7 version everywhere.
2) Apache.Ignite.exe references the same assembly that NuGet nodes use.



On Wed, Aug 17, 2016 at 5:31 PM, serrnovik  wrote:

> I'm back with the same issue. I've just changed to 1.7 version (both nuget
> and binary execution). Same test. As before if it is just nuget nodes that
> installed then everything is perfect. If I add at least one node from
> binary
> distribution - execution fails when I try to compute with not very
> descriptive null reference:
>
>
> /
> > System.NullReferenceException : Object reference not set to an instance
> of
> > an object.
> >at Apache.Ignite.Core.Impl.Common.Future`1.Get()
> /
>
> Full error from the console attached bellow.
>
> Other than nuget vs project I run binary as following:
> C:\apache-ignite\apache-ignite-fabric-1.7.0-bin\
> platforms\dotnet\bin\Apache.Ignite.exe
> -assembly="MY ASSEMBLY PATH"
>
>
> /
> > [15:48:04,700][SEVERE][pub-#15%null%][GridJobWorker] Failed to execute
> job
> > [jobId=ba932c89651-5dd85491-0bce-4872-83c5-49d4943cc246,
> > ses=GridJobSessionImpl [ses=GridTaskSessionImpl
> > [taskName=o.a.i.i.processors.platform.compute.
> PlatformBalancingMultiClosureTask,
> > dep=LocalDeployment [super=GridDeployment [ts=1471441453448,
> > depMode=SHARED, clsLdr=sun.misc.Launcher$AppClassLoader@c387f44,
> > clsLdrId=3dabeb89651-469415f2-3eb4-4221-a5db-d5d3e6f39441, userVer=0,
> > loc=true, sampleClsName=java.lang.String, pendingUndeploy=false,
> > undeployed=false, usage=0]],
> > taskClsName=o.a.i.i.processors.platform.compute.
> PlatformBalancingMultiClosureTask,
> > sesId=aa932c89651-5dd85491-0bce-4872-83c5-49d4943cc246,
> > startTime=1471441684674, endTime=9223372036854775807,
> > taskNodeId=5dd85491-0bce-4872-83c5-49d4943cc246,
> > clsLdr=sun.misc.Launcher$AppClassLoader@c387f44, closed=false,
> cpSpi=null,
> > failSpi=null, loadSpi=null, usage=1, fullSup=false,
> > subjId=5dd85491-0bce-4872-83c5-49d4943cc246, mapFut=IgniteFuture
> > [orig=GridFutureAdapter [resFlag=0, res=null, startTime=1471441684699,
> > endTime=0, ignoreInterrupts=false, state=INIT]]],
> > jobId=ba932c89651-5dd85491-0bce-4872-83c5-49d4943cc246]]
> > class org.apache.ignite.IgniteException: Native platform exception
> > occurred.
> > at
> > org.apache.ignite.internal.util.IgniteUtils.
> convertException(IgniteUtils.java:908)
> > at
> > org.apache.ignite.internal.processors.platform.compute.
> PlatformAbstractJob.execute(PlatformAbstractJob.java:83)
> > at
> > org.apache.ignite.internal.processors.job.GridJobWorker$
> 2.call(GridJobWorker.java:509)
> > at
> > org.apache.ignite.internal.util.IgniteUtils.
> wrapThreadLoader(IgniteUtils.java:6521)
> > at
> > org.apache.ignite.internal.processors.job.GridJobWorker.
> execute0(GridJobWorker.java:503)
> > at
> > org.apache.ignite.internal.processors.job.GridJobWorker.
> body(GridJobWorker.java:456)
> > at
> > org.apache.ignite.internal.util.worker.GridWorker.run(
> GridWorker.java:110)
> > at
> > org.apache.ignite.internal.processors.job.GridJobProcessor.
> processJobExecuteRequest(GridJobProcessor.java:1161)
> > at
> > org.apache.ignite.internal.processors.job.GridJobProcessor$
> JobExecutionListener.onMessage(GridJobProcessor.java:1766)
> > at
> > org.apache.ignite.internal.managers.communication.
> GridIoManager.invokeListener(GridIoManager.java:1238)
> > at
> > org.apache.ignite.internal.managers.communication.GridIoManager.
> processRegularMessage0(GridIoManager.java:866)
> > at
> > org.apache.ignite.internal.managers.communication.
> GridIoManager.access$1700(GridIoManager.java:106)
> > at
> > org.apache.ignite.internal.managers.communication.GridIoManager$5.run(
> GridIoManager.java:829)
> > at
> > java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> > at
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:745)
> > Caused by: PlatformNativeException [cause=BinaryObject
> [idHash=1737396909,
> > hash=28379535, typeId=87]]
> > at
> > org.apache.ignite.internal.processors.platform.PlatformContextImpl.
> createNativeException(PlatformContextImpl.java:652)
> > at
> > org.apache.ignite.internal.processors.platform.utils.PlatformUtils.
> readInvocationResult(PlatformUtils.java:770)
> > at
> > org.apache.ignite.internal.processors.platform.compute.
> PlatformClosureJob.execute0(PlatformClosureJob.java:74)
> > at
> > org.apache.ignite.internal.proce

Combined off heap size?

[yucigou]

According to page http://apacheignite.gridgain.org/docs/off-heap-memory, I
can set the off heap size using the offHeapMaxMemory.

If I understand correctly, this is for a single cache. Considering the
following configuration, I've got three caches, namely session-cache,
data-cache and penalty-cache. And the off heap size for each cache is set to
10GB. That is, I need 30GB RAM to accommodate the off heap memory for all
the three caches.










   







   







   





It is likely that as time goes by, 10G is not enough for one cache (e.g.,
data-cache), but still more than enough for another (e.g., penalty-cache).
That is, the off heap memory is not used efficiently.

My question is, is there a way to set the combined off heap size? For
example, I want to set the combined off heap size to 30G for all three
caches, regardless of which cache uses what amount of it.

Is it possible? Thanks.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Combined-off-heap-size-tp7137.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Who is going to take care of the off-heap memory?

[yucigou]

If I understand correctly, JVM GC takes care of the on-heap memory. My
question is, who is going to take care of the off-heap memory?

I guess the answer is Ignite :-)

But something I'm not so sure is, can I just make all caches off-heap, and
my JVM GC will work with minimal memory, so that my application latency and
throughput would be maximised. Is this true? Will the reclamation of
off-heap memory cause any latency to the application, or lower the
application throughput as Ignite threads run concurrently.

Is there a way to monitor the off-heap activities, which is similar to
Visual GC in JVisualVM?

Could you help give some insight please? Also any best practice here? Thank
you.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Who-is-going-to-take-care-of-the-off-heap-memory-tp7138.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Combined off heap size?

[Vladislav Pyatkov]

Hello,

About offHeapMaxMemory you need to know next:
1) The size is limit off heap size on one node of cache.
2) Ignite do not allocate fully off heap size immediatly. Allocation will
occure by required.

If you want to allocate N GB for off heap, you can to set offHeapMaxMemory
to N for each from caches, but you risk to get "out of memory" in unsave
code.

While there is no way to establish a common limitation to all caches.

On Wed, Aug 17, 2016 at 6:44 PM, yucigou  wrote:

> According to page http://apacheignite.gridgain.org/docs/off-heap-memory, I
> can set the off heap size using the offHeapMaxMemory.
>
> If I understand correctly, this is for a single cache. Considering the
> following configuration, I've got three caches, namely session-cache,
> data-cache and penalty-cache. And the off heap size for each cache is set
> to
> 10GB. That is, I need 30GB RAM to accommodate the off heap memory for all
> the three caches.
>
>  class="org.apache.ignite.configuration.IgniteConfiguration">
> 
> 
> 
>  value="session-cache" />
>  value="PARTITIONED"/>
>  value="1"/>
>  value="OFFHEAP_VALUES"/>
>  value="#{10 * 1024L * 1024L *
> 1024L}"/>
> 
> 
>  value="data-cache" />
>  value="PARTITIONED"/>
>  value="1"/>
>  value="OFFHEAP_VALUES"/>
>  value="#{10 * 1024L * 1024L *
> 1024L}"/>
> 
> 
>  value="penalty-cache" />
>  value="PARTITIONED"/>
>  value="1"/>
>  value="OFFHEAP_VALUES"/>
>  value="#{10 * 1024L * 1024L *
> 1024L}"/>
> 
> 
> 
> 
>
> It is likely that as time goes by, 10G is not enough for one cache (e.g.,
> data-cache), but still more than enough for another (e.g., penalty-cache).
> That is, the off heap memory is not used efficiently.
>
> My question is, is there a way to set the combined off heap size? For
> example, I want to set the combined off heap size to 30G for all three
> caches, regardless of which cache uses what amount of it.
>
> Is it possible? Thanks.
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Combined-off-heap-size-tp7137.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>



-- 
Vladislav Pyatkov

Re: Cannot get cache through the REST API

[yucigou]

Hi Alexey,

As I wrote in another thread, you would be able to see the CPU and heap
activities there captured in JVisualVM:

http://apache-ignite-users.70518.x6.nabble.com/Puzzled-about-JVM-configuration-td7091.html

The cause of it is the JVM configuration:

-XX:NewSize=128m 
-XX:MaxNewSize=128m 
-XX:MaxTenuringThreshold=0 
-XX:SurvivorRatio=1024 

Two reasons:

(1) Small young gen space size 
(2) Tiny survivor space with max tenuring threshold set to zero (which means
no aging) 

As a result, a lot of minor GCs and also a large number of short-lifespan
objects are promoted to old gen unnecessarily.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Cannot-get-cache-through-the-REST-API-tp6806p7140.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IncompatibleClassChange error in Java/Spark/Ignite program

[Ranjit]

Hi Val,

That worked!  

Is Spark 1.5.2 the recommended version to use with Ignite 1.7.0? 

We want to be at Spark 2.0.0, so what should be our next steps in
determining why Ignite 1.7.0 isn't working with Spark 2.0.0 for us?

Are you guys able to try this out and let us know if it is something on our
end or if this is genuinely an issue?

Thanks,
Ranjit




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/IncompatibleClassChange-error-in-Java-Spark-Ignite-program-tp7053p7141.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: yarn deployment

[prasanth]

Yes, that worked. Thank you.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/yarn-deployment-tp6843p7142.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: LOOK THORUGH THIS ERROR

[Ravi Puri]

is it necessary to provide project? Error can be resolved without that also
ryt?? 

-jars version matched . still error not resolved
-dependencies send to u and i changed it from beta version to normal 1.5
finl version jars. still error not resolved

what will be other reasons for this error as this is major problem i am
facing from last 3 to 4 days for my project ?? 



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/LOOK-THORUGH-THIS-ERROR-tp6977p7143.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: LOOK THORUGH THIS ERROR

[vkulichenko]

Hi Ravi,

There is only one possible reason for this issue - incorrect classpath. And
this thread doesn't have enough information to resolve this remotely.

I suggest you to do the following:

1. Switch to Maven. There is a very big chance it will resolve the issue
right away and will also make your life much easier.
2. Use debugger to find out where the wrong class is coming from.
3. If above doesn't help, please create a project that will reproduce the
problem, upload it to GitHub and provide the link here. Someone will take a
look and provide a solution then.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/LOOK-THORUGH-THIS-ERROR-tp6977p7144.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.