Re: Data Streamer error

[Alexey Kuznetsov]

Hi, @matt!

I see in stack trace "Caused by: java.util.ConcurrentModificationException".
May be your application concurrently changing your POJOs while Ignite in
progress of marshaling?

On Wed, Sep 14, 2016 at 9:04 AM, matt  wrote:

> Looks like it's a problem with the object I'm sending in. I swapped out the
> object (custom POJO) with an Integer, and all works fine. Is there a way I
> can find out what is it about this object that's causing the marshaling
> error? Any sample code around that shows how to run the marshaler
> stand-alone against a few test objects?
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Data-Streamer-error-tp7725p7727.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>



-- 
Alexey Kuznetsov
GridGain Systems
www.gridgain.com

Data Streamer error

[matt]

Hi,

I have Ignite (version 1.7.0) setup to do streaming, and a
autoFlushFrequency of 15 (for now) - in the logs, I'm getting this during
processing - any ideas on what this is coming from?

2016-09-13T17:31:52,580 - ERROR
[grid-data-loader-flusher-#55%null%:Slf4jLogger@112] - Runtime error caught
during grid runnable execution: GridWorker [name=grid-data-loader-flusher,
gridName=null, finished=false, isCancelled=false, hashCode=1315116657,
interrupted=false, runner=grid-data-loader-flusher-#55%null%]
org.apache.ignite.binary.BinaryObjectException: Failed to marshal object
with optimized marshaller: org.apache.ignite.stream.StreamVisitor$1@4e8b328a
at
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:167)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:132)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:239)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.internal.binary.BinaryMarshaller.marshal(BinaryMarshaller.java:92)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.submit(DataStreamerImpl.java:1362)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.flush(DataStreamerImpl.java:1248)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.tryFlush(DataStreamerImpl.java:970)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor$2.body(DataStreamProcessor.java:108)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
[ignite-core-1.7.0.jar:1.7.0]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_31]
Caused by: org.apache.ignite.IgniteCheckedException: Failed to serialize
object: org.apache.ignite.stream.StreamVisitor$1@4e8b328a
at
org.apache.ignite.marshaller.optimized.OptimizedMarshaller.marshal(OptimizedMarshaller.java:197)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:160)
~[ignite-core-1.7.0.jar:1.7.0]
... 9 more
Caused by: java.io.IOException: java.util.ConcurrentModificationException
at
org.apache.ignite.marshaller.optimized.OptimizedObjectOutputStream.writeSerializable(OptimizedObjectOutputStream.java:347)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.marshaller.optimized.OptimizedClassDescriptor.write(OptimizedClassDescriptor.java:800)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.marshaller.optimized.OptimizedObjectOutputStream.writeObject0(OptimizedObjectOutputStream.java:247)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.marshaller.optimized.OptimizedObjectOutputStream.writeFields(OptimizedObjectOutputStream.java:539)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.marshaller.optimized.OptimizedObjectOutputStream.writeSerializable(OptimizedObjectOutputStream.java:351)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.marshaller.optimized.OptimizedClassDescriptor.write(OptimizedClassDescriptor.java:800)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.marshaller.optimized.OptimizedObjectOutputStream.writeObject0(OptimizedObjectOutputStream.java:247)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.marshaller.optimized.OptimizedObjectOutputStream.writeFields(OptimizedObjectOutputStream.java:539)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.marshaller.optimized.OptimizedObjectOutputStream.writeSerializable(OptimizedObjectOutputStream.java:351)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.marshaller.optimized.OptimizedClassDescriptor.write(OptimizedClassDescriptor.java:800)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.marshaller.optimized.OptimizedObjectOutputStream.writeObject0(OptimizedObjectOutputStream.java:247)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.marshaller.optimized.OptimizedObjectOutputStream.writeFields(OptimizedObjectOutputStream.java:539)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.marshaller.optimized.OptimizedObjectOutputStream.writeSerializable(OptimizedObjectOutputStream.java:351)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.marshaller.optimized.OptimizedClassDescriptor.write(OptimizedClassDescriptor.java:800)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.marshaller.optimized.OptimizedObjectOutputStream.writeObject0(OptimizedObjectOutputStream.java:247)
~[ignite-core-1.7.0.jar:1.7.0]
at
org.apache.ignite.marshaller.optimized.OptimizedObjectOutputStream.writeFields(OptimizedObjectOutputStream.java:539)
~[ignite-core-1.7.0.jar:1.7.0]
at

Re: Automatic Persistence Example - Demo

[Abhishek Jain]

Thanks Denis, it worked!!! you made my day.

I appreciate your efforts and suggestions.

Regards
Abhishek

On Tue, Sep 13, 2016 at 11:43 AM, Denis Magda  wrote:

> Looks like CLASSPATH env variable is ignored on this Linux distribution
> side. You should refer to basic documentation from Oracle (JRE/JDK owner)
> or use ignite.sh script in the following way:
>
>
>1. add MAIN_CLASS env variable referring to Demo class (export
>MAIN_CLASS=org.apache.ignite.schema.Demo)
>2. add ignite-examples jar into {apache}/libs folder
>3. execute ignite.sh script. Refer to the source of ignite.sh for more
>details on how the classpath is prepared and MAIN_CLASS is used
>
>
> —
> Denis
>
> On Sep 12, 2016, at 4:11 PM, Abhishek Jain 
> wrote:
>
> Thanks Denis for your quick response. Our requirement is to make it work
> in the linux command line. The method mentioned in the Automatic
> Persistence page, is to import the pom.xml from the IDE and execute the
> Demo.java. With Eclipse in Windows is working. To take it to the next step
> I am trying to build and execute from the linux, where the jar file is
> creating but NOT able to execute the Demo application.
>
> I hope you sugested to add the /libs folder as the class
> path, that I had already included (*as part of the environment variable
> CLASSPATH*) and checked which is giving the "no main menifest attribute"
> error, please see my previous post for the console output.
>
>
> [root@dbtestvm1 target]# echo $CLASSPATH
> /root/ignite/apache-ignite-fabric-1.6.0-bin/libs:/usr/java/
> default/jre/lib/
> [root@dbtestvm1 target]#
> [root@dbtestvm1 target]# java -jar ignite-examples-1.6.0.jar
> org.apache.ignite.schema.Demo
> no main manifest attribute, in ignite-examples-1.6.0.jar
>
>
> Please correct me if I misunderstood your suggestion.
>
> Regards
> Abhishek
>
>
>
> On Mon, Sep 12, 2016 at 5:02 PM, Denis Magda  wrote:
>
>> Alternatively, you can follow this guide [1] that relies on
>> ignite-schema-import.sh script that does all the work for you.
>>
>> [1] https://apacheignite.readme.io/docs/automatic-persistence#demo
>>
>> —
>> Denis
>>
>> On Sep 12, 2016, at 2:58 PM, Denis Magda  wrote:
>>
>> Hi,
>>
>> You need to add all the libs that are located in “apache-version/libs”
>> folder to your classpath.
>>
>> —
>> Denis
>>
>> On Sep 12, 2016, at 2:46 PM, Abhishek Jain 
>> wrote:
>>
>> Hi,
>>
>> Not able to execute the Demo example given in the Automatic Persistence
>> from the command line in linux. Below is the console output, please let me
>> know if I am missing out anything. I tried couple of ways below with NO
>> success
>>
>> ===
>>
>> [root@dbtestvm1 target]# java -classpath ignite-examples-1.6.0.jar
>> org.apache.ignite.schema.Demo
>> Error: A JNI error has occurred, please check your installation and try
>> again
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> javax/cache/configuration/Factory
>> at java.lang.Class.getDeclaredMethods0(Native Method)
>> at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
>> at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
>> at java.lang.Class.getMethod0(Class.java:3018)
>> at java.lang.Class.getMethod(Class.java:1784)
>> at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
>> at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
>> Caused by: java.lang.ClassNotFoundException:
>> javax.cache.configuration.Factory
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>> ... 7 more
>> [root@dbtestvm1 target]#
>> [root@dbtestvm1 target]#
>> [root@dbtestvm1 target]#
>> [root@dbtestvm1 target]#
>> [root@dbtestvm1 target]# echo $CLASSPATH
>> /root/ignite/apache-ignite-fabric-1.6.0-bin/libs:/usr/java/
>> default/jre/lib/
>> [root@dbtestvm1 target]#
>> [root@dbtestvm1 target]# java -jar ignite-examples-1.6.0.jar
>> org.apache.ignite.schema.Demo
>> no main manifest attribute, in ignite-examples-1.6.0.jar
>> [root@dbtestvm1 target]#
>> [root@dbtestvm1 target]#
>>
>> ==
>>
>> Regards
>> Abhishek
>>
>>
>>
>>
>
>

Re: Query does not include objects added into Cache from within a transaction

[akaptsan]

Yes in theory I could know all changes I made. But in practice it will make
application logic so much more complicated! 
E.g in one part of transaction I create an object and in some other part
execute query which should include this object in result cursor taking into
account WHERE and ORDER BY. Due to this problem I need to merge somehow the
created object into the cursor considering WHERE and ORDER BY. Not easy
task. 
I'm thinking how to port our application from Oracle to Ignition. This kind
of fundamental difference in transaction behaviour makes the task very
difficult: we'll need to redesign huge amount of application code.  





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Query-does-not-include-objects-added-into-Cache-from-within-a-transaction-tp7651p7723.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Autonomous transaction

[vkulichenko]

Got it. It's not possible then. I would recommend you to revisit your logic
and check if it's possible to avoid using these features. I don't think
nested and/or autonomous transactions will be supported in Ignite in a
foreseeable future.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Autonomous-transaction-tp7672p7722.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Write Behind data safety guarantees II

[vkulichenko]

Hi,

In current implementation write-behind store can lose writes even if only
one nodes fails. This can be improved by adding backup queues [1], but it's
not implemented yet. Of course, this will not help if you lose more nodes
than number of backups you have (in this case data in memory is also lost).
Only synchronous write-through can guarantee that there are no data loss.

[1] https://issues.apache.org/jira/browse/IGNITE-1897

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Write-Behind-data-safety-guarantees-II-tp7696p7721.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite Atomic Long

[Caio Nishibe]

Hi,

I'm using a wrapper to start the nodes as service.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Atomic-Long-tp7706p7720.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite Atomic Long

[vkulichenko]

Hi,

This just means that the thread was interrupted. You should check the log
before the exception to see what could be a reason. It also looks like
you're using some kind of wrapper, probably it tried to stop the process?

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-Atomic-Long-tp7706p7719.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Custom (de)serialization for a single field

[vkulichenko]

Hi Josh,

I think Binarylizable is the right choice. But you're right that you will
have to write all the fields manually in this case. This works similar to
Externalizable in plain Java serialization.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Custom-de-serialization-for-a-single-field-tp7710p7718.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Query does not include objects added into Cache from within a transaction

[vkulichenko]

If you're executing a query from the same transaction where you updated the
data, I'm pretty sure you can find a workaround, because you know everything
about the updates made within the transaction. Transactional SQL feature is
mostly designed to avoid dirty reads in case a query transaction and update
transaction are two separate concurrent transactions.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Query-does-not-include-objects-added-into-Cache-from-within-a-transaction-tp7651p7717.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Automatic Persistence Example - Demo

[Denis Magda]

Looks like CLASSPATH env variable is ignored on this Linux distribution side. 
You should refer to basic documentation from Oracle (JRE/JDK owner) or use 
ignite.sh script in the following way:

add MAIN_CLASS env variable referring to Demo class (export 
MAIN_CLASS=org.apache.ignite.schema.Demo)
add ignite-examples jar into {apache}/libs folder
execute ignite.sh script. Refer to the source of ignite.sh for more details on 
how the classpath is prepared and MAIN_CLASS is used

—
Denis

> On Sep 12, 2016, at 4:11 PM, Abhishek Jain  
> wrote:
> 
> Thanks Denis for your quick response. Our requirement is to make it work in 
> the linux command line. The method mentioned in the Automatic Persistence 
> page, is to import the pom.xml from the IDE and execute the Demo.java. With 
> Eclipse in Windows is working. To take it to the next step I am trying to 
> build and execute from the linux, where the jar file is creating but NOT able 
> to execute the Demo application.
> 
> I hope you sugested to add the /libs folder as the class path, 
> that I had already included (as part of the environment variable CLASSPATH) 
> and checked which is giving the "no main menifest attribute" error, please 
> see my previous post for the console output. 
>>> 
>>> [root@dbtestvm1 target]# echo $CLASSPATH
>>> /root/ignite/apache-ignite-fabric-1.6.0-bin/libs:/usr/java/default/jre/lib/
>>> [root@dbtestvm1 target]# 
>>> [root@dbtestvm1 target]# java -jar ignite-examples-1.6.0.jar 
>>> org.apache.ignite.schema.Demo
>>> no main manifest attribute, in ignite-examples-1.6.0.jar
> 
> 
> Please correct me if I misunderstood your suggestion.
> 
> Regards
> Abhishek
> 
> 
> 
> On Mon, Sep 12, 2016 at 5:02 PM, Denis Magda  > wrote:
> Alternatively, you can follow this guide [1] that relies on 
> ignite-schema-import.sh script that does all the work for you.
> 
> [1] https://apacheignite.readme.io/docs/automatic-persistence#demo 
> 
> 
> —
> Denis
> 
>> On Sep 12, 2016, at 2:58 PM, Denis Magda > > wrote:
>> 
>> Hi,
>> 
>> You need to add all the libs that are located in “apache-version/libs” 
>> folder to your classpath.
>> 
>> —
>> Denis
>> 
>>> On Sep 12, 2016, at 2:46 PM, Abhishek Jain >> > wrote:
>>> 
>>> Hi,
>>> 
>>> Not able to execute the Demo example given in the Automatic Persistence 
>>> from the command line in linux. Below is the console output, please let me 
>>> know if I am missing out anything. I tried couple of ways below with NO 
>>> success
>>> 
>>> ===
>>> 
>>> [root@dbtestvm1 target]# java -classpath ignite-examples-1.6.0.jar 
>>> org.apache.ignite.schema.Demo
>>> Error: A JNI error has occurred, please check your installation and try 
>>> again
>>> Exception in thread "main" java.lang.NoClassDefFoundError: 
>>> javax/cache/configuration/Factory
>>> at java.lang.Class.getDeclaredMethods0(Native Method)
>>> at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
>>> at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
>>> at java.lang.Class.getMethod0(Class.java:3018)
>>> at java.lang.Class.getMethod(Class.java:1784)
>>> at 
>>> sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
>>> at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
>>> Caused by: java.lang.ClassNotFoundException: 
>>> javax.cache.configuration.Factory
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>> ... 7 more
>>> [root@dbtestvm1 target]# 
>>> [root@dbtestvm1 target]# 
>>> [root@dbtestvm1 target]# 
>>> [root@dbtestvm1 target]# 
>>> [root@dbtestvm1 target]# echo $CLASSPATH
>>> /root/ignite/apache-ignite-fabric-1.6.0-bin/libs:/usr/java/default/jre/lib/
>>> [root@dbtestvm1 target]# 
>>> [root@dbtestvm1 target]# java -jar ignite-examples-1.6.0.jar 
>>> org.apache.ignite.schema.Demo
>>> no main manifest attribute, in ignite-examples-1.6.0.jar
>>> [root@dbtestvm1 target]# 
>>> [root@dbtestvm1 target]#
>>> 
>>> ==
>>> 
>>> Regards
>>> Abhishek
>> 
> 
> 

Re: Query does not include objects added into Cache from within a transaction

[Denis Magda]

It depends on when someone from the community takes over this task getting it 
done. If to refer to previous discussion we should expect the feature appear in 
Ignite in the beginning of the next year.

—
Denis

> On Sep 13, 2016, at 3:15 AM, akaptsan  wrote:
> 
> I would say it's critical problem. Can you estimate when it will be fixed? 
> (I don't have access to jira)
> 
> 
> 
> --
> View this message in context: 
> http://apache-ignite-users.70518.x6.nabble.com/Query-does-not-include-objects-added-into-Cache-from-within-a-transaction-tp7651p7703.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Nested Transactions

[Denis Magda]

You can work with Ignite using JDBC [1] or ODBC [2] drivers executing SQL 
queries. DML and DDL should be a part of the product soon. Until they are not 
available you need to use Ignite Cache API for transaction executions.

However, nested transactions are not a part of Ignite.

[1] https://apacheignite.readme.io/docs/jdbc-driver
[2] https://apacheignite.readme.io/docs/odbc-driver

—
Denis

> On Sep 13, 2016, at 2:50 AM, akaptsan  wrote:
> 
> We have OLTP system based on Oracle. And we havily use the nested
> transactions 
> We would like to replace Oracle with Ignite. That's why we need all this
> features: nested transaction, autonomous transactions, DDL ... 
> 
> 
> 
> 
> 
> --
> View this message in context: 
> http://apache-ignite-users.70518.x6.nabble.com/Nested-Transactions-tp7662p7702.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Custom (de)serialization for a single field

[Vladislav Pyatkov]

Hello Josh,

I think  BinarySerializer will be matched.
Look at the article[1] and javadoc[2].

[1]:
http://apacheignite.gridgain.org/docs/binary-marshaller#configuring-binary-objects
[2]:
https://ignite.apache.org/releases/1.5.0.final/javadoc/org/apache/ignite/binary/BinarySerializer.html


On Tue, Sep 13, 2016 at 6:26 PM, Josh Cummings 
wrote:

> I have an object I need to cache which has more than 100 fields. One
> particular field needs some custom handling when being (de)serialized. I've
> considered implementing Binarylizable; however, I don't care to manage all
> of the fields via the read and write binary methods.
>
> Where should I look in the api to configure custom (de)serialization for a
> single field?
>
> --
>
> *JOSH CUMMINGS*
>
> Principal Engineer
>
> [image: Workfront] 
>
> *O*  801.477.1234  |  *M*  8015562751
>
> joshcummi...@workfront.com | www.workfront.com
> Address   |  Twitter
>   |  LinkedIn
>   |  Facebook
> 
>
> [image: Workfront] 
>



-- 
Vladislav Pyatkov

Re: Nested Transactions

[Vladislav Pyatkov]

Hello,

Unfortunately, Ignite has not support anything from the list (nested
transaction, autonomous transactions, DDL) yet.
I think, if you want to use key-value approach, changes of logic will be
significant.

On Tue, Sep 13, 2016 at 12:50 PM, akaptsan  wrote:

> We have OLTP system based on Oracle. And we havily use the nested
> transactions
> We would like to replace Oracle with Ignite. That's why we need all this
> features: nested transaction, autonomous transactions, DDL ...
>
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Nested-Transactions-tp7662p7702.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>



-- 
Vladislav Pyatkov

Custom (de)serialization for a single field

[Josh Cummings]

I have an object I need to cache which has more than 100 fields. One
particular field needs some custom handling when being (de)serialized. I've
considered implementing Binarylizable; however, I don't care to manage all
of the fields via the read and write binary methods.

Where should I look in the api to configure custom (de)serialization for a
single field?

-- 

*JOSH CUMMINGS*

Principal Engineer

[image: Workfront] 

*O*  801.477.1234  |  *M*  8015562751

joshcummi...@workfront.com | www.workfront.com
Address   |  Twitter
  |  LinkedIn
  |  Facebook


[image: Workfront] 

Re: Trouble with Using Ignite 1.8 ODBC Driver

[Igor Sapego]

You can use installer from here: [1].

What kind of problem do you mean?

[1] -
https://github.com/isapego/ignite/tree/ignite-3868/modules/platforms/cpp/odbc/install

Best Regards,
Igor

On Tue, Sep 13, 2016 at 4:57 PM, amitpa  wrote:

> Also is this a problem with Visual Studio 2015 and Ignite and it doesnt
> happen when we use other VS versions like 2010?
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Trouble-with-Using-Ignite-1-8-ODBC-
> Driver-tp7656p7707.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Services not injected into CacheStore if deployed using Spring configuration

[Evgeniy Ignatiev]

Hello, everyone.

Recently, while using service injection in custom CacheStore 
implementation, we faced the problem that with startup deployment of 
services and caches through Spring, the services are not injected into 
CacheStore. It doesn't happen when we deploy our services and caches 
manually, so we assume the behavior is not intended.


Here is the small demo proejct: 
https://github.com/YevIgn/ignite-cache-store-service-injection - when 
running example.Main I get NPE on cache.put(..)


Could you, please, look into it and advise on possible resolution of 
this issue?

Re: Trouble with Using Ignite 1.8 ODBC Driver

[amitpa]

Also is this a problem with Visual Studio 2015 and Ignite and it doesnt
happen when we use other VS versions like 2010? 



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Trouble-with-Using-Ignite-1-8-ODBC-Driver-tp7656p7707.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Ignite Atomic Long

[Caio Nishibe]

Hi, 

I'm having some trouble with atomic long inside my compute job. I'm getting
the following exception once a while:


FINEST|4532/0|Service AntheusMatchServerNode|16-09-13
10:51:23|[13:51:23,382][SEVERE][pub-#46%null%][GridCacheAtomicLongImpl]
 Failed to add and get:
o.a.i.i.processors.datastructures.GridCacheAtomicLongImpl$7@3f009e9e
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23|class
org.apache.ignite.internal.IgniteInterruptedCheckedException: null
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:175)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:117)
INFO|wrapper|Service AntheusMatchServerNode|16-09-13 10:51:23|executed
script: scripts/trayMessage.gv 1
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4667)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4650)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1391)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.apache.ignite.internal.processors.datastructures.GridCacheAtomicLongImpl$7.call(GridCacheAtomicLongImpl.java:434)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.apache.ignite.internal.processors.datastructures.GridCacheAtomicLongImpl$7.call(GridCacheAtomicLongImpl.java:431)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.apache.ignite.internal.processors.cache.GridCacheUtils$23.call(GridCacheUtils.java:1648)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.apache.ignite.internal.processors.cache.GridCacheUtils.outTx(GridCacheUtils.java:891)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.apache.ignite.internal.processors.datastructures.GridCacheAtomicLongImpl.addAndGet(GridCacheAtomicLongImpl.java:274)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.antheus.matchserver.dependencies.jobs.MatchJob.execute(MatchJob.java:367)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:509)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6459)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:503)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:456)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
java.lang.Thread.run(Thread.java:745)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23|Caused by:
java.lang.InterruptedException
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:998)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
INFO|wrapper|Service AntheusMatchServerNode|16-09-13 10:51:23|executed
script: scripts/trayMessage.gv 1
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:159)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| ... 18 more
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23|class
org.apache.ignite.IgniteInterruptedException: null
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.apache.ignite.internal.util.IgniteUtils$2.apply(IgniteUtils.java:745)
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at
org.apache.ignite.internal.util.IgniteUtils$2.apply(IgniteUtils.java:743)
INFO|wrapper|Service AntheusMatchServerNode|16-09-13 10:51:23|executed
script: scripts/trayMessage.gv 1
FINEST|4532/0|Service AntheusMatchServerNode|16-09-13 10:51:23| at

Re: JVM tuning

[Caio Nishibe]

Hi Alexey,

thank you for your response. I've implemented your suggestions and now my
cluster is with 100% cpu utilization.

Best regards,

Caio



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/JVM-tuning-tp7532p7705.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cache Problems

[Semyon Boikov]

It looks like known issue https://issues.apache.org/jira/browse/IGNITE-2714

On Tue, Sep 13, 2016 at 11:13 AM, kromulan  wrote:

> I've had same problem in the past but it was a memory leak in indexing
> code.
> Do you use indexing on your entities ?
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Cache-Problems-tp7477p7697.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Query does not include objects added into Cache from within a transaction

[akaptsan]

I would say it's critical problem. Can you estimate when it will be fixed? 
(I don't have access to jira)



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Query-does-not-include-objects-added-into-Cache-from-within-a-transaction-tp7651p7703.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Nested Transactions

[akaptsan]

We have OLTP system based on Oracle. And we havily use the nested
transactions 
We would like to replace Oracle with Ignite. That's why we need all this
features: nested transaction, autonomous transactions, DDL ... 





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Nested-Transactions-tp7662p7702.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Write Behind data safety guarantees II

[Jörn Franke]

You can have also the case both nodes crash ... The bottom line is that a write 
loss can occur in any system. I am always surprised to hear even senior 
consultants saying that in a high reliability database no write loss can occur 
or the risk is low (think about the human factor! Eg an admin accidentally 
shuts down both or some automated maintenance procedure of the underlying 
operating system).


> On 13 Sep 2016, at 09:59, Christian Reiniger  
> wrote:
> 
> Hi,
>  
> I’ve been trying to find information on cache persistence write behind 
> behavior that might lead to entries not being written to the persistent 
> store.  Most notably: what happens in the following scenario:
>  
> · Cache ‘cache’ has two backup copies on nodes A and B respectively 
> and is configured with persistence to a RDBMS, with write behind activated.
> · Node A collects some write operations, but crashes before the 
> entries can actually be written.
>  
> Will the loss of node A cause node B jump in and write these entries or can 
> ignite “forget” to write them?
>  
> Thank you
> --
> 
> Christian Reiniger
> 
>  

Re: Re: Increase Ignite instances can't increase the speed of compute

[胡永亮/Bob]

Only 118 jobs for this test.



Bob
 
From: Taras Ledkov
Date: 2016-09-13 14:52
To: user@ignite.apache.org
Subject: Re: Re: Increase Ignite instances can't increase the speed of compute
Hi,

How many MatchingJobs do you submit?


On Tue, Sep 13, 2016 at 12:29 PM, 胡永亮/Bob  wrote:
Hello,  Vladislav

The following is some code. 

...
IgniteCompute compute = ignite.compute();//.withAsync();

compute.run(new MatchingJob(m_mapReadyDataPara));
}

private static class MatchingJob implements IgniteRunnable{

private Map> m_mapReadyData;
private IgniteCache> mapMatchingData; 
//This is a cache in Ignite cluster.
...

public void run() {
...
Iterator>> entryKeyIterator1 = 
m_mapReadyData.entrySet().iterator();//m_mapReadyData is the input data, 
its size is 5000 for every job now.
Map> local_writeCache = new 
HashMap>();
...
//Then the job read detail data from m_mapReadyData, and compute.
while (entryKeyIterator1.hasNext()) {
Entry> eKey1 = entryKeyIterator1.next();
String carKey = eKey1.getKey();
Map value1 = eKey1.getValue();

//local node cache
Map existMapbaselink = 
local_mapMatchingData.get(carKey);
if(existMapbaselink == null){
existMapbaselink = mapMatchingData.get(carKey);  //Read 
data to compute with it from Ignite cache. This data's size is 154M for 
PARTITIONED mode.
if(existMapbaselink != null)
local_mapMatchingData.putIfAbsent(carKey, 
existMapbaselink);
}

//some compute logic code

mapbaselink = local_writeCache.get(carKey);
if(mapbaselink == null){
mapbaselink = new TreeMap();
}
mapbaselink.put(stdtime, ListBaseLink);
local_writeCache.put(carKey, mapbaselink);
}

//batch to write data into Ignite.
Iterator>> it = 
local_writeCache.entrySet().iterator();
while(it.hasNext()){
Entry> entry = it.next();
String carKey = entry.getKey();
final Map value = entry.getValue();

if(!mapMatchingData.containsKey(carKey)){
mapMatchingData.put(carKey, value);
}else{
mapMatchingData.invoke(carKey, new EntryProcessor, Void>() {
  @Override 
  public Void process(MutableEntry> entry, Object... args) {
  Map map = entry.getValue();
  map.putAll(value);
  entry.setValue(map);
  return null;
  }
  });
}

}




bob
 
From: Vladislav Pyatkov
Date: 2016-09-12 18:37
To: user@ignite.apache.org
Subject: Re: Increase Ignite instances can't increase the speed of compute
Hello,

I don't understand, what do you try to measure, without code.
Size of calculation task, size of data moved into network have importance.

Could you please provide code example?

On Mon, Sep 12, 2016 at 12:33 PM, 胡永亮/Bob  wrote:
Hi, everyone:

I am using Ignite for computing and cache.

I use the same input data and the same compute logic.
When my ignite cluster's node is 2 in 2 machines, the total cost time is 
38s.

But, when I increase the Ignite cluster nodes to 3 in 3 machines, the cost 
time is 32s/51s/41s
4 instances in 4 machines, the cost time is 32s/40s.

 The compute speed can't change faster, what may the reason be?

Thanks.

Bob

---
Confidentiality Notice: The information contained in this e-mail and any 
accompanying attachment(s) 
is intended only for the use of the intended recipient and may be confidential 
and/or privileged of 
Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of 
this communication is 
not the intended recipient, unauthorized use, forwarding, printing,  storing, 
disclosure or copying 
is strictly prohibited, and may be unlawful.If you have received this 
communication in error,please 
immediately notify the sender by return e-mail, and delete the original message 
and all copies from 
your system. Thank you. 
---



-- 
Vladislav Pyatkov

Re: SQL Queries - propagate "new" CacheConfiguration.queryEntities over the cluster on an already started cache

[akaptsan]

Great news! 
Could you please give me a hint how to access this function?



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/SQL-Queries-propagate-new-CacheConfiguration-queryEntities-over-the-cluster-on-an-already-started-cae-tp5802p7699.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Autonomous transaction

[akaptsan]

Not exactly. Authonomous transaction should see uncommited data of parent
transaction



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Autonomous-transaction-tp7672p7698.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cache Problems

[kromulan]

I've had same problem in the past but it was a memory leak in indexing code.
Do you use indexing on your entities ?



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Cache-Problems-tp7477p7697.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Write Behind data safety guarantees II

[Christian Reiniger]

Hi,

I've been trying to find information on cache persistence write behind behavior 
that might lead to entries not being written to the persistent store.  Most 
notably: what happens in the following scenario:


* Cache 'cache' has two backup copies on nodes A and B respectively and 
is configured with persistence to a RDBMS, with write behind activated.

* Node A collects some write operations, but crashes before the entries 
can actually be written.

Will the loss of node A cause node B jump in and write these entries or can 
ignite "forget" to write them?

Thank you
--
Christian Reiniger

Re: Re: Increase Ignite instances can't increase the speed of compute

[Taras Ledkov]

Hi,

How many MatchingJobs do you submit?


On Tue, Sep 13, 2016 at 12:29 PM, 胡永亮/Bob  wrote:

> Hello,  Vladislav 
>
> The following is some code.
>
> ...
> IgniteCompute compute = ignite.compute();//.withAsync();
>
> compute.run(new MatchingJob(m_mapReadyDataPara));
> }
>
> private static class MatchingJob implements IgniteRunnable{
>
> private Map> m_mapReadyData;
> private IgniteCache> mapMatchingData;
> *//This is a cache in Ignite cluster.*
> ...
>
> public void run() {
> ...
> Iterator>> entryKeyIterator1 = m_
> mapReadyData.entrySet().iterator();*//**m_mapReadyData is the input
> data, its size is 5000 for every job now.*
> Map> local_writeCache =
>  new HashMap>();
> ...
> *//Then the job read detail data from m_mapReadyData, and compute.*
> while (entryKeyIterator1.hasNext()) {
> Entry> eKey1 =
> entryKeyIterator1.next();
> String carKey = eKey1.getKey();
> Map value1 = eKey1.getValue();
>
> *//local node cache*
> Map existMapbaselink = local_
> mapMatchingData.get(carKey);
> if(existMapbaselink == null){
> existMapbaselink = mapMatchingData.get(carKey); * //Read
> data to compute with it from Ignite cache. This data's size is 154M for *
> PARTITIONED *mode.*
> if(existMapbaselink != null)
> local_mapMatchingData.putIfAbsent(
> carKey, existMapbaselink);
> }
>
>* //some compute logic code*
>
> mapbaselink = local_writeCache.get(carKey);
> if(mapbaselink == null){
> mapbaselink = new TreeMap();
> }
> mapbaselink.put(stdtime, ListBaseLink);
> local_writeCache.put(carKey, mapbaselink);
> }
>
> *//batch to write data into Ignite.*
> Iterator>> it = local_
> writeCache.entrySet().iterator();
> while(it.hasNext()){
> Entry>
> entry = it.next();
> String carKey = entry.getKey();
> final Map value = entry.getValue();
>
> if(!mapMatchingData.containsKey(carKey)){
> mapMatchingData.put(carKey, value);
> }else{
> mapMatchingData.invoke(carKey,
>  new EntryProcessor, Void>() {
>   @Override
>   public Void process(MutableEntry<
> String, Map> entry, Object... args) {
>   Map map =
> entry.getValue();
>   map.putAll(value);
>   entry.setValue(map);
>   return null;
>   }
>   });
> }
>
> }
>
>
> --
> bob
>
>
> *From:* Vladislav Pyatkov 
> *Date:* 2016-09-12 18:37
> *To:* user@ignite.apache.org
> *Subject:* Re: Increase Ignite instances can't increase the speed of
> compute
> Hello,
>
> I don't understand, what do you try to measure, without code.
> Size of calculation task, size of data moved into network have importance.
>
> Could you please provide code example?
>
> On Mon, Sep 12, 2016 at 12:33 PM, 胡永亮/Bob  wrote:
>
>> Hi, everyone:
>>
>> I am using Ignite for computing and cache.
>>
>> I use the same input data and the same compute logic.
>> When my ignite cluster's node is 2 in 2 machines, the total cost time
>> is 38s.
>>
>> But, when I increase the Ignite cluster nodes to 3 in 3 machines, the
>> cost time is 32s/51s/41s
>> 4 instances in 4 machines, the cost time is 32s/40s.
>>
>>  The compute speed can't change faster, what may the reason be?
>>
>> Thanks.
>>
>> Bob
>>
>> 
>> ---
>> Confidentiality Notice: The information contained in this e-mail and any
>> accompanying attachment(s)
>> is intended only for the use of the intended recipient and may be
>> confidential and/or privileged of
>> Neusoft Corporation, its subsidiaries and/or its affiliates. If any
>> reader of this communication is
>> not the intended recipient, unauthorized use, forwarding, printing,
>> storing, disclosure or copying
>> is strictly prohibited, and may be unlawful.If you have received this
>> communication in error,please
>> immediately notify the sender by return e-mail, and delete the original
>> message and all copies from
>> your system. Thank you.
>>