Re: Ignite with Spark on Yarn

Thanks Alexey. I believe I have tried all the 4 methods you suggested below without any success. 1. Specified the absolute path of the default-config.xml using the file:/// as well as hdfs:/// 2. passed the config file as file parameter to spark-submit as

getAverageGetTime/getAveragePutTime APIs of CacheMetrics always return 0

We are investigating performance issue on our cluster of 2 nodes. We are trying to understand time it takes to get or put a cache entry from/to the cache. We are looking at out of box metrics for a cache . Both getAverageGetTime and getAveragePutTime apis of CacheMetrics are always giving value 0.

Re: write behind performance impacting main thread. Write behind buffer is never full

Alexey, With our use case setting the coalesce off will probably make it worse, for at least some caches we are doing many updates to the same key, one of the reasons I am setting the batch size to 500. I will send the cachestore implementation and some logs that show the phenomenon early next

Re: implement shared RDD in spark streaming in scala

Hi, > 1. Added config/default-config.xml directly under the project and added the > below snippet. Make sure the config is located under IGNITE_HOME or provide an absolute path to it. > 2. Commented the above snippet and use the below: > val ic = new IgniteContext(sparkContext, () => new

Re: ignite web agent still has "monitoring" tab?

Missed the second question. Here is how to install the console with a docker image: https://apacheignite-tools.readme.io/docs/docker-deployment — Denis > On Nov 2, 2017, at 11:31 AM, sherryhw wrote: > > Another question is, for the web agent, is that possible that we

Re: ignite web agent still has "monitoring" tab?

Hi Sherry, The monitoring tab is provided as an add-on by GridGain. However, Ignite users can use it for free via Ignite console version hosted here: https://console.gridgain.com/ If you prefer to have it installed in your infrastructure then you need to contact the vendor or create a similar

Re: ignite 2.3.0 docker image contains 2.2.0 files.

Good catch, thank you! The latest image contains 2.3.0 (docker pull apacheignite/ignite) but image with 2.3.0 tag contains binary files for 2.2.0 version. I've fixed it. On Fri, Nov 3, 2017 at 8:13 AM, Denis Magda wrote: > Nick, Vovan, > > Have we really upgraded docker and

Re: Deadlock detected while accessing caches

I also experienced a deadlock while attempting putAll using a HashMap. I used AtomicityMode as ATOMIC. If you are like me, the following process might be helpful. Deadlock jmc picture

Re: Deadlock detected while accessing caches

Thanks Andrew for the response. I don't have any getAll/putAll operations but have discovered transaction operations on unsorted keysets on a cache which can cause a deadlock. The deadlock detected message though shows two different caches whereas the unsorted keys are accessed on a single cache

RE: Node failed to startup due to deadlock

Hi Alexey, Even after commenting all the code in the event handlers (and offloading to application threadpools), the problem persists. Your description about this related to topology changes, leads me to suspect this is the same defect as: https://issues.apache.org/jira/browse/IGNITE-6380# I have

Re: Threads waiting

Hi, Your thread is waiting for partition exchange that possibly hangs due to some reason. Is it possible, to share logs and thread dumps at a time when you observed hanging? On Sun, Oct 29, 2017 at 4:09 AM, naresh.goty wrote: > Hi, > > We are seeing threads blocked when

Re: Quick question on Atomic Mode

Hello, The answer to your question depends on WAL mode you configured. Apache ignite provides 4 modes: - DEFAULT (The changes are guaranteed to be persisted to disk for every atomic write or transactional commit) - LOG_ONLY (The changes are guaranteed to be flushed to the OS buffer cache for

Re: Why do I get table not found exception when trying do execute SQL on some tables but not others...

Hi, Most likely that this table was created not in Public schema, but in sql schema by cache name. To check this you can run H2 Debug console - just start ignite with JVM parameter -DIGNITE_H2_DEBUG_CONSOLE=true Regards, Evgenii 2017-11-03 17:42 GMT+03:00 Tobias Eriksson

Re: ignite web agent still has "monitoring" tab?

Hi, Ignite web console never has "monitoring tab". On Thu, Nov 2, 2017 at 9:31 PM, sherryhw wrote: > t1372/2017-11-02_14-29-18.png> > > Hi I am new to Ignite web agent. I deployed the web console on the server >

Re: write behind performance impacting main thread. Write behind buffer is never full

Hi, Can you share your cache store implementation? It could be several reasons for possible performance degradation in write-behind mode. Ignite can start flushing your cache values in main() thread if cacheSize becomes greater than 1,5 x setWriteBehindFlushSize. It is a common case but it does

Re: Deadlock detected while accessing caches

I also experienced a deadlock while attempting putAll using a HashMap. I used AtomicityMode as ATOMIC. If you are like me, the following process might be helpful. Deadlock jmc picture

Re: Deadlock detected while accessing caches

Hi Amit, Do you use getAll, putAll or invokeAll on non-sorted keysets? On Thu, Nov 2, 2017 at 9:43 PM, Amit Pundir wrote: > Hi, > I have a Ignite 2.0 cluster of 8 servers and 8 clients. It has 6 caches > which are transactional and partitioned. > I use pessimistic

Why do I get table not found exception when trying do execute SQL on some tables but not others...

Hi I have my tables in Cassandra as a persistence store for Apache Ignite Now one of my tables “alpha” works just fine, I can run SQL from “SQuirreL SQL” BUT some of the other tables it cannot Yet in the UI of “SQuirreL SQL”, all the tables are visible BUT doing a SELECT count(*) FROM

Re: Transactions and IgniteQueue

/** * Commits this transaction by initiating {@code two-phase-commit} process. * * @throws IgniteException If commit failed. * @throws TransactionTimeoutException If transaction is timed out. * @throws TransactionRollbackException If transaction is automatically rolled back. * @throws

Re: Transactions and IgniteQueue

Yes, you could see it in the javadoc for Transaction class: 2017-11-03 14:43 GMT+03:00 arunkjn : > Thanks Evgenii. > > If I get a TransactionOptimisticException during the transaction will it be > thrown at the time of tx.commit() ? > > > > -- > Sent from:

Re: Ignite Cluster Disconnects After Idle Time

Unfortunately, it's not enough data to determine the problem. As I said earlier, please provide full logs from all nodes. Also, it would be great to see your config files. Regards, Evgenii 2017-11-03 13:29 GMT+03:00 Ramzinator : > Updated above post with logs. Text was

Re: Ignite with Spark on Yarn

Hi, The problem is Ignite cannot find your configuration file. Ignite tries these 3 steps to find configuration file: 1. Try to resolve it as a URI 2. If previous step fails, try to resolve it as $IGNITE_HOME + 3. If previous step fails, try to resolve it in CLASSPATH. Thus, you have

Re: Does ignite-schedule not upgrade after 2.1.0?

Aurora, Could you provide more details about your problem? Why did you need ignite-schedule? And what upgrade are you expecting? On Fri, Nov 3, 2017 at 6:00 PM, Aurora <2565003...@qq.com> wrote: > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > -- Alexey Kuznetsov

Ignite with Spark on Yarn

Below are the steps I followed for ignite with spark on Yarn but I am stuck after spending almost all day on this. - copied apache-ignite-fabric-2.1.0-bin (tried 2.3.0 initially but the it didnt work) to all the 4 slave nodes of Spark cluster and started ignite with ./bin/ignite.sh. It

Re: Transactions and IgniteQueue

Thanks Evgenii. If I get a TransactionOptimisticException during the transaction will it be thrown at the time of tx.commit() ? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Does ignite-schedule not upgrade after 2.1.0?

-- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Transactions and IgniteQueue

Hi Arun, As for now, IgniteTransaction works only for caches. You can find information about it in documentation: https://apacheignite.readme.io/docs/transactions#section-ignitetransactions Evgenii 2017-11-03 12:43 GMT+03:00 arunkjn : > Hi, > > I am using an optimistic

Re: Ignite Cluster Disconnects After Idle Time

Updated above post with logs. Text was not showing at first attempt. Thanks, Rami -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite Cluster Disconnects After Idle Time

Hi ezhuravlev, I do not have a firewall configured. I was able to reproduce and obtain these logs by enabling the TcpCommunicationSpi logs to DEBUG. One of the nodes seems to be fine when i try to clear the caches: The others seem to fail establishing a Tcp connection: Node 1: Node 2:

Transactions and IgniteQueue

Hi, I am using an optimistic transaction with serializable isolation level on a transactional cache as follows- while (true){ try(Transaction tx = ignite.transactions().txStart(TransactionConcurrency.OPTIMISTIC, TransactionIsolation.SERIALIZABLE)){ // update

RE: Node failed to startup due to deadlock

Rajeev, I see “WARNING: Failed to wait for partition release future” section in your logs. It includes “WARNING: Pending explicit locks:” with 7 cache locks (GridCacheExplicitLockSpan) GridCacheExplicitLockSpan details don’t have specific information about cache operation but they have