Can ignite-spark support spark 1.6.x?

hi, I have found that ignite-spark-1.7.0 doesn't support spark 2.0.x. Does it support spark 1.6.x? -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Can-ignite-spark-support-spark-1-6-x-tp9358.html Sent from the Apache Ignite Users mailing list archive at

Re: How IGFS keep sync with HDFS?

If IGFS can't hold more input data, will it flush data to HDFS? How IGFS flush data to HDFS? Async or sync? If it a async mode, when IGFS can't hold more data, it will using cache expire policy. And if the data which is expired is not persisted to HDFS timely, the data will be lost. Is is

How IGFS keep sync with HDFS?

I am using ignite hadoop accelerate to improve my map-reduce job. I am always encountered "java heap size" problem, though I have offer more than 80GB to ignite. The map-reduce job will write more than 150GB data to IGFS. When IGFS will sync data to HDFS? When processing a map-reduce job, do I

Re: MapReduce Job stuck when using ignite hadoop accelerator

I can find the WARN in logs. The stuck should be caused by a long running cache operations. How to locate what cause the long running cache operations. My map-reduce job can run successfully without ignite hadoop accelerator. [GridCachePartitionExchangeManager] Found long running cache

Re: Failed to perform request when running MR job with ignite hadoop accelerator

I use the following configuration to avoid this problem. However, I after doing this, I encounter an another problem which makes my job stuck. The issue is described in another message: MapReduce Job stuck when using ignite hadoop accelerator

MapReduce Job stuck when using ignite hadoop accelerator

Hi, Recently, I did some experiments to test the map-reduce job's performance with ignite hadoop accelerator. It really make sense to accelerate a map-reduce job. In my previous experiment, I store a 10GB file into IGFS. I design a I/O intensive job whose duty is only to read the 10GB file.

Re: How to config timeout configuration?

I use the following configuration to avoid this problem. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-config-timeout-configuration-tp9206p9215.html Sent from the Apache

Re: Why ignite hadoop accelerator can't improve performance with cluster mode?

Maybe it is caused by the configuration. And the first load is always slow. I tuned the configuration and the job goes well on my cluster. -- View this message in context:

Re: Failed to running hadoop accelerator

I solved the problem by myself. Is it ignite's bug? Though I have set the following variable in .bashrc, ignite still can't get the related hadoop jar. *export IGNITE_LIBS=$IGNITE_LIBS:"$IGNITE_HOME/libs/*":$HADOOP_CLASSPATH* Now I have solved this problem by copying related hadoop jars

Failed to running hadoop accelerator

hi, I am new to ignite. I downloaded "apache-ignite-hadoop-1.7.0-bin.zip". I unzip it and configured it as the doc "https://apacheignite-fs.readme.io/docs/installing-on-apache-hadoop; says. When I try to run the script "ignite.sh", it gave out the exception: class

Re: Can ignite accelerate a single spark job by using IGFS?

I will try it. Thanks. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Can-ignite-accelerate-a-single-spark-job-by-using-IGFS-tp9167p9170.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Can ignite accelerate a single spark job by using IGFS?

Hi, I know igniteRDD can be used to improve spark jobs by sharing the RDD among spark jobs. However, in our situation, all the spark jobs are independent and they don't need to share others' RDDs. Can I put all my data in IGFS to improve a single spark job's performance? -- View this

Re: Apache Ignite vs alluxio

Hi Franke, Let me clarify our situation more detailed. We want to store all our data in memory and all the job will never access the backend data. However, we still need backend storage such as HDFS to persist in memory data asynchronously in case of data lose which is caused by node failure.

Re: Apache Ignite vs alluxio

hi vincent, I've got the idea that load backend data in the cache will effect the performance. However, in our situation, we will store all the data in memory. For example, assume that we have a 10GB file all in memory. All the data is pinned in memory(Does ignite support pinning date in

Apache Ignite vs alluxio

Hi, Recently, I have used alluxio for about two month. However, it does't satisfy our requirement. We want to use alluxio as a memory cache layer which can easily interagte with Hadoop or Spark to accelerate our spark or map-reduce job. Unfortunately, it doesn't work. I have done many tests