I’m not sure there are any — and you’re right, there probably should.
Having said that, integration is very straight-forward.
You run pyspark (or spark-submit), passing in the Ignite jar files (using the
—jars parameter). For example:
$SPARK_HOME/bin/spark-submit --jars
Thanks for that Stephen. That is a start.
Would you know any documentation/tutorials/examples for Ignite and Spark
integration (**in Python**)?
Anthony
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
You can use PySpark in exactly as you normally do. So something like this works:
stuff = spark.read \
.format("ignite") \
.option("config", "ignite-client.xml") \
.option("table", “Stuff") \
.option("primaryKeyFields", "ID") \
.load()
You might need to check the Java
I am interested to use Ignite to speedup Spark as in
https://apacheignite-fs.readme.io/docs/ignite-for-spark, but all the example
seems to be in Java/Scala. Is there an easy way to do the same in Python? I
read somewhere that Ignite has an ODBC driver and perhaps a RESTful api as
an alternative.
