Re: Error with Spark + IGFS (HDFS cache) through Hive

2019-01-14 Thread ilya.kasnacheev
Hello! If you can write a test which demonstrates problematic behavior, and a fix on top of that, and fill a ticket against IGNITE project in Apache JIRA, I think we could merge it all right. Regards, -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Error with Spark + IGFS (HDFS cache) through Hive

2018-09-13 Thread Maximiliano Patricio Méndez
Hi evgenii, thanks for answering. I'm fairly new to ignite, but from what I can see in the docs the ignite rdd or the ignite dataframe are only used to serve data that is already in Ignite in the form of a Table or cache. Unfortunately I currently have a couple hundred tables in HDFS referenced

Re: Error with Spark + IGFS (HDFS cache) through Hive

2018-09-11 Thread Evgenii Zhuravlev
Hi, Do you really need to use Hive here? You can just use Spark integration with Ignite, which allows to run sql: DataFrame( https://apacheignite-fs.readme.io/docs/ignite-data-frame) or RDD( https://apacheignite-fs.readme.io/docs/ignitecontext-igniterdd). For sure, this solution will work much

Error with Spark + IGFS (HDFS cache) through Hive

2018-09-10 Thread Maximiliano Patricio Méndez
Hi, I'm having an LinkageError in spark trying to read a hive table that has the external location in IGFS: java.lang.LinkageError: loader constraint violation: when resolving field "LOG" the class loader (instance of org/apache/spark/sql/hive/client/IsolatedClientLoader$$anon$1) of the referring