[jira] [Commented] (TOREE-345) Spark 2.0.0 Fails when trying to use spark-avro

2016-10-30 Thread Marius Van Niekerk (JIRA) 

[ 
https://issues.apache.org/jira/browse/TOREE-345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15620532#comment-15620532
 ] 

Marius Van Niekerk commented on TOREE-345:
--

Hi Hollin, I can help guide you on our gitter channel 
https://gitter.im/ibm-et/spark-kernel.

We probably need to build some unit tests that test spark-avro and other common 
dependencies in particular.

> Spark 2.0.0 Fails when trying to use spark-avro
> ---
>
> Key: TOREE-345
> URL: https://issues.apache.org/jira/browse/TOREE-345
> Project: TOREE
>  Issue Type: Bug
> Environment: Spark 2.0.0, Scala 2.11, Hadoop 2.7, Toree Git SHA: 
> 7c1bfb6df7130477c558e69bbb518b0af364e06a
>Reporter: Hollin Wilkins
>
> When trying to use the spark-avro project to load Avro files from Jupyter, we 
> get errors.
> First:
> {code}
> %AddDeps com.databricks spark-avro_2.11 3.0.1 --transitive --trace
> {code}
> Then try to load an Avro file and show it:
> {code}
> spark.sqlContext.read.format("com.databricks.spark.avro").load("/tmp/test.avro").show()
> {code}
> And we get an error. I will attach the trace as a file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TOREE-345) Spark 2.0.0 Fails when trying to use spark-avro

2016-10-30 Thread Hollin Wilkins (JIRA) 

[ 
https://issues.apache.org/jira/browse/TOREE-345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15620341#comment-15620341
 ] 

Hollin Wilkins commented on TOREE-345:
--

Is there any work being done on this? I would very much like to use Toree for 
demoing the MLeap project. I can help with this ticket if I can get some 
guidance through Slack or something.

Thank you

> Spark 2.0.0 Fails when trying to use spark-avro
> ---
>
> Key: TOREE-345
> URL: https://issues.apache.org/jira/browse/TOREE-345
> Project: TOREE
>  Issue Type: Bug
> Environment: Spark 2.0.0, Scala 2.11, Hadoop 2.7, Toree Git SHA: 
> 7c1bfb6df7130477c558e69bbb518b0af364e06a
>Reporter: Hollin Wilkins
>
> When trying to use the spark-avro project to load Avro files from Jupyter, we 
> get errors.
> First:
> {code}
> %AddDeps com.databricks spark-avro_2.11 3.0.1 --transitive --trace
> {code}
> Then try to load an Avro file and show it:
> {code}
> spark.sqlContext.read.format("com.databricks.spark.avro").load("/tmp/test.avro").show()
> {code}
> And we get an error. I will attach the trace as a file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TOREE-345) Spark 2.0.0 Fails when trying to use spark-avro

2016-10-06 Thread Chip Senkbeil (JIRA) 

[ 
https://issues.apache.org/jira/browse/TOREE-345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15552803#comment-15552803
 ] 

Chip Senkbeil commented on TOREE-345:
-

Yeah, that is what the Ivy files generated in the resource directories were 
for. In fact, we're still supposedly loading up the base dependencies and 
excluding them here: 
https://github.com/apache/incubator-toree/blob/a0f75a1c0e5c66d2d5787a78668d31d44ffb1be0/kernel-api/src/main/scala/org/apache/toree/dependencies/CoursierDependencyDownloader.scala#L82

Either the Ivy files are not being loaded 
(https://github.com/apache/incubator-toree/blob/a0f75a1c0e5c66d2d5787a78668d31d44ffb1be0/kernel-api/src/main/scala/org/apache/toree/dependencies/CoursierDependencyDownloader.scala#L286)
 or they are not being generated with up-to-date info. Or Coursier is not 
excluding properly.

> Spark 2.0.0 Fails when trying to use spark-avro
> ---
>
> Key: TOREE-345
> URL: https://issues.apache.org/jira/browse/TOREE-345
> Project: TOREE
>  Issue Type: Bug
> Environment: Spark 2.0.0, Scala 2.11, Hadoop 2.7, Toree Git SHA: 
> 7c1bfb6df7130477c558e69bbb518b0af364e06a
>Reporter: Hollin Wilkins
>
> When trying to use the spark-avro project to load Avro files from Jupyter, we 
> get errors.
> First:
> {code}
> %AddDeps com.databricks spark-avro_2.11 3.0.1 --transitive --trace
> {code}
> Then try to load an Avro file and show it:
> {code}
> spark.sqlContext.read.format("com.databricks.spark.avro").load("/tmp/test.avro").show()
> {code}
> And we get an error. I will attach the trace as a file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)