Re: Can't use UDFs with Dataframes in spark-2.0-preview scala-2.10

Please go ahead. On Tue, Jun 7, 2016 at 4:45 PM, franklyn wrote: > Thanks for reproducing it Ted, should i make a Jira Issue?. > > > > -- > View this message in context: >

Re: Can't use UDFs with Dataframes in spark-2.0-preview scala-2.10

Thanks for reproducing it Ted, should i make a Jira Issue?. -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Can-t-use-UDFs-with-Dataframes-in-spark-2-0-preview-scala-2-10-tp17845p17852.html Sent from the Apache Spark Developers List mailing list

Re: Can't use UDFs with Dataframes in spark-2.0-preview scala-2.10

I built with Scala 2.10 >>> df.select(add_one(df.a).alias('incremented')).collect() The above just hung. On Tue, Jun 7, 2016 at 3:31 PM, franklyn wrote: > Thanks Ted !. > > I'm using > >

Re: Can't use UDFs with Dataframes in spark-2.0-preview scala-2.10

Thanks Ted !. I'm using https://github.com/apache/spark/commit/8f5a04b6299e3a47aca13cbb40e72344c0114860 and building with scala-2.10 I can confirm that it works with scala-2.11 -- View this message in context:

Re: Can't use UDFs with Dataframes in spark-2.0-preview scala-2.10

With commit 200f01c8fb15680b5630fbd122d44f9b1d096e02 using Scala 2.11: Using Python version 2.7.9 (default, Apr 29 2016 10:48:06) SparkSession available as 'spark'. >>> from pyspark.sql import SparkSession >>> from pyspark.sql.types import IntegerType, StructField, StructType >>> from

Can't use UDFs with Dataframes in spark-2.0-preview scala-2.10

I've built spark-2.0-preview (8f5a04b) with scala-2.10 using the following > > > ./dev/change-version-to-2.10.sh > ./dev/make-distribution.sh -DskipTests -Dzookeeper.version=3.4.5 > -Dcurator.version=2.4.0 -Dscala-2.10 -Phadoop-2.6 -Pyarn -Phive and then ran the following code in a pyspark