Re: Spark 1.5: How to trigger expression execution through UnsafeRow/TungstenProject

thanks that worked -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Spark-1-5-How-to-trigger-expression-execution-through-UnsafeRow-TungstenProject-tp14026p14053.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

Spark 1.5: How to trigger expression execution through UnsafeRow/TungstenProject

The tungsten, cogegen etc options are enabled by default. But I am not able to get the execution through the UnsafeRow/TungstenProject. It still executes using InternalRow/Project. I see this in the SparkStrategies.scala: If unsafe mode is enabled and we support these data types in Unsafe, use

Re: Spark 1.5: How to trigger expression execution through UnsafeRow/TungstenProject

Here is the example from Reynold ( http://search-hadoop.com/m/q3RTtfvs1P1YDK8d) : scala> val data = sc.parallelize(1 to size, 5).map(x => (util.Random.nextInt(size / repetitions),util.Random.nextDouble)).toDF("key", "value") data: org.apache.spark.sql.DataFrame = [key: int, value: double] scala>