Re: Does spark supports the Hive function posexplode function?

2015-07-12 Thread David Sabater Dinter
It seems this feature was added in Hive 0.13. https://issues.apache.org/jira/browse/HIVE-4943 I would assume this is supported as Spark is by default compiled using Hive 0.13.1. On Sun, Jul 12, 2015 at 7:42 PM, Ruslan Dautkhanov dautkha...@gmail.com wrote: You can see what Spark SQL functions

Re: Does spark supports the Hive function posexplode function?

2015-07-12 Thread ayan guha
Spark already provides an explode function on lateral views. Please see https://issues.apache.org/jira/browse/SPARK-5573. On Mon, Jul 13, 2015 at 6:47 AM, David Sabater Dinter david.sabater.maill...@gmail.com wrote: It seems this feature was added in Hive 0.13.

Re: Does spark supports the Hive function posexplode function?

2015-07-12 Thread Ruslan Dautkhanov
You can see what Spark SQL functions are supported in Spark by doing the following in a notebook: %sql show functions https://forums.databricks.com/questions/665/is-hive-coalesce-function-supported-in-sparksql.html I think Spark SQL support is currently around Hive ~0.11? -- Ruslan

Does spark supports the Hive function posexplode function?

2015-07-07 Thread Jeff J Li
I am trying to use the posexplode function in the HiveContext to auto-generate a sequence number. This feature is supposed to be available Hive 0.13.0. SELECT name, phone FROM contact LATERAL VIEW posexplode(phoneList.phoneNumber) phoneTable AS pos, phone My test program failed with the