Re: SQL JSON array operations
Hi @jvuillermet, I am encountering a similar problem. Did you manage to figure out parsing of complicated unstructured json files. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/SQL-JSON-array-operations-tp21164p27113.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: SQL JSON array operations
yeah that's where I ended up. Thanks ! I'll give it a try. On Thu, Jan 15, 2015 at 8:46 PM, Ayoub [via Apache Spark User List] < ml-node+s1001560n21172...@n3.nabble.com> wrote: > You could try to use hive context which bring HiveQL, it would allow you > to query nested structures using "LATERAL VIEW explode..." > > see doc > <https://cwiki.apache.org/confluence/display/Hive/LanguageManual+LateralView> > here > > > -- > If you reply to this email, your message will be added to the discussion > below: > > http://apache-spark-user-list.1001560.n3.nabble.com/SQL-JSON-array-operations-tp21164p21172.html > To unsubscribe from SQL JSON array operations, click here > <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=21164&code=amVyZW15LnZ1aWxsZXJtZXRAZ21haWwuY29tfDIxMTY0fC0yMzc2NjY3MDI=> > . > NAML > <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/SQL-JSON-array-operations-tp21164p21173.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: SQL JSON array operations
You could try to use hive context which bring HiveQL, it would allow you to query nested structures using "LATERAL VIEW explode..." see doc <https://cwiki.apache.org/confluence/display/Hive/LanguageManual+LateralView> here -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/SQL-JSON-array-operations-tp21164p21172.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: SQL JSON array operations
You could try yo use hive context which bring HiveQL, it would allow you to
query nested structures using "LATERAL VIEW explode..."
On Jan 15, 2015 4:03 PM, "jvuillermet" wrote:
> let's say my json file lines looks like this
>
> {"user": "baz", "tags" : ["foo", "bar"] }
>
>
> sqlContext.jsonFile("data.json")
> ...
> How could I query for user with "bar" tags using SQL
>
> sqlContext.sql("select user from users where tags ?contains? 'bar' ")
>
> I could simplify the request and use the returned RDD to filter on tags but
> I'm exploring an app where users can write their SQL queries
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/SQL-JSON-array-operations-tp21164.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
>
SQL JSON array operations
let's say my json file lines looks like this
{"user": "baz", "tags" : ["foo", "bar"] }
sqlContext.jsonFile("data.json")
...
How could I query for user with "bar" tags using SQL
sqlContext.sql("select user from users where tags ?contains? 'bar' ")
I could simplify the request and use the returned RDD to filter on tags but
I'm exploring an app where users can write their SQL queries
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/SQL-JSON-array-operations-tp21164.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
