答复: [SparkSQL] pre-check syntex before running spark job?

2017-02-21 Thread Linyuxin
apila.pl>; user <user@spark.apache.org> 主题: RE: [SparkSQL] pre-check syntex before running spark job? Hi, you can use spark sql Antlr grammer for pre check you syntax. https://github.com/apache/spark/blob/acf71c63cdde8dced8d108260cdd35e1cc992248/sql/catalyst/src/main/antlr4/org/apache/spark/sql/cat

RE: [SparkSQL] pre-check syntex before running spark job?

2017-02-21 Thread Gurdit Singh
, 2017 7:34 AM To: Irving Duran <irving.du...@gmail.com>; Yong Zhang <java8...@hotmail.com> Cc: Jacek Laskowski <ja...@japila.pl>; user <user@spark.apache.org> Subject: 答复: [SparkSQL] pre-check syntex before running spark job? Actually，I want a standalone jar as I can check

答复: [SparkSQL] pre-check syntex before running spark job?

2017-02-21 Thread Linyuxin
m>; user <user@spark.apache.org> 主题: Re: [SparkSQL] pre-check syntex before running spark job? You can also run it on REPL and test to see if you are getting the expected result. Thank You, Irving Duran On Tue, Feb 21, 2017 at 8:01 AM, Yong Zhang <java8...@hotmail.com<mailto:java8...

Re: [SparkSQL] pre-check syntex before running spark job?

2017-02-21 Thread Irving Duran
. > > > Yong > > > -- > *From:* Jacek Laskowski <ja...@japila.pl> > *Sent:* Tuesday, February 21, 2017 4:34 AM > *To:* Linyuxin > *Cc:* user > *Subject:* Re: [SparkSQL] pre-check syntex before running spark job? > > Hi, > > Never he

Re: [SparkSQL] pre-check syntex before running spark job?

2017-02-21 Thread Yong Zhang
You can always use explain method to validate your DF or SQL, before any action. Yong From: Jacek Laskowski <ja...@japila.pl> Sent: Tuesday, February 21, 2017 4:34 AM To: Linyuxin Cc: user Subject: Re: [SparkSQL] pre-check syntex before running spark jo

Re: [SparkSQL] pre-check syntex before running spark job?

2017-02-21 Thread Jacek Laskowski
Hi, Never heard about such a tool before. You could use Antlr to parse SQLs (just as Spark SQL does while parsing queries). I think it's a one-hour project. Jacek On 21 Feb 2017 4:44 a.m., "Linyuxin" wrote: Hi All, Is there any tool/api to check the sql syntax without

[SparkSQL] pre-check syntex before running spark job?

2017-02-20 Thread Linyuxin
Hi All, Is there any tool/api to check the sql syntax without running spark job actually? Like the siddhiQL on storm here: SiddhiManagerService. validateExecutionPlan https://github.com/wso2/siddhi/blob/master/modules/siddhi-core/src/main/java/org/wso2/siddhi/core/SiddhiManagerService.java it