Hi, you can use spark sql Antlr grammer for pre check you syntax. https://github.com/apache/spark/blob/acf71c63cdde8dced8d108260cdd35e1cc992248/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
From: Linyuxin [mailto:linyu...@huawei.com] Sent: Wednesday, February 22, 2017 7:34 AM To: Irving Duran <irving.du...@gmail.com>; Yong Zhang <java8...@hotmail.com> Cc: Jacek Laskowski <ja...@japila.pl>; user <user@spark.apache.org> Subject: 答复: [SparkSQL] pre-check syntex before running spark job? Actually，I want a standalone jar as I can check the syntax without spark execution environment 发件人: Irving Duran [mailto:irving.du...@gmail.com] 发送时间: 2017年2月21日 23:29 收件人: Yong Zhang <java8...@hotmail.com<mailto:java8...@hotmail.com>> 抄送: Jacek Laskowski <ja...@japila.pl<mailto:ja...@japila.pl>>; Linyuxin <linyu...@huawei.com<mailto:linyu...@huawei.com>>; user <user@spark.apache.org<mailto:user@spark.apache.org>> 主题: Re: [SparkSQL] pre-check syntex before running spark job? You can also run it on REPL and test to see if you are getting the expected result. Thank You, Irving Duran On Tue, Feb 21, 2017 at 8:01 AM, Yong Zhang <java8...@hotmail.com<mailto:java8...@hotmail.com>> wrote: You can always use explain method to validate your DF or SQL, before any action. Yong ________________________________ From: Jacek Laskowski <ja...@japila.pl<mailto:ja...@japila.pl>> Sent: Tuesday, February 21, 2017 4:34 AM To: Linyuxin Cc: user Subject: Re: [SparkSQL] pre-check syntex before running spark job? Hi, Never heard about such a tool before. You could use Antlr to parse SQLs (just as Spark SQL does while parsing queries). I think it's a one-hour project. Jacek On 21 Feb 2017 4:44 a.m., "Linyuxin" <linyu...@huawei.com<mailto:linyu...@huawei.com>> wrote: Hi All, Is there any tool/api to check the sql syntax without running spark job actually? Like the siddhiQL on storm here: SiddhiManagerService. validateExecutionPlan https://github.com/wso2/siddhi/blob/master/modules/siddhi-core/src/main/java/org/wso2/siddhi/core/SiddhiManagerService.java it can validate the syntax before running the sql on storm this is very useful for exposing sql string as a DSL of the platform. --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org<mailto:user-unsubscr...@spark.apache.org>
