The latest Derby SQL Reference manual (version 10.11) can be found here: https://db.apache.org/derby/docs/10.11/ref/index.html. It is, indeed, very useful to have a comprehensive reference guide. The Derby build scripts can also produce a BNF description of the grammar--but that is not part of the public documentation for the project. The BNF is trivial to generate because it is an artifact of the JavaCC grammar generator which Derby uses.
I appreciate the difficulty of maintaining a formal reference guide for a rapidly evolving SQL dialect like Spark's. A machine-generated BNF, however, is easy to imagine. But perhaps not so easy to implement. Spark's SQL grammar is implemented in Scala, extending the DSL support provided by the Scala language. I am new to programming in Scala, so I don't know whether the Scala ecosystem provides any good tools for reverse-engineering a BNF from a class which extends scala.util.parsing.combinator.syntactical.StandardTokenParsers. Thanks, -Rick vivekw...@gmail.com wrote on 09/11/2015 05:05:47 AM: > From: vivek bhaskar <vivekw...@gmail.com> > To: Ted Yu <yuzhih...@gmail.com> > Cc: user <user@spark.apache.org> > Date: 09/11/2015 05:06 AM > Subject: Re: Is there any Spark SQL reference manual? > Sent by: vivekw...@gmail.com > > Hi Ted, > > The link you mention do not have complete list of supported syntax. > For example, few supported syntax are listed as "Supported Hive > features" but that do not claim to be exhaustive (even if it is so, > one has to filter out a lot many lines from Hive QL reference and > still will not be sure if its all - due to versions mismatch). > > Quickly searching online gives me link for another popular open > source project which has good sql reference: https://db.apache.org/ > derby/docs/10.1/ref/crefsqlj23296.html. > > I had similar expectation when I was looking for all supported DDL > and DML syntax along with their extensions. For example, > a. Select expression along with supported extensions i.e. where > clause, group by, different supported joins etc. > b. SQL format for Create, Insert, Alter table etc. > c. SQL for Insert, Update, Delete, etc along with their extensions. > d. Syntax for view creation, if supported > e. Syntax for explain mechanism > f. List of supported functions, operators, etc. I can see that 100s > of function are added in 1.5 but then you have to make lot of cross > check from code to JIRA tickets. > > So I wanted a piece of documentation that can provide all such > information at a single place. > > Regards, > Vivek > > On Fri, Sep 11, 2015 at 4:29 PM, Ted Yu <yuzhih...@gmail.com> wrote: > You may have seen this: > https://spark.apache.org/docs/latest/sql-programming-guide.html > > Please suggest what should be added. > > Cheers > > On Fri, Sep 11, 2015 at 3:43 AM, vivek bhaskar <vivekw...@gmail.com> wrote: > Hi all, > > I am looking for a reference manual for Spark SQL some thing like > many database vendors have. I could find one for hive ql https:// > cwiki.apache.org/confluence/display/Hive/LanguageManual but not > anything specific to spark sql. > > Please suggest. SQL reference specific to latest release will be of > great help. > > Regards, > Vivek
