Re: Spark sql jdbc fails for Oracle NUMBER type columns

[Richard Hillegas]

Hi Rajesh,

The 1.6 schedule is available on the front page of the Spark wiki:
https://cwiki.apache.org/confluence/display/SPARK/Wiki+Homepage. I don't
know of any workarounds for this problem.

Thanks,
Rick


Madabhattula Rajesh Kumar <mrajaf...@gmail.com> wrote on 11/05/2015
06:35:22 PM:

> From: Madabhattula Rajesh Kumar <mrajaf...@gmail.com>
> To: Richard Hillegas/San Francisco/IBM@IBMUS
> Cc: "u...@spark.incubator.apache.org"
> <u...@spark.incubator.apache.org>, "user@spark.apache.org"
> <user@spark.apache.org>
> Date: 11/05/2015 06:35 PM
> Subject: Re: Spark sql jdbc fails for Oracle NUMBER type columns
>
> Hi Richard,

> Thank you for the updates. Do you know tentative timeline for 1.6
> release? Mean while, any workaround solution for this issue?

> Regards,
> Rajesh
>

>
> On Thu, Nov 5, 2015 at 10:57 PM, Richard Hillegas <rhil...@us.ibm.com>
wrote:
> Or you may be referring to
https://issues.apache.org/jira/browse/SPARK-10648
> . That issue has a couple pull requests but I think that the limited
> bandwidth of the committers still applies.
>
> Thanks,
> Rick
>
>
> Richard Hillegas/San Francisco/IBM@IBMUS wrote on 11/05/2015 09:16:42 AM:
>
> > From: Richard Hillegas/San Francisco/IBM@IBMUS
> > To: Madabhattula Rajesh Kumar <mrajaf...@gmail.com>
> > Cc: "user@spark.apache.org" <user@spark.apache.org>,
> > "u...@spark.incubator.apache.org" <u...@spark.incubator.apache.org>
> > Date: 11/05/2015 09:17 AM
> > Subject: Re: Spark sql jdbc fails for Oracle NUMBER type columns
>
> >
> > Hi Rajesh,
> >
> > I think that you may be referring to https://issues.apache.org/jira/
> > browse/SPARK-10909. A pull request on that issue was submitted more
> > than a month ago but it has not been committed. I think that the
> > committers are busy working on issues which were targeted for 1.6
> > and I doubt that they will have the spare cycles to vet that pull
request.
> >
> > Thanks,
> > Rick
> >
> >
> > Madabhattula Rajesh Kumar <mrajaf...@gmail.com> wrote on 11/05/2015
> > 05:51:29 AM:
> >
> > > From: Madabhattula Rajesh Kumar <mrajaf...@gmail.com>
> > > To: "user@spark.apache.org" <user@spark.apache.org>,
> > > "u...@spark.incubator.apache.org" <u...@spark.incubator.apache.org>
> > > Date: 11/05/2015 05:51 AM
> > > Subject: Spark sql jdbc fails for Oracle NUMBER type columns
> > >
> > > Hi,
> >
> > > Is this issue fixed in 1.5.1 version?
> >
> > > Regards,
> > > Rajesh

Re: Spark sql jdbc fails for Oracle NUMBER type columns

[Richard Hillegas]

Or you may be referring to
https://issues.apache.org/jira/browse/SPARK-10648. That issue has a couple
pull requests but I think that the limited bandwidth of the committers
still applies.

Thanks,
Rick


Richard Hillegas/San Francisco/IBM@IBMUS wrote on 11/05/2015 09:16:42 AM:

> From: Richard Hillegas/San Francisco/IBM@IBMUS
> To: Madabhattula Rajesh Kumar <mrajaf...@gmail.com>
> Cc: "user@spark.apache.org" <user@spark.apache.org>,
> "u...@spark.incubator.apache.org" <u...@spark.incubator.apache.org>
> Date: 11/05/2015 09:17 AM
> Subject: Re: Spark sql jdbc fails for Oracle NUMBER type columns
>
> Hi Rajesh,
>
> I think that you may be referring to https://issues.apache.org/jira/
> browse/SPARK-10909. A pull request on that issue was submitted more
> than a month ago but it has not been committed. I think that the
> committers are busy working on issues which were targeted for 1.6
> and I doubt that they will have the spare cycles to vet that pull
request.
>
> Thanks,
> Rick
>
>
> Madabhattula Rajesh Kumar <mrajaf...@gmail.com> wrote on 11/05/2015
> 05:51:29 AM:
>
> > From: Madabhattula Rajesh Kumar <mrajaf...@gmail.com>
> > To: "user@spark.apache.org" <user@spark.apache.org>,
> > "u...@spark.incubator.apache.org" <u...@spark.incubator.apache.org>
> > Date: 11/05/2015 05:51 AM
> > Subject: Spark sql jdbc fails for Oracle NUMBER type columns
> >
> > Hi,
>
> > Is this issue fixed in 1.5.1 version?
>
> > Regards,
> > Rajesh

Re: Spark sql jdbc fails for Oracle NUMBER type columns

[Richard Hillegas]

Hi Rajesh,

I think that you may be referring to
https://issues.apache.org/jira/browse/SPARK-10909. A pull request on that
issue was submitted more than a month ago but it has not been committed. I
think that the committers are busy working on issues which were targeted
for 1.6 and I doubt that they will have the spare cycles to vet that pull
request.

Thanks,
Rick


Madabhattula Rajesh Kumar  wrote on 11/05/2015
05:51:29 AM:

> From: Madabhattula Rajesh Kumar 
> To: "user@spark.apache.org" ,
> "u...@spark.incubator.apache.org" 
> Date: 11/05/2015 05:51 AM
> Subject: Spark sql jdbc fails for Oracle NUMBER type columns
>
> Hi,

> Is this issue fixed in 1.5.1 version?

> Regards,
> Rajesh

Re: Spark scala REPL - Unable to create sqlContext

[Richard Hillegas]

Note that embedded Derby supports multiple, simultaneous connections, that
is, multiple simultaneous users. But a Derby database is owned by the
process which boots it. Only one process can boot a Derby database at a
given time. The creation of multiple SQL contexts must be spawning multiple
attempts to boot and own the database. If multiple different processes want
to access the same Derby database simultaneously, then the database should
be booted by the Derby network server. After that, the processes which want
to access the database simultaneously can use the Derby network client
driver, not the Derby embedded driver. For more information, see the Derby
Server and Administration Guide:
http://db.apache.org/derby/docs/10.12/adminguide/index.html

Thanks,
Rick Hillegas



Deenar Toraskar  wrote on 10/25/2015 11:29:54
PM:

> From: Deenar Toraskar 
> To: "Ge, Yao (Y.)" 
> Cc: Ted Yu , user 
> Date: 10/25/2015 11:30 PM
> Subject: Re: Spark scala REPL - Unable to create sqlContext
>
> Embedded Derby, which Hive/Spark SQL uses as the default metastore
> only supports a single user at a time. Till this issue is fixed, you
> could use another metastore that supports multiple concurrent users
> (e.g. networked derby or mysql) to get around it.
>
> On 25 October 2015 at 16:15, Ge, Yao (Y.)  wrote:
> Thanks. I wonder why this is not widely reported in the user forum.
> The RELP shell is basically broken in 1.5 .0 and 1.5.1
> -Yao
>
> From: Ted Yu [mailto:yuzhih...@gmail.com]
> Sent: Sunday, October 25, 2015 12:01 PM
> To: Ge, Yao (Y.)
> Cc: user
> Subject: Re: Spark scala REPL - Unable to create sqlContext
>
> Have you taken a look at the fix for SPARK-11000 which is in the
> upcoming 1.6.0 release ?
>
> Cheers
>
> On Sun, Oct 25, 2015 at 8:42 AM, Yao  wrote:
> I have not been able to start Spark scala shell since 1.5 as it was not
able
> to create the sqlContext during the startup. It complains the
metastore_db
> is already locked: "Another instance of Derby may have already booted the
> database". The Derby log is attached.
>
> I only have this problem with starting the shell in yarn-client mode. I
am
> working with HDP2.2.6 which runs Hadoop 2.6.
>
> -Yao derby.log
>

>
>
>
> --
> View this message in context: http://apache-spark-user-list.
> 1001560.n3.nabble.com/Spark-scala-REPL-Unable-to-create-sqlContext-
> tp25195.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>

Re: Can not subscript to mailing list

[Richard Hillegas]

Hi Jeff,

Hard to say what's going on. I have had problems subscribing to the Apache
lists in the past. My problems, which may be different than yours, were
caused by replying to the confirmation request from a different email
account than the account I was trying to subscribe from. It was easy for me
to get confused because I was using a single mail tool to manage multiple
email accounts (a personal account, a yahoo account, and a gmail account).
Check your confirmation response to see which email account responded to
the confirmation request.

Hope this helps,
Rick


"jeff.sadow...@gmail.com"  wrote on 10/20/2015
08:48:49 AM:

> From: "jeff.sadow...@gmail.com" 
> To: user@spark.apache.org
> Date: 10/20/2015 08:49 AM
> Subject: Can not subscript to mailing list
>
> I am having issues subscribing to the user@spark.apache.org mailing list.
>
> I would like to be added to the mailing list so I can post some
> configuration questions I have to the list that I do not see asked on the
> list.
>
> When I tried adding myself I got an email titled "confirm subscribe to
> user@spark.apache.org" but after replying as it says to do I get nothing.
I
> tried today to remove and re-add myself and I got a reply back saying I
was
> not on the list when trying to unsubscribe. When I tried to add myself
again
> I don't get any emails from it this time. I'm getting other email from
other
> people and nothing is in spam. I tried with a second email account as
well
> and the same thing is happening on it. I got the initial "confirm
subscribe
> to user@spark.apache.org" email but after replying I get nothing. I can't
> even get another "confirm subscribe to user@spark.apache.org" message.
Both
> of my emails are from google servers one is an organization email the
first
> is a personal google email
>
>
>
>
> --
> View this message in context: http://apache-spark-user-list.
> 1001560.n3.nabble.com/Can-not-subscript-to-mailing-list-tp25143.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>

Re: Spark SQL: Preserving Dataframe Schema

[Richard Hillegas]

As an academic aside, note that all datatypes are nullable according to the
SQL Standard. NOT NULL is modelled in the Standard as a constraint on data
values, not as a parallel universe of special data types. However, very few
databases implement NOT NULL via integrity constraints. Instead, almost all
relational database type systems model NOT NULL as an extra bit of metadata
alongside precision, scale, and length.

Thanks,
Rick Hillegas


Xiao Li  wrote on 10/20/2015 01:17:43 PM:

> From: Xiao Li 
> To: Michael Armbrust 
> Cc: Jerry Lam , "user@spark.apache.org"
> 
> Date: 10/20/2015 01:18 PM
> Subject: Re: Spark SQL: Preserving Dataframe Schema
>
> Sure. Will try to do a pull request this week.
>
> Schema evolution is always painful for database people. IMO, NULL is
> a bad design in the original system R. It introduces a lot of
> problems during the system migration and data integration.
>
> Let me find a possible scenario: RDBMS is used as an ODS. Spark is
> used as an external online data analysis engine. The results could
> be stored in Parquet files and inserted back RDBMS every interval.
> In this case, we could face a few options:
>
> - Change the data types of columns in RDBMS tables to support the
> possible nullable values and the logics of RDBMS applications that
> consume these results must also support NULL. When the applications
> are third-party, changing the applications become harder.
>
> - As what you suggested, before loading the data from the Parquet
> files, we need to add an extra step to do a possible data cleaning,
> value transformation or exception reporting in case of finding NULL.
>
> If having such an external parameter, when writing data schema to
> external data store, Spark will do its best to keep the original
> schema without any change (e.g., keep the initial definition of
> nullability). If some data type/schema conversions are not
> avoidable, it will issue warnings or errors to the users. Does that
> make sense?
>
> Thanks,
>
> Xiao Li
>
>  In this case,
>
> 2015-10-20 12:38 GMT-07:00 Michael Armbrust :
> First, this is not documented in the official document. Maybe we
> should do it?
http://spark.apache.org/docs/latest/sql-programming-guide.html
>
> Pull requests welcome.
>
> Second, nullability is a significant concept in the database people.
> It is part of schema. Extra codes are needed for evaluating if a
> value is null for all the nullable data types. Thus, it might cause
> a problem if you need to use Spark to transfer the data between
> parquet and RDBMS. My suggestion is to introduce another external
parameter?
>
> Sure, but a traditional RDBMS has the opportunity to do validation
> before loading data in.  Thats not really an option when you are
> reading random files from S3.  This is why Hive and many other
> systems in this space treat all columns as nullable.
>
> What would the semantics of this proposed external parameter be?

Re: SQL Context error in 1.5.1 - any work around ?

[Richard Hillegas]

A crude workaround may be to run your spark shell with a sudo command.

Hope this helps,
Rick Hillegas


Sourav Mazumder  wrote on 10/15/2015 09:59:02
AM:

> From: Sourav Mazumder 
> To: user 
> Date: 10/15/2015 09:59 AM
> Subject: SQL Context error in 1.5.1 - any work around ?
>
> I keep on getting this error whenever I'm starting spark-shell : The
> root scratch dir: /tmp/hive on HDFS should be writable. Current
> permissions are: rwx--.

> I cannot work with this if I need to do anything with sqlContext as
> that does not get created.
>
> I could see that a bug is raised for this https://issues.apache.org/
> jira/browse/SPARK-10066.

> However, is there any work around for this.

> I didn't face this problem in 1.4.1

> Regards,
> Sourav

Re: pagination spark sq

[Richard Hillegas]

Hi Ravi,

If you build Spark with Hive support, then your sqlContext variable will be
an instance of HiveContext and you will enjoy the full capabilities of the
Hive query language rather than the more limited capabilities of Spark SQL.
However, even Hive QL does not support the OFFSET clause, at least
according to the Hive language manual:
https://cwiki.apache.org/confluence/display/Hive/LanguageManual. Hive does
support the LIMIT clause. The following works for me:

import org.apache.spark.sql._
import org.apache.spark.sql.types._

val hc = sqlContext

val schema =
  StructType(
StructField("x", IntegerType, nullable=false) ::
StructField("y", DoubleType, nullable=false) :: Nil)

val rdd = sc.parallelize(
  Row(1, 1.0) :: Row(2, 1.34) :: Row(3, 2.3) :: Row(4, 2.5) :: Nil)

val df = hc.createDataFrame(rdd, schema)

df.registerTempTable("test_data")

hc.sql("SELECT * FROM test_data LIMIT 3").show()

exit()


So, to sum up, Hive QL supports a subset of the MySQL LIMIT/OFFSET syntax
(limit, no offset) but does not support the SQL Standard language for
returning a block of rows offset into a large query result.

Hope this helps,
Rick Hillegas



Ravisankar Mani  wrote on 10/12/2015 07:05:05 AM:

> From: Ravisankar Mani 
> To: user@spark.apache.org
> Date: 10/12/2015 07:05 AM
> Subject: pagination spark sq
>
> Hi everyone,
>
> Can you please share optimized query for pagination spark sql?
>

> In Ms SQL Server, They have supported "offset" method query for
> specific row selection.

> Please find the following query

> Select BusinessEntityID,[FirstName], [LastName],[JobTitle]
> from HumanResources.vEmployee
> Order By BusinessEntityID
> --OFFSET 10 ROWS
> FETCH NEXT 10 ROWS ONLY

> Is this support OFFSET method in spark sql? Kindly share the useful
details.

> Regards,
> Ravi

Re: This post has NOT been accepted by the mailing list yet.

[Richard Hillegas]

Hi Akhandeshi,

It may be that you are not seeing your own posts because you are sending
from a gmail account. See for instance
https://support.google.com/a/answer/1703601?hl=en

Hope this helps,
Rick Hillegas
STSM, IBM Analytics, Platform - IBM USA


akhandeshi  wrote on 10/07/2015 08:10:32 AM:

> From: akhandeshi 
> To: user@spark.apache.org
> Date: 10/07/2015 08:10 AM
> Subject: This post has NOT been accepted by the mailing list yet.
>
> I seem to see this for many of my posts... does anyone have solution?
>
>
>
>
> --
> View this message in context: http://apache-spark-user-list.
> 1001560.n3.nabble.com/This-post-has-NOT-been-accepted-by-the-
> mailing-list-yet-tp24969.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>

Re: save DF to JDBC

[Richard Hillegas]

Hi Ruslan,

Here is some sample code which writes a DataFrame to a table in a Derby
database:

import org.apache.spark.sql._
import org.apache.spark.sql.types._

val binaryVal = Array[Byte] ( 1, 2, 3, 4 )
val timestampVal = java.sql.Timestamp.valueOf("1996-01-01 03:30:36")
val dateVal = java.sql.Date.valueOf("1996-01-01")

val allTypes = sc.parallelize(
Array(
  (1,
  1.toLong,
  1.toDouble,
  1.toFloat,
  1.toShort,
  1.toByte,
  "true".toBoolean,
  "one ring to rule them all",
  binaryVal,
  timestampVal,
  dateVal,
  BigDecimal.valueOf(42549.12)
  )
)).toDF(
  "int_col",
  "long_col",
  "double_col",
  "float_col",
  "short_col",
  "byte_col",
  "boolean_col",
  "string_col",
  "binary_col",
  "timestamp_col",
  "date_col",
  "decimal_col"
  )

val properties = new java.util.Properties()

allTypes.write.jdbc("jdbc:derby:/Users/rhillegas/derby/databases/derby1",
"all_spark_types", properties)

Hope this helps,

Rick Hillegas
STSM, IBM Analytics, Platform - IBM USA


Ruslan Dautkhanov  wrote on 10/05/2015 02:44:20 PM:

> From: Ruslan Dautkhanov 
> To: user 
> Date: 10/05/2015 02:45 PM
> Subject: save DF to JDBC
>
> http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-
> to-other-databases
>
> Spark JDBC can read data from JDBC, but can it save back to JDBC?
> Like to an Oracle database through its jdbc driver.
>
> Also looked at SQL Context documentation
> https://spark.apache.org/docs/1.4.0/api/java/org/apache/spark/sql/
> SQLContext.html
> and can't find anything relevant.
>
> Thanks!
>
>
> --
> Ruslan Dautkhanov

Re: unsubscribe

[Richard Hillegas]

Hi Sukesh,

To unsubscribe from the dev list, please send a message to
dev-unsubscr...@spark.apache.org. To unsubscribe from the user list, please
send a message user-unsubscr...@spark.apache.org. Please see:
http://spark.apache.org/community.html#mailing-lists.

Thanks,
-Rick

sukesh kumar  wrote on 09/28/2015 11:39:01 PM:

> From: sukesh kumar 
> To: "user@spark.apache.org" ,
> "d...@spark.apache.org" 
> Date: 09/28/2015 11:39 PM
> Subject: unsubscribe
>
> unsubscribe
>
> --
> Thanks & Best Regards
> Sukesh Kumar

Re: unsubscribe

[Richard Hillegas]

Hi Ntale,

To unsubscribe from the user list, please send a message to
user-unsubscr...@spark.apache.org as described here:
http://spark.apache.org/community.html#mailing-lists.

Thanks,
-Rick

Ntale Lukama  wrote on 09/23/2015 04:34:48 AM:

> From: Ntale Lukama 
> To: user 
> Date: 09/23/2015 04:35 AM
> Subject: unsubscribe

Re: Count for select not matching count for group by

[Richard Hillegas]

For what it's worth, I get the expected result that "filter" behaves like
"group by" when I run the same experiment against a DataFrame which was
loaded from a relational store:

import org.apache.spark.sql._
import org.apache.spark.sql.types._

val df = sqlContext.read.format("jdbc").options(
  Map("url" -> "jdbc:derby:/Users/rhillegas/derby/databases/derby1",
  "dbtable" -> "app.outcomes")).load()

df.select("OUTCOME").groupBy("OUTCOME").count.show
#
# returns:
#
# +---+-+
# |OUTCOME|count|
# +---+-+
# |  A|  128|
# |  B|  256|
# +---+-+

df.filter("OUTCOME = 'A'").count
#
# returns:
#
# res1: Long = 128


df.registerTempTable("test_data")
sqlContext.sql("select OUTCOME, count( OUTCOME ) from test_data group by
OUTCOME").show
#
# returns:
#
# +---+---+
# |OUTCOME|_c1|
# +---+---+
# |  A|128|
# |  B|256|
# +---+---+

Thanks,
-Rick

Michael Kelly  wrote on 09/21/2015 08:06:29
AM:

> From: Michael Kelly 
> To: user@spark.apache.org
> Date: 09/21/2015 08:08 AM
> Subject: Count for select not matching count for group by
>
> Hi,
>
> I'm seeing some strange behaviour with spark 1.5, I have a dataframe
> that I have built from loading and joining some hive tables stored in
> s3.
>
> The dataframe is cached in memory, using df.cache.
>
> What I'm seeing is that the counts I get when I do a group by on a
> column are different from what I get when I filter/select and count.
>
> df.select("outcome").groupBy("outcome").count.show
> outcome | count
> --
> 'A'   |  100
> 'B'   |  200
>
> df.filter("outcome = 'A'").count
> # 50
>
> df.filter(df("outcome") === "A").count
> # 50
>
> I expect the count of columns that match 'A' in the groupBy to match
> the count when filtering. Any ideas what might be happening?
>
> Thanks,
>
> Michael
>
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>

Re: unsubscribe

[Richard Hillegas]

To unsubscribe from the user list, please send a message to
user-unsubscr...@spark.apache.org as described here:
http://spark.apache.org/community.html#mailing-lists.

Thanks,
-Rick

Re: Is there any Spark SQL reference manual?

[Richard Hillegas]

The latest Derby SQL Reference manual (version 10.11) can be found here:
https://db.apache.org/derby/docs/10.11/ref/index.html. It is, indeed, very
useful to have a comprehensive reference guide. The Derby build scripts can
also produce a BNF description of the grammar--but that is not part of the
public documentation for the project. The BNF is trivial to generate
because it is an artifact of the JavaCC grammar generator which Derby uses.

I appreciate the difficulty of maintaining a formal reference guide for a
rapidly evolving SQL dialect like Spark's.

A machine-generated BNF, however, is easy to imagine. But perhaps not so
easy to implement. Spark's SQL grammar is implemented in Scala, extending
the DSL support provided by the Scala language. I am new to programming in
Scala, so I don't know whether the Scala ecosystem provides any good tools
for reverse-engineering a BNF from a class which extends
scala.util.parsing.combinator.syntactical.StandardTokenParsers.

Thanks,
-Rick

vivekw...@gmail.com wrote on 09/11/2015 05:05:47 AM:

> From: vivek bhaskar 
> To: Ted Yu 
> Cc: user 
> Date: 09/11/2015 05:06 AM
> Subject: Re: Is there any Spark SQL reference manual?
> Sent by: vivekw...@gmail.com
>
> Hi Ted,
>
> The link you mention do not have complete list of supported syntax.
> For example, few supported syntax are listed as "Supported Hive
> features" but that do not claim to be exhaustive (even if it is so,
> one has to filter out a lot many lines from Hive QL reference and
> still will not be sure if its all - due to versions mismatch).
>
> Quickly searching online gives me link for another popular open
> source project which has good sql reference: https://db.apache.org/
> derby/docs/10.1/ref/crefsqlj23296.html.
>
> I had similar expectation when I was looking for all supported DDL
> and DML syntax along with their extensions. For example,
> a. Select expression along with supported extensions i.e. where
> clause, group by, different supported joins etc.
> b. SQL format for Create, Insert, Alter table etc.
> c. SQL for Insert, Update, Delete, etc along with their extensions.
> d. Syntax for view creation, if supported
> e. Syntax for explain mechanism
> f. List of supported functions, operators, etc. I can see that 100s
> of function are added in 1.5 but then you have to make lot of cross
> check from code to JIRA tickets.
>
> So I wanted a piece of documentation that can provide all such
> information at a single place.
>
> Regards,
> Vivek
>
> On Fri, Sep 11, 2015 at 4:29 PM, Ted Yu  wrote:
> You may have seen this:
> https://spark.apache.org/docs/latest/sql-programming-guide.html
>
> Please suggest what should be added.
>
> Cheers
>
> On Fri, Sep 11, 2015 at 3:43 AM, vivek bhaskar 
wrote:
> Hi all,
>
> I am looking for a reference manual for Spark SQL some thing like
> many database vendors have. I could find one for hive ql https://
> cwiki.apache.org/confluence/display/Hive/LanguageManual but not
> anything specific to spark sql.
>
> Please suggest. SQL reference specific to latest release will be of
> great help.
>
> Regards,
> Vivek