Re: HiveThriftServer and spark.sql.hive.thriftServer.singleSession setting

What command did you use to connect? Try this: beeline> !connect jdbc:hive2://localhost:1?hive.server2.transport.mode=http;hive.server2.thrift.http.path=cliservice On Thu, Aug 11, 2016 at 9:23 AM, Richard M [via Apache Spark User List] < ml-node+s1001560n27513...@n3.nabble.com> wrote: >

Re: HiveThriftServer and spark.sql.hive.thriftServer.singleSession setting

It's an issue with the preview build. Switched to RC5 and all is working. Thanks to Michael Armbrust. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/HiveThriftServer-and-spark-sql-hive-thriftServer-singleSession-setting-tp27340p27379.html Sent from the

Re: HiveThriftServer and spark.sql.hive.thriftServer.singleSession setting

Would appreciate if someone: 1. Can confirm if this is an issue or 2. Share on how to get HiveThriftServer2.startWithContext working with shared temp table. I am using Beeline as the JDBC client to access the temp tables of the running Spark app. -- View this message in context:

HiveThriftServer and spark.sql.hive.thriftServer.singleSession setting

Hi, I am on Spark 2.0 Review release. According to Spark 2.0 docs, to share TempTable/View, I need to: "to run the Thrift server in the old single-session mode, please set option spark.sql.hive.thriftServer.singleSession to true." Question: *When using HiveThriftServer2.startWithContext(),

Re: Spark 2.0 Streaming and Event Time

o find it. Keeping it short as I am on the mobile. == Chang Lim <chang...@gmail.com> Jun 9 to Tathagata Hi TD, Thanks for the reply. But I was thinking of "sorting the events by logical time" - more like what yesterday, the Microsoft presenter introduced "reorder

Running of Continuous Aggregation example

Has anyone been able to run the code in The Future of Real-Time in Spark Slide 24 :"Continuous Aggregation"? Specifically, the line: stream("jdbc:mysql//..."), Using Spark 2.0 preview build, I am getting the error when

Re: Spark 2.0 Streaming and Event Time

ks.com> wrote: > There is no special setting for event time (though we will be adding one > for setting a watermark in 2.1 to allow us to reduce the amount of state > that needs to be kept around). Just window/groupBy on the on the column > that is your event time. > > On Wed,

Spark 2.0 Streaming and Event Time

Hi All, Does Spark 2.0 Streaming [sqlContext.read.format(...).stream(...)] support Event Time? In TD's Spark Summit talk yesterday, this is listed as a 2.0 feature. Of so, where is the API or how to set it? Thanks in advanced, Chang -- View this message in context:

Re: HQL function Rollup and Cube

select * from dw_bid (dw_bid is stored in Hive) it says table not found. On Thu, Mar 26, 2015 at 11:43 PM, Chang Lim chang...@gmail.com wrote: Solved. In IDE, project settings was missing the dependent lib jars (jar files under spark-xx/lib). When theses jar is not set, I got class

Re: HQL function Rollup and Cube

Clarification on how the HQL was invoked: hiveContext.sql(select a, b, count(*) from t group by a, b with rollup) Thanks, Chang -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/HQL-function-Rollup-and-Cube-tp22241p22244.html Sent from the Apache Spark

HQL function Rollup and Cube

Has anyone been able to use Hive 0.13 ROLLUP and CUBE functions in Spark 1.3's Hive Context? According to https://issues.apache.org/jira/browse/SPARK-2663, this has been resolved in Spark 1.3. I created an in-memory temp table (t) and tried to execute a ROLLUP(and CUBE) function: select a,

Re: HQL function Rollup and Cube

Solved. In IDE, project settings was missing the dependent lib jars (jar files under spark-xx/lib). When theses jar is not set, I got class not found error about datanucleus classes (compared to an out of memory error in Spark Shell). In the context of Spark Shell, these dependent jars needs to

Re: RDD to DataFrame for using ALS under org.apache.spark.ml.recommendation.ALS

After this line: val sc = new SparkContext(conf) You need to add this line: import sc.implicits._ //this is used to implicitly convert an RDD to a DataFrame. Hope this helps -- View this message in context:

SparkSQL with Streaming RDD

Would appreciate help on: 1. How to convert streaming RDD into JavaSchemaRDD 2. How to structure the driver program to do interactive SparkSQL Using Spark 1.0 with Java. I have steaming code that does upateStateByKey resulting in JavaPairDStream. I am using JavaDStream::compute(time) to get

DStream are not processed after upgrade to Spark 1.0

Hi All, I've some Streaming code in Java that works on 0.9.1. After upgrading to 1.0 (with fix to minor API changes) DStream does not seem to be executing. The tasks got killed in 1 second by the worker. Any idea what is causing it? The worker log file is not logging my debug statements. The

Re: Spark Streaming RDD to Shark table

OK...I needed to set the JVM class.path for the worker to find the fb class: env.put(SPARK_JAVA_OPTS, -Djava.class.path=/home/myInc/hive-0.9.0-bin/lib/libfb303.jar); Now I am seeing the following spark.httpBroadcast.uri error. What am I missing? java.util.NoSuchElementException: