Re: Tableau beta connector
Thanks Todd. great stuff :)
Regards,
Ashu
From: Todd Nist
Sent: Thursday, February 19, 2015 7:46 PM
To: Ashutosh Trivedi (MT2013030)
Cc: user@spark.apache.org
Subject: Re: Tableau beta connector
I am able to connect by doing the following using the Tableau Initial SQL and a
custom query:
1. First ingest csv file or json and save out to file system:
import org.apache.spark.sql.SQLContext
import com.databricks.spark.csv._
val sqlContext = new SQLContext(sc)
val demo = sqlContext.csvFile("/user/data/csv/demo.csv")
demo.toJSON.saveAsTextFile("/user/data/json/test")
2. Start $SPARK_HOME/sbin/start-thirftserver:
./sbin/start-thriftserver.sh --master
spark://radtech.io:7077<http://radtech.io:7077> --total-executor-cores 2
--driver-class-path --hiveconf hive.server2.thrift.port=10001 --hiveconf
hive.server2.thrift.bind.host radtech.io<http://radtech.io>
3. Start tableau session. Create a connection to thrift server via SparkSQL
(Beta) connector.
4. In Tableau add the following to the "Initial SQL"
create temporary table test
using org.apache.spark.sql.json
options (path '/user/data/json/test/*');
cache table test;
* Refresh connection.
Then select "New Custom SQL" and issue something like:
select * from test;
You will see your table appear.
HTH.
-Todd
On Thu, Feb 19, 2015 at 5:41 AM, ashu
mailto:ashutosh.triv...@iiitb.org>> wrote:
Hi,
I would like you to read my stack overflow answer to this question. If you need
more clarification feel free to drop a msg.
http://stackoverflow.com/questions/28403664/connect-to-existing-hive-in-intellij-using-sbt-as-build
Regards,
Ashutosh
From: ganterm [via Apache Spark User List] http:///user/SendEmail.jtp?type=node&node=21719&i=0>>
Sent: Thursday, February 19, 2015 12:49 AM
To: Ashutosh Trivedi (MT2013030)
Subject: Re: Tableau beta connector
Ashutosh,
Were you able to figure this out? I am having the exact some question.
I think the answer is to use Spark SQL to create/load a table in Hive (e.g.
execute the HiveQL CREATE TABLE statement) but I am not sure. Hoping for
something more simple than that.
Anybody?
Thanks!
If you reply to this email, your message will be added to the discussion below:
http://apache-spark-user-list.1001560.n3.nabble.com/Tableau-beta-connector-tp21512p21709.html
To unsubscribe from Tableau beta connector, click here.
NAML<http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
View this message in context: Re: Tableau beta
connector<http://apache-spark-user-list.1001560.n3.nabble.com/Tableau-beta-connector-tp21512p21719.html>
Sent from the Apache Spark User List mailing list
archive<http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com.
Re: Tableau beta connector
I am able to connect by doing the following using the Tableau Initial SQL
and a custom query:
1.
First ingest csv file or json and save out to file system:
import org.apache.spark.sql.SQLContext
import com.databricks.spark.csv._
val sqlContext = new SQLContext(sc)
val demo = sqlContext.csvFile("/user/data/csv/demo.csv")
demo.toJSON.saveAsTextFile("/user/data/json/test”)
2.
Start $SPARK_HOME/sbin/start-thirftserver:
./sbin/start-thriftserver.sh --master spark://radtech.io:7077
--total-executor-cores 2 --driver-class-path --hiveconf
hive.server2.thrift.port=10001 --hiveconf
hive.server2.thrift.bind.host radtech.io
3.
Start tableau session. Create a connection to thrift server via SparkSQL
(Beta) connector.
4.
In Tableau add the following to the “Initial SQL”
create temporary table test
using org.apache.spark.sql.json
options (path '/user/data/json/test/*’);
cache table test;
1. Refresh connection.
Then select “New Custom SQL” and issue something like:
select * from test;
You will see your table appear.
HTH.
-Todd
On Thu, Feb 19, 2015 at 5:41 AM, ashu wrote:
> Hi,
>
> I would like you to read my stack overflow answer to this question. If you
> need more clarification feel free to drop a msg.
>
>
> http://stackoverflow.com/questions/28403664/connect-to-existing-hive-in-intellij-using-sbt-as-build
>
>
> Regards,
>
> Ashutosh
> --
> *From:* ganterm [via Apache Spark User List] <http:///user/SendEmail.jtp?type=node&node=21719&i=0>>
> *Sent:* Thursday, February 19, 2015 12:49 AM
> *To:* Ashutosh Trivedi (MT2013030)
> *Subject:* Re: Tableau beta connector
>
> Ashutosh,
>
> Were you able to figure this out? I am having the exact some question.
> I think the answer is to use Spark SQL to create/load a table in Hive
> (e.g. execute the HiveQL CREATE TABLE statement) but I am not sure. Hoping
> for something more simple than that.
>
> Anybody?
>
> Thanks!
>
> --
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-spark-user-list.1001560.n3.nabble.com/Tableau-beta-connector-tp21512p21709.html
> To unsubscribe from Tableau beta connector, click here.
> NAML
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
> --
> View this message in context: Re: Tableau beta connector
> <http://apache-spark-user-list.1001560.n3.nabble.com/Tableau-beta-connector-tp21512p21719.html>
>
> Sent from the Apache Spark User List mailing list archive
> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com.
>
Re: Tableau beta connector
Hi, I would like you to read my stack overflow answer to this question. If you need more clarification feel free to drop a msg. http://stackoverflow.com/questions/28403664/connect-to-existing-hive-in-intellij-using-sbt-as-build Regards, Ashutosh From: ganterm [via Apache Spark User List] Sent: Thursday, February 19, 2015 12:49 AM To: Ashutosh Trivedi (MT2013030) Subject: Re: Tableau beta connector Ashutosh, Were you able to figure this out? I am having the exact some question. I think the answer is to use Spark SQL to create/load a table in Hive (e.g. execute the HiveQL CREATE TABLE statement) but I am not sure. Hoping for something more simple than that. Anybody? Thanks! If you reply to this email, your message will be added to the discussion below: http://apache-spark-user-list.1001560.n3.nabble.com/Tableau-beta-connector-tp21512p21709.html To unsubscribe from Tableau beta connector, click here<http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=21512&code=YXNodXRvc2gudHJpdmVkaUBpaWl0Yi5vcmd8MjE1MTJ8LTM5MzMxOTc2MQ==>. NAML<http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
Re: Tableau beta connector
Ashutosh, Were you able to figure this out? I am having the exact some question. I think the answer is to use Spark SQL to create/load a table in Hive (e.g. execute the HiveQL CREATE TABLE statement) but I am not sure. Hoping for something more simple than that. Anybody? Thanks! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Tableau-beta-connector-tp21512p21709.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: Tableau beta connector
Could you clarify what you mean by "build another Spark and work through Spark Submit"? If you are referring to utilizing Spark spark and thrift, you could start the Spark service and then have your spark-shell, spark-submit, and/or thrift service aim at the master you have started. On Thu Feb 05 2015 at 2:02:04 AM Ashutosh Trivedi (MT2013030) < ashutosh.triv...@iiitb.org> wrote: > Hi Denny , Ismail one last question.. > > > Is it necessary to build another Spark and work through Spark-submit ? > > > I work on IntelliJ using SBT as build script, I have Hive set up with > postgres as metastore, I can run the hive server using command > > *hive --service metastore* > > *hive --service hiveserver2* > > > After that if I can use hive-context in my code > > val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc) > > > Do some processing on RDD and persist it on hive using registerTempTable > > and tableau can extract that RDD persisted on hive. > > > Regards, > > Ashutosh > > > -- > *From:* Denny Lee > > *Sent:* Thursday, February 5, 2015 1:27 PM > *To:* Ashutosh Trivedi (MT2013030); İsmail Keskin > *Cc:* user@spark.apache.org > *Subject:* Re: Tableau beta connector > The context is that you would create your RDDs and then persist them in > Hive. Once in Hive, the data is accessible from the Tableau extract through > Spark thrift server. > On Wed, Feb 4, 2015 at 23:36 Ashutosh Trivedi (MT2013030) < > ashutosh.triv...@iiitb.org> wrote: > >> Thanks Denny and Ismail. >> >> >> Denny ,I went through your blog, It was great help. I guess tableau >> beta connector also following the same procedure,you described in blog. I >> am building the Spark now. >> >> Basically what I don't get is, where to put my data so that tableau can >> extract. >> >> >> So Ismail,its just Spark SQL. No RDDs I think I am getting it now . We >> use spark for our big data processing and we want *processed data (Rdd)* >> into tableau. So we should put our data in hive metastore and tableau will >> extract it from there using this connector? Correct me if I am wrong. >> >> >> I guess I have to look at how thrift server works. >> -- >> *From:* Denny Lee >> *Sent:* Thursday, February 5, 2015 12:20 PM >> *To:* İsmail Keskin; Ashutosh Trivedi (MT2013030) >> *Cc:* user@spark.apache.org >> *Subject:* Re: Tableau beta connector >> >> Some quick context behind how Tableau interacts with Spark / Hive >> can also be found at >> https://www.concur.com/blog/en-us/connect-tableau-to-sparksql - its for >> how to connect from Tableau to the thrift server before the official >> Tableau beta connector but should provide some of the additional context >> called out. HTH! >> >> On Wed Feb 04 2015 at 10:47:23 PM İsmail Keskin < >> ismail.kes...@dilisim.com> wrote: >> >>> Tableau connects to Spark Thrift Server via an ODBC driver. So, none of >>> the RDD stuff applies, you just issue SQL queries from Tableau. >>> >>> The table metadata can come from Hive Metastore if you place your >>> hive-site.xml to configuration directory of Spark. >>> >>> On Thu, Feb 5, 2015 at 8:11 AM, ashu wrote: >>> >>>> Hi, >>>> I am trying out the tableau beta connector to Spark SQL. I have few >>>> basics >>>> question: >>>> Will this connector be able to fetch the schemaRDDs into tableau. >>>> Will all the schemaRDDs be exposed to tableau? >>>> Basically I am not getting what tableau will fetch at data-source? Is it >>>> existing files in HDFS? RDDs or something else. >>>> Question may be naive but I did not get answer anywhere else. Would >>>> really >>>> appreciate if someone has already tried it, can help me with this. >>>> >>>> Thanks, >>>> Ashutosh >>>> >>>> >>>> >>>> -- >>>> View this message in context: >>>> http://apache-spark-user-list.1001560.n3.nabble.com/Tableau-beta-connector-tp21512.html >>>> Sent from the Apache Spark User List mailing list archive at Nabble.com. >>>> >>>> - >>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>>> For additional commands, e-mail: user-h...@spark.apache.org >>>> >>>> >>>
Re: Tableau beta connector
okay. So the queries tableau will run on the persisted data will be through SPARK SQL to improve performance and to take advantage of SPARK SQL. Thanks again Denny From: Denny Lee Sent: Thursday, February 5, 2015 1:27 PM To: Ashutosh Trivedi (MT2013030); İsmail Keskin Cc: user@spark.apache.org Subject: Re: Tableau beta connector The context is that you would create your RDDs and then persist them in Hive. Once in Hive, the data is accessible from the Tableau extract through Spark thrift server. On Wed, Feb 4, 2015 at 23:36 Ashutosh Trivedi (MT2013030) mailto:ashutosh.triv...@iiitb.org>> wrote: Thanks Denny and Ismail. Denny ,I went through your blog, It was great help. I guess tableau beta connector also following the same procedure,you described in blog. I am building the Spark now. Basically what I don't get is, where to put my data so that tableau can extract. So Ismail,its just Spark SQL. No RDDs I think I am getting it now . We use spark for our big data processing and we want processed data (Rdd) into tableau. So we should put our data in hive metastore and tableau will extract it from there using this connector? Correct me if I am wrong. I guess I have to look at how thrift server works. From: Denny Lee mailto:denny.g@gmail.com>> Sent: Thursday, February 5, 2015 12:20 PM To: İsmail Keskin; Ashutosh Trivedi (MT2013030) Cc: user@spark.apache.org<mailto:user@spark.apache.org> Subject: Re: Tableau beta connector Some quick context behind how Tableau interacts with Spark / Hive can also be found at https://www.concur.com/blog/en-us/connect-tableau-to-sparksql - its for how to connect from Tableau to the thrift server before the official Tableau beta connector but should provide some of the additional context called out. HTH! On Wed Feb 04 2015 at 10:47:23 PM İsmail Keskin mailto:ismail.kes...@dilisim.com>> wrote: Tableau connects to Spark Thrift Server via an ODBC driver. So, none of the RDD stuff applies, you just issue SQL queries from Tableau. The table metadata can come from Hive Metastore if you place your hive-site.xml to configuration directory of Spark. On Thu, Feb 5, 2015 at 8:11 AM, ashu mailto:ashutosh.triv...@iiitb.org>> wrote: Hi, I am trying out the tableau beta connector to Spark SQL. I have few basics question: Will this connector be able to fetch the schemaRDDs into tableau. Will all the schemaRDDs be exposed to tableau? Basically I am not getting what tableau will fetch at data-source? Is it existing files in HDFS? RDDs or something else. Question may be naive but I did not get answer anywhere else. Would really appreciate if someone has already tried it, can help me with this. Thanks, Ashutosh -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Tableau-beta-connector-tp21512.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org<mailto:user-unsubscr...@spark.apache.org> For additional commands, e-mail: user-h...@spark.apache.org<mailto:user-h...@spark.apache.org>
Re: Tableau beta connector
The context is that you would create your RDDs and then persist them in Hive. Once in Hive, the data is accessible from the Tableau extract through Spark thrift server. On Wed, Feb 4, 2015 at 23:36 Ashutosh Trivedi (MT2013030) < ashutosh.triv...@iiitb.org> wrote: > Thanks Denny and Ismail. > > > Denny ,I went through your blog, It was great help. I guess tableau beta > connector also following the same procedure,you described in blog. I am > building the Spark now. > > Basically what I don't get is, where to put my data so that tableau can > extract. > > > So Ismail,its just Spark SQL. No RDDs I think I am getting it now . We > use spark for our big data processing and we want *processed data (Rdd)* > into tableau. So we should put our data in hive metastore and tableau will > extract it from there using this connector? Correct me if I am wrong. > > > I guess I have to look at how thrift server works. > -- > *From:* Denny Lee > *Sent:* Thursday, February 5, 2015 12:20 PM > *To:* İsmail Keskin; Ashutosh Trivedi (MT2013030) > *Cc:* user@spark.apache.org > *Subject:* Re: Tableau beta connector > > Some quick context behind how Tableau interacts with Spark / Hive can > also be found at > https://www.concur.com/blog/en-us/connect-tableau-to-sparksql - its for > how to connect from Tableau to the thrift server before the official > Tableau beta connector but should provide some of the additional context > called out. HTH! > > On Wed Feb 04 2015 at 10:47:23 PM İsmail Keskin > wrote: > >> Tableau connects to Spark Thrift Server via an ODBC driver. So, none of >> the RDD stuff applies, you just issue SQL queries from Tableau. >> >> The table metadata can come from Hive Metastore if you place your >> hive-site.xml to configuration directory of Spark. >> >> On Thu, Feb 5, 2015 at 8:11 AM, ashu wrote: >> >>> Hi, >>> I am trying out the tableau beta connector to Spark SQL. I have few >>> basics >>> question: >>> Will this connector be able to fetch the schemaRDDs into tableau. >>> Will all the schemaRDDs be exposed to tableau? >>> Basically I am not getting what tableau will fetch at data-source? Is it >>> existing files in HDFS? RDDs or something else. >>> Question may be naive but I did not get answer anywhere else. Would >>> really >>> appreciate if someone has already tried it, can help me with this. >>> >>> Thanks, >>> Ashutosh >>> >>> >>> >>> -- >>> View this message in context: >>> http://apache-spark-user-list.1001560.n3.nabble.com/Tableau-beta-connector-tp21512.html >>> Sent from the Apache Spark User List mailing list archive at Nabble.com. >>> >>> - >>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>> For additional commands, e-mail: user-h...@spark.apache.org >>> >>> >>
Re: Tableau beta connector
Thanks Denny and Ismail. Denny ,I went through your blog, It was great help. I guess tableau beta connector also following the same procedure,you described in blog. I am building the Spark now. Basically what I don't get is, where to put my data so that tableau can extract. So Ismail,its just Spark SQL. No RDDs I think I am getting it now . We use spark for our big data processing and we want processed data (Rdd) into tableau. So we should put our data in hive metastore and tableau will extract it from there using this connector? Correct me if I am wrong. I guess I have to look at how thrift server works. From: Denny Lee Sent: Thursday, February 5, 2015 12:20 PM To: İsmail Keskin; Ashutosh Trivedi (MT2013030) Cc: user@spark.apache.org Subject: Re: Tableau beta connector Some quick context behind how Tableau interacts with Spark / Hive can also be found at https://www.concur.com/blog/en-us/connect-tableau-to-sparksql - its for how to connect from Tableau to the thrift server before the official Tableau beta connector but should provide some of the additional context called out. HTH! On Wed Feb 04 2015 at 10:47:23 PM İsmail Keskin mailto:ismail.kes...@dilisim.com>> wrote: Tableau connects to Spark Thrift Server via an ODBC driver. So, none of the RDD stuff applies, you just issue SQL queries from Tableau. The table metadata can come from Hive Metastore if you place your hive-site.xml to configuration directory of Spark. On Thu, Feb 5, 2015 at 8:11 AM, ashu mailto:ashutosh.triv...@iiitb.org>> wrote: Hi, I am trying out the tableau beta connector to Spark SQL. I have few basics question: Will this connector be able to fetch the schemaRDDs into tableau. Will all the schemaRDDs be exposed to tableau? Basically I am not getting what tableau will fetch at data-source? Is it existing files in HDFS? RDDs or something else. Question may be naive but I did not get answer anywhere else. Would really appreciate if someone has already tried it, can help me with this. Thanks, Ashutosh -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Tableau-beta-connector-tp21512.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org<mailto:user-unsubscr...@spark.apache.org> For additional commands, e-mail: user-h...@spark.apache.org<mailto:user-h...@spark.apache.org>
Re: Tableau beta connector
Some quick context behind how Tableau interacts with Spark / Hive can also be found at https://www.concur.com/blog/en-us/connect-tableau-to-sparksql - its for how to connect from Tableau to the thrift server before the official Tableau beta connector but should provide some of the additional context called out. HTH! On Wed Feb 04 2015 at 10:47:23 PM İsmail Keskin wrote: > Tableau connects to Spark Thrift Server via an ODBC driver. So, none of > the RDD stuff applies, you just issue SQL queries from Tableau. > > The table metadata can come from Hive Metastore if you place your > hive-site.xml to configuration directory of Spark. > > On Thu, Feb 5, 2015 at 8:11 AM, ashu wrote: > >> Hi, >> I am trying out the tableau beta connector to Spark SQL. I have few basics >> question: >> Will this connector be able to fetch the schemaRDDs into tableau. >> Will all the schemaRDDs be exposed to tableau? >> Basically I am not getting what tableau will fetch at data-source? Is it >> existing files in HDFS? RDDs or something else. >> Question may be naive but I did not get answer anywhere else. Would really >> appreciate if someone has already tried it, can help me with this. >> >> Thanks, >> Ashutosh >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Tableau-beta-connector-tp21512.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> - >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> >> >
Re: Tableau beta connector
Tableau connects to Spark Thrift Server via an ODBC driver. So, none of the RDD stuff applies, you just issue SQL queries from Tableau. The table metadata can come from Hive Metastore if you place your hive-site.xml to configuration directory of Spark. On Thu, Feb 5, 2015 at 8:11 AM, ashu wrote: > Hi, > I am trying out the tableau beta connector to Spark SQL. I have few basics > question: > Will this connector be able to fetch the schemaRDDs into tableau. > Will all the schemaRDDs be exposed to tableau? > Basically I am not getting what tableau will fetch at data-source? Is it > existing files in HDFS? RDDs or something else. > Question may be naive but I did not get answer anywhere else. Would really > appreciate if someone has already tried it, can help me with this. > > Thanks, > Ashutosh > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Tableau-beta-connector-tp21512.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > - > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
Tableau beta connector
Hi, I am trying out the tableau beta connector to Spark SQL. I have few basics question: Will this connector be able to fetch the schemaRDDs into tableau. Will all the schemaRDDs be exposed to tableau? Basically I am not getting what tableau will fetch at data-source? Is it existing files in HDFS? RDDs or something else. Question may be naive but I did not get answer anywhere else. Would really appreciate if someone has already tried it, can help me with this. Thanks, Ashutosh -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Tableau-beta-connector-tp21512.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
