Re: Spark SQL odbc on Windows
Have you looked at Kylin? http://www.ebaytechblog.com/2014/10/20/announcing-kylin-extreme-olap-engine-for-big-data/#.VOtXUUsqnUk Pretty new but has the backing of eBay. On 23 Feb 2015, at 15:38, Denny Lee wrote: > Makes complete sense - I became a fan of Spark for pretty much the same > reasons. Best of luck, eh?! > > On Mon Feb 23 2015 at 12:08:49 AM Francisco Orchard > wrote: > Hi Denny & Ashic, > > You are putting us on the right direction. Thanks! > > We will try following your advice and provide feeback to the list. > > Regarding your question Denny. We feel MS is lacking on an scalable solution > for SSAS (tabular or multidim) so when it comes to big data, the only answer > they have is their expensive appliance (APS) which can be used as a rolap > engine. We are interesting into testing how Spark escalate to check if it can > be offered as an less expensive alternative when a single machine is not > enough to our client needs. The reason why we do not go with tabular in the > first place is because its rolap mode (direct query) is still too limited. > And thanks for writing the klout paper!! We were already using it as a > guideline for our tests. > > Best regards, > Francisco > From: Denny Lee > Sent: 22/02/2015 17:56 > To: Ashic Mahtab; Francisco Orchard; Apache Spark > Subject: Re: Spark SQL odbc on Windows > > Back to thrift, there was an earlier thread on this topic at > http://mail-archives.apache.org/mod_mbox/spark-user/201411.mbox/%3CCABPQxsvXA-ROPeXN=wjcev_n9gv-drqxujukbp_goutvnyx...@mail.gmail.com%3E > that may be useful as well. > > On Sun Feb 22 2015 at 8:42:29 AM Denny Lee wrote: > Hi Francisco, > > Out of curiosity - why ROLAP mode using multi-dimensional mode (vs tabular) > from SSAS to Spark? As a past SSAS guy you've definitely piqued my interest. > > The one thing that you may run into is that the SQL generated by SSAS can be > quite convoluted. When we were doing the same thing to try to get SSAS to > connect to Hive (ref paper at > http://download.microsoft.com/download/D/2/0/D20E1C5F-72EA-4505-9F26-FEF9550EFD44/MOLAP2HIVE_KLOUT.docx) > that was definitely a blocker. Note that Spark SQL is different than HIVEQL > but you may run into the same issue. If so, the trick you may want to use is > similar to the paper - use a SQL Server linked server connection and have SQL > Server be your "translator" for the SQL generated by SSAS. > > HTH! > Denny > > On Sun, Feb 22, 2015 at 01:44 Ashic Mahtab wrote: > Hi Francisco, > While I haven't tried this, have a look at the contents of > start-thriftserver.sh - all it's doing is setting up a few variables and > calling: > > /bin/spark-submit --class > org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 > > and passing some additional parameters. Perhaps doing the same would work? > > I also believe that this hosts a jdbc server (not odbc), but there's a free > odbc connector from databricks built by Simba, with which I've been able to > connect to a spark cluster hosted on linux. > > -Ashic. > > To: user@spark.apache.org > From: forch...@gmail.com > Subject: Spark SQL odbc on Windows > Date: Sun, 22 Feb 2015 09:45:03 +0100 > > > Hello, > I work on a MS consulting company and we are evaluating including SPARK on > our BigData offer. We are particulary interested into testing SPARK as rolap > engine for SSAS but we cannot find a way to activate the odbc server (thrift) > on a Windows custer. There is no start-thriftserver.sh command available for > windows. > > Somebody knows if there is a way to make this work? > > Thanks in advance!! > Francisco
Re: Spark SQL odbc on Windows
Makes complete sense - I became a fan of Spark for pretty much the same reasons. Best of luck, eh?! On Mon Feb 23 2015 at 12:08:49 AM Francisco Orchard wrote: > Hi Denny & Ashic, > > You are putting us on the right direction. Thanks! > > We will try following your advice and provide feeback to the list. > > Regarding your question Denny. We feel MS is lacking on an scalable > solution for SSAS (tabular or multidim) so when it comes to big data, the > only answer they have is their expensive appliance (APS) which can be used > as a rolap engine. We are interesting into testing how Spark escalate to > check if it can be offered as an less expensive alternative when a single > machine is not enough to our client needs. The reason why we do not go with > tabular in the first place is because its rolap mode (direct query) is > still too limited. And thanks for writing the klout paper!! We were already > using it as a guideline for our tests. > > Best regards, > Francisco > -- > From: Denny Lee > Sent: 22/02/2015 17:56 > To: Ashic Mahtab ; Francisco Orchard ; > Apache Spark > Subject: Re: Spark SQL odbc on Windows > > Back to thrift, there was an earlier thread on this topic at > http://mail-archives.apache.org/mod_mbox/spark-user/201411.mbox/%3CCABPQxsvXA-ROPeXN=wjcev_n9gv-drqxujukbp_goutvnyx...@mail.gmail.com%3E > that may be useful as well. > > On Sun Feb 22 2015 at 8:42:29 AM Denny Lee wrote: > >> Hi Francisco, >> >> Out of curiosity - why ROLAP mode using multi-dimensional mode (vs >> tabular) from SSAS to Spark? As a past SSAS guy you've definitely piqued my >> interest. >> >> The one thing that you may run into is that the SQL generated by SSAS can >> be quite convoluted. When we were doing the same thing to try to get SSAS >> to connect to Hive (ref paper at >> http://download.microsoft.com/download/D/2/0/D20E1C5F-72EA-4505-9F26-FEF9550EFD44/MOLAP2HIVE_KLOUT.docx) >> that was definitely a blocker. Note that Spark SQL is different than HIVEQL >> but you may run into the same issue. If so, the trick you may want to use >> is similar to the paper - use a SQL Server linked server connection and >> have SQL Server be your "translator" for the SQL generated by SSAS. >> >> HTH! >> Denny >> >> On Sun, Feb 22, 2015 at 01:44 Ashic Mahtab wrote: >> >>> Hi Francisco, >>> While I haven't tried this, have a look at the contents of >>> start-thriftserver.sh - all it's doing is setting up a few variables and >>> calling: >>> >>> /bin/spark-submit --class org.apache.spark.sql.hive. >>> thriftserver.HiveThriftServer2 >>> >>> and passing some additional parameters. Perhaps doing the same would >>> work? >>> >>> I also believe that this hosts a jdbc server (not odbc), but there's a >>> free odbc connector from databricks built by Simba, with which I've been >>> able to connect to a spark cluster hosted on linux. >>> >>> -Ashic. >>> >>> -- >>> To: user@spark.apache.org >>> From: forch...@gmail.com >>> Subject: Spark SQL odbc on Windows >>> Date: Sun, 22 Feb 2015 09:45:03 +0100 >>> >>> >>> Hello, >>> I work on a MS consulting company and we are evaluating including SPARK >>> on our BigData offer. We are particulary interested into testing SPARK as >>> rolap engine for SSAS but we cannot find a way to activate the odbc server >>> (thrift) on a Windows custer. There is no start-thriftserver.sh command >>> available for windows. >>> >>> Somebody knows if there is a way to make this work? >>> >>> Thanks in advance!! >>> Francisco >>> >>
RE: Spark SQL odbc on Windows
Hi Denny & Ashic, You are putting us on the right direction. Thanks! We will try following your advice and provide feeback to the list. Regarding your question Denny. We feel MS is lacking on an scalable solution for SSAS (tabular or multidim) so when it comes to big data, the only answer they have is their expensive appliance (APS) which can be used as a rolap engine. We are interesting into testing how Spark escalate to check if it can be offered as an less expensive alternative when a single machine is not enough to our client needs. The reason why we do not go with tabular in the first place is because its rolap mode (direct query) is still too limited. And thanks for writing the klout paper!! We were already using it as a guideline for our tests. Best regards, Francisco -Original Message- From: "Denny Lee" Sent: 22/02/2015 17:56 To: "Ashic Mahtab" ; "Francisco Orchard" ; "Apache Spark" Subject: Re: Spark SQL odbc on Windows Back to thrift, there was an earlier thread on this topic at http://mail-archives.apache.org/mod_mbox/spark-user/201411.mbox/%3CCABPQxsvXA-ROPeXN=wjcev_n9gv-drqxujukbp_goutvnyx...@mail.gmail.com%3E that may be useful as well. On Sun Feb 22 2015 at 8:42:29 AM Denny Lee wrote: Hi Francisco, Out of curiosity - why ROLAP mode using multi-dimensional mode (vs tabular) from SSAS to Spark? As a past SSAS guy you've definitely piqued my interest. The one thing that you may run into is that the SQL generated by SSAS can be quite convoluted. When we were doing the same thing to try to get SSAS to connect to Hive (ref paper at http://download.microsoft.com/download/D/2/0/D20E1C5F-72EA-4505-9F26-FEF9550EFD44/MOLAP2HIVE_KLOUT.docx) that was definitely a blocker. Note that Spark SQL is different than HIVEQL but you may run into the same issue. If so, the trick you may want to use is similar to the paper - use a SQL Server linked server connection and have SQL Server be your "translator" for the SQL generated by SSAS. HTH! Denny On Sun, Feb 22, 2015 at 01:44 Ashic Mahtab wrote: Hi Francisco, While I haven't tried this, have a look at the contents of start-thriftserver.sh - all it's doing is setting up a few variables and calling: /bin/spark-submit --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 and passing some additional parameters. Perhaps doing the same would work? I also believe that this hosts a jdbc server (not odbc), but there's a free odbc connector from databricks built by Simba, with which I've been able to connect to a spark cluster hosted on linux. -Ashic. To: user@spark.apache.org From: forch...@gmail.com Subject: Spark SQL odbc on Windows Date: Sun, 22 Feb 2015 09:45:03 +0100 Hello, I work on a MS consulting company and we are evaluating including SPARK on our BigData offer. We are particulary interested into testing SPARK as rolap engine for SSAS but we cannot find a way to activate the odbc server (thrift) on a Windows custer. There is no start-thriftserver.sh command available for windows. Somebody knows if there is a way to make this work? Thanks in advance!! Francisco
Re: Spark SQL odbc on Windows
Back to thrift, there was an earlier thread on this topic at http://mail-archives.apache.org/mod_mbox/spark-user/201411.mbox/%3CCABPQxsvXA-ROPeXN=wjcev_n9gv-drqxujukbp_goutvnyx...@mail.gmail.com%3E that may be useful as well. On Sun Feb 22 2015 at 8:42:29 AM Denny Lee wrote: > Hi Francisco, > > Out of curiosity - why ROLAP mode using multi-dimensional mode (vs > tabular) from SSAS to Spark? As a past SSAS guy you've definitely piqued my > interest. > > The one thing that you may run into is that the SQL generated by SSAS can > be quite convoluted. When we were doing the same thing to try to get SSAS > to connect to Hive (ref paper at > http://download.microsoft.com/download/D/2/0/D20E1C5F-72EA-4505-9F26-FEF9550EFD44/MOLAP2HIVE_KLOUT.docx) > that was definitely a blocker. Note that Spark SQL is different than HIVEQL > but you may run into the same issue. If so, the trick you may want to use > is similar to the paper - use a SQL Server linked server connection and > have SQL Server be your "translator" for the SQL generated by SSAS. > > HTH! > Denny > > On Sun, Feb 22, 2015 at 01:44 Ashic Mahtab wrote: > >> Hi Francisco, >> While I haven't tried this, have a look at the contents of >> start-thriftserver.sh - all it's doing is setting up a few variables and >> calling: >> >> /bin/spark-submit --class org.apache.spark.sql.hive. >> thriftserver.HiveThriftServer2 >> >> and passing some additional parameters. Perhaps doing the same would work? >> >> I also believe that this hosts a jdbc server (not odbc), but there's a >> free odbc connector from databricks built by Simba, with which I've been >> able to connect to a spark cluster hosted on linux. >> >> -Ashic. >> >> -- >> To: user@spark.apache.org >> From: forch...@gmail.com >> Subject: Spark SQL odbc on Windows >> Date: Sun, 22 Feb 2015 09:45:03 +0100 >> >> >> Hello, >> I work on a MS consulting company and we are evaluating including SPARK >> on our BigData offer. We are particulary interested into testing SPARK as >> rolap engine for SSAS but we cannot find a way to activate the odbc server >> (thrift) on a Windows custer. There is no start-thriftserver.sh command >> available for windows. >> >> Somebody knows if there is a way to make this work? >> >> Thanks in advance!! >> Francisco >> >
Re: Spark SQL odbc on Windows
Hi Francisco, Out of curiosity - why ROLAP mode using multi-dimensional mode (vs tabular) from SSAS to Spark? As a past SSAS guy you've definitely piqued my interest. The one thing that you may run into is that the SQL generated by SSAS can be quite convoluted. When we were doing the same thing to try to get SSAS to connect to Hive (ref paper at http://download.microsoft.com/download/D/2/0/D20E1C5F-72EA-4505-9F26-FEF9550EFD44/MOLAP2HIVE_KLOUT.docx) that was definitely a blocker. Note that Spark SQL is different than HIVEQL but you may run into the same issue. If so, the trick you may want to use is similar to the paper - use a SQL Server linked server connection and have SQL Server be your "translator" for the SQL generated by SSAS. HTH! Denny On Sun, Feb 22, 2015 at 01:44 Ashic Mahtab wrote: > Hi Francisco, > While I haven't tried this, have a look at the contents of > start-thriftserver.sh - all it's doing is setting up a few variables and > calling: > > /bin/spark-submit --class > org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 > > and passing some additional parameters. Perhaps doing the same would work? > > I also believe that this hosts a jdbc server (not odbc), but there's a > free odbc connector from databricks built by Simba, with which I've been > able to connect to a spark cluster hosted on linux. > > -Ashic. > > -- > To: user@spark.apache.org > From: forch...@gmail.com > Subject: Spark SQL odbc on Windows > Date: Sun, 22 Feb 2015 09:45:03 +0100 > > > Hello, > I work on a MS consulting company and we are evaluating including SPARK on > our BigData offer. We are particulary interested into testing SPARK as > rolap engine for SSAS but we cannot find a way to activate the odbc server > (thrift) on a Windows custer. There is no start-thriftserver.sh command > available for windows. > > Somebody knows if there is a way to make this work? > > Thanks in advance!! > Francisco >
RE: Spark SQL odbc on Windows
Hi Francisco,While I haven't tried this, have a look at the contents of start-thriftserver.sh - all it's doing is setting up a few variables and calling: /bin/spark-submit --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 and passing some additional parameters. Perhaps doing the same would work? I also believe that this hosts a jdbc server (not odbc), but there's a free odbc connector from databricks built by Simba, with which I've been able to connect to a spark cluster hosted on linux. -Ashic. To: user@spark.apache.org From: forch...@gmail.com Subject: Spark SQL odbc on Windows Date: Sun, 22 Feb 2015 09:45:03 +0100 Hello, I work on a MS consulting company and we are evaluating including SPARK on our BigData offer. We are particulary interested into testing SPARK as rolap engine for SSAS but we cannot find a way to activate the odbc server (thrift) on a Windows custer. There is no start-thriftserver.sh command available for windows. Somebody knows if there is a way to make this work? Thanks in advance!! Francisco
