Re: Spark-sql can replace Hive ?
OK you mean use spark.sql as opposed to HiveContext.sql?
val HiveContext = new org.apache.spark.sql.hive.HiveContext(sc)
HiveContext.sql("")
replace with
spark.sql("")
?
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Tue, 15 Jun 2021 at 18:00, Battula, Brahma Reddy
wrote:
> Currently I am using hive sql engine for adhoc queries. As spark-sql also
> supports this, I want migrate from hive.
>
>
>
>
>
>
>
>
>
> *From: *Mich Talebzadeh
> *Date: *Thursday, 10 June 2021 at 8:12 PM
> *To: *Battula, Brahma Reddy
> *Cc: *ayan guha , dev@spark.apache.org <
> dev@spark.apache.org>, u...@spark.apache.org
> *Subject: *Re: Spark-sql can replace Hive ?
>
> These are different things. Spark provides a computational layer and a
> dialogue of SQL based on Hive.
>
>
>
> Hive is a DW on top of HDFS. What are you trying to replace?
>
>
>
> HTH
>
>
>
>
>
>
>view my Linkedin profile
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fin%2Fmich-talebzadeh-ph-d-5205b2%2F=04%7C01%7Cbbattula%40visa.com%7C826c6f38277b4af22f2a08d92c1e034e%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637589329681823204%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000=nt1cQcvCszeHYupi9gHEeHi0RuDEyDoNXEPTVExLxgY%3D=0>
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
>
>
>
> On Thu, 10 Jun 2021 at 12:09, Battula, Brahma Reddy
> wrote:
>
> Thanks for prompt reply.
>
>
>
> I want to replace hive with spark.
>
>
>
>
>
>
>
>
>
> *From: *ayan guha
> *Date: *Thursday, 10 June 2021 at 4:35 PM
> *To: *Battula, Brahma Reddy
> *Cc: *dev@spark.apache.org , u...@spark.apache.org <
> u...@spark.apache.org>
> *Subject: *Re: Spark-sql can replace Hive ?
>
> Would you mind expanding the ask? Spark Sql can use hive by itaelf
>
>
>
> On Thu, 10 Jun 2021 at 8:58 pm, Battula, Brahma Reddy
> wrote:
>
> Hi
>
>
>
> Would like know any refences/docs to replace hive with spark-sql
> completely like how migrate the existing data in hive.?
>
>
>
> thanks
>
>
>
>
>
> --
>
> Best Regards,
> Ayan Guha
>
>
Re: Spark-sql can replace Hive ?
Currently I am using hive sql engine for adhoc queries. As spark-sql also supports this, I want migrate from hive. From: Mich Talebzadeh Date: Thursday, 10 June 2021 at 8:12 PM To: Battula, Brahma Reddy Cc: ayan guha , dev@spark.apache.org , u...@spark.apache.org Subject: Re: Spark-sql can replace Hive ? These are different things. Spark provides a computational layer and a dialogue of SQL based on Hive. Hive is a DW on top of HDFS. What are you trying to replace? HTH [https://docs.google.com/uc?export=download=1-q7RFGRfLMObPuQPWSd9sl_H1UPNFaIZ=0B1BiUVX33unjMWtVUWpINWFCd0ZQTlhTRHpGckh4Wlg4RG80PQ] view my Linkedin profile<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fin%2Fmich-talebzadeh-ph-d-5205b2%2F=04%7C01%7Cbbattula%40visa.com%7C826c6f38277b4af22f2a08d92c1e034e%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637589329681823204%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000=nt1cQcvCszeHYupi9gHEeHi0RuDEyDoNXEPTVExLxgY%3D=0> Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On Thu, 10 Jun 2021 at 12:09, Battula, Brahma Reddy wrote: Thanks for prompt reply. I want to replace hive with spark. From: ayan guha mailto:guha.a...@gmail.com>> Date: Thursday, 10 June 2021 at 4:35 PM To: Battula, Brahma Reddy Cc: dev@spark.apache.org<mailto:dev@spark.apache.org> mailto:dev@spark.apache.org>>, u...@spark.apache.org<mailto:u...@spark.apache.org> mailto:u...@spark.apache.org>> Subject: Re: Spark-sql can replace Hive ? Would you mind expanding the ask? Spark Sql can use hive by itaelf On Thu, 10 Jun 2021 at 8:58 pm, Battula, Brahma Reddy wrote: Hi Would like know any refences/docs to replace hive with spark-sql completely like how migrate the existing data in hive.? thanks -- Best Regards, Ayan Guha
Re: Spark-sql can replace Hive ?
These are different things. Spark provides a computational layer and a dialogue of SQL based on Hive. Hive is a DW on top of HDFS. What are you trying to replace? HTH view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On Thu, 10 Jun 2021 at 12:09, Battula, Brahma Reddy wrote: > Thanks for prompt reply. > > > > I want to replace hive with spark. > > > > > > > > > > *From: *ayan guha > *Date: *Thursday, 10 June 2021 at 4:35 PM > *To: *Battula, Brahma Reddy > *Cc: *dev@spark.apache.org , u...@spark.apache.org < > u...@spark.apache.org> > *Subject: *Re: Spark-sql can replace Hive ? > > Would you mind expanding the ask? Spark Sql can use hive by itaelf > > > > On Thu, 10 Jun 2021 at 8:58 pm, Battula, Brahma Reddy > wrote: > > Hi > > > > Would like know any refences/docs to replace hive with spark-sql > completely like how migrate the existing data in hive.? > > > > thanks > > > > > > -- > > Best Regards, > Ayan Guha >
Re: Spark-sql can replace Hive ?
Thanks for prompt reply. I want to replace hive with spark. From: ayan guha Date: Thursday, 10 June 2021 at 4:35 PM To: Battula, Brahma Reddy Cc: dev@spark.apache.org , u...@spark.apache.org Subject: Re: Spark-sql can replace Hive ? Would you mind expanding the ask? Spark Sql can use hive by itaelf On Thu, 10 Jun 2021 at 8:58 pm, Battula, Brahma Reddy wrote: Hi Would like know any refences/docs to replace hive with spark-sql completely like how migrate the existing data in hive.? thanks -- Best Regards, Ayan Guha
