Re: How To Access Hive 2 Through JDBC Using Kerberos
Sorry for the misunderstanding, I found out today that actually my colleagues didn't make Spark work with Kerberos authentication for Hive JDBC. Spark can't give Kerberos parameters to the executors. Sorry again for the misunderstanding. On Thu, Jul 9, 2020 at 9:52 PM Jeff Evans wrote: > There are various sample JDBC URLs documented here, depending on the > driver vendor, Kerberos (or not), and SSL (or not). Often times, > unsurprisingly, SSL is used in conjunction with Kerberos. Even if you > don't use StreamSets software at all, you might find these useful. > > > https://ask.streamsets.com/question/7/how-do-you-configure-a-hive-impala-jdbc-driver-for-data-collector/?answer=8#post-id-8 > > On Thu, Jul 9, 2020 at 11:28 AM Daniel de Oliveira Mantovani < > daniel.oliveira.mantov...@gmail.com> wrote: > >> One of my colleagues found this solution: >> >> >> https://github.com/morfious902002/impala-spark-jdbc-kerberos/blob/master/src/main/java/ImpalaSparkJDBC.java >> >> If you need to connect to Hive or Impala using JDBC with Kerberos >> authentication from Apache Spark, you can use it and will work. >> >> You can download the driver from Cloudera here: >> https://www.cloudera.com/downloads/connectors/hive/jdbc/2-6-1.html >> >> >> >> On Tue, Jul 7, 2020 at 12:03 AM Daniel de Oliveira Mantovani < >> daniel.oliveira.mantov...@gmail.com> wrote: >> >>> Hello Gabor, >>> >>> I meant, third-party connector* not "connection". >>> >>> Thank you so much! >>> >>> On Mon, Jul 6, 2020 at 1:09 PM Gabor Somogyi >>> wrote: >>> Hi Daniel, I'm just working on the developer API where any custom JDBC connection provider(including Hive) can be added. Not sure what you mean by third-party connection but AFAIK there is no workaround at the moment. BR, G On Mon, Jul 6, 2020 at 12:09 PM Daniel de Oliveira Mantovani < daniel.oliveira.mantov...@gmail.com> wrote: > Hello List, > > Is it possible to access Hive 2 through JDBC with Kerberos > authentication from Apache Spark JDBC interface ? If it's possible do you > have an example ? > > I found this tickets on JIRA: > https://issues.apache.org/jira/browse/SPARK-12312 > https://issues.apache.org/jira/browse/SPARK-31815 > > Do you know if there's a workaround for this ? Maybe using a > third-party connection ? > > Thank you so much > -- > > -- > Daniel Mantovani > > >>> >>> -- >>> >>> -- >>> Daniel Mantovani >>> >>> >> >> -- >> >> -- >> Daniel Mantovani >> >> -- -- Daniel Mantovani
Re: How To Access Hive 2 Through JDBC Using Kerberos
There are various sample JDBC URLs documented here, depending on the driver vendor, Kerberos (or not), and SSL (or not). Often times, unsurprisingly, SSL is used in conjunction with Kerberos. Even if you don't use StreamSets software at all, you might find these useful. https://ask.streamsets.com/question/7/how-do-you-configure-a-hive-impala-jdbc-driver-for-data-collector/?answer=8#post-id-8 On Thu, Jul 9, 2020 at 11:28 AM Daniel de Oliveira Mantovani < daniel.oliveira.mantov...@gmail.com> wrote: > One of my colleagues found this solution: > > > https://github.com/morfious902002/impala-spark-jdbc-kerberos/blob/master/src/main/java/ImpalaSparkJDBC.java > > If you need to connect to Hive or Impala using JDBC with Kerberos > authentication from Apache Spark, you can use it and will work. > > You can download the driver from Cloudera here: > https://www.cloudera.com/downloads/connectors/hive/jdbc/2-6-1.html > > > > On Tue, Jul 7, 2020 at 12:03 AM Daniel de Oliveira Mantovani < > daniel.oliveira.mantov...@gmail.com> wrote: > >> Hello Gabor, >> >> I meant, third-party connector* not "connection". >> >> Thank you so much! >> >> On Mon, Jul 6, 2020 at 1:09 PM Gabor Somogyi >> wrote: >> >>> Hi Daniel, >>> >>> I'm just working on the developer API where any custom JDBC connection >>> provider(including Hive) can be added. >>> Not sure what you mean by third-party connection but AFAIK there is no >>> workaround at the moment. >>> >>> BR, >>> G >>> >>> >>> On Mon, Jul 6, 2020 at 12:09 PM Daniel de Oliveira Mantovani < >>> daniel.oliveira.mantov...@gmail.com> wrote: >>> Hello List, Is it possible to access Hive 2 through JDBC with Kerberos authentication from Apache Spark JDBC interface ? If it's possible do you have an example ? I found this tickets on JIRA: https://issues.apache.org/jira/browse/SPARK-12312 https://issues.apache.org/jira/browse/SPARK-31815 Do you know if there's a workaround for this ? Maybe using a third-party connection ? Thank you so much -- -- Daniel Mantovani >> >> -- >> >> -- >> Daniel Mantovani >> >> > > -- > > -- > Daniel Mantovani > >
Re: How To Access Hive 2 Through JDBC Using Kerberos
One of my colleagues found this solution: https://github.com/morfious902002/impala-spark-jdbc-kerberos/blob/master/src/main/java/ImpalaSparkJDBC.java If you need to connect to Hive or Impala using JDBC with Kerberos authentication from Apache Spark, you can use it and will work. You can download the driver from Cloudera here: https://www.cloudera.com/downloads/connectors/hive/jdbc/2-6-1.html On Tue, Jul 7, 2020 at 12:03 AM Daniel de Oliveira Mantovani < daniel.oliveira.mantov...@gmail.com> wrote: > Hello Gabor, > > I meant, third-party connector* not "connection". > > Thank you so much! > > On Mon, Jul 6, 2020 at 1:09 PM Gabor Somogyi > wrote: > >> Hi Daniel, >> >> I'm just working on the developer API where any custom JDBC connection >> provider(including Hive) can be added. >> Not sure what you mean by third-party connection but AFAIK there is no >> workaround at the moment. >> >> BR, >> G >> >> >> On Mon, Jul 6, 2020 at 12:09 PM Daniel de Oliveira Mantovani < >> daniel.oliveira.mantov...@gmail.com> wrote: >> >>> Hello List, >>> >>> Is it possible to access Hive 2 through JDBC with Kerberos >>> authentication from Apache Spark JDBC interface ? If it's possible do you >>> have an example ? >>> >>> I found this tickets on JIRA: >>> https://issues.apache.org/jira/browse/SPARK-12312 >>> https://issues.apache.org/jira/browse/SPARK-31815 >>> >>> Do you know if there's a workaround for this ? Maybe using a third-party >>> connection ? >>> >>> Thank you so much >>> -- >>> >>> -- >>> Daniel Mantovani >>> >>> > > -- > > -- > Daniel Mantovani > > -- -- Daniel Mantovani
Re: How To Access Hive 2 Through JDBC Using Kerberos
Hello Gabor, I meant, third-party connector* not "connection". Thank you so much! On Mon, Jul 6, 2020 at 1:09 PM Gabor Somogyi wrote: > Hi Daniel, > > I'm just working on the developer API where any custom JDBC connection > provider(including Hive) can be added. > Not sure what you mean by third-party connection but AFAIK there is no > workaround at the moment. > > BR, > G > > > On Mon, Jul 6, 2020 at 12:09 PM Daniel de Oliveira Mantovani < > daniel.oliveira.mantov...@gmail.com> wrote: > >> Hello List, >> >> Is it possible to access Hive 2 through JDBC with Kerberos authentication >> from Apache Spark JDBC interface ? If it's possible do you have an example ? >> >> I found this tickets on JIRA: >> https://issues.apache.org/jira/browse/SPARK-12312 >> https://issues.apache.org/jira/browse/SPARK-31815 >> >> Do you know if there's a workaround for this ? Maybe using a third-party >> connection ? >> >> Thank you so much >> -- >> >> -- >> Daniel Mantovani >> >> -- -- Daniel Mantovani
Re: How To Access Hive 2 Through JDBC Using Kerberos
Hi Daniel, I'm just working on the developer API where any custom JDBC connection provider(including Hive) can be added. Not sure what you mean by third-party connection but AFAIK there is no workaround at the moment. BR, G On Mon, Jul 6, 2020 at 12:09 PM Daniel de Oliveira Mantovani < daniel.oliveira.mantov...@gmail.com> wrote: > Hello List, > > Is it possible to access Hive 2 through JDBC with Kerberos authentication > from Apache Spark JDBC interface ? If it's possible do you have an example ? > > I found this tickets on JIRA: > https://issues.apache.org/jira/browse/SPARK-12312 > https://issues.apache.org/jira/browse/SPARK-31815 > > Do you know if there's a workaround for this ? Maybe using a third-party > connection ? > > Thank you so much > -- > > -- > Daniel Mantovani > >
How To Access Hive 2 Through JDBC Using Kerberos
Hello List, Is it possible to access Hive 2 through JDBC with Kerberos authentication from Apache Spark JDBC interface ? If it's possible do you have an example ? I found this tickets on JIRA: https://issues.apache.org/jira/browse/SPARK-12312 https://issues.apache.org/jira/browse/SPARK-31815 Do you know if there's a workaround for this ? Maybe using a third-party connection ? Thank you so much -- -- Daniel Mantovani
