[jira] [Commented] (LIVY-489) Expose a JDBC endpoint for Livy
[ https://issues.apache.org/jira/browse/LIVY-489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924894#comment-16924894 ] Marco Gaido commented on LIVY-489: -- The right place is the user mailing list, please see https://livy.apache.org/community/. > Expose a JDBC endpoint for Livy > --- > > Key: LIVY-489 > URL: https://issues.apache.org/jira/browse/LIVY-489 > Project: Livy > Issue Type: New Feature > Components: API, Server >Affects Versions: 0.6.0 >Reporter: Marco Gaido >Assignee: Marco Gaido >Priority: Major > Fix For: 0.6.0 > > > Many users and BI tools use JDBC connections in order to retrieve data. As > Livy exposes only a REST API, this is a limitation in its adoption. Hence, > adding a JDBC endpoint may be a very useful feature, which could also make > Livy a more attractive solution for end user to adopt. > Moreover, currently, Spark exposes a JDBC interface, but this has many > limitations, including that all the queries are submitted to the same > application, therefore there is no isolation/security, which can be offered > by Livy, making a Livy JDBC API a better solution for companies/users who > want to use Spark in order to run they queries through JDBC. > In order to make the transition from existing solutions to the new JDBC > server seamless, the proposal is to use the Hive thrift-server and extend it > as it was done by the STS. > [Here, you can find the design > doc.|https://docs.google.com/document/d/18HAR_VnQLegbYyzGg8f4zwD4GtDP5q_t3K21eXecZC4/edit] > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (LIVY-489) Expose a JDBC endpoint for Livy
[ https://issues.apache.org/jira/browse/LIVY-489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924865#comment-16924865 ] Von Han Yu commented on LIVY-489: - Noted and thank you! If I have some more noob question, what is the appropriate thread for it? > Expose a JDBC endpoint for Livy > --- > > Key: LIVY-489 > URL: https://issues.apache.org/jira/browse/LIVY-489 > Project: Livy > Issue Type: New Feature > Components: API, Server >Affects Versions: 0.6.0 >Reporter: Marco Gaido >Assignee: Marco Gaido >Priority: Major > Fix For: 0.6.0 > > > Many users and BI tools use JDBC connections in order to retrieve data. As > Livy exposes only a REST API, this is a limitation in its adoption. Hence, > adding a JDBC endpoint may be a very useful feature, which could also make > Livy a more attractive solution for end user to adopt. > Moreover, currently, Spark exposes a JDBC interface, but this has many > limitations, including that all the queries are submitted to the same > application, therefore there is no isolation/security, which can be offered > by Livy, making a Livy JDBC API a better solution for companies/users who > want to use Spark in order to run they queries through JDBC. > In order to make the transition from existing solutions to the new JDBC > server seamless, the proposal is to use the Hive thrift-server and extend it > as it was done by the STS. > [Here, you can find the design > doc.|https://docs.google.com/document/d/18HAR_VnQLegbYyzGg8f4zwD4GtDP5q_t3K21eXecZC4/edit] > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (LIVY-489) Expose a JDBC endpoint for Livy
[ https://issues.apache.org/jira/browse/LIVY-489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924817#comment-16924817 ] Marco Gaido commented on LIVY-489: -- Yes, sure. The point is: that template in not exhaustive and does not contain all the configs which can be set. > Expose a JDBC endpoint for Livy > --- > > Key: LIVY-489 > URL: https://issues.apache.org/jira/browse/LIVY-489 > Project: Livy > Issue Type: New Feature > Components: API, Server >Affects Versions: 0.6.0 >Reporter: Marco Gaido >Assignee: Marco Gaido >Priority: Major > Fix For: 0.6.0 > > > Many users and BI tools use JDBC connections in order to retrieve data. As > Livy exposes only a REST API, this is a limitation in its adoption. Hence, > adding a JDBC endpoint may be a very useful feature, which could also make > Livy a more attractive solution for end user to adopt. > Moreover, currently, Spark exposes a JDBC interface, but this has many > limitations, including that all the queries are submitted to the same > application, therefore there is no isolation/security, which can be offered > by Livy, making a Livy JDBC API a better solution for companies/users who > want to use Spark in order to run they queries through JDBC. > In order to make the transition from existing solutions to the new JDBC > server seamless, the proposal is to use the Hive thrift-server and extend it > as it was done by the STS. > [Here, you can find the design > doc.|https://docs.google.com/document/d/18HAR_VnQLegbYyzGg8f4zwD4GtDP5q_t3K21eXecZC4/edit] > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (LIVY-489) Expose a JDBC endpoint for Livy
[
https://issues.apache.org/jira/browse/LIVY-489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924813#comment-16924813
]
Von Han Yu commented on LIVY-489:
-
I see now. All this time I've been looking in the wrong file! I thought you
were referring to the {{livy.conf.template}} file under {{/conf}} folder.
Thank you so much!
Made me think: If I put {{livy.server.thrift.enabled}} and
{{livy.server.thrift.port}} in the {{livy.conf}}, will that work?
> Expose a JDBC endpoint for Livy
> ---
>
> Key: LIVY-489
> URL: https://issues.apache.org/jira/browse/LIVY-489
> Project: Livy
> Issue Type: New Feature
> Components: API, Server
>Affects Versions: 0.6.0
>Reporter: Marco Gaido
>Assignee: Marco Gaido
>Priority: Major
> Fix For: 0.6.0
>
>
> Many users and BI tools use JDBC connections in order to retrieve data. As
> Livy exposes only a REST API, this is a limitation in its adoption. Hence,
> adding a JDBC endpoint may be a very useful feature, which could also make
> Livy a more attractive solution for end user to adopt.
> Moreover, currently, Spark exposes a JDBC interface, but this has many
> limitations, including that all the queries are submitted to the same
> application, therefore there is no isolation/security, which can be offered
> by Livy, making a Livy JDBC API a better solution for companies/users who
> want to use Spark in order to run they queries through JDBC.
> In order to make the transition from existing solutions to the new JDBC
> server seamless, the proposal is to use the Hive thrift-server and extend it
> as it was done by the STS.
> [Here, you can find the design
> doc.|https://docs.google.com/document/d/18HAR_VnQLegbYyzGg8f4zwD4GtDP5q_t3K21eXecZC4/edit]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (LIVY-489) Expose a JDBC endpoint for Livy
[
https://issues.apache.org/jira/browse/LIVY-489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924812#comment-16924812
]
Marco Gaido commented on LIVY-489:
--
[~vonatzki] nothing to be sorry about. Please see
https://github.com/apache/incubator-livy/blob/master/server/src/main/scala/org/apache/livy/LivyConf.scala#L101.
{{livy.server.thrift.enabled}} must be set to true, the port is controlled by
https://github.com/apache/incubator-livy/blob/master/server/src/main/scala/org/apache/livy/LivyConf.scala#L109.
Then you should be able to connect using the beeline you can find in the repo
or with any Hive 3 client. Thanks.
> Expose a JDBC endpoint for Livy
> ---
>
> Key: LIVY-489
> URL: https://issues.apache.org/jira/browse/LIVY-489
> Project: Livy
> Issue Type: New Feature
> Components: API, Server
>Affects Versions: 0.6.0
>Reporter: Marco Gaido
>Assignee: Marco Gaido
>Priority: Major
> Fix For: 0.6.0
>
>
> Many users and BI tools use JDBC connections in order to retrieve data. As
> Livy exposes only a REST API, this is a limitation in its adoption. Hence,
> adding a JDBC endpoint may be a very useful feature, which could also make
> Livy a more attractive solution for end user to adopt.
> Moreover, currently, Spark exposes a JDBC interface, but this has many
> limitations, including that all the queries are submitted to the same
> application, therefore there is no isolation/security, which can be offered
> by Livy, making a Livy JDBC API a better solution for companies/users who
> want to use Spark in order to run they queries through JDBC.
> In order to make the transition from existing solutions to the new JDBC
> server seamless, the proposal is to use the Hive thrift-server and extend it
> as it was done by the STS.
> [Here, you can find the design
> doc.|https://docs.google.com/document/d/18HAR_VnQLegbYyzGg8f4zwD4GtDP5q_t3K21eXecZC4/edit]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (LIVY-489) Expose a JDBC endpoint for Livy
[ https://issues.apache.org/jira/browse/LIVY-489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924797#comment-16924797 ] Von Han Yu commented on LIVY-489: - Hi [~mgaido], sorry for being persistent. I can't figure it out even when browsing through the conf options. Which options pertain to the thriftserver? I tried connecting using beeline to both port 1 and 8998 to no avail. I think I am missing a step that would boot the thriftserver. I am really sorry for being such a newbie on this. > Expose a JDBC endpoint for Livy > --- > > Key: LIVY-489 > URL: https://issues.apache.org/jira/browse/LIVY-489 > Project: Livy > Issue Type: New Feature > Components: API, Server >Affects Versions: 0.6.0 >Reporter: Marco Gaido >Assignee: Marco Gaido >Priority: Major > Fix For: 0.6.0 > > > Many users and BI tools use JDBC connections in order to retrieve data. As > Livy exposes only a REST API, this is a limitation in its adoption. Hence, > adding a JDBC endpoint may be a very useful feature, which could also make > Livy a more attractive solution for end user to adopt. > Moreover, currently, Spark exposes a JDBC interface, but this has many > limitations, including that all the queries are submitted to the same > application, therefore there is no isolation/security, which can be offered > by Livy, making a Livy JDBC API a better solution for companies/users who > want to use Spark in order to run they queries through JDBC. > In order to make the transition from existing solutions to the new JDBC > server seamless, the proposal is to use the Hive thrift-server and extend it > as it was done by the STS. > [Here, you can find the design > doc.|https://docs.google.com/document/d/18HAR_VnQLegbYyzGg8f4zwD4GtDP5q_t3K21eXecZC4/edit] > -- This message was sent by Atlassian Jira (v8.3.2#803003)
