+1 for the new thrift server to get rid of the Hive dependencies! On Mon, Dec 23, 2019 at 7:55 PM Yuming Wang <wgy...@gmail.com> wrote:
> I'm +1 for this SPIP for these two reasons: > > 1. The current thriftserver has some issues that are not easy to solve, > such as: SPARK-28636 <https://issues.apache.org/jira/browse/SPARK-28636>. > 2. The difference between the version of ORC we are using and the built-in > Hive is using is getting bigger and bigger. We can't ensure that there will > be no compatibility issues in the future. If thriftserver does not depend > on Hive, it will be much easier to upgrade the built-in Hive in the future. > > On Sat, Dec 21, 2019 at 9:28 PM angers.zhu <angers....@gmail.com> wrote: > >> Hi all, >> >> I have complete a Design doc about how to use and config this new thrift >> server, and some design detail about change and impersonation. >> >> Hope for your suggestions and ideas. >> >> SPIP DOC : >> https://docs.google.com/document/d/1ug4K5e2okF5Q2Pzi3qJiUILwwqkn0fVQaQ-Q95HEcJQ/edit#heading=h.x97c6tj78zo0 >> Design DOC : >> https://docs.google.com/document/d/1UKE9QTtHqSZBq0V_vEn54PlWaWPiRAKf_JvcT0skaSo/edit#heading=h.q1ed5q1ldh14 >> Thrift server about configurations >> https://docs.google.com/document/d/1uI35qJmQO4FKE6pr0h3zetZqww-uI8QsQjxaYY_qb1s/edit?usp=drive_web&ouid=110963191229426834922 >> >> Best Regards >> angers.zhu >> angers....@gmail.com >> >> <https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=angers.zhu&uid=angers.zhu%40gmail.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22angers.zhu%40gmail.com%22%5D> >> 签名由 网易邮箱大师 <https://mail.163.com/dashi/dlpro.html?from=mail81> 定制 >> >> --------- Forwarded Message --------- >> From： angers.zhu <angers....@gmail.com> <angers....@gmail.com> >> Date： 12/18/2019 22:29 >> To： dev-ow...@spark.apache.org <dev-ow...@spark.apache.org> >> <dev-ow...@spark.apache.org> >> Subject： Re: [VOTE][SPARK-29018][SPIP]:Build spark thrift server based >> on protocol v11 >> >> Add spark-dev group access privilege to google. >> angers.zhu >> angers....@gmail.com >> >> <https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=angers.zhu&uid=angers.zhu%40gmail.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22angers.zhu%40gmail.com%22%5D> >> 签名由 网易邮箱大师 <https://mail.163.com/dashi/dlpro.html?from=mail81> 定制 >> >> On 12/18/2019 22:02，Sandeep Katta<sandeep0102.opensou...@gmail.com> >> <sandeep0102.opensou...@gmail.com> wrote： >> >> I couldn't access the doc, please give permission to the spark-dev group >> >> On Wed, 18 Dec 2019 at 18:05, angers.zhu <angers....@gmail.com> wrote: >> >>> With the development of Spark and Hive，in current sql/hive-thriftserver >>> module, >>> >>> we need to do a lot of work to solve code conflicts for different >>> built-in hive versions. >>> >>> It's an annoying and unending work in current ways. And these issues >>> have limited >>> >>> our ability and convenience to develop new features for Spark’s thrift >>> server. >>> >>> We suppose to implement a new thrift server and JDBC driver based on >>> Hive’s >>> latest v11 >>> >>> TCLService.thrift thrift protocol. Finally, the new thrift server have >>> below feature: >>> >>> 1. >>> >>> Build new module spark-service as spark’s thrift server >>> 2. >>> >>> Don't need as much reflection and inherited code as `hive-thriftser` >>> modules >>> 3. >>> >>> Support all functions current `sql/hive-thriftserver` support >>> 4. >>> >>> Use all code maintained by spark itself, won’t depend on Hive >>> 5. >>> >>> Support origin functions use spark’s own way, won't limited by >>> Hive's code >>> 6. >>> >>> Support running without hive metastore or with hive metastore >>> 7. >>> >>> Support user impersonation by Multi-tenant splited hive >>> authentication and DFS authentication >>> 8. >>> >>> Support session hook for with spark’s own code >>> 9. >>> >>> Add a new jdbc driver spark-jdbc, with spark’s own connection url >>> “jdbc:spark:<host>:<port>/<db>” >>> 10. >>> >>> Support both hive-jdbc and spark-jdbc client, then we can support >>> most clients and BI platform >>> >>> >>> >>> https://issues.apache.org/jira/browse/SPARK-29018 >>> >>> Google Doc: >>> https://docs.google.com/document/d/1ug4K5e2okF5Q2Pzi3qJiUILwwqkn0fVQaQ-Q95HEcJQ/edit# >>> >>> [ ] +1: Accept the proposal as an official SPIP >>> [ ] +0 >>> [ ] -1: I don't think this is a good idea because ... >>> >>> I'll start with my +1 >>> angers.zhu >>> angers....@gmail.com >>> >>> <https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=angers.zhu&uid=angers.zhu%40gmail.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22angers.zhu%40gmail.com%22%5D> >>> 签名由 网易邮箱大师 <https://mail.163.com/dashi/dlpro.html?from=mail81> 定制 >>> >>>
