[ https://issues.apache.org/jira/browse/SPARK-30022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17685861#comment-17685861 ]
jorge arada commented on SPARK-30022: ------------------------------------- We are facing the same problem. The presto/trino views are not "readable" from spark. What can we do to push this request forward? > Supporting Parsing of Simple Hive Virtual View created from Presto > ------------------------------------------------------------------ > > Key: SPARK-30022 > URL: https://issues.apache.org/jira/browse/SPARK-30022 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 3.1.0 > Reporter: Arun Ravi M V > Priority: Major > > > We have an environment were we use Apache Spark and Presto (both backed by > Apache Hive Metastore). Currently, Views created from Presto fail to get > parsed in Apache Spark. This is because Presto stores the View definition and > View schema in a base64 encoded fashion and Spark is unable to process it. I > would like to propose a minor change that will allow us to read these encoded > definitions created by Presto in a Spark Program. > Assuming that the UDFs are made available, the user should be able to read > presto views after the fix. > > I would like to propose a change to > [https://github.com/apache/spark/blob/9459833eae7fae887af560f3127997e023c51d00/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala#L440] > to have support creation of CatlogTable for Views created from Presto. > > Hive Metastore DB, the table definition for Presto views (*select* * *from* > TBLS *where* `TBL_TYPE` *like* '%VIRTUAL_VIEW%') shows that the > VIEW_EXPANDED_TEXT is hardcoded as `/* Presto View **/` and > VIEW_ORIGINAL_TEXT is `/** Presto View: base64({ "originalSql": "" "catalog": > "", "schema": "", "columns": [ > { "name": "", "type": "" } > ], "owner": ""}) */` > Refer: > [https://github.com/prestodb/presto/blob/3242715959a169dbcdd88946c28488d2365c8886/presto-hive/src/main/java/com/facebook/presto/hive/HiveUtil.java#L614] > -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org