[
https://issues.apache.org/jira/browse/HIVE-24217?focusedWorklogId=501083&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-501083
]
ASF GitHub Bot logged work on HIVE-24217:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 15/Oct/20 12:28
Start Date: 15/Oct/20 12:28
Worklog Time Spent: 10m
Work Description: zeroflag commented on a change in pull request #1542:
URL: https://github.com/apache/hive/pull/1542#discussion_r505499995
##########
File path: standalone-metastore/metastore-server/src/main/resources/package.jdo
##########
@@ -1549,6 +1549,83 @@
<column name="RM_DUMP_EXECUTION_ID"/>
</index>
</class>
+
+ <class name="MStoredProc" table="STORED_PROCS" identity-type="datastore"
detachable="true">
+ <datastore-identity>
+ <column name="SP_ID"/>
+ </datastore-identity>
+ <field name="createTime">
+ <column name="CREATE_TIME" jdbc-type="integer" allows-null="false"/>
+ </field>
+ <field name="lastAccessTime">
+ <column name="LAST_ACCESS_TIME" jdbc-type="integer"
allows-null="false"/>
+ </field>
+ <field name="database">
+ <column name="DB_ID" allows-null="false"/>
+ </field>
+ <field name="name">
+ <column name="NAME" length="256" jdbc-type="VARCHAR"
allows-null="false"/>
+ </field>
+ <field name="owner">
+ <column name="OWNER_NAME" length="128" jdbc-type="VARCHAR"
allows-null="false"/>
+ </field>
+ <field name="source">
+ <column name="SOURCE" jdbc-type="MEDIUMTEXT" allows-null="false"/>
+ </field>
+ <field name="language">
+ <column name="LANG" jdbc-type="varchar" length="128"
allows-null="false"/>
+ </field>
+ <field name="returnType">
+ <column name="RET_TYPE" jdbc-type="varchar" length="128"
allows-null="true"/>
+ </field>
+ <field name="parameters" table="SP_POS_ARGS" >
+ <collection element-type="MPosParam"/>
+ <join>
+ <column name="SP_ID"/>
+ </join>
+ <element>
+ <embedded>
+ <field name="name">
+ <column name="NAME" jdbc-type="varchar" length="256"
allows-null="false"/>
+ </field>
+ <field name="type">
+ <column name="TYPE" jdbc-type="varchar" length="128"
allows-null="false"/>
+ </field>
+ <field name="length">
+ <column name="length" jdbc-type="integer" allows-null="true"/>
+ </field>
+ <field name="scale">
+ <column name="scale" jdbc-type="integer" allows-null="true"/>
+ </field>
+ <field name="isOut">
+ <column name="OUT" allows-null="false"/>
+ </field>
+ </embedded>
+ </element>
+ </field>
Review comment:
Hey @jcamachor thanks for the feedback, I'm glad you chimed in. Maybe
there is a misunderstanding, this information is not redundant in any way,
we're making use of it when invoking a procedure. This information must be
stored somewhere in some form the only question is what representation to use.
Just to clarify, currently I store the signature in a structured way +
procedure body (without the signature) in text.
What was proposed (but @kgyrtkirk correct me if I'm wrong) as an alternative
solution is store the signature together with the body and optionally add a
signature string to the table as well (this would be redundant). I think
regardless of the representation, backward compatibility will always be a
concern.
One thing I don't like about storing the signature in text is that the
assumption that invoking the procedure will always require parsing. This is
only true for now because HPL/SQL is an AST interpreter but if we ever want to
make this performant probably we'll need to introduce a byte code VM at some
point in the near future. While creating a procedure needs parsing but invoking
it wouldn't, if we stored the byte code. How would this work in this case? I
suppose we can add runtime information into the byte code, but that's not
always an option. For example postgres allows you to invoke procedures
implemented in C where runtime information about the signature is not
available. This might be one reason why they also choose to store the signature
separately in a structured way.
Multi language support was already raised by customers and adding it would
be the easiest if we had common bytecode for different languages. One might
want to call a procedure implemented in language A from a different language B.
Then A would need to use the parser of language B to get the signature
information, if the signature was stored in text.
We can keep speculating on this, but at this point this is still an
experimental and undocumented feature, I'm open to change it later if we have
proof that one way is better than the other.
But if we decide to go with the alternative solution from now on, I suggest
we choose a language agnostic representation (JSON or whatever) of the
signature, instead of the unparsed text.
> Also, checking the documentation, it seems HPL/SQL can apply some
transformations to the field type. Are those transformations applied before
storing the definition or later on?
Yes, and that only affects create table statements.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 501083)
Time Spent: 2h (was: 1h 50m)
> HMS storage backend for HPL/SQL stored procedures
> -------------------------------------------------
>
> Key: HIVE-24217
> URL: https://issues.apache.org/jira/browse/HIVE-24217
> Project: Hive
> Issue Type: Bug
> Components: Hive, hpl/sql, Metastore
> Reporter: Attila Magyar
> Assignee: Attila Magyar
> Priority: Major
> Labels: pull-request-available
> Attachments: HPL_SQL storedproc HMS storage.pdf
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> HPL/SQL procedures are currently stored in text files. The goal of this Jira
> is to implement a Metastore backend for storing and loading these procedures.
> This is an incremental step towards having fully capable stored procedures in
> Hive.
>
> See the attached design for more information.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
