[
https://issues.apache.org/jira/browse/TRAFODION-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999105#comment-14999105
]
Roberta Marton commented on TRAFODION-1578:
-------------------------------------------
I am just coming up to speed on this proposal. Part of this is trying to
understand how things will work. Let's start with the new "upload" command.
Let me know if I understand things correctly.
It looks like you are proposing a new system SPJ that will upload JAR files to
be accessible by Trafodion for procedures, functions, and table-mapping
functions and maybe more things in the future. To do this, you will be adding
a new system SPJ that will accept a JAR file, an associated .md5 file, the type
of JAR, and perhaps the CREATE text. The upload command will check the JAR
file against the .md5 file, it will create a library specifying some predefined
location, create the function/procedure/table-mapping functions so it is part
of the Trafodion metadata. Then it will store the JAR as a LOB and/or copy it
out to some predefined location on each node in the cluster.
General comments and questions on the DDL aspect:
- This requires us to create a new system library and SPJ. Today, we don't
have any system SPJs. So there will be upgrade considerations to create a
schema such as "_SPJ_", create the library, and load the SPJ. Have you
considered how we will be adding this infrastructure?
- We can, at upgrade/install time, create the upload SPJ. As mentioned before,
we probably want to create a system schema along with this. I assume that
DB__ROOT will own the schema which, in effect, makes DB__ROOT owner of the
library and system SPJ and that initially only DB__ROOT can execute the upload
SPJ. We can use existing grant commands to give all the required permission to
other roles/users. Do we need to add any additional privileges to these
commands? Should we allow anyone granted DB__ROOTROLE permission run this
command also?
- Since the upload command performs SQL statements, we need to be concerned
about who can execute it. Today we only support invoker rights so anyone who
can execute the command has to be trusted. We should probably consider definer
rights; that is allow people to run the upload command only and not grant them
the world.
- When you upload a jar, you need to add it to SQL metadata through a CREATE
FUNCTION or similar command. Will each new JAR belong to a different library
or will a set of JAR's belong to the same library? If you combine JAR together
in a single library, how will this work?
- while we are designing this, we should remember that we claim to be SQL ANSI
compliant. So we do need to make sure we follow their guidelines. Is anything
you are proposing differ much from ANSI?
- In your proposal "add a record in metadata table in case users upload the
same JAR many times", it looks like you want to keep a record on how many times
a JAR is uploaded. Will this information be stored in a repository table or in
the system metadata? Again there are upgrade considerations. I like the idea
of keeping track of usages. We might want to add some rules that specify how
often and frequently a JAR is uploaded to perhaps detect security issues. Of
course, we want to associate the real user with the upload operation.
- will you be able to upload JAR through both sqlci and trafci?
This is an interesting proposal and am looking forward to helping out with it.
There is a lot to be considered so we want to make sure we cover everything.
Will you be writing up a design specification for this? It would be helpful
since there are a lot of conversations and a single document codify them in one
place.
I will comment later on the overall security of things once I understand things
better.
> Proposal for SPJ management
> ---------------------------
>
> Key: TRAFODION-1578
> URL: https://issues.apache.org/jira/browse/TRAFODION-1578
> Project: Apache Trafodion
> Issue Type: Improvement
> Components: connectivity-dcs
> Reporter: Kevin Xu
>
> JAR upload process:
> 1. Initialize JAR upload procedure by default
> 2. JAR upload by Trafci(add library LIB_NAME JAR_LOCAL_PATH). Upload and
> create library will be done here. And also, you can only upload the JARs by
> UPLOAD command on Trafci that it will not create a lib.
> Tips: Before put the JAR into HDFS check MD5 first, if the file exists,
> only add a record in metadata table in case users upload the same JAR many
> times on platform.
> 3. On server-side, the JAR will store in HDFS. At the same time JAR
> metadata(path in HDFS, MD5 of the file, and others) stores in store procedure
> metadata table.
> 4. create procedure is the same as now.
> JAR perform process:
> 1. Send a CALL by Trafci/JDBC/ODBC/ADO.NET.
> 2. DCSMaster assign a DCSServer for the CALL.
> 3. DCSServer start a JVM for the user. User can modify JVM options, program
> properties and JAVA classpath. At the same time, a monitor class will be
> starting in the JVM witch will register a node on Zookeeper for this JVM as
> well as metadata info( process id, server info and so on) and the node will
> be removed while JVM exiting. It allows customer to specify JVM idle time in
> case of some realtime senarior like Kafka consumer.
> 4. Useful commands on Trafci: list all JVMs in user; kill one of them that no
> long in use; Restart JVMs with latest JARs and so on.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
