[
https://issues.apache.org/jira/browse/HIVE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835112#action_12835112
]
Edward Capriolo commented on HIVE-1096:
---------------------------------------
>>
Using the HiveConf namespace for user variables seems like a bad idea,
especially since there are no checks in place to prevent people from defining
variables with names like "hive.foo.bar" Giving users unprotected access to
your configuration namespace usually leads to problems down the road.
Philosophically I agree. In actuality have Hive/Hadoop conf is easily
manipulated by changing your hadoop-site.xml or hive-site.xml. Users do have
unprotected access to the namespace that is the nature of hadoop. Users of hive
are setting variables all the time.
>>
It would be nice to be able to reference Java system properties using this
syntax.
Hive /Hadoop do not often refer to system properties during normal operation. I
am on the fence about this.
>>Driver.replace() iterates over the list of defined variables. Instead, I
>>think it should iterate over the tokens in the command that match the pattern
>>'${.*}'.
The hive CLI really needs some type of top level parser. Because we don't have
this there are many ways we could do certain things but all of them are a
little 'hackish'. If we had a real parser, reading character by character, we
would not need a regex or string replace to do variable processing.
>>
This would make it easy to log any cases where the command contains
"${foo.bar}" and foo.bar is undefined.
I think the code should replace "" for an undefined variable.
>>Driver.replace(String) should have a name like
>>Driver.interpolateCommandVariables(), or Driver.replaceVariables().
Agreed, my notorious spelling usually causes me to avoid words like
'interpolate' :)
>>It would be nice to be able to prevent interpolation using an escape
>>character, e.g. "\${somevar}".
I punted on that by being able to turn the replacement on and off. As I said
above true parser would do stuff like this.
>>It would be nice to be able to nest variable definitions, e.g.
>>version="0.6.0", jar_name="hive-exec-${version}". The variable interpolation
>>code in Hadoop's Configuration class does this.
Yes. Carl I started with your code, the bug I mentioned above (comments before
set), caused me to rip the entire thing apart, before I found out what I was
doing wrong I had ripped out all the regex. I am not very comfortable with
regex, and I was not in love with the while construct with a return in the
middle. Again nesting would be easy with a true parser. I do not mind going
back to your code.
>>The last thing we want is two completely separate variable interpolation
>>mechanisms.
Agreed. The only true difference in implementation is that your doing it with
properties and I am doing it with HiveConf Vars. If we support both I think we
are both happy. Any ideas?
> Hive Variables
> --------------
>
> Key: HIVE-1096
> URL: https://issues.apache.org/jira/browse/HIVE-1096
> Project: Hadoop Hive
> Issue Type: New Feature
> Reporter: Edward Capriolo
> Assignee: Edward Capriolo
> Attachments: 1096-9.diff, hive-1096-2.diff, hive-1096-7.diff,
> hive-1096-8.diff, hive-1096.diff
>
>
> From mailing list:
> --Amazon Elastic MapReduce version of Hive seems to have a nice feature
> called "Variables." Basically you can define a variable via command-line
> while invoking hive with -d DT=2009-12-09 and then refer to the variable via
> ${DT} within the hive queries. This could be extremely useful. I can't seem
> to find this feature even on trunk. Is this feature currently anywhere in the
> roadmap?--
> This could be implemented in many places.
> A simple place to put this is
> in Driver.compile or Driver.run we can do string substitutions at that level,
> and further downstream need not be effected.
> There could be some benefits to doing this further downstream, parser,plan.
> but based on the simple needs we may not need to overthink this.
> I will get started on implementing in compile unless someone wants to discuss
> this more.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
