[jira] Updated: (HIVE-1096) Hive Variables
[
https://issues.apache.org/jira/browse/HIVE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Namit Jain updated HIVE-1096:
-
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed. Thanks Edward
> Hive Variables
> --
>
> Key: HIVE-1096
> URL: https://issues.apache.org/jira/browse/HIVE-1096
> Project: Hive
> Issue Type: New Feature
> Components: Query Processor
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.7.0
>
> Attachments: 1096-9.diff, hive-1096-10-patch.txt,
> hive-1096-11-patch.txt, hive-1096-12.patch.txt, hive-1096-15.patch.txt,
> hive-1096-15.patch.txt, hive-1096-2.diff, hive-1096-20.patch.txt,
> hive-1096-7.diff, hive-1096-8.diff, hive-1096.diff
>
>
> From mailing list:
> --Amazon Elastic MapReduce version of Hive seems to have a nice feature
> called "Variables." Basically you can define a variable via command-line
> while invoking hive with -d DT=2009-12-09 and then refer to the variable via
> ${DT} within the hive queries. This could be extremely useful. I can't seem
> to find this feature even on trunk. Is this feature currently anywhere in the
> roadmap?--
> This could be implemented in many places.
> A simple place to put this is
> in Driver.compile or Driver.run we can do string substitutions at that level,
> and further downstream need not be effected.
> There could be some benefits to doing this further downstream, parser,plan.
> but based on the simple needs we may not need to overthink this.
> I will get started on implementing in compile unless someone wants to discuss
> this more.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1096) Hive Variables
[
https://issues.apache.org/jira/browse/HIVE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-1096:
--
Attachment: hive-1096-20.patch.txt
Were two 15 patches. I bumped the number to 20 to clear any confusion.
> Hive Variables
> --
>
> Key: HIVE-1096
> URL: https://issues.apache.org/jira/browse/HIVE-1096
> Project: Hive
> Issue Type: New Feature
> Components: Query Processor
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.7.0
>
> Attachments: 1096-9.diff, hive-1096-10-patch.txt,
> hive-1096-11-patch.txt, hive-1096-12.patch.txt, hive-1096-15.patch.txt,
> hive-1096-15.patch.txt, hive-1096-2.diff, hive-1096-20.patch.txt,
> hive-1096-7.diff, hive-1096-8.diff, hive-1096.diff
>
>
> From mailing list:
> --Amazon Elastic MapReduce version of Hive seems to have a nice feature
> called "Variables." Basically you can define a variable via command-line
> while invoking hive with -d DT=2009-12-09 and then refer to the variable via
> ${DT} within the hive queries. This could be extremely useful. I can't seem
> to find this feature even on trunk. Is this feature currently anywhere in the
> roadmap?--
> This could be implemented in many places.
> A simple place to put this is
> in Driver.compile or Driver.run we can do string substitutions at that level,
> and further downstream need not be effected.
> There could be some benefits to doing this further downstream, parser,plan.
> but based on the simple needs we may not need to overthink this.
> I will get started on implementing in compile unless someone wants to discuss
> this more.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1096) Hive Variables
[
https://issues.apache.org/jira/browse/HIVE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-1096:
--
Attachment: hive-1096-15.patch.txt
Adds xdocs.
> Hive Variables
> --
>
> Key: HIVE-1096
> URL: https://issues.apache.org/jira/browse/HIVE-1096
> Project: Hive
> Issue Type: New Feature
> Components: Query Processor
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.7.0
>
> Attachments: 1096-9.diff, hive-1096-10-patch.txt,
> hive-1096-11-patch.txt, hive-1096-12.patch.txt, hive-1096-15.patch.txt,
> hive-1096-15.patch.txt, hive-1096-2.diff, hive-1096-7.diff, hive-1096-8.diff,
> hive-1096.diff
>
>
> From mailing list:
> --Amazon Elastic MapReduce version of Hive seems to have a nice feature
> called "Variables." Basically you can define a variable via command-line
> while invoking hive with -d DT=2009-12-09 and then refer to the variable via
> ${DT} within the hive queries. This could be extremely useful. I can't seem
> to find this feature even on trunk. Is this feature currently anywhere in the
> roadmap?--
> This could be implemented in many places.
> A simple place to put this is
> in Driver.compile or Driver.run we can do string substitutions at that level,
> and further downstream need not be effected.
> There could be some benefits to doing this further downstream, parser,plan.
> but based on the simple needs we may not need to overthink this.
> I will get started on implementing in compile unless someone wants to discuss
> this more.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1096) Hive Variables
[
https://issues.apache.org/jira/browse/HIVE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-1096:
--
Status: Patch Available (was: Open)
* trunk/conf/hive-default.xml:
Spelling: substituation
Fixed
*
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/VariableSubstitution.java:
Make these variables private?
Private variables are what got us into the mess with hadoop. I am not going to
repeat the problem.
*
trunk/ql/src/java/org/apache/hadoop/hive/ql/processors/AddResourceProcessor.java:
Since we want to do substitution for all commands it would probably make
sense to do the substitution in CommandProcessorFactory.get() and make
CommandProcessor an abstract class with the following implementation:
...
In other words, CommandProcessorFactory would return a CommandProcessor object
that has been initialized with a substituted copy of the command.
No. No more re factoring. It is working the way it is. Using factories going to
be major. I'm tired. It does not prove anything since this entire process is
not very clever anyway. Currently it is slightly baked, but I believe that
better then being over designed.
* trunk/ql/src/java/org/apache/hadoop/hive/ql/processors/SetProcessor.java:
Replace these string literals with constants, e.g:
public static final String ENV_PREFIX = "env:";
public static final String SYSTEM_PREFIX = "system:"
public static final String HIVECONF_PREFIX = "hiveconf:"
Fixed
* trunk/ql/src/java/org/apache/hadoop/hive/ql/processors/SetProcessor.java:
String propName = varname.substring(SYSTEM_PREFIX.length());
Fixed
* trunk/ql/src/java/org/apache/hadoop/hive/ql/processors/SetProcessor.java:
Can we remove this special case for "silent"? In SessionState this
actually maps to "hive.session.silent" and I don't see any test cases that
cover this case, i.e. that call "set silent" or "set silent=x". It also seems
that this introduces in inconsistency since "set silent" will show the value of
"hive.session.silent", but the output of "set" will not list a value for the
property "silent".
Anyone know if there is any older code that depends on this behavior?
Do not really know. do not really care :) Out of scope. It is there I am
leaving it.
As for the VAR. Turns out supporting this is not very easy. Adding Options
Parsing to the CLI works, however the session state gives you no where to store
"variables" except in the hive conf. SetProcessor works with SessionState not
CLI SessionState. Again big re factoring is needed.
What I did do is move remove support for set y=${x}. This patch only adds set
y=${hiveconf:x}. Thus if someone cares to add "VAR X" or ${x} or determine how
to change the CLI to add this other map that can be shared across the session
state this patch is not in the way.
Thus substitution only works for ${hiveconf:x} ${system:x} and ${env:x}.
implementing ${x} and var can be done in a separate issue.
> Hive Variables
> --
>
> Key: HIVE-1096
> URL: https://issues.apache.org/jira/browse/HIVE-1096
> Project: Hive
> Issue Type: New Feature
> Components: Query Processor
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.7.0
>
> Attachments: 1096-9.diff, hive-1096-10-patch.txt,
> hive-1096-11-patch.txt, hive-1096-12.patch.txt, hive-1096-15.patch.txt,
> hive-1096-2.diff, hive-1096-7.diff, hive-1096-8.diff, hive-1096.diff
>
>
> From mailing list:
> --Amazon Elastic MapReduce version of Hive seems to have a nice feature
> called "Variables." Basically you can define a variable via command-line
> while invoking hive with -d DT=2009-12-09 and then refer to the variable via
> ${DT} within the hive queries. This could be extremely useful. I can't seem
> to find this feature even on trunk. Is this feature currently anywhere in the
> roadmap?--
> This could be implemented in many places.
> A simple place to put this is
> in Driver.compile or Driver.run we can do string substitutions at that level,
> and further downstream need not be effected.
> There could be some benefits to doing this further downstream, parser,plan.
> but based on the simple needs we may not need to overthink this.
> I will get started on implementing in compile unless someone wants to discuss
> this more.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1096) Hive Variables
[
https://issues.apache.org/jira/browse/HIVE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-1096:
--
Attachment: hive-1096-15.patch.txt
> Hive Variables
> --
>
> Key: HIVE-1096
> URL: https://issues.apache.org/jira/browse/HIVE-1096
> Project: Hive
> Issue Type: New Feature
> Components: Query Processor
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.7.0
>
> Attachments: 1096-9.diff, hive-1096-10-patch.txt,
> hive-1096-11-patch.txt, hive-1096-12.patch.txt, hive-1096-15.patch.txt,
> hive-1096-2.diff, hive-1096-7.diff, hive-1096-8.diff, hive-1096.diff
>
>
> From mailing list:
> --Amazon Elastic MapReduce version of Hive seems to have a nice feature
> called "Variables." Basically you can define a variable via command-line
> while invoking hive with -d DT=2009-12-09 and then refer to the variable via
> ${DT} within the hive queries. This could be extremely useful. I can't seem
> to find this feature even on trunk. Is this feature currently anywhere in the
> roadmap?--
> This could be implemented in many places.
> A simple place to put this is
> in Driver.compile or Driver.run we can do string substitutions at that level,
> and further downstream need not be effected.
> There could be some benefits to doing this further downstream, parser,plan.
> but based on the simple needs we may not need to overthink this.
> I will get started on implementing in compile unless someone wants to discuss
> this more.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
