[jira] Commented: (HIVE-1611) Add alternative search-provider to Hive site
[ https://issues.apache.org/jira/browse/HIVE-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917016#action_12917016 ] Edward Capriolo commented on HIVE-1611: --- Now that hive is TLP we likely have to get the ball rolling and cut the cord with hadoop. I will contact infra and see what our options are. We have a few issues. -we need to move the SVN from a hadoop subproject to a toplevel svn. -after we do that we need to take the forest docs and move them into hive then can change the search box If we want to see the skinconf change done first, we should open/transfer this ticket to core I believe. > Add alternative search-provider to Hive site > > > Key: HIVE-1611 > URL: https://issues.apache.org/jira/browse/HIVE-1611 > Project: Hadoop Hive > Issue Type: Improvement >Reporter: Alex Baranau >Assignee: Alex Baranau >Priority: Minor > Attachments: HIVE-1611.patch > > > Use search-hadoop.com service to make available search in Hive sources, MLs, > wiki, etc. > This was initially proposed on user mailing list. The search service was > already added in site's skin (common for all Hadoop related projects) before > so this issue is about enabling it for Hive. The ultimate goal is to use it > at all Hadoop's sub-projects' sites. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1668) Move HWI out to Github
[
https://issues.apache.org/jira/browse/HIVE-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914741#action_12914741
]
Edward Capriolo commented on HIVE-1668:
---
{quote}It should also help mature the product for eventual inclusion in
trunk.{quote}
Why would we move something from hive out to github, just to move it back to
hive?
{quote}Empirically, they don't. The value of the web interface to users is not
nearly as high as the pain it causes the developers for maintenance.{quote}
Who are these developers who maintain it? Has anyone every added a feature
beside me? I'm not complaining.
http://blog.milford.io/2010/06/getting-the-hive-web-interface-hwi-to-work-on-centos/
{quote}The Hive Web Interface is a pretty sweet deal.{quote}
Sounds like people like it.
Why are we debating the past state of hwi? It works now. If someone reports a
bug I typically investigate and patch that same day.
I challenge anyone to open a ticket on core user, called "remove name node web
interface to github" and tried to say " now offers a better name node
interface using python." The ticket would instantly get a "RESOLVED: WILL NOT
FIX". Why is this any different?
> Move HWI out to Github
> --
>
> Key: HIVE-1668
> URL: https://issues.apache.org/jira/browse/HIVE-1668
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Web UI
>Reporter: Jeff Hammerbacher
>
> I have seen HWI cause a number of build and test errors, and it's now going
> to cost us some extra work for integration with security. We've worked on
> hundreds of clusters at Cloudera and I've never seen anyone use HWI. With the
> Beeswax UI available in Hue, it's unlikely that anyone would prefer to stick
> with HWI. I think it's time to move it out to Github.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1668) Move HWI out to Github
[
https://issues.apache.org/jira/browse/HIVE-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914669#action_12914669
]
Edward Capriolo commented on HIVE-1668:
---
{quote}That's not a great argument for keeping code that's onerous to maintain
in trunk.{quote}
Its not onerous to maintain. As you can see from the tickets I pointed out it
broke because it was not tested.
For example,
https://issues.apache.org/jira/browse/HIVE-752 when designing SHIM classes that
specify a classname in a string, one has to make sure they get the class name
correct. I know it was an over site, but I am sure someone fired up the CLI and
made sure the class name was correct.
As for https://issues.apache.org/jira/browse/HIVE-978, I specifically mentioned
how to test this any why it should be tested in the patch and it still turned
out not to work right.
pragmatic is the perfect word. HWI was never made to be fancy. Anyone who has
hive can build and run the web interface. With no extra dependencies. It looks
like to use Beeswax you need Hue, which means you need to go somewhere else and
get it and install it. It seems like you need to patch or load extra plugins to
your namenode and datanode like org.apache.hadoop.thriftfs.NamenodePlugin, It
looks like (http://archive.cloudera.com/cdh/3/hue/manual.html#_install_hue) you
need:
gcc gcc
libxml2-devel libxml2-dev
libxslt-devel libxslt-dev
mysql-devel librarysqlclient-dev
python-develpython-dev
python-setuptools python-setuptools
sqlite-devellibsqlite3-dev
The pragmatic approach, is to use the web interface provided by hive. You do
not need anything external like python, or have to make any changes to their
environment. That is why I think we should stay part of the hive distribution.
I'm -1 on taking it out.
> Move HWI out to Github
> --
>
> Key: HIVE-1668
> URL: https://issues.apache.org/jira/browse/HIVE-1668
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Web UI
>Reporter: Jeff Hammerbacher
>
> I have seen HWI cause a number of build and test errors, and it's now going
> to cost us some extra work for integration with security. We've worked on
> hundreds of clusters at Cloudera and I've never seen anyone use HWI. With the
> Beeswax UI available in Hue, it's unlikely that anyone would prefer to stick
> with HWI. I think it's time to move it out to Github.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1668) Move HWI out to Github
[ https://issues.apache.org/jira/browse/HIVE-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914605#action_12914605 ] Edward Capriolo commented on HIVE-1668: --- Plus, not to get too far off topic, but there is a huge portion of the hadoop community that thinks "Security? So what? Who cares?" I am not going to run Active Directory or Kerberos just so I can say "My hadoop is is secure" . It adds latency to many processes, complexity to the overall design of hadoop, and does not even encrypt data in transit. Many people are going to elect not to use "hadoop security" for those reasons. Is "extra work" a reason not to do something? Are we going to move the Hive Thrift server out to github too because of the burden of "extra work"? It is a lot of extra work for me when hadoop renames all its jmx counters or tells me "all my code is deprecated because of our new slick mapreduce.* api". I have learned to roll with the punches. > Move HWI out to Github > -- > > Key: HIVE-1668 > URL: https://issues.apache.org/jira/browse/HIVE-1668 > Project: Hadoop Hive > Issue Type: Improvement > Components: Web UI >Reporter: Jeff Hammerbacher > > I have seen HWI cause a number of build and test errors, and it's now going > to cost us some extra work for integration with security. We've worked on > hundreds of clusters at Cloudera and I've never seen anyone use HWI. With the > Beeswax UI available in Hue, it's unlikely that anyone would prefer to stick > with HWI. I think it's time to move it out to Github. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1668) Move HWI out to Github
[
https://issues.apache.org/jira/browse/HIVE-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914584#action_12914584
]
Edward Capriolo commented on HIVE-1668:
---
Jeff,
I disagree. The build and test errors are not insurmountable. In fact some if
not most of the "ERRORS" were cascading changes that were not tested properly.
For example:
https://issues.apache.org/jira/browse/HIVE-1183 was a fix I had to do because
someone broke it here. https://issues.apache.org/jira/browse/HIVE-978 because
someone wanted all jars to be named whatever.${version} and did not bother to
look across all the shell script files that startup hive.
https://issues.apache.org/jira/browse/HIVE-1294 again someone changed some
shell scripts and only tested the cli.
https://issues.apache.org/jira/browse/HIVE-752 again someone broke hwi without
testing it.
https://issues.apache.org/jira/browse/HIVE-1615, not really anyone's fault but
no API stability across hive. I do not see why one method went away and another
similar method took its place.
I have been of course talking about moving HWI to wikit for a while moving from
JSP to Servlet/ Java code will fix errors, but the little time I do have I
usually have to spend detecting and cleaning up other breakages.
HUE and Beeswax I honestly do not know, but sounds like you need extra magical
stuff to make this work, and HWI works with hive on its own (onless people
break it)
> Move HWI out to Github
> --
>
> Key: HIVE-1668
> URL: https://issues.apache.org/jira/browse/HIVE-1668
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Web UI
>Reporter: Jeff Hammerbacher
>
> I have seen HWI cause a number of build and test errors, and it's now going
> to cost us some extra work for integration with security. We've worked on
> hundreds of clusters at Cloudera and I've never seen anyone use HWI. With the
> Beeswax UI available in Hue, it's unlikely that anyone would prefer to stick
> with HWI. I think it's time to move it out to Github.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-842) Authentication Infrastructure for Hive
[ https://issues.apache.org/jira/browse/HIVE-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12913741#action_12913741 ] Edward Capriolo commented on HIVE-842: -- By attack the Web UI separately what is meant? Will it be broken or non-functional at any phase here? That is what I find happens often, some of it is really the WUI's fault for using JSP and not servlets, but there is no simple way to code cover the wui and all the different ways its gets broken. > Authentication Infrastructure for Hive > -- > > Key: HIVE-842 > URL: https://issues.apache.org/jira/browse/HIVE-842 > Project: Hadoop Hive > Issue Type: New Feature > Components: Server Infrastructure >Reporter: Edward Capriolo >Assignee: Todd Lipcon > Attachments: HiveSecurityThoughts.pdf > > > This issue deals with the authentication (user name,password) infrastructure. > Not the authorization components that specify what a user should be able to > do. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-268) "Insert Overwrite Directory" to accept configurable table row format
[
https://issues.apache.org/jira/browse/HIVE-268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12910656#action_12910656
]
Edward Capriolo commented on HIVE-268:
--
Still not exactly what you want but with CTAS you can essentially get a folder
in /user/hive/warehouse/ with the format you want.
> "Insert Overwrite Directory" to accept configurable table row format
>
>
> Key: HIVE-268
> URL: https://issues.apache.org/jira/browse/HIVE-268
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
>Reporter: Zheng Shao
>Assignee: Paul Yang
>
> There is no way for the users to control the file format when they are
> outputting the result into a directory.
> We should allow:
> {code}
> INSERT OVERWRITE DIRECTORY "/user/zshao/result"
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY '9'
> SELECT tablea.* from tablea;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1615) Web Interface JSP needs Refactoring for removed meta store methods
[ https://issues.apache.org/jira/browse/HIVE-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1615: -- Fix Version/s: 0.7.0 (was: 0.6.0) Affects Version/s: 0.6.0 (was: 0.5.1) > Web Interface JSP needs Refactoring for removed meta store methods > -- > > Key: HIVE-1615 > URL: https://issues.apache.org/jira/browse/HIVE-1615 > Project: Hadoop Hive > Issue Type: Bug > Components: Web UI >Affects Versions: 0.6.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo >Priority: Blocker > Fix For: 0.7.0 > > Attachments: hive-1615.patch.2.txt, hive-1615.patch.txt > > > Some meta store methods being called from JSP have been removed. Really > should prioritize compiling jsp into servlet code again. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1613) hive --service jar looks for hadoop version but was not defined
[ https://issues.apache.org/jira/browse/HIVE-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1613: -- Fix Version/s: 0.6.0 (was: 0.7.0) Affects Version/s: 0.5.1 (was: 0.6.0) Priority: Blocker (was: Major) I think we should patch this as well functionality was broken. > hive --service jar looks for hadoop version but was not defined > --- > > Key: HIVE-1613 > URL: https://issues.apache.org/jira/browse/HIVE-1613 > Project: Hadoop Hive > Issue Type: Bug > Components: Clients >Affects Versions: 0.5.1 >Reporter: Edward Capriolo >Assignee: Edward Capriolo >Priority: Blocker > Fix For: 0.6.0 > > Attachments: hive-1613.patch.txt > > > hive --service jar fails. I have to open another ticket to clean up the > scripts and unify functions like version detection. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1615) Web Interface JSP needs Refactoring for removed meta store methods
[ https://issues.apache.org/jira/browse/HIVE-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1615: -- Fix Version/s: 0.6.0 (was: 0.7.0) Affects Version/s: 0.5.1 (was: 0.7.0) Priority: Blocker (was: Major) Moving this up the meta store changes recently broke the web interface > Web Interface JSP needs Refactoring for removed meta store methods > -- > > Key: HIVE-1615 > URL: https://issues.apache.org/jira/browse/HIVE-1615 > Project: Hadoop Hive > Issue Type: Bug > Components: Web UI >Affects Versions: 0.5.1 >Reporter: Edward Capriolo >Assignee: Edward Capriolo >Priority: Blocker > Fix For: 0.6.0 > > Attachments: hive-1615.patch.2.txt, hive-1615.patch.txt > > > Some meta store methods being called from JSP have been removed. Really > should prioritize compiling jsp into servlet code again. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1615) Web Interface JSP needs Refactoring for removed meta store methods
[ https://issues.apache.org/jira/browse/HIVE-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1615: -- Attachment: hive-1615.patch.2.txt > Web Interface JSP needs Refactoring for removed meta store methods > -- > > Key: HIVE-1615 > URL: https://issues.apache.org/jira/browse/HIVE-1615 > Project: Hadoop Hive > Issue Type: Bug > Components: Web UI >Affects Versions: 0.7.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.7.0 > > Attachments: hive-1615.patch.2.txt, hive-1615.patch.txt > > > Some meta store methods being called from JSP have been removed. Really > should prioritize compiling jsp into servlet code again. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1615) Web Interface JSP needs Refactoring for removed meta store methods
[ https://issues.apache.org/jira/browse/HIVE-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1615: -- Status: Patch Available (was: Open) Affects Version/s: 0.7.0 Fix Version/s: 0.7.0 > Web Interface JSP needs Refactoring for removed meta store methods > -- > > Key: HIVE-1615 > URL: https://issues.apache.org/jira/browse/HIVE-1615 > Project: Hadoop Hive > Issue Type: Bug > Components: Web UI >Affects Versions: 0.7.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.7.0 > > Attachments: hive-1615.patch.txt > > > Some meta store methods being called from JSP have been removed. Really > should prioritize compiling jsp into servlet code again. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1615) Web Interface JSP needs Refactoring for removed meta store methods
[ https://issues.apache.org/jira/browse/HIVE-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1615: -- Summary: Web Interface JSP needs Refactoring for removed meta store methods (was: Web Interface JSP needs Refactoring for deprecated meta store methods) > Web Interface JSP needs Refactoring for removed meta store methods > -- > > Key: HIVE-1615 > URL: https://issues.apache.org/jira/browse/HIVE-1615 > Project: Hadoop Hive > Issue Type: Bug > Components: Web UI >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Attachments: hive-1615.patch.txt > > > Some meta store methods being called from JSP have been removed. Really > should prioritize compiling jsp into servlet code again. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1615) Web Interface JSP needs Refactoring for deprecated meta store methods
[ https://issues.apache.org/jira/browse/HIVE-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1615: -- Attachment: hive-1615.patch.txt > Web Interface JSP needs Refactoring for deprecated meta store methods > - > > Key: HIVE-1615 > URL: https://issues.apache.org/jira/browse/HIVE-1615 > Project: Hadoop Hive > Issue Type: Bug > Components: Web UI >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Attachments: hive-1615.patch.txt > > > Some meta store methods being called from JSP have been removed. Really > should prioritize compiling jsp into servlet code again. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1615) Web Interface JSP needs Refactoring for deprecated meta store methods
Web Interface JSP needs Refactoring for deprecated meta store methods - Key: HIVE-1615 URL: https://issues.apache.org/jira/browse/HIVE-1615 Project: Hadoop Hive Issue Type: Bug Components: Web UI Reporter: Edward Capriolo Assignee: Edward Capriolo Some meta store methods being called from JSP have been removed. Really should prioritize compiling jsp into servlet code again. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1613) hive --service jar looks for hadoop version but was not defined
[ https://issues.apache.org/jira/browse/HIVE-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1613: -- Status: Patch Available (was: Open) > hive --service jar looks for hadoop version but was not defined > --- > > Key: HIVE-1613 > URL: https://issues.apache.org/jira/browse/HIVE-1613 > Project: Hadoop Hive > Issue Type: Bug > Components: Clients >Affects Versions: 0.6.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.7.0 > > Attachments: hive-1613.patch.txt > > > hive --service jar fails. I have to open another ticket to clean up the > scripts and unify functions like version detection. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1613) hive --service jar looks for hadoop version but was not defined
[ https://issues.apache.org/jira/browse/HIVE-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1613: -- Attachment: hive-1613.patch.txt > hive --service jar looks for hadoop version but was not defined > --- > > Key: HIVE-1613 > URL: https://issues.apache.org/jira/browse/HIVE-1613 > Project: Hadoop Hive > Issue Type: Bug > Components: Clients >Affects Versions: 0.6.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.7.0 > > Attachments: hive-1613.patch.txt > > > hive --service jar fails. I have to open another ticket to clean up the > scripts and unify functions like version detection. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1613) hive --service jar looks for hadoop version but was not defined
hive --service jar looks for hadoop version but was not defined --- Key: HIVE-1613 URL: https://issues.apache.org/jira/browse/HIVE-1613 Project: Hadoop Hive Issue Type: Bug Components: Clients Affects Versions: 0.6.0 Reporter: Edward Capriolo Assignee: Edward Capriolo Fix For: 0.7.0 hive --service jar fails. I have to open another ticket to clean up the scripts and unify functions like version detection. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-471:
-
Status: Patch Available (was: Open)
> A UDF for simple reflection
> ---
>
> Key: HIVE-471
> URL: https://issues.apache.org/jira/browse/HIVE-471
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
>Affects Versions: 0.6.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Fix For: 0.7.0
>
> Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch,
> HIVE-471.3.patch, HIVE-471.4.patch, HIVE-471.5.patch, HIVE-471.6.patch.txt,
> hive-471.diff
>
>
> There are many methods in java that are static and have no arguments or can
> be invoked with one simple parameter. More complicated functions will require
> a UDF but one generic one can work as a poor-mans UDF.
> {noformat}
> SELECT reflect("java.lang.String", "valueOf", 1), reflect("java.lang.String",
> "isEmpty")
> FROM src LIMIT 1;
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-471:
-
Attachment: HIVE-471.6.patch.txt
> A UDF for simple reflection
> ---
>
> Key: HIVE-471
> URL: https://issues.apache.org/jira/browse/HIVE-471
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
>Affects Versions: 0.6.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Fix For: 0.7.0
>
> Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch,
> HIVE-471.3.patch, HIVE-471.4.patch, HIVE-471.5.patch, HIVE-471.6.patch.txt,
> hive-471.diff
>
>
> There are many methods in java that are static and have no arguments or can
> be invoked with one simple parameter. More complicated functions will require
> a UDF but one generic one can work as a poor-mans UDF.
> {noformat}
> SELECT reflect("java.lang.String", "valueOf", 1), reflect("java.lang.String",
> "isEmpty")
> FROM src LIMIT 1;
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902883#action_12902883
]
Edward Capriolo commented on HIVE-471:
--
@Namit,
I thought we were getting rid of negative tests. Am I misinformed?
> A UDF for simple reflection
> ---
>
> Key: HIVE-471
> URL: https://issues.apache.org/jira/browse/HIVE-471
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
>Affects Versions: 0.6.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Fix For: 0.7.0
>
> Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch,
> HIVE-471.3.patch, HIVE-471.4.patch, HIVE-471.5.patch, hive-471.diff
>
>
> There are many methods in java that are static and have no arguments or can
> be invoked with one simple parameter. More complicated functions will require
> a UDF but one generic one can work as a poor-mans UDF.
> {noformat}
> SELECT reflect("java.lang.String", "valueOf", 1), reflect("java.lang.String",
> "isEmpty")
> FROM src LIMIT 1;
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1434) Cassandra Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12902616#action_12902616
]
Edward Capriolo commented on HIVE-1434:
---
Maven, I am on the fence about it. We actually do not need all the libs I
included. Having them in a tarball sounds good, but making a maven repo for
only this purpose seems to be a lot of work.
{quote}
Should we attempt to factor out the HBase commonality immediately, or commit
the overlapping code and then do refactoring as a followup? I'm fine either
way; I can give suggestions on how to create the reusable abstract bases and
where to package+name them.{quote}
If you can specify specific instances then sure. The code may be 99% the same,
but that one nuance is going to make the abstractions confusing and useless.
I await further review.
> Cassandra Storage Handler
> -
>
> Key: HIVE-1434
> URL: https://issues.apache.org/jira/browse/HIVE-1434
> Project: Hadoop Hive
> Issue Type: New Feature
>Affects Versions: 0.7.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.7.0
>
> Attachments: cas-handle.tar.gz, hive-1434-1.txt,
> hive-1434-2-patch.txt, hive-1434-3-patch.txt, hive-1434-4-patch.txt
>
>
> Add a cassandra storage handler.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-471:
-
Status: Patch Available (was: Open)
Affects Version/s: 0.6.0
(was: 0.5.1)
Fix Version/s: 0.7.0
> A UDF for simple reflection
> ---
>
> Key: HIVE-471
> URL: https://issues.apache.org/jira/browse/HIVE-471
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
>Affects Versions: 0.6.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Fix For: 0.7.0
>
> Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch,
> HIVE-471.3.patch, HIVE-471.4.patch, HIVE-471.5.patch, hive-471.diff
>
>
> There are many methods in java that are static and have no arguments or can
> be invoked with one simple parameter. More complicated functions will require
> a UDF but one generic one can work as a poor-mans UDF.
> {noformat}
> SELECT reflect("java.lang.String", "valueOf", 1), reflect("java.lang.String",
> "isEmpty")
> FROM src LIMIT 1;
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-471:
-
Attachment: HIVE-471.5.patch
Almost two years in the making!
> A UDF for simple reflection
> ---
>
> Key: HIVE-471
> URL: https://issues.apache.org/jira/browse/HIVE-471
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
>Affects Versions: 0.5.1
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch,
> HIVE-471.3.patch, HIVE-471.4.patch, HIVE-471.5.patch, hive-471.diff
>
>
> There are many methods in java that are static and have no arguments or can
> be invoked with one simple parameter. More complicated functions will require
> a UDF but one generic one can work as a poor-mans UDF.
> {noformat}
> SELECT reflect("java.lang.String", "valueOf", 1), reflect("java.lang.String",
> "isEmpty")
> FROM src LIMIT 1;
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1505) Support non-UTF8 data
[ https://issues.apache.org/jira/browse/HIVE-1505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12900697#action_12900697 ] Edward Capriolo commented on HIVE-1505: --- Maybe you should fork hive and call it chive. On a serious node . Great job. Would you consider editing the cli.xml in the xdocs to explain this feature? I think it would be very helpful look in docs/xdocs/. > Support non-UTF8 data > - > > Key: HIVE-1505 > URL: https://issues.apache.org/jira/browse/HIVE-1505 > Project: Hadoop Hive > Issue Type: New Feature > Components: Serializers/Deserializers >Affects Versions: 0.5.0 >Reporter: bc Wong >Assignee: Ted Xu > Attachments: trunk-encoding.patch > > > I'd like to work with non-UTF8 data easily. > Suppose I have data in latin1. Currently, doing a "select *" will return the > upper ascii characters in '\xef\xbf\xbd', which is the replacement character > '\ufffd' encoded in UTF-8. Would be nice for Hive to understand different > encodings, or to have a concept of byte string. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12900039#action_12900039 ] Edward Capriolo commented on HIVE-1555: --- I wonder if this could end up being a very effective way to query shared data stores. I think I saw something like this in futurama.. Dont worry about querying blank, let me worry about querying blank. http://www.google.com/url?sa=t&source=web&cd=2&ved=0CBcQFjAB&url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DB5cAwTEEGNE&ei=Qk9sTLAThIqXB__DzDw&usg=AFQjCNH_TOUS1cl6t0gZXefRURw0a_feZg > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hadoop Hive > Issue Type: New Feature >Affects Versions: 0.5.0 >Reporter: Bob Robertson > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1434) Cassandra Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1434: -- Attachment: hive-1434-4-patch.txt Refactored the code, added xdoc, more extensive testing. > Cassandra Storage Handler > - > > Key: HIVE-1434 > URL: https://issues.apache.org/jira/browse/HIVE-1434 > Project: Hadoop Hive > Issue Type: New Feature >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Attachments: cas-handle.tar.gz, hive-1434-1.txt, > hive-1434-2-patch.txt, hive-1434-3-patch.txt, hive-1434-4-patch.txt > > > Add a cassandra storage handler. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1530) Include hive-default.xml and hive-log4j.properties in hive-common JAR
[ https://issues.apache.org/jira/browse/HIVE-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898266#action_12898266 ] Edward Capriolo commented on HIVE-1530: --- @Joydeep +1 > Include hive-default.xml and hive-log4j.properties in hive-common JAR > - > > Key: HIVE-1530 > URL: https://issues.apache.org/jira/browse/HIVE-1530 > Project: Hadoop Hive > Issue Type: Improvement > Components: Configuration >Reporter: Carl Steinbach > > hive-common-*.jar should include hive-default.xml and hive-log4j.properties, > and similarly hive-exec-*.jar should include hive-exec-log4j.properties. The > hive-default.xml file that currently sits in the conf/ directory should be > removed. > Motivations for this change: > * We explicitly tell users that they should never modify hive-default.xml yet > give them the opportunity to do so by placing the file in the conf dir. > * Many users are familiar with the Hadoop configuration mechanism that does > not require *-default.xml files to be present in the HADOOP_CONF_DIR, and > assume that the same is true for HIVE_CONF_DIR. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1530) Include hive-default.xml and hive-log4j.properties in hive-common JAR
[ https://issues.apache.org/jira/browse/HIVE-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898030#action_12898030 ] Edward Capriolo commented on HIVE-1530: --- I like the default xml. Hive has many undocumented options, new ones are being added often. Are end users going to know which jar the default.xml are in? Users want to extracting a jar just to get the conf out of it to read the description of the setting. As for what hadoop does...I personally find it annoying to have navigate to hadoop/src/mapred/mapred-default.xml or to hadoop/src/hdfs/hdfs-default.xml to figure out what options I have for settings. So i do not really thing we should just do it to be like hadoop it it makes peoples life harder. If anything please keep it as hive-site.xml.sample. > Include hive-default.xml and hive-log4j.properties in hive-common JAR > - > > Key: HIVE-1530 > URL: https://issues.apache.org/jira/browse/HIVE-1530 > Project: Hadoop Hive > Issue Type: Improvement > Components: Configuration >Reporter: Carl Steinbach > > hive-common-*.jar should include hive-default.xml and hive-log4j.properties, > and similarly hive-exec-*.jar should include hive-exec-log4j.properties. The > hive-default.xml file that currently sits in the conf/ directory should be > removed. > Motivations for this change: > * We explicitly tell users that they should never modify hive-default.xml yet > give them the opportunity to do so by placing the file in the conf dir. > * Many users are familiar with the Hadoop configuration mechanism that does > not require *-default.xml files to be present in the HADOOP_CONF_DIR, and > assume that the same is true for HIVE_CONF_DIR. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1434) Cassandra Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1434: -- Status: Patch Available (was: Open) This patch has full read/write functionality. I am going to do another patch later today with xdocs, but do not expect any code changes. > Cassandra Storage Handler > - > > Key: HIVE-1434 > URL: https://issues.apache.org/jira/browse/HIVE-1434 > Project: Hadoop Hive > Issue Type: New Feature >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Attachments: cas-handle.tar.gz, hive-1434-1.txt, > hive-1434-2-patch.txt, hive-1434-3-patch.txt > > > Add a cassandra storage handler. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1434) Cassandra Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1434: -- Attachment: hive-1434-3-patch.txt > Cassandra Storage Handler > - > > Key: HIVE-1434 > URL: https://issues.apache.org/jira/browse/HIVE-1434 > Project: Hadoop Hive > Issue Type: New Feature >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Attachments: cas-handle.tar.gz, hive-1434-1.txt, > hive-1434-2-patch.txt, hive-1434-3-patch.txt > > > Add a cassandra storage handler. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1513) hive starter scripts should load admin/user supplied script for configurability
[ https://issues.apache.org/jira/browse/HIVE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12895698#action_12895698 ] Edward Capriolo commented on HIVE-1513: --- Anything you put in the bin/ext is sourced as part of the bootstrap process. Could you do something like bin/ext/mystuff.sh? > hive starter scripts should load admin/user supplied script for > configurability > --- > > Key: HIVE-1513 > URL: https://issues.apache.org/jira/browse/HIVE-1513 > Project: Hadoop Hive > Issue Type: Improvement > Components: CLI >Reporter: Joydeep Sen Sarma > > it's difficult to add environment variables to Hive starter scripts except by > modifying the scripts directly. this is undesirable (since they are source > code). Hive starter scripts should load a admin supplied shell script for > configurability. This would be similar to what hadoop does with hadoop-env.sh -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1511) Hive plan serialization is slow
[ https://issues.apache.org/jira/browse/HIVE-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12895352#action_12895352 ] Edward Capriolo commented on HIVE-1511: --- Also possibly a clever way to remove duplicate expressions that evaluate to the same result such as multiple key=0 > Hive plan serialization is slow > --- > > Key: HIVE-1511 > URL: https://issues.apache.org/jira/browse/HIVE-1511 > Project: Hadoop Hive > Issue Type: Improvement >Affects Versions: 0.7.0 >Reporter: Ning Zhang > > As reported by Edward Capriolo: > For reference I did this as a test case > SELECT * FROM src where > key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 > OR key=0 OR key=0 OR key=0 OR > key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 > OR key=0 OR key=0 OR key=0 OR > ...(100 more of these) > No OOM but I gave up after the test case did not go anywhere for about > 2 minutes. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1434) Cassandra Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1434: -- Attachment: hive-1434-2-patch.txt Closing in on this one. This patch sets up build environment correctly. Proper test infrastructure. Patch is much cleaner. Still working on Serializing/Deserialing correctly so not very functional. 80% I think. > Cassandra Storage Handler > - > > Key: HIVE-1434 > URL: https://issues.apache.org/jira/browse/HIVE-1434 > Project: Hadoop Hive > Issue Type: New Feature >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Attachments: cas-handle.tar.gz, hive-1434-1.txt, hive-1434-2-patch.txt > > > Add a cassandra storage handler. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1441) Extend ivy offline mode to cover metastore downloads
[
https://issues.apache.org/jira/browse/HIVE-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894133#action_12894133
]
Edward Capriolo commented on HIVE-1441:
---
Fresh checkout before any after patch. Still looking into it.
{noformat}
M
etaException(message:Could not connect to meta store using any of the URIs
provided)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:160)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:128)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:71)
at
org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote.setUp(TestHiveMetaStoreRemote.java:64)
at junit.framework.TestCase.runBare(TestCase.java:125)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)
>From eclipse:
Running metastore!
MetaException(message:hive.metastore.warehouse.dir is not set in the config or
blank)
at org.apache.hadoop.hive.metastore.Warehouse.(Warehouse.java:58)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:155)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.(HiveMetaStore.java:125)
at
org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:1965)
at
org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote$RunMS.run(TestHiveMetaStoreRemote.java:39)
at java.lang.Thread.run(Thread.java:619)
10/07/30 16:03:22 ERROR metastore.HiveMetaStore: Metastore Thrift Server threw
an exception. Exiting...
10/07/30 16:03:22 ERROR metastore.HiveMetaStore:
MetaException(message:hive.metastore.warehouse.dir is not set in the config or
blank)
at org.apache.hadoop.hive.metastore.Warehouse.(Warehouse.java:58)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:155)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.(HiveMetaStore.java:125)
at
org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:1965)
at
org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote$RunMS.run(TestHiveMetaStoreRemote.java:39)
at java.lang.Thread.run(Thread.java:619)
{noformat}
> Extend ivy offline mode to cover metastore downloads
>
>
> Key: HIVE-1441
> URL: https://issues.apache.org/jira/browse/HIVE-1441
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Build Infrastructure
>Affects Versions: 0.6.0
>Reporter: John Sichi
>Assignee: John Sichi
> Fix For: 0.7.0
>
> Attachments: HIVE-1441.1.patch
>
>
> We recently started downloading datanucleus jars via ivy, and the existing
> ivy offilne mode doesn't cover this, so we still end up trying to contact the
> ivy repository even with offline mode enabled.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1441) Extend ivy offline mode to cover metastore downloads
[ https://issues.apache.org/jira/browse/HIVE-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894028#action_12894028 ] Edward Capriolo commented on HIVE-1441: --- Testing. test: [junit] Running org.apache.hadoop.hive.metastore.TestHiveMetaStore [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 24.539 sec [junit] Running org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote [junit] Running metastore! [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 8.238 sec [junit] Test org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote FAILED Can this be skipped? > Extend ivy offline mode to cover metastore downloads > > > Key: HIVE-1441 > URL: https://issues.apache.org/jira/browse/HIVE-1441 > Project: Hadoop Hive > Issue Type: Improvement > Components: Build Infrastructure >Affects Versions: 0.6.0 >Reporter: John Sichi >Assignee: John Sichi > Fix For: 0.7.0 > > Attachments: HIVE-1441.1.patch > > > We recently started downloading datanucleus jars via ivy, and the existing > ivy offilne mode doesn't cover this, so we still end up trying to contact the > ivy repository even with offline mode enabled. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1492) FileSinkOperator should remove duplicated files from the same task based on file sizes
[ https://issues.apache.org/jira/browse/HIVE-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893772#action_12893772 ] Edward Capriolo commented on HIVE-1492: --- "the largest file is the correct file" Is that generally true or an absolute fact? > FileSinkOperator should remove duplicated files from the same task based on > file sizes > -- > > Key: HIVE-1492 > URL: https://issues.apache.org/jira/browse/HIVE-1492 > Project: Hadoop Hive > Issue Type: Bug >Affects Versions: 0.7.0 >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.7.0 > > Attachments: HIVE-1492.patch, HIVE-1492_branch-0.6.patch > > > FileSinkOperator.jobClose() calls Utilities.removeTempOrDuplicateFiles() to > retain only one file for each task. A task could produce multiple files due > to failed attempts or speculative runs. The largest file should be retained > rather than the first file for each task. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1294) HIVE_AUX_JARS_PATH interferes with startup of Hive Web Interface
[
https://issues.apache.org/jira/browse/HIVE-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-1294:
--
Status: Patch Available (was: Open)
Fix Version/s: 0.6.0
Hwi does not start correctly without this patch.
> HIVE_AUX_JARS_PATH interferes with startup of Hive Web Interface
>
>
> Key: HIVE-1294
> URL: https://issues.apache.org/jira/browse/HIVE-1294
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Web UI
>Affects Versions: 0.5.0
>Reporter: Dilip Joseph
>Assignee: Edward Capriolo
>Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: hive-1294.patch.txt
>
>
> The Hive Webserver fails to startup with the following error message, if
> HIVE_AUX_JARS_PATH environment variable is set (works fine if unset).
> $ build/dist/bin/hive --service hwi
> Exception in thread "main" java.io.IOException: Error opening job jar:
> -libjars
>at org.apache.hadoop.util.RunJar.main(RunJar.java:90)
> Caused by: java.util.zip.ZipException: error in opening zip file
>at java.util.zip.ZipFile.open(Native Method)
>at java.util.zip.ZipFile.(ZipFile.java:114)
>at java.util.jar.JarFile.(JarFile.java:133)
>at java.util.jar.JarFile.(JarFile.java:70)
>at org.apache.hadoop.util.RunJar.main(RunJar.java:88)
> Slightly modifying the command line to launch hadoop in hwi.sh solves the
> problem:
> $ diff bin/ext/hwi.sh /tmp/new-hwi.sh
> 28c28
> < exec $HADOOP jar $AUX_JARS_CMD_LINE ${HWI_JAR_FILE} $CLASS $HIVE_OPTS "$@"
> ---
> > exec $HADOOP jar ${HWI_JAR_FILE} $CLASS $AUX_JARS_CMD_LINE $HIVE_OPTS
> > "$@"
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1294) HIVE_AUX_JARS_PATH interferes with startup of Hive Web Interface
[
https://issues.apache.org/jira/browse/HIVE-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-1294:
--
Attachment: hive-1294.patch.txt
> HIVE_AUX_JARS_PATH interferes with startup of Hive Web Interface
>
>
> Key: HIVE-1294
> URL: https://issues.apache.org/jira/browse/HIVE-1294
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Web UI
>Affects Versions: 0.5.0
>Reporter: Dilip Joseph
>Assignee: Edward Capriolo
>Priority: Blocker
> Attachments: hive-1294.patch.txt
>
>
> The Hive Webserver fails to startup with the following error message, if
> HIVE_AUX_JARS_PATH environment variable is set (works fine if unset).
> $ build/dist/bin/hive --service hwi
> Exception in thread "main" java.io.IOException: Error opening job jar:
> -libjars
>at org.apache.hadoop.util.RunJar.main(RunJar.java:90)
> Caused by: java.util.zip.ZipException: error in opening zip file
>at java.util.zip.ZipFile.open(Native Method)
>at java.util.zip.ZipFile.(ZipFile.java:114)
>at java.util.jar.JarFile.(JarFile.java:133)
>at java.util.jar.JarFile.(JarFile.java:70)
>at org.apache.hadoop.util.RunJar.main(RunJar.java:88)
> Slightly modifying the command line to launch hadoop in hwi.sh solves the
> problem:
> $ diff bin/ext/hwi.sh /tmp/new-hwi.sh
> 28c28
> < exec $HADOOP jar $AUX_JARS_CMD_LINE ${HWI_JAR_FILE} $CLASS $HIVE_OPTS "$@"
> ---
> > exec $HADOOP jar ${HWI_JAR_FILE} $CLASS $AUX_JARS_CMD_LINE $HIVE_OPTS
> > "$@"
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-1294) HIVE_AUX_JARS_PATH interferes with startup of Hive Web Interface
[
https://issues.apache.org/jira/browse/HIVE-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo reassigned HIVE-1294:
-
Assignee: Edward Capriolo
> HIVE_AUX_JARS_PATH interferes with startup of Hive Web Interface
>
>
> Key: HIVE-1294
> URL: https://issues.apache.org/jira/browse/HIVE-1294
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Web UI
>Affects Versions: 0.5.0
>Reporter: Dilip Joseph
>Assignee: Edward Capriolo
>Priority: Minor
>
> The Hive Webserver fails to startup with the following error message, if
> HIVE_AUX_JARS_PATH environment variable is set (works fine if unset).
> $ build/dist/bin/hive --service hwi
> Exception in thread "main" java.io.IOException: Error opening job jar:
> -libjars
>at org.apache.hadoop.util.RunJar.main(RunJar.java:90)
> Caused by: java.util.zip.ZipException: error in opening zip file
>at java.util.zip.ZipFile.open(Native Method)
>at java.util.zip.ZipFile.(ZipFile.java:114)
>at java.util.jar.JarFile.(JarFile.java:133)
>at java.util.jar.JarFile.(JarFile.java:70)
>at org.apache.hadoop.util.RunJar.main(RunJar.java:88)
> Slightly modifying the command line to launch hadoop in hwi.sh solves the
> problem:
> $ diff bin/ext/hwi.sh /tmp/new-hwi.sh
> 28c28
> < exec $HADOOP jar $AUX_JARS_CMD_LINE ${HWI_JAR_FILE} $CLASS $HIVE_OPTS "$@"
> ---
> > exec $HADOOP jar ${HWI_JAR_FILE} $CLASS $AUX_JARS_CMD_LINE $HIVE_OPTS
> > "$@"
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1294) HIVE_AUX_JARS_PATH interferes with startup of Hive Web Interface
[
https://issues.apache.org/jira/browse/HIVE-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-1294:
--
Priority: Blocker (was: Minor)
> HIVE_AUX_JARS_PATH interferes with startup of Hive Web Interface
>
>
> Key: HIVE-1294
> URL: https://issues.apache.org/jira/browse/HIVE-1294
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Web UI
>Affects Versions: 0.5.0
>Reporter: Dilip Joseph
>Assignee: Edward Capriolo
>Priority: Blocker
>
> The Hive Webserver fails to startup with the following error message, if
> HIVE_AUX_JARS_PATH environment variable is set (works fine if unset).
> $ build/dist/bin/hive --service hwi
> Exception in thread "main" java.io.IOException: Error opening job jar:
> -libjars
>at org.apache.hadoop.util.RunJar.main(RunJar.java:90)
> Caused by: java.util.zip.ZipException: error in opening zip file
>at java.util.zip.ZipFile.open(Native Method)
>at java.util.zip.ZipFile.(ZipFile.java:114)
>at java.util.jar.JarFile.(JarFile.java:133)
>at java.util.jar.JarFile.(JarFile.java:70)
>at org.apache.hadoop.util.RunJar.main(RunJar.java:88)
> Slightly modifying the command line to launch hadoop in hwi.sh solves the
> problem:
> $ diff bin/ext/hwi.sh /tmp/new-hwi.sh
> 28c28
> < exec $HADOOP jar $AUX_JARS_CMD_LINE ${HWI_JAR_FILE} $CLASS $HIVE_OPTS "$@"
> ---
> > exec $HADOOP jar ${HWI_JAR_FILE} $CLASS $AUX_JARS_CMD_LINE $HIVE_OPTS
> > "$@"
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1434) Cassandra Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1434: -- Attachment: cas-handle.tar.gz This is not a quality patch yet. I am still experimenting with some ideas. Everying is free form and will likely change before the final patch. There are a few junk files (HiveIColumn,etc) which will not be part of the release. Thus far: CassandraSplit.java HiveCassandraTableInputFormat.java CassandraSerDe.java TestColumnFamilyInputFormat.java TestCassandraPut.java TestColumnFamilyInputFormat.java Are working and can give you an idea of where the code is going. > Cassandra Storage Handler > - > > Key: HIVE-1434 > URL: https://issues.apache.org/jira/browse/HIVE-1434 > Project: Hadoop Hive > Issue Type: New Feature >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Attachments: cas-handle.tar.gz, hive-1434-1.txt > > > Add a cassandra storage handler. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1414) automatically invoke .hiverc init script
[ https://issues.apache.org/jira/browse/HIVE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1414: -- Status: Patch Available (was: Open) > automatically invoke .hiverc init script > > > Key: HIVE-1414 > URL: https://issues.apache.org/jira/browse/HIVE-1414 > Project: Hadoop Hive > Issue Type: Improvement > Components: Clients >Affects Versions: 0.5.0 >Reporter: John Sichi >Assignee: Edward Capriolo > Fix For: 0.7.0 > > Attachments: hive-1414-2.txt, hive-1414-patch-1.txt > > > Similar to .bashrc but run Hive SQL commands. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1414) automatically invoke .hiverc init script
[ https://issues.apache.org/jira/browse/HIVE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1414: -- Attachment: hive-1414-2.txt New version only reads hiverc if -i option is not specified. Includes xdocs. > automatically invoke .hiverc init script > > > Key: HIVE-1414 > URL: https://issues.apache.org/jira/browse/HIVE-1414 > Project: Hadoop Hive > Issue Type: Improvement > Components: Clients >Affects Versions: 0.5.0 >Reporter: John Sichi >Assignee: Edward Capriolo > Fix For: 0.7.0 > > Attachments: hive-1414-2.txt, hive-1414-patch-1.txt > > > Similar to .bashrc but run Hive SQL commands. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-471:
-
Attachment: HIVE-471.4.patch
> A UDF for simple reflection
> ---
>
> Key: HIVE-471
> URL: https://issues.apache.org/jira/browse/HIVE-471
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
>Affects Versions: 0.5.1
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch,
> HIVE-471.3.patch, HIVE-471.4.patch, hive-471.diff
>
>
> There are many methods in java that are static and have no arguments or can
> be invoked with one simple parameter. More complicated functions will require
> a UDF but one generic one can work as a poor-mans UDF.
> {noformat}
> SELECT reflect("java.lang.String", "valueOf", 1), reflect("java.lang.String",
> "isEmpty")
> FROM src LIMIT 1;
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-471:
-
Status: Patch Available (was: Open)
> A UDF for simple reflection
> ---
>
> Key: HIVE-471
> URL: https://issues.apache.org/jira/browse/HIVE-471
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
>Affects Versions: 0.5.1
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch,
> HIVE-471.3.patch, HIVE-471.4.patch, hive-471.diff
>
>
> There are many methods in java that are static and have no arguments or can
> be invoked with one simple parameter. More complicated functions will require
> a UDF but one generic one can work as a poor-mans UDF.
> {noformat}
> SELECT reflect("java.lang.String", "valueOf", 1), reflect("java.lang.String",
> "isEmpty")
> FROM src LIMIT 1;
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1446) Move Hive Documentation from the wiki to version control
[ https://issues.apache.org/jira/browse/HIVE-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1446: -- Attachment: hive-1446-part-1.diff Got most of the language manual > Move Hive Documentation from the wiki to version control > > > Key: HIVE-1446 > URL: https://issues.apache.org/jira/browse/HIVE-1446 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Reporter: Carl Steinbach >Assignee: Carl Steinbach > Fix For: 0.6.0, 0.7.0 > > Attachments: hive-1446-part-1.diff, hive-1446.diff, hive-logo-wide.png > > > Move the Hive Language Manual (and possibly some other documents) from the > Hive wiki to version control. This work needs to be coordinated with the > hive-dev and hive-user community in order to avoid missing any edits as well > as to avoid or limit unavailability of the docs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1096) Hive Variables
[
https://issues.apache.org/jira/browse/HIVE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886912#action_12886912
]
Edward Capriolo commented on HIVE-1096:
---
I am having trouble uploading with the update diff function of the review
board. As I mentioned several times, I really had one simple requirement
{noformat}
hive -hiveconf DAY=5 -e "LOAD DATA INFILE '/tmp/${DAY}' into logs
partition=${DAY}"
{noformat}
I am all for doing things 100% correct, but this is such a simple thing I am
really getting worn out by the endless revisions and doing lots of fancy things
just because someone might want to do ${x${y}bla}.
Really, I would like to make this ticket go +1, and get on with something more
interesting.
> Hive Variables
> --
>
> Key: HIVE-1096
> URL: https://issues.apache.org/jira/browse/HIVE-1096
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.6.0, 0.7.0
>
> Attachments: 1096-9.diff, hive-1096-10-patch.txt,
> hive-1096-11-patch.txt, hive-1096-12.patch.txt, hive-1096-2.diff,
> hive-1096-7.diff, hive-1096-8.diff, hive-1096.diff
>
>
> From mailing list:
> --Amazon Elastic MapReduce version of Hive seems to have a nice feature
> called "Variables." Basically you can define a variable via command-line
> while invoking hive with -d DT=2009-12-09 and then refer to the variable via
> ${DT} within the hive queries. This could be extremely useful. I can't seem
> to find this feature even on trunk. Is this feature currently anywhere in the
> roadmap?--
> This could be implemented in many places.
> A simple place to put this is
> in Driver.compile or Driver.run we can do string substitutions at that level,
> and further downstream need not be effected.
> There could be some benefits to doing this further downstream, parser,plan.
> but based on the simple needs we may not need to overthink this.
> I will get started on implementing in compile unless someone wants to discuss
> this more.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1096) Hive Variables
[
https://issues.apache.org/jira/browse/HIVE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-1096:
--
Attachment: hive-1096-12.patch.txt
Change interpolate to substituteAdded the substitution logic to file, dfs,
set , and query processor
> Hive Variables
> --
>
> Key: HIVE-1096
> URL: https://issues.apache.org/jira/browse/HIVE-1096
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.6.0, 0.7.0
>
> Attachments: 1096-9.diff, hive-1096-10-patch.txt,
> hive-1096-11-patch.txt, hive-1096-12.patch.txt, hive-1096-2.diff,
> hive-1096-7.diff, hive-1096-8.diff, hive-1096.diff
>
>
> From mailing list:
> --Amazon Elastic MapReduce version of Hive seems to have a nice feature
> called "Variables." Basically you can define a variable via command-line
> while invoking hive with -d DT=2009-12-09 and then refer to the variable via
> ${DT} within the hive queries. This could be extremely useful. I can't seem
> to find this feature even on trunk. Is this feature currently anywhere in the
> roadmap?--
> This could be implemented in many places.
> A simple place to put this is
> in Driver.compile or Driver.run we can do string substitutions at that level,
> and further downstream need not be effected.
> There could be some benefits to doing this further downstream, parser,plan.
> but based on the simple needs we may not need to overthink this.
> I will get started on implementing in compile unless someone wants to discuss
> this more.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1096) Hive Variables
[
https://issues.apache.org/jira/browse/HIVE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-1096:
--
Status: Patch Available (was: Open)
> Hive Variables
> --
>
> Key: HIVE-1096
> URL: https://issues.apache.org/jira/browse/HIVE-1096
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.6.0, 0.7.0
>
> Attachments: 1096-9.diff, hive-1096-10-patch.txt,
> hive-1096-11-patch.txt, hive-1096-12.patch.txt, hive-1096-2.diff,
> hive-1096-7.diff, hive-1096-8.diff, hive-1096.diff
>
>
> From mailing list:
> --Amazon Elastic MapReduce version of Hive seems to have a nice feature
> called "Variables." Basically you can define a variable via command-line
> while invoking hive with -d DT=2009-12-09 and then refer to the variable via
> ${DT} within the hive queries. This could be extremely useful. I can't seem
> to find this feature even on trunk. Is this feature currently anywhere in the
> roadmap?--
> This could be implemented in many places.
> A simple place to put this is
> in Driver.compile or Driver.run we can do string substitutions at that level,
> and further downstream need not be effected.
> There could be some benefits to doing this further downstream, parser,plan.
> but based on the simple needs we may not need to overthink this.
> I will get started on implementing in compile unless someone wants to discuss
> this more.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1447) Speed up reflection method calls
[
https://issues.apache.org/jira/browse/HIVE-1447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12884488#action_12884488
]
Edward Capriolo commented on HIVE-1447:
---
Does this have an affect on all generic UDFs? I had someone telling me that
they added one of my genUDF's to a query on a large table and the time to
complete the job went up dramatically.
> Speed up reflection method calls
>
>
> Key: HIVE-1447
> URL: https://issues.apache.org/jira/browse/HIVE-1447
> Project: Hadoop Hive
> Issue Type: Improvement
>Reporter: Zheng Shao
>Assignee: Zheng Shao
> Attachments: A.java, HIVE-1447.1.patch
>
>
> See http://www.cowtowncoder.com/blog/archives/2010/04/entry_396.html and
> http://www.jguru.com/faq/view.jsp?EID=246569
> There is a huge drop of overhead (more than half) if we do
> "field.setAccessible(true)" for the field that we want to access.
> I did a simple experiment and that worked well with method as well.
> The results are (note that the method just add 1 to an integer):
> {code}
> 1 regular method calls:26 milliseconds.
> 1 reflective method calls without lookup:4029 milliseconds.
> 1 accessible reflective method calls without lookup:1810 milliseconds.
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1434) Cassandra Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12884384#action_12884384 ] Edward Capriolo commented on HIVE-1434: --- I actually got pretty far with this simply duplicating the logic in the Hbase Storage handler. Unfortunately I hit a snafu. Cassandra is not using the deprecated mapred.*, their input format is using mapreduce.*. I have seen a few tickets for this, and as far as I know hive is 100% mapred. So to get this done we either have to wait until hive is converted to mapreduce, or I have to make an "old school" mapred based input format for cassandra. @John am I wrong? Is there a way to work with mapreduce input formats that I am not understanding? > Cassandra Storage Handler > - > > Key: HIVE-1434 > URL: https://issues.apache.org/jira/browse/HIVE-1434 > Project: Hadoop Hive > Issue Type: New Feature >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Attachments: hive-1434-1.txt > > > Add a cassandra storage handler. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1446) Move Hive Documentation from the wiki to version control
[ https://issues.apache.org/jira/browse/HIVE-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1446: -- Attachment: hive-1446.diff Includes the image in the vsl to fix the alignment > Move Hive Documentation from the wiki to version control > > > Key: HIVE-1446 > URL: https://issues.apache.org/jira/browse/HIVE-1446 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Reporter: Carl Steinbach >Assignee: Carl Steinbach > Fix For: 0.6.0, 0.7.0 > > Attachments: hive-1446.diff, hive-logo-wide.png > > > Move the Hive Language Manual (and possibly some other documents) from the > Hive wiki to version control. This work needs to be coordinated with the > hive-dev and hive-user community in order to avoid missing any edits as well > as to avoid or limit unavailability of the docs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1446) Move Hive Documentation from the wiki to version control
[ https://issues.apache.org/jira/browse/HIVE-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1446: -- Attachment: hive-logo-wide.png We need this wise logo to fix the align of the generated docs > Move Hive Documentation from the wiki to version control > > > Key: HIVE-1446 > URL: https://issues.apache.org/jira/browse/HIVE-1446 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Reporter: Carl Steinbach >Assignee: Carl Steinbach > Fix For: 0.6.0, 0.7.0 > > Attachments: hive-logo-wide.png > > > Move the Hive Language Manual (and possibly some other documents) from the > Hive wiki to version control. This work needs to be coordinated with the > hive-dev and hive-user community in order to avoid missing any edits as well > as to avoid or limit unavailability of the docs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1446) Move Hive Documentation from the wiki to version control
[ https://issues.apache.org/jira/browse/HIVE-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12884142#action_12884142 ] Edward Capriolo commented on HIVE-1446: --- I will make a xdoc the CLI page > Move Hive Documentation from the wiki to version control > > > Key: HIVE-1446 > URL: https://issues.apache.org/jira/browse/HIVE-1446 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Reporter: Carl Steinbach >Assignee: Carl Steinbach > Fix For: 0.6.0, 0.7.0 > > > Move the Hive Language Manual (and possibly some other documents) from the > Hive wiki to version control. This work needs to be coordinated with the > hive-dev and hive-user community in order to avoid missing any edits as well > as to avoid or limit unavailability of the docs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1135) Use Anakia for version controlled documentation
[ https://issues.apache.org/jira/browse/HIVE-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12884141#action_12884141 ] Edward Capriolo commented on HIVE-1135: --- Carl, thank you for the assist! > Use Anakia for version controlled documentation > --- > > Key: HIVE-1135 > URL: https://issues.apache.org/jira/browse/HIVE-1135 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1135-3-patch.txt, hive-1135-4-patch.txt, > hive-1135-5-patch.txt, hive-1135-6-patch.txt, hive-1335-1.patch.txt, > hive-1335-2.patch.txt, jdom-1.1.jar, jdom-1.1.LICENSE, wtf.png > > > Currently the Hive Language Manual and many other critical pieces of > documentation are on the Hive wiki. > Right now we count on the author of a patch to follow up and add wiki > entries. While we do a decent job with this, new features can be missed. Or > using running older/newer branches can not locate relevant documentation for > their branch. > ..example of a perception I do not think we want to give off... > http://dev.hubspot.com/bid/30170/Who-Loves-the-Magic-Undocumented-Hive-Mapjoin-This-Guy > We should generate our documentation in the way hadoop & hbase does, inline > using forest. I would like to take the lead on this, but we need a lot of > consensus on doing this properly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1434) Cassandra Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1434: -- Attachment: hive-1434-1.txt Just a start. (To prove that I am doing something with this ticket) > Cassandra Storage Handler > - > > Key: HIVE-1434 > URL: https://issues.apache.org/jira/browse/HIVE-1434 > Project: Hadoop Hive > Issue Type: New Feature >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Attachments: hive-1434-1.txt > > > Add a cassandra storage handler. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1434) Cassandra Storage Handler
Cassandra Storage Handler - Key: HIVE-1434 URL: https://issues.apache.org/jira/browse/HIVE-1434 Project: Hadoop Hive Issue Type: New Feature Reporter: Edward Capriolo Assignee: Edward Capriolo Add a cassandra storage handler. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1135) Use Anakia for version controlled documentation
[ https://issues.apache.org/jira/browse/HIVE-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882349#action_12882349 ] Edward Capriolo commented on HIVE-1135: --- Bump:: I will fix the formatting later. Can we commit this we do not really need any unit tests here? > Use Anakia for version controlled documentation > --- > > Key: HIVE-1135 > URL: https://issues.apache.org/jira/browse/HIVE-1135 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1135-3-patch.txt, hive-1135-4-patch.txt, > hive-1135-5-patch.txt, hive-1135-6-patch.txt, hive-1335-1.patch.txt, > hive-1335-2.patch.txt, jdom-1.1.jar, jdom-1.1.LICENSE, wtf.png > > > Currently the Hive Language Manual and many other critical pieces of > documentation are on the Hive wiki. > Right now we count on the author of a patch to follow up and add wiki > entries. While we do a decent job with this, new features can be missed. Or > using running older/newer branches can not locate relevant documentation for > their branch. > ..example of a perception I do not think we want to give off... > http://dev.hubspot.com/bid/30170/Who-Loves-the-Magic-Undocumented-Hive-Mapjoin-This-Guy > We should generate our documentation in the way hadoop & hbase does, inline > using forest. I would like to take the lead on this, but we need a lot of > consensus on doing this properly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1431) Hive CLI can't handle query files that begin with comments
[
https://issues.apache.org/jira/browse/HIVE-1431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882024#action_12882024
]
Edward Capriolo commented on HIVE-1431:
---
We have a few tickets open, we really need to move all this stuff to a real
parser so we can properly deal with things like ';' or comments like this or
whatever. It is painfully hard to work around all these type of things and we
never get to the root of the problem.
> Hive CLI can't handle query files that begin with comments
> --
>
> Key: HIVE-1431
> URL: https://issues.apache.org/jira/browse/HIVE-1431
> Project: Hadoop Hive
> Issue Type: Bug
> Components: CLI
>Reporter: Carl Steinbach
> Fix For: 0.6.0, 0.7.0
>
>
> {code}
> % cat test.q
> -- This is a comment, followed by a command
> set -v;
> --
> -- Another comment
> --
> show tables;
> -- Last comment
> (master) [ ~/Projects/hive ]
> % hive < test.q
> Hive history file=/tmp/carl/hive_job_log_carl_201006231606_1140875653.txt
> hive> -- This is a comment, followed by a command
> > set -v;
> FAILED: Parse Error: line 2:0 cannot recognize input 'set'
> hive> --
> > -- Another comment
> > --
> > show tables;
> OK
> rawchunks
> Time taken: 5.334 seconds
> hive> -- Last comment
> > (master) [ ~/Projects/hive ]
> %
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1096) Hive Variables
[
https://issues.apache.org/jira/browse/HIVE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-1096:
--
Attachment: hive-1096-11-patch.txt
Was not interpolating system:vars. Fixed with better test case.
> Hive Variables
> --
>
> Key: HIVE-1096
> URL: https://issues.apache.org/jira/browse/HIVE-1096
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.6.0, 0.7.0
>
> Attachments: 1096-9.diff, hive-1096-10-patch.txt,
> hive-1096-11-patch.txt, hive-1096-2.diff, hive-1096-7.diff, hive-1096-8.diff,
> hive-1096.diff
>
>
> From mailing list:
> --Amazon Elastic MapReduce version of Hive seems to have a nice feature
> called "Variables." Basically you can define a variable via command-line
> while invoking hive with -d DT=2009-12-09 and then refer to the variable via
> ${DT} within the hive queries. This could be extremely useful. I can't seem
> to find this feature even on trunk. Is this feature currently anywhere in the
> roadmap?--
> This could be implemented in many places.
> A simple place to put this is
> in Driver.compile or Driver.run we can do string substitutions at that level,
> and further downstream need not be effected.
> There could be some benefits to doing this further downstream, parser,plan.
> but based on the simple needs we may not need to overthink this.
> I will get started on implementing in compile unless someone wants to discuss
> this more.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1096) Hive Variables
[
https://issues.apache.org/jira/browse/HIVE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-1096:
--
Status: Patch Available (was: Open)
> Hive Variables
> --
>
> Key: HIVE-1096
> URL: https://issues.apache.org/jira/browse/HIVE-1096
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
>Affects Versions: 0.5.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.6.0
>
> Attachments: 1096-9.diff, hive-1096-10-patch.txt, hive-1096-2.diff,
> hive-1096-7.diff, hive-1096-8.diff, hive-1096.diff
>
>
> From mailing list:
> --Amazon Elastic MapReduce version of Hive seems to have a nice feature
> called "Variables." Basically you can define a variable via command-line
> while invoking hive with -d DT=2009-12-09 and then refer to the variable via
> ${DT} within the hive queries. This could be extremely useful. I can't seem
> to find this feature even on trunk. Is this feature currently anywhere in the
> roadmap?--
> This could be implemented in many places.
> A simple place to put this is
> in Driver.compile or Driver.run we can do string substitutions at that level,
> and further downstream need not be effected.
> There could be some benefits to doing this further downstream, parser,plan.
> but based on the simple needs we may not need to overthink this.
> I will get started on implementing in compile unless someone wants to discuss
> this more.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1096) Hive Variables
[
https://issues.apache.org/jira/browse/HIVE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-1096:
--
Attachment: hive-1096-10-patch.txt
Patch adds variable interpretation.
> Hive Variables
> --
>
> Key: HIVE-1096
> URL: https://issues.apache.org/jira/browse/HIVE-1096
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
>Affects Versions: 0.5.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.6.0
>
> Attachments: 1096-9.diff, hive-1096-10-patch.txt, hive-1096-2.diff,
> hive-1096-7.diff, hive-1096-8.diff, hive-1096.diff
>
>
> From mailing list:
> --Amazon Elastic MapReduce version of Hive seems to have a nice feature
> called "Variables." Basically you can define a variable via command-line
> while invoking hive with -d DT=2009-12-09 and then refer to the variable via
> ${DT} within the hive queries. This could be extremely useful. I can't seem
> to find this feature even on trunk. Is this feature currently anywhere in the
> roadmap?--
> This could be implemented in many places.
> A simple place to put this is
> in Driver.compile or Driver.run we can do string substitutions at that level,
> and further downstream need not be effected.
> There could be some benefits to doing this further downstream, parser,plan.
> but based on the simple needs we may not need to overthink this.
> I will get started on implementing in compile unless someone wants to discuss
> this more.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1414) automatically invoke .hiverc init script
[ https://issues.apache.org/jira/browse/HIVE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881206#action_12881206 ] Edward Capriolo commented on HIVE-1414: --- 1 & 3 ok. As for #2 is there every a condition where getenv() or home will be null? I doubt hive would work in the face in either of these conditions. I will add the null checks anyway. > automatically invoke .hiverc init script > > > Key: HIVE-1414 > URL: https://issues.apache.org/jira/browse/HIVE-1414 > Project: Hadoop Hive > Issue Type: Improvement > Components: Clients >Affects Versions: 0.5.0 >Reporter: John Sichi >Assignee: Edward Capriolo > Fix For: 0.7.0 > > Attachments: hive-1414-patch-1.txt > > > Similar to .bashrc but run Hive SQL commands. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1419) Policy on deserialization errors
[ https://issues.apache.org/jira/browse/HIVE-1419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880908#action_12880908 ] Edward Capriolo commented on HIVE-1419: --- I am looking through this and trying to wrap my head around it. Off hand do you know what happens in this situation. We have a table that we have added columns to over time create table tab (a int, b int); Over time we have added more columns alter table tab (a int, b int, c int) This works fine for us as selecting column c on older data returns null for that column. Will this behaviour be preserved ? > Policy on deserialization errors > > > Key: HIVE-1419 > URL: https://issues.apache.org/jira/browse/HIVE-1419 > Project: Hadoop Hive > Issue Type: Improvement > Components: Serializers/Deserializers >Affects Versions: 0.5.0 >Reporter: Vladimir Klimontovich >Assignee: Vladimir Klimontovich >Priority: Minor > Fix For: 0.5.1, 0.6.0 > > Attachments: corrupted_records_0.5.patch, > corrupted_records_0.5_ver2.patch, corrupted_records_trunk.patch, > corrupted_records_trunk_ver2.patch > > > When deserializer throws an exception the whole map tasks fails (see > MapOperator.java file). It's not always an convenient behavior especially on > huge datasets where several corrupted lines could be a normal practice. > Proposed solution: > 1) Have a counter of corrupted records > 2) When a counter exceeds a limit (configurable via > hive.max.deserializer.errors property, 0 by default) throw an exception. > Otherwise just log and exception with WARN level. > Patches for 0.5 branch and trunk are attached -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1414) automatically invoke .hiverc init script
[ https://issues.apache.org/jira/browse/HIVE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880414#action_12880414 ] Edward Capriolo commented on HIVE-1414: --- Files automatically: ql sourced env[HIVE_HOME]/bin/.hiverc, property(user.home)/.hiverc. I think only the CLI needs these features. Users of hive service are accessing the session though code repetition is not a problem, the same is true with JDBC. CLI users get the most benefit from the .hiverc. What do you think? > automatically invoke .hiverc init script > > > Key: HIVE-1414 > URL: https://issues.apache.org/jira/browse/HIVE-1414 > Project: Hadoop Hive > Issue Type: Improvement > Components: Clients >Affects Versions: 0.5.0 >Reporter: John Sichi >Assignee: Edward Capriolo > Attachments: hive-1414-patch-1.txt > > > Similar to .bashrc but run Hive SQL commands. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1414) automatically invoke .hiverc init script
[ https://issues.apache.org/jira/browse/HIVE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1414: -- Attachment: hive-1414-patch-1.txt First attempt at patch. > automatically invoke .hiverc init script > > > Key: HIVE-1414 > URL: https://issues.apache.org/jira/browse/HIVE-1414 > Project: Hadoop Hive > Issue Type: Improvement > Components: Clients >Affects Versions: 0.5.0 >Reporter: John Sichi >Assignee: Edward Capriolo > Attachments: hive-1414-patch-1.txt > > > Similar to .bashrc but run Hive SQL commands. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-1414) automatically invoke .hiverc init script
[ https://issues.apache.org/jira/browse/HIVE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo reassigned HIVE-1414: - Assignee: Edward Capriolo > automatically invoke .hiverc init script > > > Key: HIVE-1414 > URL: https://issues.apache.org/jira/browse/HIVE-1414 > Project: Hadoop Hive > Issue Type: Improvement > Components: Clients >Affects Versions: 0.5.0 >Reporter: John Sichi >Assignee: Edward Capriolo > > Similar to .bashrc but run Hive SQL commands. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1135) Use Anakia for version controlled documentation
[ https://issues.apache.org/jira/browse/HIVE-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880303#action_12880303 ] Edward Capriolo commented on HIVE-1135: --- Great on ivy. As for the wiki I think we should just put a node at the top of the page that says "Do not edit me. Edit xdocs instead." For the pages we have migrated. I want to do like a page every other day so it should be done soon enough. I actually have commit access but I usually leave the commits up to the experts. Also since I worked on this ticket I really should not be the commit person. Anyone else? > Use Anakia for version controlled documentation > --- > > Key: HIVE-1135 > URL: https://issues.apache.org/jira/browse/HIVE-1135 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1135-3-patch.txt, hive-1135-4-patch.txt, > hive-1135-5-patch.txt, hive-1135-6-patch.txt, hive-1335-1.patch.txt, > hive-1335-2.patch.txt, jdom-1.1.jar, jdom-1.1.LICENSE, wtf.png > > > Currently the Hive Language Manual and many other critical pieces of > documentation are on the Hive wiki. > Right now we count on the author of a patch to follow up and add wiki > entries. While we do a decent job with this, new features can be missed. Or > using running older/newer branches can not locate relevant documentation for > their branch. > ..example of a perception I do not think we want to give off... > http://dev.hubspot.com/bid/30170/Who-Loves-the-Magic-Undocumented-Hive-Mapjoin-This-Guy > We should generate our documentation in the way hadoop & hbase does, inline > using forest. I would like to take the lead on this, but we need a lot of > consensus on doing this properly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1405) Implement a .hiverc startup file
[ https://issues.apache.org/jira/browse/HIVE-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880207#action_12880207 ] Edward Capriolo commented on HIVE-1405: --- I was thinking we just look for hive_rc in the users home directory and/or in hive_home/bin. If we find that file we have to read it line by line and process it just like other hive commands. We could restrict this to just set or add commands but there is no reason it could not have a full query. > Implement a .hiverc startup file > > > Key: HIVE-1405 > URL: https://issues.apache.org/jira/browse/HIVE-1405 > Project: Hadoop Hive > Issue Type: New Feature >Reporter: Jonathan Chang >Assignee: John Sichi > > When deploying hive, it would be nice to have a .hiverc file containing > statements that would be automatically run whenever hive is launched. This > way, we can automatically add JARs, create temporary functions, set flags, > etc. for all users quickly. > This should ideally be set up like .bashrc and the like with a global version > and a user-local version. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1405) Implement a .hiverc startup file
[
https://issues.apache.org/jira/browse/HIVE-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880053#action_12880053
]
Edward Capriolo commented on HIVE-1405:
---
{noformat}
[edw...@ec dist]$ echo show tables > a.sql
[edw...@ec dist]$ bin/hive
[edw...@ec dist]$ chmod a+x a.sql
[edw...@ec dist]$ bin/hive
Hive history file=/tmp/edward/hive_job_log_edward_201006172223_1189860304.txt
[edw...@ec dist]$ pwd
/mnt/data/hive/hive/build/dist
[edw...@ec dist]$ bin/hive
Hive history file=/tmp/edward/hive_job_log_edward_201006172223_310534855.txt
hive> ! /mnt/data/hive/hive/build/dist/a.sql;
/mnt/data/hive/hive/build/dist/a.sql: line 1: show: command not found
Command failed with exit code = 127
{noformat}
! seems to execute bash commands
Dont we want to execute hive commands inside hive like add jar
> Implement a .hiverc startup file
>
>
> Key: HIVE-1405
> URL: https://issues.apache.org/jira/browse/HIVE-1405
> Project: Hadoop Hive
> Issue Type: New Feature
>Reporter: Jonathan Chang
>Assignee: John Sichi
>
> When deploying hive, it would be nice to have a .hiverc file containing
> statements that would be automatically run whenever hive is launched. This
> way, we can automatically add JARs, create temporary functions, set flags,
> etc. for all users quickly.
> This should ideally be set up like .bashrc and the like with a global version
> and a user-local version.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1135) Use Anakia for version controlled documentation
[ https://issues.apache.org/jira/browse/HIVE-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1135: -- Attachment: hive-1135-5-patch.txt Added the join page as well. > Use Anakia for version controlled documentation > --- > > Key: HIVE-1135 > URL: https://issues.apache.org/jira/browse/HIVE-1135 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1135-3-patch.txt, hive-1135-4-patch.txt, > hive-1135-5-patch.txt, hive-1335-1.patch.txt, hive-1335-2.patch.txt, > jdom-1.1.jar, jdom-1.1.LICENSE, wtf.png > > > Currently the Hive Language Manual and many other critical pieces of > documentation are on the Hive wiki. > Right now we count on the author of a patch to follow up and add wiki > entries. While we do a decent job with this, new features can be missed. Or > using running older/newer branches can not locate relevant documentation for > their branch. > ..example of a perception I do not think we want to give off... > http://dev.hubspot.com/bid/30170/Who-Loves-the-Magic-Undocumented-Hive-Mapjoin-This-Guy > We should generate our documentation in the way hadoop & hbase does, inline > using forest. I would like to take the lead on this, but we need a lot of > consensus on doing this properly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1405) Implement a .hiverc startup file
[ https://issues.apache.org/jira/browse/HIVE-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880028#action_12880028 ] Edward Capriolo commented on HIVE-1405: --- I like Carl's approach. The entire point of the hiverc is not to explicitly have to do invoke anything explicit to add jars. > Implement a .hiverc startup file > > > Key: HIVE-1405 > URL: https://issues.apache.org/jira/browse/HIVE-1405 > Project: Hadoop Hive > Issue Type: New Feature >Reporter: Jonathan Chang >Assignee: John Sichi > > When deploying hive, it would be nice to have a .hiverc file containing > statements that would be automatically run whenever hive is launched. This > way, we can automatically add JARs, create temporary functions, set flags, > etc. for all users quickly. > This should ideally be set up like .bashrc and the like with a global version > and a user-local version. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1135) Use Anakia for version controlled documentation
[ https://issues.apache.org/jira/browse/HIVE-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1135: -- Attachment: wtf.png Cool on adding the logoBut something went wrong here. Unless I applied the patch from the left table looks wrong now. check the screen shot. > Use Anakia for version controlled documentation > --- > > Key: HIVE-1135 > URL: https://issues.apache.org/jira/browse/HIVE-1135 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1135-3-patch.txt, hive-1135-4-patch.txt, > hive-1335-1.patch.txt, hive-1335-2.patch.txt, jdom-1.1.jar, jdom-1.1.LICENSE, > wtf.png > > > Currently the Hive Language Manual and many other critical pieces of > documentation are on the Hive wiki. > Right now we count on the author of a patch to follow up and add wiki > entries. While we do a decent job with this, new features can be missed. Or > using running older/newer branches can not locate relevant documentation for > their branch. > ..example of a perception I do not think we want to give off... > http://dev.hubspot.com/bid/30170/Who-Loves-the-Magic-Undocumented-Hive-Mapjoin-This-Guy > We should generate our documentation in the way hadoop & hbase does, inline > using forest. I would like to take the lead on this, but we need a lot of > consensus on doing this properly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1135) Move hive language manual and tutorial to version control
[ https://issues.apache.org/jira/browse/HIVE-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1135: -- Status: Patch Available (was: Open) > Move hive language manual and tutorial to version control > - > > Key: HIVE-1135 > URL: https://issues.apache.org/jira/browse/HIVE-1135 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1135-3-patch.txt, hive-1335-1.patch.txt, > hive-1335-2.patch.txt, jdom-1.1.jar, jdom-1.1.LICENSE > > > Currently the Hive Language Manual and many other critical pieces of > documentation are on the Hive wiki. > Right now we count on the author of a patch to follow up and add wiki > entries. While we do a decent job with this, new features can be missed. Or > using running older/newer branches can not locate relevant documentation for > their branch. > ..example of a perception I do not think we want to give off... > http://dev.hubspot.com/bid/30170/Who-Loves-the-Magic-Undocumented-Hive-Mapjoin-This-Guy > We should generate our documentation in the way hadoop & hbase does, inline > using forest. I would like to take the lead on this, but we need a lot of > consensus on doing this properly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1135) Move hive language manual and tutorial to version control
[ https://issues.apache.org/jira/browse/HIVE-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1135: -- Attachment: jdom-1.1.LICENSE > Move hive language manual and tutorial to version control > - > > Key: HIVE-1135 > URL: https://issues.apache.org/jira/browse/HIVE-1135 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1135-3-patch.txt, hive-1335-1.patch.txt, > hive-1335-2.patch.txt, jdom-1.1.jar, jdom-1.1.LICENSE > > > Currently the Hive Language Manual and many other critical pieces of > documentation are on the Hive wiki. > Right now we count on the author of a patch to follow up and add wiki > entries. While we do a decent job with this, new features can be missed. Or > using running older/newer branches can not locate relevant documentation for > their branch. > ..example of a perception I do not think we want to give off... > http://dev.hubspot.com/bid/30170/Who-Loves-the-Magic-Undocumented-Hive-Mapjoin-This-Guy > We should generate our documentation in the way hadoop & hbase does, inline > using forest. I would like to take the lead on this, but we need a lot of > consensus on doing this properly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1135) Move hive language manual and tutorial to version control
[ https://issues.apache.org/jira/browse/HIVE-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1135: -- Attachment: hive-1135-3-patch.txt Fixed all items. > Move hive language manual and tutorial to version control > - > > Key: HIVE-1135 > URL: https://issues.apache.org/jira/browse/HIVE-1135 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1135-3-patch.txt, hive-1335-1.patch.txt, > hive-1335-2.patch.txt, jdom-1.1.jar > > > Currently the Hive Language Manual and many other critical pieces of > documentation are on the Hive wiki. > Right now we count on the author of a patch to follow up and add wiki > entries. While we do a decent job with this, new features can be missed. Or > using running older/newer branches can not locate relevant documentation for > their branch. > ..example of a perception I do not think we want to give off... > http://dev.hubspot.com/bid/30170/Who-Loves-the-Magic-Undocumented-Hive-Mapjoin-This-Guy > We should generate our documentation in the way hadoop & hbase does, inline > using forest. I would like to take the lead on this, but we need a lot of > consensus on doing this properly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1096) Hive Variables
[
https://issues.apache.org/jira/browse/HIVE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879233#action_12879233
]
Edward Capriolo commented on HIVE-1096:
---
This is my next target after the xdoc ticket. I should not be to far off.
> Hive Variables
> --
>
> Key: HIVE-1096
> URL: https://issues.apache.org/jira/browse/HIVE-1096
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
>Affects Versions: 0.5.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.6.0
>
> Attachments: 1096-9.diff, hive-1096-2.diff, hive-1096-7.diff,
> hive-1096-8.diff, hive-1096.diff
>
>
> From mailing list:
> --Amazon Elastic MapReduce version of Hive seems to have a nice feature
> called "Variables." Basically you can define a variable via command-line
> while invoking hive with -d DT=2009-12-09 and then refer to the variable via
> ${DT} within the hive queries. This could be extremely useful. I can't seem
> to find this feature even on trunk. Is this feature currently anywhere in the
> roadmap?--
> This could be implemented in many places.
> A simple place to put this is
> in Driver.compile or Driver.run we can do string substitutions at that level,
> and further downstream need not be effected.
> There could be some benefits to doing this further downstream, parser,plan.
> but based on the simple needs we may not need to overthink this.
> I will get started on implementing in compile unless someone wants to discuss
> this more.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1255) Add mathamatical UDFs PI, E, degrees, radians, tan, sign, and atan
[ https://issues.apache.org/jira/browse/HIVE-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1255: -- Attachment: hive-1255-patch-4.txt show_functions.q is patched > Add mathamatical UDFs PI, E, degrees, radians, tan, sign, and atan > -- > > Key: HIVE-1255 > URL: https://issues.apache.org/jira/browse/HIVE-1255 > Project: Hadoop Hive > Issue Type: New Feature > Components: Query Processor >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1255-patch-2.txt, hive-1255-patch-3.txt, > hive-1255-patch-4.txt, hive-1255-patch.txt > > > Add support for PI, E, degrees, radians, tan, sign and atan -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1255) Add mathamatical UDFs PI, E, degrees, radians, tan, sign, and atan
[ https://issues.apache.org/jira/browse/HIVE-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1255: -- Status: Patch Available (was: Open) > Add mathamatical UDFs PI, E, degrees, radians, tan, sign, and atan > -- > > Key: HIVE-1255 > URL: https://issues.apache.org/jira/browse/HIVE-1255 > Project: Hadoop Hive > Issue Type: New Feature > Components: Query Processor >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1255-patch-2.txt, hive-1255-patch-3.txt, > hive-1255-patch-4.txt, hive-1255-patch.txt > > > Add support for PI, E, degrees, radians, tan, sign and atan -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1135) Move hive language manual and all wiki based documentation to forest
[ https://issues.apache.org/jira/browse/HIVE-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878818#action_12878818 ] Edward Capriolo commented on HIVE-1135: --- >JDOM ivy. Right now most/all the stuff in /lib DOES NOT comes from ivy. We should open up a another ticket and convert the entire project to ivy. > Move hive language manual and all wiki based documentation to forest > > > Key: HIVE-1135 > URL: https://issues.apache.org/jira/browse/HIVE-1135 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1335-1.patch.txt, hive-1335-2.patch.txt, > jdom-1.1.jar > > > Currently the Hive Language Manual and many other critical pieces of > documentation are on the Hive wiki. > Right now we count on the author of a patch to follow up and add wiki > entries. While we do a decent job with this, new features can be missed. Or > using running older/newer branches can not locate relevant documentation for > their branch. > ..example of a perception I do not think we want to give off... > http://dev.hubspot.com/bid/30170/Who-Loves-the-Magic-Undocumented-Hive-Mapjoin-This-Guy > We should generate our documentation in the way hadoop & hbase does, inline > using forest. I would like to take the lead on this, but we need a lot of > consensus on doing this properly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1135) Move hive language manual and all wiki based documentation to forest
[ https://issues.apache.org/jira/browse/HIVE-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878816#action_12878816 ] Edward Capriolo commented on HIVE-1135: --- FMI. What is the hbase review board? > Stash this stuff under docs/ instead of creating another top level directory > (xdocs/) It seems like xdocs is convention, also did not want to step on whatever is in docs. I will see if they both can live in docs happily. >JDOM ivy. Right now most/all the stuff in /lib comes from ivy. We should open up a another ticket and convert the entire project to ivy. >velocity.log Yes my local version (next patch already fixes that) > * Limit the initial import to the convents of the Hive Language Manual. I > think some things should actually stay on the wiki, but the language manual > is definitely one of those things that we want to have in VCS. I agree the initial import should come from the Hive Language Manual only. To me wiki just screems, "I did not have time to write a full complete doc." Generalization coming: 99% of the things in the wiki should be in xdocs. Users only want one place for authoritative information. Wikis and xdoc will fall out of sync, confusion follows. > Move hive language manual and all wiki based documentation to forest > > > Key: HIVE-1135 > URL: https://issues.apache.org/jira/browse/HIVE-1135 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1335-1.patch.txt, hive-1335-2.patch.txt, > jdom-1.1.jar > > > Currently the Hive Language Manual and many other critical pieces of > documentation are on the Hive wiki. > Right now we count on the author of a patch to follow up and add wiki > entries. While we do a decent job with this, new features can be missed. Or > using running older/newer branches can not locate relevant documentation for > their branch. > ..example of a perception I do not think we want to give off... > http://dev.hubspot.com/bid/30170/Who-Loves-the-Magic-Undocumented-Hive-Mapjoin-This-Guy > We should generate our documentation in the way hadoop & hbase does, inline > using forest. I would like to take the lead on this, but we need a lot of > consensus on doing this properly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1135) Move hive language manual and all wiki based documentation to forest
[ https://issues.apache.org/jira/browse/HIVE-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1135: -- Attachment: hive-1335-2.patch.txt Edited the build.xml a bit to deal with some incorrect paths. Also added the HiveDataDefinitionStatements wiki page for reference. Is everyone ok with using anakia and this structure. I would like to get this cleaned up with the current docs in place. Then I will do some follow up tickets and add some wiki pages, let me know if everyone is happy with the overall xdocs->docs process. > Move hive language manual and all wiki based documentation to forest > > > Key: HIVE-1135 > URL: https://issues.apache.org/jira/browse/HIVE-1135 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1335-1.patch.txt, hive-1335-2.patch.txt, > jdom-1.1.jar > > > Currently the Hive Language Manual and many other critical pieces of > documentation are on the Hive wiki. > Right now we count on the author of a patch to follow up and add wiki > entries. While we do a decent job with this, new features can be missed. Or > using running older/newer branches can not locate relevant documentation for > their branch. > ..example of a perception I do not think we want to give off... > http://dev.hubspot.com/bid/30170/Who-Loves-the-Magic-Undocumented-Hive-Mapjoin-This-Guy > We should generate our documentation in the way hadoop & hbase does, inline > using forest. I would like to take the lead on this, but we need a lot of > consensus on doing this properly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1135) Move hive language manual and all wiki based documentation to forest
[ https://issues.apache.org/jira/browse/HIVE-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1135: -- Status: Patch Available (was: Open) > Move hive language manual and all wiki based documentation to forest > > > Key: HIVE-1135 > URL: https://issues.apache.org/jira/browse/HIVE-1135 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1335-1.patch.txt, jdom-1.1.jar > > > Currently the Hive Language Manual and many other critical pieces of > documentation are on the Hive wiki. > Right now we count on the author of a patch to follow up and add wiki > entries. While we do a decent job with this, new features can be missed. Or > using running older/newer branches can not locate relevant documentation for > their branch. > ..example of a perception I do not think we want to give off... > http://dev.hubspot.com/bid/30170/Who-Loves-the-Magic-Undocumented-Hive-Mapjoin-This-Guy > We should generate our documentation in the way hadoop & hbase does, inline > using forest. I would like to take the lead on this, but we need a lot of > consensus on doing this properly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1135) Move hive language manual and all wiki based documentation to forest
[ https://issues.apache.org/jira/browse/HIVE-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1135: -- Attachment: hive-1335-1.patch.txt Patch, docs command beings anakia from xdocs directory. > Move hive language manual and all wiki based documentation to forest > > > Key: HIVE-1135 > URL: https://issues.apache.org/jira/browse/HIVE-1135 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1335-1.patch.txt, jdom-1.1.jar > > > Currently the Hive Language Manual and many other critical pieces of > documentation are on the Hive wiki. > Right now we count on the author of a patch to follow up and add wiki > entries. While we do a decent job with this, new features can be missed. Or > using running older/newer branches can not locate relevant documentation for > their branch. > ..example of a perception I do not think we want to give off... > http://dev.hubspot.com/bid/30170/Who-Loves-the-Magic-Undocumented-Hive-Mapjoin-This-Guy > We should generate our documentation in the way hadoop & hbase does, inline > using forest. I would like to take the lead on this, but we need a lot of > consensus on doing this properly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1135) Move hive language manual and all wiki based documentation to forest
[ https://issues.apache.org/jira/browse/HIVE-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1135: -- Fix Version/s: 0.6.0 Affects Version/s: 0.5.0 > Move hive language manual and all wiki based documentation to forest > > > Key: HIVE-1135 > URL: https://issues.apache.org/jira/browse/HIVE-1135 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: hive-1335-1.patch.txt, jdom-1.1.jar > > > Currently the Hive Language Manual and many other critical pieces of > documentation are on the Hive wiki. > Right now we count on the author of a patch to follow up and add wiki > entries. While we do a decent job with this, new features can be missed. Or > using running older/newer branches can not locate relevant documentation for > their branch. > ..example of a perception I do not think we want to give off... > http://dev.hubspot.com/bid/30170/Who-Loves-the-Magic-Undocumented-Hive-Mapjoin-This-Guy > We should generate our documentation in the way hadoop & hbase does, inline > using forest. I would like to take the lead on this, but we need a lot of > consensus on doing this properly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1135) Move hive language manual and all wiki based documentation to forest
[ https://issues.apache.org/jira/browse/HIVE-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1135: -- Attachment: jdom-1.1.jar > Move hive language manual and all wiki based documentation to forest > > > Key: HIVE-1135 > URL: https://issues.apache.org/jira/browse/HIVE-1135 > Project: Hadoop Hive > Issue Type: Task > Components: Documentation >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Attachments: jdom-1.1.jar > > > Currently the Hive Language Manual and many other critical pieces of > documentation are on the Hive wiki. > Right now we count on the author of a patch to follow up and add wiki > entries. While we do a decent job with this, new features can be missed. Or > using running older/newer branches can not locate relevant documentation for > their branch. > ..example of a perception I do not think we want to give off... > http://dev.hubspot.com/bid/30170/Who-Loves-the-Magic-Undocumented-Hive-Mapjoin-This-Guy > We should generate our documentation in the way hadoop & hbase does, inline > using forest. I would like to take the lead on this, but we need a lot of > consensus on doing this properly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1401) Web Interface can ony browse default
[ https://issues.apache.org/jira/browse/HIVE-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1401: -- Status: Patch Available (was: Open) This patch is to support hive getting multiple catalog support. > Web Interface can ony browse default > > > Key: HIVE-1401 > URL: https://issues.apache.org/jira/browse/HIVE-1401 > Project: Hadoop Hive > Issue Type: New Feature > Components: Web UI >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: HIVE-1401-1-patch.txt > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1401) Web Interface can ony browse default
[ https://issues.apache.org/jira/browse/HIVE-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Capriolo updated HIVE-1401: -- Attachment: HIVE-1401-1-patch.txt > Web Interface can ony browse default > > > Key: HIVE-1401 > URL: https://issues.apache.org/jira/browse/HIVE-1401 > Project: Hadoop Hive > Issue Type: New Feature > Components: Web UI >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: HIVE-1401-1-patch.txt > > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1401) Web Interface can ony browse default
Web Interface can ony browse default Key: HIVE-1401 URL: https://issues.apache.org/jira/browse/HIVE-1401 Project: Hadoop Hive Issue Type: New Feature Components: Web UI Affects Versions: 0.5.0 Reporter: Edward Capriolo Assignee: Edward Capriolo Fix For: 0.6.0 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1397) histogram() UDAF for a numerical column
[ https://issues.apache.org/jira/browse/HIVE-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12877220#action_12877220 ] Edward Capriolo commented on HIVE-1397: --- Looks great. Can not wait. > histogram() UDAF for a numerical column > --- > > Key: HIVE-1397 > URL: https://issues.apache.org/jira/browse/HIVE-1397 > Project: Hadoop Hive > Issue Type: New Feature > Components: Query Processor >Affects Versions: 0.6.0 >Reporter: Mayank Lahiri >Assignee: Mayank Lahiri > Fix For: 0.6.0 > > > A histogram() UDAF to generate an approximate histogram of a numerical (byte, > short, double, long, etc.) column. The result is returned as a map of (x,y) > histogram pairs, and can be plotted in Gnuplot using impulses (for example). > The algorithm is currently adapted from "A streaming parallel decision tree > algorithm" by Ben-Haim and Tom-Tov, JMLR 11 (2010), and uses space > proportional to the number of histogram bins specified. It has no > approximation guarantees, but seems to work well when there is a lot of data > and a large number (e.g. 50-100) of histogram bins specified. > A typical call might be: > SELECT histogram(val, 10) FROM some_table; > where the result would be a histogram with 10 bins, returned as a Hive map > object. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1386) HiveQL SQL Compliance (Umbrella)
[ https://issues.apache.org/jira/browse/HIVE-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12877204#action_12877204 ] Edward Capriolo commented on HIVE-1386: --- As a side note maybe with something total order partitioner we can do a true map/reduce order by. > HiveQL SQL Compliance (Umbrella) > > > Key: HIVE-1386 > URL: https://issues.apache.org/jira/browse/HIVE-1386 > Project: Hadoop Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Carl Steinbach > > This is an umbrella ticket to track work related to HiveQL compliance with > the SQL standard, e.g. supported query syntax, data types, views, catalog > access, etc. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1373) Missing connection pool plugin in Eclipse classpath
[
https://issues.apache.org/jira/browse/HIVE-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12876250#action_12876250
]
Edward Capriolo commented on HIVE-1373:
---
{quote}
1 copy is anyway done from lib to dist/lib for these jars. If we go directly to
ivy we would copy things from the ivy cache to dist/lib. So the number of
copies in the build process
would remain the same, no? There is of course the first time overhead of
downloading these jars from their repos to the ivy cache.
{quote}
I follow what you are thinking. Currently the code I did takes specifc jars
from metastore ivy dowloads. We could probably have ivy download directly to
build/lib. I just think we should watch to make sure many unneeded jars do not
appear.
> Missing connection pool plugin in Eclipse classpath
> ---
>
> Key: HIVE-1373
> URL: https://issues.apache.org/jira/browse/HIVE-1373
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Environment: Eclipse, Linux
>Reporter: Vinithra Varadharajan
>Assignee: Vinithra Varadharajan
> Attachments: HIVE-1373.patch
>
>
> In a recent checkin, connection pool dependency was introduced but eclipse
> .classpath file was not updated. This causes launch configurations from
> within Eclipse to fail.
> {code}
> hive> show tables;
> show tables;
> 10/05/26 14:59:46 INFO parse.ParseDriver: Parsing command: show tables
> 10/05/26 14:59:46 INFO parse.ParseDriver: Parse Completed
> 10/05/26 14:59:46 INFO ql.Driver: Semantic Analysis Completed
> 10/05/26 14:59:46 INFO ql.Driver: Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from
> deserializer)], properties:null)
> 10/05/26 14:59:46 INFO ql.Driver: query plan =
> file:/tmp/vinithra/hive_2010-05-26_14-59-46_058_1636674338194744357/queryplan.xml
> 10/05/26 14:59:46 INFO ql.Driver: Starting command: show tables
> 10/05/26 14:59:46 INFO metastore.HiveMetaStore: 0: Opening raw store with
> implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
> 10/05/26 14:59:46 INFO metastore.ObjectStore: ObjectStore, initialize called
> FAILED: Error in metadata: javax.jdo.JDOFatalInternalException: Error
> creating transactional connection factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> 10/05/26 14:59:47 ERROR exec.DDLTask: FAILED: Error in metadata:
> javax.jdo.JDOFatalInternalException: Error creating transactional connection
> factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> org.apache.hadoop.hive.ql.metadata.HiveException:
> javax.jdo.JDOFatalInternalException: Error creating transactional connection
> factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> at org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:491)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.getTablesByPattern(Hive.java:472)
> at org.apache.hadoop.hive.ql.metadata.Hive.getAllTables(Hive.java:458)
> at org.apache.hadoop.hive.ql.exec.DDLTask.showTables(DDLTask.java:504)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:176)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:631)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:504)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:382)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:303)
> Caused by: javax.jdo.JDOFatalInternalException: Error creating transactional
> connection factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> at
> org.datanucleus.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:395)
> at
> org.datanucleus.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:547)
> at
> org.datanucleus.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:175)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at javax.jdo.JDOHelper$16.run(JDOHelper.java:1956)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.jdo.JDOHelper.invoke(JDOHelper.java:
[jira] Commented: (HIVE-1369) LazySimpleSerDe should be able to read classes that support some form of toString()
[ https://issues.apache.org/jira/browse/HIVE-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12875909#action_12875909 ] Edward Capriolo commented on HIVE-1369: --- This seems interesting. In some cases toString() output can change between java versions for some objects. Do we need to compensate for that. > LazySimpleSerDe should be able to read classes that support some form of > toString() > --- > > Key: HIVE-1369 > URL: https://issues.apache.org/jira/browse/HIVE-1369 > Project: Hadoop Hive > Issue Type: Improvement >Reporter: Alex Kozlov >Assignee: Alex Kozlov >Priority: Minor > Attachments: HIVE-1369.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > Currently LazySimpleSerDe is able to deserialize only BytesWritable or Text > objects. It should be pretty easy to extend the class to read any object > that implements toString() method. > Ideas or concerns? > Alex K -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1265) Function Registry should should auto-detect UDFs from UDF Description
[
https://issues.apache.org/jira/browse/HIVE-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12872996#action_12872996
]
Edward Capriolo commented on HIVE-1265:
---
{noformat}
public static List getClassesForPackage(String packageName, Class
classType){
+List matchingClasses = new ArrayList();
+File directory = null;
+System.out.println(packageName.replace('.', File.separatorChar));
+URL u = Thread.currentThread().getContextClassLoader()
+//URL u = new Object().getClass().c
+.getResource(packageName.replace('.', File.separatorChar));
{noformat}
It seems like this section of code only picks up classes in
ql/test/org.apache.hadoop.hive.ql.udf. This must have something to do with
classloaders/threads/ and getResource(). It seems like getResource is unaware
that two folders could be responsible for the same resource. Or I have to find
a better way to do this.
> Function Registry should should auto-detect UDFs from UDF Description
> --
>
> Key: HIVE-1265
> URL: https://issues.apache.org/jira/browse/HIVE-1265
> Project: Hadoop Hive
> Issue Type: Improvement
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Attachments: hive-1265-patch.diff
>
>
> We should be able to register functions dynamically.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1373) Missing connection pool plugin in Eclipse classpath
[
https://issues.apache.org/jira/browse/HIVE-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12872337#action_12872337
]
Edward Capriolo commented on HIVE-1373:
---
I was thinking to move everything that came from ivy to build lib. I see the
benefit, but I saw this technique adding more copies and moves into the ant
process. Try different approaches. I found none of them were better then the
next. All involved doing more work hear and less there, or changing this
classpath insteasd of putting a file into X folder. I was kinda confused on the
"best way" to handle that. I would be interested to swee what you come up with.
> Missing connection pool plugin in Eclipse classpath
> ---
>
> Key: HIVE-1373
> URL: https://issues.apache.org/jira/browse/HIVE-1373
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Environment: Eclipse, Linux
>Reporter: Vinithra Varadharajan
>Assignee: Vinithra Varadharajan
>Priority: Minor
> Attachments: HIVE-1373.patch
>
>
> In a recent checkin, connection pool dependency was introduced but eclipse
> .classpath file was not updated. This causes launch configurations from
> within Eclipse to fail.
> {code}
> hive> show tables;
> show tables;
> 10/05/26 14:59:46 INFO parse.ParseDriver: Parsing command: show tables
> 10/05/26 14:59:46 INFO parse.ParseDriver: Parse Completed
> 10/05/26 14:59:46 INFO ql.Driver: Semantic Analysis Completed
> 10/05/26 14:59:46 INFO ql.Driver: Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from
> deserializer)], properties:null)
> 10/05/26 14:59:46 INFO ql.Driver: query plan =
> file:/tmp/vinithra/hive_2010-05-26_14-59-46_058_1636674338194744357/queryplan.xml
> 10/05/26 14:59:46 INFO ql.Driver: Starting command: show tables
> 10/05/26 14:59:46 INFO metastore.HiveMetaStore: 0: Opening raw store with
> implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
> 10/05/26 14:59:46 INFO metastore.ObjectStore: ObjectStore, initialize called
> FAILED: Error in metadata: javax.jdo.JDOFatalInternalException: Error
> creating transactional connection factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> 10/05/26 14:59:47 ERROR exec.DDLTask: FAILED: Error in metadata:
> javax.jdo.JDOFatalInternalException: Error creating transactional connection
> factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> org.apache.hadoop.hive.ql.metadata.HiveException:
> javax.jdo.JDOFatalInternalException: Error creating transactional connection
> factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> at org.apache.hadoop.hive.ql.metadata.Hive.getTablesForDb(Hive.java:491)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.getTablesByPattern(Hive.java:472)
> at org.apache.hadoop.hive.ql.metadata.Hive.getAllTables(Hive.java:458)
> at org.apache.hadoop.hive.ql.exec.DDLTask.showTables(DDLTask.java:504)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:176)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:631)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:504)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:382)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:303)
> Caused by: javax.jdo.JDOFatalInternalException: Error creating transactional
> connection factory
> NestedThrowables:
> java.lang.reflect.InvocationTargetException
> at
> org.datanucleus.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:395)
> at
> org.datanucleus.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:547)
> at
> org.datanucleus.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:175)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at javax.jdo.JDOHelper$16.run(JDOHelper.java:1956)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.jdo.JDOHelper.invoke(JDOHelper.java:1951)
> at
> javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImp
[jira] Commented: (HIVE-802) Bug in DataNucleus prevents Hive from building if inside a dir with '+' in it
[ https://issues.apache.org/jira/browse/HIVE-802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12872244#action_12872244 ] Edward Capriolo commented on HIVE-802: -- I just did a patch that adds connection pooling to DataNucleas. (Sorry that I jumped ahead of you). It should be easy to update now just bump the versions in metastore/ivy.xml. Please make sure the version you pick works with the connect pooling libs, as ivy fetches versions and dependants that do not work well together. > Bug in DataNucleus prevents Hive from building if inside a dir with '+' in it > - > > Key: HIVE-802 > URL: https://issues.apache.org/jira/browse/HIVE-802 > Project: Hadoop Hive > Issue Type: Bug > Components: Build Infrastructure >Reporter: Todd Lipcon >Assignee: Arvind Prabhakar > > There's a bug in DataNucleus that causes this issue: > http://www.jpox.org/servlet/jira/browse/NUCCORE-371 > To reproduce, simply put your hive source tree in a directory that contains a > '+' character. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1096) Hive Variables
[
https://issues.apache.org/jira/browse/HIVE-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12870480#action_12870480
]
Edward Capriolo commented on HIVE-1096:
---
I am back on this one. Keep your eye out for the next patch.
> Hive Variables
> --
>
> Key: HIVE-1096
> URL: https://issues.apache.org/jira/browse/HIVE-1096
> Project: Hadoop Hive
> Issue Type: New Feature
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Attachments: 1096-9.diff, hive-1096-2.diff, hive-1096-7.diff,
> hive-1096-8.diff, hive-1096.diff
>
>
> From mailing list:
> --Amazon Elastic MapReduce version of Hive seems to have a nice feature
> called "Variables." Basically you can define a variable via command-line
> while invoking hive with -d DT=2009-12-09 and then refer to the variable via
> ${DT} within the hive queries. This could be extremely useful. I can't seem
> to find this feature even on trunk. Is this feature currently anywhere in the
> roadmap?--
> This could be implemented in many places.
> A simple place to put this is
> in Driver.compile or Driver.run we can do string substitutions at that level,
> and further downstream need not be effected.
> There could be some benefits to doing this further downstream, parser,plan.
> but based on the simple needs we may not need to overthink this.
> I will get started on implementing in compile unless someone wants to discuss
> this more.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-471) A UDF for simple reflection
[
https://issues.apache.org/jira/browse/HIVE-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward Capriolo updated HIVE-471:
-
Fix Version/s: 0.6.0
Affects Version/s: 0.5.1
(was: 0.6.0)
Should be good for trunk.
> A UDF for simple reflection
> ---
>
> Key: HIVE-471
> URL: https://issues.apache.org/jira/browse/HIVE-471
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
>Affects Versions: 0.5.1
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Fix For: 0.6.0
>
> Attachments: hive-471-gen.diff, HIVE-471.1.patch, HIVE-471.2.patch,
> HIVE-471.3.patch, hive-471.diff
>
>
> There are many methods in java that are static and have no arguments or can
> be invoked with one simple parameter. More complicated functions will require
> a UDF but one generic one can work as a poor-mans UDF.
> {noformat}
> SELECT reflect("java.lang.String", "valueOf", 1), reflect("java.lang.String",
> "isEmpty")
> FROM src LIMIT 1;
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1335) DataNucleus should use connection pooling
[ https://issues.apache.org/jira/browse/HIVE-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12870477#action_12870477 ] Edward Capriolo commented on HIVE-1335: --- Can we go +1? > DataNucleus should use connection pooling > - > > Key: HIVE-1335 > URL: https://issues.apache.org/jira/browse/HIVE-1335 > Project: Hadoop Hive > Issue Type: Improvement >Affects Versions: 0.5.0 >Reporter: Edward Capriolo >Assignee: Edward Capriolo > Fix For: 0.6.0 > > Attachments: commons-dbcp-1.2.2.jar, commons-dbcp.LICENSE, > commons-pool-1.2.jar, commons-pool.LICENSE, > datanucleus-connectionpool-1.0.2.jar, datanucleus-connectionpool.LICENSE, > hive-1335-1.patch.txt, hive-1335-2.patch.txt, hive-1335-3.patch.txt, > hive-1335.patch.txt > > > Currently each Data Nucleus operation disconnects and reconnects to the > MetaStore over jdbc. Queries fail to even explain properly in cases where a > table has many partitions. This is fixed by enabling one parameter and > including several jars. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
