[jira] Updated: (HIVE-1405) hive command line option -i to run an init file before other SQL commands

[John Sichi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sichi updated HIVE-1405:
-

   Status: Patch Available  (was: Open)
Affects Version/s: 0.5.0
Fix Version/s: 0.6.0

> hive command line option -i to run an init file before other SQL commands
> -
>
> Key: HIVE-1405
> URL: https://issues.apache.org/jira/browse/HIVE-1405
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Clients
>Affects Versions: 0.5.0
>Reporter: Jonathan Chang
>Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1405.1.patch
>
>
> When deploying hive, it would be nice to have a .hiverc file containing 
> statements that would be automatically run whenever hive is launched.  This 
> way, we can automatically add JARs, create temporary functions, set flags, 
> etc. for all users quickly. 
> This should ideally be set up like .bashrc and the like with a global version 
> and a user-local version.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1414) automatically invoke .hiverc init script

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880414#action_12880414
 ] 

Edward Capriolo commented on HIVE-1414:
---

Files automatically: ql sourced env[HIVE_HOME]/bin/.hiverc, 
property(user.home)/.hiverc. I think only the CLI needs these features. Users 
of hive service are accessing the session though code repetition is not a 
problem, the same is true with JDBC. CLI users get the most benefit from the 
.hiverc. What do you think?

> automatically invoke .hiverc init script
> 
>
> Key: HIVE-1414
> URL: https://issues.apache.org/jira/browse/HIVE-1414
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Clients
>Affects Versions: 0.5.0
>Reporter: John Sichi
>Assignee: Edward Capriolo
> Attachments: hive-1414-patch-1.txt
>
>
> Similar to .bashrc but run Hive SQL commands.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1414) automatically invoke .hiverc init script

[Edward Capriolo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Capriolo updated HIVE-1414:
--

Attachment: hive-1414-patch-1.txt

First attempt at patch.

> automatically invoke .hiverc init script
> 
>
> Key: HIVE-1414
> URL: https://issues.apache.org/jira/browse/HIVE-1414
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Clients
>Affects Versions: 0.5.0
>Reporter: John Sichi
>Assignee: Edward Capriolo
> Attachments: hive-1414-patch-1.txt
>
>
> Similar to .bashrc but run Hive SQL commands.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Assigned: (HIVE-1414) automatically invoke .hiverc init script

[Edward Capriolo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward Capriolo reassigned HIVE-1414:
-

Assignee: Edward Capriolo

> automatically invoke .hiverc init script
> 
>
> Key: HIVE-1414
> URL: https://issues.apache.org/jira/browse/HIVE-1414
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Clients
>Affects Versions: 0.5.0
>Reporter: John Sichi
>Assignee: Edward Capriolo
>
> Similar to .bashrc but run Hive SQL commands.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1418) column pruning not working with lateral view

[He Yongqiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

He Yongqiang updated HIVE-1418:
---

   Status: Patch Available  (was: Open)
Affects Version/s: 0.5.0
Fix Version/s: 0.6.0

> column pruning not working with lateral view
> 
>
> Key: HIVE-1418
> URL: https://issues.apache.org/jira/browse/HIVE-1418
> Project: Hadoop Hive
>  Issue Type: Bug
>Affects Versions: 0.5.0
>Reporter: He Yongqiang
>Assignee: He Yongqiang
> Fix For: 0.6.0
>
> Attachments: hive.1418.1.patch
>
>
> select myCol from tmp_pyang_lv LATERAL VIEW explode(array(1,2,3)) myTab as 
> myCol limit 3;

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1418) column pruning not working with lateral view

[He Yongqiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

He Yongqiang updated HIVE-1418:
---

Attachment: hive.1418.1.patch

> column pruning not working with lateral view
> 
>
> Key: HIVE-1418
> URL: https://issues.apache.org/jira/browse/HIVE-1418
> Project: Hadoop Hive
>  Issue Type: Bug
>Affects Versions: 0.5.0
>Reporter: He Yongqiang
>Assignee: He Yongqiang
> Fix For: 0.6.0
>
> Attachments: hive.1418.1.patch
>
>
> select myCol from tmp_pyang_lv LATERAL VIEW explode(array(1,2,3)) myTab as 
> myCol limit 3;

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (HIVE-1418) column pruning not working with lateral view

[He Yongqiang (JIRA)]

column pruning not working with lateral view


 Key: HIVE-1418
 URL: https://issues.apache.org/jira/browse/HIVE-1418
 Project: Hadoop Hive
  Issue Type: Bug
Reporter: He Yongqiang
Assignee: He Yongqiang


select myCol from tmp_pyang_lv LATERAL VIEW explode(array(1,2,3)) myTab as 
myCol limit 3;

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (HIVE-1417) Archived partitions throw error with queries calling getContentSummary

[Paul Yang (JIRA)]

Archived partitions throw error with queries calling getContentSummary
--

 Key: HIVE-1417
 URL: https://issues.apache.org/jira/browse/HIVE-1417
 Project: Hadoop Hive
  Issue Type: Bug
Affects Versions: 0.6.0
Reporter: Paul Yang
Assignee: Paul Yang


Assuming you have a src table with a ds='1' partition that is archived, the 
following table will throw an exception

{code}
select count(1) from src where ds='1' group by key;
{code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (HIVE-1416) Dynamic partition inserts left empty files uncleaned in hadoop 0.17 local mode

[Ning Zhang (JIRA)]

Dynamic partition inserts left empty files uncleaned in hadoop 0.17 local mode
--

 Key: HIVE-1416
 URL: https://issues.apache.org/jira/browse/HIVE-1416
 Project: Hadoop Hive
  Issue Type: Bug
Reporter: Ning Zhang
Assignee: Ning Zhang


Hive parses the file name generated by tasks to figure out the task ID in order 
to generate files for empty buckets. Different hadoop versions and execution 
mode have different ways of naming  output files by mappers/reducers. We need 
to move the parsing code to shims. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1229) replace dependencies on HBase deprecated API

[John Sichi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880372#action_12880372
 ] 

John Sichi commented on HIVE-1229:
--

Hey Basab,

Looks great.  I have a few review questions before we commit this, but first, I 
need to you to submit a new patch rebased against latest trunk now that 
HIVE-1383 has been committed.  (Note that there were some more minor changes in 
between HIVE-1383.3.patch and HIVE-1383.4.patch.)

For the last patch, you had a typo in the filename (1129 instead of 1229); call 
the new one HIVE-1229.2.patch and then delete the old one to avoid confusion.



> replace dependencies on HBase deprecated API
> 
>
> Key: HIVE-1229
> URL: https://issues.apache.org/jira/browse/HIVE-1229
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: HBase Handler
>Affects Versions: 0.6.0
>Reporter: John Sichi
>Assignee: Basab Maulik
> Attachments: HIVE-1129.1.patch
>
>
> Some of these dependencies are on the old Hadoop mapred packages; others are 
> HBase-specific.  The former have to wait until the rest of Hive moves over to 
> the new Hadoop mapreduce package, but the HBase-specific ones don't have to 
> wait.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1405) hive command line option -i to run an init file before other SQL commands

[John Sichi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sichi updated HIVE-1405:
-

Attachment: HIVE-1405.1.patch

> hive command line option -i to run an init file before other SQL commands
> -
>
> Key: HIVE-1405
> URL: https://issues.apache.org/jira/browse/HIVE-1405
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Clients
>Reporter: Jonathan Chang
>Assignee: John Sichi
> Attachments: HIVE-1405.1.patch
>
>
> When deploying hive, it would be nice to have a .hiverc file containing 
> statements that would be automatically run whenever hive is launched.  This 
> way, we can automatically add JARs, create temporary functions, set flags, 
> etc. for all users quickly. 
> This should ideally be set up like .bashrc and the like with a global version 
> and a user-local version.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

evolving the GenericUDAF interface

[John Sichi]

There's a discussion in HIVE-287 regarding whether this is warranted for a 
particular feature:

https://issues.apache.org/jira/browse/HIVE-287

We have a proposal there for how to do it in a non-breaking fashion, while at 
the same time building in insulation for making it easier in the future.

If you have opinions, please chime in.

JVS

[jira] Commented: (HIVE-287) count distinct on multiple columns does not work

[John Sichi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880325#action_12880325
 ] 

John Sichi commented on HIVE-287:
-

The object creation and method call here are negligible since this is in the 
semantic analyzer context (not an execution codepath).


> count distinct on multiple columns does not work
> 
>
> Key: HIVE-287
> URL: https://issues.apache.org/jira/browse/HIVE-287
> Project: Hadoop Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Arvind Prabhakar
> Attachments: HIVE-287-1.patch, HIVE-287-2.patch, HIVE-287-3.patch
>
>
> The following query does not work:
> select count(distinct col1, col2) from Tbl

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1135) Use Anakia for version controlled documentation

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880303#action_12880303
 ] 

Edward Capriolo commented on HIVE-1135:
---

Great on ivy. 
As for the wiki I think we should just put a node at the top of the page that 
says "Do not edit me. Edit xdocs instead." For the pages we have migrated. I 
want to do like a page every other day so it should be done soon enough. I 
actually have commit access but I usually leave the commits up to the experts. 
Also since I worked on this ticket I really should not be the commit person. 
Anyone else?

> Use Anakia for version controlled documentation
> ---
>
> Key: HIVE-1135
> URL: https://issues.apache.org/jira/browse/HIVE-1135
> Project: Hadoop Hive
>  Issue Type: Task
>  Components: Documentation
>Affects Versions: 0.5.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.6.0
>
> Attachments: hive-1135-3-patch.txt, hive-1135-4-patch.txt, 
> hive-1135-5-patch.txt, hive-1135-6-patch.txt, hive-1335-1.patch.txt, 
> hive-1335-2.patch.txt, jdom-1.1.jar, jdom-1.1.LICENSE, wtf.png
>
>
> Currently the Hive Language Manual and many other critical pieces of 
> documentation are on the Hive wiki. 
> Right now we count on the author of a patch to follow up and add wiki 
> entries. While we do a decent job with this, new features can be missed. Or 
> using running older/newer branches can not locate relevant documentation for 
> their branch. 
> ..example of a perception I do not think we want to give off...
> http://dev.hubspot.com/bid/30170/Who-Loves-the-Magic-Undocumented-Hive-Mapjoin-This-Guy
> We should generate our documentation in the way hadoop & hbase does, inline 
> using forest. I would like to take the lead on this, but we need a lot of 
> consensus on doing this properly. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1135) Use Anakia for version controlled documentation

[Carl Steinbach (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880297#action_12880297
 ] 

Carl Steinbach commented on HIVE-1135:
--

@Ed:
* I updated the patch to use Ivy for managing the JDOM dependency.
* I'm not sure how to fix the image alignment issue. Let's defer worrying about 
this for a followup ticket.
* I will file a followup ticket that covers migrating the wiki docs over to 
version control. We need to do this in a consistent fashion in order to avoid 
missing any updates to the wiki that people make in the meantime. We also need 
to figure out how to push the generated docs to the website.
* I'm +1 for committing this patch. Can you take care of that?



> Use Anakia for version controlled documentation
> ---
>
> Key: HIVE-1135
> URL: https://issues.apache.org/jira/browse/HIVE-1135
> Project: Hadoop Hive
>  Issue Type: Task
>  Components: Documentation
>Affects Versions: 0.5.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.6.0
>
> Attachments: hive-1135-3-patch.txt, hive-1135-4-patch.txt, 
> hive-1135-5-patch.txt, hive-1135-6-patch.txt, hive-1335-1.patch.txt, 
> hive-1335-2.patch.txt, jdom-1.1.jar, jdom-1.1.LICENSE, wtf.png
>
>
> Currently the Hive Language Manual and many other critical pieces of 
> documentation are on the Hive wiki. 
> Right now we count on the author of a patch to follow up and add wiki 
> entries. While we do a decent job with this, new features can be missed. Or 
> using running older/newer branches can not locate relevant documentation for 
> their branch. 
> ..example of a perception I do not think we want to give off...
> http://dev.hubspot.com/bid/30170/Who-Loves-the-Magic-Undocumented-Hive-Mapjoin-This-Guy
> We should generate our documentation in the way hadoop & hbase does, inline 
> using forest. I would like to take the lead on this, but we need a lot of 
> consensus on doing this properly. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1405) hive command line option -i to run an init file before other SQL commands

[John Sichi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880296#action_12880296
 ] 

John Sichi commented on HIVE-1405:
--

The -i scripts will always run in silent (-S) mode.  However, if they encounter 
an error, CLI will print the message and abort.


> hive command line option -i to run an init file before other SQL commands
> -
>
> Key: HIVE-1405
> URL: https://issues.apache.org/jira/browse/HIVE-1405
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Clients
>Reporter: Jonathan Chang
>Assignee: John Sichi
>
> When deploying hive, it would be nice to have a .hiverc file containing 
> statements that would be automatically run whenever hive is launched.  This 
> way, we can automatically add JARs, create temporary functions, set flags, 
> etc. for all users quickly. 
> This should ideally be set up like .bashrc and the like with a global version 
> and a user-local version.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1135) Use Anakia for version controlled documentation

[Carl Steinbach (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach updated HIVE-1135:
-

Attachment: hive-1135-6-patch.txt

hive-1136-6-patch.txt:
* Use Ivy to manage jdom dependency for docs target.


> Use Anakia for version controlled documentation
> ---
>
> Key: HIVE-1135
> URL: https://issues.apache.org/jira/browse/HIVE-1135
> Project: Hadoop Hive
>  Issue Type: Task
>  Components: Documentation
>Affects Versions: 0.5.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
> Fix For: 0.6.0
>
> Attachments: hive-1135-3-patch.txt, hive-1135-4-patch.txt, 
> hive-1135-5-patch.txt, hive-1135-6-patch.txt, hive-1335-1.patch.txt, 
> hive-1335-2.patch.txt, jdom-1.1.jar, jdom-1.1.LICENSE, wtf.png
>
>
> Currently the Hive Language Manual and many other critical pieces of 
> documentation are on the Hive wiki. 
> Right now we count on the author of a patch to follow up and add wiki 
> entries. While we do a decent job with this, new features can be missed. Or 
> using running older/newer branches can not locate relevant documentation for 
> their branch. 
> ..example of a perception I do not think we want to give off...
> http://dev.hubspot.com/bid/30170/Who-Loves-the-Magic-Undocumented-Hive-Mapjoin-This-Guy
> We should generate our documentation in the way hadoop & hbase does, inline 
> using forest. I would like to take the lead on this, but we need a lot of 
> consensus on doing this properly. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1405) hive command line option -i to run an init file before other SQL commands

[John Sichi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880270#action_12880270
 ] 

John Sichi commented on HIVE-1405:
--

Created HIVE-1414 and HIVE-1415.

> hive command line option -i to run an init file before other SQL commands
> -
>
> Key: HIVE-1405
> URL: https://issues.apache.org/jira/browse/HIVE-1405
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Clients
>Reporter: Jonathan Chang
>Assignee: John Sichi
>
> When deploying hive, it would be nice to have a .hiverc file containing 
> statements that would be automatically run whenever hive is launched.  This 
> way, we can automatically add JARs, create temporary functions, set flags, 
> etc. for all users quickly. 
> This should ideally be set up like .bashrc and the like with a global version 
> and a user-local version.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (HIVE-1415) add CLI command for executing a SQL script

[John Sichi (JIRA)]

add CLI command for executing a SQL script
--

 Key: HIVE-1415
 URL: https://issues.apache.org/jira/browse/HIVE-1415
 Project: Hadoop Hive
  Issue Type: Improvement
  Components: Clients
Affects Versions: 0.5.0
Reporter: John Sichi


Suggestion in HIVE-1405 was "source", e.g.

source somescript.sql;



-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1414) automatically invoke .hiverc init script

[John Sichi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880267#action_12880267
 ] 

John Sichi commented on HIVE-1414:
--

I'm leaving this vague on purpose since there are a few issues which would need 
to be decided (what the file should be named, where it should be located, 
whether to invoke from CLI only or also other services when they start up, etc).

> automatically invoke .hiverc init script
> 
>
> Key: HIVE-1414
> URL: https://issues.apache.org/jira/browse/HIVE-1414
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Clients
>Affects Versions: 0.5.0
>Reporter: John Sichi
>
> Similar to .bashrc but run Hive SQL commands.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Created: (HIVE-1414) automatically invoke .hiverc init script

[John Sichi (JIRA)]

automatically invoke .hiverc init script


 Key: HIVE-1414
 URL: https://issues.apache.org/jira/browse/HIVE-1414
 Project: Hadoop Hive
  Issue Type: Improvement
  Components: Clients
Affects Versions: 0.5.0
Reporter: John Sichi


Similar to .bashrc but run Hive SQL commands.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1405) hive command line option -i to run an init file before other SQL commands

[John Sichi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880263#action_12880263
 ] 

John Sichi commented on HIVE-1405:
--

For this JIRA issue, I'm just going to do the -i option proposed above.  I'll 
create separate issues for automatically reading a .hiverc and for the source 
command.

> hive command line option -i to run an init file before other SQL commands
> -
>
> Key: HIVE-1405
> URL: https://issues.apache.org/jira/browse/HIVE-1405
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Clients
>Reporter: Jonathan Chang
>Assignee: John Sichi
>
> When deploying hive, it would be nice to have a .hiverc file containing 
> statements that would be automatically run whenever hive is launched.  This 
> way, we can automatically add JARs, create temporary functions, set flags, 
> etc. for all users quickly. 
> This should ideally be set up like .bashrc and the like with a global version 
> and a user-local version.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-1405) hive command line option -i to run an init file before other SQL commands

[John Sichi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sichi updated HIVE-1405:
-

Summary: hive command line option -i to run an init file before other 
SQL commands  (was: Implement a .hiverc startup file)
Component/s: Clients

> hive command line option -i to run an init file before other SQL commands
> -
>
> Key: HIVE-1405
> URL: https://issues.apache.org/jira/browse/HIVE-1405
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Clients
>Reporter: Jonathan Chang
>Assignee: John Sichi
>
> When deploying hive, it would be nice to have a .hiverc file containing 
> statements that would be automatically run whenever hive is launched.  This 
> way, we can automatically add JARs, create temporary functions, set flags, 
> etc. for all users quickly. 
> This should ideally be set up like .bashrc and the like with a global version 
> and a user-local version.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Build failed in Hudson: Hive-trunk-h0.17 #473

[Apache Hudson Server]

See 

Changes:

[nzhang] HIVE-1383. Allow HBase WAL to be disabled (John Sichi via Ning Zhang)

--
[...truncated 4130 lines...]
[junit] Done query: transform_ppr1.q
[junit] Begin query: transform_ppr2.q
[junit] diff -a -I file: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I 
lastAccessTime -I owner -I transient_lastDdlTime -I java.lang.RuntimeException 
-I at org -I at sun -I at java -I at junit -I Caused by: -I [.][.][.] [0-9]* 
more 

 

[junit] Done query: transform_ppr2.q
[junit] Begin query: type_cast_1.q
[junit] diff -a -I file: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I 
lastAccessTime -I owner -I transient_lastDdlTime -I java.lang.RuntimeException 
-I at org -I at sun -I at java -I at junit -I Caused by: -I [.][.][.] [0-9]* 
more 

 

[junit] Done query: type_cast_1.q
[junit] Begin query: udaf_avg.q
[junit] diff -a -I file: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I 
lastAccessTime -I owner -I transient_lastDdlTime -I java.lang.RuntimeException 
-I at org -I at sun -I at java -I at junit -I Caused by: -I [.][.][.] [0-9]* 
more 

 

[junit] Done query: udaf_avg.q
[junit] Begin query: udaf_count.q
[junit] diff -a -I file: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I 
lastAccessTime -I owner -I transient_lastDdlTime -I java.lang.RuntimeException 
-I at org -I at sun -I at java -I at junit -I Caused by: -I [.][.][.] [0-9]* 
more 

 

[junit] Done query: udaf_count.q
[junit] Begin query: udaf_histogram_numeric.q
[junit] diff -a -I file: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I 
lastAccessTime -I owner -I transient_lastDdlTime -I java.lang.RuntimeException 
-I at org -I at sun -I at java -I at junit -I Caused by: -I [.][.][.] [0-9]* 
more 

 

[junit] Done query: udaf_histogram_numeric.q
[junit] Begin query: udaf_max.q
[junit] diff -a -I file: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I 
lastAccessTime -I owner -I transient_lastDdlTime -I java.lang.RuntimeException 
-I at org -I at sun -I at java -I at junit -I Caused by: -I [.][.][.] [0-9]* 
more 

 

[junit] Done query: udaf_max.q
[junit] Begin query: udaf_min.q
[junit] diff -a -I file: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I 
lastAccessTime -I owner -I transient_lastDdlTime -I java.lang.RuntimeException 
-I at org -I at sun -I at java -I at junit -I Caused by: -I [.][.][.] [0-9]* 
more 

 

[junit] Done query: udaf_min.q
[junit] Begin query: udaf_number_format.q
[junit] diff -a -I file: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I 
lastAccessTime -I owner -I transient_lastDdlTime -I java.lang.RuntimeException 
-I at org -I at sun -I at java -I at junit -I Caused by: -I [.][.][.] [0-9]* 
more 

 

[junit] Done query: udaf_number_format.q
[junit] Begin query: udaf_std.q
[junit] diff -a -I file: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I 
lastAccessTime -I owner -I transient_lastDdlTime

[jira] Commented: (HIVE-1405) Implement a .hiverc startup file

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880207#action_12880207
 ] 

Edward Capriolo commented on HIVE-1405:
---

I was thinking we just look for hive_rc in the users home directory and/or in 
hive_home/bin. If we find that file we have to read it line by line and process 
it just like other hive commands. We could restrict this to just set or add 
commands but there is no reason it could not have a full query.

> Implement a .hiverc startup file
> 
>
> Key: HIVE-1405
> URL: https://issues.apache.org/jira/browse/HIVE-1405
> Project: Hadoop Hive
>  Issue Type: New Feature
>Reporter: Jonathan Chang
>Assignee: John Sichi
>
> When deploying hive, it would be nice to have a .hiverc file containing 
> statements that would be automatically run whenever hive is launched.  This 
> way, we can automatically add JARs, create temporary functions, set flags, 
> etc. for all users quickly. 
> This should ideally be set up like .bashrc and the like with a global version 
> and a user-local version.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1402) Add parallel ORDER BY to Hive

[Jeff Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880120#action_12880120
 ] 

Jeff Zhang commented on HIVE-1402:
--

Hi, I make a draft implementation for one special case. And it works, but since 
it is only for one special case, so I have some hard coding. I hope someone can 
give some help or instruction for the next step. 
One big problem of parallel ORDER BY is that the output  key type of ExecMapper 
is HiveKey, and it has been serialized by LazyBinarySerDe, so the original 
column type is lost here. But when do sampling and partition, I should use the 
original column type.

The following is my initial design.

1. During parse stage, extract one SampleOperator which has two children: 
TableScanOperator, SelectOperator ( I am not familiar with Hive Parse Stage, 
and the code is not clear for me, could anyone give some help or recommend some 
documentation about the Hive parser ? )

2. Modify the TotalOrderPartitioner.  Add a Deserializer to convert the HiveKey 
to its original column type. and deserialie the HiveKey in method 
getPartition(). 

Welcome any comments and help.



> Add parallel ORDER BY to Hive
> -
>
> Key: HIVE-1402
> URL: https://issues.apache.org/jira/browse/HIVE-1402
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Affects Versions: 0.5.0
>Reporter: Jeff Hammerbacher
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-943) Hive jdbc client - result is NULL when I run a query to select a large of data (with starting mapreduce)

[Vu Hoang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880095#action_12880095
 ] 

Vu Hoang commented on HIVE-943:
---

I used the method like this:
{code:java|title=ClientCodeTest|borderStyle=solid}
select("default", "select * from keyword_frequency", 
"/home/vhoang/hadoop/hive", true);
{code}

method definition select(String schema, String query, String path, boolean 
header)
{code:java|title=ClientCode.select(String schema, String query, String path, 
boolean header)|borderStyle=solid}
try
{
String url = METADATA_URL.replace(VAR_META_HOST, 
getConfig().get(HIVE_JDBC_SERVER)).replace(VAR_META_NAME, schema);
Class.forName(METADATA_DRIVER);

Log.debug(getLogString() + "url='" + url + "'");
Connection conn = DriverManager.getConnection(url, "", "");
Statement stmt = conn.createStatement(ResultSet.TYPE_SCROLL_SENSITIVE, 
ResultSet.CONCUR_UPDATABLE);

ResultSet recs = stmt.executeQuery(query);
ResultSetMetaData meta = recs.getMetaData();

if (path == null) print(recs, meta);
else write(recs, meta, new File(path), header);
}
catch (ClassNotFoundException ex) { error(ex); }
catch (SQLException ex) { error(ex); }
{code}

method definition print(ResultSet recs, ResultSetMetaData meta)
{code:java|title=ClientCode.print(ResultSet recs, ResultSetMetaData 
meta)|borderStyle=solid}
if (getLogString().equals(BLANK))
setLogString("HiveQuery.print()|");

init();
try
{
int columnSize = meta.getColumnCount();
Log.debug(getLogString() + "columnSize=" + columnSize);
for (int index = 0; index < columnSize; index++)
{
String columnName = meta.getColumnName(index + 1);
Log.debug(getLogString() + "column='" + columnName + "'");
if (index == columnSize - 1) System.out.print(columnName);
else System.out.print(columnName + StringConst.COMMA);
}
System.out.println();

while (recs.next())
{
for (int index = 0; index < columnSize; index++)
{
String record = recs.getString(index + 1);
Log.debug(getLogString() + "record='" + record + "'");
if (index == columnSize - 1) System.out.print(record);
else System.out.print(record + StringConst.COMMA);
}
System.out.println();
}
}
catch (SQLException ex) { error(ex); }
{code}

sorry about the delay feedback

> Hive jdbc client - result is NULL when I run a query to select a large of 
> data (with starting mapreduce)
> 
>
> Key: HIVE-943
> URL: https://issues.apache.org/jira/browse/HIVE-943
> Project: Hadoop Hive
>  Issue Type: Bug
>  Components: Clients
>Affects Versions: 0.4.0
>Reporter: Vu Hoang
> Fix For: 0.4.2
>
>
> - some main output messages i got from console:
> Total MapReduce jobs = 1
> 09/11/18 15:56:03 INFO ql.Driver: Total MapReduce jobs = 1
> 09/11/18 15:56:03 INFO exec.ExecDriver: BytesPerReducer=10 
> maxReducers=999 totalInputFileSize=1289288953
> Number of reduce tasks not specified. Estimated from input data size: 2
> 09/11/18 15:56:03 INFO exec.ExecDriver: Number of reduce tasks not specified. 
> Estimated from input data size: 2
> In order to change the average load for a reducer (in bytes):
> 09/11/18 15:56:03 INFO exec.ExecDriver: In order to change the average load 
> for a reducer (in bytes):
>   set hive.exec.reducers.bytes.per.reducer=
> 09/11/18 15:56:03 INFO exec.ExecDriver:   set 
> hive.exec.reducers.bytes.per.reducer=
> In order to limit the maximum number of reducers:
> 09/11/18 15:56:03 INFO exec.ExecDriver: In order to limit the maximum number 
> of reducers:
>   set hive.exec.reducers.max=
> 09/11/18 15:56:03 INFO exec.ExecDriver:   set hive.exec.reducers.max=
> In order to set a constant number of reducers:
> 09/11/18 15:56:03 INFO exec.ExecDriver: In order to set a constant number of 
> reducers:
>   set mapred.reduce.tasks=
> 09/11/18 15:56:03 INFO exec.ExecDriver:   set mapred.reduce.tasks=
> 09/11/18 15:56:03 INFO exec.ExecDriver: Using 
> org.apache.hadoop.hive.ql.io.HiveInputFormat
> Starting Job = job_200911122011_0639, Tracking URL = 
> http://**/jobdetails.jsp?jobid=job_200911122011_0639
> 09/11/18 15:56:04 INFO exec.ExecDriver: Starting Job = job_200911122011_0639, 
> Tracking URL = http://**/jobdetails.jsp?jobid=job_200911122011_0639
> Kill Command = /data/hadoop-hive/bin/../bin/hadoop job  
> -Dmapred.job.tracker=** -kill job_200911122011_0639