Re: Hive in maven

2010-01-19 Thread Zheng Shao 
This is a good idea. Can you point us to some references on how to upload it
to maven?

Zheng

On Mon, Jan 18, 2010 at 1:20 PM, Gerrit wrote:

> Hi guys,
>
> Would it be possible to add the hive jars to the main maven repo? If
> there is not objections I can make the request to the main repo if you
> agree.
>
> The reason for this need is that I've created a Loader for the pig
> project to read HiveRCTables
> (https://issues.apache.org/jira/browse/PIG-1117) and currently use ant
> to directly download the libraries from the apache site using:
>  src="${apache.dist.site}/${hive.groupId}/${hive.artifactId}/${hive.artifactId}-${hive.version}/${hive.artifactId}-${hive.version}-hadoop-${hadoop.version}-bin.tar.gz"
> dest="lib-hivedeps/${hive.artifactId}-${hive.version}-hadoop-${hadoop.version}-bin.tar.gz"/>
>
> I would much prefer using ivy or maven and it makes this much cleaner.
>
> Thanks,
>
>  Gerrit
>



-- 
Yours,
Zheng

[jira] Commented: (HIVE-78) Authorization infrastructure for Hive

2010-01-19 Thread Amr Awadallah (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802720#action_12802720
 ] 

Amr Awadallah commented on HIVE-78:
---

I am also very curious what is latest on this jira, no updates since Sept of 
last year. Min, did you stop working on this? 

-- amr


> Authorization infrastructure for Hive
> -
>
> Key: HIVE-78
> URL: https://issues.apache.org/jira/browse/HIVE-78
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Server Infrastructure
>Reporter: Ashish Thusoo
>Assignee: Edward Capriolo
> Attachments: createuser-v1.patch, hive-78-metadata-v1.patch, 
> hive-78-syntax-v1.patch, hive-78.diff
>
>
> Allow hive to integrate with existing user repositories for authentication 
> and authorization infromation.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-972) support views

2010-01-19 Thread John Sichi (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802713#action_12802713
 ] 

John Sichi commented on HIVE-972:
-

The wiki page describing the development so far is here:

http://wiki.apache.org/hadoop/Hive/ViewDev

I'll go ahead and update the LanguageManual and other relevant pages to 
describe the support that will be available starting with 0.6.


> support views
> -
>
> Key: HIVE-972
> URL: https://issues.apache.org/jira/browse/HIVE-972
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Metastore, Query Processor
>Reporter: Namit Jain
>Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-972.1.patch, HIVE-972.2.patch, HIVE-972.3.patch
>
>
> Hive currently does not support views. 
> It would be a very nice feature to have.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-972) support views

2010-01-19 Thread Zheng Shao (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802689#action_12802689
 ] 

Zheng Shao commented on HIVE-972:
-

I guess not yet. A good place to put it is 
http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL. We should mention it's 
available from 0.6.


> support views
> -
>
> Key: HIVE-972
> URL: https://issues.apache.org/jira/browse/HIVE-972
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Metastore, Query Processor
>Reporter: Namit Jain
>Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-972.1.patch, HIVE-972.2.patch, HIVE-972.3.patch
>
>
> Hive currently does not support views. 
> It would be a very nice feature to have.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-972) support views

2010-01-19 Thread Jeff Hammerbacher (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802688#action_12802688
 ] 

Jeff Hammerbacher commented on HIVE-972:


Was there a documentation update for this feature? I didn't notice a wiki 
change go by, but I may have missed it.

> support views
> -
>
> Key: HIVE-972
> URL: https://issues.apache.org/jira/browse/HIVE-972
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Metastore, Query Processor
>Reporter: Namit Jain
>Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-972.1.patch, HIVE-972.2.patch, HIVE-972.3.patch
>
>
> Hive currently does not support views. 
> It would be a very nice feature to have.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1064) NPE when operating HiveCLI in distributed mode

2010-01-19 Thread Ning Zhang (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802686#action_12802686
 ] 

Ning Zhang commented on HIVE-1064:
--

Also can you add a unit test?

> NPE when operating HiveCLI in distributed mode
> --
>
> Key: HIVE-1064
> URL: https://issues.apache.org/jira/browse/HIVE-1064
> Project: Hadoop Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.6.0
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Fix For: 0.6.0
>
> Attachments: HIVE-1064.patch
>
>
> {code}
> hive> select id, name from tab_a;
> select id, name from tab_a;
> 10/01/18 03:55:59 INFO parse.ParseDriver: Parsing command: select id, name 
> from tab_a
> 10/01/18 03:55:59 INFO parse.ParseDriver: Parse Completed
> 10/01/18 03:55:59 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
> 10/01/18 03:55:59 INFO parse.SemanticAnalyzer: Completed phase 1 of Semantic 
> Analysis
> 10/01/18 03:55:59 INFO parse.SemanticAnalyzer: Get metadata for source tables
> 10/01/18 03:55:59 INFO metastore.HiveMetaStore: 0: Opening raw store with 
> implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
> 10/01/18 03:55:59 INFO metastore.ObjectStore: ObjectStore, initialize called
> 10/01/18 03:56:03 INFO metastore.ObjectStore: Initialized ObjectStore
> 10/01/18 03:56:03 INFO metastore.HiveMetaStore: 0: get_table : db=default 
> tbl=tab_a
> 10/01/18 03:56:03 INFO hive.log: DDL: struct tab_a { i32 id, string file, 
> string name}
> 10/01/18 03:56:03 INFO parse.SemanticAnalyzer: Get metadata for subqueries
> 10/01/18 03:56:03 INFO parse.SemanticAnalyzer: Get metadata for destination 
> tables
> 10/01/18 03:56:04 INFO parse.SemanticAnalyzer: Completed getting MetaData in 
> Semantic Analysis
> 10/01/18 03:56:04 INFO ppd.OpProcFactory: Processing for FS(2)
> 10/01/18 03:56:04 INFO ppd.OpProcFactory: Processing for SEL(1)
> 10/01/18 03:56:04 INFO ppd.OpProcFactory: Processing for TS(0)
> 10/01/18 03:56:04 INFO hive.log: DDL: struct tab_a { i32 id, string file, 
> string name}
> 10/01/18 03:56:04 INFO hive.log: DDL: struct tab_a { i32 id, string file, 
> string name}
> 10/01/18 03:56:04 INFO parse.SemanticAnalyzer: Completed plan generation
> 10/01/18 03:56:04 INFO ql.Driver: Semantic Analysis Completed
> 10/01/18 03:56:04 INFO ql.Driver: Starting command: select id, name from tab_a
> Total MapReduce jobs = 1
> 10/01/18 03:56:04 INFO ql.Driver: Total MapReduce jobs = 1
> Launching Job 1 out of 1
> 10/01/18 03:56:04 INFO ql.Driver: Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> 10/01/18 03:56:04 INFO exec.ExecDriver: Number of reduce tasks is set to 0 
> since there's no reduce operator
> FAILED: Unknown exception : null
> 10/01/18 03:56:04 ERROR ql.Driver: FAILED: Unknown exception : null
> java.lang.NullPointerException
>   at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:288)
>   at 
> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:475)
>   at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:103)
>   at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:64)
>   at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:589)
>   at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:469)
>   at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:329)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:317)
>   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
>   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
>   at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>   at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)
> hive> 
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-1064) NPE when operating HiveCLI in distributed mode

2010-01-19 Thread Ning Zhang (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802685#action_12802685
 ] 

Ning Zhang commented on HIVE-1064:
--

Carl, there are a lot of conflicts when applying this patch to the latest 
trunk. It may be because of the two large patches committed recently. There 
should not be further large patches in recently, can you  resolve the conflicts 
and resubmit a new patch? I'll start reviewing right away from there.

> NPE when operating HiveCLI in distributed mode
> --
>
> Key: HIVE-1064
> URL: https://issues.apache.org/jira/browse/HIVE-1064
> Project: Hadoop Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.6.0
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Fix For: 0.6.0
>
> Attachments: HIVE-1064.patch
>
>
> {code}
> hive> select id, name from tab_a;
> select id, name from tab_a;
> 10/01/18 03:55:59 INFO parse.ParseDriver: Parsing command: select id, name 
> from tab_a
> 10/01/18 03:55:59 INFO parse.ParseDriver: Parse Completed
> 10/01/18 03:55:59 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
> 10/01/18 03:55:59 INFO parse.SemanticAnalyzer: Completed phase 1 of Semantic 
> Analysis
> 10/01/18 03:55:59 INFO parse.SemanticAnalyzer: Get metadata for source tables
> 10/01/18 03:55:59 INFO metastore.HiveMetaStore: 0: Opening raw store with 
> implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
> 10/01/18 03:55:59 INFO metastore.ObjectStore: ObjectStore, initialize called
> 10/01/18 03:56:03 INFO metastore.ObjectStore: Initialized ObjectStore
> 10/01/18 03:56:03 INFO metastore.HiveMetaStore: 0: get_table : db=default 
> tbl=tab_a
> 10/01/18 03:56:03 INFO hive.log: DDL: struct tab_a { i32 id, string file, 
> string name}
> 10/01/18 03:56:03 INFO parse.SemanticAnalyzer: Get metadata for subqueries
> 10/01/18 03:56:03 INFO parse.SemanticAnalyzer: Get metadata for destination 
> tables
> 10/01/18 03:56:04 INFO parse.SemanticAnalyzer: Completed getting MetaData in 
> Semantic Analysis
> 10/01/18 03:56:04 INFO ppd.OpProcFactory: Processing for FS(2)
> 10/01/18 03:56:04 INFO ppd.OpProcFactory: Processing for SEL(1)
> 10/01/18 03:56:04 INFO ppd.OpProcFactory: Processing for TS(0)
> 10/01/18 03:56:04 INFO hive.log: DDL: struct tab_a { i32 id, string file, 
> string name}
> 10/01/18 03:56:04 INFO hive.log: DDL: struct tab_a { i32 id, string file, 
> string name}
> 10/01/18 03:56:04 INFO parse.SemanticAnalyzer: Completed plan generation
> 10/01/18 03:56:04 INFO ql.Driver: Semantic Analysis Completed
> 10/01/18 03:56:04 INFO ql.Driver: Starting command: select id, name from tab_a
> Total MapReduce jobs = 1
> 10/01/18 03:56:04 INFO ql.Driver: Total MapReduce jobs = 1
> Launching Job 1 out of 1
> 10/01/18 03:56:04 INFO ql.Driver: Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> 10/01/18 03:56:04 INFO exec.ExecDriver: Number of reduce tasks is set to 0 
> since there's no reduce operator
> FAILED: Unknown exception : null
> 10/01/18 03:56:04 ERROR ql.Driver: FAILED: Unknown exception : null
> java.lang.NullPointerException
>   at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:288)
>   at 
> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:475)
>   at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:103)
>   at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:64)
>   at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:589)
>   at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:469)
>   at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:329)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:317)
>   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
>   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>   at java.lang.reflect.Method.invoke(Method.java:597)
>   at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
>   at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>   at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)
> hive> 
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Resolved: (HIVE-972) support views

2010-01-19 Thread Zheng Shao (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Shao resolved HIVE-972.
-

   Resolution: Fixed
Fix Version/s: 0.6.0
 Release Note: HIVE-972. Support views. (John Sichi via zshao)
 Hadoop Flags: [Reviewed]

Committed. Thanks John! Please open a follow-up issue and let's get the rest of 
the TODO items done!


> support views
> -
>
> Key: HIVE-972
> URL: https://issues.apache.org/jira/browse/HIVE-972
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Metastore, Query Processor
>Reporter: Namit Jain
>Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-972.1.patch, HIVE-972.2.patch, HIVE-972.3.patch
>
>
> Hive currently does not support views. 
> It would be a very nice feature to have.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-972) support views

2010-01-19 Thread Zheng Shao (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802659#action_12802659
 ] 

Zheng Shao commented on HIVE-972:
-

Thanks John! I will test and commit it now.
Please open an follow-up issue and let's discuss the rest of things there.


> support views
> -
>
> Key: HIVE-972
> URL: https://issues.apache.org/jira/browse/HIVE-972
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Metastore, Query Processor
>Reporter: Namit Jain
>Assignee: John Sichi
> Attachments: HIVE-972.1.patch, HIVE-972.2.patch, HIVE-972.3.patch
>
>
> Hive currently does not support views. 
> It would be a very nice feature to have.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-972) support views

2010-01-19 Thread John Sichi (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802656#action_12802656
 ] 

John Sichi commented on HIVE-972:
-

I set up a MySQL metastore configuration, then created a table and view 
successfully. Browsing the resulting MySQL database, it looks like JDO chose 
MEDIUMTEXT for the type of the lob attributes. MEDIUMTEXT allows up to 2^24 
bytes (16MB), which should be good enough for even very large views.

Also, all tests are passing when I apply the .3 patch to svn head, so it's 
ready for commit. 


> support views
> -
>
> Key: HIVE-972
> URL: https://issues.apache.org/jira/browse/HIVE-972
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Metastore, Query Processor
>Reporter: Namit Jain
>Assignee: John Sichi
> Attachments: HIVE-972.1.patch, HIVE-972.2.patch, HIVE-972.3.patch
>
>
> Hive currently does not support views. 
> It would be a very nice feature to have.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-990) Incorporate CheckStyle into Hive's build.xml

2010-01-19 Thread Zheng Shao (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zheng Shao updated HIVE-990:


   Resolution: Fixed
Fix Version/s: 0.6.0
 Release Note: HIVE-990. Incorporate CheckStyle into Hive's build.xml. 
(Carl Steinbach via zshao)
 Hadoop Flags: [Reviewed]
   Status: Resolved  (was: Patch Available)

Committed. Thanks Carl!

> Incorporate CheckStyle into Hive's build.xml
> 
>
> Key: HIVE-990
> URL: https://issues.apache.org/jira/browse/HIVE-990
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Fix For: 0.6.0
>
> Attachments: checkstyle-errors.html, HIVE-990.patch
>
>
> Hadoop and Pig both have CheckStyle integrated into their build. This is 
> useful for catching
> a variety of errors as well as for enforcing a specific coding style and 
> maintaining good code hygiene.
> We just need to snatch Hadoop's checkstyle.xml and integrate it into Hive's 
> build.xml file.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-972) support views

2010-01-19 Thread John Sichi (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802622#action_12802622
 ] 

John Sichi commented on HIVE-972:
-

Yes, will do.

> support views
> -
>
> Key: HIVE-972
> URL: https://issues.apache.org/jira/browse/HIVE-972
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Metastore, Query Processor
>Reporter: Namit Jain
>Assignee: John Sichi
> Attachments: HIVE-972.1.patch, HIVE-972.2.patch, HIVE-972.3.patch
>
>
> Hive currently does not support views. 
> It would be a very nice feature to have.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-259) Add PERCENTILE aggregate function

2010-01-19 Thread Zheng Shao (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802616#action_12802616
 ] 

Zheng Shao commented on HIVE-259:
-

This is a good first step. We can provide some UDFs to "bucketize" the values 
first in case the user needs it.


> Add PERCENTILE aggregate function
> -
>
> Key: HIVE-259
> URL: https://issues.apache.org/jira/browse/HIVE-259
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Venky Iyer
>
> Compute atleast 25, 50, 75th percentiles

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-972) support views

2010-01-19 Thread Zheng Shao (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802608#action_12802608
 ] 

Zheng Shao commented on HIVE-972:
-

John, the last point, can you do "4"? We definitely need to test it out with 
mysql.


> support views
> -
>
> Key: HIVE-972
> URL: https://issues.apache.org/jira/browse/HIVE-972
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Metastore, Query Processor
>Reporter: Namit Jain
>Assignee: John Sichi
> Attachments: HIVE-972.1.patch, HIVE-972.2.patch, HIVE-972.3.patch
>
>
> Hive currently does not support views. 
> It would be a very nice feature to have.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-990) Incorporate CheckStyle into Hive's build.xml

2010-01-19 Thread Prasad Chakka (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802601#action_12802601
 ] 

Prasad Chakka commented on HIVE-990:


zheng, with java i don't think 80 is a realistic limit. imagine trying to fit 
initializations of generics into 80chars. :) i remember setting 100char as 
limit in eclipse as well.

> Incorporate CheckStyle into Hive's build.xml
> 
>
> Key: HIVE-990
> URL: https://issues.apache.org/jira/browse/HIVE-990
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: checkstyle-errors.html, HIVE-990.patch
>
>
> Hadoop and Pig both have CheckStyle integrated into their build. This is 
> useful for catching
> a variety of errors as well as for enforcing a specific coding style and 
> maintaining good code hygiene.
> We just need to snatch Hadoop's checkstyle.xml and integrate it into Hive's 
> build.xml file.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-990) Incorporate CheckStyle into Hive's build.xml

2010-01-19 Thread Zheng Shao (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802602#action_12802602
 ] 

Zheng Shao commented on HIVE-990:
-

Looks good. Will commit after test passes.

> Incorporate CheckStyle into Hive's build.xml
> 
>
> Key: HIVE-990
> URL: https://issues.apache.org/jira/browse/HIVE-990
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: checkstyle-errors.html, HIVE-990.patch
>
>
> Hadoop and Pig both have CheckStyle integrated into their build. This is 
> useful for catching
> a variety of errors as well as for enforcing a specific coding style and 
> maintaining good code hygiene.
> We just need to snatch Hadoop's checkstyle.xml and integrate it into Hive's 
> build.xml file.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-990) Incorporate CheckStyle into Hive's build.xml

2010-01-19 Thread Zheng Shao (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802593#action_12802593
 ] 

Zheng Shao commented on HIVE-990:
-

Can you explain why you changed the max line length from 80 to 100?

> Incorporate CheckStyle into Hive's build.xml
> 
>
> Key: HIVE-990
> URL: https://issues.apache.org/jira/browse/HIVE-990
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: checkstyle-errors.html, HIVE-990.patch
>
>
> Hadoop and Pig both have CheckStyle integrated into their build. This is 
> useful for catching
> a variety of errors as well as for enforcing a specific coding style and 
> maintaining good code hygiene.
> We just need to snatch Hadoop's checkstyle.xml and integrate it into Hive's 
> build.xml file.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-990) Incorporate CheckStyle into Hive's build.xml

2010-01-19 Thread Carl Steinbach (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach updated HIVE-990:


Status: Patch Available  (was: Open)

* I used Hadoop's checkstyle.xml configuration file. See HADOOP-1051 for the 
rationale behind the various settings values.
* I increased the max line length from 80 to 100 characters. 


> Incorporate CheckStyle into Hive's build.xml
> 
>
> Key: HIVE-990
> URL: https://issues.apache.org/jira/browse/HIVE-990
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: checkstyle-errors.html, HIVE-990.patch
>
>
> Hadoop and Pig both have CheckStyle integrated into their build. This is 
> useful for catching
> a variety of errors as well as for enforcing a specific coding style and 
> maintaining good code hygiene.
> We just need to snatch Hadoop's checkstyle.xml and integrate it into Hive's 
> build.xml file.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-990) Incorporate CheckStyle into Hive's build.xml

2010-01-19 Thread Carl Steinbach (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach updated HIVE-990:


Attachment: checkstyle-errors.html
HIVE-990.patch

> Incorporate CheckStyle into Hive's build.xml
> 
>
> Key: HIVE-990
> URL: https://issues.apache.org/jira/browse/HIVE-990
> Project: Hadoop Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: checkstyle-errors.html, HIVE-990.patch
>
>
> Hadoop and Pig both have CheckStyle integrated into their build. This is 
> useful for catching
> a variety of errors as well as for enforcing a specific coding style and 
> maintaining good code hygiene.
> We just need to snatch Hadoop's checkstyle.xml and integrate it into Hive's 
> build.xml file.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (HIVE-972) support views

2010-01-19 Thread John Sichi (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sichi updated HIVE-972:


Attachment: HIVE-972.3.patch

Here's the new patch (.3).  Also adds a couple of testcases for exercising 
EXPLAIN with CREATE VIEW and with SELECT from view.


> support views
> -
>
> Key: HIVE-972
> URL: https://issues.apache.org/jira/browse/HIVE-972
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Metastore, Query Processor
>Reporter: Namit Jain
>Assignee: John Sichi
> Attachments: HIVE-972.1.patch, HIVE-972.2.patch, HIVE-972.3.patch
>
>
> Hive currently does not support views. 
> It would be a very nice feature to have.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-972) support views

2010-01-19 Thread Zheng Shao (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802553#action_12802553
 ] 

Zheng Shao commented on HIVE-972:
-

Good to me. Please upload the new patch.


> support views
> -
>
> Key: HIVE-972
> URL: https://issues.apache.org/jira/browse/HIVE-972
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Metastore, Query Processor
>Reporter: Namit Jain
>Assignee: John Sichi
> Attachments: HIVE-972.1.patch, HIVE-972.2.patch
>
>
> Hive currently does not support views. 
> It would be a very nice feature to have.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-972) support views

2010-01-19 Thread John Sichi (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802549#action_12802549
 ] 

John Sichi commented on HIVE-972:
-

I have a patch which addresses items 8/9/10/11/12.  Is it OK if I run that 
through tests and submit it, then address 1/5/6 in followup JIRA issues?  They 
are easier to work on independent of the main patch.


> support views
> -
>
> Key: HIVE-972
> URL: https://issues.apache.org/jira/browse/HIVE-972
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Metastore, Query Processor
>Reporter: Namit Jain
>Assignee: John Sichi
> Attachments: HIVE-972.1.patch, HIVE-972.2.patch
>
>
> Hive currently does not support views. 
> It would be a very nice feature to have.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Assigned: (HIVE-958) support ALTER PARTITION

2010-01-19 Thread Namit Jain (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain reassigned HIVE-958:
---

Assignee: Paul Yang  (was: He Yongqiang)

> support ALTER PARTITION
> ---
>
> Key: HIVE-958
> URL: https://issues.apache.org/jira/browse/HIVE-958
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Paul Yang
>
> alter table  ...
> should be enhanced to also support:
> alter table  partition  ...
> Only 1 partition can be specified at a time

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

RE: Unit test result depends on platform.

2010-01-19 Thread Ashish Thusoo 
Can you file a JIRA and give us the unit tests that fail. That would be very 
helpful. I suspect some of the test queries may be missing a sort by predicate 
so they could have different sort orders as compared to the expected output.

Ashish 

-Original Message-
From: Mafish Liu [mailto:maf...@gmail.com] 
Sent: Monday, January 18, 2010 5:30 PM
To: hive-dev@hadoop.apache.org
Subject: Re: Unit test result depends on platform.

Attachments are listing programs.
--
maf...@gmail.com

[jira] Commented: (HIVE-1032) Better Error Messages for Execution Errors

2010-01-19 Thread Zheng Shao (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-1032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802448#action_12802448
 ] 

Zheng Shao commented on HIVE-1032:
--

Another error that we might want to include in the same patch.

The solution for this error is: "Data file split 
hdfs://dfs:9000/user/hive/warehouse/mytable/ds=2009-10-04/part-00232, range: 
0-0 is corrupted".

{code}
2010-01-19 11:53:30,581 INFO org.apache.hadoop.mapred.MapTask: split: 
hdfs://dfs:9000/user/hive/warehouse/mytable/ds=2009-10-04/part-00232, range: 0-0
2010-01-19 11:53:30,795 WARN org.apache.hadoop.mapred.Child: Error running child
java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:180)
at java.io.DataInputStream.readFully(DataInputStream.java:152)
at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1450)
at 
org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1428)
at 
org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1417)
at 
org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1412)
at 
org.apache.hadoop.mapred.SequenceFileRecordReader.(SequenceFileRecordReader.java:43)
at 
org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFileInputFormat.java:63)
at 
org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:236)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:338)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:159)
2010-01-19 11:53:30,801 INFO org.apache.hadoop.mapred.Task: Runnning cleanup 
for the task
{code}


> Better Error Messages for Execution Errors
> --
>
> Key: HIVE-1032
> URL: https://issues.apache.org/jira/browse/HIVE-1032
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Paul Yang
>Assignee: Paul Yang
> Attachments: HIVE-1032.1.patch, HIVE-1032.2.patch, HIVE-1032.3.patch
>
>
> Three common errors that occur during execution are:
> 1. Map-side group-by causing an out of memory exception due to large 
> aggregation hash tables
> 2. ScriptOperator failing due to the user's script throwing an exception or 
> otherwise returning a non-zero error code
> 3. Incorrectly specifying the join order of small and large tables, causing 
> the large table to be loaded into memory and producing an out of memory 
> exception.
> These errors are typically discovered by manually examining the error log 
> files of the failed task. This task proposes to create a feature that would 
> automatically read the error logs and output a probable cause and solution to 
> the command line.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HIVE-972) support views

2010-01-19 Thread John Sichi (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802417#action_12802417
 ] 

John Sichi commented on HIVE-972:
-

Re-running tests now.


> support views
> -
>
> Key: HIVE-972
> URL: https://issues.apache.org/jira/browse/HIVE-972
> Project: Hadoop Hive
>  Issue Type: New Feature
>  Components: Metastore, Query Processor
>Reporter: Namit Jain
>Assignee: John Sichi
> Attachments: HIVE-972.1.patch, HIVE-972.2.patch
>
>
> Hive currently does not support views. 
> It would be a very nice feature to have.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Build failed in Hudson: Hive-trunk-h0.20 #163

2010-01-19 Thread Apache Hudson Server 
See 

Changes:

[namit] HIVE-1066. Fix test outputs serde_typedbytes*.q
(Zheng Shao via namit)

--
[...truncated 13394 lines...]
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table src
[junit] POSTHOOK: Output: defa...@src
[junit] OK
[junit] Loading data to table src1
[junit] POSTHOOK: Output: defa...@src1
[junit] OK
[junit] Loading data to table src_sequencefile
[junit] POSTHOOK: Output: defa...@src_sequencefile
[junit] OK
[junit] Loading data to table src_thrift
[junit] POSTHOOK: Output: defa...@src_thrift
[junit] OK
[junit] Loading data to table src_json
[junit] POSTHOOK: Output: defa...@src_json
[junit] OK
[junit] diff 

 

[junit] Done query: unknown_function4.q
[junit] Begin query: unknown_table1.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=11
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=12
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=11
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=12
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table src
[junit] POSTHOOK: Output: defa...@src
[junit] OK
[junit] Loading data to table src1
[junit] POSTHOOK: Output: defa...@src1
[junit] OK
[junit] Loading data to table src_sequencefile
[junit] POSTHOOK: Output: defa...@src_sequencefile
[junit] OK
[junit] Loading data to table src_thrift
[junit] POSTHOOK: Output: defa...@src_thrift
[junit] OK
[junit] Loading data to table src_json
[junit] POSTHOOK: Output: defa...@src_json
[junit] OK
[junit] diff 

 

[junit] Done query: unknown_table1.q
[junit] Begin query: unknown_table2.q
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=11
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-08/hr=12
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=11
[junit] OK
[junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
[junit] POSTHOOK: Output: defa...@srcpart@ds=2008-04-09/hr=12
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] Loading data to table srcbucket
[junit] POSTHOOK: Output: defa...@srcbucket
[junit] OK
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output: defa...@srcbucket2
[junit] OK
[junit] Loading data to table srcbucket2
[junit] POSTHOOK: Output:

Hudson build is back to normal: Hive-trunk-h0.19 #341

2010-01-19 Thread Apache Hudson Server 
See

[jira] Commented: (HIVE-887) Allow SELECT without a mapreduce job

2010-01-19 Thread E. Sammer (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802311#action_12802311
 ] 

E. Sammer commented on HIVE-887:


Ning:

That sounds great. The syntax isn't really important. I was just trying to 
think of a way of providing a hint to the query execution layer so the user can 
indicate when they're willing to take the performance hit in favor of a faster 
initial response. I would think that in many cases you still probably want to 
exec as a MR job even for small results; this fetch behavior should be a user 
requested special case, in my opinion.

> Allow SELECT  without a mapreduce job
> --
>
> Key: HIVE-887
> URL: https://issues.apache.org/jira/browse/HIVE-887
> Project: Hadoop Hive
>  Issue Type: New Feature
> Environment: All
>Reporter: Eric Sun
>Assignee: Ning Zhang
>
> I often find myself needing to take a quick look at a particular column of a 
> Hive table.
> I usually do this by doing a 
> SELECT * from  LIMIT 20;
> from the CLI.  Doing this is pretty fast since it doesn't require a mapreduce 
> job.  However, it's tough to examine just 1 or 2 columns when the table is 
> very wide.
> So, I might do
> SELECT  from  LIMIT 20;
> but it's much slower since it requires a map-reduce.  It'd be really 
> convenient if a map-reduce wasn't necessary.
> Currently a good work around is to do
> hive -e "select * from table" | cut --key=n
> but it'd be more convenient if it were built in since it alleviates the need 
> for column counting.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Hudson build is back to normal: Hive-trunk-h0.18 #341

2010-01-19 Thread Apache Hudson Server 
See

Hudson build is back to normal: Hive-trunk-h0.17 #338

2010-01-19 Thread Apache Hudson Server 
See