Re: hadoop core 0.20.2 not found

[abhinav narain]

I compiled the code on remote machine without a proxy and it worked.
Then, I copied the .ant folder to my comp(in lab) and the hive code also

Now, I can see some ivy*.xml files and jar files in the
org.apache.hbase/hbase folder .

I build the same code using the new .ant folder in my home and I again get
the same error as before of hbase-0.89.0-SNAPSHOT.jar not found.

Is everyone on the development of hive having public ips and none behind a
proxy ? facing similar issues ?

I am unable to understand, why is it unable to fetch the jars when they are
already present in the cache ?


same error again :

http://repo1.maven.org/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.pom
[ivy:resolve]   -- artifact
org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar(test-jar):
[ivy:resolve]
http://repo1.maven.org/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar
[ivy:resolve]   -- artifact
org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar:
[ivy:resolve]
http://repo1.maven.org/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar
[ivy:resolve]  datanucleus-repo: tried
[ivy:resolve]   -- artifact
org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar:
[ivy:resolve]
http://www.datanucleus.org/downloads/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar
[ivy:resolve]   -- artifact
org.apache.hbase#hbase;0.89.0-SNAPSHOT!hbase.jar(test-jar):
[ivy:resolve]
http://www.datanucleus.org/downloads/maven2/org/apache/hbase/hbase/0.89.0-SNAPSHOT/hbase-0.89.0-SNAPSHOT.jar
[ivy:resolve] ::
[ivy:resolve] ::  UNRESOLVED DEPENDENCIES ::
[ivy:resolve] ::
[ivy:resolve] :: org.apache.hbase#hbase;0.89.0-SNAPSHOT: not found

Abhinav Narain

On Wed, Feb 9, 2011 at 2:31 AM, Carl Steinbach  wrote:
>
>> Hi Abhinav,
>>
>> Please make sure the .ant cache directory contains the following files:
>>
>> .ant/cache/org.apache.hbase/hbase/jars/hbase-0.89.0-SNAPSHOT.jar
>>
>> .ant/cache/org.apache.hbase/hbase/test-jars/hbase-0.89.0-SNAPSHOT-tests.jar
>>
>> I don't have any more suggestions if this does not work. Sorry.
>>
>> Carl
>>
>

[jira] Updated: (HIVE-1979) fix hbase_bulk.m by setting HiveInputFormat

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-1979:
-

  Resolution: Fixed
Hadoop Flags: [Reviewed]
  Status: Resolved  (was: Patch Available)

Committed. Thanks John

> fix hbase_bulk.m by setting HiveInputFormat
> ---
>
> Key: HIVE-1979
> URL: https://issues.apache.org/jira/browse/HIVE-1979
> Project: Hive
>  Issue Type: Bug
>  Components: HBase Handler
>Affects Versions: 0.7.0
>Reporter: John Sichi
>Assignee: John Sichi
>Priority: Blocker
> Fix For: 0.7.0
>
> Attachments: HIVE-1979.1.patch
>
>
> This was broken by the switch to CombineHiveInputFormat in HIVE-1942.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1948) Have audit logging in the Metastore

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992345#comment-12992345
 ] 

Namit Jain commented on HIVE-1948:
--

Can you regenerate the patch - I am getting some merge conflicts.

> Have audit logging in the Metastore
> ---
>
> Key: HIVE-1948
> URL: https://issues.apache.org/jira/browse/HIVE-1948
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Devaraj Das
>Assignee: Devaraj Das
> Fix For: 0.7.0
>
> Attachments: audit-log-2.patch, audit-log.1.patch, audit-log.patch
>
>
> It would be good to have audit logging in the metastore, similar to Hadoop's 
> NameNode audit logging. This would allow administrators to dig into details 
> about which user performed metadata operations (like create/drop 
> tables/partitions) and from where (IP address).

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (HIVE-1948) Have audit logging in the Metastore

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-1948:
-

Status: Open  (was: Patch Available)

> Have audit logging in the Metastore
> ---
>
> Key: HIVE-1948
> URL: https://issues.apache.org/jira/browse/HIVE-1948
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Devaraj Das
>Assignee: Devaraj Das
> Fix For: 0.7.0
>
> Attachments: audit-log-2.patch, audit-log.1.patch, audit-log.patch
>
>
> It would be good to have audit logging in the metastore, similar to Hadoop's 
> NameNode audit logging. This would allow administrators to dig into details 
> about which user performed metadata operations (like create/drop 
> tables/partitions) and from where (IP address).

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1979) fix hbase_bulk.m by setting HiveInputFormat

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992346#comment-12992346
 ] 

Namit Jain commented on HIVE-1979:
--

+1

> fix hbase_bulk.m by setting HiveInputFormat
> ---
>
> Key: HIVE-1979
> URL: https://issues.apache.org/jira/browse/HIVE-1979
> Project: Hive
>  Issue Type: Bug
>  Components: HBase Handler
>Affects Versions: 0.7.0
>Reporter: John Sichi
>Assignee: John Sichi
>Priority: Blocker
> Fix For: 0.7.0
>
> Attachments: HIVE-1979.1.patch
>
>
> This was broken by the switch to CombineHiveInputFormat in HIVE-1942.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1918) Add export/import facilities to the hive system

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992344#comment-12992344
 ] 

Namit Jain commented on HIVE-1918:
--

Reading from:
http://download.oracle.com/docs/cd/B10500_01/server.920/a96652/ch02.htm#1005081

Importing into Existing Tables

This section describes factors to take into account when you import data into 
existing tables.

Manually Creating Tables Before Importing Data

When you choose to create tables manually before importing data into them from 
an export file, you should use either the same table definition previously used 
or a compatible format. For example, although you can increase the width of 
columns and change their order, you cannot do the following:

Add NOT NULL columns
Change the datatype of a column to an incompatible datatype (LONG to NUMBER, 
for example)
Change the definition of object types used in a table
Change DEFAULT column values

Note:
When tables are manually created before data is imported, the CREATE TABLE 
statement in the export dump file will fail because the table already exists. 
To avoid this failure and continue loading data into the table, set the import 
parameter IGNORE=y. Otherwise, no data will be loaded into the table because of 
the table creation error.


Do you want to support this ? Seems like a reasonable thing to have - 
currently, an error is thrown during import
if the table already exists ?


> Add export/import facilities to the hive system
> ---
>
> Key: HIVE-1918
> URL: https://issues.apache.org/jira/browse/HIVE-1918
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Krishna Kumar
>Assignee: Krishna Kumar
> Attachments: HIVE-1918.patch.1.txt, HIVE-1918.patch.2.txt, 
> HIVE-1918.patch.3.txt, HIVE-1918.patch.txt, hive-metastore-er.pdf
>
>
> This is an enhancement request to add export/import features to hive.
> With this language extension, the user can export the data of the table - 
> which may be located in different hdfs locations in case of a partitioned 
> table - as well as the metadata of the table into a specified output 
> location. This output location can then be moved over to another different 
> hadoop/hive instance and imported there.  
> This should work independent of the source and target metastore dbms used; 
> for instance, between derby and mysql.
> For partitioned tables, the ability to export/import a subset of the 
> partition must be supported.
> Howl will add more features on top of this: The ability to create/use the 
> exported data even in the absence of hive, using MR or Pig. Please see 
> http://wiki.apache.org/pig/Howl/HowlImportExport for these details.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (HIVE-1918) Add export/import facilities to the hive system

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-1918:
-

Status: Open  (was: Patch Available)

> Add export/import facilities to the hive system
> ---
>
> Key: HIVE-1918
> URL: https://issues.apache.org/jira/browse/HIVE-1918
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Krishna Kumar
>Assignee: Krishna Kumar
> Attachments: HIVE-1918.patch.1.txt, HIVE-1918.patch.2.txt, 
> HIVE-1918.patch.3.txt, HIVE-1918.patch.txt, hive-metastore-er.pdf
>
>
> This is an enhancement request to add export/import features to hive.
> With this language extension, the user can export the data of the table - 
> which may be located in different hdfs locations in case of a partitioned 
> table - as well as the metadata of the table into a specified output 
> location. This output location can then be moved over to another different 
> hadoop/hive instance and imported there.  
> This should work independent of the source and target metastore dbms used; 
> for instance, between derby and mysql.
> For partitioned tables, the ability to export/import a subset of the 
> partition must be supported.
> Howl will add more features on top of this: The ability to create/use the 
> exported data even in the absence of hive, using MR or Pig. Please see 
> http://wiki.apache.org/pig/Howl/HowlImportExport for these details.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1918) Add export/import facilities to the hive system

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992339#comment-12992339
 ] 

Namit Jain commented on HIVE-1918:
--

1. Just moving the serialization/deserialization methods
3. inputs and outputs are present in SemanticAnalyzer - 

Look at ReadEntity/WriteEntity - they are populated by the appropriate semantic 
analyzer and
then used later by:

a. concurrency : uses inputs/outputs to lock those objects
b. authorization: uses inputs/outputs for permission checking
c. execution hooks: they can be used for a variety of things: in facebook, we 
use them for replication

> Add export/import facilities to the hive system
> ---
>
> Key: HIVE-1918
> URL: https://issues.apache.org/jira/browse/HIVE-1918
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Krishna Kumar
>Assignee: Krishna Kumar
> Attachments: HIVE-1918.patch.1.txt, HIVE-1918.patch.2.txt, 
> HIVE-1918.patch.3.txt, HIVE-1918.patch.txt, hive-metastore-er.pdf
>
>
> This is an enhancement request to add export/import features to hive.
> With this language extension, the user can export the data of the table - 
> which may be located in different hdfs locations in case of a partitioned 
> table - as well as the metadata of the table into a specified output 
> location. This output location can then be moved over to another different 
> hadoop/hive instance and imported there.  
> This should work independent of the source and target metastore dbms used; 
> for instance, between derby and mysql.
> For partitioned tables, the ability to export/import a subset of the 
> partition must be supported.
> Howl will add more features on top of this: The ability to create/use the 
> exported data even in the absence of hive, using MR or Pig. Please see 
> http://wiki.apache.org/pig/Howl/HowlImportExport for these details.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1978) Hive SymlinkTextInputFormat does not estimate input size correctly

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992338#comment-12992338
 ] 

Namit Jain commented on HIVE-1978:
--

Also, it might be simpler to add the new function 'getContentSummary' in all 
existing
input formats.

You can create a dummy class which all other hive input formats (other than 
symlinktextinputformat) extend.
In the abstract dummy class, the existing defn. can be there.

FileSystem fs = p.getFileSystem(ctx.getConf()); 
  
cs = fs.getContentSummary(p);


That waym you dont need any special checking in Utilities.java - it calls 
getContentSummary(),
which is implemented by all input formats that hive supports.




> Hive SymlinkTextInputFormat does not estimate input size correctly
> --
>
> Key: HIVE-1978
> URL: https://issues.apache.org/jira/browse/HIVE-1978
> Project: Hive
>  Issue Type: Improvement
>Reporter: He Yongqiang
>Assignee: He Yongqiang
> Attachments: HIVE-1978.1.patch
>
>


-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (HIVE-1978) Hive SymlinkTextInputFormat does not estimate input size correctly

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-1978:
-

Status: Open  (was: Patch Available)

> Hive SymlinkTextInputFormat does not estimate input size correctly
> --
>
> Key: HIVE-1978
> URL: https://issues.apache.org/jira/browse/HIVE-1978
> Project: Hive
>  Issue Type: Improvement
>Reporter: He Yongqiang
>Assignee: He Yongqiang
> Attachments: HIVE-1978.1.patch
>
>


-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1978) Hive SymlinkTextInputFormat does not estimate input size correctly

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992335#comment-12992335
 ] 

Namit Jain commented on HIVE-1978:
--

It might be simpler to add a .q file testcase.
Just load 2 files (say a1.q and a2.q in a hdfs directory).
Then load a new file, say foo, for the table 'T' - the contents of the file 
'foo' are

a1.q
a2.q


Then, 'T' can be queried

> Hive SymlinkTextInputFormat does not estimate input size correctly
> --
>
> Key: HIVE-1978
> URL: https://issues.apache.org/jira/browse/HIVE-1978
> Project: Hive
>  Issue Type: Improvement
>Reporter: He Yongqiang
>Assignee: He Yongqiang
> Attachments: HIVE-1978.1.patch
>
>


-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Resolved: (HIVE-1818) Call frequency and duration metrics for HiveMetaStore via jmx

[Sushanth Sowmyan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sushanth Sowmyan resolved HIVE-1818.


  Resolution: Fixed
Hadoop Flags: [Reviewed]

> Call frequency and duration metrics for HiveMetaStore via jmx
> -
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
>  Issue Type: New Feature
>  Components: Metastore
>Reporter: Sushanth Sowmyan
>Assignee: Sushanth Sowmyan
>Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, 
> HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the 
> HiveMetaStore had some sort of instrumentation capability so as to measure 
> frequency of calls to various calls on the HiveMetaStore and the duration of 
> time spent in these calls. 
> There are already incrementCounter() and logStartFunction() / 
> logStartTableFunction() ,etc calls in HiveMetaStore, and they could be 
> refactored/repurposed to make calls that expose JMX MBeans as well. Or, a 
> Metrics subsystem could be introduced which made calls to 
> incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics 
> subsystem could use to determine whether or not to be enabled, and if so, on 
> to what port. And once we have the capability to instrument and expose 
> MBeans, it might also be possible for other subsystems to also adopt and use 
> this system.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1818) Call frequency and duration metrics for HiveMetaStore via jmx

[Sushanth Sowmyan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992322#comment-12992322
 ] 

Sushanth Sowmyan commented on HIVE-1818:


Thanks!

> Call frequency and duration metrics for HiveMetaStore via jmx
> -
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
>  Issue Type: New Feature
>  Components: Metastore
>Reporter: Sushanth Sowmyan
>Assignee: Sushanth Sowmyan
>Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, 
> HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the 
> HiveMetaStore had some sort of instrumentation capability so as to measure 
> frequency of calls to various calls on the HiveMetaStore and the duration of 
> time spent in these calls. 
> There are already incrementCounter() and logStartFunction() / 
> logStartTableFunction() ,etc calls in HiveMetaStore, and they could be 
> refactored/repurposed to make calls that expose JMX MBeans as well. Or, a 
> Metrics subsystem could be introduced which made calls to 
> incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics 
> subsystem could use to determine whether or not to be enabled, and if so, on 
> to what port. And once we have the capability to instrument and expose 
> MBeans, it might also be possible for other subsystems to also adopt and use 
> this system.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Hudson: Hive-trunk-h0.20 #542

[Apache Hudson Server]

See 

Changes:

[pauly] HIVE-1818 Call frequency and duration metrics for HiveMetaStore via jmx
(Sushanth Sowmyan via pauly)

[jvs] HIVE-1970. Modify build to run all tests regardless of subproject 
failures.
(Carl Steinbach via jvs)

--
[...truncated 22603 lines...]
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] OK
[junit] PREHOOK: query: create table testhivedrivertable (num int)
[junit] PREHOOK: type: CREATETABLE
[junit] POSTHOOK: query: create table testhivedrivertable (num int)
[junit] POSTHOOK: type: CREATETABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] Hive history 
file=
[junit] Hive history 
file=
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] OK
[junit] PREHOOK: query: create table testhivedrivertable (key int, value 
string)
[junit] PREHOOK: type: CREATETABLE
[junit] POSTHOOK: query: create table testhivedrivertable (key int, value 
string)
[junit] POSTHOOK: type: CREATETABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: load data local inpath 
'
 into table testhivedrivertable
[junit] PREHOOK: type: LOAD
[junit] Copying data from 

[junit] Loading data to table testhivedrivertable
[junit] POSTHOOK: query: load data local inpath 
'
 into table testhivedrivertable
[junit] POSTHOOK: type: LOAD
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: select key, value from testhivedrivertable
[junit] PREHOOK: type: QUERY
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: 
file:/tmp/hudson/hive_2011-02-08_19-43-39_257_7044498286683889035/-mr-1
[junit] Total MapReduce jobs = 1
[junit] Launching Job 1 out of 1
[junit] Number of reduce tasks is set to 0 since there's no reduce operator
[junit] Job running in-process (local Hadoop)
[junit] 2011-02-08 19:43:41,823 null map = 100%,  reduce = 0%
[junit] Ended Job = job_local_0001
[junit] POSTHOOK: query: select key, value from testhivedrivertable
[junit] POSTHOOK: type: QUERY
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: 
file:/tmp/hudson/hive_2011-02-08_19-43-39_257_7044498286683889035/-mr-1
[junit] OK
[junit] PREHOOK: query: select key, value from testhivedrivertable
[junit] PREHOOK: type: QUERY
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: 
file:/tmp/hudson/hive_2011-02-08_19-43-41_974_4178845283739002672/-mr-1
[junit] Total MapReduce jobs = 1
[junit] Launching Job 1 out of 1
[junit] Number of reduce tasks is set to 0 since there's no reduce operator
[junit] Job running in-process (local Hadoop)
[junit] 2011-02-08 19:43:44,534 null map = 100%,  reduce = 0%
[junit] Ended Job = job_local_0001
[junit] POSTHOOK: query: select key, value from testhivedrivertable
[junit] POSTHOOK: type: QUERY
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: 
file:/tmp/hudson/hive_2011-02-08_19-43-41_974_4178845283739002672/-mr-1
[junit] OK
[junit] PREHOOK: query: select key, value from testhivedrivertable
[junit] PREHOOK: type: QUERY
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: 
file:/tmp/hudson/hive_2011-02-08_19-43-44_715_3499711993732994623/-mr-1
[junit] Total MapReduce jobs = 1
[junit] Launching Job 1 out of 1
[junit] Number of reduce tasks is set to 0 since there's no reduce operator
[junit] Job running in-process (local Hadoop)
[junit] 2011-02-08 19:43:47,282 null map = 100%,  reduce = 0%
[junit] Ended Job =

[jira] Updated: (HIVE-1979) fix hbase_bulk.m by setting HiveInputFormat

[John Sichi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sichi updated HIVE-1979:
-

Attachment: HIVE-1979.1.patch

> fix hbase_bulk.m by setting HiveInputFormat
> ---
>
> Key: HIVE-1979
> URL: https://issues.apache.org/jira/browse/HIVE-1979
> Project: Hive
>  Issue Type: Bug
>  Components: HBase Handler
>Affects Versions: 0.7.0
>Reporter: John Sichi
>Assignee: John Sichi
>Priority: Blocker
> Fix For: 0.7.0
>
> Attachments: HIVE-1979.1.patch
>
>
> This was broken by the switch to CombineHiveInputFormat in HIVE-1942.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (HIVE-1979) fix hbase_bulk.m by setting HiveInputFormat

[John Sichi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sichi updated HIVE-1979:
-

Status: Patch Available  (was: Open)

HIVE-1979.1.patch

> fix hbase_bulk.m by setting HiveInputFormat
> ---
>
> Key: HIVE-1979
> URL: https://issues.apache.org/jira/browse/HIVE-1979
> Project: Hive
>  Issue Type: Bug
>  Components: HBase Handler
>Affects Versions: 0.7.0
>Reporter: John Sichi
>Assignee: John Sichi
>Priority: Blocker
> Fix For: 0.7.0
>
> Attachments: HIVE-1979.1.patch
>
>
> This was broken by the switch to CombineHiveInputFormat in HIVE-1942.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (HIVE-1979) fix hbase_bulk.m by setting HiveInputFormat

[John Sichi (JIRA)]

fix hbase_bulk.m by setting HiveInputFormat
---

 Key: HIVE-1979
 URL: https://issues.apache.org/jira/browse/HIVE-1979
 Project: Hive
  Issue Type: Bug
  Components: HBase Handler
Affects Versions: 0.7.0
Reporter: John Sichi
Assignee: John Sichi
Priority: Blocker
 Fix For: 0.7.0


This was broken by the switch to CombineHiveInputFormat in HIVE-1942.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

Review Request: This patch adds the ability to get tables by owners from the metastore.

[Ashish Thusoo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/409/
---

Review request for hive and Paul Yang.


Summary
---

This patch adds the ability to get tables by owners from the metastore. The api 
added to the metastore is

list get_tables_by_owner(string owner, long offset, int limit)

The offset and limit is included so that the tables can be fetched in small 
batches. The tables are returned
sorted in the order of database name, table name.


Diffs
-

  http://svn.apache.org/repos/asf/hive/trunk/metastore/if/hive_metastore.thrift 
1068698 
  
http://svn.apache.org/repos/asf/hive/trunk/metastore/scripts/upgrade/derby/upgrade-0.8.0.derby.sql
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/hive/trunk/metastore/scripts/upgrade/mysql/upgrade-0.8.0.mysql.sql
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/hive/trunk/metastore/scripts/upgrade/postgres/upgrade-0.8.0.postgres.sql
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h
 1068698 
  
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
 1068698 
  
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp
 1068698 
  
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
 1068698 
  
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php
 1068698 
  
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote
 1068698 
  
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py
 1068698 
  
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb
 1068698 
  
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
 1068698 
  
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
 1068698 
  
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
 1068698 
  
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
 1068698 
  
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java
 1068698 
  http://svn.apache.org/repos/asf/hive/trunk/metastore/src/model/package.jdo 
1068698 
  
http://svn.apache.org/repos/asf/hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
 1068698 

Diff: https://reviews.apache.org/r/409/diff


Testing
---

Metastore tests pass.
Some other unit tests seem to be broken.


Thanks,

Ashish

[jira] Updated: (HIVE-818) Create a Hive CLI that connects to hive ThriftServer

[Ning Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ning Zhang updated HIVE-818:


Status: Patch Available  (was: Open)

> Create a Hive CLI that connects to hive ThriftServer
> 
>
> Key: HIVE-818
> URL: https://issues.apache.org/jira/browse/HIVE-818
> Project: Hive
>  Issue Type: New Feature
>  Components: Clients, Server Infrastructure
>Reporter: Edward Capriolo
>Assignee: Ning Zhang
> Attachments: HIVE-818.patch
>
>
> We should have an alternate CLI that works by interacting with the 
> HiveServer, in this way it will be ready when/if we deprecate the current CLI.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1818) Call frequency and duration metrics for HiveMetaStore via jmx

[Paul Yang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992249#comment-12992249
 ] 

Paul Yang commented on HIVE-1818:
-

Committed. Thanks Sushanth!

> Call frequency and duration metrics for HiveMetaStore via jmx
> -
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
>  Issue Type: New Feature
>  Components: Metastore
>Reporter: Sushanth Sowmyan
>Assignee: Sushanth Sowmyan
>Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, 
> HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the 
> HiveMetaStore had some sort of instrumentation capability so as to measure 
> frequency of calls to various calls on the HiveMetaStore and the duration of 
> time spent in these calls. 
> There are already incrementCounter() and logStartFunction() / 
> logStartTableFunction() ,etc calls in HiveMetaStore, and they could be 
> refactored/repurposed to make calls that expose JMX MBeans as well. Or, a 
> Metrics subsystem could be introduced which made calls to 
> incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics 
> subsystem could use to determine whether or not to be enabled, and if so, on 
> to what port. And once we have the capability to instrument and expose 
> MBeans, it might also be possible for other subsystems to also adopt and use 
> this system.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: Review for HIVE-1818 : jmx instrumentation for HiveMetaStore call frequency and duration

[Paul Yang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/352/#review203
---

Ship it!


+1

- Paul


On 2011-02-07 14:14:06, Sushanth Sowmyan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/352/
> ---
> 
> (Updated 2011-02-07 14:14:06)
> 
> 
> Review request for hive.
> 
> 
> Summary
> ---
> 
> HIVE-1818 : jmx instrumentation for HiveMetaStore call frequency and duration
> 
> 
> This addresses bug HIVE-1818.
> https://issues.apache.org/jira/browse/HIVE-1818
> 
> 
> Diffs
> -
> 
>   
> http://svn.apache.org/repos/asf/hive/trunk/common/src/java/org/apache/hadoop/hive/common/metrics/Metrics.java
>  PRE-CREATION 
>   
> http://svn.apache.org/repos/asf/hive/trunk/common/src/java/org/apache/hadoop/hive/common/metrics/MetricsMBean.java
>  PRE-CREATION 
>   
> http://svn.apache.org/repos/asf/hive/trunk/common/src/java/org/apache/hadoop/hive/common/metrics/MetricsMBeanImpl.java
>  PRE-CREATION 
>   
> http://svn.apache.org/repos/asf/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
>  1068155 
>   
> http://svn.apache.org/repos/asf/hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
>  1068155 
> 
> Diff: https://reviews.apache.org/r/352/diff
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sushanth
> 
>

[jira] Commented: (HIVE-1818) Call frequency and duration metrics for HiveMetaStore via jmx

[Paul Yang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992247#comment-12992247
 ] 

Paul Yang commented on HIVE-1818:
-

+1 tests passed

> Call frequency and duration metrics for HiveMetaStore via jmx
> -
>
> Key: HIVE-1818
> URL: https://issues.apache.org/jira/browse/HIVE-1818
> Project: Hive
>  Issue Type: New Feature
>  Components: Metastore
>Reporter: Sushanth Sowmyan
>Assignee: Sushanth Sowmyan
>Priority: Minor
> Fix For: 0.7.0
>
> Attachments: HIVE-1818-vs-1054860.patch, HIVE-1818-vs-1063088.patch, 
> HIVE-1818.patch
>
>
> As recently brought up in the hive-dev mailing list, it'd be useful if the 
> HiveMetaStore had some sort of instrumentation capability so as to measure 
> frequency of calls to various calls on the HiveMetaStore and the duration of 
> time spent in these calls. 
> There are already incrementCounter() and logStartFunction() / 
> logStartTableFunction() ,etc calls in HiveMetaStore, and they could be 
> refactored/repurposed to make calls that expose JMX MBeans as well. Or, a 
> Metrics subsystem could be introduced which made calls to 
> incrementCounter()/etc as a refactor.
> It might also be possible to specify a -D parameter that the Metrics 
> subsystem could use to determine whether or not to be enabled, and if so, on 
> to what port. And once we have the capability to instrument and expose 
> MBeans, it might also be possible for other subsystems to also adopt and use 
> this system.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

slogging on through test failures

[John Sichi]

I just committed Carl's patch for HIVE-1970.  This will cause ant test to 
continue on through all subdirs even if tests for one fail.

The plus is that this allows you to see all test failures at once instead of 
seeing a known failure in one subdir and forgetting about the fact that the 
subsequent ones may also have new failures.

The minus is that you may see some spurious failures from tests whose setup 
encounters a problem left behind by a previous failure.  (But this is already 
the case within one subdir, so it's not really new.)

You can get the old behavior back with -Dtest.continue.on.failure=false

JVS

[jira] Updated: (HIVE-1978) Hive SymlinkTextInputFormat does not estimate input size correctly

[He Yongqiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

He Yongqiang updated HIVE-1978:
---

Status: Patch Available  (was: Open)

> Hive SymlinkTextInputFormat does not estimate input size correctly
> --
>
> Key: HIVE-1978
> URL: https://issues.apache.org/jira/browse/HIVE-1978
> Project: Hive
>  Issue Type: Improvement
>Reporter: He Yongqiang
>Assignee: He Yongqiang
> Attachments: HIVE-1978.1.patch
>
>


-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (HIVE-1978) Hive SymlinkTextInputFormat does not estimate input size correctly

[He Yongqiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

He Yongqiang updated HIVE-1978:
---

Attachment: HIVE-1978.1.patch

> Hive SymlinkTextInputFormat does not estimate input size correctly
> --
>
> Key: HIVE-1978
> URL: https://issues.apache.org/jira/browse/HIVE-1978
> Project: Hive
>  Issue Type: Improvement
>Reporter: He Yongqiang
>Assignee: He Yongqiang
> Attachments: HIVE-1978.1.patch
>
>


-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (HIVE-1978) Hive SymlinkTextInputFormat does not estimate input size correctly

[He Yongqiang (JIRA)]

Hive SymlinkTextInputFormat does not estimate input size correctly
--

 Key: HIVE-1978
 URL: https://issues.apache.org/jira/browse/HIVE-1978
 Project: Hive
  Issue Type: Improvement
Reporter: He Yongqiang
Assignee: He Yongqiang




-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1970) Modify build to run all tests regardless of subproject failures

[John Sichi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992229#comment-12992229
 ] 

John Sichi commented on HIVE-1970:
--

I was able to run all the way through with the first version of the patch; I'll 
commit the second.

Damage report:

* HIVE-1969 (already known)
* HIVE-1939 (already known)
* TestHBaseMinimrCliDriver (may be the same as HIVE-1969 since it's minimr)

There were some other errors and failures, but they look like setup problems 
due to earlier failures.  (That noise is one of the downsides of continuing to 
run through after failures.)


> Modify build to run all tests regardless of subproject failures
> ---
>
> Key: HIVE-1970
> URL: https://issues.apache.org/jira/browse/HIVE-1970
> Project: Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: HIVE-1970.1.patch.txt, HIVE-1970.2.patch.txt
>
>
> We need to modify the Hive build so that it runs all subproject tests 
> regardless of any failures it encounters along the way. Currently, the build 
> stops at the first subproject it encounters with test failures.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1950) Block merge for RCFile

[Ning Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992227#comment-12992227
 ] 

Ning Zhang commented on HIVE-1950:
--

As discussed offline, this patch should be able to handle stats update 
(creating a StatsTask as child). 

Also please keep in mind that the design and implementation of the new 
MergeTask should be easy to be used in the merge process in INSERT OVERWRITE. 

> Block merge for RCFile
> --
>
> Key: HIVE-1950
> URL: https://issues.apache.org/jira/browse/HIVE-1950
> Project: Hive
>  Issue Type: New Feature
>Reporter: He Yongqiang
>Assignee: He Yongqiang
> Attachments: HIVE-1950.1.patch, HIVE-1950.2.patch
>
>
> In our env, there are a lot of small files inside one partition/table. In 
> order to reduce the namenode load, we have one dedicated housekeeping job 
> running to merge these file. Right now the merge is an 'insert overwrite' in 
> hive, and requires decompress the data and compress it. This jira is to add a 
> command in Hive to do the merge without decompress and recompress the data.
> Something like "alter table tbl_name [partition ()] merge files". In this 
> jira the new command will only support RCFile, since there need some new APIs 
> to the fileformat.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1971) Verbose/echo mode for the Hive CLI

[Ning Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992223#comment-12992223
 ] 

Ning Zhang commented on HIVE-1971:
--

+1. Will commit if tests pass.

> Verbose/echo mode for the Hive CLI
> --
>
> Key: HIVE-1971
> URL: https://issues.apache.org/jira/browse/HIVE-1971
> Project: Hive
>  Issue Type: New Feature
>  Components: CLI
>Reporter: Jonathan Natkins
>Assignee: Jonathan Natkins
> Attachments: HIVE-1971.1.patch.txt, HIVE-1971.2.patch.txt
>
>
> It would be very beneficial to have a mode which allows a user to run a SQL 
> script, and have each command echoed to the console as it's executed.  This 
> would be useful in figuring out which SQL statement is causing failures 
> during test runs, especially when running particularly long scripts.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1517) ability to select across a database

[Siying Dong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992215#comment-12992215
 ] 

Siying Dong commented on HIVE-1517:
---

The concurrency issue seems to be that in the codes of acquiring locks, we 
always lock current database but when we locking every table or partition, we 
don't lock database of them, so that it breaks when we reference table from 
other databases. When I lock database for every table/partitions, the test 
passes.

Namit, do you think I should remove the codes to lock the current database? Is 
there a reason we always lock current database?

> ability to select across a database
> ---
>
> Key: HIVE-1517
> URL: https://issues.apache.org/jira/browse/HIVE-1517
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Siying Dong
>Priority: Blocker
> Fix For: 0.7.0
>
> Attachments: HIVE-1517.1.patch.txt, HIVE-1517.2.patch.txt
>
>
> After  https://issues.apache.org/jira/browse/HIVE-675, we need a way to be 
> able to select across a database for this feature to be useful.
> For eg:
> use db1
> create table foo();
> use db2
> select .. from db1.foo.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1941) support explicit view partitioning

[John Sichi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992211#comment-12992211
 ] 

John Sichi commented on HIVE-1941:
--

Chatted with Paul:  hook should get view+view partitions as output, underlying 
table+table partitions as input.


> support explicit view partitioning
> --
>
> Key: HIVE-1941
> URL: https://issues.apache.org/jira/browse/HIVE-1941
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Affects Versions: 0.6.0
>Reporter: John Sichi
>Assignee: John Sichi
> Attachments: HIVE-1941.1.patch, HIVE-1941.2.patch
>
>
> Allow creation of a view with an explicit partitioning definition, and 
> support ALTER VIEW ADD/DROP PARTITION for instantiating partitions.
> For more information, see
> http://wiki.apache.org/hadoop/Hive/PartitionedViews

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (HIVE-1971) Verbose/echo mode for the Hive CLI

[Jonathan Natkins (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated HIVE-1971:
---

Attachment: HIVE-1971.2.patch.txt

Patch updated

> Verbose/echo mode for the Hive CLI
> --
>
> Key: HIVE-1971
> URL: https://issues.apache.org/jira/browse/HIVE-1971
> Project: Hive
>  Issue Type: New Feature
>  Components: CLI
>Reporter: Jonathan Natkins
>Assignee: Jonathan Natkins
> Attachments: HIVE-1971.1.patch.txt, HIVE-1971.2.patch.txt
>
>
> It would be very beneficial to have a mode which allows a user to run a SQL 
> script, and have each command echoed to the console as it's executed.  This 
> would be useful in figuring out which SQL statement is causing failures 
> during test runs, especially when running particularly long scripts.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1517) ability to select across a database

[Carl Steinbach (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992212#comment-12992212
 ] 

Carl Steinbach commented on HIVE-1517:
--

The syntax for DESCRIBE is broken. It should be:

{code}
DESCRIBE [EXTENDED] [database DOT]table [column]
{code}

but is actually

{code}
DESCRIBE [EXTENDED] table[DOT col_name]
{code}

Ref: http://dev.mysql.com/doc/refman/5.0/en/describe.html

One solution (which I'm not eager to see us take) is to extend Hive's 
non-standard as follows:

{code}
DESCRIBE [EXTENDED] [database] table[DOT col_name]
{code}

Anyway, I think it's OK to fix this as a followup.


> ability to select across a database
> ---
>
> Key: HIVE-1517
> URL: https://issues.apache.org/jira/browse/HIVE-1517
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Siying Dong
>Priority: Blocker
> Fix For: 0.7.0
>
> Attachments: HIVE-1517.1.patch.txt, HIVE-1517.2.patch.txt
>
>
> After  https://issues.apache.org/jira/browse/HIVE-675, we need a way to be 
> able to select across a database for this feature to be useful.
> For eg:
> use db1
> create table foo();
> use db2
> select .. from db1.foo.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: HIVE-1971. Verbose/echo mode for the Hive CLI.

[Jonathan Natkins]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/404/
---

(Updated 2011-02-08 14:17:58.969501)


Review request for hive.


Summary
---

This would add a mode to the CLI that echoes any commands that are executed 
back to the command line.


This addresses bug HIVE-1971.
https://issues.apache.org/jira/browse/HIVE-1971


Diffs (updated)
-

  cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 2b52826 
  cli/src/java/org/apache/hadoop/hive/cli/OptionsProcessor.java 93a7c95 
  ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 2b0dba8 

Diff: https://reviews.apache.org/r/404/diff


Testing
---

Manual


Thanks,

Jonathan

Re: Review Request: HIVE-1971. Verbose/echo mode for the Hive CLI.

[Jonathan Natkins]

> On 2011-02-08 08:33:08, Ning Zhang wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 194
> > 
> >
> > I don't see why this should be set in HiveConf. IMO, verbose should be 
> > pure client side setting, it should not been passed to the server side.

My intention had been to mimic what I was seeing done with other options, but 
having it be client-side only definitely makes sense.  I've removed all the 
server-side settings.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/404/#review191
---


On 2011-02-07 23:16:17, Jonathan Natkins wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/404/
> ---
> 
> (Updated 2011-02-07 23:16:17)
> 
> 
> Review request for hive.
> 
> 
> Summary
> ---
> 
> This would add a mode to the CLI that echoes any commands that are executed 
> back to the command line.
> 
> 
> This addresses bug HIVE-1971.
> https://issues.apache.org/jira/browse/HIVE-1971
> 
> 
> Diffs
> -
> 
>   cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 2b52826 
>   cli/src/java/org/apache/hadoop/hive/cli/OptionsProcessor.java 93a7c95 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a091b95 
>   ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 2b0dba8 
> 
> Diff: https://reviews.apache.org/r/404/diff
> 
> 
> Testing
> ---
> 
> Manual
> 
> 
> Thanks,
> 
> Jonathan
> 
>

[jira] Updated: (HIVE-1950) Block merge for RCFile

[He Yongqiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

He Yongqiang updated HIVE-1950:
---

Status: Patch Available  (was: Open)

> Block merge for RCFile
> --
>
> Key: HIVE-1950
> URL: https://issues.apache.org/jira/browse/HIVE-1950
> Project: Hive
>  Issue Type: New Feature
>Reporter: He Yongqiang
>Assignee: He Yongqiang
> Attachments: HIVE-1950.1.patch, HIVE-1950.2.patch
>
>
> In our env, there are a lot of small files inside one partition/table. In 
> order to reduce the namenode load, we have one dedicated housekeeping job 
> running to merge these file. Right now the merge is an 'insert overwrite' in 
> hive, and requires decompress the data and compress it. This jira is to add a 
> command in Hive to do the merge without decompress and recompress the data.
> Something like "alter table tbl_name [partition ()] merge files". In this 
> jira the new command will only support RCFile, since there need some new APIs 
> to the fileformat.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (HIVE-1950) Block merge for RCFile

[He Yongqiang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

He Yongqiang updated HIVE-1950:
---

Attachment: HIVE-1950.2.patch

A new patch addressed the review comments.

Will put a few into followup including the stat update.

> Block merge for RCFile
> --
>
> Key: HIVE-1950
> URL: https://issues.apache.org/jira/browse/HIVE-1950
> Project: Hive
>  Issue Type: New Feature
>Reporter: He Yongqiang
>Assignee: He Yongqiang
> Attachments: HIVE-1950.1.patch, HIVE-1950.2.patch
>
>
> In our env, there are a lot of small files inside one partition/table. In 
> order to reduce the namenode load, we have one dedicated housekeeping job 
> running to merge these file. Right now the merge is an 'insert overwrite' in 
> hive, and requires decompress the data and compress it. This jira is to add a 
> command in Hive to do the merge without decompress and recompress the data.
> Something like "alter table tbl_name [partition ()] merge files". In this 
> jira the new command will only support RCFile, since there need some new APIs 
> to the fileformat.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (HIVE-1977) DESCRIBE TABLE syntax doesn't support specifying a database qualified table name

[Carl Steinbach (JIRA)]

DESCRIBE TABLE syntax doesn't support specifying a database qualified table name


 Key: HIVE-1977
 URL: https://issues.apache.org/jira/browse/HIVE-1977
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Carl Steinbach


The syntax for DESCRIBE is broken. It should be:

{code}
DESCRIBE [EXTENDED] [database DOT]table [column]
{code}

but is actually

{code}
DESCRIBE [EXTENDED] table[DOT col_name]
{code}


Ref: http://dev.mysql.com/doc/refman/5.0/en/describe.html

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1517) ability to select across a database

[Siying Dong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992199#comment-12992199
 ] 

Siying Dong commented on HIVE-1517:
---

Looks like ANALYZE TABLE doesn't need the table.xxx syntax. I'll change that. 
Then the only problem left is DESCRIBE.

> ability to select across a database
> ---
>
> Key: HIVE-1517
> URL: https://issues.apache.org/jira/browse/HIVE-1517
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Siying Dong
>Priority: Blocker
> Fix For: 0.7.0
>
> Attachments: HIVE-1517.1.patch.txt, HIVE-1517.2.patch.txt
>
>
> After  https://issues.apache.org/jira/browse/HIVE-675, we need a way to be 
> able to select across a database for this feature to be useful.
> For eg:
> use db1
> create table foo();
> use db2
> select .. from db1.foo.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (HIVE-1973) Getting error when join on tables where name of table has uppercase letters

[Carl Steinbach (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach updated HIVE-1973:
-

Affects Version/s: 0.7.0

Verified that this fails on trunk:

{code}
hive> select * from a join tabForJoin on (a.b = tabForJoin.b);
select * from a join tabForJoin on (a.b = tabForJoin.b);
FAILED: Error in semantic analysis: line 1:42 Invalid Table Alias tabForJoin
hive> select * from a join tabforjoin on (a.b = tabForJoin.b);
OK
Time taken: 5.618 seconds
hive>
{code}

> Getting error when join on tables where name of table has uppercase letters
> ---
>
> Key: HIVE-1973
> URL: https://issues.apache.org/jira/browse/HIVE-1973
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.5.0, 0.7.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise 
> Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
>Reporter: Chinna Rao Lalam
>Assignee: Chinna Rao Lalam
>
> When execute a join query on tables containing Uppercase letters in the table 
> names hit an exception
>  Ex:
> {noformat}
>   create table a(b int);
>   create table tabForJoin(b int,c int);
>   select * from a join tabForJoin on(a.b=tabForJoin.b);
>   Got an exception like this
>   FAILED: Error in semantic analysis:  Invalid Table Alias tabForJoin
> {noformat}
> But if i give without capital letters ,It is working

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1970) Modify build to run all tests regardless of subproject failures

[Carl Steinbach (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992184#comment-12992184
 ] 

Carl Steinbach commented on HIVE-1970:
--

The new version of the patch adds the build property 
test.continue.on.failure=true


> Modify build to run all tests regardless of subproject failures
> ---
>
> Key: HIVE-1970
> URL: https://issues.apache.org/jira/browse/HIVE-1970
> Project: Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: HIVE-1970.1.patch.txt, HIVE-1970.2.patch.txt
>
>
> We need to modify the Hive build so that it runs all subproject tests 
> regardless of any failures it encounters along the way. Currently, the build 
> stops at the first subproject it encounters with test failures.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: HIVE-1970: Modify build to run all tests regardless of subproject failures

[Carl Steinbach]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/402/
---

(Updated 2011-02-08 13:04:14.542972)


Review request for hive.


Changes
---

Added a new property test.continue.on.failure to build.xml (defaults to true), 
which controls whether or not the build should continue past failures.


Summary
---

Review for HIVE-1970


This addresses bug HIVE-1970.
https://issues.apache.org/jira/browse/HIVE-1970


Diffs (updated)
-

  build.xml f1b558c 

Diff: https://reviews.apache.org/r/402/diff


Testing
---


Thanks,

Carl

[jira] Updated: (HIVE-1970) Modify build to run all tests regardless of subproject failures

[Carl Steinbach (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach updated HIVE-1970:
-

Attachment: HIVE-1970.2.patch.txt

> Modify build to run all tests regardless of subproject failures
> ---
>
> Key: HIVE-1970
> URL: https://issues.apache.org/jira/browse/HIVE-1970
> Project: Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: HIVE-1970.1.patch.txt, HIVE-1970.2.patch.txt
>
>
> We need to modify the Hive build so that it runs all subproject tests 
> regardless of any failures it encounters along the way. Currently, the build 
> stops at the first subproject it encounters with test failures.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: hadoop core 0.20.2 not found

[Carl Steinbach]

Hi Abhinav,

Please make sure the .ant cache directory contains the following files:

.ant/cache/org.apache.hbase/hbase/jars/hbase-0.89.0-SNAPSHOT.jar
.ant/cache/org.apache.hbase/hbase/test-jars/hbase-0.89.0-SNAPSHOT-tests.jar

I don't have any more suggestions if this does not work. Sorry.

Carl

On Tue, Feb 8, 2011 at 8:40 AM, abhinav narain wrote:

> I am waiting, if Carl can suggest anything ?
>
> On Tue, Feb 8, 2011 at 9:08 PM, abhinav narain  >wrote:
>
> > Only if I had a machine, that was not behind a proxy !
> >
> >
> > On Tue, Feb 8, 2011 at 8:43 PM, Edward Capriolo  >wrote:
> >
> >> On Tue, Feb 8, 2011 at 5:40 AM, abhinav narain
> >>  wrote:
> >> > On Mon, Feb 7, 2011 at 11:29 PM, Carl Steinbach 
> >> wrote:
> >> >
> >> >> Hi Abhinav,
> >> >>
> >> >> I am using a proxy.
> >> >> > I am using cntlm for authentication.
> >> >> > I have added localhost:1234 in ANT_OPTS ... as above before
> compiling
> >> >> >
> >> >> >
> >> >> It looks like Ivy supports NTLM authentication:
> >> >>
> >> >>
> >> >>
> >>
> https://issues.apache.org/jira/browse/IVY-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> >> >>
> >> >> Can you try also setting -Dhttp.proxyRealm= in ANT_OPTS
> >> and
> >> >> let
> >> >> us know if it works?
> >> >>
> >> >> Sadly, the above does not help.
> >> > I get the same error.
> >> > The following is a line from my .bashrc
> >> > export ANT_OPTS="-Dhttp.proxyHost=localhost -Dhttp.proxyPort=1234
> >> > -Dhttp.proxyRealm=iitm.ac.in"
> >> >
> >> >
> >> >
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hbase/hbase/0.89.0-SNAPSHOT/
> >> >
> >> >> > >
> >> $HOME/.ant/cache/org.apache.hbase/hbase/jars/hbase-0.89.0-SNAPSHOT.jar
> >> >> > >
> >> >> > > There is no folder as org.apache.hbase 
> >> >> > but there are folders as org.apache.{zookeeper},{commons} etc ...
> >> >> > Shall i create one and place the downloaded hbase jar there ?
> >> >>
> >> >>
> >> >> Yes, if you can't get Ivy to work then you'll need to satisfy the
> >> >> dependency
> >> >> manually by downloading the hbase jar and copying it to
> >> >>
> $HOME/.ant/cache/org/apache.hbase/hbase/jars/hbase-0.89.0-SNAPSHOT.jar
> >> >>
> >> > I have added the following jars in the folder
> >> > $HOME/.ant/cache/org{.}apache.hbase/hbase/jars/
> >> >
> >> > Please note the dot{.} in the path instead of a slash{/}
> >> >
> >> > hbase-0.89.0-SNAPSHOT-tests.jar,  hbase-0.89.0-SNAPSHOT.jar,
> >> > hbase-0.89.0-SNAPSHOT.pom
> >> >
> >> > But, I still get the same error as before !
> >> > What should I do.
> >> > Are there any xml fiels that have to be manually added too ?
> >> >
> >> > Abhinav Narain
> >> >
> >>
> >> I am a big fan of quixotic quests, but have you considering building
> >> on a machine without this proxy? Java byte-code is portable after all.
> >>
> >
> >
>

[jira] Commented: (HIVE-1517) ability to select across a database

[Siying Dong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992165#comment-12992165
 ] 

Siying Dong commented on HIVE-1517:
---

Sorry, I mean DESCRIBE. DROP TABLE is fine.
You can do something like 'desc extended src.key;'

For ANALYZE TABLE, the syntax is similar to DESCRIBE, though I'm not sure how 
it is used.

Those are the two that I found you didn't replace Identification to tableName.

> ability to select across a database
> ---
>
> Key: HIVE-1517
> URL: https://issues.apache.org/jira/browse/HIVE-1517
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Siying Dong
>Priority: Blocker
> Fix For: 0.7.0
>
> Attachments: HIVE-1517.1.patch.txt, HIVE-1517.2.patch.txt
>
>
> After  https://issues.apache.org/jira/browse/HIVE-675, we need a way to be 
> able to select across a database for this feature to be useful.
> For eg:
> use db1
> create table foo();
> use db2
> select .. from db1.foo.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1517) ability to select across a database

[Carl Steinbach (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992158#comment-12992158
 ] 

Carl Steinbach commented on HIVE-1517:
--

I'm not sure I understand the problem with DROP TABLE. How is it ambiguous?

As for ANALYZE TABLE, it looks like the definition of the tabTypeExpr
rule in the grammar is more complicated than it needs to be? Isn't this
just tableName from my patch?

> ability to select across a database
> ---
>
> Key: HIVE-1517
> URL: https://issues.apache.org/jira/browse/HIVE-1517
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Siying Dong
>Priority: Blocker
> Fix For: 0.7.0
>
> Attachments: HIVE-1517.1.patch.txt, HIVE-1517.2.patch.txt
>
>
> After  https://issues.apache.org/jira/browse/HIVE-675, we need a way to be 
> able to select across a database for this feature to be useful.
> For eg:
> use db1
> create table foo();
> use db2
> select .. from db1.foo.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1517) ability to select across a database

[Siying Dong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992139#comment-12992139
 ] 

Siying Dong commented on HIVE-1517:
---

I notice that Carl's patch added cross database support for 'create table', 
'load', etc, but now 'drop table' and 'analyze table', which are very hard 
since they currently support dots as other meaning and would cause the 
ambiguity that we are not ready to handle. Also, it's hard for 'drop table' or 
'analyze table' to give accurate error message if people try to use db.table. 
Do you think we want to keep it or we should remove all supports for DDL 
queries?

> ability to select across a database
> ---
>
> Key: HIVE-1517
> URL: https://issues.apache.org/jira/browse/HIVE-1517
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Siying Dong
>Priority: Blocker
> Fix For: 0.7.0
>
> Attachments: HIVE-1517.1.patch.txt, HIVE-1517.2.patch.txt
>
>
> After  https://issues.apache.org/jira/browse/HIVE-675, we need a way to be 
> able to select across a database for this feature to be useful.
> For eg:
> use db1
> create table foo();
> use db2
> select .. from db1.foo.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1970) Modify build to run all tests regardless of subproject failures

[Carl Steinbach (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992136#comment-12992136
 ] 

Carl Steinbach commented on HIVE-1970:
--

Good idea. I'll make the change.

> Modify build to run all tests regardless of subproject failures
> ---
>
> Key: HIVE-1970
> URL: https://issues.apache.org/jira/browse/HIVE-1970
> Project: Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: HIVE-1970.1.patch.txt
>
>
> We need to modify the Hive build so that it runs all subproject tests 
> regardless of any failures it encounters along the way. Currently, the build 
> stops at the first subproject it encounters with test failures.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-818) Create a Hive CLI that connects to hive ThriftServer

[Ning Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992134#comment-12992134
 ] 

Ning Zhang commented on HIVE-818:
-

Review board:  https://reviews.apache.org/r/407/

> Create a Hive CLI that connects to hive ThriftServer
> 
>
> Key: HIVE-818
> URL: https://issues.apache.org/jira/browse/HIVE-818
> Project: Hive
>  Issue Type: New Feature
>  Components: Clients, Server Infrastructure
>Reporter: Edward Capriolo
>Assignee: Ning Zhang
> Attachments: HIVE-818.patch
>
>
> We should have an alternate CLI that works by interacting with the 
> HiveServer, in this way it will be ready when/if we deprecate the current CLI.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

Review Request: HIVE-818. Create a Hive CLI that connects to hive ThriftServer

[Ning Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/407/
---

Review request for hive.


Summary
---

Copied from JIRA HIVE-818: 

This patch does the following:

add 2 options (-h, -p) in CLI to specify the hostname and port of Hive server.
change the HiveServer to output non-Hive commands (non Driver) to a temp file 
and change the fetchOne/fetchN/fetchAll functions to get results from the temp 
file.
change the fetchOne function to throw a HiveServerException (error code 0) when 
reaching the end of result rather than sending an empty string.
Caveats:

session.err from the HiveServer is still not sending back to client. So the 
progress of a Hadoop job is not shown in the client side in remote mode (I 
think there is a JIRA opened already. If not I wil file a follow-up JIRA for 
this).
now end-to-end unit test for remote mode. I manually tested HiveServer and CLI 
in remote mode (set/dfs/SQL commands) and in combination of -e/-f options. I 
will file a follow-up JIRA for creating a unit test suite for remote mode CLI.


Diffs
-

  trunk/build.xml 1068272 
  trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1068272 
  trunk/cli/src/java/org/apache/hadoop/hive/cli/CliSessionState.java 1068272 
  trunk/cli/src/java/org/apache/hadoop/hive/cli/OptionsProcessor.java 1068272 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 1068272 
  trunk/service/if/hive_service.thrift 1068272 
  trunk/service/src/gen/thrift/gen-cpp/ThriftHive.h 1068272 
  trunk/service/src/gen/thrift/gen-cpp/ThriftHive.cpp 1068272 
  trunk/service/src/gen/thrift/gen-cpp/ThriftHive_server.skeleton.cpp 1068272 
  
trunk/service/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/service/ThriftHive.java
 1068272 
  trunk/service/src/gen/thrift/gen-php/hive_service/ThriftHive.php 1068272 
  trunk/service/src/gen/thrift/gen-py/hive_service/ThriftHive-remote 1068272 
  trunk/service/src/gen/thrift/gen-py/hive_service/ThriftHive.py 1068272 
  trunk/service/src/gen/thrift/gen-rb/thrift_hive.rb 1068272 
  trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java 1068272 

Diff: https://reviews.apache.org/r/407/diff


Testing
---

Passed all unit tests. 

Also manually tested HiveServer and CLI remote mode by:
 1) $ hive --service hiveserver
 2) in another terminal: hive -h localhost
 3) tested the following command:
- set; -- get all parameters
- set hive.stats.autogather;  -- check default parameter value
- set hive.stats.autogather=false;  -- change parameter value
- set hive.stats.autogather;  -- check parameter value got changed
- select * from src;  -- Hive query but no Hadoop job
- select count(*) from src; -- Hive query and Hadoop job is created
- select k from src; -- negative test case where SemanticAnalyzer throw an 
exception
- show partitions srcpart;  -- Hive Query but no hadoop job
- explain select count(*) from srcpart where ds is not null; -- explain 
query


Thanks,

Ning

[jira] Updated: (HIVE-818) Create a Hive CLI that connects to hive ThriftServer

[Ning Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ning Zhang updated HIVE-818:


Attachment: HIVE-818.patch

This patch does the following: 
 - add 2 options (-h, -p) in CLI to specify the hostname and port of Hive 
server.
 - change the HiveServer to output non-Hive commands (non Driver) to a temp 
file and change the fetchOne/fetchN/fetchAll functions to get results from the 
temp file. 
 - change the fetchOne function to throw a HiveServerException (error code 0) 
when reaching the end of result rather than sending an empty string. 

Caveats:
 - session.err from the HiveServer is still not sending back to client. So the 
progress of a Hadoop job is not shown in the client side in remote mode (I 
think there is a JIRA opened already. If not I wil file a follow-up JIRA for 
this). 
 - now end-to-end unit test for remote mode. I manually tested HiveServer and 
CLI in remote mode (set/dfs/SQL commands) and in combination of -e/-f options. 
I will file a follow-up JIRA for creating a unit test suite for remote mode CLI.

> Create a Hive CLI that connects to hive ThriftServer
> 
>
> Key: HIVE-818
> URL: https://issues.apache.org/jira/browse/HIVE-818
> Project: Hive
>  Issue Type: New Feature
>  Components: Clients, Server Infrastructure
>Reporter: Edward Capriolo
>Assignee: Ning Zhang
> Attachments: HIVE-818.patch
>
>
> We should have an alternate CLI that works by interacting with the 
> HiveServer, in this way it will be ready when/if we deprecate the current CLI.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1970) Modify build to run all tests regardless of subproject failures

[John Sichi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992125#comment-12992125
 ] 

John Sichi commented on HIVE-1970:
--

Do you think we should parameterize it so that for Hudson runs, we can tell it 
to keep going, but for developer testing, we can still make it stop on first 
error?


> Modify build to run all tests regardless of subproject failures
> ---
>
> Key: HIVE-1970
> URL: https://issues.apache.org/jira/browse/HIVE-1970
> Project: Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: HIVE-1970.1.patch.txt
>
>
> We need to modify the Hive build so that it runs all subproject tests 
> regardless of any failures it encounters along the way. Currently, the build 
> stops at the first subproject it encounters with test failures.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1970) Modify build to run all tests regardless of subproject failures

[John Sichi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992123#comment-12992123
 ] 

John Sichi commented on HIVE-1970:
--

+1.  Will commit if tests fail.  (I always wanted to say that!)

Basically, the current state of Hudson is a great test case for this patch to 
make sure I can see all of the failures at once.


> Modify build to run all tests regardless of subproject failures
> ---
>
> Key: HIVE-1970
> URL: https://issues.apache.org/jira/browse/HIVE-1970
> Project: Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Attachments: HIVE-1970.1.patch.txt
>
>
> We need to modify the Hive build so that it runs all subproject tests 
> regardless of any failures it encounters along the way. Currently, the build 
> stops at the first subproject it encounters with test failures.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1950) Block merge for RCFile

[He Yongqiang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992112#comment-12992112
 ] 

He Yongqiang commented on HIVE-1950:


review comments from internal review:
1) if the stats present, try to correct it
2) jobClose of RCFileMergeMapper should share the code in FileSinkOperator
3) move the original data to a dump loc first
4) remove getRecordWriter() and RCFileBlockMergeOutputFormat
5) ioCxt for input file changed
6) disable merge for archived table/partition and bucketized table/partition
7) comments
8) negative tests for hiveinputformat



> Block merge for RCFile
> --
>
> Key: HIVE-1950
> URL: https://issues.apache.org/jira/browse/HIVE-1950
> Project: Hive
>  Issue Type: New Feature
>Reporter: He Yongqiang
>Assignee: He Yongqiang
> Attachments: HIVE-1950.1.patch
>
>
> In our env, there are a lot of small files inside one partition/table. In 
> order to reduce the namenode load, we have one dedicated housekeeping job 
> running to merge these file. Right now the merge is an 'insert overwrite' in 
> hive, and requires decompress the data and compress it. This jira is to add a 
> command in Hive to do the merge without decompress and recompress the data.
> Something like "alter table tbl_name [partition ()] merge files". In this 
> jira the new command will only support RCFile, since there need some new APIs 
> to the fileformat.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hudson status

[John Sichi]

Hey all,

I'm not sure how people are still committing patches given the current bad 
state of the tests.  I'd suggest everyone stop committing new functionality and 
work on committing test fixes until we see a clean build from Hudson again.  
Carl is blocked on cutting the 0.7 branch until this happens.

Thanks,
JVS

Build failed in Hudson: Hive-trunk-h0.20 #541

[Apache Hudson Server]

See 

--
[...truncated 17967 lines...]
[junit] POSTHOOK: Output: default@srcbucket
[junit] OK
[junit] PREHOOK: query: LOAD DATA LOCAL INPATH 
'
 INTO TABLE srcbucket
[junit] PREHOOK: type: LOAD
[junit] Copying data from 

[junit] Loading data to table srcbucket
[junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 
'
 INTO TABLE srcbucket
[junit] POSTHOOK: type: LOAD
[junit] POSTHOOK: Output: default@srcbucket
[junit] OK
[junit] PREHOOK: query: CREATE TABLE srcbucket2(key int, value string) 
CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE
[junit] PREHOOK: type: CREATETABLE
[junit] POSTHOOK: query: CREATE TABLE srcbucket2(key int, value string) 
CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE
[junit] POSTHOOK: type: CREATETABLE
[junit] POSTHOOK: Output: default@srcbucket2
[junit] OK
[junit] PREHOOK: query: LOAD DATA LOCAL INPATH 
'
 INTO TABLE srcbucket2
[junit] PREHOOK: type: LOAD
[junit] Copying data from 

[junit] Loading data to table srcbucket2
[junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 
'
 INTO TABLE srcbucket2
[junit] POSTHOOK: type: LOAD
[junit] POSTHOOK: Output: default@srcbucket2
[junit] OK
[junit] PREHOOK: query: LOAD DATA LOCAL INPATH 
'
 INTO TABLE srcbucket2
[junit] PREHOOK: type: LOAD
[junit] Copying data from 

[junit] Loading data to table srcbucket2
[junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 
'
 INTO TABLE srcbucket2
[junit] POSTHOOK: type: LOAD
[junit] POSTHOOK: Output: default@srcbucket2
[junit] OK
[junit] PREHOOK: query: LOAD DATA LOCAL INPATH 
'
 INTO TABLE srcbucket2
[junit] PREHOOK: type: LOAD
[junit] Copying data from 

[junit] Loading data to table srcbucket2
[junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 
'
 INTO TABLE srcbucket2
[junit] POSTHOOK: type: LOAD
[junit] POSTHOOK: Output: default@srcbucket2
[junit] OK
[junit] PREHOOK: query: LOAD DATA LOCAL INPATH 
'
 INTO TABLE srcbucket2
[junit] PREHOOK: type: LOAD
[junit] Copying data from 

[junit] Loading data to table srcbucket2
[junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 
'
 INTO TABLE srcbucket2
[junit] POSTHOOK: type: LOAD
[junit] POSTHOOK: Output: default@srcbucket2
[junit] OK
[junit] PREHOOK: query: LOAD DATA LOCAL INPATH 
'
 INTO TABLE src
[junit] PREHOOK: type: LOAD
[junit] Copying data from 

[junit] Loading data to table src
[junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 
'
 INTO TABLE src
[junit] POSTHOOK: type: LOAD
[junit] POSTHOOK: Output: default@src
[junit] OK
[junit] PREHOOK: query: LOAD DATA LOCAL INPATH 
'
 INTO TABLE src1
[junit] PREHOOK: type: LOAD
[junit] Copying data from 

[junit] Loading data to table src1
[junit] POSTHOOK: query: LOAD DATA LOCAL INPATH 
'
 INTO TABLE src1
[junit] POSTHOOK: type: LOAD
[junit] POSTHOOK: Output: default@src1
[junit] OK
[junit] PREHOOK: query: LOAD DATA LOCAL INPATH 
'

Re: hadoop core 0.20.2 not found

[abhinav narain]

I am waiting, if Carl can suggest anything ?

On Tue, Feb 8, 2011 at 9:08 PM, abhinav narain wrote:

> Only if I had a machine, that was not behind a proxy !
>
>
> On Tue, Feb 8, 2011 at 8:43 PM, Edward Capriolo wrote:
>
>> On Tue, Feb 8, 2011 at 5:40 AM, abhinav narain
>>  wrote:
>> > On Mon, Feb 7, 2011 at 11:29 PM, Carl Steinbach 
>> wrote:
>> >
>> >> Hi Abhinav,
>> >>
>> >> I am using a proxy.
>> >> > I am using cntlm for authentication.
>> >> > I have added localhost:1234 in ANT_OPTS ... as above before compiling
>> >> >
>> >> >
>> >> It looks like Ivy supports NTLM authentication:
>> >>
>> >>
>> >>
>> https://issues.apache.org/jira/browse/IVY-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>> >>
>> >> Can you try also setting -Dhttp.proxyRealm= in ANT_OPTS
>> and
>> >> let
>> >> us know if it works?
>> >>
>> >> Sadly, the above does not help.
>> > I get the same error.
>> > The following is a line from my .bashrc
>> > export ANT_OPTS="-Dhttp.proxyHost=localhost -Dhttp.proxyPort=1234
>> > -Dhttp.proxyRealm=iitm.ac.in"
>> >
>> >
>> >
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hbase/hbase/0.89.0-SNAPSHOT/
>> >
>> >> > >
>> $HOME/.ant/cache/org.apache.hbase/hbase/jars/hbase-0.89.0-SNAPSHOT.jar
>> >> > >
>> >> > > There is no folder as org.apache.hbase 
>> >> > but there are folders as org.apache.{zookeeper},{commons} etc ...
>> >> > Shall i create one and place the downloaded hbase jar there ?
>> >>
>> >>
>> >> Yes, if you can't get Ivy to work then you'll need to satisfy the
>> >> dependency
>> >> manually by downloading the hbase jar and copying it to
>> >> $HOME/.ant/cache/org/apache.hbase/hbase/jars/hbase-0.89.0-SNAPSHOT.jar
>> >>
>> > I have added the following jars in the folder
>> > $HOME/.ant/cache/org{.}apache.hbase/hbase/jars/
>> >
>> > Please note the dot{.} in the path instead of a slash{/}
>> >
>> > hbase-0.89.0-SNAPSHOT-tests.jar,  hbase-0.89.0-SNAPSHOT.jar,
>> > hbase-0.89.0-SNAPSHOT.pom
>> >
>> > But, I still get the same error as before !
>> > What should I do.
>> > Are there any xml fiels that have to be manually added too ?
>> >
>> > Abhinav Narain
>> >
>>
>> I am a big fan of quixotic quests, but have you considering building
>> on a machine without this proxy? Java byte-code is portable after all.
>>
>
>

[jira] Commented: (HIVE-1918) Add export/import facilities to the hive system

[Krishna Kumar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992036#comment-12992036
 ] 

Krishna Kumar commented on HIVE-1918:
-

Thanks, Namit, for the comments.

1. Ok re moving serialization/deserialization methods to EximUtil, but did not 
understand the first part. Are you suggesting moving EximUtil, 
ImportSemanticAnalyzer and ExportSemanticAnalyzer to a new package? Does not 
seem to warrant it; today all parsing/semantic analysis classes are in 
o.a.h.h.ql.parse package...

2. You mean Hive.java's API? The existing first createPartition remains as it 
is, the second createPartition used in DDLTasek is changing to allow the 
creation of a partition with all the partition-specific configurations. Since 
AddPartitionDesc is initialized with nulls/-1 for these extra parameters, the 
existing behaviour is not altered.

3. Can you expand a little? What are inputs/outputs (classes?, tables?) - if 
they are part of the existing object model/data model, I think they are 
exported and imported. 

> Add export/import facilities to the hive system
> ---
>
> Key: HIVE-1918
> URL: https://issues.apache.org/jira/browse/HIVE-1918
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Krishna Kumar
>Assignee: Krishna Kumar
> Attachments: HIVE-1918.patch.1.txt, HIVE-1918.patch.2.txt, 
> HIVE-1918.patch.3.txt, HIVE-1918.patch.txt, hive-metastore-er.pdf
>
>
> This is an enhancement request to add export/import features to hive.
> With this language extension, the user can export the data of the table - 
> which may be located in different hdfs locations in case of a partitioned 
> table - as well as the metadata of the table into a specified output 
> location. This output location can then be moved over to another different 
> hadoop/hive instance and imported there.  
> This should work independent of the source and target metastore dbms used; 
> for instance, between derby and mysql.
> For partitioned tables, the ability to export/import a subset of the 
> partition must be supported.
> Howl will add more features on top of this: The ability to create/use the 
> exported data even in the absence of hive, using MR or Pig. Please see 
> http://wiki.apache.org/pig/Howl/HowlImportExport for these details.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: HIVE-1971. Verbose/echo mode for the Hive CLI.

[Ning Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/404/#review191
---



cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java


Do you also need to add this piece of code for the case where proc is not 
an instance of Driver? Commands like 'set' and 'dfs' fall into that category.



common/src/java/org/apache/hadoop/hive/conf/HiveConf.java


I don't see why this should be set in HiveConf. IMO, verbose should be pure 
client side setting, it should not been passed to the server side.



ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java


Again, setting it in HiveConf may not be a good idea.



ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java


isSilent -> isVerbose.



ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java


same as above


- Ning


On 2011-02-07 23:16:17, Jonathan Natkins wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/404/
> ---
> 
> (Updated 2011-02-07 23:16:17)
> 
> 
> Review request for hive.
> 
> 
> Summary
> ---
> 
> This would add a mode to the CLI that echoes any commands that are executed 
> back to the command line.
> 
> 
> This addresses bug HIVE-1971.
> https://issues.apache.org/jira/browse/HIVE-1971
> 
> 
> Diffs
> -
> 
>   cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 2b52826 
>   cli/src/java/org/apache/hadoop/hive/cli/OptionsProcessor.java 93a7c95 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java a091b95 
>   ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 2b0dba8 
> 
> Diff: https://reviews.apache.org/r/404/diff
> 
> 
> Testing
> ---
> 
> Manual
> 
> 
> Thanks,
> 
> Jonathan
> 
>

[jira] Commented: (HIVE-1971) Verbose/echo mode for the Hive CLI

[Ning Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992027#comment-12992027
 ] 

Ning Zhang commented on HIVE-1971:
--

I will take a look.

> Verbose/echo mode for the Hive CLI
> --
>
> Key: HIVE-1971
> URL: https://issues.apache.org/jira/browse/HIVE-1971
> Project: Hive
>  Issue Type: New Feature
>  Components: CLI
>Reporter: Jonathan Natkins
>Assignee: Jonathan Natkins
> Attachments: HIVE-1971.1.patch.txt
>
>
> It would be very beneficial to have a mode which allows a user to run a SQL 
> script, and have each command echoed to the console as it's executed.  This 
> would be useful in figuring out which SQL statement is causing failures 
> during test runs, especially when running particularly long scripts.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Assigned: (HIVE-1969) TestMinimrCliDriver merge_dynamic_partition2 and 3 are failing on trunk

[Ning Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ning Zhang reassigned HIVE-1969:


Assignee: Ning Zhang

> TestMinimrCliDriver merge_dynamic_partition2 and 3 are failing on trunk
> ---
>
> Key: HIVE-1969
> URL: https://issues.apache.org/jira/browse/HIVE-1969
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: John Sichi
>Assignee: Ning Zhang
>Priority: Blocker
> Fix For: 0.7.0
>
>
> I haven't looked into it yet but saw this at the end of the .q.out:
> +Ended Job = job_201102071402_0020 with errors
> +FAILED: Execution Error, return code 2 from 
> org.apache.hadoop.hive.ql.exec.MapRedTask

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (HIVE-1969) TestMinimrCliDriver merge_dynamic_partition2 and 3 are failing on trunk

[Ning Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992024#comment-12992024
 ] 

Ning Zhang commented on HIVE-1969:
--

I will take a look.

> TestMinimrCliDriver merge_dynamic_partition2 and 3 are failing on trunk
> ---
>
> Key: HIVE-1969
> URL: https://issues.apache.org/jira/browse/HIVE-1969
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: John Sichi
>Assignee: Ning Zhang
>Priority: Blocker
> Fix For: 0.7.0
>
>
> I haven't looked into it yet but saw this at the end of the .q.out:
> +Ended Job = job_201102071402_0020 with errors
> +FAILED: Execution Error, return code 2 from 
> org.apache.hadoop.hive.ql.exec.MapRedTask

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (HIVE-1976) Exception should be thrown when invalid jar,file,archive is given to add command

[Chinna Rao Lalam (JIRA)]

Exception should be thrown when invalid jar,file,archive is given to add command


 Key: HIVE-1976
 URL: https://issues.apache.org/jira/browse/HIVE-1976
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 0.5.0
 Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 
10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam


When executed add command with non existing jar it should throw exception 
through   HiveStatement

Ex:
{noformat}
  add jar /root/invalidpath/testjar.jar
{noformat}

Here testjar.jar is not exist so it should throw exception.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: hadoop core 0.20.2 not found

[abhinav narain]

Only if I had a machine, that was not behind a proxy !

On Tue, Feb 8, 2011 at 8:43 PM, Edward Capriolo wrote:

> On Tue, Feb 8, 2011 at 5:40 AM, abhinav narain
>  wrote:
> > On Mon, Feb 7, 2011 at 11:29 PM, Carl Steinbach 
> wrote:
> >
> >> Hi Abhinav,
> >>
> >> I am using a proxy.
> >> > I am using cntlm for authentication.
> >> > I have added localhost:1234 in ANT_OPTS ... as above before compiling
> >> >
> >> >
> >> It looks like Ivy supports NTLM authentication:
> >>
> >>
> >>
> https://issues.apache.org/jira/browse/IVY-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> >>
> >> Can you try also setting -Dhttp.proxyRealm= in ANT_OPTS and
> >> let
> >> us know if it works?
> >>
> >> Sadly, the above does not help.
> > I get the same error.
> > The following is a line from my .bashrc
> > export ANT_OPTS="-Dhttp.proxyHost=localhost -Dhttp.proxyPort=1234
> > -Dhttp.proxyRealm=iitm.ac.in"
> >
> >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hbase/hbase/0.89.0-SNAPSHOT/
> >
> >> > >
> $HOME/.ant/cache/org.apache.hbase/hbase/jars/hbase-0.89.0-SNAPSHOT.jar
> >> > >
> >> > > There is no folder as org.apache.hbase 
> >> > but there are folders as org.apache.{zookeeper},{commons} etc ...
> >> > Shall i create one and place the downloaded hbase jar there ?
> >>
> >>
> >> Yes, if you can't get Ivy to work then you'll need to satisfy the
> >> dependency
> >> manually by downloading the hbase jar and copying it to
> >> $HOME/.ant/cache/org/apache.hbase/hbase/jars/hbase-0.89.0-SNAPSHOT.jar
> >>
> > I have added the following jars in the folder
> > $HOME/.ant/cache/org{.}apache.hbase/hbase/jars/
> >
> > Please note the dot{.} in the path instead of a slash{/}
> >
> > hbase-0.89.0-SNAPSHOT-tests.jar,  hbase-0.89.0-SNAPSHOT.jar,
> > hbase-0.89.0-SNAPSHOT.pom
> >
> > But, I still get the same error as before !
> > What should I do.
> > Are there any xml fiels that have to be manually added too ?
> >
> > Abhinav Narain
> >
>
> I am a big fan of quixotic quests, but have you considering building
> on a machine without this proxy? Java byte-code is portable after all.
>

[jira] Created: (HIVE-1975) "insert overwrite directory" Not able to insert data with multi level directory path

[Chinna Rao Lalam (JIRA)]

"insert overwrite directory" Not able to insert data with multi level directory 
path


 Key: HIVE-1975
 URL: https://issues.apache.org/jira/browse/HIVE-1975
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 0.5.0
 Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 
10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam


Below query execution is failed

Ex:
{noformat}
   insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
{noformat}

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (HIVE-1974) In error scenario some opened streams may not closed in ScriptOperator.java, Utilities.java

[Chinna Rao Lalam (JIRA)]

In error scenario some opened streams may not closed in ScriptOperator.java, 
Utilities.java 


 Key: HIVE-1974
 URL: https://issues.apache.org/jira/browse/HIVE-1974
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 0.5.0
 Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 
10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam


1)In error scenario StreamProcessor may not be closed in ScriptOperator.java
2)In error scenario XMLEncoder may not be closed in Utilities.java

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (HIVE-1973) Getting error when join on tables where name of table has uppercase letters

[Chinna Rao Lalam (JIRA)]

Getting error when join on tables where name of table has uppercase letters
---

 Key: HIVE-1973
 URL: https://issues.apache.org/jira/browse/HIVE-1973
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 0.5.0
 Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 
10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam


When execute a join query on tables containing Uppercase letters in the table 
names hit an exception

 Ex:
{noformat}
  create table a(b int);
  create table tabForJoin(b int,c int);

  select * from a join tabForJoin on(a.b=tabForJoin.b);

  Got an exception like this
  FAILED: Error in semantic analysis:  Invalid Table Alias tabForJoin
{noformat}

But if i give without capital letters ,It is working

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Created: (HIVE-1972) HiveResultset is always returning null for Array Data types in the select Query

[Chinna Rao Lalam (JIRA)]

HiveResultset is always returning null for Array Data types in the select Query
---

 Key: HIVE-1972
 URL: https://issues.apache.org/jira/browse/HIVE-1972
 Project: Hive
  Issue Type: Bug
  Components: JDBC
Affects Versions: 0.5.0
 Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 
10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam
Priority: Minor


Execute the following Hive Queries 
{noformat}
1) create table samplearray(a int,b int,c array)row format delimited 
fields terminated by '@' collection items terminated by '$' stored as textfile;

2) LOAD DATA INPATH '/user/dataloc/details3.txt' OVERWRITE INTO TABLE 
samplearray

3) Now execute the select statement "select c from emp;" using HiveStatement API

4) Now Iterate through the returned HiveResultSet, the array column is always 
null.
{noformat}

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: hadoop core 0.20.2 not found

[Edward Capriolo]

On Tue, Feb 8, 2011 at 5:40 AM, abhinav narain
 wrote:
> On Mon, Feb 7, 2011 at 11:29 PM, Carl Steinbach  wrote:
>
>> Hi Abhinav,
>>
>> I am using a proxy.
>> > I am using cntlm for authentication.
>> > I have added localhost:1234 in ANT_OPTS ... as above before compiling
>> >
>> >
>> It looks like Ivy supports NTLM authentication:
>>
>>
>> https://issues.apache.org/jira/browse/IVY-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>>
>> Can you try also setting -Dhttp.proxyRealm= in ANT_OPTS and
>> let
>> us know if it works?
>>
>> Sadly, the above does not help.
> I get the same error.
> The following is a line from my .bashrc
> export ANT_OPTS="-Dhttp.proxyHost=localhost -Dhttp.proxyPort=1234
> -Dhttp.proxyRealm=iitm.ac.in"
>
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hbase/hbase/0.89.0-SNAPSHOT/
>
>> > > $HOME/.ant/cache/org.apache.hbase/hbase/jars/hbase-0.89.0-SNAPSHOT.jar
>> > >
>> > > There is no folder as org.apache.hbase 
>> > but there are folders as org.apache.{zookeeper},{commons} etc ...
>> > Shall i create one and place the downloaded hbase jar there ?
>>
>>
>> Yes, if you can't get Ivy to work then you'll need to satisfy the
>> dependency
>> manually by downloading the hbase jar and copying it to
>> $HOME/.ant/cache/org/apache.hbase/hbase/jars/hbase-0.89.0-SNAPSHOT.jar
>>
> I have added the following jars in the folder
> $HOME/.ant/cache/org{.}apache.hbase/hbase/jars/
>
> Please note the dot{.} in the path instead of a slash{/}
>
> hbase-0.89.0-SNAPSHOT-tests.jar,  hbase-0.89.0-SNAPSHOT.jar,
> hbase-0.89.0-SNAPSHOT.pom
>
> But, I still get the same error as before !
> What should I do.
> Are there any xml fiels that have to be manually added too ?
>
> Abhinav Narain
>

I am a big fan of quixotic quests, but have you considering building
on a machine without this proxy? Java byte-code is portable after all.

Query regarding Hive-307 against latest Hive trunk

[Mohit]

Hello Namit/Kirk/Authors,

 

I have raise some concerns regarding fix of Hive-307, put comment in JIRA as
well.

 

It seems it is not working for query like (giving NullPointerException)

 

LOAD DATA INPATH './kv2.txt' INTO TABLE tst_src1;

 

which has the same purpose as the query being mentioned in test case for
this defect.

 

LOAD DATA LOCAL INPATH '/home/test/kv2.txt' INTO TABLE tst_src1;

 

Kindly confirm, if I'm doing anything wrong or it is known(and fixed) or it
is really an bug.

 

Thanks,

Mohit

 


***
This e-mail and attachments contain confidential information from HUAWEI,
which is intended only for the person or entity whose address is listed
above. Any use of the information contained herein in any way (including,
but not limited to, total or partial disclosure, reproduction, or
dissemination) by persons other than the intended recipient's) is
prohibited. If you receive this e-mail in error, please notify the sender by
phone or email immediately and delete it!

 

[jira] Commented: (HIVE-307) "LOAD DATA LOCAL INPATH" fails when the table already contains a file of the same name

[Mohit Sikri (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12991955#comment-12991955
 ] 

Mohit Sikri commented on HIVE-307:
--

Hello Namit/Kirk/He Yongqiang,

I was evaluating this defect, but it seems it is incomplete(falling short) of 
its cause.
It should also work fine for queries like

LOAD DATA INPATH './kv2.txt' INTO TABLE tst_src1;

But unfortunately :-( it is failing, giving null pointer exception in 
copyFiles() method [Hive.java]. I tested it against Hive0.7 snapshot of the 
latest build.

The difference between the query present in the test case and the one I 
mentioned   is the Path from which kv2.txt is fetched, the source path in 
former is a hdfs temp directory(say /tmp/hive-root/2097947077/1 under which 
kv2.txt is present) whereas in the later it is a hdfs file which is being 
actually referred. 

So the changes i.e. renaming done by checkPaths() method to avoid conflict is 
not visible outside it's scope ;i.e. not detected by fs.listStatus() where in 
actual the renamed file(say /user/root/kv2.txt_copy_1) is moved to tst_src1 
warehouse (say /user/hive/warehouse/tst_src1).

Kindly confirm, so that I can raise a new bug or re-open the existing one.

> "LOAD DATA LOCAL INPATH" fails when the table already contains a file of the 
> same name
> --
>
> Key: HIVE-307
> URL: https://issues.apache.org/jira/browse/HIVE-307
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.5.0, 0.6.0
>Reporter: Zheng Shao
>Assignee: Kirk True
>Priority: Critical
> Fix For: 0.7.0
>
> Attachments: HIVE-307.patch, HIVE-307.patch, HIVE-307.v3.patch, 
> hive-307.1.svn.patch
>
>
> Failed with exception checkPaths: 
> /user/zshao/warehouse/tmp_user_msg_history/test_user_msg_history already 
> exists
> FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.MoveTask

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: hadoop core 0.20.2 not found

[abhinav narain]

On Mon, Feb 7, 2011 at 11:29 PM, Carl Steinbach  wrote:

> Hi Abhinav,
>
> I am using a proxy.
> > I am using cntlm for authentication.
> > I have added localhost:1234 in ANT_OPTS ... as above before compiling
> >
> >
> It looks like Ivy supports NTLM authentication:
>
>
> https://issues.apache.org/jira/browse/IVY-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
>
> Can you try also setting -Dhttp.proxyRealm= in ANT_OPTS and
> let
> us know if it works?
>
> Sadly, the above does not help.
I get the same error.
The following is a line from my .bashrc
export ANT_OPTS="-Dhttp.proxyHost=localhost -Dhttp.proxyPort=1234
-Dhttp.proxyRealm=iitm.ac.in"


https://repository.apache.org/content/repositories/snapshots/org/apache/hbase/hbase/0.89.0-SNAPSHOT/

> > > $HOME/.ant/cache/org.apache.hbase/hbase/jars/hbase-0.89.0-SNAPSHOT.jar
> > >
> > > There is no folder as org.apache.hbase 
> > but there are folders as org.apache.{zookeeper},{commons} etc ...
> > Shall i create one and place the downloaded hbase jar there ?
>
>
> Yes, if you can't get Ivy to work then you'll need to satisfy the
> dependency
> manually by downloading the hbase jar and copying it to
> $HOME/.ant/cache/org/apache.hbase/hbase/jars/hbase-0.89.0-SNAPSHOT.jar
>
I have added the following jars in the folder
$HOME/.ant/cache/org{.}apache.hbase/hbase/jars/

Please note the dot{.} in the path instead of a slash{/}

hbase-0.89.0-SNAPSHOT-tests.jar,  hbase-0.89.0-SNAPSHOT.jar,
hbase-0.89.0-SNAPSHOT.pom

But, I still get the same error as before !
What should I do.
Are there any xml fiels that have to be manually added too ?

Abhinav Narain