Hive-trunk-h0.21 - Build # 1167 - Still Failing

2011-12-24 Thread Apache Jenkins Server
Changes for Build #1144
[jvs] HIVE-1040 [jira] use sed rather than diff for masking out noise in 
diff-based
tests
(Marek Sapota via John Sichi)

Summary:
Replace diff -I with regex masking in Java

The current diff -I approach has two problems:  (1) it does not allow resolution
finer than line-level, so it's impossible to mask out pattern occurrences within
a line, and (2) it produces unmasked files, so if you run diff on the command
line to compare the result .q.out with the checked-in file, you see the noise.

My suggestion is to first run sed to replace noise patterns with an
unlikely-to-occur string like ZYZZYZVA, and then diff the pre-masked files
without using any -I.

This would require a one-time hit to update all existing .q.out files so that
they would contain the pre-masked results.

Test Plan: EMPTY

Reviewers: JIRA, jsichi

Reviewed By: jsichi

CC: jsichi

Differential Revision: 597


Changes for Build #1145

Changes for Build #1146
[namit] HIVE-2640 Add alterPartition to AlterHandler interface
(Kevin Wilfong via namit)


Changes for Build #1147
[namit] HIVE-2617 Insert overwrite table db.tname fails if partition already 
exists
(Chinna Rao Lalam via namit)


Changes for Build #1148
[heyongqiang] HIVE-2651 [jira] The variable hive.exec.mode.local.auto.tasks.max 
should be
changed
(Namit Jain via Yongqiang He)

Summary:
HIVE-2651

It should be called hive.exec.mode.local.auto.input.files.max instead.
The number of input files are checked currently.

Test Plan: EMPTY

Reviewers: JIRA, heyongqiang

Reviewed By: heyongqiang

CC: heyongqiang

Differential Revision: 861

[cws] HIVE-727. Hive Server getSchema() returns wrong schema for 'Explain' 
queries (Prasad Mujumdar via cws)

[namit] HIVE-2611 Make index table output of create index command if
index is table based (Kevin Wilfong via namit)


Changes for Build #1150
[jvs] HIVE-2657 [jira] builtins JAR is not being published to Maven repo & 
hive-cli
POM does not depend on it either
(Carl Steinbach via John Sichi)

Summary: Make hive-cli and hive-ql depend on hive-builtins

Test Plan: EMPTY

Reviewers: JIRA, jsichi

Reviewed By: jsichi

CC: jsichi

Differential Revision: 897

[namit] HIVE-2654 "hive.querylog.location" requires parent directory to be 
exist or
  else folder creation fails (Chinna Rao Lalam via namit)


Changes for Build #1151
[hashutosh] HIVE-1892 : show functions also returns internal operators 
(Priyadarshini via Ashutosh Chauhan)


Changes for Build #1152

Changes for Build #1153
[namit] HIVE-2660 Need better exception handling in RCFile tolerate corruptions
mode (Ramkumar Vadali via namit)


Changes for Build #1154
[cws] HIVE-2631. Make Hive work with Hadoop 1.0.0 (Ashutosh Chauhan via cws)


Changes for Build #1155
[cws] HIVE-BUILD. Update RELEASE_NOTES.txt with 0.8.0 release information (cws)


Changes for Build #1156

Changes for Build #1157

Changes for Build #1158
[namit] HIVE-2602 add support for insert partition overwrite(...) if not
  exists (Chinna Rao Lalam via namit)


Changes for Build #1159

Changes for Build #1160
[cws] HIVE-2005. Implement BETWEEN operator (Navis via cws)


Changes for Build #1161
[jvs] HIVE-2433. add DOAP file for Hive


Changes for Build #1162

Changes for Build #1163

Changes for Build #1164
[heyongqiang] HIVE-2666 [jira] StackOverflowError when using custom UDF in map 
join
(Kevin Wilfong via Yongqiang He)

Summary:
Resource files are now added to the class path as soon as they are added via the
CLI.  This fixes the stack overflow error mentioned in the JIRA by ensuring a
consistent class loader between serializers and deserializers for the same
query.

Note that now serdes which contain a static block to register themselves are now
registered twice, once when adding the file to the class loader, and once when
an instance of the class is created.  Previously, registering a serde twice
resulted in an exception, to avoid this, I have downgraded it to a warning.

When a custom UDF is used as part of a join which is converted to a map join,
the XMLEncoder enters an infinite loop when serializing the map reduce task for
the second time, as part of sending it to be executed.  This results in a stack
overflow error.

Test Plan:
I ran the unit tests to verify nothing was broken.

I ran several queries which used custom UDFs and involved a join which was
converted to a map join.  I verified these completed successfully consistently

Reviewers: JIRA, heyongqiang

Reviewed By: heyongqiang

CC: heyongqiang, kevinwilfong

Differential Revision: 957

[namit] HIVE-2642 fix Hive-2566 and make union optimization more aggressive
(Yongqiang He via namit)


Changes for Build #1166

Changes for Build #1167



8 tests failed.
REGRESSION:  
org.apache.hadoop.hive.cli.TestHBaseMinimrCliDriver.testCliDriver_hbase_bulk

Error Message:
Unexpected exception

Stack Trace:
junit.framework.AssertionFailedError: Unexpected exception
at junit.framework.Assert.fail(Assert.java:50)
at 
org.apac

[jira] [Updated] (HIVE-2600) Enable/Add type-specific compression for rcfile

2011-12-24 Thread He Yongqiang (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

He Yongqiang updated HIVE-2600:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

committed, thanks Krishna Kumar!

> Enable/Add type-specific compression for rcfile
> ---
>
> Key: HIVE-2600
> URL: https://issues.apache.org/jira/browse/HIVE-2600
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor, Serializers/Deserializers
>Reporter: Krishna Kumar
>Assignee: Krishna Kumar
>Priority: Minor
> Attachments: HIVE-2600.D1005.1.patch, HIVE-2600.D1005.2.patch, 
> HIVE-2600.D1005.3.patch, HIVE-2600.v0.patch, HIVE-2600.v1.patch
>
>
> Enable schema-aware compression codecs which can perform type-specific 
> compression on a per-column basis. I see this as in three-parts
> 1. Add interfaces for the rcfile to communicate column information to the 
> codec
> 2. Add an "uber compressor" which can perform column-specific compression on 
> a per-block basis. Initially, this can be config driven, but we can go for a 
> dynamic implementation later.
> 3. A bunch of type-specific compressors
> This jira is for the first part of the effort.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HIVE-2621) Allow multiple group bys with the same input data and spray keys to be run on the same reducer.

2011-12-24 Thread Phabricator (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175775#comment-13175775
 ] 

Phabricator commented on HIVE-2621:
---

heyongqiang has accepted the revision "HIVE-2621 [jira] Allow multiple group 
bys with the same input data and spray keys to be run on the same reducer.".

REVISION DETAIL
  https://reviews.facebook.net/D567


> Allow multiple group bys with the same input data and spray keys to be run on 
> the same reducer.
> ---
>
> Key: HIVE-2621
> URL: https://issues.apache.org/jira/browse/HIVE-2621
> Project: Hive
>  Issue Type: New Feature
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-2621.1.patch.txt, HIVE-2621.D567.1.patch, 
> HIVE-2621.D567.2.patch, HIVE-2621.D567.3.patch, HIVE-2621.D567.4.patch
>
>
> Currently, when a user runs a query, such as a multi-insert, where each 
> insertion subclause consists of a simple query followed by a group by, the 
> group bys for each clause are run on a separate reducer.  This requires 
> writing the data for each group by clause to an intermediate file, and then 
> reading it back.  This uses a significant amount of the total CPU consumed by 
> the query for an otherwise simple query.
> If the subclauses are grouped by their distinct expressions and group by 
> keys, with all of the group by expressions for a group of subclauses run on a 
> single reducer, this would reduce the amount of reading/writing to 
> intermediate files for some queries.
> To do this, for each group of subclauses, in the mapper we would execute a 
> the filters for each subclause 'or'd together (provided each subclause has a 
> filter) followed by a reduce sink.  In the reducer, the child operators would 
> be each subclauses filter followed by the group by and any subsequent 
> operations.
> Note that this would require turning off map aggregation, so we would need to 
> make using this type of plan configurable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Hive-trunk-h0.23.0 - Build # 46 - Still Failing

2011-12-24 Thread Apache Jenkins Server
Changes for Build #1

Changes for Build #2
[jvs] HIVE-2625. Fix maven-build Ant target
(Carl Steinbach via jvs)


Changes for Build #3

Changes for Build #4
[jvs] HIVE-2630 [jira] TestHiveServer doesn't produce a JUnit report file
(Marek Sapota via John Sichi)

Summary:
Remove property overrides

Run `ant test Dtestcase=TestHiveServer && ant testreport`, the test report
will be empty because not 'TEST*.xml' file was produced by JUnit.

Test Plan: EMPTY

Reviewers: JIRA, jsichi

Reviewed By: jsichi

CC: mareksapotafb, jsichi

Differential Revision: 621

[jvs] HIVE-2362. HiveConf properties not appearing in the output of 'set' or 
'set -v'
(Carl Steinbach via jvs)


Changes for Build #5
[hashutosh] HIVE-2467 : HA Support for Metastore Server (Thomas Weise via 
Ashutosh Chauhan)

[hashutosh] HIVE-2622 : Hive POMs reference the wrong Hadoop artifacts (Carl 
Steinbach via Ashutosh Chauhan)


Changes for Build #6

Changes for Build #7
[heyongqiang] HIVE-1003 [jira] optimize metadata only queries
(Namit Jain via Yongqiang He)

Summary:
testing

Queries like:

select max(ds) from T

where ds is a partitioning column should be optimized.

Test Plan: EMPTY

Reviewers: JIRA, heyongqiang

Reviewed By: heyongqiang

CC: njain, heyongqiang

Differential Revision: 105


Changes for Build #8

Changes for Build #9
[heyongqiang] HIVE-2634 [jira] revert HIVE-2566
(Namit Jain via Yongqiang He)

Summary:
HIVE-2634

This is leading to some problems.   

I will upload the offending testcase in a new jira.

Test Plan: EMPTY

Reviewers: JIRA, heyongqiang

Reviewed By: heyongqiang

CC: heyongqiang

Differential Revision: 651


Changes for Build #10
[heyongqiang] HIVE-2632 [jira] ignore exception for external jars via reflection
(Namit Jain via Yongqiang He)

Summary:
HIVE-2632



Test Plan: EMPTY

Reviewers: JIRA, heyongqiang

Reviewed By: heyongqiang

CC: heyongqiang

Differential Revision: 639


Changes for Build #11

Changes for Build #12
[heyongqiang] HIVE-2635 [jira] wrong class loader used for external jars
(Namit Jain via Yongqiang He)

Summary:
HIVE-2635



Test Plan: EMPTY

Reviewers: JIRA, heyongqiang

Reviewed By: heyongqiang

CC: heyongqiang

Differential Revision: 687

[namit] HIVE-2329 Not using map aggregation, fails to execute group-by after
cluster-by with same key (Navis via namit)


Changes for Build #13
[namit] HIVE-2628 move one line log from MapOperator to 
HiveContextAwareRecordReader
(He Yongqiang via namit)

[namit] HIVE-2520 left semi join will duplicate data
(binlijin via namit)

[jvs] HIVE-1487 [jira] Add ptest script
(Marek Sapota via John Sichi)

Summary: HIVE-1464 speeded up serial runs somewhat - but looks like it's still
too slow. we should use parallel junit or some similar setup to run test queries
in parallel. this should be really easy as well need to just use a separate
warehouse/metadb and potentiall mapred system dir location.

Test Plan: EMPTY

Reviewers: JIRA, jsichi

Reviewed By: jsichi

CC: mareksapotafb, jsichi

Differential Revision: 603


Changes for Build #14

Changes for Build #15

Changes for Build #16

Changes for Build #17
[jvs] Fix executable permissions left out of commit for HIVE-1487.


Changes for Build #18
[jvs] HIVE-2647 [jira] Force Bash shell on parallel test slave nodes
(Marek Sapota via John Sichi)

Summary:
Forcing Bash on each remote command call.

Force use of Bash shell, so user default shell choice doesn't impact the test
script.

Test Plan: EMPTY

Reviewers: JIRA, jsichi, njain

Reviewed By: jsichi

CC: mareksapotafb, jsichi

Differential Revision: 771

[jvs] HIVE-2648 [jira] Parallel tests fail if master directory is not present
(Marek Sapota via John Sichi)

Summary:
Fix the task downloading Ant.

Parallel tests should create directories as needed.

Test Plan: EMPTY

Reviewers: JIRA, jsichi

Reviewed By: jsichi

CC: mareksapotafb, jsichi

Differential Revision: 777


Changes for Build #19
[jvs] HIVE-2649 [jira] Allow multiple ptest runs by the same person
(Marek Sapota via John Sichi)

Summary:
Allow multiple ptest runs.

Allow running ptest simultaneously, multiple times by one person on the same
machines.

Test Plan: EMPTY

Reviewers: JIRA, jsichi

Reviewed By: jsichi

CC: mareksapotafb, jsichi

Differential Revision: 783

[jvs] HIVE-2650 [jira] Parallel test commands that include cd fail
(Marek Sapota via John Sichi)

Summary: Add quotes around bash -c to fix commands using cd.

Test Plan: EMPTY

Reviewers: JIRA, jsichi

Reviewed By: jsichi

CC: mareksapotafb, jsichi

Differential Revision: 801

[jvs] HIVE-2643. Recent patch prevents Hadoop confs from loading in 0.20.204
(Carl Steinbach via jvs)


Changes for Build #20

Changes for Build #21
[jvs] HIVE-1040 [jira] use sed rather than diff for masking out noise in 
diff-based
tests
(Marek Sapota via John Sichi)

Summary:
Replace diff -I with regex masking in Java

The current diff -I approach has two problems:  (1) it does not allow resolution
finer than line-level, so it's impossible to mask out pattern occurr

[jira] [Commented] (HIVE-2600) Enable/Add type-specific compression for rcfile

2011-12-24 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175777#comment-13175777
 ] 

Hudson commented on HIVE-2600:
--

Integrated in Hive-trunk-h0.23.0 #46 (See 
[https://builds.apache.org/job/Hive-trunk-h0.23.0/46/])
HIVE-2600: Enable/Add type-specific compression for rcfile (Krishna Kumar 
via He Yongqiang)

heyongqiang : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1223025
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SchemaAwareCompressionInputStream.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SchemaAwareCompressionOutputStream.java


> Enable/Add type-specific compression for rcfile
> ---
>
> Key: HIVE-2600
> URL: https://issues.apache.org/jira/browse/HIVE-2600
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor, Serializers/Deserializers
>Reporter: Krishna Kumar
>Assignee: Krishna Kumar
>Priority: Minor
> Attachments: HIVE-2600.D1005.1.patch, HIVE-2600.D1005.2.patch, 
> HIVE-2600.D1005.3.patch, HIVE-2600.v0.patch, HIVE-2600.v1.patch
>
>
> Enable schema-aware compression codecs which can perform type-specific 
> compression on a per-column basis. I see this as in three-parts
> 1. Add interfaces for the rcfile to communicate column information to the 
> codec
> 2. Add an "uber compressor" which can perform column-specific compression on 
> a per-block basis. Initially, this can be config driven, but we can go for a 
> dynamic implementation later.
> 3. A bunch of type-specific compressors
> This jira is for the first part of the effort.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Hive-trunk-h0.21 - Build # 1168 - Still Failing

2011-12-24 Thread Apache Jenkins Server
Changes for Build #1144
[jvs] HIVE-1040 [jira] use sed rather than diff for masking out noise in 
diff-based
tests
(Marek Sapota via John Sichi)

Summary:
Replace diff -I with regex masking in Java

The current diff -I approach has two problems:  (1) it does not allow resolution
finer than line-level, so it's impossible to mask out pattern occurrences within
a line, and (2) it produces unmasked files, so if you run diff on the command
line to compare the result .q.out with the checked-in file, you see the noise.

My suggestion is to first run sed to replace noise patterns with an
unlikely-to-occur string like ZYZZYZVA, and then diff the pre-masked files
without using any -I.

This would require a one-time hit to update all existing .q.out files so that
they would contain the pre-masked results.

Test Plan: EMPTY

Reviewers: JIRA, jsichi

Reviewed By: jsichi

CC: jsichi

Differential Revision: 597


Changes for Build #1145

Changes for Build #1146
[namit] HIVE-2640 Add alterPartition to AlterHandler interface
(Kevin Wilfong via namit)


Changes for Build #1147
[namit] HIVE-2617 Insert overwrite table db.tname fails if partition already 
exists
(Chinna Rao Lalam via namit)


Changes for Build #1148
[heyongqiang] HIVE-2651 [jira] The variable hive.exec.mode.local.auto.tasks.max 
should be
changed
(Namit Jain via Yongqiang He)

Summary:
HIVE-2651

It should be called hive.exec.mode.local.auto.input.files.max instead.
The number of input files are checked currently.

Test Plan: EMPTY

Reviewers: JIRA, heyongqiang

Reviewed By: heyongqiang

CC: heyongqiang

Differential Revision: 861

[cws] HIVE-727. Hive Server getSchema() returns wrong schema for 'Explain' 
queries (Prasad Mujumdar via cws)

[namit] HIVE-2611 Make index table output of create index command if
index is table based (Kevin Wilfong via namit)


Changes for Build #1150
[jvs] HIVE-2657 [jira] builtins JAR is not being published to Maven repo & 
hive-cli
POM does not depend on it either
(Carl Steinbach via John Sichi)

Summary: Make hive-cli and hive-ql depend on hive-builtins

Test Plan: EMPTY

Reviewers: JIRA, jsichi

Reviewed By: jsichi

CC: jsichi

Differential Revision: 897

[namit] HIVE-2654 "hive.querylog.location" requires parent directory to be 
exist or
  else folder creation fails (Chinna Rao Lalam via namit)


Changes for Build #1151
[hashutosh] HIVE-1892 : show functions also returns internal operators 
(Priyadarshini via Ashutosh Chauhan)


Changes for Build #1152

Changes for Build #1153
[namit] HIVE-2660 Need better exception handling in RCFile tolerate corruptions
mode (Ramkumar Vadali via namit)


Changes for Build #1154
[cws] HIVE-2631. Make Hive work with Hadoop 1.0.0 (Ashutosh Chauhan via cws)


Changes for Build #1155
[cws] HIVE-BUILD. Update RELEASE_NOTES.txt with 0.8.0 release information (cws)


Changes for Build #1156

Changes for Build #1157

Changes for Build #1158
[namit] HIVE-2602 add support for insert partition overwrite(...) if not
  exists (Chinna Rao Lalam via namit)


Changes for Build #1159

Changes for Build #1160
[cws] HIVE-2005. Implement BETWEEN operator (Navis via cws)


Changes for Build #1161
[jvs] HIVE-2433. add DOAP file for Hive


Changes for Build #1162

Changes for Build #1163

Changes for Build #1164
[heyongqiang] HIVE-2666 [jira] StackOverflowError when using custom UDF in map 
join
(Kevin Wilfong via Yongqiang He)

Summary:
Resource files are now added to the class path as soon as they are added via the
CLI.  This fixes the stack overflow error mentioned in the JIRA by ensuring a
consistent class loader between serializers and deserializers for the same
query.

Note that now serdes which contain a static block to register themselves are now
registered twice, once when adding the file to the class loader, and once when
an instance of the class is created.  Previously, registering a serde twice
resulted in an exception, to avoid this, I have downgraded it to a warning.

When a custom UDF is used as part of a join which is converted to a map join,
the XMLEncoder enters an infinite loop when serializing the map reduce task for
the second time, as part of sending it to be executed.  This results in a stack
overflow error.

Test Plan:
I ran the unit tests to verify nothing was broken.

I ran several queries which used custom UDFs and involved a join which was
converted to a map join.  I verified these completed successfully consistently

Reviewers: JIRA, heyongqiang

Reviewed By: heyongqiang

CC: heyongqiang, kevinwilfong

Differential Revision: 957

[namit] HIVE-2642 fix Hive-2566 and make union optimization more aggressive
(Yongqiang He via namit)


Changes for Build #1166

Changes for Build #1167

Changes for Build #1168
[heyongqiang] HIVE-2600: Enable/Add type-specific compression for rcfile 
(Krishna Kumar via He Yongqiang)




3 tests failed.
REGRESSION:  org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppr_pushdown

Error Message:
Unexpected exception See build/ql/tmp/hive.log

[jira] [Commented] (HIVE-2600) Enable/Add type-specific compression for rcfile

2011-12-24 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175790#comment-13175790
 ] 

Hudson commented on HIVE-2600:
--

Integrated in Hive-trunk-h0.21 #1168 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1168/])
HIVE-2600: Enable/Add type-specific compression for rcfile (Krishna Kumar 
via He Yongqiang)

heyongqiang : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1223025
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SchemaAwareCompressionInputStream.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/SchemaAwareCompressionOutputStream.java


> Enable/Add type-specific compression for rcfile
> ---
>
> Key: HIVE-2600
> URL: https://issues.apache.org/jira/browse/HIVE-2600
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor, Serializers/Deserializers
>Reporter: Krishna Kumar
>Assignee: Krishna Kumar
>Priority: Minor
> Attachments: HIVE-2600.D1005.1.patch, HIVE-2600.D1005.2.patch, 
> HIVE-2600.D1005.3.patch, HIVE-2600.v0.patch, HIVE-2600.v1.patch
>
>
> Enable schema-aware compression codecs which can perform type-specific 
> compression on a per-column basis. I see this as in three-parts
> 1. Add interfaces for the rcfile to communicate column information to the 
> codec
> 2. Add an "uber compressor" which can perform column-specific compression on 
> a per-block basis. Initially, this can be config driven, but we can go for a 
> dynamic implementation later.
> 3. A bunch of type-specific compressors
> This jira is for the first part of the effort.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HIVE-2604) Add UberCompressor Serde/Codec to contrib which allows per-column compression strategies

2011-12-24 Thread Phabricator (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-2604:
--

Attachment: HIVE-2604.D1011.1.patch

krishnakumar requested code review of "HIVE-2604 [jira] Add UberCompressor 
Serde/Codec to contrib which allows per-column compression strategies".
Reviewers: JIRA

  added serde/codec which provide for type-specific compression mechanisms

  The strategies supported are
  1. using a specified codec on the column
  2. using a specific codec on the column which is serialized via a specific 
serde
  3. using a specific "TypeSpecificCompressor" instance

TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D1011

AFFECTED FILES
  
contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/InputReader.java
  
contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/OutputWriter.java
  
contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/TypeSpecificCompressor.java
  
contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/UberCompressionCodec.java
  
contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/UberCompressionInputStream.java
  
contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/UberCompressionOutputStream.java
  
contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/UberCompressorColumnConfig.java
  
contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/UberCompressorConfig.java
  
contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/UberCompressorSerde.java
  
contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/UberCompressorSerdeField.java
  
contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/UberCompressorUtils.java
  
contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/compressors/DummyIntegerCompressor.java
  
contrib/src/java/org/apache/hadoop/hive/contrib/ubercompressor/dsalg/Tuple.java
  contrib/src/test/queries/clientpositive/ubercompressor.q
  contrib/src/test/results/clientpositive/ubercompressor.q.out

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/2121/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


> Add UberCompressor Serde/Codec to contrib which allows per-column compression 
> strategies
> 
>
> Key: HIVE-2604
> URL: https://issues.apache.org/jira/browse/HIVE-2604
> Project: Hive
>  Issue Type: Sub-task
>  Components: Contrib
>Reporter: Krishna Kumar
>Assignee: Krishna Kumar
> Attachments: HIVE-2604.D1011.1.patch, HIVE-2604.v0.patch, 
> HIVE-2604.v1.patch, HIVE-2604.v2.patch
>
>
> The strategies supported are
> 1. using a specified codec on the column
> 2. using a specific codec on the column which is serialized via a specific 
> serde
> 3. using a specific "TypeSpecificCompressor" instance

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira