[
https://issues.apache.org/jira/browse/HIVE-6147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13909967#comment-13909967
]
Swarnim Kulkarni commented on HIVE-6147:
----------------------------------------
I am not sure what's going on here. I checked out the latest trunk, applied the
patch and re-ran the failing tests locally and they all passed!
{noformat}
mac-swarnim:hive swarnim$ git pull --rebase
Current branch trunk is up to date.
mac-swarnim:hive swarnim$ wget
https://issues.apache.org/jira/secure/attachment/12629556/HIVE-6147.3.patch.txt
--2014-02-23 18:40:27--
https://issues.apache.org/jira/secure/attachment/12629556/HIVE-6147.3.patch.txt
Resolving issues.apache.org... 140.211.11.121
Connecting to issues.apache.org|140.211.11.121|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 212743 (208K) [text/plain]
Saving to: `HIVE-6147.3.patch.txt'
100%[=====================================================================================================================>]
212,743 29.5K/s in 7.0s
2014-02-23 18:40:37 (29.5 KB/s) - `HIVE-6147.3.patch.txt' saved [212743/212743]
mac-swarnim:hive swarnim$ git status
# On branch trunk
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# HIVE-6147.3.patch.txt
nothing added to commit but untracked files present (use "git add" to track)
mac-swarnim:hive swarnim$ patch -p0 < HIVE-6147.3.patch.txt
patching file hbase-handler/pom.xml
patching file
hbase-handler/src/gen/avro/gen-java/org/apache/hadoop/hive/hbase/avro/Address.java
patching file
hbase-handler/src/gen/avro/gen-java/org/apache/hadoop/hive/hbase/avro/ContactInfo.java
patching file
hbase-handler/src/gen/avro/gen-java/org/apache/hadoop/hive/hbase/avro/Employee.java
patching file
hbase-handler/src/gen/avro/gen-java/org/apache/hadoop/hive/hbase/avro/EmployeeAvro.java
patching file
hbase-handler/src/gen/avro/gen-java/org/apache/hadoop/hive/hbase/avro/Gender.java
patching file
hbase-handler/src/gen/avro/gen-java/org/apache/hadoop/hive/hbase/avro/HomePhone.java
patching file
hbase-handler/src/gen/avro/gen-java/org/apache/hadoop/hive/hbase/avro/Magic.java
patching file
hbase-handler/src/gen/avro/gen-java/org/apache/hadoop/hive/hbase/avro/OfficePhone.java
patching file hbase-handler/src/if/avro/avro_test.avpr
patching file
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseCompositeKey.java
patching file
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
patching file
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDeHelper.java
patching file
hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseCellMap.java
patching file
hbase-handler/src/test/org/apache/hadoop/hive/hbase/HBaseTestAvroSchemaRetriever.java
patching file
hbase-handler/src/test/org/apache/hadoop/hive/hbase/HBaseTestCompositeKey.java
patching file
hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestHBaseSerDe.java
patching file
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroGenericRecordWritable.java
patching file
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroLazyObjectInspector.java
patching file
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroObjectInspectorException.java
patching file
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroObjectInspectorGenerator.java
patching file
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSchemaRetriever.java
patching file
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java
patching file serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java
patching file serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyStruct.java
patching file serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUnion.java
patching file
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/LazyObjectInspectorFactory.java
patching file
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/LazySimpleStructObjectInspector.java
patching file
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java
mac-swarnim:hive swarnim$ git status
# On branch trunk
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: hbase-handler/pom.xml
# modified:
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseCompositeKey.java
# modified:
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
# modified:
hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseCellMap.java
# modified:
hbase-handler/src/test/org/apache/hadoop/hive/hbase/HBaseTestCompositeKey.java
# modified:
hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestHBaseSerDe.java
# modified:
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroGenericRecordWritable.java
# modified:
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroObjectInspectorGenerator.java
# modified:
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java
# modified:
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java
# modified:
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyStruct.java
# modified:
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUnion.java
# modified:
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/LazyObjectInspectorFactory.java
# modified:
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/LazySimpleStructObjectInspector.java
# modified:
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java
#
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# HIVE-6147.3.patch.txt
# hbase-handler/src/gen/
# hbase-handler/src/if/
#
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDeHelper.java
#
hbase-handler/src/test/org/apache/hadoop/hive/hbase/HBaseTestAvroSchemaRetriever.java
#
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroLazyObjectInspector.java
#
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroObjectInspectorException.java
#
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSchemaRetriever.java
mac-swarnim:hive swarnim$ cd hcatalog/
mac-swarnim:hcatalog swarnim$ mvn clean install -pl hcatalog-pig-adapter -P
hadoop-1
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Hive HCatalog Pig Adapter 0.13.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @
hive-hcatalog-pig-adapter ---
[INFO] Deleting
/Users/swarnim/git-repo/apache/hive/hcatalog/hcatalog-pig-adapter/target
[INFO] Deleting
/Users/swarnim/git-repo/apache/hive/hcatalog/hcatalog-pig-adapter (includes =
[datanucleus.log, derby.log], excludes = [])
[INFO] Deleting
/Users/swarnim/git-repo/apache/hive/hcatalog/hcatalog-pig-adapter/build
(includes = [], excludes = [])
[INFO]
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @
hive-hcatalog-pig-adapter ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/Users/swarnim/git-repo/apache/hive/hcatalog/hcatalog-pig-adapter/src/main/resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @
hive-hcatalog-pig-adapter ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @
hive-hcatalog-pig-adapter ---
[INFO] Compiling 10 source files to
/Users/swarnim/git-repo/apache/hive/hcatalog/hcatalog-pig-adapter/target/classes
[WARNING] Note: Some input files use or override a deprecated API.
[WARNING] Note: Recompile with -Xlint:deprecation for details.
[WARNING] Note: Some input files use unchecked or unsafe operations.
[WARNING] Note: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @
hive-hcatalog-pig-adapter ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/Users/swarnim/git-repo/apache/hive/hcatalog/hcatalog-pig-adapter/src/test/resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @
hive-hcatalog-pig-adapter ---
[INFO] Executing tasks
main:
[mkdir] Created dir:
/Users/swarnim/git-repo/apache/hive/hcatalog/hcatalog-pig-adapter/target/tmp
[mkdir] Created dir:
/Users/swarnim/git-repo/apache/hive/hcatalog/hcatalog-pig-adapter/target/warehouse
[mkdir] Created dir:
/Users/swarnim/git-repo/apache/hive/hcatalog/hcatalog-pig-adapter/target/tmp/conf
[copy] Copying 5 files to
/Users/swarnim/git-repo/apache/hive/hcatalog/hcatalog-pig-adapter/target/tmp/conf
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @
hive-hcatalog-pig-adapter ---
[INFO] Compiling 26 source files to
/Users/swarnim/git-repo/apache/hive/hcatalog/hcatalog-pig-adapter/target/test-classes
[WARNING] Note: Some input files use or override a deprecated API.
[WARNING] Note: Recompile with -Xlint:deprecation for details.
[INFO]
[INFO] --- maven-surefire-plugin:2.16:test (default-test) @
hive-hcatalog-pig-adapter ---
[INFO] Surefire report directory:
/Users/swarnim/git-repo/apache/hive/hcatalog/hcatalog-pig-adapter/target/surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.hcatalog.pig.TestHCatLoader
2014-02-23 16:42:49.909 java[65062:1903] Unable to load realm info from
SCDynamicStore
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 55.476 sec - in
org.apache.hcatalog.pig.TestHCatLoader
Running org.apache.hcatalog.pig.TestHCatLoaderComplexSchema
2014-02-23 16:43:45.590 java[65112:1903] Unable to load realm info from
SCDynamicStore
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.887 sec - in
org.apache.hcatalog.pig.TestHCatLoaderComplexSchema
Running org.apache.hcatalog.pig.TestHCatLoaderStorer
2014-02-23 16:44:14.765 java[65319:1903] Unable to load realm info from
SCDynamicStore
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.618 sec - in
org.apache.hcatalog.pig.TestHCatLoaderStorer
Running org.apache.hcatalog.pig.TestHCatStorer
2014-02-23 16:44:26.599 java[65328:1903] Unable to load realm info from
SCDynamicStore
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.412 sec -
in org.apache.hcatalog.pig.TestHCatStorer
Running org.apache.hcatalog.pig.TestHCatStorerMulti
2014-02-23 16:44:47.482 java[65363:1903] Unable to load realm info from
SCDynamicStore
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.681 sec - in
org.apache.hcatalog.pig.TestHCatStorerMulti
Running org.apache.hcatalog.pig.TestHCatStorerWrapper
2014-02-23 16:45:01.495 java[65377:1903] Unable to load realm info from
SCDynamicStore
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.07 sec - in
org.apache.hcatalog.pig.TestHCatStorerWrapper
Running org.apache.hcatalog.pig.TestOrcHCatLoader
2014-02-23 16:45:06.985 java[65384:1903] Unable to load realm info from
SCDynamicStore
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 56.718 sec - in
org.apache.hcatalog.pig.TestOrcHCatLoader
Running org.apache.hcatalog.pig.TestOrcHCatStorer
2014-02-23 16:46:04.004 java[65438:1903] Unable to load realm info from
SCDynamicStore
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.293 sec - in
org.apache.hcatalog.pig.TestOrcHCatStorer
Running org.apache.hcatalog.pig.TestPigHCatUtil
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.512 sec - in
org.apache.hcatalog.pig.TestPigHCatUtil
Running org.apache.hive.hcatalog.pig.TestE2EScenarios
2014-02-23 16:46:16.502 java[65454:1903] Unable to load realm info from
SCDynamicStore
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.143 sec - in
org.apache.hive.hcatalog.pig.TestE2EScenarios
Running org.apache.hive.hcatalog.pig.TestHCatLoader
2014-02-23 16:46:25.991 java[65461:1903] Unable to load realm info from
SCDynamicStore
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 64.261 sec - in
org.apache.hive.hcatalog.pig.TestHCatLoader
Running org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema
2014-02-23 16:47:30.547 java[65536:1903] Unable to load realm info from
SCDynamicStore
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.867 sec - in
org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema
Running org.apache.hive.hcatalog.pig.TestHCatLoaderStorer
2014-02-23 16:47:59.693 java[65744:1903] Unable to load realm info from
SCDynamicStore
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.167 sec - in
org.apache.hive.hcatalog.pig.TestHCatLoaderStorer
Running org.apache.hive.hcatalog.pig.TestHCatStorer
2014-02-23 16:48:11.201 java[65754:1903] Unable to load realm info from
SCDynamicStore
Tests run: 27, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 101.175 sec -
in org.apache.hive.hcatalog.pig.TestHCatStorer
Running org.apache.hive.hcatalog.pig.TestHCatStorerMulti
2014-02-23 16:49:50.434 java[65845:1903] Unable to load realm info from
SCDynamicStore
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.798 sec - in
org.apache.hive.hcatalog.pig.TestHCatStorerMulti
Running org.apache.hive.hcatalog.pig.TestHCatStorerWrapper
2014-02-23 16:50:05.471 java[65860:1903] Unable to load realm info from
SCDynamicStore
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.097 sec - in
org.apache.hive.hcatalog.pig.TestHCatStorerWrapper
Running org.apache.hive.hcatalog.pig.TestOrcHCatLoader
2014-02-23 16:50:11.008 java[65866:1903] Unable to load realm info from
SCDynamicStore
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 64.125 sec - in
org.apache.hive.hcatalog.pig.TestOrcHCatLoader
Running org.apache.hive.hcatalog.pig.TestOrcHCatLoaderComplexSchema
2014-02-23 16:51:15.432 java[65940:1903] Unable to load realm info from
SCDynamicStore
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.779 sec - in
org.apache.hive.hcatalog.pig.TestOrcHCatLoaderComplexSchema
Running org.apache.hive.hcatalog.pig.TestOrcHCatStorer
2014-02-23 16:51:42.577 java[66137:1903] Unable to load realm info from
SCDynamicStore
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.179 sec - in
org.apache.hive.hcatalog.pig.TestOrcHCatStorer
Running org.apache.hive.hcatalog.pig.TestPigHCatUtil
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.513 sec - in
org.apache.hive.hcatalog.pig.TestPigHCatUtil
Results :
Tests run: 101, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-hcatalog-pig-adapter
---
[INFO] Building jar:
/Users/swarnim/git-repo/apache/hive/hcatalog/hcatalog-pig-adapter/target/hive-hcatalog-pig-adapter-0.13.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @
hive-hcatalog-pig-adapter ---
[INFO] Installing
/Users/swarnim/git-repo/apache/hive/hcatalog/hcatalog-pig-adapter/target/hive-hcatalog-pig-adapter-0.13.0-SNAPSHOT.jar
to
/Users/swarnim/.m2/repository/org/apache/hive/hcatalog/hive-hcatalog-pig-adapter/0.13.0-SNAPSHOT/hive-hcatalog-pig-adapter-0.13.0-SNAPSHOT.jar
[INFO] Installing
/Users/swarnim/git-repo/apache/hive/hcatalog/hcatalog-pig-adapter/pom.xml to
/Users/swarnim/.m2/repository/org/apache/hive/hcatalog/hive-hcatalog-pig-adapter/0.13.0-SNAPSHOT/hive-hcatalog-pig-adapter-0.13.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 9:11.629s
[INFO] Finished at: Sun Feb 23 18:51:54 CST 2014
[INFO] Final Memory: 24M/81M
[INFO] ------------------------------------------------------------------------
{noformat}
> Support avro data stored in HBase columns
> -----------------------------------------
>
> Key: HIVE-6147
> URL: https://issues.apache.org/jira/browse/HIVE-6147
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
> Affects Versions: 0.12.0
> Reporter: Swarnim Kulkarni
> Assignee: Swarnim Kulkarni
> Attachments: HIVE-6147.1.patch.txt, HIVE-6147.2.patch.txt,
> HIVE-6147.3.patch.txt
>
>
> Presently, the HBase Hive integration supports querying only primitive data
> types in columns. It would be nice to be able to store and query Avro objects
> in HBase columns by making them visible as structs to Hive. This will allow
> Hive to perform ad hoc analysis of HBase data which can be deeply structured.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
