-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8611/
-----------------------------------------------------------
(Updated Dec. 15, 2012, 1:21 a.m.)
Review request for giraph.
Description (updated)
-------
GIRAPH-453: Pure Hive I/O (nitay)
For now this is only the Input side of things. One particular thing I added was
the concept of "profiles", allowing for easily reading from multiple tables.
This should remove a lot of the cruft around the GiraphHCat* classes.
In terms of performance this is on par with our current HCatalog stuff. I ran a
few jobs and noticed at most a few seconds of difference between the input
supersteps.
This addresses bug GIRAPH-453.
https://issues.apache.org/jira/browse/GIRAPH-453
Diffs
-----
giraph-formats-contrib/pom.xml 9326c28420be2749fa724ebd86d9555ae209a2ee
giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/HCatGiraphRunner.java
PRE-CREATION
giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/HiveGiraphRunner.java
7a7c2f87bc50eaebe879e2eff2da661b504096b0
giraph-hive/pom.xml PRE-CREATION
giraph-hive/src/main/assembly/compile.xml PRE-CREATION
giraph-hive/src/main/java/org/apache/giraph/hive/HiveConstants.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/giraph/hive/HiveEdgeInputFormat.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/giraph/hive/HiveEdgeReader.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/giraph/hive/HiveGiraphRunner.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/giraph/hive/HiveUtils.java PRE-CREATION
giraph-hive/src/main/java/org/apache/giraph/hive/HiveVertexCreator.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/giraph/hive/HiveVertexInputFormat.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/giraph/hive/HiveVertexReader.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/giraph/hive/VertexCreator.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/HiveApiInputFormat.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/HiveApiOutputFormat.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/HiveInputDescription.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/HiveOutputDescription.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/HiveRecord.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/common/Classes.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/common/HadoopUtils.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/common/ProgressReporter.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/common/SerDes.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/common/Writables.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/input/ApiInputSplit.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/input/ApiRecord.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/input/ApiRecordReader.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/input/InputConf.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/input/InputPartition.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/input/InputPartitions.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/input/InputSplitData.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/inspect/ListParser.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/inspect/MapParser.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/inspect/ObjectParser.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/inspect/Parsers.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/inspect/PrimitiveParser.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/inspect/StructParser.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/inspect/UnionParser.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/output/OutputConf.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/schema/MaterializedField.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/schema/MaterializedList.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/schema/MaterializedMap.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/schema/MaterializedPrimitive.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/schema/MaterializedSchema.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/schema/MaterializedStruct.java
PRE-CREATION
giraph-hive/src/main/java/org/apache/hadoop/hive/api/schema/MaterializedUnion.java
PRE-CREATION
pom.xml 47736351c45f4cdb55df762506069eaed7126a7e
Diff: https://reviews.apache.org/r/8611/diff/
Testing
-------
Thanks,
Nitay Joffe
