> On Jan. 29, 2019, 2:04 a.m., Eugene Koifman wrote:
> > itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCrudCompactorOnTez.java
> > Lines 299 (patched)
> > <https://reviews.apache.org/r/69367/diff/7-9/?file=2121174#file2121174line328>
> >
> > testMoreBucketsThanReducers/testMoreBucketsThanReducers2 in
> > TestTxnCommands force a specific number of reducers
I used conf.setIntVar(HiveConf.ConfVars.HADOOPNUMREDUCERS, 2) on an unbucketed
table. However, the inserts created 2 different buckets. For example:
vgumashta:hive vgumashta$
../orc-git/build/_CPack_Packages/Darwin/TGZ/ORC-1.5.4-Darwin/bin/orc-contents
/Users/vgumashta/Documents/workspace/hive/itests/hive-unit/target/tmp/org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez-1549010487358_1258013156/warehouse/testcompactionwithschemaevolutionnobucketsmultiplereducers/ds\=today/delta_0000003_0000003_0000/bucket_00000
{"operation": 0, "originalTransaction": 3, "bucket": 536870912, "rowId": 0,
"currentTransaction": 3, "row": {"a": 3, "b": 3, "c": 1001}}
{"operation": 0, "originalTransaction": 3, "bucket": 536870912, "rowId": 1,
"currentTransaction": 3, "row": {"a": 4, "b": 2, "c": 1003}}
{"operation": 0, "originalTransaction": 3, "bucket": 536870912, "rowId": 2,
"currentTransaction": 3, "row": {"a": 4, "b": 4, "c": 1005}}
vgumashta:hive vgumashta$
../orc-git/build/_CPack_Packages/Darwin/TGZ/ORC-1.5.4-Darwin/bin/orc-contents
/Users/vgumashta/Documents/workspace/hive/itests/hive-unit/target/tmp/org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez-1549010487358_1258013156/warehouse/testcompactionwithschemaevolutionnobucketsmultiplereducers/ds\=yesterday/delta_0000003_0000003_0000/bucket_00001
{"operation": 0, "originalTransaction": 3, "bucket": 536936448, "rowId": 0,
"currentTransaction": 3, "row": {"a": 3, "b": 2, "c": 1000}}
{"operation": 0, "originalTransaction": 3, "bucket": 536936448, "rowId": 1,
"currentTransaction": 3, "row": {"a": 3, "b": 4, "c": 1002}}
{"operation": 0, "originalTransaction": 3, "bucket": 536936448, "rowId": 2,
"currentTransaction": 3, "row": {"a": 4, "b": 3, "c": 1004}}
- Vaibhav
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69367/#review212399
-----------------------------------------------------------
On Jan. 28, 2019, 7:49 p.m., Vaibhav Gumashta wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69367/
> -----------------------------------------------------------
>
> (Updated Jan. 28, 2019, 7:49 p.m.)
>
>
> Review request for hive and Eugene Koifman.
>
>
> Bugs: HIVE-20699
> https://issues.apache.org/jira/browse/HIVE-20699
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> https://jira.apache.org/jira/browse/HIVE-20699
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java b3a475478d
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/TestAcidOnTez.java
> d6a41919bf
>
> itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCrudCompactorOnTez.java
> PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e7aa041c25
> ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HiveSplitGenerator.java
> 15c14c9be5
> ql/src/java/org/apache/hadoop/hive/ql/exec/tez/SplitGrouper.java 7f8bd229a6
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRawRecordMerger.java
> fbb931cbcd
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRecordUpdater.java
> 6d4578e7a0
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSplit.java 4d55592b63
> ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
> db3b427adc
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java
> dc05e1990e
> ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Initiator.java
> a0df82cb20
>
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFValidateAcidSortOrder.java
> PRE-CREATION
> ql/src/test/results/clientpositive/show_functions.q.out c9716e904c
>
>
> Diff: https://reviews.apache.org/r/69367/diff/9/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Vaibhav Gumashta
>
>
