[
https://issues.apache.org/jira/browse/HIVE-396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12805574#action_12805574
]
Vasilis Liaskovitis commented on HIVE-396:
------------------------------------------
I am trying to run the hive benchmarks on a small 8-node hadoop cluster,
starting from the 2009-07-21 tarball. Iam using hadoop-0.20.1 and hive-0.4.1. I
have a few questions, any help is welcome
- datasize: the report states 60GB of data used for the uservisitis table. The
default datagen/htmlgen/config.txt uses ~15GB of Uservisits tables per node. I
believe the way the generatedata.py script runs, we get one rankings/
uservisits table per node i.e. on an 10 data nodes we 'd get
10tablesx15GB=150GB of data. Do the benchmarks in the report use all of the
generated data i.e. UserVistis_0, UserVists_1,..., UserVistis_10 across the 10
data nodes? Or a subset of these? Same question for the rankings table.
- I assume that the hadoop-site.xml and hive-site.xml files under confs/ (from
the 2009-07-21 tarball) accurately reflect the hadoop/hive configuration used
to produce the results in the report. Let me know if that's not the case, I ve
been using these configs to run the benchmarks.
- Regarding query4 (join) specifically , the reduce phase of the first job
(main job with ) shows a strange behaviour. Looking at the jobtracker GUI, out
of 60 reducers, 59 reducers finish relatively fast (~1 min), but one reducer
task takes much longer (~19min). Does that point to a specific hadoop/ hive
misconfiguration? No other hadoop job that I 've run exhibits this. Is this
intuitive behaviour given a join operation?
> Hive performance benchmarks
> ---------------------------
>
> Key: HIVE-396
> URL: https://issues.apache.org/jira/browse/HIVE-396
> Project: Hadoop Hive
> Issue Type: New Feature
> Reporter: Zheng Shao
> Assignee: Yuntao Jia
> Attachments: AlansMRcode.tgz, hive_benchmark_2009-06-18.pdf,
> hive_benchmark_2009-06-18.tar.gz, hive_benchmark_2009-07-12.pdf,
> hive_benchmark_2009-07-21.tar.gz
>
>
> We need some performance benchmark to measure and track the performance
> improvements of Hive.
> Some references:
> PIG performance benchmarks PIG-200
> PigMix: http://wiki.apache.org/pig/PigMix
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
