[
https://issues.apache.org/jira/browse/BIGTOP-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177006#comment-14177006
]
jay vyas commented on BIGTOP-1450:
----------------------------------
[~jbx] agreed... So, Shall we just gut them entirely... and evolve the hive
tests in **smoke-tests** over time ? In favor of the new and easy to run
{{smoke-tests}} framework ? The new smoke-tests are here
https://github.com/apache/bigtop/tree/master/bigtop-tests/smoke-tests ... I
beleive ive discussed this with you before breifly, and would love to migrate
entirely over to them (no jar required, no complex directory bifurcation,
totally modifiable and hackable, uses gradle instead of maven, and so on),.
> hive smoke test : possibly out of sync, need review, and hard to debug
> ----------------------------------------------------------------------
>
> Key: BIGTOP-1450
> URL: https://issues.apache.org/jira/browse/BIGTOP-1450
> Project: Bigtop
> Issue Type: Improvement
> Components: tests
> Reporter: jay vyas
> Assignee: jay vyas
> Fix For: 0.9.0
>
>
> *Overall: The hive tests in {{test-artifacts}} are prone to failures from
> missing data sets and generally need a thorough review*
> When testing bigtop 0.8.0 release candidate, I found that I got some errors
> {noformat}
> [--- /dev/fd/63 2014-09-16 10:12:54.579647323 +0000, +++ /dev/fd/62
> 2014-09-16 10:12:54.579647323 +0000, @@ -14,4 +14,4 @@, INSERT OVERWRITE
> DIRECTORY '/tmp/count', SELECT COUNT(1) FROM u_data, dfs -cat /tmp/count/*,
> -0, +100000] err=[14/09/16 10:12:17 WARN mapred.JobConf: The variable
> mapred.child.ulimit is no longer used., , Logging initialized using
> configuration in file:/etc/hive/conf.dist/hive-log4j.properties, OK, Time
> taken: 2.609 seconds, OK, Time taken: 0.284 seconds, Total jobs = 1,
> Launching Job 1 out of 1, Number of reduce tasks determined at compile time:
> 1, In order to change the average load for a reducer (in bytes):, set
> hive.exec.reducers.bytes.per.reducer=<number>, In order to limit the
> maximum number of reducers:, set hive.exec.reducers.max=<number>, In
> order to set a constant number of reducers:, set
> mapreduce.job.reduces=<number>, Starting Job = job_1410830363557_0019,
> Tracking URL =
> http://bigtop1.vagrant:20888/proxy/application_1410830363557_0019/, Kill
> Command = /usr/lib/hadoop/bin/hadoop job -kill job_1410830363557_0019,
> Hadoop job information for Stage-1: number of mappers: 1; number of reducers:
> 1, 2014-09-16 10:12:38,870 Stage-1 map = 0%, reduce = 0%, 2014-09-16
> 10:12:45,516 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 0.81 sec,
> 2014-09-16 10:12:53,036 Stage-1 map = 100%, reduce = 100%, Cumulative CPU
> 1.73 sec, MapReduce Total cumulative CPU time: 1 seconds 730 msec, Ended Job
> = job_1410830363557_0019, Moving data to: /tmp/count, MapReduce Jobs
> Launched: , Job 0: Map: 1 Reduce: 1 Cumulative CPU: 1.73 sec HDFS Read:
> 272 HDFS Write: 2 SUCCESS, Total MapReduce CPU Time Spent: 1 seconds 730
> msec, OK, Time taken: 24.594 seconds
> {noformat}
> I know there is a diff error in here - some kind of diff is going on , but I
> forgot how the actual,output,and filter are working.
> In any case, I think these tests can be simplified to just grep for a output
> string and check error code, or else, at least add some very clear assertions
> as to what failures may be.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
