2014-07-18 15:03:37,774 INFO mr.ExecDriver
(SessionState.java:printInfo(537)) - Execution log at:
/tmp/myuser/myuser_2014071815030
3_56bf6bb0-db30-4dbc-807c-9023ce4103f4.log
2014-07-18 15:03:37,864 WARN conf.Configuration
(Configuration.java:loadProperty(2358)) -
file:/tmp/myuser/hive_2014-07-18_15-03-30_423_
6799963466906099923-1/-local-10011/jobconf.xml:an attempt to override final
parameter: mapreduce.job.end-notification.max.retry.interval; Igno
ring.
2014-07-18 15:03:37,871 WARN conf.Configuration
(Configuration.java:loadProperty(2358)) -
file:/tmp/myuser/hive_2014-07-18_15-03-30_423_
6799963466906099923-1/-local-10011/jobconf.xml:an attempt to override final
parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-07-18 15:03:37,951 INFO log.PerfLogger
(PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=deserializePlan
from=org.apache.hadoop.hive.
ql.exec.Utilities>
2014-07-18 15:03:37,951 INFO exec.Utilities
(Utilities.java:deserializePlan(822)) - Deserializing MapredLocalWork via
kryo
2014-07-18 15:03:38,237 INFO log.PerfLogger
(PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=deserializePlan
start=1405721017951 end=14057
21018237 duration=286 from=org.apache.hadoop.hive.ql.exec.Utilities>
2014-07-18 15:03:38,246 INFO mr.MapredLocalTask
(SessionState.java:printInfo(537)) - 2014-07-18 03:03:38 Starting to
launch local task t
o process map join; maximum memory = 4261937152
2014-07-18 15:03:38,261 INFO mr.MapredLocalTask
(MapredLocalTask.java:initializeOperators(406)) - fetchoperator for
null-subquery2:a-subquery2
:dpkg_cntr:dpkg_wtransaction_p2_id_user_30m created
2014-07-18 15:03:38,263 INFO mr.MapredLocalTask
(MapredLocalTask.java:initializeOperators(406)) - fetchoperator for
null-subquery2:a-subquery2
:dpkg:dpkg_wtransaction_p2_id_user_30m created
2014-07-18 15:03:38,264 INFO mr.MapredLocalTask
(MapredLocalTask.java:initializeOperators(406)) - fetchoperator for
null-subquery2:a-subquery2
:xclick:b:wtrans_data_map_p2_30m created
2014-07-18 15:03:38,266 INFO mr.MapredLocalTask
(MapredLocalTask.java:initializeOperators(406)) - fetchoperator for
null-subquery1:a-subquery1
:dpkg_cntr:dpkg_wtransaction_id_user_30m created
2014-07-18 15:03:38,268 INFO mr.MapredLocalTask
(MapredLocalTask.java:initializeOperators(406)) - fetchoperator for
null-subquery1:a-subquery1
:dpkg:dpkg_wtransaction_id_user_30m created
2014-07-18 15:03:38,269 INFO mr.MapredLocalTask
(MapredLocalTask.java:initializeOperators(406)) - fetchoperator for
null-subquery1:a-subquery1
:xclick:b:wtrans_data_map_30m created
---------------------------------------------
whole bunch of stuff omitted here
--------------------------------------
--------------------------------------
2014-07-18 15:04:08,678 INFO exec.HashTableSinkOperator
(HashTableSinkOperator.java:flushToFile(278)) - Temp URI for side table:
file:/tmp/myuser/hive_2014-07-18_15-03-30_423_6799963466906099923-1/-local-10008/HashTable-Stage-2
2014-07-18 15:04:08,678 INFO exec.HashTableSinkOperator
(SessionState.java:printInfo(537)) - 2014-07-18 03:04:08 Dump the
side-table into file:
file:/tmp/myuser/hive_2014-07-18_15-03-30_423_6799963466906099923-1/-local-10008/HashTable-Stage-2/MapJoin-mapfile11--.hashtable
2014-07-18 15:04:09,943 INFO exec.HashTableSinkOperator
(SessionState.java:printInfo(537)) - 2014-07-18 03:04:09 Uploaded 1
File to:
file:/tmp/myuser/hive_2014-07-18_15-03-30_423_6799963466906099923-1/-local-10008/HashTable-Stage-2/MapJoin-mapfile11--.hashtable
(58010217 bytes)
2014-07-18 15:04:09,943 INFO exec.HashTableSinkOperator
(Operator.java:close(591)) - 6 Close done
2014-07-18 15:04:09,943 INFO exec.SelectOperator
(Operator.java:close(591)) - 5 Close done
2014-07-18 15:04:09,943 INFO exec.TableScanOperator
(Operator.java:close(591)) - 4 Close done
2014-07-18 15:04:09,951 INFO mapred.FileInputFormat
(FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2014-07-18 15:04:10,008 INFO mapred.FileInputFormat
(FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2014-07-18 15:04:11,526 INFO exec.HashTableSinkOperator
(SessionState.java:printInfo(537)) - 2014-07-18 03:04:11 Processing
rows:
200000 Hashtable size: 199999 Memory usage: 190041576
percentage: 0.045
2014-07-18 15:04:11,950 INFO exec.HashTableSinkOperator
(SessionState.java:printInfo(537)) - 2014-07-18 03:04:11 Processing
rows:
300000 Hashtable size: 299999 Memory usage: 250890416
percentage: 0.059
2014-07-18 15:04:12,456 INFO exec.HashTableSinkOperator
(SessionState.java:printInfo(537)) - 2014-07-18 03:04:12 Processing
rows:
400000 Hashtable size: 399999 Memory usage: 304697120
percentage: 0.071
2014-07-18 15:04:12,744 INFO exec.TableScanOperator
(Operator.java:close(574)) - 11 finished. closing...
2014-07-18 15:04:12,745 INFO exec.FilterOperator
(Operator.java:close(574)) - 12 finished. closing...
2014-07-18 15:04:12,745 INFO exec.FilterOperator
(Operator.java:logStats(838)) - PASSED:471403
2014-07-18 15:04:12,745 INFO exec.FilterOperator
(Operator.java:logStats(838)) - FILTERED:278619
2014-07-18 15:04:12,745 INFO exec.SelectOperator
(Operator.java:close(574)) - 13 finished. closing...
2014-07-18 15:04:12,745 INFO exec.HashTableSinkOperator
(Operator.java:close(574)) - 14 finished. closing...
2014-07-18 15:04:12,745 INFO exec.HashTableSinkOperator
(HashTableSinkOperator.java:flushToFile(278)) - Temp URI for side table:
file:/tmp/myuser/hive_2014-07-18_15-03-30_423_6799963466906099923-1/-local-10008/HashTable-Stage-2
2014-07-18 15:04:12,745 INFO exec.HashTableSinkOperator
(SessionState.java:printInfo(537)) - 2014-07-18 03:04:12 Dump the
side-table into file:
file:/tmp/myuser/hive_2014-07-18_15-03-30_423_6799963466906099923-1/-local-10008/HashTable-Stage-2/MapJoin-mapfile31--.hashtable
2014-07-18 15:04:13,836 INFO exec.HashTableSinkOperator
(SessionState.java:printInfo(537)) - 2014-07-18 03:04:13 Uploaded 1
File to:
file:/tmp/myuser/hive_2014-07-18_15-03-30_423_6799963466906099923-1/-local-10008/HashTable-Stage-2/MapJoin-mapfile31--.hashtable
(47251316 bytes)
2014-07-18 15:04:13,836 INFO exec.HashTableSinkOperator
(Operator.java:close(591)) - 14 Close done
2014-07-18 15:04:13,836 INFO exec.SelectOperator
(Operator.java:close(591)) - 13 Close done
2014-07-18 15:04:13,836 INFO exec.FilterOperator
(Operator.java:close(591)) - 12 Close done
2014-07-18 15:04:13,836 INFO exec.TableScanOperator
(Operator.java:close(591)) - 11 Close done
2014-07-18 15:04:13,837 INFO mr.MapredLocalTask
(SessionState.java:printInfo(537)) - 2014-07-18 03:04:13 End of local
task; Time Taken: 35.59 sec.
