dear all,
---------
Key: PIG-2500
URL: https://issues.apache.org/jira/browse/PIG-2500
Project: Pig
Issue Type: Bug
Reporter: vaibhav
input file :
cat input13
(3,8,9) {(3,8,9)} [open#apache]
(1,4,7) {(1,4,7)} [apache#hadoop]
(2,5,8) {(2,5,8)} [open#apache]
A = LOAD '/data/input13' AS (T1:tuple(f1:int, f2:int),
B:bag{T2:tuple(t1:float,t2:float)}, M:map[] );
output :
dump A ;
(3,),,)
((1,),,)
((2,),,)
but it should be the same as input?
2)
cat input15
(3,8,9) (mary,19)
(1,4,7) (john,18)
(2,5,8) (joe,18)
o/p
((3,8,9),)
((1,4,7),)
((2,5,8),)
---------------------------------------
first logs
--------------------------------------------------------------------------------
grunt> A = LOAD '/data/input13' AS (T1:tuple(f1:int, f2:int),
B:bag{T2:tuple(t1:float,t2:float)}, M:map[] );
grunt> dump A ;
2012-02-01 20:22:14,025 [main] INFO
org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No column pruned for A
2012-02-01 20:22:14,025 [main] INFO
org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No map keys pruned
for A
2012-02-01 20:22:14,032 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics -
Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2012-02-01 20:22:14,034 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name:
Store(hdfs://localhost:54310/tmp/temp537168513/tmp899939258:org.apache.pig.builtin.BinStorage)
- 1-246 Operator Key: 1-246)
2012-02-01 20:22:14,035 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size before optimization: 1
2012-02-01 20:22:14,035 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size after optimization: 1
2012-02-01 20:22:14,040 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics -
Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2012-02-01 20:22:14,040 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics -
Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2012-02-01 20:22:14,040 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2012-02-01 20:22:15,334 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting up single store job
2012-02-01 20:22:15,335 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics -
Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2012-02-01 20:22:15,336 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 1 map-reduce job(s) waiting for submission.
2012-02-01 20:22:15,336 [Thread-149] WARN org.apache.hadoop.mapred.JobClient -
Use GenericOptionsParser for parsing the arguments. Applications should
implement Tool for the same.
2012-02-01 20:22:15,378 [Thread-149] INFO
org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
2012-02-01 20:22:15,382 [Thread-149] INFO
org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
2012-02-01 20:22:15,388 [Thread-149] INFO
org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
process : 1
2012-02-01 20:22:15,388 [Thread-149] INFO
org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
paths to process : 1
2012-02-01 20:22:15,425 [Thread-157] INFO
org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
2012-02-01 20:22:15,427 [Thread-157] INFO
org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
process : 1
2012-02-01 20:22:15,427 [Thread-157] INFO
org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
paths to process : 1
2012-02-01 20:22:15,437 [Thread-157] INFO
org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
2012-02-01 20:22:15,438 [Thread-157] INFO
org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
2012-02-01 20:22:15,440 [Thread-157] INFO
org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
2012-02-01 20:22:15,444 [Thread-157] INFO
org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
2012-02-01 20:22:15,463 [Thread-157] INFO org.apache.hadoop.mapred.TaskRunner
- Task:attempt_local_0011_m_000000_0 is done. And is in the process of commiting
2012-02-01 20:22:15,464 [Thread-157] INFO
org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
2012-02-01 20:22:15,465 [Thread-157] INFO
org.apache.hadoop.mapred.LocalJobRunner -
2012-02-01 20:22:15,465 [Thread-157] INFO org.apache.hadoop.mapred.TaskRunner
- Task attempt_local_0011_m_000000_0 is allowed to commit now
2012-02-01 20:22:15,466 [Thread-157] INFO
org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
2012-02-01 20:22:15,471 [Thread-157] INFO
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - Saved output of
task 'attempt_local_0011_m_000000_0' to
hdfs://localhost:54310/tmp/temp537168513/tmp899939258
2012-02-01 20:22:15,471 [Thread-157] INFO
org.apache.hadoop.mapred.LocalJobRunner -
2012-02-01 20:22:15,471 [Thread-157] INFO org.apache.hadoop.mapred.TaskRunner
- Task 'attempt_local_0011_m_000000_0' done.
2012-02-01 20:22:15,837 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- HadoopJobId: job_local_0011
2012-02-01 20:22:15,837 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 0% complete
2012-02-01 20:22:20,843 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete
2012-02-01 20:22:20,843 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Successfully stored result in:
"hdfs://localhost:54310/tmp/temp537168513/tmp899939258"
2012-02-01 20:22:20,843 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Records written : 0
2012-02-01 20:22:20,843 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Bytes written : 0
2012-02-01 20:22:20,843 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Spillable Memory Manager spill count : 0
2012-02-01 20:22:20,843 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Proactive spill count : 0
2012-02-01 20:22:20,843 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Success!
2012-02-01 20:22:20,855 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics -
Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2012-02-01 20:22:20,859 [main] INFO
org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
process : 1
2012-02-01 20:22:20,859 [main] INFO
org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
paths to process : 1
((3,),,)
((1,),,)
((2,),,)
----------------------------------------------------------------------
second logs
grunt> D = LOAD '/data/input15' AS
(F:tuple(f1:int,f2:int,f3:int),T:tuple(t1:chararray,t2:int));
grunt> dump D ;
2012-02-01 20:28:32,287 [main] INFO
org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No column pruned for D
2012-02-01 20:28:32,287 [main] INFO
org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No map keys pruned
for D
2012-02-01 20:28:32,330 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics -
Initializing JVM Metrics with processName=JobTracker, sessionId=
2012-02-01 20:28:32,399 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name:
Store(hdfs://localhost:54310/tmp/temp2086651143/tmp-1826566586:org.apache.pig.builtin.BinStorage)
- 1-14 Operator Key: 1-14)
2012-02-01 20:28:32,428 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size before optimization: 1
2012-02-01 20:28:32,428 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size after optimization: 1
2012-02-01 20:28:32,443 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics -
Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2012-02-01 20:28:32,448 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics -
Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2012-02-01 20:28:32,448 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2012-02-01 20:28:33,845 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting up single store job
2012-02-01 20:28:33,869 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics -
Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2012-02-01 20:28:33,870 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 1 map-reduce job(s) waiting for submission.
2012-02-01 20:28:33,873 [Thread-8] WARN org.apache.hadoop.mapred.JobClient -
Use GenericOptionsParser for parsing the arguments. Applications should
implement Tool for the same.
2012-02-01 20:28:33,968 [Thread-8] INFO
org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
2012-02-01 20:28:33,977 [Thread-8] INFO
org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
2012-02-01 20:28:33,985 [Thread-8] INFO
org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
process : 1
2012-02-01 20:28:33,985 [Thread-8] INFO
org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
paths to process : 1
2012-02-01 20:28:34,102 [Thread-17] INFO
org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
2012-02-01 20:28:34,104 [Thread-17] INFO
org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
process : 1
2012-02-01 20:28:34,105 [Thread-17] INFO
org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
paths to process : 1
2012-02-01 20:28:34,136 [Thread-17] INFO
org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
2012-02-01 20:28:34,145 [Thread-17] INFO
org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
2012-02-01 20:28:34,149 [Thread-17] INFO
org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
2012-02-01 20:28:34,153 [Thread-17] INFO
org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
2012-02-01 20:28:34,179 [Thread-17] INFO org.apache.hadoop.mapred.TaskRunner -
Task:attempt_local_0001_m_000000_0 is done. And is in the process of commiting
2012-02-01 20:28:34,181 [Thread-17] INFO
org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
2012-02-01 20:28:34,183 [Thread-17] INFO
org.apache.hadoop.mapred.LocalJobRunner -
2012-02-01 20:28:34,184 [Thread-17] INFO org.apache.hadoop.mapred.TaskRunner -
Task attempt_local_0001_m_000000_0 is allowed to commit now
2012-02-01 20:28:34,185 [Thread-17] INFO
org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
2012-02-01 20:28:34,192 [Thread-17] INFO
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - Saved output of
task 'attempt_local_0001_m_000000_0' to
hdfs://localhost:54310/tmp/temp2086651143/tmp-1826566586
2012-02-01 20:28:34,193 [Thread-17] INFO
org.apache.hadoop.mapred.LocalJobRunner -
2012-02-01 20:28:34,193 [Thread-17] INFO org.apache.hadoop.mapred.TaskRunner -
Task 'attempt_local_0001_m_000000_0' done.
2012-02-01 20:28:34,371 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- HadoopJobId: job_local_0001
2012-02-01 20:28:34,372 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 0% complete
2012-02-01 20:28:39,379 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete
2012-02-01 20:28:39,379 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Successfully stored result in:
"hdfs://localhost:54310/tmp/temp2086651143/tmp-1826566586"
2012-02-01 20:28:39,380 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Records written : 0
2012-02-01 20:28:39,380 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Bytes written : 0
2012-02-01 20:28:39,381 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Spillable Memory Manager spill count : 0
2012-02-01 20:28:39,381 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Proactive spill count : 0
2012-02-01 20:28:39,381 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Success!
2012-02-01 20:28:39,394 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics -
Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2012-02-01 20:28:39,400 [main] INFO
org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
process : 1
2012-02-01 20:28:39,400 [main] INFO
org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
paths to process : 1
((3,8,9),)
((1,4,7),)
((2,5,8),)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
