[
https://issues.apache.org/jira/browse/PIG-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198643#comment-13198643
]
vaibhav commented on PIG-2500:
------------------------------
hi, it is not working again,i didn;t get proper output
i have done indivisible MAP,TUPLE and BAG file also,But they are working and
giving o/p also but i want how to combine two TUPLE or BAG., all of these three.
like:
1)
cat input15
(3,8,9) (mary,19)
(1,4,7) (john,18)
(2,5,8) (joe,18)
D = LOAD '/data/input15' AS
(F:tuple(f1:int,f2:int,f3:int),T:tuple(t1:chararray,t2:int));
it gives no output
2)cat input13
(3,8,9) {(3,8,9)} open#apache
(1,4,7) {(1,4,7)} apache#hadoop
(2,5,8) {(2,5,8)} open#apache
A = LOAD '/data/input13' AS (T1:tuple(f1:int, f2:int,f3:int),
B:bag{T2:tuple(t1:float,t2:float,t2:float)}, M:map[] );
output :
dump A ;
(3,),,)
((1,),,)
((2,),,)
> bag and tuple issues
> --------------------
>
> Key: PIG-2500
> URL: https://issues.apache.org/jira/browse/PIG-2500
> Project: Pig
> Issue Type: Bug
> Reporter: vaibhav
> Assignee: Daniel Dai
>
> input file :
> cat input13
> (3,8,9) {(3,8,9)} [open#apache]
> (1,4,7) {(1,4,7)} [apache#hadoop]
> (2,5,8) {(2,5,8)} [open#apache]
> A = LOAD '/data/input13' AS (T1:tuple(f1:int, f2:int),
> B:bag{T2:tuple(t1:float,t2:float)}, M:map[] );
> output :
> dump A ;
> (3,),,)
> ((1,),,)
> ((2,),,)
> but it should be the same as input?
> 2)
> cat input15
> (3,8,9) (mary,19)
> (1,4,7) (john,18)
> (2,5,8) (joe,18)
> o/p
> ((3,8,9),)
> ((1,4,7),)
> ((2,5,8),)
> ---------------------------------------
> first logs
> --------------------------------------------------------------------------------
> grunt> A = LOAD '/data/input13' AS (T1:tuple(f1:int, f2:int),
> B:bag{T2:tuple(t1:float,t2:float)}, M:map[] );
> grunt> dump A ;
> 2012-02-01 20:22:14,025 [main] INFO
> org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No column pruned
> for A
> 2012-02-01 20:22:14,025 [main] INFO
> org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No map keys pruned
> for A
> 2012-02-01 20:22:14,032 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics
> - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= -
> already initialized
> 2012-02-01 20:22:14,034 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name:
> Store(hdfs://localhost:54310/tmp/temp537168513/tmp899939258:org.apache.pig.builtin.BinStorage)
> - 1-246 Operator Key: 1-246)
> 2012-02-01 20:22:14,035 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size before optimization: 1
> 2012-02-01 20:22:14,035 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size after optimization: 1
> 2012-02-01 20:22:14,040 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics
> - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= -
> already initialized
> 2012-02-01 20:22:14,040 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics
> - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= -
> already initialized
> 2012-02-01 20:22:14,040 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
> 2012-02-01 20:22:15,334 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Setting up single store job
> 2012-02-01 20:22:15,335 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics
> - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= -
> already initialized
> 2012-02-01 20:22:15,336 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 1 map-reduce job(s) waiting for submission.
> 2012-02-01 20:22:15,336 [Thread-149] WARN org.apache.hadoop.mapred.JobClient
> - Use GenericOptionsParser for parsing the arguments. Applications should
> implement Tool for the same.
> 2012-02-01 20:22:15,378 [Thread-149] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
> processName=JobTracker, sessionId= - already initialized
> 2012-02-01 20:22:15,382 [Thread-149] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
> processName=JobTracker, sessionId= - already initialized
> 2012-02-01 20:22:15,388 [Thread-149] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
> process : 1
> 2012-02-01 20:22:15,388 [Thread-149] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> 2012-02-01 20:22:15,425 [Thread-157] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
> processName=JobTracker, sessionId= - already initialized
> 2012-02-01 20:22:15,427 [Thread-157] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
> process : 1
> 2012-02-01 20:22:15,427 [Thread-157] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> 2012-02-01 20:22:15,437 [Thread-157] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
> processName=JobTracker, sessionId= - already initialized
> 2012-02-01 20:22:15,438 [Thread-157] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
> processName=JobTracker, sessionId= - already initialized
> 2012-02-01 20:22:15,440 [Thread-157] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
> processName=JobTracker, sessionId= - already initialized
> 2012-02-01 20:22:15,444 [Thread-157] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
> processName=JobTracker, sessionId= - already initialized
> 2012-02-01 20:22:15,463 [Thread-157] INFO
> org.apache.hadoop.mapred.TaskRunner - Task:attempt_local_0011_m_000000_0 is
> done. And is in the process of commiting
> 2012-02-01 20:22:15,464 [Thread-157] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
> processName=JobTracker, sessionId= - already initialized
> 2012-02-01 20:22:15,465 [Thread-157] INFO
> org.apache.hadoop.mapred.LocalJobRunner -
> 2012-02-01 20:22:15,465 [Thread-157] INFO
> org.apache.hadoop.mapred.TaskRunner - Task attempt_local_0011_m_000000_0 is
> allowed to commit now
> 2012-02-01 20:22:15,466 [Thread-157] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
> processName=JobTracker, sessionId= - already initialized
> 2012-02-01 20:22:15,471 [Thread-157] INFO
> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - Saved output of
> task 'attempt_local_0011_m_000000_0' to
> hdfs://localhost:54310/tmp/temp537168513/tmp899939258
> 2012-02-01 20:22:15,471 [Thread-157] INFO
> org.apache.hadoop.mapred.LocalJobRunner -
> 2012-02-01 20:22:15,471 [Thread-157] INFO
> org.apache.hadoop.mapred.TaskRunner - Task 'attempt_local_0011_m_000000_0'
> done.
> 2012-02-01 20:22:15,837 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - HadoopJobId: job_local_0011
> 2012-02-01 20:22:15,837 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 0% complete
> 2012-02-01 20:22:20,843 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 100% complete
> 2012-02-01 20:22:20,843 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Successfully stored result in:
> "hdfs://localhost:54310/tmp/temp537168513/tmp899939258"
> 2012-02-01 20:22:20,843 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Records written : 0
> 2012-02-01 20:22:20,843 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Bytes written : 0
> 2012-02-01 20:22:20,843 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Spillable Memory Manager spill count : 0
> 2012-02-01 20:22:20,843 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Proactive spill count : 0
> 2012-02-01 20:22:20,843 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Success!
> 2012-02-01 20:22:20,855 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics
> - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= -
> already initialized
> 2012-02-01 20:22:20,859 [main] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
> process : 1
> 2012-02-01 20:22:20,859 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> ((3,),,)
> ((1,),,)
> ((2,),,)
> ----------------------------------------------------------------------
> second logs
> grunt> D = LOAD '/data/input15' AS
> (F:tuple(f1:int,f2:int,f3:int),T:tuple(t1:chararray,t2:int));
> grunt> dump D ;
> 2012-02-01 20:28:32,287 [main] INFO
> org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No column pruned
> for D
> 2012-02-01 20:28:32,287 [main] INFO
> org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No map keys pruned
> for D
> 2012-02-01 20:28:32,330 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics
> - Initializing JVM Metrics with processName=JobTracker, sessionId=
> 2012-02-01 20:28:32,399 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name:
> Store(hdfs://localhost:54310/tmp/temp2086651143/tmp-1826566586:org.apache.pig.builtin.BinStorage)
> - 1-14 Operator Key: 1-14)
> 2012-02-01 20:28:32,428 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size before optimization: 1
> 2012-02-01 20:28:32,428 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size after optimization: 1
> 2012-02-01 20:28:32,443 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics
> - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= -
> already initialized
> 2012-02-01 20:28:32,448 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics
> - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= -
> already initialized
> 2012-02-01 20:28:32,448 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
> 2012-02-01 20:28:33,845 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Setting up single store job
> 2012-02-01 20:28:33,869 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics
> - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= -
> already initialized
> 2012-02-01 20:28:33,870 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 1 map-reduce job(s) waiting for submission.
> 2012-02-01 20:28:33,873 [Thread-8] WARN org.apache.hadoop.mapred.JobClient -
> Use GenericOptionsParser for parsing the arguments. Applications should
> implement Tool for the same.
> 2012-02-01 20:28:33,968 [Thread-8] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
> processName=JobTracker, sessionId= - already initialized
> 2012-02-01 20:28:33,977 [Thread-8] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
> processName=JobTracker, sessionId= - already initialized
> 2012-02-01 20:28:33,985 [Thread-8] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
> process : 1
> 2012-02-01 20:28:33,985 [Thread-8] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> 2012-02-01 20:28:34,102 [Thread-17] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
> processName=JobTracker, sessionId= - already initialized
> 2012-02-01 20:28:34,104 [Thread-17] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
> process : 1
> 2012-02-01 20:28:34,105 [Thread-17] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> 2012-02-01 20:28:34,136 [Thread-17] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
> processName=JobTracker, sessionId= - already initialized
> 2012-02-01 20:28:34,145 [Thread-17] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
> processName=JobTracker, sessionId= - already initialized
> 2012-02-01 20:28:34,149 [Thread-17] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
> processName=JobTracker, sessionId= - already initialized
> 2012-02-01 20:28:34,153 [Thread-17] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
> processName=JobTracker, sessionId= - already initialized
> 2012-02-01 20:28:34,179 [Thread-17] INFO org.apache.hadoop.mapred.TaskRunner
> - Task:attempt_local_0001_m_000000_0 is done. And is in the process of
> commiting
> 2012-02-01 20:28:34,181 [Thread-17] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
> processName=JobTracker, sessionId= - already initialized
> 2012-02-01 20:28:34,183 [Thread-17] INFO
> org.apache.hadoop.mapred.LocalJobRunner -
> 2012-02-01 20:28:34,184 [Thread-17] INFO org.apache.hadoop.mapred.TaskRunner
> - Task attempt_local_0001_m_000000_0 is allowed to commit now
> 2012-02-01 20:28:34,185 [Thread-17] INFO
> org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with
> processName=JobTracker, sessionId= - already initialized
> 2012-02-01 20:28:34,192 [Thread-17] INFO
> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - Saved output of
> task 'attempt_local_0001_m_000000_0' to
> hdfs://localhost:54310/tmp/temp2086651143/tmp-1826566586
> 2012-02-01 20:28:34,193 [Thread-17] INFO
> org.apache.hadoop.mapred.LocalJobRunner -
> 2012-02-01 20:28:34,193 [Thread-17] INFO org.apache.hadoop.mapred.TaskRunner
> - Task 'attempt_local_0001_m_000000_0' done.
> 2012-02-01 20:28:34,371 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - HadoopJobId: job_local_0001
> 2012-02-01 20:28:34,372 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 0% complete
> 2012-02-01 20:28:39,379 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 100% complete
> 2012-02-01 20:28:39,379 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Successfully stored result in:
> "hdfs://localhost:54310/tmp/temp2086651143/tmp-1826566586"
> 2012-02-01 20:28:39,380 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Records written : 0
> 2012-02-01 20:28:39,380 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Bytes written : 0
> 2012-02-01 20:28:39,381 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Spillable Memory Manager spill count : 0
> 2012-02-01 20:28:39,381 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Proactive spill count : 0
> 2012-02-01 20:28:39,381 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Success!
> 2012-02-01 20:28:39,394 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics
> - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= -
> already initialized
> 2012-02-01 20:28:39,400 [main] INFO
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
> process : 1
> 2012-02-01 20:28:39,400 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input
> paths to process : 1
> ((3,8,9),)
> ((1,4,7),)
> ((2,5,8),)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
