[ https://issues.apache.org/jira/browse/SPARK-21733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jepson updated SPARK-21733: --------------------------- Description: Kafka+Spark streaming ,throw these error: {code:java} 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored as bytes in memory (estimated size 1895.0 B, free 1643.2 MB) 17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable 8003 took 11 ms 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as values in memory (estimated size 2.9 KB, free 1643.2 MB) 17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the same as ending offset skipping kssh 5 17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0 (TID 64178). 1740 bytes result sent to driver 17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002 17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 64186 17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0 (TID 64186) 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast variable 8004 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored as bytes in memory (estimated size 1895.0 B, free 1643.2 MB) 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable 8004 took 8 ms 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as values in memory (estimated size 2.9 KB, free 1643.2 MB) 17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the same as ending offset skipping kssh 5 17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0 (TID 64186). 1740 bytes result sent to driver 17/08/15 09:34:29 {color:#59afe1}ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM{color} 17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called 17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called {code} was: Kafka+Spark streaming ,throw these error: {code:java} 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored as bytes in memory (estimated size 1895.0 B, free 1643.2 MB) 17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable 8003 took 11 ms 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as values in memory (estimated size 2.9 KB, free 1643.2 MB) 17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the same as ending offset skipping kssh 5 17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0 (TID 64178). 1740 bytes result sent to driver 17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002 17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 64186 17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0 (TID 64186) 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast variable 8004 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored as bytes in memory (estimated size 1895.0 B, free 1643.2 MB) 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable 8004 took 8 ms 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as values in memory (estimated size 2.9 KB, free 1643.2 MB) 17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the same as ending offset skipping kssh 5 17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0 (TID 64186). 1740 bytes result sent to driver 17/08/15 09:34:29 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM 17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called 17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called {code} > ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM > ----------------------------------------------------------------- > > Key: SPARK-21733 > URL: https://issues.apache.org/jira/browse/SPARK-21733 > Project: Spark > Issue Type: Bug > Components: DStreams > Affects Versions: 2.1.1 > Environment: Apache Spark2.1.1 > CDH5.12.0 Yarn > Reporter: Jepson > Labels: patch > Original Estimate: 96h > Remaining Estimate: 96h > > Kafka+Spark streaming ,throw these error: > {code:java} > 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003_piece0 stored > as bytes in memory (estimated size 1895.0 B, free 1643.2 MB) > 17/08/15 09:34:14 INFO broadcast.TorrentBroadcast: Reading broadcast variable > 8003 took 11 ms > 17/08/15 09:34:14 INFO memory.MemoryStore: Block broadcast_8003 stored as > values in memory (estimated size 2.9 KB, free 1643.2 MB) > 17/08/15 09:34:14 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the > same as ending offset skipping kssh 5 > 17/08/15 09:34:14 INFO executor.Executor: Finished task 7.0 in stage 8003.0 > (TID 64178). 1740 bytes result sent to driver > 17/08/15 09:34:21 INFO storage.BlockManager: Removing RDD 8002 > 17/08/15 09:34:21 INFO executor.CoarseGrainedExecutorBackend: Got assigned > task 64186 > 17/08/15 09:34:21 INFO executor.Executor: Running task 7.0 in stage 8004.0 > (TID 64186) > 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Started reading broadcast > variable 8004 > 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004_piece0 stored > as bytes in memory (estimated size 1895.0 B, free 1643.2 MB) > 17/08/15 09:34:21 INFO broadcast.TorrentBroadcast: Reading broadcast variable > 8004 took 8 ms > 17/08/15 09:34:21 INFO memory.MemoryStore: Block broadcast_8004 stored as > values in memory (estimated size 2.9 KB, free 1643.2 MB) > 17/08/15 09:34:21 INFO kafka010.KafkaRDD: Beginning offset 10130733 is the > same as ending offset skipping kssh 5 > 17/08/15 09:34:21 INFO executor.Executor: Finished task 7.0 in stage 8004.0 > (TID 64186). 1740 bytes result sent to driver > 17/08/15 09:34:29 {color:#59afe1}ERROR executor.CoarseGrainedExecutorBackend: > RECEIVED SIGNAL TERM{color} > 17/08/15 09:34:29 INFO storage.DiskBlockManager: Shutdown hook called > 17/08/15 09:34:29 INFO util.ShutdownHookManager: Shutdown hook called > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org