[jira] [Updated] (SPARK-4133) PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0
[
https://issues.apache.org/jira/browse/SPARK-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tathagata Das updated SPARK-4133:
-
Priority: Major (was: Blocker)
PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0
--
Key: SPARK-4133
URL: https://issues.apache.org/jira/browse/SPARK-4133
Project: Spark
Issue Type: Bug
Components: Streaming
Affects Versions: 1.1.0
Reporter: Antonio Jesus Navarro
Attachments: spark_ex.logs
Snappy related problems found when trying to upgrade existing Spark Streaming
App from 1.0.2 to 1.1.0.
We can not run an existing 1.0.2 spark app if upgraded to 1.1.0
IOException is thrown by snappy (parsing_error(2))
{code}
Executor task launch worker-0 DEBUG storage.BlockManager - Getting local
block broadcast_0
Executor task launch worker-0 DEBUG storage.BlockManager - Level for block
broadcast_0 is StorageLevel(true, true, false, true, 1)
Executor task launch worker-0 DEBUG storage.BlockManager - Getting block
broadcast_0 from memory
Executor task launch worker-0 DEBUG storage.BlockManager - Getting local
block broadcast_0
Executor task launch worker-0 DEBUG executor.Executor - Task 0's epoch is 0
Executor task launch worker-0 DEBUG storage.BlockManager - Block broadcast_0
not registered locally
Executor task launch worker-0 INFO broadcast.TorrentBroadcast - Started
reading broadcast variable 0
sparkDriver-akka.actor.default-dispatcher-4 INFO
receiver.ReceiverSupervisorImpl - Registered receiver 0
Executor task launch worker-0 INFO util.RecurringTimer - Started timer for
BlockGenerator at time 1414656492400
Executor task launch worker-0 INFO receiver.BlockGenerator - Started
BlockGenerator
Thread-87 INFO receiver.BlockGenerator - Started block pushing thread
Executor task launch worker-0 INFO receiver.ReceiverSupervisorImpl -
Starting receiver
sparkDriver-akka.actor.default-dispatcher-5 INFO scheduler.ReceiverTracker -
Registered receiver for stream 0 from akka://sparkDriver
Executor task launch worker-0 INFO kafka.KafkaReceiver - Starting Kafka
Consumer Stream with group: stratioStreaming
Executor task launch worker-0 INFO kafka.KafkaReceiver - Connecting to
Zookeeper: node.stratio.com:2181
sparkDriver-akka.actor.default-dispatcher-2 DEBUG local.LocalActor - [actor]
received message StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0
cap=0]) from Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-2 DEBUG local.LocalActor - [actor]
received message StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0
cap=0]) from Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-6 DEBUG local.LocalActor - [actor]
received message StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0
cap=0]) from Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-2 DEBUG local.LocalActor - [actor]
handled message (8.442354 ms)
StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0 cap=0]) from
Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-2 DEBUG local.LocalActor - [actor]
handled message (8.412421 ms)
StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0 cap=0]) from
Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-6 DEBUG local.LocalActor - [actor]
handled message (8.385471 ms)
StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0 cap=0]) from
Actor[akka://sparkDriver/deadLetters]
Executor task launch worker-0 INFO utils.VerifiableProperties - Verifying
properties
Executor task launch worker-0 INFO utils.VerifiableProperties - Property
group.id is overridden to stratioStreaming
Executor task launch worker-0 INFO utils.VerifiableProperties - Property
zookeeper.connect is overridden to node.stratio.com:2181
Executor task launch worker-0 INFO utils.VerifiableProperties - Property
zookeeper.connection.timeout.ms is overridden to 1
Executor task launch worker-0 INFO broadcast.TorrentBroadcast - Reading
broadcast variable 0 took 0.033998997 s
Executor task launch worker-0 INFO consumer.ZookeeperConsumerConnector -
[stratioStreaming_ajn-stratio-1414656492293-8ecb3e3a], Connecting to
zookeeper instance at node.stratio.com:2181
Executor task launch worker-0 DEBUG zkclient.ZkConnection - Creating new
ZookKeeper instance to connect to node.stratio.com:2181.
ZkClient-EventThread-169-node.stratio.com:2181 INFO zkclient.ZkEventThread -
Starting ZkClient event thread.
Executor task launch worker-0 INFO zookeeper.ZooKeeper - Initiating client
connection, connectString=node.stratio.com:2181 sessionTimeout=6000
watcher=org.I0Itec.zkclient.ZkClient@5b4bdc81
Executor task launch worker-0 DEBUG
[jira] [Updated] (SPARK-4133) PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0
[
https://issues.apache.org/jira/browse/SPARK-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vitaliy Migov updated SPARK-4133:
-
Attachment: spark_ex.logs
PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0
--
Key: SPARK-4133
URL: https://issues.apache.org/jira/browse/SPARK-4133
Project: Spark
Issue Type: Bug
Components: Streaming
Affects Versions: 1.1.0
Reporter: Antonio Jesus Navarro
Priority: Blocker
Attachments: spark_ex.logs
Snappy related problems found when trying to upgrade existing Spark Streaming
App from 1.0.2 to 1.1.0.
We can not run an existing 1.0.2 spark app if upgraded to 1.1.0
IOException is thrown by snappy (parsing_error(2))
{code}
Executor task launch worker-0 DEBUG storage.BlockManager - Getting local
block broadcast_0
Executor task launch worker-0 DEBUG storage.BlockManager - Level for block
broadcast_0 is StorageLevel(true, true, false, true, 1)
Executor task launch worker-0 DEBUG storage.BlockManager - Getting block
broadcast_0 from memory
Executor task launch worker-0 DEBUG storage.BlockManager - Getting local
block broadcast_0
Executor task launch worker-0 DEBUG executor.Executor - Task 0's epoch is 0
Executor task launch worker-0 DEBUG storage.BlockManager - Block broadcast_0
not registered locally
Executor task launch worker-0 INFO broadcast.TorrentBroadcast - Started
reading broadcast variable 0
sparkDriver-akka.actor.default-dispatcher-4 INFO
receiver.ReceiverSupervisorImpl - Registered receiver 0
Executor task launch worker-0 INFO util.RecurringTimer - Started timer for
BlockGenerator at time 1414656492400
Executor task launch worker-0 INFO receiver.BlockGenerator - Started
BlockGenerator
Thread-87 INFO receiver.BlockGenerator - Started block pushing thread
Executor task launch worker-0 INFO receiver.ReceiverSupervisorImpl -
Starting receiver
sparkDriver-akka.actor.default-dispatcher-5 INFO scheduler.ReceiverTracker -
Registered receiver for stream 0 from akka://sparkDriver
Executor task launch worker-0 INFO kafka.KafkaReceiver - Starting Kafka
Consumer Stream with group: stratioStreaming
Executor task launch worker-0 INFO kafka.KafkaReceiver - Connecting to
Zookeeper: node.stratio.com:2181
sparkDriver-akka.actor.default-dispatcher-2 DEBUG local.LocalActor - [actor]
received message StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0
cap=0]) from Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-2 DEBUG local.LocalActor - [actor]
received message StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0
cap=0]) from Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-6 DEBUG local.LocalActor - [actor]
received message StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0
cap=0]) from Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-2 DEBUG local.LocalActor - [actor]
handled message (8.442354 ms)
StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0 cap=0]) from
Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-2 DEBUG local.LocalActor - [actor]
handled message (8.412421 ms)
StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0 cap=0]) from
Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-6 DEBUG local.LocalActor - [actor]
handled message (8.385471 ms)
StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0 cap=0]) from
Actor[akka://sparkDriver/deadLetters]
Executor task launch worker-0 INFO utils.VerifiableProperties - Verifying
properties
Executor task launch worker-0 INFO utils.VerifiableProperties - Property
group.id is overridden to stratioStreaming
Executor task launch worker-0 INFO utils.VerifiableProperties - Property
zookeeper.connect is overridden to node.stratio.com:2181
Executor task launch worker-0 INFO utils.VerifiableProperties - Property
zookeeper.connection.timeout.ms is overridden to 1
Executor task launch worker-0 INFO broadcast.TorrentBroadcast - Reading
broadcast variable 0 took 0.033998997 s
Executor task launch worker-0 INFO consumer.ZookeeperConsumerConnector -
[stratioStreaming_ajn-stratio-1414656492293-8ecb3e3a], Connecting to
zookeeper instance at node.stratio.com:2181
Executor task launch worker-0 DEBUG zkclient.ZkConnection - Creating new
ZookKeeper instance to connect to node.stratio.com:2181.
ZkClient-EventThread-169-node.stratio.com:2181 INFO zkclient.ZkEventThread -
Starting ZkClient event thread.
Executor task launch worker-0 INFO zookeeper.ZooKeeper - Initiating client
connection, connectString=node.stratio.com:2181 sessionTimeout=6000
watcher=org.I0Itec.zkclient.ZkClient@5b4bdc81
Executor task
[jira] [Updated] (SPARK-4133) PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0
[
https://issues.apache.org/jira/browse/SPARK-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Antonio Jesus Navarro updated SPARK-4133:
-
Description:
Snappy related problems found when trying to upgrade existing Spark Streaming
App from 1.0.2 to 1.1.0.
We can not run an existing 1.0.2 spark app if upgraded to 1.1.0
IOException is thrown by snappy (parsing_error(2))
Only spark version changed
As far as we have checked, snappy will throw this error when dealing with zero
bytes length arrays.
We have tried:
Changing from snappy to LZF,
Changing broadcast.compression false
Changing from TorrentBroadcast to HTTPBroadcast.
but with no luck for the moment.
{code}
Executor task launch worker-0 DEBUG storage.BlockManager - Getting local block
broadcast_0
Executor task launch worker-0 DEBUG storage.BlockManager - Level for block
broadcast_0 is StorageLevel(true, true, false, true, 1)
Executor task launch worker-0 DEBUG storage.BlockManager - Getting block
broadcast_0 from memory
Executor task launch worker-0 DEBUG storage.BlockManager - Getting local block
broadcast_0
Executor task launch worker-0 DEBUG executor.Executor - Task 0's epoch is 0
Executor task launch worker-0 DEBUG storage.BlockManager - Block broadcast_0
not registered locally
Executor task launch worker-0 INFO broadcast.TorrentBroadcast - Started
reading broadcast variable 0
sparkDriver-akka.actor.default-dispatcher-4 INFO
receiver.ReceiverSupervisorImpl - Registered receiver 0
Executor task launch worker-0 INFO util.RecurringTimer - Started timer for
BlockGenerator at time 1414656492400
Executor task launch worker-0 INFO receiver.BlockGenerator - Started
BlockGenerator
Thread-87 INFO receiver.BlockGenerator - Started block pushing thread
Executor task launch worker-0 INFO receiver.ReceiverSupervisorImpl - Starting
receiver
sparkDriver-akka.actor.default-dispatcher-5 INFO scheduler.ReceiverTracker -
Registered receiver for stream 0 from akka://sparkDriver
Executor task launch worker-0 INFO kafka.KafkaReceiver - Starting Kafka
Consumer Stream with group: stratioStreaming
Executor task launch worker-0 INFO kafka.KafkaReceiver - Connecting to
Zookeeper: node.stratio.com:2181
sparkDriver-akka.actor.default-dispatcher-2 DEBUG local.LocalActor - [actor]
received message StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0
cap=0]) from Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-2 DEBUG local.LocalActor - [actor]
received message StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0
cap=0]) from Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-6 DEBUG local.LocalActor - [actor]
received message StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0
cap=0]) from Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-2 DEBUG local.LocalActor - [actor]
handled message (8.442354 ms)
StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0 cap=0]) from
Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-2 DEBUG local.LocalActor - [actor]
handled message (8.412421 ms)
StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0 cap=0]) from
Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-6 DEBUG local.LocalActor - [actor]
handled message (8.385471 ms)
StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0 cap=0]) from
Actor[akka://sparkDriver/deadLetters]
Executor task launch worker-0 INFO utils.VerifiableProperties - Verifying
properties
Executor task launch worker-0 INFO utils.VerifiableProperties - Property
group.id is overridden to stratioStreaming
Executor task launch worker-0 INFO utils.VerifiableProperties - Property
zookeeper.connect is overridden to node.stratio.com:2181
Executor task launch worker-0 INFO utils.VerifiableProperties - Property
zookeeper.connection.timeout.ms is overridden to 1
Executor task launch worker-0 INFO broadcast.TorrentBroadcast - Reading
broadcast variable 0 took 0.033998997 s
Executor task launch worker-0 INFO consumer.ZookeeperConsumerConnector -
[stratioStreaming_ajn-stratio-1414656492293-8ecb3e3a], Connecting to zookeeper
instance at node.stratio.com:2181
Executor task launch worker-0 DEBUG zkclient.ZkConnection - Creating new
ZookKeeper instance to connect to node.stratio.com:2181.
ZkClient-EventThread-169-node.stratio.com:2181 INFO zkclient.ZkEventThread -
Starting ZkClient event thread.
Executor task launch worker-0 INFO zookeeper.ZooKeeper - Initiating client
connection, connectString=node.stratio.com:2181 sessionTimeout=6000
watcher=org.I0Itec.zkclient.ZkClient@5b4bdc81
Executor task launch worker-0 DEBUG zkclient.ZkClient - Awaiting connection to
Zookeeper server
Executor task launch worker-0 DEBUG zkclient.ZkClient - Waiting for keeper
state SyncConnected
Executor task launch
[jira] [Updated] (SPARK-4133) PARSING_ERROR(2) when upgrading issues from 1.0.2 to 1.1.0
[
https://issues.apache.org/jira/browse/SPARK-4133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Antonio Jesus Navarro updated SPARK-4133:
-
Description:
Snappy related problems found when trying to upgrade existing Spark Streaming
App from 1.0.2 to 1.1.0.
We can not run an existing 1.0.2 spark app if upgraded to 1.1.0
IOException is thrown by snappy (parsing_error(2))
{code}
Executor task launch worker-0 DEBUG storage.BlockManager - Getting local block
broadcast_0
Executor task launch worker-0 DEBUG storage.BlockManager - Level for block
broadcast_0 is StorageLevel(true, true, false, true, 1)
Executor task launch worker-0 DEBUG storage.BlockManager - Getting block
broadcast_0 from memory
Executor task launch worker-0 DEBUG storage.BlockManager - Getting local block
broadcast_0
Executor task launch worker-0 DEBUG executor.Executor - Task 0's epoch is 0
Executor task launch worker-0 DEBUG storage.BlockManager - Block broadcast_0
not registered locally
Executor task launch worker-0 INFO broadcast.TorrentBroadcast - Started
reading broadcast variable 0
sparkDriver-akka.actor.default-dispatcher-4 INFO
receiver.ReceiverSupervisorImpl - Registered receiver 0
Executor task launch worker-0 INFO util.RecurringTimer - Started timer for
BlockGenerator at time 1414656492400
Executor task launch worker-0 INFO receiver.BlockGenerator - Started
BlockGenerator
Thread-87 INFO receiver.BlockGenerator - Started block pushing thread
Executor task launch worker-0 INFO receiver.ReceiverSupervisorImpl - Starting
receiver
sparkDriver-akka.actor.default-dispatcher-5 INFO scheduler.ReceiverTracker -
Registered receiver for stream 0 from akka://sparkDriver
Executor task launch worker-0 INFO kafka.KafkaReceiver - Starting Kafka
Consumer Stream with group: stratioStreaming
Executor task launch worker-0 INFO kafka.KafkaReceiver - Connecting to
Zookeeper: node.stratio.com:2181
sparkDriver-akka.actor.default-dispatcher-2 DEBUG local.LocalActor - [actor]
received message StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0
cap=0]) from Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-2 DEBUG local.LocalActor - [actor]
received message StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0
cap=0]) from Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-6 DEBUG local.LocalActor - [actor]
received message StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0
cap=0]) from Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-2 DEBUG local.LocalActor - [actor]
handled message (8.442354 ms)
StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0 cap=0]) from
Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-2 DEBUG local.LocalActor - [actor]
handled message (8.412421 ms)
StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0 cap=0]) from
Actor[akka://sparkDriver/deadLetters]
sparkDriver-akka.actor.default-dispatcher-6 DEBUG local.LocalActor - [actor]
handled message (8.385471 ms)
StatusUpdate(0,RUNNING,java.nio.HeapByteBuffer[pos=0 lim=0 cap=0]) from
Actor[akka://sparkDriver/deadLetters]
Executor task launch worker-0 INFO utils.VerifiableProperties - Verifying
properties
Executor task launch worker-0 INFO utils.VerifiableProperties - Property
group.id is overridden to stratioStreaming
Executor task launch worker-0 INFO utils.VerifiableProperties - Property
zookeeper.connect is overridden to node.stratio.com:2181
Executor task launch worker-0 INFO utils.VerifiableProperties - Property
zookeeper.connection.timeout.ms is overridden to 1
Executor task launch worker-0 INFO broadcast.TorrentBroadcast - Reading
broadcast variable 0 took 0.033998997 s
Executor task launch worker-0 INFO consumer.ZookeeperConsumerConnector -
[stratioStreaming_ajn-stratio-1414656492293-8ecb3e3a], Connecting to zookeeper
instance at node.stratio.com:2181
Executor task launch worker-0 DEBUG zkclient.ZkConnection - Creating new
ZookKeeper instance to connect to node.stratio.com:2181.
ZkClient-EventThread-169-node.stratio.com:2181 INFO zkclient.ZkEventThread -
Starting ZkClient event thread.
Executor task launch worker-0 INFO zookeeper.ZooKeeper - Initiating client
connection, connectString=node.stratio.com:2181 sessionTimeout=6000
watcher=org.I0Itec.zkclient.ZkClient@5b4bdc81
Executor task launch worker-0 DEBUG zkclient.ZkClient - Awaiting connection to
Zookeeper server
Executor task launch worker-0 DEBUG zkclient.ZkClient - Waiting for keeper
state SyncConnected
Executor task launch worker-0-SendThread(node.stratio.com:2181) INFO
zookeeper.ClientCnxn - Opening socket connection to server
node.stratio.com/172.19.0.96:2181. Will not attempt to authenticate using SASL
(unknown error)
Executor task launch worker-0-SendThread(node.stratio.com:2181) INFO
zookeeper.ClientCnxn - Socket connection
