[ https://issues.apache.org/jira/browse/SPARK-31604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Divya Paliwal updated SPARK-31604: ---------------------------------- Description: Hi, I am currently facing the below error when I run my code to stream data from Couchbase in spark cluster. 2020-04-29 00:04:06,061 WARN server.TransportChannelHandler: Exception in connection from /[host]:56910 java.lang.IllegalArgumentException: Frame length should be positive: -9223371863711366549 at org.spark_project.guava.base.Preconditions.checkArgument(Preconditions.java:119) at org.apache.spark.network.util.TransportFrameDecoder.decodeNext(TransportFrameDecoder.java:134) at org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:81) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:138) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138) at java.lang.Thread.run(Thread.java:748) When ever I run the spark-submit command with all the arguments on the host I get the above error. Command run: bin/spark-submit \ --deploy-mode cluster \ --class "com.apple.pst.dc.CouchbaseRawMain" \ --master [spark://master-host:8091] \ --jars /ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/core-io-1.7.6.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/java-client-2.7.6.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/rxjava-1.3.8.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/couchbase-spark-connector_2.11-2.4.0.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/dcp-client-0.23.0.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/opentracing-api-0.31.0.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/rxscala_2.11-0.27.0.jar \ --conf spark.rpc.message.maxSize=1000 —-conf spark.shuffle.service.enabled=true —-conf spark.network.crypto.saslFallback=true —-conf spark.authenticate=true —-conf spark.network.crypto.enabled=true --driver-memory 3g --driver-cores 2 --num-executors 1 --executor-memory 3g --total-executor-cores 2 --executor-cores 2 \ /ngs/app/dcf5d/HD-cluster/spark/code/generic-couchbase-raw-0.1.0-SNAPSHOT.jar [spark://master-host:8091] Couchbase-DC-Bucket-Raw-QA-OG-28 host1:8091,host2:8091 DC [hdfs://master-host:9000/tables/couchbase/QA/DC-bucket/] Administrator dcadmin welcome true Thanks, Divya was: Hi, I am currently facing the below error when I run my code to stream data from Couchbase in spark cluster. 2020-04-29 00:04:06,061 WARN server.TransportChannelHandler: Exception in connection from /[host]:56910 java.lang.IllegalArgumentException: Frame length should be positive: -9223371863711366549 at org.spark_project.guava.base.Preconditions.checkArgument(Preconditions.java:119) at org.apache.spark.network.util.TransportFrameDecoder.decodeNext(TransportFrameDecoder.java:134) at org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:81) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:138) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138) at java.lang.Thread.run(Thread.java:748) When ever I run the spark-submit command with all the arguments on the host I get the above error. Command run: bin/spark-submit \ --deploy-mode cluster \ --class "com.apple.pst.dc.CouchbaseRawMain" \ --master [spark://rn-dcf5t-lapp85.rno.apple.com:8091] \ --jars /ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/core-io-1.7.6.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/java-client-2.7.6.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/rxjava-1.3.8.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/couchbase-spark-connector_2.11-2.4.0.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/dcp-client-0.23.0.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/opentracing-api-0.31.0.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/rxscala_2.11-0.27.0.jar \ --conf spark.rpc.message.maxSize=1000 —-conf spark.shuffle.service.enabled=true —-conf spark.network.crypto.saslFallback=true —-conf spark.authenticate=true —-conf spark.network.crypto.enabled=true --driver-memory 3g --driver-cores 2 --num-executors 1 --executor-memory 3g --total-executor-cores 2 --executor-cores 2 \ /ngs/app/dcf5d/HD-cluster/spark/code/generic-couchbase-raw-0.1.0-SNAPSHOT.jar [spark://rn-dcf5t-lapp85.rno.apple.com:8091] Couchbase-DC-Bucket-Raw-QA-OG-28 rn-dcf5t-lapp87.rno.apple.com:8091,rn-dcf5t-lapp88.rno.apple.com:8091 DC [hdfs://rn-dcf5t-lapp85.rno.apple.com:9000/tables/couchbase/QA/DC-bucket/] Administrator dcadmin welcome true Thanks, Divya > java.lang.IllegalArgumentException: Frame length should be positive > ------------------------------------------------------------------- > > Key: SPARK-31604 > URL: https://issues.apache.org/jira/browse/SPARK-31604 > Project: Spark > Issue Type: Bug > Components: Deploy > Affects Versions: 2.4.4 > Environment: Scala version 2.11.12 > Spark version 2.4.4 > Reporter: Divya Paliwal > Priority: Major > Fix For: 2.4.4 > > > Hi, > I am currently facing the below error when I run my code to stream data from > Couchbase in spark cluster. > 2020-04-29 00:04:06,061 WARN server.TransportChannelHandler: Exception in > connection from /[host]:56910 > java.lang.IllegalArgumentException: Frame length should be positive: > -9223371863711366549 > at > org.spark_project.guava.base.Preconditions.checkArgument(Preconditions.java:119) > at > org.apache.spark.network.util.TransportFrameDecoder.decodeNext(TransportFrameDecoder.java:134) > at > org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:81) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) > at > io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) > at > io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935) > at > io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:138) > at > io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) > at > io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) > at > io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497) > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) > at > io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) > at > io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138) > at java.lang.Thread.run(Thread.java:748) > > When ever I run the spark-submit command with all the arguments on the host I > get the above error. > > Command run: > bin/spark-submit \ > --deploy-mode cluster \ > --class "com.apple.pst.dc.CouchbaseRawMain" \ > --master [spark://master-host:8091] \ > --jars > /ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/core-io-1.7.6.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/java-client-2.7.6.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/rxjava-1.3.8.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/couchbase-spark-connector_2.11-2.4.0.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/dcp-client-0.23.0.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/opentracing-api-0.31.0.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/rxscala_2.11-0.27.0.jar > \ > --conf spark.rpc.message.maxSize=1000 —-conf > spark.shuffle.service.enabled=true —-conf > spark.network.crypto.saslFallback=true —-conf spark.authenticate=true —-conf > spark.network.crypto.enabled=true > --driver-memory 3g --driver-cores 2 --num-executors 1 --executor-memory 3g > --total-executor-cores 2 --executor-cores 2 \ > > /ngs/app/dcf5d/HD-cluster/spark/code/generic-couchbase-raw-0.1.0-SNAPSHOT.jar > [spark://master-host:8091] Couchbase-DC-Bucket-Raw-QA-OG-28 > host1:8091,host2:8091 DC > [hdfs://master-host:9000/tables/couchbase/QA/DC-bucket/] Administrator > dcadmin welcome true > > Thanks, > Divya -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org