Wampler deanwamp...@gmail.com
To: Lin Hao Xu/China/IBM@IBMCN
Cc: Hai Shan Wu/China/IBM@IBMCN, user user@spark.apache.org
Date: 2015/04/29 09:40
Subject: Re: A problem of using spark streaming to capture network packets
--
Are the tasks on the slaves also running
command line you use to submit the job?
From: Dean Wampler deanwamp...@gmail.com
To: Hai Shan Wu/China/IBM@IBMCN
Cc: user user@spark.apache.org, Lin Hao Xu/China/IBM@IBMCN
Date: 2015/04/28 20:07
Subject: Re: A problem of using spark streaming to capture network packets
...@gmail.com
To: Hai Shan Wu/China/IBM@IBMCN
Cc: user user@spark.apache.org, Lin Hao Xu/China/IBM@IBMCN
Date: 2015/04/28 20:07
Subject:Re: A problem of using spark streaming to capture network
packets
It's probably not your code.
What's the full command line you use
, user user@spark.apache.org
Date: 2015/04/29 09:40
Subject:Re: A problem of using spark streaming to capture network
packets
Are the tasks on the slaves also running as root? If not, that might
explain the problem.
dean
Dean Wampler, Ph.D.
Author: Programming Scala, 2nd
It's probably not your code.
What's the full command line you use to submit the job?
Are you sure the job on the cluster has access to the network interface?
Can you test the receiver by itself without Spark? For example, does this
line work as expected:
ListPcapNetworkInterface nifs =
to submit the job?
From: Dean Wampler deanwamp...@gmail.com
To: Hai Shan Wu/China/IBM@IBMCN
Cc: user user@spark.apache.org, Lin Hao Xu/China/IBM@IBMCN
Date: 2015/04/28 20:07
Subject: Re: A problem of using spark streaming to capture network
packets
It's probably not your code.
What's
Hi Everyone
We use pcap4j to capture network packets and then use spark streaming to
analyze captured packets. However, we met a strange problem.
If we run our application on spark locally (for example, spark-submit
--master local[2]), then the program runs successfully.
If we run our
