You will have to define your own stream-to-iterator function and use the
socketStream. The function should return custom delimited object as bytes
are continuously coming in. When data is insufficient, the function should
block.
TD
On Jul 23, 2014 6:52 PM, "kytay" wrote:
> Hi TD
>
> You are righ
Hi TD
You are right, I did not include "\n" to delimit the string flushed. That's
the reason.
Is there a way for me to define the delimiter? Like SOH or ETX instead of
"\n"
Regards
kytay
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Solved-Streaming-Can
When you are sending data using simple socket code to send messages, are
those messages "\n" delimited? If its not, then the receiver of
socketTextSTream, wont identify them as separate events, and keep buffering
them.
TD
On Sun, Jul 13, 2014 at 10:49 PM, kytay wrote:
> Hi Tobias
>
> I have be
Hi Tobias
I have been using "local[4]" to test.
My problem is likely caused by the tcp host server that I am trying the
emulate. I was trying to emulate the tcp host to send out messages.
(although I am not sure at the moment :D)
First way I tried was to use a tcp tool called, Hercules.
Second w
Hi,
I experienced exactly the same problems when using SparkContext with
"local[1]" master specification, because in that case one thread is used
for receiving data, the others for processing. As there is only one thread
running, no processing will take place. Once you shut down the connection,
th
Hi Akhil Das
Thanks.
I tried the codes. and it works.
There's a problem with my socket codes that is not flushing the content out,
and for the test tool, Hercules, I have to close the socket connection to
"flush" the content out.
I am going to troubleshoot why nc works, and the codes and test t
Can you try this piece of code?
SparkConf sparkConf = new SparkConf().setAppName("JavaNetworkWordCount"
);
JavaStreamingContext ssc = new JavaStreamingContext(sparkConf, new
Duration(1000));
JavaReceiverInputDStream lines = ssc.socketTextStream(
args[0], Integer.parseInt(a
I think I should be seeing any line of text that I have typed in the nc
command.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Streaming-Cannot-get-socketTextStream-to-receive-anything-tp9382p9410.html
Sent from the Apache Spark User List mailing list arch
Hi Akhil Das
I have tried the
nc -lk
command too.
I was hoping the "System.out.println("Print text:" + arg0);" is printed when
a stream is processed when lines.flatMap(...) is called.
But from my test with "nc -lk ", nothing is printed on the console at
all.
==
To test out whether t
Sorry, the command is
nc -lk 12345
Thanks
Best Regards
On Fri, Jul 11, 2014 at 6:46 AM, Akhil Das
wrote:
> You simply use the *nc* command to do this. like:
>
> nc -p 12345
>
> will open the 12345 port and from the terminal you can provide whatever
> input you require for your StreamingCode.
You simply use the *nc* command to do this. like:
nc -p 12345
will open the 12345 port and from the terminal you can provide whatever
input you require for your StreamingCode.
Thanks
Best Regards
On Fri, Jul 11, 2014 at 2:41 AM, kytay wrote:
> Hi
>
> I am learning spark streaming, and is try
11 matches
Mail list logo
