Re: Join streams Apache Spark

2017-05-09 Thread tencas
Hi scorpio, thanks for your reply. I don't understand your approach. Is it possible to receive data from different clients throught the same port on Spark? Surely I'm confused and I'd appreciate your opinion. Regarding the word count example , from Spark Streaming documentation, Spark acts as

Re: Join streams Apache Spark

2017-05-08 Thread tencas
Yep, I mean the first script you posted. So, you can compile it to Java binaries for example ? Ok, I have no idea about Go. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Join-streams-Apache-Spark-tp28603p28662.html Sent from the Apache Spark User List mail

Re: Join streams Apache Spark

2017-05-06 Thread tencas
There exists an Spark Streaming example of the classic word count, using apache kafka connector: https://github.com/apache/spark/blob/master/examples/src/main/java/org/apache/spark/examples/streaming/JavaKafkaWordCount.java (maybe you already know) The point is, what are the benefits from using

Re: Join streams Apache Spark

2017-05-06 Thread tencas
Thanks @saulshanabrook, I'll have a look at it. I think apache kafka could be an alternative solution, but I haven't checked it yet. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Join-streams-Apache-Spark-tp28603p28656.html Sent from the Apache Spark User

Join streams Apache Spark

2017-04-15 Thread tencas
Hi everybody, I am using Apache Spark Streaming using a TCP connector to receive data. I have a python application that connects to a sensor, and create a TCP server that waits connection from Apache Spark, and then, sends json data through this socket. How can I manage to join many independent

Re: Spark Streaming. Real-time save data and visualize on dashboard

2017-04-12 Thread tencas
Hi Gaurav1809 , I was thinking about using elasticsearch + kibana too (actually don't know the differences between ELK and elasticsearch). I was wondering about pros and cons of using a document indexer vs NoSQL database. -- View this message in context: http://apache-spark-user-list.1001560.n

Spark Streaming. Real-time save data and visualize on dashboard

2017-04-11 Thread tencas
I've developed an application using Apache Spark Streaming, that reads simple info from plane sensors like acceleration, via TCP sockets on json format, and analyse it. I'd like to be able to persist this info from each "flight" on real-time, while it is shown on any responsive dashboard. I just