Re: Re: Selecting the top 100 records per group by?

It's not Spark specific, but it answers your question: https://blog.jooq.org/2014/08/12/the-difference-between-row_number-rank-and-dense_rank/ On 12 September 2016 at 12:42, Mich Talebzadeh wrote: > Hi, > > I don't understand why you need to add a column row_number

Re: Submit and Monitor standalone cluster application

Sorry, my mistake (quick copy-paste), livy doesn't let me submit applications the classic way (with assembly jars) and force me to change all my current applications. -- *Mariano Semelman* P13N - IT Av. Corrientes Nº 746 - piso 13 - C.A.B.A. (C1043AAU) Teléfono (54

Submit and Monitor standalone cluster application

​Hello everybody, I'm developing an application to submit batch and streaming apps in a fault tolerant fashion. For that I need a programatically way to submit and monitor my apps and relaunch them in case of failure. Right now I'm using spark standalone (1.6.x) and submitting in cluster mode.

Re: SparkLauncher not receiving events

2016 at 11:37, Mariano Semelman < mariano.semel...@despegar.com> wrote: > Hello, > > I'm having problems to receive events from the submited app. The app > succesfuly submits, but the listener I'm passing to SparkLauncher is not > receiving events. > > Spark Vers

SparkLauncher not receiving events

Hello, I'm having problems to receive events from the submited app. The app succesfuly submits, but the listener I'm passing to SparkLauncher is not receiving events. Spark Version: 1.6.1 (both client app and master) here are the relevant snippets I'm using in my code:

Re: Master OOM in "master-rebuild-ui-thread" while running stream app

Thanks, I would go with log disabling. BTW, the master crashed while the application was still running. -- *Mariano Semelman* P13N - IT Av. Corrientes Nº 746 - piso 13 - C.A.B.A. (C1043AAU) Teléfono (54) 11- *4894-3500* [image: Seguinos en Twitter!] <h

Master OOM in "master-rebuild-ui-thread" while running stream app

Hello everybody, I am running a spark streaming app and I am planning to use it as a long running service. However while trying the app in a rc environment I got this exception in the master daemon after 1 hour of running: ​​Exception in thread "master-rebuild-ui-thread"

Q: Multiple spark streaming app, one kafka topic, same consumer group

Hello everybody, I am trying to understand how Kafka Direct Stream works. I'm interested in having a production ready Spark Streaming application that consumes a Kafka topic. But I need to guarantee there's (almost) no downtime, specially during deploys (and submit) of new versions. What it seems