Re: Spark streaming: StorageLevel.MEMORY_AND_DISK_SER setting for KafkaUtils.createDirectStream

the input data ahead of time. Only >> receiver-based approach could specify the storage level. >> >> Thanks >> Saisai >> >> On Wed, Mar 2, 2016 at 7:08 PM, Vinti Maheshwari <vinti.u...@gmail.com> >> wrote: >> >>> Hi

Re: Spark streaming: StorageLevel.MEMORY_AND_DISK_SER setting for KafkaUtils.createDirectStream

ire to store the input data ahead of time. Only receiver-based > approach could specify the storage level. > > Thanks > Saisai > > On Wed, Mar 2, 2016 at 7:08 PM, Vinti Maheshwari <vinti.u...@gmail.com> > wrote: > >> Hi All, >> >> I wanted to set *Storag

Re: Spark streaming: StorageLevel.MEMORY_AND_DISK_SER setting for KafkaUtils.createDirectStream

t; Hi All, > > I wanted to set *StorageLevel.MEMORY_AND_DISK_SER* in my spark-streaming > program as currently i am getting > MetadataFetchFailedException*. *I am not sure where i should pass > StorageLevel.MEMORY_AND_DISK, as it seems like createDirectStream doesn't > allow to pas

Spark streaming: StorageLevel.MEMORY_AND_DISK_SER setting for KafkaUtils.createDirectStream

Hi All, I wanted to set *StorageLevel.MEMORY_AND_DISK_SER* in my spark-streaming program as currently i am getting MetadataFetchFailedException*. *I am not sure where i should pass StorageLevel.MEMORY_AND_DISK, as it seems like createDirectStream doesn't allow to pass that parameter. val

Re: StorageLevel.MEMORY_AND_DISK_SER

[GenericRecord]], writeJob.getConfiguration) } On Wed, Jul 1, 2015 at 8:11 AM, Koert Kuipers ko...@tresata.com wrote: rdd.persist(StorageLevel.MEMORY_AND_DISK_SER) On Wed, Jul 1, 2015 at 11:01 AM, ÐΞ€ρ@Ҝ (๏̯͡๏) deepuj...@gmail.com wrote: How do i persist an RDD using

Re: StorageLevel.MEMORY_AND_DISK_SER

(StorageLevel.MEMORY_AND_DISK_SER) On Wed, Jul 1, 2015 at 11:01 AM, ÐΞ€ρ@Ҝ (๏̯͡๏) deepuj...@gmail.com wrote: How do i persist an RDD using StorageLevel.MEMORY_AND_DISK_SER ? -- Deepak -- Deepak -- Deepak

Re: StorageLevel.MEMORY_AND_DISK_SER

at 8:11 AM, Koert Kuipers ko...@tresata.com wrote: rdd.persist(StorageLevel.MEMORY_AND_DISK_SER) On Wed, Jul 1, 2015 at 11:01 AM, ÐΞ€ρ@Ҝ (๏̯͡๏) deepuj...@gmail.com wrote: How do i persist an RDD using StorageLevel.MEMORY_AND_DISK_SER ? -- Deepak -- Deepak -- Deepak

Re: StorageLevel.MEMORY_AND_DISK_SER

at 8:11 AM, Koert Kuipers ko...@tresata.com wrote: rdd.persist(StorageLevel.MEMORY_AND_DISK_SER) On Wed, Jul 1, 2015 at 11:01 AM, ÐΞ€ρ@Ҝ (๏̯͡๏) deepuj...@gmail.com wrote: How do i persist an RDD using StorageLevel.MEMORY_AND_DISK_SER ? -- Deepak -- Deepak -- Deepak

StorageLevel.MEMORY_AND_DISK_SER

How do i persist an RDD using StorageLevel.MEMORY_AND_DISK_SER ? -- Deepak

Re: StorageLevel.MEMORY_AND_DISK_SER

rdd.persist(StorageLevel.MEMORY_AND_DISK_SER) On Wed, Jul 1, 2015 at 11:01 AM, ÐΞ€ρ@Ҝ (๏̯͡๏) deepuj...@gmail.com wrote: How do i persist an RDD using StorageLevel.MEMORY_AND_DISK_SER ? -- Deepak

Re: StorageLevel.MEMORY_AND_DISK_SER

, 2015 at 8:11 AM, Koert Kuipers ko...@tresata.com wrote: rdd.persist(StorageLevel.MEMORY_AND_DISK_SER) On Wed, Jul 1, 2015 at 11:01 AM, ÐΞ€ρ@Ҝ (๏̯͡๏) deepuj...@gmail.com wrote: How do i persist an RDD using StorageLevel.MEMORY_AND_DISK_SER ? -- Deepak -- Deepak

Re: Executor Lost with StorageLevel.MEMORY_AND_DISK_SER

-Original Message- From: Marius Soutier [mailto:mps@gmail.com] Sent: Monday, February 09, 2015 2:19 AM To: user Subject: Executor Lost with StorageLevel.MEMORY_AND_DISK_SER Hi there, I'm trying to improve performance on a job that has GC troubles and takes longer

Executor Lost with StorageLevel.MEMORY_AND_DISK_SER

Hi there, I’m trying to improve performance on a job that has GC troubles and takes longer to compute simply because it has to recompute failed tasks. After deferring object creation as much as possible, I’m now trying to improve memory usage with StorageLevel.MEMORY_AND_DISK_SER and a custom