Hi,
I am creating DataFrame from a json file and the schema of json as truely
depicted by dataframe.printschema() is:
root
|-- 1-F2: struct (nullable = true)
||-- A: string (nullable = true)
||-- B: string (nullable = true)
||-- C: string (nullable = true)
|-- 10-C4: struct (nullable
...@sigmoidanalytics.com]
Sent: Tuesday, June 16, 2015 12:44 PM
To: Chaudhary, Umesh
Cc: user@spark.apache.org
Subject: Re: Optimizing Streaming from Websphere MQ
Each receiver will run on 1 core. So if your network is not the bottleneck then
to test the consumption speed of the receivers you can
not
experiencing any performance benefit from it.
Is it something related to the bottleneck of MQ or Reliable Receiver?
From: Akhil Das [mailto:ak...@sigmoidanalytics.com]
Sent: Saturday, June 13, 2015 1:10 AM
To: Chaudhary, Umesh
Cc: user@spark.apache.org
Subject: Re: Optimizing Streaming from Websphere MQ
Hi,
I have created a Custom Receiver in Java which receives data from Websphere MQ
and I am only writing the received records on HDFS.
I have referred many forums for optimizing speed of spark streaming
application. Here I am listing a few:
* Spark
issue.
From: Tathagata Das [mailto:t...@databricks.com]
Sent: Friday, May 29, 2015 3:30 AM
To: Chaudhary, Umesh
Cc: Arush Kharbanda; user@spark.apache.org
Subject: Re: FW: Websphere MQ as a data source for Apache Spark Streaming
Are you sure that the data can be saved as strings?
Another, more
Thanks for the suggestion, I will try and post the outcome.
From: Arush Kharbanda [mailto:ar...@sigmoidanalytics.com]
Sent: Monday, May 25, 2015 12:24 PM
To: Chaudhary, Umesh; user@spark.apache.org
Subject: Re: Websphere MQ as a data source for Apache Spark Streaming
Hi Umesh,
You can write
