Re: Change for submitting to yarn in 1.3.1

sc.applicationId gives the yarn appid. On 11 May 2015 at 08:13, Mridul Muralidharan wrote: > We had a similar requirement, and as a stopgap, I currently use a > suboptimal impl specific workaround - parsing it out of the > stdout/stderr (based on log config). > A better means to get to this is i

Re: Change for submitting to yarn in 1.3.1

We had a similar requirement, and as a stopgap, I currently use a suboptimal impl specific workaround - parsing it out of the stdout/stderr (based on log config). A better means to get to this is indeed required ! Regards, Mridul On Sun, May 10, 2015 at 7:33 PM, Ron's Yahoo! wrote: > Hi, > I u

Change for submitting to yarn in 1.3.1

Hi, I used to submit my Spark yarn applications by using org.apache.spark.yarn.deploy.Client api so I can get the application id after I submit it. The following is the code that I have, but after upgrading to 1.3.1, the yarn Client class was made into a private class. Is there a particular r

RE: [SparkSQL] cannot filter by a DateType column

Sorry, I was using Spark 1.3.x. I cannot reproduce it in master. But should I still open a JIRA because can I request it to be back ported to 1.3.x branch? Thanks again! From: Michael Armbrust [mailto:mich...@databricks.com] Sent: Saturday, May 09, 201