Hi, Tested for calculating values for 300 columns. Analyser takes around 4 minutes to generate the plan. Is this normal?
Regards, Madhukara Phatak http://datamantra.io/ On Tue, May 19, 2015 at 4:35 PM, madhu phatak <phatak....@gmail.com> wrote: > Hi, > I am using spark 1.3.1 > > > > > Regards, > Madhukara Phatak > http://datamantra.io/ > > On Tue, May 19, 2015 at 4:34 PM, Wangfei (X) <wangf...@huawei.com> wrote: > >> And which version are you using >> >> 发自我的 iPhone >> >> 在 2015年5月19日，18:29，"ayan guha" <guha.a...@gmail.com> 写道： >> >> can you kindly share your code? >> >> On Tue, May 19, 2015 at 8:04 PM, madhu phatak <phatak....@gmail.com> >> wrote: >> >>> Hi, >>> I am trying run spark sql aggregation on a file with 26k columns. No of >>> rows is very small. I am running into issue that spark is taking huge >>> amount of time to parse the sql and create a logical plan. Even if i have >>> just one row, it's taking more than 1 hour just to get pass the parsing. >>> Any idea how to optimize in these kind of scenarios? >>> >>> >>> Regards, >>> Madhukara Phatak >>> http://datamantra.io/ >>> >> >> >> >> -- >> Best Regards, >> Ayan Guha >> >> >
