How many process are you runing in parallel? In build cube step On 1 Feb 2018 7:39 a.m., "Kumar, Manoj H" <manoj.h.ku...@jpmorgan.com> wrote:
> We have 15 nodes & each nodes have 8 cores. RAM= 256 MB. > > > > I don’t think memory is issue here. > > > > Regards, > > Manoj > > > > *From:* Alberto Ramón [mailtoa.ramonporto...@gmail.com] > *Sent:* Thursday, February 01, 2018 1:33 AM > *To:* user <user@kylin.apache.org> > *Subject:* Re: Optimize Cube Build process > > > > How many nodes do you have? > > how many RAM and CPU do you have per node? > > > > On 31 January 2018 at 05:07, Kumar, Manoj H <manoj.h.ku...@jpmorgan.com> > wrote: > > It has close to 68 mapper & reducers 500.. It keeps running on this. Pls. > advise. > > [image: cid:image001.png@01D39B5D.E7ED9FD0] > > > > Regards, > > Manoj > > > > *From:* Kumar, Manoj H > *Sent:* Wednesday, January 31, 2018 9:24 AM > *To:* 'user@kylin.apache.org' <user@kylin.apache.org> > *Subject:* Optimize Cube Build process > > > > Hi Folks – I have close to 33 million of fact data to be processed, Data > is having lot of unique/Distinct values such Loan_unique_code, > Facility_code,card_id such.. Dimension looks up are made of these. > > > > Fact table – 33 millions > > Looks up tables having to 3 to 4 millions > > Cube build type I have chosen – inmem > > Engine – Mapreduce > > > > Cube build step is taking 90 minutes which is seems to be high. What I can > do in order to minimize build time? What Parameter I should tweak so that > Build time gets reduced. Thanks. > > > > > > I have Followed the same steps as given below but it doesn’t help in this > case > > > > http://kylin.apache.org/docs21/howto/howto_optimize_build.html > > > > > > Regards, > > Manoj > > > > This message is confidential and subject to terms at: http:// > www.jpmorgan.com/emaildisclaimer including on confidentiality, legal > privilege, viruses and monitoring of electronic messages. If you are not > the intended recipient, please delete this message and notify the sender > immediately. Any unauthorized use is strictly prohibited. > > > > This message is confidential and subject to terms at: http:// > www.jpmorgan.com/emaildisclaimer including on confidentiality, legal > privilege, viruses and monitoring of electronic messages. If you are not > the intended recipient, please delete this message and notify the sender > immediately. Any unauthorized use is strictly prohibited. >