Hello, does someone know, why we have a huge memory consumption with Oozie 3.1.3 (cdh4) and YARN ? Oozie needs around 4GB Java heap for just some very short pig workflows that run under a minute and never more than 5 simultaneously. It looks almost like a memory leak, since the memory used slowly rises over around 1 hour to be kept constantly at around 3.8GB... This might be due to (higher) GC activity, but we didn't check explicitly.
After we switched back to MRv1 the memory consumption is now below 300MB however. Plus the same jobs now complete in roughly half the time to a tenth of the time. I would be curious to know if this is a known issue and maybe get some insight on how and when we could switch to YARN for production?! Greetings, Johannes Schwenk -- Softwareentwickler (Reporting) ________________________________________________________ ADITION technologies AG Schwarzwaldstraße 78b 79117 Freiburg http://www.adition.com T +49 / (0)761 / 88147 - 30 F +49 / (0)761 / 88147 - 77 SUPPORT +49 / (0)1805 - ADITION (Festnetzpreis 14 ct/min; Mobilfunkpreise maximal 42 ct/min) Eingetragen beim Amtsgericht Düsseldorf unter HRB 54076 Vorstände: Andreas Kleiser, Jörg Klekamp, Tihomir Perkovic, Marcus Schlüter Aufsichtsratsvorsitzender: Rechtsanwalt Daniel Raimer UStIDNr.: DE 218 858 434
signature.asc
Description: OpenPGP digital signature
