Recently some of us encount a problem in loading when the executor side throw an OOM exception but the driver side said "unable to generate the mdkey". Actually no one will throw KeyGenException. Anyway, the error message in driver is unrelevant.
Dive into the code, we found that it is caused by Wrapped Exception as below. Actual exception is throw in addDataToStore when new a column page. We should make the error message more helpful for debugging instead of becoming confusing. ``` private void processBatch(CarbonRowBatch batch, CarbonFactHandler dataHandler, int iteratorIndex) throws CarbonDataLoadingException { try { while (batch.hasNext()) { CarbonRow row = batch.next(); CarbonRow converted = convertRow(row); dataHandler.addDataToStore(converted); readCounter[iteratorIndex]++; } writeCounter[iteratorIndex] += batch.getSize(); } catch (Exception e) { throw new CarbonDataLoadingException("unable to generate the mdkey", e); } rowCounter.getAndAdd(batch.getSize()); } ``` By using regex expression, we found so many codes of "catch \([^}]*throw new" in java and "catch \{[^}]*throw new" in scala to deal with exception. Any good ideas to decide whether there is problem for the wrapped exception and how to fix it? For this example, shall we change the error message to "e.getMessage()" to keep original error? ----- Regards Manhua -- Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/