Paul Rogers created DRILL-7658: ---------------------------------- Summary: Vector allocateNew() has poor error reporting Key: DRILL-7658 URL: https://issues.apache.org/jira/browse/DRILL-7658 Project: Apache Drill Issue Type: Bug Affects Versions: 1.17.0 Reporter: Paul Rogers
See posting by Charles on 2020-03-24 on the user and dev lists of a message forwarded from another user where a query ran out of memory. Stack trace: {noformat} Caused by: org.apache.drill.exec.exception.OutOfMemoryException: null at org.apache.drill.exec.vector.complex.AbstractContainerVector.allocateNew(AbstractContainerVector.java:59) at org.apache.drill.exec.test.generated.PartitionerGen5$OutgoingRecordBatch.allocateOutgoingRecordBatch(PartitionerTemplate. {noformat} Notice the complete lack of context. The method in question: {code:java} public void allocateNew() throws OutOfMemoryException { if (!allocateNewSafe()) { throw new OutOfMemoryException(); } } {code} A generated implementation of the {{allocateNewSafe()}} method: {code:java} @Override public boolean allocateNewSafe() { long curAllocationSize = allocationSizeInBytes; if (allocationMonitor > 10) { curAllocationSize = Math.max(8, curAllocationSize / 2); allocationMonitor = 0; } else if (allocationMonitor < -2) { curAllocationSize = allocationSizeInBytes * 2L; allocationMonitor = 0; } try{ allocateBytes(curAllocationSize); } catch (DrillRuntimeException ex) { return false; } return true; } {code} Note that the {{allocateNew()}} method is not "safe" (it throws an exception), but it does so by discarding the underlying exception. What should happen is that the "non-safe" {{allocateNew()}} should call the {{allocateBytes()}} method and simply forward the {{DrillRuntimeException}}. It probably does not do so because the author wanted to reuse the extra size calcs in {{allocateNewSafe()}}. The solution is to put the calcs and the call to {{allocateBytes()}} in a "non-safe" method, and call that entire method from {{allocateNew()}} and {{allocateNewSafe()}}. Or, better, generate {{allocateNew()}} using the above code, but have the base class define {{allocateNewSafe()}} as a wrapper. Note an extra complexity: although the base class provides the method shown above, each generated vector also provides: {code:java} @Override public void allocateNew() { if (!allocateNewSafe()) { throw new OutOfMemoryException("Failure while allocating buffer."); } } {code} Which is both redundant and inconsistent (one has a message, the other does not.) -- This message was sent by Atlassian Jira (v8.3.4#803005)