Bharath Vissapragada has posted comments on this change. Change subject: IMPALA-5881: Use native allocation while building catalog updates ......................................................................
Patch Set 8: (8 comments) http://gerrit.cloudera.org:8080/#/c/7955/8/be/src/catalog/catalog.cc File be/src/catalog/catalog.cc: Line 116: DCHECK(len < std::numeric_limits<uint32_t>::max()); > DCHECK_LE Done Line 170: LOG(ERROR) << "Failed to free buffers. Potential memory leak of: " > Failed to free native buffer through JNI. Leaked N bytes. Done http://gerrit.cloudera.org:8080/#/c/7955/8/fe/src/main/java/org/apache/impala/service/JniCatalog.java File fe/src/main/java/org/apache/impala/service/JniCatalog.java: Line 295: TException { > Does this really throw TException? good catch, no! http://gerrit.cloudera.org:8080/#/c/7955/8/fe/src/main/java/org/apache/impala/thrift/NativeByteArrayOutputStream.java File fe/src/main/java/org/apache/impala/thrift/NativeByteArrayOutputStream.java: Line 94: private void tryAllocate(long len, boolean reAllocate) { > Can we simplify this to just always use reallocateMemory()? Realloc from a Good point, checked the JVM sources, it already includes the check to call malloc/realloc depending on whether we pass 0. http://hg.openjdk.java.net/jdk8/jdk8/hotspot/file/87ee5ee27509/src/share/vm/prims/unsafe.cpp#l613 Line 95: try { > Preconditions.checkState(bufferPtr_ >= 0); Done Line 113: } catch (Exception e) { > catch Throwable for extra paranoia Done Line 139: if (newBufferSize >= BUFFER_MAX_SIZE_LIMIT) { > || newBufferSize < 0 (overflow) Don't think thats possible given we cap at BUFFER_MAX_SIZE_LIMIT which is <<< long_max. I'm still including it to be on the safe side. Line 155: public synchronized void reset() {} > What's this for? Is it an @Override? Not an ovveride, removed, this was actually being called from serializer, but does nothing. Ideally useful to reset the state of NBOAS, but given we don't allow calling serialize twice, I don't think that is necessary. -- To view, visit http://gerrit.cloudera.org:8080/7955 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I383684effa9524734ce3c6c0fb7ed37de0e15782 Gerrit-PatchSet: 8 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com> Gerrit-Reviewer: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogian...@cloudera.com> Gerrit-Reviewer: Mostafa Mokhtar <mmokh...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-HasComments: Yes
