[ https://issues.apache.org/jira/browse/HDFS-9265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
James Clampffer updated HDFS-9265: ---------------------------------- Description: The remote block reader relies on undefined behavior in how it uses enable_shared_from_this. http://en.cppreference.com/w/cpp/memory/enable_shared_from_this The spec states a shared_ptr to an object inheriting from enable_shared_from_this must be live before calling make_shared_from_this. Calling make_shared_from_this without an existing shared_ptr is undefined behavior and causes deterministic crashes when the code is built with GCC. example: class foo : public enable_shared_from_this {/*bar*/}; safe: auto ptr1 = std::make_shared<foo>(); auto ptr2 = foo->make_shared_from_this(); broken: foo *ptr = new foo(); auto ptr2 = foo->make_shared_from_this(); //no existing live shared_ptr In order to fix the input stream should call std::make_shared and hang onto a shared_ptr to the block reader. The block reader will then be free to call make_shared_from this as much as it wants without issue. was: The remote block reader relies on undefined behavior in how it uses enable_shared_from_this. http://en.cppreference.com/w/cpp/memory/enable_shared_from_this The spec states a shared_ptr to an object inheriting from enable_shared_from_this must be live before calling make_shared_from_this. Calling make_shared_from_this without an existing shared_ptr is undefined behavior and causes deterministic crashes when the code is built with GCC. example: class foo : public enable_shared_from_this {/*bar*/}; safe: auto ptr1 = std::make_shared<foo>(); auto ptr2 = foo->make_shared_from_this(); broken: foo *ptr = new foo(); auto ptr2 = foo->make_shared_from_this(); //no existing live shared_ptr In order to fix the input stream should call std::make_shared and hang onto a shared_ptr rather than unique_ptr to the block reader. The block reader will then be free to call make_shared_from this as much as it wants without issue. I think this will fix some double deletes and pure virtual method call exceptions as a bonus. Both the shared_ptr and unique pointer think they own the reader, whichever calls delete on the reader second will either segfault if the memory has been reused or end up calling a pure virtual dtor which throws (because after the first dtor executes vptr points to the base class vtable and gcc stubs those in to throw). > Use of undefined behavior in remote_block_reader causing deterministic > crashes. > ------------------------------------------------------------------------------- > > Key: HDFS-9265 > URL: https://issues.apache.org/jira/browse/HDFS-9265 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs-client > Reporter: James Clampffer > Priority: Blocker > > The remote block reader relies on undefined behavior in how it uses > enable_shared_from_this. > http://en.cppreference.com/w/cpp/memory/enable_shared_from_this > The spec states a shared_ptr to an object inheriting from > enable_shared_from_this must be live before calling make_shared_from_this. > Calling make_shared_from_this without an existing shared_ptr is undefined > behavior and causes deterministic crashes when the code is built with GCC. > example: > class foo : public enable_shared_from_this {/*bar*/}; > safe: > auto ptr1 = std::make_shared<foo>(); > auto ptr2 = foo->make_shared_from_this(); > broken: > foo *ptr = new foo(); > auto ptr2 = foo->make_shared_from_this(); //no existing live shared_ptr > In order to fix the input stream should call std::make_shared and hang onto a > shared_ptr to the block reader. The block reader will then be free to call > make_shared_from this as much as it wants without issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)