bneradt opened a new issue, #12225:
URL: https://github.com/apache/trafficserver/issues/12225
We saw this on a few of our ATS 10.0.x boxes:
```
(gdb) bt
#0 0x00007f6402be4e92 in __memmove_avx_unaligned_erms () from
/lib64/libc.so.6
#1 0x00000000006c2440 in XpackDynamicTableStorage::write (value_len=0,
value=0x0, name_len=23, name=<optimized out>, this=0x7f59da9e8b20) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/proxy/hdrs/XPACK.cc:569
#2 ExpandCapacityContext::copy_field (len=23, offset=<optimized out>,
this=<synthetic pointer>) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/proxy/hdrs/XPACK.cc:620
#3 XpackDynamicTable::_expand_storage_size (this=0x7f59da9e8af8,
new_storage_size=<optimized out>) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/proxy/hdrs/XPACK.cc:492
#4 0x00000000006c2681 in XpackDynamicTable::update_maximum_size
(this=this@entry=0x7f59da9e8af8, new_max_size=4096) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/proxy/hdrs/XPACK.cc:416
#5 0x000000000065bd78 in HpackIndexingTable::update_maximum_size
(new_size=<optimized out>, this=0x7f59da9e8af0) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/proxy/http2/HPACK.cc:685
#6 update_dynamic_table_size (maximum_table_size=4096, indexing_table=...,
buf_end=0x7f58ee6bc79a "t cabp", buf_start=<optimized out>) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/proxy/http2/HPACK.cc:685
#7 hpack_decode_header_block (indexing_table=...,
hdr=hdr@entry=0x7f5a25cc9f80,
in_buf=0x7f58ee6bc750
"?\341\037\202ه\004\260b*\210\066\"\250\276R\302\022\064\230\214\022{j\352\206\031d\017.\344o\266ۃ
]i\311\032\033\213\364G\371%t@\211ư\n㮺?\324\322\326\325\302\301\300\277\276\314\313\312\311\310\307\306\305\304t
cabp", in_buf_len=<optimized out>, max_header_size=131072,
maximum_table_size=maximum_table_size@entry=4096) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/proxy/http2/HPACK.cc:737
#8 0x0000000000636c35 in http2_decode_header_blocks
(hdr=hdr@entry=0x7f5a25cc9f80, buf_start=<optimized out>, buf_len=<optimized
out>, len_read=len_read@entry=0x0, handle=..., is_trailing_header=<optimized
out>, maximum_table_size=4096, is_outbound=false)
at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/proxy/http2/HTTP2.cc:443
#9 0x000000000064b10e in Http2Stream::decode_header_blocks
(this=this@entry=0x7f5a25cc9e00, hpack_handle=...,
maximum_table_size=<optimized out>) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/proxy/http2/Http2Stream.cc:691
#10 0x000000000064566c in Http2ConnectionState::rcv_headers_frame
(this=0x7f5acd6e5320, frame=...) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/proxy/http2/Http2ConnectionState.cc:1234
#11 0x0000000000641e91 in Http2ConnectionState::rcv_frame
(this=this@entry=0x7f5acd6e5320, frame=frame@entry=0x7f63ff3fc2e0) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/proxy/http2/Http2ConnectionState.cc:1464
#12 0x0000000000639569 in Http2CommonSession::do_complete_frame_read
(this=this@entry=0x7f5acd6e5318) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/proxy/http2/Http2CommonSession.cc:338
#13 0x00000000006399d4 in Http2CommonSession::do_process_frame_read
(this=0x7f5acd6e5318, vio=0x7f54bff5ab58, inside_frame=<optimized out>) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/proxy/http2/Http2CommonSession.cc:400
#14 0x000000000065fbf0 in Http2ClientSession::main_event_handler
(this=0x7f5acd6e5000, event=100, edata=0x7f54bff5ab58) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/proxy/http2/Http2ClientSession.cc:183
#15 0x000000000080f604 in Continuation::handleEvent (data=0x7f54bff5ab58,
event=100, this=0x7f5acd6e5000) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/include/iocore/eventsystem/Continuation.h:228
#16 Continuation::handleEvent (data=0x7f54bff5ab58, event=100,
this=0x7f5acd6e5000) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/include/iocore/eventsystem/Continuation.h:224
#17 read_signal_and_update (vc=0x7f54bff5a900, event=100) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/iocore/net/UnixNetVConnection.cc:95
#18 UnixNetVConnection::readSignalAndUpdate (this=this@entry=0x7f54bff5a900,
event=event@entry=100) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/iocore/net/UnixNetVConnection.cc:989
#19 0x00000000007d968c in SSLNetVConnection::net_read_io
(this=0x7f54bff5a900, nh=0x7f63fdc4df10, lthread=<optimized out>) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/iocore/net/SSLNetVConnection.cc:731
#20 0x000000000083b088 in NetHandler::process_ready_list
(this=this@entry=0x7f63fdc4df10) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/iocore/net/NetHandler.cc:286
#21 0x000000000083b38a in NetHandler::waitForActivity (this=0x7f63fdc4df10,
timeout=<optimized out>) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/iocore/net/NetHandler.cc:374
#22 0x0000000000876eac in EThread::execute_regular
(this=this@entry=0x7f63fdc4d380) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/include/tsutil/Histogram.h:156
#23 0x0000000000877069 in EThread::execute (this=0x7f63fdc4d380) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/iocore/eventsystem/UnixEThread.cc:348
#24 EThread::execute (this=0x7f63fdc4d380) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/iocore/eventsystem/UnixEThread.cc:326
#25 0x0000000000873df2 in spawn_thread_internal (a=0x7f6401c90180) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/iocore/eventsystem/Thread.cc:75
#26 0x00007f6402ee31ca in start_thread () from /lib64/libpthread.so.0
#27 0x00007f6402b4fe73 in clone () from /lib64/libc.so.6
(gdb) f 2
#2 ExpandCapacityContext::copy_field (len=23, offset=<optimized out>,
this=<synthetic pointer>) at
/sd/workspace/src/git.ouryahoo.com/Edge/build/_scm/trafficserver10.0/src/proxy/hdrs/XPACK.cc:620
620 return this->_storage.write(field, len, nullptr, 0);
(gdb) l
615
616 uint32_t
617 ExpandCapacityContext::copy_field(uint32_t offset, uint32_t len)
618 {
619 char const *field = reinterpret_cast<char const
*>(this->_storage._old_data + offset);
620 return this->_storage.write(field, len, nullptr, 0);
621 }
(gdb) p this->_storage._old_data
$20 = (uint8_t *) 0x0
```
We are in the table storage expansion phase, and having `_old_data` point to
nullptr implies that `_data` was nullptr somehow. The possibilities are:
* It was initialized as such and we're now expanding it from 0. But then why
would the logic think there are entries to copy?
* It's a use after free (destruction sets `_data` to nullptr)
* We incorrectly allowed it to shrink to 0.
The last option seems the most likely.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
