JosiahWI commented on issue #12124: URL: https://github.com/apache/trafficserver/issues/12124#issuecomment-2749614801
In #11825 I unwarily moved the place where the `static_cast` happened from the unsigned stripe length to the signed preservation table size. I believe if the resulting size does not fit in the positive part of an `int`, the size will be negative, which violates a debug assert (not triggered in the reported case because it was a release build), and is divided by the bucket size to make a smaller negative number and cast back to unsigned for the allocation call. That's why it would attempt to allocate a very large amount of memory... approaching the maximum allocation value. The issue will only be observed if the configuration specifies a stripe length that can't be represented as a positive `int`, so the unit tests in CI do not catch it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@trafficserver.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
