Hello, I've updated patches for std::string sanitization and disabling CXX11 string SSO usage for correct sanitization.
>> _M_destroy(_M_allocated_capacity); >>+ else >>+ __annotate_delete(); > >Do these calls definitely optimize away completely when not >sanitizing? Even for -O1, -Os and -Og? > >For std::vector annotation I used macros to add these annotations, so >there is no change to the generated code when annotations are >disabled. But it makes the code quite ugly. I've checked asm code for string-inst.o and it looks like this calls are optimized away, but there are some light changes after patch fir . > Right, I was wondering specifically about the <fstream> > instantiations. I could be wrong but I don't think anything in > <fstream> creates, destroys or modifies a std::basic_string. I was confused by reinterpret_cast's on strings in fstream.tcc, looks like this is not needed, you are right. >> // calling 4.0.x+ _S_create. >> __p->_M_set_sharable(); >>+#if _GLIBCXX_SANITIZER_ANNOTATE_STRING >>+ __p->_M_length = 0; >>+#endif > > Why is this needed? I think a comment explaining it would help (like > the one above explaining why calling _M_set_sharable() is needed). Checked current version without this change, looks like now it works, reverted. Short summary: The reason for changing strings layout under sanitization is to place string char buffer on heap for correct aligning in 32-bit environments, both pre-CXX11 and CXX11 strings ABI. | Sanitize string | string type | ABI is changed? | 32-bit | 64-bit | |-----------------+-------------+-----------------+--------+--------| | FULL | SSO-string | yes | + | + | | | COW-string | yes | + | + | |-----------------+-------------+-----------------+--------+--------| | PARTIAL | SSO-string | no | -+(*) | + | | | COW-string | no | - | + | *only longs strings are sanitized for 32-bit Some functions with new define looks a bit messy without changing internal functions(operator=), I'm also not sure if disabling string SSO usage define should also affects other parts that relies on basic_string class size (checks with static_assert in exceptions/shim-facets). Any thoughts? On 05/29/2018 06:55 PM, Jonathan Wakely wrote: > On 29/05/18 18:18 +0300, Kashkarov Mikhail wrote: >> Jonathan Wakely <jwak...@redhat.com> writes: >>>> --- a/libstdc++-v3/include/bits/fstream.tcc >>>> +++ b/libstdc++-v3/include/bits/fstream.tcc >>>> @@ -1081,6 +1081,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION >>>> >>>> // Inhibit implicit instantiations for required instantiations, >>>> // which are defined via explicit instantiations elsewhere. >>>> +#if !_GLIBCXX_SANITIZE_STRING >>>> #if _GLIBCXX_EXTERN_TEMPLATE >>>> extern template class basic_filebuf<char>; >>>> extern template class basic_ifstream<char>; >>>> @@ -1094,6 +1095,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION >>>> extern template class basic_fstream<wchar_t>; >>>> #endif >>>> #endif >>>> +#endif // !_GLIBCXX_SANITIZE_STRING >>> >>> Why do we need to disable these explicit instantiation declarations? >>> Are they affected by the std::string layout changes? Is that just >>> because of the constructors taking std::string, or something else? >> >> Libstdc++ build is not sanitized, so macroses that requires >> AddressSanitizer support will not applied and these templates will be >> instantate without support for ASan annotations. > > Right, I was wondering specifically about the <fstream> > instantiations. I could be wrong but I don't think anything in > <fstream> creates, destroys or modifies a std::basic_string. > > > > > -- Best regards, Kashkarov Mikhail Samsung R&D Institute Russia
From 4b8de0240ac091cdd43b690276f09e94bfb0ef4d Mon Sep 17 00:00:00 2001 From: Mikhail Kashkarov <m.kashka...@partner.samsung.com> Date: Fri, 8 Jun 2018 14:11:11 +0300 Subject: [PATCH 2/2] Add AddressSanitizer annotations to std::string. * include/bits/c++config: define (_GLIBCXX_SANITIZE_STRING_PARTIAL, _GLIBCXX_SANITIZE_STRING_FULL) (_GLIBCXX_SANTIZE_STRING, _GLIBCXX_SANITIZER_ANNOTATE_STRING) (_GLIBCXX_SANITIZER_DISABLE_LOCAL_STRING_ANNOTATION) (_GLIBCXX_SANITIZER_ALIGN_COW_STRING) * doc/xml/manual/using.xml: document GLIBCXX_SANITIZE_STRING_PARTIAL, _GLIBCXX_SANITIZE_STRING_FULL * include/bits/basic_string.h [_GLIBCXX_USE_DUAL_ABI] (_asan_traits<_CharT, _Alloc>, _asan_traits<_CharT, allocator<_CharT>) (_asan_traits::__annotate_delete, _asan_traits::__annotate_new) (_asan_traits::__annotate_grow): New traits for annotation. (basic_string::__RAII_IncreaseAnnotator::__RAII_Increaseannotator) (basic_string::__RAII_IncreaseAnnotator::done) (basic_string::__RAII_IncreaseAnnotator::~RAII_Increaseannotator): New annotation helpers in case of exceptions. (basic_string::__get_beg, basic_string::__get_mid) (basic_string::__get_end): New annotation helpers. (basis_string::_M_dispose, basic_string::_M_destroy) (basic_string::basic_string(basic_string&& __str)) (basic_string::basic_string(basic_string&& __str, const _Alloc& __a)) (basic_string::operator=(constbasic_string& __str)) (basic_string::operator=(basic_string&& __str)) (basic_string::clear, basic_string::push_back): Annotate. [!_GLIBCXX_USE_DUAL_ABI] (_asan_traits<_CharT, _Alloc>, _asan_traits<_CharT, allocator<_CharT>) (_asan_traits::__annotate_delete, _asan_traits::__annotate_new) (_asan_traits::__annotate_grow): New traits for annotation. (basic_string::__RAII_IncreaseAnnotator::__RAII_Increaseannotator) (basic_string::__RAII_IncreaseAnnotator::done) (basic_string::__RAII_IncreaseAnnotator::~RAII_Increaseannotator): New annotation helpers in case of exceptions. (basic_string::__get_beg, basic_string::__get_mid) (basic_string::__get_end): New annotation helpers. (basic_string::_Rep_base): Align by 8 when annotations are anabled. (basic_string::push_back): Annotate * include/bits/basic_string.tcc [_GLIBCXX_USE_DUAL_ABI] (basic_string::swap, basic_string::_M_construct) (basic_string::_M_assign, basic_string::reserve) (basic_string::_M_mutate, basic_string::_M_erase) (basic_string::resize, basic_string::_M_append) (basic_string::_M_replace): Annotate. [!_GLIBCXX_USE_DUAL_ABI] (basic_string::_S_construct) (basic_string::append(const _CharT* __s, size_type __n)) (basic_string::append(const basic_string& __str)) (basic_string::_M_destroy, basic_string::_M_mutate) (basic_string::_S_create, basic_string::_M_clone): Annotate. Disable template implicit instantiation declarations for annotations. * libstdc++-v3/include/bits/sstream.tcc (basic_stringbuf::overflow, basic_stringbuf::_M_sync): Annotate Disable template implicit instantiation declarations for annotations. * libstdc++-v3/include/bits/locale_facets.tcc: Disable template implicit instantiation declarations for annotations. * libstdc++-v3/include/bits/fstream.tcc: Likewise. --- libstdc++-v3/doc/xml/manual/using.xml | 41 ++++ libstdc++-v3/include/bits/basic_string.h | 314 +++++++++++++++++++++++++++- libstdc++-v3/include/bits/basic_string.tcc | 66 +++++- libstdc++-v3/include/bits/c++config | 28 +++ libstdc++-v3/include/bits/locale_facets.tcc | 2 + libstdc++-v3/include/bits/sstream.tcc | 8 + 6 files changed, 450 insertions(+), 9 deletions(-) diff --git a/libstdc++-v3/doc/xml/manual/using.xml b/libstdc++-v3/doc/xml/manual/using.xml index 67f9cf5..ba5af0d 100644 --- a/libstdc++-v3/doc/xml/manual/using.xml +++ b/libstdc++-v3/doc/xml/manual/using.xml @@ -992,6 +992,47 @@ g++ -Winvalid-pch -I. -include stdc++.h -H -g -O2 hello.cc -o test.exe </para> </listitem></varlistentry> + <varlistentry><term><code>_GLIBCXX_SANITIZE_STRING_PARTIAL</code></term> + <listitem> + <para> + Undefined by default. When defined, <classname>std::string</classname> + operations will be annotated so that AddressSanitizer can detect + invalid accesses to the unused capacity of a + <classname>std::string</classname>. These annotations are only + enabled for + <classname>std::basic_string<T, std::allocator<T>></classname> + and only when <classname>std::allocator</classname> is derived from + <xref linkend="allocator.impl"><classname>new_allocator</classname> + or <classname>malloc_allocator</classname></xref>. The annotations + must be present on all string operations or none, so this macro must be + defined to the same value for all translation units that create, destroy or + modify strings. With this partial sanitization CXX11 small strings wiil be not + annotated, 64-bit environment is required for COW-strings to align + character buffer correctly. + </para> + </listitem></varlistentry> + + <varlistentry><term><code>_GLIBCXX_SANITIZE_STRING_FULL</code></term> + <listitem> + <para> + Undefined by default. When defined, <classname>std::string</classname> + operations will be annotated so that AddressSanitizer can detect + invalid accesses to the unused capacity of a + <classname>std::string</classname>. These annotations are only + enabled for + <classname>std::basic_string<T, std::allocator<T>></classname> + and only when <classname>std::allocator</classname> is derived from + <xref linkend="allocator.impl"><classname>new_allocator</classname> + or <classname>malloc_allocator</classname></xref>. The annotations + must be present on all string operations or none, so this macro must + be defined to the same value for all translation units that create, + destroy or modify strings. Strings ABI is changed: pre-CXX11 string + character buffer is forced to be aligned by 8 with annotations, disable + small-string optimization for CXX11 stings to allow correct AddressSanitizer + poisoning. + </para> + </listitem></varlistentry> + <varlistentry><term><code>_GLIBCXX_SANITIZE_VECTOR</code></term> <listitem> <para> diff --git a/libstdc++-v3/include/bits/basic_string.h b/libstdc++-v3/include/bits/basic_string.h index 9d971c0..d0448d9 100644 --- a/libstdc++-v3/include/bits/basic_string.h +++ b/libstdc++-v3/include/bits/basic_string.h @@ -48,11 +48,84 @@ # include <string_view> #endif +#if _GLIBCXX_SANITIZER_ANNOTATE_STRING +extern "C" void +__sanitizer_annotate_contiguous_container(const void *, const void *, + const void *, const void *); +#endif namespace std _GLIBCXX_VISIBILITY(default) { _GLIBCXX_BEGIN_NAMESPACE_VERSION + template <typename _CharT, typename _Alloc> + struct _Asan_traits + { + typedef typename + __gnu_cxx::__alloc_traits<_Alloc>::const_pointer const_pointer; + + static void __annotate_delete(const_pointer __beg, + const_pointer __mid, + const_pointer __end) { } + + static void __annotate_new(const_pointer __beg, const_pointer __mid, + const_pointer __end) { } + + static void __annotate_grow(const_pointer __beg, + const_pointer __old_mid, + const_pointer __new_mid, + const_pointer __end) { } + }; + + // AddressSanitizer is enabled only for default allocator. +#if _GLIBCXX_SANITIZER_ANNOTATE_STRING + template <typename _CharT> + struct _Asan_traits<_CharT, allocator<_CharT> > + { + typedef typename + __gnu_cxx::__alloc_traits<allocator_CharT>::const_pointer const_pointer; + + static void _GLIBCXX_VISIBILITY(hidden) __annotate_delete( + const_pointer __beg, + const_pointer __mid, + const_pointer __end) + { + __annotate_contiguous_container(__beg, __end, __mid, __end); + } + + static void _GLIBCXX_VISIBILITY(hidden) __annotate_new( + const_pointer __beg, + const_pointer __mid, + const_pointer __end) + { + __annotate_contiguous_container(__beg, __end, __end, __mid); + } + + static void _GLIBCXX_VISIBILITY(hidden) __annotate_grow( + const_pointer __beg, + const_pointer __old_mid, + const_pointer __new_mid, + const_pointer __end) + { + __annotate_contiguous_container(__beg, __end, __old_mid, __new_mid); + } + + private: + static void _GLIBCXX_VISIBILITY(hidden) __annotate_contiguous_container( + const void* __beg, + const void* __end, + const void* __old_mid, + const void* __new_mid) + { + if (__beg) + { + __sanitizer_annotate_contiguous_container(__beg, __end, __old_mid, + __new_mid); + } + } + }; +#endif // _GLIBCXX_SANITIZER_ANNOTATE_STRING + #if _GLIBCXX_USE_CXX11_ABI _GLIBCXX_BEGIN_NAMESPACE_CXX11 /** @@ -96,11 +169,130 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11 const_iterator; typedef std::reverse_iterator<const_iterator> const_reverse_iterator; typedef std::reverse_iterator<iterator> reverse_iterator; + typedef _Asan_traits<_CharT, _Char_alloc_type> asan_traits; /// Value returned by various member functions when they fail. static const size_type npos = static_cast<size_type>(-1); private: +#if _GLIBCXX_SANITIZER_ANNOTATE_STRING + // The annotation for size increase should happen before the actual + // increase, but if an exception is thrown after that the annotation has + // to be undone. + struct _GLIBCXX_VISIBILITY(hidden) _RAII_IncreaseAnnotator + { + _RAII_IncreaseAnnotator (const basic_string& __s, + difference_type __n = 1) + : __commit(false), __s(__s), __n(__n) + { + if (__n > 0) // grow before use + { + size_type __old_size = __s.size(); + if (__s._M_is_local() + && ((__s.size() + __n) > __s.capacity())) + { + __n = __s.capacity() - __s.size(); + } + __s.__annotate_grow(__old_size, __old_size + __n); + } + } + + void __done() + { + if (__n < 0) // shrink after use + { + size_type __old_size = __s.size(); + __s.__annotate_grow(__old_size, __old_size + __n); + } + __commit = true; + } + + ~_RAII_IncreaseAnnotator() + { + if (__commit) return; + size_type __cur_size = __s.size(); + if (__s._M_is_local()) + __s.__annotate_grow(__s.capacity(), __cur_size); + else + __s.__annotate_grow(__cur_size + __n, __cur_size); + } + + bool __commit; + difference_type __n; + const basic_string &__s; + }; +#else // _GLIBCXX_SANITIZER_ANNOTATE_STRING + struct _GLIBCXX_VISIBILITY(hidden) _RAII_IncreaseAnnotator + { + inline _RAII_IncreaseAnnotator(const basic_string &, size_type __n = 1) + {} + inline void __done() { } + ~_RAII_IncreaseAnnotator() { } + }; +#endif // _GLIBCXX_SANITIZER_ANNOTATE_STRING + + const_pointer _S_get_beg () const _GLIBCXX_VISIBILITY(hidden) + { + if (!_M_is_local()) + return _M_data(); + return _M_local_data(); + } + + const_pointer _S_get_mid(size_type __sz) const _GLIBCXX_VISIBILITY(hidden) + { + if (!_M_is_local()) + return _M_data() + __sz + 1 /* terminator */; + else + return _M_local_data() + __sz + 1; + } + + const_pointer _S_get_end() const _GLIBCXX_VISIBILITY(hidden) + { + if (!_M_is_local()) + return _M_data() + _M_allocated_capacity + 1; + + const_pointer __p = _S_get_beg() + _S_local_capacity + 1; + + return __p; + } + + void __annotate_new(size_type __sz) const _GLIBCXX_VISIBILITY(hidden) + { +#ifdef _GLIBCXX_SANITIZER_DISABLE_LOCAL_STRING_ANNOTATION + if (_M_is_local()) + return; +#endif + asan_traits::__annotate_new(_S_get_beg(), _S_get_mid(__sz), + _S_get_end()); + } + + void __annotate_grow(size_type __old_size, size_type __new_size) const + _GLIBCXX_VISIBILITY(hidden) + { +#ifdef _GLIBCXX_SANITIZER_DISABLE_LOCAL_STRING_ANNOTATION + if (_M_is_local()) + return; +#endif + asan_traits::__annotate_grow(_S_get_beg(), _S_get_mid(__old_size), + _S_get_mid(__new_size), _S_get_end()); + } + + basic_string<_CharT, _Traits, _Alloc> const *__annotate_delete() const + _GLIBCXX_VISIBILITY(hidden) + { +#ifdef _GLIBCXX_SANITIZER_DISABLE_LOCAL_STRING_ANNOTATION + if (_M_is_local()) + return this; +#endif + if (_M_is_local()) + asan_traits::__annotate_delete(_S_get_beg(), _S_get_mid(0), + _S_get_end()); + else + asan_traits::__annotate_delete(_S_get_beg(), _S_get_mid(size()), + _S_get_end()); + return this; + } + // type used for positions in insert, erase etc. #if __cplusplus < 201103L typedef iterator __const_iterator; @@ -225,6 +417,8 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11 { if (!_M_is_local()) _M_destroy(_M_allocated_capacity); + else + __annotate_delete(); } void @@ -234,6 +428,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11 if (!_M_allocated_capacity) return; #endif + __annotate_delete(); _Alloc_traits::deallocate(_M_get_allocator(), _M_data(), __size + 1); } @@ -560,8 +755,10 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11 #if !_GLIBCXX_DISABLE_STRING_SSO_USAGE if (__str._M_is_local()) { + __str.__annotate_delete(); traits_type::copy(_M_local_buf, __str._M_local_buf, _S_local_capacity + 1); + __annotate_new(__str.length()); } else #endif @@ -582,6 +779,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11 __str._M_data(__str._M_local_data()); __str._M_set_length(0); #endif + __str.__annotate_new(0); } /** @@ -607,6 +805,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11 traits_type::copy(_M_local_buf, __str._M_local_buf, _S_local_capacity + 1); _M_length(__str.length()); + __annotate_new(__str.length()); __str._M_set_length(0); } else @@ -617,6 +816,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11 _M_data(__str._M_data()); _M_length(__str.length()); _M_capacity(__str._M_allocated_capacity); + __annotate_new(__str.length()); #if _GLIBCXX_DISABLE_STRING_SSO_USAGE __str._M_data(nullptr); __str._M_length(0); @@ -625,7 +825,6 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11 __str._M_data(__str._M_local_buf); __str._M_set_length(0); #endif - } else _M_construct(__str.begin(), __str.end()); @@ -729,6 +928,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11 _M_data(__ptr); _M_capacity(__new_cap); _M_set_length(__len); + __annotate_new(__len); } } std::__alloc_on_copy(_M_get_allocator(), __str._M_get_allocator()); @@ -774,6 +974,10 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11 operator=(basic_string&& __str) noexcept(_Alloc_traits::_S_nothrow_move()) { +#if _GLIBCXX_SANITIZER_ANNOTATE_STRING + size_type __str_capacity = __str.capacity(); + size_type __str_length = __str.length(); +#endif if (!_M_is_local() && _Alloc_traits::_S_propagate_on_move_assign() && !_Alloc_traits::_S_always_equal() && _M_get_allocator() != __str._M_get_allocator()) @@ -786,6 +990,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11 #else _M_data(_M_local_data()); _M_set_length(0); + __annotate_new(0); #endif } // Replace allocator if POCMA is true. @@ -797,16 +1002,18 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11 { pointer __data = nullptr; size_type __capacity; -#if _GLIBCXX_DISABLE_STRING_SSO_USAGE +#if _GLIBCXX_DISABLE_STRING_SSO_USAGE || _GLIBCXX_SANITIZER_ANNOTATE_STRING size_type __length; #endif + __annotate_delete(); + __str.__annotate_delete(); if (!_M_is_local()) { if (_Alloc_traits::_S_always_equal()) { __data = _M_data(); __capacity = _M_allocated_capacity; -#if _GLIBCXX_DISABLE_STRING_SSO_USAGE +#if _GLIBCXX_DISABLE_STRING_SSO_USAGE || _GLIBCXX_SANITIZER_ANNOTATE_STRING __length = _M_string_length; #endif } @@ -826,8 +1033,9 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11 { __str._M_data(__data); __str._M_capacity(__capacity); -#if _GLIBCXX_DISABLE_STRING_SSO_USAGE +#if _GLIBCXX_DISABLE_STRING_SSO_USAGE || _GLIBCXX_SANITIZER_ANNOTATE_STRING __str._M_length(__length); + __str.__annotate_new(__str.length()); #endif } else { @@ -837,8 +1045,10 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11 __str._M_capacity(0); #else __str._M_data(__str._M_local_buf); + __str.__annotate_new(__str.length()); #endif } + __annotate_new(length()); } else assign(__str); @@ -1081,6 +1291,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11 void clear() _GLIBCXX_NOEXCEPT { + __annotate_grow(_M_string_length, 0); #if _GLIBCXX_DISABLE_STRING_SSO_USAGE _M_allocated_capacity ? _M_set_length(0) : _M_length(0); #else @@ -1416,6 +1627,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11 const size_type __size = this->size(); if (__size + 1 > this->capacity()) this->_M_mutate(__size, size_type(0), 0, size_type(1)); + this->__annotate_grow(__size, __size + 1); traits_type::assign(this->_M_data()[__size], __c); this->_M_set_length(__size + 1); } @@ -3208,12 +3420,57 @@ _GLIBCXX_END_NAMESPACE_CXX11 size_type _M_capacity; _Atomic_word _M_refcount; }; + // } _GLIBCXX_SANITIZER_ALIGN_COW_STRING; struct _Rep : _Rep_base { // Types: typedef typename _Alloc::template rebind<char>::other _Raw_bytes_alloc; + typedef _Asan_traits<_CharT, _CharT_alloc_type> asan_traits; + #if _GLIBCXX_SANITIZE_ANNOTATE_STRING + const_pointer _S_get_beg() const _GLIBCXX_NOEXCEPT + { + return _M_refdata(); + } + + const_pointer _S_get_mid(size_type __sz) const _GLIBCXX_NOEXCEPT + { + return std::min(_M_refdata() + __sz + 1, /* terminator */ + _S_get_end()); + } + const_pointer _S_get_end() const _GLIBCXX_NOEXCEPT + { + return _M_refdata() + this->_M_capacity + 1 /* terminator */; + } + + void __annotate_new(size_type __sz) const + { + asan_traits::__annotate_new(_S_get_beg(), _S_get_mid(__sz), + _S_get_end()); + } + + void __annotate_grow(size_type __old_size, size_type __new_size) const + { + asan_traits::__annotate_grow(_S_get_beg(), _S_get_mid(__old_size), + _S_get_mid(__new_size), _S_get_end()); + } + + _Rep const *__annotate_delete() const + { + asan_traits::__annotate_delete(_S_get_beg(), + _S_get_mid(this->_M_length), + _S_get_end()); + return this; + } +#else // _GLIBCXX_SANITIZE_ANNOTATE_STRING + const_pointer _S_get_beg() const _GLIBCXX_NOEXCEPT { return 0; } + const_pointer _S_get_mid(size_type __sz) const _GLIBCXX_NOEXCEPT { return 0; } + const_pointer _S_get_end() const _GLIBCXX_NOEXCEPT { return 0; } + void __annotate_new(size_type __sz) const { } + void __annotate_grow(size_type __old_size, size_type __new_size) const { } + _Rep const *__annotate_delete() const { return this; } +#endif // _GLIBCXX_SANITIZE_ANNOTATE_STRING // (Public) Data members: // The maximum number of individual char_type elements of an @@ -3296,6 +3553,10 @@ _GLIBCXX_END_NAMESPACE_CXX11 } } + const_pointer + _M_refdata() const throw() + { return reinterpret_cast<const_pointer>(this + 1); } + _CharT* _M_refdata() throw() { return reinterpret_cast<_CharT*>(this + 1); } @@ -3354,6 +3615,50 @@ _GLIBCXX_END_NAMESPACE_CXX11 _M_clone(const _Alloc&, size_type __res = 0); }; +#if _GLIBCXX_SANITIZER_ANNOTATE_STRING + // The annotation for size increase should happen before the actual + // increase, but if an exception is thrown after that the annotation has + // to be undone. + struct _GLIBCXX_VISIBILITY(hidden) _RAII_IncreaseAnnotator + { + _RAII_IncreaseAnnotator (const _Rep& __s, difference_type __n = 1) + : __commit(false), __s(__s), __n(__n) + { + if (__n > 0) // grow before use + { + size_type __old_size = __s._M_length; + __s.__annotate_grow(__old_size, __old_size + __n); + } + } + void __done() + { + if (__n < 0) // shrink after use + { + size_type __old_size = __s._M_length; + __s.__annotate_grow(__old_size, __old_size + __n); + } + __commit = true; + } + ~_RAII_IncreaseAnnotator() + { + if (__commit) return; + size_type __cur_size = __s._M_length; + __s.__annotate_grow(__cur_size + __n, __cur_size); + } + bool __commit; + difference_type __n; + const _Rep &__s; + }; +#else // _GLIBCXX_SANITIZER_ANNOTATE_STRING + struct _GLIBCXX_VISIBILITY(hidden) _RAII_IncreaseAnnotator + { + inline _RAII_IncreaseAnnotator(const _Rep &, size_type __n = 1) + {} + inline void __done() {} + ~_RAII_IncreaseAnnotator() {} + }; +#endif // _GLIBCXX_SANITIZER_ANNOTATE_STRING + // Use empty-base optimization: http://www.cantrip.org/emptyopt.html struct _Alloc_hider : _Alloc { @@ -4318,6 +4623,7 @@ _GLIBCXX_END_NAMESPACE_CXX11 push_back(_CharT __c) { const size_type __len = 1 + this->size(); + _M_rep()->__annotate_grow(this->size(), this->size() + 1); if (__len > this->capacity() || _M_rep()->_M_is_shared()) this->reserve(__len); traits_type::assign(_M_data()[this->size()], __c); diff --git a/libstdc++-v3/include/bits/basic_string.tcc b/libstdc++-v3/include/bits/basic_string.tcc index 6b6c7eb..89d57d3 100644 --- a/libstdc++-v3/include/bits/basic_string.tcc +++ b/libstdc++-v3/include/bits/basic_string.tcc @@ -62,6 +62,8 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION return; _Alloc_traits::_S_on_swap(_M_get_allocator(), __s._M_get_allocator()); + __annotate_delete(); + __s.__annotate_delete(); if (_M_is_local()) if (__s._M_is_local()) @@ -124,7 +126,9 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION const size_type __tmp_length = length(); _M_length(__s.length()); + __annotate_new(length()); __s._M_length(__tmp_length); + __s.__annotate_new(__s.length()); } template<typename _CharT, typename _Traits, typename _Alloc> @@ -175,8 +179,10 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION while (__beg != __end && __len < __capacity) { + _RAII_IncreaseAnnotator __annotator(*this); _M_data()[__len++] = *__beg; ++__beg; + __annotator.__done(); } __try @@ -196,9 +202,12 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION _M_dispose(); _M_data(__another); _M_capacity(__capacity); + __annotate_new(__len); } + _RAII_IncreaseAnnotator __annotator(*this); _M_data()[__len++] = *__beg; ++__beg; + __annotator.__done(); } } __catch(...) @@ -235,6 +244,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION { _M_data(_M_create(__new_capacity, size_type(0))); _M_capacity(__new_capacity); + __annotate_new(__dnew); } // Check for out_of_range and length_error exceptions. @@ -266,6 +276,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION { _M_data(_M_create(__new_capacity, size_type(0))); _M_capacity(__new_capacity); + __annotate_new(__n); } if (__n) @@ -281,6 +292,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION { if (this != &__str) { + __annotate_delete(); const size_type __rsize = __str.length(); const size_type __capacity = capacity(); @@ -301,6 +313,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION #else _M_set_length(__rsize); #endif + __annotate_new(__rsize); } } @@ -335,6 +348,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION _M_data(_M_local_data()); } #endif + __annotate_new(length()); } } @@ -360,6 +374,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION _M_dispose(); _M_data(__r); _M_capacity(__new_capacity); + __annotate_new(__pos + __len2 + __how_much); } template<typename _CharT, typename _Traits, typename _Alloc> @@ -372,7 +387,9 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION if (__how_much && __n) this->_S_move(_M_data() + __pos, _M_data() + __pos + __n, __how_much); + const size_type __old_length = length(); _M_set_length(length() - __n); + __annotate_grow(__old_length, __old_length - __n); } template<typename _CharT, typename _Traits, typename _Alloc> @@ -384,7 +401,10 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION if (__size < __n) this->append(__n - __size, __c); else if (__n < __size) - this->_M_set_length(__n); + { + this->_M_set_length(__n); + this->__annotate_grow(__size, __n); + } } template<typename _CharT, typename _Traits, typename _Alloc> @@ -392,12 +412,16 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION basic_string<_CharT, _Traits, _Alloc>:: _M_append(const _CharT* __s, size_type __n) { - const size_type __len = __n + this->size(); + const size_type __old_len = this->size(); + const size_type __len = __n + __old_len; if (__len <= this->capacity()) { if (__n) - this->_S_copy(this->_M_data() + this->size(), __s, __n); + { + __annotate_grow(__old_len, __len); + this->_S_copy(this->_M_data() + __old_len, __s, __n); + } } else this->_M_mutate(this->size(), size_type(0), __s, __n); @@ -436,8 +460,10 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION pointer __p = this->_M_data() + __pos1; const size_type __how_much = __old_size - __pos1 - __n1; + _RAII_IncreaseAnnotator __annotator(*this, __n2 - __n1); if (__how_much && __n1 != __n2) this->_S_move(__p + __n2, __p + __n1, __how_much); + __annotator.__done(); } else this->_M_mutate(__pos1, __n1, 0, __n2); @@ -462,6 +488,9 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION if (__new_size <= this->capacity()) { + // grow before use + if (__old_size < __new_size) + __annotate_grow(__old_size, __new_size); pointer __p = this->_M_data() + __pos; const size_type __how_much = __old_size - __pos - __len1; @@ -494,6 +523,10 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION } } } + // shrink after use + if (__old_size > __new_size) { + __annotate_grow(__old_size, __new_size); + } } else this->_M_mutate(__pos, __len1, __s, __len2); @@ -591,6 +624,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION __r->_M_destroy(__a); __throw_exception_again; } + __r->__annotate_new(__len); __r->_M_set_length_and_sharable(__len); return __r->_M_refdata(); } @@ -621,6 +655,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION __r->_M_destroy(__a); __throw_exception_again; } + __r->__annotate_new(__dnew); __r->_M_set_length_and_sharable(__dnew); return __r->_M_refdata(); } @@ -636,6 +671,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION #endif // Check for out_of_range and length_error exceptions. _Rep* __r = _Rep::_S_create(__n, size_type(0), __a); + __r->__annotate_new(__n); if (__n) _M_assign(__r->_M_refdata(), __n, __c); @@ -771,10 +807,13 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION if (__n) { _M_check_length(size_type(0), __n, "basic_string::append"); - const size_type __len = __n + this->size(); + const size_type __old_len = this->size(); + const size_type __len = __n + __old_len; if (__len > this->capacity() || _M_rep()->_M_is_shared()) this->reserve(__len); + _RAII_IncreaseAnnotator __annotator(*_M_rep(), __n); _M_assign(_M_data() + this->size(), __n, __c); + __annotator.__done(); _M_rep()->_M_set_length_and_sharable(__len); } return *this; @@ -801,6 +840,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION __s = _M_data() + __off; } } + _M_rep()->__annotate_grow(this->size(), __len); _M_copy(_M_data() + this->size(), __s, __n); _M_rep()->_M_set_length_and_sharable(__len); } @@ -815,9 +855,11 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION const size_type __size = __str.size(); if (__size) { - const size_type __len = __size + this->size(); + const size_type __old_len = this->size(); + const size_type __len = __size + __old_len; if (__len > this->capacity() || _M_rep()->_M_is_shared()) this->reserve(__len); + _M_rep()->__annotate_grow(__old_len, __len); _M_copy(_M_data() + this->size(), __str._M_data(), __size); _M_rep()->_M_set_length_and_sharable(__len); } @@ -934,6 +976,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION { const size_type __size = sizeof(_Rep_base) + (this->_M_capacity + 1) * sizeof(_CharT); + __annotate_delete(); _Raw_bytes_alloc(__a).deallocate(reinterpret_cast<char*>(this), __size); } @@ -965,12 +1008,16 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION // Must reallocate. const allocator_type __a = get_allocator(); _Rep* __r = _Rep::_S_create(__new_size, this->capacity(), __a); + __r->_M_length = 0; + __r->__annotate_new(0); + _RAII_IncreaseAnnotator __annotator(*__r, __new_size); if (__pos) _M_copy(__r->_M_refdata(), _M_data(), __pos); if (__how_much) _M_copy(__r->_M_refdata() + __pos + __len2, _M_data() + __pos + __len1, __how_much); + __annotator.__done(); _M_rep()->_M_dispose(__a); _M_data(__r->_M_refdata()); @@ -978,9 +1025,15 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION else if (__how_much && __len1 != __len2) { // Work in-place. + if (__new_size > __old_size) + _M_rep()->__annotate_grow(__old_size, __new_size); _M_move(_M_data() + __pos + __len2, _M_data() + __pos + __len1, __how_much); + if (__new_size < __old_size) + _M_rep()->__annotate_grow(__old_size, __new_size); } + else + _M_rep()->__annotate_grow(__old_size, __new_size); _M_rep()->_M_set_length_and_sharable(__new_size); } @@ -1115,6 +1168,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION const size_type __requested_cap = this->_M_length + __res; _Rep* __r = _Rep::_S_create(__requested_cap, this->_M_capacity, __alloc); + __r->__annotate_new(this->_M_length); if (this->_M_length) _M_copy(__r->_M_refdata(), _M_refdata(), this->_M_length); @@ -1635,6 +1689,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION // Inhibit implicit instantiations for required instantiations, // which are defined via explicit instantiations elsewhere. +#if !_GLIBCXX_SANITIZE_STRING #if _GLIBCXX_EXTERN_TEMPLATE > 0 && __cplusplus <= 201402L extern template class basic_string<char>; extern template @@ -1666,6 +1721,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION getline(basic_istream<wchar_t>&, wstring&); #endif #endif +#endif _GLIBCXX_END_NAMESPACE_VERSION } // namespace std diff --git a/libstdc++-v3/include/bits/c++config b/libstdc++-v3/include/bits/c++config index 280f65e..0f29f9c 100644 --- a/libstdc++-v3/include/bits/c++config +++ b/libstdc++-v3/include/bits/c++config @@ -420,6 +420,34 @@ namespace std # define _GLIBCXX_END_NAMESPACE_LDBL_OR_CXX11 _GLIBCXX_END_NAMESPACE_LDBL #endif +// Partial AddressSanitizer std::basic_string annotation: +// ABI is unchanged, short strings are not sanitized, +// works with cow-strings only under 64-bits environment. +#if _GLIBCXX_SANITIZE_STRING_PARTIAL +# define _GLIBCXX_SANITIZE_STRING 1 +# define _GLIBCXX_DISABLE_STRING_SSO_USAGE 0 +# define _GLIBCXX_SANITIZER_ALIGN_COW_STRING +# if _GLIBCXX_SANITIZE_STD_ALLOCATOR +# define _GLIBCXX_SANITIZER_ANNOTATE_STRING 1 +# define _GLIBCXX_SANITIZER_DISABLE_LOCAL_STRING_ANNOTATION +# endif +#endif + +// Full AddressSanitizer std::basic_string annotation: ABI is changed. +#if _GLIBCXX_SANITIZE_STRING_FULL +# define _GLIBCXX_SANITIZE_STRING 1 +# define _GLIBCXX_DISABLE_STRING_SSO_USAGE 1 +# define _GLIBCXX_SANITIZER_ALIGN_COW_STRING __attribute__((aligned(8))) +# if _GLIBCXX_SANITIZE_STD_ALLOCATOR +# define _GLIBCXX_SANITIZER_ANNOTATE_STRING 1 +# undef _GLIBCXX_SANITIZER_DISABLE_LOCAL_STRING_ANNOTATION +# endif +#endif + +#if !_GLIBCXX_SANITIZE_STRING +# define _GLIBCXX_SANITIZER_ALIGN_COW_STRING +#endif + // Debug Mode implies checking assertions. #if defined(_GLIBCXX_DEBUG) && !defined(_GLIBCXX_ASSERTIONS) # define _GLIBCXX_ASSERTIONS 1 diff --git a/libstdc++-v3/include/bits/locale_facets.tcc b/libstdc++-v3/include/bits/locale_facets.tcc index 39da5766..31ee6a4 100644 --- a/libstdc++-v3/include/bits/locale_facets.tcc +++ b/libstdc++-v3/include/bits/locale_facets.tcc @@ -1291,6 +1291,7 @@ _GLIBCXX_END_NAMESPACE_LDBL // Inhibit implicit instantiations for required instantiations, // which are defined via explicit instantiations elsewhere. +#if !_GLIBCXX_SANITIZE_STRING #if _GLIBCXX_EXTERN_TEMPLATE extern template class _GLIBCXX_NAMESPACE_CXX11 numpunct<char>; extern template class _GLIBCXX_NAMESPACE_CXX11 numpunct_byname<char>; @@ -1370,6 +1371,7 @@ _GLIBCXX_END_NAMESPACE_LDBL has_facet<num_get<wchar_t> >(const locale&); #endif #endif +#endif // !_GLIBCXX_SANITIZE_STRING _GLIBCXX_END_NAMESPACE_VERSION } // namespace diff --git a/libstdc++-v3/include/bits/sstream.tcc b/libstdc++-v3/include/bits/sstream.tcc index 5f90b76..ca1363b 100644 --- a/libstdc++-v3/include/bits/sstream.tcc +++ b/libstdc++-v3/include/bits/sstream.tcc @@ -92,6 +92,9 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION #if _GLIBCXX_USE_CXX11_ABI if ((this->epptr() - this->pbase()) < __capacity) { +#if _GLIBCXX_SANITIZER_ANNOTATE_STRING + _M_string.__annotate_grow(_M_string.size(), __capacity); +#endif // There is additional capacity in _M_string that can be used. char_type* __base = const_cast<char_type*>(_M_string.data()); _M_pbump(__base, __base + __capacity, this->pptr() - this->pbase()); @@ -269,6 +272,9 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION if (!__testin) this->setg(__endg, __endg, __endg); } +#if _GLIBCXX_USE_CXX11_ABI && _GLIBCXX_SANITIZER_ANNOTATE_STRING + _M_string.__annotate_grow(_M_string.size(), _M_string.capacity()); +#endif } template <class _CharT, class _Traits, class _Alloc> @@ -287,6 +293,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION // Inhibit implicit instantiations for required instantiations, // which are defined via explicit instantiations elsewhere. +#if !_GLIBCXX_SANITIZE_STRING #if _GLIBCXX_EXTERN_TEMPLATE extern template class basic_stringbuf<char>; extern template class basic_istringstream<char>; @@ -300,6 +307,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION extern template class basic_stringstream<wchar_t>; #endif #endif +#endif // !_GLIBCXX_SANITIZE_STRING _GLIBCXX_END_NAMESPACE_VERSION } // namespace std -- 2.7.4