Re: Should Arrow adopt C++14 / 17?
I opened https://issues.apache.org/jira/browse/ARROW-6800. On Fri, Oct 4, 2019 at 3:42 PM Wes McKinney wrote: > > On Fri, Oct 4, 2019 at 3:18 PM Zhuo Peng wrote: > > > > > > > > On 2019/10/04 19:43:04, Wes McKinney wrote: > > > On Fri, Oct 4, 2019 at 12:45 PM Zhuo Peng wrote: > > > > > > > > > > > > > > > > On 2019/10/04 17:05:00, Antoine Pitrou wrote: > > > > > > > > > > Le 04/10/2019 à 19:01, Zhuo Peng a écrit : > > > > > > > > > > > > backports are cool for internal use, but probably not so if a > > > > > > public API accepts it? (because you vendor the headers in (i.e. > > > > > > namespace, symbol names unchanged), they might clash with headers > > > > > > that a client uses). > > > > > > > > > > This is true unfortunately. > > > > > > > > > > >>> And btw, was -std=gnu++11 an intentional choice? what gnu > > > > > >>> extensions does the library rely on? > > > > > >> > > > > > >> None, AFAIK. Arrow compiles on MSVC fine. Where is -std=gnu++11 > > > > > >> added? > > > > > > https://github.com/apache/arrow/blob/3129e3ed90219ecfffe2a25ce5820eec8cc947d0/cpp/cmake_modules/SetupCxxFlags.cmake#L33 > > > > > > > > > > > > https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_STANDARD.html > > > > > > > > > > Right, so this is a CMake decision. I think we require only plain > > > > > C++11 > > > > > (but we may enable additional features on some compilers, provided > > > > > there's a fallback). > > > > Extensions can be disabled through: > > > > set(CMAKE_CXX_EXTENSIONS OFF) > > > > > > > > https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_EXTENSIONS.html > > > > > > > > Is that something more desirable than the current state? > > > > > > Yes, I think so, I don't think we need to be relying on GNU gcc > > > extensions, but we should open a JIRA issue about disabling it in case > > > some tests break because of something we didn't realize we were > > > depending on. > > sg. I'll create one then. > > > > > > As far as C++14/17 upgrading, it seems like it will be at least 2 > > > years before we could upgrade to C++17 given the state of compiler > > > support across the spectrum. Using C++17 would mean requiring at least > > > VS 2017 on Windows, since at least in the Python world I think > > > everything is on VS 2015. > > > > > > Are there ways we could create defines to switch between backports and > > > STL things (like string_view, optional, etc.) so that developers using > > > the Arrow library in a C++17 application can use the built-in types? > > This is dangerous unless they build the Arrow library from source with > > C++17. if libarrow takes arrow::string_view but the user gives it a > > std::string_view, it's UB. > > > > If we are talking about allowing users to build Arrow with C++17 and > > support transparently the new STL types in the public APIs, the ABSL[1] > > library could be something to consider.. > > absl::{string_view,optional,variant} becomes their std:: counterparts when > > compiled under C++17, e.g. [2]. > > > > Yes, the presumption would be a monotoolchain environment, and Arrow > would need to have some CMake options to build in C++17 mode > > > And inline namespaces are used [3] to make sure different libraries can > > depend on different version of absl. > > > > [1] https://abseil.io/ > > [2] > > https://github.com/abseil/abseil-cpp/blob/25597bdfc148e91e27678ec30efa52f4fc8c164f/absl/strings/string_view.h#L38 > > [3] > > https://github.com/abseil/abseil-cpp/blob/aa844899c937bde5d2b24f276b59997e5b668bde/absl/strings/string_view.h#L38 > > > > > > > > > > > > > Regards > > > > > > > > > > Antoine. > > > > > > > >
Re: Should Arrow adopt C++14 / 17?
On Fri, Oct 4, 2019 at 3:18 PM Zhuo Peng wrote: > > > > On 2019/10/04 19:43:04, Wes McKinney wrote: > > On Fri, Oct 4, 2019 at 12:45 PM Zhuo Peng wrote: > > > > > > > > > > > > On 2019/10/04 17:05:00, Antoine Pitrou wrote: > > > > > > > > Le 04/10/2019 à 19:01, Zhuo Peng a écrit : > > > > > > > > > > backports are cool for internal use, but probably not so if a public > > > > > API accepts it? (because you vendor the headers in (i.e. namespace, > > > > > symbol names unchanged), they might clash with headers that a client > > > > > uses). > > > > > > > > This is true unfortunately. > > > > > > > > >>> And btw, was -std=gnu++11 an intentional choice? what gnu > > > > >>> extensions does the library rely on? > > > > >> > > > > >> None, AFAIK. Arrow compiles on MSVC fine. Where is -std=gnu++11 > > > > >> added? > > > > > https://github.com/apache/arrow/blob/3129e3ed90219ecfffe2a25ce5820eec8cc947d0/cpp/cmake_modules/SetupCxxFlags.cmake#L33 > > > > > > > > > > https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_STANDARD.html > > > > > > > > Right, so this is a CMake decision. I think we require only plain C++11 > > > > (but we may enable additional features on some compilers, provided > > > > there's a fallback). > > > Extensions can be disabled through: > > > set(CMAKE_CXX_EXTENSIONS OFF) > > > > > > https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_EXTENSIONS.html > > > > > > Is that something more desirable than the current state? > > > > Yes, I think so, I don't think we need to be relying on GNU gcc > > extensions, but we should open a JIRA issue about disabling it in case > > some tests break because of something we didn't realize we were > > depending on. > sg. I'll create one then. > > > > As far as C++14/17 upgrading, it seems like it will be at least 2 > > years before we could upgrade to C++17 given the state of compiler > > support across the spectrum. Using C++17 would mean requiring at least > > VS 2017 on Windows, since at least in the Python world I think > > everything is on VS 2015. > > > > Are there ways we could create defines to switch between backports and > > STL things (like string_view, optional, etc.) so that developers using > > the Arrow library in a C++17 application can use the built-in types? > This is dangerous unless they build the Arrow library from source with C++17. > if libarrow takes arrow::string_view but the user gives it a > std::string_view, it's UB. > > If we are talking about allowing users to build Arrow with C++17 and support > transparently the new STL types in the public APIs, the ABSL[1] library could > be something to consider.. absl::{string_view,optional,variant} becomes their > std:: counterparts when compiled under C++17, e.g. [2]. > Yes, the presumption would be a monotoolchain environment, and Arrow would need to have some CMake options to build in C++17 mode > And inline namespaces are used [3] to make sure different libraries can > depend on different version of absl. > > [1] https://abseil.io/ > [2] > https://github.com/abseil/abseil-cpp/blob/25597bdfc148e91e27678ec30efa52f4fc8c164f/absl/strings/string_view.h#L38 > [3] > https://github.com/abseil/abseil-cpp/blob/aa844899c937bde5d2b24f276b59997e5b668bde/absl/strings/string_view.h#L38 > > > > > > > > > > Regards > > > > > > > > Antoine. > > > > > >
Re: Should Arrow adopt C++14 / 17?
On 2019/10/04 19:43:04, Wes McKinney wrote: > On Fri, Oct 4, 2019 at 12:45 PM Zhuo Peng wrote: > > > > > > > > On 2019/10/04 17:05:00, Antoine Pitrou wrote: > > > > > > Le 04/10/2019 à 19:01, Zhuo Peng a écrit : > > > > > > > > backports are cool for internal use, but probably not so if a public > > > > API accepts it? (because you vendor the headers in (i.e. namespace, > > > > symbol names unchanged), they might clash with headers that a client > > > > uses). > > > > > > This is true unfortunately. > > > > > > >>> And btw, was -std=gnu++11 an intentional choice? what gnu extensions > > > >>> does the library rely on? > > > >> > > > >> None, AFAIK. Arrow compiles on MSVC fine. Where is -std=gnu++11 > > > >> added? > > > > https://github.com/apache/arrow/blob/3129e3ed90219ecfffe2a25ce5820eec8cc947d0/cpp/cmake_modules/SetupCxxFlags.cmake#L33 > > > > > > > > https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_STANDARD.html > > > > > > Right, so this is a CMake decision. I think we require only plain C++11 > > > (but we may enable additional features on some compilers, provided > > > there's a fallback). > > Extensions can be disabled through: > > set(CMAKE_CXX_EXTENSIONS OFF) > > > > https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_EXTENSIONS.html > > > > Is that something more desirable than the current state? > > Yes, I think so, I don't think we need to be relying on GNU gcc > extensions, but we should open a JIRA issue about disabling it in case > some tests break because of something we didn't realize we were > depending on. sg. I'll create one then. > > As far as C++14/17 upgrading, it seems like it will be at least 2 > years before we could upgrade to C++17 given the state of compiler > support across the spectrum. Using C++17 would mean requiring at least > VS 2017 on Windows, since at least in the Python world I think > everything is on VS 2015. > > Are there ways we could create defines to switch between backports and > STL things (like string_view, optional, etc.) so that developers using > the Arrow library in a C++17 application can use the built-in types? This is dangerous unless they build the Arrow library from source with C++17. if libarrow takes arrow::string_view but the user gives it a std::string_view, it's UB. If we are talking about allowing users to build Arrow with C++17 and support transparently the new STL types in the public APIs, the ABSL[1] library could be something to consider.. absl::{string_view,optional,variant} becomes their std:: counterparts when compiled under C++17, e.g. [2]. And inline namespaces are used [3] to make sure different libraries can depend on different version of absl. [1] https://abseil.io/ [2] https://github.com/abseil/abseil-cpp/blob/25597bdfc148e91e27678ec30efa52f4fc8c164f/absl/strings/string_view.h#L38 [3] https://github.com/abseil/abseil-cpp/blob/aa844899c937bde5d2b24f276b59997e5b668bde/absl/strings/string_view.h#L38 > > > > > > > Regards > > > > > > Antoine. > > > >
Re: Should Arrow adopt C++14 / 17?
On Fri, Oct 4, 2019 at 12:45 PM Zhuo Peng wrote: > > > > On 2019/10/04 17:05:00, Antoine Pitrou wrote: > > > > Le 04/10/2019 à 19:01, Zhuo Peng a écrit : > > > > > > backports are cool for internal use, but probably not so if a public API > > > accepts it? (because you vendor the headers in (i.e. namespace, symbol > > > names unchanged), they might clash with headers that a client uses). > > > > This is true unfortunately. > > > > >>> And btw, was -std=gnu++11 an intentional choice? what gnu extensions > > >>> does the library rely on? > > >> > > >> None, AFAIK. Arrow compiles on MSVC fine. Where is -std=gnu++11 added? > > > https://github.com/apache/arrow/blob/3129e3ed90219ecfffe2a25ce5820eec8cc947d0/cpp/cmake_modules/SetupCxxFlags.cmake#L33 > > > > > > https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_STANDARD.html > > > > Right, so this is a CMake decision. I think we require only plain C++11 > > (but we may enable additional features on some compilers, provided > > there's a fallback). > Extensions can be disabled through: > set(CMAKE_CXX_EXTENSIONS OFF) > > https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_EXTENSIONS.html > > Is that something more desirable than the current state? Yes, I think so, I don't think we need to be relying on GNU gcc extensions, but we should open a JIRA issue about disabling it in case some tests break because of something we didn't realize we were depending on. As far as C++14/17 upgrading, it seems like it will be at least 2 years before we could upgrade to C++17 given the state of compiler support across the spectrum. Using C++17 would mean requiring at least VS 2017 on Windows, since at least in the Python world I think everything is on VS 2015. Are there ways we could create defines to switch between backports and STL things (like string_view, optional, etc.) so that developers using the Arrow library in a C++17 application can use the built-in types? > > > > Regards > > > > Antoine. > >
Re: Should Arrow adopt C++14 / 17?
On 2019/10/04 17:05:00, Antoine Pitrou wrote: > > Le 04/10/2019 à 19:01, Zhuo Peng a écrit : > > > > backports are cool for internal use, but probably not so if a public API > > accepts it? (because you vendor the headers in (i.e. namespace, symbol > > names unchanged), they might clash with headers that a client uses). > > This is true unfortunately. > > >>> And btw, was -std=gnu++11 an intentional choice? what gnu extensions does > >>> the library rely on? > >> > >> None, AFAIK. Arrow compiles on MSVC fine. Where is -std=gnu++11 added? > > https://github.com/apache/arrow/blob/3129e3ed90219ecfffe2a25ce5820eec8cc947d0/cpp/cmake_modules/SetupCxxFlags.cmake#L33 > > > > https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_STANDARD.html > > Right, so this is a CMake decision. I think we require only plain C++11 > (but we may enable additional features on some compilers, provided > there's a fallback). Extensions can be disabled through: set(CMAKE_CXX_EXTENSIONS OFF) https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_EXTENSIONS.html Is that something more desirable than the current state? > > Regards > > Antoine. >
Re: Should Arrow adopt C++14 / 17?
Le 04/10/2019 à 19:01, Zhuo Peng a écrit : > > backports are cool for internal use, but probably not so if a public API > accepts it? (because you vendor the headers in (i.e. namespace, symbol names > unchanged), they might clash with headers that a client uses). This is true unfortunately. >>> And btw, was -std=gnu++11 an intentional choice? what gnu extensions does >>> the library rely on? >> >> None, AFAIK. Arrow compiles on MSVC fine. Where is -std=gnu++11 added? > https://github.com/apache/arrow/blob/3129e3ed90219ecfffe2a25ce5820eec8cc947d0/cpp/cmake_modules/SetupCxxFlags.cmake#L33 > > https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_STANDARD.html Right, so this is a CMake decision. I think we require only plain C++11 (but we may enable additional features on some compilers, provided there's a fallback). Regards Antoine.
Re: Should Arrow adopt C++14 / 17?
C++14 isn't very interesting. C++17 is, but it's probably too young given the diversity of platform and toolchain requirements that constrai us. Regards Antoine. Le 04/10/2019 à 18:13, Neal Richardson a écrit : > We do have to care about more than just conda and Python. For R, for > example, C++14 support is limited: > https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Using-C_002b_002b14-code > > That's not to say that we can't try (if C++ devs wanted to, and I > can't speak for them), but it might be time-consuming (or worse) to > figure out what features are supported on all of the platforms and > languages that Arrow C++ intends to work for. That said, hopefully our > CI coverage is sufficient to let someone try it out and see what > breaks. > > Neal > > On Fri, Oct 4, 2019 at 9:05 AM Zhuo Peng wrote: >> >> Dear Arrow maintainers, >> >> Sorry if this was raised before. I did search the mailing list but "C++" >> matched too many results.. >> >> With manylinux1 (GCC4.8) being sunset, both Conda and Pypa are providing a >> modern enough toolchain (Conda Forge - GCC7; Pypa manylinux2010 docker - >> devtoolset-8(GCC8)). And full C++17 support has been included in GCC7 [1]. I >> wonder what are the concerns of adopting a newer standard? >> >> C++14 might not bring a whole lot of interesting features, but C++17 brings: >> >> std::string_view >> std::optional >> std::variant (the newly added Result class is based on some form of variant >> implementation I suppose?) >> >> and many syntax sugar.. (like emplace_back() returning back(), so you can do >> RETURN_NOT_OK(CreateArray(my_array_sp_vector.emplace_back( >> >> And btw, was -std=gnu++11 an intentional choice? what gnu extensions does >> the library rely on? >> >> [1] https://gcc.gnu.org/projects/cxx-status.html >>
Re: Should Arrow adopt C++14 / 17?
On 2019/10/04 16:53:59, Antoine Pitrou wrote: > > Le 04/10/2019 à 18:05, Zhuo Peng a écrit : > > Dear Arrow maintainers, > > > > Sorry if this was raised before. I did search the mailing list but "C++" > > matched too many results.. > > > > With manylinux1 (GCC4.8) being sunset, both Conda and Pypa are providing a > > modern enough toolchain (Conda Forge - GCC7; Pypa manylinux2010 docker - > > devtoolset-8(GCC8)). And full C++17 support has been included in GCC7 [1]. > > I wonder what are the concerns of adopting a newer standard? > > > > C++14 might not bring a whole lot of interesting features, but C++17 brings: > > > > std::string_view > > std::optional > > std::variant (the newly added Result class is based on some form of variant > > implementation I suppose?) > > We already have `string_view` and `variant` backports. We could > reasonably add a `optional` backport. > backports are cool for internal use, but probably not so if a public API accepts it? (because you vendor the headers in (i.e. namespace, symbol names unchanged), they might clash with headers that a client uses). > > And btw, was -std=gnu++11 an intentional choice? what gnu extensions does > > the library rely on? > > None, AFAIK. Arrow compiles on MSVC fine. Where is -std=gnu++11 added? https://github.com/apache/arrow/blob/3129e3ed90219ecfffe2a25ce5820eec8cc947d0/cpp/cmake_modules/SetupCxxFlags.cmake#L33 https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_STANDARD.html > > Regards > > Antoine. >
Re: Should Arrow adopt C++14 / 17?
Le 04/10/2019 à 18:05, Zhuo Peng a écrit : > Dear Arrow maintainers, > > Sorry if this was raised before. I did search the mailing list but "C++" > matched too many results.. > > With manylinux1 (GCC4.8) being sunset, both Conda and Pypa are providing a > modern enough toolchain (Conda Forge - GCC7; Pypa manylinux2010 docker - > devtoolset-8(GCC8)). And full C++17 support has been included in GCC7 [1]. I > wonder what are the concerns of adopting a newer standard? > > C++14 might not bring a whole lot of interesting features, but C++17 brings: > > std::string_view > std::optional > std::variant (the newly added Result class is based on some form of variant > implementation I suppose?) We already have `string_view` and `variant` backports. We could reasonably add a `optional` backport. > And btw, was -std=gnu++11 an intentional choice? what gnu extensions does the > library rely on? None, AFAIK. Arrow compiles on MSVC fine. Where is -std=gnu++11 added? Regards Antoine.
Re: Should Arrow adopt C++14 / 17?
We do have to care about more than just conda and Python. For R, for example, C++14 support is limited: https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Using-C_002b_002b14-code That's not to say that we can't try (if C++ devs wanted to, and I can't speak for them), but it might be time-consuming (or worse) to figure out what features are supported on all of the platforms and languages that Arrow C++ intends to work for. That said, hopefully our CI coverage is sufficient to let someone try it out and see what breaks. Neal On Fri, Oct 4, 2019 at 9:05 AM Zhuo Peng wrote: > > Dear Arrow maintainers, > > Sorry if this was raised before. I did search the mailing list but "C++" > matched too many results.. > > With manylinux1 (GCC4.8) being sunset, both Conda and Pypa are providing a > modern enough toolchain (Conda Forge - GCC7; Pypa manylinux2010 docker - > devtoolset-8(GCC8)). And full C++17 support has been included in GCC7 [1]. I > wonder what are the concerns of adopting a newer standard? > > C++14 might not bring a whole lot of interesting features, but C++17 brings: > > std::string_view > std::optional > std::variant (the newly added Result class is based on some form of variant > implementation I suppose?) > > and many syntax sugar.. (like emplace_back() returning back(), so you can do > RETURN_NOT_OK(CreateArray(my_array_sp_vector.emplace_back( > > And btw, was -std=gnu++11 an intentional choice? what gnu extensions does the > library rely on? > > [1] https://gcc.gnu.org/projects/cxx-status.html >
Should Arrow adopt C++14 / 17?
Dear Arrow maintainers, Sorry if this was raised before. I did search the mailing list but "C++" matched too many results.. With manylinux1 (GCC4.8) being sunset, both Conda and Pypa are providing a modern enough toolchain (Conda Forge - GCC7; Pypa manylinux2010 docker - devtoolset-8(GCC8)). And full C++17 support has been included in GCC7 [1]. I wonder what are the concerns of adopting a newer standard? C++14 might not bring a whole lot of interesting features, but C++17 brings: std::string_view std::optional std::variant (the newly added Result class is based on some form of variant implementation I suppose?) and many syntax sugar.. (like emplace_back() returning back(), so you can do RETURN_NOT_OK(CreateArray(my_array_sp_vector.emplace_back( And btw, was -std=gnu++11 an intentional choice? what gnu extensions does the library rely on? [1] https://gcc.gnu.org/projects/cxx-status.html