Re: Should Arrow adopt C++14 / 17?
I opened https://issues.apache.org/jira/browse/ARROW-6800.
On Fri, Oct 4, 2019 at 3:42 PM Wes McKinney wrote:
>
> On Fri, Oct 4, 2019 at 3:18 PM Zhuo Peng wrote:
> >
> >
> >
> > On 2019/10/04 19:43:04, Wes McKinney wrote:
> > > On Fri, Oct 4, 2019 at 12:45 PM Zhuo Peng wrote:
> > > >
> > > >
> > > >
> > > > On 2019/10/04 17:05:00, Antoine Pitrou wrote:
> > > > >
> > > > > Le 04/10/2019 à 19:01, Zhuo Peng a écrit :
> > > > > >
> > > > > > backports are cool for internal use, but probably not so if a
> > > > > > public API accepts it? (because you vendor the headers in (i.e.
> > > > > > namespace, symbol names unchanged), they might clash with headers
> > > > > > that a client uses).
> > > > >
> > > > > This is true unfortunately.
> > > > >
> > > > > >>> And btw, was -std=gnu++11 an intentional choice? what gnu
> > > > > >>> extensions does the library rely on?
> > > > > >>
> > > > > >> None, AFAIK. Arrow compiles on MSVC fine. Where is -std=gnu++11
> > > > > >> added?
> > > > > > https://github.com/apache/arrow/blob/3129e3ed90219ecfffe2a25ce5820eec8cc947d0/cpp/cmake_modules/SetupCxxFlags.cmake#L33
> > > > > >
> > > > > > https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_STANDARD.html
> > > > >
> > > > > Right, so this is a CMake decision. I think we require only plain
> > > > > C++11
> > > > > (but we may enable additional features on some compilers, provided
> > > > > there's a fallback).
> > > > Extensions can be disabled through:
> > > > set(CMAKE_CXX_EXTENSIONS OFF)
> > > >
> > > > https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_EXTENSIONS.html
> > > >
> > > > Is that something more desirable than the current state?
> > >
> > > Yes, I think so, I don't think we need to be relying on GNU gcc
> > > extensions, but we should open a JIRA issue about disabling it in case
> > > some tests break because of something we didn't realize we were
> > > depending on.
> > sg. I'll create one then.
> > >
> > > As far as C++14/17 upgrading, it seems like it will be at least 2
> > > years before we could upgrade to C++17 given the state of compiler
> > > support across the spectrum. Using C++17 would mean requiring at least
> > > VS 2017 on Windows, since at least in the Python world I think
> > > everything is on VS 2015.
> > >
> > > Are there ways we could create defines to switch between backports and
> > > STL things (like string_view, optional, etc.) so that developers using
> > > the Arrow library in a C++17 application can use the built-in types?
> > This is dangerous unless they build the Arrow library from source with
> > C++17. if libarrow takes arrow::string_view but the user gives it a
> > std::string_view, it's UB.
> >
> > If we are talking about allowing users to build Arrow with C++17 and
> > support transparently the new STL types in the public APIs, the ABSL[1]
> > library could be something to consider..
> > absl::{string_view,optional,variant} becomes their std:: counterparts when
> > compiled under C++17, e.g. [2].
> >
>
> Yes, the presumption would be a monotoolchain environment, and Arrow
> would need to have some CMake options to build in C++17 mode
>
> > And inline namespaces are used [3] to make sure different libraries can
> > depend on different version of absl.
> >
> > [1] https://abseil.io/
> > [2]
> > https://github.com/abseil/abseil-cpp/blob/25597bdfc148e91e27678ec30efa52f4fc8c164f/absl/strings/string_view.h#L38
> > [3]
> > https://github.com/abseil/abseil-cpp/blob/aa844899c937bde5d2b24f276b59997e5b668bde/absl/strings/string_view.h#L38
> > >
> > > > >
> > > > > Regards
> > > > >
> > > > > Antoine.
> > > > >
> > >
Re: Should Arrow adopt C++14 / 17?
On Fri, Oct 4, 2019 at 3:18 PM Zhuo Peng wrote:
>
>
>
> On 2019/10/04 19:43:04, Wes McKinney wrote:
> > On Fri, Oct 4, 2019 at 12:45 PM Zhuo Peng wrote:
> > >
> > >
> > >
> > > On 2019/10/04 17:05:00, Antoine Pitrou wrote:
> > > >
> > > > Le 04/10/2019 à 19:01, Zhuo Peng a écrit :
> > > > >
> > > > > backports are cool for internal use, but probably not so if a public
> > > > > API accepts it? (because you vendor the headers in (i.e. namespace,
> > > > > symbol names unchanged), they might clash with headers that a client
> > > > > uses).
> > > >
> > > > This is true unfortunately.
> > > >
> > > > >>> And btw, was -std=gnu++11 an intentional choice? what gnu
> > > > >>> extensions does the library rely on?
> > > > >>
> > > > >> None, AFAIK. Arrow compiles on MSVC fine. Where is -std=gnu++11
> > > > >> added?
> > > > > https://github.com/apache/arrow/blob/3129e3ed90219ecfffe2a25ce5820eec8cc947d0/cpp/cmake_modules/SetupCxxFlags.cmake#L33
> > > > >
> > > > > https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_STANDARD.html
> > > >
> > > > Right, so this is a CMake decision. I think we require only plain C++11
> > > > (but we may enable additional features on some compilers, provided
> > > > there's a fallback).
> > > Extensions can be disabled through:
> > > set(CMAKE_CXX_EXTENSIONS OFF)
> > >
> > > https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_EXTENSIONS.html
> > >
> > > Is that something more desirable than the current state?
> >
> > Yes, I think so, I don't think we need to be relying on GNU gcc
> > extensions, but we should open a JIRA issue about disabling it in case
> > some tests break because of something we didn't realize we were
> > depending on.
> sg. I'll create one then.
> >
> > As far as C++14/17 upgrading, it seems like it will be at least 2
> > years before we could upgrade to C++17 given the state of compiler
> > support across the spectrum. Using C++17 would mean requiring at least
> > VS 2017 on Windows, since at least in the Python world I think
> > everything is on VS 2015.
> >
> > Are there ways we could create defines to switch between backports and
> > STL things (like string_view, optional, etc.) so that developers using
> > the Arrow library in a C++17 application can use the built-in types?
> This is dangerous unless they build the Arrow library from source with C++17.
> if libarrow takes arrow::string_view but the user gives it a
> std::string_view, it's UB.
>
> If we are talking about allowing users to build Arrow with C++17 and support
> transparently the new STL types in the public APIs, the ABSL[1] library could
> be something to consider.. absl::{string_view,optional,variant} becomes their
> std:: counterparts when compiled under C++17, e.g. [2].
>
Yes, the presumption would be a monotoolchain environment, and Arrow
would need to have some CMake options to build in C++17 mode
> And inline namespaces are used [3] to make sure different libraries can
> depend on different version of absl.
>
> [1] https://abseil.io/
> [2]
> https://github.com/abseil/abseil-cpp/blob/25597bdfc148e91e27678ec30efa52f4fc8c164f/absl/strings/string_view.h#L38
> [3]
> https://github.com/abseil/abseil-cpp/blob/aa844899c937bde5d2b24f276b59997e5b668bde/absl/strings/string_view.h#L38
> >
> > > >
> > > > Regards
> > > >
> > > > Antoine.
> > > >
> >
Re: Should Arrow adopt C++14 / 17?
On 2019/10/04 19:43:04, Wes McKinney wrote:
> On Fri, Oct 4, 2019 at 12:45 PM Zhuo Peng wrote:
> >
> >
> >
> > On 2019/10/04 17:05:00, Antoine Pitrou wrote:
> > >
> > > Le 04/10/2019 à 19:01, Zhuo Peng a écrit :
> > > >
> > > > backports are cool for internal use, but probably not so if a public
> > > > API accepts it? (because you vendor the headers in (i.e. namespace,
> > > > symbol names unchanged), they might clash with headers that a client
> > > > uses).
> > >
> > > This is true unfortunately.
> > >
> > > >>> And btw, was -std=gnu++11 an intentional choice? what gnu extensions
> > > >>> does the library rely on?
> > > >>
> > > >> None, AFAIK. Arrow compiles on MSVC fine. Where is -std=gnu++11
> > > >> added?
> > > > https://github.com/apache/arrow/blob/3129e3ed90219ecfffe2a25ce5820eec8cc947d0/cpp/cmake_modules/SetupCxxFlags.cmake#L33
> > > >
> > > > https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_STANDARD.html
> > >
> > > Right, so this is a CMake decision. I think we require only plain C++11
> > > (but we may enable additional features on some compilers, provided
> > > there's a fallback).
> > Extensions can be disabled through:
> > set(CMAKE_CXX_EXTENSIONS OFF)
> >
> > https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_EXTENSIONS.html
> >
> > Is that something more desirable than the current state?
>
> Yes, I think so, I don't think we need to be relying on GNU gcc
> extensions, but we should open a JIRA issue about disabling it in case
> some tests break because of something we didn't realize we were
> depending on.
sg. I'll create one then.
>
> As far as C++14/17 upgrading, it seems like it will be at least 2
> years before we could upgrade to C++17 given the state of compiler
> support across the spectrum. Using C++17 would mean requiring at least
> VS 2017 on Windows, since at least in the Python world I think
> everything is on VS 2015.
>
> Are there ways we could create defines to switch between backports and
> STL things (like string_view, optional, etc.) so that developers using
> the Arrow library in a C++17 application can use the built-in types?
This is dangerous unless they build the Arrow library from source with C++17.
if libarrow takes arrow::string_view but the user gives it a std::string_view,
it's UB.
If we are talking about allowing users to build Arrow with C++17 and support
transparently the new STL types in the public APIs, the ABSL[1] library could
be something to consider.. absl::{string_view,optional,variant} becomes their
std:: counterparts when compiled under C++17, e.g. [2].
And inline namespaces are used [3] to make sure different libraries can depend
on different version of absl.
[1] https://abseil.io/
[2]
https://github.com/abseil/abseil-cpp/blob/25597bdfc148e91e27678ec30efa52f4fc8c164f/absl/strings/string_view.h#L38
[3]
https://github.com/abseil/abseil-cpp/blob/aa844899c937bde5d2b24f276b59997e5b668bde/absl/strings/string_view.h#L38
>
> > >
> > > Regards
> > >
> > > Antoine.
> > >
>
Re: Should Arrow adopt C++14 / 17?
On 2019/10/04 17:05:00, Antoine Pitrou wrote: > > Le 04/10/2019 à 19:01, Zhuo Peng a écrit : > > > > backports are cool for internal use, but probably not so if a public API > > accepts it? (because you vendor the headers in (i.e. namespace, symbol > > names unchanged), they might clash with headers that a client uses). > > This is true unfortunately. > > >>> And btw, was -std=gnu++11 an intentional choice? what gnu extensions does > >>> the library rely on? > >> > >> None, AFAIK. Arrow compiles on MSVC fine. Where is -std=gnu++11 added? > > https://github.com/apache/arrow/blob/3129e3ed90219ecfffe2a25ce5820eec8cc947d0/cpp/cmake_modules/SetupCxxFlags.cmake#L33 > > > > https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_STANDARD.html > > Right, so this is a CMake decision. I think we require only plain C++11 > (but we may enable additional features on some compilers, provided > there's a fallback). Extensions can be disabled through: set(CMAKE_CXX_EXTENSIONS OFF) https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_EXTENSIONS.html Is that something more desirable than the current state? > > Regards > > Antoine. >
Re: Should Arrow adopt C++14 / 17?
Le 04/10/2019 à 19:01, Zhuo Peng a écrit : > > backports are cool for internal use, but probably not so if a public API > accepts it? (because you vendor the headers in (i.e. namespace, symbol names > unchanged), they might clash with headers that a client uses). This is true unfortunately. >>> And btw, was -std=gnu++11 an intentional choice? what gnu extensions does >>> the library rely on? >> >> None, AFAIK. Arrow compiles on MSVC fine. Where is -std=gnu++11 added? > https://github.com/apache/arrow/blob/3129e3ed90219ecfffe2a25ce5820eec8cc947d0/cpp/cmake_modules/SetupCxxFlags.cmake#L33 > > https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_STANDARD.html Right, so this is a CMake decision. I think we require only plain C++11 (but we may enable additional features on some compilers, provided there's a fallback). Regards Antoine.
Re: Should Arrow adopt C++14 / 17?
C++14 isn't very interesting. C++17 is, but it's probably too young given the diversity of platform and toolchain requirements that constrai us. Regards Antoine. Le 04/10/2019 à 18:13, Neal Richardson a écrit : > We do have to care about more than just conda and Python. For R, for > example, C++14 support is limited: > https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Using-C_002b_002b14-code > > That's not to say that we can't try (if C++ devs wanted to, and I > can't speak for them), but it might be time-consuming (or worse) to > figure out what features are supported on all of the platforms and > languages that Arrow C++ intends to work for. That said, hopefully our > CI coverage is sufficient to let someone try it out and see what > breaks. > > Neal > > On Fri, Oct 4, 2019 at 9:05 AM Zhuo Peng wrote: >> >> Dear Arrow maintainers, >> >> Sorry if this was raised before. I did search the mailing list but "C++" >> matched too many results.. >> >> With manylinux1 (GCC4.8) being sunset, both Conda and Pypa are providing a >> modern enough toolchain (Conda Forge - GCC7; Pypa manylinux2010 docker - >> devtoolset-8(GCC8)). And full C++17 support has been included in GCC7 [1]. I >> wonder what are the concerns of adopting a newer standard? >> >> C++14 might not bring a whole lot of interesting features, but C++17 brings: >> >> std::string_view >> std::optional >> std::variant (the newly added Result class is based on some form of variant >> implementation I suppose?) >> >> and many syntax sugar.. (like emplace_back() returning back(), so you can do >> RETURN_NOT_OK(CreateArray(my_array_sp_vector.emplace_back( >> >> And btw, was -std=gnu++11 an intentional choice? what gnu extensions does >> the library rely on? >> >> [1] https://gcc.gnu.org/projects/cxx-status.html >>
Re: Should Arrow adopt C++14 / 17?
On 2019/10/04 16:53:59, Antoine Pitrou wrote: > > Le 04/10/2019 à 18:05, Zhuo Peng a écrit : > > Dear Arrow maintainers, > > > > Sorry if this was raised before. I did search the mailing list but "C++" > > matched too many results.. > > > > With manylinux1 (GCC4.8) being sunset, both Conda and Pypa are providing a > > modern enough toolchain (Conda Forge - GCC7; Pypa manylinux2010 docker - > > devtoolset-8(GCC8)). And full C++17 support has been included in GCC7 [1]. > > I wonder what are the concerns of adopting a newer standard? > > > > C++14 might not bring a whole lot of interesting features, but C++17 brings: > > > > std::string_view > > std::optional > > std::variant (the newly added Result class is based on some form of variant > > implementation I suppose?) > > We already have `string_view` and `variant` backports. We could > reasonably add a `optional` backport. > backports are cool for internal use, but probably not so if a public API accepts it? (because you vendor the headers in (i.e. namespace, symbol names unchanged), they might clash with headers that a client uses). > > And btw, was -std=gnu++11 an intentional choice? what gnu extensions does > > the library rely on? > > None, AFAIK. Arrow compiles on MSVC fine. Where is -std=gnu++11 added? https://github.com/apache/arrow/blob/3129e3ed90219ecfffe2a25ce5820eec8cc947d0/cpp/cmake_modules/SetupCxxFlags.cmake#L33 https://cmake.org/cmake/help/v3.1/prop_tgt/CXX_STANDARD.html > > Regards > > Antoine. >
Re: Should Arrow adopt C++14 / 17?
Le 04/10/2019 à 18:05, Zhuo Peng a écrit : > Dear Arrow maintainers, > > Sorry if this was raised before. I did search the mailing list but "C++" > matched too many results.. > > With manylinux1 (GCC4.8) being sunset, both Conda and Pypa are providing a > modern enough toolchain (Conda Forge - GCC7; Pypa manylinux2010 docker - > devtoolset-8(GCC8)). And full C++17 support has been included in GCC7 [1]. I > wonder what are the concerns of adopting a newer standard? > > C++14 might not bring a whole lot of interesting features, but C++17 brings: > > std::string_view > std::optional > std::variant (the newly added Result class is based on some form of variant > implementation I suppose?) We already have `string_view` and `variant` backports. We could reasonably add a `optional` backport. > And btw, was -std=gnu++11 an intentional choice? what gnu extensions does the > library rely on? None, AFAIK. Arrow compiles on MSVC fine. Where is -std=gnu++11 added? Regards Antoine.
Re: Should Arrow adopt C++14 / 17?
We do have to care about more than just conda and Python. For R, for example, C++14 support is limited: https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Using-C_002b_002b14-code That's not to say that we can't try (if C++ devs wanted to, and I can't speak for them), but it might be time-consuming (or worse) to figure out what features are supported on all of the platforms and languages that Arrow C++ intends to work for. That said, hopefully our CI coverage is sufficient to let someone try it out and see what breaks. Neal On Fri, Oct 4, 2019 at 9:05 AM Zhuo Peng wrote: > > Dear Arrow maintainers, > > Sorry if this was raised before. I did search the mailing list but "C++" > matched too many results.. > > With manylinux1 (GCC4.8) being sunset, both Conda and Pypa are providing a > modern enough toolchain (Conda Forge - GCC7; Pypa manylinux2010 docker - > devtoolset-8(GCC8)). And full C++17 support has been included in GCC7 [1]. I > wonder what are the concerns of adopting a newer standard? > > C++14 might not bring a whole lot of interesting features, but C++17 brings: > > std::string_view > std::optional > std::variant (the newly added Result class is based on some form of variant > implementation I suppose?) > > and many syntax sugar.. (like emplace_back() returning back(), so you can do > RETURN_NOT_OK(CreateArray(my_array_sp_vector.emplace_back( > > And btw, was -std=gnu++11 an intentional choice? what gnu extensions does the > library rely on? > > [1] https://gcc.gnu.org/projects/cxx-status.html >
