[jira] [Commented] (ARROW-17580) [Doc][C++][Python] Unclear how to influence compilation flags
[
https://issues.apache.org/jira/browse/ARROW-17580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601724#comment-17601724
]
Alenka Frim commented on ARROW-17580:
-
Happy it works. I suggest we close this issue and I will do a PR with some
corrections for the compilation flags for PyArrow in
https://issues.apache.org/jira/browse/ARROW-17579.
> [Doc][C++][Python] Unclear how to influence compilation flags
> -
>
> Key: ARROW-17580
> URL: https://issues.apache.org/jira/browse/ARROW-17580
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Documentation, Python
>Reporter: Antoine Pitrou
>Priority: Critical
>
> Frequently people need to customize compilation flags for C++ and/or C files.
> Unfortunately, both for Arrow C++ and PyArrow, it is very difficult to find
> out the proper way to do this.
> For Arrow C++, it seems {{ARROW_CXXFLAGS}} should be passed to CMake, while
> the {{CXXFLAGS}} environment variable is ignored (it probably shouldn't?).
> For PyArrow, I have not found a way to do it. -The {{CXXFLAGS}} environment
> variable is ignored, and -the {{PYARROW_CXXFLAGS}} CMake variable has two
> problems:
> * it is only recognized for Cython-generated files, not for PyArrow C++
> sources
> * it only affects linker calls, while it should actually affect compiler
> calls (edit: does it?)
> *Edit: I was wrong, the {{CXXFLAGS}} environment variable is not ignored.*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17580) [Doc][C++][Python] Unclear how to influence compilation flags
[
https://issues.apache.org/jira/browse/ARROW-17580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601714#comment-17601714
]
Antoine Pitrou commented on ARROW-17580:
Yes, it appears I had misinterpreted what I saw...
> [Doc][C++][Python] Unclear how to influence compilation flags
> -
>
> Key: ARROW-17580
> URL: https://issues.apache.org/jira/browse/ARROW-17580
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Documentation, Python
>Reporter: Antoine Pitrou
>Priority: Critical
>
> Frequently people need to customize compilation flags for C++ and/or C files.
> Unfortunately, both for Arrow C++ and PyArrow, it is very difficult to find
> out the proper way to do this.
> For Arrow C++, it seems {{ARROW_CXXFLAGS}} should be passed to CMake, while
> the {{CXXFLAGS}} environment variable is ignored (it probably shouldn't?).
> For PyArrow, I have not found a way to do it. -The {{CXXFLAGS}} environment
> variable is ignored, and -the {{PYARROW_CXXFLAGS}} CMake variable has two
> problems:
> * it is only recognized for Cython-generated files, not for PyArrow C++
> sources
> * it only affects linker calls, while it should actually affect compiler
> calls (edit: does it?)
> *Edit: I was wrong, the {{CXXFLAGS}} environment variable is not ignored.*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17580) [Doc][C++][Python] Unclear how to influence compilation flags
[
https://issues.apache.org/jira/browse/ARROW-17580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601712#comment-17601712
]
Alenka Frim commented on ARROW-17580:
-
_Update about the {{{}CXXFLAGS{}}}:_
CMake does in fact read them. I have tried again with a clean built. I have set
{{CXXFLAGS}} env var to a random value and the built failed. See:
{code:java}
(pyarrow-dev-9) (base) alenkafrim@Alenkas-MacBook-Pro arrow % printenv CXXFLAGS
-Woverloaded-virtual ../dist/configure{code}
{code:java}
(pyarrow-dev-9) (base) alenkafrim@Alenkas-MacBook-Pro arrow % arrow cmake-debug
ARROW_BUILD_TYPE=debug
~/repos/arrow/cpp/build ~/repos/arrow
-- Building using CMake version: 3.24.1
-- The C compiler identification is AppleClang 13.1.6.13160021
-- The CXX compiler identification is AppleClang 13.1.6.13160021
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
- skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - failed
-- Check for working CXX compiler:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
- broken
CMake Error at
/opt/homebrew/Cellar/cmake/3.24.1/share/cmake/Modules/CMakeTestCXXCompiler.cmake:62
(message):
The C++ compiler
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++"
is not able to compile a simple test program.
It fails with the following output:
Change Dir: /Users/alenkafrim/repos/arrow/cpp/build/CMakeFiles/CMakeTmp
Run Build Command(s):/opt/homebrew/bin/ninja cmTC_3527f && [1/2] Building
CXX object CMakeFiles/cmTC_3527f.dir/testCXXCompiler.cxx.o
FAILED: CMakeFiles/cmTC_3527f.dir/testCXXCompiler.cxx.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-Woverloaded-virtual ../dist/configure -arch arm64 -isysroot
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk
-MD -MT CMakeFiles/cmTC_3527f.dir/testCXXCompiler.cxx.o -MF
CMakeFiles/cmTC_3527f.dir/testCXXCompiler.cxx.o.d -o
CMakeFiles/cmTC_3527f.dir/testCXXCompiler.cxx.o -c
/Users/alenkafrim/repos/arrow/cpp/build/CMakeFiles/CMakeTmp/testCXXCompiler.cxx
clang: error: no such file or directory: '../dist/configure'
ninja: build stopped: subcommand failed.
{code}
If I change the value of {{CXXFLAGS to -O0 it gets added to the CMAKE_CXX_FLAGS
at the beginning of the list:}}
{code:java}
...
-- **
-- ARROW_CXXFLAGS: -ggdb
-- PYARROW_CXXFLAGS:
-- CXXFLAGS:
-- CXX_COMMON_FLAGS: -Wall -Wextra -Wdocumentation -Wshorten-64-to-32
-Wno-missing-braces -Wno-unused-parameter -Wno-constant-logical-operand
-Wno-return-stack-address -Werror -Wno-unknown-warning-option -Wno-pass-failed
-stdlib=libc++ -march=armv8-a
-- CMAKE_CXX_FLAGS: -O0 -Qunused-arguments -fcolor-diagnostics -ggdb -O0
-Wall -Wextra -Wdocumentation -Wshorten-64-to-32 -Wno-missing-braces
-Wno-unused-parameter -Wno-constant-logical-operand -Wno-return-stack-address
-Werror -Wno-unknown-warning-option -Wno-pass-failed -stdlib=libc++
-march=armv8-a -ggdb
-- **
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/alenkafrim/repos/arrow/cpp/build
~/repos/arrow
(pyarrow-dev-9) (base) alenkafrim@Alenkas-MacBook-Pro arrow % printenv CXXFLAGS
-O0{code}
> [Doc][C++][Python] Unclear how to influence compilation flags
> -
>
> Key: ARROW-17580
> URL: https://issues.apache.org/jira/browse/ARROW-17580
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Documentation, Python
>Reporter: Antoine Pitrou
>Priority: Critical
>
> Frequently people need to customize compilation flags for C++ and/or C files.
> Unfortunately, both for Arrow C++ and PyArrow, it is very difficult to find
> out the proper way to do this.
> For Arrow C++, it seems {{ARROW_CXXFLAGS}} should be passed to CMake, while
> the {{CXXFLAGS}} environment variable is ignored (it probably shouldn't?).
> For PyArrow, I have not found a way to do it. -The {{CXXFLAGS}} environment
> variable is ignored, and -the {{PYARROW_CXXFLAGS}} CMake variable has two
> problems:
> * it is only recognized for Cython-generated files, not for PyArrow C++
> sources
> * it only affects linker calls, while it should actually affect compiler
> calls (edit: does it?)
> *Edit: I was wrong, the {{CXXFLAGS}} environment variable is not
[jira] [Commented] (ARROW-17580) [Doc][C++][Python] Unclear how to influence compilation flags
[
https://issues.apache.org/jira/browse/ARROW-17580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599308#comment-17599308
]
Alenka Frim commented on ARROW-17580:
-
I did some research and found:
- {{CMAKE_CXX_FLAGS}} in pyarrow include {{CXX_COMMON_FLAGS}} but this is not
true for pyarrow c++ *->* I need to add {{set(CMAKE_CXX_FLAGS
"${CXX_COMMON_FLAGS} ${CMAKE_CXX_FLAGS}")}} to pyarrow c++ CMakeLists file
- {{PYARROW_CXXFLAGS}} is ignored by pyarrow c++ and pyarrow *->* I need to
add it to the cmake_options in setup.py, then they are read by the cmake
- {{ARROW_CXXFLAGS}} are recognised by all three builds
{code:bash}
arrow c++
-- ARROW_CXXFLAGS: -ggdb
-- PYARROW_CXXFLAGS:
-- CXXFLAGS:
-- CXX_COMMON_FLAGS: -Wall -Wextra -Wdocumentation -Wshorten-64-to-32
-Wno-missing-braces -Wno-unused-parameter -Wno-constant-logical-operand
-Wno-return-stack-address -Werror -Wno-unknown-warning-option -Wno-pass-failed
-stdlib=libc++ -march=armv8-a
--CMAKE_CXX_FLAGS: -Qunused-arguments -fcolor-diagnostics -ggdb -O0 -Wall
-Wextra -Wdocumentation -Wshorten-64-to-32 -Wno-missing-braces
-Wno-unused-parameter -Wno-constant-logical-operand -Wno-return-stack-address
-Werror -Wno-unknown-warning-option -Wno-pass-failed -stdlib=libc++
-march=armv8-a -ggdb
pyarrow c++
-- ARROW_CXXFLAGS: -ggdb
-- PYARROW_CXXFLAGS:
-- CXXFLAGS:
-- CXX_COMMON_FLAGS: -Wall -Wextra -Wdocumentation -Wshorten-64-to-32
-Wno-missing-braces -Wno-unused-parameter -Wno-constant-logical-operand
-Wno-return-stack-address -Werror -Wno-unknown-warning-option -Wno-pass-failed
-stdlib=libc++ -march=armv8-a
--CMAKE_CXX_FLAGS: -Qunused-arguments -fcolor-diagnostics -ggdb -O0
pyarrow
-- ARROW_CXXFLAGS: -ggdb
-- PYARROW_CXXFLAGS:
-- CXXFLAGS:
-- CXX_COMMON_FLAGS: -Wall -Wno-unknown-warning-option -Wno-pass-failed
-stdlib=libc++ -march=armv8-a
-- CMAKE_CXX_FLAGS: -Wall -Wno-unknown-warning-option -Wno-pass-failed
-stdlib=libc++ -march=armv8-a -Qunused-arguments -fcolor-diagnostics -g -O0
-fno-omit-frame-pointer -Wno-unused-variable -Wno-maybe-uninitialized
-Wno-parentheses-equality -Wno-constant-logical-operand
-Wno-missing-declarations -Wno-sometimes-uninitialized
-Wno-return-type-c-linkage
{code}
- I think {{CXXFLAGS}} doesn't get included in the {{CMAKE_CXX_FLAGS}} list.
Maybe even setting {{CMAKE_CXX_FLAGS}} in
[https://github.com/apache/arrow/blob/413e49385f52b8c76aa02f67588f4dc66d5ed86b/cpp/CMakeLists.txt#L555]
makes the variable to be ignnored?
{code:bash}
(pyarrow-dev-9) (base) alenkafrim@Alenkas-MacBook-Pro arrow % printenv CXXFLAGS
-Woverloaded-virtual ../dist/configure
(pyarrow-dev-9) (base) alenkafrim@Alenkas-MacBook-Pro arrow % arrow cmake-debug
...
-- ARROW_CXXFLAGS: -ggdb
-- PYARROW_CXXFLAGS:
-- CXXFLAGS:
-- CXX_COMMON_FLAGS: -Wall -Wextra -Wdocumentation -Wshorten-64-to-32
-Wno-missing-braces -Wno-unused-parameter -Wno-constant-logical-operand
-Wno-return-stack-address -Werror -Wno-unknown-warning-option -Wno-pass-failed
-stdlib=libc++ -march=armv8-a
-- CMAKE_CXX_FLAGS: -Qunused-arguments -fcolor-diagnostics -ggdb -O0 -Wall
-Wextra -Wdocumentation -Wshorten-64-to-32 -Wno-missing-braces
-Wno-unused-parameter -Wno-constant-logical-operand -Wno-return-stack-address
-Werror -Wno-unknown-warning-option -Wno-pass-failed -stdlib=libc++
-march=armv8-a -ggdb
-- **
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/alenkafrim/repos/arrow/cpp/build
~/repos/arrow
{code}
> [Doc][C++][Python] Unclear how to influence compilation flags
> -
>
> Key: ARROW-17580
> URL: https://issues.apache.org/jira/browse/ARROW-17580
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Documentation, Python
>Reporter: Antoine Pitrou
>Priority: Critical
>
> Frequently people need to customize compilation flags for C++ and/or C files.
> Unfortunately, both for Arrow C++ and PyArrow, it is very difficult to find
> out the proper way to do this.
> For Arrow C++, it seems {{ARROW_CXXFLAGS}} should be passed to CMake, while
> the {{CXXFLAGS}} environment variable is ignored (it probably shouldn't?).
> For PyArrow, I have not found a way to do it. -The {{CXXFLAGS}} environment
> variable is ignored, and -the {{PYARROW_CXXFLAGS}} CMake variable has two
> problems:
> * it is only recognized for Cython-generated files, not for PyArrow C++
> sources
> * it only affects linker calls, while it should actually affect compiler
> calls (edit: does it?)
> *Edit: I was wrong, the {{CXXFLAGS}} environment variable is not ignored.*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17580) [Doc][C++][Python] Unclear how to influence compilation flags
[
https://issues.apache.org/jira/browse/ARROW-17580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599051#comment-17599051
]
Antoine Pitrou commented on ARROW-17580:
Hmm, it seems I was wrong. Apparently the CXXFLAGS environment variable is not
ignored, at least in my local tests. I'm a bit puzzled...
> [Doc][C++][Python] Unclear how to influence compilation flags
> -
>
> Key: ARROW-17580
> URL: https://issues.apache.org/jira/browse/ARROW-17580
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Documentation, Python
>Reporter: Antoine Pitrou
>Priority: Critical
>
> Frequently people need to customize compilation flags for C++ and/or C files.
> Unfortunately, both for Arrow C++ and PyArrow, it is very difficult to find
> out the proper way to do this.
> For Arrow C++, it seems {{ARROW_CXXFLAGS}} should be passed to CMake, while
> the {{CXXFLAGS}} environment variable is ignored (it probably shouldn't?).
> For PyArrow, I have not found a way to do it. The {{CXXFLAGS}} environment
> variable is ignored, and the {{PYARROW_CXXFLAGS}} CMake variable has two
> problems:
> * it is only recognized for Cython-generated files, not for PyArrow C++
> sources
> * it only affects linker calls, while it should actually affect compiler calls
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17580) [Doc][C++][Python] Unclear how to influence compilation flags
[
https://issues.apache.org/jira/browse/ARROW-17580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599042#comment-17599042
]
Alenka Frim commented on ARROW-17580:
-
{quote}For PyArrow:
* We should unify {{python/CMakeLists.txt}} and
{{{}python/pyarrow/src/CMakeLists.txt{}}}.
* We need to look into why {{CXXFLAGS}} environment variable is ignored.{quote}
I will look into why CXXFLAGS env var is ignored.
I also agree that we should unify the CMakeLists files for PyArrow. I would
fist deal with the issues in the follow-up of the refactoring
https://issues.apache.org/jira/browse/ARROW-17122 and then try unifying the
CMakeLists. Would that work for you [~apitrou], if I make sure that the
compilation flags for C++ and/or C files are equal between the builds?
> [Doc][C++][Python] Unclear how to influence compilation flags
> -
>
> Key: ARROW-17580
> URL: https://issues.apache.org/jira/browse/ARROW-17580
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Documentation, Python
>Reporter: Antoine Pitrou
>Priority: Critical
>
> Frequently people need to customize compilation flags for C++ and/or C files.
> Unfortunately, both for Arrow C++ and PyArrow, it is very difficult to find
> out the proper way to do this.
> For Arrow C++, it seems {{ARROW_CXXFLAGS}} should be passed to CMake, while
> the {{CXXFLAGS}} environment variable is ignored (it probably shouldn't?).
> For PyArrow, I have not found a way to do it. The {{CXXFLAGS}} environment
> variable is ignored, and the {{PYARROW_CXXFLAGS}} CMake variable has two
> problems:
> * it is only recognized for Cython-generated files, not for PyArrow C++
> sources
> * it only affects linker calls, while it should actually affect compiler calls
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17580) [Doc][C++][Python] Unclear how to influence compilation flags
[
https://issues.apache.org/jira/browse/ARROW-17580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17598685#comment-17598685
]
Kouhei Sutou commented on ARROW-17580:
--
{quote}
For Arrow C++, it seems {{ARROW_CXXFLAGS}} should be passed to CMake, while the
{{CXXFLAGS}} environment variable is ignored (it probably shouldn't?).
{quote}
Really? {{CXXFLAGS}} environment variable should be respected because we don't
override {{CMAKE_CXX_FLAGS}} that is initialized with {{CXXFLAGS}} environment
variable.
See also:
* https://github.com/apache/arrow/blob/master/cpp/CMakeLists.txt#L555
* https://cmake.org/cmake/help/latest/envvar/CXXFLAGS.html
For PyArrow:
* We should unify {{python/CMakeLists.txt}} and
{{python/pyarrow/src/CMakeLists.txt}}.
* We need to look into why {{CXXFLAGS}} environment variable is ignored.
> [Doc][C++][Python] Unclear how to influence compilation flags
> -
>
> Key: ARROW-17580
> URL: https://issues.apache.org/jira/browse/ARROW-17580
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Documentation, Python
>Reporter: Antoine Pitrou
>Priority: Critical
>
> Frequently people need to customize compilation flags for C++ and/or C files.
> Unfortunately, both for Arrow C++ and PyArrow, it is very difficult to find
> out the proper way to do this.
> For Arrow C++, it seems {{ARROW_CXXFLAGS}} should be passed to CMake, while
> the {{CXXFLAGS}} environment variable is ignored (it probably shouldn't?).
> For PyArrow, I have not found a way to do it. The {{CXXFLAGS}} environment
> variable is ignored, and the {{PYARROW_CXXFLAGS}} CMake variable has two
> problems:
> * it is only recognized for Cython-generated files, not for PyArrow C++
> sources
> * it only affects linker calls, while it should actually affect compiler calls
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17580) [Doc][C++][Python] Unclear how to influence compilation flags
[
https://issues.apache.org/jira/browse/ARROW-17580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17598375#comment-17598375
]
Antoine Pitrou commented on ARROW-17580:
Another problem seems to be that PyArrow uses two independent CMakeLists files
that are not related to each other (meaning two different sets of CMake
invocations, with different possible options...).
> [Doc][C++][Python] Unclear how to influence compilation flags
> -
>
> Key: ARROW-17580
> URL: https://issues.apache.org/jira/browse/ARROW-17580
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Documentation, Python
>Reporter: Antoine Pitrou
>Priority: Critical
>
> Frequently people need to customize compilation flags for C++ and/or C files.
> Unfortunately, both for Arrow C++ and PyArrow, it is very difficult to find
> out the proper way to do this.
> For Arrow C++, it seems {{ARROW_CXXFLAGS}} should be passed to CMake, while
> the {{CXXFLAGS}} environment variable is ignored (it probably shouldn't?).
> For PyArrow, I have not found a way to do it. The {{CXXFLAGS}} environment
> variable is ignored, and the {{PYARROW_CXXFLAGS}} CMake variable has two
> problems:
> * it is only recognized for Cython-generated files, not for PyArrow C++
> sources
> * it only affects linker calls, while it should actually affect compiler calls
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17580) [Doc][C++][Python] Unclear how to influence compilation flags
[
https://issues.apache.org/jira/browse/ARROW-17580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17598372#comment-17598372
]
Antoine Pitrou commented on ARROW-17580:
cc [~alenka] [~kou]
> [Doc][C++][Python] Unclear how to influence compilation flags
> -
>
> Key: ARROW-17580
> URL: https://issues.apache.org/jira/browse/ARROW-17580
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Documentation, Python
>Reporter: Antoine Pitrou
>Priority: Critical
>
> Frequently people need to customize compilation flags for C++ and/or C files.
> Unfortunately, both for Arrow C++ and PyArrow, it is very difficult to find
> out the proper way to do this.
> For Arrow C++, it seems {{ARROW_CXXFLAGS}} should be passed to CMake, while
> the {{CXXFLAGS}} environment variable is ignored (it probably shouldn't?).
> For PyArrow, I have not found a way to do it. The {{CXXFLAGS}} environment is
> ignored, and the {{PYARROW_CXXFLAGS}} has two problems:
> * it is only recognized for Cython-generated files, not for PyArrow C++
> sources
> * it only affects linker calls, while it should actually affect compiler calls
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
