[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643582#comment-17643582
]
Arjan van der Velde commented on ARROW-17374:
-
Thanks!
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 8.0.1, 9.0.0
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Assignee: Kouhei Sutou
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 11.0.0
>
> Attachments: build-images.out, environment.yml
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642281#comment-17642281
]
Kouhei Sutou commented on ARROW-17374:
--
Thanks!
I could reproduce this problem.
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 8.0.1, 9.0.0
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: build-images.out, environment.yml
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642256#comment-17642256
]
Arjan van der Velde commented on ARROW-17374:
-
I tested this in a docker image and got the same error. Turned out 'which' UNIX
command was missing. After installing 'which' with yum, R starts up fine. I
hope that helps.
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 8.0.1, 9.0.0
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: build-images.out, environment.yml
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642209#comment-17642209
]
Kouhei Sutou commented on ARROW-17374:
--
I tried with {{centos:7}} image but I got the following error:
{noformat}
(testme) [root@f186ef90db02 /]# R -e 'install.packages("decor",
repos="http://cran.us.r-project.org;)'
R version 4.1.3 (2022-03-10) -- "One Push-Up"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: x86_64-conda-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
During startup - Warning messages:
1: package 'utils' in options("defaultPackages") was not found
2: package 'stats' in options("defaultPackages") was not found
> install.packages("decor", repos="http://cran.us.r-project.org;)
Error in install.packages("decor", repos = "http://cran.us.r-project.org;) :
could not find function "install.packages"
Execution halted
{noformat}
What is missing?
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 8.0.1, 9.0.0
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: build-images.out, environment.yml
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642172#comment-17642172
]
Arjan van der Velde commented on ARROW-17374:
-
Yes, this is on CentOS 7.
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 8.0.1, 9.0.0
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: build-images.out, environment.yml
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642170#comment-17642170
]
Kouhei Sutou commented on ARROW-17374:
--
Sorry. I forgot to try this.
Could you show your CentOS version? 7?
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 8.0.1, 9.0.0
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: build-images.out, environment.yml
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642042#comment-17642042
]
Arjan van der Velde commented on ARROW-17374:
-
[~kou] fyi
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 8.0.1, 9.0.0
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: build-images.out, environment.yml
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637494#comment-17637494
]
Arjan van der Velde commented on ARROW-17374:
-
I have a conda environment file. I haven't stripped it down to the minimum
requirements for arrow but it illustrates the issue. The issue happens on a
linux (centos) install and not on macOS where it seems to be using imported
libraries from brew.
[^environment.yml]
{code:java}
# Set up conda environment and activate
conda env create -n testme -f environment.yml
conda activate testme
# Build required 'decor' which doesn't exist in any conda repo
R -e 'install.packages("decor", repos="http://cran.us.r-project.org;)'
# Build arrow
mkdir -p /tmp/git && \
cd /tmp/git && git clone 'https://github.com/apache/arrow' arrow &&
cd /tmp/git/arrow/r && \
ARROW_S3=OFF LIBARROW_MINIMAL=FALSE ARROW_R_DEV=TRUE R CMD INSTALL . {code}
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 8.0.1, 9.0.0
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: build-images.out, environment.yml
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627330#comment-17627330
]
Kouhei Sutou commented on ARROW-17374:
--
Could you provide command lines to reproduce this with conda?
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 8.0.1, 9.0.0
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: build-images.out
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627216#comment-17627216
]
Arjan van der Velde commented on ARROW-17374:
-
I'm running into this issue while building arrow for R from a conda
environment. The issue seems to have been introduced by
https://issues.apache.org/jira/browse/ARROW-16999. Versions prior to that build
fine.
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 8.0.1, 9.0.0
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: build-images.out
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605546#comment-17605546
]
Vincent Nijs commented on ARROW-17374:
--
The code below works fine on x86_64 but fails with the errors below on aarch64
{{cp -a /usr/local/lib/R/etc/Makeconf /usr/local/lib/R/etc/Makeconf.bak;}}
{{sed -i 's/fpic/fPIC/g' /usr/local/lib/R/etc/Makeconf;}}
{{R -e "options(HTTPUserAgent = sprintf('R/%s R (%s)', getRversion(),
paste(getRversion(), R.version['platform'], R.version['arch'],
R.version['os']))); Sys.setenv('ARROW_R_DEV' = TRUE); install.packages('arrow',
repos = 'https://cran.r-project.org', type='source', Ncpus=-1)"}}
{{mv /usr/local/lib/R/etc/Makeconf.bak /usr/local/lib/R/etc/Makeconf;}}
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘arrow’ in dyn.load(file, DLLpath =
DLLpath, ...):
unable to load shared object
'/usr/local/lib/R/site-library/00LOCK-arrow/00new/arrow/libs/arrow.so':
/usr/local/lib/R/site-library/00LOCK-arrow/00new/arrow/libs/arrow.so:
undefined symbol: _ZTIN6apache6thrift8protocol9TProtocolE
Error: loading failed
Execution halted
ERROR: loading failed
* removing '/usr/local/lib/R/site-library/arrow'
The downloaded source packages are in
'/tmp/RtmpnU3zrU/downloaded_packages'
Warning message:
In install.packages("arrow", repos = "https://cran.r-project.org;, :
installation of package 'arrow' had non-zero exit status
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 9.0.0, 8.0.1
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: build-images.out
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600881#comment-17600881
]
Vincent Nijs commented on ARROW-17374:
--
I haven't tried that tbh. The docker file below is for x86_64. However, it will
use binaries by default.
[https://github.com/radiant-rstats/docker/blob/master/rsm-jupyter-rs/Dockerfile]
Could be tested for x86_64 with a tweak to the script below to install from
source.
[https://github.com/radiant-rstats/docker/blob/master/files/setup-radiant.sh]
I'm on an M1 so building that would take a 1+ day.
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 9.0.0, 8.0.1
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: build-images.out
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600643#comment-17600643
]
Kouhei Sutou commented on ARROW-17374:
--
Thanks．
I could build the {{Dockerfile}} successfully... (It takes 1 day+ because I use
QEMU)
This is related to aarch64? Can we reproduce this on x86_64?
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 9.0.0, 8.0.1
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: build-images.out
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599968#comment-17599968
]
Vincent Nijs commented on ARROW-17374:
--
[~kou] See below
Docker file: [https://github.com/radiant-rstats/docker/tree/master/rsm-jupyter]
Script used by docker to install arrow:
[https://github.com/radiant-rstats/docker/blob/master/files/setup-radiant.sh]
Docker image: [https://hub.docker.com/repository/docker/vnijs/rsm-jupyter]
Build script:
[https://github.com/radiant-rstats/docker/blob/master/scripts/build-images.sh]
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 9.0.0, 8.0.1
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: build-images.out
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599917#comment-17599917
]
Kouhei Sutou commented on ARROW-17374:
--
Could you also provide {{Dockerfile}}?
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 9.0.0, 8.0.1
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: build-images.out
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599759#comment-17599759
]
Vincent Nijs commented on ARROW-17374:
--
Made some progress. I had to install `libsnappy-dev` and `libre2-dev` in Ubuntu
22.04 and use
[https://github.com/duckdb/duckdb/issues/3049#issuecomment-1096671708]
Arrow still doesn't compile successfully, however, and ends with the below. The
build logs are attached ("build-images.out").
```
#21 1786.1 ** testing if installed package can be loaded from temporary location
#21 1786.4 Error: package or namespace load failed for ‘arrow’ in
dyn.load(file, DLLpath = DLLpath, ...):
#21 1786.4 unable to load shared object
'/usr/local/lib/R/site-library/00LOCK-arrow/00new/arrow/libs/arrow.so':
#21 1786.4
/usr/local/lib/R/site-library/00LOCK-arrow/00new/arrow/libs/arrow.so: undefined
symbol: _ZTIN6apache6thrift8protocol9TProtocolE
```
```
# workaround for fPIC issue
[https://github.com/duckdb/duckdb/issues/3049#issuecomment-1096671708]
cp -a /usr/local/lib/R/etc/Makeconf /usr/local/lib/R/etc/Makeconf.bak;
sed -i 's/fpic/fPIC/g' /usr/local/lib/R/etc/Makeconf;
R -e "options(HTTPUserAgent = sprintf('R/%s R (%s)', getRversion(),
paste(getRversion(), R.version['platform'], R.version['arch'],
R.version['os']))); Sys.setenv('ARROW_R_DEV' = TRUE);
install.packages(c('duckdb', 'arrow'), repo='${CRAN}', Ncpus=${NCPUS})"
mv /usr/local/lib/R/etc/Makeconf.bak /usr/local/lib/R/etc/Makeconf;
```
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 9.0.0, 8.0.1
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: build-images.out
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17585662#comment-17585662
]
Vincent Nijs commented on ARROW-17374:
--
I ran into the same issue installing arrow from source for R on aarch64. Python
is installed using conda but R was installed from source. The docker container
for aarch64 is
[vnijs/rsm-jupyter]([https://hub.docker.com/repository/docker/vnijs/rsm-jupyter).]
Note that the image is rather large.
FYI I also tried installing using binaries but that gives the error below even
though the file clearly exists.
!image-2022-08-26-21-42-13-338.png!
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 9.0.0, 8.0.1
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: image-2022-08-26-21-42-13-338.png
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581086#comment-17581086
]
Kouhei Sutou commented on ARROW-17374:
--
Could you provide a {{Dockerfile}} or something that reproduces this case?
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 9.0.0, 8.0.1
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17580981#comment-17580981
]
Felipe Matas commented on ARROW-17374:
--
Hi, this maybe is weird, I'm in gentoo 64, R too, and I have the same error
a difference, would be I have already installed snappy, I can load and use it
from python (3.9). I have python 3.10 too but is not the main to use in the
system.
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 9.0.0, 8.0.1
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579851#comment-17579851
]
SHIMA Tatsuya commented on ARROW-17374:
---
I think what you are seeing in the error message is Python 3.1, not Python 3.10.
I am not sure, but are you specifying the Python version in yaml or something?
In yaml, if you just write {{3.10}}, it
will be interpreted as the number {{3.1}}, so you need to string {{"3.10"}}.
If this is an irrelevant comment, please ignore it.
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 9.0.0, 8.0.1
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578526#comment-17578526
]
Shane Brennan commented on ARROW-17374:
---
Thanks Kouhei and Neal for responding.
The conda install fails both for the release and the nightly build. Seems like
the Python 3.10 on the system doesn't play too well with the conda install
package. The conda error is.
{code:java}
(R) sh-4.2$ conda install -c conda-forge --strict-channel-priority r-arrow
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible
solve.
Solving environment: failed with repodata from current_repodata.json, will
retry with next repodata source.
ResolvePackageNotFound:
- python=3.1{code}
However, I mostly was trying to install arrow within R using this setup
{code:java}
Sys.setenv(NOT_CRAN = TRUE)
Sys.setenv("ARROW_PARQUET" = "ON")
Sys.setenv("ARROW_S3" = "ON")
Sys.setenv("ARROW_WITH_SNAPPY" = "ON")
Sys.setenv("ARROW_R_DEV" = TRUE)
install.packages("arrow", repos =
"https://arrow-r-nightly.s3.amazonaws.com;){code}
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 9.0.0, 8.0.1
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578156#comment-17578156
]
Neal Richardson commented on ARROW-17374:
-
hmm, I wonder how {{SNAPPY_LIB-NOTFOUND}} gets inserted into the link command
with -l libs?
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 9.0.0, 8.0.1
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578155#comment-17578155
]
Kouhei Sutou commented on ARROW-17374:
--
It seems that you use Conda. Did you install Snappy by Conda? Or did you
install Snappy by {{yum}}? I think that you need to install Snappy by Conda.
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 9.0.0, 8.0.1
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
