[
https://issues.apache.org/jira/browse/ARROW-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacob Wujciak-Jens updated ARROW-16404:
---------------------------------------
Description:
Find a way to host the R nightly binaries in some form of artifactory under ASF
umbrella. Currently they are hosted on s3. See ARROW-16401
Python wheels are hosted gemfury.io
cc: [~kszucs] [~kou] [~amol-] [~raulcd]
Possible solutions, both of which could be extended for other Components that
don't need an active server for their binary repos:
* Host nightly builds on apache artifactory
** [https://apache.jfrog.io/artifactory]
*** (?) Is this even possible as nightlies are not signed official releases?
*** Size considerations (-> limit to 5 days as with conda?)
** [https://nightlies.apache.org/]
*** {-}(y){-}Made for this purpose, extendable to other components like java
that currently have no "nice" solution
*** {-}(y){-}Entirely under ASF control and on-brand{-}{{}}{-}
*** {-}{{(y)}}{-}{{{}We could host M1 binaries compiled on Crossbow (by
downloading them in arrow ci){}}}
* Host nightly builds on Github via Github Pages using either
[drat|https://github.com/eddelbuettel/drat] or manually creating the repo
structure.
** Building and hosting on apache/arrow:
*** (y) The existing [R
CI|https://github.com/apache/arrow/blob/master/.github/workflows/r.yml] could
easily be extended to also commit the binaries to a gh_pages based repository,
either on push to master or as a cron job.
*** (y) Entirely under ASF control and on-brand for users:
{{install.packages("arrow", repos = "https://apache.github.io/arrow/r";)}} vs.
{{install.packages("arrow", repos =
"https://arrow-r-nightly.s3.amazonaws.com";)}}
*** (y) Github Pages can be turned on via
[.asf.yaml|https://cwiki.apache.org/confluence/display/INFRA/git+-+.asf.yaml+features#Git.asf.yamlfeatures-DependabotAlertsandUpdates]
- no ticket needed for Infra?
*** (n) Not possible to create M1 binaries ([for
now?|https://github.com/actions/virtual-environments/issues/2187])
** Hosting on a newly created utility repository apache/arrow-nightly:
*** Build binaries in Crossbow and upload via PAT with write access (not an
option for apache/arrow due to security concerns)
*** {{(y) We could host M1 binaries compiled on Crossbow}}
*** (y) Entirely under ASF control and on-brand for users:
{{install.packages("arrow", repos =
"https://apache.github.io/arrow-nightly/";)}} vs. {{install.packages("arrow",
repos = "https://arrow-r-nightly.s3.amazonaws.com";)}}
*** (?) Does INFRA even allow such "utility" repos?
** Building and hosting on ursacomputing/crossbow:
*** (n) Neither under ASF control nor on-brand
*** {{(y) We could host M1 binaries}}
was:
Find a way to host the R nightly binaries in some form of artifactory under ASF
umbrella. Currently they are hosted on s3. See ARROW-16401
Python wheels are hosted gemfury.io
cc: [~kszucs] [~kou] [~amol-] [~raulcd]
Possible solutions, both of which could be extended for other Components that
don't need an active server for their binary repos:
* Host nightly builds on apache artifactory
** [https://apache.jfrog.io/artifactory]
*** (?) Is this even possible as nightlies are not signed official releases?
*** Size considerations (-> limit to 5 days as with conda?)
* -Host nightly builds on Github via Github Pages using either
[drat|https://github.com/eddelbuettel/drat] or manually creating the repo
structure.-
** -Building and hosting on apache/arrow:-
*** -(y) The existing [R
CI|https://github.com/apache/arrow/blob/master/.github/workflows/r.yml] could
easily be extended to also commit the binaries to a gh_pages based repository,
either on push to master or as a cron job.-
*** -(y) Entirely under ASF control and on-brand for users:
{{install.packages("arrow", repos = "https://apache.github.io/arrow/r";)}} vs.
{{install.packages("arrow", repos =
"https://arrow-r-nightly.s3.amazonaws.com";)}}-
*** -(y) Github Pages can be turned on via
[.asf.yaml|https://cwiki.apache.org/confluence/display/INFRA/git+-+.asf.yaml+features#Git.asf.yamlfeatures-DependabotAlertsandUpdates]
- no ticket needed for Infra?-
*** -(n) Not possible to create M1 binaries ([for
now?|https://github.com/actions/virtual-environments/issues/2187])-
** -Hosting on a newly created utility repository apache/arrow-nightly:-
*** -Build binaries in Crossbow and upload via PAT with write access (not an
option for apache/arrow due to security concerns)-
*** -{{(y) We could host M1 binaries compiled on Crossbow}}-
*** -(y) Entirely under ASF control and on-brand for users:
{{install.packages("arrow", repos =
"https://apache.github.io/arrow-nightly/";)}} vs. {{install.packages("arrow",
repos = "https://arrow-r-nightly.s3.amazonaws.com";)}}-
*** -(?) Does INFRA even allow such "utility" repos?-
** -Building and hosting on ursacomputing/crossbow:-
*** -(n) Neither under ASF control nor on-brand-
*** -{{(y) We could host M1 binaries}}-
> [R][CI] Research alternative binary hosting
> --------------------------------------------
>
> Key: ARROW-16404
> URL: https://issues.apache.org/jira/browse/ARROW-16404
> Project: Apache Arrow
> Issue Type: Sub-task
> Components: Continuous Integration, R
> Reporter: Jacob Wujciak-Jens
> Assignee: Jacob Wujciak-Jens
> Priority: Major
> Fix For: 9.0.0
>
>
> Find a way to host the R nightly binaries in some form of artifactory under
> ASF umbrella. Currently they are hosted on s3. See ARROW-16401
> Python wheels are hosted gemfury.io
> cc: [~kszucs] [~kou] [~amol-] [~raulcd]
> Possible solutions, both of which could be extended for other Components that
> don't need an active server for their binary repos:
> * Host nightly builds on apache artifactory
> ** [https://apache.jfrog.io/artifactory]
> *** (?) Is this even possible as nightlies are not signed official releases?
> *** Size considerations (-> limit to 5 days as with conda?)
> ** [https://nightlies.apache.org/]
> *** {-}(y){-}Made for this purpose, extendable to other components like java
> that currently have no "nice" solution
> *** {-}(y){-}Entirely under ASF control and on-brand{-}{{}}{-}
> *** {-}{{(y)}}{-}{{{}We could host M1 binaries compiled on Crossbow (by
> downloading them in arrow ci){}}}
> * Host nightly builds on Github via Github Pages using either
> [drat|https://github.com/eddelbuettel/drat] or manually creating the repo
> structure.
> ** Building and hosting on apache/arrow:
> *** (y) The existing [R
> CI|https://github.com/apache/arrow/blob/master/.github/workflows/r.yml] could
> easily be extended to also commit the binaries to a gh_pages based
> repository, either on push to master or as a cron job.
> *** (y) Entirely under ASF control and on-brand for users:
> {{install.packages("arrow", repos = "https://apache.github.io/arrow/r";)}} vs.
> {{install.packages("arrow", repos =
> "https://arrow-r-nightly.s3.amazonaws.com";)}}
> *** (y) Github Pages can be turned on via
> [.asf.yaml|https://cwiki.apache.org/confluence/display/INFRA/git+-+.asf.yaml+features#Git.asf.yamlfeatures-DependabotAlertsandUpdates]
> - no ticket needed for Infra?
> *** (n) Not possible to create M1 binaries ([for
> now?|https://github.com/actions/virtual-environments/issues/2187])
> ** Hosting on a newly created utility repository apache/arrow-nightly:
> *** Build binaries in Crossbow and upload via PAT with write access (not an
> option for apache/arrow due to security concerns)
> *** {{(y) We could host M1 binaries compiled on Crossbow}}
> *** (y) Entirely under ASF control and on-brand for users:
> {{install.packages("arrow", repos =
> "https://apache.github.io/arrow-nightly/";)}} vs. {{install.packages("arrow",
> repos = "https://arrow-r-nightly.s3.amazonaws.com";)}}
> *** (?) Does INFRA even allow such "utility" repos?
> ** Building and hosting on ursacomputing/crossbow:
> *** (n) Neither under ASF control nor on-brand
> *** {{(y) We could host M1 binaries}}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
