[GitHub] [arrow-datafusion-python] alamb commented on pull request #5: Latest from datafusion-contrib
alamb commented on PR #5: URL: https://github.com/apache/arrow-datafusion-python/pull/5#issuecomment-1191888186 👍 to me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-datafusion-python] matthewmturner commented on pull request #5: Latest from datafusion-contrib
matthewmturner commented on PR #5: URL: https://github.com/apache/arrow-datafusion-python/pull/5#issuecomment-1191605004 @andygrove good for me - i just left a governance related question on the original issue proposing this move -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-datafusion-python] andygrove commented on pull request #5: Latest from datafusion-contrib
andygrove commented on PR #5: URL: https://github.com/apache/arrow-datafusion-python/pull/5#issuecomment-1191250037 @jimexist @cpcloud @samkaufman @jychen7 @alamb @matthewmturner Could you please review when you have time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-datafusion-python] dependabot[bot] opened a new pull request, #1: Bump numpy from 1.21.3 to 1.22.0
dependabot[bot] opened a new pull request, #1: URL: https://github.com/apache/arrow-datafusion-python/pull/1 Bumps [numpy](https://github.com/numpy/numpy) from 1.21.3 to 1.22.0. Release notes Sourced from https://github.com/numpy/numpy/releases";>numpy's releases. v1.22.0 NumPy 1.22.0 Release Notes NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are: Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release. A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX. NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data. New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature. A new configurable allocator for use by downstream projects. These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation. The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays. Expired deprecations Deprecated numeric style dtype strings have been removed Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError. (https://github-redirect.dependabot.com/numpy/numpy/pull/19539";>gh-19539) Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter. (https://github-redirect.dependabot.com/numpy/numpy/pull/19615";>gh-19615) ... (truncated) Commits https://github.com/numpy/numpy/commit/4adc87dff15a247e417d50f10cc4def8e1c17a03";>4adc87d Merge pull request https://github-redirect.dependabot.com/numpy/numpy/issues/20685";>#20685 from charris/prepare-for-1.22.0-release https://github.com/numpy/numpy/commit/fd66547557f57c430d41be2fc0764f74a62e8ccf";>fd66547 REL: Prepare for the NumPy 1.22.0 release. https://github.com/numpy/numpy/commit/125304b035effcd82e366e601b102e7347eaa9ba";>125304b wip https://github.com/numpy/numpy/commit/c283859128b1a4b57014581570a23ed7950a24ea";>c283859 Merge pull request https://github-redirect.dependabot.com/numpy/numpy/issues/20682";>#20682 from charris/backport-20416 https://github.com/numpy/numpy/commit/5399c03d4a069fe81a1616be0184c9749d7271ee";>5399c03 Merge pull request https://github-redirect.dependabot.com/numpy/numpy/issues/20681";>#20681 from charris/backport-20954 https://github.com/numpy/numpy/commit/f9c45f8ebf31340b1a5a0371bfca25afcfc4794e";>f9c45f8 Merge pull request https://github-redirect.dependabot.com/numpy/numpy/issues/20680";>#20680 from charris/backport-20663 https://github.com/numpy/numpy/commit/794b36f7e1bf2a8c42774ab0db86a74bd32f674b";>794b36f Update armccompiler.py https://github.com/numpy/numpy/commit/d93b14e3d7abaa1d837825e51671f817788e120f";>d93b14e Update test_public_api.py https://github.com/numpy/numpy/commit/7662c0789cc6a70d5ad4d950ee2e95f3afef7df6";>7662c07 Update init.py https://github.com/numpy/numpy/commit/311ab52488a7d096ac3bc4c2de0fdae17ecd13ef";>311ab52 Update armccompiler.py Additional commits viewable in https://github.com/numpy/numpy/compare/v1.21.3...v1.22.0";>compare view [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: -
[GitHub] [arrow-datafusion-python] andygrove opened a new pull request, #5: Latest from datafusion-contrib
andygrove opened a new pull request, #5: URL: https://github.com/apache/arrow-datafusion-python/pull/5 This PR contributes the changes that were made to the project since it moved to the `datafusion-contrib` github organization. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-datafusion-python] dependabot[bot] opened a new pull request, #4: Bump regex from 1.5.4 to 1.6.0
dependabot[bot] opened a new pull request, #4: URL: https://github.com/apache/arrow-datafusion-python/pull/4 Bumps [regex](https://github.com/rust-lang/regex) from 1.5.4 to 1.6.0. Changelog Sourced from https://github.com/rust-lang/regex/blob/master/CHANGELOG.md";>regex's changelog. 1.6.0 (2022-07-05) This release principally includes an upgrade to Unicode 14. New features: [FEATURE https://github-redirect.dependabot.com/rust-lang/regex/issues/832";>#832](https://github-redirect.dependabot.com/rust-lang/regex/pull/832";>rust-lang/regex#832): Clarify that Captures::len includes all groups, not just matching groups. [FEATURE https://github-redirect.dependabot.com/rust-lang/regex/issues/857";>#857](https://github-redirect.dependabot.com/rust-lang/regex/pull/857";>rust-lang/regex#857): Add an ExactSizeIterator impl for SubCaptureMatches. [FEATURE https://github-redirect.dependabot.com/rust-lang/regex/issues/861";>#861](https://github-redirect.dependabot.com/rust-lang/regex/pull/861";>rust-lang/regex#861): Improve RegexSet documentation examples. [FEATURE https://github-redirect.dependabot.com/rust-lang/regex/issues/877";>#877](https://github-redirect.dependabot.com/rust-lang/regex/issues/877";>rust-lang/regex#877): Upgrade to Unicode 14. Bug fixes: [BUG https://github-redirect.dependabot.com/rust-lang/regex/issues/792";>#792](https://github-redirect.dependabot.com/rust-lang/regex/issues/792";>rust-lang/regex#792): Fix error message rendering bug. 1.5.6 (2022-05-20) This release includes a few bug fixes, including a bug that produced incorrect matches when a non-greedy ? operator was used. [BUG https://github-redirect.dependabot.com/rust-lang/regex/issues/680";>#680](https://github-redirect.dependabot.com/rust-lang/regex/issues/680";>rust-lang/regex#680): Fixes a bug where [[:alnum:][:^ascii:]] dropped [:alnum:] from the class. [BUG https://github-redirect.dependabot.com/rust-lang/regex/issues/859";>#859](https://github-redirect.dependabot.com/rust-lang/regex/issues/859";>rust-lang/regex#859): Fixes a bug where Hir::is_match_empty returned false for \b. [BUG https://github-redirect.dependabot.com/rust-lang/regex/issues/862";>#862](https://github-redirect.dependabot.com/rust-lang/regex/issues/862";>rust-lang/regex#862): Fixes a bug where 'ab??' matches 'ab' instead of 'a' in 'ab'. 1.5.5 (2022-03-08) This releases fixes a security bug in the regex compiler. This bug permits a vector for a denial-of-service attack in cases where the regex being compiled is untrusted. There are no known problems where the regex is itself trusted, including in cases of untrusted haystacks. https://github.com/rust-lang/regex/security/advisories/GHSA-m5pq-gvj9-9vr8";>SECURITY #GHSA-m5pq-gvj9-9vr8: Fixes a bug in the regex compiler where empty sub-expressions subverted the existing mitigations in place to enforce a size limit on compiled regexes. The Rust Security Response WG published an advisory about this: https://groups.google.com/g/rustlang-security-announcements/c/NcNNL1Jq7Yw";>https://groups.google.com/g/rustlang-security-announcements/c/NcNNL1Jq7Yw Commits https://github.com/rust-lang/regex/commit/fc9ee6a249f2d594713c16b1646670924b4d";>fc9ee6a 1.6.0 https://github.com/rust-lang/regex/commit/8ddc8ece4bd3ca4cdd068c6895f44fd57b682951";>8ddc8ec deps: bump regex-syntax to 0.6.27 https://github.com/rust-lang/regex/commit/8c0eccd0c6e775eae8a258d6bd43aca2e5ff92e5";>8c0eccd regex-syntax-0.6.27 https://github.com/rust-lang/regex/commit/dee5c25d8e4f030182bf61ca56b8819ae0c2f7f2";>dee5c25 changelog: 1.6.0 https://github.com/rust-lang/regex/commit/9ca3099037dcb2faf1b49e6493f4c758532f2da1";>9ca3099 regex: fix clippy lints up to rust 1.41.1 https://github.com/rust-lang/regex/commit/de838287bb9ebd554c078ee75a8e69a59f6e3820";>de83828 syntax: fix clippy lints up to rust 1.41.1 https://github.com/rust-lang/regex/commit/b87cd88476a0eb4c0187321a6ab42c7644531b67";>b87cd88 syntax: include only the start of the character class on error https://github.com/rust-lang/regex/commit/9d1478cfb5616a4f289c015fffaf7f6228b72b56";>9d1478c doc: fix typos https://github.com/rust-lang/regex/commit/7d21662735ffdf585ca6bff773d997fe44a4f868";>7d21662 doc: clarify Captures::len includes non-matching captures https://github.com/rust-lang/regex/commit/5e98788947b28da3da27f4e156b877eb0cb1593e";>5e98788 perf: use is_match_at instead of shortest_match_at Additional commits viewable in https://github.com/rust-lang/regex/compare/1.5.4...1.6.0";>compare view [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-securi
[GitHub] [arrow-datafusion-python] dependabot[bot] opened a new pull request, #3: Bump tokio from 1.12.0 to 1.13.1
dependabot[bot] opened a new pull request, #3:
URL: https://github.com/apache/arrow-datafusion-python/pull/3
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.12.0 to 1.13.1.
Release notes
Sourced from https://github.com/tokio-rs/tokio/releases";>tokio's releases.
Tokio v1.13.1
1.13.1 (November 15, 2021)
This release fixes a data race when sending and receiving on a
closed oneshot channel (https://rustsec.org/advisories/RUSTSEC-2021-0124.html";>RUSTSEC-2021-0124).
Fixed
sync: fix a data race between oneshot::Sender::send and
awaiting a
oneshot::Receiver when the oneshot has been closed (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4226";>#4226)
Tokio v1.13.0
1.13.0 (October 29, 2021)
Fixed
sync: fix Notify to clone the waker before locking its
waiter list (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4129";>#4129)
tokio: add riscv32 to non atomic64 architectures (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4185";>#4185)
Added
net: add poll_{recv,send}_ready methods to udp
and uds_datagram (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4131";>#4131)
net: add try_*, readable,
writable, ready, and peer_addr methods
to split halves (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4120";>#4120)
sync: add blocking_lock to Mutex (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4130";>#4130)
sync: add watch::Sender::send_replace (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/3962";>#3962,
https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4195";>#4195)
sync: expand Debug for Mutex impl to
unsized T (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4134";>#4134)
tracing: instrument time::Sleep (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4072";>#4072)
tracing: use structured location fields for spawned tasks (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4128";>#4128)
Changed
io: add assert in copy_bidirectional that
poll_write is sensible (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4125";>#4125)
macros: use qualified syntax when polling in select! (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4192";>#4192)
runtime: handle block_on wakeups better (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4157";>#4157)
task: allocate callback on heap immediately in debug mode (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4203";>#4203)
tokio: assert platform-minimum requirements at build time (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/3797";>#3797)
Documented
docs: conversion of doc comments to indicative mood (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4174";>#4174)
docs: add returning on the first error example for
try_join! (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4133";>#4133)
docs: fixing broken links in tokio/src/lib.rs (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4132";>#4132)
signal: add example with background listener (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4171";>#4171)
sync: add more oneshot examples (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4153";>#4153)
time: document Interval::tick cancel safety (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4152";>#4152)
https://github-redirect.dependabot.com/tokio-rs/tokio/issues/3797";>#3797:
https://github-redirect.dependabot.com/tokio-rs/tokio/pull/3797";>tokio-rs/tokio#3797
https://github-redirect.dependabot.com/tokio-rs/tokio/issues/3962";>#3962:
https://github-redirect.dependabot.com/tokio-rs/tokio/pull/3962";>tokio-rs/tokio#3962
... (truncated)
Commits
https://github.com/tokio-rs/tokio/commit/2a3c803dd1c487868eabcef547066952cc07624e";>2a3c803
chore: prepare Tokio v1.13.1 (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4235";>#4235)
https://github.com/tokio-rs/tokio/commit/7d8de50482a077349db721c7cad3034b093a0083";>7d8de50
oneshot: document UnsafeCell invariants (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4229";>#4229)
https://github.com/tokio-rs/tokio/commit/ab0e60ddeaf1b617ed4990901d2410ef810f1a61";>ab0e60d
sync: fix racy UnsafeCell access on a closed oneshot (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4226";>#4226)
https://github.com/tokio-rs/tokio/commit/ac89d8926d0c06eeb62936b0843a5278cd98ade0";>ac89d89
chore: prepare Tokio v1.13.0 (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4196";>#4196)
https://github.com/tokio-rs/tokio/commit/e184205421e7a5e6d397805adb516e661819240c";>e184205
chore: prepare tokio-macros 1.6.0 (https://github-redirect.dependabot.com/tokio-rs/tokio/issues/4197";>#4197)
https://github.com/tokio-rs/tokio/c
[GitHub] [arrow-datafusion-python] dependabot[bot] opened a new pull request, #2: Bump num-bigint from 0.4.2 to 0.4.3
dependabot[bot] opened a new pull request, #2: URL: https://github.com/apache/arrow-datafusion-python/pull/2 Bumps [num-bigint](https://github.com/rust-num/num-bigint) from 0.4.2 to 0.4.3. Changelog Sourced from https://github.com/rust-num/num-bigint/blob/master/RELEASES.md";>num-bigint's changelog. Release 0.4.3 (2021-11-02) https://github.com/rust-num/num-bigint/security/advisories/GHSA-v935-pqmr-g8v9";>GHSA-v935-pqmr-g8v9: https://github-redirect.dependabot.com/rust-num/num-bigint/pull/228";>Fix unexpected panics in multiplication. Contributors: https://github.com/arvidn";>@arvidn, https://github.com/cuviper";>@cuviper, https://github.com/guidovranken";>@guidovranken Commits https://github.com/rust-num/num-bigint/commit/e77ffacafef7e8fa2c2b990489b7aa30dceaed64";>e77ffac Merge https://github-redirect.dependabot.com/rust-num/num-bigint/issues/228";>#228 https://github.com/rust-num/num-bigint/commit/056e0d42fe98f3889b10db6bc8d637df2e113e42";>056e0d4 Release 0.4.3 https://github.com/rust-num/num-bigint/commit/8008707fea97b9215f3b949a0eb6044aec709a31";>8008707 Fix an undersized buffer panic in multiplication https://github.com/rust-num/num-bigint/commit/0940e509dca55d19197adbe7cb5c1d5423a390cf";>0940e50 Fix a mac3 panic when an operand is all-zero https://github.com/rust-num/num-bigint/commit/1fc25378c1caf8f97ce56da58ed8bc40a13e3e76";>1fc2537 Add 0.3.3 release notes See full diff in https://github.com/rust-num/num-bigint/compare/num-bigint-0.4.2...num-bigint-0.4.3";>compare view [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/arrow-datafusion-python/network/alerts). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-datafusion] yjshen commented on a diff in pull request #2132: WIP: Reduce sort memory usage v1
yjshen commented on code in PR #2132:
URL: https://github.com/apache/arrow-datafusion/pull/2132#discussion_r841342087
##
datafusion/core/src/physical_plan/sorts/sort.rs:
##
@@ -105,13 +107,21 @@ impl ExternalSorter {
}
}
-async fn insert_batch(&self, input: RecordBatch) -> Result<()> {
+async fn insert_batch(
+&self,
+input: RecordBatch,
+tracking_metrics: &MemTrackingMetrics,
+) -> Result<()> {
if input.num_rows() > 0 {
let size = batch_byte_size(&input);
self.try_grow(size).await?;
self.metrics.mem_used().add(size);
let mut in_mem_batches = self.in_mem_batches.lock().await;
-in_mem_batches.push(input);
+// NB timer records time taken on drop, so there are no
+// calls to `timer.done()` below.
+let _timer = tracking_metrics.elapsed_compute().timer();
+let partial = sort_batch(input, self.schema.clone(), &self.expr)?;
Review Comment:
Performance would deteriorate significantly without this change:
```
Running benchmarks with the following options: DataFusionBenchmarkOpt {
query: 1, debug: false, iterations: 3, partitions: 2, batch_size: 4096, path:
"/home/yijie/sort_test/tpch-parquet", file_format: "parquet", mem_table: false,
output_path: None }
Query 1 iteration 0 took 4619.9 ms and returned 6001214 rows
Query 1 iteration 1 took 4561.0 ms and returned 6001214 rows
Query 1 iteration 2 took 4527.7 ms and returned 6001214 rows
```
The main reason I think is caused by random memory access while constructing
output batches. Without this per-batch sort, while collecting cells from
unsorted batches, the memory access would be fully randomized. With this
per-batch sort, we are accessing memory linearly for each column in each batch,
this would results in much predictable memory access pattern and benefits the
CPU cache.
I think the perf counter confirms the above speculation:
```
sudo perf stat -a -e
cache-misses,cache-references,l3_cache_accesses,l3_misses,dTLB-load-misses,dTLB-loads
target/release/tpch benchmark datafusion --iterations 3 --path
/home/yijie/sort_test/tpch-parquet --format parquet --query 1 --batch-size 4096
```
Without this per-batch sort:
```
Performance counter stats for 'system wide':
1,340,359,889 cache-misses # 35.817 % of all cache
refs
3,742,289,458 cache-references
1,984,089,839 l3_cache_accesses
540,429,658 l3_misses
303,508,234 dTLB-load-misses # 49.51% of all dTLB
cache accesses
613,048,439 dTLB-loads
14.222309739 seconds time elapsed
```
With this per-batch sort:
```
Performance counter stats for 'system wide':
1,059,913,512 cache-misses # 30.715 % of all cache
refs
3,450,839,405 cache-references
1,388,975,765 l3_cache_accesses
235,570,805 l3_misses
239,390,511 dTLB-load-misses # 51.36% of all dTLB
cache accesses
466,141,655 dTLB-loads
8.675278258 seconds time elapsed
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [arrow] sanjibansg commented on a diff in pull request #12755: ARROW-16014: [C++] Create more benchmarks for measuring expression evaluation overhead
sanjibansg commented on code in PR #12755: URL: https://github.com/apache/arrow/pull/12755#discussion_r841298581 ## cpp/src/arrow/compute/exec/expression_execution_benchmark.cc: ## @@ -0,0 +1,56 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +#include "benchmark/benchmark.h" + +#include "arrow/compute/cast.h" +#include "arrow/compute/exec/expression.h" +#include "arrow/compute/exec/test_util.h" +#include "arrow/dataset/partition.h" +#include "arrow/testing/generator.h" +#include "arrow/testing/gtest_util.h" Review Comment: Removed the file and moved the benchmark to the expression benchmark file. ## cpp/src/arrow/compute/exec/CMakeLists.txt: ## @@ -36,6 +36,7 @@ add_arrow_compute_test(union_node_test PREFIX "arrow-compute") add_arrow_compute_test(util_test PREFIX "arrow-compute") add_arrow_benchmark(expression_benchmark PREFIX "arrow-compute") +add_arrow_benchmark(expression_execution_benchmark PREFIX "arrow-compute") Review Comment: Removed the file and moved the benchmark to the expression benchmark file. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] sanjibansg commented on a diff in pull request #12755: ARROW-16014: [C++] Create more benchmarks for measuring expression evaluation overhead
sanjibansg commented on code in PR #12755: URL: https://github.com/apache/arrow/pull/12755#discussion_r841298550 ## cpp/src/arrow/compute/exec/expression_execution_benchmark.cc: ## @@ -0,0 +1,56 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +#include "benchmark/benchmark.h" + +#include "arrow/compute/cast.h" +#include "arrow/compute/exec/expression.h" +#include "arrow/compute/exec/test_util.h" +#include "arrow/dataset/partition.h" Review Comment: Removed the file and moved the benchmark to the expression benchmark file. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] sanjibansg commented on a diff in pull request #12755: ARROW-16014: [C++] Create more benchmarks for measuring expression evaluation overhead
sanjibansg commented on code in PR #12755:
URL: https://github.com/apache/arrow/pull/12755#discussion_r841298493
##
cpp/src/arrow/compute/exec/expression_execution_benchmark.cc:
##
@@ -0,0 +1,56 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "benchmark/benchmark.h"
+
+#include "arrow/compute/cast.h"
+#include "arrow/compute/exec/expression.h"
+#include "arrow/compute/exec/test_util.h"
+#include "arrow/dataset/partition.h"
+#include "arrow/testing/generator.h"
+#include "arrow/testing/gtest_util.h"
+#include "arrow/type.h"
+
+namespace arrow {
+namespace compute {
+
+auto expression =
+and_(less(field_ref("x"), literal(20)), greater(field_ref("x"),
literal(0)));
+
+static void Execution(benchmark::State& state) {
+ const auto array_batches = static_cast(state.range(0));
+ const auto array_size = 1000 / array_batches;
+
+ ExecContext ctx;
+ auto dataset_schema = schema({
+ field("x", int64()),
+ });
+ ExecBatch input({Datum(ConstantArrayGenerator::Int64(array_size, 5))},
+ /*length=*/1);
+
+ ASSIGN_OR_ABORT(auto bound, expression.Bind(*dataset_schema));
+ for (auto _ : state) {
+for (int it = 0; it < array_batches; ++it)
+ ABORT_NOT_OK(ExecuteScalarExpression(bound, input, &ctx).status());
+ }
+ state.SetBytesProcessed(state.iterations() * 8 * array_size);
Review Comment:
Made the change, now we are using `items per second`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [arrow] sanjibansg commented on a diff in pull request #12755: ARROW-16014: [C++] Create more benchmarks for measuring expression evaluation overhead
sanjibansg commented on code in PR #12755:
URL: https://github.com/apache/arrow/pull/12755#discussion_r841298365
##
cpp/src/arrow/compute/exec/expression_execution_benchmark.cc:
##
@@ -0,0 +1,56 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "benchmark/benchmark.h"
+
+#include "arrow/compute/cast.h"
+#include "arrow/compute/exec/expression.h"
+#include "arrow/compute/exec/test_util.h"
+#include "arrow/dataset/partition.h"
+#include "arrow/testing/generator.h"
+#include "arrow/testing/gtest_util.h"
+#include "arrow/type.h"
+
+namespace arrow {
+namespace compute {
+
+auto expression =
+and_(less(field_ref("x"), literal(20)), greater(field_ref("x"),
literal(0)));
Review Comment:
For a zero-copy benchmark, now I am trying to return the values as is, does
this work?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [arrow-datafusion] renato2099 commented on a diff in pull request #2142: implement 'StringConcat' operator to support sql like "select 'aa' || 'b' "
renato2099 commented on code in PR #2142:
URL: https://github.com/apache/arrow-datafusion/pull/2142#discussion_r841282224
##
datafusion/core/tests/sql/expr.rs:
##
@@ -280,6 +280,47 @@ async fn query_scalar_minus_array() -> Result<()> {
Ok(())
}
+#[tokio::test]
+async fn test_string_concat_operator() -> Result<()> {
+let ctx = SessionContext::new();
+// concat 2 strings
+let sql = "SELECT 'aa' || 'b'";
+let actual = execute_to_batches(&ctx, sql).await;
+let expected = vec![
+"+-+",
+"| Utf8(\"aa\") || Utf8(\"b\") |",
+"+-+",
+"| aab |",
+"+-+",
+];
+assert_batches_eq!(expected, &actual);
+
+// concat 4 strings as a string concat pipe.
+let sql = "SELECT 'aa' || 'b' || 'cc' || 'd'";
+let expected = vec![
+"++",
+"| Utf8(\"aa\") || Utf8(\"b\") || Utf8(\"cc\") || Utf8(\"d\") |",
+"++",
+"| aabccd |",
+"++",
+];
+let actual = execute_to_batches(&ctx, sql).await;
+assert_batches_eq!(expected, &actual);
+
+// concat 3 strings with one is NULL
+let sql = "SELECT 'aa' || NULL || 'd'";
+let expected = vec![
Review Comment:
as a user I would also expect that any operation involving a `null` value
would actually return `null` since `null != empty`, maybe there is a SQL spec
somewhere we could check?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [arrow-datafusion] renato2099 commented on a diff in pull request #1510: Add factorial function
renato2099 commented on code in PR #1510:
URL: https://github.com/apache/arrow-datafusion/pull/1510#discussion_r841278856
##
datafusion/Cargo.toml:
##
@@ -77,6 +77,7 @@ rand = "0.8"
avro-rs = { version = "0.13", features = ["snappy"], optional = true }
num-traits = { version = "0.2", optional = true }
pyo3 = { version = "0.14", optional = true }
+statrs = "0.15"
Review Comment:
thanks for the suggestion, I removed the dependency
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [arrow-datafusion] renato2099 commented on a diff in pull request #1510: Add factorial function
renato2099 commented on code in PR #1510: URL: https://github.com/apache/arrow-datafusion/pull/1510#discussion_r841278823 ## datafusion/src/physical_plan/functions.rs: ## @@ -546,6 +550,7 @@ pub fn return_type( BuiltinScalarFunction::Lpad => utf8_to_str_type(&input_expr_types[0], "lpad"), BuiltinScalarFunction::Ltrim => utf8_to_str_type(&input_expr_types[0], "ltrim"), BuiltinScalarFunction::MD5 => utf8_to_str_type(&input_expr_types[0], "md5"), +BuiltinScalarFunction::Factorial => Ok(DataType::Float64), Review Comment: right but that is at a logical level, i.e., the `numeric` data type is not necessarily a physical data type. That said I am extremely not familiar with datafusion 😅 so if you could point me to some code, I'd appreciate it Also I took a quick look at PG and I think it converts `int64` to the actual data type (`numeric`) but maybe I also missed something in PG's code base https://github.com/postgres/postgres/blob/46ab07ffda9d6c8e63360ded2d4568aa160a7700/src/backend/utils/adt/numeric.c#L3566 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-julia] kou merged pull request #272: Configure repository metadata
kou merged pull request #272: URL: https://github.com/apache/arrow-julia/pull/272 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-julia] kou closed issue #271: Configure repository metadata
kou closed issue #271: URL: https://github.com/apache/arrow-julia/issues/271 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-julia] kou commented on pull request #272: Configure repository metadata
kou commented on pull request #272: URL: https://github.com/apache/arrow-julia/pull/272#issuecomment-1002765711 @quinnj I'll merge this pull request to change notifications to `git...@arrow.apache.org` from `dev@arrow.apache.org`. We can work on extra configurations in follow-up pull requests. Please comment missing configurations to #271. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-julia] kou commented on pull request #272: Configure repository metadata
kou commented on pull request #272: URL: https://github.com/apache/arrow-julia/pull/272#issuecomment-1002764280 @pitrou Could you "approve" this pull request? We can't merge a pull request without at least one "approve" in this repository. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-julia] pitrou commented on pull request #272: Configure repository metadata
pitrou commented on pull request #272: URL: https://github.com/apache/arrow-julia/pull/272#issuecomment-1002763678 Can we get this merged, at least the notifications part? dev@arrow.apache.org is getting notifications for each PR comment... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-julia] pitrou commented on a change in pull request #272: Configure repository metadata
pitrou commented on a change in pull request #272: URL: https://github.com/apache/arrow-julia/pull/272#discussion_r776482306 ## File path: .asf.yaml ## @@ -0,0 +1,42 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features + +notifications: + commits: comm...@arrow.apache.org + issues: iss...@arrow.apache.org + pullrequests: git...@arrow.apache.org + +github: + description: "Apache Arrow Julia" Review comment: ```suggestion description: "Official Julia implementation of Apache Arrow" ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-julia] kou commented on a change in pull request #272: Configure repository metadata
kou commented on a change in pull request #272: URL: https://github.com/apache/arrow-julia/pull/272#discussion_r776482184 ## File path: .asf.yaml ## @@ -0,0 +1,42 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features + +notifications: + commits: comm...@arrow.apache.org + issues: iss...@arrow.apache.org + pullrequests: git...@arrow.apache.org + +github: + description: "Apache Arrow Julia" + homepage: https://arrow.apache.org/ + labels: +- apache-arrow +- julia + features: +issues: true + enabled_merge_buttons: +squash: true +merge: false +rebase: false + protected_branches: +main: + required_linear_history: true + required_pull_request_reviews: +dismiss_stale_reviews: true +required_approving_review_count: 1 Review comment: Thanks. It seems that we can enable the feature by https://docs.github.com/en/rest/reference/repos#update-a-repository API. But the `.asf.yaml` mechanism doesn't support it yet: https://github.com/apache/infrastructure-p6/blob/production/modules/gitbox/files/asfgit/package/asfyaml.py#L234 It seems that the `.asf.yaml` mechanism doesn't change the current configuration. I hope that we can still use the feature. We can confirm it by merging this pull request. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-julia] kou commented on a change in pull request #272: Configure repository metadata
kou commented on a change in pull request #272: URL: https://github.com/apache/arrow-julia/pull/272#discussion_r776481132 ## File path: .asf.yaml ## @@ -0,0 +1,42 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features + +notifications: + commits: comm...@arrow.apache.org + issues: iss...@arrow.apache.org + pullrequests: git...@arrow.apache.org + +github: + description: "Apache Arrow Julia" + homepage: https://arrow.apache.org/ + labels: +- apache-arrow +- julia + features: +issues: true + enabled_merge_buttons: +squash: true +merge: false +rebase: false Review comment: OK. I've enabled rebase. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-julia] quinnj commented on a change in pull request #272: Configure repository metadata
quinnj commented on a change in pull request #272: URL: https://github.com/apache/arrow-julia/pull/272#discussion_r776181901 ## File path: .asf.yaml ## @@ -0,0 +1,42 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features + +notifications: + commits: comm...@arrow.apache.org + issues: iss...@arrow.apache.org + pullrequests: git...@arrow.apache.org + +github: + description: "Apache Arrow Julia" + homepage: https://arrow.apache.org/ + labels: +- apache-arrow +- julia + features: +issues: true + enabled_merge_buttons: +squash: true +merge: false +rebase: false + protected_branches: +main: + required_linear_history: true + required_pull_request_reviews: +dismiss_stale_reviews: true +required_approving_review_count: 1 Review comment: We also had "Automatically delete head branches " feature enabled, but I think that's the only one I don't see in the list here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-julia] quinnj commented on a change in pull request #272: Configure repository metadata
quinnj commented on a change in pull request #272: URL: https://github.com/apache/arrow-julia/pull/272#discussion_r776181151 ## File path: .asf.yaml ## @@ -0,0 +1,42 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features + +notifications: + commits: comm...@arrow.apache.org + issues: iss...@arrow.apache.org + pullrequests: git...@arrow.apache.org + +github: + description: "Apache Arrow Julia" + homepage: https://arrow.apache.org/ + labels: +- apache-arrow +- julia + features: +issues: true + enabled_merge_buttons: +squash: true +merge: false +rebase: false Review comment: We allowed squash and rebase, but not merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-julia] kou commented on a change in pull request #272: Configure repository metadata
kou commented on a change in pull request #272: URL: https://github.com/apache/arrow-julia/pull/272#discussion_r776180085 ## File path: .asf.yaml ## @@ -0,0 +1,42 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features + +notifications: + commits: comm...@arrow.apache.org + issues: iss...@arrow.apache.org + pullrequests: git...@arrow.apache.org + +github: + description: "Apache Arrow Julia" + homepage: https://arrow.apache.org/ + labels: +- apache-arrow +- julia + features: +issues: true + enabled_merge_buttons: +squash: true +merge: false +rebase: false + protected_branches: +main: + required_linear_history: true + required_pull_request_reviews: +dismiss_stale_reviews: true +required_approving_review_count: 1 Review comment: @quinnj It seems that JuliaData/Arrow.jl uses protected branch feature but I don't know the configuration. Are there some missing configurations? See also https://docs.github.com/en/rest/reference/branches#update-branch-protection for available configurations. ## File path: .asf.yaml ## @@ -0,0 +1,42 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features + +notifications: + commits: comm...@arrow.apache.org + issues: iss...@arrow.apache.org + pullrequests: git...@arrow.apache.org + +github: + description: "Apache Arrow Julia" + homepage: https://arrow.apache.org/ + labels: +- apache-arrow +- julia + features: +issues: true + enabled_merge_buttons: +squash: true +merge: false +rebase: false Review comment: @quinnj It seems that JuliaData/Arrow.jl uses only squash merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-julia] kou opened a new pull request #272: Configure repository metadata
kou opened a new pull request #272: URL: https://github.com/apache/arrow-julia/pull/272 fix #271 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-julia] kou opened a new issue #271: Configure repository metadata
kou opened a new issue #271: URL: https://github.com/apache/arrow-julia/issues/271 We can configure this repository's metadata by `.asf.yaml`. See also: https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features We should change at least notification configurations. All issue changes will be notified to dev@arrow.apache.org with the default configuration. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-julia] kou opened a new issue #270: Document the Apache way development process
kou opened a new issue #270: URL: https://github.com/apache/arrow-julia/issues/270 JuliaData/Arrow.jl has been transferred to apache/arrow-julia. See also: #265 and https://issues.apache.org/jira/browse/INFRA-22663 We discussed the development process for apache/arrow-julia on dev@arrow.apache.org. See the VOTE thread for summary of it: https://lists.apache.org/thread/6q333y875v7mfyl2g988b01hqtgr52pt apache/arrow-julia uses apache/arrow-rs like development process: https://docs.google.com/document/d/1TyrUP8_UWXqk97a8Hvb1d0UYWigch0HAephIjW7soSI/edit Note: Contributors will be required to write issues for planned features and bug fixes so that we have visibility and opportunities for collaboration before a PR shows up. This is for the Apache way. See also http://theapacheway.com/ for the Apache way. For example, apache/arrow-rs mentions it as comment in https://raw.githubusercontent.com/apache/arrow-rs/master/.github/pull_request_template.md . We need something like this to keep the Apache way development process. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] pitrou commented on issue #6984: ARROW-8520: [Developer] Use .asf.yaml to direct GitHub notifications to JIRA and mailing lists
pitrou commented on issue #6984: URL: https://github.com/apache/arrow/pull/6984#issuecomment-616237297 Wow, thank you for solving this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] wesm commented on issue #6984: ARROW-8520: [Developer] Use .asf.yaml to direct GitHub notifications to JIRA and mailing lists
wesm commented on issue #6984: URL: https://github.com/apache/arrow/pull/6984#issuecomment-616237030 +1. This can only be "tested in production" so I'm merging this This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] wesm opened a new pull request #6984: ARROW-8520: [Developer] Use .asf.yaml to direct GitHub notifications to JIRA and mailing lists
wesm opened a new pull request #6984: URL: https://github.com/apache/arrow/pull/6984 Context: INFRA-20149 Following instructions per https://s.apache.org/asfyaml-notify This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] wesm commented on issue #6984: ARROW-8520: [Developer] Use .asf.yaml to direct GitHub notifications to JIRA and mailing lists
wesm commented on issue #6984: URL: https://github.com/apache/arrow/pull/6984#issuecomment-616236838 I just requested the creation of github@ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on issue #6983: ARROW-8519: [C++][Packaging] Reduce disk usage for external projects
github-actions[bot] commented on issue #6983: URL: https://github.com/apache/arrow/pull/6983#issuecomment-616234088 https://issues.apache.org/jira/browse/ARROW-8519 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on issue #6983: ARROW-8519: [C++][Packaging] Reduce disk usage for external projects
github-actions[bot] commented on issue #6983: URL: https://github.com/apache/arrow/pull/6983#issuecomment-616233760 Revision: ac4a31458b83fa39e659c2a0551a07f4f645bcbb Submitted crossbow builds: [ursa-labs/crossbow @ actions-148](https://github.com/ursa-labs/crossbow/branches/all?query=actions-148) |Task|Status| ||--| |centos-6-amd64|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-148-github-centos-6-amd64)| |centos-7-amd64|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-148-github-centos-7-amd64)| |centos-8-amd64|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-148-github-centos-8-amd64)| |debian-buster-amd64|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-148-github-debian-buster-amd64)| |debian-stretch-amd64|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-148-github-debian-stretch-amd64)| |ubuntu-bionic-amd64|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-148-github-ubuntu-bionic-amd64)| |ubuntu-eoan-amd64|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-148-github-ubuntu-eoan-amd64)| |ubuntu-focal-amd64|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-148-github-ubuntu-focal-amd64)| |ubuntu-xenial-amd64|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-148-github-ubuntu-xenial-amd64)| This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] kou commented on issue #6983: ARROW-8519: [C++][Packaging] Reduce disk usage for external projects
kou commented on issue #6983: URL: https://github.com/apache/arrow/pull/6983#issuecomment-616233577 @github-actions crossbow submit -g linux This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow] kou opened a new pull request #6983: ARROW-8519: [C++][Packaging] Reduce disk usage for external projects
kou opened a new pull request #6983: URL: https://github.com/apache/arrow/pull/6983 This change removes built files after installing built files for external projects. It reduces disk usage for external projects. This may resolve "No space left on device" caused by building Linux packages on GitHub Actions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-site] nealrichardson commented on a change in pull request #7: ARROW-6139: [Documentation][R] Build R docs (pkgdown) site and add to arrow-site
nealrichardson commented on a change in pull request #7: ARROW-6139:
[Documentation][R] Build R docs (pkgdown) site and add to arrow-site
URL: https://github.com/apache/arrow-site/pull/7#discussion_r310816056
##
File path: assets/.sprockets-manifest-af5b3e33562477ba4f207e3ea1e798c0.json
##
@@ -0,0 +1 @@
+{}
Review comment:
This is autogenerated by jekyll. There's some sorcery in the
Jekyll/bootstrap asset rendering that I intend to clean up once
https://lists.apache.org/thread.html/73e8a7d2ee667e83371e8bb99c13338144c8af5444f55918994713be@%3Cdev.arrow.apache.org%3E
is resolved.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [arrow-site] kou commented on a change in pull request #7: ARROW-6139: [Documentation][R] Build R docs (pkgdown) site and add to arrow-site
kou commented on a change in pull request #7: ARROW-6139: [Documentation][R]
Build R docs (pkgdown) site and add to arrow-site
URL: https://github.com/apache/arrow-site/pull/7#discussion_r310807654
##
File path: assets/.sprockets-manifest-af5b3e33562477ba4f207e3ea1e798c0.json
##
@@ -0,0 +1 @@
+{}
Review comment:
Should we remove this?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [arrow-site] wesm closed pull request #8: ARROW-4473: [Website] Support test site deployment
wesm closed pull request #8: ARROW-4473: [Website] Support test site deployment URL: https://github.com/apache/arrow-site/pull/8 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [arrow-site] fsaintjacques commented on a change in pull request #8: ARROW-4473: [Website] Support test site deployment
fsaintjacques commented on a change in pull request #8: ARROW-4473: [Website]
Support test site deployment
URL: https://github.com/apache/arrow-site/pull/8#discussion_r310730531
##
File path: build-and-deploy.sh
##
@@ -0,0 +1,51 @@
+#!/bin/bash
+set -ev
+
+if [ "${TRAVIS_BRANCH}" = "master" ] && [ "${TRAVIS_PULL_REQUEST}" = "false"
]; then
+
+if [ -z "${GITHUB_PAT}" ]; then
+# Don't build because we can't publish
+echo "To publish the site, you must set a GITHUB_PAT at"
+echo "https://travis-ci.org/"${TRAVIS_REPO_SLUG}"/settings";
+exit 1
+fi
+
+# Set git config so that the author of the deployed site commit is the same
+# as the author of the commit we're building
+export AUTHOR_EMAIL=$(git log -1 --pretty=format:%ae)
+export AUTHOR_NAME=$(git log -1 --pretty=format:%an)
+git config --global user.email "${AUTHOR_EMAIL}"
+git config --global user.name "${AUTHOR_NAME}"
+
+if [ "${TRAVIS_REPO_SLUG}" = "apache/arrow-site" ]; then
+# Production
+export TARGET_BRANCH=asf-site
+export BASE_URL=https://arrow.apache.org
+else
+# On a fork, so we'll deploy to GitHub Pages
+export TARGET_BRANCH=gh-pages
+# You could supply an alternate BASE_URL, but that's not necessary
+# because we can infer it based on GitHub Pages conventions
+if [ -z "${BASE_URL}" ]; then
+export BASE_URL="https://"$(echo $TRAVIS_REPO_SLUG | sed
's@/@.github.io/@')
+fi
+fi
+# Set the site.baseurl
+perl -pe 's@^baseurl.*@baseurl: '"${BASE_URL}"'@' -i _config.yml
+
+# Build
+gem install jekyll bundler
+bundle install
+JEKYLL_ENV=production bundle exec jekyll build
+
+# Publish
+git clone -b ${TARGET_BRANCH}
https://${GITHUB_PAT}@github.com/$TRAVIS_REPO_SLUG.git OUTPUT
+rsync -r build/ OUTPUT/
+cd OUTPUT
+
+git add .
Review comment:
Because it swallows the return code with true, normally such error would
bubble up. Makes it easier to chain/re-use the script.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [arrow-site] nealrichardson commented on a change in pull request #8: ARROW-4473: [Website] Support test site deployment
nealrichardson commented on a change in pull request #8: ARROW-4473: [Website]
Support test site deployment
URL: https://github.com/apache/arrow-site/pull/8#discussion_r310729936
##
File path: build-and-deploy.sh
##
@@ -0,0 +1,51 @@
+#!/bin/bash
+set -ev
+
+if [ "${TRAVIS_BRANCH}" = "master" ] && [ "${TRAVIS_PULL_REQUEST}" = "false"
]; then
+
+if [ -z "${GITHUB_PAT}" ]; then
+# Don't build because we can't publish
+echo "To publish the site, you must set a GITHUB_PAT at"
+echo "https://travis-ci.org/"${TRAVIS_REPO_SLUG}"/settings";
+exit 1
+fi
+
+# Set git config so that the author of the deployed site commit is the same
+# as the author of the commit we're building
+export AUTHOR_EMAIL=$(git log -1 --pretty=format:%ae)
+export AUTHOR_NAME=$(git log -1 --pretty=format:%an)
+git config --global user.email "${AUTHOR_EMAIL}"
+git config --global user.name "${AUTHOR_NAME}"
+
+if [ "${TRAVIS_REPO_SLUG}" = "apache/arrow-site" ]; then
+# Production
+export TARGET_BRANCH=asf-site
+export BASE_URL=https://arrow.apache.org
+else
+# On a fork, so we'll deploy to GitHub Pages
+export TARGET_BRANCH=gh-pages
+# You could supply an alternate BASE_URL, but that's not necessary
+# because we can infer it based on GitHub Pages conventions
+if [ -z "${BASE_URL}" ]; then
+export BASE_URL="https://"$(echo $TRAVIS_REPO_SLUG | sed
's@/@.github.io/@')
+fi
+fi
+# Set the site.baseurl
+perl -pe 's@^baseurl.*@baseurl: '"${BASE_URL}"'@' -i _config.yml
+
+# Build
+gem install jekyll bundler
+bundle install
+JEKYLL_ENV=production bundle exec jekyll build
+
+# Publish
+git clone -b ${TARGET_BRANCH}
https://${GITHUB_PAT}@github.com/$TRAVIS_REPO_SLUG.git OUTPUT
+rsync -r build/ OUTPUT/
+cd OUTPUT
+
+git add .
Review comment:
What is the race problem? If `git push` fails because GitHub is down, then
the build will complete without pushing. That seems reasonable under the
circumstances.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [arrow-site] fsaintjacques commented on a change in pull request #8: ARROW-4473: [Website] Support test site deployment
fsaintjacques commented on a change in pull request #8: ARROW-4473: [Website]
Support test site deployment
URL: https://github.com/apache/arrow-site/pull/8#discussion_r310724697
##
File path: build-and-deploy.sh
##
@@ -0,0 +1,51 @@
+#!/bin/bash
+set -ev
+
+if [ "${TRAVIS_BRANCH}" = "master" ] && [ "${TRAVIS_PULL_REQUEST}" = "false"
]; then
+
+if [ -z "${GITHUB_PAT}" ]; then
+# Don't build because we can't publish
+echo "To publish the site, you must set a GITHUB_PAT at"
+echo "https://travis-ci.org/"${TRAVIS_REPO_SLUG}"/settings";
+exit 1
+fi
+
+# Set git config so that the author of the deployed site commit is the same
+# as the author of the commit we're building
+export AUTHOR_EMAIL=$(git log -1 --pretty=format:%ae)
+export AUTHOR_NAME=$(git log -1 --pretty=format:%an)
+git config --global user.email "${AUTHOR_EMAIL}"
+git config --global user.name "${AUTHOR_NAME}"
+
+if [ "${TRAVIS_REPO_SLUG}" = "apache/arrow-site" ]; then
+# Production
+export TARGET_BRANCH=asf-site
+export BASE_URL=https://arrow.apache.org
+else
+# On a fork, so we'll deploy to GitHub Pages
+export TARGET_BRANCH=gh-pages
+# You could supply an alternate BASE_URL, but that's not necessary
+# because we can infer it based on GitHub Pages conventions
+if [ -z "${BASE_URL}" ]; then
+export BASE_URL="https://"$(echo $TRAVIS_REPO_SLUG | sed
's@/@.github.io/@')
+fi
+fi
+# Set the site.baseurl
+perl -pe 's@^baseurl.*@baseurl: '"${BASE_URL}"'@' -i _config.yml
+
+# Build
+gem install jekyll bundler
+bundle install
+JEKYLL_ENV=production bundle exec jekyll build
+
+# Publish
+git clone -b ${TARGET_BRANCH}
https://${GITHUB_PAT}@github.com/$TRAVIS_REPO_SLUG.git OUTPUT
+rsync -r build/ OUTPUT/
+cd OUTPUT
+
+git add .
Review comment:
Wrap the git stuff in a block guarded by a check with `git status
--porcelain | wc -l`, otherwise there's a race if git push fails for a real
reason, e.g. github being down.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [arrow-site] fsaintjacques commented on a change in pull request #8: ARROW-4473: [Website] Support test site deployment
fsaintjacques commented on a change in pull request #8: ARROW-4473: [Website]
Support test site deployment
URL: https://github.com/apache/arrow-site/pull/8#discussion_r310727786
##
File path: build-and-deploy.sh
##
@@ -0,0 +1,51 @@
+#!/bin/bash
+set -ev
+
+if [ "${TRAVIS_BRANCH}" = "master" ] && [ "${TRAVIS_PULL_REQUEST}" = "false"
]; then
+
+if [ -z "${GITHUB_PAT}" ]; then
+# Don't build because we can't publish
+echo "To publish the site, you must set a GITHUB_PAT at"
+echo "https://travis-ci.org/"${TRAVIS_REPO_SLUG}"/settings";
+exit 1
+fi
+
+# Set git config so that the author of the deployed site commit is the same
+# as the author of the commit we're building
+export AUTHOR_EMAIL=$(git log -1 --pretty=format:%ae)
+export AUTHOR_NAME=$(git log -1 --pretty=format:%an)
+git config --global user.email "${AUTHOR_EMAIL}"
+git config --global user.name "${AUTHOR_NAME}"
+
+if [ "${TRAVIS_REPO_SLUG}" = "apache/arrow-site" ]; then
+# Production
+export TARGET_BRANCH=asf-site
+export BASE_URL=https://arrow.apache.org
+else
+# On a fork, so we'll deploy to GitHub Pages
+export TARGET_BRANCH=gh-pages
+# You could supply an alternate BASE_URL, but that's not necessary
+# because we can infer it based on GitHub Pages conventions
+if [ -z "${BASE_URL}" ]; then
+export BASE_URL="https://"$(echo $TRAVIS_REPO_SLUG | sed
's@/@.github.io/@')
+fi
+fi
+# Set the site.baseurl
+perl -pe 's@^baseurl.*@baseurl: '"${BASE_URL}"'@' -i _config.yml
Review comment:
Why not sed?
```
01:arrow-site/ (master) $ grep -E '^baseurl' _config.yml
baseurl:
01:arrow-site/ (master) $ sed -i -e's#^baseurl:.*#baseurl: HI#' _config.yml
01:arrow-site/ (master✗) $ grep -E '^baseurl' _config.yml
baseurl: HI
01:arrow-site/ (master✗) $ sed -i -e's#^baseurl:.*#baseurl: NOPENOPENOPE#'
_config.yml
01:arrow-site/ (master✗) $ grep -E '^baseurl' _config.yml
baseurl: NOPENOPENOPE
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [arrow-site] nealrichardson opened a new pull request #8: ARROW-4473: [Website] Support test site deployment
nealrichardson opened a new pull request #8: ARROW-4473: [Website] Support test site deployment URL: https://github.com/apache/arrow-site/pull/8 See https://lists.apache.org/thread.html/73e8a7d2ee667e83371e8bb99c13338144c8af5444f55918994713be@%3Cdev.arrow.apache.org%3E This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [arrow-site] nealrichardson commented on issue #7: ARROW-6139: [Documentation][R] Build R docs (pkgdown) site and add to arrow-site
nealrichardson commented on issue #7: ARROW-6139: [Documentation][R] Build R docs (pkgdown) site and add to arrow-site URL: https://github.com/apache/arrow-site/pull/7#issuecomment-518334974 > Currently there are no links to and from the main arrow.apache.org/docs/ site, but I'd like to take that up in a followup issue because there's more cleanup to do there than just adding a link. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [arrow-site] fsaintjacques commented on issue #7: ARROW-6139: [Documentation][R] Build R docs (pkgdown) site and add to arrow-site
fsaintjacques commented on issue #7: ARROW-6139: [Documentation][R] Build R docs (pkgdown) site and add to arrow-site URL: https://github.com/apache/arrow-site/pull/7#issuecomment-518332661 I don't see the R documentation in the drop-down menu of documentations. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [arrow-site] nealrichardson commented on issue #7: ARROW-6139: [Documentation][R] Build R docs (pkgdown) site and add to arrow-site
nealrichardson commented on issue #7: ARROW-6139: [Documentation][R] Build R docs (pkgdown) site and add to arrow-site URL: https://github.com/apache/arrow-site/pull/7#issuecomment-518327887 Full site preview at https://enpiar.com/arrow-site/ and R docs at https://enpiar.com/arrow-site/docs/r/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [arrow-site] nealrichardson opened a new pull request #7: ARROW-6139: [Documentation][R] Build R docs (pkgdown) site and add to arrow-site
nealrichardson opened a new pull request #7: ARROW-6139: [Documentation][R] Build R docs (pkgdown) site and add to arrow-site URL: https://github.com/apache/arrow-site/pull/7 This renders the site that was configured in https://issues.apache.org/jira/browse/ARROW-5452, using the 0.14.1 R CRAN (so-called) "release" branch. Currently there are no links to and from the main arrow.apache.org/docs/ site, but I'd like to take that up in a followup issue because there's more cleanup to do there than just adding a link. This patch also includes the last two patches to the website source in apache/arrow, which have not yet been published, and it has some relative URL fixes from my fork of the site source. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [arrow-site] wesm merged pull request #6: Remove very stale docs/latest
wesm merged pull request #6: Remove very stale docs/latest URL: https://github.com/apache/arrow-site/pull/6 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [arrow-site] wesm commented on issue #6: Remove very stale docs/latest
wesm commented on issue #6: Remove very stale docs/latest URL: https://github.com/apache/arrow-site/pull/6#issuecomment-501073903 +1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [arrow-site] nealrichardson opened a new pull request #6: Remove very stale docs/latest
nealrichardson opened a new pull request #6: Remove very stale docs/latest URL: https://github.com/apache/arrow-site/pull/6 See discussion on https://issues.apache.org/jira/browse/ARROW-5548 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [arrow-site] riboflavin closed pull request #5: [Website] Update website for Gandiva donation
riboflavin closed pull request #5: [Website] Update website for Gandiva donation URL: https://github.com/apache/arrow-site/pull/5 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] BryanCutler commented on issue #19: Build 0.8.0 wheels
BryanCutler commented on issue #19: Build 0.8.0 wheels URL: https://github.com/apache/arrow-dist/pull/19#issuecomment-352680792 Thanks for fixing this @xhochy! I uploaded the macOS wheels for pip after this update This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kou closed pull request #20: Support Arrow C++ and Arrow GLib 0.8.0 packages for Debian, Ubuntu and CentOS
kou closed pull request #20: Support Arrow C++ and Arrow GLib 0.8.0 packages
for Debian, Ubuntu and CentOS
URL: https://github.com/apache/arrow-dist/pull/20
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/cpp-linux/apt/debian-stretch/Dockerfile
b/cpp-linux/apt/debian-stretch/Dockerfile
index 797cc1b..815349f 100644
--- a/cpp-linux/apt/debian-stretch/Dockerfile
+++ b/cpp-linux/apt/debian-stretch/Dockerfile
@@ -27,6 +27,7 @@ RUN \
quiet=$([ "${DEBUG}" = "yes" ] || echo "-qq") && \
apt update ${quiet} && \
apt install -y -V ${quiet} \
+autoconf-archive \
build-essential \
cmake \
debhelper\
diff --git a/cpp-linux/apt/ubuntu-artful/Dockerfile
b/cpp-linux/apt/ubuntu-artful/Dockerfile
index 6daea3a..9cf779b 100644
--- a/cpp-linux/apt/ubuntu-artful/Dockerfile
+++ b/cpp-linux/apt/ubuntu-artful/Dockerfile
@@ -25,6 +25,7 @@ RUN \
quiet=$([ "${DEBUG}" = "yes" ] || echo "-qq") && \
apt update ${quiet} && \
apt install -y -V ${quiet} \
+autoconf-archive \
build-essential \
cmake \
debhelper\
diff --git a/cpp-linux/apt/ubuntu-xenial/Dockerfile
b/cpp-linux/apt/ubuntu-xenial/Dockerfile
index 85d1e1a..86dc7d3 100644
--- a/cpp-linux/apt/ubuntu-xenial/Dockerfile
+++ b/cpp-linux/apt/ubuntu-xenial/Dockerfile
@@ -25,10 +25,12 @@ RUN \
quiet=$([ "${DEBUG}" = "yes" ] || echo "-qq") && \
apt update ${quiet} && \
apt install -y -V ${quiet} \
+autoconf-archive \
build-essential \
cmake \
debhelper\
devscripts \
+dh-autoreconf \
git \
gtk-doc-tools \
libboost-filesystem-dev \
diff --git a/cpp-linux/apt/ubuntu-zesty/Dockerfile
b/cpp-linux/apt/ubuntu-zesty/Dockerfile
index 03b75c8..f652fed 100644
--- a/cpp-linux/apt/ubuntu-zesty/Dockerfile
+++ b/cpp-linux/apt/ubuntu-zesty/Dockerfile
@@ -25,6 +25,7 @@ RUN \
quiet=$([ "${DEBUG}" = "yes" ] || echo "-qq") && \
apt update ${quiet} && \
apt install -y -V ${quiet} \
+autoconf-archive \
build-essential \
cmake \
debhelper\
diff --git a/cpp-linux/debian/changelog b/cpp-linux/debian/changelog
index 852bac9..691acb7 100644
--- a/cpp-linux/debian/changelog
+++ b/cpp-linux/debian/changelog
@@ -2,6 +2,8 @@ apache-arrow (0.8.0-1) unstable; urgency=low
* New upstream release.
+ * Add libarrow-gpu-glib0, libarrow-gpu-glib-dev and gir1.2-arrow-gpu-1.0.
+
-- Uwe L. Korn Sun, 17 Dec 2017 20:24:44 -
apache-arrow (0.7.1-2) unstable; urgency=low
diff --git a/cpp-linux/debian/compat b/cpp-linux/debian/compat
index ec63514..f599e28 100644
--- a/cpp-linux/debian/compat
+++ b/cpp-linux/debian/compat
@@ -1 +1 @@
-9
+10
diff --git a/cpp-linux/debian/control b/cpp-linux/debian/control
index 26b3b10..5b3fa6f 100644
--- a/cpp-linux/debian/control
+++ b/cpp-linux/debian/control
@@ -3,7 +3,9 @@ Section: devel
Priority: optional
Maintainer: Kouhei Sutou
Build-Depends:
- debhelper (>= 9),
+ debhelper (>= 9.20160115),
+ dh-autoreconf,
+ autoconf-archive,
pkg-config,
cmake,
git,
@@ -147,3 +149,43 @@ Recommends: libglib2.0-doc
Description: Apache Arrow is a data processing library for analysis
.
This package provides documentations.
+
+Package: libarrow-gpu-glib0
+Section: libs
+Architecture: any
+Multi-Arch: same
+Pre-Depends: ${misc:Pre-Depends}
+Depends:
+ ${misc:Depends},
+ ${shlibs:Depends},
+ libarrow-glib0 (= ${binary:Version}),
+ libarrow-gpu0 (= ${binary:Version})
+Description: Apache Arrow is a data processing library for analysis
+ .
+ This package provides library files for GPU support.
+
+Package: gir1.2-arrow-gpu-1.0
+Section: introspection
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${gir:Depends},
+ ${misc:Depends}
+Description: Apache Arrow is a data processing library for analysis
+ .
+ This package provides GObject Introspection typelib files for GPU support.
+
+Package: libarrow-gpu-glib-dev
+Section: libdevel
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${misc:Depends},
+ libarrow-dev (= ${binary:Version}),
+ libarrow-glib-dev (= ${binary:Version}),
+ libarrow-gpu-dev (= ${binary:Version}),
+ libarrow-gpu-glib0 (= ${binary:Version}),
+ gir1.2-arrow-gpu-1.0 (= ${binary:Version})
+Description: Apache Arrow is a data processing library for analysis
+ .
+ This package provides header files for GPU support.
diff --git a/cpp-linux/debian/gir1.2-arrow-1.0.install
b/cpp-linux/debian/gir1.2-arrow-1.0.install
index 9038727..e0197fc 100644
--- a/cpp-linux/debian/gir1.2-arrow-1.0.install
+++ b/cpp-linux/debian/gir1.2-arrow-1.0.install
@@ -1 +1 @@
-usr/lib/*/girepository-1.0/
+usr/lib/*/girepository-1.0/Arrow-1.0.typelib
diff --git a/cpp-linux/debian/gir1.2-arrow-gpu-1.0.install
b/cpp-linux/debian/gir1.2-arrow-gpu-1.0.install
new file mode 100644
index 000..10e0ca9
--- /dev
[GitHub] kou commented on issue #20: Support Arrow C++ and Arrow GLib 0.8.0 packages for Debian, Ubuntu and CentOS
kou commented on issue #20: Support Arrow C++ and Arrow GLib 0.8.0 packages for Debian, Ubuntu and CentOS URL: https://github.com/apache/arrow-dist/pull/20#issuecomment-352669961 +1 All builds are green. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kou opened a new pull request #20: Support Arrow C++ and Arrow GLib 0.8.0 packages for Debian, Ubuntu and CentOS
kou opened a new pull request #20: Support Arrow C++ and Arrow GLib 0.8.0 packages for Debian, Ubuntu and CentOS URL: https://github.com/apache/arrow-dist/pull/20 Arrow GLib packages for Debian and Ubuntu support GPU. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] xhochy commented on issue #19: Build 0.8.0 wheels
xhochy commented on issue #19: Build 0.8.0 wheels URL: https://github.com/apache/arrow-dist/pull/19#issuecomment-352554021 Merging as all builds except the APT/YUM packages passed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] xhochy closed pull request #19: Build 0.8.0 wheels
xhochy closed pull request #19: Build 0.8.0 wheels
URL: https://github.com/apache/arrow-dist/pull/19
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/.travis.yml b/.travis.yml
index 6431e31..389c6d5 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,13 +1,11 @@
env:
global:
- PLAT=x86_64
- - BUILD_REF=apache-arrow-0.7.1
+ - BUILD_REF=apache-arrow-0.8.0
language: python
python: 3.5
sudo: required
dist: trusty
-services:
-- docker
matrix:
exclude:
- python: 3.5
@@ -35,6 +33,8 @@ matrix:
- PANDAS_DEP="0.20.3"
- language: cpp
os: linux
+services:
+- docker
before_script:
- docker pull quay.io/xhochy/arrow_manylinux1_x86_64_base:latest
before_install:
@@ -94,7 +94,8 @@ matrix:
key:
secure:
"XXRJFBIOGS9urF047KcGlCqnMwKWa/EhW1Q2A3JGDd7Au+NHd99Rl/iRbtd44xVnfhosKGEH9mBGrqIZb/Tfn3RTtSi/KvN2RIoWmdsc3Kxk1eKd6O9kCLo4Y1GqboK8JVIHD4OYiyoVoFmnYLuinfdtrZrME5aU4pENnvwR3E0qvjhnhfwRG/Mtb93ePAqzih95gT7S0VWlsBV3FPa21aY7ctFteUP+DdqXZX1BmUnJB6cBxUGjpL8Bu8O/mt3l3yIrT9UkWM1g75l7zrP354x5+AMbHWBv7yGFa+JvbB+LXkrE2dV6j3Upf6ChnQKoTKv5Ur5LJBbCHyRieW3/uE/h47IR3c/3mRnhoSIl4QX7UoQkvTwHSE1Xpry3I2jE0WVHmq9NqT4iaxIv5oga+3I/W6h8brV3yCYfVUi1F90aRlnZHsCDmtAqzW1u9s1xqkO4z4/dennI0gniEn4kWfx0I7aCQQI/XmDri/33uTINprwBB5eHt6a7WGysrcO6Mv5I8EFEAJ5vkx2mDVw+rDFC7/R8YNA77PcJABlF7G/ZajcHqPnN5uyJC+P2m6JyHO+kTHemjW8pgf5vKvdNzdTN1AVFSO9Mv4O3NPyaADRl9DCCcqAk2LwL+SYqCVvIXu98Mb6uEoSOb+ZnpvseyydxNB6DtQSVtVUNdis5Ibw="
before_install:
-- brew uninstall boost
+# Also remove artefacts that depend on Boost
+- brew uninstall boost cgal postgis sfcgal
- brew update
- brew upgrade cmake
- BUILD_DEPENDS="numpy==$NP_BUILD_DEP six cython"
@@ -104,8 +105,11 @@ before_install:
- source multibuild/common_utils.sh
- source multibuild/travis_steps.sh
- before_install
-- popd
+- unset -f cd
+- cd ..
install:
+- unset -f pushd
+- unset -f popd
- clean_code arrow $BUILD_REF
- build_wheel arrow $PLAT
- mv arrow/python/dist/* dist/
diff --git a/appveyor.yml b/appveyor.yml
index 040808a..b910a8d 100644
--- a/appveyor.yml
+++ b/appveyor.yml
@@ -31,9 +31,9 @@ environment:
MSVC_DEFAULT_OPTIONS: ON
BOOST_ROOT: C:\Libraries\boost_1_63_0
BOOST_LIBRARYDIR: C:\Libraries\boost_1_63_0\lib64-msvc-14.0
- pyarrow_version: 0.7.1
- pyarrow_commit: 0e21f84c2fc26dba949a03ee7d7ebfade0a65b81
- parquet_commit: b69669a230e59e36a193b679fc2c5b4d9150f8c9
+ pyarrow_version: 0.8.0
+ pyarrow_commit: 1d689e5a3e417d3a4b3de16fea7a6fe195c8f8df
+ parquet_commit: e2814ccb224a01289ad3a736ce89a956cb7d52fb
init:
- set MINICONDA=C:\Miniconda35-x64
diff --git a/arrow b/arrow
index 0e21f84..1d689e5 16
--- a/arrow
+++ b/arrow
@@ -1 +1 @@
-Subproject commit 0e21f84c2fc26dba949a03ee7d7ebfade0a65b81
+Subproject commit 1d689e5a3e417d3a4b3de16fea7a6fe195c8f8df
diff --git a/cpp-linux/apt/descriptor.json b/cpp-linux/apt/descriptor.json
index 9e1ff7a..cc94323 100644
--- a/cpp-linux/apt/descriptor.json
+++ b/cpp-linux/apt/descriptor.json
@@ -7,7 +7,7 @@
"vcs_url": "htttps://github.com/apache/arrow.git"
},
"version": {
-"name": "0.7.1"
+"name": "0.8.0"
},
"files": [
{
diff --git a/cpp-linux/debian.ubuntu-trusty/changelog
b/cpp-linux/debian.ubuntu-trusty/changelog
index 9317adc..95f470d 100644
--- a/cpp-linux/debian.ubuntu-trusty/changelog
+++ b/cpp-linux/debian.ubuntu-trusty/changelog
@@ -1,3 +1,9 @@
+apache-arrow (0.8.0-1) unstable; urgency=low
+
+ * New upstream release.
+
+ -- Uwe L. Korn Sun, 17 Dec 2017 20:24:44 -
+
apache-arrow (0.7.1-1) unstable; urgency=low
* New upstream release.
diff --git a/cpp-linux/debian/changelog b/cpp-linux/debian/changelog
index 601579b..852bac9 100644
--- a/cpp-linux/debian/changelog
+++ b/cpp-linux/debian/changelog
@@ -1,3 +1,9 @@
+apache-arrow (0.8.0-1) unstable; urgency=low
+
+ * New upstream release.
+
+ -- Uwe L. Korn Sun, 17 Dec 2017 20:24:44 -
+
apache-arrow (0.7.1-2) unstable; urgency=low
* Add libarrow-gpu0 and libarrow-gpu-dev.
diff --git a/cpp-linux/yum/arrow.spec.in b/cpp-linux/yum/arrow.spec.in
index 71e2afb..85405d3 100644
--- a/cpp-linux/yum/arrow.spec.in
+++ b/cpp-linux/yum/arrow.spec.in
@@ -220,6 +220,9 @@ Documentation for Apache Arrow GLib.
%endif
%changelog
+* Sun Dec 17 2017 Uwe Korn - 0.8.0-1
+- New upstream release.
+
* Wed Sep 27 2017 Kouhei Sutou - 0.7.1-1
- New upstream release.
diff --git a/cpp-linux/yum/descriptor.json b/cpp-linux/yum/descriptor.json
index 22152ed..2c0ea47 100644
--- a/cpp-linux/yum/descriptor.json
+++ b/cpp-linux/yum/descriptor.json
@@ -7,7 +7,7 @@
"vcs_url": "htttps://github.com/apache/arrow.git"
},
"version": {
-"name": "0.7.1"
+"name": "0.8.0"
},
"files": [
{
diff --git
[GitHub] xhochy commented on issue #19: Build 0.8.0 wheels
xhochy commented on issue #19: Build 0.8.0 wheels URL: https://github.com/apache/arrow-dist/pull/19#issuecomment-35240 @matthew-brett Awesome, that was the problem! Wheels now build. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] matthew-brett commented on issue #19: Build 0.8.0 wheels
matthew-brett commented on issue #19: Build 0.8.0 wheels URL: https://github.com/apache/arrow-dist/pull/19#issuecomment-352396398 I wonder if you're hitting https://github.com/travis-ci/travis-ci/issues/8703#issuecomment-348866918 ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kou commented on issue #19: Build 0.8.0 wheels
kou commented on issue #19: Build 0.8.0 wheels URL: https://github.com/apache/arrow-dist/pull/19#issuecomment-352379958 I'll put a patch to fix deb/rpm package builds later. It's OK to me that this change is merged my patch because I can put my change after this change is merged. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] wesm commented on issue #19: Build 0.8.0 wheels
wesm commented on issue #19: Build 0.8.0 wheels URL: https://github.com/apache/arrow-dist/pull/19#issuecomment-352327398 Most strange, perhaps @matthew-brett could advise on the macOS failures? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] xhochy opened a new pull request #19: Build 0.8.0 wheels
xhochy opened a new pull request #19: Build 0.8.0 wheels URL: https://github.com/apache/arrow-dist/pull/19 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] xhochy closed pull request #18: ARROW-1752: Add GPU packages for Debian and Ubuntu
xhochy closed pull request #18: ARROW-1752: Add GPU packages for Debian and
Ubuntu
URL: https://github.com/apache/arrow-dist/pull/18
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/.travis.yml b/.travis.yml
index 098afe6..6431e31 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -54,7 +54,22 @@ matrix:
before_install:
install:
script:
-- (cd cpp-linux && travis_wait 40 rake apt:build PARALLEL=yes DEBUG=no)
+- (cd cpp-linux && travis_wait 40 rake apt:build
APT_TARGETS=debian-stretch,ubuntu-trusty,ubuntu-xenial PARALLEL=yes DEBUG=no)
+deploy:
+ provider: bintray
+ file: cpp-linux/apt/descriptor.json
+ user: kou
+ on:
+all_branches: true
+ key:
+secure:
"XXRJFBIOGS9urF047KcGlCqnMwKWa/EhW1Q2A3JGDd7Au+NHd99Rl/iRbtd44xVnfhosKGEH9mBGrqIZb/Tfn3RTtSi/KvN2RIoWmdsc3Kxk1eKd6O9kCLo4Y1GqboK8JVIHD4OYiyoVoFmnYLuinfdtrZrME5aU4pENnvwR3E0qvjhnhfwRG/Mtb93ePAqzih95gT7S0VWlsBV3FPa21aY7ctFteUP+DdqXZX1BmUnJB6cBxUGjpL8Bu8O/mt3l3yIrT9UkWM1g75l7zrP354x5+AMbHWBv7yGFa+JvbB+LXkrE2dV6j3Upf6ChnQKoTKv5Ur5LJBbCHyRieW3/uE/h47IR3c/3mRnhoSIl4QX7UoQkvTwHSE1Xpry3I2jE0WVHmq9NqT4iaxIv5oga+3I/W6h8brV3yCYfVUi1F90aRlnZHsCDmtAqzW1u9s1xqkO4z4/dennI0gniEn4kWfx0I7aCQQI/XmDri/33uTINprwBB5eHt6a7WGysrcO6Mv5I8EFEAJ5vkx2mDVw+rDFC7/R8YNA77PcJABlF7G/ZajcHqPnN5uyJC+P2m6JyHO+kTHemjW8pgf5vKvdNzdTN1AVFSO9Mv4O3NPyaADRl9DCCcqAk2LwL+SYqCVvIXu98Mb6uEoSOb+ZnpvseyydxNB6DtQSVtVUNdis5Ibw="
+ - language: ruby
+os: linux
+before_script:
+before_install:
+install:
+script:
+- (cd cpp-linux && travis_wait 40 rake apt:build
APT_TARGETS=ubuntu-zesty,ubuntu-artful PARALLEL=yes DEBUG=no)
deploy:
provider: bintray
file: cpp-linux/apt/descriptor.json
diff --git a/cpp-linux/apt/debian-stretch/Dockerfile
b/cpp-linux/apt/debian-stretch/Dockerfile
index 87c9672..797cc1b 100644
--- a/cpp-linux/apt/debian-stretch/Dockerfile
+++ b/cpp-linux/apt/debian-stretch/Dockerfile
@@ -21,6 +21,8 @@ ENV DEBIAN_FRONTEND noninteractive
ARG DEBUG
+RUN sed -i'' -e 's/main$/main contrib non-free/g' /etc/apt/sources.list
+
RUN \
quiet=$([ "${DEBUG}" = "yes" ] || echo "-qq") && \
apt update ${quiet} && \
@@ -37,6 +39,7 @@ RUN \
libglib2.0-doc \
libjemalloc-dev \
lsb-release \
+nvidia-cuda-toolkit \
pkg-config \
python3-dev \
python3-numpy && \
diff --git a/cpp-linux/apt/ubuntu-artful/Dockerfile
b/cpp-linux/apt/ubuntu-artful/Dockerfile
index 2d05483..6daea3a 100644
--- a/cpp-linux/apt/ubuntu-artful/Dockerfile
+++ b/cpp-linux/apt/ubuntu-artful/Dockerfile
@@ -37,6 +37,7 @@ RUN \
libglib2.0-doc \
libjemalloc-dev \
lsb-release \
+nvidia-cuda-toolkit \
pkg-config \
python3-dev \
python3-numpy && \
diff --git a/cpp-linux/apt/ubuntu-xenial/Dockerfile
b/cpp-linux/apt/ubuntu-xenial/Dockerfile
index b61549d..85d1e1a 100644
--- a/cpp-linux/apt/ubuntu-xenial/Dockerfile
+++ b/cpp-linux/apt/ubuntu-xenial/Dockerfile
@@ -37,6 +37,7 @@ RUN \
libglib2.0-doc \
libjemalloc-dev \
lsb-release \
+nvidia-cuda-toolkit \
pkg-config \
python3-dev \
python3-numpy && \
diff --git a/cpp-linux/apt/ubuntu-zesty/Dockerfile
b/cpp-linux/apt/ubuntu-zesty/Dockerfile
index b0f2335..03b75c8 100644
--- a/cpp-linux/apt/ubuntu-zesty/Dockerfile
+++ b/cpp-linux/apt/ubuntu-zesty/Dockerfile
@@ -37,6 +37,7 @@ RUN \
libglib2.0-doc \
libjemalloc-dev \
lsb-release \
+nvidia-cuda-toolkit \
pkg-config \
python3-dev \
python3-numpy && \
diff --git a/cpp-linux/debian/changelog b/cpp-linux/debian/changelog
index 9317adc..601579b 100644
--- a/cpp-linux/debian/changelog
+++ b/cpp-linux/debian/changelog
@@ -1,3 +1,11 @@
+apache-arrow (0.7.1-2) unstable; urgency=low
+
+ * Add libarrow-gpu0 and libarrow-gpu-dev.
+
+ * Add libarrow-python-dev.
+
+ -- Kouhei Sutou Sun, 29 Oct 2017 21:59:13 +0900
+
apache-arrow (0.7.1-1) unstable; urgency=low
* New upstream release.
diff --git a/cpp-linux/debian/control b/cpp-linux/debian/control
index 147a73c..26b3b10 100644
--- a/cpp-linux/debian/control
+++ b/cpp-linux/debian/control
@@ -12,6 +12,7 @@ Build-Depends:
libjemalloc-dev,
python3-dev,
python3-numpy,
+ nvidia-cuda-toolkit,
libgirepository1.0-dev,
gtk-doc-tools,
gobject-introspection
@@ -45,7 +46,22 @@ Depends:
python3-numpy
Description: Apache Arrow is a data processing library for analysis
.
- This package provides library files for Python.
+ This package provides library files for Python support.
+
+Package: libarrow-gpu0
+Section: libs
+Architecture: any
+Multi-Arch: same
+Pre-Depends: ${misc:Pre-Depends}
+Depends:
+ ${misc:Depends},
+ ${shlibs:Depends},
+ libarrow0 (= ${binary:Version}),
+ libnvidia-fatbinaryloader,
+ libcuda1
+Descri
[GitHub] kou opened a new pull request #18: ARROW-1752: Add GPU packages for Debian and Ubuntu
kou opened a new pull request #18: ARROW-1752: Add GPU packages for Debian and Ubuntu URL: https://github.com/apache/arrow-dist/pull/18 We need two Travis CI entry to build all .debs because we don't have enough space to build all .debs in one Travis CI entry. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kou closed pull request #17: ARROW-1725: Upload .deb for Ubuntu 17.10
kou closed pull request #17: ARROW-1725: Upload .deb for Ubuntu 17.10
URL: https://github.com/apache/arrow-dist/pull/17
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/.travis.yml b/.travis.yml
index 048d464..098afe6 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -54,7 +54,7 @@ matrix:
before_install:
install:
script:
-- (cd cpp-linux && rake apt:build DEBUG=no)
+- (cd cpp-linux && travis_wait 40 rake apt:build PARALLEL=yes DEBUG=no)
deploy:
provider: bintray
file: cpp-linux/apt/descriptor.json
diff --git a/cpp-linux/apt/descriptor.json b/cpp-linux/apt/descriptor.json
index fde23e4..9e1ff7a 100644
--- a/cpp-linux/apt/descriptor.json
+++ b/cpp-linux/apt/descriptor.json
@@ -39,6 +39,16 @@
"deb_architecture": "amd64",
"override": 1
}
+},
+{
+"includePattern":
"cpp-linux/apt/repositories/([^/]+)/pool/artful/universe/a/apache-arrow/([^/]+\\.deb)\\z",
+"uploadPattern": "pool/artful/universe/$2",
+"matrixParams": {
+"deb_distribution": "artful",
+"deb_component": "universe",
+"deb_architecture": "amd64",
+"override": 1
+}
}
],
"publish": true
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] kou commented on issue #17: ARROW-1725: Upload .deb for Ubuntu 17.10
kou commented on issue #17: ARROW-1725: Upload .deb for Ubuntu 17.10 URL: https://github.com/apache/arrow-dist/pull/17#issuecomment-339220628 +1 We can't test upload is succeeded with pull request. We need to use master branch. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kou opened a new pull request #17: ARROW-1725: Upload .deb for Ubuntu 17.10
kou opened a new pull request #17: ARROW-1725: Upload .deb for Ubuntu 17.10 URL: https://github.com/apache/arrow-dist/pull/17 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kou closed pull request #16: ARROW-1724: Support Ubuntu 17.10
kou closed pull request #16: ARROW-1724: Support Ubuntu 17.10
URL: https://github.com/apache/arrow-dist/pull/16
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/cpp-linux/apt/ubuntu-artful/Dockerfile
b/cpp-linux/apt/ubuntu-artful/Dockerfile
new file mode 100644
index 000..2d05483
--- /dev/null
+++ b/cpp-linux/apt/ubuntu-artful/Dockerfile
@@ -0,0 +1,44 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+FROM ubuntu:17.10
+
+ENV DEBIAN_FRONTEND noninteractive
+
+ARG DEBUG
+
+RUN \
+ quiet=$([ "${DEBUG}" = "yes" ] || echo "-qq") && \
+ apt update ${quiet} && \
+ apt install -y -V ${quiet} \
+build-essential \
+cmake \
+debhelper\
+devscripts \
+git \
+gtk-doc-tools \
+libboost-filesystem-dev \
+libboost-system-dev \
+libgirepository1.0-dev \
+libglib2.0-doc \
+libjemalloc-dev \
+lsb-release \
+pkg-config \
+python3-dev \
+python3-numpy && \
+ apt clean && \
+ rm -rf /var/lib/apt/lists/*
diff --git a/cpp-linux/package-task.rb b/cpp-linux/package-task.rb
index 6426e37..adaf9da 100644
--- a/cpp-linux/package-task.rb
+++ b/cpp-linux/package-task.rb
@@ -211,6 +211,7 @@ def define_apt_task
"ubuntu-trusty",
"ubuntu-xenial",
"ubuntu-zesty",
+ "ubuntu-artful",
]
end
targets.each do |target|
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] kou commented on issue #16: ARROW-1724: Support Ubuntu 17.10
kou commented on issue #16: ARROW-1724: Support Ubuntu 17.10 URL: https://github.com/apache/arrow-dist/pull/16#issuecomment-339188039 +1 .deb package build is succeeded on Travis CI. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kou opened a new pull request #16: ARROW--1724: Support Ubuntu 17.10
kou opened a new pull request #16: ARROW--1724: Support Ubuntu 17.10 URL: https://github.com/apache/arrow-dist/pull/16 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
