Re: [C++]: cmake: about parallel build of third party modules
Hi Yibo We’ve found flakiness with protobuf_ep when building in parallel with GNU Make. There are comments about this in ThirdpartyToolchain.cmake I think and prior JIRA issues related to this. For faster builds we generally are focused on using -GNinja so recommend using the ninja build tool instead of make. Wes On Fri, Jan 3, 2020 at 8:28 AM Yibo Cai wrote: > Maybe just install protobuf dev package is the simplest solution. > > On 1/2/20 1:55 PM, Yibo Cai wrote: > > I noticed a fresh build always stuck at compiling protobuf for a long > time. We've decided to use single job building for each third party module > [1], partly because different thirty party modules are built concurrently > (protobuf is built concurrently with jemalloc, but protobuf itself is built > with only one job). > > > > The problem is that protobuf takes much more time than other modules to > finish, which blocks the whole building process. I tried adding "-j4" > manually when compiling protobuf [2], it significantly reduced time for a > fresh build. Below is my testing. > > > > test setting > > > > cpu: Intel E5-2650, 20 logical cores > > memory: 64G > > > > test command > > > > cmake -DCMAKE_BUILD_TYPE=Release -DARROW_BUILD_TESTS=ON > -DARROW_COMPUTE=ON -DARROW_GANDIVA=ON .. > > make -j8 > > > > test result > > --- > > build protobuf with single job(default): 10min 32sec > > build protobuf with four jobs(add -j4): 6min 23sec > > > > Build time dropped 40% from 632s to 383s. Even bigger gap is observed on > Arm platform. > > > > I would suggest enabling multi job build for protobuf, maybe set half > total jobs. Say, it we launch arrow build with "make -j8", we compile > protobuf with "-j4". Code may be kind of ugly and not consistent, but > deserves the effort IMHO. Comments? > > > > [1] https://github.com/apache/arrow/pull/2779 > > [2] > https://github.com/apache/arrow/blob/master/cpp/cmake_modules/ThirdpartyToolchain.cmake#L1163 >
[NIGHTLY] Arrow Build Report for Job nightly-2020-01-03-0
Arrow Build Report for Job nightly-2020-01-03-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0 Failed Tasks: - debian-stretch: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-debian-stretch - gandiva-jar-osx: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-travis-gandiva-jar-osx - homebrew-cpp: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-travis-homebrew-cpp - test-ubuntu-18.04-cpp-static: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-ubuntu-18.04-cpp-static Succeeded Tasks: - centos-6: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-centos-6 - centos-7: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-centos-7 - centos-8: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-centos-8 - conda-linux-gcc-py27: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-linux-gcc-py27 - conda-linux-gcc-py36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-linux-gcc-py36 - conda-linux-gcc-py37: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-linux-gcc-py37 - conda-linux-gcc-py38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-linux-gcc-py38 - conda-osx-clang-py27: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-osx-clang-py27 - conda-osx-clang-py36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-osx-clang-py36 - conda-osx-clang-py37: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-osx-clang-py37 - conda-osx-clang-py38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-osx-clang-py38 - conda-win-vs2015-py36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-win-vs2015-py36 - conda-win-vs2015-py37: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-win-vs2015-py37 - conda-win-vs2015-py38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-win-vs2015-py38 - debian-buster: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-debian-buster - gandiva-jar-trusty: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-travis-gandiva-jar-trusty - macos-r-autobrew: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-travis-macos-r-autobrew - test-conda-cpp: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-cpp - test-conda-python-2.7-pandas-latest: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-2.7-pandas-latest - test-conda-python-2.7: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-2.7 - test-conda-python-3.6: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-3.6 - test-conda-python-3.7-dask-latest: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-3.7-dask-latest - test-conda-python-3.7-hdfs-2.9.2: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-3.7-hdfs-2.9.2 - test-conda-python-3.7-pandas-latest: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-3.7-pandas-latest - test-conda-python-3.7-pandas-master: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-3.7-pandas-master - test-conda-python-3.7-spark-master: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-3.7-spark-master - test-conda-python-3.7-turbodbc-latest: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-3.7-turbodbc-latest - test-conda-python-3.7-turbodbc-master: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-3.7-turbodbc-master - test-conda-python-3.7: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-3.7 - test-conda-python-3.8-dask-master: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-3.8-dask-master - test-
[jira] [Created] (ARROW-7492) [CI][Crossbow] Nightly homebrew-cpp job fails on Python installation
Neal Richardson created ARROW-7492:
--
Summary: [CI][Crossbow] Nightly homebrew-cpp job fails on Python
installation
Key: ARROW-7492
URL: https://issues.apache.org/jira/browse/ARROW-7492
Project: Apache Arrow
Issue Type: Bug
Components: C++, Continuous Integration
Reporter: Neal Richardson
Fix For: 1.0.0
Oddly, it looks like the formula builds successfully
(https://travis-ci.org/ursa-labs/crossbow/builds/631795636#L3698)
{code}
🍺 /usr/local/Cellar/apache-arrow/HEAD-096c78c: 366 files, 57.5MB, built in 9
minutes 46 seconds
{code}
But it exits with status 1. AFAICT the error is here:
https://travis-ci.org/ursa-labs/crossbow/builds/631795636#L1745
{code}
==> Pouring python-3.7.6_1.high_sierra.bottle.tar.gz
tar xof
/Users/travis/Library/Caches/Homebrew/downloads/3d94ccb6613548e55aed20c7ee59f0e0b7fb045a5d9c8885aa3504ea31f8ab0e--python-3.7.6_1.high_sierra.bottle.tar.gz
-C /var/folders/nz/vv4_9tw56nv9k3tkvyszvwg8gn/T/d20200102-25789-k5jazs
cp -pR
/var/folders/nz/vv4_9tw56nv9k3tkvyszvwg8gn/T/d20200102-25789-k5jazs/python/.
/usr/local/Cellar/python
chmod -Rf +w
/var/folders/nz/vv4_9tw56nv9k3tkvyszvwg8gn/T/d20200102-25789-k5jazs
Error: The `brew link` step did not complete successfully
{code}
According to
https://github.com/ursa-labs/crossbow/branches/all?utf8=%E2%9C%93&query=-0-travis-homebrew-cpp,
this started failing on 2019-12-28.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [NIGHTLY] Arrow Build Report for Job nightly-2020-01-03-0
Homebrew has been failing for about a week so I ticketed it here: https://issues.apache.org/jira/browse/ARROW-7492 On Fri, Jan 3, 2020 at 6:18 AM Crossbow wrote: > > Arrow Build Report for Job nightly-2020-01-03-0 > > All tasks: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0 > > Failed Tasks: > - debian-stretch: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-debian-stretch > - gandiva-jar-osx: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-travis-gandiva-jar-osx > - homebrew-cpp: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-travis-homebrew-cpp > - test-ubuntu-18.04-cpp-static: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-ubuntu-18.04-cpp-static > > Succeeded Tasks: > - centos-6: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-centos-6 > - centos-7: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-centos-7 > - centos-8: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-centos-8 > - conda-linux-gcc-py27: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-linux-gcc-py27 > - conda-linux-gcc-py36: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-linux-gcc-py36 > - conda-linux-gcc-py37: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-linux-gcc-py37 > - conda-linux-gcc-py38: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-linux-gcc-py38 > - conda-osx-clang-py27: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-osx-clang-py27 > - conda-osx-clang-py36: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-osx-clang-py36 > - conda-osx-clang-py37: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-osx-clang-py37 > - conda-osx-clang-py38: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-osx-clang-py38 > - conda-win-vs2015-py36: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-win-vs2015-py36 > - conda-win-vs2015-py37: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-win-vs2015-py37 > - conda-win-vs2015-py38: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-conda-win-vs2015-py38 > - debian-buster: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-azure-debian-buster > - gandiva-jar-trusty: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-travis-gandiva-jar-trusty > - macos-r-autobrew: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-travis-macos-r-autobrew > - test-conda-cpp: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-cpp > - test-conda-python-2.7-pandas-latest: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-2.7-pandas-latest > - test-conda-python-2.7: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-2.7 > - test-conda-python-3.6: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-3.6 > - test-conda-python-3.7-dask-latest: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-3.7-dask-latest > - test-conda-python-3.7-hdfs-2.9.2: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-3.7-hdfs-2.9.2 > - test-conda-python-3.7-pandas-latest: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-3.7-pandas-latest > - test-conda-python-3.7-pandas-master: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-3.7-pandas-master > - test-conda-python-3.7-spark-master: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-3.7-spark-master > - test-conda-python-3.7-turbodbc-latest: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-python-3.7-turbodbc-latest > - test-conda-python-3.7-turbodbc-master: > URL: > https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-01-03-0-circle-test-conda-p
[jira] [Created] (ARROW-7493) [Python] Expose sum kernel in pyarrow.compute and support ChunkedArray inputs
Uwe Korn created ARROW-7493: --- Summary: [Python] Expose sum kernel in pyarrow.compute and support ChunkedArray inputs Key: ARROW-7493 URL: https://issues.apache.org/jira/browse/ARROW-7493 Project: Apache Arrow Issue Type: Improvement Components: C++ - Compute, Python Reporter: Uwe Korn Assignee: Uwe Korn Fix For: 1.0.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
Interested in leveraging (or if not existent building) an Apache Arrow Query Engine
Hi. I’m interested in leveraging (or if does not yet existent co-building) an "Apache Arrow Query Engine” in C++. I found various references on the web, eg. https://docs.google.com/document/d/10RoUZmiMQRi_J1FcPeVAUAMJ6d_ZuiEbaM2Y33sNPu4/edit#heading=h.2k6k5a4y9b8y, or Gandiva (for Dremio’s engine) or DataFusion (for Rust), but no AQE. Further all these efforts seem to have stalled. Can someone explain what the status of AQE is? Thanks much!,- Wolfram
Re: Interested in leveraging (or if not existent building) an Apache Arrow Query Engine
Hi Wolfram, The effort hasn’t stalled. As a volunteer-driven project with developers coming from modestly funded organizations the progress has not been overnight. My team (Ursa Labs) has been building infrastructure in support of this effort (for example, the datasets API and filesystem API, both necessary pieces), as have developers from Dremio and others (eg on Gandiva). It would be great if engineers from Facebook would contribute to the effort. Thanks Wes On Fri, Jan 3, 2020 at 10:16 PM Wolfram Schulte wrote: > Hi. > > I’m interested in leveraging (or if does not yet existent co-building) an > "Apache Arrow Query Engine” in C++. > > I found various references on the web, eg. > https://docs.google.com/document/d/10RoUZmiMQRi_J1FcPeVAUAMJ6d_ZuiEbaM2Y33sNPu4/edit#heading=h.2k6k5a4y9b8y, > or Gandiva (for Dremio’s engine) or DataFusion (for Rust), but no AQE. > > Further all these efforts seem to have stalled. Can someone explain what > the status of AQE is? > > Thanks much!,- Wolfram > >
[jira] [Created] (ARROW-7494) [Java] Remove reader index and writer index from ArrowBuf
Jacques Nadeau created ARROW-7494: - Summary: [Java] Remove reader index and writer index from ArrowBuf Key: ARROW-7494 URL: https://issues.apache.org/jira/browse/ARROW-7494 Project: Apache Arrow Issue Type: Task Reporter: Jacques Nadeau Reader and writer index and functionality doesn't belong on a chunk of memory and is due to inheritance from ByteBuf. As part of removing ByteBuf inheritance, we should also remove reader and writer indexes from ArrowBuf functionality. It wastes heap memory for rare utility. In general, a slice can be used instead of a reader/writer index pattern. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-7495) [Java] Remove "empty" concept from ArrowBuf, replace with custom referencemanager
Jacques Nadeau created ARROW-7495: - Summary: [Java] Remove "empty" concept from ArrowBuf, replace with custom referencemanager Key: ARROW-7495 URL: https://issues.apache.org/jira/browse/ARROW-7495 Project: Apache Arrow Issue Type: Task Reporter: Jacques Nadeau With the introduction of ReferenceManager in the codebase, the need for a separate ArrowBuf is no longer necessary. Instead, once can create a new reference manager that is used for the empty ArrowBuf. For reminder/review, empty arrowbufs have a special behavior in that they don't actually have any reference counting semantics and always stay at one. This allow us to better troubleshoot unallocated memory than what would otherwise be an NPE after calling ValueVector.clear() -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Looking to 1.0
I identified three things in the java library that I think are top of mind and should be fixed before 1.0 to avoid weird incompatibility changes in the java apis (technical debt). I've tagged them as pre-1.0 as I don't exactly see what is the right way to tag/label a target release for a ticket. https://issues.apache.org/jira/browse/ARROW-7495?jql=labels%20%3D%20pre-1.0 For the three tickets I identified, does anyone have interest in trying to resolve? thanks, Jacques On Thu, Jan 2, 2020 at 11:55 AM Neal Richardson wrote: > Hi all, > Happy new year! As we look ahead to 2020, it's time to start mobilizing for > the Arrow 1.0 release. At 0.15, I believe we decided that our next release > should be 1.0, and it's been a couple of months since 0.15, so we're due to > release again this month, give or take. (See [1] for when we most recently > discussed doing 1.0 back in June, or if you're a fan of ancient history, > see [2] for a similar discussion from July 2017.) > > Since there appeared to be consensus before that it is time for 1.0, let's > discuss how to get it done. One first step would be to make sure that we've > identified all format/specification issues we think we must resolve before > declaring 1.0. [3] shows 3 "blockers" for the 1.0 release already. There > are an additional 14 "Format" issues ([4]); perhaps some of those should > also be labeled blockers for 1.0. > > It would be great if folks could review Jira in their areas of expertise > and make sure everything essential for 1.0 is ticketed and prioritized > appropriately. Once we've identified the required tasks for making a 1.0 > release, we can work together on burning those down. > > Neal > > [1]: > > https://lists.apache.org/thread.html/44a7a3d256ab5dbd62da6fe45b56951b435697426bf4adedb6520907@%3Cdev.arrow.apache.org%3E > > [2]: > > https://lists.apache.org/thread.html/0aca401e8906e1adbb37228b38569a9a7736b864da854007dad111c3%40%3Cdev.arrow.apache.org%3E > [3]: https://cwiki.apache.org/confluence/display/ARROW/Arrow+1.0.0+Release > [4]: > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20ARROW%20AND%20status%20in%20(%22In%20Review%22%2C%20Open%2C%20%22In%20Progress%22)%20AND%20fixVersion%20%3D%201.0.0%20AND%20component%20%3D%20Format >
Re: Looking to 1.0
Thanks for reviewing, Jacques. Re: Jira tagging, the convention I've seen is to use "Fix Version" for open tickets to indicate a target release (that's what the confluence board draws from, for example). So by my understanding, Fix Version == 1.0.0 and Priority == Blocker says that they're things we have to resolve before 1.0.0. Happy to follow a different convention though if folks can think of something better. Neal On Fri, Jan 3, 2020 at 3:16 PM Jacques Nadeau wrote: > I identified three things in the java library that I think are top of mind > and should be fixed before 1.0 to avoid weird incompatibility changes in > the java apis (technical debt). I've tagged them as pre-1.0 as I don't > exactly see what is the right way to tag/label a target release for a > ticket. > https://issues.apache.org/jira/browse/ARROW-7495?jql=labels%20%3D%20pre-1.0 > > For the three tickets I identified, does anyone have interest in trying to > resolve? > > thanks, > Jacques > > > > On Thu, Jan 2, 2020 at 11:55 AM Neal Richardson < > neal.p.richard...@gmail.com> > wrote: > > > Hi all, > > Happy new year! As we look ahead to 2020, it's time to start mobilizing > for > > the Arrow 1.0 release. At 0.15, I believe we decided that our next > release > > should be 1.0, and it's been a couple of months since 0.15, so we're due > to > > release again this month, give or take. (See [1] for when we most > recently > > discussed doing 1.0 back in June, or if you're a fan of ancient history, > > see [2] for a similar discussion from July 2017.) > > > > Since there appeared to be consensus before that it is time for 1.0, > let's > > discuss how to get it done. One first step would be to make sure that > we've > > identified all format/specification issues we think we must resolve > before > > declaring 1.0. [3] shows 3 "blockers" for the 1.0 release already. There > > are an additional 14 "Format" issues ([4]); perhaps some of those should > > also be labeled blockers for 1.0. > > > > It would be great if folks could review Jira in their areas of expertise > > and make sure everything essential for 1.0 is ticketed and prioritized > > appropriately. Once we've identified the required tasks for making a 1.0 > > release, we can work together on burning those down. > > > > Neal > > > > [1]: > > > > > https://lists.apache.org/thread.html/44a7a3d256ab5dbd62da6fe45b56951b435697426bf4adedb6520907@%3Cdev.arrow.apache.org%3E > > > > [2]: > > > > > https://lists.apache.org/thread.html/0aca401e8906e1adbb37228b38569a9a7736b864da854007dad111c3%40%3Cdev.arrow.apache.org%3E > > [3]: > https://cwiki.apache.org/confluence/display/ARROW/Arrow+1.0.0+Release > > [4]: > > > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20ARROW%20AND%20status%20in%20(%22In%20Review%22%2C%20Open%2C%20%22In%20Progress%22)%20AND%20fixVersion%20%3D%201.0.0%20AND%20component%20%3D%20Format > > >
Re: Looking to 1.0
Hi, Jacques Nadeau I only found two tickets[1][2], and I am interested in these issues. If you don't mind, I would like to take a close watch and try to resolve later. Thanks, Ji Liu [1] https://issues.apache.org/jira/browse/ARROW-7494 [2] https://issues.apache.org/jira/browse/ARROW-7495 -- From:Jacques Nadeau Send Time:2020年1月4日(星期六) 07:16 To:dev Subject:Re: Looking to 1.0 I identified three things in the java library that I think are top of mind and should be fixed before 1.0 to avoid weird incompatibility changes in the java apis (technical debt). I've tagged them as pre-1.0 as I don't exactly see what is the right way to tag/label a target release for a ticket. https://issues.apache.org/jira/browse/ARROW-7495?jql=labels%20%3D%20pre-1.0 For the three tickets I identified, does anyone have interest in trying to resolve? thanks, Jacques On Thu, Jan 2, 2020 at 11:55 AM Neal Richardson wrote: > Hi all, > Happy new year! As we look ahead to 2020, it's time to start mobilizing for > the Arrow 1.0 release. At 0.15, I believe we decided that our next release > should be 1.0, and it's been a couple of months since 0.15, so we're due to > release again this month, give or take. (See [1] for when we most recently > discussed doing 1.0 back in June, or if you're a fan of ancient history, > see [2] for a similar discussion from July 2017.) > > Since there appeared to be consensus before that it is time for 1.0, let's > discuss how to get it done. One first step would be to make sure that we've > identified all format/specification issues we think we must resolve before > declaring 1.0. [3] shows 3 "blockers" for the 1.0 release already. There > are an additional 14 "Format" issues ([4]); perhaps some of those should > also be labeled blockers for 1.0. > > It would be great if folks could review Jira in their areas of expertise > and make sure everything essential for 1.0 is ticketed and prioritized > appropriately. Once we've identified the required tasks for making a 1.0 > release, we can work together on burning those down. > > Neal > > [1]: > > https://lists.apache.org/thread.html/44a7a3d256ab5dbd62da6fe45b56951b435697426bf4adedb6520907@%3Cdev.arrow.apache.org%3E > > [2]: > > https://lists.apache.org/thread.html/0aca401e8906e1adbb37228b38569a9a7736b864da854007dad111c3%40%3Cdev.arrow.apache.org%3E > [3]: https://cwiki.apache.org/confluence/display/ARROW/Arrow+1.0.0+Release > [4]: > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20ARROW%20AND%20status%20in%20(%22In%20Review%22%2C%20Open%2C%20%22In%20Progress%22)%20AND%20fixVersion%20%3D%201.0.0%20AND%20component%20%3D%20Format >
Re: Looking to 1.0
Hi Jacques, I am interested in the issues, and if it is possible, I would like to try to resolve them. Thanks. Liya Fan On Sat, Jan 4, 2020 at 7:16 AM Jacques Nadeau wrote: > I identified three things in the java library that I think are top of mind > and should be fixed before 1.0 to avoid weird incompatibility changes in > the java apis (technical debt). I've tagged them as pre-1.0 as I don't > exactly see what is the right way to tag/label a target release for a > ticket. > https://issues.apache.org/jira/browse/ARROW-7495?jql=labels%20%3D%20pre-1.0 > > For the three tickets I identified, does anyone have interest in trying to > resolve? > > thanks, > Jacques > > > > On Thu, Jan 2, 2020 at 11:55 AM Neal Richardson < > neal.p.richard...@gmail.com> > wrote: > > > Hi all, > > Happy new year! As we look ahead to 2020, it's time to start mobilizing > for > > the Arrow 1.0 release. At 0.15, I believe we decided that our next > release > > should be 1.0, and it's been a couple of months since 0.15, so we're due > to > > release again this month, give or take. (See [1] for when we most > recently > > discussed doing 1.0 back in June, or if you're a fan of ancient history, > > see [2] for a similar discussion from July 2017.) > > > > Since there appeared to be consensus before that it is time for 1.0, > let's > > discuss how to get it done. One first step would be to make sure that > we've > > identified all format/specification issues we think we must resolve > before > > declaring 1.0. [3] shows 3 "blockers" for the 1.0 release already. There > > are an additional 14 "Format" issues ([4]); perhaps some of those should > > also be labeled blockers for 1.0. > > > > It would be great if folks could review Jira in their areas of expertise > > and make sure everything essential for 1.0 is ticketed and prioritized > > appropriately. Once we've identified the required tasks for making a 1.0 > > release, we can work together on burning those down. > > > > Neal > > > > [1]: > > > > > https://lists.apache.org/thread.html/44a7a3d256ab5dbd62da6fe45b56951b435697426bf4adedb6520907@%3Cdev.arrow.apache.org%3E > > > > [2]: > > > > > https://lists.apache.org/thread.html/0aca401e8906e1adbb37228b38569a9a7736b864da854007dad111c3%40%3Cdev.arrow.apache.org%3E > > [3]: > https://cwiki.apache.org/confluence/display/ARROW/Arrow+1.0.0+Release > > [4]: > > > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20ARROW%20AND%20status%20in%20(%22In%20Review%22%2C%20Open%2C%20%22In%20Progress%22)%20AND%20fixVersion%20%3D%201.0.0%20AND%20component%20%3D%20Format > > >
Re: Looking to 1.0
I am sorry. I did not notice the issues have already been assigned. Best, Liya Fan On Sat, Jan 4, 2020 at 12:15 PM Fan Liya wrote: > Hi Jacques, > > I am interested in the issues, and if it is possible, I would like to try > to resolve them. > > Thanks. > > Liya Fan > > On Sat, Jan 4, 2020 at 7:16 AM Jacques Nadeau wrote: > >> I identified three things in the java library that I think are top of mind >> and should be fixed before 1.0 to avoid weird incompatibility changes in >> the java apis (technical debt). I've tagged them as pre-1.0 as I don't >> exactly see what is the right way to tag/label a target release for a >> ticket. >> >> https://issues.apache.org/jira/browse/ARROW-7495?jql=labels%20%3D%20pre-1.0 >> >> For the three tickets I identified, does anyone have interest in trying to >> resolve? >> >> thanks, >> Jacques >> >> >> >> On Thu, Jan 2, 2020 at 11:55 AM Neal Richardson < >> neal.p.richard...@gmail.com> >> wrote: >> >> > Hi all, >> > Happy new year! As we look ahead to 2020, it's time to start mobilizing >> for >> > the Arrow 1.0 release. At 0.15, I believe we decided that our next >> release >> > should be 1.0, and it's been a couple of months since 0.15, so we're >> due to >> > release again this month, give or take. (See [1] for when we most >> recently >> > discussed doing 1.0 back in June, or if you're a fan of ancient history, >> > see [2] for a similar discussion from July 2017.) >> > >> > Since there appeared to be consensus before that it is time for 1.0, >> let's >> > discuss how to get it done. One first step would be to make sure that >> we've >> > identified all format/specification issues we think we must resolve >> before >> > declaring 1.0. [3] shows 3 "blockers" for the 1.0 release already. There >> > are an additional 14 "Format" issues ([4]); perhaps some of those should >> > also be labeled blockers for 1.0. >> > >> > It would be great if folks could review Jira in their areas of expertise >> > and make sure everything essential for 1.0 is ticketed and prioritized >> > appropriately. Once we've identified the required tasks for making a 1.0 >> > release, we can work together on burning those down. >> > >> > Neal >> > >> > [1]: >> > >> > >> https://lists.apache.org/thread.html/44a7a3d256ab5dbd62da6fe45b56951b435697426bf4adedb6520907@%3Cdev.arrow.apache.org%3E >> > >> > [2]: >> > >> > >> https://lists.apache.org/thread.html/0aca401e8906e1adbb37228b38569a9a7736b864da854007dad111c3%40%3Cdev.arrow.apache.org%3E >> > [3]: >> https://cwiki.apache.org/confluence/display/ARROW/Arrow+1.0.0+Release >> > [4]: >> > >> > >> https://issues.apache.org/jira/issues/?jql=project%20%3D%20ARROW%20AND%20status%20in%20(%22In%20Review%22%2C%20Open%2C%20%22In%20Progress%22)%20AND%20fixVersion%20%3D%201.0.0%20AND%20component%20%3D%20Format >> > >> >
