Bintray sunsetting
Appears that JFrog is sunsetting Bintray, so we will need to sort out alternative hosting for Linux packages for the 4.0.0 release: https://jfrog.com/blog/into-the-sunset-bintray-jcenter-gocenter-and-chartcenter/
Re: [RUST] Fields and schema metadata
I see. I guess you are doing key comparison on the Field's metadata. Thanks for the clarification On Sat, Feb 6, 2021 at 12:27 PM Neville Dipale wrote: > We had to use the BTreeMap because a HashMap doesn't implement Hash, so > can't be used in the Field. > > The easiest way to see this, is to replace it with a HashMap, and try > compile the arrow crate. > > Neville > > On Sat, 06 Feb 2021, 13:50 Fernando Herrera, > > wrote: > > > Hi all, Is there a reason why the Field metadata is a BTreeMap and > Schema's > > metadata is a HashMap? > > > > I'm just curious why different structures were selected for the same > thing. > > Sorry if this is explained somewhere in the code, but I couldn't find > > anything about it. > > > > Fernando, > > >
Re: [RUST] Fields and schema metadata
We had to use the BTreeMap because a HashMap doesn't implement Hash, so can't be used in the Field. The easiest way to see this, is to replace it with a HashMap, and try compile the arrow crate. Neville On Sat, 06 Feb 2021, 13:50 Fernando Herrera, wrote: > Hi all, Is there a reason why the Field metadata is a BTreeMap and Schema's > metadata is a HashMap? > > I'm just curious why different structures were selected for the same thing. > Sorry if this is explained somewhere in the code, but I couldn't find > anything about it. > > Fernando, >
Re: [Rust] DataFusion TPCH benchmark overview
Update: I created the list of issues in JIRA so we can keep track of progress there. https://issues.apache.org/jira/browse/ARROW-11519https://issues.apache.org/jira/browse/ARROW-11519 Op do 4 feb. 2021 om 12:45 schreef Daniël Heres : > Thanks all for your input! > > I will create an umbrella ticket + linked failures / issues to track > progress for TPCH support coming days and will share it here. > > Daniël > > Op do 4 feb. 2021 om 00:13 schreef Andrew Lamb : > >> This is awesome, thank you Daniel. I agree that focusing on enough SQL for >> TPCH queries would be a great idea and way to focus our efforts. >> >> Subqueries may be the largest remaining outstanding item that I see -- I >> have some ideas of how to implement them on the planner side if others are >> interested in collaborating. >> >> Andrew >> >> On Wed, Feb 3, 2021 at 4:02 PM Andy Grove wrote: >> >> > Thanks for the update on this, Daniël. It is great to see the progress >> with >> > this! >> > >> > Perhaps it is worth creating one JIRA issue per failing query detailing >> the >> > errors and we can link these to the issues that are causing the >> failures? >> > >> > On Wed, Feb 3, 2021 at 1:57 PM Mike Seddon wrote: >> > >> > > Hi Daniël, >> > > >> > > I am working on 22 as part of >> https://github.com/apache/arrow/pull/9243 >> > > >> > > We also need to convert all the Float64 schema types to Decimal(n). >> > > >> > > Cheers, >> > > Mike >> > > >> > > On Thu, Feb 4, 2021 at 5:44 AM Daniël Heres >> > wrote: >> > > >> > > > Hey all, >> > > > >> > > > Quite some features have been added to DataFusion in the last >> couple of >> > > > months. >> > > > >> > > > One test of the functionality we support this is the TPC-H >> benchmark. >> > We >> > > > now can run 7 out of 22 queries without errors. >> > > > I think a nice goal would be having complete support for the full >> > suite, >> > > as >> > > > it means a lot of functionality is included, helps optimization and >> > helps >> > > > us to test against other engines. >> > > > >> > > > These queries fail currently because of missing features or bugs: >> > > > >> > > > * 2 IN (Subquery) in (WHERE) expression >> > > > * 4 Intervals https://github.com/apache/arrow/pull/9373 >> > > > * 7 Fails with error "Schema contains duplicate unqualified field >> name >> > > > \'n_nationkey\'" https://issues.apache.org/jira/browse/ARROW-11432 >> > > > * 8 Fails with error "Schema contains duplicate unqualified field >> name >> > > > \'n_nationkey\'" https://issues.apache.org/jira/browse/ARROW-11432 >> > > > * 9 Fails with error "Cartesian joins are not supported" >> > > > * 11 HAVING support https://github.com/apache/arrow/pull/9364 (but >> > also >> > > > requires IN (subquery) in expression) >> > > > * 13 Filters in JOIN condition: "Unsupported expression \'NotLike\' >> in >> > > JOIN >> > > > condition" >> > > > * 14 CASE WHEN expressions are not coerced yet, query fails with >> error >> > > > "false_values downcast failed" >> > > > * 15 VIEW/multiple statement support: "The context currently only >> > > supports >> > > > a single SQL statement" >> > > > * 16 IN (Subquery) in (WHERE) expression >> > > > * 17 Subquery in (WHERE) expression >> > > > * 18 IN (Subquery) in (WHERE) expression >> > > > * 19 Fails with error "Cartesian joins are not supported" >> > > > * 20 IN (Subquery) in (WHERE) expression >> > > > * 21 Compound identifier not supported: "Unsupported compound >> > identifier >> > > > \'[\"l1\", \"l_suppkey\"]" >> > > > * 22 Fails with parser error for the syntax SUBSTRING(col FROM 1) >> > > > >> > > > FOR 2) >> > > > Other functionality not causing failures now, but needed: >> > > > >> > > > * EXTRACT https://github.com/apache/arrow/pull/9359 >> > > > * EXISTS >> > > > >> > > > Am I missing any JIRA issues / PRs or features in this list? I would >> > like >> > > > to create some issues on JIRA so we can tackle this. >> > > > >> > > > Best regards, >> > > > >> > > > Daniël >> > > > >> > > >> > >> > > > -- > Daniël Heres > -- Daniël Heres
[RUST] Fields and schema metadata
Hi all, Is there a reason why the Field metadata is a BTreeMap and Schema's metadata is a HashMap? I'm just curious why different structures were selected for the same thing. Sorry if this is explained somewhere in the code, but I couldn't find anything about it. Fernando,
[NIGHTLY] Arrow Build Report for Job nightly-2021-02-06-0
Arrow Build Report for Job nightly-2021-02-06-0 All tasks: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0 Failed Tasks: - conda-linux-gcc-py36-aarch64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-drone-conda-linux-gcc-py36-aarch64 - conda-linux-gcc-py37-aarch64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-drone-conda-linux-gcc-py37-aarch64 - conda-linux-gcc-py38-aarch64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-drone-conda-linux-gcc-py38-aarch64 - conda-linux-gcc-py39-aarch64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-drone-conda-linux-gcc-py39-aarch64 - test-conda-python-3.7-dask-latest: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-github-test-conda-python-3.7-dask-latest - test-conda-python-3.8-dask-master: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-github-test-conda-python-3.8-dask-master - test-conda-python-3.8-jpype: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-github-test-conda-python-3.8-jpype - test-ubuntu-18.04-docs: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-azure-test-ubuntu-18.04-docs Succeeded Tasks: - centos-7-amd64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-github-centos-7-amd64 - centos-8-amd64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-github-centos-8-amd64 - conda-clean: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-azure-conda-clean - conda-linux-gcc-py36-cpu-r36: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-azure-conda-linux-gcc-py36-cpu-r36 - conda-linux-gcc-py36-cuda: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-azure-conda-linux-gcc-py36-cuda - conda-linux-gcc-py37-cpu-r40: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-azure-conda-linux-gcc-py37-cpu-r40 - conda-linux-gcc-py37-cuda: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-azure-conda-linux-gcc-py37-cuda - conda-linux-gcc-py38-cpu: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-azure-conda-linux-gcc-py38-cpu - conda-linux-gcc-py38-cuda: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-azure-conda-linux-gcc-py38-cuda - conda-linux-gcc-py39-cpu: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-azure-conda-linux-gcc-py39-cpu - conda-linux-gcc-py39-cuda: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-azure-conda-linux-gcc-py39-cuda - conda-osx-clang-py36-r36: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-azure-conda-osx-clang-py36-r36 - conda-osx-clang-py37-r40: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-azure-conda-osx-clang-py37-r40 - conda-osx-clang-py38: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-azure-conda-osx-clang-py38 - conda-osx-clang-py39: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-azure-conda-osx-clang-py39 - conda-win-vs2017-py36-r36: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-azure-conda-win-vs2017-py36-r36 - conda-win-vs2017-py37-r40: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-azure-conda-win-vs2017-py37-r40 - conda-win-vs2017-py38: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-azure-conda-win-vs2017-py38 - debian-buster-amd64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-github-debian-buster-amd64 - example-cpp-minimal-build-static-system-dependency: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-github-example-cpp-minimal-build-static-system-dependency - example-cpp-minimal-build-static: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-github-example-cpp-minimal-build-static - gandiva-jar-osx: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-github-gandiva-jar-osx - gandiva-jar-ubuntu: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-github-gandiva-jar-ubuntu - homebrew-cpp: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-02-06-0-github-homebrew-cpp - homebrew-r-autobrew: URL
