comphead opened a new pull request, #211: URL: https://github.com/apache/datafusion-site/pull/211
This adds a draft blog post about the nested benchmark derived from TPC-H that was added to Apache DataFusion Comet in apache/datafusion-comet#7044. The post covers: - Why a flat TPC-H schema leaves out the choke points that nested data adds, and how the nested benchmark keeps the TPC-H questions while changing the shape of the tables. - SF1000 results for Comet and Spark, for a starting-point run and a latest run, as fastest-of-3 per query, with charts and an interactive dashboard (`content/images/comet-nested-tpch/dashboard.html`). - The issues the benchmark helped to surface: apache/datafusion-comet#6467 and #6698, #6724 and #6725 (follow-up #7019), and #5397. - Future work, including a nested TPC-DS. ## Draft status - The `author:` line is a placeholder (`comphead`) and needs a full name and affiliation. - The paper references in the introduction sections are written from memory and have not been checked. - Each result set is a single run per engine. The two sets use different Spark, Iceberg and Comet builds and the cluster size is not recorded with the runs, so the post does not attribute the change in Comet's time to individual changes. - The preview has not been built with Pelican. The `site/` branch name only stages for branches in this repository, so a maintainer may need to push a branch here, or the site can be built locally with `make`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
