dimas-b commented on code in PR #2851: URL: https://github.com/apache/polaris/pull/2851#discussion_r2455945559
########## site/content/blog/2025/10/21/starrocks-polaris-integration.md: ########## @@ -0,0 +1,322 @@ +--- +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +title: "StarRocks and Apache Polaris Integration: Building a Unified, High-Performance Data Lakehouse" +date: 2025-10-21 +author: Wayne Yang +--- +## Introduction: Why StarRocks + Polaris? + +### Modern Data-Architecture Pain Points: Silos & Engine Lock-in +Today’s data-driven enterprises face two chronic ailments: + +* **Data silos** : transactional data sits in RDBMS, click-streams in an S3 data lake, CRM data in a SaaS vault, etc. Cross-domain analysis requires expensive ETL and still arrives stale. + +* **Engine lock-in** : every OLAP engine optimises for its own metadata layer and file layout. Migrating to a faster or cheaper engine means re-formatting, re-writing and re-governing years of data. + +An open, interoperable architecture is no longer a luxury, it is a survival requirement. + +### StarRocks at a Glance: Speed & Simplicity +[StarRocks](https://www.starrocks.io/) is a modern MPP database designed for high-performance analytics and real-time data processing. Key features include: + +* MPP, fully-vectorised execution, CBO that thrives on complex multi-table joins. + +* Sub-second response on TB-scale data without pre-aggregation. + +* Compute-storage separation since v3.0: scale stateless compute pods in seconds, keep data in cheap object storage. + +### Apache Polaris (Incubating): Vendor-Neutral Iceberg Catalog +* 100% open-source implementation of the Iceberg REST Catalog API. + +* Pluggable metadata backend (PostgreSQL, in-mem) and multi-cloud storage support. + +* Fine-grained RBAC + credential-vending (temporary STS tokens) for secure, governed sharing across engines. + +### Business Value of the Pairing +**StarRocks** delivers performance; **Polaris** delivers openness. Together they let you: +* Keep **ONE** copy of data in Iceberg on S3, queryable by Spark, Flink, Trino, and StarRocks concurrently. + +* Maintain **ONE** set of role-based permissions in Polaris that apply to every engine. + +* Deliver BI dashboards, ad-hoc exploration and lightweight ETL operations on the same fresh dataset, zero data movement, zero lock-in. + +## Architecture + + +**Polaris** acts as the single source of truth for: + +* Table schema, partitions, snapshots + +* Role-based access control (RBAC) + +* Short-lived, scoped cloud credentials (credential vending) + +**StarRocks** acts as a stateless compute layer: + +* Discover Iceberg metadata via REST calls + +* Directly reads Parquet/ORC files from cloud storage using the vendored credentials + +* Applies its own CBO and vectorised execution for query acceleration + +## Deploy and Configure Polaris + +### Clone Source Code and Start Polaris + +1. Clone source code and checkout to released version + +User can get the latest released version from https://github.com/apache/polaris/releases +```bash +# download the latest released version 1.1.0-incubating +wget https://dlcdn.apache.org/incubator/polaris/1.1.0-incubating/apache-polaris-1.1.0-incubating.tar.gz +tar -xvzf apache-polaris-1.1.0-incubating.tar.gz +cd apache-polaris-1.1.0-incubating +``` + +2. Build Polaris Review Comment: Fair point. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
