FrankChen021 commented on code in PR #20544: URL: https://github.com/apache/druid/pull/20544#discussion_r4325136212
########## web-console/e2e-tests/README.md: ########## @@ -0,0 +1,192 @@ +<!-- + ~ Licensed to the Apache Software Foundation (ASF) under one + ~ or more contributor license agreements. See the NOTICE file + ~ distributed with this work for additional information + ~ regarding copyright ownership. The ASF licenses this file + ~ to you under the Apache License, Version 2.0 (the + ~ "License"); you may not use this file except in compliance + ~ with the License. You may obtain a copy of the License at + ~ + ~ http://www.apache.org/licenses/LICENSE-2.0 + ~ + ~ Unless required by applicable law or agreed to in writing, + ~ software distributed under the License is distributed on an + ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + ~ KIND, either express or implied. See the License for the + ~ specific language governing permissions and limitations + ~ under the License. + --> + +# End-to-end tests + +These tests drive the web console in a real (headless) Chromium against a real Druid cluster. They catch what the unit +tests can't: the console and Druid's APIs disagreeing, a flow across several views breaking, or a request that only +fails against a live cluster. + +They use the Playwright test runner (`@playwright/test`, configured in `../playwright.config.ts`). The Chromium it +drives is downloaded by the `playwright-chromium` package's install script, so keep the two packages on the same +version. They run in CI on embedded clusters (see below), as part of `.github/scripts/web-checks.sh`. + +## Running them + +The tests need a Druid cluster with its Router on :8888 (every request goes through the console's service), running +on the same machine (its tasks read the `examples/quickstart/tutorial` data files of this checkout): + +```bash +script/druid build # once, builds a distribution with the extensions the tests need +script/druid start +npm run test-e2e # all the tests, one at a time (they share the cluster) +script/druid stop +``` + +Before the tests start, a global setup (`util/global-setup.ts`) waits for the console's SQL endpoint to answer. + +- **One file**: `npm run test-e2e -- cancel-query` (any `playwright test` arguments go after `--`) +- **See the browser**: `npm run test-e2e -- --headed`, step through a test with `--debug`, or pick and watch tests in + the UI mode with `--ui` +- **Test a dev server instead of the bundled console**: `DRUID_E2E_TEST_UNIFIED_CONSOLE_PORT=18081 npm run test-e2e`. + Without it the tests run against the console that the router on :8888 serves, which was bundled when Druid was + built, not your working tree. +- **On failure** a test leaves a screenshot, a trace (`npx playwright show-trace <path>`: every action with the DOM, + console and network at that point) and an `error-context.md` (the page as an accessibility tree) in `test-results/` + (git ignored). Every HTTP response with a status of 400 or more is also logged with its body + (`util/fixtures.ts`). +- **See the steps a test takes**: `E2E_SHOW_STEPS=true npm run test-e2e`. Each test saves a screenshot at each of + its steps (each step of the data loader, each view it reads, a query's results, a dialog it fills in, and the + page as it ends) to `steps/<spec>-001.png`, `-002.png`, ... (git ignored, replaced by the next run of the spec). + The quickest way to see what a test does, especially one someone else wrote. It works on the embedded clusters + too: `E2E_SHOW_STEPS=true mvn -pl embedded-tests verify -Pweb-console-tests ...`. + +### On an embedded cluster (as in CI) + +Each spec is run by a `*WebConsoleTest` class in `embedded-tests`, in the package of the functionality it covers. The +class starts an embedded cluster (in one JVM, with an in-memory metadata store, its Router on :8888) with the resources +the spec needs, and runs the spec on it with `npx playwright test <spec>`: + +| Spec | Run by (in `org.apache.druid.testing.embedded`) | +|-----------------------------------------------|-------------------------------------------------| +| `tutorial-batch.spec.ts`, `reindexing.spec.ts` | `indexing.BatchIndexingWebConsoleTest` | +| `auto-compaction.spec.ts` | `compact.AutoCompactionWebConsoleTest` | +| `multi-stage-query.spec.ts` | `msq.MultiStageQueryWebConsoleTest` | +| `cancel-query.spec.ts` | `query.SqlQueryCancelWebConsoleTest` | +| `s3-ingestion.spec.ts` | `s3.S3WebConsoleTest` (with an S3 container) | +| `input-formats.spec.ts` | `indexer.InputFormatsWebConsoleTest` | +| `kafka-ingestion.spec.ts` | `indexing.KafkaWebConsoleTest` (with Kafka) | +| `sql-ingestion.spec.ts` | `msq.SqlIngestionWebConsoleTest` | +| `datasource-actions.spec.ts` | `indexing.DatasourceActionsWebConsoleTest` | +| `retention-rules.spec.ts` | `server.RetentionRulesWebConsoleTest` | +| `jdbc-lookup.spec.ts` | `lookup.JdbcLookupWebConsoleTest` | +| `dart.spec.ts` | `msq.DartWebConsoleTest` | +| `input-formats.spec.ts` | Loads the same 10 rows (over 2 days) of Wikipedia edits from local CSV, TSV, Parquet, ORC and Avro OCF files through the data loader, one test per format. Checks that the **Parse data** step picks the right input format by itself and parses the columns, and that the datasource ends up as 2 segments with 10 rows. Skipped without `DRUID_E2E_TEST_DATA_DIR` (the data files of embedded-tests): it runs on an embedded cluster with the Avro, Parquet and ORC extensions, from `InputFormatsWebConsoleTest`. | Review Comment: [P3] Keep the embedded-test table columns consistent **Finding:** The table beginning at line 66 declares its second column as `Run by`, but lines 80-86 append long `What it does` descriptions as additional rows without a new header. These rows render in the wrong column and duplicate several spec entries, making the embedded-test mapping misleading. **Suggestion:** Move the descriptions under a separate `Spec | What it does` table or remove the duplicate block so the embedded-cluster table contains only runner mappings. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
