FrankChen021 commented on code in PR #19698:
URL: https://github.com/apache/druid/pull/19698#discussion_r3703838753
##########
server/src/main/java/org/apache/druid/metadata/SQLMetadataConnector.java:
##########
@@ -1038,6 +1045,185 @@ public void createAuditTable()
}
}
+ @Override
+ public void exportTable(
+ final String tableName,
+ final String outputPath
+ )
+ {
+ exportTable(tableName, outputPath, null);
+ }
+
+ /**
+ * Exports a table to a CSV file, emitting the given columns in the given
order.
+ *
+ * @param columns columns to export in the desired order, or null to export
all columns in the
+ * order reported by the database
+ */
+ public void exportTable(
+ final String tableName,
+ final String outputPath,
+ @Nullable final List<String> columns
+ )
+ {
+ exportTableWithJdbc(tableName, outputPath, columns);
+ }
+
+ /**
+ * Returns the columns of the given table, in the order reported by the
database.
+ * Returns an empty list if the table does not exist or the metadata cannot
be read.
+ *
+ * The lookup is scoped to the schema of the current connection, which is
the schema an unqualified
+ * table name resolves to, and the table name is escaped so that it is
matched literally rather than
+ * as a {@link DatabaseMetaData#getColumns} search pattern.
+ */
+ public List<String> getTableColumns(final String tableName)
+ {
+ return getDBI().withHandle(handle -> {
+ final List<String> columns = new ArrayList<>();
+ try {
+ if (tableExists(handle, tableName)) {
+ final Connection conn = handle.getConnection();
+ final DatabaseMetaData dbMetaData = conn.getMetaData();
+ try (ResultSet rs = dbMetaData.getColumns(
+ null,
+ escapeMetaDataSearchString(dbMetaData, conn.getSchema()),
+ escapeMetaDataSearchString(dbMetaData, tableName),
Review Comment:
[P2] Normalize PostgreSQL identifiers before metadata lookup
PostgreSQL folds unquoted identifiers to lowercase, while
DatabaseMetaData.getColumns uses a case-sensitive LIKE. With a valid mixed-case
--base, tableExists succeeds via ILIKE and ordinary SQL resolves the lowercase
table, but this lookup returns no columns, causing segment export to abort.
Normalize according to JDBC identifier rules or resolve the actual table name
first.
##########
docs/operations/export-metadata.md:
##########
@@ -151,14 +164,18 @@ In the example command above:
After running the tool, the output directory will contain
`<table-name>_raw.csv` and `<table-name>.csv` files.
-The `<table-name>_raw.csv` files are intermediate files used by the tool,
containing the table data as exported by Derby without modification.
+The `<table-name>_raw.csv` files are intermediate files used by the tool,
containing the table data as exported from the source database without
deep-storage rewrites. BLOB columns are hex-encoded and booleans are written as
`true`/`false` strings.
The `<table-name>.csv` files are used for import into another database such as
MySQL and PostgreSQL and have any configured deep storage location rewrites
applied.
Example import commands for Derby, MySQL, and PostgreSQL are shown below.
These example import commands expect `/tmp/csv` and its contents to be
accessible from the server. For other options, such as importing from the
client filesystem, please refer to the database's documentation.
+The segments table is exported in a fixed column order, independent of the
physical column order of the source table: `id`, `dataSource`, `created_date`,
`start`, `end`, `partitioned`, `version`, `used`, `payload`,
`used_status_last_updated`, `indexing_state_fingerprint`,
`upgraded_from_segment_id`, followed by `schema_fingerprint` and `num_rows` if
the source table has them. Add `schema_fingerprint,num_rows` to the end of the
segments column list in the import commands below if those columns are present.
Review Comment:
[P1] Keep import lists aligned with older segment schemas
orderSegmentsColumns skips every absent column, and the new test explicitly
supports legacy nine-column tables, but the documented PostgreSQL COPY always
expects the three later columns. A nine-field export therefore fails with
missing data. The instructions or generated output must identify every optional
post-payload column and omit absent ones from the import list and FORCE_NULL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
