[
https://issues.apache.org/jira/browse/ATLAS-5021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pinal Shah updated ATLAS-5021:
------------------------------
Description:
h2. Description
*Overview:*
* A standalone service to extract metadata from Trino via JDBC connection.
Trino's catalogs, schemas, tables, and columns information are fetched and
converted into Atlas-compatible json format and ingested to Atlas through REST
API
* Handle creation/updation/deletion of metadata
* Handle deletion of native hook entity metadata
*Configuration Setup: configure the `atlas-trino-extractor.properties` file:*
{code:java}
# Atlas connection
atlas.rest.address=http://localhost:21000/
# Trino jdbc url, If SSL is enabled, append below value with ";SSL=true"
atlas.trino.jdbc.address=jdbc:trino://<host>:<port>/
atlas.trino.jdbc.user=your-username
# Catalogs to extract
atlas.trino.catalogs.registered=hive_catalog,iceberg_catalog {code}
*Basic Execution:*
{code:java}
# Extract all registered catalogs
./bin/run-trino-extractor.sh
# Extract specific catalog
./bin/run-trino-extractor.sh -c my_catalog
# Schedule periodic extraction (every 6 hours)
./bin/run-trino-extractor.sh -cx "0 0 */6 * * ?"{code}
> Extract Metadata from Trino periodically
> ----------------------------------------
>
> Key: ATLAS-5021
> URL: https://issues.apache.org/jira/browse/ATLAS-5021
> Project: Atlas
> Issue Type: Sub-task
> Components: atlas-core
> Reporter: Pinal Shah
> Assignee: Pinal Shah
> Priority: Major
> Fix For: 3.0.0, 2.5.0
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> h2. Description
>
> *Overview:*
> * A standalone service to extract metadata from Trino via JDBC connection.
> Trino's catalogs, schemas, tables, and columns information are fetched and
> converted into Atlas-compatible json format and ingested to Atlas through
> REST API
> * Handle creation/updation/deletion of metadata
> * Handle deletion of native hook entity metadata
> *Configuration Setup: configure the `atlas-trino-extractor.properties` file:*
>
> {code:java}
> # Atlas connection
> atlas.rest.address=http://localhost:21000/
> # Trino jdbc url, If SSL is enabled, append below value with ";SSL=true"
> atlas.trino.jdbc.address=jdbc:trino://<host>:<port>/
> atlas.trino.jdbc.user=your-username
> # Catalogs to extract
> atlas.trino.catalogs.registered=hive_catalog,iceberg_catalog {code}
> *Basic Execution:*
>
> {code:java}
> # Extract all registered catalogs
> ./bin/run-trino-extractor.sh
> # Extract specific catalog
> ./bin/run-trino-extractor.sh -c my_catalog
> # Schedule periodic extraction (every 6 hours)
> ./bin/run-trino-extractor.sh -cx "0 0 */6 * * ?"{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
