markhoerth opened a new issue, #12688:
URL: https://github.com/apache/gravitino/issues/12688
### What would you like to be improved?
### What's wrong
When the Trino connector fails to register a catalog, nothing tells
the operator. The plugin installs, the entry catalog loads, SHOW
CATALOGS lists the entry catalog and none of the metalake's catalogs,
and there is no error anywhere.
This was traced on a customer deployment over several days. The
sequence a customer would go through:
- SHOW CATALOGS shows nothing from their metalake. No error.
- The connector logs nothing at all, at any level, from any of its own
loggers, while io.trino and com.starburstdata log normally. The
connector carries its own log4j binding, so slf4j resolves to it
rather than to the server's logging, and its lines go nowhere unless
the operator finds the one line in the installation doc about a JVM
argument.
- With logging enabled, the error read "Trino server is not started",
which was wrong. Trino was running. The actual failure was TLS on the
connector's JDBC loopback to the coordinator.
- The fix was `trino.jdbc.ssl.verification`, a property absent from the
installation page, whose default is FULL, against a coordinator with
a self-signed certificate.
- Separately, when Iceberg REST discovery fails against a server that
predates the endpoint, the connector logs one WARN and then DEBUG
forever, and silently falls back to catalog-backend translation. The
feature the routing exists to deliver becomes a no-op with no
ongoing signal.
Every step of that is a diagnosability failure rather than a functional
one. The connector was working as designed at each point.
### What should change
- [ ] **1. The connector logs through the server's logging.** Every
other Trino plugin lets slf4j resolve to the server's logging and
needs no extra configuration to produce an error. Excluding
log4j-slf4j2-impl and log4j-core from the distribution would achieve
this. If the binding must stay, the distribution needs a log4j2
config that writes to the console.
- [ ] **2. A failed registerCatalog surfaces on the entry catalog's
status as ERROR by default,** rather than being swallowed.
- [ ] **3. "Trino server is not started" names what actually failed:**
the loopback connection, the URI it used, and the cause.
- [ ] **4. Discovery fallback is loud and repeated.** One WARN then
DEBUG forever is the worst available default for a silent no-op of
the feature's whole purpose.
- [ ] **5. The address returned by /api/system/iceberg-rest is
reachable by the caller.** On the deployment tested it returned
`http://localhost:9001/iceberg`, derived from the server's own bind
address, so a connector in a different pod dials its own localhost
and finds nothing. Derive it from the host the client used to reach
the API with the IRC port substituted, or take an explicitly
configured advertised URI. A discovery endpoint whose answer only
works inside the server's own pod cannot serve the remote callers it
exists for.
### Related
A separate operational problem found alongside these: any alterCatalog
property change causes the connector to unregister the catalog from
Trino and never re-create it, requiring a coordinator restart.
Removing a property self-heals within a poll cycle; adding or changing
one does not.
### How should we improve?
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
