Gabriel39 opened a new pull request, #69969:
URL: https://github.com/apache/doris/pull/69969
### What problem does this PR solve?
Issue Number: N/A
MongoDB exposes `ObjectId` as a plugin-defined Trino type. Doris did not
register those types in either plugin loader, rejected ObjectId during schema
mapping, and discarded the connector's default hidden `_id` column.
Register plugin-defined types in both FE and BE loaders, map ObjectId to
`VARBINARY(12)` recursively, and expose the default MongoDB `_id` while
retaining the hiding of other columns. The existing binary scanner preserves
all twelve bytes without UTF-8 conversion. MongoDB remains an
operator-installed Trino 435 plugin; no MongoDB runtime dependency is added to
Doris.
Add an opt-in MongoDB 8.0.20 Docker fixture and four regression suites with
**99 result checks and eight expected-error checks**. Coverage includes
primitive/nested types, null and missing fields, numeric/date boundaries,
schema discovery and permissions, filters, joins, aggregates, subqueries,
windows, views, internal-table joins, materialization, multi-page reads and
repeated early termination. Golden files were generated and then compared using
the standard regression runner.
ObjectId retains the existing VARBINARY operator/storage restrictions. Use
`from_binary(_id)` when a hexadecimal string is needed for filtering, joining,
grouping or internal-table materialization. The regression tests cover both
these usable forms and the unsupported raw binary forms.
### Validation
- FE package and relevant BE Java plugins built with `build.sh --fe`; UI,
Hive UDF, OBS/COS dependencies and unrelated BE Java plugins were excluded for
the local build.
- Standard `run-fe-ut.sh --run` entry point passed for the relevant test
classes; focused module validation after the final predicate tests passed **48
unit checks** in total.
- `run-regression-test.sh --run -d
external_table_p0/trino_connector/mongodb`: **four suites passed**, with
MongoDB enabled and the official complete Trino 435 MongoDB plugin ZIP
installed.
- Checkstyle/`validate` passed for all changed Java modules; launcher shell
syntax, Compose configuration and fixture-readiness command passed.
- Integration used the newly built FE/Java plugins with an existing master
ASAN BE binary at `bbe765a93d9`. BE C++ was not rebuilt locally; full
compilation is left to PR CI.
The fixture README documents setup, the opt-in switch, coverage and current
limitations. The MongoDB tests are disabled by default and require the matching
plugin to be installed on FE and BE.
### Release note
Support MongoDB ObjectId columns, including the default `_id` and nested
ObjectId fields, through Trino connector catalogs as `VARBINARY(12)`.
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- [x] Manual test: real MongoDB fixture, complete connector ZIP, FE/BE
queries and exact byte validation
- Behavior changed:
- [x] Yes. ObjectId fields become queryable as VARBINARY(12), and the
default hidden MongoDB `_id` is exposed.
- Does this need documentation?
- [x] Yes. Fixture setup and supported query forms are documented in
`docker/thirdparties/docker-compose/mongodb/README.md`.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
