Zhangg7723 opened a new pull request, #6498:
URL: https://github.com/apache/iceberg/pull/6498
This PR gives a support for Flink lookup join like:
INSERT INTO sink_table SELECT s.id, d.name from (SELECT *, PROCTIME() AS
proctime FROM source_table) AS s LEFT JOIN dim_table FOR SYSTEM_TIME AS OF
s.proctime as d ON s.id = d.id.
It loads the right table records into cache synchronized or asynchronized
when lookup keys are matched . the cache supports these properties:
- lookup.cache.type
value "all" loads all rows for initialization work, value "lru" manages
rows cache in LRU mode.
- lookup.cache.ttl
- lookup.cache.max-rows
According to my tests, lookup.cache.type=all has a good performance, and
loading cache lazily has a terrible performance, maybe it will be better when
secondary index is mature。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
