Github user arina-ielchiieva commented on the issue:
https://github.com/apache/drill/pull/701
@jinfengni
1. Depending on how often udfs are added, we don't expect it to happen
often though. But you are correct about the overhead for the queries that do
not use dynamic UDFs.
2. You are right, function registry can be checked several times and can
slow down the entire query, It's hard to say how much performance will be slow
down, as it may depend on many factors, like number of parallel queries, number
of not exact functions in query, ZK time of response and so on).
3. Refresh function registry function is considered but as part of MVCC. It
could help in current approach but still it could not guarantee that after
issuing the refresh command all drillbits will sync their local function
registries with remote one, unless refresh function would wait for all
drillbits to send their confirmation that sync was done. But what if one of
drillbits fails to sync, should refresh function have retry mechanism or fail
immediately, how long it could take the user to wait for refresh command to
finish execution etc. With MVCC refresh command would need to guarantee that
only current drillbit is in sync and all above questions will be dropped (more
in MVCC doc).
Anyway, you are totally right that current approach is covering only the
gap with function overloading and not optimal and may slow down the queries.
Having refresh command might partially solve the problem as well but might have
some issues to be covered. So it's better to dive in MVCC for the most optimal
implementation.
Regarding this pull request I don't have strong feelings if it should be
merged or not. Yes, it would solve problem with functions overloading but it
may impact performance but it's hard to say how much since many factors may
have influence.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
