Re: [QUESTION]: Caching UDFs

2019-08-08 Thread Igor Guzenko
Hello Charles, Although the idea seems good in general, actual implementation may cause much more issues than solve. If you are thinking about distributed cache, then every cache miss on current drillbit will cause network request to other drillbits, passing function arguments along with ids of

Re: [QUESTION]: Caching UDFs

2019-08-08 Thread Paul Rogers
Hi Charles, In general, we cannot know if a function is deterministic. Your function might be rand(seed, max). It might do a JDBC lookup or a REST call. Drill can't know (unless we add some way to know that a function is deterministic: maybe a @Deterministic annotation.) That said, you can

[QUESTION]: Caching UDFs

2019-08-08 Thread Charles Givre
Hello Drill Devs, I have a question about UDFs. Let's say you have a non-trivial UDF called foo(x,y) which returns some value. Assuming that if the arguments are the same, the function foo() will return the same result, does Drill have any optimizations to prevent running the non-trivial