Hi Dawid, I think defering the instantiation of temporary functions to compile time is quite a good idea but needs further discussion. As it is orthogonal with this FLIP, we could continue the discussion in a new thread later. What do you think?
Best, Wei > 在 2020年3月5日，21:11，Wei Zhong <weizhong0...@gmail.com> 写道： > > Hi Dawid, > > Thanks for your suggestion. > > After some investigation, there are two designs in my mind about how to defer > the instantiation of temporary system function and temporary catalog function > to compile time. > > 1. FunctionCatalog accepts both FunctionDefinitions and uninstantiated > temporary functions. The uninstantiated temporary functions will be > instantiated when compiling. There is no public API change in this design, > but the FunctionCatalog needs to store and process both FunctionDefinitions > and uninstantiated temporary functions. > > 2. FunctionCatalog accepts only uninstantiated temporary functions. In this > design we need to remove those APIs that accepts FunctionDefinitions from > TableEnvironment, i.e. `void createTemporaryFunction(String path, > UserDefinedFunction functionInstance)` and `void > createTemporarySystemFunction(String name, UserDefinedFunction > functionInstance)`. But the FunctionCatalog only needs to store and process > uninstantiated temporary functions. > > As I don't know the details about the plan to store temporary functions as > catalog functions instead of FunctionDefinitions, I'm not sure which solution > fits more. It would be great if you could share more details or share some > thoughts on these two solutions? > > Best, > Wei > >> 在 2020年3月4日，16:17，Dawid Wysakowicz <dwysakow...@apache.org> 写道： >> >> Hi all, >> I had a really quick look and from my perspective the proposal looks fine. >> I share Jarks opinion that the instantiation could be done at a later >> stage. I agree with Wei it requires some changes in the internal >> implementation of the FunctionCatalog, to store temporary functions as >> catalog functions instead of FunctionDefinitions, but we have that on our >> agenda anyway. I would suggest investigating if we could do that as part of >> this flip already. Nevertheless this in theory can be also done later. >> >> Best, >> Dawid >> >> On Mon, 2 Mar 2020, 14:58 Jark Wu, <imj...@gmail.com> wrote: >> >>> Thanks for the explanation, Wei! >>> >>> On Mon, 2 Mar 2020 at 20:59, Wei Zhong <weizhong0...@gmail.com> wrote: >>> >>>> Hi Jark, >>>> >>>> Thanks for your suggestion. >>>> >>>> Actually, the timing of starting a Python process depends on the UDF >>> type, >>>> because the Python process is used to provide the necessary information >>> to >>>> instantiate the FunctionDefinition object of the Python UDF. For catalog >>>> function, the FunctionDefinition will be instantiated when compiling the >>>> job, which means the Python process is required during the compilation >>>> instead of the registeration. For temporary system function and temporary >>>> catalog function, the FunctionDefinition will be instantiated during the >>>> UDF registeration, so the Python process need to be started at that time. >>>> >>>> But this FLIP will only support registering the temporary system function >>>> and temporary catalog function in SQL DDL because registering Python UDF >>> to >>>> catalog is not supported yet. We plan to support the registeration of >>>> Python catalog function (via Table API and SQL DDL) in a separate FLIP. >>>> I'll add a non-goal section to the FLIP page to illustrate this. >>>> >>>> Best, >>>> Wei >>>> >>>> >>>>> 在 2020年3月2日，15:11，Jark Wu <imj...@gmail.com> 写道： >>>>> >>>>> Hi Weizhong, >>>>> >>>>> Thanks for proposing this feature. In geneal, I'm +1 from the table's >>>> view. >>>>> >>>>> I have one suggestion: I think the register python function into >>> catalog >>>>> doesn't need to startup python process (the "High Level Sequence >>> Diagram" >>>>> in your FLIP). >>>>> Because only meta-information is persisted into catalog, we don't need >>> to >>>>> store "return type", "input types" into catalog. >>>>> I guess the python process is required when compiling a SQL job. >>>>> >>>>> Best, >>>>> Jark >>>>> >>>>> >>>>> >>>>> On Fri, 28 Feb 2020 at 19:04, Benchao Li <libenc...@gmail.com> wrote: >>>>> >>>>>> Big +1 for this feature. >>>>>> >>>>>> We built our SQL platform on Java Table API, and most common UDF are >>>>>> implemented in Java. However some python developers are not familiar >>>> with >>>>>> Java/Scala, and it's very inconvenient for these users to use UDF in >>>> SQL. >>>>>> >>>>>> Wei Zhong <weizhong0...@gmail.com> 于2020年2月28日周五 下午6:58写道： >>>>>> >>>>>>> Thank for your reply Dan! >>>>>>> >>>>>>> By the way, this FLIP is closely related to the SQL API. @Jark Wu < >>>>>>> imj...@gmail.com> @Timo <twal...@apache.org> could you please take a >>>>>>> look? >>>>>>> >>>>>>> Thanks, >>>>>>> Wei >>>>>>> >>>>>>>> 在 2020年2月25日，16:25，zoudan <zoud...@163.com> 写道： >>>>>>>> >>>>>>>> +1 for supporting Python UDF in Java/Scala Table API. >>>>>>>> This is a great feature and would be helpful for python users! >>>>>>>> >>>>>>>> Best, >>>>>>>> Dan Zou >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Benchao Li >>>>>> School of Electronics Engineering and Computer Science, Peking >>>> University >>>>>> Tel:+86-15650713730 >>>>>> Email: libenc...@gmail.com; libenc...@pku.edu.cn >>>>>> >>>>>> >>>> >>>> >>> >
