Hi all, Thanks for all of your quick response! I will bring up the VOTE.
Best, Wei > 在 2019年12月5日，10:39，Jark Wu <imj...@gmail.com> 写道： > > Hi Wei, > > Thanks for bringing this discussion up, the changes look good to me. > Looking forward to the vote. And you can prepare the pull request at the > same time (in order to checkin in time). > > Best, > Jark > > On Thu, 5 Dec 2019 at 10:27, Hequn Cheng <chenghe...@gmail.com> wrote: > >> Hi all, >> >> Thanks a lot for the discussion! Using "#" also makes sense to me. >> And +1 to have these improvements in 1.10 as we don't want to >> introduce compatibility problems later. >> >> Looking forward to the vote! >> >> Best, Hequn >> >> >> >> On Thu, Dec 5, 2019 at 10:02 AM jincheng sun <sunjincheng...@gmail.com> >> wrote: >> >>> Hi all, >>> >>> Thanks for the quick response Aljoscha & Wei ! >>> >>> It seems unify the options is necessary, and 1.10 will be code frozen. I >>> would be like to bring up the VOTE thread for this change ASAP, and more >>> detail can continue discuss in the PR. >>> >>> What do you think? >>> >>> Best, >>> Jincheng >>> >>> Aljoscha Krettek <aljos...@apache.org> 于2019年12月4日周三 下午5:11写道： >>> >>>> Perfect, thanks for the background info! I also found this section now, >>>> which mentions that it comes from Hadoop: >>>> >>> >> https://spark.apache.org/docs/latest/running-on-yarn.html#important-notes. >>>> >>>> I think the proposed changes are good! >>>> >>>> Best, >>>> Aljoscha >>>> >>>>> On 4. Dec 2019, at 04:34, Wei Zhong <weizhong0...@gmail.com> wrote: >>>>> >>>>> Hi Aljoscha, >>>>> >>>>> Thanks for your reply! Before bringing up this discussion I did some >>>> research on commonly used separators for options that take multiple >>> values. >>>> I have considered ",", ":" and "#". Finally I chose "#" as the >> separator >>> of >>>> "--pyRequirements". >>>>> >>>>> For ",", it is the most widely used separator. Many projects use it >> as >>>> the separator of the values in same level. e.g. "-Dexcludes" in Maven, >>>> "--files" in Spark and "-pyFiles" in Flink. But the second parameter of >>>> "--pyRequirements", the requirement cached directory, is not at the >> same >>>> level as its first parameter (the requirements file). It is secondary >> and >>>> is only needed when the packages in the requirements file can not be >>>> downloaded from the package index server. >>>>> >>>>> For ":", it is used as a path separator in most cases. e.g. main >>>> arguments of scp (secure copy), "--volume" in Docker and "-cp" in Java. >>> But >>>> as we support accept a URI as the file path, which contains ":" in most >>>> cases, ":" can not be used as the separator of "--pyRequirements". >>>>> >>>>> For "#", it is really rarely used as a separator for multiple >> values. I >>>> only find Spark using "#" as the separator for option "--files" and >>>> "--archives" between file path and target file/directory name. After >> some >>>> research I find that this usage comes from the URI fragment. We can >>> append >>>> a secondary resource as the fragment of the URI after a number sign >> ("#") >>>> character. As we treat user file paths as URIs when parsing command >> line, >>>> using "#" as the separator of "--pyRequirements" makes sense to me, >> which >>>> means the second parameter is the fragment of the first parameter. The >>>> definition of URI fragment can be found here [1]. >>>>> >>>>> The reason of using "#" in "--pyArchives" as the separator of file >> path >>>> and targer directory name is the same as above. >>>>> >>>>> Best, >>>>> Wei >>>>> >>>>> [1] https://tools.ietf.org/html/rfc3986#section-3.5 >>>>> >>>>>> 在 2019年12月3日，22:02，Aljoscha Krettek <aljos...@apache.org> 写道： >>>>>> >>>>>> Hi, >>>>>> >>>>>> Yes, I think it’s a good idea to make the options uniform. Using ‘#’ >>> as >>>> a separator for options that take two values seems a bit strange to me, >>> did >>>> you research if any other CLI tools have this convention? >>>>>> >>>>>> Side note: I don’t like that our options use camel-case, I think >>> that’s >>>> very non-standard. But that’s how it is now… >>>>>> >>>>>> Best, >>>>>> Aljoscha >>>>>> >>>>>>> On 3. Dec 2019, at 10:14, jincheng sun <sunjincheng...@gmail.com> >>>> wrote: >>>>>>> >>>>>>> Thanks for bringup this discussion Wei! >>>>>>> I think this is very important for Flink User, we should contains >>> this >>>>>>> changes in Flink 1.10. >>>>>>> +1 for the optimization from the perspective of user convenience >> and >>>> the >>>>>>> unified use of Flink command line parameters. >>>>>>> >>>>>>> Best, >>>>>>> Jincheng >>>>>>> >>>>>>> Wei Zhong <weizhong0...@gmail.com> 于2019年12月2日周一 下午3:26写道： >>>>>>> >>>>>>>> Hi everyone, >>>>>>>> >>>>>>>> I wanted to bring up the discussion of improving the Pyflink >> command >>>> line >>>>>>>> options. >>>>>>>> >>>>>>>> A few command line options have been introduced in the FLIP-78 >> [1], >>>> i.e. >>>>>>>> "python-executable-path", "python-requirements","python-archive", >>> etc. >>>>>>>> There are a few problems with these options, i.e. the naming >> style, >>>>>>>> variable argument options, etc. >>>>>>>> >>>>>>>> We want to make some adjustment of FLIP-78 to improve the newly >>>> introduced >>>>>>>> command line options, here is the design doc: >>>>>>>> >>>>>>>> >>>> >>> >> https://docs.google.com/document/d/1R8CaDa3908V1SnTxBkTBzeisWqBF40NAYYjfRl680eg/edit?usp=sharing >>>>>>>> < >>>>>>>> >>>> >>> >> https://docs.google.com/document/d/1R8CaDa3908V1SnTxBkTBzeisWqBF40NAYYjfRl680eg/edit?usp=sharing >>>>>>>>> >>>>>>>> Looking forward to your feedback! >>>>>>>> >>>>>>>> Best, >>>>>>>> Wei >>>>>>>> >>>>>>>> [1] >>>>>>>> >>>> >>> >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-78%3A+Flink+Python+UDF+Environment+and+Dependency+Management >>>>>>>> < >>>>>>>> >>>> >>> >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-78:+Flink+Python+UDF+Environment+and+Dependency+Management >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >>>> >>> >>
