Re: [VOTE] Update the committer guidelines to clarify when to commit changes.

+1 Xiao On Fri, Jul 31, 2020 at 9:32 AM Mridul Muralidharan wrote: > > +1 > > Thanks, > Mridul > > On Thu, Jul 30, 2020 at 4:49 PM Holden Karau wrote: > >> Hi Spark Developers, >> >> After the discussion of the proposal to amend Spark committer guidelines, >> it appears folks are generally in

Re: [VOTE] Update the committer guidelines to clarify when to commit changes.

+1 Thanks, Mridul On Thu, Jul 30, 2020 at 4:49 PM Holden Karau wrote: > Hi Spark Developers, > > After the discussion of the proposal to amend Spark committer guidelines, > it appears folks are generally in agreement on policy clarifications. (See >

Re: [VOTE] Update the committer guidelines to clarify when to commit changes.

+1 Thanks Shivaram On Thu, Jul 30, 2020 at 11:56 PM Wenchen Fan wrote: > > +1, thanks for driving it, Holden! > > On Fri, Jul 31, 2020 at 10:24 AM Holden Karau wrote: >> >> +1 from myself :) >> >> On Thu, Jul 30, 2020 at 2:53 PM Jungtaek Lim >> wrote: >>> >>> +1 (non-binding, I guess) >>>

Re: [DISCUSS] "latestFirst" option and metadata growing issue in File stream source

Hi German, option 1 isn't about "deleting" the old files, as your input directory may be accessed by multiple queries. Kafka centralizes the maintenance of input data hence possible to apply retention without problem. option 1 is more about "hiding" the old files being read, so that end users