Thanks! Now we get to debate what knobs to twiddle :-P

FYI, I did a simple run (just pushed to
https://github.com/apache/beam/compare/master...robertwb:yapf) to see
the impact. The diff is

    $ git diff --stat master
    ...
     547 files changed, 22118 insertions(+), 21129 deletions(-)

For reference

    $ find sdks/python/apache_beam -name '*.py' | xargs wc
    ...
    200424  612002 7431637 total

which means a little over 10% of lines get touched. I think there are
some options, such as SPLIT_ALL_TOP_LEVEL_COMMA_SEPARATED_VALUES and
COALESCE_BRACKETS, that will conform more to the style we are already
(mostly) following.


On Thu, Jan 23, 2020 at 1:59 AM Kamil Wasilewski
<kamil.wasilew...@polidea.com> wrote:
>
> Thank you Michał for creating the ticket. I have some free time and I'd like 
> to volunteer myself for this task.
> Indeed, it looks like there's consensus for `yapf`, so I'll try `yapf` first.
>
> Best,
> Kamil
>
>
> On Thu, Jan 23, 2020 at 10:37 AM Michał Walenia <michal.wale...@polidea.com> 
> wrote:
>>
>> Hi all,
>> I created a JIRA issue for this and summarized the available tools
>>
>> https://issues.apache.org/jira/browse/BEAM-9175
>>
>> Cheers,
>> Michal
>>
>> On Thu, Jan 23, 2020 at 1:49 AM Udi Meiri <eh...@google.com> wrote:
>>>
>>> Sorry, backing off on this due to time constraints.
>>>
>>> On Wed, Jan 22, 2020 at 3:39 PM Udi Meiri <eh...@google.com> wrote:
>>>>
>>>> It sounds like there's a consensus for yapf. I volunteer to take this on
>>>>
>>>> On Wed, Jan 22, 2020, 10:31 Udi Meiri <eh...@google.com> wrote:
>>>>>
>>>>> +1 to autoformatting
>>>>>
>>>>> On Wed, Jan 22, 2020 at 9:57 AM Luke Cwik <lc...@google.com> wrote:
>>>>>>
>>>>>> +1 to autoformatters. Also the Beam Java SDK went through a one time 
>>>>>> pass to apply the spotless formatting.
>>>>>>
>>>>>> On Tue, Jan 21, 2020 at 9:52 PM Ahmet Altay <al...@google.com> wrote:
>>>>>>>
>>>>>>> +1 to autoformatters and yapf. It appears to be a well maintained 
>>>>>>> project. I do support making a one time pass to apply formatting the 
>>>>>>> whole code base.
>>>>>>>
>>>>>>> On Tue, Jan 21, 2020 at 5:38 PM Chad Dombrova <chad...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> It'd be good if there was a way to only apply to violating (or at
>>>>>>>>> least changed) lines.
>>>>>>>>
>>>>>>>>
>>>>>>>> I assumed the first thing we’d do is convert all of the code in one 
>>>>>>>> go, since it’s a very safe operation. Did you have something else in 
>>>>>>>> mind?
>>>>>>>>
>>>>>>>> -chad
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Jan 21, 2020 at 1:56 PM Chad Dombrova <chad...@gmail.com> 
>>>>>>>>> wrote:
>>>>>>>>> >
>>>>>>>>> > +1 to autoformatting
>>>>>>>>> >
>>>>>>>>> > Let me add some nuance to that.
>>>>>>>>> >
>>>>>>>>> > The way I see it there are 2 varieties of formatters:  those which 
>>>>>>>>> > take the original formatting into consideration (autopep8) and 
>>>>>>>>> > those which disregard it (yapf, black).
>>>>>>>>> >
>>>>>>>>> > I much prefer yapf to black, because you have plenty of options to 
>>>>>>>>> > tweak with yapf (enough to make the output a pretty close match to 
>>>>>>>>> > the current Beam style), and you can mark areas to preserve the 
>>>>>>>>> > original formatting, which could be very useful with Pipeline 
>>>>>>>>> > building with pipe operators.  Please don't pick black.
>>>>>>>>> >
>>>>>>>>> > autopep8 is more along the lines of spotless in Java -- it only 
>>>>>>>>> > corrects code that breaks the project's style rules.  The big 
>>>>>>>>> > problem with Beam's current style is that it is so esoteric that 
>>>>>>>>> > autopep8 can't enforce it -- and I'm not just talking about 
>>>>>>>>> > 2-spaces, which I don't really have a problem with -- the problem 
>>>>>>>>> > is the use of either 2 or 4 spaces depending on context (expression 
>>>>>>>>> > start vs hanging indent, etc).  This is my *biggest* gripe about 
>>>>>>>>> > the current style.  PyCharm doesn't have enough control either.  
>>>>>>>>> > So, if we can choose a style that can be expressed by flake8 or 
>>>>>>>>> > pycodestyle then we can use autopep8 to enforce it.
>>>>>>>>> >
>>>>>>>>> > I'd prefer autopep8 to yapf because I like having a little wiggle 
>>>>>>>>> > room to influence the style, but on a big project like Beam all 
>>>>>>>>> > that wiggle room ends up to minor but noticeable inconsistencies in 
>>>>>>>>> > style throughout the project.  yapf ensures completely consistent 
>>>>>>>>> > style, but the tradeoff is that it's sometimes ugly, especially in 
>>>>>>>>> > scenarios with similar repeated entries like argparse, where yapf 
>>>>>>>>> > might insert line breaks in visually inconsistent and unappealing 
>>>>>>>>> > ways depending on the lengths of the keywords and expressions 
>>>>>>>>> > involved.
>>>>>>>>> >
>>>>>>>>> > Either way (but especially if we choose yapf) I think it'd be a 
>>>>>>>>> > nice addition to setup a pre-commit [1] config so that people can 
>>>>>>>>> > opt in to running *lightweight* autofixers prior to commit.  This 
>>>>>>>>> > will not only reduce dev frustration but will also reduce the 
>>>>>>>>> > amount of cpu cycles that Jenkins spends pointing out lint errors.
>>>>>>>>> >
>>>>>>>>> > [1] https://pre-commit.com/
>>>>>>>>> >
>>>>>>>>> > -chad
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > On Tue, Jan 21, 2020 at 12:52 PM Ismaël Mejía <ieme...@gmail.com> 
>>>>>>>>> > wrote:
>>>>>>>>> >>
>>>>>>>>> >> Last time we discussed this there seems not to be much progress 
>>>>>>>>> >> into autoformatting.
>>>>>>>>> >> This tool looks more tweakable, so maybe it could be more 
>>>>>>>>> >> appropriate for Beam's use case.
>>>>>>>>> >> https://github.com/google/yapf/
>>>>>>>>> >> WDYT?
>>>>>>>>> >>
>>>>>>>>> >>
>>>>>>>>> >> On Thu, May 30, 2019 at 10:50 AM Łukasz Gajowy 
>>>>>>>>> >> <lgaj...@apache.org> wrote:
>>>>>>>>> >>>
>>>>>>>>> >>> +1 for any autoformatter for Python SDK that does the job. My 
>>>>>>>>> >>> experience is that since spotless in Java SDK I would never start 
>>>>>>>>> >>> a new Java project without it. So many great benefits not only 
>>>>>>>>> >>> for one person coding but for all community.
>>>>>>>>> >>>
>>>>>>>>> >>> It is a GitHub UI issue that you cannot easily browse past the 
>>>>>>>>> >>> reformat. It is not actually that hard, but does take a couple 
>>>>>>>>> >>> extra clicks to get GitHub to display blame before a reformat. It 
>>>>>>>>> >>> is easier with the command line. I do a lot of code history 
>>>>>>>>> >>> digging and the global Java reformat is not really a problem.
>>>>>>>>> >>>
>>>>>>>>> >>> It's actually one more click on Github but I agree it's not the 
>>>>>>>>> >>> best way to search the history. The most convenient and clear one 
>>>>>>>>> >>> I've found so far is in Jetbrains IDEs (Intelij) where you can:
>>>>>>>>> >>>
>>>>>>>>> >>> right click on line number -> "annotate" -> click again -> 
>>>>>>>>> >>> "annotate previous revision" -> ...
>>>>>>>>> >>>
>>>>>>>>> >>> You can also use "compare with" to see the diff between two 
>>>>>>>>> >>> revisions.
>>>>>>>>> >>>
>>>>>>>>> >>> Łukasz
>>>>>>>>> >>>
>>>>>>>>> >>>
>>>>>>>>> >>>
>>>>>>>>> >>>
>>>>>>>>> >>>
>>>>>>>>> >>> czw., 30 maj 2019 o 06:15 Kenneth Knowles <k...@apache.org> 
>>>>>>>>> >>> napisał(a):
>>>>>>>>> >>>>
>>>>>>>>> >>>> +1 pending good enough tooling (I can't quite tell - seems there 
>>>>>>>>> >>>> are some issues?)
>>>>>>>>> >>>>
>>>>>>>>> >>>> On Wed, May 29, 2019 at 2:40 PM Katarzyna Kucharczyk 
>>>>>>>>> >>>> <ka.kucharc...@gmail.com> wrote:
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> What else actually we gain? My guess is faster PR review 
>>>>>>>>> >>>>> iteration. We will skip some of conversations about code style.
>>>>>>>>> >>>>
>>>>>>>>> >>>> ...
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> Last but not least, new contributor may be less discouraged. 
>>>>>>>>> >>>>> When I started contribute I didn’t know how to format my code 
>>>>>>>>> >>>>> and I lost a lot of time to add pylint and adjust IntelliJ. I 
>>>>>>>>> >>>>> eventually failed. Currently I write code intuitively and when 
>>>>>>>>> >>>>> I don’t forget I rerun tox.
>>>>>>>>> >>>>
>>>>>>>>> >>>>
>>>>>>>>> >>>> This is a huge benefit. This is why I supported it so much for 
>>>>>>>>> >>>> Java. It is a community benefit. You do not have to be a 
>>>>>>>>> >>>> contributor to the Python SDK to support this. That is why I am 
>>>>>>>>> >>>> writing here. Just eliminate all discussion of formatting. It 
>>>>>>>>> >>>> doesn't really matter what the resulting format is, if it is not 
>>>>>>>>> >>>> crazy to read. I strongly oppose maintaining a non-default 
>>>>>>>>> >>>> format.
>>>>>>>>> >>>>
>>>>>>>>> >>>> Reformating 20k lines or 200k is not hard. The Java global 
>>>>>>>>> >>>> reformat touched 50k lines. It does not really matter how big it 
>>>>>>>>> >>>> is. Definitely do it all at once if you think the tool is good 
>>>>>>>>> >>>> enough. And you should pin a version, so churn is not a problem. 
>>>>>>>>> >>>> You can upgrade the version and reformat in a PR later and that 
>>>>>>>>> >>>> is also easy.
>>>>>>>>> >>>>
>>>>>>>>> >>>> It is a GitHub UI issue that you cannot easily browse past the 
>>>>>>>>> >>>> reformat. It is not actually that hard, but does take a couple 
>>>>>>>>> >>>> extra clicks to get GitHub to display blame before a reformat. 
>>>>>>>>> >>>> It is easier with the command line. I do a lot of code history 
>>>>>>>>> >>>> digging and the global Java reformat is not really a problem.
>>>>>>>>> >>>>
>>>>>>>>> >>>> Kenn
>>>>>>>>> >>>>
>>>>>>>>> >>>>
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> Also everything will be formatted in a same way, so eventually 
>>>>>>>>> >>>>> it would be easier to read.
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> Moreover, as it was mentioned in previous emails - a lot of 
>>>>>>>>> >>>>> Jenkins failures won’t take place, so we save time and 
>>>>>>>>> >>>>> resources.
>>>>>>>>> >>>>>
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> One of disadvantages is that our pipelines has custom syntax 
>>>>>>>>> >>>>> and after formatting they looks a little bit weird, but maybe 
>>>>>>>>> >>>>> extending the only configurable option in Black - lines, from 
>>>>>>>>> >>>>> 88 to 110 would be solution.
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> Second one is that Black requires Python 3 to be run. I don’t 
>>>>>>>>> >>>>> know how big obstacle it would be.
>>>>>>>>> >>>>>
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> I believe there are two options how it would be possible to 
>>>>>>>>> >>>>> introduce Black. First: just do it, it will hurt but then it 
>>>>>>>>> >>>>> would be ok (same as a dentist appointment). Of course it may 
>>>>>>>>> >>>>> require some work to adjust linters. On the other hand we can 
>>>>>>>>> >>>>> do it gradually and start including sdk parts one by one - 
>>>>>>>>> >>>>> maybe it will be less painful?
>>>>>>>>> >>>>>
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> As an example I can share one of projects [2] I know that uses 
>>>>>>>>> >>>>> Black (they use also other cool checkers and pre-commit [3]). 
>>>>>>>>> >>>>> This is how looks their build with all checks [4].
>>>>>>>>> >>>>>
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> To sum up I believe that if we want improve our coding 
>>>>>>>>> >>>>> experience, we should improve our toolset. Black seems be 
>>>>>>>>> >>>>> recent and quite popular tool what makes think they won’t stop 
>>>>>>>>> >>>>> developing it.
>>>>>>>>> >>>>>
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> [1] 
>>>>>>>>> >>>>> https://stackoverflow.com/questions/4112410/git-change-styling-whitespace-without-changing-ownership-blame
>>>>>>>>> >>>>>
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> [2]  https://github.com/GoogleCloudPlatform/oozie-to-airflow
>>>>>>>>> >>>>>
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> [3] https://pre-commit.com
>>>>>>>>> >>>>>
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> [4] 
>>>>>>>>> >>>>> https://travis-ci.org/GoogleCloudPlatform/oozie-to-airflow/builds/538725689
>>>>>>>>> >>>>>
>>>>>>>>> >>>>>
>>>>>>>>> >>>>>
>>>>>>>>> >>>>> On Wed, May 29, 2019 at 2:01 PM Robert Bradshaw 
>>>>>>>>> >>>>> <rober...@google.com> wrote:
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> Reformatting to 4 spaces seems a non-starter to me, as it 
>>>>>>>>> >>>>>> would change nearly every single line in the codebase (and the 
>>>>>>>>> >>>>>> loss of all context as well as that particular line).
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> This is probably why the 2-space fork exists. However, we 
>>>>>>>>> >>>>>> don't conform to that either--we use 2 spaces for indentation, 
>>>>>>>>> >>>>>> but 4 for continuation indentation. (As for the history of 
>>>>>>>>> >>>>>> this, this goes back to Google's internal style guide, 
>>>>>>>>> >>>>>> probably motivated by consistency with C++, Java, ... and the 
>>>>>>>>> >>>>>> fact that with an indent level of 4 one ends up wrapping lines 
>>>>>>>>> >>>>>> quite frequently (it's telling that black's default line 
>>>>>>>>> >>>>>> length is 88)). This turns out to be an easy change to the 
>>>>>>>>> >>>>>> codebase.
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> Once we move beyond the 2 vs. 4 whitespace thing, I found that 
>>>>>>>>> >>>>>> this tool introduces a huge amount of vertical whitespace 
>>>>>>>>> >>>>>> (e.g. closing parentheses on their own line), e.g.
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> def foo(
>>>>>>>>> >>>>>>     args
>>>>>>>>> >>>>>> ):
>>>>>>>>> >>>>>>   if (
>>>>>>>>> >>>>>>       long expression)
>>>>>>>>> >>>>>>   ):
>>>>>>>>> >>>>>>     func(
>>>>>>>>> >>>>>>         args
>>>>>>>>> >>>>>>     )
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> I wrote a simple post-processor to put closing parentheses on 
>>>>>>>>> >>>>>> the same lines, as well as omit the newline after "if (", and 
>>>>>>>>> >>>>>> disabling formatting of strings, which reduce the churn in our 
>>>>>>>>> >>>>>> codebase to 15k lines (adding about 4k) out of 200k total.
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> https://github.com/apache/beam/pull/8712/files
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> It's still very opinionated, often in different ways then me, 
>>>>>>>>> >>>>>> and doesn't understand the semantics of the code, but possibly 
>>>>>>>>> >>>>>> something we could live with given the huge advantages of an 
>>>>>>>>> >>>>>> autoformatter.
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> An intermediate point would be to allow, but not require, 
>>>>>>>>> >>>>>> autoformatting of changed lines.
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> As for being beta quality, it looks like it's got a decent 
>>>>>>>>> >>>>>> number of contributors and in my book being in the python 
>>>>>>>>> >>>>>> github project is a strong positive signal. But, due to the 
>>>>>>>>> >>>>>> above issues, I think we'd have to maintain a fork. (The code 
>>>>>>>>> >>>>>> is pretty lightweight, the 2 vs. 4 space issue is a 2-line 
>>>>>>>>> >>>>>> change, and the rest implemented as a post-processing step 
>>>>>>>>> >>>>>> (for now, incomplete), so it'd be easy to stay in sync with 
>>>>>>>>> >>>>>> upstream.)
>>>>>>>>> >>>>>>
>>>>>>>>> >>>>>> On Wed, May 29, 2019 at 11:03 AM Ismaël Mejía 
>>>>>>>>> >>>>>> <ieme...@gmail.com> wrote:
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> > > I think the question is if it can be configured in a way 
>>>>>>>>> >>>>>> > > to fit our
>>>>>>>>> >>>>>> > > current linter's style. I don't think it is feasible to 
>>>>>>>>> >>>>>> > > reformat the
>>>>>>>>> >>>>>> > > entire Python SDK.
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> > It cannot be configured to do what we actually do because 
>>>>>>>>> >>>>>> > Black is
>>>>>>>>> >>>>>> > configurable only to support the standard python codestyle 
>>>>>>>>> >>>>>> > guidelines
>>>>>>>>> >>>>>> > (PEP-8) which recommends 4 spaces and is what most projects 
>>>>>>>>> >>>>>> > in the
>>>>>>>>> >>>>>> > python world use.
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> > > Reformatted lines don't allow quick access to the Git 
>>>>>>>>> >>>>>> > > history. This
>>>>>>>>> >>>>>> > > effect is still visible in the Java SDK. However, I have 
>>>>>>>>> >>>>>> > > the feeling
>>>>>>>>> >>>>>> > > that this might be less of a problem with Python because 
>>>>>>>>> >>>>>> > > the linter has
>>>>>>>>> >>>>>> > > more rules than Checkstyle had.
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> > Yes that’s the bad side effect but there are always 
>>>>>>>>> >>>>>> > tradeoffs we have
>>>>>>>>> >>>>>> > to deal with.
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> >
>>>>>>>>> >>>>>> > On Wed, May 29, 2019 at 10:52 AM Maximilian Michels 
>>>>>>>>> >>>>>> > <m...@apache.org> wrote:
>>>>>>>>> >>>>>> > >
>>>>>>>>> >>>>>> > > I think the question is if it can be configured in a way 
>>>>>>>>> >>>>>> > > to fit our
>>>>>>>>> >>>>>> > > current linter's style. I don't think it is feasible to 
>>>>>>>>> >>>>>> > > reformat the
>>>>>>>>> >>>>>> > > entire Python SDK.
>>>>>>>>> >>>>>> > >
>>>>>>>>> >>>>>> > > Reformatted lines don't allow quick access to the Git 
>>>>>>>>> >>>>>> > > history. This
>>>>>>>>> >>>>>> > > effect is still visible in the Java SDK. However, I have 
>>>>>>>>> >>>>>> > > the feeling
>>>>>>>>> >>>>>> > > that this might be less of a problem with Python because 
>>>>>>>>> >>>>>> > > the linter has
>>>>>>>>> >>>>>> > > more rules than Checkstyle had.
>>>>>>>>> >>>>>> > >
>>>>>>>>> >>>>>> > > -Max
>>>>>>>>> >>>>>> > >
>>>>>>>>> >>>>>> > > On 29.05.19 10:16, Ismaël Mejía wrote:
>>>>>>>>> >>>>>> > > >> My concerns are:
>>>>>>>>> >>>>>> > > >> - The product is clearly marked as beta with a big 
>>>>>>>>> >>>>>> > > >> warning.
>>>>>>>>> >>>>>> > > >> - It looks like mostly a single person project. For the 
>>>>>>>>> >>>>>> > > >> same reason I also strongly prefer not using a fork for 
>>>>>>>>> >>>>>> > > >> a specific setting. Fork will only have less people 
>>>>>>>>> >>>>>> > > >> looking at it.
>>>>>>>>> >>>>>> > > >
>>>>>>>>> >>>>>> > > > I suppose the project is marked as beta because it is 
>>>>>>>>> >>>>>> > > > recent, it was
>>>>>>>>> >>>>>> > > > presented in 2018’s pycon, and because some things can 
>>>>>>>>> >>>>>> > > > change since
>>>>>>>>> >>>>>> > > > auto-formatters are pretty tricky beasts, I think beta 
>>>>>>>>> >>>>>> > > > in that case is
>>>>>>>>> >>>>>> > > > like our own ‘@Experimental’. If you look at the 
>>>>>>>>> >>>>>> > > > contribution page [1]
>>>>>>>>> >>>>>> > > > you can notice that it is less and less a single person 
>>>>>>>>> >>>>>> > > > project, there
>>>>>>>>> >>>>>> > > > have been 93 independent contributions since the project 
>>>>>>>>> >>>>>> > > > became
>>>>>>>>> >>>>>> > > > public, and the fact that it is hosted in the python 
>>>>>>>>> >>>>>> > > > organization
>>>>>>>>> >>>>>> > > > github [2] gives some confidence on the project 
>>>>>>>>> >>>>>> > > > continuity.
>>>>>>>>> >>>>>> > > >
>>>>>>>>> >>>>>> > > > You are right however about the fact that the main 
>>>>>>>>> >>>>>> > > > author seems to be
>>>>>>>>> >>>>>> > > > the ‘benevolent’ dictator, and in the 2-spaces issue he 
>>>>>>>>> >>>>>> > > > can seem
>>>>>>>>> >>>>>> > > > arbitrary, but he is just following pep8 style guide 
>>>>>>>>> >>>>>> > > > recommendations
>>>>>>>>> >>>>>> > > > [3]. I am curious of why we (Beam) do not follow the 4 
>>>>>>>>> >>>>>> > > > spaces
>>>>>>>>> >>>>>> > > > recommendation of PEP-8 or even Google's own Python 
>>>>>>>>> >>>>>> > > > style guide [4],
>>>>>>>>> >>>>>> > > > So, probably it should be to us to reconsider the 
>>>>>>>>> >>>>>> > > > current policy to
>>>>>>>>> >>>>>> > > > adapt to the standards (and the tool).
>>>>>>>>> >>>>>> > > >
>>>>>>>>> >>>>>> > > > I did a quick run of black with python 2.7 compatibility 
>>>>>>>>> >>>>>> > > > on
>>>>>>>>> >>>>>> > > > sdks/python and got only 4 parsing errors which is 
>>>>>>>>> >>>>>> > > > positive given the
>>>>>>>>> >>>>>> > > > size of our code base.
>>>>>>>>> >>>>>> > > >
>>>>>>>>> >>>>>> > > > 415 files reformatted, 45 files left unchanged, 4 files 
>>>>>>>>> >>>>>> > > > failed to reformat.
>>>>>>>>> >>>>>> > > >
>>>>>>>>> >>>>>> > > > error: cannot format
>>>>>>>>> >>>>>> > > > /home/ismael/upstream/beam/sdks/python/apache_beam/runners/interactive/display/display_manager.py:
>>>>>>>>> >>>>>> > > > Cannot parse: 47:22:   _display_progress = print
>>>>>>>>> >>>>>> > > > error: cannot format
>>>>>>>>> >>>>>> > > > /home/ismael/upstream/beam/sdks/python/apache_beam/runners/worker/log_handler.py:
>>>>>>>>> >>>>>> > > > Cannot parse: 151:18:               file=sys.stderr)
>>>>>>>>> >>>>>> > > > error: cannot format
>>>>>>>>> >>>>>> > > > /home/ismael/upstream/beam/sdks/python/apache_beam/runners/worker/sdk_worker.py:
>>>>>>>>> >>>>>> > > > Cannot parse: 160:34:       print(traceback_string, 
>>>>>>>>> >>>>>> > > > file=sys.stderr)
>>>>>>>>> >>>>>> > > > error: cannot format
>>>>>>>>> >>>>>> > > > /home/ismael/upstream/beam/sdks/python/apache_beam/typehints/trivial_inference.py:
>>>>>>>>> >>>>>> > > > Cannot parse: 335:51:       print('-->' if pc == last_pc 
>>>>>>>>> >>>>>> > > > else '    ',
>>>>>>>>> >>>>>> > > > end=' ')
>>>>>>>>> >>>>>> > > >
>>>>>>>>> >>>>>> > > > I still think this can be positive for the project but 
>>>>>>>>> >>>>>> > > > well I am
>>>>>>>>> >>>>>> > > > barely a contributor to the python code base so I let 
>>>>>>>>> >>>>>> > > > you the python
>>>>>>>>> >>>>>> > > > maintainers to reconsider this, in any case it seems 
>>>>>>>>> >>>>>> > > > like a good
>>>>>>>>> >>>>>> > > > improvement for the project.
>>>>>>>>> >>>>>> > > >
>>>>>>>>> >>>>>> > > > [1] https://github.com/python/black/graphs/contributors
>>>>>>>>> >>>>>> > > > [2] https://github.com/python
>>>>>>>>> >>>>>> > > > [3] https://www.python.org/dev/peps/pep-0008/#indentation
>>>>>>>>> >>>>>> > > > [4] 
>>>>>>>>> >>>>>> > > > https://github.com/google/styleguide/blob/gh-pages/pyguide.md#34-indentation
>>>>>>>>> >>>>>> > > >
>>>>>>>>> >>>>>> > > > On Tue, May 28, 2019 at 11:15 PM Ahmet Altay 
>>>>>>>>> >>>>>> > > > <al...@google.com> wrote:
>>>>>>>>> >>>>>> > > >>
>>>>>>>>> >>>>>> > > >> I am in the same boat with Robert, I am in favor of 
>>>>>>>>> >>>>>> > > >> autoformatters but I am not familiar with this one. My 
>>>>>>>>> >>>>>> > > >> concerns are:
>>>>>>>>> >>>>>> > > >> - The product is clearly marked as beta with a big 
>>>>>>>>> >>>>>> > > >> warning.
>>>>>>>>> >>>>>> > > >> - It looks like mostly a single person project. For the 
>>>>>>>>> >>>>>> > > >> same reason I also strongly prefer not using a fork for 
>>>>>>>>> >>>>>> > > >> a specific setting. Fork will only have less people 
>>>>>>>>> >>>>>> > > >> looking at it.
>>>>>>>>> >>>>>> > > >>
>>>>>>>>> >>>>>> > > >> IMO, this is in an early stage for us. That said lint 
>>>>>>>>> >>>>>> > > >> issues are real as pointed in the thread. If someone 
>>>>>>>>> >>>>>> > > >> would like to give it a try and see how it would look 
>>>>>>>>> >>>>>> > > >> like for us that would be interesting.
>>>>>>>>> >>>>>> > > >>
>>>>>>>>> >>>>>> > > >> On Tue, May 28, 2019 at 4:44 AM Katarzyna Kucharczyk 
>>>>>>>>> >>>>>> > > >> <ka.kucharc...@gmail.com> wrote:
>>>>>>>>> >>>>>> > > >>>
>>>>>>>>> >>>>>> > > >>> This sounds really good. A lot of Jenkins jobs 
>>>>>>>>> >>>>>> > > >>> failures are caused by lint problems.
>>>>>>>>> >>>>>> > > >>> I think it would be great to have something similar to 
>>>>>>>>> >>>>>> > > >>> Spotless in Java SDK (I heard there is problem with 
>>>>>>>>> >>>>>> > > >>> configuring Black with IntelliJ).
>>>>>>>>> >>>>>> > > >>>
>>>>>>>>> >>>>>> > > >>> On Mon, May 27, 2019 at 10:52 PM Robert Bradshaw 
>>>>>>>>> >>>>>> > > >>> <rober...@google.com> wrote:
>>>>>>>>> >>>>>> > > >>>>
>>>>>>>>> >>>>>> > > >>>> I'm generally in favor of autoformatters, though I 
>>>>>>>>> >>>>>> > > >>>> haven't looked at
>>>>>>>>> >>>>>> > > >>>> how well this particular one works. We might have to 
>>>>>>>>> >>>>>> > > >>>> go with
>>>>>>>>> >>>>>> > > >>>> https://github.com/desbma/black-2spaces given
>>>>>>>>> >>>>>> > > >>>> https://github.com/python/black/issues/378 .
>>>>>>>>> >>>>>> > > >>>>
>>>>>>>>> >>>>>> > > >>>> On Mon, May 27, 2019 at 10:43 PM Pablo Estrada 
>>>>>>>>> >>>>>> > > >>>> <pabl...@google.com> wrote:
>>>>>>>>> >>>>>> > > >>>>>
>>>>>>>>> >>>>>> > > >>>>> This looks pretty good:) I know at least a couple 
>>>>>>>>> >>>>>> > > >>>>> people (myself included) who've been annoyed by 
>>>>>>>>> >>>>>> > > >>>>> having to take care of lint issues that maybe a code 
>>>>>>>>> >>>>>> > > >>>>> formatter could save us.
>>>>>>>>> >>>>>> > > >>>>> Thanks for sharing Ismael.
>>>>>>>>> >>>>>> > > >>>>> -P.
>>>>>>>>> >>>>>> > > >>>>>
>>>>>>>>> >>>>>> > > >>>>>
>>>>>>>>> >>>>>> > > >>>>> On Mon, May 27, 2019, 12:24 PM Ismaël Mejía 
>>>>>>>>> >>>>>> > > >>>>> <ieme...@gmail.com> wrote:
>>>>>>>>> >>>>>> > > >>>>>>
>>>>>>>>> >>>>>> > > >>>>>> I stumbled by chance into Black [1] a python code 
>>>>>>>>> >>>>>> > > >>>>>> auto formatter that
>>>>>>>>> >>>>>> > > >>>>>> is becoming the 'de-facto' auto-formatter for 
>>>>>>>>> >>>>>> > > >>>>>> python, and wanted to
>>>>>>>>> >>>>>> > > >>>>>> bring to the ML Is there interest from the python 
>>>>>>>>> >>>>>> > > >>>>>> people to get this
>>>>>>>>> >>>>>> > > >>>>>> into the build?
>>>>>>>>> >>>>>> > > >>>>>>
>>>>>>>>> >>>>>> > > >>>>>> The introduction of spotless for Java has been a 
>>>>>>>>> >>>>>> > > >>>>>> good improvement and
>>>>>>>>> >>>>>> > > >>>>>> maybe the python code base may benefit of this too.
>>>>>>>>> >>>>>> > > >>>>>>
>>>>>>>>> >>>>>> > > >>>>>> WDYT?
>>>>>>>>> >>>>>> > > >>>>>>
>>>>>>>>> >>>>>> > > >>>>>> [1] https://github.com/python/black
>>
>>
>>
>> --
>>
>> Michał Walenia
>> Polidea | Software Engineer
>>
>> M: +48 791 432 002
>> E: michal.wale...@polidea.com
>>
>> Unique Tech
>> Check out our projects!

Reply via email to