Re: spark code formatter?
BTW, we also run Scalariform, but we don't turn it on automatically. We
find that for the most part it is good, but there are a few places where it
reformats things and doesn't look good, and requires cleanup. I think
Scalariform requires some more rules to make it more generally useful.
-Evan
On Thu, Jan 9, 2014 at 12:23 PM, DB Tsai wrote:
> Initially, we also had the same concern, so we started from limited
> set of rules. Gradually, we found that it increases the productivity
> and readability of our codebase.
>
> PS, Scalariform is compatible with the Scala Style Guide in the sense
> that, given the right preference settings, source code that is
> initially compiliant with the Style Guide will not become uncompliant
> after formatting. In a number of cases, running the formatter will
> make uncompliant source more compliant.
>
> I added the configuration option in the latest PR to limit the set of
> rules. The options are https://github.com/mdr/scalariform
>
> When developers wants to choose their own style for whatever reasons,
> they can source directives to turn it off by `// format: OFF`.
>
> Just quickly run the formatter, and I found that Spark is in general
> in good shape; most of the changes are extra space after semicolon.
>
> - def run[K: Manifest, V <: Vertex : Manifest, M <: Message[K] :
> Manifest, C: Manifest](
>
> + def run[K: Manifest, V <: Vertex: Manifest, M <: Message[K]:
> Manifest, C: Manifest](
>
> - def addFile(file: File) : String = {
> + def addFile(file: File): String = {
>
> Sincerely,
>
> DB Tsai
> Machine Learning Engineer
> Alpine Data Labs
> --
> Web: http://alpinenow.com/
>
>
> On Thu, Jan 9, 2014 at 12:32 AM, Patrick Wendell
> wrote:
> > I'm also very wary of using a code formatter for the reasons already
> > mentioned by Reynold.
> >
> > Does scaliform have a mode where it just provides style checks rather
> > than reformat the code? This is something we really need for, e.g.,
> > reviewing the many submissions to the project.
> >
> > - Patrick
> >
> > On Wed, Jan 8, 2014 at 11:51 PM, Reynold Xin
> wrote:
> >> Thanks for doing that, DB. Not sure about others, but I'm actually
> strongly
> >> against blanket automatic code formatters, given that they can be
> >> disruptive. Often humans would intentionally choose to style things in a
> >> certain way for more clear semantics and better readability. Code
> >> formatters don't capture these nuances. It is pretty dangerous to just
> auto
> >> format everything.
> >>
> >> Maybe it'd be ok if we restrict the code formatters to a very limited
> set
> >> of things, such as indenting function parameters, etc.
> >>
> >>
> >> On Wed, Jan 8, 2014 at 10:28 PM, DB Tsai wrote:
> >>
> >>> A pull request for scalariform.
> >>> https://github.com/apache/incubator-spark/pull/365
> >>>
> >>> Sincerely,
> >>>
> >>> DB Tsai
> >>> Machine Learning Engineer
> >>> Alpine Data Labs
> >>> --
> >>> Web: http://alpinenow.com/
> >>>
> >>>
> >>> On Wed, Jan 8, 2014 at 10:09 PM, DB Tsai wrote:
> >>> > We use sbt-scalariform in our company, and it can automatically
> format
> >>> > the coding style when runs `sbt compile`.
> >>> >
> >>> > https://github.com/sbt/sbt-scalariform
> >>> >
> >>> > We ask our developers to run `sbt compile` before commit, and it's
> >>> > really nice to see everyone has the same spacing and indentation.
> >>> >
> >>> > Sincerely,
> >>> >
> >>> > DB Tsai
> >>> > Machine Learning Engineer
> >>> > Alpine Data Labs
> >>> > --
> >>> > Web: http://alpinenow.com/
> >>> >
> >>> >
> >>> > On Wed, Jan 8, 2014 at 9:50 PM, Reynold Xin
> wrote:
> >>> >> We have a Scala style configuration file in Shark:
> >>> >> https://github.com/amplab/shark/blob/master/scalastyle-config.xml
> >>> >>
> >>> >> However, the scalastyle project is still pretty primitive and
> doesn't
> >>> cover
> >>> >> most of the use cases. It is still great to include it to cover
> basic
> >>> >> checks such as 100-char wide lines.
> >>> >>
> >>> >>
> >>> >> On Wed, Jan 8, 2014 at 8:02 PM, Matei Zaharia <
> matei.zaha...@gmail.com
> >>> >wrote:
> >>> >>
> >>> >>> Not that I know of. This would be very useful to add, especially
> if we
> >>> can
> >>> >>> make SBT automatically check the code style (or we can somehow plug
> >>> this
> >>> >>> into Jenkins).
> >>> >>>
> >>> >>> Matei
> >>> >>>
> >>> >>> On Jan 8, 2014, at 11:00 AM, Michael Allman wrote:
> >>> >>>
> >>> >>> > Hi,
> >>> >>> >
> >>> >>> > I've read the spark code style guide for contributors here:
> >>> >>> >
> >>> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide
> >>> >>> >
> >>> >>> > For scala code, do you have a scalariform configuration that you
> use
> >>> to
> >>> >>> format your code to these specs?
> >>> >>> >
> >>> >>> > Cheers,
> >>> >>> >
> >>> >>> > Michael
> >>> >>>
> >>> >>>
> >>>
>
--
--
Evan Chan
Staff Engineer
e...@ooyala.c
Re: spark code formatter?
Initially, we also had the same concern, so we started from limited
set of rules. Gradually, we found that it increases the productivity
and readability of our codebase.
PS, Scalariform is compatible with the Scala Style Guide in the sense
that, given the right preference settings, source code that is
initially compiliant with the Style Guide will not become uncompliant
after formatting. In a number of cases, running the formatter will
make uncompliant source more compliant.
I added the configuration option in the latest PR to limit the set of
rules. The options are https://github.com/mdr/scalariform
When developers wants to choose their own style for whatever reasons,
they can source directives to turn it off by `// format: OFF`.
Just quickly run the formatter, and I found that Spark is in general
in good shape; most of the changes are extra space after semicolon.
- def run[K: Manifest, V <: Vertex : Manifest, M <: Message[K] :
Manifest, C: Manifest](
+ def run[K: Manifest, V <: Vertex: Manifest, M <: Message[K]:
Manifest, C: Manifest](
- def addFile(file: File) : String = {
+ def addFile(file: File): String = {
Sincerely,
DB Tsai
Machine Learning Engineer
Alpine Data Labs
--
Web: http://alpinenow.com/
On Thu, Jan 9, 2014 at 12:32 AM, Patrick Wendell wrote:
> I'm also very wary of using a code formatter for the reasons already
> mentioned by Reynold.
>
> Does scaliform have a mode where it just provides style checks rather
> than reformat the code? This is something we really need for, e.g.,
> reviewing the many submissions to the project.
>
> - Patrick
>
> On Wed, Jan 8, 2014 at 11:51 PM, Reynold Xin wrote:
>> Thanks for doing that, DB. Not sure about others, but I'm actually strongly
>> against blanket automatic code formatters, given that they can be
>> disruptive. Often humans would intentionally choose to style things in a
>> certain way for more clear semantics and better readability. Code
>> formatters don't capture these nuances. It is pretty dangerous to just auto
>> format everything.
>>
>> Maybe it'd be ok if we restrict the code formatters to a very limited set
>> of things, such as indenting function parameters, etc.
>>
>>
>> On Wed, Jan 8, 2014 at 10:28 PM, DB Tsai wrote:
>>
>>> A pull request for scalariform.
>>> https://github.com/apache/incubator-spark/pull/365
>>>
>>> Sincerely,
>>>
>>> DB Tsai
>>> Machine Learning Engineer
>>> Alpine Data Labs
>>> --
>>> Web: http://alpinenow.com/
>>>
>>>
>>> On Wed, Jan 8, 2014 at 10:09 PM, DB Tsai wrote:
>>> > We use sbt-scalariform in our company, and it can automatically format
>>> > the coding style when runs `sbt compile`.
>>> >
>>> > https://github.com/sbt/sbt-scalariform
>>> >
>>> > We ask our developers to run `sbt compile` before commit, and it's
>>> > really nice to see everyone has the same spacing and indentation.
>>> >
>>> > Sincerely,
>>> >
>>> > DB Tsai
>>> > Machine Learning Engineer
>>> > Alpine Data Labs
>>> > --
>>> > Web: http://alpinenow.com/
>>> >
>>> >
>>> > On Wed, Jan 8, 2014 at 9:50 PM, Reynold Xin wrote:
>>> >> We have a Scala style configuration file in Shark:
>>> >> https://github.com/amplab/shark/blob/master/scalastyle-config.xml
>>> >>
>>> >> However, the scalastyle project is still pretty primitive and doesn't
>>> cover
>>> >> most of the use cases. It is still great to include it to cover basic
>>> >> checks such as 100-char wide lines.
>>> >>
>>> >>
>>> >> On Wed, Jan 8, 2014 at 8:02 PM, Matei Zaharia >> >wrote:
>>> >>
>>> >>> Not that I know of. This would be very useful to add, especially if we
>>> can
>>> >>> make SBT automatically check the code style (or we can somehow plug
>>> this
>>> >>> into Jenkins).
>>> >>>
>>> >>> Matei
>>> >>>
>>> >>> On Jan 8, 2014, at 11:00 AM, Michael Allman wrote:
>>> >>>
>>> >>> > Hi,
>>> >>> >
>>> >>> > I've read the spark code style guide for contributors here:
>>> >>> >
>>> >>> >
>>> https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide
>>> >>> >
>>> >>> > For scala code, do you have a scalariform configuration that you use
>>> to
>>> >>> format your code to these specs?
>>> >>> >
>>> >>> > Cheers,
>>> >>> >
>>> >>> > Michael
>>> >>>
>>> >>>
>>>
Re: spark code formatter?
I'm also very wary of using a code formatter for the reasons already mentioned by Reynold. Does scaliform have a mode where it just provides style checks rather than reformat the code? This is something we really need for, e.g., reviewing the many submissions to the project. - Patrick On Wed, Jan 8, 2014 at 11:51 PM, Reynold Xin wrote: > Thanks for doing that, DB. Not sure about others, but I'm actually strongly > against blanket automatic code formatters, given that they can be > disruptive. Often humans would intentionally choose to style things in a > certain way for more clear semantics and better readability. Code > formatters don't capture these nuances. It is pretty dangerous to just auto > format everything. > > Maybe it'd be ok if we restrict the code formatters to a very limited set > of things, such as indenting function parameters, etc. > > > On Wed, Jan 8, 2014 at 10:28 PM, DB Tsai wrote: > >> A pull request for scalariform. >> https://github.com/apache/incubator-spark/pull/365 >> >> Sincerely, >> >> DB Tsai >> Machine Learning Engineer >> Alpine Data Labs >> -- >> Web: http://alpinenow.com/ >> >> >> On Wed, Jan 8, 2014 at 10:09 PM, DB Tsai wrote: >> > We use sbt-scalariform in our company, and it can automatically format >> > the coding style when runs `sbt compile`. >> > >> > https://github.com/sbt/sbt-scalariform >> > >> > We ask our developers to run `sbt compile` before commit, and it's >> > really nice to see everyone has the same spacing and indentation. >> > >> > Sincerely, >> > >> > DB Tsai >> > Machine Learning Engineer >> > Alpine Data Labs >> > -- >> > Web: http://alpinenow.com/ >> > >> > >> > On Wed, Jan 8, 2014 at 9:50 PM, Reynold Xin wrote: >> >> We have a Scala style configuration file in Shark: >> >> https://github.com/amplab/shark/blob/master/scalastyle-config.xml >> >> >> >> However, the scalastyle project is still pretty primitive and doesn't >> cover >> >> most of the use cases. It is still great to include it to cover basic >> >> checks such as 100-char wide lines. >> >> >> >> >> >> On Wed, Jan 8, 2014 at 8:02 PM, Matei Zaharia > >wrote: >> >> >> >>> Not that I know of. This would be very useful to add, especially if we >> can >> >>> make SBT automatically check the code style (or we can somehow plug >> this >> >>> into Jenkins). >> >>> >> >>> Matei >> >>> >> >>> On Jan 8, 2014, at 11:00 AM, Michael Allman wrote: >> >>> >> >>> > Hi, >> >>> > >> >>> > I've read the spark code style guide for contributors here: >> >>> > >> >>> > >> https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide >> >>> > >> >>> > For scala code, do you have a scalariform configuration that you use >> to >> >>> format your code to these specs? >> >>> > >> >>> > Cheers, >> >>> > >> >>> > Michael >> >>> >> >>> >>
Re: spark code formatter?
Thanks for doing that, DB. Not sure about others, but I'm actually strongly against blanket automatic code formatters, given that they can be disruptive. Often humans would intentionally choose to style things in a certain way for more clear semantics and better readability. Code formatters don't capture these nuances. It is pretty dangerous to just auto format everything. Maybe it'd be ok if we restrict the code formatters to a very limited set of things, such as indenting function parameters, etc. On Wed, Jan 8, 2014 at 10:28 PM, DB Tsai wrote: > A pull request for scalariform. > https://github.com/apache/incubator-spark/pull/365 > > Sincerely, > > DB Tsai > Machine Learning Engineer > Alpine Data Labs > -- > Web: http://alpinenow.com/ > > > On Wed, Jan 8, 2014 at 10:09 PM, DB Tsai wrote: > > We use sbt-scalariform in our company, and it can automatically format > > the coding style when runs `sbt compile`. > > > > https://github.com/sbt/sbt-scalariform > > > > We ask our developers to run `sbt compile` before commit, and it's > > really nice to see everyone has the same spacing and indentation. > > > > Sincerely, > > > > DB Tsai > > Machine Learning Engineer > > Alpine Data Labs > > -- > > Web: http://alpinenow.com/ > > > > > > On Wed, Jan 8, 2014 at 9:50 PM, Reynold Xin wrote: > >> We have a Scala style configuration file in Shark: > >> https://github.com/amplab/shark/blob/master/scalastyle-config.xml > >> > >> However, the scalastyle project is still pretty primitive and doesn't > cover > >> most of the use cases. It is still great to include it to cover basic > >> checks such as 100-char wide lines. > >> > >> > >> On Wed, Jan 8, 2014 at 8:02 PM, Matei Zaharia >wrote: > >> > >>> Not that I know of. This would be very useful to add, especially if we > can > >>> make SBT automatically check the code style (or we can somehow plug > this > >>> into Jenkins). > >>> > >>> Matei > >>> > >>> On Jan 8, 2014, at 11:00 AM, Michael Allman wrote: > >>> > >>> > Hi, > >>> > > >>> > I've read the spark code style guide for contributors here: > >>> > > >>> > > https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide > >>> > > >>> > For scala code, do you have a scalariform configuration that you use > to > >>> format your code to these specs? > >>> > > >>> > Cheers, > >>> > > >>> > Michael > >>> > >>> >
Re: spark code formatter?
A pull request for scalariform. https://github.com/apache/incubator-spark/pull/365 Sincerely, DB Tsai Machine Learning Engineer Alpine Data Labs -- Web: http://alpinenow.com/ On Wed, Jan 8, 2014 at 10:09 PM, DB Tsai wrote: > We use sbt-scalariform in our company, and it can automatically format > the coding style when runs `sbt compile`. > > https://github.com/sbt/sbt-scalariform > > We ask our developers to run `sbt compile` before commit, and it's > really nice to see everyone has the same spacing and indentation. > > Sincerely, > > DB Tsai > Machine Learning Engineer > Alpine Data Labs > -- > Web: http://alpinenow.com/ > > > On Wed, Jan 8, 2014 at 9:50 PM, Reynold Xin wrote: >> We have a Scala style configuration file in Shark: >> https://github.com/amplab/shark/blob/master/scalastyle-config.xml >> >> However, the scalastyle project is still pretty primitive and doesn't cover >> most of the use cases. It is still great to include it to cover basic >> checks such as 100-char wide lines. >> >> >> On Wed, Jan 8, 2014 at 8:02 PM, Matei Zaharia wrote: >> >>> Not that I know of. This would be very useful to add, especially if we can >>> make SBT automatically check the code style (or we can somehow plug this >>> into Jenkins). >>> >>> Matei >>> >>> On Jan 8, 2014, at 11:00 AM, Michael Allman wrote: >>> >>> > Hi, >>> > >>> > I've read the spark code style guide for contributors here: >>> > >>> > https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide >>> > >>> > For scala code, do you have a scalariform configuration that you use to >>> format your code to these specs? >>> > >>> > Cheers, >>> > >>> > Michael >>> >>>
Re: spark code formatter?
We use sbt-scalariform in our company, and it can automatically format the coding style when runs `sbt compile`. https://github.com/sbt/sbt-scalariform We ask our developers to run `sbt compile` before commit, and it's really nice to see everyone has the same spacing and indentation. Sincerely, DB Tsai Machine Learning Engineer Alpine Data Labs -- Web: http://alpinenow.com/ On Wed, Jan 8, 2014 at 9:50 PM, Reynold Xin wrote: > We have a Scala style configuration file in Shark: > https://github.com/amplab/shark/blob/master/scalastyle-config.xml > > However, the scalastyle project is still pretty primitive and doesn't cover > most of the use cases. It is still great to include it to cover basic > checks such as 100-char wide lines. > > > On Wed, Jan 8, 2014 at 8:02 PM, Matei Zaharia wrote: > >> Not that I know of. This would be very useful to add, especially if we can >> make SBT automatically check the code style (or we can somehow plug this >> into Jenkins). >> >> Matei >> >> On Jan 8, 2014, at 11:00 AM, Michael Allman wrote: >> >> > Hi, >> > >> > I've read the spark code style guide for contributors here: >> > >> > https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide >> > >> > For scala code, do you have a scalariform configuration that you use to >> format your code to these specs? >> > >> > Cheers, >> > >> > Michael >> >>
Re: spark code formatter?
We have a Scala style configuration file in Shark: https://github.com/amplab/shark/blob/master/scalastyle-config.xml However, the scalastyle project is still pretty primitive and doesn't cover most of the use cases. It is still great to include it to cover basic checks such as 100-char wide lines. On Wed, Jan 8, 2014 at 8:02 PM, Matei Zaharia wrote: > Not that I know of. This would be very useful to add, especially if we can > make SBT automatically check the code style (or we can somehow plug this > into Jenkins). > > Matei > > On Jan 8, 2014, at 11:00 AM, Michael Allman wrote: > > > Hi, > > > > I've read the spark code style guide for contributors here: > > > > https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide > > > > For scala code, do you have a scalariform configuration that you use to > format your code to these specs? > > > > Cheers, > > > > Michael > >
Re: spark code formatter?
Not that I know of. This would be very useful to add, especially if we can make SBT automatically check the code style (or we can somehow plug this into Jenkins). Matei On Jan 8, 2014, at 11:00 AM, Michael Allman wrote: > Hi, > > I've read the spark code style guide for contributors here: > > https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide > > For scala code, do you have a scalariform configuration that you use to > format your code to these specs? > > Cheers, > > Michael
spark code formatter?
Hi, I've read the spark code style guide for contributors here: https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide For scala code, do you have a scalariform configuration that you use to format your code to these specs? Cheers, Michael
