Re: Scalastyle improvements / large code reformatting

2014-10-13 Thread Patrick Wendell
Hey Nick, I think the best solution is really to find a way to only apply certain rules to code modified after a certain date. I also don't think it would be that hard to implement because git can output per-line information about modification times. So you'd just run the scalastyle rules and

Re: Scalastyle improvements / large code reformatting

2014-10-13 Thread Marcelo Vanzin
Another option is to add new style rules that trigger too many errors as warnings, and slowly clean them up. This means that reviewers will be burdened with manually enforcing the rules for a while, and we need to remember to turn them to errors once some threshold is reached. (The Hadoop build

Re: Scalastyle improvements / large code reformatting

2014-10-13 Thread Nicholas Chammas
On Mon, Oct 13, 2014 at 11:57 AM, Patrick Wendell pwend...@gmail.com wrote: That would even work for imports as well, you'd just have a thing where if anyone modified some imports they would have to fix all the imports in that file. It's at least worth a try. OK, that sounds like a fair

Re: Scalastyle improvements / large code reformatting

2014-10-13 Thread Erik Erlandson
- Original Message - I'm also against these huge reformattings. They slow down development and backporting for trivial reasons. Let's not do that at this point, the style of the current code is quite consistent and we have plenty of other things to worry about. Instead, what you can

Scalastyle improvements / large code reformatting

2014-10-12 Thread Josh Rosen
There are a number of open pull requests that aim to extend Spark’s automated style checks (see https://issues.apache.org/jira/browse/SPARK-3849 for an umbrella JIRA).  These fixes are mostly good, but I have some concerns about merging these patches.  Several of these patches make large

Re: Scalastyle improvements / large code reformatting

2014-10-12 Thread Reynold Xin
I actually think we should just take the bite and follow through with the reformatting. Many rules are simply not possible to enforce only on deltas (e.g. import ordering). That said, maybe there are better windows to do this, e.g. during the QA period. On Sun, Oct 12, 2014 at 9:37 PM, Josh

Re: Scalastyle improvements / large code reformatting

2014-10-12 Thread Patrick Wendell
Another big problem with these patches are that they make it almost impossible to backport changes to older branches cleanly (there becomes like 100% chance of a merge conflict). One proposal is to do this: 1. We only consider new style rules at the end of a release cycle, when there is the