[jira] [Commented] (SPARK-5312) Use sbt to detect new or changed public classes in PRs
[
https://issues.apache.org/jira/browse/SPARK-5312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409767#comment-15409767
]
Nicholas Chammas commented on SPARK-5312:
-
[~boyork] - Shall we close this? It doesn't look like it has any momentum.
> Use sbt to detect new or changed public classes in PRs
> --
>
> Key: SPARK-5312
> URL: https://issues.apache.org/jira/browse/SPARK-5312
> Project: Spark
> Issue Type: Improvement
> Components: Project Infra
>Reporter: Nicholas Chammas
>Priority: Minor
>
> We currently use an [unwieldy grep/sed
> contraption|https://github.com/apache/spark/blob/19556454881e05a6c2470d406d50f004b88088a2/dev/run-tests-jenkins#L152-L174]
> to detect new public classes in PRs.
> -Apparently, sbt lets you get a list of public classes [much more
> directly|http://www.scala-sbt.org/0.13/docs/Howto-Inspect-the-Build.html] via
> {{show compile:discoveredMainClasses}}. We should use that instead.-
> There is a tool called [ClassUtil|http://software.clapper.org/classutil/]
> that seems to help give this kind of information much more directly. We
> should look into using that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-5312) Use sbt to detect new or changed public classes in PRs
[
https://issues.apache.org/jira/browse/SPARK-5312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15037808#comment-15037808
]
Sean Owen commented on SPARK-5312:
--
[~boyork] are you going to do more with this?
> Use sbt to detect new or changed public classes in PRs
> --
>
> Key: SPARK-5312
> URL: https://issues.apache.org/jira/browse/SPARK-5312
> Project: Spark
> Issue Type: Improvement
> Components: Project Infra
>Reporter: Nicholas Chammas
>Priority: Minor
>
> We currently use an [unwieldy grep/sed
> contraption|https://github.com/apache/spark/blob/19556454881e05a6c2470d406d50f004b88088a2/dev/run-tests-jenkins#L152-L174]
> to detect new public classes in PRs.
> -Apparently, sbt lets you get a list of public classes [much more
> directly|http://www.scala-sbt.org/0.13/docs/Howto-Inspect-the-Build.html] via
> {{show compile:discoveredMainClasses}}. We should use that instead.-
> There is a tool called [ClassUtil|http://software.clapper.org/classutil/]
> that seems to help give this kind of information much more directly. We
> should look into using that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-5312) Use sbt to detect new or changed public classes in PRs
[
https://issues.apache.org/jira/browse/SPARK-5312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14355190#comment-14355190
]
Brennon York commented on SPARK-5312:
-
Hey [~srowen] I was actually looking at this again and I think I can make a
much better solution given the link [~nchammas] put up. I figure I can add a
{{GeneratePublicClasses}} tool just like we have a {{GenerateMIMAIgnore}} tool
(and call it the same way). Thoughts? Was *just* starting to work on it so I'll
put that on hold until I hear something.
Use sbt to detect new or changed public classes in PRs
--
Key: SPARK-5312
URL: https://issues.apache.org/jira/browse/SPARK-5312
Project: Spark
Issue Type: Improvement
Components: Project Infra
Reporter: Nicholas Chammas
Priority: Minor
We currently use an [unwieldy grep/sed
contraption|https://github.com/apache/spark/blob/19556454881e05a6c2470d406d50f004b88088a2/dev/run-tests-jenkins#L152-L174]
to detect new public classes in PRs.
Apparently, sbt lets you get a list of public classes [much more
directly|http://www.scala-sbt.org/0.13/docs/Howto-Inspect-the-Build.html] via
{{show compile:discoveredMainClasses}}. We should use that instead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-5312) Use sbt to detect new or changed public classes in PRs
[
https://issues.apache.org/jira/browse/SPARK-5312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14355196#comment-14355196
]
Sean Owen commented on SPARK-5312:
--
Oh OK, I resolved given the discussion above, but if you think you have a good
shot at proposing a better solution, reopen it.
Use sbt to detect new or changed public classes in PRs
--
Key: SPARK-5312
URL: https://issues.apache.org/jira/browse/SPARK-5312
Project: Spark
Issue Type: Improvement
Components: Project Infra
Reporter: Nicholas Chammas
Priority: Minor
We currently use an [unwieldy grep/sed
contraption|https://github.com/apache/spark/blob/19556454881e05a6c2470d406d50f004b88088a2/dev/run-tests-jenkins#L152-L174]
to detect new public classes in PRs.
Apparently, sbt lets you get a list of public classes [much more
directly|http://www.scala-sbt.org/0.13/docs/Howto-Inspect-the-Build.html] via
{{show compile:discoveredMainClasses}}. We should use that instead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-5312) Use sbt to detect new or changed public classes in PRs
[
https://issues.apache.org/jira/browse/SPARK-5312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14355622#comment-14355622
]
Nicholas Chammas commented on SPARK-5312:
-
Thanks for looking into this [~boyork]. I'm looking forward to see what comes
of it.
The goal, as you hinted at, is basically to give reviewers a complement to the
MIMA check that lets them see public API changes for each PR very easily.
Use sbt to detect new or changed public classes in PRs
--
Key: SPARK-5312
URL: https://issues.apache.org/jira/browse/SPARK-5312
Project: Spark
Issue Type: Improvement
Components: Project Infra
Reporter: Nicholas Chammas
Priority: Minor
We currently use an [unwieldy grep/sed
contraption|https://github.com/apache/spark/blob/19556454881e05a6c2470d406d50f004b88088a2/dev/run-tests-jenkins#L152-L174]
to detect new public classes in PRs.
Apparently, sbt lets you get a list of public classes [much more
directly|http://www.scala-sbt.org/0.13/docs/Howto-Inspect-the-Build.html] via
{{show compile:discoveredMainClasses}}. We should use that instead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-5312) Use sbt to detect new or changed public classes in PRs
[
https://issues.apache.org/jira/browse/SPARK-5312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14335565#comment-14335565
]
Brennon York commented on SPARK-5312:
-
New thoughts... So I checked out the link which seems like it would def. do
what we want it to do, but at the cost of much more complexity (unless I'm
missing something). It looks like the easiest way to get the tool to work would
be to add a separate scala program that would need to be executed (in the
{{run-tests-jenkins}} script) producing the public classes. Pros? It should get
us exactly the classes we want, then we could {{diff}} that against the master
branch and be on our way. Cons? An entire other scala program (again, unless
I'm missing a way to integrate it) that would need to be maintained / managed.
Not sure how important this is given the amount of code added to maintain into
the project. Thoughts?
Use sbt to detect new or changed public classes in PRs
--
Key: SPARK-5312
URL: https://issues.apache.org/jira/browse/SPARK-5312
Project: Spark
Issue Type: Improvement
Components: Project Infra
Reporter: Nicholas Chammas
Priority: Minor
We currently use an [unwieldy grep/sed
contraption|https://github.com/apache/spark/blob/19556454881e05a6c2470d406d50f004b88088a2/dev/run-tests-jenkins#L152-L174]
to detect new public classes in PRs.
Apparently, sbt lets you get a list of public classes [much more
directly|http://www.scala-sbt.org/0.13/docs/Howto-Inspect-the-Build.html] via
{{show compile:discoveredMainClasses}}. We should use that instead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-5312) Use sbt to detect new or changed public classes in PRs
[
https://issues.apache.org/jira/browse/SPARK-5312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14335573#comment-14335573
]
Nicholas Chammas commented on SPARK-5312:
-
It's something to consider I guess. Spark provides strong guarantees about API
stability and the like.
Making it easy for reviewers to catch changes to public classes is supposed to
help with that. What we have is perhaps good for now, and perhaps the
foreseeable future.
So maybe we should resolve this issue for now and just keep it in mind in the
future.
cc [~pwendell]
Use sbt to detect new or changed public classes in PRs
--
Key: SPARK-5312
URL: https://issues.apache.org/jira/browse/SPARK-5312
Project: Spark
Issue Type: Improvement
Components: Project Infra
Reporter: Nicholas Chammas
Priority: Minor
We currently use an [unwieldy grep/sed
contraption|https://github.com/apache/spark/blob/19556454881e05a6c2470d406d50f004b88088a2/dev/run-tests-jenkins#L152-L174]
to detect new public classes in PRs.
Apparently, sbt lets you get a list of public classes [much more
directly|http://www.scala-sbt.org/0.13/docs/Howto-Inspect-the-Build.html] via
{{show compile:discoveredMainClasses}}. We should use that instead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-5312) Use sbt to detect new or changed public classes in PRs
[
https://issues.apache.org/jira/browse/SPARK-5312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14334168#comment-14334168
]
Brennon York commented on SPARK-5312:
-
I'm not so sure the sbt {{show compile:discoveredMainClasses}} call will give
the output we want in this case. For reference, from the documentation of that
call states that sbt detects the classes with public, static main methods for
use by the run method and, to replicate the grep/sed code, we'd need a sbt
call to gather *all* {{public}} classes and not just the {{public static main}}
classes. If the goal is merely to rework the grep/sed code I could see what I
can do, but I'm not sure its going to get much better. I checked the sbt doc's
for something resembling what we're looking for and it doesn't seem they have
that feature. Thoughts?
Use sbt to detect new or changed public classes in PRs
--
Key: SPARK-5312
URL: https://issues.apache.org/jira/browse/SPARK-5312
Project: Spark
Issue Type: Improvement
Components: Project Infra
Reporter: Nicholas Chammas
Priority: Minor
We currently use an [unwieldy grep/sed
contraption|https://github.com/apache/spark/blob/19556454881e05a6c2470d406d50f004b88088a2/dev/run-tests-jenkins#L152-L174]
to detect new public classes in PRs.
Apparently, sbt lets you get a list of public classes [much more
directly|http://www.scala-sbt.org/0.13/docs/Howto-Inspect-the-Build.html] via
{{show compile:discoveredMainClasses}}. We should use that instead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-5312) Use sbt to detect new or changed public classes in PRs
[
https://issues.apache.org/jira/browse/SPARK-5312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14334355#comment-14334355
]
Nicholas Chammas commented on SPARK-5312:
-
Yeah, this is not a priority really. I looked into sbt and agree it's probably
not suited to the task. I found something else that looks interesting:
http://software.clapper.org/classutil/
But I don't have time to look into it.
Use sbt to detect new or changed public classes in PRs
--
Key: SPARK-5312
URL: https://issues.apache.org/jira/browse/SPARK-5312
Project: Spark
Issue Type: Improvement
Components: Project Infra
Reporter: Nicholas Chammas
Priority: Minor
We currently use an [unwieldy grep/sed
contraption|https://github.com/apache/spark/blob/19556454881e05a6c2470d406d50f004b88088a2/dev/run-tests-jenkins#L152-L174]
to detect new public classes in PRs.
Apparently, sbt lets you get a list of public classes [much more
directly|http://www.scala-sbt.org/0.13/docs/Howto-Inspect-the-Build.html] via
{{show compile:discoveredMainClasses}}. We should use that instead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
