[jira] [Commented] (SPARK-26944) Python unit-tests.log not available in artifacts for a build in Jenkins
[
https://issues.apache.org/jira/browse/SPARK-26944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835074#comment-16835074
]
shane knapp commented on SPARK-26944:
-
hmm, actually... i don't think i will need to do anything. the code is a
little confusing, but this is actually behaving as intended. :)
if all py* unit tests pass, nothing is written to python/unit-tests.log
if a test fails, then that output is written to python/unit-tests.log and
testing stops.
the *actual* output from the python unit tests (failing or successful) is
written to target/unit-tests.log along with EVERY OTHER BIT OF UNIT TEST OUTPUT
FROM THE ENTIRE BUILD (emphasis mine).
this isn't actually that useful, and the target/unit-tests.log file is
gigabytes in size. however, breaking the python-only output out of here will
most likely be difficult as this output is logged through scala (ie:
core/src/main/scala/org/apache/spark/util/ShutdownHookManager.scala).
does this make sense?
> Python unit-tests.log not available in artifacts for a build in Jenkins
> ---
>
> Key: SPARK-26944
> URL: https://issues.apache.org/jira/browse/SPARK-26944
> Project: Spark
> Issue Type: Improvement
> Components: Build
>Affects Versions: 2.4.0
>Reporter: Alessandro Bellina
>Assignee: shane knapp
>Priority: Minor
> Attachments: Screen Shot 2019-03-05 at 12.08.43 PM.png
>
>
> I had a pr where the python unit tests failed. The tests point at the
> `/home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log` file,
> but I can't get to that from jenkins UI it seems (are all prs writing to the
> same file?).
> {code:java}
>
> Running PySpark tests
>
> Running PySpark tests. Output is in
> /home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log{code}
> For reference, please see this build:
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/102518/console
> This Jira is to make it available under the artifacts for each build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26944) Python unit-tests.log not available in artifacts for a build in Jenkins
[
https://issues.apache.org/jira/browse/SPARK-26944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835031#comment-16835031
]
shane knapp commented on SPARK-26944:
-
going through my queue, i found this and realized i hadn't verified that it
worked. and it doesn't.
because there's a bug in python/run-tests.py that overrides the output to a
different location *after* it prints out where it plans on writing the log file
to.
PR incoming. this will need to be backported to 2.3 and 2.4.
> Python unit-tests.log not available in artifacts for a build in Jenkins
> ---
>
> Key: SPARK-26944
> URL: https://issues.apache.org/jira/browse/SPARK-26944
> Project: Spark
> Issue Type: Improvement
> Components: Build
>Affects Versions: 2.4.0
>Reporter: Alessandro Bellina
>Assignee: shane knapp
>Priority: Minor
> Attachments: Screen Shot 2019-03-05 at 12.08.43 PM.png
>
>
> I had a pr where the python unit tests failed. The tests point at the
> `/home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log` file,
> but I can't get to that from jenkins UI it seems (are all prs writing to the
> same file?).
> {code:java}
>
> Running PySpark tests
>
> Running PySpark tests. Output is in
> /home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log{code}
> For reference, please see this build:
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/102518/console
> This Jira is to make it available under the artifacts for each build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26944) Python unit-tests.log not available in artifacts for a build in Jenkins
[
https://issues.apache.org/jira/browse/SPARK-26944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784853#comment-16784853
]
Alessandro Bellina commented on SPARK-26944:
[~shaneknapp] nice!! thank you
> Python unit-tests.log not available in artifacts for a build in Jenkins
> ---
>
> Key: SPARK-26944
> URL: https://issues.apache.org/jira/browse/SPARK-26944
> Project: Spark
> Issue Type: Improvement
> Components: Build
>Affects Versions: 2.4.0
>Reporter: Alessandro Bellina
>Assignee: shane knapp
>Priority: Minor
> Attachments: Screen Shot 2019-03-05 at 12.08.43 PM.png
>
>
> I had a pr where the python unit tests failed. The tests point at the
> `/home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log` file,
> but I can't get to that from jenkins UI it seems (are all prs writing to the
> same file?).
> {code:java}
>
> Running PySpark tests
>
> Running PySpark tests. Output is in
> /home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log{code}
> For reference, please see this build:
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/102518/console
> This Jira is to make it available under the artifacts for each build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26944) Python unit-tests.log not available in artifacts for a build in Jenkins
[
https://issues.apache.org/jira/browse/SPARK-26944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784852#comment-16784852
]
shane knapp commented on SPARK-26944:
-
added a glob to store these (see attached image). !Screen Shot 2019-03-05 at
12.08.43 PM.png!
> Python unit-tests.log not available in artifacts for a build in Jenkins
> ---
>
> Key: SPARK-26944
> URL: https://issues.apache.org/jira/browse/SPARK-26944
> Project: Spark
> Issue Type: Improvement
> Components: Build
>Affects Versions: 2.4.0
>Reporter: Alessandro Bellina
>Assignee: shane knapp
>Priority: Minor
> Attachments: Screen Shot 2019-03-05 at 12.08.43 PM.png
>
>
> I had a pr where the python unit tests failed. The tests point at the
> `/home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log` file,
> but I can't get to that from jenkins UI it seems (are all prs writing to the
> same file?).
> {code:java}
>
> Running PySpark tests
>
> Running PySpark tests. Output is in
> /home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log{code}
> For reference, please see this build:
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/102518/console
> This Jira is to make it available under the artifacts for each build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26944) Python unit-tests.log not available in artifacts for a build in Jenkins
[
https://issues.apache.org/jira/browse/SPARK-26944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784856#comment-16784856
]
shane knapp commented on SPARK-26944:
-
ill confirm that this works after the current PRB builds finish before closing
this.
> Python unit-tests.log not available in artifacts for a build in Jenkins
> ---
>
> Key: SPARK-26944
> URL: https://issues.apache.org/jira/browse/SPARK-26944
> Project: Spark
> Issue Type: Improvement
> Components: Build
>Affects Versions: 2.4.0
>Reporter: Alessandro Bellina
>Assignee: shane knapp
>Priority: Minor
> Attachments: Screen Shot 2019-03-05 at 12.08.43 PM.png
>
>
> I had a pr where the python unit tests failed. The tests point at the
> `/home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log` file,
> but I can't get to that from jenkins UI it seems (are all prs writing to the
> same file?).
> {code:java}
>
> Running PySpark tests
>
> Running PySpark tests. Output is in
> /home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log{code}
> For reference, please see this build:
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/102518/console
> This Jira is to make it available under the artifacts for each build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26944) Python unit-tests.log not available in artifacts for a build in Jenkins
[
https://issues.apache.org/jira/browse/SPARK-26944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782085#comment-16782085
]
Sean Owen commented on SPARK-26944:
---
I can see things like
https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-branch-2.4-test-sbt-hadoop-2.7/315/artifact/target/unit-tests.log
but not the python one. [~shaneknapp] just in case there's an easy way to
make Jenkins save that?
> Python unit-tests.log not available in artifacts for a build in Jenkins
> ---
>
> Key: SPARK-26944
> URL: https://issues.apache.org/jira/browse/SPARK-26944
> Project: Spark
> Issue Type: Improvement
> Components: Build
>Affects Versions: 2.4.0
>Reporter: Alessandro Bellina
>Priority: Minor
>
> I had a pr where the python unit tests failed. The tests point at the
> `/home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log` file,
> but I can't get to that from jenkins UI it seems (are all prs writing to the
> same file?).
> {code:java}
>
> Running PySpark tests
>
> Running PySpark tests. Output is in
> /home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log{code}
> For reference, please see this build:
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/102518/console
> This Jira is to make it available under the artifacts for each build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26944) Python unit-tests.log not available in artifacts for a build in Jenkins
[
https://issues.apache.org/jira/browse/SPARK-26944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775117#comment-16775117
]
Alessandro Bellina commented on SPARK-26944:
Hmm, I have a subsequent build from the same PR, and I don't see a link to the
python tests either. Maybe I am looking in the wrong place?
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/102590/artifact/
> Python unit-tests.log not available in artifacts for a build in Jenkins
> ---
>
> Key: SPARK-26944
> URL: https://issues.apache.org/jira/browse/SPARK-26944
> Project: Spark
> Issue Type: Improvement
> Components: Build
>Affects Versions: 2.4.0
>Reporter: Alessandro Bellina
>Priority: Minor
>
> I had a pr where the python unit tests failed. The tests point at the
> `/home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log` file,
> but I can't get to that from jenkins UI it seems (are all prs writing to the
> same file?).
> {code:java}
>
> Running PySpark tests
>
> Running PySpark tests. Output is in
> /home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log{code}
> For reference, please see this build:
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/102518/console
> This Jira is to make it available under the artifacts for each build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26944) Python unit-tests.log not available in artifacts for a build in Jenkins
[
https://issues.apache.org/jira/browse/SPARK-26944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16774892#comment-16774892
]
Hyukjin Kwon commented on SPARK-26944:
--
Actually it's usually able to see it. It's usually included in the artifact of
the built image IIRC.
> Python unit-tests.log not available in artifacts for a build in Jenkins
> ---
>
> Key: SPARK-26944
> URL: https://issues.apache.org/jira/browse/SPARK-26944
> Project: Spark
> Issue Type: Improvement
> Components: Build
>Affects Versions: 2.4.0
>Reporter: Alessandro Bellina
>Priority: Minor
>
> I had a pr where the python unit tests failed. The tests point at the
> `/home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log` file,
> but I can't get to that from jenkins UI it seems (are all prs writing to the
> same file?).
> {code:java}
>
> Running PySpark tests
>
> Running PySpark tests. Output is in
> /home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log{code}
> For reference, please see this build:
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/102518/console
> This Jira is to make it available under the artifacts for each build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26944) Python unit-tests.log not available in artifacts for a build in Jenkins
[
https://issues.apache.org/jira/browse/SPARK-26944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16774800#comment-16774800
]
Alessandro Bellina commented on SPARK-26944:
[~shivuson...@gmail.com] [~hyukjin.kwon] I added a link to the description. I
think we should look at the Jenkins configuration to see what is getting copied
to artifacts, though I believe you need special access for that.
> Python unit-tests.log not available in artifacts for a build in Jenkins
> ---
>
> Key: SPARK-26944
> URL: https://issues.apache.org/jira/browse/SPARK-26944
> Project: Spark
> Issue Type: Improvement
> Components: Build
>Affects Versions: 2.4.0
>Reporter: Alessandro Bellina
>Priority: Minor
>
> I had a pr where the python unit tests failed. The tests point at the
> `/home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log` file,
> but I can't get to that from jenkins UI it seems (are all prs writing to the
> same file?).
> {code:java}
>
> Running PySpark tests
>
> Running PySpark tests. Output is in
> /home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log{code}
> For reference, please see this build:
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/102518/console
> This Jira is to make it available under the artifacts for each build.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26944) Python unit-tests.log not available in artifacts for a build in Jenkins
[ https://issues.apache.org/jira/browse/SPARK-26944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16774768#comment-16774768 ] Hyukjin Kwon commented on SPARK-26944: -- Yes, can you point out your PR? > Python unit-tests.log not available in artifacts for a build in Jenkins > --- > > Key: SPARK-26944 > URL: https://issues.apache.org/jira/browse/SPARK-26944 > Project: Spark > Issue Type: Improvement > Components: Build >Affects Versions: 2.4.0 >Reporter: Alessandro Bellina >Priority: Minor > > I had a pr where the python unit tests failed. The tests point at the > `/home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log` file, > but I can't get to that from jenkins UI it seems (are all prs writing to the > same file?). > This Jira is to make it available under the artifacts for each build. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26944) Python unit-tests.log not available in artifacts for a build in Jenkins
[ https://issues.apache.org/jira/browse/SPARK-26944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16774725#comment-16774725 ] Shivu Sondur commented on SPARK-26944: -- [~abellina] Can you mention more info, like Jenkins link, where issue is happening and other details. > Python unit-tests.log not available in artifacts for a build in Jenkins > --- > > Key: SPARK-26944 > URL: https://issues.apache.org/jira/browse/SPARK-26944 > Project: Spark > Issue Type: Improvement > Components: Build >Affects Versions: 2.4.0 >Reporter: Alessandro Bellina >Priority: Minor > > I had a pr where the python unit tests failed. The tests point at the > `/home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log` file, > but I can't get to that from jenkins UI it seems (are all prs writing to the > same file?). > This Jira is to make it available under the artifacts for each build. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
