Re: amplab jenkins is down
i'll take a look at this today. On Mon, Sep 8, 2014 at 1:13 AM, Josh Rosen wrote: > Yeah, I think https://github.com/apache/spark/pull/2315 should have fixed > the Mima issue. We're still seeing some intermittent failures due to > DriverSuite and SparkSubmitSuite tests failing, so I'd appreciate any help > in diagnosing that issue. > > On Sun, Sep 7, 2014 at 10:08 PM, Prashant Sharma > wrote: > >> Looks like this is already taken care of ? >> >> Prashant Sharma >> >> >> >> On Mon, Sep 8, 2014 at 4:37 AM, Josh Rosen wrote: >> >>> Does anyone know why some of the MiMa tests have started failing? >>> >>> See >>> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19948/consoleFull >>> for >>> an example. >>> >>> On September 6, 2014 at 12:48:27 PM, Josh Rosen (rosenvi...@gmail.com) >>> wrote: >>> >>> It looks like Jenkins is up and running, but there seems to be a delay >>> in responding to requests to re-test patches. It seems like Jenkins is >>> promptly testing new PRs, or new commits as they’re added to existing PRs, >>> but taking a very long time to respond to requests to re-test PRs. >>> >>> I’m going to continue monitoring this today. I’m considering creating >>> my own fork of the Jenkins pull request builder plugin so that we can add >>> extra logging in order to diagnose what’s causing this lag. >>> >>> - Josh >>> On September 5, 2014 at 5:49:32 PM, Josh Rosen (rosenvi...@gmail.com) >>> wrote: >>> >>> We have successfully purged Jenkins’ build queue. If you want a PR to >>> be re-tested, please ask Jenkins again. >>> >>> On September 5, 2014 at 5:36:30 PM, shane knapp (skn...@berkeley.edu) >>> wrote: >>> >>> yeah, it was a problem w/the PRB's OAuth key. josh rosen added a new key, >>> and magique! >>> >>> we're about to clear the queue of all builds as most aren't >>> wanted/needed. >>> >>> >>> On Fri, Sep 5, 2014 at 5:33 PM, Nicholas Chammas < >>> nicholas.cham...@gmail.com >>> > wrote: >>> >>> > Looks like Jenkins is back! >>> > >>> > lol The poor guy has like a million builds >>> > < >>> https://amplab.cs.berkeley.edu/jenkins/view/Pull%20Request%20Builders/job/SparkPullRequestBuilder/ >>> > >>> > to catch up on. >>> > >>> > >>> > On Fri, Sep 5, 2014 at 4:15 PM, Nicholas Chammas < >>> > nicholas.cham...@gmail.com> wrote: >>> > >>> >> How's it going? >>> >> >>> >> It looks like during the last build >>> >> < >>> https://amplab.cs.berkeley.edu/jenkins/view/Pull%20Request%20Builders/job/SparkPullRequestBuilder/lastBuild/console >>> > >>> >> from about 30 min ago Jenkins was still having trouble fetching from >>> >> GitHub. It also looks like not all requests for testing are triggering >>> >> builds. >>> >> >>> >> >>> >> On Fri, Sep 5, 2014 at 1:23 PM, shane knapp >>> wrote: >>> >> >>> >>> it's looking like everything except the pull request builders are >>> >>> working. i'm going to be working on getting this resolved today. >>> >>> >>> >>> >>> >>> On Fri, Sep 5, 2014 at 8:18 AM, Nicholas Chammas < >>> >>> nicholas.cham...@gmail.com> wrote: >>> >>> >>> Hmm, looks like at least some builds >>> < >>> https://amplab.cs.berkeley.edu/jenkins/view/Pull%20Request%20Builders/job/SparkPullRequestBuilder/19804/consoleFull >>> > >>> are working now, though this last one was from ~5 hours ago. >>> >>> >>> On Fri, Sep 5, 2014 at 1:02 AM, shane knapp >>> wrote: >>> >>> > yep. that's exactly the behavior i saw earlier, and will be >>> figuring >>> > out first thing tomorrow morning. i bet it's an environment issues >>> on the >>> > slaves. >>> > >>> > >>> > On Thu, Sep 4, 2014 at 7:10 PM, Nicholas Chammas < >>> > nicholas.cham...@gmail.com> wrote: >>> > >>> >> Looks like during the last build >>> >> < >>> https://amplab.cs.berkeley.edu/jenkins/view/Pull%20Request%20Builders/job/SparkPullRequestBuilder/19797/console >>> > >>> >> Jenkins was unable to execute a git fetch? >>> >> >>> >> >>> >> On Thu, Sep 4, 2014 at 7:58 PM, shane knapp >>> >> wrote: >>> >> >>> >>> i'm going to restart jenkins and see if that fixes things. >>> >>> >>> >>> >>> >>> On Thu, Sep 4, 2014 at 4:56 PM, shane knapp >> > >>> >>> wrote: >>> >>> >>> looking >>> >>> >>> On Thu, Sep 4, 2014 at 4:21 PM, Nicholas Chammas < >>> nicholas.cham...@gmail.com> wrote: >>> >>> > It appears that our main man is having trouble >>> > < >>> https://amplab.cs.berkeley.edu/jenkins/view/Pull%20Request%20Builders/job/SparkPullRequestBuilder/ >>> > >>> > hearing new requests >>> > < >>> https://github.com/apache/spark/pull/2277#issuecomment-54549106>. >>> > >>> > Do we need some smelling salts? >>> > >>> > >>> > On Thu, Sep 4, 2014 at 5:49 PM, shane knapp < >>> skn...@berkeley.edu> >>> > wrote: >>> > >>> >> i'd ping the Jenkinsmench... the master was completely >>>
Re: amplab jenkins is down
Yeah, I think https://github.com/apache/spark/pull/2315 should have fixed the Mima issue. We're still seeing some intermittent failures due to DriverSuite and SparkSubmitSuite tests failing, so I'd appreciate any help in diagnosing that issue. On Sun, Sep 7, 2014 at 10:08 PM, Prashant Sharma wrote: > Looks like this is already taken care of ? > > Prashant Sharma > > > > On Mon, Sep 8, 2014 at 4:37 AM, Josh Rosen wrote: > >> Does anyone know why some of the MiMa tests have started failing? >> >> See >> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19948/consoleFull >> for >> an example. >> >> On September 6, 2014 at 12:48:27 PM, Josh Rosen (rosenvi...@gmail.com) >> wrote: >> >> It looks like Jenkins is up and running, but there seems to be a delay in >> responding to requests to re-test patches. It seems like Jenkins is >> promptly testing new PRs, or new commits as they’re added to existing PRs, >> but taking a very long time to respond to requests to re-test PRs. >> >> I’m going to continue monitoring this today. I’m considering creating my >> own fork of the Jenkins pull request builder plugin so that we can add >> extra logging in order to diagnose what’s causing this lag. >> >> - Josh >> On September 5, 2014 at 5:49:32 PM, Josh Rosen (rosenvi...@gmail.com) >> wrote: >> >> We have successfully purged Jenkins’ build queue. If you want a PR to be >> re-tested, please ask Jenkins again. >> >> On September 5, 2014 at 5:36:30 PM, shane knapp (skn...@berkeley.edu) >> wrote: >> >> yeah, it was a problem w/the PRB's OAuth key. josh rosen added a new key, >> and magique! >> >> we're about to clear the queue of all builds as most aren't wanted/needed. >> >> >> On Fri, Sep 5, 2014 at 5:33 PM, Nicholas Chammas < >> nicholas.cham...@gmail.com >> > wrote: >> >> > Looks like Jenkins is back! >> > >> > lol The poor guy has like a million builds >> > < >> https://amplab.cs.berkeley.edu/jenkins/view/Pull%20Request%20Builders/job/SparkPullRequestBuilder/ >> > >> > to catch up on. >> > >> > >> > On Fri, Sep 5, 2014 at 4:15 PM, Nicholas Chammas < >> > nicholas.cham...@gmail.com> wrote: >> > >> >> How's it going? >> >> >> >> It looks like during the last build >> >> < >> https://amplab.cs.berkeley.edu/jenkins/view/Pull%20Request%20Builders/job/SparkPullRequestBuilder/lastBuild/console >> > >> >> from about 30 min ago Jenkins was still having trouble fetching from >> >> GitHub. It also looks like not all requests for testing are triggering >> >> builds. >> >> >> >> >> >> On Fri, Sep 5, 2014 at 1:23 PM, shane knapp >> wrote: >> >> >> >>> it's looking like everything except the pull request builders are >> >>> working. i'm going to be working on getting this resolved today. >> >>> >> >>> >> >>> On Fri, Sep 5, 2014 at 8:18 AM, Nicholas Chammas < >> >>> nicholas.cham...@gmail.com> wrote: >> >>> >> Hmm, looks like at least some builds >> < >> https://amplab.cs.berkeley.edu/jenkins/view/Pull%20Request%20Builders/job/SparkPullRequestBuilder/19804/consoleFull >> > >> are working now, though this last one was from ~5 hours ago. >> >> >> On Fri, Sep 5, 2014 at 1:02 AM, shane knapp >> wrote: >> >> > yep. that's exactly the behavior i saw earlier, and will be figuring >> > out first thing tomorrow morning. i bet it's an environment issues >> on the >> > slaves. >> > >> > >> > On Thu, Sep 4, 2014 at 7:10 PM, Nicholas Chammas < >> > nicholas.cham...@gmail.com> wrote: >> > >> >> Looks like during the last build >> >> < >> https://amplab.cs.berkeley.edu/jenkins/view/Pull%20Request%20Builders/job/SparkPullRequestBuilder/19797/console >> > >> >> Jenkins was unable to execute a git fetch? >> >> >> >> >> >> On Thu, Sep 4, 2014 at 7:58 PM, shane knapp >> >> wrote: >> >> >> >>> i'm going to restart jenkins and see if that fixes things. >> >>> >> >>> >> >>> On Thu, Sep 4, 2014 at 4:56 PM, shane knapp >> >>> wrote: >> >>> >> looking >> >> >> On Thu, Sep 4, 2014 at 4:21 PM, Nicholas Chammas < >> nicholas.cham...@gmail.com> wrote: >> >> > It appears that our main man is having trouble >> > < >> https://amplab.cs.berkeley.edu/jenkins/view/Pull%20Request%20Builders/job/SparkPullRequestBuilder/ >> > >> > hearing new requests >> > < >> https://github.com/apache/spark/pull/2277#issuecomment-54549106>. >> > >> > Do we need some smelling salts? >> > >> > >> > On Thu, Sep 4, 2014 at 5:49 PM, shane knapp < >> skn...@berkeley.edu> >> > wrote: >> > >> >> i'd ping the Jenkinsmench... the master was completely offline, >> >> so any new >> >> jobs wouldn't have reached it. any jobs that were queued when >> >> power was >> >> lost probably started up, but jobs that were running would >> fail. >> >> >> >> >> >
Re: [mllib] Add multiplying large scale matrices
Hi Xiangrui Meng, Thank you for your comment and creating tickets. The ticket which I created would be moved to your tickets. I will close my ticket, and then will link it to yours later. Best, Yu Ishikawa -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/mllib-Add-multiplying-large-scale-matrices-tp8291p8333.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [mllib] Add multiplying large scale matrices
Sorry for my late reply! I'm also very interested in the implementation of distributed matrix multiplication. As Shivaram mentioned, the communication is the concern here. But maybe we can start with a reasonable implementation and then iterate on its performance. It would be great if eventually we can implement an algorithm close to the 2.5D algorithm (http://www.netlib.org/lapack/lawnspdf/lawn248.pdf). I created two JIRAs for this topic: 1. Distributed block matrix: https://issues.apache.org/jira/browse/SPARK-3434 2. Distributed matrix multiplication: https://issues.apache.org/jira/browse/SPARK-3435 We can move our discussion there. Rong, I'm really happy to see the Saury project. It would be great if you can share your design and experience (maybe on the JIRA page so it is easier to track). I will read the reports on CSDN and ping you if I ran into problems. Thanks! Best, Xiangrui On Sat, Sep 6, 2014 at 1:28 AM, Yu Ishikawa wrote: > Hi Rong, > > Great job! Thank you for let me know your work. > I will read the source code of saury later. > > Although AMPLab is working to implement them, would you like to merge it > into Spark? > > Best, > > -- Yu Ishikawa > > > > > -- > View this message in context: > http://apache-spark-developers-list.1001551.n3.nabble.com/mllib-Add-multiplying-large-scale-matrices-tp8291p8310.html > Sent from the Apache Spark Developers List mailing list archive at Nabble.com. > > - > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > For additional commands, e-mail: dev-h...@spark.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
