OK, I'll stop future observations after the current observation ends. I think everything on the digital side is in good shape except that we need to fix or replace the pf4 roach2. I know we talked about obtaining another ROACH2 to send to SA when pf4 first had power problems, but I don't know how far we got with that once Matthys gave pf4 some TLC and got it back on the air.
Dave On Feb 19, 2015, at 12:42 PM, Aaron Parsons wrote: > We agreed to stop at yesterday's meeting. Let it be done. > > *But* let's make an effort to get everything in working order nowish, rather > than later. > > On Thu, Feb 19, 2015 at 12:39 PM, danny jacobs <[email protected]> > wrote: > Hi Dave et al, > > The still is actually basically at capacity <1TB free on each pot, I'd say > its about time to stop. > > > Cheers, > ~Dany > > > On Thu, Feb 19, 2015 at 1:10 PM, David MacMahon <[email protected]> > wrote: > Is anybody taking any action on this or are we just going to let things run > their course as-is? > > In case it wasn't clear, the ROACH2 named "pf4" is down again, most likely > due to a power problem like before. This means that we are missing 16 > antennas(!) as well as 4/32 == 1/8 of the band. The "autoplots" on the blog > show the missing parts of the band, but they (and probably the MIRIAD > datasets) deceptively show old (recycled) data for the missing antennas. > > Thanks, > Dave > > On Feb 17, 2015, at 9:40 PM, David MacMahon wrote: > > > Hi, Danny at al., > > > > The problem is not with px4, but instead with pf4 which has gone down > > (again). I had forgotten that px4 is directly connected to pf4 instead of > > the switch. > > > > How many observing days are left? Would it be reasonable to ask Matthys to > > pay another visit to the PAPER container? > > > > Dave > > > > On Feb 17, 2015, at 8:14 PM, David MacMahon wrote: > > > >> Hi, Danny, > >> > >> This is NOT another IERR event. px4 is up, but it is not getting any > >> packets. In fact, it looks like none of the 10 GbE interfaces even have a > >> link! > >> > >> davidm@px4:~$ for e in eth{2..5}; do sudo ethtool $e | grep detected; done > >> Link detected: no > >> Link detected: no > >> Link detected: no > >> Link detected: no > >> > >> I have no idea what would have caused this other than cabling problems. > >> I'm going to reboot. > >> > >> Dave > >> > >> On Feb 17, 2015, at 7:58 PM, David MacMahon wrote: > >> > >>> Hi, Danny, > >>> > >>> Looks like px4 had another IERR event. Observing is about to end for the > >>> night there (in about 3 minutes from this writing) so I'll fix it up > >>> after that and it should be all good to go for tomorrow night. > >>> > >>> Dave > >>> > >>> > >>> Begin forwarded message: > >>> > >>>> From: [email protected] (David MacMahon) > >>>> Date: February 17, 2015 6:50:01 PM PST > >>>> To: [email protected], [email protected] > >>>> Subject: PAPER Correlator Status Report > >>>> > >>>> 2015-02-18T02:50:01+00:00 > >>>> > >>>> Got OUTDUMPS for all X engine instances > >>>> Bad OUTDUMPS value for X engine instance px4/0 > >>>> Bad OUTDUMPS value for X engine instance px4/1 > >>>> Bad OUTDUMPS value for X engine instance px4/2 > >>>> Bad OUTDUMPS value for X engine instance px4/3 > >>>> > >>>> Got GPUGBPS for all X engine instances > >>>> GPUGBPS for present X engine instances all in range > >>>> > >>>> Detected 4 errors > >>> > >> > > > > > > > > -- > > National Science Foundation Fellow > Arizona State University > School of Earth and Space Exploration > Low Frequency Cosmology > Phone: (505) 500 4521 > Homepage: http://loco.lab.asu.edu/danny_jacobs/ > > > > -- > Aaron Parsons > > 510-406-4322 (cell) > Campbell Hall 523, UCB
