Re: [casper] Timing distribution over fiber
On Wed, 6 May 2015 at 07:35 Johan Burger wrote: > > Hi there, > > Do you maybe have any idea of requirement specifications for the HERA's RF > phase stability and time (?) - this might determine what technology could > be appropriate. > Hi Johan, Thanks for your response. We're sampling at 500 MHz, so we'd like to have a stability of "few" degrees, preferably over timescales of many hours but perhaps more reasonably on a calibration cadence of O(10 minutes) PPS is not such a big deal, and synchronization to a couple of ADC clock cycles is probably fine. We're investigating simple-ish ways to calibrate these out with signal injection. > > We at SKA Africa have after some iteration come up, with a precision RF > distribution system for many antennas. The type of laser and integrated > modulator have been proven in the field on large arrays (not just > MeerKAT). The RF can be directly transmitted (in our case up to 2-3 GHz > limited by our synthesizer - the precise frequency is 1.712GHz). 500MHz RF > over fibre can be done by this as well. There is conditioning of the RF > taking place on MeerKAT at the receiving end. As Jason said, not any or all > modules really do the job properly - we converged on a solution after > testing, that implicitly included modules evaluated from KAT-7 days, and > more recent modules from other manufacturers. > > "Low precision" timing ~100ns can indeed be done using PTP. If PPS is > required instead of an Ethernet package a special conversion board (PCIe) > is necessary. This is really enough for fringe finding - used in MeerKAT > S-band for example. That digitiser is mounted in an RFI shielded pedestal > of the antenna though. We supply the high precision PPS using our custom > system as described below. > > For our L-band digitisers mounted on the outside we had to come up with > special low power, low cost, high accuracy solution - this is being > implemented by Renier and Etienne and others here at SKA Africa (so a joint > effort by our time and frequency and digitiser team). The reason is that > White Rabbit is not compatible with 10Gbe links used on this system. > Furthermore Ethernet is actually quite noisy as per MeerKAT measurements, > and White Rabbit and PTP uses that (and with highish power consumption and > largish board size), and is not preferable in a high purity clock signal > and PPS module. We found that measurement based PPS system will meet our > requirements though, for stabilized links and provides us with accurate > absolute time references at antennas, using analog methodologies. This for > example being important in pulsar science. > > I am not sure what level of RFI shielding you would be able to mount > around modules, but as said RFI from Ethernet has certainly been found to > be an RFI culprit, and cannot be therefore be used in MeerKAT close to > sensitive modules - and needs to separately shielded. This therefore means > that if PPS is generated from White Rabbit/PTP there is still some > uncertain propagation paths left (important at least for MeerKAT) up to the > point of digitization where a timing edge is inserted. We are using > seperate fibres for PPS and RF, to further limit self-RFI and as it was > found that requirements could only be met in this way. > This is a good point, and something we'll make sure to keep in mind... Thanks again, Jack > > Regards > > Johan Burger > > > > > > For MeerKAT high precision timing a special PPS solution is used. There > are seperate PPS transmitters and > On Tue, May 5, 2015 at 7:56 PM, Michael Inggs wrote: > >> Hi >> >> We have a complete White Rabbit setup, including 5km fibre test reels. It >> is driven by a Meinberg Plenum 1 clock. We are quite happy to chat with >> anyone on this topic of phase coherence. As you all know, the hardware and >> firmware is all Open Source, so as Jason mentions, easy to incorporate. I >> think I saw that the new CASPER card has provision for WR? Members of the >> team have specialised GPSDO systems developed in the lab that seem to give >> below 10 ns of jitter. >> >> Someone has tested it over 900km and beyond in Norway and sub ps jitter >> is achieved. >> >> One drawback is it needs a 1Ge fibre link, which I guess means, these >> days, its own fibre. We spoke to CERN about developing 10Ge and above, but >> it looks complex due to the many channels embedded in these streams. There >> has been some good work on sending it over microwave, but I guess that is a >> no-no for astronomy reserves. >> >> Regards >> >> >> >> On 5 May 2015 at 19:27, Jason Manley wrote: >> >>> White rabbit is the version of IEEE1588 that locks bit clocks. Normal >>> IEEE1588 doesn't do this. For this reason, (last I looked), there was no >>> 10G implementation. 1G speeds is the highest White Rabbit implementation. >>> The HW support for "normal" IEEE1588 is used to timestamp the packets >>> without software in the loop (where the CPU will introduce jitter). >>> >>> But
Re: [casper] Timing distribution over fiber
Jason, Mike, Jonathan, others!, White rabbit, and PTP are both pretty interesting. My suspicion is that it may end up being no more expensive and less worrysome (if not necessarily actually better) to distribute PPS/ some ref. clock directly, but HERA is a good excuse to investigate these alternative solutions. Mike -- there's much reading I need to do about White Rabbit before I can usefully have a conversation with anyone about it, but at some point in the future I'd certainly be interested in taking you up on the offer to chat more about your system. Thanks again Jack On Tue, 5 May 2015 at 10:57 Michael Inggs wrote: > Hi > > We have a complete White Rabbit setup, including 5km fibre test reels. It > is driven by a Meinberg Plenum 1 clock. We are quite happy to chat with > anyone on this topic of phase coherence. As you all know, the hardware and > firmware is all Open Source, so as Jason mentions, easy to incorporate. I > think I saw that the new CASPER card has provision for WR? Members of the > team have specialised GPSDO systems developed in the lab that seem to give > below 10 ns of jitter. > > Someone has tested it over 900km and beyond in Norway and sub ps jitter is > achieved. > > One drawback is it needs a 1Ge fibre link, which I guess means, these > days, its own fibre. We spoke to CERN about developing 10Ge and above, but > it looks complex due to the many channels embedded in these streams. There > has been some good work on sending it over microwave, but I guess that is a > no-no for astronomy reserves. > > Regards > > > > On 5 May 2015 at 19:27, Jason Manley wrote: > >> White rabbit is the version of IEEE1588 that locks bit clocks. Normal >> IEEE1588 doesn't do this. For this reason, (last I looked), there was no >> 10G implementation. 1G speeds is the highest White Rabbit implementation. >> The HW support for "normal" IEEE1588 is used to timestamp the packets >> without software in the loop (where the CPU will introduce jitter). >> >> But you don't necessarily need any special HW for IEEE1588... it can all >> be done in a normal FPGA using the standard Xilinx 10G IP core. There are >> FPGA cores available for the IEEE1588 part, so you don't have to implement >> it yourself. We considered this for MeerKAT at one stage, but in the end we >> couldn't achieve the required performance. It might be quite workable for >> HERA, though. Anyway, just a thought. It'd save you buying special HW and >> running additional fibres, if it meets your performance targets. >> >> Jason Manley >> CBF Manager >> SKA-SA >> >> Cell: +27 82 662 7726 >> Work: +27 21 506 7300 >> >> On 05 May 2015, at 18:49, Dan Werthimer wrote: >> >> > >> > >> > hi dave, >> > >> > i also think distributing clock and 1 PPS is simpler than IEEE1588. >> > >> > some of the IEEE1588 and white rabbit experts are here at berkeley. >> > see for example: >> > http://chess.eecs.berkeley.edu/pubs/881/dreams.pdf >> > >> > my limited understanding is that 1588 phase locks the bit clocks of >> > routers and switches together. 1588 routers and switches have SMA >> > connectors on them so you can use external maser/rubidium/GPS >> references. >> > >> > you can achieve spectacular accuracy and stabilitity with white rabbit >> > if you employ really good oscillators at each node, >> > i think white rabbit can acheive 70 picosecond RMS time transfer. >> > >> > best, >> > >> > dan >> > >> > >> > >> > >> > >> > On Tue, May 5, 2015 at 8:52 AM, David MacMahon < >> dav...@astro.berkeley.edu> wrote: >> > Hi, Jason, >> > >> > I have a great deal of curiosity about IEEE-1588, but I've always >> wondered about the precision/stability that's attainable. Compared with >> multiple sample clocks, correlating signals sampled with one common clock >> seems far more forgiving vis a vis clock frequency/phase stability. If you >> or John could point me to any information about this, please do! >> > >> > Thanks, >> > Dave >> > >> > On May 4, 2015, at 11:44 PM, Jason Manley wrote: >> > >> > > On the far end of the concept spectrum, have you considered >> distributing time over your existing ethernet network with IEEE-1588, and >> using this to discipline local ovenised 10MHz oscs at each antenna? >> > > >> > > I'm cc'ing Johan Burger, who heads up our Timing and Frequency >> Reference subsystem, who might be able to offer some additional insight. I >> know they've tried a few different lasers and detectors, with varying >> levels of success. >> > > >> > > Jason Manley >> > > CBF Manager >> > > SKA-SA >> > > >> > > Cell: +27 82 662 7726 >> > > Work: +27 21 506 7300 >> > > >> > > On 05 May 2015, at 5:18, Bob Stricklin wrote: >> > > >> > >> Hi Jack and John, >> > >> >> > >> I wanted to add an input here….. >> > >> >> > >> I am working on a 10 MHz GPS slaved reference for my personal use. I >> am working with a Analog Devices AD9548 Evaluation board (~$250) , GPS with >> 1 PPS, and a ovenized 10 MHz osc. I also plan to distribute this clock and >> have considere
Re: [casper] Timing distribution over fiber
Hi Bob, Sounds interesting -- for what it's worth, I've just learnt some fellow astronomers in Italy (Andrea Maccaferri at INAF's Medicina observatory) have built some PPS/refclk distribution systems based on the Avago fiber parts -- perhaps you might be interested in getting in touch with him... Thanks (to everyone on this thread) Jack On Mon, 4 May 2015 at 20:19 Bob Stricklin wrote: > Hi Jack and John, > > I wanted to add an input here….. > > I am working on a 10 MHz GPS slaved reference for my personal use. I am > working with a Analog Devices AD9548 Evaluation board (~$250) , GPS with 1 > PPS, and a ovenized 10 MHz osc. I also plan to distribute this clock and > have considered the Avago fiber product line. One of the older generation > Avago fiber parts should work fine for <$25 per channel. With careful > control of lengths and delays it should be possible to maintain good > phasing between channels. The analog devices chip is <$50 so a custom > solution should be <$500/reference but with considerable development time. > > Bob Stricklin > > > On May 4, 2015, at 10:02 PM, Jack Hickish wrote: > > Hi John, > > Thanks for the info. I'll add Litelink to my list of suppliers to > investigate. > We have no particular urge to multiplex the signals on to the fiber unless > there's a particularly neat/cheap solution to do that. There's no great > appetite to go custom. We've got about ~30 nodes, and my first stab at > getting an off-the-shelf solution turned up at a few k$ / node, not > including any cleanup electronics. > > Thanks again, > > Jack > > On Mon, 4 May 2015 at 19:25 John Ford wrote: > >> > Hi CASPERites, >> > >> > For HERA, we're looking at distributing timing signals (PPS & 10Mhz ref >> or >> > 500 MHz clock) over O(100m) fibers to various digitization nodes. >> > I figure some folks in CASPERland have experience with this kind of >> > system? >> > Did you use custom RF-over-fiber kit, or off-the-shelf PPS/10MHz >> > solutions? >> > Any words of wisdom/caution to share? >> > >> > Any responses much appreciated! >> >> We have several different schemes for the different signals. Are you >> planning for one fiber per signal per node? or one fiber with the signals >> multiplexed on them? >> >> If the signals are one per signal, you can use some off-the-shelf >> solutions, but they are kind of pricey, and if you have a lot of nodes to >> supply, it might be worth working on something custom. We have used Math >> Associates stuff for this kind of work. Math Associates is now litelink, >> and they tout the affordability of their stuff, so maybe it's >> reasonable... >> >> >> On the 10 MHz, we send the 10 MHz reference over fiber, and at the far end >> use a crystal oscillator locked to the reference to clean up the noise >> from the fiber electronics. This is essential for interferometry, but >> maybe not for single-dish use. >> >> John >> >> > >> > Jack >> > >> >> >> >
[casper] ??help??xps failed half when EDK
Dear casper, I made an easy .mdl file according to tutorial1, but when i run casper_xps, it comes out an error,shows below: XSG generation complete. # ## Copying base system ## # ## Copying custom IPs ## ## ## Creating Simulink IP ## ## ## ## Creating EDK files ## ## Error trying to backup system.xmp: No matching files were found. Error trying to overwrite system.xmp: No matching files were found. Error trying to backup system.mhs: No matching files were found. Error trying to backup core_info.tab: No matching files were found. Running off sys_clk @100MHz Error using fgets Invalid file identifier. Use fopen to generate a valid file identifier. I cant't find anywhere wrong, as simulink simulation is correct. Thanks a lot! -- Xinying Yu B332, National Astronomical Observatories 20A Datun Road, Chaoyang District Beijing 100012 China yuxinying199110...@163.com Tel: 15600602147 ***
[casper] support for 40G infiniband?
Hi list, there are some plans for another hybrid (hw/sw) wideband spectrometer and multi-beam correlator. Data input is the main issue, options are network attached samplers versus PCIe cards. Has any casperite by chance been working on 40 Gbps? Might it be possible to have Roach2 CX4 run at 40 GbE and interface to Infiniband cards? Or would this require the HXT version of Virtex-6 (in addition to probably extra work on the ten_gbe core)? cheers, Jan
