On Wed, Jul 20, 2016 at 6:31 PM, Thomas Leonard <[email protected]> wrote:
> On 20 July 2016 at 18:10, Hannes Mehnert <[email protected]> wrote: > > Hi Kia, > > > > On 20/07/2016 18:54, Kia wrote: > >> I have a working NTP client written in ocaml and that runs in mirageos. > I still > >> need to do cleanup work, write and verify code that properly handles > leap > >> second events, write .mli files, and write tests -- but the client code > works. > >> I have a unikernel which periodically queries an NTP server to generate > data > >> that lets other code in the unikernel convert the current value of > RDTSC [0] to > >> an absolute time: > >> > >> > https://github.com/matildah/mirage-ntp/blob/a5d06d05cc20bb9af680238437aeea5835fc54fa/unikernel/unikernel.ml#L58-L63 > >> > >> The algorithms used to convert timestamped (with RDTSC when they are > >> sent/received) NTP packets to offset/rate information that can be used > to > >> convert the current RDTSC value to a time (or a pair of counter values > to a > >> difference of time) are a reimplementation of RADclock's (source at > >> https://github.com/synclab/radclock), papers at > http://www.synclab.org/docs/). > >> RADclock has been extensively tested with months of data (and shown to > give > >> accuracy an order of magnitude better than the reference NTP > implementation in > >> the same conditions) -- http://www.synclab.org/testbed/ has details > and my post > >> https://matildah.github.io/posts/2016-05-23-ntp-status.html concisely > explains > >> the advantages of RADclock's feed-forward clock synchronization and its > >> suitability for a unikernel environment. > >> > >> > >> The user-facing parts of the client code are: > >> A type representing the current state of the NTP client, "state": > >> > https://github.com/matildah/mirage-ntp/blob/a5d06d05cc20bb9af680238437aeea5835fc54fa/lib/client/types.ml#L196-L202 > >> A function that generates a new NTP query, "new_query": > >> > https://github.com/matildah/mirage-ntp/blob/a5d06d05cc20bb9af680238437aeea5835fc54fa/lib/client/ntp_client.ml#L66-L69 > >> A function that updates the current state with the NTP server's reply, > "add_sample": > >> > https://github.com/matildah/mirage-ntp/blob/a5d06d05cc20bb9af680238437aeea5835fc54fa/lib/client/ntp_client.ml#L118-L121 > >> A type that contains current rate/offset estimates, "output": > >> > https://github.com/matildah/mirage-ntp/blob/a5d06d05cc20bb9af680238437aeea5835fc54fa/lib/client/types.ml#L73-L80 > >> A function that takes the current NTP client's state and extracts the > current rate/offset estimates, "output_of_state": > >> > https://github.com/matildah/mirage-ntp/blob/a5d06d05cc20bb9af680238437aeea5835fc54fa/lib/client/ntp_client.ml#L123-L136 > >> > >> and you can see them called in the demonstration unikernel code at > >> > https://github.com/matildah/mirage-ntp/blob/a5d06d05cc20bb9af680238437aeea5835fc54fa/unikernel/unikernel.ml#L51-L63 > >> > >> The client code can take the value returned by output_of_state and the > current value of RDTSC > >> and use > https://github.com/matildah/mirage-ntp/blob/a5d06d05cc20bb9af680238437aeea5835fc54fa/lib/clocklib/diffabs.ml#L3-L10 > >> to generate an absolute time. (I haven't written the difference time > function yet). > > > > This is great! > > > >> I have a few questions about how to properly integrate my client code > into a > >> module that can be used like https://github.com/mirage/mirage-clock by > any > >> unikernel that needs the time (but doesn't depend on time information > from > >> dom0). > >> > >> 1. How can I make my NTP client module depend on the mirage RANDOM > module and > >> call int32 from it? (The new_query function needs to be able to get some > >> randomness to put in the packet as a nonce) > > > > There are two options here > > - if it is code already depending on mirage-types, define a functor (as > > done e.g. in UDP > > > https://github.com/mirage/mirage-tcpip/blob/ed6dd67f7ed47acba49c8021546a4b9c38d3359c/lib/udp/udp.ml > ). > > this will then use the MirageOS Random device. > > - if the code otherwise does not depend on mirage, depending on which > > random you need (real or fake), use Nocrypto.Rng.generate 4 (and depend > > in opam on nocrypto), or Random.int32 > > > >> 2. Is there way to have a module that starts a background thread (in my > case, > >> that queries the NTP server, updates its estimates of time counter > rate/offset, > >> and then sleeps until the next poll interval) but also enable other > threads in > >> the unikernel to get the current value of a variable in its context? > > The CLOCK signature uses > > val time: unit -> float > > rather than > > val time: t -> float > > for some reason, which makes this difficult. I suggest changing the > signature to work like the other devices. > I agree, it would make an NTP-corrected clock using Kia's work much easier, and I think it will also simplify writing fake / test clocks. I worked on this a little bit and added a PCLOCK.t and DEVICE signature to PCLOCK see https://github.com/mattgray/mirage/commit/5b83035c1dcfc7e5aec8e7d9fc46f96f8e57ffc7 https://github.com/mattgray/mirage-clock/commit/b9b225e64b6ff7e9a714d13af65803ea6845a055 It compiles and does the right thing (ie. nothing in connect and disconnect) but I'm pretty confused around V1_LWT and implementing the DEVICE signature correctly, so some review / help there would be appreciated. Should MCLOCK also implement the DEVICE signature? I think so, for consistency. > > You could use a global variable to hold the current time, but that's > quite messy. > > Starting a background thread is easy enough, but stopping it is > trickier. Perhaps there should be `listen` and `disconnect` functions > on CLOCK too. For the current clocks these aren't needed, but for an > NTP clock it would be useful. > > > There is atm no such way AFAIK. In the mirage tool there is hardcoded > > some code to start entropy harvesting, and there is the mirage-logs > > which is started "magically" (but wasn't the mirage-logs integration > > done in a more general way)? > > Logging had to be magical because: > > 1. We need to start it before everything else, except things it depends on. > 2. Things use logging without explicitly depending on it. > 3. We wanted logging to be enabled by default even in unikernels that > don't explicitly request logging. > > I don't think these apply to time. Components that want time should > ask for it explicitly. > > >> Failing that, what way can I have a background thread that continually > runs and > >> always makes available the latest calculated value of "output" to other > threads > >> (so they can compute the current time based on the current RDTSC value)? > >> > >> Morally, my NTP client thread needs to let other threads have access to > its > >> latest rate/offset information so they can calculate the current time > (much how > >> Xen makes available vcpu_time_info in a shared page), but I do not know > how > >> to set this up in mirage. > > > > At the moment, mirage-clock-unix/xen provide the CLOCK (soon PCLOCK) > > interface. The NTP client should implement the same interface, and as a > > user I'd like to use this and pass it on to other CLOCK consumers. > > > > This reminds me of the hardcoded tcpip.arpv4 stuff inside the mirage > > tool. Maybe we should re-think the code stubs in there now that we have > > multiple implementation of core interfaces? Maybe stackv4 etc. should > > receive optional parameters for arp/clock/... (a brief look into > > lib/mirage.ml shows that they already do!?! (maybe I just have to try > > this out))? > > > -- > talex5 (GitHub/Twitter) http://roscidus.com/blog/ > GPG: 5DD5 8D70 899C 454A 966D 6A51 7513 3C8F 94F6 E0CC > GPG: DA98 25AE CAD0 8975 7CDA BD8E 0713 3F96 CA74 D8BA > > _______________________________________________ > MirageOS-devel mailing list > [email protected] > https://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel >
_______________________________________________ MirageOS-devel mailing list [email protected] https://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel
