[riot-devel] ztimer - a new high-level timer for RIOT
Hey everyone, since the RIOT Summit in Helsinki, I've put quite some work into ztimer, a possible successor to xtimer. If you're interested, please see an updated design document here: [1] Cheers, Kaspar [1] https://github.com/RIOT-OS/RIOT/wiki/ztimer-problem-statement-and-design-document ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hi Kaspar, why are 8-bit timers not listed? Intentional or unintentional? Regards, Robert On 09.12.19 14:49, Kaspar Schleiser wrote: > Hey everyone, > > since the RIOT Summit in Helsinki, I've put quite some work into ztimer, > a possible successor to xtimer. > > If you're interested, please see an updated design document here: [1] > > Cheers, > Kaspar > > [1] > https://github.com/RIOT-OS/RIOT/wiki/ztimer-problem-statement-and-design-document > ___ > devel mailing list > devel@riot-os.org > https://lists.riot-os.org/mailman/listinfo/devel > -- Robert Hartung, M.Sc. Technische Universität Braunschweig Institut für Betriebssysteme und Rechnerverbund Mühlenpfordtstr. 23, Raum 115 38106 Braunschweig Fon: +49 (531) 391 - 3246 Fax: +49 (531) 391 - 5936 E-Mail: hart...@ibr.cs.tu-bs.de WWW: https://www.ibr.cs.tu-bs.de/users/hartung/ ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hey again ;) Do we need to put any thoughts in power management / low_power / integration with pm_layered? Or are the possible issues addreses / already talked about? Regards Robert On 09.12.19 14:49, Kaspar Schleiser wrote: > Hey everyone, > > since the RIOT Summit in Helsinki, I've put quite some work into ztimer, > a possible successor to xtimer. > > If you're interested, please see an updated design document here: [1] > > Cheers, > Kaspar > > [1] > https://github.com/RIOT-OS/RIOT/wiki/ztimer-problem-statement-and-design-document > ___ > devel mailing list > devel@riot-os.org > https://lists.riot-os.org/mailman/listinfo/devel > -- Robert Hartung, M.Sc. Technische Universität Braunschweig Institut für Betriebssysteme und Rechnerverbund Mühlenpfordtstr. 23, Raum 115 38106 Braunschweig Fon: +49 (531) 391 - 3246 Fax: +49 (531) 391 - 5936 E-Mail: hart...@ibr.cs.tu-bs.de WWW: https://www.ibr.cs.tu-bs.de/users/hartung/ ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hi Robert, On 12/9/19 4:19 PM, Robert Hartung wrote: > why are 8-bit timers not listed? Intentional or unintentional? Unintentional! Kaspar ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hi Robert, On 12/9/19 4:25 PM, Robert Hartung wrote: > Do we need to put any thoughts in power management / low_power / > integration with pm_layered? Or are the possible issues addreses / > already talked about? Yes and yes. ;) I'll my thoughts so far. Thanks! Kaspar ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hi, On 12/9/19 4:52 PM, Kaspar Schleiser wrote: > Hi Robert, > > On 12/9/19 4:25 PM, Robert Hartung wrote: >> Do we need to put any thoughts in power management / low_power / >> integration with pm_layered? Or are the possible issues addreses / >> already talked about? > > Yes and yes. ;) > > I'll my thoughts so far. I've added this to the wiki page: # Power management considerations - currently, ztimer is pm_layered agnostic. If a timer is set on a periph_timer, this would probably not prevent sleep (timer would not trigger), whereas if a ztimer is set on a rtt, it would behave as expected (timer hardware keeps running in sleep, timer isr wakes up MCU). - (TODO) if a timeout has been set (e.g., `ztimer_set(clock, timeout)`), the backend device blocks sleeping if necessary. IMO this is the minimum requirement, but still needs to be implemented. - Idea: we specify that by convention, ZTIMER_MSEC (and ZTIMER_SEC) *keep running in sleep mode*, whereas ZTIMER_USEC stops when the MCU enters sleep (unless a timeout is scheduled). This is current behaviour *if* ZTIMER_USEC is using periph_timer as backend and ZTIMER_MSEC is using RTT/RTC. This would mean that `before = ztimer_now(clock); do something; diff = ztimer_now(clock) - before;` only works if either `do_something` does not schedule away the thread causing sleep *or* a clock is used that runs in sleep mode. - the behaviour could be accessible either through defines (ZTIMER_USEC_LPM_MODE or ZTIMER_USEC_DEEPSLEEP ...), *or* be made part of the ztimer API - in addition, we could add functions to explicitly tell the clocks to stay available until released, e.g., `ztimer_acquire(clock); before = ztimer_now(clock); do something; diff = ztimer_now(clock) - before; ztimer_release(clock);`. Once the "if timer is scheduled, don't sleep" is implemented, this could also be worked around by: `ztimer_set(clock, dummy, 0x); ...; ztimer_cancel(clock, dummy);` Feedback appreciated! Kaspar ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hi, if this is a "problem statement and design document", then concise and measurable requirements on power management should go into the corresponding section. Also, a clear and falsifiable problem statement should be given. This should IMO address the question, why timer problems cannot be fixed by simply repairing the xtimer (+ underlying HW abstractions). A long list of ztimer promises appears rather unessential and confusing. Thomas On 09/12/2019 17:45, Kaspar Schleiser wrote: Hi, On 12/9/19 4:52 PM, Kaspar Schleiser wrote: Hi Robert, On 12/9/19 4:25 PM, Robert Hartung wrote: Do we need to put any thoughts in power management / low_power / integration with pm_layered? Or are the possible issues addreses / already talked about? Yes and yes. ;) I'll my thoughts so far. I've added this to the wiki page: # Power management considerations - currently, ztimer is pm_layered agnostic. If a timer is set on a periph_timer, this would probably not prevent sleep (timer would not trigger), whereas if a ztimer is set on a rtt, it would behave as expected (timer hardware keeps running in sleep, timer isr wakes up MCU). - (TODO) if a timeout has been set (e.g., `ztimer_set(clock, timeout)`), the backend device blocks sleeping if necessary. IMO this is the minimum requirement, but still needs to be implemented. - Idea: we specify that by convention, ZTIMER_MSEC (and ZTIMER_SEC) *keep running in sleep mode*, whereas ZTIMER_USEC stops when the MCU enters sleep (unless a timeout is scheduled). This is current behaviour *if* ZTIMER_USEC is using periph_timer as backend and ZTIMER_MSEC is using RTT/RTC. This would mean that `before = ztimer_now(clock); do something; diff = ztimer_now(clock) - before;` only works if either `do_something` does not schedule away the thread causing sleep *or* a clock is used that runs in sleep mode. - the behaviour could be accessible either through defines (ZTIMER_USEC_LPM_MODE or ZTIMER_USEC_DEEPSLEEP ...), *or* be made part of the ztimer API - in addition, we could add functions to explicitly tell the clocks to stay available until released, e.g., `ztimer_acquire(clock); before = ztimer_now(clock); do something; diff = ztimer_now(clock) - before; ztimer_release(clock);`. Once the "if timer is scheduled, don't sleep" is implemented, this could also be worked around by: `ztimer_set(clock, dummy, 0x); ...; ztimer_cancel(clock, dummy);` Feedback appreciated! Kaspar ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel -- Prof. Dr. Thomas C. Schmidt ° Hamburg University of Applied Sciences Berliner Tor 7 ° ° Dept. Informatik, Internet Technologies Group 20099 Hamburg, Germany ° ° http://inet.haw-hamburg.de/members/schmidt Fon: +49-40-42875-8452 ° ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hi Thomas, > Also, a clear and falsifiable problem statement should be given. could you elaborate on what you mean by a problem statement being falsifiable? Do you want to be able to check that a given problem cannot be solved by existing features? > This should IMO address the question, why timer problems cannot be fixed by > simply repairing the xtimer (+ underlying HW abstractions). The mayor issue is that the API uses a fixed 1 µs resolution. As an uint32_t in 1 µs resolution would overflow after ~72 minutes, an uint64_t is needed as a direct consequence. This in turn results in the use of 64 bit arithmetic, which is very ill suited on IoT devices, especially on 8 bit and 16 bit platforms. Additionally, an API using 1µs resolution can be best implemented with fast timer hardware. But those usually prevent any power saving modes. This is very ill suited for the huge majority of IoT scenarios. Simply changing xtimer to use an RTT instead would solve the power saving issue. But RTTs usually operate at frequencies in the range of 1kHz - 32.678kHz. A base unit of 1µs makes very little sense in that case. And I'm aware of places in RIOT that depend on xtimer having a resolution in the order of microseconds; those would just break. All those issues are direct consequences of the use of a fixed 1 µs resolution. Allowing callers to specify the timer resolution would fix these. But that requires an API change. (All that reasoning is part of the wiki page already.) Kind regards, Marian On Mon, 9 Dec 2019 18:48:39 +0100 "Thomas C. Schmidt" wrote: > Hi, > > if this is a "problem statement and design document", then concise and > measurable requirements on power management should go into the > corresponding section. > > Also, a clear and falsifiable problem statement should be given. This > should IMO address the question, why timer problems cannot be fixed by > simply repairing the xtimer (+ underlying HW abstractions). > > A long list of ztimer promises appears rather unessential and confusing. > > Thomas > > On 09/12/2019 17:45, Kaspar Schleiser wrote: > > Hi, > > > > On 12/9/19 4:52 PM, Kaspar Schleiser wrote: > >> Hi Robert, > >> > >> On 12/9/19 4:25 PM, Robert Hartung wrote: > >>> Do we need to put any thoughts in power management / low_power / > >>> integration with pm_layered? Or are the possible issues addreses / > >>> already talked about? > >> > >> Yes and yes. ;) > >> > >> I'll my thoughts so far. > > > > I've added this to the wiki page: > > > > # Power management considerations > > - currently, ztimer is pm_layered agnostic. If a timer is set on a > > periph_timer, this would probably not prevent sleep (timer would not > > trigger), whereas if a ztimer is set on a rtt, it would behave as > > expected (timer hardware keeps running in sleep, timer isr wakes up MCU). > > > > - (TODO) if a timeout has been set (e.g., `ztimer_set(clock, timeout)`), > > the backend device blocks sleeping if necessary. IMO this is the minimum > > requirement, but still needs to be implemented. > > > > - Idea: we specify that by convention, ZTIMER_MSEC (and ZTIMER_SEC) > > *keep running in sleep mode*, whereas ZTIMER_USEC stops when the MCU > > enters sleep (unless a timeout is scheduled). This is current behaviour > > *if* ZTIMER_USEC is using periph_timer as backend and ZTIMER_MSEC is > > using RTT/RTC. > > > >This would mean that `before = ztimer_now(clock); do something; diff = > > ztimer_now(clock) - before;` only works if either `do_something` does > > not schedule away the thread causing sleep *or* a clock is used that > > runs in sleep mode. > > > > - the behaviour could be accessible either through defines > > (ZTIMER_USEC_LPM_MODE or ZTIMER_USEC_DEEPSLEEP ...), *or* be made part > > of the ztimer API > > > > - in addition, we could add functions to explicitly tell the clocks to > > stay available until released, e.g., `ztimer_acquire(clock); before = > > ztimer_now(clock); do something; diff = ztimer_now(clock) - before; > > ztimer_release(clock);`. > >Once the "if timer is scheduled, don't sleep" is implemented, this > > could also be worked around by: > > `ztimer_set(clock, dummy, 0x); ...; ztimer_cancel(clock, > > dummy);` > > > > > > Feedback appreciated! > > > > Kaspar > > ___ > > devel mailing list > > devel@riot-os.org > > https://lists.riot-os.org/mailman/listinfo/devel > > > pgp9YsMqnufSk.pgp Description: OpenPGP digital signature ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hi Marian,
On 09/12/2019 20:06, Marian Buschsieweke wrote:
Also, a clear and falsifiable problem statement should be given.
could you elaborate on what you mean by a problem statement being falsifiable?
"falsifiable" is a standard principle in science: It is sometimes
difficult to verify a statement, if application contexts cannot be
exhaustively enumerated ("It never rains in California"). Falsification
is often simpler, since only a counter-example is needed. A statement
that is neither verifiable nor falsifiable is useless for rigorous
argumentation.
"RIOT needs an easy to use, efficient and flexible timer system" is such
a poster child of a non-arguable statement and may move to the introduction.
Regarding xtimer:
If the current 1 us resolution in the API is the key problem, then this
should be stated clearly in the problem statement so that it can be
questioned and discussed.
Best,
Thomas
Do you want to be able to check that a given problem cannot be solved by
existing features?
This should IMO address the question, why timer problems cannot be fixed by
simply repairing the xtimer (+ underlying HW abstractions).
The mayor issue is that the API uses a fixed 1 µs resolution. As an uint32_t in
1 µs resolution would overflow after ~72 minutes, an uint64_t is needed as a
direct consequence. This in turn results in the use of 64 bit arithmetic, which
is very ill suited on IoT devices, especially on 8 bit and 16 bit platforms.
Additionally, an API using 1µs resolution can be best implemented with fast
timer hardware. But those usually prevent any power saving modes. This is very
ill suited for the huge majority of IoT scenarios.
Simply changing xtimer to use an RTT instead would solve the power saving
issue. But RTTs usually operate at frequencies in the range of 1kHz -
32.678kHz. A base unit of 1µs makes very little sense in that case. And I'm
aware of places in RIOT that depend on xtimer having a resolution in the order
of microseconds; those would just break.
All those issues are direct consequences of the use of a fixed 1 µs resolution.
Allowing callers to specify the timer resolution would fix these. But that
requires an API change.
(All that reasoning is part of the wiki page already.)
Kind regards,
Marian
On Mon, 9 Dec 2019 18:48:39 +0100
"Thomas C. Schmidt" wrote:
Hi,
if this is a "problem statement and design document", then concise and
measurable requirements on power management should go into the
corresponding section.
Also, a clear and falsifiable problem statement should be given. This
should IMO address the question, why timer problems cannot be fixed by
simply repairing the xtimer (+ underlying HW abstractions).
A long list of ztimer promises appears rather unessential and confusing.
Thomas
On 09/12/2019 17:45, Kaspar Schleiser wrote:
Hi,
On 12/9/19 4:52 PM, Kaspar Schleiser wrote:
Hi Robert,
On 12/9/19 4:25 PM, Robert Hartung wrote:
Do we need to put any thoughts in power management / low_power /
integration with pm_layered? Or are the possible issues addreses /
already talked about?
Yes and yes. ;)
I'll my thoughts so far.
I've added this to the wiki page:
# Power management considerations
- currently, ztimer is pm_layered agnostic. If a timer is set on a
periph_timer, this would probably not prevent sleep (timer would not
trigger), whereas if a ztimer is set on a rtt, it would behave as
expected (timer hardware keeps running in sleep, timer isr wakes up MCU).
- (TODO) if a timeout has been set (e.g., `ztimer_set(clock, timeout)`),
the backend device blocks sleeping if necessary. IMO this is the minimum
requirement, but still needs to be implemented.
- Idea: we specify that by convention, ZTIMER_MSEC (and ZTIMER_SEC)
*keep running in sleep mode*, whereas ZTIMER_USEC stops when the MCU
enters sleep (unless a timeout is scheduled). This is current behaviour
*if* ZTIMER_USEC is using periph_timer as backend and ZTIMER_MSEC is
using RTT/RTC.
This would mean that `before = ztimer_now(clock); do something; diff =
ztimer_now(clock) - before;` only works if either `do_something` does
not schedule away the thread causing sleep *or* a clock is used that
runs in sleep mode.
- the behaviour could be accessible either through defines
(ZTIMER_USEC_LPM_MODE or ZTIMER_USEC_DEEPSLEEP ...), *or* be made part
of the ztimer API
- in addition, we could add functions to explicitly tell the clocks to
stay available until released, e.g., `ztimer_acquire(clock); before =
ztimer_now(clock); do something; diff = ztimer_now(clock) - before;
ztimer_release(clock);`.
Once the "if timer is scheduled, don't sleep" is implemented, this
could also be worked around by:
`ztimer_set(clock, dummy, 0x); ...; ztimer_cancel(clock,
dummy);`
Feedback appreciated!
Kaspar
___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel
--
Prof. Dr. Thomas C. Schmidt
° Hamburg
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Dear Thomas,
I'd like to point out that the research community has largely dismissed Karl
Poppers contribution to the demarcation problem, as largely accepted fields of
research are not falsifiable. E.g. the evolution theory cannot be falsified.
Maybe it is time for you to move on as well?
Kind regards,
Marian
On Mon, 9 Dec 2019 20:35:24 +0100
"Thomas C. Schmidt" wrote:
> Hi Marian,
>
> On 09/12/2019 20:06, Marian Buschsieweke wrote:
>
> >> Also, a clear and falsifiable problem statement should be given.
> >
> > could you elaborate on what you mean by a problem statement being
> > falsifiable?
>
> "falsifiable" is a standard principle in science: It is sometimes
> difficult to verify a statement, if application contexts cannot be
> exhaustively enumerated ("It never rains in California"). Falsification
> is often simpler, since only a counter-example is needed. A statement
> that is neither verifiable nor falsifiable is useless for rigorous
> argumentation.
> "RIOT needs an easy to use, efficient and flexible timer system" is such
> a poster child of a non-arguable statement and may move to the introduction.
>
> Regarding xtimer:
>
> If the current 1 us resolution in the API is the key problem, then this
> should be stated clearly in the problem statement so that it can be
> questioned and discussed.
>
> Best,
> Thomas
>
> > Do you want to be able to check that a given problem cannot be solved by
> > existing features?
> >
> >> This should IMO address the question, why timer problems cannot be fixed by
> >> simply repairing the xtimer (+ underlying HW abstractions).
> >
> > The mayor issue is that the API uses a fixed 1 µs resolution. As an
> > uint32_t in
> > 1 µs resolution would overflow after ~72 minutes, an uint64_t is needed as a
> > direct consequence. This in turn results in the use of 64 bit arithmetic,
> > which
> > is very ill suited on IoT devices, especially on 8 bit and 16 bit platforms.
> >
> > Additionally, an API using 1µs resolution can be best implemented with fast
> > timer hardware. But those usually prevent any power saving modes. This is
> > very
> > ill suited for the huge majority of IoT scenarios.
> >
> > Simply changing xtimer to use an RTT instead would solve the power saving
> > issue. But RTTs usually operate at frequencies in the range of 1kHz -
> > 32.678kHz. A base unit of 1µs makes very little sense in that case. And I'm
> > aware of places in RIOT that depend on xtimer having a resolution in the
> > order
> > of microseconds; those would just break.
> >
> > All those issues are direct consequences of the use of a fixed 1 µs
> > resolution.
> > Allowing callers to specify the timer resolution would fix these. But that
> > requires an API change.
> >
> > (All that reasoning is part of the wiki page already.)
> >
> > Kind regards,
> > Marian
> >
> > On Mon, 9 Dec 2019 18:48:39 +0100
> > "Thomas C. Schmidt" wrote:
> >
> >> Hi,
> >>
> >> if this is a "problem statement and design document", then concise and
> >> measurable requirements on power management should go into the
> >> corresponding section.
> >>
> >> Also, a clear and falsifiable problem statement should be given. This
> >> should IMO address the question, why timer problems cannot be fixed by
> >> simply repairing the xtimer (+ underlying HW abstractions).
> >>
> >> A long list of ztimer promises appears rather unessential and confusing.
> >>
> >> Thomas
> >>
> >> On 09/12/2019 17:45, Kaspar Schleiser wrote:
> >>> Hi,
> >>>
> >>> On 12/9/19 4:52 PM, Kaspar Schleiser wrote:
> Hi Robert,
>
> On 12/9/19 4:25 PM, Robert Hartung wrote:
> > Do we need to put any thoughts in power management / low_power /
> > integration with pm_layered? Or are the possible issues addreses /
> > already talked about?
>
> Yes and yes. ;)
>
> I'll my thoughts so far.
> >>>
> >>> I've added this to the wiki page:
> >>>
> >>> # Power management considerations
> >>> - currently, ztimer is pm_layered agnostic. If a timer is set on a
> >>> periph_timer, this would probably not prevent sleep (timer would not
> >>> trigger), whereas if a ztimer is set on a rtt, it would behave as
> >>> expected (timer hardware keeps running in sleep, timer isr wakes up MCU).
> >>>
> >>> - (TODO) if a timeout has been set (e.g., `ztimer_set(clock, timeout)`),
> >>> the backend device blocks sleeping if necessary. IMO this is the minimum
> >>> requirement, but still needs to be implemented.
> >>>
> >>> - Idea: we specify that by convention, ZTIMER_MSEC (and ZTIMER_SEC)
> >>> *keep running in sleep mode*, whereas ZTIMER_USEC stops when the MCU
> >>> enters sleep (unless a timeout is scheduled). This is current behaviour
> >>> *if* ZTIMER_USEC is using periph_timer as backend and ZTIMER_MSEC is
> >>> using RTT/RTC.
> >>>
> >>> This would mean that `before = ztimer_now(clock); do something; diff =
> >>> ztimer_now(clock) - before;` only works if either `do_som
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hi Marian,
On 09/12/2019 21:00, Marian Buschsieweke wrote:
I'd like to point out that the research community has largely dismissed Karl
Poppers contribution to the demarcation problem, as largely accepted fields of
research are not falsifiable. E.g. the evolution theory cannot be falsified.
https://rationalwiki.org/wiki/Falsifiability_of_evolution ?
Maybe it is time for you to move on as well?
Well, if we give up rational argumentation, then we end up in blind
guesswork or insignificant conversation.
Statements such as "I really like my timer drift today!" are amusing,
but not helpful.
Cheers,
Thomas
On Mon, 9 Dec 2019 20:35:24 +0100
"Thomas C. Schmidt" wrote:
Hi Marian,
On 09/12/2019 20:06, Marian Buschsieweke wrote:
Also, a clear and falsifiable problem statement should be given.
could you elaborate on what you mean by a problem statement being falsifiable?
"falsifiable" is a standard principle in science: It is sometimes
difficult to verify a statement, if application contexts cannot be
exhaustively enumerated ("It never rains in California"). Falsification
is often simpler, since only a counter-example is needed. A statement
that is neither verifiable nor falsifiable is useless for rigorous
argumentation.
"RIOT needs an easy to use, efficient and flexible timer system" is such
a poster child of a non-arguable statement and may move to the introduction.
Regarding xtimer:
If the current 1 us resolution in the API is the key problem, then this
should be stated clearly in the problem statement so that it can be
questioned and discussed.
Best,
Thomas
Do you want to be able to check that a given problem cannot be solved by
existing features?
This should IMO address the question, why timer problems cannot be fixed by
simply repairing the xtimer (+ underlying HW abstractions).
The mayor issue is that the API uses a fixed 1 µs resolution. As an uint32_t in
1 µs resolution would overflow after ~72 minutes, an uint64_t is needed as a
direct consequence. This in turn results in the use of 64 bit arithmetic, which
is very ill suited on IoT devices, especially on 8 bit and 16 bit platforms.
Additionally, an API using 1µs resolution can be best implemented with fast
timer hardware. But those usually prevent any power saving modes. This is very
ill suited for the huge majority of IoT scenarios.
Simply changing xtimer to use an RTT instead would solve the power saving
issue. But RTTs usually operate at frequencies in the range of 1kHz -
32.678kHz. A base unit of 1µs makes very little sense in that case. And I'm
aware of places in RIOT that depend on xtimer having a resolution in the order
of microseconds; those would just break.
All those issues are direct consequences of the use of a fixed 1 µs resolution.
Allowing callers to specify the timer resolution would fix these. But that
requires an API change.
(All that reasoning is part of the wiki page already.)
Kind regards,
Marian
On Mon, 9 Dec 2019 18:48:39 +0100
"Thomas C. Schmidt" wrote:
Hi,
if this is a "problem statement and design document", then concise and
measurable requirements on power management should go into the
corresponding section.
Also, a clear and falsifiable problem statement should be given. This
should IMO address the question, why timer problems cannot be fixed by
simply repairing the xtimer (+ underlying HW abstractions).
A long list of ztimer promises appears rather unessential and confusing.
Thomas
On 09/12/2019 17:45, Kaspar Schleiser wrote:
Hi,
On 12/9/19 4:52 PM, Kaspar Schleiser wrote:
Hi Robert,
On 12/9/19 4:25 PM, Robert Hartung wrote:
Do we need to put any thoughts in power management / low_power /
integration with pm_layered? Or are the possible issues addreses /
already talked about?
Yes and yes. ;)
I'll my thoughts so far.
I've added this to the wiki page:
# Power management considerations
- currently, ztimer is pm_layered agnostic. If a timer is set on a
periph_timer, this would probably not prevent sleep (timer would not
trigger), whereas if a ztimer is set on a rtt, it would behave as
expected (timer hardware keeps running in sleep, timer isr wakes up MCU).
- (TODO) if a timeout has been set (e.g., `ztimer_set(clock, timeout)`),
the backend device blocks sleeping if necessary. IMO this is the minimum
requirement, but still needs to be implemented.
- Idea: we specify that by convention, ZTIMER_MSEC (and ZTIMER_SEC)
*keep running in sleep mode*, whereas ZTIMER_USEC stops when the MCU
enters sleep (unless a timeout is scheduled). This is current behaviour
*if* ZTIMER_USEC is using periph_timer as backend and ZTIMER_MSEC is
using RTT/RTC.
This would mean that `before = ztimer_now(clock); do something; diff =
ztimer_now(clock) - before;` only works if either `do_something` does
not schedule away the thread causing sleep *or* a clock is used that
runs in sleep mode.
- the behaviour could be accessible either through defines
(ZTIMER_USEC_LPM_MODE or ZTIMER_USEC
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Folks! Can we get back to the actual problem at hand, please? Let me recap: Kaspar came up with a proposal for a new timer API, since xtimer has flaws (as identified by multiple members of the RIOT community during the last ~4 years) and is apparently not fixable (at least no one came up with a concrete proposal to fix it to the best of my knowledge). Having a concrete implementation proposal is great. Thanks for the effort of designing and implementing the current state of ztimer and thanks for the documentation of this effort! I think the problem statement and the requirements could indeed be more precise - while I must admit that a lack of precise requirements is a failure of the RIOT community. Anyway, I think we need to define what "very efficient timers for use in time-critical drivers" means in order to being able to check whether the proposal fulfills the requirement or not. Besides I'm missing a requirement regarding the maximum granularity and the maximum duration of a timer. Cheers Oleg -- The good thing about object oriented jokes is they bring their own laughter method. ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hi Oleg et all, On 12/9/19 9:25 PM, Oleg Hahm wrote: > I think the problem statement and the requirements could indeed be more > precise - while I must admit that a lack of precise requirements is a failure > of the RIOT community. Yes, that could be. I intentionally did not add requirements. I just added a clarification on "adaptable to varying configurations of timers, RTTs, RTCs (use RTC if available for super-long-time timers)", trying to convey later on that xtimer just doesn't fulfill this requirement, and "fixing" that involves changing most of it. > Anyway, I think we need to define what "very efficient timers for use in > time-critical drivers" means in order to being able to check whether the > proposal fulfills the requirement or not. We can try. What would that look like? Something like "must not incur more than x us overhead on hardware of class y"? > Besides I'm missing a requirement regarding the maximum granularity and the > maximum duration of a timer. You mean minimum granularity? Anyway, good point. xtimer has 64bit range with 1us precision. ztimer makes the trade-off of only offering 32bit range, but with flexible precision. I'm not sure we can get away with that, if just for the fact that we have code using the 64bit functions, which means automatic code conversion using coccinelle is not (easily) possible. I think I'll just implement a 64bit version (as an extra module) so a possible transition gets easier. And to checkbox "any precision possible (if hardware keeps up), 64bit range supported". Kaspar ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hey! On Mon, Dec 09, 2019 at 10:07:16PM +0100, Kaspar Schleiser wrote: > > Anyway, I think we need to define what "very efficient timers for use in > > time-critical drivers" means in order to being able to check whether the > > proposal fulfills the requirement or not. > > We can try. What would that look like? > Something like "must not incur more than x us overhead on hardware of > class y"? Hm, to be honest, I'm not so sure of what kind of efficiency we're speaking here. CPU time or memory? Probably both, right? Regarding the CPU efficiency, I would assume that this also dictates the maximum precision, right? Regarding memory we have probably different requirements: ROM for the whole thing, RAM per instance and for the module itself. > > Besides I'm missing a requirement regarding the maximum granularity and the > > maximum duration of a timer. > > You mean minimum granularity? In my understanding of the term maximum granularity translates to the finest granularity. Cheers Oleg -- T he bes thin gabou tTCPfl owcontr oljokesi sthatthey knowwhento backo ff ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hey, On 12/9/19 10:32 PM, Oleg Hahm wrote: > Hm, to be honest, I'm not so sure of what kind of efficiency we're speaking > here. CPU time or memory? Probably both, right? Regarding the CPU efficiency, > I would assume that this also dictates the maximum precision, right? I don't think so. The hardware dictates the maximum precision. I wrote a test (its in tests/ztimer_overhead in the ztimer PR), that sets a timer, which in its callback gets the current time, and then calculates the differenve. Uncorrected (with XTIMER_OVERHEAD=0 or the ztimer equivalent), both add a pretty constant ~7us, on an nrf52dk using a periph timer clocked at 1MHz. There's not much happening in this simplest case. I don't think we can carve off much in any high level timer implementation, compared to using periph_timer or even skipping that and using plain register writes. Callback based (high-level) timers are only suitable for timeouts down to a certain level (somewhere below 10us on our hardware scope). Below that, context switching takes the bulk of the time, so spinning (not using a callback) is probably preferable. I mean, if a device can do 100k context switches per second, each takes 10us. Setting a timer to anything below that might work, but doesn't make much sense. Even ztimer using only 32bit arithmetic vs. xtimer sometimes needing 64bit comparisons might show measurable differences when there are many timers active, but I honestly don't know if this difference can become a deal breaker, or even just relevant. Cycle wise, xtimer is not terrible to begin with. That said, if we get an alternative that matches or improves on most metrics and is still more flexible, and allows proper sleeping, then I take any percent improvement with a happy face. > Regarding > memory we have probably different requirements: ROM for the whole thing, RAM > per instance and for the module itself. > Here, the requirement should be "needs to fulfill all other requirements", and from that point on, less is better. xtimer and ztimer are comparable in ROM and RAM usage. Again, I don't think that any high level timer implementation can bring disruptive reduction here. I'm not looking forward to coming up with requirements for accuracy. Simple tests (like the overhead test from above) are easy to optimize so a timer hits bang on the target time, by subtracting a measured overhead from any interval. Figuring out the impact of having N timers in the list is difficult, and more difficult to correct. Here we could come up with a test setting as many timers as fit in RAM to random values, then measuring each timer's punctuality. All making sure that the target times don't overlap. But all the requirements don't help over the fact that xtimer currently just doesn't work for many identified use cases. Fixing that would require substantial code changes, probably including API changes. Knowning the complexity of xtimer, I decided to give a rewrite from scratch a try, and IMO, the result is quite nice. I'd maybe roll up the discussion from a different side: If noone can identify a conceptual flaw, something where a different implementation or design might yield benefits compared to ztimer, and if ztimer does not show regressions compared to xtimer, but improves on some relevant aspects, why don't we consider going with better in the short term and pursue provably best as a longer term goal? >>> Besides I'm missing a requirement regarding the maximum granularity and the >>> maximum duration of a timer. >> >> You mean minimum granularity? > > In my understanding of the term maximum granularity translates to the finest > granularity. Yup, got it. Kaspar ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hi, I was just literally about to send an email with pretty much the same arguments Kaspar wrote right now. So I skip them and throw in a +1 instead. > Below that, context switching takes the bulk of the time, so spinning (not > using a callback) is probably preferable. I think that the ws281x driver is currently one of the most timing-critical driver in RIOT. And in the worst case scenario it currently is useable, it needs a delay of exactly 2-3 CPU cycles. On the same hardware, it takes longer from an interrupt request coming in to the ISR being run than those 2-3 CPU cycles. To me, it is safe to say that there are simply use cases that just cannot be addressed with any high level timer API. So in my opinion focusing on the use cases that can reasonably be addressed with a high level API makes most sense. And based on this, follow up PRs can try to squeeze as much efficiency out of the implementation as possible without sacrificing something on the other metrics (e.g. ROM/RAM consumption, maintainability, flexibility, etc.). Kind regards, Marian On Mon, 9 Dec 2019 23:28:19 +0100 Kaspar Schleiser wrote: > Hey, > > On 12/9/19 10:32 PM, Oleg Hahm wrote: > > Hm, to be honest, I'm not so sure of what kind of efficiency we're speaking > > here. CPU time or memory? Probably both, right? Regarding the CPU > > efficiency, I would assume that this also dictates the maximum precision, > > right? > > I don't think so. The hardware dictates the maximum precision. > > I wrote a test (its in tests/ztimer_overhead in the ztimer PR), that > sets a timer, which in its callback gets the current time, and then > calculates the differenve. > > Uncorrected (with XTIMER_OVERHEAD=0 or the ztimer equivalent), both add > a pretty constant ~7us, on an nrf52dk using a periph timer clocked at > 1MHz. There's not much happening in this simplest case. I don't think we > can carve off much in any high level timer implementation, compared to > using periph_timer or even skipping that and using plain register writes. > > Callback based (high-level) timers are only suitable for timeouts down > to a certain level (somewhere below 10us on our hardware scope). Below > that, context switching takes the bulk of the time, so spinning (not > using a callback) is probably preferable. > I mean, if a device can do 100k context switches per second, each takes > 10us. Setting a timer to anything below that might work, but doesn't > make much sense. > > Even ztimer using only 32bit arithmetic vs. xtimer sometimes needing > 64bit comparisons might show measurable differences when there are many > timers active, but I honestly don't know if this difference can become a > deal breaker, or even just relevant. > > Cycle wise, xtimer is not terrible to begin with. > > That said, if we get an alternative that matches or improves on most > metrics and is still more flexible, and allows proper sleeping, then I > take any percent improvement with a happy face. > > > Regarding > > memory we have probably different requirements: ROM for the whole thing, RAM > > per instance and for the module itself. > > > > Here, the requirement should be "needs to fulfill all other > requirements", and from that point on, less is better. > xtimer and ztimer are comparable in ROM and RAM usage. Again, I don't > think that any high level timer implementation can bring disruptive > reduction here. > > I'm not looking forward to coming up with requirements for accuracy. > Simple tests (like the overhead test from above) are easy to optimize so > a timer hits bang on the target time, by subtracting a measured overhead > from any interval. Figuring out the impact of having N timers in the > list is difficult, and more difficult to correct. > > Here we could come up with a test setting as many timers as fit in RAM > to random values, then measuring each timer's punctuality. All making > sure that the target times don't overlap. > > But all the requirements don't help over the fact that xtimer currently > just doesn't work for many identified use cases. Fixing that would > require substantial code changes, probably including API changes. > Knowning the complexity of xtimer, I decided to give a rewrite from > scratch a try, and IMO, the result is quite nice. > > I'd maybe roll up the discussion from a different side: If noone can > identify a conceptual flaw, something where a different implementation > or design might yield benefits compared to ztimer, and if ztimer does > not show regressions compared to xtimer, but improves on some relevant > aspects, why don't we consider going with better in the short term and > pursue provably best as a longer term goal? > > >>> Besides I'm missing a requirement regarding the maximum granularity and > >>> the maximum duration of a timer. > >> > >> You mean minimum granularity? > > > > In my understanding of the term maximum granularity translates to the finest > > granularity. > > Yup, got it. > > Kaspar >
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hey Kaspar,
On Mon, Dec 09, 2019 at 11:28:19PM +0100, Kaspar Schleiser wrote:
> > Hm, to be honest, I'm not so sure of what kind of efficiency we're speaking
> > here. CPU time or memory? Probably both, right? Regarding the CPU
> > efficiency,
> > I would assume that this also dictates the maximum precision, right?
>
> I don't think so. The hardware dictates the maximum precision.
my thinking here was: when I set the timer to fire in 1 time units, but the
function to set a timer takes 2 time units, then the maximum precision is 2
time units.
> > Regarding memory we have probably different requirements: ROM for the
> > whole thing, RAM per instance and for the module itself.
>
> Here, the requirement should be "needs to fulfill all other
> requirements", and from that point on, less is better.
What do you mean by "all other requirements"?
> But all the requirements don't help over the fact that xtimer currently
> just doesn't work for many identified use cases.
> require substantial code changes, probably including API changes.
> Knowning the complexity of xtimer, I decided to give a rewrite from
> scratch a try, and IMO, the result is quite nice.
I got this. But since we are going for a clean re-design we should take the
opportunity and get the requirements straight first.
But to get this straight: I don't see the need that you must collect all the
requirements yourself.
Cheers
Oleg
--
panic("Tell me what a watchpoint trap is, and I'll then
deal with such a beast...");
linux-2.2.16/arch/arch/sparc/kernel/traps.c
___
devel mailing list
devel@riot-os.org
https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hi, Thanks for starting this! It's very much appreciated. Discussing these things, reaching common ground and documenting decisions and findings during this process is IMO one of the most important things to do before we move on. I'm really sorry to write this wall of text, but there are so many things to this timer topic where we IMO lack a facts driven analysis, decision- and development-process. This is combined with a lack of documentation of decisions and their implications. TL;DR: let's try to decide what is good for our high level timer based on measurable facts. For that we need detailed knowledge on the hardware we target, the software that will use the high level timer, and measurements/benchmarks that tell us what are the ups an downs of different ways of implementation. Side note: I think we should move the discussion to an RDM PR for the design document and discuss there to not get lost. In the much too long version below I comment mostly on the design document, other quotes as indicated. RIOT needs an easy to use, efficient and flexible timer system that allows precise high-frequency (microsecond scale) timings *alongside* low-power timers. For example, an application might include a driver requiring high-frequency (microsecond scale) timings, but at the same time needs to run on batteries and thus needs to make use of low-power timers that can wake up the device from deep sleep. I fully agree that this currently is a problem and it needs to be resolved. But what this statement essentially points out, is that the xtimer API either misses an instance parameter to be used on different low level timers or that it misses the functionality to internally handle multiple low level timers to multiplex timeouts to the (different) available hardware instances to fulfill low-power and high precision sleep requirements. The problem statement implies nothing related to the rest of the high level timer API and design at all. Thus, it is not a problem statement that shows ztimer is our only option. *efficient* -in which dimensions? Do we favor small RAM, ROM, CPU overhead? -how do we prefer this to scale if more or less timers are used? -does it make sense to decide for only one of these things at all? (one implementation may not suit everyone) -just thinking: would it hurt to have one memory efficient implementation and one that is "bigger, but faster"? *flexible* -in level of manual control it provides? -in level of automatic functionality and abstraction it provides? *precision (& accuracy)* -the bounds are defined by the hardware. -the question is how do we come as close as possible to the capabilities the HW provides. -where do we need trade-offs? *low-power* -dependencies are mostly defined by the hardware -How do we model these dependencies? -for our power management we try to do things implicit. -how does this work together with a timer that needs to be called with explicit instances? -why should the timer API differ from that principle? The following general questions pop up: -What does the hardware provide that is abstracted by this high level timer? -How much abstraction do we need? -Where do we need trade-offs? -How do we balance them? -Based on what information? Please don't get me wrong, I'm not in principle for or against xtimer, ztimer (or even wtimer, the whatever timer;) Yes, 64 bit time intuitively doesn't sound like a perfect fit for the IoT, but how does this translate to numbers (also different scenarios)? Yes, xtimer is broken. Yes, it needs fixing or replacement. *But: the functional problems related to xtimer are not related to it's API, it is the implementation!* General requirements: - very efficient timers for use in time-critical drivers This statement touches memory, runtime overhead and precision, but isn't precise on how to balance between them. easy-to-use interface (unified interface) Very much a matter of taste, but also what is the importance of "unified"? If there are two completely distinct use-cases (requirements) why enforce unified API for that? work with varying MCU timer widths (16, 24, 32-bit timers) Agree, an absolute must! (though I'd rather specify it more as "any width") - adaptable to varying configurations of timers, RTTs, RTCs (use RTC if available for super-long-time timers) - this means that applications and / or system modules need to be able to set timers on different timer hardware (configurations) *simultaneously* To me this feels like it is asking for an API below the high level timer that fits various kind of hardware (?). We currently miss such an API. Though, an extended periph_timer could be used for most (all?) of it. API (necessary functionality): - Wait for a time period (e.g. timer_usleep(howlong)) - receive message after given period of time (e.g. timer_msg(howlong)) - receive message periodically - Await a point in time - Wait for a (past) time
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hi Michel, thanks for all that input. It is *a lot*. I guess it is a complex subject.. Would it make sense to make a micro conference? Get everyone interested in improving timers in a room and lock it until solutions are presented? On 12/10/19 6:23 PM, Michel Rottleuthner wrote: >> RIOT needs an easy to use, efficient and flexible timer system that >> allows precise high-frequency (microsecond scale) timings *alongside* >> low-power timers. >> >> For example, an application might include a driver requiring >> high-frequency (microsecond scale) timings, but at the same time needs >> to run on batteries and thus needs to make use of low-power timers >> that can wake up the device from deep sleep. >> > > I fully agree that this currently is a problem and it needs to be resolved. > But what this statement essentially points out, is that the xtimer API > either misses > an instance parameter to be used on different low level timers or that it > misses the functionality to internally handle multiple low level timers > to multiplex > timeouts to the (different) available hardware instances to fulfill > low-power and high precision sleep requirements. Agreed. Adding the instance parameter is not trivial. The implementation needs to provide the underlying functionality. xtimer currently doesn't allow that. Much of it needs to be rewritten. Handling this internally is IMO not feasible for reasons I will point out later. > The problem statement implies nothing related to the rest of the high > level timer API and design at all. > Thus, it is not a problem statement that shows ztimer is our only option. > > *efficient* > -in which dimensions? Do we favor small RAM, ROM, CPU overhead? > -how do we prefer this to scale if more or less timers are used? > -does it make sense to decide for only one of these things at all? (one > implementation may not suit everyone) > -just thinking: would it hurt to have one memory efficient > implementation and one that is "bigger, but faster"? > > *flexible* > -in level of manual control it provides? > -in level of automatic functionality and abstraction it provides? > > *precision (& accuracy)* > -the bounds are defined by the hardware. > -the question is how do we come as close as possible to the capabilities > the HW provides. > -where do we need trade-offs? > > *low-power* > -dependencies are mostly defined by the hardware > -How do we model these dependencies? > -for our power management we try to do things implicit. > -how does this work together with a timer that needs to be called > with explicit instances? > -why should the timer API differ from that principle? > > > The following general questions pop up: > -What does the hardware provide that is abstracted by this high level > timer? > -How much abstraction do we need? > -Where do we need trade-offs? > -How do we balance them? > -Based on what information? > > Please don't get me wrong, I'm not in principle for or against xtimer, > ztimer (or even wtimer, the whatever timer;) > Yes, 64 bit time intuitively doesn't sound like a perfect fit for the > IoT, but how does this translate to numbers (also different scenarios)? > Yes, xtimer is broken. Yes, it needs fixing or replacement. > *But: the functional problems related to xtimer are not related to it's > API, it is the implementation!* > > >> General requirements: >> - very efficient timers for use in time-critical drivers > > This statement touches memory, runtime overhead and precision, but isn't > precise on how to balance between them. These are all very valid questions. IMO, coming up with definite answers is quite difficult, unless we move towards defining bounds. (Assuming xtimer would be stable / reliable), performance wise, it is *in the acceptable range* on RAM, ROM, cycle use. We have not yet seen an application that was in any way limited by it, apart from someone trying to use it for bit banging with the sub-10-us timings, which as I described in another mail is (probably) just not doable with a callback based high level timer. With "acceptable range" I mean that any implementation that is in the same ballpark (+- maybe 20%) would be acceptable (if it provides the necessary functionality). The difference would just don't matter in practice. Smaller and faster aabsolutely becomes a *nice to have* compared to the *musts* of reliability, accuracy, low-power friendlyness and the general possibility to be usable for a high percentage of our applications. For (RAM, ROM, CPU) performance, I'd not ditch xtimer. It performs alright (if it works). Maybe we can agree that xtimer's performance tradeoffs so far have not shown to be wrong. >> easy-to-use interface (unified interface) > > Very much a matter of taste, but also what is the importance of "unified"? > If there are two completely distinct use-cases (requirements) why > enforce unified API for that? With "unified" I actually mean something that xtimer already does: no matter the tim
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hi Kaspar,
thanks a lot for reading thru that and for the reply!
Would it make sense to make a micro conference? Get everyone interested
in improving timers in a room and lock it until solutions are presented?
Not convinced about the "lock in a room" ;) - but otherwise: absolutely yes!
What do you think about an RDM PR?
We could just use your design document as a starting point.
RIOT needs an easy to use, efficient and flexible timer system that
allows precise high-frequency (microsecond scale) timings *alongside*
low-power timers.
For example, an application might include a driver requiring
high-frequency (microsecond scale) timings, but at the same time needs
to run on batteries and thus needs to make use of low-power timers
that can wake up the device from deep sleep.
I fully agree that this currently is a problem and it needs to be resolved.
But what this statement essentially points out, is that the xtimer API
either misses
an instance parameter to be used on different low level timers or that it
misses the functionality to internally handle multiple low level timers
to multiplex
timeouts to the (different) available hardware instances to fulfill
low-power and high precision sleep requirements.
Agreed.
Adding the instance parameter is not trivial. The implementation needs
to provide the underlying functionality. xtimer currently doesn't allow
that. Much of it needs to be rewritten.
Handling this internally is IMO not feasible for reasons I will point
out later.
I disagree on the last sentence, but maybe we are just not talking about
the same thing.
These are all very valid questions.
IMO, coming up with definite answers is quite difficult, unless we move
towards defining bounds.
Yes. To at least move in that direction, I think we should try to work
towards
a clear description of the problem space including, but not limited to:
-Hardware capabilities
-Requirements
-Use-Cases
-Quality metrics
-Benchmarks for these metrics
(...)
With this in place we can try to flesh out the key findings, design
decisions and
implications of the timer design.
This will help us a lot once another re-design is considered.
Basically, I'd like to transfer what you call "experience from
iterating" into a document
that manifests this experience in a form that our community can build upon.
Maybe we can agree that xtimer's performance tradeoffs so far have not
shown to be wrong.
Agree.
periph_timer IMO should be the slimmest layer of hardware abstraction
that makes sense, so users that don't want to do direct non-portable
register based applications get the next "closest to the metal".
Agree, but there are some things that we should add to the periph_timer.
E.g. adding support for dedicated overflow interrupts together with an
API to read
the corresponding IRQ status bit.
The high level timer would benefit from that on many platforms.
E.g. Ztimer wouldn't require the code for the time partitioning
mechanism then.
But thats yet another part of the story...
We currently miss such an API. Though, an extended periph_timer could be
used for most (all?) of it.
What would that extension look like? Would it add a "clock" parameter so
it can deal with "varying configurations of timers, RTTs, RTCs"? Would
it do any kind of timer width extension? Would it add multiplexing?
Would it implement frequency conversion?
Why can't xtimer solve this? (I think ztimer does.)
That's one of the questions the future RDM should investigate and answer.
It would probably do mostly a slim extension in a way that fits the
platform and peripheral.
But no multiplexing or frequency conversion stuff thats what high level
timer is for, right?
Also the term "frequency conversion" is a bit misleading I think.
With a discrete clock you won't be able to just precisely convert a
frequency to any other frequency in software.
Especially if you want to increase the frequency - it will just be a
calculation.
As already pointed out regarding the problem statement:
This only points out that we either need to add an instance parameter to
the API (for explicit control)
or that xtimer needs to have access to multiple low level timers
internally (for implicit adaption).
As a stupid example, using a perfectly working ztimer and wrapping it in
the API of xtimer like that:
xtimer_xxx(uint64_t time) {
if (time < whatever_threshold) {
ztimer_xxx(HIGH_FREQ_INSTANCE, time);
} else{
ztimer_xxx(LOW_FREQ_INSTANCE, time / somediv);
}
}
Clearly this is very simplified but you get the idea.
Yeah, but simplified doesn't cut it. Sleeping one second on an 1HZ timer
is a different thing than sleeping 1000ms on an ms timer. Even with a
perfect implementation, the former will sleep anything from zero to two
seconds. The latter anything from 999 to 1001ms, if they each need to
work with hardware of 1Hz resp. 1000Hz. There's not much to be done there.
That is one of the main issues with an API that doesn't ha
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hi Michel,
On 12/11/19 4:50 PM, Michel Rottleuthner wrote:
> Hi Kaspa
>> Would it make sense to make a micro conference? Get everyone interested
>> in improving timers in a room and lock it until solutions are presented?
> Not convinced about the "lock in a room" ;) - but otherwise: absolutely
> yes!
>
> What do you think about an RDM PR?
> We could just use your design document as a starting point.
Let me propose the following: we make merging ztimer *as a distinct and
optional module* independent of changing RIOT's default timer
implementation. The latter can be done within the RDM.
IMO, ztimer is quite usable already, and even if only used as
multiplexer for the RTT, it provides a lot of benefit. I don't see a
reason not to merge it (when it's been reviewed properly), as an
optional module.
We can, in parallel, work on the RDM. If it turns out there's some
better option than ztimer, no harm was done, I'll of course happily
accept that.
I already have a basic xtimer benchmark application (for timing
set()/remove()/now() in pathetic cases), which can provide at least some
numbers. I'll PR that today.
Regarding "fixing" xtimer vs a rewrite from scratch, I'd like to point
out that #9503 alone changes (well, removes) ~450 lines of *logic code*.
That is three quarters of xtimer's total code, including definitions and
prototypes. IMO, we need to acknowledge that changing that amount of
code does not result in the same code base. We should call it "ytimer".
The amount of reviewing, validation and testing should be the same as
for a rewrite. Or maybe just be measured in "amount of lines changed".
Regarding whether a "clock" parameter makes sense, this is something we
should explore within the RDM. I think you need to prototype a function
that chooses a suitable frequency from multiple options (without relying
on an explicit parameter for that). (I'd actually suggest you use ztimer
as basis, as there you have multiple, multiplexed backends using the
same API. :) ). You might even be successful. At that point, an RDM can
decide if that functionality should move down the layers.
More details following:
>> periph_timer IMO should be the slimmest layer of hardware abstraction
>> that makes sense, so users that don't want to do direct non-portable
>> register based applications get the next "closest to the metal".
> Agree, but there are some things that we should add to the periph_timer.
> E.g. adding support for dedicated overflow interrupts together with an
> API to read
> the corresponding IRQ status bit.
> The high level timer would benefit from that on many platforms.
> E.g. Ztimer wouldn't require the code for the time partitioning
> mechanism then.
> But thats yet another part of the story...
Yes. Also, do all platforms support that overflow interrupt?
I don't think so, in which case this cannot be relied upon to be available.
> Also the term "frequency conversion" is a bit misleading I think.
> With a discrete clock you won't be able to just precisely convert a
> frequency to any other frequency in software.
> Especially if you want to increase the frequency - it will just be a
> calculation.
Yup. Frequency conversion makes sense if an application wants to sleep
12345ms, but the timer is clocked at e.g., 1024Hz.
>> That is one of the main issues with an API that doesn't have the clock
>> parameter, but a fixed (probably high frequency) frequency, as xtimer
>> has.
>
> Of course there is a difference.
> Here I just wanted to point out that the quality defect of xtimer
> not mapping to multiple peripherals is not directly tied to its API.
>
> Further, adding a convention to the xtimer API would allow to for
> automatic selection of an appropriate low-level timer.
> E.g. think of something like "will always use the lowest-power timer
> that still ensures x.xx% precision".
That's what ztimer does.
> Again, this is just a simple example to explain what I think we should
> also consider as part of the solution.
> Forcing the application / developer to select a specific instance also
> has it's downsides.
With convention ("ZTIMER_MSEC provides ~1ms accuracy"), the application
developer chooses the intended precision. Without that, and with a fixed
API time base, 1s (100us) cannot be distinguished from 1000ms or
100us.
Maybe it can, this is where you can maybe come up with a prototype.
> I mostly agree. But as I tried to clarify before:
> ztimer is mixing "relevant and valid fixes" and "introducing new design
> concepts".
> We should strive for being able to tell what is done because it fixes
> something
> and what is done because the concept is "considered better".
> Next to that the "considered better" should then be put to the test.
Ok. That might be necessary to choose one implementation over xtimer. -> RDM
>>> For timeouts that are calculated at runtime do we really want to always
>>> add some code to decide which instance to use?
>> If there are multiple instances, there is code that
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hi, here are my thoughts on the discussion. # Not Getting Lost in Requirement Analysis and Problem Specifications A good requirement analysis is a valuable tool for both development and evaluation of a software component. But once a solid understanding of the problem to solve is reached, additional effort put into a requirement analysis doesn't yield enough additional benefit to justify the work. And every requirement analysis is flawed: Assembling a complete list of all current requirements in RIOT's code base is hard. Predicting how future developments influence the requirements we have is impossible. There has to be a point when we just stop on collecting more requirements and consider the current list as good enough; a perfect, complete and definite result cannot be reached. High level timer API are no new concept and the basic goals and requirements are well understood. On top of these basic requirements, benchmarks could be a good tool to quantify how well specific timer implementations perform. To me, writing a set of benchmarks would be more useful than additional requirements collection. Not only would it allow to see how good ztimer/xtimer are performing. They will also be useful for development and reviews of future PRs targeting RIOTs timer framework. In the end, RIOT will be judged upon the features it provides. Not on the features on RIOT's to do lists. Not on how tough and rigorous the requirements are we have formulated on some yet-to-be-implemented feature. And not on how much documents and emails have been written about an yet-to-be-implemented feature. # Complete Rewrites can be the Best Option @Michel: You seem to dismiss the development approach of a complete rewrite fundamentally, apparently for ideological reasons. While most of the time a complete rewrite is not the best option, there are good reasons for doing so in some cases: When fundamental architectural changes are needed, the effort of an rewrite can be less compared to iterative transform the architecture. In such cases, a rewrite is the better option. I don't think that there is a reason to see a complete rewrite as some kind of failure that we should try very hard to prevent for the future. And ultimately, any attempt to prevent rewrites can fail. Any design decision is based on the current experience, the available tools, and the current requirements. But with time, more experience is gained, better tools become available, and completely new use cases with potentially fully distinct requirements pop up. Perfectly solid design decisions can be rendered obsolete by this. And sometimes, fundamental architecture changes are needed in response. In that cases, a rewrite can be the best option. And this could happen very well without any mistakes being made in the original implementation. # Suggestion of a Path Forward There seems to be an agreement, that an additional parameter is needed in the API. How about we specify an implementation independent high level timer API based on xtimer with an additional parameter. The type of the first argument could just by some `foo_t` that is provided by the implementation of that API. The implementation could be freely chosen as module, and xtimer could be the default backend when the user has not specified a backend. I bet it is trivial to extend xtimer in a way to comply with that API by just adding an ignored parameter. Everyone with the motivation, knowledge and dedication to fix the discussed issue in xtimer has still the opportunity to do so. And providing ztimer as alternative implementation of the same API would not hurt anyone. Not defining the semantics and contents of the first parameter sounds a bit crazy. But I bet that 90% of all use cases will only use two different values there: One low power low resolution setting for long sleeps and one high power high precision setting e.g. in drivers. Just providing these two settings as global variables (or preprocessor macros) under defined names would be enough to cater 90% of the use cases in an implementation agnostic way. (And the other use cases could provide their own global variables for the additional settings, so that only the assignment of those variable depend on the used implementation.) And if there is indeed enough interest in fixing xtimer, this would allow them to freely decide if they want this parameter to be a pointer to a clock struct as in ztimer, or maybe a bitmask holding a set of flags, or something completely different. It might be very well possible that the future will bring use cases will mutually exclusive requirements. A implementation independent API would allow to just let the users choose what they need. Kind regards, Marian On Tue, 10 Dec 2019 18:23:53 +0100 Michel Rottleuthner wrote: > Hi, > > Thanks for starting this! It's very much appreciated. > Discussing these things, reaching common ground and documenting > decisions and findings > during this process is IMO one of the most
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hi Kaspar, Marian,
thanks for responding.
Let me propose the following: we make merging ztimer *as a distinct and
optional module* independent of changing RIOT's default timer
implementation. The latter can be done within the RDM.
IMO, ztimer is quite usable already, and even if only used as
multiplexer for the RTT, it provides a lot of benefit. I don't see a
reason not to merge it (when it's been reviewed properly), as an
optional module.
We can, in parallel, work on the RDM. If it turns out there's some
better option than ztimer, no harm was done, I'll of course happily
accept that.
I'd favor to not merge it before we worked on the RDM as it may show
that there are things that should be done differently.
But with our current state of xtimer there is not much harm to be done
anyway ;)
Also ztimer is providing solutions to problems we currently have no
alternatives for - so I won't be blocking this ;)
I already have a basic xtimer benchmark application (for timing
set()/remove()/now() in pathetic cases), which can provide at least some
numbers. I'll PR that today.
perfect!
Regarding "fixing" xtimer vs a rewrite from scratch, I'd like to point
out that #9503 alone changes (well, removes) ~450 lines of *logic code*.
That is three quarters of xtimer's total code, including definitions and
prototypes. IMO, we need to acknowledge that changing that amount of
code does not result in the same code base. We should call it "ytimer".
The amount of reviewing, validation and testing should be the same as
for a rewrite. Or maybe just be measured in "amount of lines changed".
Agree. (Just to be sure: I understand "calling it ytimer" in a
metaphorical way)
Regarding whether a "clock" parameter makes sense, this is something we
should explore within the RDM. I think you need to prototype a function
that chooses a suitable frequency from multiple options (without relying
on an explicit parameter for that). (I'd actually suggest you use ztimer
as basis, as there you have multiple, multiplexed backends using the
same API. :) ). You might even be successful. At that point, an RDM can
decide if that functionality should move down the layers.
Sounds reasonable.
To clarify a bit: I don't say we shouldn't have an additional parameter.
I'm just saying that it may be more flexible to decouple the precision
parameter from the instance parameter.
Explicit values instead of conventions might also help the usability.
periph_timer IMO should be the slimmest layer of hardware abstraction
that makes sense, so users that don't want to do direct non-portable
register based applications get the next "closest to the metal".
Agree, but there are some things that we should add to the periph_timer.
E.g. adding support for dedicated overflow interrupts together with an
API to read
the corresponding IRQ status bit.
The high level timer would benefit from that on many platforms.
E.g. Ztimer wouldn't require the code for the time partitioning
mechanism then.
But thats yet another part of the story...
Yes. Also, do all platforms support that overflow interrupt?
I don't think so, in which case this cannot be relied upon to be available.
We are currently working on transforming "I don't think so" into numbers.
It will still take some time to work thru all of those, *cough* lovely
written data sheets.
I'll provide the collected information to the RDM.
Currently it looks like *most* platforms have this option.
It would absolutely make sense to use that feature if available and only
work around
if needed. Especially if this can be decided at compile time like with
the ztimer instances.
Also the term "frequency conversion" is a bit misleading I think.
With a discrete clock you won't be able to just precisely convert a
frequency to any other frequency in software.
Especially if you want to increase the frequency - it will just be a
calculation.
Yup. Frequency conversion makes sense if an application wants to sleep
12345ms, but the timer is clocked at e.g., 1024Hz.
Why?
This won't change the fact that the timer is clocked at 1024 Hz so there
is no frequency conversion.
It's a conversion from period to ticks.
You won't suddenly get accurate discrete ms ticks because of this
calculation.
Again, this is just a simple example to explain what I think we should
also consider as part of the solution.
Forcing the application / developer to select a specific instance also
has it's downsides.
With convention ("ZTIMER_MSEC provides ~1ms accuracy"), the application
developer chooses the intended precision. Without that, and with a fixed
API time base, 1s (100us) cannot be distinguished from 1000ms or
100us.
Maybe it can, this is where you can maybe come up with a prototype.
I'm not saying we should stay with a fixed time base API.
Regarding 1s cannot be distinguished from 100 µs:
That's not required if you specify the required accuracy as percentage.
Coincidentally, that's also how the accuracy of the clocks fe
Re: [riot-devel] ztimer - a new high-level timer for RIOT
As far as I understand, the new timer implementation would not use 64 bit for the timer and the user is responsible for not overrunning the timer? Note that I haven't looked a the implementation yet, so forgive my ignorance. Over the years my experience is that it's no good idea to burden the user with the knowledge of timer overflow. The latest example was a bunch of HPE SSDs that stop working after 32.768 hours (a little less than 4 years). Bad when you have several of them in a RAID for redundancy :-) https://www.techradar.com/news/hpe-ssd-drives-could-fail-at-this-critical-moment So I'd vote for the (small) additional overhead, even on 8-bit µCs due to safety reasons. Unless the implementation can produce the correct timer representation with, say, C-preprocessor magic at compile-time. Ralf -- Dr. Ralf Schlatterbeck Tel: +43/2243/26465-16 Open Source Consulting www: http://www.runtux.com Reichergasse 131, A-3411 Weidling email: off...@runtux.com ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hi Ralf, On 12/13/19 6:41 PM, Ralf Schlatterbeck wrote: > As far as I understand, the new timer implementation would not use 64 > bit for the timer and the user is responsible for not overrunning the > timer? Note that I haven't looked a the implementation yet, so forgive > my ignorance. I think you got it right. In ztimer, the clocks' now() value will overflow after 32bits. E.g., `t1 = now(); t2 = now(); assert(t1 < t2)` might fail. Setting timer values is always relative (target = now() + X), and X can be any 32bit value, the user does not need to handle any overflow there. > Over the years my experience is that it's no good idea to burden the > user with the knowledge of timer overflow. The latest example was a > bunch of HPE SSDs that stop working after 32.768 hours (a little less > than 4 years). Bad when you have several of them in a RAID for > redundancy :-) Failing after 32768h strongly points to using a signed 16bit variable for the hours count. That's an odd choice to begin with. > https://www.techradar.com/news/hpe-ssd-drives-could-fail-at-this-critical-moment > So I'd vote for the (small) additional overhead, even on 8-bit µCs due > to safety reasons. Unless the implementation can produce the correct > timer representation with, say, C-preprocessor magic at compile-time. You mean, you'd vote for all timer values to be always 64bit? I don't think the overhead is small enough to justify that, but that needs to be evaluated. The increased CPU usage needs benchmarks, in RAM we can estimate easily. IMO, users should just not use these timer values as timestamps. That maybe needs to be stated at a prominent place (or multiple) in the documentation. For that, we can (should?) re-introduce a 64bit API in ztimer. It might make sense to add a (very large or possibly randomized) constant offset to all 32bit timers, maybe limited to development builds, that causes the timer value to overflow early(er), so applications would break early in tests. Kaspar ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
Re: [riot-devel] ztimer - a new high-level timer for RIOT
Hi, in order to push this forward I just opened a PR for an RDM at https://github.com/RIOT-OS/RIOT/pull/12970. This PR is in an early state and both feedback and help will be greatly appreciated. > IMO, users should just not use these timer values as timestamps. > That maybe needs to be stated at a prominent place (or multiple) in the > documentation. > For that, we can (should?) re-introduce a 64bit API in ztimer. I don't think that 64 bit is needed for every feature of a high-level timer API; especially for delaying the execution of the calling thread or for software timers. A 64 bit system clock would however be quite nice in some use cases. > > E.g. adding support for dedicated overflow interrupts together with an > > API to read > > the corresponding IRQ status bit. > > The high level timer would benefit from that on many platforms. > > E.g. Ztimer wouldn't require the code for the time partitioning > > mechanism then. > > But thats yet another part of the story... > > Yes. Also, do all platforms support that overflow interrupt? > I don't think so, in which case this cannot be relied upon to be available. This would not strictly be needed. In case either an overflow interrupt or a alarm interrupt is supported, the other could be implemented by the timer driver. (If only an overflow is supported, the driver could set the current timer value so that the overflow interrupt happens exactly when the alarm should happen. And if only an alarm is supported, the driver could multiplex two alarms with one being the overflow.) It needs however to be determined if this would indeed be better. > Marian: would you also be interested in attending a timer meeting? If I can find the time, sure! Some weekend in the first half of January might work for me. How about Kaspar or Michel can organize a room and use one of the many online scheduling services to find out a date most interested developers could join? Kind regards, Marian On Mon, 16 Dec 2019 11:31:34 +0100 Kaspar Schleiser wrote: > Hi Ralf, > > On 12/13/19 6:41 PM, Ralf Schlatterbeck wrote: > > As far as I understand, the new timer implementation would not use 64 > > bit for the timer and the user is responsible for not overrunning the > > timer? Note that I haven't looked a the implementation yet, so forgive > > my ignorance. > > I think you got it right. In ztimer, the clocks' now() value will > overflow after 32bits. > > E.g., `t1 = now(); t2 = now(); assert(t1 < t2)` might fail. > > Setting timer values is always relative (target = now() + X), and X can > be any 32bit value, the user does not need to handle any overflow there. > > > Over the years my experience is that it's no good idea to burden the > > user with the knowledge of timer overflow. The latest example was a > > bunch of HPE SSDs that stop working after 32.768 hours (a little less > > than 4 years). Bad when you have several of them in a RAID for > > redundancy :-) > > Failing after 32768h strongly points to using a signed 16bit variable > for the hours count. That's an odd choice to begin with. > > > https://www.techradar.com/news/hpe-ssd-drives-could-fail-at-this-critical-moment > > So I'd vote for the (small) additional overhead, even on 8-bit µCs due > > to safety reasons. Unless the implementation can produce the correct > > timer representation with, say, C-preprocessor magic at compile-time. > > You mean, you'd vote for all timer values to be always 64bit? I don't > think the overhead is small enough to justify that, but that needs to be > evaluated. > The increased CPU usage needs benchmarks, in RAM we can estimate easily. > > IMO, users should just not use these timer values as timestamps. > That maybe needs to be stated at a prominent place (or multiple) in the > documentation. > For that, we can (should?) re-introduce a 64bit API in ztimer. > > It might make sense to add a (very large or possibly randomized) > constant offset to all 32bit timers, maybe limited to development > builds, that causes the timer value to overflow early(er), so > applications would break early in tests. > > Kaspar > > ___ > devel mailing list > devel@riot-os.org > https://lists.riot-os.org/mailman/listinfo/devel pgpxImlMJaxBj.pgp Description: OpenPGP digital signature ___ devel mailing list devel@riot-os.org https://lists.riot-os.org/mailman/listinfo/devel
