Hi Syed,
On 14/05/2024 15:37, Syed Aftab Rashid wrote:
Thanks for the response Ralf,
I came across a paper that compares the performance of a few static
partitioning hypervisors including Jailhouse. Probably, you are already
aware of it, the paper is available here.
https://arxiv.org/pdf/2303.11186 <https://arxiv.org/pdf/2303.11186>.
Yes, the paper from the Portuguese colleagues.
One interesting thing I noticed is that for ARM with direct interrupt
injection enabled, the interrupt latency for Jailhouse is around ~200ns
(please see the attached image). However, on my x86 system I am getting
Yes, and in the above case, jailhouse and bao performs exactly the same,
as they use the same hardware mechanisms.
an average latency of ~800ns (from the APIC demo example). I think the
APIC example is also doing the same thing as was done for the experiment
in the figure. Do you have any idea why there is such a difference
between the values for both cases? is it architecture specific?
As we see no XApic traps in your case: Yes, it must have to do something
with /the rest/ of the hardware resp. architecture.
At least we can say, it's not the interrupt delivery system.
What remains? Lots of other architectural specifics, such as:
- Cache configuration(s)
- CPU Frequency scaling
- Power saving mechanisms
If you see lower latencies on your bare-metal configuration, I'd suggest
do investigate how the bare-metal application tunes and sets up the system.
Ralf
On Fri, May 3, 2024 at 5:31 PM Ralf Ramsauer
<ralf.ramsa...@oth-regensburg.de
<mailto:ralf.ramsa...@oth-regensburg.de>> wrote:
Hi,
On 02/05/2024 15:45, Syed Aftab Rashid wrote:
[snip]
> > I just wanted to understand what is the best possible
jitter that
> I can
> > get with Jailhouse. We are working on a project that needs
cycle
> times
> > between 1-2 microseconds. The bare metal implementation of a
> similar
> > application results in a much lower jitter ~50ns.
>
> So the application does structurally the same like apic-demo?
> (i.e., full IRQ arrival and dispatching, "computational task"
(i.e.,
> calculating jitter), return). 50ns are ~100 TSC cycles –
athletic.
>
> I really doubt those latencies are achievable with IRQs. With
active
> waiting for events (i.e., active polling for interruption) -
maybe.
>
>
> Yes, it is a similar application implemented in bare metal code
directly
> on the hardware. The jitter is even smaller with the max being 30ns.
Interesting. W/o having the possibility to have a close look at this
issue, I'd say that this comes from the additional need of hardware
lookups through IRQ rerouting tables. I cannot say.
[snip]
> > apic-demo doesn't seem to have much impact on the jitter.
How is
> cache
> > allocated under Jailhouse house? What does a cache
allocation of
> size 2
> > employ?
>
> Cache allocation is only supported on Intel X86, where we
have CAT,
> cache allocation technology.
>
> I don't know precise details of cat (would have to look them up),
> but as
> far as I remember, assigning a size of 2 to a non-root-cell
means that
> 2MiB of L3$ are exclusively assigned to the non-root cell.
(Look at the
> code, I'm not 100% sure).
>
> Why do we do that? Neighbouring noise from other cells.
>
> Just like the non-root cell might "disturb" itself, it's also
possible
> that Linux, running aside of the non-root cell disturbes it:
Imagine
> Linux those heavy number crunching while the non-root cell
waits for an
> IRQ. In that case, Linux will mess up caches of the non-root
cell.
>
> CAT prevents full cache pollution by ensuring that there is
always a
> dedicated, but limited, amount of L3$ for the non-root cell.
>
>
> Thanks for the explanation, I understand the idea of cache
pollution. It
> is just that I don't see much impact with or without cache
pollution on
> the values I get for the jitter. Ideally, the jitter should be
higher if
> I enable cache pollution and lower when I disable it. But, so far
I am
> not able to see that.
>
> Also, another question regarding direct interrupt injection, as
we saw
> that it is enabled by default. Can you tell me how can I disable it?
> Again, this is just to see how it will impact the jitter if we have
> direct interrupt injection disabled.
There's no actual support for disabling this, it's used when being
available. When Jailhouse tries to query the capability of IRQ
remapping
support during startup, you will have to mask out the bits that show
support for it. I guess that should work.
Ralf
--
Best Regards,
Syed Aftab Rashid
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/jailhouse-dev/81cdbced-ce98-4479-99b0-9774d9bf5f38%40oth-regensburg.de.
