Hello James, Julien,
regarding your previous mails. I was able to single step every instruction
of my module. The problem (or rather the solution) was to _disable_ the IRQ
interrupts from within my guest module. This solves the problem of
singlestepping a module which previously ended in a spinlock. But it does
not solve the problem with system that is singlestepped with enabled IRQ's,
as it still will be locked within a spinlock.
My module starts the singlestep routine by executing an SMC instruction.
This will be catched from within my xen-access in the Dom0 (which also will
increase the PC accordingly). Xen-access will then trigger an event to set
the domain flag for my VM (the Domain flag determines if SS should be
enabled when entering the guest) and by this start singlestepping. Every SS
exception is also caught within xen-access which then lets me decide how to
continue the execution.
This is my module code which is executed in the DomU:
int init_module()
> {
> printk(KERN_INFO "### Init address 0x%lx\n", &init_module);
> printk(KERN_INFO " Set function hook\n");
> patch_function_hook();
>
> printk(KERN_INFO " Starting singlestep\n");
>
This following part is the importand bit. I disable the local_irq's and
execute two SMCs.
> local_irq_disable();
> __asm__ __volatile__ ("SMC 1");
> __asm__ __volatile__ ("SMC 1");
>
> if(!already_trapped)
> {
> __asm__ __volatile__ ("NOP");
> __asm__ __volatile__ ("NOP");
> __asm__ __volatile__ ("NOP");
>
> //Just for keeping module busy while singlestep
> for ( c = 0 ; c < n ; c++ )
>
When executing these exact steps, it is possible to singlestep the whole
module. Without the local_irq_disable() the system will stop the module
execution right after the first SMC.
My Xen Access version got, just like the original one, a main loop where
events will be catched:
> printf("Starting loop\n" );
> for (;;)
> {
> if ( interrupted )
> {
> [...]
> }
>
> while ( RING_HAS_UNCONSUMED_REQUESTS(&xenaccess->vm_event.back_ring)
> )
> {
> [...]
>
> switch (req.reason) {
> case VM_EVENT_REASON_PRIVILEGED_CALL:
> printf("++++++++++++++++++++++++++++Privileged call:
> pc=%"PRIx64" (vcpu %d)\n",
> req.data.regs.arm.pc,
> req.vcpu_id);
>
>
> if (!ss_started)
> {
> ss_started = 1;
> toggle_single_step_domain(1);
>
>
This part enables singlestepping the VM with the first SMC. (The function
toggle_single_step_domain just sets the domain flag i mentioned in earlier
mails)
> }else
> {
> printf("Signlestep already activated\n");
> }
>
> rsp.data.regs.arm = req.data.regs.arm;
> rsp.data.regs.arm.pc += 4;
>
This is the part which sets the VM.PC to the next instruction after the SMC
instruction within the guest module.
>
> case VM_EVENT_REASON_SINGLESTEP:
>
> printf("Singlestep: PC=%016"PRIx64", vcpu %d, \n",
> req.data.regs.arm.pc,req.vcpu_id);
>
>
> if(!run)
> {
>
> printf("Enter command: s:step; c:end; r:regs:;x:end :"
> );
>
> do
> {
> command = getchar();
> }
> while (isspace(command));
>
>
> switch(command)
> {
> case 'c': printf("Singlestep weiterlaufen \n");
> // rsp.flags |= VM_EVENT_FLAG_TOGGLE_
> SINGLESTEP;
> run = 1;
> break;
> [More possible commands here]
> default: printf("Wrong command \"%c\"!\n",
> command );
> break;
>
> }
>
The above part is the handling of software step exceptions. As you can
see, i print the PC value and read a command from the user in order to let
the guest execute further or print its registers.
>
>
These two are the main control flow of xen-access.
Finally my Xen Code within the leave_hypervisor_tail looks like this. The
leave_hypervisor_tail function is called right before the control flow
switches to the guest. Because of this, every time when the guest gets to
run, we set the bits needed to singlestep.
asmlinkage void leave_hypervisor_tail(void)
> {
> struct vcpu *v = current;
>
> if ( unlikely(v->domain->arch.monitor.singlestep_enabled ) )
> {
>
> if(!(guest_cpu_user_regs()->cpsr & 0b1000))
> {
>
> WRITE_SYSREG(READ_SYSREG(MDSCR_EL1) | 0x1, MDSCR_EL1);
> WRITE_SYSREG(READ_SYSREG(MDCR_EL2) | HDCR_TDE, MDCR_EL2);
> guest_cpu_user_regs()->cpsr = guest_cpu_user_regs()->cpsr |
> 0x200000;
>
> v->arch.single_step = 1;
>
> }
>
>
> }else
> {
> //single_step Domain flag not set
> if( v->arch.single_step )
> {
> gprintk(XENLOG_ERR, "Domain flag not set, but vcpu flag is
> set\n");
> WRITE_SYSREG(READ_SYSREG(MDSCR_EL1) & ~0x1, MDSCR_EL1);
> guest_cpu_user_regs()->cpsr = guest_cpu_user_regs()->cpsr &
> ~0x200000;
>
> v->arch.single_step = 0;
> }
>
> }
>
> while (1)
> {
> local_irq_disable();
>
[...]
>
So, I adapted your comments and as I mentioned. The general SingleStep
functionality works fine.
But: It's not possible to singlestep a system as long as the VM IRQ's are
enabled. If we would activate single stepping with enabled interrupts, we
will be locked in the mentioned spinlock.
Because of this it is not possible to singlestep other application.
Additionally it is not possible to print anything while singlestepping
because, as far as I understood, the system will wait within a spinlock
until the terminal is free to print.
Do you have any idea why it's not possible to escape the lock while
singlestepping? Like I mentioned, my guess is on timer interrupts, which
should unlock the spinlock but generate problems with singlestep enabled at
the same time. This would also explain why i can observe the control flow
of my guest module with IRQ's being disabled.
Below is an abstract of the program flow. The first one is with disabled
IRQ's. You can see that every instruction is from my module (as the
addresses are around 0xffff0000008e0000). The second one shows instructions
which, directly after the SMC, starts the "go to spinlock" routine because
of enabled IRQ's.
Kind regards,
Florian
Working Singlestep in module:
> root@avocet:~# ./xen-access -m 2 singlestep
> Singlestep request found
> Starting loop
> ++++++++++++++++++++++++++++Privileged call: pc=ffff0000008e0618 (vcpu 0)
> Seeting Singlestep to 1
> SingleStep succesfull rc=0
> ++++++++++++++++++++++++++++Privileged call: pc=ffff0000008e061c (vcpu 0)
> Singlestep: PC=ffff0000008e0624, vcpu 0,
> Singlestep: PC=ffff0000008e0628, vcpu 0,
> Singlestep: PC=ffff0000008e062c, vcpu 0,
> Singlestep: PC=ffff0000008e0630, vcpu 0,
> Singlestep: PC=ffff0000008e0634, vcpu 0,
>
>
Singlestep with enabled IRQ's:
> root@avocet:~# ./xen-access -m 2 singlestep
> Singlestep request found
> Starting loop
++++++++++++++++++++++++++++Privileged call: pc=ffff0000008e0000 (vcpu 0)
>
(vcpu 0)
> Seeting Singlestep to 1
> SingleStep succesfull rc=0
>
Singlestep: PC=ffff000008081a80
>
[...]
>
Singlestep: PC=ffff000008082700
> [...]
Singlestep: PC=ffff0000080814e8
> [...]
Singlestep: PC=ffff000008100f60
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
