Hi guys,
As part of my master's thesis I am facing a challenging problem.
I am trying to let two processes be each other's debuggers using the
ptrace syscall. However, my proof-of-concept implementation always
results in a deadlock state (both processes get stuck in 't+' state as
shown by 'ps aux').
Here is my code, it's pretty simple:
https://pastebin.com/A1iBA3nh
I have compiled and run this on an ARMv7 developer board with kernel
version 3.0.35 (Linaro 13.08).
The output of the above code is this:
A waiting to continue...
B attachTo: 0
B setOptions: 0
B setVarData: 0
B setVarData: 0
B cont: 0
B waiting to continue...
B waiting to continue...
A attachTo: 0
As you can see it never reaches the "finished" printf code, and gets
stuck as soon as the other process attempts to attach to the debugger.
I have done a similar experiment for 3 processes, such that each one
attempts to attach to the other in a circular fashion: A -> B -> C -> A
The result in this case was exactly the same. However, here I was able
to detect a race condition, because sometimes the code executed
properly without getting stuck in a deadlock (but it's hard to
reproduce).
If you wish, you can test this by using a lightweight debugger I've
developed and three console terminals. Here's the code:
https://pastebin.com/fPJb8ZNb
Once you've compiled the above code, you simply run the binary on each
console and enter the PID of another process to establish a 3-way
circle.
I am far from an expert on the kernel, but I did have a look at the
ARM specific kernel implementation which left me puzzled. I couldn't
find where/how/why this code does not work.
Now I'm wondering if it's possible at all to make this work without a
deadlock occurring? Does anyone have any experience with this, or can
provide some clues/feedback?
Thank you greatly for your time, attention and effort!
Ilya Nevolin
[email protected]
