Re: [Xenomai-core] Frozen timer IRQ - now traced with kgdb :)
Jan Kiszka wrote: Yep, I dug deeper meanwhile and also came across this. I already have a trivial hack running here. The most tricky part for me was to learn quilt, but now I start to love it :). Here is a snapshot series for 2.6.15.5: prepare-ipipe-x86.patch adeos-ipipe-2.6.15-i386-1.2-01.patch kgdb-ipipe-x86.patch In order to ease patch maintenance, we should move the relevant portions of this infrastructure to the I-pipe patch directly (i.e. I-pipe specific kgdb-ipipe-* code). I'm currently wondering if it makes sense to register a kgdb domain and "officially" capture all involved IRQs and events. So far the serial line IRQ is hard-coded (should be retrieved from some internal kgdb structure later). Anyway, it seems to work quite well, I'm currently stepping through a network IRQ at ipipe-level. Having a separate domain would allow to break into any runaway code from lower priority domains even with disabled interrupts, except the ipipe itself. This said, pushing a domain on top of Xenomai would break the assumption that hw interrupts are indeed disabled when operating due to the 'last domain optimization' feature, and introduce additional jittery. The other option would be to install a KGDB 'redirector' in __ipipe_handle_irq so that serial or network interrupts to KGDB would never be blocked by the stall bit; I would actually prefer this one. While playing with this tool a bit, displaying the the ipipe structures, and thinking about the original problem again, I wondered what could cause a temporary (as I think to found out now) stalled xeno domain without locking up the system? Some irq-lock leaks at driver level (i.e. inside our own code)? At first sight, it might be related to the way __ipipe_unstall_iret_root operates. Basically, the idea is to make sure that the stall flag of the root domain upon return from the pipelining process always reflects the state of the hw interrupt flag at the time the processed event was taken by the CPU. It seems that your testcase shows that under some cicumstances, the root stage might be spuriously left in a stalled state by __ipipe_unstall_iret_root. -- Philippe. ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] [BUG?] stalled xeno domain
Jan Kiszka wrote: Jan Kiszka wrote: Hi Philippe, debugging is nice, tracing is still nicer. As you suggested, I extended the tracer with per-domain stall flags (needs some output clean-up, preliminary patch attached nevertheless). And here is the result (full trace attached): :|(0x51) 0x00c8 -1113+ 1.112 __ipipe_sync_stage+0x142 (ipipe_suspend_domain+0x56) :|fn-1112+ 1.789 __ipipe_sync_stage+0xe (ipipe_suspend_domain+0x56) :| *(0x50) 0x0064 -1110+ 1.293 __ipipe_sync_stage+0x97 (ipipe_suspend_domain+0x56) :*fn-1109+ 1.398 do_IRQ+0x8 (__ipipe_sync_stage+0xcf) :*fn-1107+ 2.105 __do_IRQ+0xc (do_IRQ+0x21) :*fn-1105+ 1.631 handle_IRQ_event+0xd (__do_IRQ+0x9e) :*fn-1104+ 2.413 timer_interrupt+0x9 (handle_IRQ_event+0x40) :*fn-1101+ 3.022 mark_offset_tsc+0xe (timer_interrupt+0x31) :*fn-1098+ 2.721 do_timer+0x9 (timer_interrupt+0x37) :| *fn-1096+ 2.112 __ipipe_handle_irq+0xe (common_interrupt+0x18) :| *fn-1093+ 1.210 __ipipe_ack_common_irq+0xc (__ipipe_handle_irq+0xc0) :| *(0x50) 0x0064 -1092+ 1.218 __ipipe_ack_common_irq+0x31 (__ipipe_handle_irq+0xc0) :| *fn-1091+ 1.834 mask_and_ack_8259A+0xb (__ipipe_ack_common_irq+0x5d) :| *(0x50) 0x0064 -1089+ 1.345 __ipipe_ack_common_irq+0x9d (__ipipe_handle_irq+0xc0) :| *fn-10880.924 ipipe_suspend_domain+0xb (__ipipe_handle_irq+0x174) :| *(0x51) 0x00c8 -1087+ 1.172 ipipe_suspend_domain+0x26 (__ipipe_handle_irq+0x174) :| *fn-1086+ 2.030 __ipipe_sync_stage+0xe (ipipe_suspend_domain+0x56) :| **(0x50) 0x00c8 -1084+ 1.330 __ipipe_sync_stage+0x97 (ipipe_suspend_domain+0x56) :| **fn-10820.879 xnintr_clock_handler+0x8 (__ipipe_sync_stage+0x12b) :| **fn-1082+ 1.218 xnintr_irq_handler+0xb (xnintr_clock_handler+0x18) :| **fn-1080+ 1.233 xnpod_announce_tick+0x9 (xnintr_irq_handler+0x2a) :| **(0x50) 0x00c8 -1079+ 1.736 xnpod_announce_tick+0x20 (xnintr_irq_handler+0x2a) :| **fn-1077+ 2.984 xntimer_do_tick_aperiodic+0xe (xnpod_announce_tick+0x36) :| **fn-1074+ 2.751 xnthread_timeout_handler+0x8 (xntimer_do_tick_aperiodic+0x18d) :| **fn-1072+ 1.864 xnpod_resume_thread+0xe (xnthread_timeout_handler+0x1d) :| **(0x50) 0x00c8 -1070+ 4.699 xnpod_resume_thread+0x2b (xnthread_timeout_handler+0x1d) :| **(0x50) 0x00c8 -1065+ 1.586 xnpod_resume_thread+0x2bf (xnthread_timeout_handler+0x1d) :| **(0x01) 0x1237 -1063+ 2.661 xntimer_do_tick_aperiodic+0x309 (xnpod_announce_tick+0x36) :| **(0x50) 0x00c8 -1061+ 1.263 xnpod_announce_tick+0x4f (xnintr_irq_handler+0x2a) :| **fn-1060+ 1.255 rthal_irq_end+0x8 (xnintr_irq_handler+0x46) :| **fn-1058+ 2.007 enable_8259A_irq+0x9 (rthal_irq_end+0x2e) :| **fn-1056+ 1.924 xnpod_schedule+0xe (xnintr_irq_handler+0x6e) :| **(0x50) 0x00c8 -1054! 15.368 xnpod_schedule+0x53 (xnintr_irq_handler+0x6e) :| **fn-1039! 13.300 __switch_to+0xc (xnpod_schedule+0x689) :| **(0x50) 0x00c8 -1026+ 1.766 xnpod_schedule+0x951 (xnpod_suspend_thread+0x27c) :| **(0x50) 0x00c8 -1024! 18.195 xnpod_suspend_thread+0x2bb (rt_task_sleep+0x54) : **fn-1006+ 1.624 __ipipe_syscall_root+0x9 (system_call+0x20) : **fn-1004+ 1.406 __ipipe_dispatch_event+0xe (__ipipe_syscall_root+0x58) : **fn-1003+ 4.323 hisyscall_event+0xe (__ipipe_dispatch_event+0x5e) : **fn -998+ 1.398 __rt_task_sleep+0xa (hisyscall_event+0x23c) : **fn -997+ 1.789 __copy_from_user_ll+0xa (__rt_task_sleep+0x1a) : **fn -995! 15.345 rt_task_sleep+0xa (__rt_task_sleep+0x25) : **fn -9800.879 __ipipe_syscall_root+0x9 (system_call+0x20) : **fn -979+ 1.308 __ipipe_dispatch_event+0xe (__ipipe_syscall_root+0x58) : **fn -978+ 2.451 hisyscall_event+0xe (__ipipe_dispatch_event+0x5e) : **fn -975+ 1.631 sys_rtdm_ioctl+0x8 (hisyscall_event+0x23c) : **fn -974+ 1.255 _rtdm_ioctl+0xc (sys_rtdm_ioctl+0x1b) : **fn -972+ 4.180 rtdm_context_get+0xa (_rtdm_ioctl+0x20) : **fn -968+ 1.203 __ipipe_syscall_root+0x9 (system_call+0x20) : **fn -967+ 1.345 __ipipe_dispatch_event+0xe (__ipipe_syscall_root+0x58) The '*' mark a stalled domain, root domain on the right. It seems to me that xnpod_suspend_thread() leaves the xeno domain stalled on wake up. This gets recovered somehow later. But here we see a different effect which finally causes the tick to get
Re: [Xenomai-core] Proposal to use buildbot for Xenomai
Niklaus Giger wrote: > Am Samstag, 8. April 2006 10:44 schrieb Jan Kiszka: >> Niklaus Giger wrote: >>> Hi everybody > <..> >> This is a really great idea! Of course, I already have another test >> candidate in mind: RTnet 8). Specifically the PPC environment would be >> interesting, as our "buildbot" (sorry, Wolfgang G. ;) ) is typically >> very busy so that build regressions are sometimes only detected with >> delay on that platform. Is it also possible to explicitly trigger an >> update and rebuild? > Yes of course. Just select a buildslave (e.g. ppc or hcu3 -> > http://ngiger.dyndns.org/buildbot/hcu3) and you may manually trigger it. I > intentionally left this facility open for everybody to test it. But it would > of course be possible to restrict it only to a few selected developers. > > If you want, I can help you to setup the master.cfg for a rtnet buildbot. I would love to, but you know - time... > >> But also for Xenomai I would see this as a very useful tool, e.g. for >> 2.4 build tests (I must confess I only test sporadically against this >> kernel). > I will add one. Could you please e-mail me (privately if you want) a working > config (ppc, no cross compiling if possible). Or do you want to add a build > slave under your control? Hmm, I would prefer to just use something, especially as my PPC experiences are quite limited :). But I guess here are qualified people listening who can quickly dig out some config. > <..> >> Is there no reset button you could control via a master station, e.g. by >> attaching some cheap electronic to a parallel or serial port? > There is a reset button, but then you have it running and consuming > electricity all the times. Yep, understandable. >> I just remember that DENX once had or still have a remote PPC test-lab >> running. I CC'ed Wolfgang, maybe he could comment on this if and how it >> could be used. > I would be willing to setup a buildbot for the u-boot, too, as my board boots > with a very outdated version of u-boot. > > Best regards > Jan signature.asc Description: OpenPGP digital signature ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
[Xenomai-core] [PATCH] display domain status in I-ipipe tracer
Hi,
as promised, here is the patch to extend the I-ipipe trace so that it
displays also the domain stall flags of the first 4 domains (I don't
expect more in practice yet ;) ). The information is shown if you switch
on the verbose mode (echo 1 > /proc/ipipe/tracer/verbose). Also, more
explanation of the shown columns is given now.
Please apply.
Jan
---
kernel/ipipe/core.c |4 --
kernel/ipipe/tracer.c | 77 +-
2 files changed, 71 insertions(+), 10 deletions(-)
Index: linux-2.6.15.3-kgdb/kernel/ipipe/core.c
===
--- linux-2.6.15.3-kgdb.orig/kernel/ipipe/core.c
+++ linux-2.6.15.3-kgdb/kernel/ipipe/core.c
@@ -40,7 +40,7 @@ struct ipipe_domain *ipipe_percpu_domain
ipipe_spinlock_t __ipipe_pipelock = IPIPE_SPIN_LOCK_UNLOCKED;
-struct list_head __ipipe_pipeline;
+LIST_HEAD(__ipipe_pipeline);
unsigned long __ipipe_virtual_irq_map = 0;
@@ -66,8 +66,6 @@ void ipipe_init(void)
* secondary CPUs are still lost in space.
*/
- INIT_LIST_HEAD(&__ipipe_pipeline);
-
ipd->name = "Linux";
ipd->domid = IPIPE_ROOT_ID;
ipd->priority = IPIPE_ROOT_PRIO;
Index: linux-2.6.15.3-kgdb/kernel/ipipe/tracer.c
===
--- linux-2.6.15.3-kgdb.orig/kernel/ipipe/tracer.c
+++ linux-2.6.15.3-kgdb/kernel/ipipe/tracer.c
@@ -51,6 +51,7 @@
#define IPIPE_TFLG_HWIRQ_OFF0x0100
#define IPIPE_TFLG_FREEZING 0x0200
+#define IPIPE_TFLG_DOMSTATE_SHIFT 12 /* bits 12..15: domain stalled? */
struct ipipe_trace_point{
@@ -118,6 +119,24 @@ __ipipe_trace_point_type(char *buf, stru
static void __ipipe_print_symname(struct seq_file *m, unsigned long eip);
+static notrace void
+__ipipe_store_domain_states(struct ipipe_trace_point *point, int cpu_id)
+{
+ int i = IPIPE_TFLG_DOMSTATE_SHIFT;
+ struct list_head *pos;
+
+ list_for_each_prev(pos, &__ipipe_pipeline) {
+ struct ipipe_domain *ipd =
+ list_entry(pos, struct ipipe_domain, p_link);
+
+ if (test_bit(IPIPE_STALL_FLAG, &ipd->cpudata[cpu_id].status))
+ point->flags |= (1 << i);
+
+ if (++i > IPIPE_TFLG_DOMSTATE_SHIFT+3)
+ break;
+ }
+}
+
static notrace int __ipipe_get_free_trace_path(int old, int cpu_id)
{
int new_active = old;
@@ -282,6 +301,8 @@ restart:
point->v = v;
ipipe_read_tsc(point->timestamp);
+ __ipipe_store_domain_states(point, cpu_id);
+
/* forward to next point buffer */
next_pos = WRAP_POINT_NO(pos+1);
tp->trace_pos = next_pos;
@@ -617,6 +638,7 @@ __ipipe_print_pathmark(struct seq_file *
{
char mark = ' ';
int point_no = point - print_path->point;
+ int i;
if (print_path->end == point_no)
mark = '<';
@@ -626,6 +648,12 @@ __ipipe_print_pathmark(struct seq_file *
mark = ':';
seq_printf(m, "%c%c", mark,
(point->flags & IPIPE_TFLG_HWIRQ_OFF) ? '|' : ' ');
+
+ if (verbose_trace)
+ for (i = IPIPE_TFLG_DOMSTATE_SHIFT + 3;
+ i >= IPIPE_TFLG_DOMSTATE_SHIFT; i--)
+ seq_printf(m, "%c",
+ (point->flags & (1 << i)) ? '*' : ' ');
}
static void
@@ -695,6 +723,9 @@ static void __ipipe_print_dbgwarning(str
#ifdef CONFIG_XENO_OPT_DEBUG
" o CONFIG_XENO_OPT_DEBUG\n"
#endif /* CONFIG_XENO_OPT_DEBUG */
+#ifdef CONFIG_XENO_OPT_DEBUG_QUEUES
+ " o CONFIG_XENO_OPT_DEBUG_QUEUES (very costly)\n"
+#endif /* CONFIG_XENO_OPT_DEBUG */
#ifdef CONFIG_DEBUG_PREEMPT
" o CONFIG_DEBUG_PREEMPT\n"
#endif /* CONFIG_DEBUG_PREEMPT */
@@ -706,9 +737,44 @@ static void __ipipe_print_dbgwarning(str
static void __ipipe_print_headline(struct seq_file *m)
{
- seq_puts(m, verbose_trace ?
- " Type User Val. TimeDelay Function (Parent)\n" :
- " TypeTime Function (Parent)\n");
+ if (verbose_trace) {
+ const char *name[4] = { [0 ... 3] = "" };
+ struct list_head *pos;
+ int i = 0;
+
+ list_for_each_prev(pos, &__ipipe_pipeline) {
+ struct ipipe_domain *ipd =
+list_entry(pos, struct ipipe_domain, p_link);
+
+ name[i] = ipd->name;
+ if (++i > 3)
+break;
+ }
+
+ seq_printf(m,
+ " +- Hard IRQs ('|': locked)\n"
+ " |+ %s\n"
+ " ||+--- %s\n"
+ " |||+-- %s\n"
+ " +- %s%s\n"
+ " | +-- "
+ "Delay flag ('+': > %d us, '!': > %d us)\n"
+ " | |+- "
+ "NMI noise ('N')\n"
+ " | ||\n"
+ " Type User Val. TimeDelay Function "
+ "(Parent)\n",
+ name[3], name[2], name[1], name[0],
+ name[0] ? " ('*': domain stalled)" : "",
+ IPIPE_DELAY_NOTE/1000, IPIPE_DELAY_WARN/1000);
+ } else
+ seq_printf(m,
+ " +-- Hard IRQs ('|': locked)\n"
+ " |+- Delay flag "
+ "('+': > %d us, '!': > %d us)\n"
+ " ||\n"
+
[Xenomai-core] [PATCH] trigger I-pipe trace freezing via proc
Hi Philippe,
here is a tiny patch to re-trigger trace freezing by writing a positive
number to /proc/ipipe/trace/frozen. Writing 0 provides the old
behaviour, i.e. resets the frozen trace so that ipipe_trace_freeze() can
capture a new trace.
Please apply.
Jan
Index: linux-2.6.15.3-kgdb/kernel/ipipe/tracer.c
===
--- linux-2.6.15.3-kgdb.orig/kernel/ipipe/tracer.c
+++ linux-2.6.15.3-kgdb/kernel/ipipe/tracer.c
@@ -1005,11 +1005,28 @@ static int __ipipe_frozen_prtrace_open(s
}
static ssize_t
-__ipipe_frozen_reset(struct file *file, const char __user *pbuffer,
- size_t count, loff_t *data)
+__ipipe_frozen_ctrl(struct file *file, const char __user *pbuffer,
+size_t count, loff_t *data)
{
+ char *end, buf[16];
+ int val;
+ int n;
+
+ n = (count > sizeof(buf) - 1) ? sizeof(buf) - 1 : count;
+
+ if (copy_from_user(buf, pbuffer, n))
+ return -EFAULT;
+
+ buf[n] = '\0';
+ val = simple_strtol(buf, &end, 0);
+
+ if (((*end != '\0') && !isspace(*end)) || (val < 0))
+ return -EINVAL;
+
down(&out_mutex);
ipipe_trace_frozen_reset();
+ if (val > 0)
+ ipipe_trace_freeze(-1);
up(&out_mutex);
return count;
@@ -1018,7 +1035,7 @@ __ipipe_frozen_reset(struct file *file,
struct file_operations __ipipe_frozen_prtrace_fops = {
.open = __ipipe_frozen_prtrace_open,
.read = seq_read,
- .write = __ipipe_frozen_reset,
+ .write = __ipipe_frozen_ctrl,
.llseek = seq_lseek,
.release= seq_release,
};
signature.asc
Description: OpenPGP digital signature
___
Xenomai-core mailing list
Xenomai-core@gna.org
https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] kgdb over ipipe
Jan Kiszka wrote:
> Hi,
>
> this is the preliminary, though already usable result of my recent
> effort to extend the tool situation for Xenomai: A kgdb patch series for
> 2.6.15 on x86. It already works quite well but likely does not yet catch
> all fatal scenarios (e.g. page faults in the Xenomai domain).
>
And here comes another revision (prepare patch remains unmodified).
It gets closer to what Philippe also just suggested in the original
thread: hook KGDB into I-pipe in favour of registering a dedicated
domain. The latter approach modifies the I-pipe state in a way which may
blur the picture of I-pipe itself to the debugger. This revision hooks
exception events into the I-pipe core so that they are delivered the
normal way when the root domain is active, but get catched early for
higher domains like Xenomai. I'm just not sure about the best way to
handle the serial line IRQ. Philippe, do you see problems with current
approach? Should we better hook into __ipipe_handle_irq (which would
make things more complicated, I'm afraid)?
In contrast to the first version, exceptions happening in the Xenomai
domain now also get reported to KGDB. Debugging mostly works fine, I'm
just facing unknown problems with intercepting and then continuing
kernel-only RT threads. KGDB sometimes reports "E22" back in this case,
but always locks up. Maybe it gets confused by the fact the there is no
Linux task behind Xenomai kernel threads? I tested this by putting a
breakpoint into xnpod_suspend_thread and running latency in mode 0 and
1. 0 works fine, 1 not.
Jan
Index: linux-2.6.15.3-kgdb/kernel/kgdb.c
===
--- linux-2.6.15.3-kgdb.orig/kernel/kgdb.c
+++ linux-2.6.15.3-kgdb/kernel/kgdb.c
@@ -740,7 +740,7 @@ static void kgdb_wait(struct pt_regs *re
unsigned long flags;
int processor;
- local_irq_save(flags);
+ local_irq_save_hw(flags);
processor = smp_processor_id();
kgdb_info[processor].debuggerinfo = regs;
kgdb_info[processor].task = current;
@@ -770,7 +770,7 @@ static void kgdb_wait(struct pt_regs *re
/* Signal the master processor that we are done */
atomic_set(&procindebug[processor], 0);
spin_unlock(&slavecpulocks[processor]);
- local_irq_restore(flags);
+ local_irq_restore_hw(flags);
}
#endif
@@ -1033,7 +1033,7 @@ int kgdb_handle_exception(int ex_vector,
* Interrupts will be restored by the 'trap return' code, except when
* single stepping.
*/
- local_irq_save(flags);
+ local_irq_save_hw(flags);
/* Hold debugger_active */
procid = smp_processor_id();
@@ -1056,7 +1056,7 @@ int kgdb_handle_exception(int ex_vector,
if (atomic_read(&cpu_doing_single_step) != -1 &&
atomic_read(&cpu_doing_single_step) != procid) {
atomic_set(&debugger_active, 0);
- local_irq_restore(flags);
+ local_irq_restore_hw(flags);
goto acquirelock;
}
@@ -1556,7 +1556,7 @@ int kgdb_handle_exception(int ex_vector,
kgdb_restore:
/* Free debugger_active */
atomic_set(&debugger_active, 0);
- local_irq_restore(flags);
+ local_irq_restore_hw(flags);
return error;
}
@@ -1925,9 +1925,9 @@ static int kgdb_notify_reboot(struct not
if (!kgdb_connected || atomic_read(&debugger_active) != 0)
return 0;
if ((code == SYS_RESTART) || (code == SYS_HALT) || (code == SYS_POWER_OFF)){
- local_irq_save(flags);
+ local_irq_save_hw(flags);
put_packet("X00");
- local_irq_restore(flags);
+ local_irq_restore_hw(flags);
}
return NOTIFY_DONE;
}
@@ -1942,9 +1942,9 @@ void kgdb_console_write(struct console *
if (!kgdb_connected || atomic_read(&debugger_active) != 0)
return;
- local_irq_save(flags);
+ local_irq_save_hw(flags);
kgdb_msg_write(s, count);
- local_irq_restore(flags);
+ local_irq_restore_hw(flags);
}
static struct console kgdbcons = {
Index: linux-2.6.15.3-kgdb/drivers/serial/8250_kgdb.c
===
--- linux-2.6.15.3-kgdb.orig/drivers/serial/8250_kgdb.c
+++ linux-2.6.15.3-kgdb/drivers/serial/8250_kgdb.c
@@ -301,6 +301,10 @@ static void __init kgdb8250_late_init(vo
"GDB-stub", current_port) < 0)
printk(KERN_ERR "KGDB failed to request the serial IRQ (%d)\n",
current_port->irq);
+#ifdef CONFIG_IPIPE
+ ipipe_control_irq(current_port->irq, 0,
+ IPIPE_HANDLE_MASK|IPIPE_STICKY_MASK|IPIPE_SYSTEM_MASK);
+#endif /* CONFIG_IPIPE */
}
static __init int kgdb_init_io(void)
Index: linux-2.6.15.3-kgdb/lib/Kconfig.debug
===
--- linux-2.6.15.3-kgdb.orig/lib/Kconfig.debug
+++ linux-2.6.15.3-kgdb/lib/Kconfig.debug
@@ -250,7 +250,7 @@ choice
config KGDB_ONLY_MODULES
bool "KGDB: Use only kernel modules for I/O"
- depends on MODULES
+ depends on MODULES && !IPIPE
help
Use only kernel modules to configure KGDB I/O after the
kernel is booted.
@@ -295,7 +295,7 @@ config KGDB_SIBYTE
endchoice
config KGDBOE
- tristate "KGDB: On ethernet" if !KGDBOE_NOMODULE
+ tristate "KGDB:
[Xenomai-core] Re: [PATCH] display domain status in I-ipipe tracer
Jan Kiszka wrote: Hi, as promised, here is the patch to extend the I-ipipe trace so that it displays also the domain stall flags of the first 4 domains (I don't expect more in practice yet ;) ). The information is shown if you switch on the verbose mode (echo 1 > /proc/ipipe/tracer/verbose). Also, more explanation of the shown columns is given now. Please apply. Applied, thanks. -- Philippe. ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
[Xenomai-core] Re: [PATCH] trigger I-pipe trace freezing via proc
Jan Kiszka wrote: Hi Philippe, here is a tiny patch to re-trigger trace freezing by writing a positive number to /proc/ipipe/trace/frozen. Writing 0 provides the old behaviour, i.e. resets the frozen trace so that ipipe_trace_freeze() can capture a new trace. Please apply. Applied, thanks. -- Philippe. ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
[Xenomai-core] latency -t 1 crashes on SMP.
I tried latency -t 1 on an SMP machines, and observed a very
reproducible crash. Most of the time, the machine locks up
completely. The rest of the time, I get :
Xenomai: suspending kernel thread e7559044 ('timerbench') at 0xb01051f2 after ex
ception #14
And the system remains runnable, but 0xb01051f2 is not a valid kernel
module text address.
The lock up does not seem to be detected by any Xenomai or Linux
debug or watchdog. Only enabling the NMI watchdog seems to
systematically produce the exception 14 instead of the lockup.
I ve tried to put a printk at the beginning of timer_task_proc outer
loop. It get printed once when getting the exception 14, and twice when
getting the lock up.
Any idea where to look ?
--
Gilles Chanteperdrix.
___
Xenomai-core mailing list
Xenomai-core@gna.org
https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] latency -t 1 crashes on SMP.
Gilles Chanteperdrix wrote:
> I tried latency -t 1 on an SMP machines, and observed a very
> reproducible crash. Most of the time, the machine locks up
> completely. The rest of the time, I get :
>
> Xenomai: suspending kernel thread e7559044 ('timerbench') at 0xb01051f2 after
> ex
> ception #14
>
> And the system remains runnable, but 0xb01051f2 is not a valid kernel
> module text address.
>
> The lock up does not seem to be detected by any Xenomai or Linux
> debug or watchdog. Only enabling the NMI watchdog seems to
> systematically produce the exception 14 instead of the lockup.
>
> I ve tried to put a printk at the beginning of timer_task_proc outer
> loop. It get printed once when getting the exception 14, and twice when
> getting the lock up.
>
> Any idea where to look ?
>
Two proposals to collect information:
o give KGDB a try
o instrument the xenomai exception handler with an ipipe_trace_freeze()
(something which should be merged into SVN later)
Jan
signature.asc
Description: OpenPGP digital signature
___
Xenomai-core mailing list
Xenomai-core@gna.org
https://mail.gna.org/listinfo/xenomai-core
[Xenomai-core] GDB 6.x + simulator
The following patch enables GDB 6.x for the simulator. Please give this
a try if you happen to use the Xenosim. TIA,
--- sim/scope/gdbhelper.cc (revision 904)
+++ sim/scope/gdbhelper.cc (working copy)
@@ -423,6 +423,8 @@
char *ibuf = gdb_ibuf.gets(), *estart = gdb_ibuf.gets();
+Tcl_ResetResult(tclInterp);
+
for (;;)
{
if (*ibuf == '\0' || *ibuf == '\n')
@@ -504,7 +506,7 @@
// the contents of the log did not match anything known to
// the caller. We cannot return -1, which value is reserved
// to indicate that the connection with GDB has been lost.
-
+
Tcl_AppendElement(tclInterp,CString(rc2 ? rc2 : nre).gets());
Tcl_AppendElement(tclInterp,matched);
Tcl_AppendElement(tclInterp,Tcl_DStringValue(&gdb_ilog));
Index: sim/scope/tcl/gdb.tcl
===
--- sim/scope/tcl/gdb.tcl (revision 904)
+++ sim/scope/tcl/gdb.tcl (working copy)
@@ -850,8 +850,10 @@
regexp "\[^\"\]+.(\[^\"\]+).*" $matched mvar curfocus
}
-# query stack information
-set rl [gdb:command where ls]
+# query stack information -- auto-limit to the inner last 32
+# frames in order to work-around the issue GDB 6.x has with
+# ucontext(2) driven co-routines.
+set rl [gdb:command "where 32" ls]
set stackinfo [lindex $rl 2]
if {$stackinfo == {}} {
--
Philippe.
___
Xenomai-core mailing list
Xenomai-core@gna.org
https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Proposal to use buildbot for Xenomai
Niklaus Giger wrote: Hi everybody If you point your browser at http://ngiger.dyndns.org/buildbot/ (with ending slash please), you will find a first prototype of the continuos integration tool buildbot (http://buildbot.sourceforge.net/) Great stuff. It proves that it possible to automatically retrieve each revison of the Xenomai and compile it for (at the moment) two targets, a stock PPC and a custom PPC405 board (cross-compiling). At the moment it already is useful for me, but I would like to ask your opinion about its usefulness. Would this be useful for you too? Do you have a special target to propose (other architectures, other target like skins, all xenomai components as modules/built-ins, mvm simulator)? With the growing number of supported archs, not to speak of the two major kernel versions supported (2.4/2.6) for some of them, a tool providing continuous integration is clearly extremely useful, since it's now virtually impossible for all developers to systematically check their work against all possible arch/config combinations. Additionally, setting up a build slave for any particular arch of interest is something users wanting to contribute back could do rather easily, and specifically companies who happen to be happy Xenomai users. AFAIC, the upside would be to be able to produce performance numbers after the builds by running regression tests, so that we could evaluate the impact of core changes almost immediately. -- Philippe. ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Proposal to use buildbot for Xenomai
Jan Kiszka wrote: Niklaus Giger wrote: Am Samstag, 8. April 2006 10:44 schrieb Jan Kiszka: Niklaus Giger wrote: Hi everybody <..> This is a really great idea! Of course, I already have another test candidate in mind: RTnet 8). Specifically the PPC environment would be interesting, as our "buildbot" (sorry, Wolfgang G. ;) ) is typically very busy so that build regressions are sometimes only detected with delay on that platform. Is it also possible to explicitly trigger an update and rebuild? Yes of course. Just select a buildslave (e.g. ppc or hcu3 -> http://ngiger.dyndns.org/buildbot/hcu3) and you may manually trigger it. I intentionally left this facility open for everybody to test it. But it would of course be possible to restrict it only to a few selected developers. If you want, I can help you to setup the master.cfg for a rtnet buildbot. I would love to, but you know - time... But also for Xenomai I would see this as a very useful tool, e.g. for 2.4 build tests (I must confess I only test sporadically against this kernel). I will add one. Could you please e-mail me (privately if you want) a working config (ppc, no cross compiling if possible). Or do you want to add a build slave under your control? Hmm, I would prefer to just use something, especially as my PPC experiences are quite limited :). But I guess here are qualified people listening who can quickly dig out some config. In arch/ppc/configs of the (DENX) 2.4 kernel are plenty of configurations for the PowerPC boards, e.g. TQM8620_defconfig, icecube_5200_defconfig, TQM866M_defconfig, etc. But we need cross compilation for most of the embedded boards. Do you need the configuration with IPIPE and Xenomai options enabled? <..> Is there no reset button you could control via a master station, e.g. by attaching some cheap electronic to a parallel or serial port? There is a reset button, but then you have it running and consuming electricity all the times. Yep, understandable. I just remember that DENX once had or still have a remote PPC test-lab running. I CC'ed Wolfgang, maybe he could comment on this if and how it could be used. I would be willing to setup a buildbot for the u-boot, too, as my board boots with a very outdated version of u-boot. We simply use remote controllable power switches to switch the boards on and off. Wolfgang. ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] GDB 6.x + simulator
Works fine here now with gdb 6.3 and svn trunk version.
Thanks !
--
Bruno
On 4/9/06, Philippe Gerum <[EMAIL PROTECTED]> wrote:
The following patch enables GDB 6.x for the simulator. Please give thisa try if you happen to use the Xenosim. TIA,--- sim/scope/gdbhelper.cc (revision 904)+++ sim/scope/gdbhelper.cc (working copy)
@@ -423,6 +423,8 @@ char *ibuf = gdb_ibuf.gets(), *estart = gdb_ibuf.gets();+Tcl_ResetResult(tclInterp);+ for (;;){if (*ibuf == '\0' || *ibuf == '\n')@@ -504,7 +506,7 @@
//
the contents of the log did not match anything known to//
the caller. We cannot return -1, which value is reserved//
to indicate that the connection with GDB has been lost.-+Tcl_AppendElement(tclInterp,CString(rc2 ? rc2 : nre).gets());Tcl_AppendElement(tclInterp,matched);Tcl_AppendElement(tclInterp,Tcl_DStringValue(&gdb_ilog));
Index: sim/scope/tcl/gdb.tcl===--- sim/scope/tcl/gdb.tcl (revision 904)+++ sim/scope/tcl/gdb.tcl (working copy)@@ -850,8 +850,10 @@
regexp "\[^\"\]+.(\[^\"\]+).*" $matched mvar curfocus }-# query stack information-set rl [gdb:command where ls]+# query stack information -- auto-limit to the inner last 32
+# frames in order to work-around the issue GDB 6.x has with+# ucontext(2) driven co-routines.+set rl [gdb:command "where 32" ls] set stackinfo [lindex $rl 2] if {$stackinfo == {}} {
--Philippe.___Xenomai-core mailing listXenomai-core@gna.org
https://mail.gna.org/listinfo/xenomai-core
___
Xenomai-core mailing list
Xenomai-core@gna.org
https://mail.gna.org/listinfo/xenomai-core
[Xenomai-core] [PATCH] improve XENO_ASSERT verbosity
Hi,
this patch adds xnlogerr to the XENO_ASSERT macro, making the failure
visible so that the user does not have to bother. Furthermore, it adds
panic tracing.
Tested and found useful with latest RTDM debugging, ok to apply?
Jan
Index: include/rtdm/rtdm_driver.h
===
--- include/rtdm/rtdm_driver.h (Revision 793)
+++ include/rtdm/rtdm_driver.h (Arbeitskopie)
@@ -40,6 +40,13 @@
#include
#include
+/* debug support */
+#include
+
+#ifndef CONFIG_XENO_OPT_DEBUG_RTDM
+#define CONFIG_XENO_OPT_DEBUG_RTDM 0
+#endif
+
struct rtdm_dev_context;
@@ -891,6 +898,7 @@ static inline rtdm_task_t *rtdm_task_cur
static inline int rtdm_task_wait_period(void)
{
+XENO_ASSERT(RTDM, !xnpod_unblockable_p(), return -EPERM;);
return xnpod_wait_thread_period(NULL);
}
Index: include/nucleus/assert.h
===
--- include/nucleus/assert.h (Revision 793)
+++ include/nucleus/assert.h (Arbeitskopie)
@@ -22,20 +22,12 @@
#include
-#ifndef CONFIG_XENO_OPT_DEBUG_LEVEL
-#define CONFIG_XENO_OPT_DEBUG_LEVEL 0
-#endif
-
-#if CONFIG_XENO_OPT_DEBUG_LEVEL > 0
-#define XENO_ASSERT(cond,action) do { \
-if (unlikely((cond) != 0)) \
+#define XENO_ASSERT(group,cond,action) do { \
+if (unlikely(CONFIG_XENO_OPT_DEBUG_##group > 0 && !(cond))) \
do { action; } while(0); \
} while(0)
-#else /* CONFIG_XENO_OPT_DEBUG_LEVEL == 0 */
-#define XENO_ASSERT(cond,action) do { } while(0)
-#endif /* CONFIG_XENO_OPT_DEBUG_LEVEL > 0 */
-#define XENO_BUGON(cond) \
-XENO_ASSERT(cond,xnpod_fatal("assertion failed at %s:%d",__FILE__,__LINE__))
+#define XENO_BUGON(group,cond) \
+XENO_ASSERT(group,cond,xnpod_fatal("assertion failed at %s:%d",__FILE__,__LINE__))
#endif /* !_XENO_NUCLEUS_ASSERT_H */
Index: ChangeLog
===
--- ChangeLog (Revision 793)
+++ ChangeLog (Arbeitskopie)
@@ -1,3 +1,13 @@
+2006-03-26 Jan Kiszka <[EMAIL PROTECTED]>
+
+ * ksrc/include/nucleus/assert.h: XENO_ASSERT with built-in
+ activation at compile time.
+
+ * ksrc/skins/rtdm/Kconfig, ksrc/skins/rtdm/Config.in,
+ ksrc/skins/rtdm/drvlib.c, include/rtdm/rtdm_driver.h: Use
+ XENO_ASSERT to check for correct RTDM driver API invocation
+ contexts.
+
2006-03-24 Gilles Chanteperdrix <[EMAIL PROTECTED]>
* ksrc/nucleus/pod.c (xnpod_init): Return immediately with an
Index: ksrc/skins/rtdm/Kconfig
===
--- ksrc/skins/rtdm/Kconfig (Revision 793)
+++ ksrc/skins/rtdm/Kconfig (Arbeitskopie)
@@ -7,3 +7,12 @@ config XENO_SKIN_RTDM
This API skin allows to write real-time drivers against a common
light weight interface in kernel mode, but use them across all other
skins in both kernel and user mode.
+
+config XENO_OPT_DEBUG_RTDM
+bool "RTDM Debugging support"
+depends on XENO_OPT_DEBUG && XENO_SKIN_RTDM
+help
+
+This option activates debugging checks for the RTDM subsystem.
+It is a recommended option for analysing potential issues in RTDM
+drivers. A minor runtime overhead is added.
Index: ksrc/skins/rtdm/drvlib.c
===
--- ksrc/skins/rtdm/drvlib.c (Revision 793)
+++ ksrc/skins/rtdm/drvlib.c (Arbeitskopie)
@@ -279,6 +279,8 @@ void rtdm_task_join_nrt(rtdm_task_t *tas
spl_t s;
+XENO_ASSERT(RTDM, xnpod_root_p(), return;);
+
xnlock_get_irqsave(&nklock, s);
while (!xnthread_test_flags(task, XNZOMBIE)) {
@@ -305,6 +307,9 @@ EXPORT_SYMBOL(rtdm_task_join_nrt);
* - -EINTR is returned if calling task has been unblock by a signal or
* explicitely via rtdm_task_unblock().
*
+ * - -EPERM @e may be returned if an illegal invocation environment is
+ * detected.
+ *
* Environments:
*
* This service can be called from:
@@ -319,6 +324,8 @@ int rtdm_task_sleep(uint64_t delay)
xnthread_t *thread = xnpod_current_thread();
+XENO_ASSERT(RTDM, !xnpod_unblockable_p(), return -EPERM;);
+
xnpod_suspend_thread(thread, XNDELAY, xnpod_ns2ticks(delay), NULL);
return xnthread_test_flags(thread, XNBREAK) ? -EINTR : 0;
@@ -337,6 +344,9 @@ EXPORT_SYMBOL(rtdm_task_sleep);
* - -EINTR is returned if calling task has been unblock by a signal or
* explicitely via rtdm_task_unblock().
*
+ * - -EPERM @e may be returned if an illegal invocation environment is
+ * detected.
+ *
* Environments:
*
* This service can be called from:
@@ -354,6 +364,8 @@ int rtdm_task_sleep_until(uint64_t wakeu
int err = 0;
+XENO_ASSERT(RTDM, !xnpod_unblockable_p(), return -EPERM;);
+
xnlock_get_irqsave(&nklock, s);
delay = xnpod_ns2ticks(wakeup_time) - xnpod_get_time();
@@ -626,6 +638,9 @@ EXPORT_SYMBOL(rtdm_event_signal);
*
* - -EIDRM is returned if @a event has been destroyed.
*
+ * - -EPERM @e may be returned if an illegal invocation environment is
+ * detected.
+ *
* Env
Re: [Xenomai-core] [PATCH] improve XENO_ASSERT verbosity
Jan Kiszka wrote: Hi, this patch adds xnlogerr to the XENO_ASSERT macro, making the failure visible so that the user does not have to bother. Furthermore, it adds panic tracing. Tested and found useful with latest RTDM debugging, ok to apply? Ok. -- Philippe. ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] latency -t 1 crashes on SMP.
Gilles Chanteperdrix wrote:
I tried latency -t 1 on an SMP machines, and observed a very
reproducible crash. Most of the time, the machine locks up
completely. The rest of the time, I get :
Xenomai: suspending kernel thread e7559044 ('timerbench') at 0xb01051f2 after ex
ception #14
And the system remains runnable, but 0xb01051f2 is not a valid kernel
module text address.
Looks like a kernel-based task stack address on x86.
--
Philippe.
___
Xenomai-core mailing list
Xenomai-core@gna.org
https://mail.gna.org/listinfo/xenomai-core
Re: [Adeos-main] Re: [Xenomai-core] kgdb over ipipe
Jan Kiszka wrote: Jan Kiszka wrote: Hi, this is the preliminary, though already usable result of my recent effort to extend the tool situation for Xenomai: A kgdb patch series for 2.6.15 on x86. It already works quite well but likely does not yet catch all fatal scenarios (e.g. page faults in the Xenomai domain). And here comes another revision (prepare patch remains unmodified). It gets closer to what Philippe also just suggested in the original thread: hook KGDB into I-pipe in favour of registering a dedicated domain. The latter approach modifies the I-pipe state in a way which may blur the picture of I-pipe itself to the debugger. This revision hooks exception events into the I-pipe core so that they are delivered the normal way when the root domain is active, but get catched early for higher domains like Xenomai. I'm just not sure about the best way to handle the serial line IRQ. Philippe, do you see problems with current approach? Should we better hook into __ipipe_handle_irq (which would make things more complicated, I'm afraid)? The current approach works fine unless a runaway thread goes wild with interrupts disabled (i.e. stall bit set) in the root stage or in any higher priority domain regardless of the root domain state, in which case the serial IRQ won't make it through the pipeline to KGDB. In contrast to the first version, exceptions happening in the Xenomai domain now also get reported to KGDB. Debugging mostly works fine, I'm just facing unknown problems with intercepting and then continuing kernel-only RT threads. KGDB sometimes reports "E22" back in this case, but always locks up. Maybe it gets confused by the fact the there is no Linux task behind Xenomai kernel threads? I tested this by putting a breakpoint into xnpod_suspend_thread and running latency in mode 0 and 1. 0 works fine, 1 not. KGDB is relying on "current", so it's reading garbage over Xenomai's kernel threads. -- Philippe. ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
