Re: Where to ask about a 7.2 bug, and debugging sys/queue.h errors

2011-08-25 Thread Brandon Gooch 
2011/8/25 Kostik Belousov :
> On Thu, Aug 25, 2011 at 05:12:09PM -0500, Brandon Gooch wrote:
>> On Thu, Aug 25, 2011 at 4:53 PM, Kostik Belousov  wrote:
>> > On Thu, Aug 25, 2011 at 03:16:09PM -0600, Charlie Martin wrote:
>> >> We're having a crash in some internal code running on FreeBSD 7.2
>> >> (specifically  7.2-PRERELEASE FreeBSD 7.2-PRERELEASE and yeah, I know
>> >> it's quite a bit behind) in which after 18-30 hours of running load
>> >> tests, the code panics with:
>> >>
>> >> panic: Bad link elm 0xff0044c09600 next->prev != elm
>> >> cpuid = 0
>> >> KDB: stack backtrace:
>> >> db_trace_self_wrapper() at 0x8019119a = db_trace_self_wrapper+0x2a
>> >> panic() at 0x80307c72 = panic+0x182
>> >> devfs_populate_loop() at 0x802a43a8 = devfs_populate_loop+0x548
>> >>
>> >>
>> >> First question: where's the most appropriate place to ask about this
>> >> kind of bug on a back version.
>> > It is fine to ask there.
>> >
>> >>
>> >> Second: does this remind anyone of any bugs?  Googling came up with a
>> >> few somewhat similar things but hasn't provided much insight so far.
>> > In 99% of the cases, it means that you forgot to dev_ref() some cdev.
>>
>> So dev_ref increments the reference count for a cdev. Even though the
>> work "loop" seems to indicate that we will iterate over a list of
>> objects (one of which we may be missing a reference to via a missing
>> dev_ref()), I'm not seeing how this can cause a panic from inside
>> devfs_populate_loop().
>>
>> Can you help me understand this?
>>
> Missing dev_ref() means that the memory for the cdev (and cdev_priv) is
> freed prematurely. If this happens before destroy_dev() is called,
> then the list which is iterated over by populate_loop(), is corrupted.
>
> See e.g. MAKEDEV_REF flag for make_dev(9) and its use in the (old) clone
> handlers.
>

Ahhh, thanks Kostik. Reading make_dev(9) (and more source code) now...

-Brandon
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Where to ask about a 7.2 bug, and debugging sys/queue.h errors

2011-08-25 Thread Brandon Gooch 
On Thu, Aug 25, 2011 at 4:53 PM, Kostik Belousov  wrote:
> On Thu, Aug 25, 2011 at 03:16:09PM -0600, Charlie Martin wrote:
>> We're having a crash in some internal code running on FreeBSD 7.2
>> (specifically  7.2-PRERELEASE FreeBSD 7.2-PRERELEASE and yeah, I know
>> it's quite a bit behind) in which after 18-30 hours of running load
>> tests, the code panics with:
>>
>> panic: Bad link elm 0xff0044c09600 next->prev != elm
>> cpuid = 0
>> KDB: stack backtrace:
>> db_trace_self_wrapper() at 0x8019119a = db_trace_self_wrapper+0x2a
>> panic() at 0x80307c72 = panic+0x182
>> devfs_populate_loop() at 0x802a43a8 = devfs_populate_loop+0x548
>>
>>
>> First question: where's the most appropriate place to ask about this
>> kind of bug on a back version.
> It is fine to ask there.
>
>>
>> Second: does this remind anyone of any bugs?  Googling came up with a
>> few somewhat similar things but hasn't provided much insight so far.
> In 99% of the cases, it means that you forgot to dev_ref() some cdev.

So dev_ref increments the reference count for a cdev. Even though the
work "loop" seems to indicate that we will iterate over a list of
objects (one of which we may be missing a reference to via a missing
dev_ref()), I'm not seeing how this can cause a panic from inside
devfs_populate_loop().

Can you help me understand this?

-Brandon
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Where to ask about a 7.2 bug, and debugging sys/queue.h errors

2011-08-25 Thread Kip Macy 
On Thu, Aug 25, 2011 at 11:16 PM, Charlie Martin  wrote:
> We're having a crash in some internal code running on FreeBSD 7.2
> (specifically  7.2-PRERELEASE FreeBSD 7.2-PRERELEASE and yeah, I know it's
> quite a bit behind) in which after 18-30 hours of running load tests, the
> code panics with:
>
> panic: Bad link elm 0xff0044c09600 next->prev != elm
> cpuid = 0
> KDB: stack backtrace:
> db_trace_self_wrapper() at 0x8019119a = db_trace_self_wrapper+0x2a
> panic() at 0x80307c72 = panic+0x182
> devfs_populate_loop() at 0x802a43a8 = devfs_populate_loop+0x548
>
>
> First question: where's the most appropriate place to ask about this kind of
> bug on a back version.

Probably -stable. I don't know how many developers are still running
7. Most are on 8 at this point.

> Second: does this remind anyone of any bugs?  Googling came up with a few
> somewhat similar things but hasn't provided much insight so far.

This panic is very common when list updates aren't adequately serialized.

> Third: I tried compiling with the sys/queue.h QUEUE_MACRO_DEBUG defined in
> order to get more useful information from the panic.  The kernel build fails
> in pmap.c when this macro is defined, giving an error saying the CTASSERT
> macro is resolving to a negative array size.  Is there any particular secret
> to using this macro (like, no one goes there any more?)

This is because you are running amd64 and the the pv_entry constants
were defined assuming the default (smaller) list entry structure. I
once fixed this in a local tree, but I think I was so dismayed at the
"obviousness" of the bug I was tracking down that I neglected to
commit the pmap update. It shouldn't be too hard to calculate the
correct constants.

Cheers
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Where to ask about a 7.2 bug, and debugging sys/queue.h errors

2011-08-25 Thread Kostik Belousov 
On Thu, Aug 25, 2011 at 05:12:09PM -0500, Brandon Gooch wrote:
> On Thu, Aug 25, 2011 at 4:53 PM, Kostik Belousov  wrote:
> > On Thu, Aug 25, 2011 at 03:16:09PM -0600, Charlie Martin wrote:
> >> We're having a crash in some internal code running on FreeBSD 7.2
> >> (specifically  7.2-PRERELEASE FreeBSD 7.2-PRERELEASE and yeah, I know
> >> it's quite a bit behind) in which after 18-30 hours of running load
> >> tests, the code panics with:
> >>
> >> panic: Bad link elm 0xff0044c09600 next->prev != elm
> >> cpuid = 0
> >> KDB: stack backtrace:
> >> db_trace_self_wrapper() at 0x8019119a = db_trace_self_wrapper+0x2a
> >> panic() at 0x80307c72 = panic+0x182
> >> devfs_populate_loop() at 0x802a43a8 = devfs_populate_loop+0x548
> >>
> >>
> >> First question: where's the most appropriate place to ask about this
> >> kind of bug on a back version.
> > It is fine to ask there.
> >
> >>
> >> Second: does this remind anyone of any bugs?  Googling came up with a
> >> few somewhat similar things but hasn't provided much insight so far.
> > In 99% of the cases, it means that you forgot to dev_ref() some cdev.
> 
> So dev_ref increments the reference count for a cdev. Even though the
> work "loop" seems to indicate that we will iterate over a list of
> objects (one of which we may be missing a reference to via a missing
> dev_ref()), I'm not seeing how this can cause a panic from inside
> devfs_populate_loop().
> 
> Can you help me understand this?
> 
Missing dev_ref() means that the memory for the cdev (and cdev_priv) is
freed prematurely. If this happens before destroy_dev() is called,
then the list which is iterated over by populate_loop(), is corrupted.

See e.g. MAKEDEV_REF flag for make_dev(9) and its use in the (old) clone
handlers.


pgpWJlf9huRNl.pgp
Description: PGP signature

Re: Where to ask about a 7.2 bug, and debugging sys/queue.h errors

2011-08-25 Thread Kostik Belousov 
On Thu, Aug 25, 2011 at 03:16:09PM -0600, Charlie Martin wrote:
> We're having a crash in some internal code running on FreeBSD 7.2 
> (specifically  7.2-PRERELEASE FreeBSD 7.2-PRERELEASE and yeah, I know 
> it's quite a bit behind) in which after 18-30 hours of running load 
> tests, the code panics with:
> 
> panic: Bad link elm 0xff0044c09600 next->prev != elm
> cpuid = 0
> KDB: stack backtrace:
> db_trace_self_wrapper() at 0x8019119a = db_trace_self_wrapper+0x2a
> panic() at 0x80307c72 = panic+0x182
> devfs_populate_loop() at 0x802a43a8 = devfs_populate_loop+0x548
> 
> 
> First question: where's the most appropriate place to ask about this 
> kind of bug on a back version.
It is fine to ask there.

> 
> Second: does this remind anyone of any bugs?  Googling came up with a 
> few somewhat similar things but hasn't provided much insight so far.
In 99% of the cases, it means that you forgot to dev_ref() some cdev.


pgpG6DLjTvGZh.pgp
Description: PGP signature

Where to ask about a 7.2 bug, and debugging sys/queue.h errors

2011-08-25 Thread Charlie Martin 
We're having a crash in some internal code running on FreeBSD 7.2 
(specifically  7.2-PRERELEASE FreeBSD 7.2-PRERELEASE and yeah, I know 
it's quite a bit behind) in which after 18-30 hours of running load 
tests, the code panics with:


panic: Bad link elm 0xff0044c09600 next->prev != elm
cpuid = 0
KDB: stack backtrace:
db_trace_self_wrapper() at 0x8019119a = db_trace_self_wrapper+0x2a
panic() at 0x80307c72 = panic+0x182
devfs_populate_loop() at 0x802a43a8 = devfs_populate_loop+0x548


First question: where's the most appropriate place to ask about this 
kind of bug on a back version.


Second: does this remind anyone of any bugs?  Googling came up with a 
few somewhat similar things but hasn't provided much insight so far.


Third: I tried compiling with the sys/queue.h QUEUE_MACRO_DEBUG defined 
in order to get more useful information from the panic.  The kernel 
build fails in pmap.c when this macro is defined, giving an error saying 
the CTASSERT macro is resolving to a negative array size.  Is there any 
particular secret to using this macro (like, no one goes there any more?)


Thanks
--

Charles R. (Charlie) Martin
Senior Software Engineer
SGI logo
1900 Pike Road
Longmont, CO 80501
Phone: 303-532-0209
E-Mail: crmar...@sgi.com 
Website: www.sgi.com 

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Concurrent execution of rc-scripts with rcorder(8)

2011-08-25 Thread joris dedieu 
2011/8/24 Vitaly Magerya :
>> the idea to start services concurrently during boot isn't new and the
>> question why FreeBSD doesn't do it has popped up on the forum and
>> mailing list occasionally. So, why not give it a shot?
>
> As someone who uses FreeBSD on hist laptop and is constantly annoyed
> by the lack of suspend-to-disk, every second trimmed of from boot
> time is a win.
>
> In line of the recent FreeBSD problems & solutions discussion, would
> any commiter take time to review and commit this? "FreeBSD 9.1
> introduces concurrent startup, improves boot speed" is the kind of
> buzz we're after.
>
>> Any ideas and feedback are very welcome!
>
> One thing to try is to attach a diagnostics feature that will produce
> data about rc script dependencies and execution times, which can
> be used to visualize which scripts take most time, and how to
> reorganize dependencies to improve boot time (one example I noticed
> is moused: it is only started after network is up, which is a shame,
> since it could easily start while DHCP negotiation is in progress).

Perhaps   background_dhclient="YES" should solve it ? I think
background approach (which is current archlinux one [1] ) is not so
bad. It's clearly  less powerful than automagic parallelization  but
it's maybe less invasive and more flexible for sysadmins.

I gave it a try with a little patch for rc.subr that introduces a
background keyword  (eg: moused_enable="background"). It's surly buggy
with some variables like rc_quiet. I have to check more.

[1] https://wiki.archlinux.org/index.php/DAEMONS

Joris

> ___
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
>
--- /etc/rc.subr	2011-05-02 08:49:11.0 +0200
+++ rc.subr	2011-08-25 13:50:29.300275783 +0200
@@ -142,8 +142,9 @@
 	debug "checkyesno: $1 is set to $_value."
 	case $_value in
 
-		#	"yes", "true", "on", or "1"
-	[Yy][Ee][Ss]|[Tt][Rr][Uu][Ee]|[Oo][Nn]|1)
+		#	"yes", "true", "on", "1", "bg" or "background"
+	[Yy][Ee][Ss]|[Tt][Rr][Uu][Ee]|[Oo][Nn]|1|[Bb][Gg]|\
+[Bb][Aa][Cc][Kk][Gg][Rr][Oo][Uu][Nn][Dd])
 		return 0
 		;;
 
@@ -159,6 +160,25 @@
 }
 
 #
+# checkbg var
+#	Test $1 variable, and return 0 if it's background or bg.
+#	Return nonzero otherwise.
+#
+checkbg()
+{
+	eval _value=\$${1}
+	debug "checkbg: $1 is set to $_value."
+	case $_value in
+	[Bb][Gg]|[Bb][Aa][Cc][Kk][Gg][Rr][Oo][Uu][Nn][Dd])
+		return 0
+		;;
+	*)
+		return 1
+		;;
+	esac
+}
+
+#
 # reverse_list list
 #	print the list in reverse order
 #
@@ -735,54 +755,11 @@
 			;;
 
 		start)
-			if [ -z "$rc_fast" -a -n "$rc_pid" ]; then
-echo 1>&2 "${name} already running? (pid=$rc_pid)."
-return 1
+			if checkbg ${rcvar}; then
+eval _run_rc_start &
+			else 
+_run_rc_start
 			fi
-
-			if [ ! -x ${_chroot}${_chroot:+"/"}${command} ]; then
-warn "run_rc_command: cannot run $command"
-return 1
-			fi
-
-			if ! _run_rc_precmd; then
-warn "failed precmd routine for ${name}"
-return 1
-			fi
-
-	# setup the full command to run
-	#
-			check_startmsgs && echo "Starting ${name}."
-			if [ -n "$_chroot" ]; then
-_doit="\
-${_nice:+nice -n $_nice }\
-chroot ${_user:+-u $_user }${_group:+-g $_group }${_groups:+-G $_groups }\
-$_chroot $command $rc_flags $command_args"
-			else
-_doit="\
-${_chdir:+cd $_chdir && }\
-$command $rc_flags $command_args"
-if [ -n "$_user" ]; then
-_doit="su -m $_user -c 'sh -c \"$_doit\"'"
-fi
-if [ -n "$_nice" ]; then
-	if [ -z "$_user" ]; then
-		_doit="sh -c \"$_doit\""
-	fi
-	_doit="nice -n $_nice $_doit"
-fi
-			fi
-
-	# run the full command
-	#
-			if ! _run_rc_doit "$_doit"; then
-warn "failed to start ${name}"
-return 1
-			fi
-
-	# finally, run postcmd
-	#
-			_run_rc_postcmd
 			;;
 
 		stop)
@@ -985,6 +962,59 @@
 	echo "$_cmd"
 }
 
+_run_rc_start()
+{
+	if [ -z "$rc_fast" -a -n "$rc_pid" ]; then
+		echo 1>&2 "${name} already running? (pid=$rc_pid)."
+		return 1
+	fi
+
+	if [ ! -x ${_chroot}${_chroot:+"/"}${command} ]; then
+		warn "run_rc_command: cannot run $command"
+		return 1
+	fi
+
+	if ! _run_rc_precmd; then
+		warn "failed precmd routine for ${name}"
+		return 1
+	fi
+
+	# setup the full command to run
+	#
+	check_startmsgs && echo "Starting ${name}."
+	if [ -n "$_chroot" ]; then
+_doit="\
+${_nice:+nice -n $_nice }\
+chroot ${_user:+-u $_user }${_group:+-g $_group }${_groups:+-G $_groups }\
+$_chroot $command $rc_flags $command_args"
+			else
+_doit="\
+${_chdir:+cd $_chdir && }\
+$command $rc_flags $command_args"
+		if [ -n "$_user" ]; then
+			_doit="su -m $_user -c 'sh -c \"$_doit\"'"
+		fi
+		if [ -n "$_nice" ]; then
+			if [ -z "$_user" ]; then
+_doit="sh -c \"$_doit\""
+			fi
+			_doit="nice -n $_nice $_doit"
+		fi
+	fi
+
+	# run the full command
+	#
+	if ! _run_rc_doit "$_doit"; then
+		warn

Re: USB Stack Hooking

2011-08-25 Thread Hans Petter Selasky 
On Thursday 25 August 2011 13:04:26 Daniel Grech wrote:
> Hi, I need to write a kernel module that would hook onto some function at a
> low level in the USB stack in order to have direct access to the raw data
> that is received from the USB Controller Hardware. Has this ever been done
> before ? Any ideas as to which function I could hook onto to have access to
> this data ?

Hi,

Recently there was an USB Packet Filter solution added, which can TAP data at 
the most common places.

See /sys/dev/usb/usb_pf.c

And: usbpf_xfertap()

If you want to grab information lower than this, there is no formal API at the 
present moment.

Are you thinking about USB Gadget/Device or USB Host data grabbing?

--HPS
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

USB Stack Hooking

2011-08-25 Thread Daniel Grech 
Hi, I need to write a kernel module that would hook onto some function at a
low level in the USB stack in order to have direct access to the raw data
that is received from the USB Controller Hardware. Has this ever been done
before ? Any ideas as to which function I could hook onto to have access to
this data ?
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"