Need some explanation on a field in struct vmtotal
All,
I'm trying to determine if this is a bug or for real. We have a customer that
pointed his
NMS at our appliance (running FBSD 9.0). These are 64-bit intel platforms (8
core Xeons)
with 8 Gbytes of RAM and a 16 Gbyte swap.
The customer claims that the NMS shows him that these boxes have a tetrabyte of
VM, which
surprised him somewhat. These platforms are using net-snmp 5.7.1 out of the
ports tree.
I did an snmpget query on one of our boxes here in our lab and saw the
following:
comet1# snmpget -v2c -c public localhost hrMemorySize.0
HOST-RESOURCES-MIB::hrMemorySize.0 = INTEGER: 8351368 KBytes
SNMP table: HOST-RESOURCES-MIB::hrStorageTable
hrStorageIndex hrStorageType hrStorageDescr hrStorageAllocationUnits
hrStorageSize hrStorageUsed hrStorageAllocationFailures
...
3 HOST-RESOURCES-TYPES::hrStorageVirtualMemory Virtual memory 4096
Bytes 269040967 268460681 0
My understanding is that the 'hrStorageSize' is in 'hrStorageAllocationUnits',
so the total
byte size should be
4096 x 269040967 = 1085607800832
Now, I have looked at the net-snmp code for retrieving this value and it seems
to be
via a sysctl to vm.vmtotal. This value is the field t_vm in the vmtotal
structure
(defined in sys/sys/vmmeter.h).
Looking at the code in sys/vm/vm_meter.c I see the following:
TAILQ_FOREACH(object, vm_object_list, object_list) {
/*
* Perform unsynchronized reads on the object to avoid
* a lock-order reversal. In this case, the lack of
* synchronization should not impair the accuracy of
* the reported statistics.
*/
if (object-type == OBJT_DEVICE || object-type == OBJT_SG) {
/*
* Devices, like /dev/mem, will badly skew our totals.
*/
continue;
}
if (object-ref_count == 0) {
/*
* Also skip unreferenced objects, including
* vnodes representing mounted file systems.
*/
continue;
}
total.t_vm += object-size;
But I cannot find any description of what object-size is defined. The
vm_object structure is defined in vm/vm_object.h as type vm_pindex_t.
Which is an architecturally defined in machine/_types.h (for amd64 this
is defined as __uint64_t.
My FreeBSD internals (McKusick's book) for 5.5 doesn't seem to reference it.
So my question is - Is the object-size in bytes? If this is so, then
total.t_vm is in bytes and net-snmp is using that value unmodified. Instead
it should divide that value by 4096 and report that number instead?
Or is my understanding screwed up here?
Thanks,
Patrick
Patrick Mahan
Lead Technical Kernel Engineer
Adara Networks
Disclaimer: The opinions expressed here are solely the responsibility of the
author and are not to be
construed as an official opinion of Adara Networks.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Help with building private .iso media
All, Asking this here as I don't see a mailing list specifically devoted to build/release issues and because I think it is a little beyond the standard -question genre. Our product here is based on FreeBSD (currently FreeBSD 9/STABLE) and it has reached the stage where we need to create our own .iso/memstick media to handle installation of our appliances. So yesterday, after perusing release(7), I cranked up /usr/src/release/generate-release.sh and capture the output to see how it was done. And I was successful in getting the .iso, memstick and FTP site created. So taking what I learned yesterday, I tried to apply it to our copy of the FreeBSD source tree. Please note that we use the provided FreeBSD makefiles to build everything we just make heavy use of MAKEOBJDIRPREFIX/OBJTREE/etc environment variables to control where every- thing goes. So today, I tried to build the world package by doing the following at the top of our FreeBSD source tree (after doing a buildworld) - cd /usr/home/pmahan/work/pm_ipr/ipr/src export MAKEOBJDIRPREFIX=/usr/home/pmahan/work/pm_ipr/ipr/amd64/obj sudo -E make TARGET_ARCH=amd64 TARGET=amd64 distrubteworld DIST=/usr/home/pmahan/work/pm_ipr/ipr/amd64/dist It failed with the following: find //usr/home/pmahan/work/pm_ipr/ipr/amd64/dist/doc -empty -delete find: -delete: //usr/home/pmahan/work/pm_ipr/ipr/amd64/dist/doc: relative path potentially not safe But when I check the logs for yesterday's successful run, I see - find //usr/obj/usr/src/release/dist/doc -empty -delete find //usr/obj/usr/src/release/dist/games -empty -delete find //usr/obj/usr/src/release/dist/lib32 -empty -delete sh /usr/src/release/scripts/mm-mtree.sh -m /usr/src/release/.. -F TARGET_ARCH=amd64 TARGET=amd64 -D /usr/obj/usr/src/release/dist/base I read find(1) description about '-delete' and did a quick test that seemed to prove out that attempting to delete a fully qualified path seems to be a no-no. But I don't understand why it did not fail yesterday. I suspect it is pilot error, but I am at a loss to explain why. Any help, steers, slap downs appreciated Thanks, Patrick Patrick Mahan Lead Technical Kernel Engineer Adara Networks Disclaimer: The opinions expressed here are solely the responsibility of the author and are not to be construed as an official opinion of Adara Networks. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
RE: crypto module
-Original Message- From: owner-freebsd-hack...@freebsd.org [mailto:owner-freebsd- hack...@freebsd.org] On Behalf Of aram baghomian Sent: Wednesday, February 01, 2012 6:54 AM To: freebsd-hackers@freebsd.org Subject: crypto module Hi, I want to know which kernel modules keep the information about implemented algorithms in crypto.ko. for example in linux there is xfrm_algo modules that keep the information about these algorithms. No, the algorithms are access via statically allocated structures defined in sys/opencrypto/xform.c Not that there couldn't be one written, but the xforms would need to be dynamically done and have each crypto algorithm register themselves. Patrick Patrick Mahan Lead Technical Kernel Engineer Adara Networks Disclaimer: The opinions expressed here are solely the responsibility of the author and are not to be construed as an official opinion of Adara Networks. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
RE: mtx_init/lock_init and uninitialized struct mtx
-Original Message-
From: owner-freebsd-hack...@freebsd.org [mailto:owner-freebsd-
hack...@freebsd.org] On Behalf Of Dmitry Krivenok
Sent: Thursday, February 24, 2011 7:47 AM
To: freebsd-hackers@freebsd.org
Subject: mtx_init/lock_init and uninitialized struct mtx
Hello Hackers,
Is it allowed to call mtx_init on a mutex defined as an auto variable
and not initialized explicitly, i.e.:
static int foo()
{
struct mtx m; // Uninitialized auto variable, so it's value is
undefined.
mtx_init(m, my_mutex, NULL, MTX_DEF);
...
// Do something
...
mtx_destroy(m);
return 0;
}
I encountered a problem with such code on a kernel compiled with
INVARIANTS option.
The problem is that mtx_init calls lock_init(m-lock_object) and
lock_init, in turn, calls:
79 /* Check for double-init and zero object. */
80 KASSERT(!lock_initalized(lock), (lock \%s\ %p already
initialized,
81 name, lock));
lock_initialized() just checks that a bit is set in lo_flags field of
struct lock_object:
178 #define lock_initalized(lo) ((lo)-lo_flags LO_INITIALIZED)
However, the structure containing this field is never initialized
(neither in mtx_init nor in lock_init).
So, assuming that the mutex was defined as auto variable, the content
of lock_object field of struct mtx
is also undefined:
37 struct mtx {
38 struct lock_object lock_object;/* Common lock
properties. */
39 volatile uintptr_t mtx_lock; /* Owner and flags. */
40 };
In some cases, the initial value of lo_flags _may_ have the
initialized bit set and KASSERT will call panic.
Is it user's responsibility to properly (how exactly?) initialize
struct mtx, e.g.
memset(m, '\0', sizeof(struct mtx));
Or should mtx_init() explicitly initialize all fields of struct mtx?
Thanks in advance!
When dealing with stack variables, I always initialize them to a known state,
(mostly doing as you do above, with memset() though in the kernel I then to
use bzero())
Given that if it is global then it is either in .bss or in .data if it has
been initialized. If it is part of a structure then most often you zero out
the structure right after allocation. So there is an implicit assumption
that the structure will be zero'd before calling mtx_init().
Patrick
Patrick Mahan
Lead Technical Kernel Engineer
Adara Networks
Disclaimer: The opinions expressed here are solely the responsibility of the
author and are not to be
construed as an official opinion of Adara Networks.
--
Sincerely yours, Dmitry V. Krivenok
e-mail: krivenok.dmi...@gmail.com
skype: krivenok_dmitry
jabber: krivenok_dmi...@jabber.ru
icq: 242-526-443
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Help with determining a system hang
Good day, I am running a FreeBSD 8.0 kernel with my code in the kernel that does some deep packet diving. This is mostly working, but I am having occasional system hangs. No response to the console keyboard, stops receiving packets, etc. I have enabled INVARIANTS, WITNESS and WATCHDOG. The watchdog fires (though not always after the 20 sec wait, sometimes it fires immediately). I also have DDB and KDB enabled in the kernel. What is puzzling me is when the watchdog fires and I get the DDB prompt, the first thing I do is list all cpus: 'show allpcpus'. I would expect to see one of the CPUs having something happening, but most of the time all I see is that all of the CPU's are idle. The couple of times this was not true the CPU showed it was in em_handle_que in dev/e1000/if_em.c. But this code is pretty straight forward, though I could see if it would block on reading it's registers. Can anyone give me a suggestion on possible causes? At first I thought that maybe I was having a deadlock issue with my code, but while WITNESS does report a few lock-order reversals, they are not in my code and seem to be false positives. I next looked for some type of resource wait, but cannot find one (or I don't know how to find it). 'show locks' does not show any locks being held. 'show threads' shows almost every thread sitting in an idle state. I am at a loss to explain it. I know it is probably my code that is causing this behavior in some way because I never seen the hang when my code is bypassed. When I do the packet diving, I am getting called in either ip_input() or ip_output() directly. In ip_input() I get called either in the forwarding path or just before calling the upper protocol layer via the protosw. In ip_output(), I get called just before ip_output() deals with IP fragmentation. This is a Intel Xeon that FreeBSD reports as a 8 CPUS (duo core + 4 threads/core). However, I am more experienced in MIPS hardware than Intel. I have not yet dug into the interrupt handling for the Intel in FreeBSD, but it is one of my suspects since the system is not even responding to the console keyboard. This is going to be a learning experience for me :-) Thanks for any and all help, Patrick Mahan ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: odd issues with DDB vs GDB
John Baldwin wrote: On Wednesday, September 15, 2010 8:01:19 pm Patrick Mahan wrote: All, I am trying to debug a system hang occurring on my HP Proliant G6 running some of our kernel software. I am seeing that under certain test loads, the system will hang-up complete, no keyboard, no console, etc. I suspect it is some of the kernel code that I have inherited that contains a lot of locking (lots of data structure, each having their own mutex lock (sleepable)). You need to use 'kgdb' rather than 'gdb' on kernel.debug. Doh! *-( I'm so used to gdb even though I use kgdb for looking at crash dumps. Thanks, Patrick ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
odd issues with DDB vs GDB
All, I am trying to debug a system hang occurring on my HP Proliant G6 running some of our kernel software. I am seeing that under certain test loads, the system will hang-up complete, no keyboard, no console, etc. I suspect it is some of the kernel code that I have inherited that contains a lot of locking (lots of data structure, each having their own mutex lock (sleepable)). I rebuilt the kernel to include the following: options KDB options DDB options GDB options MUTEX_NOINLINE options MUTEX_DEBUG options WITNESS options WITNESS_SKIPSPIN options SW_WATCHDOG # Enable to force us into the debugger on a hang This places me in the kernel DDB debugger. The backtrace show by DDB makes a lot of sense, it is showing we are blocked in _mtx_lock_flags()+0x6f. Great, so I go to enable GDB - db gdb Step to enter the remote GDB backend. db s $T0510:a6f86c80fff*;thread:186c0;#62 gdb kernel.debug Current directory is ~/devel/pm_bz5486/FBSD80REL/amd64/obj/usr/home/pmahan/devel/pm_bz5486/FBSD80REL/src/sys/MPATH/ GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type show copying to see the conditions. There is absolutely no warranty for GDB. Type show warranty for details. This GDB was configured as amd64-marcel-freebsd... (gdb) target remote 10.10.29.111:7028 Remote debugging using 10.10.29.111:7028 0x806cf8a6 in kdb_init () at /usr/home/pmahan/devel/pm_bz5486/FBSD80REL/src/sys/kern/subr_kdb.c:361 warning: Unable to find dynamic linker breakpoint function. GDB will be unable to debug shared library initializers and track explicitly loaded dynamic code. warning: shared library handler failed to enable breakpoint gdb So right away I am somewhat suspicious as it is showing me a completely different entry point. DDB showed Tracing pid 0 tid 100032 td 0xff0002668390 breakpoint() at breakpoint+0x5 kdb_enter() at kdb_enter+0x52 watchdog_fire() at watchdog_fire+0xda hardclock() at hardclock+0x73 lapic_handle_timer() at lapic_handle_timer+0x120 Xtimerint() at Xtimerint+0x8c But GDB is showing the above. A backtrace (bt) in GDB does not show the same stack signature. I have attached the complete log for those who are interested. Is there a reason for the wide difference between DDB and GDB? Am I invoking gdb incorrectly? Thanks for the education, as always! Patrick Debugging a system hang. Enabled watchdog(4) built kernel with KDB, DDB and GDB. I am trying to debug this via remote GDB but what DDB shows for a stack trace and what GDB shows are two seperate animals. External serial port setup with the following in /boot/loader.conf console=comconsole vidconsole comconsole_speed=9600 hint.uart.0.flags=0x90 Serial is accessed via a cyclades ACS console server. 'telnet 10.10.29.111 70XX' where XX is the physical port number. System comes up fine, testing is initiated, eventually the system hangs and the watchdog fires dropping us into DDB - DDB output db trace Tracing pid 0 tid 100032 td 0xff0002668390 breakpoint() at breakpoint+0x5 kdb_enter() at kdb_enter+0x52 watchdog_fire() at watchdog_fire+0xda hardclock() at hardclock+0x73 lapic_handle_timer() at lapic_handle_timer+0x120 Xtimerint() at Xtimerint+0x8c --- interrupt, rip = 0x80688532, rsp = 0xff800011e460, rbp = 0xff800011e4c0 --- _mtx_lock_sleep() at _mtx_lock_sleep+0x92 _mtx_lock_flags() at _mtx_lock_flags+0x6f VCDgetWithIIFremote() at VCDgetWithIIFremote+0x3f ProcessDataPkt() at ProcessDataPkt+0x3dc ip_input() at ip_input+0xa24 netisr_dispatch_src() at netisr_dispatch_src+0xe3 netisr_dispatch() at netisr_dispatch+0x20 gif_input() at gif_input+0x324 in_gif_input() at in_gif_input+0x28f encap4_input() at encap4_input+0x1b8 ip_input() at ip_input+0xd1a netisr_dispatch_src() at netisr_dispatch_src+0xe3 netisr_dispatch() at netisr_dispatch+0x20 ether_demux() at ether_demux+0x1f3 ether_input() at ether_input+0x4ab em_rxeof() at em_rxeof+0x410 em_handle_que() at em_handle_que+0x6f taskqueue_run() at taskqueue_run+0xbb taskqueue_thread_loop() at taskqueue_thread_loop+0x33 fork_exit() at fork_exit+0xba fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xff800011ed30, rbp = 0 --- dbgdb Step to enter the remote GDB backend. dbs ^]enter telnet quit # # Enter the debugger via remote gdb # gdb kernel.debug Current directory is ~/devel/pm_bz5486/FBSD80REL/amd64/obj/usr/home/pmahan/devel/pm_bz5486/FBSD80REL/src/sys/MPATH/ GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type show copying to see the conditions. There is absolutely no warranty for GDB. Type show warranty for details. This GDB was configured as amd64-marcel-freebsd... (gdb) target
[Fwd: Crash dump on HP Proliant G6 broken as of V8.0]
Forwarding for a colleague, Patrick Original Message Subject: Crash dump on HP Proliant G6 broken as of V8.0 Date: Wed, 15 Sep 2010 11:53:16 -0700 From: Paul Heyman phey...@adaranet.com To: freebsd-hackers@freebsd.org freebsd-hackers@freebsd.org CC: Patrick Mahan pma...@adaranet.com References: 32ab5c9615cc494997d9abb1db12783c024c8c5...@sj-exch-1.adaranet.com,32ab5c9615cc494997d9abb1db12783c024c8de...@sj-exch-1.adaranet.com,32ab5c9615cc494997d9abb1db12783c024c8c5...@sj-exch-1.adaranet.com ALL, The crash dump worked fine in V7.3. I am debugging crash dump problem on a HP Proliant G6 which uses a SATA drive connected to a CISS Raid Controller. I have tried this on a x86 box using a non-raid ATA/SATA disk controller and it works well. I noticed that in V8.0 there is a new SCSI operating method. In the v7.3 version there was only CISS_TRANSPORT_METHOD_SIMPLE, but in v8.0 there has been CISS_TRANSPORT_METHOD_PERF method added. These methods have different function calls in ciss_poll_request. The dump comand starts with a call to dadump. This function will setup a struct ccb_scsiio structure. This is done by calling scsi_read_write. Then the meat of dump happens when it calls xpt_polled_action, which manages and simualtes interrupt functionality that is working fine. The disk operations work fine except during a crash dump. I have turned debug on for CISS and CAMDEBUG to debug this problem. In xpt_polled_action (cam_xpt.c) we get past the first polling loop at line 3013, as both devq-send_opening and dev-ccbq.dev_openings are 0 ( 256 and 254 ). But we do get stuck in the second one at line 3025. We eventually time out setting start_ccb-ccb_h.status to CAM_CMD_TIMEOUT. The timeout is set with DA_DEFAULT_TIMEOUT (scsi_da.c) which is set to 60, and is used in the call to scsi_read_write. Here is the debug trace: Dumping 1240 MB: ciss_cam_action_io: XPT_SCSI_IO 0:0:0 ciss_get_request: called ciss_start: post command 150 tag 600 ciss_map_request: called ciss_request_map_helper: called ciss_cam_poll: called ciss_perf_done: completed command 150 ciss_perf_done: completed command 150 ciss_complete: called ciss_unmap_request: called ciss_cam_complete: called _ciss_report_request: called ciss_cam_complete: SCSI_STATUS_OK ciss_release_request: called ciss_complete: called ciss_unmap_request: called ciss0: WARNING: completing non-busy request ciss_cam_complete: called _ciss_report_request: called ciss_cam_complete: SCSI_STATUS_OK . . . . after about 60 seconds ciss0: WARNING: completing non-busy request ciss0: WARNING: completed command with no submitter ciss_unmap_request: called . . . This goes on forever Thanks Paul Paul Heyman phey...@adaranetworks.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Use of printf's inside a kernel thread
Just wondering if there is some constraints to using printf() calls inside a kernel thread (created by kthread_create()). I'm currently trying to debug a worker thread but the printf's are coming out garbled. Not entirely sure why this could be occurring. OS: FreeBSD 7.3 on an Intel Xeon with 8 Gbytes of memory. Any ideas? Thanks, Patrick ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Help with some makefile hackery
Dag-Erling, I am using option 3 below. I have the following in my shell: amd64-kernel.sh #!/bin/sh trap exit 1 1 2 3 15 export SRCROOT=`pwd -P` export MAKEOBJDIRPREFIX=$SRCROOT/../amd64/obj export SRCCONF=$SRCROOT/src.conf # Use our private copy export SECKNOB=-DPRIVATE KERNCONF=TCONF make NO_CLEAN=yes NO_PROFILE=yes NO_MODULES=yes KERNCONF=$KERNCONF buildkernel || exit 1 ... In the top-level makefile I have the following label: src-kernel: src-kernel-tools cd src; ./amd64-kernel.sh 21 | tee build_amd64_kernel.log If there is a build failure with the kernel, it can be seen in the file 'build_amd64_kernel.log'. However, the top-level make file just continues on to the next label as if no error occurs. The reason we are using shell scripts is because of the environment variables that need to be defined and some other house-keeping stuff that I did not include in the above example. Also, I wanted scripts so the developer could just build individual groups without resorting to building everything. Thanks, Patrick Dag-Erling Smørgrav wrote: Patrick Mahan ma...@mahan.org writes: My issue is that if there is a build failure at any point, the status does not seem to be propagated upward. For example, if the kernel fails to build due to incorrect code, the script machine-kernel64.sh stops (verifable by examining the logfile), however, the make will continue to the next target, src-world, and continue building. How do I propagate the status up to the top-level make? Your shell script needs to exit with a non-zero status if it fails. There are several ways to do this. For instance, if your shell script looks like this: #!/bin/sh make TARGET=amd64 kernel-toolchain you can: - prefix make with exec: sh will exec make instead of running it in a subshell, and the exit status will be whatever make returns. - add the following line at the bottom: exit $? which causes the shell script to exit with the same exit status as the last command it ran, in this case make. - append || exit 1 to the the make command line; this will cause the script to exit immediately with status 1 if make fails, otherwise it will continue (in case you want to do something else if make succeeds) - wrap the make command line in an if statement, which allows you do additional work depending on the outcome, and end the failure case with exit 1 or similar - insert the following line at any point between the shebang and the make command line: set -e this will cause sh to terminate the script immediately with a non-zero exit status if any command after the set line fails. However, I don't see the point of using shell scripts in the first place... DES ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Help with some makefile hackery
Oy vey! Good point! Not something I had considered (but should have). Is there a way to propogate that status through the pipe? I am using 'tee' so that the master build (which invokes the top-level make) can also see the make output. But going back and reading the man page on 'sh' shows me that on a pipline, the status is the status of the *last* command. Maybe I should do this instead? src-kernel: src-kernel-tools cd src; ./amd64-kernel.sh 21 build_amd64_kernel.log; \ tail -f build_amd64_kernel.log It is not too clear if the status is the last one in a compound command. I will give that a try. Thanks, Patrick Dag-Erling Smørgrav wrote: Patrick Mahan ma...@mahan.org writes: In the top-level makefile I have the following label: src-kernel: src-kernel-tools cd src; ./amd64-kernel.sh 21 | tee build_amd64_kernel.log If there is a build failure with the kernel, it can be seen in the file 'build_amd64_kernel.log'. However, the top-level make file just continues on to the next label as if no error occurs. Make looks at tee's exit status, not the script's. DES ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Help with some makefile hackery
Jilles,
Thanks for the more complicated example. I am always interested in
shell hacks like these, especially when they involve interesting uses
of file I/O redirection.
The tee is there so that the master build package (a perl script)
that builds not only my groups sources but other sources as well
is required to provide make 'output' for the master build log.
How is the status handled on compound commands?
./amd64-kernel.sh 21 build.log; tail -f build.log
would that obscure the exit status of the shell script as well? The
man page is not really clear on this.
Thanks,
Patrick
Jilles Tjoelker wrote:
On Tue, Jun 22, 2010 at 11:18:43PM -0700, Patrick Mahan wrote:
src-kern-tools:
cd src; ./machine-kernel-toolchain.sh 21 | tee logfile
The pipeline will return the status of 'tee' which is almost always 0.
The exit status of the build script is ignored.
A simple fix is store the status in a file and refer to that afterwards.
Another fix uses a separate file descriptor to pass through the status,
like
{ st=$(
{
{ ./kernel-toolchain.sh 21 3- 4-; echo $? 3; } | tee logfile 4
} 31)
} 41
but this is fairly complicated.
The issue can be sidestepped entirely by sending the output to a file
only; a developer can use tail -f to follow the build if necessary.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Help with some makefile hackery
I have the following senerio - We need to build the kernel and world for multiple platforms architectures (amd64, mips, ppc, etc). Initially we had a set of shell scripts for the following: machine-kernel-toolchain.sh machine-kernel64.sh machine-world64.sh machine-bldpkg.sh Which set the correct environment variables before invoking the FreeBSD makefile under src/. As part of an effort to setup nightly builds and to make it easier for people to build the kernels they needed, I created a top-level makefile that goes something like - all: src-kern-tools src-kernel src-world src-package src-kern-tools: cd src; ./machine-kernel-toolchain.sh 21 | tee logfile src-kernel: src-kern-tools cd src; ./machine-kernel64.sh 21 | tee logfile and so on. My issue is that if there is a build failure at any point, the status does not seem to be propagated upward. For example, if the kernel fails to build due to incorrect code, the script machine-kernel64.sh stops (verifable by examining the logfile), however, the make will continue to the next target, src-world, and continue building. How do I propagate the status up to the top-level make? I've read through the PMake document but nothing jumped out at me... Any pointers would be appreciated. The build miser wants me to create status dot files that he then verifies before moving on (yuck!). Thanks for listening, Patrick ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: How to get data from kernel module ?
Lukáš Czerner wrote: Hi, I am creating a kernel module and I need to get some information from that module. I can do this with ioctl and pass the data to the user space but it seems a bit unpractical to me, because I do not know the amount of the data - it can differ. I do not know of any way to pass a list of structures to the userspace through ioctl - is there any? So my question is, is there any standard way in FreeBSD to do this ? In linux I would probably use the sysfs, but in FreeBSD I can not find anything similar, except just creating some virtual filesystem on my own and obviously this is not what I want to do. You could use ioctl(). You will need to make two calls. Once without the data pointer and once with. Take a look at src/sbin/ifconfig/ifmedia.c Take a look at sysctl(3) in particular, if you have the sources online look at src/usr.bin/netstat/inet.c or src/usr.bin/netstat/route.c. You will need to also look at their corresponding kernel code. Or finally, you could use kvm(3). Good luck, Patrick ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Loading a KLM at boot time (issues) [SOLVED]
We have found the problem, it was pilot error. Thanks to Andriy for the boot verbose suggestion it was what put me on the correct trail. A brief background to set the context: The random.ko is needed to support ssh issues and because for various reasons we could not have it compiled directly in the kernel. As a result, as part of the package's postinstall we append random_load=yes to /boot/loader.conf, after saving a copy of that file for restoration on pkg_delete. Most of our test machines are rack mounted HP Proliants running Intel Xeon processors (DL360), so they are headless on both the vidconsole and comconsole (though we have a monitor and keyboard in the lab that we move around as needed). Yesterday morning, I went over to the affected box to boot it verbosely per Andriy's suggestion. After connecting the monitor and keyboard and power-cycling it, I noticed that I was not getting the familiar boot menu. Instead, it was prompting me to hit [enter] to boot or [any] other key for the boot loader shell. I started investigating why I was not seeing the boot loader menu and determined that the file '/boot/loader.rc' was missing. It turns out that one of the other packages that make up our total product had copied it's own copy of 'loader.rc' to /boot without saving a copy of the original. Thus when the package was deleted, loader.rc was removed and since the that package was not being added back in, we were missing the loader.rc. Copying in a new loader.rc from another 'pure' FreeBSD install fixed the issue and the system properly loaded 'random.ko' at boot. Thanks, Patrick Andriy Gapon wrote: on 18/04/2010 04:36 Patrick Mahan said the following: Nothing popped up when I tried debugging loader.conf in Google. Any suggestions? First, double-check that you really have random.ko in your modules directories. Supposing you don't use nextboot.conf or haven't otherwise modified kernel and modules paths, those should be /boot/kernel and /boot/modules. Make sure that you have a large enough message buffer size and boot with verbose messages enabled. kernel will report pre-loaded modules at the start of boot. Check there for any error reports from kernel linker too. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Loading a KLM at boot time (issues)
I am looking for some guidance and perhaps a method to debug an issue I am having with a KLM. For various reasons, I have to make use of random.ko instead of having it built as part of the kernel. As a result, I am adding the following line to /boot/loader.conf - random_load=yes # I have also tried random_load=YES On some platforms (Intel Xeon L5518 running FBSD 8.0-release-p2) this works as expected and a kldstat shows random.ko as loaded. However, on other platforms (Intel Xeon E5440 running FBSD 8.0-release-p2) this is not working. random.ko does not get loaded and as a result SSHD will not start which causes havoc with QA/Release engineering. I cannot figure this one out. I don't see any messages on the console indicating that it has failed to load. But sshd complains about not finding the /dev/random and logging in and doing a kldstat shows that random.ko is not there. However, it is there because I can do a 'kldload random; /etc/rc.d/sshd start' and I get SSHD up and running. Nothing popped up when I tried debugging loader.conf in Google. Any suggestions? Thanks, Patrick ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: KLM unloading on error
*smack* sorry, should have posted it. I am running FreeBSD 8.0 Release p2 (amd64). Thanks, Patrick Andrey V. Elsukov wrote: On 13.04.2010 4:56, Patrick Mahan wrote: I have a requirement that when a KLM I am building fails during MOD_LOAD, it should be unloaded. Currently, I am seeing even if I return an error (eg. EINVAL or ENOSYS) the KLM is still shown as loaded (using kldstat). Instead, I would like it to fail and be removed. Does FreeBSD support this? Hi Patrick. It seems it is bug. I have the same problem in 9-CURRENT, which FreeBSD version do you have? ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
KLM unloading on error
All, I have a requirement that when a KLM I am building fails during MOD_LOAD, it should be unloaded. Currently, I am seeing even if I return an error (eg. EINVAL or ENOSYS) the KLM is still shown as loaded (using kldstat). Instead, I would like it to fail and be removed. Does FreeBSD support this? Thanks, Patrick ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Modifying ELF files
In my job, we are producing applications and KLM's for our product that require them to be signed so that our installer will recognize and validate our images. The signature is stored in each app as unsigned char signature[40] __attribute__((section(.compsign))); What I need to do is open the file for writing, locate the .compsign section and stuff in the signature, write it out and close the file. (simple ELF manipulation) An 'ls -l' shows the following: % ls compklm.ko -rw-r--r-- 1 pmahan pmahan 125296 Apr 6 22:50 /home/pmahan/temp/compklm.ko When I try to run my program ./signfile --signature=A203239897C8EB360D1EB2C84E8E77B16E5B7C9A compklm.ko open: Text file busy Googling and looking at the kernel sources, it seems that it detects this file contains 'shared text', that is, it is an executable file and does not allow me to open it for writing. I understand (from my google search) this is a means to keep you from shooting yourself in the foot. But there has got to be a way and I really don't want to grovel through the compiler code to find it. I looked at using libelf.so but it also requires that the file be open for writing. So I am kinda of stuck. If I cannot find a quick solution we might need to do all of our signing on our FC11 box which does not have this issue. Thanks for the education I always get from this list, Patrick ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Modifying ELF files
On Thu, 8 Apr 2010, Patrick Mahan wrote: In my job, we are producing applications and KLM's for our product that require them to be signed so that our installer will recognize and validate our images. The signature is stored in each app as unsigned char signature[40] __attribute__((section(.compsign))); What I need to do is open the file for writing, locate the .compsign section and stuff in the signature, write it out and close the file. (simple ELF manipulation) An 'ls -l' shows the following: % ls compklm.ko -rw-r--r-- 1 pmahan pmahan 125296 Apr 6 22:50 /home/pmahan/temp/compklm.ko When I try to run my program ./signfile --signature=A203239897C8EB360D1EB2C84E8E77B16E5B7C9A compklm.ko open: Text file busy Googling and looking at the kernel sources, it seems that it detects this file contains 'shared text', that is, it is an executable file and does not allow me to open it for writing. My understanding was that ETXTBSY occurs when you attempt to open for writing a file which is actually being executed, i.e. is mapped into some process. I'm not aware that open(2) actually looks at the file itself to see if it is an executable; that would be very surprising to me. What does fstat -m compklm.ko say? % fstat -m compklm.ko USER CMD PID FD MOUNT INUM MODE SZ|DV R/W NAME % What happens if you cp compklm.ko foo.ko and try to sign foo.ko? You should then be able to do mv foo.ko compklm.ko; if compklm.ko is in fact mapped into some process, it will continue to use the original version, which will be kept around (invisibly) until all mappings go away. This is what compilers, install(8), etc, normally do. Does your signfile program do anything with the target file before open(..., O_RDWR)? I've just found my problem. We have a wrapper program that basically handles parsing command line options and is suppose to adjust the argv[] array so that it only contains the remaining non-option targets starting at index zero. So I am doing 'open(argv[0], O_RDWR, 0)' expecting it to be the .ko file. Turns out it was not operating as described (whipping post to be erected later); so argv[0] actually pointed at the operating program, not the first target past the cmd line options. *-) Mystery solved. Thanks, Patrick ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Using __attribute__((weak)) in the kernel
All, Has anyone used the weak function attribute in the kernel? I have some modules providing APIs that I want to allow them to be either loaded via kldload or directly built in the kernel. However, the API functions are being referenced by code that cannot (at this time) be moved out of the kernel. If I understand the use of the weak attribute, I should be able to define these functions as weak and they should be over-written either by the real code functions at kernel link or when kldload() is called. Or am I mis-understanding the use of this attribute? Thanks, Patrick ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Need a build target
On Sunday 07 March 2010 3:57:43 pm Patrick Mahan wrote: All, Just wanted some confirmation, but I have a need where I need to build just the kernel toolchain, kernel and the full toolchain and include files. The first two I can do via 'make kernel-toolchain' and 'make buildkernel'. However, to get the complete build toolchain and include files, suitable for building user applications, I still need to do a full 'buildworld'. Perhaps 'make toolchain' would work? You can then use 'make buildenv' to do interactive builds. I think you can use SUBDIR_OVERRIDE with 'make buildworld' to build specific subdirectories. I believe you are correct. This should build exactly what I need. As always, this list helps me when I'm stuck, Patrick ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Need a build target
All,
Just wanted some confirmation, but I have a need where I need
to build just the kernel toolchain, kernel and the full toolchain
and include files. The first two I can do via 'make kernel-toolchain'
and 'make buildkernel'. However, to get the complete build toolchain
and include files, suitable for building user applications, I still
need to do a full 'buildworld'.
As you are probably inferring, this is for a cross-compile environment.
I have looked through both src/Makefile and src/Makefile.inc1 and cannot
see a target that will give me that capability.
I think I need to a target to do this for me.
WTOOLSMAKE_TGTS=
.if !defined(SUBDIR_OVERRIDE)
WTOOLSMAKE_TGTS+= _worldtmp _legacy _bootstrap-tools
.endif
WTOOLSMAKE_TGTS+= _cleanobj _obj _build_tools
.if !defined(SUBDIR_OVERRIDE)
WTOOLSMAKE_TGTS+= _cross-tools
.endif
WTOOLSMAKE_TGTS+= _includes _libraries
buildworldtools: buildwtools_prologue ${WTOOLSMAKE_TGTS} buildwtools_epilogue
.ORDER buildwtools_prologue ${WTOOLSMAKE_TGTS} buildwtools_epilogue
buildwtools_prologue:
@echo --
@echo World build tools started on `LC_ALL=C date`
@echo --
buildwtools_epilogue:
@echo --
@echo World build tools started on `LC_ALL=C date`
@echo --
This should then build the complete set of binutils, compiler, libraries and
include files, yes?
Thanks for any direction,
Patrick
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Building FreeBSD on a linux FC11 box.
Hopefully, this is not too ignorant a question. But has anyone every built the FreeBSD sources, both kernel and apps, on a linux platform? I did a google on 'cross-compile freebsd (linux)' and found (mostly) discussions regarding building the linux apps for FreeBSD, but not for building a FreeBSD kernel and world. There was a brief discussion on the mailing list back in 2005, but the poster never reported if he had success or failure. Also, there were discussions about building freebsd amd64 on a freebsd i386. I have done lots of development in linux using cross-compilers (mips embedded systems, powerpc, etc). But not with FreeBSD. I know I'll need to use 'bsdmake' no gnu 'make'. The other worry is kernel-toolchain target. I guess I'll just dive in and swim around and see what happens. Thanks for any pointers, Patrick ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
