Re: CFT: BSD-licensed grep [Fwd: cvs commit: ports/textproc/bsdgrep Makefile distinfo]

[Diomidis Spinellis]

Gabor Kovesdan wrote:
In case of sort, I understarnd that it should 
explicitly handle wide characters due to the different alphabet of the 
different languages and yes, that seems to be a difficult task...


Note that Konrad Jankowski in another SoC project is adding to our C 
library support for the Unicode collation algorithm, and importing the 
corresponding language-specific collation tables.


___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: CFT: BSD-licensed grep [Fwd: cvs commit: ports/textproc/bsdgrep Makefile distinfo]

[Diomidis Spinellis]

Doug Barton wrote:
I use the following construct in portmaster, where pdb=/var/db/pkg, 
origin is set to the origin of a given port, and ro_opd is usually 
empty, but can be another origin directory or the same one. To guarantee 
that you should get some kind of results you can test with 
origin=devel/gettext.


egrep -l "DEPORIGIN:($origin|$ro_opd)$" $pdb/*/+CONTENTS

Obviously this works in portmaster with the gnu grep, but if ro_opd is 
unset with the bsd grep I get:


egrep: empty (sub)expression


To avoid these problems I had proposed to instrument getopt to write 
options passed through argv in a file, build all our ports, and look at 
the options used.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Security Flaw in Popular Disk Encryption Technologies

[Diomidis Spinellis]

Pieter de Boer wrote:

Atom Smasher wrote:

article below. does anyone know how this affects eli/geli?

from the geli man page: "detach - Detach the given providers, which 
means remove the devfs entry and clear the keys from memory." does 
that mean that geli properly wipes keys from RAM when a laptop is 
turned off?



The attack you're referencing is carried out by cold rebooting a system. 
Simply put: pull power cord, insert power cord. The volumes are never 
detached, as the shutdown sequence is never run.


This attack has to be defended against in hardware; it exploits a 
'feature' of modern day RAM chips, which can not be controlled by 
software. Anything that is in RAM when the attack is carried out, will 
be compromised. As encrypted volumes simply require keys to be in memory 
to be able to use the volumes, the encryption software is vulnerable to 
this attack. I see no reason why GELI/GBDE wouldn't be affected.


A possible counter-measure would be to add wiping features to the RAM 
modules themselves. When power is lost, the memory could wipe itself. 
Still not perfect, but would certainly help.




There are (at least) two attack methods.  One is indeed based on memory 
chips that retain their data after a shutdown, and software can't easily 
protect against this attack.  The only way I can think would be to 
reserve a processor register for storing the encryption key.


However, a second attack method is based on rebooting a hibernated 
machine with a different operating system and retrieving the encryption 
keys from the hybernation dump.  One can protect against this attack by 
having the hybernation sequence unmount the encrypted filesystems and 
wipe out the keys from memory.


Diomidis Spinellis - http://www.spinellis.gr
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: FreeBSD on non-fpu device

[Diomidis Spinellis]

On Jan 20, 2008, at 3:42 PM, Thierry Herbelot wrote:

Le Sunday 20 January 2008, Peter Jeremy a écrit :

On Sun, Jan 20, 2008 at 02:33:14AM +0200, Heikki Suonsivu wrote:
Why anyone would be interested?  The eBOX 2300SX cpu is a SoC  
device,
which is by far the lowest power consumption I have seen, and  
this is
probably cheapest computer on the market, still suitable for  
number of

applications. While it is not exactly a speed daemon, it works well
enough for all kinds of mp3-playing, small servers and control
applications.


I very much doubt MP3 playing is going to work without an FPU.


mad is a good (integer-only) candidate for MP3 rendering with a  
slow CPU


True.  I'm using it on the non-FPU DNARD Shark (StrongArm SA-110 CPU  
running at 233 MHz) and it works fine.


Diomidis Spinellis - 
http://www.spinellis.gr___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: BSD license compatible hash algorithm?

[Diomidis Spinellis]

Garrett Cooper wrote:

On Dec 27, 2007, at 4:30 PM, Garrett Cooper wrote:


Hi all,
Just wondering if anyone knew of a good BSD license compatible 
key-based hash placement / retrieval algorithm that was available 
anywhere.
I'm looking for a reliable way to lookup objects to see if a given 
action would be performed in my revised pkg_install(1), to thus 
efficiently pre-plan out the installation dependencies and fully 
utilize multiprocessing capabilities of contemporary machines / 
eliminate duplicate dependency install requirements.
I know I can use tree structures or hash(3), but I want to avoid 
trees (inefficient with large data sets of course) and I was looking 
for a non-BDB based solution (for right now, with this given structure 
as I don't want to write everything to disk).


You can pass NULL as a file name to dbopen, and thereby not store the 
results permanently in a file.  BDB will still use a temporary file 
internally, but this will be invisible to your users.


Diomidis Spinellis - http://www.spinellis.gr
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Some diffs

[Diomidis Spinellis]

M. Warner Losh wrote:

And the last, rather long, patch converts the .y in config to a form
that more versions of yacc would grok.

Comments?

[...]

-   INCLUDE ID SEMICOLON
- = {
+   INCLUDE ID SEMICOLON {

[...]

The = { way for specifying yacc actions is item 5 in yacc's Appendix D 
titled "Old Features Supported but not Encouraged" in the Seventh 
Edition Unix Programmer's Manual (January, 1979).  I believe it's now 
time to make the switch.  Thanks!


--
Diomidis Spinellis - http://www.spinellis.gr
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

FreeBSD OLPC

[Diomidis Spinellis]

Is anybody working on running FreeBSD on the One Laptop Per Child 
platform <http://www.laptop.org>?  I'd be interested to try it, but I 
wouldn't want to duplicate work.  The only thing I've found with a web 
search are some pictures of an OLPC at BSDCan 
<http://www.zabbadoz.net/users/bz/BSDCan2007/BSDCan2007-public/>.  The 
first stumbling block would be booting with OLPC's OFW.


Diomidis Spinellis - http://www.spinellis.gr
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Process accounting changes

[Diomidis Spinellis]

The accounting changes discussed in arch@ about a month ago are now part 
of the CURRENT tree.  The changes increase the precision of the time 
values stored in acct(5) from 15ms (which for most commands was too 
large) to 1 microsecond.  The userland programs sa(8) and lastcomm(1) 
provide backward compatibility with existing record and file formats 
(including accounting files including both old and new records); 
regression tests are included to verify this.  The changes have been 
tested on i386, amd64, and (partly) sparc64; please drop me an email if 
you suspect a problem with your particular setup.


Many thanks to Garance A Drosehn, Bruce Evans, Peter Jeremy, 
Poul-Henning Kamp, Sam Leffler, Rick C. Petty, Erik Trulsson, and Robert 
Watson for their earlier comments; to Eric Anderson, Carl Johan 
Gustavsson, Larry Rosenman, and Derek Tattersall for responding to my 
hackers@ request with test data, and especially to Larry Rosenman for 
arranging access to an additional test machine.


Diomidis Spinellis - http://www.spinellis.gr
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Accounting test data

[Diomidis Spinellis]

I'm testing the backwards compatibility of the process accounting 
processing tools (sa(8) and lastcomm(1)) with the upcoming new acct(5) 
record format.  If you have root access on a FreeBSD AMD64, Sparc64, 
ia64, or PowerPC machine please run the shell script 
http://www.spinellis.gr/FreeBSD/values.sh and post the resulting file as 
a reply to this message (this will avoid replicated effort).  The script 
generates a small process accounting file containing the results of a 
dozen simple commands.  It takes less than 10s to run.  If your machine 
runs with process accounting enabled, you will need to re-enable it 
after the script terminates (the script will tell you how).  I intend to 
make the data you send me part of the process accounting regression testing.


Many thanks,

Diomidis Spinellis - http://people.freebsd.org/~dds/
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Writing to a file

[Diomidis Spinellis]

On May 5, 2007, at 10:29 PM, Diomidis Spinellis wrote:

On May 5, 2007, at 9:34 PM, Sonja Milicic wrote:
I'm working on an IO logging utility for FreeBSD as my GSoC  
project, and
I have some questions about writing a kernel functions that would  
open

an existing or create a new file (with the file name as a parameter,
returns a vnode * for the file) and write data to that file (with
pointer to data as parameter). I've found some functions in existing
code that do similar things and might help me understand how to  
solve my
problem, but as there isn't much documentation out there I still  
don't
understand a lot of things. So, could anyone please give me a  
detailed
explanation of how to open a file in kernel and write to it - best  
data
types to use, functions, what to look out for, maybe a link to  
tutorial

or manual that deals with this (if such a thing exists), etc.?


A good strategy for dealing with such questions is to look for code  
that does a task similar to the one you want to implement.  Two  
kernel subsystems that come to my mind is the kernel logging  
facility, which writes data to a user space process via a socket,  
and the process accounting facility, which writes data to an  
already opened file.  There are reasons (performance, flexibility)  
why these two facilities have been designed in this way, and it  
would be a good idea to see whether some of their design decisions  
are also applicable to your problem.


Some clarifications on the above.  You can find the kernel-side of  
the accounting code at

/usr/src/sys/kern/kern_acct.c

acct opens an existing file for appending; acct_process (look for  
vn_rdwr) will write to that file.


Similarly, you can find the kernel-side of the system log code at / 
usr/src/sys/kern/subr_prf.c.  The userland client, which actually  
writes the message buffers to files, is at /usr/src/usr.sbin/syslogd.


In general, coding in the kernel environment is tricky.  You have to  
be careful with locking, many standard C facilities are missing, and  
bugs can bring down the entire system.  Therefore, it is often better  
to split complex tasks into two: a simple part in the kernel will  
communicate with a userland process, where you can put all the  
complexity.  Another example of this pattern is the routing code.


Diomidis Spinellis - http://www.spinellis.gr
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Writing to a file

[Diomidis Spinellis]

On May 5, 2007, at 9:34 PM, Sonja Milicic wrote:
I'm working on an IO logging utility for FreeBSD as my GSoC  
project, and

I have some questions about writing a kernel functions that would open
an existing or create a new file (with the file name as a parameter,
returns a vnode * for the file) and write data to that file (with
pointer to data as parameter). I've found some functions in existing
code that do similar things and might help me understand how to  
solve my

problem, but as there isn't much documentation out there I still don't
understand a lot of things. So, could anyone please give me a detailed
explanation of how to open a file in kernel and write to it - best  
data
types to use, functions, what to look out for, maybe a link to  
tutorial

or manual that deals with this (if such a thing exists), etc.?


A good strategy for dealing with such questions is to look for code  
that does a task similar to the one you want to implement.  Two  
kernel subsystems that come to my mind is the kernel logging  
facility, which writes data to a user space process via a socket, and  
the process accounting facility, which writes data to an already  
opened file.  There are reasons (performance, flexibility) why these  
two facilities have been designed in this way, and it would be a good  
idea to see whether some of their design decisions are also  
applicable to your problem.


Diomidis Spinellis - http://www.spinellis.gr
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: extern "C" and undefined reference

[Diomidis Spinellis]

Andrew Pantyukhin wrote:

7zip developers converted some code from C++ to C,
while leaving the main stand-alone lzma app in C++.
They use 'extern "C" { }' blocks around #include's
referencing C headers.

Everything compiles fine, but "undefined reference"
errors appear at linkage. The undefined references
are to the C functions included from withing those
'extern "C"' wrappers. I tried to remove the
wrappers from some files and the amount of errors
decreased a bit. Is there a better workaround?

Google came up with two results: remove the wrappers
or use c++ instead of cc. I'm already using c++.


Many of the C headers are available in a different form for C++ 
programs.  For example, in a C++ program you can #include  
instead of .


You can use nm(1) on the .o files to see where the problem comes from.

Diomidis
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: sed -i

[Diomidis Spinellis]

Yar Tikhiy wrote:

May I take a bit more of your time?

I've started playing with the code and noticed another gray area.
Namely a `c' command won't print the text if having 2 addresses
with the 2nd address beyond the actual end of file.  For example:

%%% BEGIN
$ jot 9 |

sed '6,15c\
text
' test

1
2
3
4
5
%%% END

Ditto with RE's, if the 2nd RE doesn't match any line:

%%% BEGIN
$ jot 9 | sed '/6/,/15/c\

text
' test

1
2
3
4
5
%%% END

If we've started to delete the pattern space, we should print the
text in place of it because `c' is for `change'.  BSD and GNU seds
have this bug, but Solaris sed doesn't have it.  Do you think we
should fix it, too?

Thanks!



Yes, it sounds like a bug to me.  Note that a couple of weeks ago I 
added the regression tests I originally wrote in 1992 to the FreeBSD 
regression test suite.  Given that you are writing test cases it would 
be very nice to follow the format used in 
src/tools/regression/usr.bin/sed/regress.t, so that we can add them to 
the test suite.


Diomidis
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: sed -i

[Diomidis Spinellis]

Yar Tikhiy wrote:

Hi,

Recently noticed that our sed(1) differs from its GNU analog in
that in -i mode it considers all files as a single sequence of lines
while the latter treats each file independently.  The in-line mode
isn't in POSIX, so it isn't really clear which way is correct.

Here is a couple of practical consequences:

- our sed won't act on a numeric range of lines in each file,
  as in: sed -i '' 2,5d *, which may be counter-intuitive.
- our sed's line ranges can span file boundaries in -i mode.

If the second feature isn't important, I think we should use
a separate line space for each file edited in-line, which is
usually desired.

Comments?

P.S. Attached are a test script and outputs from it for our
sed and GNU sed as found in a Linux I have access to.



I believe the GNU interpretation of lines in -i makes sense.

Diomidis - dds@
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Filesystem layering

[Diomidis Spinellis]

John Baldwin wrote:

On Wednesday 03 January 2007 12:04, Diomidis Spinellis wrote:

static int
umap_bypass(ap)
 struct vop_generic_args /* {
 struct vnodeop_desc *a_desc;
 
 } */ *ap;
{
/* ... */


In this magic code here this function changes the vnode pointers in
ap.


I presume you mean the part starting with the comment:

/*
 * Map the vnodes going in.
 * Later, we'll invoke the operation based on
 * the first mapped vnode's operation vector.
 */

Thanks!  I don't think I'd guess it by reading the code; I was looking 
for a a simple assignment or indirection.  My next plan was to add 
debugging statements along the way, which would have been quite painful.


Diomidis
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Filesystem layering

[Diomidis Spinellis]

I've been puzzling for a couple of days to understand how the code moves 
from one layer to another in the FreeeBSD stacked filesystems, but I'm 
stuck.  I've scrutinized the code, and I've read carefully pp. 254-257 
of the "Design and Implementation of the FreeBSD Operating System", but 
one detail still escapes me.  When is control passed from one layer with 
a bypass function to a lower one?


Consider a case when umapfs is layered on top of another filesystem.  As 
far as I can tell, the following excerpted sequence of function calls 
will end up continuously calling VOP_READ_APV passing umap_vnodeop as 
its first argument.  Obviously somewhere somewhere the cycle breaks, but 
I can't see where.


static int
vn_read(fp, uio, active_cred, flags, td)
struct file *fp;
struct uio *uio;
struct ucred *active_cred;
struct thread *td;
int flags;
{
struct vnode *vp;
vp = fp->f_vnode;/* dds: vp->v_op == umap_vnodeops */
error = VOP_READ(vp, uio, ioflag, fp->f_cred);
}

static __inline int VOP_READ(
struct vnode *vp,
struct uio *uio,
int ioflag,
struct ucred *cred)
{
struct vop_read_args a;

a.a_gen.a_desc = &vop_read_desc;
a.a_vp = vp;/* dds: a.a_vp->v_op == umap_vnodeops */
a.a_uio = uio;
a.a_ioflag = ioflag;
a.a_cred = cred;
/* dds: vp->v_op == umap_vnodeops */
return (VOP_READ_APV(vp->v_op, &a));
}

int
VOP_READ_APV(struct vop_vector *vop, struct vop_read_args *a)
{
int rc;

/* dds: vop->vop_bypass == umap_bypass; while body skipped */
while(vop != NULL &&
vop->vop_read == NULL && vop->vop_bypass == NULL)
vop = vop->vop_default;
if (vop->vop_read != NULL)
rc = vop->vop_read(a);
else
rc = vop->vop_bypass(&a->a_gen); /* dds: this is executed */
}

static int
umap_bypass(ap)
struct vop_generic_args /* {
struct vnodeop_desc *a_desc;

} */ *ap;
{
/* ... */
error = VCALL(ap);  /* dds: ap->a_desc == &vop_read_desc */
}

#define VCALL(c) ((c)->a_desc->vdesc_call(c))

/* dds: vdesc_call == VOP_READ_AP */

int
VOP_READ_AP(struct vop_read_args *a)
{
/* dds: a->a_vp->v_op still == umap_vnodeops  */
return(VOP_READ_APV(a->a_vp->v_op, a));
}

/* dds: again VOP_READ_APV with the same arguments? */
int
VOP_READ_APV(struct vop_vector *vop, struct vop_read_args *a)
{
int rc;

while(vop != NULL &&
vop->vop_read == NULL && vop->vop_bypass == NULL)
    vop = vop->vop_default;
if (vop->vop_read != NULL)
rc = vop->vop_read(a);
else
rc = vop->vop_bypass(&a->a_gen);
}

Diomidis Spinellis - http://www.spinellis.gr
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Call for testers: Replace GNU gzip with NetBSD's gzip implementation

[Diomidis Spinellis]

LI Xin wrote:

Here is a patchset that replaces the GNU gzip with NetBSD's gzip
implementation, which uses zlib to do actual compress/decompress operation:

Go to your src/usr.bin and execute the following shar archive:

http://people.freebsd.org/~delphij/for_review/bsd_gzip/shar-bsd-gzip-20061009

Then, go to src/ and apply the following patch:

http://people.freebsd.org/~delphij/for_review/bsd_gzip/patch-remove-gnu-gzip-01

Currently this gzip implementation should work as a drop-in replacement
for GNU gzip.  I am still working on some style issues here, etc., so
the current focus is functional test.

Any sort of comments are welcome.


The file zuncompress.c is very closely related to
usr.bin/compress/zopen.c.  Both are derived from work I did in 1992 to 
package the compress(1) functionality into a library.  Maybe the time is 
now ripe to actually move the compression/decompression code into a 
reusable library component.


Diomidis - dds@
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Code Quality and FreeBSD

[Diomidis Spinellis]

A quick note to inform my fellow FreeBSD users and developers that my 
new book "Code Quality: The Open Source Perspective" (Addison-Wesley, 
2006) is now available.  Almost all the 623 examples I use in the book 
are drawn from actual code.  NetBSD is the primary package I used for 
source code excerpts, but aggregate measures and observations of code 
evolution trends come from FreeBSD (at home FreeBSD systems outnumber 
NetBSD ones by a factor of 4 :-).  For more details you can visit the 
book's web site at .


Diomidis - [EMAIL PROTECTED]



___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Call for FreeBSD status reports

[Diomidis Spinellis]

Max Laier wrote:
The new features from last time (categories and task-list) will be available, 
again.  As a reminder the available categories are listed bellow.  Please 
feel free to suggest additional entries:

proj   - Projects (non-specific)
docs   - Documentation
kern   - Kernel
arch   - Architectures
ports  - Ports
vendor - Vendor / 3rd party software
misc   - Miscellaneous
By comparing the categories to those of GNATS three I see missing are:
bin - Userland programs
www - FreeBSD web site
advocacy- Spreading out the word
--
Diomidis - dds@
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Discussion on the future of floppies in 5.x and 6.x

[Diomidis Spinellis]

Brooks Davis wrote:
> No, I mean a server.  The hard part about using PXE to install a box is
> setting up the other box to boot the box your are installing on.  It's
> not all the difficult, but it require a bit of knowledge, some grunt
> work, and a reasionable UNIX-like machine to start from.  What I propose
> is adding enough stuff to disk 1 that you can do pxe installs like this:
Nice idea!  My mind was fixated on older machines that do not support 
PXE; as I always have a couple of FreeBSD servers around at work/home I 
failed to realise that setting up a PXE server could ever be a problem.

Diomidis

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Discussion on the future of floppies in 5.x and 6.x

[Diomidis Spinellis]

Brooks Davis wrote:
I think it would be really cool if someone would add a feature to
disk 1 to become a PXE install server.  It should be fairly straight
forward other then dealing with sysinstall.
I presume the above means a PXE *client*.  This would be cool, but by no 
means trivial.  I looked at this in the past when I wanted to network 
boot FreeBSD on a couple of machines that did not support a boot ROM and 
reached a dead end; I ended up using PicoBSD and NFS-mounting most of 
the stuff.

Following Brook's suggestion, I looked around to see how difficult a PXE 
client project would be.  Here are some bullets and pointers:

- What we would need is a PXE emulator.  PXE stands for Portable 
Execution *Environment*, and it really does supply a (primitive) but not 
trivial environment used to bootstrap the code.

- Microsoft supplies with its Remote Installation Server (RIS) a program 
(rbfg.exe) that creates such an emulation floppy.  This PXE emulator 
only supports PCI cards.  See http://support.microsoft.com/?kbid=242920

- Apparently the same product, but with additional functionality, is 
sold by Argon Technologies.   See 
http://www.argontechnology.com/rbfg/index.shtml

- An open-source project called pxe-toolkit aimed at providing examples 
of PXE client and server code.  The project seems to have dissappeared 
from the face of the earth.  Its homepage on freshmeat and a download 
page on savannah are dead; a page with links on 
http://savannah.nongnu.org/projects/pxe-toolkit does not contain any 
useful pointers.

- The PXE specification (2.1) is freely available from Intel in PDF 
format (500K, 103 pages).  See 
ftp://download.intel.com/labs/manage/wfm/download/pxespec.pdf

- Implementing a PXE client from scratch is obviously doable, but not 
trivial.  One problem is that the API is 16-bit, so we would have to use 
16-bit development tools, libraries, and an execution environment.  The 
client should support a DHCP client, preboot functionality, and an API. 
 The API consists of 37 relatively high-level functions providing TFTP, 
UDP, and UNDI (Universal Network Driver Interface) functionality.  Here 
is a list to give you a rough idea of the functionality that has to be 
provided:

UNLOAD_STACK, GET_CACHED_INFO, RESTART_TFTP, START_UNDI, STOP_UNDI,
START_BASE, STOP_BASE, TFTP_OPEN, TFTP_CLOSE, TFTP_READ, TFTP_READ_FILE,
TFTP_GET_FSIZE, UDP_OPEN, UDP_CLOSE, UDP_WRITE, UDP_READ, UNDI_STARTUP,
UNDI_CLEANUP, UNDI_INITIALIZE, UNDI_RESET_ADAPTER, UNDI_SHUTDOWN,
UNDI_OPEN, UNDI_CLOSE, UNDI_TRANSMIT, UNDI_SET_MCAST_ADDRESS,
UNDI_SET_STATION_ADDRESS, UNDI_SET_PACKET_FILTER, UNDI_GET_INFORMATION,
UNDI_GET_STATISTICS, UNDI_CLEAR_STATISTICS, UNDI_INITIATE_DIAGS,
UNDI_FORCE_INTERRUPT, UNDI_GET_MCAST_ADDRESS, UNDI_GET_NIC_TYPE,
UNDI_GET_IFACE_INFO, UNDI_GET_STATE, UNDI_ISR.
I hope this information helps if anyone wants to take it up from here.

Diomidis
--
Diomidis Spinellis  Assistant Professor
Department of Management Science and Technology  (DMST)
Athens University of Economics and Business  (AUEB)
http://www.dmst.aueb.gr/dds/ mailto:[EMAIL PROTECTED]
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Booting a machine over the network without pxe.

[Diomidis Spinellis]

Josef Karthauser wrote:
> On Tue, Sep 02, 2003 at 09:54:18PM +0100, Josef Karthauser wrote:
> > Does anyone have any experience of booting a machine over the
> > network, like pxeboot, but without running PXE on a network card.
> > I imagine that it should be possible to load pxeboot at the boot:
> > prompt and have everything just work.
> >
> > I could really do with booting my laptop into -stable, where it's only
> > got -current installed.  I do however have a -stable server on site with
> > plenty of disk space.  It would be really cool to remote boot of that
> > via NFS mounts, etc.
> 
> I should have said, my network card is an aue (usb) device which cuts
> etherboot out of the equation.

I have used picobsd(8) to load a bare bones kernel and some
configuration files on an old Pentium-100 16MB laptop using a floppy
disk.  rc.local then NFS mounts the real directories over the net:

mount spiti.spinellis.gr:/usr /usr
mount spiti.spinellis.gr:/home /home
rm bin sbin
/stand/mkdir bin sbin
/stand/mount_nfs spiti.spinellis.gr:/bin /bin
/stand/mount_nfs spiti.spinellis.gr:/sbin /sbin

It took me about two days to jugle the binaries in a way that allowed
the initial boot to view the network via the PCMCIA card and to
configure the system remove picobsd remnants once the large filesystems
were available over NFS.  At some point during the boot process you feel
as if a straightjacket is removed: you have all your binaries and a lot
of disk space available.  Before that stage you have to be very careful
with how you allocate the floppy's disk space.

Diomidis - dds@
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Minimalist FreeBSD 4.8

[Diomidis Spinellis]

Tyler Kellen wrote:
> The information I'm looking to aquire is the absolute minimum files
> required to boot FreeBSD 4.8 into multi-user mode. If this involves
> deleting a massive amount of directories and files, or setting up a
> new drive and copying only the needed files, I think I can make it work.

You can use the system the way you intent to for two weeks, and then run

find / -atime +2w -print0 | xargs -0 rm -f

This command will delete all files that have not been accessed within
the last two weeks.  

Diomidis - http://www.spinellis.gr


___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: new routing protocol

[Diomidis Spinellis]

Jerry Toung wrote:
> I am in the process of implementing a routing protocol under 5.0.
[...]
> My problem is dealing with debuging and portability. With this raw approach I
> guess I will have to run builkernel and installkernel all the time. How can I
> avoid that? I thought about kernel modules, but I don't know what kind to use
> (SYSCALL_MODULE or DEV_MODULE,etc..) and how about netgraph.? does that make
> sense?

[ I was about to ask why we need another networking protocol, but then
noticed the sender's domain :-) ]

I suggest you first build userland applications that emulate the
kernel's networking behavior that affects your protocol.  Build and test
your protocol in userland on top of that environment, and once you are
happy with it, plug it into the kernel.  You will thus gain:

- a very fast turnaround compile/test time 
- the ability to use better debugging and profiling tools
- the flexibility compile and run your code in the environment you
prefer

You might even want to keep your code buildable under the emulated
environment and the real kernel using suitable stubs or conditional
compilation.

I have used such techniques a number of times; you typically recoup your
initial investment very quickly.

Diomidis - [EMAIL PROTECTED] - http://www.spinellis.gr
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Network pipes

[Diomidis Spinellis]

Diomidis Spinellis wrote:
> I think I can package the proposed sh changes as a separate command,
> following Luigi's suggestion.  The syntax will not include a pipe symbol
> and layout, but the performance benefits will still be there.  It will
> also be a lot more portable and also usable within any shell.

The source code and the documentation for the netpipe standalone command
are now available for download under a BSD license at
<http://www.spinellis.gr/sw/unix/netpipe/>.  

Netpipe connects over a TCP/IP socket a remote command specified to a
local input generation command and/or a local output processing command.
The input and output of the remote command are appropriately redirected
so that the remote command's input will come from the local input
generation command and the remote command's output will be sent to the
local output processing command. The remote command is executed on the
machine accessed through the login command. The netpipe executable
should be available through the execution path in the remote machine.
The braces used for delimiting the commands and their arguments should
be space-separated and can be nested. This feature allows you to setup
complex and efficient topologies of distributed communicating processes. 

Although the initial netpipe communication setup is performed through
client-server intermediaries such as ssh(1) or rsh(1), the communication
channel that netpipe establishes is a direct socket connection between
the local and the remote commands. Without the use of netpipe, when
piping remote data through ssh(1) or rsh(1), each data block is read at
the local end by the respective client, is sent to the remote daemon and
written out again to the remote process. The use of netpipe removes the
inefficiency of the multiple data copies and context switches and can in
some cases provide dramatic throughput improvements. On the other hand,
the confidentiality and integrity of the data passing through netpipe's
data channel is not protected; netpipe should therefore be used only
within a confined LAN environment. (The authentication process uses the
protocol of the underlying login program and is no more or less
vulnerable than using the program in isolation; ssh(1) remains secure,
rsh(1) continues to be insecure.) 


Diomidis - http://www.spinellis.gr
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Network pipes

[Diomidis Spinellis]

Kirk Strauser wrote:
> At 2003-07-25T06:06:01Z, Diomidis Spinellis <[EMAIL PROTECTED]> writes:
> > You still have the overhead of two nc instances copying data and context
> > switching.
> Forgive my ignorance, but is that significantly higher than two /bin/sh
> instances copying data and context switching?

When the shell connects two local processes with a pipe it just
redirects the output of the one and the input of the other to the two
ends of a pipe(2) IPC abstraction and leaves them to communicate with
each other simply wait(2)ing until they finish.  The shell does not
shuffle the data between the two processes.  The same can also be done
when connecting a local process with a remote process through a
socket(2); there is no need for an intermediary, and this is what I have
implemented.

Diomidis - http://www.spinellis.gr
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Network pipes

[Diomidis Spinellis]

Kirk Strauser wrote:
> # ssh -f remotehost "nc -l -p 54321 | dd of=/dev/st0 bs=32k"
> # tar cvf - / | nc remotehost 54321
> 
> Netcat implements a TCP/UDP transports and basically nothing else.  Isn't
> that what you're trying to achieve?

You still have the overhead of two nc instances copying data and context
switching.  The same is also the problem with the "ssh -c none"
approach.  My original proposal would setup a direct socket connection
from tar to dd.

I think I can package the proposed sh changes as a separate command,
following Luigi's suggestion.  The syntax will not include a pipe symbol
and layout, but the performance benefits will still be there.  It will
also be a lot more portable and also usable within any shell.

Many thanks to all for your feedback.

Diomidis - http://www.spinellis.gr


___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: Network pipes

[Diomidis Spinellis]

Luigi Rizzo wrote:
> * strange benchmark results! Given the description, I would expect
>   the "|@ rsh" and "|@ ssh" cases to give the same throughput, and
>   in any case "| rsh" to be faster than "| ssh". How comes, instead,
>   that the times differ by an order of magnitude ? Can you run the
>   tests in similar conditions to appreciate the gains better ?

They were executed on different machines.  The ssh result was between
freefall.freebsd.org and ref5, the rsh result was between old low-end
Pentium machines on my home network.

> * I do not understand how can you remove the pipe in the remote host
>   without modifying there both "sshd" and "sh" ?
>   I think it would be very important to understand how much
>   |@ depends on the behaviour of the remote daemon.

The remote daemon is only used for authentication.  Thus any remote host
command execution method can be used without modifying the client or the
server.  What the modified shell does is start on the remote machine a
separate command "netpipe".   Netpipe takes as arguments the originating
host, the socket port, the command to execute, and its arguments. 
Netpipe opens the socket back to the originating host, redirects its I/O
to the socket, and execs the specified command.

> * the loss of encription on the channel is certainly something that might
>   escape the attention of the user. I also wonder in how many cases you
>   really need the extra performance to justify the extra plumbing
>   mechanism.

I felt the need for such functionality when moving GB data between
different machines for creating a disk copy and backup to tape.  My
requirements may be atypical, this is why I asked for input.

> * there are subtle implications of your new plumbing in the way
>   processes are started. With "A | B | C" the shell first creates the
>   pipes, then it can start the processes in any order, and they can
>   individually fail to start without any direct consequence other
>   than an I/O failure. "A |@ B |@ C" requires that you start things
>   from the end of the chain (because you cannot start a process
>   until you have a [socket] descriptor from the next stage in the
>   chain), and if a process fails to start you cannot even start the
>   next one in the sequence. Not that this is bad, just very different
>   from regular pipes.

It is even worse.  You can not write "A |@ B |@ C" because sockets are
created on the originating host.  For the above to work you would need a
mechanism to create another socket between the B and C machines.  Maybe
the syntax should be changed to make such constructions
counterintuitive.


Diomidis
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Network pipes

[Diomidis Spinellis]

I am currently testing a set of modifications to /bin/sh that allow a
user to create a pipeline over the network using a socket as its
endpoints.  Currently a command like

tar cvf - / | ssh remotehost dd of=/dev/st0 bs=32k

has tar sending each block through a pipe to a local ssh process, ssh
communicating through a socket with a remote ssh daemon and dd
communicating with sshd through a pipe again.  The changed shell allows
you to write

tar cvf - / |@ ssh remotehost -- dd of=/dev/st0 bs=32k | :

The effect of the above command is that a socket is created between the
local and the remote host with the standard output of tar and the
standard input of dd redirected to that socket.  Authentication is still
performed using ssh (or any other remote login mechanism you specify
before the -- argument), but the flow between the two processes is from
then on not protected in terms of integrity and privacy.  Thus the
method will mostly be useful within the context of a LAN or a VPN.

The authentication design requires the users to have a special command
in their path on the remote host, but does not require an additional
privileged server or the reservation of special ports.

By eliminating two processes, the associated context switches, the data
copying, and (in the case of ssh) encryption performance is markedly
improved:

dd if=/dev/zero bs=4k count=8192 | ssh remotehost -- dd of=/dev/null
33554432 bytes transferred in 17.118648 secs (1960110 bytes/sec)
dd if=/dev/zero bs=4k count=8192 |@ ssh remotehost -- dd of=/dev/null |
:
33554432 bytes transferred in  4.452980 secs (7535276 bytes/sec)

Even eliminating the encryption overhead by using rsh you can still see 

dd if=/dev/zero bs=4k count=8192 | rsh remotehost -- dd of=/dev/null
33554432 bytes transferred in 131.907130 secs (254379 bytes/sec)
dd if=/dev/zero bs=4k count=8192 |@ rsh remotehost -- dd of=/dev/null |
:
33554432 bytes transferred in 86.545385 secs (387709 bytes/sec)

My questions are:

1. How do you feel about integrating these changes to the /bin/sh in
-CURRENT?  Note that network pipes are a different process plumbing
mechanism, so they really do belong to a shell; implementing them
through a separate command would be inelegant.

2. Do you see any problems with the new syntax introduced?

3. After the remote process starts running standard error output is
lost.  Do find this a significant problem?

4. Both sides of the remote process are communication endpoints and have
to be connected to other local processes via pipes.  Would it be enough
to document this behaviour or should it be hidden from the user by means
of forked read/write processes?

Diomidis - http://www.spinellis.gr
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Contribution: PCL-724 driver

[Diomidis Spinellis]

I have written a device driver for the Advantech PCL-724 parallel I/O
card and made it available at
.  The PCL-724 card
emulates the Intel 8255A programmable peripheral interface chip running
in mode 0 (simple I/O).  The driver has been in production use under
4.1, 4.7 and 4.7 for about two years.  Recently I polished its
implementation and added a manual page to contribute it as part of
FreeBSD.  I am not a committer and probably do not have enough spare
time to become one.  What is the procedure for having the driver become
part of FreeBSD?  I can not see it documented in
.

Diomidis

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message