RE: My project wish-list for the next 12 months

[Macy, Kip]

Mbuf clusters can easily be allocated from their own pool, but I guess
you're probably right about various bits of incidental memory
allocation.

-Kip

-Original Message-
From: Scott M. Ferris [mailto:[EMAIL PROTECTED] 
Sent: Tuesday, December 14, 2004 5:38 PM
To: Macy, Kip
Cc: Danny Braniss; [EMAIL PROTECTED]; Peter Blok
Subject: Re: My project wish-list for the next 12 months


On Tue, 14 Dec 2004 17:05:25 -0800 (PST), Kip Macy <[EMAIL PROTECTED]>
wrote:
> > Hardware-based iSCSI HBAs solve this by having their own memory and 
> > TCP stack separate from the OS.  Software-only iSCSI initiators such

> > as linux-iscsi usually just hope it doesn't happen, and that's why I

> > don't usually recommend software-only iSCSI initiators to anyone.
> 
> How is that any better than having the SW iscsi initiator pre-allocate

> a pool of memory for its use at startup?

You could preallocate, but the hard part would be getting the OS network
stack to allocate from the iSCSI driver's preallocated memory pool,
instead of calling the usual kernel memory allocator (or more likely as
a fallback if the normal kernel allocation fails).  How do you tell the
TCP stack and NIC drivers to allocate control blocks, mbufs, and
anything else they might need from the preallocated iSCSI pool?  It's
not a problem in theory, but in practice that's not something you
usually see in the API for the network stack.

-- 
Scott M. Ferris
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

iSCSI, was Re: My project wish-list for the next 12 months

[Danny Braniss]

let me start by a local saying:
Q- What's a camel
A- It's a horse designed by a committee.
BTW, the camel is a very efficient piece of equipment.

If I would plan a 24/7 life support system I would not use iSCSI.
(having to rely on packets traveling the Internet, DOS, etc, is not good for
your health)

Having said that, I can see the attractive sides of iSCSI, specially if there
are vendors trying to peddle cheap disk boxes that speak iSCSI.
At the moment, they only work under MS or Linux (BTW we tested them
under Linux and so far we were not impressed - as HRH would say :-), so
any kind of implementation is most welcome.

> How do you plan on handling cases where the user program blocks and
> can't login again (because of a page fault for code or data,
> allocating a new socket in the kernel, allocating a new socket buffer
> in the kernel, etc)?

Let me see, if a user cannot login, and that is my main service, then probably
not being able to login to my iSCSI disk is the least of my problems.

Seriuosly, I understand your missgivings, but so many other, more simple
problems might cause a meltdown.

As others have pointed out, NFS/GEOM/etc rely on the kernel for resources,
so should the iSCSI.

danny


> 
> One of the major problems any software-only iSCSI initiator has is
> dealing with memory deadlocks.  The OS may try to write out one or
> more pages in order to free up memory.  If the iSCSI initiator needs
> to allocate memory (directly or indirectly in the TCP stack) in order
> to complete that write, you've got a circular dependency where in
> order to get free memory you need to have free memory.  
> Hardware-based iSCSI HBAs solve this by having their own memory and
> TCP stack separate from the OS.  Software-only iSCSI initiators such
> as linux-iscsi usually just hope it doesn't happen, and that's why I
> don't usually recommend software-only iSCSI initiators to anyone.
> 
> Having a user program login is fine for SendTargets discovery and
> testing, but if you're planning on getting a reliable iSCSI initiator,
> you'll probably need to do the login from within the kernel, and make
> sure you have all of the resources needed to do that reserved (wire
> pages into RAM, etc).  The hard part of that is making sure you have
> all the resources needed to send and receive TCP packets reserved, as
> well as the resources to establish new connections in case the
> existing connections is closed or aborted.  The linux-iscsi initiator
> doesn't even attempt this, and just hopes deadlocks don't happen very
> often.
> 
> I used to be the main (and for most of that time, only) developer for
> the linux-iscsi initiator.  When I first took over the initiator, it
> used the approach of doing iSCSI session login from a daemon, and
> passing the socket down into a kernel module.  I had to take that out
> because it deadlocked too easily.  I wouldn't recommend that approach.
> 
> -- 
> Scott M. Ferris


___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Kip Macy]

> Hardware-based iSCSI HBAs solve this by having their own memory and
> TCP stack separate from the OS.  Software-only iSCSI initiators such
> as linux-iscsi usually just hope it doesn't happen, and that's why I
> don't usually recommend software-only iSCSI initiators to anyone.

How is that any better than having the SW iscsi initiator pre-allocate a pool 
of 
memory for its use at startup?









___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Scott M. Ferris]

On Tue, 14 Dec 2004 17:05:25 -0800 (PST), Kip Macy <[EMAIL PROTECTED]> wrote:
> > Hardware-based iSCSI HBAs solve this by having their own memory and
> > TCP stack separate from the OS.  Software-only iSCSI initiators such
> > as linux-iscsi usually just hope it doesn't happen, and that's why I
> > don't usually recommend software-only iSCSI initiators to anyone.
> 
> How is that any better than having the SW iscsi initiator pre-allocate a pool 
> of
> memory for its use at startup?

You could preallocate, but the hard part would be getting the OS
network stack to allocate from the iSCSI driver's preallocated memory
pool, instead of calling the usual kernel memory allocator (or more
likely as a fallback if the normal kernel allocation fails).  How do
you tell the TCP stack and NIC drivers to allocate control blocks,
mbufs, and anything else they might need from the preallocated iSCSI
pool?  It's not a problem in theory, but in practice that's not
something you usually see in the API for the network stack.

-- 
Scott M. Ferris
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Scott M. Ferris]

On Tue, 14 Dec 2004 22:02:40 +, Christoph Hellwig <[EMAIL PROTECTED]> wrote:
> 
> Note that this isn't different from any sufficiently complex HBA driver,
> except that the code that can operate under these conditions is more
> complex for iscsi.

I don't understand why you think the situation is at all comparable to
other HBA drivers.  The complexity of the HBA driver code isn't the
real issue.  The problem with software-only iSCSI HBA drivers is that
they usually try to make use of the OS TCP stack, and that stack
usually wasn't designed to operate under the constraints that an HBA
driver has to operate under.  The correctness of a software-only iSCSI
HBA driver depends on properties of the TCP stack, as well as the HBA
driver's code.   I don't think that's true of any other HBA driver.

-- 
Scott M. Ferris
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Wilko Bulte]

On Tue, Dec 14, 2004 at 01:47:15PM -0600, Scott M. Ferris wrote..
> On Tue, 14 Dec 2004 09:29:19 +0200, Danny Braniss <[EMAIL PROTECTED]> wrote:
> > Great!, we seem to be on the same wavelength, im now writing (at about one
> > char a minute) the login user program, and somehow - to be discovered -, the
> > socket will be passed to the kernel.
> > my main efford, at the moment, is a) to &^%$$## understand the RFC (i think
> > they used a scrambler) and b) define the data structures.
> 
> How do you plan on handling cases where the user program blocks and
> can't login again (because of a page fault for code or data,
> allocating a new socket in the kernel, allocating a new socket buffer
> in the kernel, etc)?
> 
> One of the major problems any software-only iSCSI initiator has is
> dealing with memory deadlocks.  The OS may try to write out one or
> more pages in order to free up memory.  If the iSCSI initiator needs
> to allocate memory (directly or indirectly in the TCP stack) in order
> to complete that write, you've got a circular dependency where in
> order to get free memory you need to have free memory.  
> Hardware-based iSCSI HBAs solve this by having their own memory and
> TCP stack separate from the OS.  Software-only iSCSI initiators such

Downside: TOE cards are not cheap though..

-- 
Wilko Bulte [EMAIL PROTECTED]
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Scott M. Ferris]

On Tue, 14 Dec 2004 09:29:19 +0200, Danny Braniss <[EMAIL PROTECTED]> wrote:
> Great!, we seem to be on the same wavelength, im now writing (at about one
> char a minute) the login user program, and somehow - to be discovered -, the
> socket will be passed to the kernel.
> my main efford, at the moment, is a) to &^%$$## understand the RFC (i think
> they used a scrambler) and b) define the data structures.

How do you plan on handling cases where the user program blocks and
can't login again (because of a page fault for code or data,
allocating a new socket in the kernel, allocating a new socket buffer
in the kernel, etc)?

One of the major problems any software-only iSCSI initiator has is
dealing with memory deadlocks.  The OS may try to write out one or
more pages in order to free up memory.  If the iSCSI initiator needs
to allocate memory (directly or indirectly in the TCP stack) in order
to complete that write, you've got a circular dependency where in
order to get free memory you need to have free memory.  
Hardware-based iSCSI HBAs solve this by having their own memory and
TCP stack separate from the OS.  Software-only iSCSI initiators such
as linux-iscsi usually just hope it doesn't happen, and that's why I
don't usually recommend software-only iSCSI initiators to anyone.

Having a user program login is fine for SendTargets discovery and
testing, but if you're planning on getting a reliable iSCSI initiator,
you'll probably need to do the login from within the kernel, and make
sure you have all of the resources needed to do that reserved (wire
pages into RAM, etc).  The hard part of that is making sure you have
all the resources needed to send and receive TCP packets reserved, as
well as the resources to establish new connections in case the
existing connections is closed or aborted.  The linux-iscsi initiator
doesn't even attempt this, and just hopes deadlocks don't happen very
often.

I used to be the main (and for most of that time, only) developer for
the linux-iscsi initiator.  When I first took over the initiator, it
used the approach of doing iSCSI session login from a daemon, and
passing the socket down into a kernel module.  I had to take that out
because it deadlocked too easily.  I wouldn't recommend that approach.

-- 
Scott M. Ferris
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Danny Braniss]

Great!, we seem to be on the same wavelength, im now writing (at about one
char a minute) the login user program, and somehow - to be discovered -, the
socket will be passed to the kernel.
my main efford, at the moment, is a) to &^%$$## understand the RFC (i think
they used a scrambler) and b) define the data structures.

season greatings,
danny
PS: i have the 2nd Unix V6 tape that came to this part of the world,
the first one was never used, mine got installed on a PDP 11/45.

> You are my kind of guy ;-) I grew up with PDP-1[01], RSX-11 and Version 6
> UNIX straight from Bell Labs. I am a bit rusty too. I can easily hack
> something together, but it most be perfect. Handshaking must in user mode,
> actual work in the kernel. Why enter the kernel thru the network driver, go
> to a user daemon and back to the kernel to execute and v.v. for the answer?
> 
> No I'm proto-typing and have a lot of challenges and very little time.
> Luckily holiday is around the corner.
> 
> Later.
> 
> Peter


___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

RE: My project wish-list for the next 12 months

[Peter Blok]

You are my kind of guy ;-) I grew up with PDP-1[01], RSX-11 and Version 6
UNIX straight from Bell Labs. I am a bit rusty too. I can easily hack
something together, but it most be perfect. Handshaking must in user mode,
actual work in the kernel. Why enter the kernel thru the network driver, go
to a user daemon and back to the kernel to execute and v.v. for the answer?

No I'm proto-typing and have a lot of challenges and very little time.
Luckily holiday is around the corner.

Later.

Peter



-Original Message-
From: Danny Braniss [mailto:[EMAIL PROTECTED] 
Sent: Sunday, December 12, 2004 9:05 AM
To: Peter Blok
Cc: [EMAIL PROTECTED]
Subject: Re: My project wish-list for the next 12 months 

> Hi Danny,
> 
> Great! I am still in a design proof-of-concept phase, but I appreciate
your
> help. I'll keep you posted.
> 
> Peter

hi Peter,
re-reading your email, i realize you are interested in the target
side of iSCSI, and me more on the initiator side, so i decided to bite
the bullet and start designing/coding - mind you, i haven't written a
serious
driver since the days of PDP/Vax & UDA50 (let's see if someone knows what im
talking about), so it took me most of an afternoon to get the skeleton
of a kld to compile.

danny




___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Danny Braniss]

> Hi Danny,
> 
> Great! I am still in a design proof-of-concept phase, but I appreciate your
> help. I'll keep you posted.
> 
> Peter

hi Peter,
re-reading your email, i realize you are interested in the target
side of iSCSI, and me more on the initiator side, so i decided to bite
the bullet and start designing/coding - mind you, i haven't written a serious
driver since the days of PDP/Vax & UDA50 (let's see if someone knows what im
talking about), so it took me most of an afternoon to get the skeleton
of a kld to compile.

danny


___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Chris Pressey]

On Wed, 01 Dec 2004 15:02:40 -0700
Scott Long <[EMAIL PROTECTED]> wrote:

> All,
> 
> I know that I said last month that we were going to stop promising
> specific features for the next major release.  However, I'd like to
> throw out a list of things that would be really nice to have in the
> future, whether its 6.0 or 7.0 or whatever.  Most of these tasks are
> not trivial, but I hope that talking about them will encourage some
> interest.  These are in no particular priority order.  I'd also be
> thrilled if someone wanted to dress this list up in docbook and add
> it to the webpage.  While this is just my personal list, I'd welcome
> other additions to it (in the sense of significant projects, not just
> individual PRs or bug fixes that one might be interested in).
> 
> [...]
> 
> 2.  New installer.  I know some people still consider this a joke, but
> the reality is that sysinstall is no longer state of the art.  It's
> fairly good at the simple task that it does, but it's becoming harder
> and harder to fix bugs and extend functionality in it.  It's also
> fairly unfriendly to those of us who haven't been using it since 1995.
> The DFly folks have some very interesting work in this area
> (www.bsdinstaller.com) and it would be very good to see if we can
> collaborate with them on it.

I'm glad you find it interesting.  If you have any questions, feel free
to ask.  There are definately two factors to consider in any attempt to
integrate this installer with the FreeBSD release process.  One is that
it is designed to be run from a fairly full-featured environment (i.e. a
LiveCD.)  It has been successfully integrated with the latest FreeSBIE
though, so beyond that I wouldn't expect many technical difficulties. 
The other factor is that the BSD installer project is (not unlike
DragonFly itself) quite experimental, and we don't hesitate to rewrite
entire subsystems (and break the interfaces between them) when a better
idea and/or more tractable code comes along.

Anyway, since you asked, if I were to write a wish-list for FreeBSD, I
guess it would look something like:

1.  Flesh out the mission statement, such as it is (FAQ #1.2.)  "We
provide no-strings-attached software" is an excellent start, but by
itself, it's awkwardly nonspecific.  Extend it by specifying that the
provided software does come, in fact, in the form of an operating
system, and describe the intended properties of such (performant/
stable/featureful/easy-to-use/whatever/etc/etc.)

-Chris
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[J.R. Oldroyd]

How about some form of streamlined video input support?

I realize Linux's v4l model doesn't fit too well here, but what
they have done there provides support for all sorts of common video
devices, and makes them all available to pretty much any program that
wants video input.

FreeBSD is far behind in this area.

-jr


On Dec 01, 15:02, Scott Long wrote:
> All,
> 
> I know that I said last month that we were going to stop promising
> specific features for the next major release.  However, I'd like to
> throw out a list of things that would be really nice to have in the
> future, whether its 6.0 or 7.0 or whatever.  Most of these tasks are
> not trivial, but I hope that talking about them will encourage some
> interest.  These are in no particular priority order.  I'd also be
> thrilled if someone wanted to dress this list up in docbook and add
> it to the webpage.  While this is just my personal list, I'd welcome
> other additions to it (in the sense of significant projects, not just
> individual PRs or bug fixes that one might be interested in).
> 
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[jmab]

On Thu, Dec 02, 2004 at 06:41:53PM +0100, Andre Oppermann typed:
:: Data redundancy would require a UFS/FFS redesign.  I'm 'only' talking
:: about enhancing UFS/FFS but keeping anything ondisk the same (plus
:: some more elements).

Well, from my point of view, I would see this as some strong
geom modules. ggate already allows to export disk devices as
ggate instances.

One of my project (in the long term) is to be able to use
RAID scheme through the network. I think geom is the way to go
to manipulate I/O path. This could let us create RAID over IP
without FS knowledge.

--
julien.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Jose M Rodriguez]

El Viernes, 3 de Diciembre de 2004 13:29, Devon H. O'Dell escribió:
> Jose M Rodriguez wrote:
> > El Jueves, 2 de Diciembre de 2004 02:23, Peter Kieser escribió:
> >>Scott Long wrote:
[ ... ]
> > Also, I can remenber, al last, other previous try.  Please, use a
> > safe path.  As a reference, Mandrake Drakx is accesible via cvs
> > (but gpl).
> >
> > http://www.mandrakelinux.com/en/drakx.php3
> >
> > As anaconda (which use python instead of perl) is an installer
> > working from a system interpreter in a big mdfs. works over
> > perlgtk2 and a vesa X server.  This may be used in FreeBSD for
> > cdrom/pxe installs
>
> Anaconda is also GPLed and also requires a good few changes for most
> of it to run under FreeBSD. I haven't made any of these changes, but
> we looked into using Anaconda in DragonFly before we started on our
> own installer, and it would have just been too much work for the
> deadline we had (our 1.0 installer was written in less than 3
> months!). Yes, it is in Python, but all the VESA stuff is via
> framebuffer, not an X server, so it's not something that we could use
> easily. At least, this was the case when I researched it in May.
> Anything GPL probably won't qualify in the first place, due to
> obvious license incompatibilities.
>

I'm not talking about port Anaconda or DarkX,  but on analyse the 
building blocks:

- a two mfs based apoarch (stage1/stage2)
- a powerfull system orirented interpreter (perl/python)
- proven system libraries (perlgtk2 ...)
- a vesa based Xserver, very reduced.

> While I don't want to sound shameless, I do urge you who are used to
> sysinstall to take a look at our installer --
> http://www.bsdinstaller.org. It's very simple to extend, and it's not
> very large.
>

I found bsdinstaller a good candidate, but more a WIP that a reference.  
Both anaconda and DrakX are more mature.

I don't know too much about anaconda, but DarkX have several years of 
evolution.  And maybe someone find perl more familiar than python.

> I haven't looked at Drakx at all, so I can't say anything useful
> about that.
>

You can find it on Mandrake cvs, but I don't know if they have a cvsweb 
interface.

--
  josemi

> HTH,
>
> Devon H. O'Dell

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Kamal R. Prasad]

Anaconda is also GPLed and also requires a good few changes for most 
of it to run under FreeBSD. I haven't made any of these changes, but 
we looked into using Anaconda in DragonFly before we started on our 
own installer, and it would have just been too much work for the 
deadline we had (our 1.0 installer was written in less than 3 
months!). Yes, it is in Python, but all the VESA stuff is via 
framebuffer, not an X server, so it's not something that we could use 
easily. At least, this was the case when I researched it in May. 
Anything GPL probably won't qualify in the first place, due to obvious 
license incompatibilities.
Sorry for going off on a tangent -and pl ignore if its annoying:-
Can someone elaborate on the impact of GPL if one is to use lGPL'ed code 
without modifying it. i.e. if the kernel or library is GPL'ed and a 
module is developed on top of it and sold, does the GPL require one to 
give out the src code for the developed module too?

thansks
-kamal
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Devon H. O'Dell]

Jose M Rodriguez wrote:
El Jueves, 2 de Diciembre de 2004 02:23, Peter Kieser escribió:
Scott Long wrote:
2.  New installer.  I know some people still consider this a joke,
but the reality is that sysinstall is no longer state of the art. 
It's fairly good at the simple task that it does, but it's becoming
harder and harder to fix bugs and extend functionality in it.  It's
also fairly unfriendly to those of us who haven't been using it
since 1995. The DFly folks have some very interesting work in this
area (www.bsdinstaller.com) and it would be very good to see if we
can collaborate with them on it.
Please, don't change /stand/sysinstall *too* much, there is really
nothing wrong with the interface of it, and it's what makes FreeBSD
so "quick" to install. At the very least, make sure you do NOT go for
an XFree86 installation, and keep to the "KISS" approach. Visually
wise, theres nothing wrong with the current installer.. and its one
of the things I "promote" about FreeBSD -- the ease to install. It's
small, its fast.. and it works, however in error situations it does
mess up badly.

I second that.  It's the biggest thing we can put in a floppy.
But, if works on a new installer are needed, this must be neccesary:
sysinstall is only the most user related thing of and overall process.  
I think this must begin with taking /usr/src/release out of /usr/src 
and work on a new release build system.

Also, I can remenber, al last, other previous try.  Please, use a safe 
path.  As a reference, Mandrake Drakx is accesible via cvs (but gpl).

http://www.mandrakelinux.com/en/drakx.php3
As anaconda (which use python instead of perl) is an installer working 
from a system interpreter in a big mdfs. works over perlgtk2 and a vesa 
X server.  This may be used in FreeBSD for cdrom/pxe installs 
Anaconda is also GPLed and also requires a good few changes for most of 
it to run under FreeBSD. I haven't made any of these changes, but we 
looked into using Anaconda in DragonFly before we started on our own 
installer, and it would have just been too much work for the deadline we 
had (our 1.0 installer was written in less than 3 months!). Yes, it is 
in Python, but all the VESA stuff is via framebuffer, not an X server, 
so it's not something that we could use easily. At least, this was the 
case when I researched it in May. Anything GPL probably won't qualify in 
the first place, due to obvious license incompatibilities.

While I don't want to sound shameless, I do urge you who are used to 
sysinstall to take a look at our installer -- 
http://www.bsdinstaller.org. It's very simple to extend, and it's not 
very large.

I haven't looked at Drakx at all, so I can't say anything useful about that.
HTH,
Devon H. O'Dell
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Jose M Rodriguez]

El Jueves, 2 de Diciembre de 2004 02:23, Peter Kieser escribió:
> Scott Long wrote:
> > 2.  New installer.  I know some people still consider this a joke,
> > but the reality is that sysinstall is no longer state of the art. 
> > It's fairly good at the simple task that it does, but it's becoming
> > harder and harder to fix bugs and extend functionality in it.  It's
> > also fairly unfriendly to those of us who haven't been using it
> > since 1995. The DFly folks have some very interesting work in this
> > area (www.bsdinstaller.com) and it would be very good to see if we
> > can collaborate with them on it.
>
> Please, don't change /stand/sysinstall *too* much, there is really
> nothing wrong with the interface of it, and it's what makes FreeBSD
> so "quick" to install. At the very least, make sure you do NOT go for
> an XFree86 installation, and keep to the "KISS" approach. Visually
> wise, theres nothing wrong with the current installer.. and its one
> of the things I "promote" about FreeBSD -- the ease to install. It's
> small, its fast.. and it works, however in error situations it does
> mess up badly.

I second that.  It's the biggest thing we can put in a floppy.

But, if works on a new installer are needed, this must be neccesary:

sysinstall is only the most user related thing of and overall process.  
I think this must begin with taking /usr/src/release out of /usr/src 
and work on a new release build system.

Also, I can remenber, al last, other previous try.  Please, use a safe 
path.  As a reference, Mandrake Drakx is accesible via cvs (but gpl).

http://www.mandrakelinux.com/en/drakx.php3

As anaconda (which use python instead of perl) is an installer working 
from a system interpreter in a big mdfs. works over perlgtk2 and a vesa 
X server.  This may be used in FreeBSD for cdrom/pxe installs 
> ___ 
> [EMAIL PROTECTED] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to
> "[EMAIL PROTECTED]"

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Jose M Rodriguez]

El Jueves, 2 de Diciembre de 2004 00:19, Scott Long escribió:
> Kris Kennaway wrote:
> > On Wed, Dec 01, 2004 at 04:10:57PM -0700, Ryan Sommers wrote:
> >>Another issue I had with the dfly installer was one point I believe
> >> needs to be central to any next-gen installer.
> >> Internationalisation.
> >
> > Careful not to pile on so many wishes that achieving anything
> > becomes impossible.  Our current installer doesn't do this, so it's
> > not a hard requirement that a better installer should.
> >
> > Kris
>
> Internationalization is actually quite important, and is not easy to
> bolt on after the fact but is fairly easy to program to once the core
> is in place.  The fact that sysinstall doesn't have it makes it no
> less important.  Now this isn't a reason to reject the DFly work, but
> it could certainly be a good area for someone to contribute.
>
> Scott
>

Sure.  I can confirm this.  But this issue must be take with care.

1.-  This is an overall effort that, IMHO, must begin with a real i18n 
ports work.  Actual ports doesn't have a well defined i18n behavior.

2.-  this covers several unrelated issues:
- i18n, lang related issues. iso8859-1, unicode, arabic ...
- l10n, locale related issues (country specific).
- base system oper (try hack arround loader with a foreing keyboard).

About the base system, I have really bad experiences with i18n/locale 
support in boot/system works.  This must be a low priority task.

But I'll be really glad to see some basic foreing keyboard/console 
support in boot/loader as:
boot> keyb sp

--
  josemi

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Scott Long]

Peter Wemm wrote:
On Wednesday 01 December 2004 05:17 pm, Foxfair Hu wrote:
On Wed, Dec 01, 2004 at 03:02:40PM -0700, Scott Long wrote:
All,
[]

1.  Keyboard multiplexer.  We are running into problems with making
ps/2 and USB/bluetooth keyboards work together and work with KVMs.
Having a virtual keyboard device that multiplexes the various real
keyboard devices and handles hotplug can solve this mess pretty
effectively.  I know that there has been a lot of talk about this
on mailing lists recently but I don't know how much progress is
being made so I'm listing it here.
  How about reuse NetBSD's wscons ? I've kept an eye on it and
thought it should be a good start for FreeBSD.

Only if it has 100% identical look/feel to syscons.  ie: same key maps, 
same ioctl's, same Xserver interface, same escape codes, etc.  If not, 
then over my dead body!

What if it's 100% identical except for the [Space] key, which becomes
the console pause key?
=-)
Scott
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Peter Wemm]

On Wednesday 01 December 2004 05:17 pm, Foxfair Hu wrote:
> On Wed, Dec 01, 2004 at 03:02:40PM -0700, Scott Long wrote:
> > All,
>
> []
>
> > 1.  Keyboard multiplexer.  We are running into problems with making
> > ps/2 and USB/bluetooth keyboards work together and work with KVMs.
> > Having a virtual keyboard device that multiplexes the various real
> > keyboard devices and handles hotplug can solve this mess pretty
> > effectively.  I know that there has been a lot of talk about this
> > on mailing lists recently but I don't know how much progress is
> > being made so I'm listing it here.
>
>How about reuse NetBSD's wscons ? I've kept an eye on it and
> thought it should be a good start for FreeBSD.

Only if it has 100% identical look/feel to syscons.  ie: same key maps, 
same ioctl's, same Xserver interface, same escape codes, etc.  If not, 
then over my dead body!

-- 
Peter Wemm - [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]
"All of this is for nothing if we don't go to the stars" - JMS/B5
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Andre Oppermann]

Scott Long wrote:
Stephan Uphoff wrote:
On Thu, 2004-12-02 at 09:41, Andre Oppermann wrote:
The holy grail of course is to mount
the same filesystem 'rw' on more than one box, preferrably more than 
two.
This requires some more involved synchronization and locking on top 
of the
cache invalidation.  And make sure that the multi-'rw' cluster stays 
alive
if one of the participants freezes and doesn't respond anymore.

Scrolling through the UFS/FFS code I think the first one is 2-3 days of
work.  The second 2-4 weeks and the third 2-3 month to get it right.
If someone would throw up the money...
Although I don't know the specifics of your experience, I can easily
imagine how hard it would be to make this work on UFS.  Common
operations like walking a file path to the root are nearly impossible to
do reliably without an overbearing amount of synchronization.  Then you
have all of the problems synchronizing buffered data and metadata.
You don't do it the fully synchronized way.  The semantics will be somewhat
like with NFS.  Reading in the filesystem does not invoke synchronization.
Only writing, or actually intending to write something, requires synchro-
nization with all other machines in the filesystem cluster.  Synchronization
ensures that writes to a directory or file are properly serialized and that
caches are invalidated at the same time.
Softupdates would be a nightmare, if not impossible.
To the contrary.  It provides a nice place to link code into and it helps
to keep performance up to good levels.  Any inode changes entering the
softdep code would cause a message to all other machines informing them
of the change.  If another one wants to update the same inode or something
dependend on it it has to wait until you have written it to disk.  His
backnotification would of course move this inode to the front of the softdep
work queue to not stall your request.
As long as you don't want to have mmap() work across machines with contents
in memory but not yet written to disk things stay pretty much sane.
--
Andre
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Andre Oppermann]

Stephan Uphoff wrote:
On Thu, 2004-12-02 at 09:41, Andre Oppermann wrote:
Scott Long wrote:
5.  Clustered FS support.  SANs are all the rage these days, and
clustered filesystems that allow data to be distributed across many
storage enpoints and accessed concurrently through the SAN are very
powerful.  RedHat recently bought Sistina and re-opened the GFS source
code, so exploring this would be very interesting.
There are certain steps that can be be taken one at a time.  For example
it should be relatively easy to mount snapshots (ro) from more than one
machine.  Next step would be to mount a full 'rw' filesystem as 'ro' on
other boxes.  This would require cache and sector invalidation broadcasting
from the 'rw' box to the 'ro' mounts.  
Mhhh .. if you plan to invalidate at the disk block cache layer then you
will run into race conditions with UFS/FFS (Especially with remove
operations)
I was once called in to evaluate such a multiple reader/single writer
system based on an UFS like file system and block layer invalidation and
had to convince management to kill it. (It appeared to work and actually
made it though internal and customer acceptance testing before failing
horrible in the field).
If you send me more details on your proposed cache and sector
invalidation/cluster design I will be happy to review it.
It obviously doesn't work only the disk block level.  For the rw+ro(n) case
you have to invalidate updated/changed inodes (file and directory inodes)
as well as associated data blocks.  Alternatively you could invalidate all
related data blocks together with the inode but that would kill performance
for files that are simply appended (like log files).
--
Andre
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Scott Long]

Stephan Uphoff wrote:
On Thu, 2004-12-02 at 09:41, Andre Oppermann wrote:
Scott Long wrote:
5.  Clustered FS support.  SANs are all the rage these days, and
clustered filesystems that allow data to be distributed across many
storage enpoints and accessed concurrently through the SAN are very
powerful.  RedHat recently bought Sistina and re-opened the GFS source
code, so exploring this would be very interesting.
There are certain steps that can be be taken one at a time.  For example
it should be relatively easy to mount snapshots (ro) from more than one
machine.  Next step would be to mount a full 'rw' filesystem as 'ro' on
other boxes.  This would require cache and sector invalidation broadcasting
from the 'rw' box to the 'ro' mounts.  

Mhhh .. if you plan to invalidate at the disk block cache layer then you
will run into race conditions with UFS/FFS (Especially with remove
operations)
I was once called in to evaluate such a multiple reader/single writer
system based on an UFS like file system and block layer invalidation and
had to convince management to kill it. (It appeared to work and actually
made it though internal and customer acceptance testing before failing
horrible in the field).
If you send me more details on your proposed cache and sector
invalidation/cluster design I will be happy to review it.

The holy grail of course is to mount
the same filesystem 'rw' on more than one box, preferrably more than two.
This requires some more involved synchronization and locking on top of the
cache invalidation.  And make sure that the multi-'rw' cluster stays alive
if one of the participants freezes and doesn't respond anymore.
Scrolling through the UFS/FFS code I think the first one is 2-3 days of
work.  The second 2-4 weeks and the third 2-3 month to get it right.
If someone would throw up the money...

Although I don't know the specifics of your experience, I can easily
imagine how hard it would be to make this work on UFS.  Common
operations like walking a file path to the root are nearly impossible to
do reliably without an overbearing amount of synchronization.  Then you
have all of the problems synchronizing buffered data and metadata.
Softupdates would be a nightmare, if not impossible.
Scott
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Eric Kjeldergaard]

> >> 4.  Journaled filesystem.
> >
> > The stage of the current implementation is, as I said, read-only.
> > Further, it's currently i386 only.
> 
> In theory, it shouldn't be tied to i386, but I never tested it somewhere
> else unfortunately.
> 

Got that from the reiserfs port which says it doesn't compile other
than under i386.

> > However, I think that there is enough interest in this new (and
> > relatively exciting) filesystem that we may be able to find some
> > developers with time (Possibly including myself) and desire to try to
> > implement write support and do some porting.   To ease any qualms
> > with regards to licensing, it appears
> > (http://lists.freebsd.org/pipermail/freebsd-current/2004-October/040998.html)
> >  that the current implementation is BSD licensed.
> 
> Not exactly. Only the mount_reiserfs command is under BSD. The kernel
> module is GPL.
> 

Oh, I'm sorry.  I guess I read quickly and w/o thinking enough.  I
suppose that would put is near square 1.

> I don't know neither JFS nor XFS, but Reiser4 may be a great candidate
> too (the current port is ReiserFS 3.6 only). It sure doesn't have the
> maturity of ReiserFS but brings nice features.

That would be nice if the support could be gathered for it.  I do like
it better than XFS (which I've tried).

-- 
If I write a signature, my emails will appear more personalised.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Stephan Uphoff]

On Thu, 2004-12-02 at 09:41, Andre Oppermann wrote:
> Scott Long wrote:
> > 5.  Clustered FS support.  SANs are all the rage these days, and
> > clustered filesystems that allow data to be distributed across many
> > storage enpoints and accessed concurrently through the SAN are very
> > powerful.  RedHat recently bought Sistina and re-opened the GFS source
> > code, so exploring this would be very interesting.
> 
> There are certain steps that can be be taken one at a time.  For example
> it should be relatively easy to mount snapshots (ro) from more than one
> machine.  Next step would be to mount a full 'rw' filesystem as 'ro' on
> other boxes.  This would require cache and sector invalidation broadcasting
> from the 'rw' box to the 'ro' mounts.  

Mhhh .. if you plan to invalidate at the disk block cache layer then you
will run into race conditions with UFS/FFS (Especially with remove
operations)
I was once called in to evaluate such a multiple reader/single writer
system based on an UFS like file system and block layer invalidation and
had to convince management to kill it. (It appeared to work and actually
made it though internal and customer acceptance testing before failing
horrible in the field).

If you send me more details on your proposed cache and sector
invalidation/cluster design I will be happy to review it.


> The holy grail of course is to mount
> the same filesystem 'rw' on more than one box, preferrably more than two.
> This requires some more involved synchronization and locking on top of the
> cache invalidation.  And make sure that the multi-'rw' cluster stays alive
> if one of the participants freezes and doesn't respond anymore.
> 
> Scrolling through the UFS/FFS code I think the first one is 2-3 days of
> work.  The second 2-4 weeks and the third 2-3 month to get it right.
> If someone would throw up the money...

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Brooks Davis]

On Thu, Dec 02, 2004 at 10:47:36PM +0200, Valentin Nechayev wrote:
>  Wed, Dec 01, 2004 at 15:02:40, scottl wrote about "My project wish-list for 
> the next 12 months": 
> 
> [...]
> > 2.  New installer.  I know some people still consider this a joke, but
> > the reality is that sysinstall is no longer state of the art.  It's
> > fairly good at the simple task that it does, but it's becoming harder
> > and harder to fix bugs and extend functionality in it.  It's also
> > fairly unfriendly to those of us who haven't been using it since 1995.
> > The DFly folks have some very interesting work in this area
> > (www.bsdinstaller.com) and it would be very good to see if we can
> > collaborate with them on it.
> 
> If we're going to discuss admin-friendiness...
> 
> - Resplit distribution sets. It's simply shame when unpacking base overwrites
> existing log files (e.g. during binary upgrade).
> At least /etc and /var/log _must_ be separated. Also, I consider openbsd
> scheme to be pleasant: separate build tools set (gcc, binutils, etc.)
> from main functioning set.
> As maximum, convert them into packages and register along with ports.

The log files shouldn't be there at all.  I'm planning to remove all of
them from the distribution sets and do the creation during boot.

-- Brooks

--
Any statement of the form "X is the one, true Y" is FALSE.
PGP fingerprint 655D 519C 26A7 82E7 2529  9BF0 5D8E 8BE9 F238 1AD4


pgptMz7fgzhRf.pgp
Description: PGP signature

Re: My project wish-list for the next 12 months

[Valentin Nechayev]

 Wed, Dec 01, 2004 at 15:02:40, scottl wrote about "My project wish-list for 
the next 12 months": 

[...]
> 2.  New installer.  I know some people still consider this a joke, but
> the reality is that sysinstall is no longer state of the art.  It's
> fairly good at the simple task that it does, but it's becoming harder
> and harder to fix bugs and extend functionality in it.  It's also
> fairly unfriendly to those of us who haven't been using it since 1995.
> The DFly folks have some very interesting work in this area
> (www.bsdinstaller.com) and it would be very good to see if we can
> collaborate with them on it.

If we're going to discuss admin-friendiness...

- Resplit distribution sets. It's simply shame when unpacking base overwrites
existing log files (e.g. during binary upgrade).
At least /etc and /var/log _must_ be separated. Also, I consider openbsd
scheme to be pleasant: separate build tools set (gcc, binutils, etc.)
from main functioning set.
As maximum, convert them into packages and register along with ports.

- Fix the most ugly sysinstall features. E.g. setting active flags on all
partitions by default, along with setting MBR, leads to unbootable system.
This was _constant_ students' error on FreeBSD administration courses.

- Provide more administrative tools. For now, one can't see interface queue
lengths, netgraph queue lens, tune their sizes, calculate memory occupied
by a bunch of processes without multiple counting shared pages...


-netch-
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Eric Kjeldergaard]

> 4.  Journaled filesystem.  While we can debate the merits of speed and
> data integrety of journalling vs. softupdates, the simple fact remains
> that softupdates still requires a fsck run on recovery, and the
> multi-terabyte filesystems that are possible these days make fsck a very
> long and unpleasant experience, even with bg-fsck.  There was work at
> some point at RPI to add journaling to UFS, but there hasn't been much
> status on that in a long time.  There have also been proposals and
> works-in-progress to port JFS, ReiserFS, and XFS.  Some of these efforts
> are still alive, but they need to be seen through to completion.  But at
> the risk of opening a can of worms here, I'll say that it's also
> important to explore non-GPL alternatives.

A thread 
(http://lists.freebsd.org/pipermail/freebsd-current/2004-October/040904.html
)happens to be talking about ro support for the ReiserFS.  After
having used this for quite some time in a Linux environment, I can't
help but notice that it seriously outperforms any other filesystem
I've tried on large numbers of littlish files.  A beautiful
application that I've rather wanted to try for a while was this on the
ports tree.

The stage of the current implementation is, as I said, read-only. 
Further, it's currently i386 only.  However, I think that there is
enough interest in this new (and relatively exciting) filesystem that
we may be able to find some developers with time (Possibly including
myself) and desire to try to implement write support and do some
porting.   To ease any qualms with regards to licensing, it appears
(http://lists.freebsd.org/pipermail/freebsd-current/2004-October/040998.html)
that the current implementation is BSD licensed.

-- 
If I write a signature, my emails will appear more personalised.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Wilko Bulte]

On Wed, Dec 01, 2004 at 04:37:59PM -0700, Scott Long wrote..
> Jason C. Wells wrote:
> >--On Wednesday, December 01, 2004 3:02 PM -0700 Scott Long 
> ><[EMAIL PROTECTED]> wrote:
> >
> >>5.  Clustered FS support.  SANs are all the rage these days, and
> >>clustered filesystems that allow data to be distributed across many
> >>storage enpoints and accessed concurrently through the SAN are very
> >>powerful.  RedHat recently bought Sistina and re-opened the GFS source
> >>code, so exploring this would be very interesting.
> >
> >
> >This sounds very close to OpenAFS.  I don't know what distinguishes a 
> >SAN from other types of NAS.  OpenAFS does everything you mentioned in 
> >the above paragraph.  OpenAFS _almost_ works on FreeBSD right now.
> >
> >Later,
> >Jason C. Wells
> 
> Well, AFS requires an intelligent node in front of each disk.  True SAN
> clustering means that you have a web of disks directly connected to the
> SAN (iSCSI, FibreChannel, etc), and two or more servers on the SAN that
> see those disks as a single filesystem (actually a bit more complicated
> than this, but you get the point).  If one server goes down, no access
> to data is lost since the disks can be reached from any other server on
> the SAN that is participating in the clustered FS.

Find a friendly TruCluster somewhere and take a look.  Really Neat(tm).
Alternatively find a friendly OpenVMS cluster, they have forgotten more
about clusters now than Unix will ever learn (I am afraid).

While we are talking storage: multipathing support for SANs is a
very neat thing to have.  Devices uniquely identified by WWN etc.

-- 
Wilko Bulte [EMAIL PROTECTED]


smime.p7s
Description: S/MIME cryptographic signature

Re: My project wish-list for the next 12 months

[Peter Jeremy]

On Thu, 2004-Dec-02 07:55:30 +0100, Miguel Mendez wrote:
>On Thu, 02 Dec 2004 10:44:32 +0530
>"Kamal R. Prasad" <[EMAIL PROTECTED]> wrote:
>> I find X windows to be a bit too compute intensive. Maybe something
>> like apple's interface would be a good alternative [for those who
>> don't need X-windows' powerful graphic features].
>
>What makes you think so? X was originally desgined for systems with
>little memory and processing power, certainly a lot less than today's
>AMD and Intel space heaters.

Agreed.  But I don't think performance is the issue with X.  As I see
it, there are several major problems with building an X installer:
1) It quite common in the server arena for machines not to have any
   graphics head and X is incompatible with serial terminals.
2) You need to configure the X server to support your video adapter,
   mouse, keyboard and screen.  Remember, the "standard" basic VGA
   interface doesn't necessarily exist outside the PC world.  There
   are enough problems with keyboards (see one of Scott's other wishes)
   without wanting to add mice, screens and video adapters.
3) /stand is ~2.7M on i386.  A minimal X environment is going to be
   50-70MB.  This means 50-70MB less packages on CD1.
4) X is a RAM hog by sysinstall standards.  The minimum RAM requirements
   will go up significantly.  Whilst this shouldn't worry current
   generation hardware, it will make installing FreeBSD on older hardware
   (486 and P1) very difficult.

Yes, X is network aware but this doesn't really help for system
installation.  You might solve points 1 and 2 but you replace them
with the issue of how to bring up the network and arrange appropriate
client/server communication and authentication.

-- 
Peter Jeremy
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Kamal R. Prasad]

Andre Oppermann wrote:
Sam wrote:
On Thu, 2 Dec 2004, Andre Oppermann wrote:
Scott Long wrote:
5.  Clustered FS support.  SANs are all the rage these days, and
clustered filesystems that allow data to be distributed across many
storage enpoints and accessed concurrently through the SAN are very
powerful.  RedHat recently bought Sistina and re-opened the GFS source
code, so exploring this would be very interesting.

There are certain steps that can be be taken one at a time.  For 
example
it should be relatively easy to mount snapshots (ro) from more than one
machine.  Next step would be to mount a full 'rw' filesystem as 'ro' on
other boxes.  This would require cache and sector invalidation 
broadcasting
from the 'rw' box to the 'ro' mounts.  The holy grail of course is 
to mount
the same filesystem 'rw' on more than one box, preferrably more than 
two.
This requires some more involved synchronization and locking on top 
of the
cache invalidation.  And make sure that the multi-'rw' cluster stays 
alive
if one of the participants freezes and doesn't respond anymore.

Scrolling through the UFS/FFS code I think the first one is 2-3 days of
work.  The second 2-4 weeks and the third 2-3 month to get it right.
If someone would throw up the money...

You might also design in consideration for data redundancy.  Right now
GFS largely relies on the SAN box to export already redundant RAID
disks.  GFS sits on a "cluster aware" lvm layer that is supposed to
be able to do mirroring and striping, but I'm told it's not
stable enough for "production" use.

Data redundancy would require a UFS/FFS redesign.  I'm 'only' talking
about enhancing UFS/FFS but keeping anything ondisk the same (plus
some more elements).
If you add redundancy code into UFS/FFS, that will slow down its 
performance (even for those not seeking redundancy).
A better way would  be to have another filesystem implementation like 
VxFS (veritas filesystem). Im not sure if they have published papers/put 
their techniques into public domain.

regards
-kamal
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Andre Oppermann]

Sam wrote:
On Thu, 2 Dec 2004, Andre Oppermann wrote:
Scott Long wrote:
5.  Clustered FS support.  SANs are all the rage these days, and
clustered filesystems that allow data to be distributed across many
storage enpoints and accessed concurrently through the SAN are very
powerful.  RedHat recently bought Sistina and re-opened the GFS source
code, so exploring this would be very interesting.
There are certain steps that can be be taken one at a time.  For example
it should be relatively easy to mount snapshots (ro) from more than one
machine.  Next step would be to mount a full 'rw' filesystem as 'ro' on
other boxes.  This would require cache and sector invalidation 
broadcasting
from the 'rw' box to the 'ro' mounts.  The holy grail of course is to 
mount
the same filesystem 'rw' on more than one box, preferrably more than two.
This requires some more involved synchronization and locking on top of 
the
cache invalidation.  And make sure that the multi-'rw' cluster stays 
alive
if one of the participants freezes and doesn't respond anymore.

Scrolling through the UFS/FFS code I think the first one is 2-3 days of
work.  The second 2-4 weeks and the third 2-3 month to get it right.
If someone would throw up the money...
You might also design in consideration for data redundancy.  Right now
GFS largely relies on the SAN box to export already redundant RAID
disks.  GFS sits on a "cluster aware" lvm layer that is supposed to
be able to do mirroring and striping, but I'm told it's not
stable enough for "production" use.
Data redundancy would require a UFS/FFS redesign.  I'm 'only' talking
about enhancing UFS/FFS but keeping anything ondisk the same (plus
some more elements).
--
Andre
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Sam]

On Thu, 2 Dec 2004, Andre Oppermann wrote:
Scott Long wrote:
5.  Clustered FS support.  SANs are all the rage these days, and
clustered filesystems that allow data to be distributed across many
storage enpoints and accessed concurrently through the SAN are very
powerful.  RedHat recently bought Sistina and re-opened the GFS source
code, so exploring this would be very interesting.
There are certain steps that can be be taken one at a time.  For example
it should be relatively easy to mount snapshots (ro) from more than one
machine.  Next step would be to mount a full 'rw' filesystem as 'ro' on
other boxes.  This would require cache and sector invalidation broadcasting
from the 'rw' box to the 'ro' mounts.  The holy grail of course is to mount
the same filesystem 'rw' on more than one box, preferrably more than two.
This requires some more involved synchronization and locking on top of the
cache invalidation.  And make sure that the multi-'rw' cluster stays alive
if one of the participants freezes and doesn't respond anymore.
Scrolling through the UFS/FFS code I think the first one is 2-3 days of
work.  The second 2-4 weeks and the third 2-3 month to get it right.
If someone would throw up the money...
You might also design in consideration for data redundancy.  Right now
GFS largely relies on the SAN box to export already redundant RAID
disks.  GFS sits on a "cluster aware" lvm layer that is supposed to
be able to do mirroring and striping, but I'm told it's not
stable enough for "production" use.
Sam
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Andre Oppermann]

Scott Long wrote:
5.  Clustered FS support.  SANs are all the rage these days, and
clustered filesystems that allow data to be distributed across many
storage enpoints and accessed concurrently through the SAN are very
powerful.  RedHat recently bought Sistina and re-opened the GFS source
code, so exploring this would be very interesting.
There are certain steps that can be be taken one at a time.  For example
it should be relatively easy to mount snapshots (ro) from more than one
machine.  Next step would be to mount a full 'rw' filesystem as 'ro' on
other boxes.  This would require cache and sector invalidation broadcasting
from the 'rw' box to the 'ro' mounts.  The holy grail of course is to mount
the same filesystem 'rw' on more than one box, preferrably more than two.
This requires some more involved synchronization and locking on top of the
cache invalidation.  And make sure that the multi-'rw' cluster stays alive
if one of the participants freezes and doesn't respond anymore.
Scrolling through the UFS/FFS code I think the first one is 2-3 days of
work.  The second 2-4 weeks and the third 2-3 month to get it right.
If someone would throw up the money...
--
Andre
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Adam Maloney]

On Thu, 2 Dec 2004, Brad Knowles wrote:
	It's interesting that you mention this.  I've been giving some 
thought to how I might be able to dive in and start seriously working on 
building my UltraSPARC cluster (based on the four U10 clones I have already, 
plus as many U5s as I can throw into the mix), and I was hoping to find a 
better solution than NFS, and AFS/Coda/OpenAFS was tops of my list of 
alternatives to consider.

I would be very excited to see OpenAFS become production ready on BSD.  I 
was playing with CODA a few weeks ago in a test environment.  I could get 
it to mostly work the way I wanted, but it appears that there are some 
limitations that I don't like.  For one, having to "login" to CODA using 
clog (or maybe I misunderstood the docs on this point?)  I want to be able 
to list a clustered filesystem in fstab and be usable like any other 
UFS or NFS filesystem - no logging in, permissions and ownership work, 
etc.

Better yet, an approach like Google's File System.  If I run out of space 
or speed, let me throw more boxes at it.  Without losing the filesystem. 
The machine comes onto the network, notifies one of the chunk servers that 
it's available and how much disk it's got.  The chunk servers can now send 
chunks of data to it.  Data is automatically replicated to multiple disk 
servers, and requests are shared across those servers that have copies of 
the same data.  Chunk servers share metadata so they aren't a single point 
of failure.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Danny Braniss]

> As far as the iSCSI stuff, I have the Lucent stuff and am trying to use it
> as a reference to build an iSCSI target. I have been experimenting a bit.
> 
> The design goal is to have the negotiation stuff running in a user daemon,
> while the target data handling is completely in the kernel. I was thinking
> about using netgraph for the network side of things. On the "disk" side of
> things I am thinking about a geom provider, but not initially. Initially I
> will send the SCSI CDBs directly to a tape drive that is used for testing
> purposes. The project I need this for is to offer a FreeBSD connected tape
> library to a Windoze box with iSCSI and use native NTBACKUP.
> 
> Later on the FreeBSD target will be geom based.
> 
> Peter

I would like to help/cooperate, I have a netapp with iSCSI awarness, and if
necessary i can get a SAN/iSCSI, and probably some time too.

danny


___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Garrett Wollman]

[Cc list trimmed]

< said:

> The lack of speed in some apps can be blamed mostly on the toolkits.

I'll second that.

> GTK+ 1.2 was a speed demon, GTK+ 2.x is a lot slower.

And either one is an enormous hog compared to Athena widgets.  (This
is something of an accomplishment, since people have been complaining
about the efficiency of Xt since it was first released nearly 20 years
ago.  All the credit goes to Gordon Moore: X was slow on a 1-MIPS
MicroVAX or Sun-2 workstation.)

I spend almost all of my time in two applications: xterm and emacs.
Both once set records for obscene memory consumption on an earlier
generation of systems.

-GAWollman

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Brad Knowles]

At 3:38 PM -0800 2004-12-01, Kris Kennaway quoted Jason C. Wells:
 This sounds very close to OpenAFS.  I don't know what distinguishes a SAN
 from other types of NAS.  OpenAFS does everything you mentioned in the
 above paragraph.  OpenAFS _almost_ works on FreeBSD right now.
 I'd be very interested to try using this for package builds, btw.
 Currently I have to rsync a lot of data to the remote build clients,
 which takes a very long time.
	It's interesting that you mention this.  I've been giving some 
thought to how I might be able to dive in and start seriously working 
on building my UltraSPARC cluster (based on the four U10 clones I 
have already, plus as many U5s as I can throw into the mix), and I 
was hoping to find a better solution than NFS, and AFS/Coda/OpenAFS 
was tops of my list of alternatives to consider.

	In particular, if I can get this thing working reasonably well, 
I'd like to turn this into a package building cluster for 
FreeBSD/UltraSPARC, and maybe see if there are some other 
applications I can put it to during the idle times.

	I'm still looking for instructions on how the existing FreeBSD 
package building clusters are put together.  That plus various 
bits-n-pieces I've found (mostly from Brooks) should help me figure 
out how to try to build my own.

--
Brad Knowles, <[EMAIL PROTECTED]>
"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."
-- Benjamin Franklin (1706-1790), reply of the Pennsylvania
Assembly to the Governor, November 11, 1755
  SAGE member since 1995.  See  for more info.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Mark Linimon]

On Wed, 1 Dec 2004, Scott Long wrote:

> 2.  New installer.  [... sysinstall] is fairly good at the simple
> task that it does [ ... ]

I'll put my bugmeister-hat on and simply say that query-pr suggests
otherwise.  I have not spent sufficient time examining each of the
PRs to figure out what the breakdown is between 'bugs within sysinstall'/
'unable to boot FreeBSD on hardware'/'user error' but IMHO the first
group contains more than a handful of PRs.  (the total # is 142.)

This is not to say we should abandon the KISS principle and try to
come up with some all-singing/all-dancing "thing"; in fact, the
opposite.  I'd rather we spent time on making something small and
solid which would contain enough of its own documentation to prevent
people from tearing their hair out while trying to use it.  Unlike
much of the rest of the system where we assume users have at least
some familiarity with FreeBSD (and a working browser), we have to
engineer an installer that assumes that both of those are false.

Unfortunately for me I probably will never be able to work on this
unless someone wants to pay me to work on FreeBSD full-time; too
many other things are ahead of it in my personal FreeBSD queue ...

mcl

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Garrett Wollman]

< said:

> This sounds very close to OpenAFS.  I don't know what distinguishes a SAN 
> from other types of NAS.  OpenAFS does everything you mentioned in the 
> above paragraph.  OpenAFS _almost_ works on FreeBSD right now.

AFS's consistency model is wholly unsuitable for clustering.

-GAWollman

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Jason C. Wells]

--On Wednesday, December 01, 2004 3:02 PM -0700 Scott Long 
<[EMAIL PROTECTED]> wrote:

5.  Clustered FS support.  SANs are all the rage these days, and
clustered filesystems that allow data to be distributed across many
storage enpoints and accessed concurrently through the SAN are very
powerful.  RedHat recently bought Sistina and re-opened the GFS source
code, so exploring this would be very interesting.
This sounds very close to OpenAFS.  I don't know what distinguishes a SAN 
from other types of NAS.  OpenAFS does everything you mentioned in the 
above paragraph.  OpenAFS _almost_ works on FreeBSD right now.

Later,
Jason C. Wells
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Michael Nottebrock]

On Wednesday, 1. December 2004 23:14, Andre Oppermann wrote:
> Scott Long wrote:
> > All,
> >
> > I know that I said last month that we were going to stop promising
> > specific features for the next major release.  However, I'd like to
> > throw out a list of things that would be really nice to have in the
> > future, whether its 6.0 or 7.0 or whatever.  Most of these tasks are
> > not trivial, but I hope that talking about them will encourage some
> > interest.  These are in no particular priority order.  I'd also be
> > thrilled if someone wanted to dress this list up in docbook and add
> > it to the webpage.  While this is just my personal list, I'd welcome
> > other additions to it (in the sense of significant projects, not just

Suspend-to-Disk support. I remember somebody thinking aloud about howit would 
be possible to implement it, Linux has got it recently, and it's one heck of 
a feature for mobile computing (but not only).

-- 
   ,_,   | Michael Nottebrock   | [EMAIL PROTECTED]
 (/^ ^\) | FreeBSD - The Power to Serve | http://www.freebsd.org
   \u/   | K Desktop Environment on FreeBSD | http://freebsd.kde.org


pgpOc6bBWrGhB.pgp
Description: PGP signature

RE: My project wish-list for the next 12 months

[Peter Blok]

As far as the iSCSI stuff, I have the Lucent stuff and am trying to use it
as a reference to build an iSCSI target. I have been experimenting a bit.

The design goal is to have the negotiation stuff running in a user daemon,
while the target data handling is completely in the kernel. I was thinking
about using netgraph for the network side of things. On the "disk" side of
things I am thinking about a geom provider, but not initially. Initially I
will send the SCSI CDBs directly to a tape drive that is used for testing
purposes. The project I need this for is to offer a FreeBSD connected tape
library to a Windoze box with iSCSI and use native NTBACKUP.

Later on the FreeBSD target will be geom based.

Peter

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Scott Long
Sent: Wednesday, December 01, 2004 11:03 PM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: My project wish-list for the next 12 months

All,

I know that I said last month that we were going to stop promising
specific features for the next major release.  However, I'd like to
throw out a list of things that would be really nice to have in the
future, whether its 6.0 or 7.0 or whatever.  Most of these tasks are
not trivial, but I hope that talking about them will encourage some
interest.  These are in no particular priority order.  I'd also be
thrilled if someone wanted to dress this list up in docbook and add
it to the webpage.  While this is just my personal list, I'd welcome
other additions to it (in the sense of significant projects, not just
individual PRs or bug fixes that one might be interested in).

1.  Keyboard multiplexer.  We are running into problems with making
ps/2 and USB/bluetooth keyboards work together and work with KVMs.
Having a virtual keyboard device that multiplexes the various real
keyboard devices and handles hotplug can solve this mess pretty
effectively.  I know that there has been a lot of talk about this on
mailing lists recently but I don't know how much progress is being made
so I'm listing it here.

2.  New installer.  I know some people still consider this a joke, but
the reality is that sysinstall is no longer state of the art.  It's
fairly good at the simple task that it does, but it's becoming harder
and harder to fix bugs and extend functionality in it.  It's also
fairly unfriendly to those of us who haven't been using it since 1995.
The DFly folks have some very interesting work in this area
(www.bsdinstaller.com) and it would be very good to see if we can
collaborate with them on it.

3.  Native PCI Express support.  I keep on hoping to take care of this,
but I never seem to have the time to get past designing it.  This task
includes 3 parts that are mostly independent.  The first is support for
the extended PCI config space and memio access method, the second is
MSI, and the third is link QOS management.  If anyone is interested
here, please let me know.

4.  Journaled filesystem.  While we can debate the merits of speed and
data integrety of journalling vs. softupdates, the simple fact remains
that softupdates still requires a fsck run on recovery, and the
multi-terabyte filesystems that are possible these days make fsck a very
long and unpleasant experience, even with bg-fsck.  There was work at
some point at RPI to add journaling to UFS, but there hasn't been much
status on that in a long time.  There have also been proposals and
works-in-progress to port JFS, ReiserFS, and XFS.  Some of these efforts
are still alive, but they need to be seen through to completion.  But at
the risk of opening a can of worms here, I'll say that it's also
important to explore non-GPL alternatives.

5.  Clustered FS support.  SANs are all the rage these days, and
clustered filesystems that allow data to be distributed across many
storage enpoints and accessed concurrently through the SAN are very
powerful.  RedHat recently bought Sistina and re-opened the GFS source
code, so exploring this would be very interesting.

6.  Overhaul CAM, add iSCSI.  CAM is very parallel-SCSI centric right
now.  I have some work-in-progress in Perforce to address this, but it's
pretty minimal.  The parallel SCSI knowledge needs to be separated out
and the stack need to be able to cleanly deal with iSCSI, SCSI, SAS, and
maybe even ATA transports.  There is a Lucent implementation of iSCSI
for FreeBSD 4.x that could be a useful reference, though it's a
monolithic stack that doesn't really address the shortcomings of CAM.
Having iSCSI infrastructure that supported both hardware and software
implementations would be ideal.

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Poul-Henning Kamp]

In message <[EMAIL PROTECTED]>, Scott Long writes:
>All,
>
>I know that I said last month that we were going to stop promising
>specific features for the next major release.  However, I'd like to
>throw out a list of things that would be really nice to have in the
>future, whether its 6.0 or 7.0 or whatever.
>[...]
>1.  Keyboard multiplexer.  [...]
>2.  New installer.  [...]
>3.  Native PCI Express support.  [...]
>4.  Journaled filesystem.  [...]
>5.  Clustered FS support.  [...]
>6.  Overhaul CAM, add iSCSI.  [...]

I agree on all of the above but I think we also need to have things
on the list that doesn't take super hackers, and architectural
reviews.

So let us add the following points which I think are just as, if
not more important for FreeBSDs future:

7. More people writing FreeBSD related articles for online and
   traditional media.

   If you have never written a piece about FreeBSD, how about sending
   something to your local IT trade rag ?   Heck, even your local
   paper will probably run it if you send them a piece.


8. More people stomping for FreeBSD in universities and schools.

   Have you actually offered the science/IT teachers at the local
   hi-school to pop around and give a lesson on this open source
   phenomena to their pupils ?

   Or call your local college and ask if they need a teaching
   assistant for their evening courses in IT ?


9. A band of happy 1st line reponders dealing with PRs etc.

   We're getting better at this, but one way to really gain users
   is to help them when they need it most: right when they begin.


10. A more dynamic and interesting www.freebsd.org frontpage.

Come on, at least we should be able to beat the "Congressional
Record" when it comes to being interesting.


11. More people attending BSDcon conferences.

Come to BSDcan2005! come to the next EuroBSDcon or AsiaBSDcon.

Or make your own mini conference!

Many Linux User groups would welcome you if you offered to give
a talk about FreeBSD on one of their evenings.


12. Research/Coding grants (3/6/12 months) from the FreeBSD Foundation
and other deep(er) pockets to help some of the heavy lifting happen.

We're not only in it for the money, but money surely helps.


And I'd like to stress that none of the above requires you to get
permission from the core team, just go out and do it!

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Eric Masson]

> "Scott" == Scott Long <[EMAIL PROTECTED]> writes:

Hi,

 Scott> 1. Keyboard multiplexer. We are running into problems with
 Scott> making ps/2 and USB/bluetooth keyboards work together and work
 Scott> with KVMs. Having a virtual keyboard device that multiplexes the
 Scott> various real keyboard devices and handles hotplug can solve this
 Scott> mess pretty effectively. I know that there has been a lot of
 Scott> talk about this on mailing lists recently but I don't know how
 Scott> much progress is being made so I'm listing it here.

NetBSD's wscons seems to fit in this scheme, OpenBSD has adopted it as
well (the most featured implementation is in NetBSD/i386).

 Scott> 6. Overhaul CAM, add iSCSI. CAM is very parallel-SCSI centric
 Scott> right now. I have some work-in-progress in Perforce to address
 Scott> this, but it's pretty minimal. The parallel SCSI knowledge needs
 Scott> to be separated out and the stack need to be able to cleanly
 Scott> deal with iSCSI, SCSI, SAS, and maybe even ATA transports. There
 Scott> is a Lucent implementation of iSCSI for FreeBSD 4.x that could
 Scott> be a useful reference, though it's a monolithic stack that
 Scott> doesn't really address the shortcomings of CAM. Having iSCSI
 Scott> infrastructure that supported both hardware and software
 Scott> implementations would be ideal.

It would be really nice to be able to play in the iscsi field. It seems
that work has been done in this area by Wasabi (not published in NetBSD,
iirc)

Very appealing goals, if only I were a decent coder...

Éric Masson

-- 
 je pense pas que ce soit toitu es bien trop vicieux pour agir de
 cette façon. Toi ton genre, c'est plus de contacter banque direct en
 esperant que je n'auras pas mes cadeaux de parrainages!
 -+- JD in  : Petit neuneu Noël -+-
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Devon H. O'Dell]

Ryan Sommers wrote:
Scott Long said:
2.  New installer.  I know some people still consider this a joke, but
the reality is that sysinstall is no longer state of the art.  It's
fairly good at the simple task that it does, but it's becoming harder
and harder to fix bugs and extend functionality in it.  It's also
fairly unfriendly to those of us who haven't been using it since 1995.
The DFly folks have some very interesting work in this area
(www.bsdinstaller.com) and it would be very good to see if we can
collaborate with them on it.

I've spent a good deal of time taking notes and diagrams of what I wanted
from a new installer. However, time constraints have kept me from actually
putting any of it to code yet.
I've looked at the DFly installed quite a bit and I like what it offers,
however, I have a few complaints with it. Quite honestly I wasn't
impressed with the code.
Another issue I had with the dfly installer was one point I believe needs
to be central to any next-gen installer. Internationalisation. My idea of
an installer front-end would use a dynamically loadable language library.
All resources of the front-end (ie strings, images, etc) would be packaged
into a seperate language-pack. These language-packs can then be grouped
together into a language library. A few basic packs would be distributed
with the default library but other libraries could easily be substituted
to make localized distribution sets with little trouble.
The benefit of this is that instead of translating the code you would only
need to translate the language-(pack|library). I think this would greatly
simplify translation and make a seperation between language and the
front-end code.
This is where my complaint with Dfly comes in, upon reading the source,
there are string constants everywhere. Perhaps I am missing something, but
this means that in order to supply localization support much work would
need to be done to find some scheme that doesn't mean translating the
source.
I have quite a bit of notes on seperation and even down to specific
methods and sub-libraries necessary for the back-end. Perhaps if I have
some time soon I'll put it into a PDF somewhere.
Has anyone else put much thought into this?
Yes, we have. I18n is something that we're actively working on 
implementing, and is something that we take quite seriously. I know that 
not very many FreeBSD developers use IRC, but we are all available in 
#dfinstaller on EFNet. We're using gettext for this at the moment.

Kind regards,
Devon H. O'Dell
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Miguel Mendez]

On Thu, 02 Dec 2004 10:44:32 +0530
"Kamal R. Prasad" <[EMAIL PROTECTED]> wrote:

[Please don't top-post]

> I find X windows to be a bit too compute intensive. Maybe something
> like apple's interface would be a good alternative [for those who
> don't need X-windows' powerful graphic features].

What makes you think so? X was originally desgined for systems with
little memory and processing power, certainly a lot less than today's
AMD and Intel space heaters. There are some features that do indeed
require more CPU, like antialiasing. That's the price to pay for eye
candy. Things like the composite and damage extensions do wonders to
help in those areas and make things like true transparency and alpha
blending possible. So, in time, X won't be that different from Aqua in
its use of hardware.

The lack of speed in some apps can be blamed mostly on the toolkits.
GTK+ 1.2 was a speed demon, GTK+ 2.x is a lot slower. There are some
people working on a fast Pango code path that could make english text
rendering fast again.

X gives you network transparency out of the box. I used an old SGI Indy
as an X term to connect to my FreeBSD box for years, and it worked like
a charm over a 10Mbit connection.

Replacing X means writing something that's API compatible, or writing an
X server on top of your new display system, so that you don't have to
throw the thousands of X apps into the trashcan.

About some of the other comments in the thread. IMHO using bsdinstaller
as a base could be beneficial for both groups and save duplicated work.
I'm sure they'll be glad to get internacionalization patches.

I agree that a journaled filesystem is really needed if you want to
manage multi TiB partitions. As rock solid and tested softupdates is,
fsck is not an acceptable solution in this case. Some people were
working on a port of XFS, has any progress been made? I tried contacting
them some time ago, but never got an answer. Sounds like a very
interesting, though.

Cheers,
-- 
Miguel Mendez <[EMAIL PROTECTED]> | lea gfx_lib(pc),a1
http://www.energyhq.es.eu.org| moveq   #0,d0
PGP Key: 0xDC8514F1  | move.l  4.w,a6
Note: All HTML mail goes to /dev/null| jsr -552(a6)



pgpciGqFVPARp.pgp
Description: PGP signature

Re: My project wish-list for the next 12 months

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
John Hay <[EMAIL PROTECTED]> writes:
: > > 1.  Keyboard multiplexer.
: > 
: > I actually fail to stop thinking about a complete syscons and pcvt
: > replacement. You know, the one and only console implementation that
: > makes all others obsolete. Big plans, little time, yada yada yada...
: 
: It would be nice if one would still be able to use the keyboards
: separately too, even if you have to recompile the kernel for that.
: One nice usage would be on HP's quad kiosk machine. It is a single
: processor box with 4 x screen, keyboard and mouse, and then 4 people
: can use it.

I think that making it just another driver that knows about the
keyboard mux would make this possible in the short term...

Hmmm, maybe it is time for me to STFU and hack together what I'm
thinking :-)

Warner

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[John Hay]

> > 
> > 1.  Keyboard multiplexer.
> 
> I actually fail to stop thinking about a complete syscons and pcvt
> replacement. You know, the one and only console implementation that
> makes all others obsolete. Big plans, little time, yada yada yada...

It would be nice if one would still be able to use the keyboards
separately too, even if you have to recompile the kernel for that.
One nice usage would be on HP's quad kiosk machine. It is a single
processor box with 4 x screen, keyboard and mouse, and then 4 people
can use it.

John
-- 
John Hay -- [EMAIL PROTECTED] / [EMAIL PROTECTED]
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Kamal R. Prasad]

I find X windows to be a bit too compute intensive. Maybe something like 
apple's interface would be a good alternative [for those who don't need 
X-windows' powerful graphic features].

regards
-kamal
Scott Long wrote:
Jason C. Wells wrote:
--On Wednesday, December 01, 2004 3:02 PM -0700 Scott Long 
<[EMAIL PROTECTED]> wrote:

5.  Clustered FS support.  SANs are all the rage these days, and
clustered filesystems that allow data to be distributed across many
storage enpoints and accessed concurrently through the SAN are very
powerful.  RedHat recently bought Sistina and re-opened the GFS source
code, so exploring this would be very interesting.

This sounds very close to OpenAFS.  I don't know what distinguishes a 
SAN from other types of NAS.  OpenAFS does everything you mentioned 
in the above paragraph.  OpenAFS _almost_ works on FreeBSD right now.

Later,
Jason C. Wells

Well, AFS requires an intelligent node in front of each disk.  True SAN
clustering means that you have a web of disks directly connected to the
SAN (iSCSI, FibreChannel, etc), and two or more servers on the SAN that
see those disks as a single filesystem (actually a bit more complicated
than this, but you get the point).  If one server goes down, no access
to data is lost since the disks can be reached from any other server on
the SAN that is participating in the clustered FS.
Scott
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to 
"[EMAIL PROTECTED]"

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[M. Warner Losh]

In message: <[EMAIL PROTECTED]>
Scott Long <[EMAIL PROTECTED]> writes:
: 1.  Keyboard multiplexer.  We are running into problems with making
: ps/2 and USB/bluetooth keyboards work together and work with KVMs.
: Having a virtual keyboard device that multiplexes the various real
: keyboard devices and handles hotplug can solve this mess pretty
: effectively.  I know that there has been a lot of talk about this on
: mailing lists recently but I don't know how much progress is being made
: so I'm listing it here.

There aready are multiplexers in the kernel.  The problem is that we
need a many to one mux that is the OR of all the ones installed.  We
can current set WHICH keyboard is connected to the mux, but can't say
'ALL OF THEM' at all.  I believe that Brooks Davis has said he's
working on this.

Warner
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Scott Long]

Foxfair Hu wrote:
On Wed, Dec 01, 2004 at 03:02:40PM -0700, Scott Long wrote:
All,
[]
1.  Keyboard multiplexer.  We are running into problems with making
ps/2 and USB/bluetooth keyboards work together and work with KVMs.
Having a virtual keyboard device that multiplexes the various real
keyboard devices and handles hotplug can solve this mess pretty
effectively.  I know that there has been a lot of talk about this on
mailing lists recently but I don't know how much progress is being made
so I'm listing it here.

   How about reuse NetBSD's wscons ? I've kept an eye on it and thought
it should be a good start for FreeBSD.
foxfair
If it provides keyboard mux'ing like we need, then please go and
give it a shot.
Scott
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Peter Kieser]

Scott Long wrote:
2.  New installer.  I know some people still consider this a joke, but
the reality is that sysinstall is no longer state of the art.  It's
fairly good at the simple task that it does, but it's becoming harder
and harder to fix bugs and extend functionality in it.  It's also
fairly unfriendly to those of us who haven't been using it since 1995.
The DFly folks have some very interesting work in this area
(www.bsdinstaller.com) and it would be very good to see if we can
collaborate with them on it.

Please, don't change /stand/sysinstall *too* much, there is really 
nothing wrong with the interface of it, and it's what makes FreeBSD so 
"quick" to install. At the very least, make sure you do NOT go for an 
XFree86 installation, and keep to the "KISS" approach. Visually wise, 
theres nothing wrong with the current installer.. and its one of the 
things I "promote" about FreeBSD -- the ease to install. It's small, its 
fast.. and it works, however in error situations it does mess up badly.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Foxfair Hu]

On Wed, Dec 01, 2004 at 03:02:40PM -0700, Scott Long wrote:
> All,
> 
[]
> 
> 1.  Keyboard multiplexer.  We are running into problems with making
> ps/2 and USB/bluetooth keyboards work together and work with KVMs.
> Having a virtual keyboard device that multiplexes the various real
> keyboard devices and handles hotplug can solve this mess pretty
> effectively.  I know that there has been a lot of talk about this on
> mailing lists recently but I don't know how much progress is being made
> so I'm listing it here.

   How about reuse NetBSD's wscons ? I've kept an eye on it and thought
it should be a good start for FreeBSD.


foxfair



pgphRpubvLcFv.pgp
Description: PGP signature

Re: My project wish-list for the next 12 months

[Scott Long]

Marcel Moolenaar wrote:
On Wed, Dec 01, 2004 at 03:02:40PM -0700, Scott Long wrote:
1.  Keyboard multiplexer.

I actually fail to stop thinking about a complete syscons and pcvt
replacement. You know, the one and only console implementation that
makes all others obsolete. Big plans, little time, yada yada yada...

2.  New installer.

It may actually be interesting to see if we can make an expert
system for this. When I think about implementing an installer (alas
I've been doing that), I'm not so much interested in how things are
packaged, or how it looks but rather what needs to be done, when and
how all these actions relate and interact with each other. This is
especially tricky when actions are triggered by the current
configuration of the machine onto which one tries to install. Knowing
all the possible activities and their dependencies should help
establish a control flow through the installation process in such a
way that users get asked only those questions that are relevent and
also when it matters. One puts a UI on top of this to get a nice
looking installer. At least, that's how I look at it...
Yeah, I've had many similar thoughts.  The hard part of a new installer,
or any complex UI application, is the framework that ties events and
actions together.  The easy part is writing the modules on top of that
that do the discrete actions.  While I see quite a few rough edges in
the upper layers of the DF installer, it seems like quite a bit of work
is going into the framework, and that's why I'm actually so interested
in it.
Scott
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Marcel Moolenaar]

On Wed, Dec 01, 2004 at 03:02:40PM -0700, Scott Long wrote:
> 
> 1.  Keyboard multiplexer.

I actually fail to stop thinking about a complete syscons and pcvt
replacement. You know, the one and only console implementation that
makes all others obsolete. Big plans, little time, yada yada yada...

> 2.  New installer.

It may actually be interesting to see if we can make an expert
system for this. When I think about implementing an installer (alas
I've been doing that), I'm not so much interested in how things are
packaged, or how it looks but rather what needs to be done, when and
how all these actions relate and interact with each other. This is
especially tricky when actions are triggered by the current
configuration of the machine onto which one tries to install. Knowing
all the possible activities and their dependencies should help
establish a control flow through the installation process in such a
way that users get asked only those questions that are relevent and
also when it matters. One puts a UI on top of this to get a nice
looking installer. At least, that's how I look at it...

-- 
 Marcel Moolenaar USPA: A-39004  [EMAIL PROTECTED]
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Dan Nelson]

In the last episode (Dec 01), Jason C. Wells said:
> --On Wednesday, December 01, 2004 3:02 PM -0700 Scott Long 
> <[EMAIL PROTECTED]> wrote:
> 
> >5.  Clustered FS support.  SANs are all the rage these days, and
> >clustered filesystems that allow data to be distributed across many
> >storage enpoints and accessed concurrently through the SAN are very
> >powerful.  RedHat recently bought Sistina and re-opened the GFS source
> >code, so exploring this would be very interesting.
> 
> This sounds very close to OpenAFS.  I don't know what distinguishes a
> SAN from other types of NAS.  OpenAFS does everything you mentioned
> in the above paragraph.  OpenAFS _almost_ works on FreeBSD right now.

OpenAFS is a network-centric system that replicates data across its
nodes, I think, and each node has a cache.  A clustered filesystem uses
a single block of shared storage (usually on a fibre-channel SAN, but
you can also use shared scsi on a 2-machine cluster) that all servers
access directly.  The magic is getting the locking right to make sure
the servers don't stomp on each other's data.  Extremely useful for
server farms that need to share large files, or even lots of small
files (webservers for example).

-- 
Dan Nelson
[EMAIL PROTECTED]
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Scott Long]

Jason C. Wells wrote:
--On Wednesday, December 01, 2004 3:02 PM -0700 Scott Long 
<[EMAIL PROTECTED]> wrote:

5.  Clustered FS support.  SANs are all the rage these days, and
clustered filesystems that allow data to be distributed across many
storage enpoints and accessed concurrently through the SAN are very
powerful.  RedHat recently bought Sistina and re-opened the GFS source
code, so exploring this would be very interesting.

This sounds very close to OpenAFS.  I don't know what distinguishes a 
SAN from other types of NAS.  OpenAFS does everything you mentioned in 
the above paragraph.  OpenAFS _almost_ works on FreeBSD right now.

Later,
Jason C. Wells
Well, AFS requires an intelligent node in front of each disk.  True SAN
clustering means that you have a web of disks directly connected to the
SAN (iSCSI, FibreChannel, etc), and two or more servers on the SAN that
see those disks as a single filesystem (actually a bit more complicated
than this, but you get the point).  If one server goes down, no access
to data is lost since the disks can be reached from any other server on
the SAN that is participating in the clustered FS.
Scott
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Kris Kennaway]

On Wed, Dec 01, 2004 at 03:29:10PM -0800, Jason C. Wells wrote:
> --On Wednesday, December 01, 2004 3:02 PM -0700 Scott Long 
> <[EMAIL PROTECTED]> wrote:
> 
> >5.  Clustered FS support.  SANs are all the rage these days, and
> >clustered filesystems that allow data to be distributed across many
> >storage enpoints and accessed concurrently through the SAN are very
> >powerful.  RedHat recently bought Sistina and re-opened the GFS source
> >code, so exploring this would be very interesting.
> 
> This sounds very close to OpenAFS.  I don't know what distinguishes a SAN 
> from other types of NAS.  OpenAFS does everything you mentioned in the 
> above paragraph.  OpenAFS _almost_ works on FreeBSD right now.

I'd be very interested to try using this for package builds, btw.
Currently I have to rsync a lot of data to the remote build clients,
which takes a very long time.

Kris


pgpfdwcYMSTUZ.pgp
Description: PGP signature

Re: My project wish-list for the next 12 months

[Scott Long]

Kris Kennaway wrote:
On Wed, Dec 01, 2004 at 04:10:57PM -0700, Ryan Sommers wrote:

Another issue I had with the dfly installer was one point I believe needs
to be central to any next-gen installer. Internationalisation.

Careful not to pile on so many wishes that achieving anything becomes
impossible.  Our current installer doesn't do this, so it's not a hard
requirement that a better installer should.
Kris
Internationalization is actually quite important, and is not easy to
bolt on after the fact but is fairly easy to program to once the core is
in place.  The fact that sysinstall doesn't have it makes it no less
important.  Now this isn't a reason to reject the DFly work, but it
could certainly be a good area for someone to contribute.
Scott
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Kris Kennaway]

On Wed, Dec 01, 2004 at 04:10:57PM -0700, Ryan Sommers wrote:

> Another issue I had with the dfly installer was one point I believe needs
> to be central to any next-gen installer. Internationalisation.

Careful not to pile on so many wishes that achieving anything becomes
impossible.  Our current installer doesn't do this, so it's not a hard
requirement that a better installer should.

Kris


pgp53kTmYZYNU.pgp
Description: PGP signature

Re: My project wish-list for the next 12 months

[Ryan Sommers]

Scott Long said:
> 2.  New installer.  I know some people still consider this a joke, but
> the reality is that sysinstall is no longer state of the art.  It's
> fairly good at the simple task that it does, but it's becoming harder
> and harder to fix bugs and extend functionality in it.  It's also
> fairly unfriendly to those of us who haven't been using it since 1995.
> The DFly folks have some very interesting work in this area
> (www.bsdinstaller.com) and it would be very good to see if we can
> collaborate with them on it.

I've spent a good deal of time taking notes and diagrams of what I wanted
from a new installer. However, time constraints have kept me from actually
putting any of it to code yet.

I've looked at the DFly installed quite a bit and I like what it offers,
however, I have a few complaints with it. Quite honestly I wasn't
impressed with the code.

Another issue I had with the dfly installer was one point I believe needs
to be central to any next-gen installer. Internationalisation. My idea of
an installer front-end would use a dynamically loadable language library.
All resources of the front-end (ie strings, images, etc) would be packaged
into a seperate language-pack. These language-packs can then be grouped
together into a language library. A few basic packs would be distributed
with the default library but other libraries could easily be substituted
to make localized distribution sets with little trouble.

The benefit of this is that instead of translating the code you would only
need to translate the language-(pack|library). I think this would greatly
simplify translation and make a seperation between language and the
front-end code.

This is where my complaint with Dfly comes in, upon reading the source,
there are string constants everywhere. Perhaps I am missing something, but
this means that in order to supply localization support much work would
need to be done to find some scheme that doesn't mean translating the
source.

I have quite a bit of notes on seperation and even down to specific
methods and sub-libraries necessary for the back-end. Perhaps if I have
some time soon I'll put it into a PDF somewhere.

Has anyone else put much thought into this?

-- 
Ryan Sommers
[EMAIL PROTECTED]

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: My project wish-list for the next 12 months

[Andre Oppermann]

Scott Long wrote:
All,
I know that I said last month that we were going to stop promising
specific features for the next major release.  However, I'd like to
throw out a list of things that would be really nice to have in the
future, whether its 6.0 or 7.0 or whatever.  Most of these tasks are
not trivial, but I hope that talking about them will encourage some
interest.  These are in no particular priority order.  I'd also be
thrilled if someone wanted to dress this list up in docbook and add
it to the webpage.  While this is just my personal list, I'd welcome
other additions to it (in the sense of significant projects, not just
individual PRs or bug fixes that one might be interested in).
1.  Keyboard multiplexer.  We are running into problems with making
ps/2 and USB/bluetooth keyboards work together and work with KVMs.
Having a virtual keyboard device that multiplexes the various real
keyboard devices and handles hotplug can solve this mess pretty
effectively.  I know that there has been a lot of talk about this on
mailing lists recently but I don't know how much progress is being made
so I'm listing it here.
2.  New installer.  I know some people still consider this a joke, but
the reality is that sysinstall is no longer state of the art.  It's
fairly good at the simple task that it does, but it's becoming harder
and harder to fix bugs and extend functionality in it.  It's also
fairly unfriendly to those of us who haven't been using it since 1995.
The DFly folks have some very interesting work in this area
(www.bsdinstaller.com) and it would be very good to see if we can
collaborate with them on it.
3.  Native PCI Express support.  I keep on hoping to take care of this,
but I never seem to have the time to get past designing it.  This task
includes 3 parts that are mostly independent.  The first is support for
the extended PCI config space and memio access method, the second is
MSI, and the third is link QOS management.  If anyone is interested
here, please let me know.
4.  Journaled filesystem.  While we can debate the merits of speed and
data integrety of journalling vs. softupdates, the simple fact remains
that softupdates still requires a fsck run on recovery, and the
multi-terabyte filesystems that are possible these days make fsck a very
long and unpleasant experience, even with bg-fsck.  There was work at
some point at RPI to add journaling to UFS, but there hasn't been much
status on that in a long time.  There have also been proposals and
works-in-progress to port JFS, ReiserFS, and XFS.  Some of these efforts
are still alive, but they need to be seen through to completion.  But at
the risk of opening a can of worms here, I'll say that it's also
important to explore non-GPL alternatives.
5.  Clustered FS support.  SANs are all the rage these days, and
clustered filesystems that allow data to be distributed across many
storage enpoints and accessed concurrently through the SAN are very
powerful.  RedHat recently bought Sistina and re-opened the GFS source
code, so exploring this would be very interesting.
6.  Overhaul CAM, add iSCSI.  CAM is very parallel-SCSI centric right
now.  I have some work-in-progress in Perforce to address this, but it's
pretty minimal.  The parallel SCSI knowledge needs to be separated out
and the stack need to be able to cleanly deal with iSCSI, SCSI, SAS, and
maybe even ATA transports.  There is a Lucent implementation of iSCSI
for FreeBSD 4.x that could be a useful reference, though it's a
monolithic stack that doesn't really address the shortcomings of CAM.
Having iSCSI infrastructure that supported both hardware and software
implementations would be ideal.
Seeing all this storage related stuff is very interesting because I just
stumbled across a company making a new digital Cinematography system that
has 8Mpix and they say that using this in a day's worth shooting they end
up with up to 6.63TB of raw digigal footage.  In the end this is 398TB for
an average feature movie.  The camera delivers the images over
quad-infiniband to the recording system at 400MB/s.  Pretty impressive.
 http://www.dalsa.com/dc/workflow/storage.asp
--
Andre
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"