Re: unhooking lfs from ufs

2010-02-14 Thread David Holland
On Sun, Feb 14, 2010 at 10:37:29PM +, David Holland wrote:
 > or maybe not, but still, there was a bunch of uvm hacking going on at
 > the time.

I've updated the kernels, merging tonight's HEAD, if you feel inclined
to check again:

SHA1 (netbsd-sparc64-GENERIC.gz) = fed7e0667c30fef16c25d175fa53884f0f3cb5a9
SHA1 (netbsd-sparc64-GENERIC.MP.gz) = 274ae5283b8c55aac045ad761fdcb4e32996e297

-- 
David A. Holland
dholl...@netbsd.org


Re: unhooking lfs from ufs

2010-02-14 Thread David Holland
On Sun, Feb 14, 2010 at 09:56:14PM +, David Holland wrote:
 >  > I hope you'll pardon me for being skeptical.
 > 
 > And indeed, this looks to have been an unrelated now-known problem
 > that affected the tree last weekend.

or maybe not, but still, there was a bunch of uvm hacking going on at
the time.

-- 
David A. Holland
dholl...@netbsd.org


Re: unhooking lfs from ufs

2010-02-14 Thread David Holland
On Wed, Feb 10, 2010 at 03:56:43PM +, David Holland wrote:
 > On Wed, Feb 10, 2010 at 12:50:59AM +, Eduardo Horvath wrote:
 > > I tried another build and it failed the same way.  `sync' has been broken 
 > > for me for some time, so I was unable to convince the kernel to dump core.
 > > 
 > > I then tried the same thing on my old kernel and it succeeded.
 > > 
 > > Based on this limited testing, your kernel does appear to have introduced 
 > > a bug to LFS.
 > 
 > ...you've exhibited a crash in UVM that appears to happen in execve
 > without going through the FS, and it's because I cut and pasted some
 > FS code?
 > 
 > I hope you'll pardon me for being skeptical.

And indeed, this looks to have been an unrelated now-known problem
that affected the tree last weekend.

-- 
David A. Holland
dholl...@netbsd.org


Re: unhooking lfs from ufs

2010-02-10 Thread David Holland
On Wed, Feb 10, 2010 at 12:50:59AM +, Eduardo Horvath wrote:
 > I tried another build and it failed the same way.  `sync' has been broken 
 > for me for some time, so I was unable to convince the kernel to dump core.
 > 
 > I then tried the same thing on my old kernel and it succeeded.
 > 
 > Based on this limited testing, your kernel does appear to have introduced 
 > a bug to LFS.

...you've exhibited a crash in UVM that appears to happen in execve
without going through the FS, and it's because I cut and pasted some
FS code?

I hope you'll pardon me for being skeptical.

-- 
David A. Holland
dholl...@netbsd.org


Re: unhooking lfs from ufs

2010-02-09 Thread Eduardo Horvath

Following up on myself :)

I tried another build and it failed the same way.  `sync' has been broken 
for me for some time, so I was unable to convince the kernel to dump core.

I then tried the same thing on my old kernel and it succeeded.

Based on this limited testing, your kernel does appear to have introduced 
a bug to LFS.

Eduardo


Re: unhooking lfs from ufs

2010-02-09 Thread Eduardo Horvath
On Tue, 9 Feb 2010, David Holland wrote:

> On Tue, Feb 09, 2010 at 04:14:58AM +, Eduardo Horvath wrote:
>  > I'm a bit low on disk space at the moment, but if you build me a generic 
>  > sparc64 kernel I can test it.
> 
> http://www.eecs.harvard.edu/~dholland/tmp/netbsd/ now contains:
> 
> SHA1 (netbsd-sparc64-GENERIC.MP.gz) = 0cd34756d2fce2f6deeaa4192a669b0619986d6c
> SHA1 (netbsd-sparc64-GENERIC.gz) = b80187131d38fdb1689443692ea8970dd87dfe1f
> 
> Get them while they're hot :-)
> 
> (They are straight from my lfs tree, which was synced with HEAD at
> around 2230Z on 20100206. I dunno offhand if sparc64 was expected to
> work then or not. Usual provisos about accepting kernels over the
> internet apply, although if my build machine is owned I'm in a lot of
> trouble...)

I tried out the MP kernel.  The system booted properly and mounted the lfs 
filesystems.  Then I fired up a -j4 kernel build which worked fine until 
the link phase where this happened:

 1 0   311448   4800 1663   3   3  147  207  271 22 84 1221 1978 419 25 38 
36
trap type 0x34: cpu 0, pc=14a5f40 npc=14a5f44 
pstate=0x99820006
kernel trap 34: mem address not aligned
Stopped in pid 9851.1 (sh) at   netbsd:uvm_pageactivate:lduh
[
%o0 + 0x54], %g1
db{0}> tr
data_access_fault(d269840, 30, 1009cdc, da5, da5, d266000) at 
netbsd:dat
a_access_fault+0xc4
?(40c14900, da5, 4, d269ce8, d266000, da5) at 0x10086c8
execve1(0, da5, d1b5468, 0, 0, e608c00) at netbsd:execve1+0x36c
syscall_plain(d269ed0, d269f58, 409416a0, 409416a4, , d269dc0) at 
netbsd
:syscall_plain+0x138
?(40c14b00, 40c149b0, 40c14a08, 40c16f56, 6e, 40c14b00) at 0x1008c28
db{0}> 
db{0}> show reg
tstate  0x9982000601
pc  0x14a5f40   uvm_pageactivate
npc 0x14a5f44   uvm_pageactivate+0x4
ipl 0
y   0
g0  0
g1  0x1
g2  0x1497d60   uao_get
g3  0xd810ce8
g4  0x169c510   aobj_pager
g5  0xca5
g6  0
g7  0
o0  0x1
o1  0
o2  0xd269660
o3  0xd269668
o4  0
o5  0x2
o6  0xd268c41
o7  0x149b7cc   uvm_fault_internal+0xbcc
l0  0xbd784
l1  0xfffcf828ff
l2  0xff00
l3  0
l4  0xe0018000ff
l5  0xff00
l6  0xc00
l7  0
i0  0x25674
i1  0
i2  0x4100
i3  0x2000
i4  0xf00
i5  0x18c18
i6  0xd26850100
i7  0x14da3cc00


I'm not sure I trust the stacktrace since the fault appears to have 
happened here:

db{0}> x/i 14a5f40
netbsd:uvm_pageactivate:lduh[%o0 + 0x54], %g1

Eduardo


Re: unhooking lfs from ufs

2010-02-08 Thread David Holland
On Tue, Feb 09, 2010 at 04:14:58AM +, Eduardo Horvath wrote:
 > I'm a bit low on disk space at the moment, but if you build me a generic 
 > sparc64 kernel I can test it.

http://www.eecs.harvard.edu/~dholland/tmp/netbsd/ now contains:

SHA1 (netbsd-sparc64-GENERIC.MP.gz) = 0cd34756d2fce2f6deeaa4192a669b0619986d6c
SHA1 (netbsd-sparc64-GENERIC.gz) = b80187131d38fdb1689443692ea8970dd87dfe1f

Get them while they're hot :-)

(They are straight from my lfs tree, which was synced with HEAD at
around 2230Z on 20100206. I dunno offhand if sparc64 was expected to
work then or not. Usual provisos about accepting kernels over the
internet apply, although if my build machine is owned I'm in a lot of
trouble...)

-- 
David A. Holland
dholl...@netbsd.org


Re: unhooking lfs from ufs

2010-02-08 Thread Eduardo Horvath
On Tue, 9 Feb 2010, David Holland wrote:

> On Tue, Feb 09, 2010 at 03:55:59AM +, Eduardo Horvath wrote:
>  > > What magic sauce do you have? Please share it - lfs has been broken
>  > > for everyone else.
>  > 
>  > Dunno.  Although I do disable fsck_lfs.  It usually causes more problems 
>  > than it solves.  It needs a complete overhaul.  It tries to act like 
>  > fsck_ffs instead of validating segment checksums and regenerating the 
>  > ifile.
> 
> ...except that it also apparently can't actually repair most problems
> it finds, which isn't very helpful of it.
> 
>  > A also haven't updated my sources for a while so I don't have to worry 
>  > about DEV_BSIZE breakage.
> 
> Do you feel up to testing my patch? It works for me for basic stuff,
> although I didn't try pounding on it because it wasn't expected to work.

I'm a bit low on disk space at the moment, but if you build me a generic 
sparc64 kernel I can test it.

Eduardo


Re: unhooking lfs from ufs

2010-02-08 Thread David Holland
On Tue, Feb 09, 2010 at 03:55:59AM +, Eduardo Horvath wrote:
 > > What magic sauce do you have? Please share it - lfs has been broken
 > > for everyone else.
 > 
 > Dunno.  Although I do disable fsck_lfs.  It usually causes more problems 
 > than it solves.  It needs a complete overhaul.  It tries to act like 
 > fsck_ffs instead of validating segment checksums and regenerating the 
 > ifile.

...except that it also apparently can't actually repair most problems
it finds, which isn't very helpful of it.

 > A also haven't updated my sources for a while so I don't have to worry 
 > about DEV_BSIZE breakage.

Do you feel up to testing my patch? It works for me for basic stuff,
although I didn't try pounding on it because it wasn't expected to work.

-- 
David A. Holland
dholl...@netbsd.org


Re: unhooking lfs from ufs

2010-02-08 Thread Eduardo Horvath
On Tue, 9 Feb 2010, David Holland wrote:

> On Mon, Feb 08, 2010 at 09:50:22PM +, Eduardo Horvath wrote:
>  > NetBSD nonplus 5.99.23 NetBSD 5.99.23 (GENERIC.MP) #0: Fri Dec 25 01:58:51 
>  > PST 2009  
>  > eh29...@nonplus:/export/home/work/src/sys/arch/sparc64/compile/GENERIC.MP 
>  > sparc64
> 
> What magic sauce do you have? Please share it - lfs has been broken
> for everyone else.

Dunno.  Although I do disable fsck_lfs.  It usually causes more problems 
than it solves.  It needs a complete overhaul.  It tries to act like 
fsck_ffs instead of validating segment checksums and regenerating the 
ifile.

A also haven't updated my sources for a while so I don't have to worry 
about DEV_BSIZE breakage.

Eduardo


Re: unhooking lfs from ufs

2010-02-08 Thread David Holland
On Mon, Feb 08, 2010 at 09:18:05PM +0100, Adam Hamsik wrote:
 > Make LFS not compilable is the way to make it disappear very soon.

Did you read what I wrote? :-)

 > I think that these changes should happen in separate branch and
 > should be committed back to main branch only when it will be at
 > least compilable again.

That's already been done, except with a local/private branch on my
end.

-- 
David A. Holland
dholl...@netbsd.org


Re: unhooking lfs from ufs

2010-02-08 Thread David Holland
On Mon, Feb 08, 2010 at 09:50:22PM +, Eduardo Horvath wrote:
 > NetBSD nonplus 5.99.23 NetBSD 5.99.23 (GENERIC.MP) #0: Fri Dec 25 01:58:51 
 > PST 2009  
 > eh29...@nonplus:/export/home/work/src/sys/arch/sparc64/compile/GENERIC.MP 
 > sparc64

What magic sauce do you have? Please share it - lfs has been broken
for everyone else.

-- 
David A. Holland
dholl...@netbsd.org


Re: unhooking lfs from ufs

2010-02-08 Thread Eduardo Horvath
On Mon, 8 Feb 2010, Adam Hamsik wrote:

> On Feb,Monday 8 2010, at 10:37 PM, Eduardo Horvath wrote:
> 
> > On Mon, 8 Feb 2010, Adam Hamsik wrote:
> > 
> >> On Feb,Monday 8 2010, at 9:33 PM, Eduardo Horvath wrote:
> >> 
> >>> On Mon, 8 Feb 2010, Adam Hamsik wrote:
> >>> 
>  Are you sure that you can really finish this ? Currently you are working 
>  on namei, ufs_lookup and many other issues. Make LFS not compilable is 
>  the way to make it disappear very soon. I think that these changes 
>  should happen in separate branch and should be committed back to main 
>  branch only when it will be at least compilable again. 
> >>> 
> >>> s/compilable/reasonably solid/
> >>> 
> >>> I'll be rather upset if my lfs partitions suddenly stop working.
> >> 
> >> LFS is not solid for a long time, now. 
> > 
> > The only problems I've been having with it resently are media errors.
> 
> What version of NetBSD are you using ? LFS is known to be broken after 
> vmlocking2 merge, because it heavily depended on kernel big lock model used 
> before.

NetBSD nonplus 5.99.23 NetBSD 5.99.23 (GENERIC.MP) #0: Fri Dec 25 01:58:51 
PST 2009  
eh29...@nonplus:/export/home/work/src/sys/arch/sparc64/compile/GENERIC.MP 
sparc64




Re: unhooking lfs from ufs

2010-02-08 Thread Adam Hamsik

On Feb,Monday 8 2010, at 10:37 PM, Eduardo Horvath wrote:

> On Mon, 8 Feb 2010, Adam Hamsik wrote:
> 
>> On Feb,Monday 8 2010, at 9:33 PM, Eduardo Horvath wrote:
>> 
>>> On Mon, 8 Feb 2010, Adam Hamsik wrote:
>>> 
 Are you sure that you can really finish this ? Currently you are working 
 on namei, ufs_lookup and many other issues. Make LFS not compilable is the 
 way to make it disappear very soon. I think that these changes should 
 happen in separate branch and should be committed back to main branch only 
 when it will be at least compilable again. 
>>> 
>>> s/compilable/reasonably solid/
>>> 
>>> I'll be rather upset if my lfs partitions suddenly stop working.
>> 
>> LFS is not solid for a long time, now. 
> 
> The only problems I've been having with it resently are media errors.

What version of NetBSD are you using ? LFS is known to be broken after 
vmlocking2 merge, because it heavily depended on kernel big lock model used 
before.

Regards

Adam.



Re: unhooking lfs from ufs

2010-02-08 Thread Eduardo Horvath
On Mon, 8 Feb 2010, Adam Hamsik wrote:

> On Feb,Monday 8 2010, at 9:33 PM, Eduardo Horvath wrote:
> 
> > On Mon, 8 Feb 2010, Adam Hamsik wrote:
> > 
> >> Are you sure that you can really finish this ? Currently you are working 
> >> on namei, ufs_lookup and many other issues. Make LFS not compilable is the 
> >> way to make it disappear very soon. I think that these changes should 
> >> happen in separate branch and should be committed back to main branch only 
> >> when it will be at least compilable again. 
> > 
> > s/compilable/reasonably solid/
> > 
> > I'll be rather upset if my lfs partitions suddenly stop working.
> 
> LFS is not solid for a long time, now. 

The only problems I've been having with it resently are media errors.

Eduardo


Re: unhooking lfs from ufs

2010-02-08 Thread Eduardo Horvath
On Mon, 8 Feb 2010, Thor Lancelot Simon wrote:

> On Mon, Feb 08, 2010 at 08:33:28PM +, Eduardo Horvath wrote:
> > 
> > I'll be rather upset if my lfs partitions suddenly stop working.
> 
> Personally, I'd be rather astonished if the ones I used to use
> suddenly _started_ working.  I have to say if LFS is working for you
> right now you are probably in a very small minority of those who have
> tried to use it since the vmlocking2 merge.
> 
> Are you perchance on a uniprocessor?

No.  Both CPUs are running.

Eduardo


Re: unhooking lfs from ufs

2010-02-08 Thread Adam Hamsik

On Feb,Monday 8 2010, at 9:33 PM, Eduardo Horvath wrote:

> On Mon, 8 Feb 2010, Adam Hamsik wrote:
> 
>> Are you sure that you can really finish this ? Currently you are working on 
>> namei, ufs_lookup and many other issues. Make LFS not compilable is the way 
>> to make it disappear very soon. I think that these changes should happen in 
>> separate branch and should be committed back to main branch only when it 
>> will be at least compilable again. 
> 
> s/compilable/reasonably solid/
> 
> I'll be rather upset if my lfs partitions suddenly stop working.

LFS is not solid for a long time, now. 

Regards

Adam.



Re: unhooking lfs from ufs

2010-02-08 Thread Thor Lancelot Simon
On Mon, Feb 08, 2010 at 08:33:28PM +, Eduardo Horvath wrote:
> 
> I'll be rather upset if my lfs partitions suddenly stop working.

Personally, I'd be rather astonished if the ones I used to use
suddenly _started_ working.  I have to say if LFS is working for you
right now you are probably in a very small minority of those who have
tried to use it since the vmlocking2 merge.

Are you perchance on a uniprocessor?

-- 
Thor Lancelot Simont...@rek.tjls.com
  "All of my opinions are consistent, but I cannot present them all
   at once."-Jean-Jacques Rousseau, On The Social Contract


Re: unhooking lfs from ufs

2010-02-08 Thread Eduardo Horvath
On Mon, 8 Feb 2010, Adam Hamsik wrote:

> Are you sure that you can really finish this ? Currently you are working on 
> namei, ufs_lookup and many other issues. Make LFS not compilable is the way 
> to make it disappear very soon. I think that these changes should happen in 
> separate branch and should be committed back to main branch only when it will 
> be at least compilable again. 

s/compilable/reasonably solid/

I'll be rather upset if my lfs partitions suddenly stop working.

Eduardo


Re: unhooking lfs from ufs

2010-02-08 Thread Adam Hamsik
Hi David,
> 
> 
> The copy involves 18 files from sys/ufs/ufs (out of 21; the ones
> excluded are quota.h and unsurprisingly ufs_wapbl.[ch]) which contain
> 9067 lines of code. That gives the following statistics:
> 
> 14988 size of lfs currently
>+ 9067 size of copypasted ufs
> 24055 size of resulting uncompilable lfs
>-  401 result of making it compilable
> 23654 size of new lfs
> 
> This is the size of the code in sys/ufs/lfs; the userlevel tools need
> patching but don't change size significantly.
> 
> My guess/estimate is that after several rounds of consolidation the
> total size will drop to around 18000-19000 lines. Maybe less, even,
> but I wouldn't count on that. I'll be keeping an eye on the total size
> going forward.
> 
> Anyway, I have done this much and it's ready to go. I will be
> committing it tonight, I think, unless there are sudden howls of
> protest.
> 
> The diff (from HEAD of a couple hours ago to the new compilable lfs)
> is posted here:
> 
>   http://www.eecs.harvard.edu/~dholland/netbsd/lfs-ufs-20100207.diff
> 
> I will probably commit the pasted-only uncompilable form first, and
> maybe some of the intermediate steps as well, for the historical
> record and to make future merges easier. This may make the tree
> temporarily unbuildable, but hopefully not for very long. 

Are you sure that you can really finish this ? Currently you are working on 
namei, ufs_lookup and many other issues. Make LFS not compilable is the way to 
make it disappear very soon. I think that these changes should happen in 
separate branch and should be committed back to main branch only when it will 
be at least compilable again. 


Regards

Adam.



Re: unhooking lfs from ufs

2010-02-08 Thread Johnny Billquist

Hmmm...

Eduardo Horvath wrote:

On Sun, 7 Feb 2010, David Holland wrote:


Anyhow, it seems to me that isolating it from changes to ffs is likely
to result in less breakage over time, not more. Can you expand on your
reasoning some?


The most significant parts that are shared are the directory ops and the 
read/write routeines.


The directory ops are essentially FFS code with an LFS wrapper around it.  
Right now any ffs bugfixes for those used directly by LFS.  

The read/write routines also share about 80% of the code.  I suppose there 
is a better case for separating these out if it makes code maintenance 
easier.


If we were to separate them out then every time someone fixes a problem 
with FFS, the same changes would be required to be made for LFS.  
Historically this has not happened.  When you look at UVM or UBC 
integration, there were long periods of time when LFS was unusable because 
that filesystem has been considered of secondary importance.  


So, the fact that the code was shared did indeed *not* make FFS fixes 
also flow over into LFS? (I assume that FFS was fixed for those things 
in short order.)

That would, in my eyes, mean that this argument isn't valid.

LFS has a rather small user base since it's historically been considered 
experimental and most machines can't boot from it.  Few people use it and 
fewer still work on it.


Indeed. I tried LFS a few times many years ago, but had some problems 
with it, along with the "experimenal" status of it, which made me stop 
trying.


I would love to hear someone allay my fears, but I think segregating the 
LFS code from the FFS code will accelerate the bitrot and the final result 
will be removal of the LFS code.


I can't alleviate your fears, however, it appears to me that the 
assumption that LFS benefits from sharing code with FFS is wrong, or 
atleast exaggerated. Looking at how the LFS code have had problems for 
extended periods, even with shared code with FFS suggests that LFS have 
not really gained much from that sharing.


Having the code split, and then possibly getting it leaner and cleaner, 
looks like a potential gain atleast.


Johnny


Re: unhooking lfs from ufs

2010-02-08 Thread Johnny Billquist
Perhaps not a very meaningful voice, but I think it makes sense to split 
them.


Johnny

David Holland wrote:

On Mon, Feb 08, 2010 at 11:03:44AM +0900, Masao Uebayashi wrote:
 > This thread?
 > 
 > 	http://mail-index.netbsd.org/tech-kern/2009/07/21/msg005526.html


That was later - that's about what happens to quotas afterwards. But I
can't find the original thread, which as I recall was moderately
lengthy, so maybe it wasn't on tech-kern?

Anyway, I want to hear the rest of what eeh@ has to say, which means
I'm not going ahead tonight, which means nothing will be happening for
a few days at least anyhow. So if anyone wants to object or hash
through this in detail now, go right ahead.



Re: unhooking lfs from ufs

2010-02-08 Thread Greywolf
On Sun, Feb 7, 2010 at 9:56 PM, David Holland  wrote:
> On Mon, Feb 08, 2010 at 03:37:37AM +, Eduardo Horvath wrote:
>  > I would love to hear someone allay my fears, but I think segregating the
>  > LFS code from the FFS code will accelerate the bitrot and the final result
>  > will be removal of the LFS code.
>
> Well. lfs is currently pretty broken and without a substantial
> rototill it's going to stay broken. And if it stays broken, it's
> certainly going to end up removed. Certain people have already been
> agitating to remove it.
>
> It seems to me that unhooking lfs from ufs is the necessary first step
> towards any substantial overhaul of lfs. This is why I'm proposing it.
>
> There are some people who would like this unhook done so that lfs
> stops complicating ufs; they will doubtless be happy to ignore lfs
> afterwards, but my goal is to make it work.

if lfs can be made to work comparable to or better than anything we
currently have, this will be a win.  my perspective tells me that
lfs is more or less obsolete at this point and ripping it out and
letting it die is its ultimate destiny.


-- 
--*greywolf;
/* relayer @t gmail d0t com */
/* ^ spam decoy ^ */


Re: unhooking lfs from ufs

2010-02-07 Thread Iain Hibbert
On Mon, 8 Feb 2010, David Holland wrote:

> On Mon, Feb 08, 2010 at 11:03:44AM +0900, Masao Uebayashi wrote:
>  > This thread?
>  >
>  >http://mail-index.netbsd.org/tech-kern/2009/07/21/msg005526.html
>
> That was later - that's about what happens to quotas afterwards. But I
> can't find the original thread, which as I recall was moderately
> lengthy, so maybe it wasn't on tech-kern?

I think this, crossposted to nebsd-users and current-users

   http://archive.netbsd.se/?ml=netbsd-users&a=2009-04&t=10383631

iain




Re: unhooking lfs from ufs

2010-02-07 Thread David Holland
On Sun, Feb 07, 2010 at 11:07:55AM +, Mindaugas Rasiukevicius wrote:
 > > > How would this affect UFS side?  For example, any potential code
 > > > reduction and/or simplification?
 > > 
 > > Yes. ufs_readwrite.c will become much less gross, for example. There
 > > used to be assorted LFS-only code in the ufs sources; ad@ removed the
 > > ifdefs some time ago but they could be resurrected and then used to
 > > purge the relevant code. I don't know how much code that is.
 > > 
 > > As for deeper simplifications, I don't know without digging around a
 > > lot more than I have (particularly in the ext2fs code), but there
 > > should be some.
 > 
 > Good, I think it would be great to look into this.

The ifdefs in question aren't, as it turns out, large, just a matter
of being able to remove one ufsop that only lfs cares about.

However, as I recall ad had a whole list of reasons he wanted to
remove lfs, most of which pertained to its effect on ufs. I don't have
this list; maybe he can be persuaded to share it.

-- 
David A. Holland
dholl...@netbsd.org


Re: unhooking lfs from ufs

2010-02-07 Thread David Holland
On Mon, Feb 08, 2010 at 03:37:37AM +, Eduardo Horvath wrote:
 > > Anyhow, it seems to me that isolating it from changes to ffs is likely
 > > to result in less breakage over time, not more. Can you expand on your
 > > reasoning some?
 > 
 > The most significant parts that are shared are the directory ops and the 
 > read/write routeines.
 > 
 > The directory ops are essentially FFS code with an LFS wrapper around it.  
 > Right now any ffs bugfixes for those used directly by LFS.

However, several of those wrappers are truly vile, some of them AFAICT
compromise LFS's transactional update model, and they all require
stretching the LFS code to fit a paradigm that it isn't suited to.
Plus some of that code suffers pretty badly from having been written
in 1983.

There aren't that many changes to the ufs code anyway. Those changes
that do appear are easy to bring across to portions of the copied ufs
code that haven't been altered much; once significant portions have
been, fixes for those portions are much more likely to be flowing the
other way. (Why? Because aggressive rototilling has this way of
exposing bugs.)

 > If we were to separate them out then every time someone fixes a problem 
 > with FFS, the same changes would be required to be made for LFS.  

I've been doing this in my lfs tree since July (when I prepared most of
the changes) and it's taken me negligible amounts of time.

 > Historically this has not happened.  When you look at UVM or UBC 
 > integration, there were long periods of time when LFS was unusable because 
 > that filesystem has been considered of secondary importance.  

Right, and we're in one of those periods, and the goal is to get out
of it and stay out of it...

 > I would love to hear someone allay my fears, but I think segregating the 
 > LFS code from the FFS code will accelerate the bitrot and the final result 
 > will be removal of the LFS code.

Well. lfs is currently pretty broken and without a substantial
rototill it's going to stay broken. And if it stays broken, it's
certainly going to end up removed. Certain people have already been
agitating to remove it.

It seems to me that unhooking lfs from ufs is the necessary first step
towards any substantial overhaul of lfs. This is why I'm proposing it.

There are some people who would like this unhook done so that lfs
stops complicating ufs; they will doubtless be happy to ignore lfs
afterwards, but my goal is to make it work.

-- 
David A. Holland
dholl...@netbsd.org


Re: unhooking lfs from ufs

2010-02-07 Thread Eduardo Horvath
On Sun, 7 Feb 2010, David Holland wrote:

> Anyhow, it seems to me that isolating it from changes to ffs is likely
> to result in less breakage over time, not more. Can you expand on your
> reasoning some?

The most significant parts that are shared are the directory ops and the 
read/write routeines.

The directory ops are essentially FFS code with an LFS wrapper around it.  
Right now any ffs bugfixes for those used directly by LFS.  

The read/write routines also share about 80% of the code.  I suppose there 
is a better case for separating these out if it makes code maintenance 
easier.

If we were to separate them out then every time someone fixes a problem 
with FFS, the same changes would be required to be made for LFS.  
Historically this has not happened.  When you look at UVM or UBC 
integration, there were long periods of time when LFS was unusable because 
that filesystem has been considered of secondary importance.  

LFS has a rather small user base since it's historically been considered 
experimental and most machines can't boot from it.  Few people use it and 
fewer still work on it.

I would love to hear someone allay my fears, but I think segregating the 
LFS code from the FFS code will accelerate the bitrot and the final result 
will be removal of the LFS code.

Eduardo


Re: unhooking lfs from ufs

2010-02-07 Thread David Holland
On Mon, Feb 08, 2010 at 11:03:44AM +0900, Masao Uebayashi wrote:
 > This thread?
 > 
 >  http://mail-index.netbsd.org/tech-kern/2009/07/21/msg005526.html

That was later - that's about what happens to quotas afterwards. But I
can't find the original thread, which as I recall was moderately
lengthy, so maybe it wasn't on tech-kern?

Anyway, I want to hear the rest of what eeh@ has to say, which means
I'm not going ahead tonight, which means nothing will be happening for
a few days at least anyhow. So if anyone wants to object or hash
through this in detail now, go right ahead.

-- 
David A. Holland
dholl...@netbsd.org


Re: unhooking lfs from ufs

2010-02-07 Thread Masao Uebayashi
This thread?

http://mail-index.netbsd.org/tech-kern/2009/07/21/msg005526.html


Re: unhooking lfs from ufs

2010-02-07 Thread David Holland
On Sun, Feb 07, 2010 at 06:35:12PM +, Eduardo Horvath wrote:
 > Since the original disussion I looked into this a bit and I'm not 
 > convinced it's a good idea.  I'm afraid if we unhook lfs from ffs it will 
 > suffer from extreme bitrot.  Even with the current situation it was 
 > completely nonfunctional four months ago.

Yes, the idea here is to get it fixed. Currently, it's broken largely
because it's a mess. Sharing ufs is a substantial part of why it's a
mess.

Anyhow, it seems to me that isolating it from changes to ffs is likely
to result in less breakage over time, not more. Can you expand on your
reasoning some?

-- 
David A. Holland
dholl...@netbsd.org


Re: unhooking lfs from ufs

2010-02-07 Thread Eduardo Horvath

Since the original disussion I looked into this a bit and I'm not 
convinced it's a good idea.  I'm afraid if we unhook lfs from ffs it will 
suffer from extreme bitrot.  Even with the current situation it was 
completely nonfunctional four months ago.

Eduardo


Re: unhooking lfs from ufs

2010-02-07 Thread David Holland
On Sun, Feb 07, 2010 at 11:07:55AM +, Mindaugas Rasiukevicius wrote:
 > > It was discussed months ago. This is a reminder/heads-up.
 > 
 > Where?  This mailing list is a right place where such discussions (and
 > decisions) should happen.

Right here...

-- 
David A. Holland
dholl...@netbsd.org


Re: unhooking lfs from ufs

2010-02-07 Thread Mindaugas Rasiukevicius
David Holland  wrote:
>  > How would this affect UFS side?  For example, any potential code
>  > reduction and/or simplification?
> 
> Yes. ufs_readwrite.c will become much less gross, for example. There
> used to be assorted LFS-only code in the ufs sources; ad@ removed the
> ifdefs some time ago but they could be resurrected and then used to
> purge the relevant code. I don't know how much code that is.
> 
> As for deeper simplifications, I don't know without digging around a
> lot more than I have (particularly in the ext2fs code), but there
> should be some.

Good, I think it would be great to look into this.

>  > This involves significant changes, therefore enough time should be left
>  > for mailing list readers (~1 week at least, before committing anything).
> 
> It was discussed months ago. This is a reminder/heads-up.
> 

Where?  This mailing list is a right place where such discussions (and
decisions) should happen.

-- 
Mindaugas


Re: unhooking lfs from ufs

2010-02-07 Thread David Holland
On Sun, Feb 07, 2010 at 10:10:31AM +, Mindaugas Rasiukevicius wrote:
 > > The copy involves 18 files from sys/ufs/ufs (out of 21; the ones
 > > excluded are quota.h and unsurprisingly ufs_wapbl.[ch]) which contain
 > > 9067 lines of code. That gives the following statistics:
 > > 
 > >  14988 size of lfs currently
 > > + 9067 size of copypasted ufs
 > >  24055 size of resulting uncompilable lfs
 > > -  401 result of making it compilable
 > >  23654 size of new lfs
 > 
 > How would this affect UFS side?  For example, any potential code reduction
 > and/or simplification?

Yes. ufs_readwrite.c will become much less gross, for example. There
used to be assorted LFS-only code in the ufs sources; ad@ removed the
ifdefs some time ago but they could be resurrected and then used to
purge the relevant code. I don't know how much code that is.

As for deeper simplifications, I don't know without digging around a
lot more than I have (particularly in the ext2fs code), but there
should be some.

 > > Anyway, I have done this much and it's ready to go. I will be
 > > committing it tonight, I think, unless there are sudden howls of
 > > protest.
 > 
 > This involves significant changes, therefore enough time should be left
 > for mailing list readers (~1 week at least, before committing anything).

It was discussed months ago. This is a reminder/heads-up.

-- 
David A. Holland
dholl...@netbsd.org


Re: unhooking lfs from ufs

2010-02-07 Thread Mindaugas Rasiukevicius
David Holland  wrote:
> The copy involves 18 files from sys/ufs/ufs (out of 21; the ones
> excluded are quota.h and unsurprisingly ufs_wapbl.[ch]) which contain
> 9067 lines of code. That gives the following statistics:
> 
>  14988size of lfs currently
> + 9067size of copypasted ufs
>  24055size of resulting uncompilable lfs
> -  401result of making it compilable
>  23654size of new lfs

How would this affect UFS side?  For example, any potential code reduction
and/or simplification?

> Anyway, I have done this much and it's ready to go. I will be
> committing it tonight, I think, unless there are sudden howls of
> protest.

This involves significant changes, therefore enough time should be left
for mailing list readers (~1 week at least, before committing anything).

-- 
Mindaugas