[SLUG] Ubuntu, 2.6.15 kernel, RAID and SATA and IDE

[Luke Kendall]

This relates to my attempt to move from my now too-long-in-the-tooth RH
7.2 system to something new.  I picked Ubuntu and installed it without
much trouble, highly impressed by how well it auto-detected and
configured the network, sound, and graphics (to a point).

It's basically just to sketch out my failure to build a kernel to meet
my needs, in case it's useful to others doing Google searches, so they
feel they're not alone.

What I want to do is something that seems should be the norm, not
something unusual: I want RAID mirroring.  I have both an IDE drive and
a SATA drive. (The PC bumph said it was a SATA drive system, so I bought
an extra SATA drive.  What it didn't say was that it was shipped with an
IDE drive and was actually SATA-supporting.)

I plug in the SATA drive and manage to partition it the same way I did
the IDE drive (note: until you partition the drive *and reboot* the
entries for the /dev/sda etc. won't appear so you can't do mkfs of any
sort).  After making the filesystems, preparatory to doing the
mirroring, I discover the Ubuntu kernel image I'm using (2.6.15.26),
doesn't appear to have md support.

So I need to build a new kernel because I'll need RAID compiled into the
kernel along with ext2 and ext3 and IDE drivers and SATA drivers.

First thing to do is to learn is how to build a 2.6 series kernel.

I discover that to avoid the usual nuisance of nvidia's proprietary
graphics driver, there are Ubuntu/Debian packages for automatically
performing this for you, if you install the nvidia-kernerl-common package,
or something like that.

Oh, and the Debian way is not to build the kernel directly, but to do
the make config (gconfig or xconfig) parts and then run make-kpkg and
then dpkg -i the kernel.deb image and everything should just work.

So I do that and following the tips on Troubleshooters that told me all this,
also did as it said and ran the mkinitrd command to make an initrd image too.

Reboot -> panic due to no devfs.

Much searching leads to the discovery that mkinitrd and initrd are deprecated,
and I should use mkinitramfs and initramfs.  (Same boot loader syntax, though,
namely initrd).

So I do that and reboot and it looks okay up to the "Okay, booting" message
and then the screen goes blank.  That's a first.  I wait a bit, then reset
and decide to try again.  This time, I notice that the HDD light is flashing,
so it hasn't halted, and I decide to wait even longer.  Eventually the
"failed to start X" garbage-laden screen appears, and I let it show me
the log files (garbage-decorated on all borders), and then try to see why.
"FATAL error: could not load the NVIDIA kernel module."

The nvidia-glx module is installed, and can't be removed due to dependencies.
Nor can it be refreshed, that I can see (I'm still a novice at Debian
and dpkg).  Nor can it be found if I try to modprobe it.  So much for
the Debian way of including the nvidia kernel modules automatically.  I
download the manual thing from the nvidia page and "sh" that to build
the kernel modules, and that seems to sort things out.

Then I discover that there's no SATA drive visible.  No /dev/sd*

So I go hunting around and dmesg has a whole bunch of errors about sata_via:
unsatisfied external reference. It seems most of the SATA functions are
missing.  So back to make gconfig and eventually find the SATA modules
buried inside SCSI if you turn on some other SCSI option that allows you
to unfold the category they're buried inside, and I turn off all the
drivers for all the other chipsets *not* in my PC (my best guess is that I
have a VIA chipset SATA controller), because a google search has shown
a post where my error messages turning up seems to be linked to having these
other controllers compiled in as modules instead of turned off.  Doesn't
make a lot of sense to me, but I give in and build in the "via" choice into
the kernel and turn off all the others and rebuild.

This time it fails because of multiple definitions of some symbol like
"do_get_something_GFM" in ide and sata, so I change one to have an x
suffix and rebuild.  The posters suffering this blame an untested patch
by SuSE 10.1, but like I say, I'm using Ubuntu (6.06).

Now it gets further, but fails because acpi_in_suspend is undefined.
This is defined in drivers/something/acpi/sleep/main.c.  From looking at
the Makefile there I see that CONFIG_ACPI_SLEEP{_FS} aren't defined,
and in fact I can't find a way to define them, so I manually edit the
..config file and add them.  That fixes that problem, but now four other
symbols are undefined.

I make clean and try again.  No good.  acpi_in_suspend is undefined.

So at this point, I've given up, severely depressed.


All this is the same problem that's been there forever in the
Linux kernel config system - one module depending on other
modules, but no checking of those dependencies or even any clue
that the dependencies even exist, in the make config stage -
it's all spaghetti-ed through the source code.  The

Re: [SLUG] Ubuntu, 2.6.15 kernel, RAID and SATA and IDE

[Jeff Waugh]

> What I want to do is something that seems should be the norm, not
> something unusual: I want RAID mirroring.

Easy to do straight off, from the (text mode, or "alternate") installer.

> After making the filesystems, preparatory to doing the mirroring, I
> discover the Ubuntu kernel image I'm using (2.6.15.26), doesn't appear to
> have md support.
> 
> So I need to build a new kernel because I'll need RAID compiled into the
> kernel along with ext2 and ext3 and IDE drivers and SATA drivers.

Whoooaaa, that's not the case. Unmodified vendor kernel on my 6.06 LTS box
at home:

  $ find /lib/modules/2.6.15-26-686/ | grep raid1.ko
  /lib/modules/2.6.15-26-686/kernel/drivers/md/raid1.ko

What you need to do is make sure that the raid and filesystem modules you
need are built into the *initramfs*. The kernel definitely has support for
all of them, and you are *way* better off using the vendor kernel (as
you will get security updates and feel fuzzy inside).

Rebuild your initramfs like so (as root or using sudo):

  mkinitramfs -o /boot/initrd.img-2.6.15-26-686 /lib/modules/2.6.15-26-686

You will almost certainly get all of the modules you need for raid. If not,
or if the drivers for your storage chipsets aren't correctly included, you
can add module names to /etc/mkinitramfs/modules and rebuild it. I have the
following drivers listed in mine, because for some reason (probably a few
releases ago) they weren't automagically included:

  sata_sil
  sd_mod

> Oh, and I *still* haven't found where dpkg is storing the out-of-date info
> that the root file system is hda6.  Every time it alters menu.lst I have
> to manually revert to the backup and add the new entries by hand.
> Installing grub to hda7 hasn't affected that.

Look for all the other references to hda6 in the menu.list file, especially
the one that probably looks like this (yes, it's a comment):

# kopt=root=/dev/hda6 ro

:-)

- Jeff

-- 
linux.conf.au 2007: Sydney, Australia   http://lca2007.linux.org.au/
 
   No clue is good clue.
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Ubuntu, 2.6.15 kernel, RAID and SATA and IDE

[lukekendall]

On 30 Jul, Jeff Waugh wrote:
>  
>  
> > What I want to do is something that seems should be the norm, not
> > something unusual: I want RAID mirroring.
>  
>  Easy to do straight off, from the (text mode, or "alternate") installer.

I can't handle the thousand kernel module config questions of the text
mode installer. :-)

> > After making the filesystems, preparatory to doing the mirroring, I
> > discover the Ubuntu kernel image I'm using (2.6.15.26), doesn't appear to
> > have md support.
> > 
> > So I need to build a new kernel because I'll need RAID compiled into the
> > kernel along with ext2 and ext3 and IDE drivers and SATA drivers.
>  
>  Whoooaaa, that's not the case. Unmodified vendor kernel on my 6.06 LTS box
>  at home:
>  
>$ find /lib/modules/2.6.15-26-686/ | grep raid1.ko
>/lib/modules/2.6.15-26-686/kernel/drivers/md/raid1.ko

Ah!

>  What you need to do is make sure that the raid and filesystem modules you
>  need are built into the *initramfs*. The kernel definitely has support for
>  all of them, and you are *way* better off using the vendor kernel (as
>  you will get security updates and feel fuzzy inside).

I'll agree with that!

>  Rebuild your initramfs like so (as root or using sudo):
>  
>mkinitramfs -o /boot/initrd.img-2.6.15-26-686 /lib/modules/2.6.15-26-686

How will it know to include the ide and sata drivers and raid stuff?
I'll try it, though it helps if I understood roughly *how* it figures
out what you need.

>  You will almost certainly get all of the modules you need for raid. If not,
>  or if the drivers for your storage chipsets aren't correctly included, you
>  can add module names to /etc/mkinitramfs/modules and rebuild it. I have the
>  following drivers listed in mine, because for some reason (probably a few
>  releases ago) they weren't automagically included:
>  
>sata_sil
>sd_mod

Sounds good - how do I determine the right module names, though?  (I've
never understood that.)

> > Oh, and I *still* haven't found where dpkg is storing the out-of-date info
> > that the root file system is hda6.  Every time it alters menu.lst I have
> > to manually revert to the backup and add the new entries by hand.
> > Installing grub to hda7 hasn't affected that.
>  
>  Look for all the other references to hda6 in the menu.list file, especially
>  the one that probably looks like this (yes, it's a comment):
>  
>  # kopt=root=/dev/hda6 ro
>  
>  :-)

Ah!  In a *comment*.  How evil! :-)

Many thanks, Jeff, that should get me on the right track.
 
luke

-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

[SLUG] OS or Linux equivalent of windows media streaming

[Ben Donohue]

Hi Slugs,

I'm after a Linux equivalent of windows media streaming or windows media 
services.

Anyone have some ideas, success stories, whatever.
I'm looking around and Slug input would be appreciated.
Ben

--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] OS or Linux equivalent of windows media streaming

[Jeff Waugh]

> I'm after a Linux equivalent of windows media streaming or windows media
> services.  Anyone have some ideas, success stories, whatever.  I'm looking
> around and Slug input would be appreciated.

www.flumotion.net -> Free Software, plus you can license beefier tools and
proprietary codecs from Fluendo if you need them.

- Jeff

-- 
linux.conf.au 2007: Sydney, Australia   http://lca2007.linux.org.au/
 
"I've been thinking: I get way too many pieces of e-mail, about 60 a
 day." - Microserfs
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] OS or Linux equivalent of windows media streaming

[Jan Schmidt]

On Sun, 2006-07-30 at 20:35 +1000, Ben Donohue wrote:
> Hi Slugs,
> 
> I'm after a Linux equivalent of windows media streaming or windows media 
> services.

Even if I weren't a Fluendo employee, I'd point you to
http://www.flumotion.com 

The Free (GPL) Flumotion server is focussed on Ogg/Theora/Vorbis
streaming, but the commercial server does WMV/WMA. The Streaming
Platform part provides hosting and bandwidth on top.

Cheers,
Jan
-- 
Jan Schmidt <[EMAIL PROTECTED]>

-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Ubuntu, 2.6.15 kernel, RAID and SATA and IDE

[Jeff Waugh]

> > > What I want to do is something that seems should be the norm, not
> > > something unusual: I want RAID mirroring.
> >  
> >  Easy to do straight off, from the (text mode, or "alternate")
> >  installer.
> 
> I can't handle the thousand kernel module config questions of the text
> mode installer. :-)

The Ubuntu text mode installer (found on the alternate CD) doesn't have any
kernel module config questions. It's fast and simple, as it has been since
the very first release.

> How will it know to include the ide and sata drivers and raid stuff?  I'll
> try it, though it helps if I understood roughly *how* it figures out what
> you need.

mkinitramfs is a shell script, so it's pretty easy to figure out what it's
doing. For module stuff, look in: /usr/share/initramfs-tools/hook-functions

> >sata_sil
> >sd_mod
> 
> Sounds good - how do I determine the right module names, though?  (I've
> never understood that.)

You really have to learn which drivers are appropriate for your hardware, in
this kind of situation. Look at which drivers are loaded using lsmod(1) and
see whereabouts they live under /lib/modules/ for a pretty
good idea of what kind of hardware it's for.

- Jeff

-- 
linux.conf.au 2007: Sydney, Australia   http://lca2007.linux.org.au/
 
"It's the most fun I've had without the use of a water-based
   lubricant." - Stephen Fry on directing his first film
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] OS or Linux equivalent of windows media streaming

[Morgan Storey]

Another option http://www.videolan.org/streaming/ VLC is pretty good, 
pretty low on resources, does a whole heap of codecs.


Ben Donohue wrote:

Hi Slugs,

I'm after a Linux equivalent of windows media streaming or windows 
media services.

Anyone have some ideas, success stories, whatever.
I'm looking around and Slug input would be appreciated.
Ben




--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

[SLUG] Adding RAID to Linux *after* installation?

[lukekendall]

Thanks to Jeff Waugh's advice, I now seem to have all the pieces in
place to create a RAID array (using mdadm).  (Thanks very much, Jeff!)

Except that when I try:

  mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/hda7 /dev/sda7

it refuses, as /dev/hda7 ("/") is mounted.

I read the man page with care, and have googled for info and searched
the Ubuntu forums, to no avail.  It looks like everyone expects you to
own all the hardware and setup the raid during installation - not add
the extra drive and add the raid afterwards.

I couldn't even find a howto for setting up a software raid (well, not
one that talked about mdadm - only old ones talking about the older
raidtools).

Can I do what I'm trying here (having got all the SATA etc. modules
loaded and running)?  Or do I have to boot from CD and unmount
everything before I create the array?

Also, how will it know to mirror the carefully setup hda7 to the empty
filesystem on sda7, and not the other way around, wiping out my
installation and all my work?

I'm toying with the idea of dd if=/dev/hda7 of=/dev/sda7 (given that
I've made all the partitions to be raid-ed the exact same number of
cylinders).

I'm coming to the conclusion that I'm doing something that's quite
unusual, and that it's going to be messy - that I have to do things
like change the partition type to "Linux raid autodetect", fake up the
mdam.conf entries, then tell it that the empty mirror was bad but can
now be added back to the raid array and re-mirrored ...  ugh.

Obviously I should try this on the /home partition, not slash, until I
get the steps worked out.  At least that way if I lose any data, it'll
be a tiny amount of junk instead of 10GB of carefully setup Linux
system!

luke

-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Adding RAID to Linux *after* installation?

[Jeff Waugh]

>   mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/hda7 /dev/sda7
> 
> it refuses, as /dev/hda7 ("/") is mounted.

Okay, so do this instead:

  mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda7 missing

That will create a 2 device array with one missing device. You need to bring
up the array (watch /proc/mdstat to see its status), mount it, then copy
over everything from / (using cp -a or similar). Now set up everything in
the copied partition to boot from /dev/md0 instead of /dev/hda7, and then
reboot. Boot from /dev/hda0 by using the grub menus (if you've made sure
they're changed), or edit the boot command line to do it manually. You
should find yourself running a system with / on /dev/md0. At this point, you
can use mdadm to add /dev/sda7 to the array - after checking everything is
okay, of course! :-)

This is not a wildly uncommon thing to do, but it's so easy to install with
RAID now that I think most people just do that.

> Obviously I should try this on the /home partition, not slash, until I get
> the steps worked out.  At least that way if I lose any data, it'll be a
> tiny amount of junk instead of 10GB of carefully setup Linux system!

That's a good idea. :-) Running an array reliably on / or /boot requires
some extra work anyway.

- Jeff

-- 
linux.conf.au 2007: Sydney, Australia   http://lca2007.linux.org.au/
 
   I don't know whose brain child it was, but it was quite an ugly child.
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

[SLUG] www question

[jam]

G'day

my customer has said:

---
When you have a minute can you please configure our apache server error
pages to not list the webserver build and operating system as it is a
security risk.

For example if I go to www.edc.com.au/fred I get the following information

Apache/2.0.53 (Linux/SUSE)
---
I can conceive if being a slight risk, in that 'don't bother with all the 
winders files.
Am I naive, is there a risk letting the world know WHAT os and web server you 
run?

Thanks
James
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] www question

[Michael Fox]

On 31/07/2006, at 2:34 PM, [EMAIL PROTECTED] wrote:


G'day

my customer has said:

-- 
-
When you have a minute can you please configure our apache server  
error

pages to not list the webserver build and operating system as it is a
security risk.

For example if I go to www.edc.com.au/fred I get the following  
information


Apache/2.0.53 (Linux/SUSE)
-- 
-
I can conceive if being a slight risk, in that 'don't bother with  
all the

winders files.
Am I naive, is there a risk letting the world know WHAT os and web  
server you

run?



I've noticed in recent months that certain security audit tools will  
list this as a security risk, and as such customers are following the  
recommendations from audits. So they are asking to have this stuff  
disabled/removed from view.


I guess its not a bad idea to remove it, and at the end of the day  
gives anyone looking less information about the system to work with.  
How much a risk it is, thats anyones guess. But like I said, its one  
less bit of information someone looking at the system remotely has to  
work with.

--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] www question

[Robert Collins]

On Mon, 2006-07-31 at 12:34 +0800, [EMAIL PROTECTED] wrote:
> G'day
> 
> my customer has said:
> 
> ---
> When you have a minute can you please configure our apache server error
> pages to not list the webserver build and operating system as it is a
> security risk.
> 
> For example if I go to www.edc.com.au/fred I get the following information
> 
> Apache/2.0.53 (Linux/SUSE)
> ---
> I can conceive if being a slight risk, in that 'don't bother with all the 
> winders files.
> Am I naive, is there a risk letting the world know WHAT os and web server you 
> run?

No substantial risk IMO. The rationale given is that if the os and
server are visible, crackers can use that to attack you more precisely.
The reality though is that most scripts around these days just try all
the permutations anyway - because the advertised string has every chance
of being false anyway. So not advertising the configuration does not
prevent attacks that will succeed.

Cheers,
Rob
-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] www question

[Martin Barry]

$quoted_author = "[EMAIL PROTECTED]" ;
> 
> my customer has said:
> 
> ---
> When you have a minute can you please configure our apache server error
> pages to not list the webserver build and operating system as it is a
> security risk.
> 
> For example if I go to www.edc.com.au/fred I get the following information
> 
> Apache/2.0.53 (Linux/SUSE)
> ---
> I can conceive if being a slight risk, in that 'don't bother with all the 
> winders files.
> Am I naive, is there a risk letting the world know WHAT os and web server you 
> run?

you probably need to remove it in quite a few places other than the 404
page.

http://toolbar.netcraft.com/site_report?url=http://www.edc.com.au

cheers
marty

-- 
And true love waits
In haunted attics
And true love wins
On lollipops and crisps

"True Love Waits" - Radiohead
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] www question

[Jeff Waugh]

> Am I naive, is there a risk letting the world know WHAT os and web server
> you run?

Sure. It can be mild or very serious, depending on whether the product and
version indicate the existence of particular exploits. For instance, if your
webserver published that it was running IRIX, we could have been having fun
with your web pages already. By publishing which SuSE or Apache versions
you're running, you're giving a potential cracker all the information they
need to find applicable exploits.

It's one of those "how much do you care?" things.

- Jeff

-- 
linux.conf.au 2007: Sydney, Australia   http://lca2007.linux.org.au/
 
I wanted to be Superman, but all I got were these special powers of
 self-deprecation.
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] www question

[Dean Hamstead]

the absence of a number may encourage the person attacking
to be more general in their approach. which increase server load
etc. so if there is a version number, they may not bother.

but as has been pointed out, the reliability of the dist
info is questionable as it can be faked. also, most attacked
(on small sites) are usually just scripts which will unintelligently
try everything and move on.

on the *DOWN SIDE* of removing versions and dists, it may make
assertaining the version harder for the administrator. which
in turn may lul the client into a false sense of security
in that with or without the version number present, the software
will still be vulnerable to the same attacks. if you do do an
audit, having the version numbers may (most likely will)
uncover much larger (and indeed, serious) vulernabilities in
dated software.



Dean

On Mon, July 31, 2006 2:41 pm, Michael Fox wrote:
> On 31/07/2006, at 2:34 PM, [EMAIL PROTECTED] wrote:
>
>> G'day
>>
>> my customer has said:
>>
>> --
>> -
>> When you have a minute can you please configure our apache server
>> error
>> pages to not list the webserver build and operating system as it is a
>> security risk.
>>
>> For example if I go to www.edc.com.au/fred I get the following
>> information
>>
>> Apache/2.0.53 (Linux/SUSE)
>> --
>> -
>> I can conceive if being a slight risk, in that 'don't bother with
>> all the
>> winders files.
>> Am I naive, is there a risk letting the world know WHAT os and web
>> server you
>> run?
>
>
> I've noticed in recent months that certain security audit tools will
> list this as a security risk, and as such customers are following the
> recommendations from audits. So they are asking to have this stuff
> disabled/removed from view.
>
> I guess its not a bad idea to remove it, and at the end of the day
> gives anyone looking less information about the system to work with.
> How much a risk it is, thats anyones guess. But like I said, its one
> less bit of information someone looking at the system remotely has to
> work with.
> --
> SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
> Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
>


-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html