Re: mirroring debian-security, sharing package pool with stable/testing/sid pool
On 04/11/2014 02:56 AM, Zenaan Harkness wrote: 8< snip Thus my question: Is it possible to use debmirror, to mirror stable,testing,sid as well as debian-security for stable and testing, and have them all share the same package pool? I can't see it, and I'm hoping I am simply not understanding something. If I would have to go to a more advanced setup with some other debian tool, I am willing, but that would take some time (namely finding the time, so would not be immediately). I do this using a rasberry pi (or laptop) running approx, I keep it up-to-date & cart it round to the various sites I admin, /etc/approx/approx.conf on the Pi points at local mirrors, mine looks like debian http://ftp.uk.debian.org/debian #debian http://ftp.debian.org/debian #debian-updates http://ftp.us.debian.org/debian debian-updates http://ftp.uk.debian.org/debian #debian-updates http://ftp.de.debian.org/debian debsec http://security.debian.org deb-multimedia http://www.deb-multimedia.org #deb-multimedia http://debian-mirrors.sdinet.de/debian-multimedia debian-backports http://backports.debian.org/debian-backports #debian-backports http://www.backports.org/backports.org raspberrypi http://archive.raspberrypi.org/debian #raspbian http://192.168.24.54:/raspbian raspbian http://mirrordirector.raspbian.org/raspbian #google google http://dl.google.com/linux/deb google-chrome http://dl.google.com/linux/chrome/deb #unvanquished unvanquished http://debs.unvanquished.net #Ubuntu ubuntu http://gb.archive.ubuntu.com/ubuntu ubuntu-security http://security.ubuntu.com/ubuntu ubuntu-extras http://extras.ubuntu.com/ubuntu ubuntu-partner http://archive.canonical.com/ubuntu I have a couple of alternate sources in each section but you can only use one at a time. All I do is hook the Pi up to the network of the to-be-updated machines and then apt-get update then dist-upgrade as usual, the /etc/apt/sources.list looks a bit like deb http://apt.proxy.here:/debian/ jessie main contrib non-free #deb http://ftp.uwa.edu.au/debian/ jessie main contrib non-free #deb http://mirror.nus.edu.sg/Debian/ jessie main contrib non-free deb http://apt.proxy.here:/debian/ jessie-updates main contrib non-free #deb http://ftp.debian.org/debian/ jessie-updates main contrib non-free #deb http://apt.proxy.here:/debian/ jessie-backports main contrib non-free deb http://apt.proxy.here:/debsec/ jessie/updates main contrib non-free #deb http://security.debian.org/debian-security/ jessie/updates main contrib non-free deb http://apt.proxy.here:/deb-multimedia/ jessie main non-free #deb http://apt.proxy.here:/jessie-backports/ jessie main non-free #deb http://apt.proxy.here:/debian-volatile/ jessie main non-free (I have some legacy alt sources for when I was travelling) and each computer has an entry in /etc/hosts like 192.168.X.Y apt.proxy.here that points at the static assigned IP of my Pi on that network. This isn't a 0 bandwidth solution but it's pretty good. HTH HAND etc.. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/5349c1c4.6020...@homeurl.co.uk
Re: mirroring debian-security, sharing package pool with stable/testing/sid pool
On 11/04/14 15:25, Lisi Reisz wrote:
("One" is very clumsy in
> English, but in this case I felt that the second person would appear
> to target Zenaan.)
Best Queen's English, Lisi ;)
--
Tony van der Hoff | mailto:t...@vanderhoff.org
Ariège, France |
--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/5347efa1.7020...@vanderhoff.org
Re: mirroring debian-security, sharing package pool with stable/testing/sid pool
On Friday 11 April 2014 03:46:49 Zenaan Harkness wrote:
> In my foggy memory, that at least for debian stable, I seem to
> remember something about security updates all get collected up,
> possibly with other updates (??) and they become the next stable
> point release.
Erm... No! I can accept that point releases contain all security
updates since the previous point release, but you certainly don't
have to wait until then. Security updates are released as soon as
they are available. Provided that one has the security repositories
in one's sources.list, security updates will be downloaded, and
installed as soon as one does an upgrade. ("One" is very clumsy in
English, but in this case I felt that the second person would appear
to target Zenaan.)
Lisi
--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/201404111425.24603.lisi.re...@gmail.com
Re: mirroring debian-security, sharing package pool with stable/testing/sid pool
On 4/11/14, Zenaan Harkness wrote: > On 4/11/14, Zenaan Harkness wrote: >> Thus my question: >> Is it possible to use debmirror, to mirror stable,testing,sid as well >> as debian-security for stable and testing, and have them all share the >> same package pool? > I guess another way to answer my question is: what is the time lapse > for packages to "migrate" from debian security repo, to regular debian > repos? > > If automatically part of the next 24-hour (or less) 'catch up' then > there is no point my creating a local security repo. In my foggy memory, that at least for debian stable, I seem to remember something about security updates all get collected up, possibly with other updates (??) and they become the next stable point release. A point-release is definitely too slow for security updates! BUT, users complain when their OS slows down their internet connection: "it was really slow last night" "but you've only got dialup, it was probably a security update" "a what? I had to send an important email to my parents, and it was so slow I don't even know if it ever got through! this has to be fixed! Windows didn't do this to me!?!" (Of course, WindowsXP did do this to people, but people don't properly remember the past, being creatures trained very sadly so, in the art of instant gratification and 'the consumer is king'. Oh what world we live in.) TIA Zenaan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/caosgnssjkjh5ymjgogcfxkiqlzswhz65_vvv5c1bwaqbwsw...@mail.gmail.com
Re: mirroring debian-security, sharing package pool with stable/testing/sid pool
On 4/11/14, Zenaan Harkness wrote: > Thus my question: > Is it possible to use debmirror, to mirror stable,testing,sid as well > as debian-security for stable and testing, and have them all share the > same package pool? In particular, for example, I stopped my initial debian-security (off of aarnet mirror) debmirror run, after about 20 package downloads, since I realised it said ~9GiB to download. Just now I checked one package, with results as follows: Just downloaded: nice /usr/bin/debmirror --diff=mirror --md5sums --rsync-extra=trace,doc,tools --exclude='/Translation-.*\.bz2$' --include='/Translation-(en).*\.bz2$' --include='/Translation-en.*\.bz2$' --verbose --progress --arch=amd64,i386,armhf --section=main,contrib,non-free --dist=wheezy/updates,jessie/updates --host=mirror.aarnet.edu.au --method=ftp --root=pub/debian-security /public/debian/debian-security ... Files to download: 9793 MiB [ 0%] Getting: pool/updates/main/a/a2ps/a2ps_4.14-1.1+deb7u1.diff.gz# ... [ 0%] Getting: pool/updates/main/a/asterisk/asterisk-config_1.8.13.1~dfsg1-3+deb7u3_all.deb ## ... And when I check my main repos package pool: $ find /public/debian/debian/|grep asterisk-config /public/debian/debian/pool/main/a/asterisk/asterisk-config_11.8.1~dfsg-1_all.deb /public/debian/debian/pool/main/a/asterisk/asterisk-config_1.8.13.1~dfsg1-3+deb7u3_all.deb /public/debian/debian/pool/main/a/asterisk/asterisk-config_11.8.1~dfsg-1~bpo70+1_all.deb and we see, the second result of my grep is the same package that was just downloaded for debian-security. Such inefficiency shall NOT be tolerated! :) I guess another way to answer my question is: what is the time lapse for packages to "migrate" from debian security repo, to regular debian repos? If automatically part of the next 24-hour (or less) 'catch up' then there is no point my creating a local security repo. Advice on this issue will be appreciated, Zenaan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/CAOsGNSTwTjNY17BTuyHBvy7U=rd-sr4+zslecjh6rff06vf...@mail.gmail.com
mirroring debian-security, sharing package pool with stable/testing/sid pool
I currently know to some degree, and use, debmirror.
I admin/help-desk for quite a few people in a rural area - the PCs I
admin are typically only connected to the Internet via high-latency,
low-bandwidth internet connections.
So, I run a debian mirror from a particular host which has a
high-bandwidth internet connection.
I then sync against this mirror every {random period of time,
sometimes a day, sometimes a week or more} and then use that sync'ed
mirror to update the various other PCs I admin.
Does it make sense, that when I connect my sync'ed USB drive to a
computer I am updating, that an apt-get update would also update from
a local debian-security mirror?
I could create a mirror from, eg, mirror.aarnet.edu.au (I'm in
Australia, and our ISP peers its free zone with aarnet).
I just began this process now, but stopped it when I realized that
there may be overlap between the debian-security package pool, and the
normal debian stable/testing pools.
Thus my question:
Is it possible to use debmirror, to mirror stable,testing,sid as well
as debian-security for stable and testing, and have them all share the
same package pool?
I can't see it, and I'm hoping I am simply not understanding
something. If I would have to go to a more advanced setup with some
other debian tool, I am willing, but that would take some time (namely
finding the time, so would not be immediately).
TIA
Zenaan
--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive:
https://lists.debian.org/CAOsGNSR+mJo1zqJwo7HUsyCToiCSSQJPsWFfdP=s8qeuoo6...@mail.gmail.com
Re: LVM Mirroring
On Fri, Jan 11, 2013 at 8:16 PM, Bob Proulx wrote: > Tom H wrote: >> Bob Proulx wrote: >>> I have not myself used LVM mirroring. I have only used mdadm MD >>> mirroring. This means I am fuzzy on your exact configuration. But >>> the above tells me that you have the root volume directly on lvm using >>> the rootVG-rootLV volume. Which is fine. >> >> [AIUI (I've never used this) LVM mirroring is like mdraid level1 but >> instead of mirroring partitions, you're mirroring LVs.] > > Just a small detail of words but as I understand it lvm mirrors PVs > (physical volumes) not LVs (logical volumes). But this is simply > language and words and doesn't change anything technical. You're right! >> Could the problem be the reverse of the earlier LVM-over-MDRAID thread >> whereby changing "root=/dev/mapper/rootVG-rootLV" to >> "root=/dev/mapper/rootVG/rootLV" will allow the system to boot?! > > Good idea. But not without a typo correction. I am sure you meant to > say /dev/rootVG/rootLV instead because /dev/mapper/rootVG/rootLV does > not ever exist. Yes. I was clearly in dreamland when replying earlier! > But I think that /dev/mapper/rootVG-rootLV will exist if anything > does. One of the earlier discussion threads was that > /dev/rootVG/rootLV didn't exist until udev was kicked with a reboot > but that the /dev/mapper version was available immediately. > > https://lists.debian.org/debian-user/2012/12/msg00420.html > https://lists.debian.org/debian-user/2012/12/msg00407.html Both "/dev/mapper/rootVG-rootLV" and "/dev/rootVG/rootLV" are created in my case (without any "manual" intervention from me). I thought that the latter might not be created in the initramfs but it is. You can boot using either on RHEL and Fedora so it must be possible on Debian... -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAOdo=Sy9eKAE_n=ppi=_r4ty5bsvlc+9dqkt9gkucq5xmsy...@mail.gmail.com
Re: LVM Mirroring
Tom H wrote: > Bob Proulx wrote: > > I have not myself used LVM mirroring. I have only used mdadm MD > > mirroring. This means I am fuzzy on your exact configuration. But > > the above tells me that you have the root volume directly on lvm using > > the rootVG-rootLV volume. Which is fine. > > [AIUI (I've never used this) LVM mirroring is like mdraid level1 but > instead of mirroring partitions, you're mirroring LVs.] Just a small detail of words but as I understand it lvm mirrors PVs (physical volumes) not LVs (logical volumes). But this is simply language and words and doesn't change anything technical. PV1\ - /LV1 PV2->-VG1-<-LV2 PV3/ - \LV3 > Could the problem be the reverse of the earlier LVM-over-MDRAID thread > whereby changing "root=/dev/mapper/rootVG-rootLV" to > "root=/dev/mapper/rootVG/rootLV" will allow the system to boot?! Good idea. But not without a typo correction. I am sure you meant to say /dev/rootVG/rootLV instead because /dev/mapper/rootVG/rootLV does not ever exist. But I think that /dev/mapper/rootVG-rootLV will exist if anything does. One of the earlier discussion threads was that /dev/rootVG/rootLV didn't exist until udev was kicked with a reboot but that the /dev/mapper version was available immediately. https://lists.debian.org/debian-user/2012/12/msg00420.html https://lists.debian.org/debian-user/2012/12/msg00407.html Bob signature.asc Description: Digital signature
Re: LVM Mirroring
On Fri, Jan 11, 2013 at 3:21 PM, Bob Proulx wrote:
> Luca Saletta wrote:
>>
>> menuentry 'Debian GNU/Linux, with Linux 2.6.32-5-686' --class debian
>> --class gnu-linux --class gnu --class os {
>> insmod lvm
>> insmod part_msdos
>> insmod ext2
>> set root='(rootVG-rootLV)'
>> search --no-floppy --fs-uuid --set
>> 809a5d50-9789-4e56-952d-868cb243cd0c
>> echo'Loading Linux 2.6.32-5-686 ...'
>> linux /boot/vmlinuz-2.6.32-5-686 root=/dev/mapper/rootVG-rootLV ro
>> quiet
>> echo'Loading initial ramdisk ...'
>> initrd /boot/initrd.img-2.6.32-5-686
>> }
>
> I have not myself used LVM mirroring. I have only used mdadm MD
> mirroring. This means I am fuzzy on your exact configuration. But
> the above tells me that you have the root volume directly on lvm using
> the rootVG-rootLV volume. Which is fine.
[AIUI (I've never used this) LVM mirroring is like mdraid level1 but
instead of mirroring partitions, you're mirroring LVs.]
Could the problem be the reverse of the earlier LVM-over-MDRAID thread
whereby changing "root=/dev/mapper/rootVG-rootLV" to
"root=/dev/mapper/rootVG/rootLV" will allow the system to boot?!
--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive:
http://lists.debian.org/CAOdo=SyRq-mvQ59-x36HD+c6ooCRTDbo_hFAP-BSiBoUsbG=u...@mail.gmail.com
Re: LVM Mirroring
On Fri, Jan 11, 2013 at 6:09 AM, Luca Saletta wrote: > Dear Mailinglist > ... > The mirroring ist done but as soon as i try to reboot my system, it > doesn't starts up and im entering the "grub recover"-mode. > Where you can't boot before you do the mirror, it isn't tied to the mirror. Can you let us know what error you get when you boot.? Sounds like you might need to run update-grub so that it scans and makes a new /boot/grub/grub.cfg. Also do like Bob suggested and make sure your initrd is updated. Make sure /etc/fstab is correct as well. I would do these before update-grub as I believe update-grub relies on both to be correct when it scans. -- Shane D. Johnson IT Administrator Rasmussen Equipment
Re: LVM Mirroring
Luca Saletta wrote:
> I'm trying to use LVM on my debian(6.0.6 | 2.6.32-5-686) distribution.
> Every thing went well.
Good.
> I installed the LVM partitions while i was setting up the new OS. (lvm2
> 2.02.66-5)
Good.
> I'm using GRUB2 and the entrys in the grub.cfg are written as well.
>
> menuentry 'Debian GNU/Linux, with Linux 2.6.32-5-686' --class debian
> --class gnu-linux --class gnu --class os {
> insmod lvm
> insmod part_msdos
> insmod ext2
> set root='(rootVG-rootLV)'
> search --no-floppy --fs-uuid --set
> 809a5d50-9789-4e56-952d-868cb243cd0c
> echo'Loading Linux 2.6.32-5-686 ...'
> linux /boot/vmlinuz-2.6.32-5-686 root=/dev/mapper/rootVG-rootLV ro
> quiet
> echo'Loading initial ramdisk ...'
> initrd /boot/initrd.img-2.6.32-5-686
> }
I have not myself used LVM mirroring. I have only used mdadm MD
mirroring. This means I am fuzzy on your exact configuration. But
the above tells me that you have the root volume directly on lvm using
the rootVG-rootLV volume. Which is fine.
> Resizing and adding a new Harddrive to the volume group i've created,
> was no problem. But as soon as I tried to set the mirroring my problem
> came up.
>
> The mirroring ist done but as soon as i try to reboot my system, it
> doesn't starts up and im entering the "grub recover"-mode.
This problem sounds very similar to one that was discussed just
recently on Jan 7 "Squeeze assembles one RAID array, at boot but not
the other". I think that even though you are doing things differently
that the root cause of the problem is very likely the same.
https://lists.debian.org/debian-user/2013/01/msg00184.html
I responded thusly:
https://lists.debian.org/debian-user/2013/01/msg00195.html
But there was much discussion in that thread including my suggestion
to use rescue media to repair the non-booting system.
The /boot/initrd.img-$(uname -r) file is an initial ram filesystem
that must contain all of the information needed for the kernel to boot
to the system. This includes information on how to mount the root
file system. For MD (multi-device) mirroring it includes an
mdadm.conf file instructing the bootstrap process on what partitions
to assemble into the raid. I believe (without knowing) that for LVM
it must do the same thing too.
This means that if you add an additional disk to your LVM based system
then I believe that you must also rebuild the initrd so that it will
include this new disk information. If it needs to know about the new
disk in order to get the lvm system up and running and to mount the
root file system then I am sure that the initrd needs to be re-frozen
with this new information.
> I used following commands :
> Resizing a FS - lvextend –L +500M /dev/rootVG/rootLV
> Add HDD - vgextend rootVG /dev/sdb
> Set up the Mirroring - lvconvert –m1 –-mirrorlog core /dev/rootVG/rootLV
> lvconvert –m1 –-mirrorlog mirrored /dev/rootVG/rootLV
> (Didn't work...)
I haven't used lvm mirroring and so do not know about the above.
> Now maybe you could help me with whats up to do next?
I suggest the same solution here as in the MD raid boot problem. I
would use dpkg-reconfigure to run the linux kernel image post install
script to re-freeze the initrd with the current system information. I
think that is going to be needed for you since you have added
additional disks and those will be needed in order to assemble the
root filesystem for use. Using 2.6.32-5-amd64 as an example only,
substitute your appropriate flavor:
dpkg-reconfigure linux-image-2.6.32-5-amd64
Be sure to look at the instructions in this message for how to use the
installation media as a rescue image. This will get you booted and
into your system so that you can make administrative changes.
https://lists.debian.org/debian-user/2013/01/msg00218.html
Be sure to let us know how things are going for you.
Bob
signature.asc
Description: Digital signature
LVM Mirroring
Dear Mailinglist
I'm trying to use LVM on my debian(6.0.6 | 2.6.32-5-686) distribution.
Every thing went well.
I installed the LVM partitions while i was setting up the new OS. (lvm2
2.02.66-5)
I'm using GRUB2 and the entrys in the grub.cfg are written as well.
menuentry 'Debian GNU/Linux, with Linux 2.6.32-5-686' --class debian
--class gnu-linux --class gnu --class os {
insmod lvm
insmod part_msdos
insmod ext2
set root='(rootVG-rootLV)'
search --no-floppy --fs-uuid --set
809a5d50-9789-4e56-952d-868cb243cd0c
echo'Loading Linux 2.6.32-5-686 ...'
linux /boot/vmlinuz-2.6.32-5-686
root=/dev/mapper/rootVG-rootLV ro quiet
echo'Loading initial ramdisk ...'
initrd /boot/initrd.img-2.6.32-5-686
}
Resizing and adding a new Harddrive to the volume group i've created,
was no problem. But as soon as I tried to set the mirroring my problem
came up.
The mirroring ist done but as soon as i try to reboot my system, it
doesn't starts up and im entering the "grub recover"-mode.
I used following commands :
Resizing a FS - lvextend –L +500M /dev/rootVG/rootLV
Add HDD - vgextend rootVG /dev/sdb
Set up the Mirroring - lvconvert –m1 –-mirrorlog core /dev/rootVG/rootLV
lvconvert –m1 –-mirrorlog mirrored /dev/rootVG/rootLV
(Didn't work...)
Now maybe you could help me with whats up to do next?
Best regards
Luca Saletta
--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/50f01d21025eb...@gwia.msw.ch
Re: RAID 1 (mirroring) question
On Wed, Dec 19, 2012 at 6:07 AM, Bob Proulx wrote: > Note that after a power cycle even if the RAID 1 array needs to be > sync'd between the mirrored disks that the system will still boot okay > and will operate normally. I have no idea what other systems do but > you can boot the system, log in, and it will perform its normal > tasks. If the array needs to be sync'd then it will sync in the > background. This is why the kernel implements the speed_limit_max > values so that normal system operation will not be starved of disk > bandwidth. You might not notice that it is doing this. It might > finish the task without impacting normal system functions. > with Windows 7, the system boots and I can use it but is very sluggish due to all that disk activity. Hoping linux will handle this situation better. will try md RAID and see how it performs. from man md page: "While this recovery process is happening, the md driver will monitor accesses to the array and will slow down the rate of recovery if other activity is happening, so that normal access to the array will not be unduly affected. When no other activity is happening, the recovery process proceeds at full speed. The actual speed targets for the two different situations can be controlled by the *speed_limit_min* and *speed_limit_max*control files mentioned below. " this sounds promising. I will also be building a BSD server and see how ZFSRAID handles these situations. Thanks you very much for the detailed reply. It was very useful. -- Kind regards, Yudi
Re: RAID 1 (mirroring) question
Bob Proulx wrote: > The Linus software raid also had the capability to use a block bitmap > to speed up resync after a crash because then it tracks which blocks > are dirty. > See the documentation on this mdadm command to configure an internal > bitmap to speed up a re-sync after an event such as a power loss. > mdadm /dev/md1 --grow --bitmap=internal Beware, this option can severly impact the write performance of your RAID if your system is mostly doing small writes (which is what most systems do). My home desktop system was configured with a RAID1 with write intent bitmaps and the subjective day-to-day performance was awful, like driving a car with active handbrake. After disabling the bitmaps the write latency went down and the system felt much more "snappy" and responsive. I now use an intent bitmap only an huge RAID sets which are mostly written in big chunks (for example backup files from Bacula) and would need a resync in the magnitude of days if it ever happend to get desynced due to a power failure in the wrong moment. Grüße, Sven. -- Sigmentation fault. Core dumped. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/89bthh8gk...@mids.svenhartge.de
Re: RAID 1 (mirroring) question
yudi v wrote: > I am looking at using Debian software RAID mirroring and would like > to know how it handles system crashes and disk failures. It handles it quite well. > My only experience with software RAID 1 is with windows 7 inbuilt > option. Whenever the system does not shutdown cleanly, upon reboot > the disks start resynching and the whole system becomes very > sluggish, almost impossible to use. Depending on the size of the > disks, this can be quite long (I am guessing this is because it is > resynching at the block level). If a resync is needed then the entire time to complete depends upon how much data needs to be sync'd and how much data I/O bandwidth is available from the hardware. Large disks can take a while regardless of the system. > I was speaking with someone using freeBSD/ZFS and they reckon ZFS > does not resynch after a crash and when a disk is replaced it only > copies data not each block. > > How does linux software RAID 1 handle these two scenarios? In my experience, and I just experienced a power out crash of three RAID1 systems yesterday, most of the time the array will remain in sync after a crash. All three of my power crashed systems yesterday remained in sync. This probably depends upon the activity level of the systems. A system that is more idle will be less likely to experience this. Or rather a busy system is more likely to experience this and need to be sync'd. Note that you can change the Linux kernel software raid sync speed limits by setting dev.raid.speed_limit_max. $ cat /proc/sys/dev/raid/speed_limit_min # echo 5 > /proc/sys/dev/raid/speed_limit_min Can do better with: # echo 50 > /proc/sys/dev/raid/speed_limit_min The Linus software raid also had the capability to use a block bitmap to speed up resync after a crash because then it tracks which blocks are dirty. See the documentation on this mdadm command to configure an internal bitmap to speed up a re-sync after an event such as a power loss. mdadm /dev/md1 --grow --bitmap=internal This does not speed up an array check which must by definition check all of the blocks. But it will speed up a full rebuild after a power outage. Requires --detail to report that the superblock is persistent and the array active. I was only using the bitmap on one of the three machines that had the power out yesterday. The other two machines were mostly idle and survived the power cycle without needing an array sync regardless. Note that after a power cycle even if the RAID 1 array needs to be sync'd between the mirrored disks that the system will still boot okay and will operate normally. I have no idea what other systems do but you can boot the system, log in, and it will perform its normal tasks. If the array needs to be sync'd then it will sync in the background. This is why the kernel implements the speed_limit_max values so that normal system operation will not be starved of disk bandwidth. You might not notice that it is doing this. It might finish the task without impacting normal system functions. During the hours when the disks are out of sync a disk failure would not have redundancy however. Therefore getting the system back in sync again should be a priority to restore the redundancy of RAID. I usually partition the disk into partitions no larger than about 250G each. A 1T disk I would set up with four 250G partitions. Then use them as physical extents for LVM all combined together into a single 1T volume group. Then use that to create logical partitions as desired. The advantage is that if a disk fails and needs to be replaced that each 250G partition is sync'd independently. And due to LVM if the disk is not full the extra partitions may not be used and may not need to be sync'd. However the steady state of disks is full and therefore I am never able to reap that benefit. The only real gain is that as the raid1 sync proceeds the individual partitions can be checked off as done on the scoreboard as incremental progress along the way. A subsequent reboot while the sync is proceeding would not restart at the beginning for mirrors that are back in sync again. Bob signature.asc Description: Digital signature
RAID 1 (mirroring) question
Hi all, I am looking at using Debian software RAID mirroring and would like to know how it handles system crashes and disk failures. My only experience with software RAID 1 is with windows 7 inbuilt option. Whenever the system does not shutdown cleanly, upon reboot the disks start resynching and the whole system becomes very sluggish, almost impossible to use. Depending on the size of the disks, this can be quite long (I am guessing this is because it is resynching at the block level). I was speaking with someone using freeBSD/ZFS and they reckon ZFS does not resynch after a crash and when a disk is replaced it only copies data not each block. How does linux software RAID 1 handle these two scenarios? -- Kind regards, Yudi
Re: issue with mdadm and mirroring drives
26/01/2012 23:07, Joey L wrote:
>>
>>>
>>> When I boot the system with all drives in, I get the superflous error.
>>>
>>> So the only way to boot is only to put in /dev/sdc alone and boot.
>>> when i get to a linux prompt, I insert the second drive into the system
>>> /dev/sdd
>>>
>>> To sync them, /dev/sdd has already failed, so i run
>>> sfdisk -d /dev/sdc | sfdisk /dev/sdd
>>> ** i get an error that nothing has changed - so I run it with the
>>> --force command to get the partitions identical like:
>>> sfdisk -d /dev/sdc | sfdisk --force /dev/sdd
>>
>> Why do you do that ? You are forcing the partitioning of the first disk
>> onto the second, this could work at raid creation time but isn't the
>> proper procedure to re-add a failed member to an array. You don't have
>> to "sync" the data and even less the disk partitioning manually prior to
>> re-adding it to the raid. mdadm will handle the resync.
>
> I did not mean to do this - i think this is my main issue - i can not
> zero out drives with mdadm.
> It gives an error that i can not get past - so i use force option.
It would be useful to know the exact mdadm error message when you try to
do "mdadm --zero-superblock" on the failed raid member. You need to
remove ("mdadm --remove") the failed member first, or work when the
array isn't started.
> Do you recommend any other utility to zero out drives - that will make
> them blank ?
sfdisk doesn't "zero out" the disk, with option "-d" it merely dump the
partition table in a format that can be reused to copy over another
disk. It's completely possible to restore the target disk to it's
previous state with the correct information (ideally the output of
"sfdisk -O" , but testdisk could do the trick). Everything is on the
disk, but there is no more "map" to retrieve it.
To "zero out" a disk one could use dd to fill the disk with zeros, or
random data if security is important, badblock in write mode can do that
too.
The important part is that sfdisk has no idea of anything such as
metadata, bootloaders, file-system, even extended partitions can be
clobbered by the "-d" magic trick. It works reliably only for empty
primary partitions, preferably small, on disks with msdos disk-labels,
preferably identicals (same geometry), using the "-D" option.
So it's fragile enough that you don't want to mess with the "--force"
option when sfdisk complains, it can mess things easily enough as it is
;-) .
> I think it maybe an issue with my working drive - i think the
> partitions are screwed up there - and when sfdsik copies - it does not
> copy correctly.
> But i did boot with knoppix and went to fdisk and deleted the
> partitions - but still had issues with sfdisk.
> An utilty u recommend ??
>
After clearing previous raid metadata if any, I usually use parted to
create new disklabels and empty (non formatted) partitions (if not using
the whole disk) before creating a raid.
Repeat on all disks to be included in the raid with the same values.
See "help mklabel" and "help mkpart" in parted, or "man parted".
When done create the raid (with 1.2 metadata), start it, and format the
raid device with mkfs. Nothing else is needed, no flag or file-system
type changing to "fd".
Even better, clear the disks from raid metadata, create new disklabels,
and let Debian installer take care of the rest. Yes, it means
reinstalling the system, but it's by far the easiest/safest/fastest
option. Personal data can be restored from a backup quicker than one can
read parted manual !
>>>
>>>
>>> Model: ATA ST31000340AS (scsi)
>>> Disk /dev/sdc: 1000GB
>>> Sector size (logical/physical): 512B/512B
>>> Partition Table: msdos
>>>
>>> Number Start End SizeType File system Flags
>>> 1 1049kB 996GB 996GB primary ext3raid
>>> 2 996GB 1000GB 4204MB primary linux-swap(v1)
>>>
>>>
>>> Model: ATA ST31000528AS (scsi)
>>> Disk /dev/sdd: 1000GB
>>> Sector size (logical/physical): 512B/512B
>>> Partition Table: msdos
>>>
>>> Number Start End SizeType File system Flags
>>> 1 1049kB 996GB 996GB primary ext4raid
>>> 2 996GB 1000GB 4204MB primary linux-swap(v1)
>>>
>>
>> Why do you have file-systems on your partitions ? Only the "md" raid
>> devices should be formatted with a file-system, not the underlying
>> partitions !
>> I would be curious to know what "fsck" says about your md devices (fsck
>> /dev/md0 for example) ?
>
> Again - i think sfdisk copy from working drive is causing this issue.
> Can i go into fdsik to fix ?
Having file-systems on the partitions before creating the raid doesn't
help. If you want to (try to) "fix" things at this stage, run "e2fsck
-cc" on the raid device (unmounted), and then "resize2fs" . But this
will be painfully slow with 1GB disks, think more than a day...
Did you try to run "fsck" on the raid device (from a live-cd possibly) ?
> Change the labels ? i just think they are labeled incorrectly.
Talking abou
Re: issue with mdadm and mirroring drives
> You could check whether grub's OK via bootinfoscript (although the > only real test'll be a reboot...). > > http://sourceforge.net/projects/bootinfoscript/ > thanks for the script - will check it out > >>> My parted -l is: >>> >>> Disk /dev/sda: 1000GB >>> Number Start End Size Type File system Flags >>> 1 32.3kB 1000GB 1000GB primary ext3 raid >>> >>> Disk /dev/sdb: 1000GB >>> Number Start End Size Type File system Flags >>> 1 32.3kB 1000GB 1000GB primary ext3 raid >>> >>> Disk /dev/sdc: 1000GB >>> Number Start End Size Type File system Flags >>> 1 1049kB 996GB 996GB primary ext3 raid >>> 2 996GB 1000GB 4204MB primary linux-swap(v1) >>> >>> Disk /dev/sdd: 1000GB >>> Number Start End Size Type File system Flags >>> 1 1049kB 996GB 996GB primary ext4 raid >>> 2 996GB 1000GB 4204MB primary linux-swap(v1) >> >> Why do you have file-systems on your partitions ? Only the "md" raid >> devices should be formatted with a file-system, not the underlying >> partitions ! > > sdc1 is ext3 and sdd1 is ext4; somehow. > i think it is a label issue - because when i do mount - it is all ext4 -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAK3ER7uAO2dQwJixZ=MBTGRObaY2+5oNVRjqKot21Lhu13E=x...@mail.gmail.com
Re: issue with mdadm and mirroring drives
>> >> My configuration is as such: >> /dev/md0 = /dev/sdc1 and /dev/sdd1 >> /dev/md1 = /dev/sda1 and /dev/sb1 >> >> My swap partitions are not part of the array or mirror at all - >> they are just regular partitions - they are: >> /dev/sdc2 and /dev/sdd2. > > Any particular reason do do that ? If you want your system/applications > to carry on working if a disk fails you need the swap on raid1 too. I see your point - I will create new linux raid auto partitions and mirror it as well - thanks > >> >> When I boot the system with all drives in, I get the superflous error. >> >> So the only way to boot is only to put in /dev/sdc alone and boot. >> when i get to a linux prompt, I insert the second drive into the system >> /dev/sdd >> >> To sync them, /dev/sdd has already failed, so i run >> sfdisk -d /dev/sdc | sfdisk /dev/sdd >> ** i get an error that nothing has changed - so I run it with the >> --force command to get the partitions identical like: >> sfdisk -d /dev/sdc | sfdisk --force /dev/sdd > > Why do you do that ? You are forcing the partitioning of the first disk > onto the second, this could work at raid creation time but isn't the > proper procedure to re-add a failed member to an array. You don't have > to "sync" the data and even less the disk partitioning manually prior to > re-adding it to the raid. mdadm will handle the resync. I did not mean to do this - i think this is my main issue - i can not zero out drives with mdadm. It gives an error that i can not get past - so i use force option. Do you recommend any other utility to zero out drives - that will make them blank ? I think it maybe an issue with my working drive - i think the partitions are screwed up there - and when sfdsik copies - it does not copy correctly. But i did boot with knoppix and went to fdisk and deleted the partitions - but still had issues with sfdisk. An utilty u recommend ?? >> >> >> Model: ATA ST31000340AS (scsi) >> Disk /dev/sdc: 1000GB >> Sector size (logical/physical): 512B/512B >> Partition Table: msdos >> >> Number Start End Size Type File system Flags >> 1 1049kB 996GB 996GB primary ext3 raid >> 2 996GB 1000GB 4204MB primary linux-swap(v1) >> >> >> Model: ATA ST31000528AS (scsi) >> Disk /dev/sdd: 1000GB >> Sector size (logical/physical): 512B/512B >> Partition Table: msdos >> >> Number Start End Size Type File system Flags >> 1 1049kB 996GB 996GB primary ext4 raid >> 2 996GB 1000GB 4204MB primary linux-swap(v1) >> > > Why do you have file-systems on your partitions ? Only the "md" raid > devices should be formatted with a file-system, not the underlying > partitions ! > I would be curious to know what "fsck" says about your md devices (fsck > /dev/md0 for example) ? Again - i think sfdisk copy from working drive is causing this issue. Can i go into fdsik to fix ? Change the labels ? i just think they are labeled incorrectly. > > I am starting to think that you have much lower level problems. When you > created this system, where the disks "clean", or did you use "sfdisk" > over existing formatted partitions ? Where the disks used in a raid > before ? If this is the case you should consider backing up and > recreating the raid properly. > > Also, if one disk is repeatedly dropping from the raid array, consider > looking at the "smart" values, it may be dying. > What smart values are u refering to ? is that a utility ? Again - i think i need a clean utility or at least a procedure other then --zero option of mdadm to clear out the drive. thanks mjh -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAK3ER7u6V14aCSgrNvvGs=BR3sRSnvy=vrhSv_WeXBLy5Y4=h...@mail.gmail.com
Re: issue with mdadm and mirroring drives
On Wed, Jan 25, 2012 at 5:35 PM, tv.deb...@googlemail.com wrote: > 25/01/2012 19:16, Joey L wrote: >> Okay..I am telling all in this email -:) >> >> My configuration is as such: >> /dev/md0 = /dev/sdc1 and /dev/sdd1 >> /dev/md1 = /dev/sda1 and /dev/sb1 >> >> My swap partitions are not part of the array or mirror at all - >> they are just regular partitions - they are: >> /dev/sdc2 and /dev/sdd2. > > Any particular reason do do that ? If you want your system/applications > to carry on working if a disk fails you need the swap on raid1 too. +1 >> When I boot the system with all drives in, I get the superflous error. >> >> So the only way to boot is only to put in /dev/sdc alone and boot. >> when i get to a linux prompt, I insert the second drive into the system >> /dev/sdd >> >> To sync them, /dev/sdd has already failed, so i run >> sfdisk -d /dev/sdc | sfdisk /dev/sdd >> ** i get an error that nothing has changed - so I run it with the >> --force command to get the partitions identical like: >> sfdisk -d /dev/sdc | sfdisk --force /dev/sdd > > Why do you do that ? You are forcing the partitioning of the first disk > onto the second, this could work at raid creation time but isn't the > proper procedure to re-add a failed member to an array. You don't have > to "sync" the data and even less the disk partitioning manually prior to > re-adding it to the raid. mdadm will handle the resync. It's unorthodox but it's a replacement of zeroing the superblock with mdadm before adding the partition back to into the array. >> Once that is successful - i run mdadm --manage /dev/md0 --add /dev/sdd1 >> >> And the sync process starts, and when it is done yesterday, I ran this >> and got these errors: >> >> root@rider:~# grub-install --recheck --no-floppy /dev/sdc >> error: superfluous RAID member (2 found). >> error: superfluous RAID member (2 found). >> error: superfluous RAID member (2 found). >> error: superfluous RAID member (2 found). >> error: superfluous RAID member (2 found). >> Installation finished. No error reported. >> root@rider:~# grub-install --recheck --no-floppy /dev/sdd >> error: superfluous RAID member (2 found). >> error: superfluous RAID member (2 found). >> error: superfluous RAID member (2 found). >> error: superfluous RAID member (2 found). >> error: superfluous RAID member (2 found). >> Installation finished. No error reported. There are a few bugs on b.d.o that correspond to this error but none that fit your case, AFAICT. "grub-install" does say "Installation finished. No error reported." You could check whether grub's OK via bootinfoscript (although the only real test'll be a reboot...). http://sourceforge.net/projects/bootinfoscript/ >> My parted -l is: >> >> Disk /dev/sda: 1000GB >> Number Start End Size Type File system Flags >> 1 32.3kB 1000GB 1000GB primary ext3 raid >> >> Disk /dev/sdb: 1000GB >> Number Start End Size Type File system Flags >> 1 32.3kB 1000GB 1000GB primary ext3 raid >> >> Disk /dev/sdc: 1000GB >> Number Start End Size Type File system Flags >> 1 1049kB 996GB 996GB primary ext3 raid >> 2 996GB 1000GB 4204MB primary linux-swap(v1) >> >> Disk /dev/sdd: 1000GB >> Number Start End Size Type File system Flags >> 1 1049kB 996GB 996GB primary ext4 raid >> 2 996GB 1000GB 4204MB primary linux-swap(v1) > > Why do you have file-systems on your partitions ? Only the "md" raid > devices should be formatted with a file-system, not the underlying > partitions ! sdc1 is ext3 and sdd1 is ext4; somehow. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAOdo=szbcqu8tasyn1_ycn2+huzkkth4hm2smqh5m+iecb+...@mail.gmail.com
Re: issue with mdadm and mirroring drives
25/01/2012 19:16, Joey L wrote: In-line reply ; > Okay..I am telling all in this email -:) > > My configuration is as such: > /dev/md0 = /dev/sdc1 and /dev/sdd1 > /dev/md1 = /dev/sda1 and /dev/sb1 > > My swap partitions are not part of the array or mirror at all - > they are just regular partitions - they are: > /dev/sdc2 and /dev/sdd2. Any particular reason do do that ? If you want your system/applications to carry on working if a disk fails you need the swap on raid1 too. > > When I boot the system with all drives in, I get the superflous error. > > So the only way to boot is only to put in /dev/sdc alone and boot. > when i get to a linux prompt, I insert the second drive into the system > /dev/sdd > > To sync them, /dev/sdd has already failed, so i run > sfdisk -d /dev/sdc | sfdisk /dev/sdd > ** i get an error that nothing has changed - so I run it with the > --force command to get the partitions identical like: > sfdisk -d /dev/sdc | sfdisk --force /dev/sdd Why do you do that ? You are forcing the partitioning of the first disk onto the second, this could work at raid creation time but isn't the proper procedure to re-add a failed member to an array. You don't have to "sync" the data and even less the disk partitioning manually prior to re-adding it to the raid. mdadm will handle the resync. > > Once that is successful - i run mdadm --manage /dev/md0 --add /dev/sdd1 > > And the sync process starts, and when it is done yesterday, I ran this > and got these errors: > > > root@rider:~# grub-install --recheck --no-floppy /dev/sdc > error: superfluous RAID member (2 found). > error: superfluous RAID member (2 found). > error: superfluous RAID member (2 found). > error: superfluous RAID member (2 found). > error: superfluous RAID member (2 found). > Installation finished. No error reported. > root@rider:~# grub-install --recheck --no-floppy /dev/sdd > error: superfluous RAID member (2 found). > error: superfluous RAID member (2 found). > error: superfluous RAID member (2 found). > error: superfluous RAID member (2 found). > error: superfluous RAID member (2 found). > Installation finished. No error reported. > root@rider:~# grub-install --recheck --no-floppy /dev/sdc > > [cut] > > > My parted -l is: > > > Model: ATA ST31000340AS (scsi) > Disk /dev/sda: 1000GB > Sector size (logical/physical): 512B/512B > Partition Table: msdos > > Number Start End SizeType File system Flags > 1 32.3kB 1000GB 1000GB primary ext3 raid > > > Model: ATA ST31000340AS (scsi) > Disk /dev/sdb: 1000GB > Sector size (logical/physical): 512B/512B > Partition Table: msdos > > Number Start End SizeType File system Flags > 1 32.3kB 1000GB 1000GB primary ext3 raid > > > Model: ATA ST31000340AS (scsi) > Disk /dev/sdc: 1000GB > Sector size (logical/physical): 512B/512B > Partition Table: msdos > > Number Start End SizeType File system Flags > 1 1049kB 996GB 996GB primary ext3raid > 2 996GB 1000GB 4204MB primary linux-swap(v1) > > > Model: ATA ST31000528AS (scsi) > Disk /dev/sdd: 1000GB > Sector size (logical/physical): 512B/512B > Partition Table: msdos > > Number Start End SizeType File system Flags > 1 1049kB 996GB 996GB primary ext4raid > 2 996GB 1000GB 4204MB primary linux-swap(v1) > Why do you have file-systems on your partitions ? Only the "md" raid devices should be formatted with a file-system, not the underlying partitions ! I would be curious to know what "fsck" says about your md devices (fsck /dev/md0 for example) ? > Warning: Unable to open /dev/fd0 read-write (Read-only file system). /dev/fd0 > has been opened read-only. > > Error: /dev/fd0: unrecognised disk label > > Model: Linux Software RAID Array (md) > Disk /dev/md0: 996GB > Sector size (logical/physical): 512B/512B > Partition Table: loop > > Number Start EndSize File system Flags > 1 0.00B 996GB 996GB ext4 > > > Model: Linux Software RAID Array (md) > Disk /dev/md1: 1000GB > Sector size (logical/physical): 512B/512B > Partition Table: loop > > Number Start End SizeFile system Flags > 1 0.00B 1000GB 1000GB ext4 > > > [cut] > > > > I am currently terrified to reboot my box --- any help would be appriciated - > > I have NOT run the command "update-initramfs -u -k all " and do not > know if i should. > > > thanks > mjh You seem to be walking in the dark awaiting to hit the next pole ;-) . Updating the initrd won't solve file-system or raid inconsistencies, it would only be necessary if the content of mdadm.conf had changed. I am starting to think that you have much lower level problems. When you created this system, where the disks "clean", or did you use "sfdisk" over existing formatted partitions ? Where the disks used in a raid before ? If this is the case you should consider backing up and recreatin
Re: issue with mdadm and mirroring drives
xt2
set root='(md/0)'
search --no-floppy --fs-uuid --set 611258bc-4ace-4779-b44d-73ba1949f7db
echo'Loading Linux 2.6.32-5-amd64 ...'
linux /boot/vmlinuz-2.6.32-5-amd64
root=UUID=611258bc-4ace-4779-b44d-73ba1949f7db ro text
echo'Loading initial ramdisk ...'
initrd /boot/initrd.img-2.6.32-5-amd64
}
menuentry 'Debian GNU/Linux, with Linux 2.6.32-5-amd64 (recovery
mode)' --class debian --class gnu-linux --class gnu --class os {
insmod raid
insmod mdraid
insmod part_msdos
insmod part_msdos
insmod ext2
set root='(md/0)'
search --no-floppy --fs-uuid --set 611258bc-4ace-4779-b44d-73ba1949f7db
echo'Loading Linux 2.6.32-5-amd64 ...'
linux /boot/vmlinuz-2.6.32-5-amd64
root=UUID=611258bc-4ace-4779-b44d-73ba1949f7db ro single
echo'Loading initial ramdisk ...'
initrd /boot/initrd.img-2.6.32-5-amd64
I am currently terrified to reboot my box --- any help would be appriciated -
I have NOT run the command "update-initramfs -u -k all " and do not
know if i should.
thanks
mjh
On Tue, Jan 24, 2012 at 5:12 PM, tv.deb...@googlemail.com
wrote:
> 24/01/2012 18:34, Joey L wrote:
>> In related questions - i hope you can answer - are :
>>
>> 1. why do i get an error when trying to make swap partition - i get
>> this error below after running fdisk, creating a primary partition
>> /dev/sdd2 and then changing the type - with t -
>> As in below ---
>>
>> 2. the /dev/sdd drive is the mirrored drive in the set - do i have to
>> set /dev/sdd1 active in fdisk ???
>>
>>
>> root@rider:~# mkswap /dev/sdd2
>>
>> mkswap: error: swap area needs to be at least 40 KiB
>> Usage: mkswap [-c] [-pPAGESZ] [-L label] [-U UUID] /dev/name [blocks]
>> root@rider:~# fdisk -l /dev/sdd
>>
>>
>> Disk /dev/sdd: 1000.2 GB, 1000204886016 bytes
>> 255 heads, 63 sectors/track, 121601 cylinders
>> Units = cylinders of 16065 * 512 = 8225280 bytes
>> Sector size (logical/physical): 512 bytes / 512 bytes
>> I/O size (minimum/optimal): 512 bytes / 512 bytes
>> Disk identifier: 0x0005d0fd
>>
>>
>> Device Boot Start End Blocks Id System
>> /dev/sdd1 1 121091 972663426 fd Linux raid autodetect
>> /dev/sdd2 121092 121601 4096575 82 Linux swap / Solaris
>>
>>
>> thanks
>> mjh
>>
>
> What are you trying to achieve exactly ? fdisk knows nothing about mdadm
> raid, here you say that /dev/sdd is part of a mirroring raid (each
> partitions are I guess), and later you are working with fdisk on
> /dev/sdd2. If /dev/sdd2 is part of a raid1 array you must work on the
> mdadm device (/dev/md*), this is the device you need to "mkswap". There
> is nothing else to do to make it work as swap (apart registering it in
> mdadm.conf and fstab).
> If you use fdisk on a raid member you are going to wreck your system in
> no time !
>
> I am not sure I understand what you are doing and how your raid is set.
> Could you post the output of:
>
> cat /proc/mdstat
>
> and
>
> parted -l
>
>
>
>
> --
> To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
> Archive: http://lists.debian.org/4f1f2cb4.7040...@googlemail.com
>
--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive:
http://lists.debian.org/cak3er7tw3ffr9aobojfffoqmijto14seutpkjdtr2oxgapn...@mail.gmail.com
Re: issue with mdadm and mirroring drives
24/01/2012 19:10, Joey L wrote:
> Sorry to load up on related issues in this mail, but have a big issue:
> After having synced my mirrored drives in software raid -
> I get the following error:
> error: superfluous RAID member (2 found).
There is a good chance that the error reported is more cosmetic than
alarming. Could you give the metadata version used on the affected array
("cat /proc/mdstat" or "mdadm --examine /dev/sd??" replace "??" with
affected partitions addresses, otherwise it's going to print for every
partitions).
> the boot drive is /dev/sdc1 but added another one to the mirror - /dev/sdd1
> my questions :
> 1. should i put both drives in the bootup bios scan ?
Is that a bios setting on your mainboard ? If it is I think you can as
both drive should have a working grub first stage.
> 2. should i set their partitions as bootable in fdisk ?
Usually not necessary, but won't hurt.
> 3. how do i fix this without having to zero out the drive and re-add
> it to the mirror ?
If it works you don't have to "fix" it, I encountered this error before
and thought it had been fixed as a bug in grub. Grub is detecting
metadata both on the drive and the partitions, and considers that two
sets of metadata are one too many. But it shouldn't lead to a problem,
grub will fall-back to one set.
The only nasty cases I know occur if the drives where previously used in
a raid and metadata were not cleared prior to reusing it ; if the raid
partitions have been created over an existing file-system and metadata
are misplaced ; or if a partitioned raid1 (as opposed to partitions
assembled in raid1 arrays) is wrongly detected as an unpartitioned one.
Knowing more about the way you created the raid and your setup would
help to know if you fall in one of this categories.
--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4f1f32ac.90...@googlemail.com
Re: issue with mdadm and mirroring drives
24/01/2012 18:34, Joey L wrote: > In related questions - i hope you can answer - are : > > 1. why do i get an error when trying to make swap partition - i get > this error below after running fdisk, creating a primary partition > /dev/sdd2 and then changing the type - with t - > As in below --- > > 2. the /dev/sdd drive is the mirrored drive in the set - do i have to > set /dev/sdd1 active in fdisk ??? > > > root@rider:~# mkswap /dev/sdd2 > > mkswap: error: swap area needs to be at least 40 KiB > Usage: mkswap [-c] [-pPAGESZ] [-L label] [-U UUID] /dev/name [blocks] > root@rider:~# fdisk -l /dev/sdd > > > Disk /dev/sdd: 1000.2 GB, 1000204886016 bytes > 255 heads, 63 sectors/track, 121601 cylinders > Units = cylinders of 16065 * 512 = 8225280 bytes > Sector size (logical/physical): 512 bytes / 512 bytes > I/O size (minimum/optimal): 512 bytes / 512 bytes > Disk identifier: 0x0005d0fd > > >Device Boot Start End Blocks Id System > /dev/sdd1 1 121091 972663426 fd Linux raid autodetect > /dev/sdd2 121092 121601 4096575 82 Linux swap / Solaris > > > thanks > mjh > What are you trying to achieve exactly ? fdisk knows nothing about mdadm raid, here you say that /dev/sdd is part of a mirroring raid (each partitions are I guess), and later you are working with fdisk on /dev/sdd2. If /dev/sdd2 is part of a raid1 array you must work on the mdadm device (/dev/md*), this is the device you need to "mkswap". There is nothing else to do to make it work as swap (apart registering it in mdadm.conf and fstab). If you use fdisk on a raid member you are going to wreck your system in no time ! I am not sure I understand what you are doing and how your raid is set. Could you post the output of: cat /proc/mdstat and parted -l -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4f1f2cb4.7040...@googlemail.com
Re: issue with mdadm and mirroring drives
Sorry to load up on related issues in this mail, but have a big issue: After having synced my mirrored drives in software raid - I get the following error: error: superfluous RAID member (2 found). the boot drive is /dev/sdc1 but added another one to the mirror - /dev/sdd1 my questions : 1. should i put both drives in the bootup bios scan ? 2. should i set their partitions as bootable in fdisk ? 3. how do i fix this without having to zero out the drive and re-add it to the mirror ? On Tue, Jan 24, 2012 at 12:34 PM, Joey L wrote: > In related questions - i hope you can answer - are : > > 1. why do i get an error when trying to make swap partition - i get > this error below after running fdisk, creating a primary partition > /dev/sdd2 and then changing the type - with t - > As in below --- > > 2. the /dev/sdd drive is the mirrored drive in the set - do i have to > set /dev/sdd1 active in fdisk ??? > > > root@rider:~# mkswap /dev/sdd2 > > mkswap: error: swap area needs to be at least 40 KiB > Usage: mkswap [-c] [-pPAGESZ] [-L label] [-U UUID] /dev/name [blocks] > root@rider:~# fdisk -l /dev/sdd > > > Disk /dev/sdd: 1000.2 GB, 1000204886016 bytes > 255 heads, 63 sectors/track, 121601 cylinders > Units = cylinders of 16065 * 512 = 8225280 bytes > Sector size (logical/physical): 512 bytes / 512 bytes > I/O size (minimum/optimal): 512 bytes / 512 bytes > Disk identifier: 0x0005d0fd > > > Device Boot Start End Blocks Id System > /dev/sdd1 1 121091 972663426 fd Linux raid autodetect > /dev/sdd2 121092 121601 4096575 82 Linux swap / Solaris > > > thanks > mjh > > > > On Thu, Jan 19, 2012 at 12:44 PM, Marc Auslander wrote: >> Jon Dowland writes: >> >>> >>> Pretty sure at that stage it has loaded modules that let it interpret >>> a selection of filesystem types, in order to fetch grub.cfg (and >>> further >> >> How does it decide which partition (on which disk) and what pathname >> to use to find grub.cfg. I assume one it chooses a partition it can >> look at partitions, figure out what kind they are, and figure out how >> to read them. >> >> Also - if the chosen partition is raid1 does it "assemble" the array >> or just use the chosen partition as if it where a degraded array. >> >> >> -- >> To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org >> with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org >> Archive: http://lists.debian.org/87d3afegeh@aptiva.optonline.net >> -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAK3ER7tAtAbvoNtB2t2¦ALBTxB35O7RkW5mrRC=lwf0be...@mail.gmail.com
Re: issue with mdadm and mirroring drives
In related questions - i hope you can answer - are : 1. why do i get an error when trying to make swap partition - i get this error below after running fdisk, creating a primary partition /dev/sdd2 and then changing the type - with t - As in below --- 2. the /dev/sdd drive is the mirrored drive in the set - do i have to set /dev/sdd1 active in fdisk ??? root@rider:~# mkswap /dev/sdd2 mkswap: error: swap area needs to be at least 40 KiB Usage: mkswap [-c] [-pPAGESZ] [-L label] [-U UUID] /dev/name [blocks] root@rider:~# fdisk -l /dev/sdd Disk /dev/sdd: 1000.2 GB, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x0005d0fd Device Boot Start End Blocks Id System /dev/sdd1 1 121091 972663426 fd Linux raid autodetect /dev/sdd2 121092 121601 4096575 82 Linux swap / Solaris thanks mjh On Thu, Jan 19, 2012 at 12:44 PM, Marc Auslander wrote: > Jon Dowland writes: > >> >> Pretty sure at that stage it has loaded modules that let it interpret >> a selection of filesystem types, in order to fetch grub.cfg (and >> further > > How does it decide which partition (on which disk) and what pathname > to use to find grub.cfg. I assume one it chooses a partition it can > look at partitions, figure out what kind they are, and figure out how > to read them. > > Also - if the chosen partition is raid1 does it "assemble" the array > or just use the chosen partition as if it where a degraded array. > > > -- > To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org > with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org > Archive: http://lists.debian.org/87d3afegeh@aptiva.optonline.net > -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/cak3er7tf4h+bqdbjjfzub2yn5h2ojfjhhpz38ofdap44pfd...@mail.gmail.com
Re: issue with mdadm and mirroring drives
Jon Dowland writes: > > Pretty sure at that stage it has loaded modules that let it interpret > a selection of filesystem types, in order to fetch grub.cfg (and > further How does it decide which partition (on which disk) and what pathname to use to find grub.cfg. I assume one it chooses a partition it can look at partitions, figure out what kind they are, and figure out how to read them. Also - if the chosen partition is raid1 does it "assemble" the array or just use the chosen partition as if it where a degraded array. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87d3afegeh@aptiva.optonline.net
Re: issue with mdadm and mirroring drives
On 18/01/12 16:23, Joey L wrote: When i put the drive sda into the system to add it back to the md0 - the system keeps booting from it and refuses to boot from the good sdb drive. Boot with just the good drive; degrade the array (fail the missing drive); relabel your root partition (whatever is mounted at /) with a filesystem label you have not used before; adjust your fstab and reconfigure your bootloader accordingly. You may need to regenerate your initramfs. Power off, plug both drives back in; upon boot, interrupt your boot loader (there's some chance your system still invoked the one from the bad drive,) and adjust the kernel command line so that root=LABEL=new-label. That above might do the job; if not, you may also need to play in the GRUB CLI to access the correct /boot to load the correct initramfs. -- Jon Dowland -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4f183105.7010...@debian.org
Re: issue with mdadm and mirroring drives
On 19/01/12 02:04, Marc Auslander wrote: Now, does that code contain a copy of grub.cfg? Or does it read it from someplace? If the second, how does it decide where/how to read grub.cfg. I understand how, once it has grub.cfg, it decides what to boot. It's where grub.cfg comes from that I don't understand. Pretty sure at that stage it has loaded modules that let it interpret a selection of filesystem types, in order to fetch grub.cfg (and further grub modules). Therefore you need grub modules for whatever filesystem types (and similar) are needed to get to /boot; historically, that excluded md (and lvm) and so /boot was typically not put inside LVM or md RAID — more recently I believe GRUB can interpret LVM and MD devices. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4f180019.3020...@debian.org
Re: issue with mdadm and mirroring drives
19/01/2012 01:43, Joey L wrote: > Also - is there a way in debian to realize which drive is mapped to > which linux device ?? > When I go into the bios - i see disk labels or names as DSK04, DSK02 > and also long string of numbers - i guess it is the serial number of > drive. For partition UUID: ls -l /dev/disk/by-uuid or blkid /dev/sdd? For disk ID: ls -l /dev/disk/by-id For partitions labels: ls -l /dev/disk/by-label For mdadm devices: mdadm --misc --detail --brief /dev/md?* (as seen in mdadm.conf) -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4f17eff5.90...@googlemail.com
Re: issue with mdadm and mirroring drives
19/01/2012 00:03, Joey L wrote:
> Sorry - just a couple of last things:
>
>
> 1. do i run the command - "grub-install --recheck --no-floppy
> /dev/sdc1" and "grub-install --recheck --no-floppy /dev/sdd1" ???
>
> or do I run it on the whole drive meaning - "grub-install --recheck
> --no-floppy /dev/sdc" and "grub-install --recheck --no-floppy
> /dev/sdd"
Whole drive, like "/dev/sdd", unless you really know what you are doing
and want to install to a partition instead (typically if you have
another os in multiboot that manages the bootloader).
> Is that creating a boot sector ? what is it really doing ? do i need
> to set something active in the boot partition - meaning in fdisk ?
Usually no for Linux, the "boot" flag is for the /boot partition where
the kernel resides.
grub2 (aka grub-pc) support special "bios_grub" partition, but that's
meant for gpt disklabel and is completely irrelevant for you.
Grub is installed in the Master Boot Record ("MBR" for short), and some
components in the free space between the MBR and the start of the first
partition. You don't have to worry about this unless something is going
seriously wrong during the grub installation process.
>
> 2. update-initramfs -u -k all -this command only update /boot dir ?
> and then the mirroing copied to the other drive ??
>
> 3. Big thanks for your help!
>
It update the initial ram disk (aka initrd or initramfs) that normally
resides in /boot.
Once the raid is running you don't have to worry about what's synced
anymore, everything written to the raid array is synced. At the user
level there is only one device (the raid device "md*"), only the kernel
and mdadm have to know about the physical drives.
grub deals with the drives directly, before raid is even started, that's
why you have to use the devices addresses and not the partitions.
--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4f17eee7.7000...@googlemail.com
Re: issue with mdadm and mirroring drives
This discussion opens a question I've been curious about. IIUC, bios choses a boot device and runs the MBR code. Assuming that's GRUB2 MBR code, GRUB2 then loads the 1.5 code hidden before the first partition of the same device. Next step is to process grub.cfg. Now, does that code contain a copy of grub.cfg? Or does it read it from someplace? If the second, how does it decide where/how to read grub.cfg. I understand how, once it has grub.cfg, it decides what to boot. It's where grub.cfg comes from that I don't understand. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87wr8o1m97@aptiva.optonline.net
Re: issue with mdadm and mirroring drives
Also - is there a way in debian to realize which drive is mapped to which linux device ?? When I go into the bios - i see disk labels or names as DSK04, DSK02 and also long string of numbers - i guess it is the serial number of drive. On Wed, Jan 18, 2012 at 6:03 PM, Joey L wrote: > Sorry - just a couple of last things: > > > 1. do i run the command - "grub-install --recheck --no-floppy > /dev/sdc1" and "grub-install --recheck --no-floppy /dev/sdd1" ??? > > or do I run it on the whole drive meaning - "grub-install --recheck > --no-floppy /dev/sdc" and "grub-install --recheck --no-floppy > /dev/sdd" > > Is that creating a boot sector ? what is it really doing ? do i need > to set something active in the boot partition - meaning in fdisk ? > > 2. update-initramfs -u -k all -this command only update /boot dir ? > and then the mirroing copied to the other drive ?? > > 3. Big thanks for your help! -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAK3ER7v04N31yxgqvARBF-9oX+db=cylftkhsuoque7ghgx...@mail.gmail.com
Re: issue with mdadm and mirroring drives
Sorry - just a couple of last things: 1. do i run the command - "grub-install --recheck --no-floppy /dev/sdc1" and "grub-install --recheck --no-floppy /dev/sdd1" ??? or do I run it on the whole drive meaning - "grub-install --recheck --no-floppy /dev/sdc" and "grub-install --recheck --no-floppy /dev/sdd" Is that creating a boot sector ? what is it really doing ? do i need to set something active in the boot partition - meaning in fdisk ? 2. update-initramfs -u -k all -this command only update /boot dir ? and then the mirroing copied to the other drive ?? 3. Big thanks for your help! -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/cak3er7vohjj3w3xpzuc_qf1fw519qgvzarfkc6muhebv_po...@mail.gmail.com
Re: issue with mdadm and mirroring drives
18/01/2012 22:22, Joey L wrote: > okay..thanks for the clarification. > I think it was initrd - busybox. > > my debian version is: > root@rider:~# lsb_release -a > No LSB modules are available. > Distributor ID: Debian > Description:Debian GNU/Linux 6.0.3 (squeeze) > Release:6.0.3 > Codename: squeeze > > uname -mrs > Linux 2.6.32-5-amd64 x86_64 > > > I fooled around with the bois and I was able to both drives to be seen > and the good partition running and the drive is syncing. > Not really sure why it was behaving this way - but thanks to plug and > play - the running system seen the bad drive and i was able to sync > it. > > Now, my question to you is: > 1. what do I do next ? > do i run the command - "grub-install --recheck --no-floppy /dev/sdc1" > (sdc1 is my good drive) If the system is booting normally now, it's not mandatory. But it wouldn't hurt to make sure grub is installed on all drives in the array. > 2. Do i do this command after a reboot or just after the sync is done ? Doesn't matter, I would say let it sync first out of pity for that poor drive ;-) . > 3. I get this error regarding a Floppy Error - I have no floppy > drives - how can i remove this ? Yes you can, remove the "--no-floppy" option, it's a precautionary measure. I have the habit to add it when I don't know what the hardware is, it prevents nasty grub mistakes when a floppy drive is present, and doesn't hurt if there isn't any. Floppy is something of the past, I have to get used to it. > 4. Do i have to update any of the mdadm files ? or any additional grub > files ? before or after I reboot ? Doesn't hurt do refresh your /etc/mdadm/mdadm.conf , compare it to the output of: mdadm --misc --detail --brief /dev/md?* If you change mdadm.conf, rebuild your initrd: update-initramfs -u -k all Nothing to do for grub aside from making sure it's installed on all drives in the array. > before or after I reboot ? It's always better to make sure everything is in order before rebooting, after it is sometime too late... > thanks for your help! > mjh > You helped yourself on this one, happy you managed to get the system back. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4f173e01.4030...@googlemail.com
Re: issue with mdadm and mirroring drives
okay..thanks for the clarification.
I think it was initrd - busybox.
my debian version is:
root@rider:~# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:Debian GNU/Linux 6.0.3 (squeeze)
Release:6.0.3
Codename: squeeze
uname -mrs
Linux 2.6.32-5-amd64 x86_64
I fooled around with the bois and I was able to both drives to be seen
and the good partition running and the drive is syncing.
Not really sure why it was behaving this way - but thanks to plug and
play - the running system seen the bad drive and i was able to sync
it.
Now, my question to you is:
1. what do I do next ?
do i run the command - "grub-install --recheck --no-floppy /dev/sdc1"
(sdc1 is my good drive)
2. Do i do this command after a reboot or just after the sync is done ?
3. I get this error regarding a Floppy Error - I have no floppy
drives - how can i remove this ?
4. Do i have to update any of the mdadm files ? or any additional grub
files ? before or after I reboot ?
thanks for your help!
mjh
On Wed, Jan 18, 2012 at 2:07 PM, tv.deb...@googlemail.com
wrote:
> 18/01/2012 19:38, Joey L wrote:
>> On Wed, Jan 18, 2012 at 12:49 PM, tv.deb...@googlemail.com
>> wrote:
>>> > 18/01/2012 18:03, Joey L wrote:
>> The issue I am having is that if I put into the system both drives,
>> the system always chooses the faulty drive.
>> I do not even get linux system - i get a weird text prompt - i think
>> it is initrdfs - even if i change it in the bios.
>>
>> On installing grub - can you tell me what is the procedure for that
>> after i get my drives to an okay state ?
>>
>> thanks
>> mjh
>
> initrd is a later stage than grub menu, grub comes second after bios,
> and is in charge of loading the initrd. So if you get to the initrd
> "busybox" shell it means you got past the grub menu. Otherwise what you
> think is initrd really is either bios or grub.
>
> Maybe it's time you give us some details about your setup, what kind of
> computer is this, and what flavor of Debian are you running on it (grub
> version would be nice too) ?
> How exactly is the boot sequence occurring, what do you "see" and in
> what order ?
> How do you know it "chooses" the faulty drive ?
>
> No matter which drive the system boots on, it will fail to bring up the
> raid array your root partition resides on if it's flagged as degraded,
> unless you pass the necessary option I gave you earlier. It is possible
> to assemble the array from the busybox ("initrd") shell and resume the
> init ("boot") process, but if we don't have more details, or if you
> don't tell us exactly what you try and the result (error messages), we
> can keep shooting in the dark for a long time.
>
> If you can't get it to boot by itself it may be easier to boot from a
> recovery live-cd, any will do as long as it has mdadm installed or
> installable. From there it will be possible to start the degraded raid
> array, make sure it works, "chroot" on it (means "transferring" the
> live-cd shell to the target system), and then issue commands as if they
> were issued from the target system. Or for a start you could simply
> assemble the array, start it and mount the target system's partition,
> and edit the grub menu entry to add the necessary option.
>
>
> --
> To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
> Archive: http://lists.debian.org/4f17186a.3000...@googlemail.com
>
--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive:
http://lists.debian.org/CAK3ER7sVaYQ5ghCGO7nJL2fT6p8Yj4V03w=nk7lafhqxsjy...@mail.gmail.com
Re: issue with mdadm and mirroring drives
On Wed, 2012-01-18 at 13:38 -0500, Joey L wrote: > thanks for the reply - > > 1. my issue is that i am not getting to the grub menu on the good > disk, it always goes to the bad one. > I think it puts me in a intrdfsram prompt or something similar - even > if i change the bios settings. > - so editing the grub menu option - is unavailable for me. > > 2. regarding the procedure after grub - you wrote: > > grub-install --recheck --no-floppy /dev/sda > > repeat for "/dev/sdb", or use "hd0" and "hd1" which is grub traditional > notation for first and second drives. > > my drives are /dev/sda and my boot partition is /dev/sda1 > > what do i type in that scenerio after i am able to create the mirror > back to the way it was. > > thanks > mjh That's strange that your BIOS won't let you select the primary drive. I suppose it is possible that the drives are jumpered to master and slave. One could also try physically reversing the drives to see if perhaps the bus scan order is the determining factor. I'm just guessing as I am certainly not an expert in this area - John -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1326913929.2936.71.ca...@denise.theartistscloset.com
Re: issue with mdadm and mirroring drives
18/01/2012 19:38, Joey L wrote:
> On Wed, Jan 18, 2012 at 12:49 PM, tv.deb...@googlemail.com
> wrote:
>> > 18/01/2012 18:03, Joey L wrote:
>>> >> The issue I am having is that if I put into the system both drives,
>>> >> the system always chooses the faulty drive.
>>> >> I do not even get linux system - i get a weird text prompt - i think
>>> >> it is initrdfs - even if i change it in the bios.
>>> >>
>>> >> On installing grub - can you tell me what is the procedure for that
>>> >> after i get my drives to an okay state ?
>>> >>
>>> >> thanks
>>> >> mjh
initrd is a later stage than grub menu, grub comes second after bios,
and is in charge of loading the initrd. So if you get to the initrd
"busybox" shell it means you got past the grub menu. Otherwise what you
think is initrd really is either bios or grub.
Maybe it's time you give us some details about your setup, what kind of
computer is this, and what flavor of Debian are you running on it (grub
version would be nice too) ?
How exactly is the boot sequence occurring, what do you "see" and in
what order ?
How do you know it "chooses" the faulty drive ?
No matter which drive the system boots on, it will fail to bring up the
raid array your root partition resides on if it's flagged as degraded,
unless you pass the necessary option I gave you earlier. It is possible
to assemble the array from the busybox ("initrd") shell and resume the
init ("boot") process, but if we don't have more details, or if you
don't tell us exactly what you try and the result (error messages), we
can keep shooting in the dark for a long time.
If you can't get it to boot by itself it may be easier to boot from a
recovery live-cd, any will do as long as it has mdadm installed or
installable. From there it will be possible to start the degraded raid
array, make sure it works, "chroot" on it (means "transferring" the
live-cd shell to the target system), and then issue commands as if they
were issued from the target system. Or for a start you could simply
assemble the array, start it and mount the target system's partition,
and edit the grub menu entry to add the necessary option.
--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4f17186a.3000...@googlemail.com
Re: issue with mdadm and mirroring drives
thanks for the reply - 1. my issue is that i am not getting to the grub menu on the good disk, it always goes to the bad one. I think it puts me in a intrdfsram prompt or something similar - even if i change the bios settings. - so editing the grub menu option - is unavailable for me. 2. regarding the procedure after grub - you wrote: grub-install --recheck --no-floppy /dev/sda repeat for "/dev/sdb", or use "hd0" and "hd1" which is grub traditional notation for first and second drives. my drives are /dev/sda and my boot partition is /dev/sda1 what do i type in that scenerio after i am able to create the mirror back to the way it was. thanks mjh On Wed, Jan 18, 2012 at 12:49 PM, tv.deb...@googlemail.com wrote: > 18/01/2012 18:03, Joey L wrote: >> The issue I am having is that if I put into the system both drives, >> the system always chooses the faulty drive. >> I do not even get linux system - i get a weird text prompt - i think >> it is initrdfs - even if i change it in the bios. >> >> On installing grub - can you tell me what is the procedure for that >> after i get my drives to an okay state ? >> >> thanks >> mjh > > To get the system to start on a degraded raid array you can try passing > the following option on the "linux" line of the grub menu: > > md-mod.start_dirty_degraded=1 > > To edit the grub menu entry type the letter "e", then make the changes > by adding the option at the end of the line that starts with "linux". > To boot use "ctrl" + "x" (keys "control" and "x" et the same time). > > Once you are in the system, issue (as root) the following commands > assuming that "md0" is the array, and "sdb1" the faulty drive partition: > > mdadm /dev/md0 --fail /dev/sdb1 > > mdadm /dev/md0 --remove /dev/sdb1 > > mdadm /dev/md0 --zero-superblock /dev/sdb1 > > mdadm /dev/md0 --add /dev/sdb1 > > "fail" and "remove" can be issued in the same command, but you can also > use them as shown above for clarity's sake. > > > Regarding grub, just use the usual magic formula (as root): > > grub-install --recheck --no-floppy /dev/sda > > repeat for "/dev/sdb", or use "hd0" and "hd1" which is grub traditional > notation for first and second drives. > > > -- > To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org > with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org > Archive: http://lists.debian.org/4f170619.5070...@googlemail.com > -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/cak3er7t40um+w448xogt8brp4o41crf6saihb67frdd6igq...@mail.gmail.com
Re: issue with mdadm and mirroring drives
18/01/2012 18:03, Joey L wrote: > The issue I am having is that if I put into the system both drives, > the system always chooses the faulty drive. > I do not even get linux system - i get a weird text prompt - i think > it is initrdfs - even if i change it in the bios. > > On installing grub - can you tell me what is the procedure for that > after i get my drives to an okay state ? > > thanks > mjh To get the system to start on a degraded raid array you can try passing the following option on the "linux" line of the grub menu: md-mod.start_dirty_degraded=1 To edit the grub menu entry type the letter "e", then make the changes by adding the option at the end of the line that starts with "linux". To boot use "ctrl" + "x" (keys "control" and "x" et the same time). Once you are in the system, issue (as root) the following commands assuming that "md0" is the array, and "sdb1" the faulty drive partition: mdadm /dev/md0 --fail /dev/sdb1 mdadm /dev/md0 --remove /dev/sdb1 mdadm /dev/md0 --zero-superblock /dev/sdb1 mdadm /dev/md0 --add /dev/sdb1 "fail" and "remove" can be issued in the same command, but you can also use them as shown above for clarity's sake. Regarding grub, just use the usual magic formula (as root): grub-install --recheck --no-floppy /dev/sda repeat for "/dev/sdb", or use "hd0" and "hd1" which is grub traditional notation for first and second drives. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4f170619.5070...@googlemail.com
Re: issue with mdadm and mirroring drives
The issue I am having is that if I put into the system both drives, the system always chooses the faulty drive. I do not even get linux system - i get a weird text prompt - i think it is initrdfs - even if i change it in the bios. On installing grub - can you tell me what is the procedure for that after i get my drives to an okay state ? thanks mjh On Wed, Jan 18, 2012 at 11:39 AM, tv.deb...@googlemail.com wrote: > 18/01/2012 17:23, Joey L wrote: >> I have a raid 1 mdadm setup. >> I have devices sda and sdb with 2 partitions on each - sda1 and sda2 >> on the other drive i have sdb1 and sdb2. >> >> Partition sda1 is the root partition and sda2 is the swap partition. >> >> My sda failed yesterday and now i am running in sdb only. >> >> When i put the drive sda into the system to add it back to the md0 - >> the system keeps booting from it and refuses to boot from the good sdb >> drive. >> >> What can i do to stop this from happening - I have no other system to >> plug this drive into to zero out the drive. >> >> Also if I am able to get over this issue - do i have to do other >> procedures to this drive? what are they ??? I think i have to do the >> following: >> >> mdadm --manage /dev/md0 --remove /dev/sdb1 >> >> mdadm --manage /dev/md0 --fail /dev/sdb1 >> >> sfdisk -d /dev/sda | sfdisk /dev/sdb >> >> mdadm --manage /dev/md0 --add /dev/sdb1 >> >> >> I am thinking that I need to do something with grub or do some kind of >> update so the server can boot of the good drive that I have working >> now - but do not know what it is. >> >> thanks >> mjh >> >> > > Hi, when you manage to boot with the faulty drive in, just "--fail", > "--remove" and "--zero-superblock", then "--add" it again. Off course if > the drive is damaged or faulty at a low level it may not come up or > won't survive the rebuild... > > If you can work around the boot issue, maybe you can physically change > the way the disk "sdb" is plugged-in, and use the slot previously used > by "sda". This can usually be achieved in bios too by changing in the > disk ordering. > > As an alternate, you could try booting on a rescue live-cd first, then > start your degraded raid, chroot to the system to get the faulty drive > back in the raid and start the rebuild, even reinstall grub from the > chroot, then reboot in the original system. > > Once you are done it's a good idea to reinstall grub on both drives. > > > -- > To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org > with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org > Archive: http://lists.debian.org/4f16f5cd.7060...@googlemail.com > -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAK3ER7t=XTODrfY00Dv9T=vhtqo_f8wbe+cv3ut+6s-adji...@mail.gmail.com
Re: issue with mdadm and mirroring drives
18/01/2012 17:23, Joey L wrote: > I have a raid 1 mdadm setup. > I have devices sda and sdb with 2 partitions on each - sda1 and sda2 > on the other drive i have sdb1 and sdb2. > > Partition sda1 is the root partition and sda2 is the swap partition. > > My sda failed yesterday and now i am running in sdb only. > > When i put the drive sda into the system to add it back to the md0 - > the system keeps booting from it and refuses to boot from the good sdb > drive. > > What can i do to stop this from happening - I have no other system to > plug this drive into to zero out the drive. > > Also if I am able to get over this issue - do i have to do other > procedures to this drive? what are they ??? I think i have to do the > following: > > mdadm --manage /dev/md0 --remove /dev/sdb1 > > mdadm --manage /dev/md0 --fail /dev/sdb1 > > sfdisk -d /dev/sda | sfdisk /dev/sdb > > mdadm --manage /dev/md0 --add /dev/sdb1 > > > I am thinking that I need to do something with grub or do some kind of > update so the server can boot of the good drive that I have working > now - but do not know what it is. > > thanks > mjh > > Hi, when you manage to boot with the faulty drive in, just "--fail", "--remove" and "--zero-superblock", then "--add" it again. Off course if the drive is damaged or faulty at a low level it may not come up or won't survive the rebuild... If you can work around the boot issue, maybe you can physically change the way the disk "sdb" is plugged-in, and use the slot previously used by "sda". This can usually be achieved in bios too by changing in the disk ordering. As an alternate, you could try booting on a rescue live-cd first, then start your degraded raid, chroot to the system to get the faulty drive back in the raid and start the rebuild, even reinstall grub from the chroot, then reboot in the original system. Once you are done it's a good idea to reinstall grub on both drives. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4f16f5cd.7060...@googlemail.com
issue with mdadm and mirroring drives
I have a raid 1 mdadm setup. I have devices sda and sdb with 2 partitions on each - sda1 and sda2 on the other drive i have sdb1 and sdb2. Partition sda1 is the root partition and sda2 is the swap partition. My sda failed yesterday and now i am running in sdb only. When i put the drive sda into the system to add it back to the md0 - the system keeps booting from it and refuses to boot from the good sdb drive. What can i do to stop this from happening - I have no other system to plug this drive into to zero out the drive. Also if I am able to get over this issue - do i have to do other procedures to this drive? what are they ??? I think i have to do the following: mdadm --manage /dev/md0 --remove /dev/sdb1 mdadm --manage /dev/md0 --fail /dev/sdb1 sfdisk -d /dev/sda | sfdisk /dev/sdb mdadm --manage /dev/md0 --add /dev/sdb1 I am thinking that I need to do something with grub or do some kind of update so the server can boot of the good drive that I have working now - but do not know what it is. thanks mjh -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAK3ER7s=z-p-yfkntxibmdo24kjndwn2styd9fs61byhfan...@mail.gmail.com
Re: Automatic partial archive mirroring
On Mon, May 31, 2010 at 01:40:59AM -, Cameron Hutchison wrote: > I have a few debian unstable boxes that I like to keep up-to-date. > > Currently, I run apt-cacher-ng (a proxy for apt-get which stores > packages so they don't need to be downloaded again) on my gateway box > (lenny 32-bit), so I don't double download from my couple of other boxes > (sid 64-bit). This works well enough, but I want to make it better. > > My internet connection has an off-peak period (2am-8am) where downloads > are not counted in the monthly quota. I want my downloads to occur > during this window, automatically. > > The problem is that the sid boxes are not powered-up during this window > so I cannot just simply schedule a cronjob to do an apt-get -d > dist-upgrade. > > I would like my lenny box to do this on behalf of my sid boxes. I dont > really want to mirror the whole archive. Is there some way to do an > intelligent partial mirror of the archive of what is installed on other > boxes, without too much overhead of managing a package list? (i.e. I dont > want to have to manually update some package list on my lenny box when I > install a new package on a sid box). > > I've looked at apt-proxy, apt-cacher, apt-cacher-ng and approx, but they > all appear to download on demand, not according to a schedule. Take a look at debmirror's options. You can choose to download by dist, section, protocol, arch, source/no source, include/exclude packages by regexp matching, etc... It can be easily scheduled and you can serve the packages afterwards adding a deb http://lighttpd.at.lenny.gw sid main contrib to the other boxes' /etc/apt/sources.list.d/lenny.gw.list, for example. Would that be useful for you? Obvious drawback: you'll possibly be pulling more than you need. -- Huella de clave primaria: 0FDA C36F F110 54F4 D42B D0EB 617D 396C 448B 31EB pgpEJLQO0ldqB.pgp Description: PGP signature
Re: Automatic partial archive mirroring
On Mon,31.May.10, 01:40:59, Cameron Hutchison wrote: > > I've looked at apt-proxy, apt-cacher, apt-cacher-ng and approx, but they > all appear to download on demand, not according to a schedule. Maybe apt-zip or apt-offline (not in lenny) can be used for what you need. Regards, Andrei -- Offtopic discussions among Debian users and developers: http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic signature.asc Description: Digital signature
Re: Automatic partial archive mirroring
Celejar writes: >Cameron Hutchison wrote: >> >and then have the gateway box get those packages. >> >> hmmm. How? Is there an existing tool that will do this? Doing it >> manually (in a script) would require too much work (essentially >> implementing apt-get -d dist-upgrade against a specified package index >> and package list). >How about using 'aptitude download', possibly feeding the package list >to aptitude with xargs? The 'download' action downloads to the current >directory, so I suppose that you'd use a temp directory, and that would >have the side effect of getting a copy stored by the cacher. The problem is the the dpkg architecture (and distribution) of the host is different to the targets. 'aptitude download' is going to use the native dpkg architecture, and is going to look in /etc/apt/sources.list for the sources to use. It looks like it may be possible to use something like 'aptitude extract-cache-subset' on the targets and set Apt::Architecture to the target on the command line. apt.conf(5) has a number of details of how I can override most of the host environment. Hmmm, I can see how this might work now. * periodically run aptitude extract-cache-subset on the targets and send that to the host * every night on the host, run 'aptitude -d dist-upgrade', but overriding all the directories for configs, indexes, pkgstates, etc and architecture to point to what has been captured by extract-cache-subset. Also point aptitude to the local apt-cacher-ng proxy so all downloads go through it. * delete the downloaded files, since apt-cacher-ng has already cached them. Thanks for the ideas. Perhaps I can knock this up fairly easily. I'll certainly learn more about apt than I probably want to :-) -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/eb9.4c0344bd.22...@getafix.xdna.net
Re: Automatic partial archive mirroring
On Mon, 31 May 2010 04:09:19 - Cameron Hutchison wrote: > Celejar writes: > >Cameron Hutchison wrote: > > >> Is there some way to do an > >> intelligent partial mirror of the archive of what is installed on other > >> boxes, without too much overhead of managing a package list? (i.e. I dont > >> want to have to manually update some package list on my lenny box when I > >> install a new package on a sid box). > > >You could use use the output of 'dpkg --get-selections' on the Sid > >boxes, pruning it using grep or similar to keep only the lines ending > >with 'install'. > > ok. that's easy. > > >Combine the results from the various Sid boxes, > > so's that. > > >and then have the gateway box get those packages. > > hmmm. How? Is there an existing tool that will do this? Doing it > manually (in a script) would require too much work (essentially > implementing apt-get -d dist-upgrade against a specified package index > and package list). How about using 'aptitude download', possibly feeding the package list to aptitude with xargs? The 'download' action downloads to the current directory, so I suppose that you'd use a temp directory, and that would have the side effect of getting a copy stored by the cacher. Celejar -- foffl.sourceforge.net - Feeds OFFLine, an offline RSS/Atom aggregator mailmin.sourceforge.net - remote access via secure (OpenPGP) email ssuds.sourceforge.net - A Simple Sudoku Solver and Generator -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100531003508.36f3ddde.cele...@gmail.com
Re: Automatic partial archive mirroring
Celejar writes: >Cameron Hutchison wrote: >> Is there some way to do an >> intelligent partial mirror of the archive of what is installed on other >> boxes, without too much overhead of managing a package list? (i.e. I dont >> want to have to manually update some package list on my lenny box when I >> install a new package on a sid box). >You could use use the output of 'dpkg --get-selections' on the Sid >boxes, pruning it using grep or similar to keep only the lines ending >with 'install'. ok. that's easy. >Combine the results from the various Sid boxes, so's that. >and then have the gateway box get those packages. hmmm. How? Is there an existing tool that will do this? Doing it manually (in a script) would require too much work (essentially implementing apt-get -d dist-upgrade against a specified package index and package list). -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/c07.4c03366f.8a...@getafix.xdna.net
Re: Automatic partial archive mirroring
On Mon, 31 May 2010 01:40:59 - Cameron Hutchison wrote: > I have a few debian unstable boxes that I like to keep up-to-date. > > Currently, I run apt-cacher-ng (a proxy for apt-get which stores > packages so they don't need to be downloaded again) on my gateway box > (lenny 32-bit), so I don't double download from my couple of other boxes > (sid 64-bit). This works well enough, but I want to make it better. > > My internet connection has an off-peak period (2am-8am) where downloads > are not counted in the monthly quota. I want my downloads to occur > during this window, automatically. > > The problem is that the sid boxes are not powered-up during this window > so I cannot just simply schedule a cronjob to do an apt-get -d > dist-upgrade. > > I would like my lenny box to do this on behalf of my sid boxes. I dont > really want to mirror the whole archive. Is there some way to do an > intelligent partial mirror of the archive of what is installed on other > boxes, without too much overhead of managing a package list? (i.e. I dont > want to have to manually update some package list on my lenny box when I > install a new package on a sid box). > > I've looked at apt-proxy, apt-cacher, apt-cacher-ng and approx, but they > all appear to download on demand, not according to a schedule. You could use use the output of 'dpkg --get-selections' on the Sid boxes, pruning it using grep or similar to keep only the lines ending with 'install'. Combine the results from the various Sid boxes, and then have the gateway box get those packages. This could be scripted easily enough, and there would be no need for manually updating anything. Celejar -- foffl.sourceforge.net - Feeds OFFLine, an offline RSS/Atom aggregator mailmin.sourceforge.net - remote access via secure (OpenPGP) email ssuds.sourceforge.net - A Simple Sudoku Solver and Generator -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100530232849.13be6e74.cele...@gmail.com
Automatic partial archive mirroring
I have a few debian unstable boxes that I like to keep up-to-date. Currently, I run apt-cacher-ng (a proxy for apt-get which stores packages so they don't need to be downloaded again) on my gateway box (lenny 32-bit), so I don't double download from my couple of other boxes (sid 64-bit). This works well enough, but I want to make it better. My internet connection has an off-peak period (2am-8am) where downloads are not counted in the monthly quota. I want my downloads to occur during this window, automatically. The problem is that the sid boxes are not powered-up during this window so I cannot just simply schedule a cronjob to do an apt-get -d dist-upgrade. I would like my lenny box to do this on behalf of my sid boxes. I dont really want to mirror the whole archive. Is there some way to do an intelligent partial mirror of the archive of what is installed on other boxes, without too much overhead of managing a package list? (i.e. I dont want to have to manually update some package list on my lenny box when I install a new package on a sid box). I've looked at apt-proxy, apt-cacher, apt-cacher-ng and approx, but they all appear to download on demand, not according to a schedule. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/85e.4c0313ab.76...@getafix.xdna.net
Re: network data mirroring
2009/6/9 michal krajcirovic : > hello, > solves a simple problem complicated solution:) > > I have two servers, one on the disk any data that need to the second. You could try drbd Regards, -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
network data mirroring
hello, solves a simple problem complicated solution:) I have two servers, one on the disk any data that need to the second. Currently exports to the disc via NFS, however, load the disk is too big and the whole is slow. Then I need in two places at the same time maintain the same data. Among other reasons, can not be used glusterfs. has anybody experience with a very easy solution? -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Re: Local mirroring How-To?
Dennis Wicks wrote: Greetings; I remember there was a discussion sometime back about this but I can't locate it. Here is my situation. I have six Debian/Lenny machines that I have to update/upgrade at various times, and some of these runs take hours for each system over my 768Kbs connection. I have the space to install a local mirror of one or two releases of Debian so I can do updates over my local net at 100 Mbs and speed things up considerably. But ... I can't find anything current about how to do that. Does anybody have any pointers to a good, and current, set of directions on how to do this? Or have a cookbook that works for you? Many TIA! Dennis Dennis: You can user debmirror. It works very well. You can have several mirrors (stable, testing, etc.,) just put then in different directories. You can use the following command to mirror testing. Just modify the destination directory. You may want to modify the source mirror to one that is closer to you. debmirror --progress --passive --verbose --nosource --method=rsync host=ftp.at.debian.org --root=:debian --dist=testing --arch=i386 --ignore-release-gpg --ignore-small-errors --timeout=900 --pdiff=mirror /home/testing-mirror/ See the debmirror man page or /usr/share/doc/debmirror for more details. Steven. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Re: Local mirroring How-To?
> Here is my situation. I have six Debian/Lenny machines that I have to > update/upgrade at various times, and some of these runs take hours for each > system over my 768Kbs connection. Same here. For various reasons, I found apt-cacher and friends unsuitable for my needs (can't remember the reasons), so I simply use an http_proxy (polipo in my case). Works dandy, and I can also use it for anything else that crosses my mind. Stefan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Re: Local mirroring How-To?
On Mar 12, 12:40 pm, "Stackpole, Chris" wrote: > > From: Dennis Wicks [mailto:w...@mgssub.com] > > Sent: Thursday, March 12, 2009 11:22 AM > > Subject: Local mirroring How-To? > > > Greetings; > > > I remember there was a discussion sometime back about this > > but I can't locate it. > > > Here is my situation. I have six Debian/Lenny machines that > > I have to update/upgrade at various times, and some of these > > runs take hours for each system over my 768Kbs connection. > > > I have the space to install a local mirror of one or two > > releases of Debian so I can do updates over my local net at > > 100 Mbs and speed things up considerably. > > > But ... I can't find anything current about how to do that. > > > Does anybody have any pointers to a good, and current, set > > of directions on how to do this? Or have a cookbook that > > works for you? > > You probably don't want a full mirror. They are big (last doc I saw said > 60GB per architecture/version). You want something that will only > provide the packages you need/use. > I use the debmirror package to keep a local mirror of Lenny and Testing. (main, contrib and non-free). It "only" takes up about 28GB Stuart -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
RE: Local mirroring How-To?
> From: Dennis Wicks [mailto:w...@mgssub.com] > Sent: Thursday, March 12, 2009 11:22 AM > Subject: Local mirroring How-To? > > Greetings; > > I remember there was a discussion sometime back about this > but I can't locate it. > > Here is my situation. I have six Debian/Lenny machines that > I have to update/upgrade at various times, and some of these > runs take hours for each system over my 768Kbs connection. > > I have the space to install a local mirror of one or two > releases of Debian so I can do updates over my local net at > 100 Mbs and speed things up considerably. > > But ... I can't find anything current about how to do that. > > Does anybody have any pointers to a good, and current, set > of directions on how to do this? Or have a cookbook that > works for you? You probably don't want a full mirror. They are big (last doc I saw said 60GB per architecture/version). You want something that will only provide the packages you need/use. I prefer apt-cacher to do this. Try this link [1] to get you started. Yes, the guide is almost 2 years old, but it still works. There is also apt-cacher-ng and apt-proxy. They all do the same thing; they cache packages that run through them. If you search the list long enough, you will find people who recommend each of these. I prefer apt-cacher simply because it was easy to install, configure, and I have not had problems with it. The other plus for me is that I run Etch, Lenny, and Squeeze installs with a large mix of 32bit and 64bit systems plus 3 releases of Ubuntu which also have 32 and 64 bit installs. I only have one package cacher that supports all of them. My entire cache is at 9GB. Hope this helps. Have fun! ~Stack~ [1] http://www.debuntu.org/how-to-set-up-a-repository-cache-with-apt-cacher -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Re: Local mirroring How-To?
Dennis Wicks wrote: Greetings; I remember there was a discussion sometime back about this but I can't locate it. Here is my situation. I have six Debian/Lenny machines that I have to update/upgrade at various times, and some of these runs take hours for each system over my 768Kbs connection. I have the space to install a local mirror of one or two releases of Debian so I can do updates over my local net at 100 Mbs and speed things up considerably. But ... I can't find anything current about how to do that. Does anybody have any pointers to a good, and current, set of directions on how to do this? Or have a cookbook that works for you? Many TIA! Dennis its pretty simple to set up your own mirror i kept some notes on the link below on how to set up a full mirror http://www.songshu.org/index.php/setup-a-debian-apt-mirror alternatively this is actually what i use on my LAN, it caches a .deb file when you download it once so it saves a lot of disk space compared to the full mirror. http://www.songshu.org/index.php/apt-cacher -- www.songshu.org Just another collection of nuts -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Local mirroring How-To?
Greetings; I remember there was a discussion sometime back about this but I can't locate it. Here is my situation. I have six Debian/Lenny machines that I have to update/upgrade at various times, and some of these runs take hours for each system over my 768Kbs connection. I have the space to install a local mirror of one or two releases of Debian so I can do updates over my local net at 100 Mbs and speed things up considerably. But ... I can't find anything current about how to do that. Does anybody have any pointers to a good, and current, set of directions on how to do this? Or have a cookbook that works for you? Many TIA! Dennis -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Mirroring existing LVM set: software RAID or LVM mirror?
I'm going to buy a second disk to add to my system. I want to mirror my current LVM setup for redundancy. So far, on older systems, I've always used software RAID 1 or 5 arrays with LVM on top. I've never had any problems with it, mainly because I've been lucky and have never had disk failures. Currently, I have no software RAID on my system yet. Just a small /boot partition, and a huge LVM partition for everything else. I recently discovered I could use LVM's mirroring capabilities. This would probably be easiest and fastest for me, since I wouldn't have to try shove a software RAID array beneath my existing PV's. On the other hand, software RAID with LVM on top seems to be the tried and trusted Linux way. I haven't even found a comparison between the two yet. Please share your LVM mirroring experiences. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: For Lenny: LVM, LVM+MD or just MD for mirroring?
On 6 Sep., 00:40, "nate" <[EMAIL PROTECTED]> wrote: > Jens wrote: > > Just out of curiosity, what exactly did you do? > > I'll have to dig up the configs, will send a reply when I find them.. Thanks! > > OK, I do this with quotas mainly. I haven't yet come across a scenario > > where these were not sufficient any more. (Except maybe for /tmp and / > > var which have their own partitions). > > Well this is for the file system itself rather than users on the system. > e.g. allocate 1TB volume to file system thinly provisioned > format volume > (size taken on array ~20MB) > Write 100GB of data to volume > (size taken on array 100GB) > delete 100GB of data from volume > (size taken on array 100GB) But, unless you need dozens of different file systems which need to have lots of free space allocated for some reason, which doesn't really exist, you can achieve the same thing with quotas, right? > The LVM restrictions is mainly to compensate for the file systems > inherit inefficiency in not re-using existing blocks that were > freed before allocating new blocks(sometimes it does but it doesn't > do a perfect job at it). ??? I don't get it - if I delete a file, the space it used up _is_ freed, right? > It's a fairly new technology not many storage systems implement it > yet, though it's a wonderful way to grow on demand. At my > last company I was able to provision 400% more storage to > the servers than I actually had capacity. As we got closer to > the limit of the installed storage system we added more space > and re-balanced the array for maximum performance, no downtime, > no impact. This is what LVM2 does, right? Allowing to grow partitions / volumes without being restricted to physical volume sizes, by just adding new space. Saves you having to think years ahead and plan for storage that isn't going to be used until much later. Or having to add disks later on and ending up in symlink hell. > I plan to get an evaluation storage system in next month at > my company that implements the next generation of thin > provisioning, which will allow the storage array to > automatically reclaim space in the file system if it is > zeroed out. May take a year or two for the software vendors > to catch up but when they do I'll be ready for even more > storage efficiency ! I love technology. What exactly is the difference in "the array reclaiming free space" and "the file system reclaiming free space" (what it always does)? I haven't -yet- come across a FS that didn't free exactly the number of blocks that it previously allocated, when you deleted files. So the way this looks to me right now is a hugely overcomplicated feature that already exists in a much simpler way. But I'm sure I misunderstand something. Jens -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: For Lenny: LVM, LVM+MD or just MD for mirroring?
Jens wrote: > Just out of curiosity, what exactly did you do? -- snip from kickstart -- # manually isntall grub cat >/tmp/grub.txt <&1 | tee /root/grub-install.txt -- end snip -- Now that I thought about it I never did have a disk failure so I didn't actually test it.. BUT without doing that usually the system would not boot after installation(would hang where grub would normally pop up). The hang only occurred when installing onto software raid. nate -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: For Lenny: LVM, LVM+MD or just MD for mirroring?
Jens wrote: > Just out of curiosity, what exactly did you do? I'll have to dig up the configs, will send a reply when I find them.. > That was an advantage, true. > Nowadays you can mount using UUIDs or disk labels which also works > fine. Yeah that's true.. > OK, I do this with quotas mainly. I haven't yet come across a scenario > where these were not sufficient any more. (Except maybe for /tmp and / > var which have their own partitions). Well this is for the file system itself rather than users on the system. e.g. allocate 1TB volume to file system thinly provisioned format volume (size taken on array ~20MB) Write 100GB of data to volume (size taken on array 100GB) delete 100GB of data from volume (size taken on array 100GB) The LVM restrictions is mainly to compensate for the file systems inherit inefficiency in not re-using existing blocks that were freed before allocating new blocks(sometimes it does but it doesn't do a perfect job at it). It's a fairly new technology not many storage systems implement it yet, though it's a wonderful way to grow on demand. At my last company I was able to provision 400% more storage to the servers than I actually had capacity. As we got closer to the limit of the installed storage system we added more space and re-balanced the array for maximum performance, no downtime, no impact. I plan to get an evaluation storage system in next month at my company that implements the next generation of thin provisioning, which will allow the storage array to automatically reclaim space in the file system if it is zeroed out. May take a year or two for the software vendors to catch up but when they do I'll be ready for even more storage efficiency ! I love technology. nate nate -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: For Lenny: LVM, LVM+MD or just MD for mirroring?
On Fri, Sep 05, 2008 at 12:59:01PM -0700, Jens wrote:
> On 5 Sep., 21:20, "Brian Schrock" <[EMAIL PROTECTED]> wrote:
>
> > We use LVM and software raid via md on every server. We use md to run raid
> > 10 arrays and then we use LVM on top of that. We use snapshots to backup our
>
> Hello Brian,
>
> sounds good - what made you stack MD and LVM, and not use LVM's mirror
> feature?
> Any specific reason?
I do the same as brian, md base and lvm on top, my typical setup is
DiskA Partition 1 - 500M (sda1)
DiskA P2 - 10G
DiskA P3 - The rest
DiskB Partition 1 - 500M (sdb1)
DiskB P2 - 10G
DiskB P3 - The rest
. (for the number of disks)
DiskX Partition 1 - 500M (sdx)
DiskX P2 - 10G
DiskX P3 - The rest
I use
sda1 & sdb1 in a raid1 for /boot (ext2), an old habit, raid1 is the
hardest to [EMAIL PROTECTED] up and it it just hold /boot even harder, I leave
enough
room to place rescue stuff there - busbybox and tools
sda2 & sdb2 in a raid1 for / (ext3), shouldn't need more than 10G, used
to use 8G previously, again for its simplicity
Depending on if I have any other disks or not I will make a raid5 (or
raid6) with the sd?3 paritions, if I only have the 2 disks then sda3 &
sdb3 are made into a raid1 and the resultant md is placed as a lvm
physical volume, which I then carve out things like /tmp /var/log /home
swap. This means I can grow them on the fly as needed.
I also looked at lvm mirror, the difference is lvm need a bitmap (which
you have the option of keeping in memory - which is transient). The
overhead of using lvm raid1 did not seem worth it (echo by my results
from googling)
Alex
>
> I have read contradicting reports regarding performance - one said LVM
> mirroring was faster since it read from both disks in parallel, one
> said the same about MD.
> What is your experience?
>
> > mySQL servers and it works great. We do nightly snapshots and rsync/tar that
> > stuff to central on line storage server. Though we do have to issue a lock
> > tables command to our mySQL servers for the duration of the creation of the
> > LVM snapshot, but after the snapshot is created we remove the locks.
>
> That was the idea.
> Do you have to snapshot each LV separately or can you do one snapshot
> for the whole VG?
> Since I only have two disks I'll do a Raid 1 mirror and probably use a
> layout like
>
> md0: root (10G)
> md1: system VG (rest of disk)
> system/swap (probably 4G to start with)
> system/tmp (probably 4G too, tmpfs)
> system/var (10G to start with, will contain MySQL data)
> system/home (rest of disk minus 10G for snapshot data)
>
> This worked quite well so far. But I'm sure once I have the
> flexibility, I will think of another more detailed setup for
> partitioning.
>
> Do you think it's a good idea to put swap and /tmp in the VG?
>
> > For booting, we just use raid 1 on two partitions, no LVM. Debian seems to
> > have no problem with installation, but it does not install grub into both
> > drives mbr. You have to manually do that or if the hdd that does have grub
> > installed fails your system will be unable to boot. Its been a while since I
> > have double checked this, so it may have been fixed.
>
> I have done this with LILO ("raid-extra-boot=") but how do you do it
> with Grub?
> Like this? : http://faq.jensbenecke.de/wiki/SuseSystem
> (scroll down a bit to see the GRUB commands)
>
> > I was just running some bonnie tests yesterday on a test server and here are
> > those results...
> > name file_size putc putc_cpu put_block put_block_cpu rewrite rewrite_cpu
> > getc getc_cpu get_block get_block_cpu seeks seeks_cpu num_files dhcp187
> > 32368M 69293 99 209808 51 66438 13 49501 67 94315 9 777.3 0 16 dhcp187
> > 32368M 68795 97 219512 51 65161 13 49648 67 93097 9 769.1 1 16 dhcp187
> > 32368M 68501 98 222966 53 65132 13 49639 67 94043 9 762.7 1 16 dhcp187
> > 32368M 69498 98 246923 58 63480 12 49937 67 94189 9 785 0 16 dhcp187 32368M
> > 69324 98 230438 55 64656 12 49576 67 94046 9 749 1 16
>
> hm, this does not tell me too much (probably because of bad line
> wrapping).
> did you compare a single disk with a MD setup?
>
> > Brian Schrock,
>
>
> --
> To UNSUBSCRIBE, email to [EMAIL PROTECTED]
> with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
>
>
--
"We ended the rule of one of history's worst tyrants, and in so doing, we not
only freed the American people, we made our own people more secure."
- George W. Bush
05/03/2003
Crawford, TX
signature.asc
Description: Digital signature
Re: For Lenny: LVM, LVM+MD or just MD for mirroring?
On 5 Sep., 21:20, "Brian Schrock" <[EMAIL PROTECTED]> wrote:
> We use LVM and software raid via md on every server. We use md to run raid
> 10 arrays and then we use LVM on top of that. We use snapshots to backup our
Hello Brian,
sounds good - what made you stack MD and LVM, and not use LVM's mirror
feature?
Any specific reason?
I have read contradicting reports regarding performance - one said LVM
mirroring was faster since it read from both disks in parallel, one
said the same about MD.
What is your experience?
> mySQL servers and it works great. We do nightly snapshots and rsync/tar that
> stuff to central on line storage server. Though we do have to issue a lock
> tables command to our mySQL servers for the duration of the creation of the
> LVM snapshot, but after the snapshot is created we remove the locks.
That was the idea.
Do you have to snapshot each LV separately or can you do one snapshot
for the whole VG?
Since I only have two disks I'll do a Raid 1 mirror and probably use a
layout like
md0: root (10G)
md1: system VG (rest of disk)
system/swap (probably 4G to start with)
system/tmp (probably 4G too, tmpfs)
system/var (10G to start with, will contain MySQL data)
system/home (rest of disk minus 10G for snapshot data)
This worked quite well so far. But I'm sure once I have the
flexibility, I will think of another more detailed setup for
partitioning.
Do you think it's a good idea to put swap and /tmp in the VG?
> For booting, we just use raid 1 on two partitions, no LVM. Debian seems to
> have no problem with installation, but it does not install grub into both
> drives mbr. You have to manually do that or if the hdd that does have grub
> installed fails your system will be unable to boot. Its been a while since I
> have double checked this, so it may have been fixed.
I have done this with LILO ("raid-extra-boot=") but how do you do it
with Grub?
Like this? : http://faq.jensbenecke.de/wiki/SuseSystem
(scroll down a bit to see the GRUB commands)
> I was just running some bonnie tests yesterday on a test server and here are
> those results...
> name file_size putc putc_cpu put_block put_block_cpu rewrite rewrite_cpu
> getc getc_cpu get_block get_block_cpu seeks seeks_cpu num_files dhcp187
> 32368M 69293 99 209808 51 66438 13 49501 67 94315 9 777.3 0 16 dhcp187
> 32368M 68795 97 219512 51 65161 13 49648 67 93097 9 769.1 1 16 dhcp187
> 32368M 68501 98 222966 53 65132 13 49639 67 94043 9 762.7 1 16 dhcp187
> 32368M 69498 98 246923 58 63480 12 49937 67 94189 9 785 0 16 dhcp187 32368M
> 69324 98 230438 55 64656 12 49576 67 94046 9 749 1 16
hm, this does not tell me too much (probably because of bad line
wrapping).
did you compare a single disk with a MD setup?
> Brian Schrock,
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: For Lenny: LVM, LVM+MD or just MD for mirroring?
On 5 Sep., 20:10, "nate" <[EMAIL PROTECTED]> wrote: > Jens wrote: > > Generally I would like to hear any experiences with LVM2 you can > > offer. > > The last time I tried LVM I hosed everything, but that was eight years > > ago, at least partly due to a user error, and with LVM1 on Debian > > Woody with a self-compiled kernel on 2.4.2x. > > I would stick to MD for the RAID and LVM for the snapshot stuff. Hi, thanks a lot for your insights! I was leaning towards that solution as well since I have experience with MD already. LVM is supposed to be able to do fast rebuilding by creating a log when a device fails. MD (in 2.4 at least) always does a full rebuild, has this maybe changed? > It's been a while since I ran MD on Debian so I don't know how/if it > handles installing the boot loader to the other disk. Red Hat at least > did not by default and I put a workaround in my kickstart config to > take care of that for me. Just out of curiosity, what exactly did you do? > I have used LVM/LVM2 quite a bit, though have never used snapshots. > The most use I got out of LVM was using it with a multipathed SAN, > over iSCSI and Fiber Channel. LVM allowed the system to find the > volumes no matter what path they were presented down(and with That was an advantage, true. Nowadays you can mount using UUIDs or disk labels which also works fine. > Also with LVM I was able to restrict volume sizes pretty easily OK, I do this with quotas mainly. I haven't yet come across a scenario where these were not sufficient any more. (Except maybe for /tmp and / var which have their own partitions). > I believe with LVM and snapshots you have to set aside a > fixed amount of space to store the deltas for them when > configuring the volume group, though this may of changed. This is still the case. This allows LVM to rebuild within minutes, instead of hours (like MD). This would be a big plus for LVM - if it worked. Plus, I could use a MD device for the LVM log so it could be used if either disk fails. > If using MD with LVM sounds too complicated you should consider > a hardware RAID controller, that way you don't need to worry > about boot loaders and 2nd disks, or rebuilding the array etc.. The problem here is the budget ... ;) > For me, multiple simple layers are easier to work with than > fewer more complicated layers. True. Thanks a lot! -Jens -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: For Lenny: LVM, LVM+MD or just MD for mirroring?
We use LVM and software raid via md on every server. We use md to run raid 10 arrays and then we use LVM on top of that. We use snapshots to backup our mySQL servers and it works great. We do nightly snapshots and rsync/tar that stuff to central on line storage server. Though we do have to issue a lock tables command to our mySQL servers for the duration of the creation of the LVM snapshot, but after the snapshot is created we remove the locks. For booting, we just use raid 1 on two partitions, no LVM. Debian seems to have no problem with installation, but it does not install grub into both drives mbr. You have to manually do that or if the hdd that does have grub installed fails your system will be unable to boot. Its been a while since I have double checked this, so it may have been fixed. I was just running some bonnie tests yesterday on a test server and here are those results... name file_size putc putc_cpu put_block put_block_cpu rewrite rewrite_cpu getc getc_cpu get_block get_block_cpu seeks seeks_cpu num_files dhcp187 32368M 69293 99 209808 51 66438 13 49501 67 94315 9 777.3 0 16 dhcp187 32368M 68795 97 219512 51 65161 13 49648 67 93097 9 769.1 1 16 dhcp187 32368M 68501 98 222966 53 65132 13 49639 67 94043 9 762.7 1 16 dhcp187 32368M 69498 98 246923 58 63480 12 49937 67 94189 9 785 0 16 dhcp187 32368M 69324 98 230438 55 64656 12 49576 67 94046 9 749 1 16 Brian Schrock,
Re: For Lenny: LVM, LVM+MD or just MD for mirroring?
Jens wrote: > Generally I would like to hear any experiences with LVM2 you can > offer. > The last time I tried LVM I hosed everything, but that was eight years > ago, at least partly due to a user error, and with LVM1 on Debian > Woody > with a self-compiled kernel on 2.4.2x. I would stick to MD for the RAID and LVM for the snapshot stuff. It's been a while since I ran MD on Debian so I don't know how/if it handles installing the boot loader to the other disk. Red Hat at least did not by default and I put a workaround in my kickstart config to take care of that for me. I have used LVM/LVM2 quite a bit, though have never used snapshots. The most use I got out of LVM was using it with a multipathed SAN, over iSCSI and Fiber Channel. LVM allowed the system to find the volumes no matter what path they were presented down(and with software iSCSI the scsi devices changed every time iscsi was restarted). So it was a real help. I limited my snapshots to the SAN array itself, so I could snapshot the volume and export it to another system to do no impact backups. Also with LVM I was able to restrict volume sizes pretty easily to allow for growth as the storage array presented the volumes in a thinly provisioned form (some work loads don't agree with thin provisioning so I could export a 2TB volume to the hosts, restrict the file system to a couple hundred gig, and when I needed more I just expanded the LVM, only space that was written to was actually consumed on the storage end). I believe with LVM and snapshots you have to set aside a fixed amount of space to store the deltas for them when configuring the volume group, though this may of changed. If using MD with LVM sounds too complicated you should consider a hardware RAID controller, that way you don't need to worry about boot loaders and 2nd disks, or rebuilding the array etc.. For me, multiple simple layers are easier to work with than fewer more complicated layers. nate -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
For Lenny: LVM, LVM+MD or just MD for mirroring?
Hi everybody, I am a long term Debian user (since 1996, actually). I am about to set up a new debian server using Lenny, which will go into production probably in October. This machine will use lighttpd, mysql, dovecot/postfix among others. Currently I use two harddisks and MD for all servers to guard against harddisk failure, and to gain additional read speed. For the new machine, I would like to try LVM2 but after spending a week browsing Google and news articles, I am a bit confused. There are two things I want to achieve: 1. guard against harddisk failure, get notified (by mail / SMS / whateveR) when one fails MD does this beautifully and my current server has had a few harddisk failures which were all handled very well. 2. [NEW] do backups using snapshots, to minimize downtime for backups. I have read LVM2 does this very well, currently I have to shutdown everything for backups (nightly), which basically sucks. I do NOT need dynamic resizing of partitions. My current server is (still) an updated Sarge installation , running since Sarge was "testing", and it has never felt the need to add space. I will have two 400GB disks in the new machine which will be far more than I need (currently I have 2x160GB of which 75GB are used). I would like something like the following setup: 10GB root, 2GB swap, 4GB tmp, 10GB /var, and the rest for data. All mirrored 1:1 on both disks. If possible, I would like to use a stock kernel (no self-compiles) and stock Debian utilities, to ease upgrades. Questions: -- Generally: Is it a good idea to use LVM2 mirroring or should I stick to MD for mirroring, and use LVM only for the snapshots? This seems overcomplicated. -- Can Lenny create and use mirrored LVM partitions? I tried a test installation in VMware but the installer does not offer a mirroring feature, only encryption. -- Can Lenny's boot manager install itself on both mirrored harddisks' MBRs and boot from both? -- Can Lenny boot from a LVM2 logical volume? -- Lenny does not seem to contain 'dmeventd', which (I heard) is necessary to detect disk failures and deactivate failed mirrors, like 'mdadm' did with 'md'. Can it, or should it, be added, or is there an alternative way of doing this? Generally I would like to hear any experiences with LVM2 you can offer. The last time I tried LVM I hosed everything, but that was eight years ago, at least partly due to a user error, and with LVM1 on Debian Woody with a self-compiled kernel on 2.4.2x. Thank you! Jens -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Traffic Mirroring for Debugging
I suppose you have multiples interfaces on your 192.168.1.10. Normally, you have multiple IP addresses on your 192.168.1.10, one address for each interface. If you bridge interfaces, all your interfaces will have one and only one IP address 192.168.1.10. If you prefer, your machine (192.168.1.10) will work as switch. You can see the beginning of this article (no need to use the firewall part, use only D1 or D2 depending on your configuration): http://www.debian.org/doc/manuals/securing-debian-howto/ap-bridge-fw.en.html Search also on google with "debian+bridge". I'm french and I have many links but only in my language. HTH, Christophe Volkan YAZICI a écrit : On Fri, 22 Aug 2008, Bonnel Christophe <[EMAIL PROTECTED]> writes: Why don't you bridge the interfaces of 192.168.1.10? I don't have in depth experience with bridging, would you mind explaining a little bit about it please. (Example tutorial pointers will be appreciated.) In that case, you need to reconfigure client to point to 192.168.1.20 and you can sniff your network via the 192.168.1.10 machine? What I really need is not spoofing, I need to somehow redirect identical network traffic to a second machine. Applications are running on Microsoft Windows Servers on 1.2 and 1.20. Is bridging in the way you mentioned capable of doing such a thing? Regards. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Traffic Mirroring for Debugging
On Fri, 22 Aug 2008, Bonnel Christophe <[EMAIL PROTECTED]> writes: > Why don't you bridge the interfaces of 192.168.1.10? I don't have in depth experience with bridging, would you mind explaining a little bit about it please. (Example tutorial pointers will be appreciated.) > In that case, you need to reconfigure client to point to 192.168.1.20 > and you can sniff your network via the 192.168.1.10 machine? What I really need is not spoofing, I need to somehow redirect identical network traffic to a second machine. Applications are running on Microsoft Windows Servers on 1.2 and 1.20. Is bridging in the way you mentioned capable of doing such a thing? Regards. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Traffic Mirroring for Debugging
Why don't you bridge the interfaces of 192.168.1.10 ? In that case, you need to reconfigure client to point to 192.168.1.20 and you can sniff your network via the 192.168.1.10 machine ? Hope this helps, Christophe Volkan YAZICI a écrit : Hi, In one of our servers, I want to debug a network server daemon. The problem is I don't have luxury for a downtime or to iterrupt related server's network traffic. Current routing structure looks like below. VPN Switch (192.168.1.1) -> Server Machine (192.168.1.2) To debug the related server daemon, I planned to redirect the network traffic to a second sandbox server. Planned routing structure is: VPN Switch (192.168.1.1) | +-> Temporary Linux Machine (192.168.1.10) || |+-> Server Machine (192.168.1.2) +--> Sandbox Server Machine (192.168.1.20) For test purposes, I configured a client to connect to 192.168.1.10 (instead of actual server, 192.168.1.2). But I couldn't manage to redirect incoming traffic of 192.168.1.10 to 192.168.1.2 and 192.168.2.20 machines. How can I get such a mirrored redirection? Furthermore, I first considered using DNAT of iptables, but learnt that it doesn't support multiple destinations without load-balancing. Anyway, I wanted to give it a try for just a single machine. I connected to 1.10 machine and typed below iptables command: # iptables -t nat -A PREROUTING -d 192.168.1.10 -j DNAT --to-destination 192.168.1.20 # iptables -L -n -t nat Chain PREROUTING (policy ACCEPT) target prot opt source destination DNAT 0-- 0.0.0.0/0192.168.1.10 to:192.168.1.20 Chain POSTROUTING (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination And started a netcat listener on 1.20 machine: # nc -l -p 2310 Then tried to connect to the started nc listener from 1.10 machine: # nc -vvv 192.168.1.10 2310 192.168.1.10: inverse host lookup failed: Unknown host (UNKNOWN) [192.168.1.10] 2310 (?) : Connection refused sent 0, rcvd 0 Gosh! Even couldn't manage to make DNAT for a single address work properly. Any helps will be really really appreciated. Regards. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Traffic Mirroring for Debugging
Hi, In one of our servers, I want to debug a network server daemon. The problem is I don't have luxury for a downtime or to iterrupt related server's network traffic. Current routing structure looks like below. VPN Switch (192.168.1.1) -> Server Machine (192.168.1.2) To debug the related server daemon, I planned to redirect the network traffic to a second sandbox server. Planned routing structure is: VPN Switch (192.168.1.1) | +-> Temporary Linux Machine (192.168.1.10) || |+-> Server Machine (192.168.1.2) +--> Sandbox Server Machine (192.168.1.20) For test purposes, I configured a client to connect to 192.168.1.10 (instead of actual server, 192.168.1.2). But I couldn't manage to redirect incoming traffic of 192.168.1.10 to 192.168.1.2 and 192.168.2.20 machines. How can I get such a mirrored redirection? Furthermore, I first considered using DNAT of iptables, but learnt that it doesn't support multiple destinations without load-balancing. Anyway, I wanted to give it a try for just a single machine. I connected to 1.10 machine and typed below iptables command: # iptables -t nat -A PREROUTING -d 192.168.1.10 -j DNAT --to-destination 192.168.1.20 # iptables -L -n -t nat Chain PREROUTING (policy ACCEPT) target prot opt source destination DNAT 0-- 0.0.0.0/0192.168.1.10 to:192.168.1.20 Chain POSTROUTING (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination And started a netcat listener on 1.20 machine: # nc -l -p 2310 Then tried to connect to the started nc listener from 1.10 machine: # nc -vvv 192.168.1.10 2310 192.168.1.10: inverse host lookup failed: Unknown host (UNKNOWN) [192.168.1.10] 2310 (?) : Connection refused sent 0, rcvd 0 Gosh! Even couldn't manage to make DNAT for a single address work properly. Any helps will be really really appreciated. Regards. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: mirroring a site locally using wget?
On Fri, Aug 15, 2008 at 05:13:16AM -0400, Zach Uram wrote: > How exactly can I do this: > > There is a website I wish to mirror. It has many pages and images and > it uses CSS. > > I wish to run wget and have it: > 1) download all HTML/CSS/images > 2) uses local references so HREF="http://site.org/images/image.jpg";> will be saved as HREF="images/image.jpg"> > 3) do the same thing for CSS etc. Here is an example of a command we use to mirror a website. See the manpage for the meaning of the switches. Don't just copy the command. For other websites we use less switches and different ones: /usr/bin/wget -E -np -r -k -K -N -l inf http:// &> /home/ftpadm/logs/websitename.log Regards Johann -- Johann Spies Telefoon: 021-808 4036 Informasietegnologie, Universiteit van Stellenbosch "And I say unto you, Ask, and it shall be given you; seek, and ye shall find; knock, and it shall be opened unto you."Luke 11:9 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: mirroring a site locally using wget?
On Fri, 15 Aug 2008 05:13:16 -0400 "Zach Uram" <[EMAIL PROTECTED]> wrote: > How exactly can I do this: > > There is a website I wish to mirror. It has many pages and images and > it uses CSS. You also might be interested in httrack. > Zach Celejar -- mailmin.sourceforge.net - remote access via secure (OpenPGP) email ssuds.sourceforge.net - A Simple Sudoku Solver and Generator -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: mirroring a site locally using wget?
Zach Uram escreveu: How exactly can I do this: There is a website I wish to mirror. It has many pages and images and it uses CSS. I wish to run wget and have it: 1) download all HTML/CSS/images 2) uses local references so http://site.org/images/image.jpg";> will be saved as 3) do the same thing for CSS etc. I am not sure how many levels of links the site has but it is a lot so I want to be sure to get them all. So when the site is saved locally all the links will work and be local links, even if one of the pages is setup to use http://site.org/html/setup.html";> it will be saved locall as "A HREF="html/setup.html"> Enter man wget at the command prompt. Then read it. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
mirroring a site locally using wget?
How exactly can I do this: There is a website I wish to mirror. It has many pages and images and it uses CSS. I wish to run wget and have it: 1) download all HTML/CSS/images 2) uses local references so http://site.org/images/image.jpg";> will be saved as 3) do the same thing for CSS etc. I am not sure how many levels of links the site has but it is a lot so I want to be sure to get them all. So when the site is saved locally all the links will work and be local links, even if one of the pages is setup to use http://site.org/html/setup.html";> it will be saved locall as "A HREF="html/setup.html"> Thanks! Zach -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Re: mounting CD1 is not enough for mirroring and net install
Thanks for reply,
I added the official Release and the gpg files to the approprate
place dists/etch ( I am using etch/R3), but I got same error unable
to find the Release file.
Maybe this can help to understand:When I start the client
machine, the dhcp and tftp ran correctly ( the address is given, the
kernel is found ) After that I received the following message:
The network autoconfiguration was successful. However ,
NO DEFAULT ROUTE WAS SET: the system doesn't know how to communicate
with hosts on the internet.
This will make it impossible to continue , UNLESS you have first CD, a
netinst CD or package on LOCAL network.
This is my Case , local package !!!
May be I have a problem in dchp or tftp-ha ( no DNS declared)
Here is my config:
server dhcp : 172.19.6.250
subnet 172.19.6.0 netmask 255.255.255.0 {
range 172.19.6.100 172.19.6.200;
option subnet-mask 255.255.255.0 ;
option broadcast-address 172.19.6.255;
option root-path "172.19.6.250:/pxeroot";
next-server 172.19.6.250;
filename "pxelinux.0";
}
cat tftpd-hpa :
RUN_DAEMON="yes"
OPTIONS="-l -s /var/lib/tftpboot"
thanks for any help
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: mounting CD1 is not enough for mirroring and net install
Please do not start a new thread unless the new problem is totally unrelated. abdelkader belahcene wrote: Thanks for reply, I put in the server as it is described in the doc, the netboot, and used the tftp, it runs correctly, and started nice, when it asked for the mirror, If I declare an official mirror, it runs correctly, and installed, when I declare my own mirror, I fails. I mean by my own mirror an ftp server when I mount The offical CD1, ! The installation is probably failing because the official CDs do not have the Release.gpg files which are required by default. You can either have a partial mirror on your server machine using debmirror or you can pass the boot parameter debian-installer/allow_unauthenticated=true -- Raj Kiran Grandhi -- At the source of every error which is blamed on the computer, you will find at least two human errors, including the error of blaming it on the computer. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
mounting CD1 is not enough for mirroring and net install
Thanks for reply, I put in the server as it is described in the doc, the netboot, and used the tftp, it runs correctly, and started nice, when it asked for the mirror, If I declare an official mirror, it runs correctly, and installed, when I declare my own mirror, I fails. I mean by my own mirror an ftp server when I mount The offical CD1, ! Do I need and extra config to precise, may be the tree of offical installation (via network) is not the same as in CD1? thank you >>Read the netboot section in the Debian installation manual. >want to install Debian from a local server on several machines which >haven't CD,s. >In other word how to install without using neither CD nor floppy drive, usb , >just network card. from A local site. >I mean instead of reading from CD, I read from remote machine using >pxe or tftboot or something like this. >I saw somebody doing it with Fedora!! >The gool is to install several machines at the same time from the server. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: using webmin to create raid1 mirroring on exsiting system
On Fri, May 04, 2007 at 09:55:41AM -0400, harland christofferson wrote: > > i have a sarge installation. > > i have two ide drives (hda and hdc) that are partitioned identically. > > when doing the install a while ago, i tried to configure partitions for > raid1 but some how goofed. > > i cannot risk losing data on hda while getting raid1 up and running. ^^^ Make a backup. Seriously. If you can't afford to lose data, then don't do it. Regards, -Roberto -- Roberto C. Sánchez http://people.connexer.com/~roberto http://www.connexer.com signature.asc Description: Digital signature
Re: using webmin to create raid1 mirroring on exsiting system
also sprach harland christofferson <[EMAIL PROTECTED]> [2007.05.04.1555 +0200]: > has anyone successfully implemented raid1 on an existing system by using > raid tools and mdadm? README.recipes says how to do that. -- Please do not send copies of list mail to me; I read the list! .''`. martin f. krafft <[EMAIL PROTECTED]> : :' : proud Debian developer, author, administrator, and user `. `'` http://people.debian.org/~madduck - http://debiansystem.info `- Debian - when you have better things to do than fixing systems NP: Trentemøller - Like Two Strangers signature.asc Description: Digital signature (GPG/PGP)
using webmin to create raid1 mirroring on exsiting system
i have a sarge installation. i have two ide drives (hda and hdc) that are partitioned identically. when doing the install a while ago, i tried to configure partitions for raid1 but some how goofed. i cannot risk losing data on hda while getting raid1 up and running. has anyone successfully implemented raid1 on an existing system w/ webmin? has anyone successfully implemented raid1 on an existing system by using raid tools and mdadm? i tried to do this on another system years ago ... ended up wiping out data on hda. i have been skiddish about trying again but have to bite the bullet now. regards, harland -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Mirroring a failing HDD
Douglas Tutty wrote: On Fri, Nov 10, 2006 at 12:38:43PM +, Shri Shrikumar wrote: There is a server(sarge) that I maintain that used to be mirrored and all was well. However, the mirror was recently broken and when trying to rebuild, I run into an interesting problem. The array rebuilds to about 80% and then restarts. dmesg has the following: Hi Doug, Thanks for you response. You had a raid consisting of two drives, one of which is /dev/sdb3. One died, leaving sdb3. However, during rebuild /dev/sdb is having read errors. Not quite. The other drive did not die. I upgraded the kernel and on re-boot, the other drive was mis-read and not re-linked as part of the raid. It was not a hard disk failure that caused the mirror to be broken. Was the origional failed drive on the same controller? Could that drive failure also have killed the controller? Could it have been a controller failure and not a drive failure? Do you have another controller in the box to which you could connect the drive that is now hdb? Are there other drives on this controller that are working OK? The problem is that its a live server, so can't fiddle around with it much. There is another disk on it which seems to be running just fine. I am fairly confident that it is the disk that is failing and not the controller. Can you read other partitions on hdb? (and therefore prove that both the controller and drive are OK) Yes, there is another partition that is mirrored just fine with sda. I hope you have good backups. Thankfully, I do. It continues to run fine as well due to the corruption being outside of the disk being used. How does the system work if you disconnect the new drive (sda) so that the raid runs in degradded mode? Do you still get read errors? If you The raid is currently in degraded mode with just sdb. There are no read errors. I am using LVM on raid to allocate disk space and there is about 60gb unallocated. I am guessing that the failed part of the disk is in this unallocated area. I suppose the worst case would be to build a brand new mirror using just the new disk and pvmove the data across and then de-commission the dying hdd. I am hoping for an easier (and quicker solution though) Best Wishes, Shri -- Shri Shrikumar Technologist Extraordinaire Kraya t: 0845 644 4745 d: 0131 247 8021 f: 0131 478 7377 w: www.kraya.co.uk -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Mirroring a failing HDD
On Fri, Nov 10, 2006 at 12:38:43PM +, Shri Shrikumar wrote:
> There is a server(sarge) that I maintain that used to be mirrored and
> all was well. However, the mirror was recently broken and when trying to
> rebuild, I run into an interesting problem.
>
> The array rebuilds to about 80% and then restarts. dmesg has the following:
>
> RAID1 conf printout:
> --- wd:1 rd:2
> disk 0, wo:1, o:1, dev:sda3
> disk 1, wo:0, o:1, dev:sdb3
> ata2: status=0x51 { DriveReady SeekComplete Error }
> ata2: error=0x40 { UncorrectableError }
> SCSI error : <1 0 0 0> return code = 0x802
> sdb: Current: sense key: Medium Error
>Additional sense: Unrecovered read error - auto reallocate failed
> end_request: I/O error, dev sdb, sector 268475518
> //. removed few more of these lines
>
> Mirror information as follows:
>
> $ sudo mdadm -D /dev/md1
> /dev/md1:
>Version : 00.90.01
> Creation Time : Tue Nov 8 16:54:09 2005
> Raid Level : raid1
> Array Size : 155276160 (148.08 GiB 159.00 GB)
>Device Size : 155276160 (148.08 GiB 159.00 GB)
> Raid Devices : 2
> Total Devices : 2
> Preferred Minor : 1
>Persistence : Superblock is persistent
>
> State : active, degraded, recovering
> Active Devices : 1
> Working Devices : 2
> Failed Devices : 0
> Spare Devices : 1
>
> Rebuild Status : 59% complete
>
>Number Major Minor RaidDevice State
> 0 00- removed
> 1 8 191 active sync /dev/sdb3
>
> 2 830 spare rebuilding /dev/sda3
>
> of the 159GB, only 60gb is allocated(using LVM) and I am guessing that
> the failing part is within the unallocated section.
>
Hopefully, someone who has had a raid disk failure and has watched a
recovery can offer some better advice. However, my interpretation is
this:
You had a raid consisting of two drives, one of which is /dev/sdb3. One
died, leaving sdb3. However, during rebuild /dev/sdb is having read
errors.
I see three possbile sources of read erroros on sdb3:
The third partition (sdb3) is corrupted
The drive is failing (sdb)
The controller that the drive is on is failing.
Was the origional failed drive on the same controller? Could that
drive failure also have killed the controller? Could it have been a
controller failure and not a drive failure? Do you have another
controller in the box to which you could connect the drive that is now
hdb? Are there other drives on this controller that are working OK?
Can you read other partitions on hdb? (and therefore prove that both the
controller and drive are OK)
I hope you have good backups.
How does the system work if you disconnect the new drive (sda) so that the
raid runs in degradded mode? Do you still get read errors? If you
don't have backups, this would be a good time to copy the data
preferably off this box via scp or something to another computer on your
network.
Doug.
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Mirroring a failing HDD
Hi All,
I tried posting this yesterday while not subscribed and didn't look like
it got through - so trying again after subscribing. If it gets through
twice - please accept my apologies.
I wonder if someone can help me with a problem that I am having.
There is a server(sarge) that I maintain that used to be mirrored and
all was well. However, the mirror was recently broken and when trying to
rebuild, I run into an interesting problem.
The array rebuilds to about 80% and then restarts. dmesg has the following:
RAID1 conf printout:
--- wd:1 rd:2
disk 0, wo:1, o:1, dev:sda3
disk 1, wo:0, o:1, dev:sdb3
..<6>md: syncing RAID array md1
md: minimum _guaranteed_ reconstruction speed: 1000 KB/sec/disc.
md: using maximum available idle IO bandwith (but not more than 20
KB/sec) for reconstruction.
md: using 128k window, over a total of 155276160 blocks.
ata2: status=0x51 { DriveReady SeekComplete Error }
ata2: error=0x40 { UncorrectableError }
ata2: status=0x51 { DriveReady SeekComplete Error }
ata2: error=0x40 { UncorrectableError }
ata2: status=0x51 { DriveReady SeekComplete Error }
ata2: error=0x40 { UncorrectableError }
ata2: status=0x51 { DriveReady SeekComplete Error }
ata2: error=0x40 { UncorrectableError }
ata2: status=0x51 { DriveReady SeekComplete Error }
ata2: error=0x40 { UncorrectableError }
SCSI error : <1 0 0 0> return code = 0x802
sdb: Current: sense key: Medium Error
Additional sense: Unrecovered read error - auto reallocate failed
end_request: I/O error, dev sdb, sector 268475518
ata2: status=0x51 { DriveReady SeekComplete Error }
ata2: error=0x40 { UncorrectableError }
SCSI error : <1 0 0 0> return code = 0x802
sdb: Current: sense key: Medium Error
Additional sense: Unrecovered read error - auto reallocate failed
end_request: I/O error, dev sdb, sector 268475526
ata2: status=0x51 { DriveReady SeekComplete Error }
ata2: error=0x40 { UncorrectableError }
//. removed few more of these lines
..<6>md: syncing RAID array md1
md: minimum _guaranteed_ reconstruction speed: 1000 KB/sec/disc.
md: using maximum available idle IO bandwith (but not more than 20
KB/sec) for reconstruction.
md: using 128k window, over a total of 155276160 blocks.
Mirror information as follows:
$ sudo mdadm -D /dev/md1
/dev/md1:
Version : 00.90.01
Creation Time : Tue Nov 8 16:54:09 2005
Raid Level : raid1
Array Size : 155276160 (148.08 GiB 159.00 GB)
Device Size : 155276160 (148.08 GiB 159.00 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 1
Persistence : Superblock is persistent
Update Time : Thu Nov 9 17:56:27 2006
State : active, degraded, recovering
Active Devices : 1
Working Devices : 2
Failed Devices : 0
Spare Devices : 1
Rebuild Status : 59% complete
UUID : 526093bb:9a1af158:1faa9404:008c0621
Events : 0.35102711
Number Major Minor RaidDevice State
0 00- removed
1 8 191 active sync /dev/sdb3
2 830 spare rebuilding /dev/sda3
of the 159GB, only 60gb is allocated(using LVM) and I am guessing that
the failing part is within the unallocated section.
Any help in resolving this would be greatly appreciated. Please let me
know if you require any additional information.
Best Wishes,
Shri
PS: I am not subscribed to the list (too much traffic), so a cc: would
be greatly appreciated.
--
Shri Shrikumar
Technologist Extraordinaire
Kraya
t: 0845 644 4745
d: 0131 247 8021
f: 0131 478 7377
w: www.kraya.co.uk
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Fwd: Problems mirroring debian-multimedia
On 6/24/06, Marty <[EMAIL PROTECTED]> wrote: I am repeating this issue to give it it's own subject line and thread. Since the change from marillat to debian-multimedia I have not yet found a satisfactory way of mirroring debian-multimedia. The site does not seem to offer anonymous rsync or ftp, and wget and debmirror don't work as I expect them to. ... I know this won't answer your question, but I used apt-mirror, a perl very nice script for mirroring debian apt-sources, and for me it works like a charm even with the new debian-multimedia. However it depends upon wget, and still it works nice with debian-multimedia, so I don't know what your problems would be. Any ways give it a try if you'd like to use something new, it's pretty easy to setup and use. http://apt-mirror.sourceforge.net/ -- Javier -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
SOLVED (was Re: Problems mirroring debian-multimedia)
Marty wrote: Since the change from marillat to debian-multimedia I have not yet found a satisfactory way of mirroring debian-multimedia. The site does not seem to offer anonymous rsync or ftp, and wget and debmirror don't work as I expect them to. Replying to my own posting, I have found what appears to be a suitable method using lftp, which can do http transfers. I still don't know why it only works when I mirror the pool and dist subdirectories separately: lftp -c 'open http://www.debian-multimedia.org/pool; mirror -x .*-powerpc.*\ -x .*-mips.* -x .*-arm.* -x .*m68k.* -x .*sparc.* -x .*amd64.* -x .*ia64.* \ -x .*-alpha.* -x .*_alpha.* -x .*hppa.* --verbose=3 --delete . \ /mirror/debian-multimedia/pool' lftp -c 'open http://www.debian-multimedia.org/dists; mirror -x .*-powerpc.*\ -x .*-mips.* -x .*-arm.* -x .*m68k.* -x .*sparc.* -x .*amd64.* -x .*ia64.* \ -x .*-alpha.* -x .*_alpha.* -x .*hppa.* --verbose=3 --delete . \ /mirror/debian-multimedia/dists' -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Problems mirroring debian-multimedia
I am repeating this issue to give it it's own subject line and thread. Since the change from marillat to debian-multimedia I have not yet found a satisfactory way of mirroring debian-multimedia. The site does not seem to offer anonymous rsync or ftp, and wget and debmirror don't work as I expect them to. With debmirror: debmirror --root=/ -d sarge -a i386 -e http \ --ignore-release-gpg \--ignore-missing-release --ignore-small-errors \ -p -v --debug --getcontents -h www.debian-multimedia.org \ /mirror The results include the following: Ignoring missing Release file for dists/main/main/binary-i386/Packages.gz dists/main/main/binary-i386/Packages.gz needs fetch http://www.debian-multimedia.org///dists/main/main/binary-i386/Packages.gz => Getting: dists/main/main/binary-i386/Packages.gz... 404 Not Found dists/main/main/binary-i386/Packages.gz failed 404 Not Found dists/main/main/binary-i386/Packages.gz failed md5sum check, removing Some directories are created but no files are downloaded. With wget: wget -mirror www.debian-multimedia.org -nH -P /mirror This fails to download the pool directory, which contains all the packages. In addition, I haven't found a way to exclude non-i386 files, or automatically delete files not in the archive. In this sense, wget's mirror option seems like a misnomer. I also have a pre-existing issue in that I have not found a way to validate the files, since the archive has no indices file, and the last time I checked sarge did not support gpg validation. Thanks for any suggestions. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Will Debian supports Intel 64 bit processor?. How to set mirroring.
Colin wrote: Ryan Nowakowski wrote: On Fri, Dec 02, 2005 at 11:06:44PM +0800, Muthukumaran Saravanan wrote: Will Debian supports Intel 64 bit processor. Yes: http://www.debian.org/ports/ia64/ I don't think Muthukumaran means Itanium. The amd64 architecture is probably what he means. ... information for which is available here: http://www.debian.org/ports/amd64/ Greg -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Will Debian supports Intel 64 bit processor?. How to set mirroring.
Ryan Nowakowski wrote: > On Fri, Dec 02, 2005 at 11:06:44PM +0800, Muthukumaran Saravanan wrote: > >>Will Debian supports Intel 64 bit processor. > > Yes: http://www.debian.org/ports/ia64/ I don't think Muthukumaran means Itanium. The amd64 architecture is probably what he means. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Will Debian supports Intel 64 bit processor?. How to set mirroring.
On Fri, Dec 02, 2005 at 11:06:44PM +0800, Muthukumaran Saravanan wrote: > Will Debian supports Intel 64 bit processor. Yes: http://www.debian.org/ports/ia64/ > I want to install debian file server which the data should be shared with > windows clients, need to install smb server also. apt-get install samba > How to setup mirroring using debian in the same server. I have 2 73 GB SCSI > Harddisk, for this i want to setup mirroring. apt-get install mdadm signature.asc Description: Digital signature
Will Debian supports Intel 64 bit processor?. How to set mirroring.
Dear all, I want some technical details. Will Debian supports Intel 64 bit processor. I want to install debian file server which the data should be shared with windows clients, need to install smb server also. How to setup mirroring using debian in the same server. I have 2 73 GB SCSI Harddisk, for this i want to setup mirroring. Can anyof you give me technical guidance. Thanks & Regards-- M.SaravananFlat 5E 16/FKWAN YICK BUILDING PHASE 1430 - 440A DES VOEUX RS WHONG KONGPH: HOME :(852)-36452678 HAND :(852)-61000856
Re: OT: Mirroring via disk image file
On Sun, Oct 30, 2005 at 04:08:08PM -0500, Marty wrote: > Roberto C. Sanchez wrote: > > >If the drives are *exactly* identical, you can move your old drive to be > >the slave on the secondary ide channel, which makes it hdd. Then place > >the new drive on hda. Boot Knoppix, or another suitable live CD distro, > >and then do this: > > > >dd if=/dev/hdd of=/dev/hda > > That's equivalent my cp example and it doesn't solve the problem. I want > (and need) to use an intermediate image file. > Boot Knoppix - tar up the partitions on the disk, preserving permissions. Create the file system spaces on the new disk. Untar the tarred-up partitions, preserving permissions. Re-run lilo or grub. Identical drives means - exactly identical down to C,H,S sizes if you want to be sure to use a dd Alternatively, look at a cpio / tar solution. Bear in mind that an intermediate 80GB file is going to have to be stored somewhere if you're doing this with 80GB drives and you want to preserve the intermediate step. HTH, Andy > > -- > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > with a subject of "unsubscribe". Trouble? Contact > [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: OT: Mirroring via disk image file
Roberto C. Sanchez wrote: On Sun, Oct 30, 2005 at 03:15:56PM -0500, Marty wrote: I plan to mirror then replace a hard drive, using an identical model drive. Instead of a device-device copy e.g. "cp /dev/hda /dev/hdb" I hope to do "cp image" followed by cp image " where "image" is a disk image file of the original drive. Will the physical sector arrangment be preserved using the intermediate image file, or is that only possible with the device-device copy? I'm asking here because don't have the identical drives available to test my theory beforehand. [Assumption, current hard drive is hda, CD-ROM is hdc] If the drives are *exactly* identical, *exactly*, can that mean both e.g. 80GB drives? you can move your old drive to be the slave on the secondary ide channel, which makes it hdd. Then place the new drive on hda. Boot Knoppix, or another suitable live CD distro, and then do this: dd if=/dev/hdd of=/dev/hda Make absolutely 100% sure that hdd contains the old drive and hda the new drive, or you will obliterate your data with garbage. Also, make sure that you specify `if' and `of' correctly, since `i' and `o' are next to each other on a qwerty keyboard. If you would like to this reasonably quickly, I would recommend enabling DMA on the drives after booting. I believe that Knoppix leaves it off by default since it can cause problems in some rare cases. -Roberto -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: OT: Mirroring via disk image file
On Sun, Oct 30, 2005 at 04:08:08PM -0500, Marty wrote: > Roberto C. Sanchez wrote: > > >If the drives are *exactly* identical, you can move your old drive to be > >the slave on the secondary ide channel, which makes it hdd. Then place > >the new drive on hda. Boot Knoppix, or another suitable live CD distro, > >and then do this: > >dd if=/dev/hdd of=/dev/hda > > That's equivalent my cp example and it doesn't solve the problem. I want > (and need) to use an intermediate image file. Actually, it is not equivalent, as cp will modify the file metadata is it copies. If you need to use an intermediate step: dd if=/dev/hd? of=/path/to/some/place/foo.img dd if=/path/to/some/place/foo.img of=/dev/hd? The only thing is that the place where the intermediate image is placed needs enough free space to store the entire image, which will be the size of your hard drive. Optionally, you can look at a backup/restore tool like systemimager. -Roberto -- Roberto C. Sanchez http://familiasanchez.net/~roberto pgpTI77kZ1VbU.pgp Description: PGP signature
Re: OT: Mirroring via disk image file
Roberto C. Sanchez wrote: If the drives are *exactly* identical, you can move your old drive to be the slave on the secondary ide channel, which makes it hdd. Then place the new drive on hda. Boot Knoppix, or another suitable live CD distro, and then do this: dd if=/dev/hdd of=/dev/hda That's equivalent my cp example and it doesn't solve the problem. I want (and need) to use an intermediate image file. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: OT: Mirroring via disk image file
On Sun, Oct 30, 2005 at 03:15:56PM -0500, Marty wrote: > I plan to mirror then replace a hard drive, using an identical model drive. > > Instead of a device-device copy e.g. "cp /dev/hda /dev/hdb" I hope to do > "cp image" followed by cp image " where "image" is > a disk image file of the original drive. > > Will the physical sector arrangment be preserved using the intermediate image > file, or is that only possible with the device-device copy? I'm asking here > because don't have the identical drives available to test my theory > beforehand. [Assumption, current hard drive is hda, CD-ROM is hdc] If the drives are *exactly* identical, you can move your old drive to be the slave on the secondary ide channel, which makes it hdd. Then place the new drive on hda. Boot Knoppix, or another suitable live CD distro, and then do this: dd if=/dev/hdd of=/dev/hda Make absolutely 100% sure that hdd contains the old drive and hda the new drive, or you will obliterate your data with garbage. Also, make sure that you specify `if' and `of' correctly, since `i' and `o' are next to each other on a qwerty keyboard. If you would like to this reasonably quickly, I would recommend enabling DMA on the drives after booting. I believe that Knoppix leaves it off by default since it can cause problems in some rare cases. -Roberto -- Roberto C. Sanchez http://familiasanchez.net/~roberto pgp8LZjcM8rN2.pgp Description: PGP signature
OT: Mirroring via disk image file
I plan to mirror then replace a hard drive, using an identical model drive. Instead of a device-device copy e.g. "cp /dev/hda /dev/hdb" I hope to do "cp image" followed by cp image " where "image" is a disk image file of the original drive. Will the physical sector arrangment be preserved using the intermediate image file, or is that only possible with the device-device copy? I'm asking here because don't have the identical drives available to test my theory beforehand. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Webserver mirroring or web cluster tools
On Friday 30 September 2005 05:11 pm, Radhika wrote: > Hi, > > We are going to host the webserver for our ISP and we need to install > high availability.So is there any free webcluster software`with more > reliability and good performance or free mirroring software to > configure this.I am planning to install this on debian linux. if you want active/passive type cluster (HA, not load-balanced), use 'heartbeat' and 'stonith' and use apache on the machines. if you can make do with non-realtime replication of your web files, use rsync. if you need realtime replication, look into the kernel module drbd. use module-assistant to help you compile/install it if compiling kernels is not your thing. drbd is a 'RAID over TCP/IP' and has been working quite well for me. for a web server, use apache of course. configure it identically on both machines. etc.. - anoop. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Webserver mirroring or web cluster tools
Hi, We are going to host the webserver for our ISP and we need to install high availability.So is there any free webcluster software`with morereliability and good performance or free mirroring software to configure this.I am planning to install this on debian linux. Thanks for your help Yahoo! for Good Click here to donate to the Hurricane Katrina relief effort.
