Re: [vdr] mdadm software raid5 arrays?
What about a simple raid 1 mirror set? - Original Message - From: H. Langos henrik-...@prak.org To: VDR Mailing List vdr@linuxtv.org Sent: Tuesday, November 10, 2009 6:49 AM Subject: Re: [vdr] mdadm software raid5 arrays? Hi Simon, On Sat, Nov 07, 2009 at 07:38:03AM +1300, Simon Baxter wrote: Hi I've been running logical volume management (LVMs) on my production VDR box for years, but recently had a drive failure. To be honest, in the ~20 years I've had PCs in the house, this is the first time a drive failed! Anyway, I've bought 3x 1.5 TB SATA disks which I'd like to put into a software (mdadm) raid 5 array. ... I regularly record 3 and sometimes 4 channels simultaneously, while watching a recording. Under regular LVM, this sometimes seemed to cause some slow downs. I know I risk a flame war here but I feel obliged to say it: Avoid raid5 if you can avoid it! It is fun to play with but if you care for your data buy a fourth drive and do raid1+0 (mirroring and striping) instead. Raid 5 is very fast on linear read operations because basically the load will be spread onto all the available drives. But if you are going to run vdr on that drive array, you are going to do a lot of write operations, and raid5 is bad if you do a lot of writes for a very simple reason. Take a raid5 array with X devices. If you want to write just one block, you need to read 2 blocks (the old data that you are going to overwrite and the old parity) and you need to write 2 blocks (one with the actual data and one with the new parity). In the best of case, the disk block that you are going to overwrite is already in ram, but the parity block almost never will be. Only if you keep writing the same block over and over, you'll have data and parity blocks cached. In most cases (and certainly in the case of writing data streams on disk) you'll need to read two blocks before you can calculate the new parity and write it back to the disks along with your data. So in short you do two reads and two writes for every write operation. There goes your performance... Now about drive failures... if one of X disks fails, you can still read blocks on the OK drives with just one read operation but you need X-1 read operations for every read operation on the failed drive. Writes on OK drives have the same two reads/two writes as before, (only if the failed drive contained the parity for this block you can skip the additional two reads and one write). If however you need to write on the the failed drive, then you need to read every other X-1 drive in the array to first reconstruct the missing data and then you can calculate and write the new parity. (and then you throw away the actual data that you were going to write because the drive that you could write it to is gone...) Example: You have your three 1.5TB drives A B C in an array and C fails. In this situation you'd want to treat your drives as carefully as possible because one more failure and all your data is gone. Unfortunately continued operating in fail condition will put your remaining drives under much more stress than usually. Reading will cause twice the read operations on your remaining drives. block: n n+1 n+2 OK State : a b c Failstate: a b ab Writing (on a small array) will produce the same load of two reads and two writes average per write. block: n n+1n+2 OK:acAC baBA cbCB FAIL: A baBA baB Confusingly enough the read load per drive doesn't change if you have more than three drives in your array. Reads will still produce on average double the load in failed state. Writes on a failed array seem to produce the same load as on an OK array. But this is only true for very small arrays. If you add more disks you'll see that the read penalty grows for writing blocks where the data disk is missing and you need to read all other drives in order to update th parity. Reconstruction of you array after adding a new drive will take a long time and most of complete array failures (i.e. data lost forever) occure during the rebuilding phase, not during in the fail state. Thats simply because you put a lot of stress on your drives (that probably come from same batch as the one that already failed). Depending on the number and nature of your drives and the host connection they have, the limiting factor can read performance (you need to read X-1 drives completely) or it can be the write performance if your disk is slower on sustained writing than on reading. Remember that you need to read and write a whole disks worth of data, not just the used parts. Example: Your drives have 1.5tb and we assume that you have a whoopin 100MB/s on read as well as on write. (pretty much the fastest there currently is). You need to read 3tb as well as write 1.5tb. if your system can handle the load in parallel you can treat it as just writing one 1.5tb drive. 150mb/100mb/s/60s/m makes 250 minutes or 4 hours and 10 minutes. I am curious if you
Re: [vdr] strange issue with DVB-S2 and VDPAU
your channels list is wrong wrong modulation - should be everytime S1 and O35 or O25 The channels.conf was as vdr generated it, but I tried different modifications, so maybe that time I changed the modulation, but it didn't helped. read please man vdr about this parameters Believe me, I read it very often, and I had read about channels.conf parameters not in the vdr manual but on the german wiki. please try to use roll-off factor = 0,35 (as here - O35) I tried. I also installed vdr-reelchannelscan plugin, it generated a new channels.conf, but my image remanined broken, and log was full with TS continuity error. One thing: I was wrong about that single FTA radio channel from Hello HD packet: it's also not working, I can hear only some glitches if I tune to. I will reinstall Ubuntu 9.04 on this weekend, and I hope that will help. thanks for your quick answers. István -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] strange issue with DVB-S2 and VDPAU
I will reinstall Ubuntu 9.04 on this weekend, and I hope that will help. This is never a good idea, this way you will never find out the reason for the problem. So there is no warranty that it will not happen again. Gerald ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] strange issue with DVB-S2 and VDPAU
On Tue, 10 Nov 2009, Gerald Dachs wrote: I will reinstall Ubuntu 9.04 on this weekend, and I hope that will help. This is never a good idea, this way you will never find out the reason for the problem. So there is no warranty that it will not happen again. Gerald I agree. But I miss my vdr box, and for watching television I need to use that Kathrein box, which I don't like :) -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] mdadm software raid5 arrays?
On Tue, Nov 10, 2009 at 09:46:52PM +1300, Simon Baxter wrote: What about a simple raid 1 mirror set? Ok.. short comparison, using a single disk as baseline. using 2 disks raid0: (striping) ++ double read throughput, ++ double write throughput, -- half the reliability (read: only use with good backup!) raid1: (mirroring) ++ double read throughput. osame write throughput ++ double the reliability using 3 disks: raid0: striping +++ tripple read performance +++ tripple write performance --- third of reliability raid1: mirroring +++ tripple read performance osame write throughput +++ tripple reliability raid5: (distributed parity) +++ tripple read performance -lower write performance (not due to the second write but due to the necessary reads) +sustains failure of any one drive in the set using 4 disks: raid1+0: four times the read performance ++ double write performance ++ double reliability please note: these are approximations and depending on your hardware they may be off by quite a bit. cheers -henrik ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
[vdr] vdr-rotor-0.1.5
http://home.vrweb.de/~bergwinkl.thomas/downro/vdr-rotor-0.1.5.tgz vdr-rotor for vdr 179 ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] [OT] ARM based devices for VDR server and client
On Tue, Nov 3, 2009 at 7:49 PM, Paul Menzel paulepan...@users.sourceforge.net wrote: Am Dienstag, den 03.11.2009, 09:49 +0200 schrieb Michael Stepanov: IMHO nVidia Ion is much better option. Did you try both options or is this just your point of view from reading the specs? Especially for HDTV. As far as I know the Beagle Board is supposed to play 720p without any problems. See for example [1]. The NVIDIA ION is supposed to do 1080p with VDPAU, but I think I do not need it yet. + The Beagle Board should use less power. - More people run and test x86 hardware. • Both use proprietary drivers. Agree with you. But 720p on the Beagle Board is a hack while nVidia Ion is reality ;) In any case this is just my opinion. You can find easily case, MB with different options such external PSU, PCI extension or ready made HTPC based on nVidia Ion. Regarding the power consumption. I don't think that the difference between Ion and Beagle Board is extremely big. More efficient way to reduce energy consumption is replace all kitchen appliances to energy save ones, use thermostat to control temperature, switch off unnecessary electrical devices and use energy saving lamps :) Thanks, Paul [1] http://www.youtube.com/watch?v=BdnDpH3543Q ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr -- Cheers, Michael ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] Hulu for Linux
Hi Timothy, You may try LinuxMCE - http://linuxmce.com, which includes VDR, MythTV and now Hulu :) Or you can try XBMC+VDR On Sat, Oct 10, 2009 at 6:58 PM, Timothy D. Lenz tl...@vorgon.com wrote: Hulu now has a linux interface. And it seems MythTv can use it. what about vdr? Can it be done without installing a desktop? http://www.nvnews.net/vbulletin/showthread.php?t=139868 ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr -- Cheers, Michael ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] strange issue with DVB-S2 and VDPAU
wrong modulation - should be everytime S1 and O35 or O25 The channels.conf was as vdr generated it, but I tried different modifications, so maybe that time I changed the modulation, but it didn't helped. read please man vdr about this parameters Believe me, I read it very often, and I had read about channels.conf parameters not in the vdr manual but on the german wiki. please try to use roll-off factor = 0,35 (as here - O35) I tried. I also installed vdr-reelchannelscan plugin, it generated a new channels.conf, but my image remanined broken, and log was full with TS continuity error. One thing: I was wrong about that single FTA radio channel from Hello HD packet: it's also not working, I can hear only some glitches if I tune to.. what about of LOCK - do you have permanent stable LOCK ? have you any figures concerning of SNR BER ? Goga ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] Hulu for Linux
On Tuesday 10 November 2009, Michael Stepanov wrote: You may try LinuxMCE - http://linuxmce.com, which includes VDR, MythTV and linuxmce.org -- Wolfgang ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] mdadm software raid5 arrays?
Thanks - very useful! So what I'll probably do is as follows... * My system has 4x SATA ports on the motherboard, to which I'll connect my 4x 1.5TB drives. * Currently 1 drive is in use with ~30G for / /boot and swap and ~1.4TB for /media * I'll create /dev/md2, using mdadm, in RAID1 across 2 ~1.4TB partitions on 2 drives * move all active recordings (~400G) to /dev/md2 * split /dev/md2 and create a raid 1+0 (/dev/md1) using 4x partitions of ~1.4TB across 4 drives At this point I have preserved all my data, and created a raid1+0 for recordings and media. I should now use the remaining ~100G on each drive for raid protection for (root) / and /boot. I've read lots on the web on this, but what's your recommendation? RAID1 mirror across 2 of the disks for / (/dev/md0) and install grub (/boot) on both so either will boot? On Tue, Nov 10, 2009 at 09:46:52PM +1300, Simon Baxter wrote: What about a simple raid 1 mirror set? Ok.. short comparison, using a single disk as baseline. using 2 disks raid0: (striping) ++ double read throughput, ++ double write throughput, -- half the reliability (read: only use with good backup!) raid1: (mirroring) ++ double read throughput. osame write throughput ++ double the reliability using 3 disks: raid0: striping +++ tripple read performance +++ tripple write performance --- third of reliability raid1: mirroring +++ tripple read performance osame write throughput +++ tripple reliability raid5: (distributed parity) +++ tripple read performance -lower write performance (not due to the second write but due to the necessary reads) +sustains failure of any one drive in the set using 4 disks: raid1+0: four times the read performance ++ double write performance ++ double reliability please note: these are approximations and depending on your hardware they may be off by quite a bit. cheers -henrik ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] Hulu for Linux
Both are correct ;) On Tue, Nov 10, 2009 at 7:25 PM, Wolfgang Rohdewald wolfg...@rohdewald.dewrote: On Tuesday 10 November 2009, Michael Stepanov wrote: You may try LinuxMCE - http://linuxmce.com, which includes VDR, MythTV and linuxmce.org -- Wolfgang ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr -- Cheers, Michael ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] mdadm software raid5 arrays?
Hello Simon, what you also can do is to create the two RAID1 md devices with missing disks, e.g.: mdadm --create /dev/md2 --level=1 --raid-disks=2 missing /dev/sdb3 mdadm --create /dev/md3 --level=1 --raid-disks=2 missing /dev/sdd3 mdadm --create /dev/md1 --level=0 --raid-disks=2 /dev/md2 /dev/md3 Then you can create a filesystem on /dev/md1 and mount it, move all your recordings to that filesystem and lateron you can add the other two partitions to your RAID1 sets: mdadm --add /dev/md2 /dev/sda3 mdadm --add /dev/md3 /dev/sdc3 This way you don't have to split anything. You can just setup the two RAID1 arrays with one only one drive and one drive missing. All of my systems (except of VDR because it currently only has one disk) has a mirrored / and grub installed on both disks. I don't know if you can read German or if a Google translation of the following pages is usable but it might help you to get the correct keywords for a Google search: http://linuxwiki.de/mdadm http://www.howtoforge.de/howto/software-raid1-auf-einem-laufenden-system-inkl-grub-konfiguration-debian-etch-einrichten/ Best regards, Jogi On Wed, Nov 11, 2009 at 07:48:20AM +1300, Simon Baxter wrote: Thanks - very useful! So what I'll probably do is as follows... * My system has 4x SATA ports on the motherboard, to which I'll connect my 4x 1.5TB drives. * Currently 1 drive is in use with ~30G for / /boot and swap and ~1.4TB for /media * I'll create /dev/md2, using mdadm, in RAID1 across 2 ~1.4TB partitions on 2 drives * move all active recordings (~400G) to /dev/md2 * split /dev/md2 and create a raid 1+0 (/dev/md1) using 4x partitions of ~1.4TB across 4 drives At this point I have preserved all my data, and created a raid1+0 for recordings and media. I should now use the remaining ~100G on each drive for raid protection for (root) / and /boot. I've read lots on the web on this, but what's your recommendation? RAID1 mirror across 2 of the disks for / (/dev/md0) and install grub (/boot) on both so either will boot? On Tue, Nov 10, 2009 at 09:46:52PM +1300, Simon Baxter wrote: What about a simple raid 1 mirror set? Ok.. short comparison, using a single disk as baseline. using 2 disks raid0: (striping) ++ double read throughput, ++ double write throughput, -- half the reliability (read: only use with good backup!) raid1: (mirroring) ++ double read throughput. osame write throughput ++ double the reliability using 3 disks: raid0: striping +++ tripple read performance +++ tripple write performance --- third of reliability raid1: mirroring +++ tripple read performance osame write throughput +++ tripple reliability raid5: (distributed parity) +++ tripple read performance -lower write performance (not due to the second write but due to the necessary reads) +sustains failure of any one drive in the set using 4 disks: raid1+0: four times the read performance ++ double write performance ++ double reliability please note: these are approximations and depending on your hardware they may be off by quite a bit. cheers -henrik ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] strange issue with DVB-S2 and VDPAU
to upgrade. After the upgrade I had no sound at all on my system. As I saw, many others had this problem with Ubuntu 9.10. After a day and a half I managed to fix the sound (compiling alsa, etc), but the TS continuity error came back on vdr. Is ubuntu using Pulse Audio volume control? In many cases I like pulse way of allowing to manage and switch between multiple sound cards inputs and outputs even application by application but at least with Mandriva the old configs have been lost couple of times and I have needed to run both the alsaconfig and pulseaudio config for my system to get things working fine. I use spdif output and couple of times after system updates I have not had any sounds available. In those cases I have needed to: 1)alsamixer -c (-- unmute spdif output) 2)pulse-audio config (select spdif for default output channel instead of hdmi and check from the application view that apps running realized the config change. 3) check that my denon amplifier realized the config change by pressing auto button that forces it's to check both the digital and analog input... Mika ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] Hulu for Linux
On Tuesday 10 November 2009, Michael Stepanov wrote: Both are correct strange. This afternoon konqueror told me the domain linuxmce.com does not exist. Now it does. Probably my mistake. -- Wolfgang ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] vdr-rotor-0.1.5
Goga777 ha scritto: http://home.vrweb.de/~bergwinkl.thomas/downro/vdr-rotor-0.1.5.tgz vdr-rotor for vdr 179 ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr Hi, i upload here the italian translation. Bye, Diego Pierotto -- Member of the Italian VDR Wiki http://vdr.spaghettilinux.org/ rotor-0.1.5.tar.bz2 Description: application/bzip ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] Hulu for Linux
I already have debain linux/vdr setup. 64 bit on my system and 32 on my dads. I don't want to rip it all out and start over with another flavor of linux. Michael Stepanov wrote: Hi Timothy, You may try LinuxMCE - http://linuxmce.com, which includes VDR, MythTV and now Hulu :) Or you can try XBMC+VDR On Sat, Oct 10, 2009 at 6:58 PM, Timothy D. Lenz tl...@vorgon.com mailto:tl...@vorgon.com wrote: Hulu now has a linux interface. And it seems MythTv can use it. what about vdr? Can it be done without installing a desktop? http://www.nvnews.net/vbulletin/showthread.php?t=139868 ___ vdr mailing list vdr@linuxtv.org mailto:vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr -- Cheers, Michael ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] Hulu for Linux
On Wed, Nov 11, 2009 at 3:44 AM, Timothy D. Lenz tl...@vorgon.com wrote: I already have debain linux/vdr setup. 64 bit on my system and 32 on my dads. I don't want to rip it all out and start over with another flavor of linux. Then XBMC will help you ;) Here is one way to integrate VDR and XBMC using streamdev plugin - http://blog.mymediasystem.net/avchd/xbmc-on-karmic-with-vdpau-and-vdr/ There is also another way without streamdev, which is better as people talk. Michael Stepanov wrote: Hi Timothy, You may try LinuxMCE - http://linuxmce.com, which includes VDR, MythTV and now Hulu :) Or you can try XBMC+VDR On Sat, Oct 10, 2009 at 6:58 PM, Timothy D. Lenz tl...@vorgon.commailto: tl...@vorgon.com wrote: Hulu now has a linux interface. And it seems MythTv can use it. what about vdr? Can it be done without installing a desktop? http://www.nvnews.net/vbulletin/showthread.php?t=139868 ___ vdr mailing list vdr@linuxtv.org mailto:vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr -- Cheers, Michael ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr -- Cheers, Michael ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
