Re: Mirror SL repository with pulp

[Martin Surovcak]

On Wed, Jul 24, 2013 at 11:18:52AM +0200, Axel Bock wrote:
 Hello list,
 
 first a little introduction: I am as of now a Sysadmin in a German finance
 organisation, and I am responsible for the company's Linux-Systems which
 run on SL.
 
 As such I want to mirror the current 6.4 repository to our internal pulp
 server - and I fail. I now found a list post saying that the repomd.xml was
 broken, and just wanted to ask whether this is still the case.
 
 If not I would appreciate any help of people who did that already, so I can
 shamelessly copy their approach :) .
 
 Thanks in advance for any answers!
 
 /Axel.
Hi Axel,

we're using Pulp to mirror SL6 repositories too (also EPEL). Pulp is also used 
to handle our internal repositories.

There was an issue with SL's checksums in repomd.xml file but this has been 
solved already.

There is a documentation available which I followed during building our mirror.
http://pulp-user-guide.readthedocs.org/en/latest/index.html
http://pulp-rpm-user-guide.readthedocs.org/en/latest/index.html

When you have pulp up and running adding a repo to sync is trivial:

pulp-admin rpm repo create 
--feed=http://mirror.itc.virginia.edu/fedora-epel/6/x86_64/ --repo-id=epel6 
--relative-url=epel/6/x86_64/ --serve-http=true

pulp-admin rpm repo sync schedules create --schedule 2013-04-25T01:00:00Z/P1D 
--repo-id=epel6

This will create mirror for EPEL6 and sync it once a day.

As you heaven't mentioned your troubles I suggest you reading and following 
docs and/or specify your problem, I'll be happy to help.

Martin

Re: [SCIENTIFIC-LINUX-USERS] Upgrade SLC5 to SLC6 ?

[Tom H]

On Tue, Jul 23, 2013 at 9:02 PM, Nico Kadel-Garcia nka...@gmail.com wrote:
 On Tue, Jul 23, 2013 at 8:56 PM, Jeffrey Anderson jdander...@lbl.gov wrote:

 The message I get from the official voices in this thread is that there is
 no supported method of upgrading major versions, only minor point versions.
 It's no big deal. I just did a fresh install, then some cfengine magic and
 am back up and running in a couple hours. I was just surprised because
 major version upgrades have been available for every version of SL and TUV
 for the past 15+ years, and when the install media did not give me that
 option here I wanted to make sure I wasn't overlooking something.

 I'm glad for you, and startled myself. Our favorite upstream vendor
 certainly supported doing updates from major OS versions to major OS
 versoins: you just couldn''t gracefully do it *live, because changing
 things like major versions of glibc and rpm while you're in the midst
 of using them to do the update is.. intriguingly problematic.
 (Tried it once: don't recommend it!)

Fedup solves the live problem by performing the upgrade from a custom initramfs.

(I've been upgrading to rawhide for a few years by upgrading
rpm/yum/python then kernel/glibc then init/udev then everything else
but I haven't checked the Fedora wiki for a LONG time to see whether
this is still the approved method. They seem to be OTT because every
step seems to pull in more and more dependencies with every new Fedora
version!)

RE: Large filesystem recommendation

[Paul Robert Marino]

Although that said EXT4 is still an inode centric file system with a journal added so moving the journal to a faster volume wont have as big an effect as it does on ground up designed journaling file systems. So while that feature may speed up the journal for EXT4 its still limited by the speed of the in-filesystem inodes regardless of where the journal is located.The difference being XFS, JFS, ZFS, and a few others primarily rely on the journal and write the inboxes as needed after the fact for backwards compatibility for older low level binaries. XFS also uses them with the xfsrepair tool as a DR backup incase the very rare casein the journal getting corrupted (usually due to a hardware issue like raid controller backplane melt down) but even in that case XFS only thin-provisions creates the inodes it reallyneeds the first time they are written to. Which is why themkfs.xfs tool is so fast.EXT4 still pre-allocates all of the possible inodes during formating and writes to the inodes before the journal-- Sent from my HP Pre3On Jul 25, 2013 1:17, Paul Robert Marino prmari...@gmail.com wrote: That's cool I've never noticed that I the documentation but I'll look for it.-- Sent from my HP Pre3On Jul 24, 2013 18:41, Scott Weikart scot...@benetech.org wrote: 

 Though I will admit the being able to move your journal to a
 separate faster volume to increase performance is very cool
 and that's only a feature I've seen in XFS and ZFS.

ext4 supports that.

-scott


From: owner-scientific-linux-us...@listserv.fnal.gov owner-scientific-linux-us...@listserv.fnal.gov on behalf of Paul Robert Marino prmari...@gmail.com
Sent: Wednesday, July 24, 2013 3:36 PM
To: Brown, Chris (GE Healthcare); Graham Allan; John Lauro
Cc: scientific-linux-users
Subject: RE: Large filesystem recommendation


ZFS is a performance nightmare if you plan to export it via NFS because of a core design conflict with how NFS locking and the ZIL journal in ZFS. Its not just a linux issue it effects Solaris and BSD as well. My only experience with ZFS was on a Solaris
 NFS server and we had to get a dedicated flash backed ram drive for the ZIL to fix our performance issues, and let me tell you sun charged us a small fortune for the card.
Aside from that most of the cool features are available in XFS if you dive deep enough into the documentation though most of them like multi disk spanning can be handled now by LVM or MD but are at least in my opinion handled better by hardware raid. Though
 I will admit the being able to move your journal to a separate faster volume to increase performance is very cool and that's only a feature I've seen in XFS and ZFS.




-- Sent from my HP Pre3



On Jul 24, 2013 16:53, Brown, Chris (GE Healthcare) christopher.br...@med.ge.com wrote:


ZFS on Linux will provide you all the goodness that it brought to Solaris and BSD.

Check out: 
http://listserv.fnal.gov/scripts/wa.exe?A2=ind1303L=scientific-linux-usersT=0P=21739


http://listserv.fnal.gov/scripts/wa.exe?A2=ind1303L=scientific-linux-usersT=0P=21882


http://listserv.fnal.gov/scripts/wa.exe?A2=ind1307L=scientific-linux-usersT=0P=4752


- Chris 


-Original Message- 
From: owner-scientific-linux-us...@listserv.fnal.gov [mailto:owner-scientific-linux-us...@listserv.fnal.gov] On Behalf Of Graham Allan

Sent: Wednesday, July 24, 2013 3:46 PM 
To: John Lauro 
Cc: scientific-linux-users 
Subject: Re: Large filesystem recommendation 

XFS seems like the most obvious and maybe safest choice. FWIW, we use it on SL5 and SL6. Ultimately any issues we've had with it turned out to be hardware-related.


ZFS has some really nice features, and we are using it for larger filesystems than we have XFS, but so far only on BSD rather than Linux...


On Wed, Jul 24, 2013 at 01:59:03PM -0400, John Lauro wrote: 
 What is recommended for a large file system (40TB) under SL6? 
 
 In the past I have always had good luck with jfs. Might not be the 
 fastest, but very stable. It works well with being able to repair 
 huge filesystems in reasonable amount of RAM, and handle large 
 directories, and large files. Unfortunately jfs doesn't appear to be 
 supported in 6? (or is there a repo I can add?) 
 
 
 Besides for support of 40TB filesystem, also need support of files 4TB, and directories with hundreds of thousands of files. What do people recommend?


-- 
- 
Graham Allan 
School of Physics and Astronomy - University of Minnesota 
-