On 09.03.2010 15:25, Dan McGee wrote:
> We're bandwidth and rsync-slot constrained, not "slow at
> distribution". It seems like this solution is not solving a problem we
> currently have...
>
Actually that's being solved by multi tier mirroring and the more tiers
there are the slower the whole pro
On Tue, Mar 9, 2010 at 8:22 AM, Florian Pritz wrote:
> On 09.03.2010 14:37, Ionut Biru wrote:
>> On 03/09/2010 03:17 PM, Florian Pritz wrote:
>>> On 09.03.2010 02:20, keenerd wrote:
> On Mon, 8 Mar 2010 19:13:38 -0600, Dan McGee wrote
> On Mon, Mar 8, 2010 at 7:09 PM, keenerd wrote:
On 03/09/2010 03:17 PM, Florian Pritz wrote:
On 09.03.2010 02:20, keenerd wrote:
On Mon, 8 Mar 2010 19:13:38 -0600, Dan McGee wrote
On Mon, Mar 8, 2010 at 7:09 PM, keenerd wrote:
There does not seem to be a documented "standard rsync" command for
the mirrors to use, so I'm making all sorts of
On 09.03.2010 02:20, keenerd wrote:
>> On Mon, 8 Mar 2010 19:13:38 -0600, Dan McGee wrote
>> On Mon, Mar 8, 2010 at 7:09 PM, keenerd wrote:
>> > There does not seem to be a documented "standard rsync" command for
>> > the mirrors to use, so I'm making all sorts of wild assumptions about
>> > what
On 09.03.2010 14:37, Ionut Biru wrote:
> On 03/09/2010 03:17 PM, Florian Pritz wrote:
>> On 09.03.2010 02:20, keenerd wrote:
On Mon, 8 Mar 2010 19:13:38 -0600, Dan McGee wrote
On Mon, Mar 8, 2010 at 7:09 PM, keenerd wrote:
> There does not seem to be a documented "standard rsync" co
On Mon, Mar 8, 2010 at 7:20 PM, keenerd wrote:
>> On Mon, 8 Mar 2010 19:13:38 -0600, Dan McGee wrote
>> On Mon, Mar 8, 2010 at 7:09 PM, keenerd wrote:
>> > There does not seem to be a documented "standard rsync" command for
>> > the mirrors to use, so I'm making all sorts of wild assumptions abo
> On Mon, 8 Mar 2010 19:13:38 -0600, Dan McGee wrote
> On Mon, Mar 8, 2010 at 7:09 PM, keenerd wrote:
> > There does not seem to be a documented "standard rsync" command for
> > the mirrors to use, so I'm making all sorts of wild assumptions about
> > what a mirror's rsync is doing.
> http://wiki
On Mon, Mar 8, 2010 at 7:09 PM, keenerd wrote:
> There does not seem to be a documented "standard rsync" command for
> the mirrors to use, so I'm making all sorts of wild assumptions about
> what a mirror's rsync is doing.
http://wiki.archlinux.org/index.php/Local_Mirror#The_synchronization_scrip
A lot of people have been getting bit by mirrors being out of sync.
Fundamentally, this comes down to a mirror's database tarball being
ahead or behind of the packages which actually exist on the mirror.
Rsync is made for bulk updating, but it is not atomic, and bad things
happen if you interact w
>> I wonder if something like http://www.coralcdn.org/ could be used for
>> a package repository.
>
> Nope. The CoralCDN is intended to be used as a distributed web cache.
> It doesn't even serve large files, it redirects you to the original
> source:
I was thinking about using the technology not
On Sun, Feb 7, 2010 at 3:28 AM, Damjan Georgievski wrote:
> I wonder if something like http://www.coralcdn.org/ could be used for
> a package repository.
Nope. The CoralCDN is intended to be used as a distributed web cache.
It doesn't even serve large files, it redirects you to the original
sourc
On Sat, Feb 6, 2010 at 21:20, Gaurish Sharma wrote:
> Hi,
> I suggest using A Download Redirector and Metalink Generator like
> mirror brain(http://mirrorbrain.org/) to reduce work load on main
> ArchLinux Server.
I wonder if something like http://www.coralcdn.org/ could be used for
a package rep
Hi,
I suggest using A Download Redirector and Metalink Generator like
mirror brain(http://mirrorbrain.org/) to reduce work load on main
ArchLinux Server.
Also, we should have few other official mirrors apart from al.org from
which mirrors of rest of the world would sync. essentially spreading
out
Hi,
PS: A mailinglist for mirror stuff (like this discussion) with all
mirror admins would also be quite nice.
As a mirror admin (mirrors.rit.edu) , I second that request, although
perhaps one low traffic list (mandatory script updates, bulletins,
etc) and one for discussions?
Or perhaps some
On Thu, Feb 4, 2010 at 15:57, Florian Pritz wrote:
> On 02/04/2010 09:42 PM, Lee Burton wrote:
>> It probably is. Perhaps a push-primary solution (much simpler..)
>> combined with a default twice a day sync (just to make sure?) for
>> tier-1 mirrors might work.. the I believe point here is to get
On 02/04/2010 09:42 PM, Lee Burton wrote:
> It probably is. Perhaps a push-primary solution (much simpler..)
> combined with a default twice a day sync (just to make sure?) for
> tier-1 mirrors might work.. the I believe point here is to get ideas
> out there.
I'd go for arch master -> mirror with
On Thu, Feb 4, 2010 at 14:40, Dieter Plaetinck wrote:
> On Thu, 4 Feb 2010 14:27:14 -0500
> Lee Burton wrote:
>
>> To make it "multi-tiered" and to reduce load on the primary mirror
>> could have slightly more intelligent polling than just checking one
>> upstream machine.
>> In this example Let:
2010/2/4 Dieter Plaetinck :
> seems needlessly complex to me.
> Dieter
But the current way is not the best doable. We need another(a better) solution!
--
Gruß, Benedikt
On Thu, 4 Feb 2010 14:27:14 -0500
Lee Burton wrote:
> To make it "multi-tiered" and to reduce load on the primary mirror
> could have slightly more intelligent polling than just checking one
> upstream machine.
> In this example Let:
> Primary = Arch Primary Mirror/Mirrors (updated directly by th
On Wed, Feb 3, 2010 at 10:53, Florian Pritz wrote:
> On 02/03/2010 03:12 PM, Lee Burton wrote:
>> As for push mirroring, http://www.debian.org/mirror/push_server is a
>> decent example
>> An identity file with
>> no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,command="/path/to/mir
On 02/03/2010 03:12 PM, Lee Burton wrote:
> As for push mirroring, http://www.debian.org/mirror/push_server is a
> decent example
> An identity file with
> no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,command="/path/to/mirror/script",from="IPADDRESS"
> &"
> Is fairly decent..
I'v
This order can be accomplished by first running rsync without the delete flag.
Then rsync over the DB.
Then re-run the original rsync with --delete or --delete-after.
You could also google for 'atomic rsync'
First hit is
http://www.opensource.apple.com/source/rsync/rsync-35.2/rsync/support/atomic-
Hi Allan,
I am holding the upgrade till now on your advice.
Total Download Size:679.66 MB
Total Installed Size: 1811.77 MB
Is it safe to Upgrade now?
Regards,
Gaurish Sharma
www.gaurishsharma.com
On Tue, Feb 2, 2010 at 8:34 AM, Allan McRae wrote:
> On 02/02/10 13:01, Steve Holmes wr
Looks really nice, Florian ;]
I wouldn't dare to analyse the syntax, as I only know the basics,
but the idea is pretty neat.
--
Guilherme M. Nogueira
"Any sufficiently advanced technology is indistinguishable from magic."
- Arthur C. Clarke
On 02/03/2010 01:16 AM, Florian Pritz wrote:
> For those who are interested I've attached the current sync script. It
> might change in future though.
Seems it got lost :(
http://karif.server-speed.net/~flo/tmp/mirrorsync.sh.txt
--
Florian Pritz -- {flo,bluewi...@server-speed.net
I'm working on a mirrorscript that can be run as often as you want to.
Even every minute.
In short: The script fetches a md5sum of the databases and if one
database has changed it will start rsync to resync that particular repo.
The md5 it fetches is small, static and will cause nearly no load, bu
Damjan Georgievski writes:
>>> >>> - download new packages
>>> >>> - update db
>>> >>> - delete old packages
>>> >>>
>>>
>>> from http://www.debian.org/mirror/ftpmirror#how
>>> ...
>>> * MUST perform a 2-stage sync
>>> ...
>>> Rationale: if archive mirroring is done in a single stage, there will
On Tue, Feb 2, 2010 at 7:41 PM, Damjan Georgievski wrote:
>
> >> >>> - download new packages
> >> >>> - update db
> >> >>> - delete old packages
> >> ...
> > I must be missing something.. isn't --delete-after good enough?
>
> you are missing the fact that it will download the database file
> befor
>> >>> - download new packages
>> >>> - update db
>> >>> - delete old packages
>> >>>
>>
>> from http://www.debian.org/mirror/ftpmirror#how
>> ...
>> * MUST perform a 2-stage sync
>> ...
>> Rationale: if archive mirroring is done in a single stage, there will
>> be periods of time
>> during which t
On Tue, 2 Feb 2010 20:32:20 +0100
Dieter Plaetinck wrote:
> On Tue, 2 Feb 2010 14:06:35 -0500
> Andrew Antle wrote:
>
> > On Tue, Feb 2, 2010 at 1:53 PM, Hannes Rist
> > wrote:
> > > Ionut Biru wrote:
> > >>
> > >> On 02/02/2010 07:53 PM, Damjan Georgievski wrote:
> > >
> > > There's a
On Tue, 2 Feb 2010 14:06:35 -0500
Andrew Antle wrote:
> On Tue, Feb 2, 2010 at 1:53 PM, Hannes Rist wrote:
> > Ionut Biru wrote:
> >>
> >> On 02/02/2010 07:53 PM, Damjan Georgievski wrote:
> >
> > There's also the problem that some mirrors (most of the ones
> > I've tried) sync the p
On Tue, Feb 2, 2010 at 1:53 PM, Hannes Rist wrote:
> Ionut Biru wrote:
>>
>> On 02/02/2010 07:53 PM, Damjan Georgievski wrote:
>
> There's also the problem that some mirrors (most of the ones I've
> tried) sync the package database before syncing all the packages.
Actually, s
Ionut Biru wrote:
On 02/02/2010 07:53 PM, Damjan Georgievski wrote:
There's also the problem that some mirrors (most of the ones I've
tried) sync the package database before syncing all the packages.
Actually, syncing the db last is not going to improve things: if some
packages get deleted, th
On 02/02/2010 07:53 PM, Damjan Georgievski wrote:
There's also the problem that some mirrors (most of the ones I've
tried) sync the package database before syncing all the packages.
Actually, syncing the db last is not going to improve things: if some
packages get deleted, they won't be found w
>> There's also the problem that some mirrors (most of the ones I've
>> tried) sync the package database before syncing all the packages.
>
> Actually, syncing the db last is not going to improve things: if some
> packages get deleted, they won't be found when updating against the
> old db.
- down
Excerpts from Allan McRae's message of 2010-02-02 09:24:47 +0100:
> On 02/02/10 18:09, Benoit Favre wrote:
> > In any case, there should be more communication towards users about
> > what's really going on.
>
> Like posting a message saying not to update on the front page? That
> would have been
On 02/02/10 18:09, Benoit Favre wrote:
In any case, there should be more communication towards users about
what's really going on.
Like posting a message saying not to update on the front page? That
would have been nice...
Allan
On Tue, Feb 2, 2010 at 12:02 AM, Damjan Georgievski wrote:
> There's also the problem that some mirrors (most of the ones I've
> tried) sync the package database before syncing all the packages.
Actually, syncing the db last is not going to improve things: if some
packages get deleted, they won't
Allan McRae wrote:
Well, there was a news item saying it would be best to wait a couple of
days to do an update... but no-one ever listens to us.
And yet they're still using Archlinux. True love!
regards,
Hannes
--
Hannes Rist
++
| Crew Se
On 02/02/10 13:01, Steve Holmes wrote:
On Tue, Feb 02, 2010 at 04:42:32AM +0200, Ionut Biru wrote:
mirrors.kernel.org in fact is not a single mirror. is an alias to a
geolocation subdomain and from there is serving from closer
geographically position(in theory).
for you maybe you hit in an up to
On Tue, Feb 02, 2010 at 04:42:32AM +0200, Ionut Biru wrote:
> mirrors.kernel.org in fact is not a single mirror. is an alias to a
> geolocation subdomain and from there is serving from closer
> geographically position(in theory).
> for you maybe you hit in an up to date server.
I don't know but I'
On 02/02/2010 04:29 AM, Ray Kohler wrote:
On Mon, Feb 1, 2010 at 9:01 PM, Steve Holmes wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160
I'm also having major problems upgrading my system. Pacman errors out
and it tells me I have 77 packages to update and I was current two
days ago.
On Mon, Feb 1, 2010 at 9:01 PM, Steve Holmes wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: RIPEMD160
>
> I'm also having major problems upgrading my system. Pacman errors out
> and it tells me I have 77 packages to update and I was current two
> days ago. I believe it is 77 and counting.
-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160
I'm also having major problems upgrading my system. Pacman errors out
and it tells me I have 77 packages to update and I was current two
days ago. I believe it is 77 and counting. Last night it was 66.
I'm using the kernel.org site for my pack
> We have a bit update today, and we see: The syncing process is not
> really good.
There's also the problem that some mirrors (most of the ones I've
tried) sync the package database before syncing all the packages.
So "pacman -Syu" errors-out because it can't download some packages.
So I either
On Sun, 31 Jan 2010 17:36:50 +0100
Lukáš Jirkovský wrote:
> No, what I meant was that difference between having package pool to
> which packages are linked and sending some text file to all servers
> saying "Hi, please move package foo-1.2.3-1-i686.pkg.tar.gz from
> [testing] to [core]" which wou
On Sun, 31 Jan 2010 17:24:22 +0100
Lukáš Jirkovský wrote:
> I didn't understand what you meant first time. I think I got it now.
> If I understand it well you mean having all packages in one directory
> on server and the repos would be differentiated by some text files or
> symlinks. The differe
On 31 January 2010 17:15, Pierre Schmitz wrote:
> Am Sonntag, 31. Januar 2010 17:14:05 schrieb Lukáš Jirkovský:
>> I think that the syncing would be much less painful if there was some
>> possibility to tell mirrors that package foo has been moved from
>> [testing] to [extra]. Then these rebuilds
Am Sonntag, 31. Januar 2010 17:14:05 schrieb Lukáš Jirkovský:
> I think that the syncing would be much less painful if there was some
> possibility to tell mirrors that package foo has been moved from
> [testing] to [extra]. Then these rebuilds would be only a matter of
> distributing information w
On 31 January 2010 17:05, Hannes Rist wrote:
> Hi,
>
>> There are several methods to improve the situation:
>> * multi tier mirroring. Roman started to work on this but might need
> some help
>> here. It's mostly an organizing task
>
> I strongly second that. Having a geographically organized hier
Hi,
> There are several methods to improve the situation:
> * multi tier mirroring. Roman started to work on this but might need
some help
> here. It's mostly an organizing task
I strongly second that. Having a geographically organized hierarchy would
be nice, so that there are tier-1 mirrors in
Am Sonntag, 31. Januar 2010 15:27:03 schrieb Dan McGee:
> Thanks for signing that message, I wasn't sure it was from you.
OT: Can't we strip gpg-signatures from the mailinglist? It's of no use. Use
s/mime instead ;-)
> The problem here is we haven't had anyone step up and finish a two
> tier mir
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 01/31/2010 03:27 PM, Dan McGee wrote:
> As far as pushing goes, that is a bad idea for a number of reasons,
> the primary being one compromised root server gains you ssh access to
> X more servers.
Can be solved easily by using forced commands:
http
On 01/31/2010 04:30 PM, Benedikt Müller wrote:
2010/1/31 Dan McGee:
As far as pushing goes, that is a bad idea for a number of reasons,
the primary being one compromised root server gains you ssh access to
X more servers.
-Dan
I didn't say that it must be root. One user with the only permissi
2010/1/31 Dan McGee :
> As far as pushing goes, that is a bad idea for a number of reasons,
> the primary being one compromised root server gains you ssh access to
> X more servers.
>
> -Dan
>
I didn't say that it must be root. One user with the only permission
to use rsync would be the right for t
> We have a bit update today, and we see: The syncing process is not
> really good. So I suggest to change the procedure mirrorsyncs are
> done: We should have primary and secondary mirrors. When al.org is
> updated, the sync process of the primary mirrors should be started via
> ssh(or something s
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
We have a bit update today, and we see: The syncing process is not
really good. So I suggest to change the procedure mirrorsyncs are
done: We should have primary and secondary mirrors. When al.org is
updated, the sync process of the primary mirrors sho
57 matches
Mail list logo
