Re: Mirrorlist missing mirrors?

2006-03-21 Thread Jeff Spaleta
On 3/21/06, Avi Kivity  wrote:
> [...@firebolt ~]$ wget -q -O -
> http://fedora.redhat.com/download/mirrors/fedora-core-5
> http://download.fedoraproject.org/pub/fedora/linux/core/5/$ARCH/os/
>
> A few more mirrors would help reduce the load...

the file fedora-core-5-redirect is an actual mirrorlist file. 
Appearently there is suppose to be some serverside redirect magic
going on which is non-obvious from the client side yum configuration.

I've turned on verbose mode on the fastestmirror plugin I'm using to
see the list of mirrors which fastestmirror plugin is suppose to be
"timing" to make its judgement. And I'm not seeing the mirrors as
listed in the -redirect file in the fastestmirror output.  So from my
pov whatever magic on the server which is suppose to be redirecting
clients to use fedora-core-5-redirect mirrorlist isn't working afaict.

I'm not sure whom to address this problem to exactly. The redirect
issue affects both core and updates in the fc5 configs.

-jef


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Jeff Spaleta
On 3/21/06, Jeff Spaleta  wrote:
> I'm not sure whom to address this problem to exactly. The redirect
> issue affects both core and updates in the fc5 configs.


Of course I could be completely mis-interpreting how the redirect
magic works. The redirect stuff might be completely opaque to
clientside and we call all be redirected to other mirrors without
having to much in the way of clientside information that its
happening.

I'd personally love an explanation as to how the new mirror
redirection works with an eye on how to troubleshoot if there is a
problem or not without special access  from the clientside of
things.

-jef


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Ralph Angenendt
Jeff Spaleta wrote:
> On 3/21/06, Jeff Spaleta  wrote:
> > I'm not sure whom to address this problem to exactly. The redirect
> > issue affects both core and updates in the fc5 configs.
> 
> Of course I could be completely mis-interpreting how the redirect
> magic works. The redirect stuff might be completely opaque to
> clientside and we call all be redirected to other mirrors without
> having to much in the way of clientside information that its
> happening.

I don't think so. 

fastestmirror chooses download.fedora.redhat.com for me also, which
doesn't seem to be some round robin dns box. And there are mirrors in my
vicinity (Germany, Europe) which should supposedly be faster than
anything at *.redhat.com for me. 

And: If something times out while downloading, yum directly comes back
to me telling me, that it has run out of mirrors to try.

Regards,

Ralph
-- 
Ralph angenendt@br-online.de | .."Text processing has made it possible
Bayerischer Rundfunk...80300 München | to right-justify any idea, even one
Programmbereich.Bayern 3, Jugend und | .which cannot be justified on any other
Multimedia.Tl:089.5900.16023 | ..grounds." -- J. Finnegan, USC


pgpa9eKAHHFn9.pgp
Description: PGP signature


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Jeff Spaleta
On 3/21/06, Ralph Angenendt  wrote:
> I don't think so.

Or perhaps...the single mirror listed in the mirrorlist is acting as a
redirect at the http server level... redirecting you to some other
http mirror based on http server logic which we can't easily evaluate
via normal client interaction... essentially short-circuiting the
normal mirrorlist behavior we've grown used to in previous versions. 
I don't have a clear understanding as to expected behavior now and I
need an explanation from the mirror infrastructure developers as where
things stand. Something has certaintly appeared to have changed, but
without understanding the change it will be difficult to troubleshoot.

-jef


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Avi Kivity

Jeff Spaleta wrote:

On 3/21/06, Ralph Angenendt  wrote:
  

I don't think so.



Or perhaps...the single mirror listed in the mirrorlist is acting as a
redirect at the http server level... redirecting you to some other
http mirror based on http server logic which we can't easily evaluate
via normal client interaction... essentially short-circuiting the
normal mirrorlist behavior we've grown used to in previous versions. 
I don't have a clear understanding as to expected behavior now and I

need an explanation from the mirror infrastructure developers as where
things stand. Something has certaintly appeared to have changed, but
without understanding the change it will be difficult to troubleshoot.

  
no. we have a single mirror active, and doing a 'yum upgrade' is an 
exercise in futility as after 4-5 headers the mirror will reject the 
connection due to overload and yum will exit saying no more mirrors are 
available.


the corresponding file for FC4 has all the mirrors listed, someone just 
needs to go and update it.


--
error compiling committee.c: too many arguments to function


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Jesse Keating
On Tuesday 21 March 2006 07:00, Jeff Spaleta wrote:
> I'm not sure whom to address this problem to exactly. The redirect
> issue affects both core and updates in the fc5 configs.

download.fedoraproject.org is a redirector.  It uses the mirrors found in the 
-redirect version of the file (not sure if this is done automatically or 
somebody reads the -redirect file and updates some other list for 
redirection).  Elliot Lee is the person who is managing the redirector, 
perhaps he can chime in (I've cc'd him).  Theoretically each time you load a 
valid url from download.fedoraproject.org it would use a new mirror.  Firefox 
won't work for testing this, but using links again and again will show that 
it does indeed to go different hosts.

-- 
Jesse Keating
Release Engineer: Fedora


pgpY5bdgHlUZL.pgp
Description: PGP signature


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Robert Spanton
I think some of the fedora mirrors haven't caught up yet. Maybe someone's
waiting for them to catch up a little.

Rob Spanton

On 3/21/06, Avi Kivity  wrote:
>
> Jeff Spaleta wrote:
> > On 3/21/06, Ralph Angenendt  wrote:
> >
> >> I don't think so.
> >>
> >
> > Or perhaps...the single mirror listed in the mirrorlist is acting as a
> > redirect at the http server level... redirecting you to some other
> > http mirror based on http server logic which we can't easily evaluate
> > via normal client interaction... essentially short-circuiting the
> > normal mirrorlist behavior we've grown used to in previous versions.
> > I don't have a clear understanding as to expected behavior now and I
> > need an explanation from the mirror infrastructure developers as where
> > things stand. Something has certaintly appeared to have changed, but
> > without understanding the change it will be difficult to troubleshoot.
> >
> >
> no. we have a single mirror active, and doing a 'yum upgrade' is an
> exercise in futility as after 4-5 headers the mirror will reject the
> connection due to overload and yum will exit saying no more mirrors are
> available.
>
> the corresponding file for FC4 has all the mirrors listed, someone just
> needs to go and update it.
>
> --
> error compiling committee.c: too many arguments to function
>
> --
> fedora-devel-list mailing list
> fedora-devel-list@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-devel-list
>


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Jeff Spaleta
On 3/21/06, Jesse Keating  wrote:
> Theoretically each time you load a
> valid url from download.fedoraproject.org it would use a new mirror.  Firefox
> won't work for testing this, but using links again and again will show that
> it does indeed to go different hosts.

I'd love an example of a client that I can use to troubleshoot this
behavior without resorting to low level packet inspection. The
verbosity on fastestmirror plugin doesn't indicate that a redirect is
occuring.  And yum clean metadatayum -d6 list updates  does not
give me any information with regard to which mirror I'm being
redirected to when pulling primary.xml.gz or repomd.xml.gz
the yum -d6 output just lists the redirector url.  And now with the
change to the mirrorlist file... lists the same url repeatedly :->

What is the best procedure that I should be performing when following
up on complaints with regard to mirrorlist not redirecting as it
should?

-jef


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Jesse Keating
On Tuesday 21 March 2006 10:02, Jeff Spaleta wrote:
> I'd love an example of a client that I can use to troubleshoot this
> behavior without resorting to low level packet inspection. The
> verbosity on fastestmirror plugin doesn't indicate that a redirect is
> occuring.  And yum clean metadata    yum -d6 list updates  does not
> give me any information with regard to which mirror I'm being
> redirected to when pulling primary.xml.gz or repomd.xml.gz
> the yum -d6 output just lists the redirector url.  And now with the
> change to the mirrorlist file... lists the same url repeatedly :->
>
> What is the best procedure that I should be performing when following
> up on complaints with regard to mirrorlist not redirecting as it
> should?

Currently I think Elliot Lee is the pointman for this 'feature'.  I was asked 
to try this.  If it continues to fail, I will copy over the existing list of 
mirrors in -redirect to the actual mirror file that yum points to and avoid 
the redirector on download.fedoraproject.org.

I think a bug to fedora infrastructure?

-- 
Jesse Keating RHCE  (geek.j2solutions.net)
Fedora Legacy Team  (www.fedoralegacy.org)
GPG Public Key  (geek.j2solutions.net/jkeating.j2solutions.pub)


pgpJiPE8f2Z8A.pgp
Description: PGP signature


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Jeff Spaleta
On 3/21/06, Jesse Keating  wrote:
> Currently I think Elliot Lee is the pointman for this 'feature'.  I was asked
> to try this.  If it continues to fail,

Is it failing? Its very difficult to make a claim as to what is
failing and how, if competent troubleshouters can't reproduce behavior
with enough detail to pinpoint a problem. And right now the available
debugging information in yum isn't verbose enough to see the redirects
as they happen and tell the users about which mirror is being used.
This makes reproducibility and documentation of a problem...
difficult.

-jef


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Jesse Keating
On Tuesday 21 March 2006 10:27, Jeff Spaleta wrote:
> Is it failing? Its very difficult to make a claim as to what is
> failing and how, if competent troubleshouters can't reproduce behavior
> with enough detail to pinpoint a problem. And right now the available
> debugging information in yum isn't verbose enough to see the redirects
> as they happen and tell the users about which mirror is being used.
> This makes reproducibility and documentation of a problem...
> difficult.

What exactly do you want from me here?  This in itself could be considered a 
failure.  File a bug as such.  If you're getting repeated failures running 
yum, but pointing yum to the -redirect version of the mirrorlist file 
consistently works, this would indicate a failure in the way the redirect is 
working for yum.

-- 
Jesse Keating RHCE  (geek.j2solutions.net)
Fedora Legacy Team  (www.fedoralegacy.org)
GPG Public Key  (geek.j2solutions.net/jkeating.j2solutions.pub)


pgpJC6rr5lcv6.pgp
Description: PGP signature


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Jeff Spaleta
On 3/21/06, Jesse Keating  wrote:
> What exactly do you want from me here?

>From you in particular? That's a difficult question to answer because
I'm not sure in a position to provide me with what I'm seeking...
since you have already pointed out that I need to schedule a meeting
between Mr. Lee and the magic lasso of truth that I have on loan from
wonder woman.

> This in itself could be considered a
> failure.  File a bug as such.

I'm attempting to do my best to ensure than whatever bugs are filed
are not closed as worksforme because of lack of reproducibility or
consistency of behavior. I would like to be able to find a way to
produce an auditable trail of clientside output which unsuspecting
users can be directed to generate as needed per incident in an effort
to document what could very well be a sporadic symptoms in order to
pinpoint the if there is an underlying problem with the new redirect
service. People are already confused by the changes in the mirrorlist
file.

> If you're getting repeated failures running
> yum, but pointing yum to the -redirect version of the mirrorlist file
> consistently works, this would indicate a failure in the way the redirect is
> working for yum.

And if using -redirect directly also produces sporadic failures for
some people with similiar clientside symptoms?  There are other,
pre-existing issues which some users can run into.. things like
timeout settings which could be mis-diagnosed as problems associated
with the new redirect layer simply because the redirect process is new
and opaque from the client pov.

-jef


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Chris Adams
Once upon a time, Jesse Keating  said:
> download.fedoraproject.org is a redirector.  It uses the mirrors found in the 
> -redirect version of the file (not sure if this is done automatically or 
> somebody reads the -redirect file and updates some other list for 
> redirection).  Elliot Lee is the person who is managing the redirector, 
> perhaps he can chime in (I've cc'd him).  Theoretically each time you load a 
> valid url from download.fedoraproject.org it would use a new mirror.  Firefox 
> won't work for testing this, but using links again and again will show that 
> it does indeed to go different hosts.

Using a server-side redirector seems like a bad idea.  It completely
defeats the purpose of things like fastestmirror or even giving a user a
list of mirrors to choose from.
-- 
Chris Adams 
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Jeff Spaleta
On 3/21/06, Jeff Spaleta  wrote:
> And if using -redirect directly also produces sporadic failures for
> some people with similiar clientside symptoms?  There are other,
> pre-existing issues which some users can run into.. things like
> timeout settings which could be mis-diagnosed as problems associated
> with the new redirect layer simply because the redirect process is new
> and opaque from the client pov.


Very frustrating... how do I help users who are complaining about errors like..
http://download.fedoraproject.org/pub/fedora/linux/core/updates/5/i386/repodata/primary.xml.gz:
[Errno -1] Metadata file does not match checksum
Trying other mirror.
Error: failure: repodata/primary.xml.gz from updates: [Errno 256] No
more mirrors to try.

now that there is a serverside redirector in place.. how do I help
track down which mirror is out of sync so it can be reported
appropriate?

And appearantly even though there are multiple instances of the same
url in the mirrorlist, yum doesn't fail over to the next listing like
they are different mirrors like it use to when the mirrorlist was a
set of distinct mirrors. Instead the error occurs and yum bails like
there was only a single mirror in the list. The single repeated
redirecting URL appears to shortcircuit "mirrorlist" functionality.

-jef


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Jesse Keating
On Tuesday 21 March 2006 13:42, Jeff Spaleta wrote:
> Instead the error occurs and yum bails like
> there was only a single mirror in the list. The single repeated
> redirecting URL appears to shortcircuit "mirrorlist" functionality.

I am reverting to the old style of the complete mirror list.  The single 
redirector URL is not working as planned.  Please wait for the new mirror 
list files to make it to the live webserver.

-- 
Jesse Keating
Release Engineer: Fedora


pgpt4mN90v3pV.pgp
Description: PGP signature


RE: Mirrorlist missing mirrors?

2006-03-21 Thread Jeroen van Meeuwen

> I am reverting to the old style of the complete mirror list.  The single
> redirector URL is not working as planned.  Please wait for the new mirror
> list files to make it to the live webserver.

It seems to be yum that cannot cope with the redirects. Elliot Lee
recommended the fedora-users mailing list subscribers earlier today to leave
things as they are, but there's no estimated time for a solution.

Jef Spaleta:

> yum -d6 list updates  does not
> give me any information with regard to which mirror I'm being
> redirected to when pulling primary.xml.gz or repomd.xml.gz

>From my point of view yum treats the 'Baseurl(s) for repo:' as the mirror
(and does not take any redirection from that point).

Kind regards,
 
Jeroen van Meeuwen
 
--
kanarip


RE: Mirrorlist missing mirrors?

2006-03-21 Thread Elliot Lee
On Tue, 21 Mar 2006, Jeroen van Meeuwen wrote:

> It seems to be yum that cannot cope with the redirects.

The yum I have installed (yum-2.6.0-1) seems to handle the redirects fine.

I am all in favor of fixing what is broken, but so far the only specific
complaint has been from Jef about the difficulty of seeing what's going on
with things. Other than that, I guess I really need to understand the 
specifics of the problem so I can do my part to help fix them...

Best,
-- Elliot


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Jeff Spaleta
On 3/21/06, Elliot Lee  wrote:
> On Tue, 21 Mar 2006, Jeroen van Meeuwen wrote:
>
> > It seems to be yum that cannot cope with the redirects.
>
> The yum I have installed (yum-2.6.0-1) seems to handle the redirects fine.

It's difficult for me to make such a claim based on yum-2.6.0-1 output
that I have available. Things may very well be redirected as expected
but there's not information being provided which I can use to confirm
that.

On top of that.. the mirrorlist approach fails to failover to a 2nd
mirror if there is a sync problem.. when only the redirector is used
as the only mirror in the list..even if its repeated in the list. That
has to be counted as regression in functionality... even if the
redirecting is working. Again very difficult to know whats going on
here from the clientside. I'd really appreciate it if you could define
a reasonably useful recipe for generating auditable content that I can
direct users to use so we can track which mirrors are causing
problems.. or if the redirection itself is causing a problem.  Is the
only way to audit this going to be having users fire up packet
sniffers and logging individual packages for review? I'd really like
to avoid that.

-jef


RE: Mirrorlist missing mirrors?

2006-03-21 Thread Jeroen van Meeuwen
> I am all in favor of fixing what is broken, but so far the only specific
> complaint has been from Jef about the difficulty of seeing what's going on
> with things. Other than that, I guess I really need to understand the
> specifics of the problem so I can do my part to help fix them...

It's just now that I receive from
http://fedora.redhat.com/download/mirrors/fedora-core-$releasever a proper
list of mirrors.

The problem seems to be solved by not providing a mirrorlist directed to
download.fedoraproject.org, which then again redirects, but instead provide
the mirrorlist at fedora.redhat.com.

Kind regards,
 
Jeroen van Meeuwen
 
--
kanarip


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Elliot Lee
On Tue, 21 Mar 2006, Jeff Spaleta wrote:

> It's difficult for me to make such a claim based on yum-2.6.0-1 output
> that I have available. Things may very well be redirected as expected
> but there's not information being provided which I can use to confirm
> that.

I agree that the lack of debug info is sucky. Is this something that can
be fixed with a yum update, or is the fastestmirror plugin distributed
separately?

> On top of that.. the mirrorlist approach fails to failover to a 2nd
> mirror if there is a sync problem..

Hmm, do you know if yum does the mirror selection on a per-file basis or
once for all the downloads for a particular repo during a particular
session?

Out of syncedness should only really be an issue for rawhide... Maybe
Updates as well to some extent, but updates doesn't churn nearly as much
and is more likely to be in sync between mirrors.

Best,
-- Elliot


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Ralph Angenendt
Elliot Lee wrote:
> I am all in favor of fixing what is broken, but so far the only specific
> complaint has been from Jef about the difficulty of seeing what's going on
> with things. Other than that, I guess I really need to understand the 
> specifics of the problem so I can do my part to help fix them...

I had yum bailing out on my when timeouts occured. It printed out
"Trying other mirror." and then complained about having run out of
available mirrors. 

At the moment (with the mirrorlist available), using different mirrors
works again:

| Downloading Packages:
| http://fedora.inode.at/updates/5/i386/bind-config-9.3.2-10.FC5.i386.rpm:
| [Errno 14] HTTP Error 404: Content-Type: text/html
| Content-Length: 345
| Date: Tue, 21 Mar 2006 22:51:14 GMT
| Server: lighttpd/1.4.11
| Trying other mirror.
| (1/4): bind-config-9.3.2- 100% |=|  50 kB 00:00

Second: Having the redirector on the serverside makes plugins like
fastestmirror completely useless, as only one mirror will be found and
tested. What happens if the server running the redirector goes down? 

Ralph


pgpL1YX8sp1LK.pgp
Description: PGP signature


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Elliot Lee
On Tue, 21 Mar 2006, Ralph Angenendt wrote:

> Elliot Lee wrote:
> > I am all in favor of fixing what is broken, but so far the only specific
> > complaint has been from Jef about the difficulty of seeing what's going on
> > with things. Other than that, I guess I really need to understand the 
> > specifics of the problem so I can do my part to help fix them...
> 
> I had yum bailing out on my when timeouts occured. It printed out
> "Trying other mirror." and then complained about having run out of
> available mirrors. 

That particular problem should have been fixed a little while ago...

> At the moment (with the mirrorlist available), using different mirrors
> works again:
>
> | Downloading Packages:
> | http://fedora.inode.at/updates/5/i386/bind-config-9.3.2-10.FC5.i386.rpm:
> | [Errno 14] HTTP Error 404: Content-Type: text/html
> | Content-Length: 345
> | Date: Tue, 21 Mar 2006 22:51:14 GMT
> | Server: lighttpd/1.4.11
> | Trying other mirror.
> | (1/4): bind-config-9.3.2- 100% |=|  50 kB 00:00
> 
> Second: Having the redirector on the serverside makes plugins like
> fastestmirror completely useless, as only one mirror will be found and
> tested. What happens if the server running the redirector goes down?

And what will happen if the server hosting the mirrorlist goes down? :)

I can't help with the fastestmirror thing per se, and I know there are
people who are on the same backbone as a fast mirror and would rather make
use of that mirror by default. Some people will be helped by GeoIP (slated
to happen sometime). And perhaps there can be a way for clients to report
back to the server which mirrors they find fastest, while still allowing
the server to have a voice in the redirect decision. Let's try to figure
out the best of both worlds...

We do have a redundant setup that should be able to handle a reasonable 
amount of problems, but nothing is foolproof.
-- Elliot


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Ralph Angenendt
Elliot Lee wrote:
> On Tue, 21 Mar 2006, Ralph Angenendt wrote:
> > I had yum bailing out on my when timeouts occured. It printed out
> > "Trying other mirror." and then complained about having run out of
> > available mirrors. 
> 
> That particular problem should have been fixed a little while ago...

Depends on the "little while" :)

Happened to me around 4pm (GMT+0100).

> > Second: Having the redirector on the serverside makes plugins like
> > fastestmirror completely useless, as only one mirror will be found and
> > tested. What happens if the server running the redirector goes down?
> 
> And what will happen if the server hosting the mirrorlist goes down? :)

True. Fastestmirror only caches the hostname. 

> I can't help with the fastestmirror thing per se, and I know there are
> people who are on the same backbone as a fast mirror and would rather make
> use of that mirror by default. Some people will be helped by GeoIP (slated
> to happen sometime). And perhaps there can be a way for clients to report
> back to the server which mirrors they find fastest, while still allowing
> the server to have a voice in the redirect decision. Let's try to figure
> out the best of both worlds...

Well, I'd try to be "netfriendly", so GeoIP probably would be the best
solution. No need for my packets to travel across the pond :)

> We do have a redundant setup that should be able to handle a reasonable 
> amount of problems, but nothing is foolproof.

Yeah. But I hadn't seen problems like this afternoon before, as I had to
start the update process about 15 times to get all headers and packages
...

Ralph


pgpJdrHP76t4Z.pgp
Description: PGP signature


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Jeff Spaleta
On 3/21/06, Elliot Lee  wrote:
> I agree that the lack of debug info is sucky. Is this something that can
> be fixed with a yum update, or is the fastestmirror plugin distributed
> separately?

I can't answer that.. i haven't started my patent-pending process of
putting in random print statements into either the yum code, the
urlgrabber code or the fastestmirror plugin yet to see how much
information i can actually sneak out about where the redirect is
actually redirecting me.  And now that the rediector has been reverted
back to a list, I don't have a url to use my patent-pending print
statement insertion method with.

Let's be clear here... i only turned on the fastestmirror verbositing
because at the moment its the most verbose information I have without
putting random print statements in. This is also something in trying
to avoid because I want to be able to give instructions to users as
they experience problems so that the specifics of each incident can be
recorded. I'm really not prepared to tell people to patch their update
client willy-nilly, even if it is just python scripts. I don't want to
cause more errors with my patent-pending print statement insertion
method.

> Hmm, do you know if yum does the mirror selection on a per-file basis or
> once for all the downloads for a particular repo during a particular
> session?
>
> Out of syncedness should only really be an issue for rawhide...

Uhm... while i respect the "should only" point of view.. it was
definitely happening with the redirector for core and updates.

> Maybe
> Updates as well to some extent, but updates doesn't churn nearly as much
> and is more likely to be in sync between mirrors.

Again i cannot stress how difficult is to argue either way as to the
extent of mirror sync is going to be a problem if the redirector is
opaque with regard to which mirrors end up resulting in sync issues. 
If I can't track symptoms for reproducibility per mirror I'm just
going to advise people to open bug reports to try to accumulate stats
as to general occurance of the problem. But I know how that will end,
the bugs will be closed because there wont be enough information to do
anything about it.. per report.

-jef


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Robert Spanton
Chris Adams  wrote:
>
> Using a server-side redirector seems like a bad idea.  It completely
> defeats the purpose of things like fastestmirror or even giving a user a
> list of mirrors to choose from.
>

Rather than having the server side redirect done on requests for
http://download.fedoraproject.org/pub/fedora/linux/core/5/i386/os, perhaps
it would be better to shuffle the mirror list itself. Would the problems
that we're all encountering be avoided that way?

Rob


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Josh Boyer
On Tue, 2006-03-21 at 19:07 -0500, Jeff Spaleta wrote:
> On 3/21/06, Elliot Lee  wrote:
> 
> > Hmm, do you know if yum does the mirror selection on a per-file basis or
> > once for all the downloads for a particular repo during a particular
> > session?
> >
> > Out of syncedness should only really be an issue for rawhide...
> 
> Uhm... while i respect the "should only" point of view.. it was
> definitely happening with the redirector for core and updates.

I was seeing this last night as well.  Repeatedly retrying the yum
update make it finally succeed.  Presumably because it picked a new
mirror finally.

I have to agree with Jef here in that it does look like a regression to
the everyday user.

josh


Re: Mirrorlist missing mirrors?

2006-03-21 Thread Chris Adams
Once upon a time, Elliot Lee  said:
> And what will happen if the server hosting the mirrorlist goes down? :)

It would probably be easy enough for yum (or a plugin) to cache the last
known mirrorlist.

> I can't help with the fastestmirror thing per se, and I know there are
> people who are on the same backbone as a fast mirror and would rather make
> use of that mirror by default. Some people will be helped by GeoIP (slated
> to happen sometime). And perhaps there can be a way for clients to report
> back to the server which mirrors they find fastest, while still allowing
> the server to have a voice in the redirect decision. Let's try to figure
> out the best of both worlds...

Well, what problem was the server-side redirection try to solve?  What
are the cons to having the client choose a mirror instead of the server?

-- 
Chris Adams 
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.


Re: Mirrorlist missing mirrors?

2006-03-22 Thread Jeff Spaleta
On 3/21/06, Jesse Keating  wrote:
> I am reverting to the old style of the complete mirror list.  The single
> redirector URL is not working as planned.  Please wait for the new mirror
> list files to make it to the live webserver.

I just want to clarify..I take it the behavior has been reverted again
back to the redirector approach?

I'm seeing the same behavior as was seen ealier in the day yesterday
with regard to catching an out of sync mirror and not being able to
failover to another mirror.  Should I start direct people to file bugs
about this behavior per occurance? And which component do I have them
file against?

I continue to see a failure to failover to a second mirror with the
redirector mirrorlist in place.  Even after running yum clean all and
confirming by visual inspection that the cache directory is clear of
cached metadata.
Errors like:
http://download.fedoraproject.org/pub/fedora/linux/core/5/i386/os/repodata/primary.xml.gz:
[Errno 12] Timeout: 
Trying other mirror.
Error: failure: repodata/primary.xml.gz from core: [Errno 256] No more
mirrors to try.

or

http://download.fedoraproject.org/pub/fedora/linux/core/updates/testing/5/i386/repodata/primary.xml.gz:
[Errno -1] Metadata file does not match checksum
Trying other mirror.
Error: failure: repodata/primary.xml.gz from updates-testing: [Errno
256] No more mirrors to try.

When I use the -redirect mirrorlist instead.. IF there is a checksum
mismatch because of an out of sync mirror then the failover to the
second mirror in the lis occurs as expected. Also I can't seem to
cause a checksum mismatch right after do a yum clean all when using
the -redirect list, whereas the default mirrorlist is failing with the
above errors multiple times right after a yum clean all. Not 100% of
the time, because I'm sure it depends on which mirror I'm redirected
to when i pull the repomd from and then which mirror i am redirected
to for the primary pull.

With the server-side redirector active, I'm assuming that every
communication to the server means a potentially different mirror. As
in the pull to get repomd.xml can be a different mirror than the
primary.xml? If thats true.. thats russian rulette when there is a
single out of sync mirror in the pool for redirection. I really wish I
knew more about how the redirect works, or I could probe the redirect
so I could understand the potential failure modes. This is really
frustrating. I'm really considering telling people to edit their
configs so they are using a flat list instead.

This is vastly different than the logic I'm use to with yum. Normally
when you have a flat list of potential mirrors yum will start with a
mirror in the list and if there is a failure situation, yum will
failover to the next mirror and continue with that mirror unless there
is another error. Yum will continue to do this until it runs out of
mirrors never going back to a previous mirror.  With the server-side
redirect is there any garuntee that the server-side redirect won't
redirect me to a problematic out-of-sync mirror a second time during
the transaction?

-jef