RE: Are we blocked by central Maven repo?
The balancer should have a static ip, the machines behind it will be transparent to you. -Original Message- From: Ed Hillmann [mailto:[EMAIL PROTECTED] Sent: Monday, September 29, 2008 8:49 PM To: Maven Users List Subject: Re: Are we blocked by central Maven repo? On Tue, Sep 30, 2008 at 9:38 AM, Jason van Zyl <[EMAIL PROTECTED]> wrote: > We could start publishing a feed, but depending on the IP of the machine is > probably not the best idea. Given that we've installed a load balancer the > IP you get is going to be a whatever you get. > Thanks for the info. We'll have to address our setup and either find a mirror with a fixed IP or see if we can do this another way (second option is up to my IS dept, not me). Thanks, Ed - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Are we blocked by central Maven repo?
>For one, with rsync, if the network goes down, and an artifact that a >developer wants that hasn't yet been used is requested, nexus is useless. >He's still stuck. With an rsync everything is available. Only if you run rsync fairly often and kill us with bandwidth. And does every home based developer allocate 30gb to hold their own copy of central? I don't see how this is better than a RM >Second, being a command line person, I like being able to login to the server >and do something like "find . -name "*.pom" | xargs grep "somestring"" and >such to find various things. (I know, the repo managers have search things, >but gui's suck) Nexus stores the files in a file system, you can do exactly the same if you choose to not use the lucene index. >Third, httpd can run and serve static files on some very lightweight hardware >that cannot even begin to consider running java. As such, it's much faster >than Nexus or others. I run Nexus in a vm with ubuntu jeos with 128mb ram for the whole vm on an old machine. Does it get much lighter than that? Nexus uses ~64mb of ram...even our repository.sonatype.org instance that gets slammed by all the Maven builds at ci.sonatype.org. >Finally, this is the most important thing to me, each "mirrored" repository >can be kept on a unique URL. http://proxy/central, http://proxy/java.net, >http://proxy/apache-incubator, http://proxy/apache-snapshot, etc Thus, I don't get what you mean here. All the repo managers expose the repos via individual urls, grouping is recommended but optional.
Re: Are we blocked by central Maven repo?
On Tue, Sep 30, 2008 at 9:38 AM, Jason van Zyl <[EMAIL PROTECTED]> wrote: > We could start publishing a feed, but depending on the IP of the machine is > probably not the best idea. Given that we've installed a load balancer the > IP you get is going to be a whatever you get. > Thanks for the info. We'll have to address our setup and either find a mirror with a fixed IP or see if we can do this another way (second option is up to my IS dept, not me). Thanks, Ed - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Are we blocked by central Maven repo?
We could start publishing a feed, but depending on the IP of the machine is probably not the best idea. Given that we've installed a load balancer the IP you get is going to be a whatever you get. On 29-Sep-08, at 6:59 PM, Ed Hillmann wrote: 2008/9/30 Dan Tran <[EMAIL PROTECTED]>: repo1.maven.org has a new IP Address, so if you hardcoded repo1.maven.org in your /etc/hosts you will have this problem. Sorry if this s a repeat. -D We're having this exact same problem. Our Nexus repository suddenly over the weekend couldn't access repo1. The machine on which the Nexus repository runs has limited access to the internet, based on IP address. So, if repo1's address has changed, this will be the cause. Thanks for the info! It wasn't a repeat for me, as I must have missed the first time. Given that we're pretty much dependant on that IP address not changing, I wish there was a better notification system for changes to central. It's just buggered up all of our builds for the last 1 1/2 days. Thanks, Ed - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] Thanks, Jason -- Jason van Zyl Founder, Apache Maven jason at sonatype dot com -- the course of true love never did run smooth ... -- Shakespeare - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Are we blocked by central Maven repo?
2008/9/30 Dan Tran <[EMAIL PROTECTED]>: > repo1.maven.org has a new IP Address, so if you hardcoded > repo1.maven.org in your /etc/hosts you will have this problem. Sorry > if this s a repeat. > > -D We're having this exact same problem. Our Nexus repository suddenly over the weekend couldn't access repo1. The machine on which the Nexus repository runs has limited access to the internet, based on IP address. So, if repo1's address has changed, this will be the cause. Thanks for the info! It wasn't a repeat for me, as I must have missed the first time. Given that we're pretty much dependant on that IP address not changing, I wish there was a better notification system for changes to central. It's just buggered up all of our builds for the last 1 1/2 days. Thanks, Ed - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Are we blocked by central Maven repo?
repo1.maven.org has a new IP Address, so if you hardcoded
repo1.maven.org in your /etc/hosts you will have this problem. Sorry
if this s a repeat.
-D
2008/9/26 Beyer,Nathan <[EMAIL PROTECTED]>:
> It's possible that from the central repo's perspective, all traffic from your
> company may seem like it's coming from one IP address because of NAT.
>
> Using an internal mirror can help alleviate things. The most non-invasive
> mirror would be to rsync the central repo periodically and then modify
> internal DNS to point 'repo1.maven.org' to an internal IP address. You can
> save a lot of bandwidth and time this way.
>
> -Nathan
>
> -Original Message-
> From: 陈思淼 [mailto:[EMAIL PROTECTED]
> Sent: Friday, September 26, 2008 10:47 AM
> To: Maven Users List
> Subject: Re: Are we blocked by central Maven repo?
>
> we didn't do that kind of thing. we have a company-level artifactory
> repository.someone didn't follow the rule but most of us are good citizen,
> and follow the maven RULE,
> Is maven block strategy to block IP too strict?
> Can I do anything to Fix it Up?
>
>
>
> 2008/9/26 Wayne Fay <[EMAIL PROTECTED]>
>
>> It is possible to get blocked if you are acting as a bad citizen
>> (downloading the entire Central repo using wget, for example). Have
>> you (or someone else at your company) attempted to do this from your
>> IP address?
>>
>> If not, the repo is probably just busy, or you had some random
>> Internet connection failure. Try again. "Normal" Maven usage of the
>> repo will not get you blocked.
>>
>> Wayne
>>
>> On Fri, Sep 26, 2008 at 7:37 AM, 陈思淼 <[EMAIL PROTECTED]> wrote:
>> > This's log from artifactory.
>> >
>> > 2008-09-26 22:27:28,025 [WARN ] (RemoteRepoBase.java:259{10}) -
>> repo1:
>> > Error in getting information for 'org/apache/maven
>> > /maven-model/2.0.4/maven-model-2.0.4.pom.sha1'
>> > (org.apache.commons.httpclient.ConnectionPoolTimeoutException: Timeout
>> > waiting
>> > for connection).
>> >
>> > we company only have one outlet IP address ,someone may download Maven
>> from
>> > apache and didn't set the Mirror of central in the conf/setting.xml. so
>> they
>> > download the pom directly from central? Is that the reason why the
>> central
>> > repo block our IP address?
>> >
>>
>
> --
> CONFIDENTIALITY NOTICE This message and any included attachments are from
> Cerner Corporation and are intended only for the addressee. The information
> contained in this message is confidential and may constitute inside or
> non-public information under international, federal, or state securities
> laws. Unauthorized forwarding, printing, copying, distribution, or use of
> such information is strictly prohibited and may be unlawful. If you are not
> the addressee, please promptly delete this message and notify the sender of
> the delivery error by e-mail or you may call Cerner's corporate offices in
> Kansas City, Missouri, U.S.A at (+1) (816)221-1024.
>
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
Re: Are we blocked by central Maven repo?
On 29-Sep-08, at 11:54 AM, Daniel Kulp wrote: On Monday 29 September 2008 10:59:48 am Jason van Zyl wrote: There is no rsync access to central. But the crawling is doing the equivalent amount of damage. I was suggesting creating a new public mirror that would be listable on http://maven.apache.org/guides/mini/guide-mirror-settings.html as a public mirror. Thus, it could be "pushed" from central like the other mirrors. From there, people could rsync from that repo instead of central or ibiblio or such. There is no upside to using rsync over a repository manager. I disagree. For one, with rsync, if the network goes down, and an artifact that a developer wants that hasn't yet been used is requested, nexus is useless. If your rsync just finished and the network hasn't gone down, and an artifact is released your developer makes a request for an artifact then your rsync'd repository is useless. There are potential holes in both cases but given the assumption your network is relatively healthy a repository manager is a more robust solution. Provided your CI mechanism is working against the repository manager it is going to be primed the vast majority of the time. The days of the repository being down for more then 10 minutes are over. So provided you have built in the recent past you have what you need. He's still stuck. With an rsync everything is available. This HAS bitten us. We have several developers that work from home offices and thus have their own repo manager setup or similar (or maybe don't use one). They develop stuff, commit some changes. The nightly builds then run but due to network hickups, fail as the artifacts couldn't be retrieved since those builds are the first to ask for them. Managers come running and screaming saying "maven sucks" cause the build fail. Anything that keeps the managers from running to me screaming maven sucks is a good thing. They don't care about repo managers, rsyncs, etc They just want their builds to not fail for stupid reasons. This is where repository managers vastly outstrip direct use. You can't do any routing to protect yourself, so bad metadata will hose your developers. I think your chances of problems with you developers are higher without a repository manager. Second, being a command line person, I like being able to login to the server and do something like "find . -name "*.pom" | xargs grep "somestring"" and such to find various things. (I know, the repo managers have search things, but gui's suck) You can do that with a repository manager that uses a file-based system like Nexus. The repository looks just like you expect unless you have your own store implementation. Nexus also has a rest API so you can have curl script to make a REST call to do a search, get a POM, an artifact, a configuration, the status. Anything that is available via the UI in Nexus is available via the CLI. The UI is just a REST client. Third, httpd can run and serve static files on some very lightweight hardware that cannot even begin to consider running java. As such, it's much faster than Nexus or others. Jetty using memory mapped buffers are really not that different. We are not currently doing that in Nexus but once we do it won't be that different. But the mediation that a repository manager provides means you can fix all sorts of crap you cannot drinking from the fire hose. Finally, this is the most important thing to me, each "mirrored" repository can be kept on a unique URL. http://proxy/central, http://proxy/java.net , http://proxy/apache-incubator, http://proxy/apache-snapshot, etc Thus, I can be sure that poms that are checked in have the appropriate entries that can resolve artifacts from their proper location in the absense of any repo manager.(Yes, Archiva can do this via the virtual repositories. Archiva is the only repo manager I would consider using because of this.) Nexus does this for certain, and I'm sure Artifactory does as well. But repositories in POMs are a bad practice and it's far easier to control everything from the repository manager. Repositories in POM make the artifacts non-portable which we've seen make a Maven environment pretty much unworkable. In an environment where you want partitioning and you promote artifacts based on quality (whether it's promoted to a different logical or physical repository) then you can have a build that is now targeted for a QA environment pointing back to a dev environment. Nexus can actually reroute any repository request in a POM but I consider it a Maven anti-pattern. The repository management side is now starting to parallel the changes that have happened in the build management side. I think it's a natural evolution in the use of Maven. That said, there are a lot of advantages to using a repo manager as we
Re: Are we blocked by central Maven repo?
I'm not saying you are. It's the hundreds of other people trying it. You cannot sync against the central repository, you are syncing against ibiblio. Not the same thing. On 29-Sep-08, at 11:34 AM, Beyer,Nathan wrote: Rsync - http://maven.apache.org/guides/mini/guide-mirror- settings.html (see 'Creating your own mirror') We aren't crawling. -Original Message- From: Jason van Zyl [mailto:[EMAIL PROTECTED] Sent: Monday, September 29, 2008 10:00 AM To: Maven Users List Subject: Re: Are we blocked by central Maven repo? There is no rsync access to central. But the crawling is doing the equivalent amount of damage. There is no upside to using rsync over a repository manager. On 29-Sep-08, at 10:51 AM, Daniel Kulp wrote: One thing I keep thinking about doing is creating a public mirror that is synced from central (it's a public mirror, thus, they would allow that), but provide rsync acess on some sort of paid agreement. Maybe $5/month or possibly just a ontime $100 setup fee or similar. Basically, enough to cover the bandwidth/hosting charges plus deter "everyone and their mother" from just rsyncing away.Is that something that people would have interest in? If I only had the time to get it setup... :-( Dan On Monday 29 September 2008 10:21:54 am Beyer,Nathan wrote: What would you suggest then? Anything that requires customized maven installs or modifying 'settings.xml' post install is not feasible in our environment - development is too distributed. In the long-run I believe the rsync approach does reduce bandwith, but more importantly, the concurrent access to the central repo via HTTP is close to nil. Additionally, as I mentioned, the repository managers are NOT stable and require too much configuration and setup. These are not acceptable options. The repository managers aren't providing any other value beyond central repo caching for us. If you're going to cut off anonymous rsync access, you might as well just kill anonymous central repo access too, as that's the only way you'll be able to force people into use repository managers. I would suggest more granular rsync access, so that requests can be more targeted. -Nathan -Original Message- From: Jason van Zyl [mailto:[EMAIL PROTECTED] Sent: Friday, September 26, 2008 3:51 PM To: Maven Users List Subject: Re: Are we blocked by central Maven repo? On 26-Sep-08, at 9:31 PM, Beyer,Nathan wrote: I disagree. 10gb or even 20gb isn't that much data, and rsync isn't pulling that same amount down every time it runs. We're doing it and it's working quite well. It's much more stable and reliable than any other current mirroring practices. The internal DNS modification makes user setup easy, since there isn't any. The use of mirror settings per device is a non-starter for large, disparate organizations. All of the various caching servers just aren't stable enough yet, in my opinion. It is possible to get blocked by the central repo - we were contacted about our significant usage and told we were on the verge of being blacklisted, which is what lead us to rsync the mirror. There is no way you could use less bandwidth rsyncing then using a repository manager. If everyone rsynced and we allowed that against central we would get destroyed. We only allow mirrors to rsync, not users and mirrors will probably also stop providing rsync access because the first hit is just too high now if everyone did it. -Nathan -Original Message- From: Wayne Fay [mailto:[EMAIL PROTECTED] Sent: Friday, September 26, 2008 11:11 AM To: Maven Users List Subject: Re: Are we blocked by central Maven repo? IIRC Central is well over 10gb at this point (possibly 20gb) and a given organization will really only use at the most 1gb of it, so rsync'ing it is just a bad idea unless you are setting up an actual external mirror that will be available to the community. They are already using Artifactory, and I certainly hope/assume they are caching the results. This would limit their use of Central to one access per artifact (GAV) plus some hits by people not using their Artifactory instance. I would generally doubt they are actually blocked by Central, but rather this is an intermittent failure that will eventually resolve itself. Wayne 2008/9/26 Beyer,Nathan <[EMAIL PROTECTED]>: It's possible that from the central repo's perspective, all traffic from your company may seem like it's coming from one IP address because of NAT. Using an internal mirror can help alleviate things. The most non- invasive mirror would be to rsync the central repo periodically and then modify internal DNS to point 'repo1.maven.org' to an internal IP address. You can save a lot of bandwidth and time this way. -Nathan -----Original Message----- From: 陈思淼 [mailto:[EMAIL PROTECTED] Sent: Friday, September
RE: Are we blocked by central Maven repo?
As I mentioned in an earlier email, we rsync periodically, and override our internal DNS to redirect 'repo1.maven.org'. All internal developers automatically use the local copy. We don't crawl the repository. You may know people using Nexus for over a year that can't live without it. We've been using Apache web server mod_dav for three+ years with LDAP-based authentication and authorization - it has worked perfectly. As for all of your points about not having control of developers - a repository manager won't give me that either. There is no amount technology that can be applied to completely prevent any of those issues or countless others. I prefer to allow open access, facilitate developers, educate them and then review multiple times. The review points are where we begin to lock down and restrict access and the builds go through quality assurance, which can be automated. Locking down, filtering or controlling access up front prematurely limits innovation. I prefer to allow people to access what they want, as they want and as they move to later points in the process, then we tighten control as their builds move closer to manufacturing and work through dependency approval, license compliance, etc. -Nathan -Original Message- From: Jason van Zyl [mailto:[EMAIL PROTECTED] Sent: Monday, September 29, 2008 9:59 AM To: Maven Users List Subject: Re: Are we blocked by central Maven repo? On 29-Sep-08, at 10:21 AM, Beyer,Nathan wrote: > What would you suggest then? Anything that requires customized maven > installs or modifying 'settings.xml' post install is not feasible in > our environment - development is too distributed. So how do you used your rsync'd repository? How do you get all your developers to use your in-house repository which is a copy of central? Using repository managers makes distributed development an order of magnitude easier. > > > In the long-run I believe the rsync approach does reduce bandwith, > but more importantly, the concurrent access to the central repo via > HTTP is close to nil. Believe, as the one who looks at the logs and watch people crawl the repository this is not the case. In the long run you will see that you use less then 3% of what's in central so there is no point in pulling the bulk of the content. > > > Additionally, as I mentioned, the repository managers are NOT stable > and require too much configuration and setup. Hardly. We have people who have been using Nexus for over a year and they couldn't live without it now > These are not acceptable options. The repository managers aren't > providing any other value beyond central repo caching for us. > You have no idea what you're talking about. Sorry but you're are sadly mistaken. - Does your organization like your developers' builds crawling around every repository listed in a POM? You have no control over that without a repository manager. Tell your management that you're not controlling access to external repositories and see how much they like that. - You can create and manage access by all your developers from one location, if you have multiple repositories which most organizations have, this is a nightmare without a repository manager - IDE integration? Using a Nexus index you get complete autocompletion in the POM editor, ability to search for all plugins available, all archetypes available. - Routing around bad metadata protecting your developers from mis- formed POMs which can happen - Optimized searching for dependencies i.e. don't think around the world for your company's artifacts or only take Apache artifacts from the Apache repository - Repository federation, by proxing other repositories in Nexus you can search them all - Deployment with a simple PUT, no requirement for the WebDAV provider - Fine grained access to repositories i.e. far more powerful then access via Apache > If you're going to cut off anonymous rsync access, you might as well > just kill anonymous central repo access too, as that's the only way > you'll be able to force people into use repository managers. There is no anonymous rsync access to central, there never has been because the bandwidth charges would have made the situation unmanageable. We aren't forcing anyone to use repository managers, it boils down to a matter of cost in bandwidth. And people are using repository managers because it's just the smarter way to work with Maven. > > > I would suggest more granular rsync access, so that requests can be > more targeted. That pretty much amounts to the way a repository manager works. You're not going to get more targeted access then that. You get what you need and that's it. Run your CI system working against a repository manager
Re: Are we blocked by central Maven repo?
On Monday 29 September 2008 10:59:48 am Jason van Zyl wrote: > There is no rsync access to central. But the crawling is doing the > equivalent amount of damage. I was suggesting creating a new public mirror that would be listable on http://maven.apache.org/guides/mini/guide-mirror-settings.html as a public mirror. Thus, it could be "pushed" from central like the other mirrors. From there, people could rsync from that repo instead of central or ibiblio or such. > There is no upside to using rsync over a repository manager. I disagree. For one, with rsync, if the network goes down, and an artifact that a developer wants that hasn't yet been used is requested, nexus is useless. He's still stuck. With an rsync everything is available. This HAS bitten us. We have several developers that work from home offices and thus have their own repo manager setup or similar (or maybe don't use one). They develop stuff, commit some changes. The nightly builds then run but due to network hickups, fail as the artifacts couldn't be retrieved since those builds are the first to ask for them. Managers come running and screaming saying "maven sucks" cause the build fail. Anything that keeps the managers from running to me screaming maven sucks is a good thing. They don't care about repo managers, rsyncs, etc They just want their builds to not fail for stupid reasons. Second, being a command line person, I like being able to login to the server and do something like "find . -name "*.pom" | xargs grep "somestring"" and such to find various things. (I know, the repo managers have search things, but gui's suck) Third, httpd can run and serve static files on some very lightweight hardware that cannot even begin to consider running java. As such, it's much faster than Nexus or others. Finally, this is the most important thing to me, each "mirrored" repository can be kept on a unique URL. http://proxy/central, http://proxy/java.net, http://proxy/apache-incubator, http://proxy/apache-snapshot, etc Thus, I can be sure that poms that are checked in have the appropriate entries that can resolve artifacts from their proper location in the absense of any repo manager.(Yes, Archiva can do this via the virtual repositories. Archiva is the only repo manager I would consider using because of this.) That said, there are a lot of advantages to using a repo manager as well. I admit that. But using a repo manager currently does not meet ALL requirements. Dan > > On 29-Sep-08, at 10:51 AM, Daniel Kulp wrote: > > One thing I keep thinking about doing is creating a public mirror > > that is > > synced from central (it's a public mirror, thus, they would allow > > that), but > > provide rsync acess on some sort of paid agreement. Maybe $5/month > > or > > possibly just a ontime $100 setup fee or similar. Basically, > > enough to > > cover the bandwidth/hosting charges plus deter "everyone and their > > mother" > > from just rsyncing away.Is that something that people would have > > interest > > in? > > > > If I only had the time to get it setup... :-( > > > > Dan > > -- Daniel Kulp [EMAIL PROTECTED] http://www.dankulp.com/blog - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Are we blocked by central Maven repo?
Rsync - http://maven.apache.org/guides/mini/guide-mirror-settings.html (see 'Creating your own mirror') We aren't crawling. -Original Message- From: Jason van Zyl [mailto:[EMAIL PROTECTED] Sent: Monday, September 29, 2008 10:00 AM To: Maven Users List Subject: Re: Are we blocked by central Maven repo? There is no rsync access to central. But the crawling is doing the equivalent amount of damage. There is no upside to using rsync over a repository manager. On 29-Sep-08, at 10:51 AM, Daniel Kulp wrote: > > One thing I keep thinking about doing is creating a public mirror > that is > synced from central (it's a public mirror, thus, they would allow > that), but > provide rsync acess on some sort of paid agreement. Maybe $5/month > or > possibly just a ontime $100 setup fee or similar. Basically, > enough to > cover the bandwidth/hosting charges plus deter "everyone and their > mother" > from just rsyncing away.Is that something that people would have > interest > in? > > If I only had the time to get it setup... :-( > > Dan > > > > On Monday 29 September 2008 10:21:54 am Beyer,Nathan wrote: >> What would you suggest then? Anything that requires customized maven >> installs or modifying 'settings.xml' post install is not feasible >> in our >> environment - development is too distributed. >> >> In the long-run I believe the rsync approach does reduce bandwith, >> but more >> importantly, the concurrent access to the central repo via HTTP is >> close to >> nil. >> >> Additionally, as I mentioned, the repository managers are NOT >> stable and >> require too much configuration and setup. These are not acceptable >> options. >> The repository managers aren't providing any other value beyond >> central >> repo caching for us. >> >> If you're going to cut off anonymous rsync access, you might as >> well just >> kill anonymous central repo access too, as that's the only way >> you'll be >> able to force people into use repository managers. >> >> I would suggest more granular rsync access, so that requests can be >> more >> targeted. >> >> -Nathan >> >> -Original Message- >> From: Jason van Zyl [mailto:[EMAIL PROTECTED] >> Sent: Friday, September 26, 2008 3:51 PM >> To: Maven Users List >> Subject: Re: Are we blocked by central Maven repo? >> >> On 26-Sep-08, at 9:31 PM, Beyer,Nathan wrote: >>> I disagree. 10gb or even 20gb isn't that much data, and rsync isn't >>> pulling that same amount down every time it runs. We're doing it and >>> it's working quite well. It's much more stable and reliable than any >>> other current mirroring practices. The internal DNS modification >>> makes user setup easy, since there isn't any. The use of mirror >>> settings per device is a non-starter for large, disparate >>> organizations. All of the various caching servers just aren't stable >>> enough yet, in my opinion. >>> >>> It is possible to get blocked by the central repo - we were >>> contacted about our significant usage and told we were on the verge >>> of being blacklisted, which is what lead us to rsync the mirror. >> >> There is no way you could use less bandwidth rsyncing then using a >> repository manager. If everyone rsynced and we allowed that against >> central we would get destroyed. We only allow mirrors to rsync, not >> users and mirrors will probably also stop providing rsync access >> because the first hit is just too high now if everyone did it. >> >>> -Nathan >>> >>> -Original Message- >>> From: Wayne Fay [mailto:[EMAIL PROTECTED] >>> Sent: Friday, September 26, 2008 11:11 AM >>> To: Maven Users List >>> Subject: Re: Are we blocked by central Maven repo? >>> >>> IIRC Central is well over 10gb at this point (possibly 20gb) and a >>> given organization will really only use at the most 1gb of it, so >>> rsync'ing it is just a bad idea unless you are setting up an actual >>> external mirror that will be available to the community. >>> >>> They are already using Artifactory, and I certainly hope/assume they >>> are caching the results. This would limit their use of Central to >>> one >>> access per artifact (GAV) plus some hits by people not using their >>> Artifactory instance. >>> >>> I would generally doubt they are actually blocked
RE: Are we blocked by central Maven repo?
Does anyone have anecdotal proof that Nexus can handle significant loads? In my experience, it hasn't been able to scale beyond a small group of users (less than 25). I'm aware of this option, but none of the repository managers, in my experience, have been able to scale as well as a Apache web server loading artifacts from a filesystem. -Original Message- From: Stephen Connolly [mailto:[EMAIL PROTECTED] Sent: Monday, September 29, 2008 10:05 AM To: Maven Users List Subject: Re: Are we blocked by central Maven repo? Here is a quick set up for you. On your local machine that you were using for the internal mirror: 1. Install Apache httpd 2.2 with mod_proxy_ajp 2. Install Nexus 3. Front Nexus through ajp on the Apache httpd 4. Use a rewrite rule for /maven2 to /nexus/content/repositories/central/ 5. Change your internal dns records so that repo1.maven.org points to this local machine Now you have the same mirroring capabilities as before, only lower bandwidth and everything will be hunky-dorey 6. If you want to be ultra-fancy, add mod-proxy rules to map anything that's not on the server through to the real repo1.maven.org -Stephen 2008/9/29 Beyer,Nathan <[EMAIL PROTECTED]> > What would you suggest then? Anything that requires customized maven > installs or modifying 'settings.xml' post install is not feasible in our > environment - development is too distributed. > > In the long-run I believe the rsync approach does reduce bandwith, but more > importantly, the concurrent access to the central repo via HTTP is close to > nil. > > Additionally, as I mentioned, the repository managers are NOT stable and > require too much configuration and setup. These are not acceptable options. > The repository managers aren't providing any other value beyond central repo > caching for us. > > If you're going to cut off anonymous rsync access, you might as well just > kill anonymous central repo access too, as that's the only way you'll be > able to force people into use repository managers. > > I would suggest more granular rsync access, so that requests can be more > targeted. > > -Nathan > > -Original Message- > From: Jason van Zyl [mailto:[EMAIL PROTECTED] > Sent: Friday, September 26, 2008 3:51 PM > To: Maven Users List > Subject: Re: Are we blocked by central Maven repo? > > > On 26-Sep-08, at 9:31 PM, Beyer,Nathan wrote: > > > I disagree. 10gb or even 20gb isn't that much data, and rsync isn't > > pulling that same amount down every time it runs. We're doing it and > > it's working quite well. It's much more stable and reliable than any > > other current mirroring practices. The internal DNS modification > > makes user setup easy, since there isn't any. The use of mirror > > settings per device is a non-starter for large, disparate > > organizations. All of the various caching servers just aren't stable > > enough yet, in my opinion. > > > > It is possible to get blocked by the central repo - we were > > contacted about our significant usage and told we were on the verge > > of being blacklisted, which is what lead us to rsync the mirror. > > > > There is no way you could use less bandwidth rsyncing then using a > repository manager. If everyone rsynced and we allowed that against > central we would get destroyed. We only allow mirrors to rsync, not > users and mirrors will probably also stop providing rsync access > because the first hit is just too high now if everyone did it. > > > -Nathan > > > > -Original Message- > > From: Wayne Fay [mailto:[EMAIL PROTECTED] > > Sent: Friday, September 26, 2008 11:11 AM > > To: Maven Users List > > Subject: Re: Are we blocked by central Maven repo? > > > > IIRC Central is well over 10gb at this point (possibly 20gb) and a > > given organization will really only use at the most 1gb of it, so > > rsync'ing it is just a bad idea unless you are setting up an actual > > external mirror that will be available to the community. > > > > They are already using Artifactory, and I certainly hope/assume they > > are caching the results. This would limit their use of Central to one > > access per artifact (GAV) plus some hits by people not using their > > Artifactory instance. > > > > I would generally doubt they are actually blocked by Central, but > > rather this is an intermittent failure that will eventually resolve > > itself. > > > > Wayne > > > > 2008/9/26 Beyer,Nathan <[EMAIL PROTECTED]>: > >> It's possible that from the central repo's perspective, all traffic > >> from your company ma
Re: Are we blocked by central Maven repo?
There is no rsync access to central. But the crawling is doing the equivalent amount of damage. There is no upside to using rsync over a repository manager. On 29-Sep-08, at 10:51 AM, Daniel Kulp wrote: One thing I keep thinking about doing is creating a public mirror that is synced from central (it's a public mirror, thus, they would allow that), but provide rsync acess on some sort of paid agreement. Maybe $5/month or possibly just a ontime $100 setup fee or similar. Basically, enough to cover the bandwidth/hosting charges plus deter "everyone and their mother" from just rsyncing away.Is that something that people would have interest in? If I only had the time to get it setup... :-( Dan On Monday 29 September 2008 10:21:54 am Beyer,Nathan wrote: What would you suggest then? Anything that requires customized maven installs or modifying 'settings.xml' post install is not feasible in our environment - development is too distributed. In the long-run I believe the rsync approach does reduce bandwith, but more importantly, the concurrent access to the central repo via HTTP is close to nil. Additionally, as I mentioned, the repository managers are NOT stable and require too much configuration and setup. These are not acceptable options. The repository managers aren't providing any other value beyond central repo caching for us. If you're going to cut off anonymous rsync access, you might as well just kill anonymous central repo access too, as that's the only way you'll be able to force people into use repository managers. I would suggest more granular rsync access, so that requests can be more targeted. -Nathan -Original Message- From: Jason van Zyl [mailto:[EMAIL PROTECTED] Sent: Friday, September 26, 2008 3:51 PM To: Maven Users List Subject: Re: Are we blocked by central Maven repo? On 26-Sep-08, at 9:31 PM, Beyer,Nathan wrote: I disagree. 10gb or even 20gb isn't that much data, and rsync isn't pulling that same amount down every time it runs. We're doing it and it's working quite well. It's much more stable and reliable than any other current mirroring practices. The internal DNS modification makes user setup easy, since there isn't any. The use of mirror settings per device is a non-starter for large, disparate organizations. All of the various caching servers just aren't stable enough yet, in my opinion. It is possible to get blocked by the central repo - we were contacted about our significant usage and told we were on the verge of being blacklisted, which is what lead us to rsync the mirror. There is no way you could use less bandwidth rsyncing then using a repository manager. If everyone rsynced and we allowed that against central we would get destroyed. We only allow mirrors to rsync, not users and mirrors will probably also stop providing rsync access because the first hit is just too high now if everyone did it. -Nathan -Original Message- From: Wayne Fay [mailto:[EMAIL PROTECTED] Sent: Friday, September 26, 2008 11:11 AM To: Maven Users List Subject: Re: Are we blocked by central Maven repo? IIRC Central is well over 10gb at this point (possibly 20gb) and a given organization will really only use at the most 1gb of it, so rsync'ing it is just a bad idea unless you are setting up an actual external mirror that will be available to the community. They are already using Artifactory, and I certainly hope/assume they are caching the results. This would limit their use of Central to one access per artifact (GAV) plus some hits by people not using their Artifactory instance. I would generally doubt they are actually blocked by Central, but rather this is an intermittent failure that will eventually resolve itself. Wayne 2008/9/26 Beyer,Nathan <[EMAIL PROTECTED]>: It's possible that from the central repo's perspective, all traffic from your company may seem like it's coming from one IP address because of NAT. Using an internal mirror can help alleviate things. The most non- invasive mirror would be to rsync the central repo periodically and then modify internal DNS to point 'repo1.maven.org' to an internal IP address. You can save a lot of bandwidth and time this way. -Nathan -Original Message----- From: 陈思淼 [mailto:[EMAIL PROTECTED] Sent: Friday, September 26, 2008 10:47 AM To: Maven Users List Subject: Re: Are we blocked by central Maven repo? we didn't do that kind of thing. we have a company-level artifactory repository.someone didn't follow the rule but most of us are good citizen, and follow the maven RULE, Is maven block strategy to block IP too strict? Can I do anything to Fix it Up? 2008/9/26 Wayne Fay <[EMAIL PROTECTED]> It is possible to get blocked if you are acting as a bad citizen (downloading the entire Central repo using wget, for example). Have
Re: Are we blocked by central Maven repo?
On 29-Sep-08, at 10:21 AM, Beyer,Nathan wrote: What would you suggest then? Anything that requires customized maven installs or modifying 'settings.xml' post install is not feasible in our environment - development is too distributed. So how do you used your rsync'd repository? How do you get all your developers to use your in-house repository which is a copy of central? Using repository managers makes distributed development an order of magnitude easier. In the long-run I believe the rsync approach does reduce bandwith, but more importantly, the concurrent access to the central repo via HTTP is close to nil. Believe, as the one who looks at the logs and watch people crawl the repository this is not the case. In the long run you will see that you use less then 3% of what's in central so there is no point in pulling the bulk of the content. Additionally, as I mentioned, the repository managers are NOT stable and require too much configuration and setup. Hardly. We have people who have been using Nexus for over a year and they couldn't live without it now These are not acceptable options. The repository managers aren't providing any other value beyond central repo caching for us. You have no idea what you're talking about. Sorry but you're are sadly mistaken. - Does your organization like your developers' builds crawling around every repository listed in a POM? You have no control over that without a repository manager. Tell your management that you're not controlling access to external repositories and see how much they like that. - You can create and manage access by all your developers from one location, if you have multiple repositories which most organizations have, this is a nightmare without a repository manager - IDE integration? Using a Nexus index you get complete autocompletion in the POM editor, ability to search for all plugins available, all archetypes available. - Routing around bad metadata protecting your developers from mis- formed POMs which can happen - Optimized searching for dependencies i.e. don't think around the world for your company's artifacts or only take Apache artifacts from the Apache repository - Repository federation, by proxing other repositories in Nexus you can search them all - Deployment with a simple PUT, no requirement for the WebDAV provider - Fine grained access to repositories i.e. far more powerful then access via Apache If you're going to cut off anonymous rsync access, you might as well just kill anonymous central repo access too, as that's the only way you'll be able to force people into use repository managers. There is no anonymous rsync access to central, there never has been because the bandwidth charges would have made the situation unmanageable. We aren't forcing anyone to use repository managers, it boils down to a matter of cost in bandwidth. And people are using repository managers because it's just the smarter way to work with Maven. I would suggest more granular rsync access, so that requests can be more targeted. That pretty much amounts to the way a repository manager works. You're not going to get more targeted access then that. You get what you need and that's it. Run your CI system working against a repository manager will always keep your repository primed for use by your developers. -Nathan -Original Message- From: Jason van Zyl [mailto:[EMAIL PROTECTED] Sent: Friday, September 26, 2008 3:51 PM To: Maven Users List Subject: Re: Are we blocked by central Maven repo? On 26-Sep-08, at 9:31 PM, Beyer,Nathan wrote: I disagree. 10gb or even 20gb isn't that much data, and rsync isn't pulling that same amount down every time it runs. We're doing it and it's working quite well. It's much more stable and reliable than any other current mirroring practices. The internal DNS modification makes user setup easy, since there isn't any. The use of mirror settings per device is a non-starter for large, disparate organizations. All of the various caching servers just aren't stable enough yet, in my opinion. It is possible to get blocked by the central repo - we were contacted about our significant usage and told we were on the verge of being blacklisted, which is what lead us to rsync the mirror. There is no way you could use less bandwidth rsyncing then using a repository manager. If everyone rsynced and we allowed that against central we would get destroyed. We only allow mirrors to rsync, not users and mirrors will probably also stop providing rsync access because the first hit is just too high now if everyone did it. -Nathan -Original Message- From: Wayne Fay [mailto:[EMAIL PROTECTED] Sent: Friday, September 26, 2008 11:11 AM To: Maven Users List Subject: Re: Are we blocked by cent
Re: Are we blocked by central Maven repo?
Here is a quick set up for you. On your local machine that you were using for the internal mirror: 1. Install Apache httpd 2.2 with mod_proxy_ajp 2. Install Nexus 3. Front Nexus through ajp on the Apache httpd 4. Use a rewrite rule for /maven2 to /nexus/content/repositories/central/ 5. Change your internal dns records so that repo1.maven.org points to this local machine Now you have the same mirroring capabilities as before, only lower bandwidth and everything will be hunky-dorey 6. If you want to be ultra-fancy, add mod-proxy rules to map anything that's not on the server through to the real repo1.maven.org -Stephen 2008/9/29 Beyer,Nathan <[EMAIL PROTECTED]> > What would you suggest then? Anything that requires customized maven > installs or modifying 'settings.xml' post install is not feasible in our > environment - development is too distributed. > > In the long-run I believe the rsync approach does reduce bandwith, but more > importantly, the concurrent access to the central repo via HTTP is close to > nil. > > Additionally, as I mentioned, the repository managers are NOT stable and > require too much configuration and setup. These are not acceptable options. > The repository managers aren't providing any other value beyond central repo > caching for us. > > If you're going to cut off anonymous rsync access, you might as well just > kill anonymous central repo access too, as that's the only way you'll be > able to force people into use repository managers. > > I would suggest more granular rsync access, so that requests can be more > targeted. > > -Nathan > > -Original Message- > From: Jason van Zyl [mailto:[EMAIL PROTECTED] > Sent: Friday, September 26, 2008 3:51 PM > To: Maven Users List > Subject: Re: Are we blocked by central Maven repo? > > > On 26-Sep-08, at 9:31 PM, Beyer,Nathan wrote: > > > I disagree. 10gb or even 20gb isn't that much data, and rsync isn't > > pulling that same amount down every time it runs. We're doing it and > > it's working quite well. It's much more stable and reliable than any > > other current mirroring practices. The internal DNS modification > > makes user setup easy, since there isn't any. The use of mirror > > settings per device is a non-starter for large, disparate > > organizations. All of the various caching servers just aren't stable > > enough yet, in my opinion. > > > > It is possible to get blocked by the central repo - we were > > contacted about our significant usage and told we were on the verge > > of being blacklisted, which is what lead us to rsync the mirror. > > > > There is no way you could use less bandwidth rsyncing then using a > repository manager. If everyone rsynced and we allowed that against > central we would get destroyed. We only allow mirrors to rsync, not > users and mirrors will probably also stop providing rsync access > because the first hit is just too high now if everyone did it. > > > -Nathan > > > > -Original Message- > > From: Wayne Fay [mailto:[EMAIL PROTECTED] > > Sent: Friday, September 26, 2008 11:11 AM > > To: Maven Users List > > Subject: Re: Are we blocked by central Maven repo? > > > > IIRC Central is well over 10gb at this point (possibly 20gb) and a > > given organization will really only use at the most 1gb of it, so > > rsync'ing it is just a bad idea unless you are setting up an actual > > external mirror that will be available to the community. > > > > They are already using Artifactory, and I certainly hope/assume they > > are caching the results. This would limit their use of Central to one > > access per artifact (GAV) plus some hits by people not using their > > Artifactory instance. > > > > I would generally doubt they are actually blocked by Central, but > > rather this is an intermittent failure that will eventually resolve > > itself. > > > > Wayne > > > > 2008/9/26 Beyer,Nathan <[EMAIL PROTECTED]>: > >> It's possible that from the central repo's perspective, all traffic > >> from your company may seem like it's coming from one IP address > >> because of NAT. > >> > >> Using an internal mirror can help alleviate things. The most non- > >> invasive mirror would be to rsync the central repo periodically and > >> then modify internal DNS to point 'repo1.maven.org' to an internal > >> IP address. You can save a lot of bandwidth and time this way. > >> > >> -Nathan > >> > >> -Original Message- > >> From: 陈思淼 [mailt
Re: Are we blocked by central Maven repo?
One thing I keep thinking about doing is creating a public mirror that is synced from central (it's a public mirror, thus, they would allow that), but provide rsync acess on some sort of paid agreement. Maybe $5/month or possibly just a ontime $100 setup fee or similar. Basically, enough to cover the bandwidth/hosting charges plus deter "everyone and their mother" from just rsyncing away.Is that something that people would have interest in? If I only had the time to get it setup... :-( Dan On Monday 29 September 2008 10:21:54 am Beyer,Nathan wrote: > What would you suggest then? Anything that requires customized maven > installs or modifying 'settings.xml' post install is not feasible in our > environment - development is too distributed. > > In the long-run I believe the rsync approach does reduce bandwith, but more > importantly, the concurrent access to the central repo via HTTP is close to > nil. > > Additionally, as I mentioned, the repository managers are NOT stable and > require too much configuration and setup. These are not acceptable options. > The repository managers aren't providing any other value beyond central > repo caching for us. > > If you're going to cut off anonymous rsync access, you might as well just > kill anonymous central repo access too, as that's the only way you'll be > able to force people into use repository managers. > > I would suggest more granular rsync access, so that requests can be more > targeted. > > -Nathan > > -Original Message- > From: Jason van Zyl [mailto:[EMAIL PROTECTED] > Sent: Friday, September 26, 2008 3:51 PM > To: Maven Users List > Subject: Re: Are we blocked by central Maven repo? > > On 26-Sep-08, at 9:31 PM, Beyer,Nathan wrote: > > I disagree. 10gb or even 20gb isn't that much data, and rsync isn't > > pulling that same amount down every time it runs. We're doing it and > > it's working quite well. It's much more stable and reliable than any > > other current mirroring practices. The internal DNS modification > > makes user setup easy, since there isn't any. The use of mirror > > settings per device is a non-starter for large, disparate > > organizations. All of the various caching servers just aren't stable > > enough yet, in my opinion. > > > > It is possible to get blocked by the central repo - we were > > contacted about our significant usage and told we were on the verge > > of being blacklisted, which is what lead us to rsync the mirror. > > There is no way you could use less bandwidth rsyncing then using a > repository manager. If everyone rsynced and we allowed that against > central we would get destroyed. We only allow mirrors to rsync, not > users and mirrors will probably also stop providing rsync access > because the first hit is just too high now if everyone did it. > > > -Nathan > > > > -Original Message- > > From: Wayne Fay [mailto:[EMAIL PROTECTED] > > Sent: Friday, September 26, 2008 11:11 AM > > To: Maven Users List > > Subject: Re: Are we blocked by central Maven repo? > > > > IIRC Central is well over 10gb at this point (possibly 20gb) and a > > given organization will really only use at the most 1gb of it, so > > rsync'ing it is just a bad idea unless you are setting up an actual > > external mirror that will be available to the community. > > > > They are already using Artifactory, and I certainly hope/assume they > > are caching the results. This would limit their use of Central to one > > access per artifact (GAV) plus some hits by people not using their > > Artifactory instance. > > > > I would generally doubt they are actually blocked by Central, but > > rather this is an intermittent failure that will eventually resolve > > itself. > > > > Wayne > > > > 2008/9/26 Beyer,Nathan <[EMAIL PROTECTED]>: > >> It's possible that from the central repo's perspective, all traffic > >> from your company may seem like it's coming from one IP address > >> because of NAT. > >> > >> Using an internal mirror can help alleviate things. The most non- > >> invasive mirror would be to rsync the central repo periodically and > >> then modify internal DNS to point 'repo1.maven.org' to an internal > >> IP address. You can save a lot of bandwidth and time this way. > >> > >> -Nathan > >> > >> -Original Message- > >> From: 陈思淼 [mailto:[EMAIL PROTECTED] > >> Sent: Friday, September 26, 2008 10:47 AM > >> To: Maven Users List > &
RE: Are we blocked by central Maven repo?
What would you suggest then? Anything that requires customized maven installs or modifying 'settings.xml' post install is not feasible in our environment - development is too distributed. In the long-run I believe the rsync approach does reduce bandwith, but more importantly, the concurrent access to the central repo via HTTP is close to nil. Additionally, as I mentioned, the repository managers are NOT stable and require too much configuration and setup. These are not acceptable options. The repository managers aren't providing any other value beyond central repo caching for us. If you're going to cut off anonymous rsync access, you might as well just kill anonymous central repo access too, as that's the only way you'll be able to force people into use repository managers. I would suggest more granular rsync access, so that requests can be more targeted. -Nathan -Original Message- From: Jason van Zyl [mailto:[EMAIL PROTECTED] Sent: Friday, September 26, 2008 3:51 PM To: Maven Users List Subject: Re: Are we blocked by central Maven repo? On 26-Sep-08, at 9:31 PM, Beyer,Nathan wrote: > I disagree. 10gb or even 20gb isn't that much data, and rsync isn't > pulling that same amount down every time it runs. We're doing it and > it's working quite well. It's much more stable and reliable than any > other current mirroring practices. The internal DNS modification > makes user setup easy, since there isn't any. The use of mirror > settings per device is a non-starter for large, disparate > organizations. All of the various caching servers just aren't stable > enough yet, in my opinion. > > It is possible to get blocked by the central repo - we were > contacted about our significant usage and told we were on the verge > of being blacklisted, which is what lead us to rsync the mirror. > There is no way you could use less bandwidth rsyncing then using a repository manager. If everyone rsynced and we allowed that against central we would get destroyed. We only allow mirrors to rsync, not users and mirrors will probably also stop providing rsync access because the first hit is just too high now if everyone did it. > -Nathan > > -Original Message- > From: Wayne Fay [mailto:[EMAIL PROTECTED] > Sent: Friday, September 26, 2008 11:11 AM > To: Maven Users List > Subject: Re: Are we blocked by central Maven repo? > > IIRC Central is well over 10gb at this point (possibly 20gb) and a > given organization will really only use at the most 1gb of it, so > rsync'ing it is just a bad idea unless you are setting up an actual > external mirror that will be available to the community. > > They are already using Artifactory, and I certainly hope/assume they > are caching the results. This would limit their use of Central to one > access per artifact (GAV) plus some hits by people not using their > Artifactory instance. > > I would generally doubt they are actually blocked by Central, but > rather this is an intermittent failure that will eventually resolve > itself. > > Wayne > > 2008/9/26 Beyer,Nathan <[EMAIL PROTECTED]>: >> It's possible that from the central repo's perspective, all traffic >> from your company may seem like it's coming from one IP address >> because of NAT. >> >> Using an internal mirror can help alleviate things. The most non- >> invasive mirror would be to rsync the central repo periodically and >> then modify internal DNS to point 'repo1.maven.org' to an internal >> IP address. You can save a lot of bandwidth and time this way. >> >> -Nathan >> >> -Original Message- >> From: 陈思淼 [mailto:[EMAIL PROTECTED] >> Sent: Friday, September 26, 2008 10:47 AM >> To: Maven Users List >> Subject: Re: Are we blocked by central Maven repo? >> >> we didn't do that kind of thing. we have a company-level artifactory >> repository.someone didn't follow the rule but most of us are good >> citizen, >> and follow the maven RULE, >> Is maven block strategy to block IP too strict? >> Can I do anything to Fix it Up? >> >> >> >> 2008/9/26 Wayne Fay <[EMAIL PROTECTED]> >> >>> It is possible to get blocked if you are acting as a bad citizen >>> (downloading the entire Central repo using wget, for example). Have >>> you (or someone else at your company) attempted to do this from your >>> IP address? >>> >>> If not, the repo is probably just busy, or you had some random >>> Internet connection failure. Try again. "Normal" Maven usage of the >>> repo will not get you blocked. >
Re: Are we blocked by central Maven repo?
I second that.
We used to rsync the whole repo... but after the second time our internal
mirror server ran out of disk space we gave up and switched to nexus... the
whole thing is a lot more stable than the rsync nightmare... plus
we're not hitting 1/100 as much bandwith, and since we needed an internal
repo to deploy to anyway... win win win win
-Stephen
2008/9/26 Baptiste MATHUS <[EMAIL PROTECTED]>
> Rsync'ing the whole repository seems a real bad idea to me, I guess you
> could be blacklisted doing that.
> It's far better to do the downloading on demand and caching it by using a
> common maven repo manager (archiva, nexus...).
> In fact, if you mirror the whole repository, I guess you're just going to:
> * waste your bandwidth and the central repo one during a quite long time
> * waste your local disk space.
> I guess you're unlikely going to use even 5% of the thousands of existing
> artifacts from central.
>
> Well, redirecting packets to repo1.maven.org might be acceptable, though
> it's often useful to verify if the artifact is really available publicly
> before insulting your corporate repo. I do it myself quite regularly.
>
> Cheers.
>
> 2008/9/26 Beyer,Nathan <[EMAIL PROTECTED]>
>
> > It's possible that from the central repo's perspective, all traffic from
> > your company may seem like it's coming from one IP address because of
> NAT.
> >
> > Using an internal mirror can help alleviate things. The most non-invasive
> > mirror would be to rsync the central repo periodically and then modify
> > internal DNS to point 'repo1.maven.org' to an internal IP address. You
> can
> > save a lot of bandwidth and time this way.
> >
> > -Nathan
> >
> > -Original Message-
> > From: 陈思淼 [mailto:[EMAIL PROTECTED]
> > Sent: Friday, September 26, 2008 10:47 AM
> > To: Maven Users List
> > Subject: Re: Are we blocked by central Maven repo?
> >
> > we didn't do that kind of thing. we have a company-level artifactory
> > repository.someone didn't follow the rule but most of us are good
> citizen,
> > and follow the maven RULE,
> > Is maven block strategy to block IP too strict?
> > Can I do anything to Fix it Up?
> >
> >
> >
> > 2008/9/26 Wayne Fay <[EMAIL PROTECTED]>
> >
> > > It is possible to get blocked if you are acting as a bad citizen
> > > (downloading the entire Central repo using wget, for example). Have
> > > you (or someone else at your company) attempted to do this from your
> > > IP address?
> > >
> > > If not, the repo is probably just busy, or you had some random
> > > Internet connection failure. Try again. "Normal" Maven usage of the
> > > repo will not get you blocked.
> > >
> > > Wayne
> > >
> > > On Fri, Sep 26, 2008 at 7:37 AM, 陈思淼 <[EMAIL PROTECTED]> wrote:
> > > > This's log from artifactory.
> > > >
> > > > 2008-09-26 22:27:28,025 [WARN ] (RemoteRepoBase.java:259{10}) -
> > > repo1:
> > > > Error in getting information for 'org/apache/maven
> > > > /maven-model/2.0.4/maven-model-2.0.4.pom.sha1'
> > > > (org.apache.commons.httpclient.ConnectionPoolTimeoutException:
> Timeout
> > > > waiting
> > > > for connection).
> > > >
> > > > we company only have one outlet IP address ,someone may download
> Maven
> > > from
> > > > apache and didn't set the Mirror of central in the conf/setting.xml.
> so
> > > they
> > > > download the pom directly from central? Is that the reason why the
> > > central
> > > > repo block our IP address?
> > > >
> > >
> >
> > --
> > CONFIDENTIALITY NOTICE This message and any included attachments are from
> > Cerner Corporation and are intended only for the addressee. The
> information
> > contained in this message is confidential and may constitute inside or
> > non-public information under international, federal, or state securities
> > laws. Unauthorized forwarding, printing, copying, distribution, or use of
> > such information is strictly prohibited and may be unlawful. If you are
> not
> > the addressee, please promptly delete this message and notify the sender
> of
> > the delivery error by e-mail or you may call Cerner's corporate offices
> in
> > Kansas City, Missouri, U.S.A at (+1) (816)221-1024.
> >
> > -
> > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > For additional commands, e-mail: [EMAIL PROTECTED]
> >
> >
>
>
> --
> Baptiste MATHUS - http://batmat.net
> Sauvez un arbre,
> Mangez un castor !
>
Re: Are we blocked by central Maven repo?
First might be that you privately ask Jason if you are concerned by
http://blogs.sonatype.com/jvanzyl/2008/08/28/1219948661495.html
If so, then beg his pardon and promise you won't do it again, never :-).
But before annoying Jason, obviously try wget'ing something from
repo1.maven.org from your corporate maven repository server.
Cheers.
2008/9/26 陈思淼 <[EMAIL PROTECTED]>
> we didn't do that kind of thing. we have a company-level artifactory
> repository.someone didn't follow the rule but most of us are good citizen,
> and follow the maven RULE,
> Is maven block strategy to block IP too strict?
> Can I do anything to Fix it Up?
>
>
>
> 2008/9/26 Wayne Fay <[EMAIL PROTECTED]>
>
> > It is possible to get blocked if you are acting as a bad citizen
> > (downloading the entire Central repo using wget, for example). Have
> > you (or someone else at your company) attempted to do this from your
> > IP address?
> >
> > If not, the repo is probably just busy, or you had some random
> > Internet connection failure. Try again. "Normal" Maven usage of the
> > repo will not get you blocked.
> >
> > Wayne
> >
> > On Fri, Sep 26, 2008 at 7:37 AM, 陈思淼 <[EMAIL PROTECTED]> wrote:
> > > This's log from artifactory.
> > >
> > > 2008-09-26 22:27:28,025 [WARN ] (RemoteRepoBase.java:259{10}) -
> > repo1:
> > > Error in getting information for 'org/apache/maven
> > > /maven-model/2.0.4/maven-model-2.0.4.pom.sha1'
> > > (org.apache.commons.httpclient.ConnectionPoolTimeoutException: Timeout
> > > waiting
> > > for connection).
> > >
> > > we company only have one outlet IP address ,someone may download Maven
> > from
> > > apache and didn't set the Mirror of central in the conf/setting.xml. so
> > they
> > > download the pom directly from central? Is that the reason why the
> > central
> > > repo block our IP address?
> > >
> >
>
--
Baptiste MATHUS - http://batmat.net
Sauvez un arbre,
Mangez un castor !
Re: Are we blocked by central Maven repo?
Rsync'ing the whole repository seems a real bad idea to me, I guess you
could be blacklisted doing that.
It's far better to do the downloading on demand and caching it by using a
common maven repo manager (archiva, nexus...).
In fact, if you mirror the whole repository, I guess you're just going to:
* waste your bandwidth and the central repo one during a quite long time
* waste your local disk space.
I guess you're unlikely going to use even 5% of the thousands of existing
artifacts from central.
Well, redirecting packets to repo1.maven.org might be acceptable, though
it's often useful to verify if the artifact is really available publicly
before insulting your corporate repo. I do it myself quite regularly.
Cheers.
2008/9/26 Beyer,Nathan <[EMAIL PROTECTED]>
> It's possible that from the central repo's perspective, all traffic from
> your company may seem like it's coming from one IP address because of NAT.
>
> Using an internal mirror can help alleviate things. The most non-invasive
> mirror would be to rsync the central repo periodically and then modify
> internal DNS to point 'repo1.maven.org' to an internal IP address. You can
> save a lot of bandwidth and time this way.
>
> -Nathan
>
> -Original Message-
> From: 陈思淼 [mailto:[EMAIL PROTECTED]
> Sent: Friday, September 26, 2008 10:47 AM
> To: Maven Users List
> Subject: Re: Are we blocked by central Maven repo?
>
> we didn't do that kind of thing. we have a company-level artifactory
> repository.someone didn't follow the rule but most of us are good citizen,
> and follow the maven RULE,
> Is maven block strategy to block IP too strict?
> Can I do anything to Fix it Up?
>
>
>
> 2008/9/26 Wayne Fay <[EMAIL PROTECTED]>
>
> > It is possible to get blocked if you are acting as a bad citizen
> > (downloading the entire Central repo using wget, for example). Have
> > you (or someone else at your company) attempted to do this from your
> > IP address?
> >
> > If not, the repo is probably just busy, or you had some random
> > Internet connection failure. Try again. "Normal" Maven usage of the
> > repo will not get you blocked.
> >
> > Wayne
> >
> > On Fri, Sep 26, 2008 at 7:37 AM, 陈思淼 <[EMAIL PROTECTED]> wrote:
> > > This's log from artifactory.
> > >
> > > 2008-09-26 22:27:28,025 [WARN ] (RemoteRepoBase.java:259{10}) -
> > repo1:
> > > Error in getting information for 'org/apache/maven
> > > /maven-model/2.0.4/maven-model-2.0.4.pom.sha1'
> > > (org.apache.commons.httpclient.ConnectionPoolTimeoutException: Timeout
> > > waiting
> > > for connection).
> > >
> > > we company only have one outlet IP address ,someone may download Maven
> > from
> > > apache and didn't set the Mirror of central in the conf/setting.xml. so
> > they
> > > download the pom directly from central? Is that the reason why the
> > central
> > > repo block our IP address?
> > >
> >
>
> --
> CONFIDENTIALITY NOTICE This message and any included attachments are from
> Cerner Corporation and are intended only for the addressee. The information
> contained in this message is confidential and may constitute inside or
> non-public information under international, federal, or state securities
> laws. Unauthorized forwarding, printing, copying, distribution, or use of
> such information is strictly prohibited and may be unlawful. If you are not
> the addressee, please promptly delete this message and notify the sender of
> the delivery error by e-mail or you may call Cerner's corporate offices in
> Kansas City, Missouri, U.S.A at (+1) (816)221-1024.
>
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
--
Baptiste MATHUS - http://batmat.net
Sauvez un arbre,
Mangez un castor !
Re: Are we blocked by central Maven repo?
On 26-Sep-08, at 9:31 PM, Beyer,Nathan wrote:
I disagree. 10gb or even 20gb isn't that much data, and rsync isn't
pulling that same amount down every time it runs. We're doing it and
it's working quite well. It's much more stable and reliable than any
other current mirroring practices. The internal DNS modification
makes user setup easy, since there isn't any. The use of mirror
settings per device is a non-starter for large, disparate
organizations. All of the various caching servers just aren't stable
enough yet, in my opinion.
It is possible to get blocked by the central repo - we were
contacted about our significant usage and told we were on the verge
of being blacklisted, which is what lead us to rsync the mirror.
There is no way you could use less bandwidth rsyncing then using a
repository manager. If everyone rsynced and we allowed that against
central we would get destroyed. We only allow mirrors to rsync, not
users and mirrors will probably also stop providing rsync access
because the first hit is just too high now if everyone did it.
-Nathan
-Original Message-
From: Wayne Fay [mailto:[EMAIL PROTECTED]
Sent: Friday, September 26, 2008 11:11 AM
To: Maven Users List
Subject: Re: Are we blocked by central Maven repo?
IIRC Central is well over 10gb at this point (possibly 20gb) and a
given organization will really only use at the most 1gb of it, so
rsync'ing it is just a bad idea unless you are setting up an actual
external mirror that will be available to the community.
They are already using Artifactory, and I certainly hope/assume they
are caching the results. This would limit their use of Central to one
access per artifact (GAV) plus some hits by people not using their
Artifactory instance.
I would generally doubt they are actually blocked by Central, but
rather this is an intermittent failure that will eventually resolve
itself.
Wayne
2008/9/26 Beyer,Nathan <[EMAIL PROTECTED]>:
It's possible that from the central repo's perspective, all traffic
from your company may seem like it's coming from one IP address
because of NAT.
Using an internal mirror can help alleviate things. The most non-
invasive mirror would be to rsync the central repo periodically and
then modify internal DNS to point 'repo1.maven.org' to an internal
IP address. You can save a lot of bandwidth and time this way.
-Nathan
-Original Message-
From: 陈思淼 [mailto:[EMAIL PROTECTED]
Sent: Friday, September 26, 2008 10:47 AM
To: Maven Users List
Subject: Re: Are we blocked by central Maven repo?
we didn't do that kind of thing. we have a company-level artifactory
repository.someone didn't follow the rule but most of us are good
citizen,
and follow the maven RULE,
Is maven block strategy to block IP too strict?
Can I do anything to Fix it Up?
2008/9/26 Wayne Fay <[EMAIL PROTECTED]>
It is possible to get blocked if you are acting as a bad citizen
(downloading the entire Central repo using wget, for example). Have
you (or someone else at your company) attempted to do this from your
IP address?
If not, the repo is probably just busy, or you had some random
Internet connection failure. Try again. "Normal" Maven usage of the
repo will not get you blocked.
Wayne
On Fri, Sep 26, 2008 at 7:37 AM, 陈思淼 <[EMAIL PROTECTED]>
wrote:
This's log from artifactory.
2008-09-26 22:27:28,025 [WARN ] (RemoteRepoBase.java:259{10}) -
repo1:
Error in getting information for 'org/apache/maven
/maven-model/2.0.4/maven-model-2.0.4.pom.sha1'
(org.apache.commons.httpclient.ConnectionPoolTimeoutException:
Timeout
waiting
for connection).
we company only have one outlet IP address ,someone may download
Maven
from
apache and didn't set the Mirror of central in the conf/
setting.xml. so
they
download the pom directly from central? Is that the reason why the
central
repo block our IP address?
--
CONFIDENTIALITY NOTICE This message and any included attachments
are from Cerner Corporation and are intended only for the
addressee. The information contained in this message is
confidential and may constitute inside or non-public information
under international, federal, or state securities laws.
Unauthorized forwarding, printing, copying, distribution, or use of
such information is strictly prohibited and may be unlawful. If you
are not the addressee, please promptly delete this message and
notify the sender of the delivery error by e-mail or you may call
Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1)
(816)221-1024.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Thanks,
RE: Are we blocked by central Maven repo?
I disagree. 10gb or even 20gb isn't that much data, and rsync isn't pulling
that same amount down every time it runs. We're doing it and it's working quite
well. It's much more stable and reliable than any other current mirroring
practices. The internal DNS modification makes user setup easy, since there
isn't any. The use of mirror settings per device is a non-starter for large,
disparate organizations. All of the various caching servers just aren't stable
enough yet, in my opinion.
It is possible to get blocked by the central repo - we were contacted about our
significant usage and told we were on the verge of being blacklisted, which is
what lead us to rsync the mirror.
-Nathan
-Original Message-
From: Wayne Fay [mailto:[EMAIL PROTECTED]
Sent: Friday, September 26, 2008 11:11 AM
To: Maven Users List
Subject: Re: Are we blocked by central Maven repo?
IIRC Central is well over 10gb at this point (possibly 20gb) and a
given organization will really only use at the most 1gb of it, so
rsync'ing it is just a bad idea unless you are setting up an actual
external mirror that will be available to the community.
They are already using Artifactory, and I certainly hope/assume they
are caching the results. This would limit their use of Central to one
access per artifact (GAV) plus some hits by people not using their
Artifactory instance.
I would generally doubt they are actually blocked by Central, but
rather this is an intermittent failure that will eventually resolve
itself.
Wayne
2008/9/26 Beyer,Nathan <[EMAIL PROTECTED]>:
> It's possible that from the central repo's perspective, all traffic from your
> company may seem like it's coming from one IP address because of NAT.
>
> Using an internal mirror can help alleviate things. The most non-invasive
> mirror would be to rsync the central repo periodically and then modify
> internal DNS to point 'repo1.maven.org' to an internal IP address. You can
> save a lot of bandwidth and time this way.
>
> -Nathan
>
> -Original Message-
> From: 陈思淼 [mailto:[EMAIL PROTECTED]
> Sent: Friday, September 26, 2008 10:47 AM
> To: Maven Users List
> Subject: Re: Are we blocked by central Maven repo?
>
> we didn't do that kind of thing. we have a company-level artifactory
> repository.someone didn't follow the rule but most of us are good citizen,
> and follow the maven RULE,
> Is maven block strategy to block IP too strict?
> Can I do anything to Fix it Up?
>
>
>
> 2008/9/26 Wayne Fay <[EMAIL PROTECTED]>
>
>> It is possible to get blocked if you are acting as a bad citizen
>> (downloading the entire Central repo using wget, for example). Have
>> you (or someone else at your company) attempted to do this from your
>> IP address?
>>
>> If not, the repo is probably just busy, or you had some random
>> Internet connection failure. Try again. "Normal" Maven usage of the
>> repo will not get you blocked.
>>
>> Wayne
>>
>> On Fri, Sep 26, 2008 at 7:37 AM, 陈思淼 <[EMAIL PROTECTED]> wrote:
>> > This's log from artifactory.
>> >
>> > 2008-09-26 22:27:28,025 [WARN ] (RemoteRepoBase.java:259{10}) -
>> repo1:
>> > Error in getting information for 'org/apache/maven
>> > /maven-model/2.0.4/maven-model-2.0.4.pom.sha1'
>> > (org.apache.commons.httpclient.ConnectionPoolTimeoutException: Timeout
>> > waiting
>> > for connection).
>> >
>> > we company only have one outlet IP address ,someone may download Maven
>> from
>> > apache and didn't set the Mirror of central in the conf/setting.xml. so
>> they
>> > download the pom directly from central? Is that the reason why the
>> central
>> > repo block our IP address?
>> >
>>
>
> --
> CONFIDENTIALITY NOTICE This message and any included attachments are from
> Cerner Corporation and are intended only for the addressee. The information
> contained in this message is confidential and may constitute inside or
> non-public information under international, federal, or state securities
> laws. Unauthorized forwarding, printing, copying, distribution, or use of
> such information is strictly prohibited and may be unlawful. If you are not
> the addressee, please promptly delete this message and notify the sender of
> the delivery error by e-mail or you may call Cerner's corporate offices in
> Kansas City, Missouri, U.S.A at (+1) (816)221-1024.
>
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
RE: Are we blocked by central Maven repo?
No, can't do it today. We are in release week and I have all the servers locked down into our cnfiguration. I will be bringing up a new version of artifactory over the weekend (hopefully) and will see if it has something to do with artifactory versions. -Original Message- From: Wayne Fay [mailto:[EMAIL PROTECTED] Sent: Friday, September 26, 2008 12:15 PM To: Maven Users List Subject: Re: Are we blocked by central Maven repo? Could you perhaps set up Nexus and see if it is having the same trouble, or limited to Artifactory (which would be odd)? Wayne 2008/9/26 Sommers, Elizabeth <[EMAIL PROTECTED]>: > > We are having the exact same problem with artifactory today. Nobody here has been a bad citizen. > > Liz > [EMAIL PROTECTED] > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Are we blocked by central Maven repo?
Please use a mirror
http://maven.apache.org/guides/mini/guide-mirror-settings.html
http://docs.codehaus.org/display/MAVENUSER/Mirrors+Repositories
2008/9/26 Sommers, Elizabeth <[EMAIL PROTECTED]>:
>
> We are having the exact same problem with artifactory today. Nobody here has
> been a bad citizen.
>
> Liz
> [EMAIL PROTECTED]
>
> -Original Message-
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
> Sent: Friday, September 26, 2008 11:47 AM
> To: Maven Users List
> Subject: Re: Are we blocked by central Maven repo?
>
> we didn't do that kind of thing. we have a company-level artifactory
> repository.someone didn't follow the rule but most of us are good citizen,
> and follow the maven RULE, Is maven block strategy to block IP too strict?
> Can I do anything to Fix it Up?
>
>
>
> 2008/9/26 Wayne Fay <[EMAIL PROTECTED]>
>
>> It is possible to get blocked if you are acting as a bad citizen
>> (downloading the entire Central repo using wget, for example). Have
>> you (or someone else at your company) attempted to do this from your
>> IP address?
>>
>> If not, the repo is probably just busy, or you had some random
>> Internet connection failure. Try again. "Normal" Maven usage of the
>> repo will not get you blocked.
>>
>> Wayne
>>
>> On Fri, Sep 26, 2008 at 7:37 AM, 陈思淼 <[EMAIL PROTECTED]> wrote:
>> > This's log from artifactory.
>> >
>> > 2008-09-26 22:27:28,025 [WARN ] (RemoteRepoBase.java:259{10}) -
>> repo1:
>> > Error in getting information for 'org/apache/maven
>> > /maven-model/2.0.4/maven-model-2.0.4.pom.sha1'
>> > (org.apache.commons.httpclient.ConnectionPoolTimeoutException:
>> > Timeout waiting for connection).
>> >
>> > we company only have one outlet IP address ,someone may download
>> > Maven
>> from
>> > apache and didn't set the Mirror of central in the conf/setting.xml.
>> > so
>> they
>> > download the pom directly from central? Is that the reason why the
>> central
>> > repo block our IP address?
>> >
>>
>
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
Re: Are we blocked by central Maven repo?
Could you perhaps set up Nexus and see if it is having the same
trouble, or limited to Artifactory (which would be odd)?
Wayne
2008/9/26 Sommers, Elizabeth <[EMAIL PROTECTED]>:
>
> We are having the exact same problem with artifactory today. Nobody here has
> been a bad citizen.
>
> Liz
> [EMAIL PROTECTED]
>
> -Original Message-
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
> Sent: Friday, September 26, 2008 11:47 AM
> To: Maven Users List
> Subject: Re: Are we blocked by central Maven repo?
>
> we didn't do that kind of thing. we have a company-level artifactory
> repository.someone didn't follow the rule but most of us are good citizen,
> and follow the maven RULE, Is maven block strategy to block IP too strict?
> Can I do anything to Fix it Up?
>
>
>
> 2008/9/26 Wayne Fay <[EMAIL PROTECTED]>
>
>> It is possible to get blocked if you are acting as a bad citizen
>> (downloading the entire Central repo using wget, for example). Have
>> you (or someone else at your company) attempted to do this from your
>> IP address?
>>
>> If not, the repo is probably just busy, or you had some random
>> Internet connection failure. Try again. "Normal" Maven usage of the
>> repo will not get you blocked.
>>
>> Wayne
>>
>> On Fri, Sep 26, 2008 at 7:37 AM, 陈思淼 <[EMAIL PROTECTED]> wrote:
>> > This's log from artifactory.
>> >
>> > 2008-09-26 22:27:28,025 [WARN ] (RemoteRepoBase.java:259{10}) -
>> repo1:
>> > Error in getting information for 'org/apache/maven
>> > /maven-model/2.0.4/maven-model-2.0.4.pom.sha1'
>> > (org.apache.commons.httpclient.ConnectionPoolTimeoutException:
>> > Timeout waiting for connection).
>> >
>> > we company only have one outlet IP address ,someone may download
>> > Maven
>> from
>> > apache and didn't set the Mirror of central in the conf/setting.xml.
>> > so
>> they
>> > download the pom directly from central? Is that the reason why the
>> central
>> > repo block our IP address?
>> >
>>
>
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
RE: Are we blocked by central Maven repo?
We are having the exact same problem with artifactory today. Nobody here has
been a bad citizen.
Liz
[EMAIL PROTECTED]
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
Sent: Friday, September 26, 2008 11:47 AM
To: Maven Users List
Subject: Re: Are we blocked by central Maven repo?
we didn't do that kind of thing. we have a company-level artifactory
repository.someone didn't follow the rule but most of us are good citizen, and
follow the maven RULE, Is maven block strategy to block IP too strict?
Can I do anything to Fix it Up?
2008/9/26 Wayne Fay <[EMAIL PROTECTED]>
> It is possible to get blocked if you are acting as a bad citizen
> (downloading the entire Central repo using wget, for example). Have
> you (or someone else at your company) attempted to do this from your
> IP address?
>
> If not, the repo is probably just busy, or you had some random
> Internet connection failure. Try again. "Normal" Maven usage of the
> repo will not get you blocked.
>
> Wayne
>
> On Fri, Sep 26, 2008 at 7:37 AM, 陈思淼 <[EMAIL PROTECTED]> wrote:
> > This's log from artifactory.
> >
> > 2008-09-26 22:27:28,025 [WARN ] (RemoteRepoBase.java:259{10}) -
> repo1:
> > Error in getting information for 'org/apache/maven
> > /maven-model/2.0.4/maven-model-2.0.4.pom.sha1'
> > (org.apache.commons.httpclient.ConnectionPoolTimeoutException:
> > Timeout waiting for connection).
> >
> > we company only have one outlet IP address ,someone may download
> > Maven
> from
> > apache and didn't set the Mirror of central in the conf/setting.xml.
> > so
> they
> > download the pom directly from central? Is that the reason why the
> central
> > repo block our IP address?
> >
>
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Are we blocked by central Maven repo?
IIRC Central is well over 10gb at this point (possibly 20gb) and a
given organization will really only use at the most 1gb of it, so
rsync'ing it is just a bad idea unless you are setting up an actual
external mirror that will be available to the community.
They are already using Artifactory, and I certainly hope/assume they
are caching the results. This would limit their use of Central to one
access per artifact (GAV) plus some hits by people not using their
Artifactory instance.
I would generally doubt they are actually blocked by Central, but
rather this is an intermittent failure that will eventually resolve
itself.
Wayne
2008/9/26 Beyer,Nathan <[EMAIL PROTECTED]>:
> It's possible that from the central repo's perspective, all traffic from your
> company may seem like it's coming from one IP address because of NAT.
>
> Using an internal mirror can help alleviate things. The most non-invasive
> mirror would be to rsync the central repo periodically and then modify
> internal DNS to point 'repo1.maven.org' to an internal IP address. You can
> save a lot of bandwidth and time this way.
>
> -Nathan
>
> -Original Message-
> From: 陈思淼 [mailto:[EMAIL PROTECTED]
> Sent: Friday, September 26, 2008 10:47 AM
> To: Maven Users List
> Subject: Re: Are we blocked by central Maven repo?
>
> we didn't do that kind of thing. we have a company-level artifactory
> repository.someone didn't follow the rule but most of us are good citizen,
> and follow the maven RULE,
> Is maven block strategy to block IP too strict?
> Can I do anything to Fix it Up?
>
>
>
> 2008/9/26 Wayne Fay <[EMAIL PROTECTED]>
>
>> It is possible to get blocked if you are acting as a bad citizen
>> (downloading the entire Central repo using wget, for example). Have
>> you (or someone else at your company) attempted to do this from your
>> IP address?
>>
>> If not, the repo is probably just busy, or you had some random
>> Internet connection failure. Try again. "Normal" Maven usage of the
>> repo will not get you blocked.
>>
>> Wayne
>>
>> On Fri, Sep 26, 2008 at 7:37 AM, 陈思淼 <[EMAIL PROTECTED]> wrote:
>> > This's log from artifactory.
>> >
>> > 2008-09-26 22:27:28,025 [WARN ] (RemoteRepoBase.java:259{10}) -
>> repo1:
>> > Error in getting information for 'org/apache/maven
>> > /maven-model/2.0.4/maven-model-2.0.4.pom.sha1'
>> > (org.apache.commons.httpclient.ConnectionPoolTimeoutException: Timeout
>> > waiting
>> > for connection).
>> >
>> > we company only have one outlet IP address ,someone may download Maven
>> from
>> > apache and didn't set the Mirror of central in the conf/setting.xml. so
>> they
>> > download the pom directly from central? Is that the reason why the
>> central
>> > repo block our IP address?
>> >
>>
>
> --
> CONFIDENTIALITY NOTICE This message and any included attachments are from
> Cerner Corporation and are intended only for the addressee. The information
> contained in this message is confidential and may constitute inside or
> non-public information under international, federal, or state securities
> laws. Unauthorized forwarding, printing, copying, distribution, or use of
> such information is strictly prohibited and may be unlawful. If you are not
> the addressee, please promptly delete this message and notify the sender of
> the delivery error by e-mail or you may call Cerner's corporate offices in
> Kansas City, Missouri, U.S.A at (+1) (816)221-1024.
>
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
Re: Are we blocked by central Maven repo?
We didn't do anything terrible, we just try to download some frequently used
maven-plugin and build our project,
Maybe there is a Bug about artifactory repository, because after we restart
the jetty running the artifactory, It lost the cached artifact jar and
reconnect to central repository to get the jars again. Maybe that's the
reason why we reconnect to central maven again.
But is that too strictly? I don't know. I use
http://www.ibiblio.org/maven2to mirror central maven temperaroryly.
I pray the central will unblock us as quickly as possible.
2008/9/26 Beyer,Nathan <[EMAIL PROTECTED]>
> It's possible that from the central repo's perspective, all traffic from
> your company may seem like it's coming from one IP address because of NAT.
>
> Using an internal mirror can help alleviate things. The most non-invasive
> mirror would be to rsync the central repo periodically and then modify
> internal DNS to point 'repo1.maven.org' to an internal IP address. You can
> save a lot of bandwidth and time this way.
>
> -Nathan
>
> -Original Message-
> From: 陈思淼 [mailto:[EMAIL PROTECTED]
> Sent: Friday, September 26, 2008 10:47 AM
> To: Maven Users List
> Subject: Re: Are we blocked by central Maven repo?
>
> we didn't do that kind of thing. we have a company-level artifactory
> repository.someone didn't follow the rule but most of us are good citizen,
> and follow the maven RULE,
> Is maven block strategy to block IP too strict?
> Can I do anything to Fix it Up?
>
>
>
> 2008/9/26 Wayne Fay <[EMAIL PROTECTED]>
>
> > It is possible to get blocked if you are acting as a bad citizen
> > (downloading the entire Central repo using wget, for example). Have
> > you (or someone else at your company) attempted to do this from your
> > IP address?
> >
> > If not, the repo is probably just busy, or you had some random
> > Internet connection failure. Try again. "Normal" Maven usage of the
> > repo will not get you blocked.
> >
> > Wayne
> >
> > On Fri, Sep 26, 2008 at 7:37 AM, 陈思淼 <[EMAIL PROTECTED]> wrote:
> > > This's log from artifactory.
> > >
> > > 2008-09-26 22:27:28,025 [WARN ] (RemoteRepoBase.java:259{10}) -
> > repo1:
> > > Error in getting information for 'org/apache/maven
> > > /maven-model/2.0.4/maven-model-2.0.4.pom.sha1'
> > > (org.apache.commons.httpclient.ConnectionPoolTimeoutException: Timeout
> > > waiting
> > > for connection).
> > >
> > > we company only have one outlet IP address ,someone may download Maven
> > from
> > > apache and didn't set the Mirror of central in the conf/setting.xml. so
> > they
> > > download the pom directly from central? Is that the reason why the
> > central
> > > repo block our IP address?
> > >
> >
>
> --
> CONFIDENTIALITY NOTICE This message and any included attachments are from
> Cerner Corporation and are intended only for the addressee. The information
> contained in this message is confidential and may constitute inside or
> non-public information under international, federal, or state securities
> laws. Unauthorized forwarding, printing, copying, distribution, or use of
> such information is strictly prohibited and may be unlawful. If you are not
> the addressee, please promptly delete this message and notify the sender of
> the delivery error by e-mail or you may call Cerner's corporate offices in
> Kansas City, Missouri, U.S.A at (+1) (816)221-1024.
>
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
RE: Are we blocked by central Maven repo?
It's possible that from the central repo's perspective, all traffic from your
company may seem like it's coming from one IP address because of NAT.
Using an internal mirror can help alleviate things. The most non-invasive
mirror would be to rsync the central repo periodically and then modify internal
DNS to point 'repo1.maven.org' to an internal IP address. You can save a lot of
bandwidth and time this way.
-Nathan
-Original Message-
From: 陈思淼 [mailto:[EMAIL PROTECTED]
Sent: Friday, September 26, 2008 10:47 AM
To: Maven Users List
Subject: Re: Are we blocked by central Maven repo?
we didn't do that kind of thing. we have a company-level artifactory
repository.someone didn't follow the rule but most of us are good citizen,
and follow the maven RULE,
Is maven block strategy to block IP too strict?
Can I do anything to Fix it Up?
2008/9/26 Wayne Fay <[EMAIL PROTECTED]>
> It is possible to get blocked if you are acting as a bad citizen
> (downloading the entire Central repo using wget, for example). Have
> you (or someone else at your company) attempted to do this from your
> IP address?
>
> If not, the repo is probably just busy, or you had some random
> Internet connection failure. Try again. "Normal" Maven usage of the
> repo will not get you blocked.
>
> Wayne
>
> On Fri, Sep 26, 2008 at 7:37 AM, 陈思淼 <[EMAIL PROTECTED]> wrote:
> > This's log from artifactory.
> >
> > 2008-09-26 22:27:28,025 [WARN ] (RemoteRepoBase.java:259{10}) -
> repo1:
> > Error in getting information for 'org/apache/maven
> > /maven-model/2.0.4/maven-model-2.0.4.pom.sha1'
> > (org.apache.commons.httpclient.ConnectionPoolTimeoutException: Timeout
> > waiting
> > for connection).
> >
> > we company only have one outlet IP address ,someone may download Maven
> from
> > apache and didn't set the Mirror of central in the conf/setting.xml. so
> they
> > download the pom directly from central? Is that the reason why the
> central
> > repo block our IP address?
> >
>
--
CONFIDENTIALITY NOTICE This message and any included attachments are from
Cerner Corporation and are intended only for the addressee. The information
contained in this message is confidential and may constitute inside or
non-public information under international, federal, or state securities laws.
Unauthorized forwarding, printing, copying, distribution, or use of such
information is strictly prohibited and may be unlawful. If you are not the
addressee, please promptly delete this message and notify the sender of the
delivery error by e-mail or you may call Cerner's corporate offices in Kansas
City, Missouri, U.S.A at (+1) (816)221-1024.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: Are we blocked by central Maven repo?
have been noticeable net outages and delays due to weather related phenomenon
here in the US..so I would advise working 'locally' as much as possible..
Martin
__
Disclaimer and confidentiality note
Everything in this e-mail and any attachments relates to the official business
of Sender. This transmission is of a confidential nature and Sender does not
endorse distribution to any party other than intended recipient. Sender does
not necessarily endorse content contained within this transmission.
> Date: Fri, 26 Sep 2008 23:47:02 +0800
> From: [EMAIL PROTECTED]
> To: users@maven.apache.org
> Subject: Re: Are we blocked by central Maven repo?
>
> we didn't do that kind of thing. we have a company-level artifactory
> repository.someone didn't follow the rule but most of us are good citizen,
> and follow the maven RULE,
> Is maven block strategy to block IP too strict?
> Can I do anything to Fix it Up?
>
>
>
> 2008/9/26 Wayne Fay <[EMAIL PROTECTED]>
>
> > It is possible to get blocked if you are acting as a bad citizen
> > (downloading the entire Central repo using wget, for example). Have
> > you (or someone else at your company) attempted to do this from your
> > IP address?
> >
> > If not, the repo is probably just busy, or you had some random
> > Internet connection failure. Try again. "Normal" Maven usage of the
> > repo will not get you blocked.
> >
> > Wayne
> >
> > On Fri, Sep 26, 2008 at 7:37 AM, 陈思淼 <[EMAIL PROTECTED]> wrote:
> > > This's log from artifactory.
> > >
> > > 2008-09-26 22:27:28,025 [WARN ] (RemoteRepoBase.java:259{10}) -
> > repo1:
> > > Error in getting information for 'org/apache/maven
> > > /maven-model/2.0.4/maven-model-2.0.4.pom.sha1'
> > > (org.apache.commons.httpclient.ConnectionPoolTimeoutException: Timeout
> > > waiting
> > > for connection).
> > >
> > > we company only have one outlet IP address ,someone may download Maven
> > from
> > > apache and didn't set the Mirror of central in the conf/setting.xml. so
> > they
> > > download the pom directly from central? Is that the reason why the
> > central
> > > repo block our IP address?
> > >
> >
_
See how Windows Mobile brings your life together―at home, work, or on the go.
http://clk.atdmt.com/MRT/go/msnnkwxp1020093182mrt/direct/01/
Re: Are we blocked by central Maven repo?
we didn't do that kind of thing. we have a company-level artifactory
repository.someone didn't follow the rule but most of us are good citizen,
and follow the maven RULE,
Is maven block strategy to block IP too strict?
Can I do anything to Fix it Up?
2008/9/26 Wayne Fay <[EMAIL PROTECTED]>
> It is possible to get blocked if you are acting as a bad citizen
> (downloading the entire Central repo using wget, for example). Have
> you (or someone else at your company) attempted to do this from your
> IP address?
>
> If not, the repo is probably just busy, or you had some random
> Internet connection failure. Try again. "Normal" Maven usage of the
> repo will not get you blocked.
>
> Wayne
>
> On Fri, Sep 26, 2008 at 7:37 AM, 陈思淼 <[EMAIL PROTECTED]> wrote:
> > This's log from artifactory.
> >
> > 2008-09-26 22:27:28,025 [WARN ] (RemoteRepoBase.java:259{10}) -
> repo1:
> > Error in getting information for 'org/apache/maven
> > /maven-model/2.0.4/maven-model-2.0.4.pom.sha1'
> > (org.apache.commons.httpclient.ConnectionPoolTimeoutException: Timeout
> > waiting
> > for connection).
> >
> > we company only have one outlet IP address ,someone may download Maven
> from
> > apache and didn't set the Mirror of central in the conf/setting.xml. so
> they
> > download the pom directly from central? Is that the reason why the
> central
> > repo block our IP address?
> >
>
Re: Are we blocked by central Maven repo?
It is possible to get blocked if you are acting as a bad citizen
(downloading the entire Central repo using wget, for example). Have
you (or someone else at your company) attempted to do this from your
IP address?
If not, the repo is probably just busy, or you had some random
Internet connection failure. Try again. "Normal" Maven usage of the
repo will not get you blocked.
Wayne
On Fri, Sep 26, 2008 at 7:37 AM, 陈思淼 <[EMAIL PROTECTED]> wrote:
> This's log from artifactory.
>
> 2008-09-26 22:27:28,025 [WARN ] (RemoteRepoBase.java:259{10}) - repo1:
> Error in getting information for 'org/apache/maven
> /maven-model/2.0.4/maven-model-2.0.4.pom.sha1'
> (org.apache.commons.httpclient.ConnectionPoolTimeoutException: Timeout
> waiting
> for connection).
>
> we company only have one outlet IP address ,someone may download Maven from
> apache and didn't set the Mirror of central in the conf/setting.xml. so they
> download the pom directly from central? Is that the reason why the central
> repo block our IP address?
>
Re: Are we blocked by central Maven repo?
You might be getting redirected. I had to change mine to
http://repo-cogent.maven.org/maven2/, but I don't get the redirect
from my home connection, so it might depend on the IP address you
have.
I'd suggest you to do a wget and see where the redirect tells you to go to.
Gabriel
2008/9/26 陈思淼 <[EMAIL PROTECTED]>:
> This's log from artifactory.
>
> 2008-09-26 22:27:28,025 [WARN ] (RemoteRepoBase.java:259{10}) - repo1:
> Error in getting information for 'org/apache/maven
> /maven-model/2.0.4/maven-model-2.0.4.pom.sha1'
> (org.apache.commons.httpclient.ConnectionPoolTimeoutException: Timeout
> waiting
> for connection).
>
> we company only have one outlet IP address ,someone may download Maven from
> apache and didn't set the Mirror of central in the conf/setting.xml. so they
> download the pom directly from central? Is that the reason why the central
> repo block our IP address?
>
