>From the bug report: "The timeout is apache setting `Timeout` which defaults to 300."
https://bugs.launchpad.net/ubuntu/+source/charm/+bug/1592822 2016-06-20 20:32 GMT+02:00 Tom Barber <t...@analytical-labs.com>: > If I had to upload 270mb from my home I'd be waiting 3 weeks..... what's > the timeout set to? ;) > On 20 Jun 2016 19:30, "Merlijn Sebrechts" <merlijn.sebrec...@gmail.com> > wrote: > >> Thanks for the explanation, Jay. >> >> I did some further testing. Charm upload fails for a 270MB Charm both >> from my home, my work and our datacenter. >> >> - The datacenter is connected directly to Belnet (upload bandwith >> ~300Mbit/s). >> - My upload bandwidth at home is ~ 3 Mbps (speedtest.net) although >> during upload, system monitor shows ~400KiB/s. >> >> This causes me to think there is more at play here then large file + slow >> internet... Let me know if I can help to further debug this problem. >> >> >> As an aside; I don't consider 270MB to be that large. Some examples: >> >> >> - Kubernetes is ~1G >> - Ubuntu docker base image is ~200MB >> >> >> I think this is stuff we should be able to handle... >> >> 2016-06-20 18:41 GMT+02:00 Jay Wren <jay.w...@canonical.com>: >> >>> Yes, files are broken up into many TCP packets, and they are all >>> transmitted over a single TCP connection. TCP is a complex protocol which >>> is well documented, so I'll not repeat that here. If you want lots of >>> details, wikipedia is not bad: >>> https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Protocol_operation >>> >>> In the abstract, when you connect to a server using TCP it is identified >>> by a 4-tuple of source address, source port, target ip address, target >>> port. These connections consume server resources and indeed a connection >>> exhaustion is a popular denial of service attack. >>> >>> You are getting a tcp timeout due to of file size because the time it >>> takes to send the entire content is longer than the TCP connection timeout. >>> >>> Yes, the resource upload command to charmstore will also be affected by >>> this. Luckily, resources also have the ability to be uploaded specifically >>> to a model, which might have greater network data rates from the resource >>> uploader. >>> >>> -- >>> Jay >>> >>> >>> On Mon, Jun 20, 2016 at 7:04 PM, Merlijn Sebrechts < >>> merlijn.sebrec...@gmail.com> wrote: >>> >>>> Thanks for looking into this! I'll try the compression and see if that >>>> works. >>>> >>>> Just curious; why does filesize affect tcp connection timeout? Aren't >>>> the files broken up into a bunch of smaller tcp packets? Filesize should >>>> only affect the number of tcp packets, not the size of the tcp packets? So >>>> getting a tcp timeout because of filesize seems strange to me... Any idea >>>> what exactly goes wrong here? >>>> >>>> Also, now that I think of it, the resource upload command might also be >>>> affected by this, if it uses the same library and similar backend >>>> infrastructure? I'll test this out. >>>> >>>> >>>> Op maandag 20 juni 2016 heeft Jay Wren <jay.w...@canonical.com> het >>>> volgende geschreven: >>>> > Hello Merlijn, >>>> > I can replicate the problem and I can work around it by using a >>>> faster internet connection. >>>> > At some point, tcp connections have to time out. I can only replicate >>>> the issue when that timeout is reached. If you have the means to relocate >>>> to a faster internet connection temporarily for pushing to charmstore, >>>> please do. You might also try recompressing any items in the charm using a >>>> higher compression level. xz -9 instead of gzip -3 or whatever things may >>>> be using now. >>>> > We are aware this is a poor longterm solution. We are investigating >>>> better solutions for uploads. As you've mentioned, resources will also help >>>> the situation. >>>> > I am sorry that I do not have a better solution. >>>> > -- >>>> > Jay >>>> > On Fri, Jun 17, 2016 at 4:29 PM, Rick Harding < >>>> rick.hard...@canonical.com> wrote: >>>> >> >>>> >> Merlijn, thanks. I'm going to bet there's an issue with http request >>>> sizes for the charmstore that the charm command talks do as we've got some >>>> layers (Apache, Squid) in front of the actual application. The team is >>>> looking into it. Thanks for giving us the heads up. >>>> >> On Wed, Jun 15, 2016 at 10:28 AM Merlijn Sebrechts < >>>> merlijn.sebrec...@gmail.com> wrote: >>>> >>> >>>> >>> Hi all >>>> >>> >>>> >>> I've hit a roadblock in setting up my CI pipeline. I have a charm >>>> with a Java sdk blob of ~200MB. Pushing this charm to the store causes the >>>> tool to crash. I put a bug report here: >>>> https://bugs.launchpad.net/juju/+bug/1592822 >>>> >>> Juju resources would fix my problem, but then I'd need to move to >>>> Juju 2.0 and I'm not ready to do that yet. >>>> >>> >>>> >>> >>>> >>> Kind regards >>>> >>> Merlijn Sebrechts >>>> >>> -- >>>> >>> Juju mailing list >>>> >>> Juju@lists.ubuntu.com >>>> >>> Modify settings or unsubscribe at: >>>> https://lists.ubuntu.com/mailman/listinfo/juju >>>> >> >>>> >> -- >>>> >> Juju mailing list >>>> >> Juju@lists.ubuntu.com >>>> >> Modify settings or unsubscribe at: >>>> https://lists.ubuntu.com/mailman/listinfo/juju >>>> >> >>>> > >>>> > >>>> >>> >>> >> >> -- >> Juju mailing list >> Juju@lists.ubuntu.com >> Modify settings or unsubscribe at: >> https://lists.ubuntu.com/mailman/listinfo/juju >> >>
-- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
