Thanks for the further testing. Now I'm questioning how in the world I was able to see the same error.
I will continue my testing to attempt to reproduce the error. Also, the Apache Timeout 300 should behave exactly as Mark said. I'm still trying to find what is causing this failure. On Mon, Jun 20, 2016 at 9:49 PM, Merlijn Sebrechts < merlijn.sebrec...@gmail.com> wrote: > So if that's 300 days, that should give you plenty of time ;) > > 2016-06-20 20:38 GMT+02:00 Merlijn Sebrechts <merlijn.sebrec...@gmail.com> > : > >> From the bug report: "The timeout is apache setting `Timeout` which >> defaults to 300." >> >> https://bugs.launchpad.net/ubuntu/+source/charm/+bug/1592822 >> >> 2016-06-20 20:32 GMT+02:00 Tom Barber <t...@analytical-labs.com>: >> >>> If I had to upload 270mb from my home I'd be waiting 3 weeks..... what's >>> the timeout set to? ;) >>> On 20 Jun 2016 19:30, "Merlijn Sebrechts" <merlijn.sebrec...@gmail.com> >>> wrote: >>> >>>> Thanks for the explanation, Jay. >>>> >>>> I did some further testing. Charm upload fails for a 270MB Charm both >>>> from my home, my work and our datacenter. >>>> >>>> - The datacenter is connected directly to Belnet (upload bandwith >>>> ~300Mbit/s). >>>> - My upload bandwidth at home is ~ 3 Mbps (speedtest.net) although >>>> during upload, system monitor shows ~400KiB/s. >>>> >>>> This causes me to think there is more at play here then large file + >>>> slow internet... Let me know if I can help to further debug this problem. >>>> >>>> >>>> As an aside; I don't consider 270MB to be that large. Some examples: >>>> >>>> >>>> - Kubernetes is ~1G >>>> - Ubuntu docker base image is ~200MB >>>> >>>> >>>> I think this is stuff we should be able to handle... >>>> >>>> 2016-06-20 18:41 GMT+02:00 Jay Wren <jay.w...@canonical.com>: >>>> >>>>> Yes, files are broken up into many TCP packets, and they are all >>>>> transmitted over a single TCP connection. TCP is a complex protocol which >>>>> is well documented, so I'll not repeat that here. If you want lots of >>>>> details, wikipedia is not bad: >>>>> https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Protocol_operation >>>>> >>>>> In the abstract, when you connect to a server using TCP it is >>>>> identified by a 4-tuple of source address, source port, target ip address, >>>>> target port. These connections consume server resources and indeed a >>>>> connection exhaustion is a popular denial of service attack. >>>>> >>>>> You are getting a tcp timeout due to of file size because the time it >>>>> takes to send the entire content is longer than the TCP connection >>>>> timeout. >>>>> >>>>> Yes, the resource upload command to charmstore will also be affected >>>>> by this. Luckily, resources also have the ability to be uploaded >>>>> specifically to a model, which might have greater network data rates from >>>>> the resource uploader. >>>>> >>>>> -- >>>>> Jay >>>>> >>>>> >>>>> On Mon, Jun 20, 2016 at 7:04 PM, Merlijn Sebrechts < >>>>> merlijn.sebrec...@gmail.com> wrote: >>>>> >>>>>> Thanks for looking into this! I'll try the compression and see if >>>>>> that works. >>>>>> >>>>>> Just curious; why does filesize affect tcp connection timeout? Aren't >>>>>> the files broken up into a bunch of smaller tcp packets? Filesize should >>>>>> only affect the number of tcp packets, not the size of the tcp packets? >>>>>> So >>>>>> getting a tcp timeout because of filesize seems strange to me... Any idea >>>>>> what exactly goes wrong here? >>>>>> >>>>>> Also, now that I think of it, the resource upload command might also >>>>>> be affected by this, if it uses the same library and similar backend >>>>>> infrastructure? I'll test this out. >>>>>> >>>>>> >>>>>> Op maandag 20 juni 2016 heeft Jay Wren <jay.w...@canonical.com> het >>>>>> volgende geschreven: >>>>>> > Hello Merlijn, >>>>>> > I can replicate the problem and I can work around it by using a >>>>>> faster internet connection. >>>>>> > At some point, tcp connections have to time out. I can only >>>>>> replicate the issue when that timeout is reached. If you have the means >>>>>> to >>>>>> relocate to a faster internet connection temporarily for pushing to >>>>>> charmstore, please do. You might also try recompressing any items in the >>>>>> charm using a higher compression level. xz -9 instead of gzip -3 or >>>>>> whatever things may be using now. >>>>>> > We are aware this is a poor longterm solution. We are investigating >>>>>> better solutions for uploads. As you've mentioned, resources will also >>>>>> help >>>>>> the situation. >>>>>> > I am sorry that I do not have a better solution. >>>>>> > -- >>>>>> > Jay >>>>>> > On Fri, Jun 17, 2016 at 4:29 PM, Rick Harding < >>>>>> rick.hard...@canonical.com> wrote: >>>>>> >> >>>>>> >> Merlijn, thanks. I'm going to bet there's an issue with http >>>>>> request sizes for the charmstore that the charm command talks do as we've >>>>>> got some layers (Apache, Squid) in front of the actual application. The >>>>>> team is looking into it. Thanks for giving us the heads up. >>>>>> >> On Wed, Jun 15, 2016 at 10:28 AM Merlijn Sebrechts < >>>>>> merlijn.sebrec...@gmail.com> wrote: >>>>>> >>> >>>>>> >>> Hi all >>>>>> >>> >>>>>> >>> I've hit a roadblock in setting up my CI pipeline. I have a charm >>>>>> with a Java sdk blob of ~200MB. Pushing this charm to the store causes >>>>>> the >>>>>> tool to crash. I put a bug report here: >>>>>> https://bugs.launchpad.net/juju/+bug/1592822 >>>>>> >>> Juju resources would fix my problem, but then I'd need to move to >>>>>> Juju 2.0 and I'm not ready to do that yet. >>>>>> >>> >>>>>> >>> >>>>>> >>> Kind regards >>>>>> >>> Merlijn Sebrechts >>>>>> >>> -- >>>>>> >>> Juju mailing list >>>>>> >>> Juju@lists.ubuntu.com >>>>>> >>> Modify settings or unsubscribe at: >>>>>> https://lists.ubuntu.com/mailman/listinfo/juju >>>>>> >> >>>>>> >> -- >>>>>> >> Juju mailing list >>>>>> >> Juju@lists.ubuntu.com >>>>>> >> Modify settings or unsubscribe at: >>>>>> https://lists.ubuntu.com/mailman/listinfo/juju >>>>>> >> >>>>>> > >>>>>> > >>>>>> >>>>> >>>>> >>>> >>>> -- >>>> Juju mailing list >>>> Juju@lists.ubuntu.com >>>> Modify settings or unsubscribe at: >>>> https://lists.ubuntu.com/mailman/listinfo/juju >>>> >>>> >> >
-- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
