Re: Charm push crashes when uploading big charms

[Tom Barber]

Okay bzip -9'ed it, down to 760mb and tried to attach it via my server in a
data centre... same failure.

Tom

--

Director Meteorite.bi - Saiku Analytics Founder
Tel: +44(0)5603641316

(Thanks to the Saiku community we reached our Kickstart

goal, but you can always help by sponsoring the project
)

On 19 July 2016 at 07:51, Tom Barber  wrote:

> Yeah, but in life, i can ftp/sftp/http upload a file to a.n.other service
> without a timeout irrespective of size. Clearly it will take a long time on
> crappy FTTC ADSL but it does work and doesn't timeout :)
>
> --
>
> Director Meteorite.bi - Saiku Analytics Founder
> Tel: +44(0)5603641316
>
> (Thanks to the Saiku community we reached our Kickstart
> 
> goal, but you can always help by sponsoring the project
> )
>
> On 19 July 2016 at 07:49, José Antonio Rey  wrote:
>
>> Oh! Huh. Then maybe it is a size constraint :)
>>
>> On Tue, Jul 19, 2016, 01:48 Tom Barber  wrote:
>>
>>> Yeah, but you could also argue this hotel wifi is a lot quicker than my
>>> home wifi, so it still seems a bit flawed if you need to push from fast
>>> pipes! Anyway I shall try and find out.
>>>
>>> Tom
>>>
>>> --
>>>
>>> Director Meteorite.bi - Saiku Analytics Founder
>>> Tel: +44(0)5603641316
>>>
>>> (Thanks to the Saiku community we reached our Kickstart
>>> 
>>> goal, but you can always help by sponsoring the project
>>> )
>>>
>>> On 19 July 2016 at 07:45, José Antonio Rey  wrote:
>>>
 Hotel wifi may be the problem. Maybe try uploading it to a server with
 a fast upload speed, and then pusing from that server instead?

 On 07/19/2016 01:33 AM, Tom Barber wrote:

> 825mb on hotel wifi Rick, I'll squash it further  and try again on data
> centre pipes.
>
> Tom
>
> --
>
> Director Meteorite.bi - Saiku Analytics Founder
> Tel: +44(0)5603641316
>
> (Thanks to the Saiku community we reached our Kickstart
> <
> http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/
> >
> goal, but you can always help by sponsoring the project
> )
>
> On 19 July 2016 at 01:06, Rick Harding  > wrote:
>
> How big/long to upload was your resource Tom? I know the team
> bumped
> it while they get some better insight into tracking/quota'ing sizes
> and I'm curious where you got denied.
>
>
> On Mon, Jul 18, 2016, 5:53 PM Tom Barber  > wrote:
>
> Awww Merlijn
>
> You got my hopes up then I got the same error as usual
> *sadface*
>
> Tom
>
> --
>
> Director Meteorite.bi - Saiku Analytics Founder
> Tel: +44(0)5603641316 
>
> (Thanks to the Saiku community we reached our Kickstart
> <
> http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/
> >
> goal, but you can always help by sponsoring the project
> )
>
> On 18 July 2016 at 14:39, Tom Barber  > wrote:
>
> Cool thanks, I'll check it shortly.
>
> Tom
>
> --
>
> Director Meteorite.bi - Saiku Analytics Founder
> Tel: +44(0)5603641316 
>
> (Thanks to the Saiku community we reached our Kickstart
> <
> http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/
> >
> goal, but you can always help by sponsoring the project
> )
>
> On 18 July 2016 at 14:37, Merlijn Sebrechts
>  > wrote:
>
> Hi Jay
>
>
> I'm able to push a 270 mb large Charm now. This was not
> possible before, so it seems the issue is fixed for me.
> I've added Tom in cc so he can check too. He was having
> the same issue if I'm not mistaken. I'll come back to
> you if the issue magically reappears.. :)
>
>
>
> Kind regards
>>

Re: Charm push crashes when uploading big charms

[Tom Barber]

Yeah, but in life, i can ftp/sftp/http upload a file to a.n.other service
without a timeout irrespective of size. Clearly it will take a long time on
crappy FTTC ADSL but it does work and doesn't timeout :)

--

Director Meteorite.bi - Saiku Analytics Founder
Tel: +44(0)5603641316

(Thanks to the Saiku community we reached our Kickstart

goal, but you can always help by sponsoring the project
)

On 19 July 2016 at 07:49, José Antonio Rey  wrote:

> Oh! Huh. Then maybe it is a size constraint :)
>
> On Tue, Jul 19, 2016, 01:48 Tom Barber  wrote:
>
>> Yeah, but you could also argue this hotel wifi is a lot quicker than my
>> home wifi, so it still seems a bit flawed if you need to push from fast
>> pipes! Anyway I shall try and find out.
>>
>> Tom
>>
>> --
>>
>> Director Meteorite.bi - Saiku Analytics Founder
>> Tel: +44(0)5603641316
>>
>> (Thanks to the Saiku community we reached our Kickstart
>> 
>> goal, but you can always help by sponsoring the project
>> )
>>
>> On 19 July 2016 at 07:45, José Antonio Rey  wrote:
>>
>>> Hotel wifi may be the problem. Maybe try uploading it to a server with a
>>> fast upload speed, and then pusing from that server instead?
>>>
>>> On 07/19/2016 01:33 AM, Tom Barber wrote:
>>>
 825mb on hotel wifi Rick, I'll squash it further  and try again on data
 centre pipes.

 Tom

 --

 Director Meteorite.bi - Saiku Analytics Founder
 Tel: +44(0)5603641316

 (Thanks to the Saiku community we reached our Kickstart
 <
 http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/
 >
 goal, but you can always help by sponsoring the project
 )

 On 19 July 2016 at 01:06, Rick Harding >>> > wrote:

 How big/long to upload was your resource Tom? I know the team bumped
 it while they get some better insight into tracking/quota'ing sizes
 and I'm curious where you got denied.


 On Mon, Jul 18, 2016, 5:53 PM Tom Barber >>> > wrote:

 Awww Merlijn

 You got my hopes up then I got the same error as usual
 *sadface*

 Tom

 --

 Director Meteorite.bi - Saiku Analytics Founder
 Tel: +44(0)5603641316 

 (Thanks to the Saiku community we reached our Kickstart
 <
 http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/
 >
 goal, but you can always help by sponsoring the project
 )

 On 18 July 2016 at 14:39, Tom Barber >>> > wrote:

 Cool thanks, I'll check it shortly.

 Tom

 --

 Director Meteorite.bi - Saiku Analytics Founder
 Tel: +44(0)5603641316 

 (Thanks to the Saiku community we reached our Kickstart
 <
 http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/
 >
 goal, but you can always help by sponsoring the project
 )

 On 18 July 2016 at 14:37, Merlijn Sebrechts
 >>> > wrote:

 Hi Jay


 I'm able to push a 270 mb large Charm now. This was not
 possible before, so it seems the issue is fixed for me.
 I've added Tom in cc so he can check too. He was having
 the same issue if I'm not mistaken. I'll come back to
 you if the issue magically reappears.. :)



 Kind regards
 Merlijn


 2016-07-08 21:04 GMT+02:00 Jay Wren
 mailto:jay.w...@canonical.com
 >>:

 We have made a configuration change which should
 impact this. If you are able to try again, please
 do so.

 Also, we are continuing to track this issue and add
 monitoring and instrumentation around this area of
 the charmstore to be better able to respond to such
 issues in the future.

 Thanks,
>>>

Re: Charm push crashes when uploading big charms

[José Antonio Rey]

Oh! Huh. Then maybe it is a size constraint :)

On Tue, Jul 19, 2016, 01:48 Tom Barber  wrote:

> Yeah, but you could also argue this hotel wifi is a lot quicker than my
> home wifi, so it still seems a bit flawed if you need to push from fast
> pipes! Anyway I shall try and find out.
>
> Tom
>
> --
>
> Director Meteorite.bi - Saiku Analytics Founder
> Tel: +44(0)5603641316
>
> (Thanks to the Saiku community we reached our Kickstart
> 
> goal, but you can always help by sponsoring the project
> )
>
> On 19 July 2016 at 07:45, José Antonio Rey  wrote:
>
>> Hotel wifi may be the problem. Maybe try uploading it to a server with a
>> fast upload speed, and then pusing from that server instead?
>>
>> On 07/19/2016 01:33 AM, Tom Barber wrote:
>>
>>> 825mb on hotel wifi Rick, I'll squash it further  and try again on data
>>> centre pipes.
>>>
>>> Tom
>>>
>>> --
>>>
>>> Director Meteorite.bi - Saiku Analytics Founder
>>> Tel: +44(0)5603641316
>>>
>>> (Thanks to the Saiku community we reached our Kickstart
>>> <
>>> http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/
>>> >
>>> goal, but you can always help by sponsoring the project
>>> )
>>>
>>> On 19 July 2016 at 01:06, Rick Harding >> > wrote:
>>>
>>> How big/long to upload was your resource Tom? I know the team bumped
>>> it while they get some better insight into tracking/quota'ing sizes
>>> and I'm curious where you got denied.
>>>
>>>
>>> On Mon, Jul 18, 2016, 5:53 PM Tom Barber >> > wrote:
>>>
>>> Awww Merlijn
>>>
>>> You got my hopes up then I got the same error as usual
>>> *sadface*
>>>
>>> Tom
>>>
>>> --
>>>
>>> Director Meteorite.bi - Saiku Analytics Founder
>>> Tel: +44(0)5603641316 
>>>
>>> (Thanks to the Saiku community we reached our Kickstart
>>> <
>>> http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/
>>> >
>>> goal, but you can always help by sponsoring the project
>>> )
>>>
>>> On 18 July 2016 at 14:39, Tom Barber >> > wrote:
>>>
>>> Cool thanks, I'll check it shortly.
>>>
>>> Tom
>>>
>>> --
>>>
>>> Director Meteorite.bi - Saiku Analytics Founder
>>> Tel: +44(0)5603641316 
>>>
>>> (Thanks to the Saiku community we reached our Kickstart
>>> <
>>> http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/
>>> >
>>> goal, but you can always help by sponsoring the project
>>> )
>>>
>>> On 18 July 2016 at 14:37, Merlijn Sebrechts
>>> >> > wrote:
>>>
>>> Hi Jay
>>>
>>>
>>> I'm able to push a 270 mb large Charm now. This was not
>>> possible before, so it seems the issue is fixed for me.
>>> I've added Tom in cc so he can check too. He was having
>>> the same issue if I'm not mistaken. I'll come back to
>>> you if the issue magically reappears.. :)
>>>
>>>
>>>
>>> Kind regards
>>> Merlijn
>>>
>>>
>>> 2016-07-08 21:04 GMT+02:00 Jay Wren
>>> mailto:jay.w...@canonical.com
>>> >>:
>>>
>>> We have made a configuration change which should
>>> impact this. If you are able to try again, please do
>>> so.
>>>
>>> Also, we are continuing to track this issue and add
>>> monitoring and instrumentation around this area of
>>> the charmstore to be better able to respond to such
>>> issues in the future.
>>>
>>> Thanks,
>>> --
>>> Jay
>>>
>>>
>>> On Wed, Jun 22, 2016 at 10:30 AM, Stuart Bishop
>>> >> > wrote:
>>>
>>> On 21 June 2016 at 02:40, Jay Wren
>>> >> > wrote:
>>> > Thanks for the further testing.
>>> >
>>> > Now I'm questioning how in the world I was
>>> able to see the same error.
>>> >
>>> > I will continue my testing to attempt to
>>> reproduce the error.
>>>

Re: Charm push crashes when uploading big charms

[Tom Barber]

Yeah, but you could also argue this hotel wifi is a lot quicker than my
home wifi, so it still seems a bit flawed if you need to push from fast
pipes! Anyway I shall try and find out.

Tom

--

Director Meteorite.bi - Saiku Analytics Founder
Tel: +44(0)5603641316

(Thanks to the Saiku community we reached our Kickstart

goal, but you can always help by sponsoring the project
)

On 19 July 2016 at 07:45, José Antonio Rey  wrote:

> Hotel wifi may be the problem. Maybe try uploading it to a server with a
> fast upload speed, and then pusing from that server instead?
>
> On 07/19/2016 01:33 AM, Tom Barber wrote:
>
>> 825mb on hotel wifi Rick, I'll squash it further  and try again on data
>> centre pipes.
>>
>> Tom
>>
>> --
>>
>> Director Meteorite.bi - Saiku Analytics Founder
>> Tel: +44(0)5603641316
>>
>> (Thanks to the Saiku community we reached our Kickstart
>> <
>> http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/
>> >
>> goal, but you can always help by sponsoring the project
>> )
>>
>> On 19 July 2016 at 01:06, Rick Harding > > wrote:
>>
>> How big/long to upload was your resource Tom? I know the team bumped
>> it while they get some better insight into tracking/quota'ing sizes
>> and I'm curious where you got denied.
>>
>>
>> On Mon, Jul 18, 2016, 5:53 PM Tom Barber > > wrote:
>>
>> Awww Merlijn
>>
>> You got my hopes up then I got the same error as usual
>> *sadface*
>>
>> Tom
>>
>> --
>>
>> Director Meteorite.bi - Saiku Analytics Founder
>> Tel: +44(0)5603641316 
>>
>> (Thanks to the Saiku community we reached our Kickstart
>> <
>> http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/
>> >
>> goal, but you can always help by sponsoring the project
>> )
>>
>> On 18 July 2016 at 14:39, Tom Barber > > wrote:
>>
>> Cool thanks, I'll check it shortly.
>>
>> Tom
>>
>> --
>>
>> Director Meteorite.bi - Saiku Analytics Founder
>> Tel: +44(0)5603641316 
>>
>> (Thanks to the Saiku community we reached our Kickstart
>> <
>> http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/
>> >
>> goal, but you can always help by sponsoring the project
>> )
>>
>> On 18 July 2016 at 14:37, Merlijn Sebrechts
>> > > wrote:
>>
>> Hi Jay
>>
>>
>> I'm able to push a 270 mb large Charm now. This was not
>> possible before, so it seems the issue is fixed for me.
>> I've added Tom in cc so he can check too. He was having
>> the same issue if I'm not mistaken. I'll come back to
>> you if the issue magically reappears.. :)
>>
>>
>>
>> Kind regards
>> Merlijn
>>
>>
>> 2016-07-08 21:04 GMT+02:00 Jay Wren
>> mailto:jay.w...@canonical.com>>:
>>
>> We have made a configuration change which should
>> impact this. If you are able to try again, please do
>> so.
>>
>> Also, we are continuing to track this issue and add
>> monitoring and instrumentation around this area of
>> the charmstore to be better able to respond to such
>> issues in the future.
>>
>> Thanks,
>> --
>> Jay
>>
>>
>> On Wed, Jun 22, 2016 at 10:30 AM, Stuart Bishop
>> > > wrote:
>>
>> On 21 June 2016 at 02:40, Jay Wren
>> > > wrote:
>> > Thanks for the further testing.
>> >
>> > Now I'm questioning how in the world I was able
>> to see the same error.
>> >
>> > I will continue my testing to attempt to
>> reproduce the error.
>> >
>> > Also, the Apache Timeout 300 should behave
>> exactly as Mark said. I'm still
>> > trying to find what is causing this failure.
>>
>> The error from the bug report origi

Re: Charm push crashes when uploading big charms

[José Antonio Rey]

Hotel wifi may be the problem. Maybe try uploading it to a server with a 
fast upload speed, and then pusing from that server instead?


On 07/19/2016 01:33 AM, Tom Barber wrote:

825mb on hotel wifi Rick, I'll squash it further  and try again on data
centre pipes.

Tom

--

Director Meteorite.bi - Saiku Analytics Founder
Tel: +44(0)5603641316

(Thanks to the Saiku community we reached our Kickstart

goal, but you can always help by sponsoring the project
)

On 19 July 2016 at 01:06, Rick Harding mailto:rick.hard...@canonical.com>> wrote:

How big/long to upload was your resource Tom? I know the team bumped
it while they get some better insight into tracking/quota'ing sizes
and I'm curious where you got denied.


On Mon, Jul 18, 2016, 5:53 PM Tom Barber mailto:t...@analytical-labs.com>> wrote:

Awww Merlijn

You got my hopes up then I got the same error as usual *sadface*

Tom

--

Director Meteorite.bi - Saiku Analytics Founder
Tel: +44(0)5603641316 

(Thanks to the Saiku community we reached our Kickstart


goal, but you can always help by sponsoring the project
)

On 18 July 2016 at 14:39, Tom Barber mailto:t...@analytical-labs.com>> wrote:

Cool thanks, I'll check it shortly.

Tom

--

Director Meteorite.bi - Saiku Analytics Founder
Tel: +44(0)5603641316 

(Thanks to the Saiku community we reached our Kickstart


goal, but you can always help by sponsoring the project
)

On 18 July 2016 at 14:37, Merlijn Sebrechts
mailto:merlijn.sebrec...@gmail.com>> wrote:

Hi Jay


I'm able to push a 270 mb large Charm now. This was not
possible before, so it seems the issue is fixed for me.
I've added Tom in cc so he can check too. He was having
the same issue if I'm not mistaken. I'll come back to
you if the issue magically reappears.. :)



Kind regards
Merlijn


2016-07-08 21:04 GMT+02:00 Jay Wren
mailto:jay.w...@canonical.com>>:

We have made a configuration change which should
impact this. If you are able to try again, please do so.

Also, we are continuing to track this issue and add
monitoring and instrumentation around this area of
the charmstore to be better able to respond to such
issues in the future.

Thanks,
--
Jay


On Wed, Jun 22, 2016 at 10:30 AM, Stuart Bishop
mailto:stuart.bis...@canonical.com>> wrote:

On 21 June 2016 at 02:40, Jay Wren
mailto:jay.w...@canonical.com>> wrote:
> Thanks for the further testing.
>
> Now I'm questioning how in the world I was able to 
see the same error.
>
> I will continue my testing to attempt to reproduce 
the error.
>
> Also, the Apache Timeout 300 should behave exactly as 
Mark said. I'm still
> trying to find what is causing this failure.

The error from the bug report originates from
Squid, which is probably
buffering the upload. The timeout might happen
between Squid and
Apache if it takes to long to upload and there
is no keep-alive
mechanism in place.

--
Stuart Bishop mailto:stuart.bis...@canonical.com>>





--
Juju mailing list
Juju@lists.ubuntu.com 
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju







--
José Antonio Rey

--
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm push crashes when uploading big charms

[Tom Barber]

825mb on hotel wifi Rick, I'll squash it further  and try again on data
centre pipes.

Tom

--

Director Meteorite.bi - Saiku Analytics Founder
Tel: +44(0)5603641316

(Thanks to the Saiku community we reached our Kickstart

goal, but you can always help by sponsoring the project
)

On 19 July 2016 at 01:06, Rick Harding  wrote:

> How big/long to upload was your resource Tom? I know the team bumped it
> while they get some better insight into tracking/quota'ing sizes and I'm
> curious where you got denied.
>
> On Mon, Jul 18, 2016, 5:53 PM Tom Barber  wrote:
>
>> Awww Merlijn
>>
>> You got my hopes up then I got the same error as usual *sadface*
>>
>> Tom
>>
>> --
>>
>> Director Meteorite.bi - Saiku Analytics Founder
>> Tel: +44(0)5603641316
>>
>> (Thanks to the Saiku community we reached our Kickstart
>> 
>> goal, but you can always help by sponsoring the project
>> )
>>
>> On 18 July 2016 at 14:39, Tom Barber  wrote:
>>
>>> Cool thanks, I'll check it shortly.
>>>
>>> Tom
>>>
>>> --
>>>
>>> Director Meteorite.bi - Saiku Analytics Founder
>>> Tel: +44(0)5603641316
>>>
>>> (Thanks to the Saiku community we reached our Kickstart
>>> 
>>> goal, but you can always help by sponsoring the project
>>> )
>>>
>>> On 18 July 2016 at 14:37, Merlijn Sebrechts >> > wrote:
>>>
 Hi Jay


 I'm able to push a 270 mb large Charm now. This was not possible
 before, so it seems the issue is fixed for me. I've added Tom in cc so he
 can check too. He was having the same issue if I'm not mistaken. I'll come
 back to you if the issue magically reappears.. :)



 Kind regards
 Merlijn


 2016-07-08 21:04 GMT+02:00 Jay Wren :

> We have made a configuration change which should impact this. If you
> are able to try again, please do so.
>
> Also, we are continuing to track this issue and add monitoring and
> instrumentation around this area of the charmstore to be better able to
> respond to such issues in the future.
>
> Thanks,
> --
> Jay
>
>
> On Wed, Jun 22, 2016 at 10:30 AM, Stuart Bishop <
> stuart.bis...@canonical.com> wrote:
>
>> On 21 June 2016 at 02:40, Jay Wren  wrote:
>> > Thanks for the further testing.
>> >
>> > Now I'm questioning how in the world I was able to see the same
>> error.
>> >
>> > I will continue my testing to attempt to reproduce the error.
>> >
>> > Also, the Apache Timeout 300 should behave exactly as Mark said.
>> I'm still
>> > trying to find what is causing this failure.
>>
>> The error from the bug report originates from Squid, which is probably
>> buffering the upload. The timeout might happen between Squid and
>> Apache if it takes to long to upload and there is no keep-alive
>> mechanism in place.
>>
>> --
>> Stuart Bishop 
>>
>
>

>>>
>> --
>> Juju mailing list
>> Juju@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm push crashes when uploading big charms

[Rick Harding]

How big/long to upload was your resource Tom? I know the team bumped it
while they get some better insight into tracking/quota'ing sizes and I'm
curious where you got denied.

On Mon, Jul 18, 2016, 5:53 PM Tom Barber  wrote:

> Awww Merlijn
>
> You got my hopes up then I got the same error as usual *sadface*
>
> Tom
>
> --
>
> Director Meteorite.bi - Saiku Analytics Founder
> Tel: +44(0)5603641316
>
> (Thanks to the Saiku community we reached our Kickstart
> 
> goal, but you can always help by sponsoring the project
> )
>
> On 18 July 2016 at 14:39, Tom Barber  wrote:
>
>> Cool thanks, I'll check it shortly.
>>
>> Tom
>>
>> --
>>
>> Director Meteorite.bi - Saiku Analytics Founder
>> Tel: +44(0)5603641316
>>
>> (Thanks to the Saiku community we reached our Kickstart
>> 
>> goal, but you can always help by sponsoring the project
>> )
>>
>> On 18 July 2016 at 14:37, Merlijn Sebrechts 
>> wrote:
>>
>>> Hi Jay
>>>
>>>
>>> I'm able to push a 270 mb large Charm now. This was not possible before,
>>> so it seems the issue is fixed for me. I've added Tom in cc so he can check
>>> too. He was having the same issue if I'm not mistaken. I'll come back to
>>> you if the issue magically reappears.. :)
>>>
>>>
>>>
>>> Kind regards
>>> Merlijn
>>>
>>>
>>> 2016-07-08 21:04 GMT+02:00 Jay Wren :
>>>
 We have made a configuration change which should impact this. If you
 are able to try again, please do so.

 Also, we are continuing to track this issue and add monitoring and
 instrumentation around this area of the charmstore to be better able to
 respond to such issues in the future.

 Thanks,
 --
 Jay


 On Wed, Jun 22, 2016 at 10:30 AM, Stuart Bishop <
 stuart.bis...@canonical.com> wrote:

> On 21 June 2016 at 02:40, Jay Wren  wrote:
> > Thanks for the further testing.
> >
> > Now I'm questioning how in the world I was able to see the same
> error.
> >
> > I will continue my testing to attempt to reproduce the error.
> >
> > Also, the Apache Timeout 300 should behave exactly as Mark said. I'm
> still
> > trying to find what is causing this failure.
>
> The error from the bug report originates from Squid, which is probably
> buffering the upload. The timeout might happen between Squid and
> Apache if it takes to long to upload and there is no keep-alive
> mechanism in place.
>
> --
> Stuart Bishop 
>


>>>
>>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm push crashes when uploading big charms

[Tom Barber]

Awww Merlijn

You got my hopes up then I got the same error as usual *sadface*

Tom

--

Director Meteorite.bi - Saiku Analytics Founder
Tel: +44(0)5603641316

(Thanks to the Saiku community we reached our Kickstart

goal, but you can always help by sponsoring the project
)

On 18 July 2016 at 14:39, Tom Barber  wrote:

> Cool thanks, I'll check it shortly.
>
> Tom
>
> --
>
> Director Meteorite.bi - Saiku Analytics Founder
> Tel: +44(0)5603641316
>
> (Thanks to the Saiku community we reached our Kickstart
> 
> goal, but you can always help by sponsoring the project
> )
>
> On 18 July 2016 at 14:37, Merlijn Sebrechts 
> wrote:
>
>> Hi Jay
>>
>>
>> I'm able to push a 270 mb large Charm now. This was not possible before,
>> so it seems the issue is fixed for me. I've added Tom in cc so he can check
>> too. He was having the same issue if I'm not mistaken. I'll come back to
>> you if the issue magically reappears.. :)
>>
>>
>>
>> Kind regards
>> Merlijn
>>
>>
>> 2016-07-08 21:04 GMT+02:00 Jay Wren :
>>
>>> We have made a configuration change which should impact this. If you are
>>> able to try again, please do so.
>>>
>>> Also, we are continuing to track this issue and add monitoring and
>>> instrumentation around this area of the charmstore to be better able to
>>> respond to such issues in the future.
>>>
>>> Thanks,
>>> --
>>> Jay
>>>
>>>
>>> On Wed, Jun 22, 2016 at 10:30 AM, Stuart Bishop <
>>> stuart.bis...@canonical.com> wrote:
>>>
 On 21 June 2016 at 02:40, Jay Wren  wrote:
 > Thanks for the further testing.
 >
 > Now I'm questioning how in the world I was able to see the same error.
 >
 > I will continue my testing to attempt to reproduce the error.
 >
 > Also, the Apache Timeout 300 should behave exactly as Mark said. I'm
 still
 > trying to find what is causing this failure.

 The error from the bug report originates from Squid, which is probably
 buffering the upload. The timeout might happen between Squid and
 Apache if it takes to long to upload and there is no keep-alive
 mechanism in place.

 --
 Stuart Bishop 

>>>
>>>
>>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm push crashes when uploading big charms

[Tom Barber]

Cool thanks, I'll check it shortly.

Tom

--

Director Meteorite.bi - Saiku Analytics Founder
Tel: +44(0)5603641316

(Thanks to the Saiku community we reached our Kickstart

goal, but you can always help by sponsoring the project
)

On 18 July 2016 at 14:37, Merlijn Sebrechts 
wrote:

> Hi Jay
>
>
> I'm able to push a 270 mb large Charm now. This was not possible before,
> so it seems the issue is fixed for me. I've added Tom in cc so he can check
> too. He was having the same issue if I'm not mistaken. I'll come back to
> you if the issue magically reappears.. :)
>
>
>
> Kind regards
> Merlijn
>
>
> 2016-07-08 21:04 GMT+02:00 Jay Wren :
>
>> We have made a configuration change which should impact this. If you are
>> able to try again, please do so.
>>
>> Also, we are continuing to track this issue and add monitoring and
>> instrumentation around this area of the charmstore to be better able to
>> respond to such issues in the future.
>>
>> Thanks,
>> --
>> Jay
>>
>>
>> On Wed, Jun 22, 2016 at 10:30 AM, Stuart Bishop <
>> stuart.bis...@canonical.com> wrote:
>>
>>> On 21 June 2016 at 02:40, Jay Wren  wrote:
>>> > Thanks for the further testing.
>>> >
>>> > Now I'm questioning how in the world I was able to see the same error.
>>> >
>>> > I will continue my testing to attempt to reproduce the error.
>>> >
>>> > Also, the Apache Timeout 300 should behave exactly as Mark said. I'm
>>> still
>>> > trying to find what is causing this failure.
>>>
>>> The error from the bug report originates from Squid, which is probably
>>> buffering the upload. The timeout might happen between Squid and
>>> Apache if it takes to long to upload and there is no keep-alive
>>> mechanism in place.
>>>
>>> --
>>> Stuart Bishop 
>>>
>>
>>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm push crashes when uploading big charms

[Merlijn Sebrechts]

Hi Jay


I'm able to push a 270 mb large Charm now. This was not possible before, so
it seems the issue is fixed for me. I've added Tom in cc so he can check
too. He was having the same issue if I'm not mistaken. I'll come back to
you if the issue magically reappears.. :)



Kind regards
Merlijn


2016-07-08 21:04 GMT+02:00 Jay Wren :

> We have made a configuration change which should impact this. If you are
> able to try again, please do so.
>
> Also, we are continuing to track this issue and add monitoring and
> instrumentation around this area of the charmstore to be better able to
> respond to such issues in the future.
>
> Thanks,
> --
> Jay
>
>
> On Wed, Jun 22, 2016 at 10:30 AM, Stuart Bishop <
> stuart.bis...@canonical.com> wrote:
>
>> On 21 June 2016 at 02:40, Jay Wren  wrote:
>> > Thanks for the further testing.
>> >
>> > Now I'm questioning how in the world I was able to see the same error.
>> >
>> > I will continue my testing to attempt to reproduce the error.
>> >
>> > Also, the Apache Timeout 300 should behave exactly as Mark said. I'm
>> still
>> > trying to find what is causing this failure.
>>
>> The error from the bug report originates from Squid, which is probably
>> buffering the upload. The timeout might happen between Squid and
>> Apache if it takes to long to upload and there is no keep-alive
>> mechanism in place.
>>
>> --
>> Stuart Bishop 
>>
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm push crashes when uploading big charms

[Jay Wren]

We have made a configuration change which should impact this. If you are
able to try again, please do so.

Also, we are continuing to track this issue and add monitoring and
instrumentation around this area of the charmstore to be better able to
respond to such issues in the future.

Thanks,
--
Jay


On Wed, Jun 22, 2016 at 10:30 AM, Stuart Bishop  wrote:

> On 21 June 2016 at 02:40, Jay Wren  wrote:
> > Thanks for the further testing.
> >
> > Now I'm questioning how in the world I was able to see the same error.
> >
> > I will continue my testing to attempt to reproduce the error.
> >
> > Also, the Apache Timeout 300 should behave exactly as Mark said. I'm
> still
> > trying to find what is causing this failure.
>
> The error from the bug report originates from Squid, which is probably
> buffering the upload. The timeout might happen between Squid and
> Apache if it takes to long to upload and there is no keep-alive
> mechanism in place.
>
> --
> Stuart Bishop 
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm push crashes when uploading big charms

[Stuart Bishop]

On 21 June 2016 at 02:40, Jay Wren  wrote:
> Thanks for the further testing.
>
> Now I'm questioning how in the world I was able to see the same error.
>
> I will continue my testing to attempt to reproduce the error.
>
> Also, the Apache Timeout 300 should behave exactly as Mark said. I'm still
> trying to find what is causing this failure.

The error from the bug report originates from Squid, which is probably
buffering the upload. The timeout might happen between Squid and
Apache if it takes to long to upload and there is no keep-alive
mechanism in place.

-- 
Stuart Bishop 

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm push crashes when uploading big charms

[Jay Wren]

Thanks for the further testing.

Now I'm questioning how in the world I was able to see the same error.

I will continue my testing to attempt to reproduce the error.

Also, the Apache Timeout 300 should behave exactly as Mark said. I'm still
trying to find what is causing this failure.


On Mon, Jun 20, 2016 at 9:49 PM, Merlijn Sebrechts <
merlijn.sebrec...@gmail.com> wrote:

> So if that's 300 days, that should give you plenty of time ;)
>
> 2016-06-20 20:38 GMT+02:00 Merlijn Sebrechts 
> :
>
>> From the bug report: "The timeout is apache setting `Timeout` which
>> defaults to 300."
>>
>> https://bugs.launchpad.net/ubuntu/+source/charm/+bug/1592822
>>
>> 2016-06-20 20:32 GMT+02:00 Tom Barber :
>>
>>> If I had to upload 270mb from my home I'd be waiting 3 weeks. what's
>>> the timeout set to? ;)
>>> On 20 Jun 2016 19:30, "Merlijn Sebrechts" 
>>> wrote:
>>>
 Thanks for the explanation, Jay.

 I did some further testing. Charm upload fails for a 270MB Charm both
 from my home, my work and our datacenter.

- The datacenter is connected directly to Belnet (upload bandwith
~300Mbit/s).
- My upload bandwidth at home is ~ 3 Mbps (speedtest.net) although
during upload, system monitor shows ~400KiB/s.

 This causes me to think there is more at play here then large file +
 slow internet... Let me know if I can help to further debug this problem.


 As an aside; I don't consider 270MB to be that large. Some examples:


- Kubernetes is ~1G
- Ubuntu docker base image is ~200MB


 I think this is stuff we should be able to handle...

 2016-06-20 18:41 GMT+02:00 Jay Wren :

> Yes, files are broken up into many TCP packets, and they are all
> transmitted over a single TCP connection. TCP is a complex protocol which
> is well documented, so I'll not repeat that here. If you want lots of
> details, wikipedia is not bad:
> https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Protocol_operation
>
> In the abstract, when you connect to a server using TCP it is
> identified by a 4-tuple of source address, source port, target ip address,
> target port. These connections consume server resources and indeed a
> connection exhaustion is a popular denial of service attack.
>
> You are getting a tcp timeout due to of file size because the time it
> takes to send the entire content is longer than the TCP connection 
> timeout.
>
> Yes, the resource upload command to charmstore will also be affected
> by this. Luckily, resources also have the ability to be uploaded
> specifically to a model, which might have greater network data rates from
> the resource uploader.
>
> --
> Jay
>
>
> On Mon, Jun 20, 2016 at 7:04 PM, Merlijn Sebrechts <
> merlijn.sebrec...@gmail.com> wrote:
>
>> Thanks for looking into this! I'll try the compression and see if
>> that works.
>>
>> Just curious; why does filesize affect tcp connection timeout? Aren't
>> the files broken up into a bunch of smaller tcp packets? Filesize should
>> only affect the number of tcp packets, not the size of the tcp packets? 
>> So
>> getting a tcp timeout because of filesize seems strange to me... Any idea
>> what exactly goes wrong here?
>>
>> Also, now that I think of it, the resource upload command might also
>> be affected by this, if it uses the same library and similar backend
>> infrastructure? I'll test this out.
>>
>>
>> Op maandag 20 juni 2016 heeft Jay Wren  het
>> volgende geschreven:
>> > Hello Merlijn,
>> > I can replicate the problem and I can work around it by using a
>> faster internet connection.
>> > At some point, tcp connections have to time out. I can only
>> replicate the issue when that timeout is reached. If you have the means 
>> to
>> relocate to a faster internet connection temporarily for pushing to
>> charmstore, please do. You might also try recompressing any items in the
>> charm using a higher compression level. xz -9 instead of gzip -3 or
>> whatever things may be using now.
>> > We are aware this is a poor longterm solution. We are investigating
>> better solutions for uploads. As you've mentioned, resources will also 
>> help
>> the situation.
>> > I am sorry that I do not have a better solution.
>> > --
>> > Jay
>> > On Fri, Jun 17, 2016 at 4:29 PM, Rick Harding <
>> rick.hard...@canonical.com> wrote:
>> >>
>> >> Merlijn, thanks. I'm going to bet there's an issue with http
>> request sizes for the charmstore that the charm command talks do as we've
>> got some layers (Apache, Squid) in front of the actual application. The
>> team is looking into it. Thanks for giving us the heads up.
>> >> On Wed, Jun 15, 2016 at 10:28 AM Merlijn Sebrechts 

Re: Charm push crashes when uploading big charms

[Merlijn Sebrechts]

So if that's 300 days, that should give you plenty of time ;)

2016-06-20 20:38 GMT+02:00 Merlijn Sebrechts :

> From the bug report: "The timeout is apache setting `Timeout` which
> defaults to 300."
>
> https://bugs.launchpad.net/ubuntu/+source/charm/+bug/1592822
>
> 2016-06-20 20:32 GMT+02:00 Tom Barber :
>
>> If I had to upload 270mb from my home I'd be waiting 3 weeks. what's
>> the timeout set to? ;)
>> On 20 Jun 2016 19:30, "Merlijn Sebrechts" 
>> wrote:
>>
>>> Thanks for the explanation, Jay.
>>>
>>> I did some further testing. Charm upload fails for a 270MB Charm both
>>> from my home, my work and our datacenter.
>>>
>>>- The datacenter is connected directly to Belnet (upload bandwith
>>>~300Mbit/s).
>>>- My upload bandwidth at home is ~ 3 Mbps (speedtest.net) although
>>>during upload, system monitor shows ~400KiB/s.
>>>
>>> This causes me to think there is more at play here then large file +
>>> slow internet... Let me know if I can help to further debug this problem.
>>>
>>>
>>> As an aside; I don't consider 270MB to be that large. Some examples:
>>>
>>>
>>>- Kubernetes is ~1G
>>>- Ubuntu docker base image is ~200MB
>>>
>>>
>>> I think this is stuff we should be able to handle...
>>>
>>> 2016-06-20 18:41 GMT+02:00 Jay Wren :
>>>
 Yes, files are broken up into many TCP packets, and they are all
 transmitted over a single TCP connection. TCP is a complex protocol which
 is well documented, so I'll not repeat that here. If you want lots of
 details, wikipedia is not bad:
 https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Protocol_operation

 In the abstract, when you connect to a server using TCP it is
 identified by a 4-tuple of source address, source port, target ip address,
 target port. These connections consume server resources and indeed a
 connection exhaustion is a popular denial of service attack.

 You are getting a tcp timeout due to of file size because the time it
 takes to send the entire content is longer than the TCP connection timeout.

 Yes, the resource upload command to charmstore will also be affected by
 this. Luckily, resources also have the ability to be uploaded specifically
 to a model, which might have greater network data rates from the resource
 uploader.

 --
 Jay


 On Mon, Jun 20, 2016 at 7:04 PM, Merlijn Sebrechts <
 merlijn.sebrec...@gmail.com> wrote:

> Thanks for looking into this! I'll try the compression and see if that
> works.
>
> Just curious; why does filesize affect tcp connection timeout? Aren't
> the files broken up into a bunch of smaller tcp packets? Filesize should
> only affect the number of tcp packets, not the size of the tcp packets? So
> getting a tcp timeout because of filesize seems strange to me... Any idea
> what exactly goes wrong here?
>
> Also, now that I think of it, the resource upload command might also
> be affected by this, if it uses the same library and similar backend
> infrastructure? I'll test this out.
>
>
> Op maandag 20 juni 2016 heeft Jay Wren  het
> volgende geschreven:
> > Hello Merlijn,
> > I can replicate the problem and I can work around it by using a
> faster internet connection.
> > At some point, tcp connections have to time out. I can only
> replicate the issue when that timeout is reached. If you have the means to
> relocate to a faster internet connection temporarily for pushing to
> charmstore, please do. You might also try recompressing any items in the
> charm using a higher compression level. xz -9 instead of gzip -3 or
> whatever things may be using now.
> > We are aware this is a poor longterm solution. We are investigating
> better solutions for uploads. As you've mentioned, resources will also 
> help
> the situation.
> > I am sorry that I do not have a better solution.
> > --
> > Jay
> > On Fri, Jun 17, 2016 at 4:29 PM, Rick Harding <
> rick.hard...@canonical.com> wrote:
> >>
> >> Merlijn, thanks. I'm going to bet there's an issue with http
> request sizes for the charmstore that the charm command talks do as we've
> got some layers (Apache, Squid) in front of the actual application. The
> team is looking into it. Thanks for giving us the heads up.
> >> On Wed, Jun 15, 2016 at 10:28 AM Merlijn Sebrechts <
> merlijn.sebrec...@gmail.com> wrote:
> >>>
> >>> Hi all
> >>>
> >>> I've hit a roadblock in setting up my CI pipeline. I have a charm
> with a Java sdk blob of ~200MB. Pushing this charm to the store causes the
> tool to crash. I put a bug report here:
> https://bugs.launchpad.net/juju/+bug/1592822
> >>> Juju resources would fix my problem, but then I'd need to move to
> Juju 2.0 and I'm not ready to do that yet.
> >>>
> >>>
> >>> Kind regards
> >>> Me

Re: Charm push crashes when uploading big charms

[Merlijn Sebrechts]

>From the bug report: "The timeout is apache setting `Timeout` which
defaults to 300."

https://bugs.launchpad.net/ubuntu/+source/charm/+bug/1592822

2016-06-20 20:32 GMT+02:00 Tom Barber :

> If I had to upload 270mb from my home I'd be waiting 3 weeks. what's
> the timeout set to? ;)
> On 20 Jun 2016 19:30, "Merlijn Sebrechts" 
> wrote:
>
>> Thanks for the explanation, Jay.
>>
>> I did some further testing. Charm upload fails for a 270MB Charm both
>> from my home, my work and our datacenter.
>>
>>- The datacenter is connected directly to Belnet (upload bandwith
>>~300Mbit/s).
>>- My upload bandwidth at home is ~ 3 Mbps (speedtest.net) although
>>during upload, system monitor shows ~400KiB/s.
>>
>> This causes me to think there is more at play here then large file + slow
>> internet... Let me know if I can help to further debug this problem.
>>
>>
>> As an aside; I don't consider 270MB to be that large. Some examples:
>>
>>
>>- Kubernetes is ~1G
>>- Ubuntu docker base image is ~200MB
>>
>>
>> I think this is stuff we should be able to handle...
>>
>> 2016-06-20 18:41 GMT+02:00 Jay Wren :
>>
>>> Yes, files are broken up into many TCP packets, and they are all
>>> transmitted over a single TCP connection. TCP is a complex protocol which
>>> is well documented, so I'll not repeat that here. If you want lots of
>>> details, wikipedia is not bad:
>>> https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Protocol_operation
>>>
>>> In the abstract, when you connect to a server using TCP it is identified
>>> by a 4-tuple of source address, source port, target ip address, target
>>> port. These connections consume server resources and indeed a connection
>>> exhaustion is a popular denial of service attack.
>>>
>>> You are getting a tcp timeout due to of file size because the time it
>>> takes to send the entire content is longer than the TCP connection timeout.
>>>
>>> Yes, the resource upload command to charmstore will also be affected by
>>> this. Luckily, resources also have the ability to be uploaded specifically
>>> to a model, which might have greater network data rates from the resource
>>> uploader.
>>>
>>> --
>>> Jay
>>>
>>>
>>> On Mon, Jun 20, 2016 at 7:04 PM, Merlijn Sebrechts <
>>> merlijn.sebrec...@gmail.com> wrote:
>>>
 Thanks for looking into this! I'll try the compression and see if that
 works.

 Just curious; why does filesize affect tcp connection timeout? Aren't
 the files broken up into a bunch of smaller tcp packets? Filesize should
 only affect the number of tcp packets, not the size of the tcp packets? So
 getting a tcp timeout because of filesize seems strange to me... Any idea
 what exactly goes wrong here?

 Also, now that I think of it, the resource upload command might also be
 affected by this, if it uses the same library and similar backend
 infrastructure? I'll test this out.


 Op maandag 20 juni 2016 heeft Jay Wren  het
 volgende geschreven:
 > Hello Merlijn,
 > I can replicate the problem and I can work around it by using a
 faster internet connection.
 > At some point, tcp connections have to time out. I can only replicate
 the issue when that timeout is reached. If you have the means to relocate
 to a faster internet connection temporarily for pushing to charmstore,
 please do. You might also try recompressing any items in the charm using a
 higher compression level. xz -9 instead of gzip -3 or whatever things may
 be using now.
 > We are aware this is a poor longterm solution. We are investigating
 better solutions for uploads. As you've mentioned, resources will also help
 the situation.
 > I am sorry that I do not have a better solution.
 > --
 > Jay
 > On Fri, Jun 17, 2016 at 4:29 PM, Rick Harding <
 rick.hard...@canonical.com> wrote:
 >>
 >> Merlijn, thanks. I'm going to bet there's an issue with http request
 sizes for the charmstore that the charm command talks do as we've got some
 layers (Apache, Squid) in front of the actual application. The team is
 looking into it. Thanks for giving us the heads up.
 >> On Wed, Jun 15, 2016 at 10:28 AM Merlijn Sebrechts <
 merlijn.sebrec...@gmail.com> wrote:
 >>>
 >>> Hi all
 >>>
 >>> I've hit a roadblock in setting up my CI pipeline. I have a charm
 with a Java sdk blob of ~200MB. Pushing this charm to the store causes the
 tool to crash. I put a bug report here:
 https://bugs.launchpad.net/juju/+bug/1592822
 >>> Juju resources would fix my problem, but then I'd need to move to
 Juju 2.0 and I'm not ready to do that yet.
 >>>
 >>>
 >>> Kind regards
 >>> Merlijn Sebrechts
 >>> --
 >>> Juju mailing list
 >>> Juju@lists.ubuntu.com
 >>> Modify settings or unsubscribe at:
 https://lists.ubuntu.com/mailman/listinfo/juju
 >>
 >> --
 >> Juju mailing list
 >> Ju

Re: Charm push crashes when uploading big charms

[Tom Barber]

If I had to upload 270mb from my home I'd be waiting 3 weeks. what's
the timeout set to? ;)
On 20 Jun 2016 19:30, "Merlijn Sebrechts" 
wrote:

> Thanks for the explanation, Jay.
>
> I did some further testing. Charm upload fails for a 270MB Charm both from
> my home, my work and our datacenter.
>
>- The datacenter is connected directly to Belnet (upload bandwith
>~300Mbit/s).
>- My upload bandwidth at home is ~ 3 Mbps (speedtest.net) although
>during upload, system monitor shows ~400KiB/s.
>
> This causes me to think there is more at play here then large file + slow
> internet... Let me know if I can help to further debug this problem.
>
>
> As an aside; I don't consider 270MB to be that large. Some examples:
>
>
>- Kubernetes is ~1G
>- Ubuntu docker base image is ~200MB
>
>
> I think this is stuff we should be able to handle...
>
> 2016-06-20 18:41 GMT+02:00 Jay Wren :
>
>> Yes, files are broken up into many TCP packets, and they are all
>> transmitted over a single TCP connection. TCP is a complex protocol which
>> is well documented, so I'll not repeat that here. If you want lots of
>> details, wikipedia is not bad:
>> https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Protocol_operation
>>
>> In the abstract, when you connect to a server using TCP it is identified
>> by a 4-tuple of source address, source port, target ip address, target
>> port. These connections consume server resources and indeed a connection
>> exhaustion is a popular denial of service attack.
>>
>> You are getting a tcp timeout due to of file size because the time it
>> takes to send the entire content is longer than the TCP connection timeout.
>>
>> Yes, the resource upload command to charmstore will also be affected by
>> this. Luckily, resources also have the ability to be uploaded specifically
>> to a model, which might have greater network data rates from the resource
>> uploader.
>>
>> --
>> Jay
>>
>>
>> On Mon, Jun 20, 2016 at 7:04 PM, Merlijn Sebrechts <
>> merlijn.sebrec...@gmail.com> wrote:
>>
>>> Thanks for looking into this! I'll try the compression and see if that
>>> works.
>>>
>>> Just curious; why does filesize affect tcp connection timeout? Aren't
>>> the files broken up into a bunch of smaller tcp packets? Filesize should
>>> only affect the number of tcp packets, not the size of the tcp packets? So
>>> getting a tcp timeout because of filesize seems strange to me... Any idea
>>> what exactly goes wrong here?
>>>
>>> Also, now that I think of it, the resource upload command might also be
>>> affected by this, if it uses the same library and similar backend
>>> infrastructure? I'll test this out.
>>>
>>>
>>> Op maandag 20 juni 2016 heeft Jay Wren  het
>>> volgende geschreven:
>>> > Hello Merlijn,
>>> > I can replicate the problem and I can work around it by using a faster
>>> internet connection.
>>> > At some point, tcp connections have to time out. I can only replicate
>>> the issue when that timeout is reached. If you have the means to relocate
>>> to a faster internet connection temporarily for pushing to charmstore,
>>> please do. You might also try recompressing any items in the charm using a
>>> higher compression level. xz -9 instead of gzip -3 or whatever things may
>>> be using now.
>>> > We are aware this is a poor longterm solution. We are investigating
>>> better solutions for uploads. As you've mentioned, resources will also help
>>> the situation.
>>> > I am sorry that I do not have a better solution.
>>> > --
>>> > Jay
>>> > On Fri, Jun 17, 2016 at 4:29 PM, Rick Harding <
>>> rick.hard...@canonical.com> wrote:
>>> >>
>>> >> Merlijn, thanks. I'm going to bet there's an issue with http request
>>> sizes for the charmstore that the charm command talks do as we've got some
>>> layers (Apache, Squid) in front of the actual application. The team is
>>> looking into it. Thanks for giving us the heads up.
>>> >> On Wed, Jun 15, 2016 at 10:28 AM Merlijn Sebrechts <
>>> merlijn.sebrec...@gmail.com> wrote:
>>> >>>
>>> >>> Hi all
>>> >>>
>>> >>> I've hit a roadblock in setting up my CI pipeline. I have a charm
>>> with a Java sdk blob of ~200MB. Pushing this charm to the store causes the
>>> tool to crash. I put a bug report here:
>>> https://bugs.launchpad.net/juju/+bug/1592822
>>> >>> Juju resources would fix my problem, but then I'd need to move to
>>> Juju 2.0 and I'm not ready to do that yet.
>>> >>>
>>> >>>
>>> >>> Kind regards
>>> >>> Merlijn Sebrechts
>>> >>> --
>>> >>> Juju mailing list
>>> >>> Juju@lists.ubuntu.com
>>> >>> Modify settings or unsubscribe at:
>>> https://lists.ubuntu.com/mailman/listinfo/juju
>>> >>
>>> >> --
>>> >> Juju mailing list
>>> >> Juju@lists.ubuntu.com
>>> >> Modify settings or unsubscribe at:
>>> https://lists.ubuntu.com/mailman/listinfo/juju
>>> >>
>>> >
>>> >
>>>
>>
>>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lis

Re: Charm push crashes when uploading big charms

[Merlijn Sebrechts]

Thanks for the explanation, Jay.

I did some further testing. Charm upload fails for a 270MB Charm both from
my home, my work and our datacenter.

   - The datacenter is connected directly to Belnet (upload bandwith
   ~300Mbit/s).
   - My upload bandwidth at home is ~ 3 Mbps (speedtest.net) although
   during upload, system monitor shows ~400KiB/s.

This causes me to think there is more at play here then large file + slow
internet... Let me know if I can help to further debug this problem.


As an aside; I don't consider 270MB to be that large. Some examples:


   - Kubernetes is ~1G
   - Ubuntu docker base image is ~200MB


I think this is stuff we should be able to handle...

2016-06-20 18:41 GMT+02:00 Jay Wren :

> Yes, files are broken up into many TCP packets, and they are all
> transmitted over a single TCP connection. TCP is a complex protocol which
> is well documented, so I'll not repeat that here. If you want lots of
> details, wikipedia is not bad:
> https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Protocol_operation
>
> In the abstract, when you connect to a server using TCP it is identified
> by a 4-tuple of source address, source port, target ip address, target
> port. These connections consume server resources and indeed a connection
> exhaustion is a popular denial of service attack.
>
> You are getting a tcp timeout due to of file size because the time it
> takes to send the entire content is longer than the TCP connection timeout.
>
> Yes, the resource upload command to charmstore will also be affected by
> this. Luckily, resources also have the ability to be uploaded specifically
> to a model, which might have greater network data rates from the resource
> uploader.
>
> --
> Jay
>
>
> On Mon, Jun 20, 2016 at 7:04 PM, Merlijn Sebrechts <
> merlijn.sebrec...@gmail.com> wrote:
>
>> Thanks for looking into this! I'll try the compression and see if that
>> works.
>>
>> Just curious; why does filesize affect tcp connection timeout? Aren't the
>> files broken up into a bunch of smaller tcp packets? Filesize should only
>> affect the number of tcp packets, not the size of the tcp packets? So
>> getting a tcp timeout because of filesize seems strange to me... Any idea
>> what exactly goes wrong here?
>>
>> Also, now that I think of it, the resource upload command might also be
>> affected by this, if it uses the same library and similar backend
>> infrastructure? I'll test this out.
>>
>>
>> Op maandag 20 juni 2016 heeft Jay Wren  het
>> volgende geschreven:
>> > Hello Merlijn,
>> > I can replicate the problem and I can work around it by using a faster
>> internet connection.
>> > At some point, tcp connections have to time out. I can only replicate
>> the issue when that timeout is reached. If you have the means to relocate
>> to a faster internet connection temporarily for pushing to charmstore,
>> please do. You might also try recompressing any items in the charm using a
>> higher compression level. xz -9 instead of gzip -3 or whatever things may
>> be using now.
>> > We are aware this is a poor longterm solution. We are investigating
>> better solutions for uploads. As you've mentioned, resources will also help
>> the situation.
>> > I am sorry that I do not have a better solution.
>> > --
>> > Jay
>> > On Fri, Jun 17, 2016 at 4:29 PM, Rick Harding <
>> rick.hard...@canonical.com> wrote:
>> >>
>> >> Merlijn, thanks. I'm going to bet there's an issue with http request
>> sizes for the charmstore that the charm command talks do as we've got some
>> layers (Apache, Squid) in front of the actual application. The team is
>> looking into it. Thanks for giving us the heads up.
>> >> On Wed, Jun 15, 2016 at 10:28 AM Merlijn Sebrechts <
>> merlijn.sebrec...@gmail.com> wrote:
>> >>>
>> >>> Hi all
>> >>>
>> >>> I've hit a roadblock in setting up my CI pipeline. I have a charm
>> with a Java sdk blob of ~200MB. Pushing this charm to the store causes the
>> tool to crash. I put a bug report here:
>> https://bugs.launchpad.net/juju/+bug/1592822
>> >>> Juju resources would fix my problem, but then I'd need to move to
>> Juju 2.0 and I'm not ready to do that yet.
>> >>>
>> >>>
>> >>> Kind regards
>> >>> Merlijn Sebrechts
>> >>> --
>> >>> Juju mailing list
>> >>> Juju@lists.ubuntu.com
>> >>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>> >>
>> >> --
>> >> Juju mailing list
>> >> Juju@lists.ubuntu.com
>> >> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>> >>
>> >
>> >
>>
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm push crashes when uploading big charms

[Mark Shuttleworth]

On 20/06/16 17:41, Jay Wren wrote:
> You are getting a tcp timeout due to of file size because the time it
> takes to send the entire content is longer than the TCP connection
> timeout.

Unlikely. TCP timeout would be the time between packets, not the
total time of a session ;)

-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm push crashes when uploading big charms

[Jay Wren]

Yes, files are broken up into many TCP packets, and they are all
transmitted over a single TCP connection. TCP is a complex protocol which
is well documented, so I'll not repeat that here. If you want lots of
details, wikipedia is not bad:
https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Protocol_operation

In the abstract, when you connect to a server using TCP it is identified by
a 4-tuple of source address, source port, target ip address, target port.
These connections consume server resources and indeed a connection
exhaustion is a popular denial of service attack.

You are getting a tcp timeout due to of file size because the time it takes
to send the entire content is longer than the TCP connection timeout.

Yes, the resource upload command to charmstore will also be affected by
this. Luckily, resources also have the ability to be uploaded specifically
to a model, which might have greater network data rates from the resource
uploader.

--
Jay


On Mon, Jun 20, 2016 at 7:04 PM, Merlijn Sebrechts <
merlijn.sebrec...@gmail.com> wrote:

> Thanks for looking into this! I'll try the compression and see if that
> works.
>
> Just curious; why does filesize affect tcp connection timeout? Aren't the
> files broken up into a bunch of smaller tcp packets? Filesize should only
> affect the number of tcp packets, not the size of the tcp packets? So
> getting a tcp timeout because of filesize seems strange to me... Any idea
> what exactly goes wrong here?
>
> Also, now that I think of it, the resource upload command might also be
> affected by this, if it uses the same library and similar backend
> infrastructure? I'll test this out.
>
>
> Op maandag 20 juni 2016 heeft Jay Wren  het
> volgende geschreven:
> > Hello Merlijn,
> > I can replicate the problem and I can work around it by using a faster
> internet connection.
> > At some point, tcp connections have to time out. I can only replicate
> the issue when that timeout is reached. If you have the means to relocate
> to a faster internet connection temporarily for pushing to charmstore,
> please do. You might also try recompressing any items in the charm using a
> higher compression level. xz -9 instead of gzip -3 or whatever things may
> be using now.
> > We are aware this is a poor longterm solution. We are investigating
> better solutions for uploads. As you've mentioned, resources will also help
> the situation.
> > I am sorry that I do not have a better solution.
> > --
> > Jay
> > On Fri, Jun 17, 2016 at 4:29 PM, Rick Harding <
> rick.hard...@canonical.com> wrote:
> >>
> >> Merlijn, thanks. I'm going to bet there's an issue with http request
> sizes for the charmstore that the charm command talks do as we've got some
> layers (Apache, Squid) in front of the actual application. The team is
> looking into it. Thanks for giving us the heads up.
> >> On Wed, Jun 15, 2016 at 10:28 AM Merlijn Sebrechts <
> merlijn.sebrec...@gmail.com> wrote:
> >>>
> >>> Hi all
> >>>
> >>> I've hit a roadblock in setting up my CI pipeline. I have a charm with
> a Java sdk blob of ~200MB. Pushing this charm to the store causes the tool
> to crash. I put a bug report here:
> https://bugs.launchpad.net/juju/+bug/1592822
> >>> Juju resources would fix my problem, but then I'd need to move to Juju
> 2.0 and I'm not ready to do that yet.
> >>>
> >>>
> >>> Kind regards
> >>> Merlijn Sebrechts
> >>> --
> >>> Juju mailing list
> >>> Juju@lists.ubuntu.com
> >>> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
> >>
> >> --
> >> Juju mailing list
> >> Juju@lists.ubuntu.com
> >> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
> >>
> >
> >
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm push crashes when uploading big charms

[Merlijn Sebrechts]

Thanks for looking into this! I'll try the compression and see if that
works.

Just curious; why does filesize affect tcp connection timeout? Aren't the
files broken up into a bunch of smaller tcp packets? Filesize should only
affect the number of tcp packets, not the size of the tcp packets? So
getting a tcp timeout because of filesize seems strange to me... Any idea
what exactly goes wrong here?

Also, now that I think of it, the resource upload command might also be
affected by this, if it uses the same library and similar backend
infrastructure? I'll test this out.

Op maandag 20 juni 2016 heeft Jay Wren  het
volgende geschreven:
> Hello Merlijn,
> I can replicate the problem and I can work around it by using a faster
internet connection.
> At some point, tcp connections have to time out. I can only replicate the
issue when that timeout is reached. If you have the means to relocate to a
faster internet connection temporarily for pushing to charmstore, please
do. You might also try recompressing any items in the charm using a higher
compression level. xz -9 instead of gzip -3 or whatever things may be using
now.
> We are aware this is a poor longterm solution. We are investigating
better solutions for uploads. As you've mentioned, resources will also help
the situation.
> I am sorry that I do not have a better solution.
> --
> Jay
> On Fri, Jun 17, 2016 at 4:29 PM, Rick Harding 
wrote:
>>
>> Merlijn, thanks. I'm going to bet there's an issue with http request
sizes for the charmstore that the charm command talks do as we've got some
layers (Apache, Squid) in front of the actual application. The team is
looking into it. Thanks for giving us the heads up.
>> On Wed, Jun 15, 2016 at 10:28 AM Merlijn Sebrechts <
merlijn.sebrec...@gmail.com> wrote:
>>>
>>> Hi all
>>>
>>> I've hit a roadblock in setting up my CI pipeline. I have a charm with
a Java sdk blob of ~200MB. Pushing this charm to the store causes the tool
to crash. I put a bug report here:
https://bugs.launchpad.net/juju/+bug/1592822
>>> Juju resources would fix my problem, but then I'd need to move to Juju
2.0 and I'm not ready to do that yet.
>>>
>>>
>>> Kind regards
>>> Merlijn Sebrechts
>>> --
>>> Juju mailing list
>>> Juju@lists.ubuntu.com
>>> Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju
>>
>> --
>> Juju mailing list
>> Juju@lists.ubuntu.com
>> Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju
>>
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm push crashes when uploading big charms

[Jay Wren]

Hello Merlijn,

I can replicate the problem and I can work around it by using a faster
internet connection.

At some point, tcp connections have to time out. I can only replicate the
issue when that timeout is reached. If you have the means to relocate to a
faster internet connection temporarily for pushing to charmstore, please
do. You might also try recompressing any items in the charm using a higher
compression level. xz -9 instead of gzip -3 or whatever things may be using
now.

We are aware this is a poor longterm solution. We are investigating better
solutions for uploads. As you've mentioned, resources will also help the
situation.

I am sorry that I do not have a better solution.
--
Jay

On Fri, Jun 17, 2016 at 4:29 PM, Rick Harding 
wrote:

> Merlijn, thanks. I'm going to bet there's an issue with http request sizes
> for the charmstore that the charm command talks do as we've got some layers
> (Apache, Squid) in front of the actual application. The team is looking
> into it. Thanks for giving us the heads up.
>
> On Wed, Jun 15, 2016 at 10:28 AM Merlijn Sebrechts <
> merlijn.sebrec...@gmail.com> wrote:
>
>> Hi all
>>
>>
>> I've hit a roadblock in setting up my CI pipeline. I have a charm with a
>> Java sdk blob of ~200MB. Pushing this charm to the store causes the tool to
>> crash. I put a bug report here:
>> https://bugs.launchpad.net/juju/+bug/1592822
>>
>> Juju resources would fix my problem, but then I'd need to move to Juju
>> 2.0 and I'm not ready to do that yet.
>>
>>
>>
>> Kind regards
>> Merlijn Sebrechts
>> --
>> Juju mailing list
>> Juju@lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>>
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: Charm push crashes when uploading big charms

[Rick Harding]

Merlijn, thanks. I'm going to bet there's an issue with http request sizes
for the charmstore that the charm command talks do as we've got some layers
(Apache, Squid) in front of the actual application. The team is looking
into it. Thanks for giving us the heads up.

On Wed, Jun 15, 2016 at 10:28 AM Merlijn Sebrechts <
merlijn.sebrec...@gmail.com> wrote:

> Hi all
>
>
> I've hit a roadblock in setting up my CI pipeline. I have a charm with a
> Java sdk blob of ~200MB. Pushing this charm to the store causes the tool to
> crash. I put a bug report here:
> https://bugs.launchpad.net/juju/+bug/1592822
>
> Juju resources would fix my problem, but then I'd need to move to Juju 2.0
> and I'm not ready to do that yet.
>
>
>
> Kind regards
> Merlijn Sebrechts
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Charm push crashes when uploading big charms

[Merlijn Sebrechts]

Hi all


I've hit a roadblock in setting up my CI pipeline. I have a charm with a
Java sdk blob of ~200MB. Pushing this charm to the store causes the tool to
crash. I put a bug report here: https://bugs.launchpad.net/juju/+bug/1592822

Juju resources would fix my problem, but then I'd need to move to Juju 2.0
and I'm not ready to do that yet.



Kind regards
Merlijn Sebrechts
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju