Re: [Pharo-dev] Pharo Downloads are sluggish

[Marcus Denker]

Hello,

We moved the download from Inria to OVH because Inria was unreliable. (They 
found some very arcane issue with the backbone
infrastructure some month later after constantly telling us that we were wrong…)

Now the OVH server offer is not good enough, they seem to throttle us.

We are in the process of getting better infrastructure, again… 

one problem is always that there is just no time to do all these things so fast 
as they should be done…

Marcus

> On 13 Mar 2019, at 12:30, Esteban Maringolo  wrote:
> 
> More to this issue that might affect only those living below the latitude 38° 
> S...
> 
> I was having download interruptions when using the launcher to fetch 
> templates, so I changed the ZnNetworkUtils defaultSocketStreamTimeout to 120 
> (default is 30) and in the last two templates I downloaded I had no issues, 
> it was slow, but at least without interruptions. Maybe the download manager I 
> mentioned in the previous email had a highter timeout as well.
> 
> Given that the INRIA file server hasn't the fastest upload link, I think that 
> rising the timeout from 30 to something at least the double is a reasonable 
> workaround.
> 
> Regards,
> 
> Esteban A. Maringolo
> 
> 
> El vie., 1 feb. 2019 a las 14:48, Esteban Maringolo ( >) escribió:
> I'm bumping this, just in case it helps somebody.
> 
> I tried to download the latest Pharo Launcher 1.6, and I had the same issues 
> as the ones stated in this thread (slow download, timeouts, etc.).
> 
> But I downloaded it with a download manager [1], and by means of ten 
> simultaneous connections the download speed was fast enough as to download 
> the 50 MB in a minute or so.
> 
> So I don't know if there is a restriction from the file server or somewhere 
> in the way out, but you can work around it by means of a Download Manager.
> 
> Regards!
> 
> [1] FreeDownloadManager.org
> 
> Esteban A. Maringolo
> 
> 
> El mié., 3 oct. 2018 a las 5:04, Marcus Denker ( >) escribió:
> Ok, so the upgrade of the server contract did not help.
> 
> There is a CDN included, I will try to enable that next (but CDNs cache by 
> name, so we need to make sure we invalidate the cache…)
> 
> I will do a calculation what S3 would cost, too.
> 
>   Marcus
> 
>> On 3 Oct 2018, at 00:59, Esteban Maringolo > > wrote:
>> 
>> I'm reviving this because I can't finish the download of an image using the 
>> latest Pharo launcher.
>> 
>> I'm timeout errors when attempting to download it. Tried 5 times at 
>> different intervals.
>> 
>> 
>> 
>> 
>> Any idea of what could be happening?
>> 
>> Esteban A. Maringolo
>> 
>> 
>> El mar., 18 sept. 2018 a las 21:03, Esteban Maringolo (> >) escribió:
>> Is there a reason why the latest version of the builds linked from the 
>> website are not hosted in some CDN, Amazon S3 or similar?
>> 
>> Maybe in Europe the speed is fast, but where I am (Argentina) it is really 
>> slow, the screenshot shows 3.5KB/s, it averages 10KB/s, with peaks of 40KB/s.
>> 
>> This is an pitiful boarding experience, that will make many cancel it even 
>> before getting to open the downloaded launcher.
>> 
>> 
>> 
>> The screenshot shows that is not a problem with my home connection, which 
>> isn't the fastest but is "normal" for my use cases.
>> 
>> Regards,
>> 
>> Esteban A. Maringolo
>> 
>> ps: When I finished writing this email, this was the progress...
>> 
> 

Re: [Pharo-dev] Pharo Downloads are sluggish

[Esteban Maringolo]

More to this issue that might affect only those living below the latitude
38° S...

I was having download interruptions when using the launcher to fetch
templates, so I changed the ZnNetworkUtils defaultSocketStreamTimeout to
120 (default is 30) and in the last two templates I downloaded I had no
issues, it was slow, but at least without interruptions. Maybe the download
manager I mentioned in the previous email had a highter timeout as well.

Given that the INRIA file server hasn't the fastest upload link, I think
that rising the timeout from 30 to something at least the double is a
reasonable workaround.

Regards,

Esteban A. Maringolo


El vie., 1 feb. 2019 a las 14:48, Esteban Maringolo ()
escribió:

> I'm bumping this, just in case it helps somebody.
>
> I tried to download the latest Pharo Launcher 1.6, and I had the same
> issues as the ones stated in this thread (slow download, timeouts, etc.).
>
> But I downloaded it with a download manager [1], and by means of ten
> simultaneous connections the download speed was fast enough as to download
> the 50 MB in a minute or so.
>
> So I don't know if there is a restriction from the file server or
> somewhere in the way out, but you can work around it by means of a Download
> Manager.
>
> Regards!
>
> [1] FreeDownloadManager.org
>
> Esteban A. Maringolo
>
>
> El mié., 3 oct. 2018 a las 5:04, Marcus Denker ()
> escribió:
>
>> Ok, so the upgrade of the server contract did not help.
>>
>> There is a CDN included, I will try to enable that next (but CDNs cache
>> by name, so we need to make sure we invalidate the cache…)
>>
>> I will do a calculation what S3 would cost, too.
>>
>> Marcus
>>
>> On 3 Oct 2018, at 00:59, Esteban Maringolo  wrote:
>>
>> I'm reviving this because I can't finish the download of an image using
>> the latest Pharo launcher.
>>
>> I'm timeout errors when attempting to download it. Tried 5 times at
>> different intervals.
>>
>>
>> 
>>
>> Any idea of what could be happening?
>>
>> Esteban A. Maringolo
>>
>>
>> El mar., 18 sept. 2018 a las 21:03, Esteban Maringolo (<
>> emaring...@gmail.com>) escribió:
>>
>>> Is there a reason why the latest version of the builds linked from the
>>> website are not hosted in some CDN, Amazon S3 or similar?
>>>
>>> Maybe in Europe the speed is fast, but where I am (Argentina) it is
>>> really slow, the screenshot shows 3.5KB/s, it averages 10KB/s, with peaks
>>> of 40KB/s.
>>>
>>> This is an pitiful boarding experience, that will make many cancel it
>>> even before getting to open the downloaded launcher.
>>>
>>> 
>>>
>>> The screenshot shows that is not a problem with my home connection,
>>> which isn't the fastest but is "normal" for my use cases.
>>>
>>> Regards,
>>>
>>> Esteban A. Maringolo
>>>
>>> ps: When I finished writing this email, this was the progress...
>>> 
>>>
>>
>>

Re: [Pharo-dev] Pharo Downloads are sluggish

[Esteban Maringolo]

I'm bumping this, just in case it helps somebody.

I tried to download the latest Pharo Launcher 1.6, and I had the same
issues as the ones stated in this thread (slow download, timeouts, etc.).

But I downloaded it with a download manager [1], and by means of ten
simultaneous connections the download speed was fast enough as to download
the 50 MB in a minute or so.

So I don't know if there is a restriction from the file server or somewhere
in the way out, but you can work around it by means of a Download Manager.

Regards!

[1] FreeDownloadManager.org

Esteban A. Maringolo


El mié., 3 oct. 2018 a las 5:04, Marcus Denker ()
escribió:

> Ok, so the upgrade of the server contract did not help.
>
> There is a CDN included, I will try to enable that next (but CDNs cache by
> name, so we need to make sure we invalidate the cache…)
>
> I will do a calculation what S3 would cost, too.
>
> Marcus
>
> On 3 Oct 2018, at 00:59, Esteban Maringolo  wrote:
>
> I'm reviving this because I can't finish the download of an image using
> the latest Pharo launcher.
>
> I'm timeout errors when attempting to download it. Tried 5 times at
> different intervals.
>
>
> 
>
> Any idea of what could be happening?
>
> Esteban A. Maringolo
>
>
> El mar., 18 sept. 2018 a las 21:03, Esteban Maringolo (<
> emaring...@gmail.com>) escribió:
>
>> Is there a reason why the latest version of the builds linked from the
>> website are not hosted in some CDN, Amazon S3 or similar?
>>
>> Maybe in Europe the speed is fast, but where I am (Argentina) it is
>> really slow, the screenshot shows 3.5KB/s, it averages 10KB/s, with peaks
>> of 40KB/s.
>>
>> This is an pitiful boarding experience, that will make many cancel it
>> even before getting to open the downloaded launcher.
>>
>> 
>>
>> The screenshot shows that is not a problem with my home connection, which
>> isn't the fastest but is "normal" for my use cases.
>>
>> Regards,
>>
>> Esteban A. Maringolo
>>
>> ps: When I finished writing this email, this was the progress...
>> 
>>
>
>

Re: [Pharo-dev] Pharo Downloads are sluggish

[Marcus Denker]

Ok, so the upgrade of the server contract did not help.

There is a CDN included, I will try to enable that next (but CDNs cache by 
name, so we need to make sure we invalidate the cache…)

I will do a calculation what S3 would cost, too.

Marcus

> On 3 Oct 2018, at 00:59, Esteban Maringolo  wrote:
> 
> I'm reviving this because I can't finish the download of an image using the 
> latest Pharo launcher.
> 
> I'm timeout errors when attempting to download it. Tried 5 times at different 
> intervals.
> 
> 
> 
> 
> Any idea of what could be happening?
> 
> Esteban A. Maringolo
> 
> 
> El mar., 18 sept. 2018 a las 21:03, Esteban Maringolo ( >) escribió:
> Is there a reason why the latest version of the builds linked from the 
> website are not hosted in some CDN, Amazon S3 or similar?
> 
> Maybe in Europe the speed is fast, but where I am (Argentina) it is really 
> slow, the screenshot shows 3.5KB/s, it averages 10KB/s, with peaks of 40KB/s.
> 
> This is an pitiful boarding experience, that will make many cancel it even 
> before getting to open the downloaded launcher.
> 
> 
> 
> The screenshot shows that is not a problem with my home connection, which 
> isn't the fastest but is "normal" for my use cases.
> 
> Regards,
> 
> Esteban A. Maringolo
> 
> ps: When I finished writing this email, this was the progress...
> 

Re: [Pharo-dev] Pharo Downloads are sluggish

[Esteban A. Maringolo]

Hi Marcus,

Thank you and all the team for this.

Best regards,

--
Esteban A. Maringolo

On 24/09/2018 07:05, Marcus Denker wrote:
> Hello,
> 
> We are switching the hosting contract to a more professional one. This
> should be active soon.
> 
> This includes a CDN, but it will be active only after we make sure that
> the files get invalidated when the
> CI uploads them, so this will be in a week or so.
> 
> It should improve things even without the CDN active, though.
> 
> Marcus

Re: [Pharo-dev] Pharo Downloads are sluggish

[Marcus Denker]

Hello,

We are switching the hosting contract to a more professional one. This should 
be active soon.

This includes a CDN, but it will be active only after we make sure that the 
files get invalidated when the
CI uploads them, so this will be in a week or so.

It should improve things even without the CDN active, though.

Marcus

> On 19 Sep 2018, at 15:06, Marcus Denker  wrote:
> 
> I will have a look how to improve the situation.
> 
> (I hope to be able to do it by next week).
> 
>   Marcus
> 
>> On 19 Sep 2018, at 02:10, Esteban Maringolo > > wrote:
>> 
>> After sending the previous email I checked the progress and it was aborted 
>> due to a network error.
>> 
>> 
>> 
>> I retried and apparently the speed got to a more reasonable one, with peaks 
>> of 300KB/sec.
>> 
>> 
>> 
>> 
>> My guess is that the bottleneck is at where the files are hosted, and if you 
>> attempt to download while maybe there are several CI builds running or 
>> something else is using the fileserver bandwidth, then it impacts the 
>> overall download speed of those attempting to download just the basic 
>> elements.
>> 
>> Regards!
>> 
>> Esteban A. Maringolo
>> 
>> 
>> El mar., 18 sept. 2018 a las 21:03, Esteban Maringolo (> >) escribió:
>> Is there a reason why the latest version of the builds linked from the 
>> website are not hosted in some CDN, Amazon S3 or similar?
>> 
>> Maybe in Europe the speed is fast, but where I am (Argentina) it is really 
>> slow, the screenshot shows 3.5KB/s, it averages 10KB/s, with peaks of 40KB/s.
>> 
>> This is an pitiful boarding experience, that will make many cancel it even 
>> before getting to open the downloaded launcher.
>> 
>> 
>> 
>> The screenshot shows that is not a problem with my home connection, which 
>> isn't the fastest but is "normal" for my use cases.
>> 
>> Regards,
>> 
>> Esteban A. Maringolo
>> 
>> ps: When I finished writing this email, this was the progress...
>> 
> 

Re: [Pharo-dev] Pharo Downloads are sluggish

[Esteban Maringolo]

I understand, I'm a heavy user of P2P for most of what's possible, even
have a , but HTTP CDNs exists for a reason, which is "distributed
availability" and low latency.

If we add a bitorrent client as a requirement to have reasonable download
speed then you're adding one extra step to the whole onboarding experience.

Maybe instead of using a CDNs we can think of using mirrors, but then if
the mirror can't withstand the load, you end up in the same situation.

Regards,

Esteban A. Maringolo


El sáb., 22 sept. 2018 a las 2:25, ponyatov ()
escribió:

> I say not about size but *distributed availability*.
>
> Community members can hold many torrent peers let anybody download
> minimizing problems with networks, servers etc, see the topic title.
>
> PS: maybe in 2048 people will have distributed Smalltalk, but not in this
> life with the image hell
>
>
>
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
>
>

Re: [Pharo-dev] Pharo Downloads are sluggish

[ponyatov]

I say not about size but *distributed availability*.

Community members can hold many torrent peers let anybody download
minimizing problems with networks, servers etc, see the topic title.

PS: maybe in 2048 people will have distributed Smalltalk, but not in this
life with the image hell



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Re: [Pharo-dev] Pharo Downloads are sluggish

[Esteban A. Maringolo]

On 20/09/2018 02:11, ponyatov wrote:
> Just add Torrent as download variant

Torrent for a 50mb/file is overkill today.
And adds a new requirement to get started.

A Torrent based Pharo launcher is a different story :)

Regards,

-- 
Esteban A. Maringolo

Re: [Pharo-dev] Pharo Downloads are sluggish

[ponyatov]

Just add Torrent as download variant



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Re: [Pharo-dev] Pharo Downloads are sluggish

[Esteban A. Maringolo]

Hi Marcus,

Thank you.

Anybody WILLING to use it will wait whatever it takes, but my concern,
and rant, is about these that want to give it a try and will abort it
even before running it.

I'm happily using Pharo in Windows preparing it for a demo I'll be making.


Regards!

On 19/09/2018 10:06, Marcus Denker wrote:
> I will have a look how to improve the situation.
> 
> (I hope to be able to do it by next week).
> 
> Marcus
> 
>> On 19 Sep 2018, at 02:10, Esteban Maringolo > > wrote:
>>
>> After sending the previous email I checked the progress and it was
>> aborted due to a network error.
>>
>> 
>>
>> I retried and apparently the speed got to a more reasonable one, with
>> peaks of 300KB/sec.
>>
>> 
>>
>>
>> My guess is that the bottleneck is at where the files are hosted, and
>> if you attempt to download while maybe there are several CI builds
>> running or something else is using the fileserver bandwidth, then it
>> impacts the overall download speed of those attempting to download
>> just the basic elements.
>>
>> Regards!
>>
>> Esteban A. Maringolo
>>
>>
>> El mar., 18 sept. 2018 a las 21:03, Esteban Maringolo
>> (mailto:emaring...@gmail.com>>) escribió:
>>
>> Is there a reason why the latest version of the builds linked from
>> the website are not hosted in some CDN, Amazon S3 or similar?
>>
>> Maybe in Europe the speed is fast, but where I am (Argentina) it
>> is really slow, the screenshot shows 3.5KB/s, it averages 10KB/s,
>> with peaks of 40KB/s.
>>
>> This is an pitiful boarding experience, that will make many cancel
>> it even before getting to open the downloaded launcher.
>>
>> 
>>
>> The screenshot shows that is not a problem with my home
>> connection, which isn't the fastest but is "normal" for my use cases.
>>
>> Regards,
>>
>> Esteban A. Maringolo
>>
>> ps: When I finished writing this email, this was the progress...
>> 
>>
> 

-- 
Esteban A. Maringolo

Re: [Pharo-dev] Pharo Downloads are sluggish

[Marcus Denker]

I will have a look how to improve the situation.

(I hope to be able to do it by next week).

Marcus

> On 19 Sep 2018, at 02:10, Esteban Maringolo  wrote:
> 
> After sending the previous email I checked the progress and it was aborted 
> due to a network error.
> 
> 
> 
> I retried and apparently the speed got to a more reasonable one, with peaks 
> of 300KB/sec.
> 
> 
> 
> 
> My guess is that the bottleneck is at where the files are hosted, and if you 
> attempt to download while maybe there are several CI builds running or 
> something else is using the fileserver bandwidth, then it impacts the overall 
> download speed of those attempting to download just the basic elements.
> 
> Regards!
> 
> Esteban A. Maringolo
> 
> 
> El mar., 18 sept. 2018 a las 21:03, Esteban Maringolo ( >) escribió:
> Is there a reason why the latest version of the builds linked from the 
> website are not hosted in some CDN, Amazon S3 or similar?
> 
> Maybe in Europe the speed is fast, but where I am (Argentina) it is really 
> slow, the screenshot shows 3.5KB/s, it averages 10KB/s, with peaks of 40KB/s.
> 
> This is an pitiful boarding experience, that will make many cancel it even 
> before getting to open the downloaded launcher.
> 
> 
> 
> The screenshot shows that is not a problem with my home connection, which 
> isn't the fastest but is "normal" for my use cases.
> 
> Regards,
> 
> Esteban A. Maringolo
> 
> ps: When I finished writing this email, this was the progress...
> 

Re: [Pharo-dev] Pharo Downloads are sluggish

[Esteban Maringolo]

After sending the previous email I checked the progress and it was aborted
due to a network error.

[image: pharo-cancelled.PNG]

I retried and apparently the speed got to a more reasonable one, with peaks
of 300KB/sec.

[image: pharo-retry.png]


My guess is that the bottleneck is at where the files are hosted, and if
you attempt to download while maybe there are several CI builds running or
something else is using the fileserver bandwidth, then it impacts the
overall download speed of those attempting to download just the basic
elements.

Regards!

Esteban A. Maringolo


El mar., 18 sept. 2018 a las 21:03, Esteban Maringolo ()
escribió:

> Is there a reason why the latest version of the builds linked from the
> website are not hosted in some CDN, Amazon S3 or similar?
>
> Maybe in Europe the speed is fast, but where I am (Argentina) it is really
> slow, the screenshot shows 3.5KB/s, it averages 10KB/s, with peaks of
> 40KB/s.
>
> This is an pitiful boarding experience, that will make many cancel it even
> before getting to open the downloaded launcher.
>
> [image: node-pharo.png]
>
> The screenshot shows that is not a problem with my home connection, which
> isn't the fastest but is "normal" for my use cases.
>
> Regards,
>
> Esteban A. Maringolo
>
> ps: When I finished writing this email, this was the progress...
> [image: image.png]
>

[Pharo-dev] Pharo Downloads are sluggish

[Esteban Maringolo]

Is there a reason why the latest version of the builds linked from the
website are not hosted in some CDN, Amazon S3 or similar?

Maybe in Europe the speed is fast, but where I am (Argentina) it is really
slow, the screenshot shows 3.5KB/s, it averages 10KB/s, with peaks of
40KB/s.

This is an pitiful boarding experience, that will make many cancel it even
before getting to open the downloaded launcher.

[image: node-pharo.png]

The screenshot shows that is not a problem with my home connection, which
isn't the fastest but is "normal" for my use cases.

Regards,

Esteban A. Maringolo

ps: When I finished writing this email, this was the progress...
[image: image.png]