Re: freebsd-update and speed

[Philip Paeps]

On 2021-04-18 13:57:45 (+0800), Jason Tubnor wrote:


On Sun, 18 Apr 2021 at 10:51, Philip Paeps  wrote:




It looks like there were at least experiments with pointing
freebsd-update at AWS, similar to how portsnap currently works.  I 
will

check if these experiments went anywhere and possibly point
freebsd-update there instead.



The AWS freebsd-update has been working fine for quite a while.  All
project mirrors are slow in Australia so we have been using the AWS 
one
since Colin brought it online to make it a better experience for our 
team

to update the fleet.

FWIW adjust update.FreeBSD.org to aws.update.FreeBSD.org in
/etc/freebsd-update.conf and you are good to go.

Can the project look at offering traditional mirrors for base and pkgs
rather than the current offering?  Those that want to stand up 
un-official

mirrors can do so by pointing rsync to a Tier 2 mirror for updating
purposes so they can provide faster, localised access.  From Melbourne 
to
our closest geo mirror is 240ms, this latency really drags out 
updates, so

having a mirror out of a Melbourne DC would be beneficial.


We've got an ongoing action item to set up a traditional pkg and 
download mirror at IX Australia.  This has kept stalling out over the 
past year-and-a-bit because the world is on fire.  I'll try to pick this 
up again Soon.


Currently, from Australia you're either sent to a pkg or download mirror 
in Malaysia or on the west coast of America.  Neither of those are 
great.  Australia is a big island, far away from everywhere. :)


Glad to hear the AWS stuff is working for you.  I'll see what needs to 
happen to put that in the SRV record for everyone.


Philip

--
Philip Paeps
Senior Reality Engineer
Alternative Enterprises
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd-update and speed

[Jason Tubnor]

On Sun, 18 Apr 2021 at 10:51, Philip Paeps  wrote:

>
>
> It looks like there were at least experiments with pointing
> freebsd-update at AWS, similar to how portsnap currently works.  I will
> check if these experiments went anywhere and possibly point
> freebsd-update there instead.
>

The AWS freebsd-update has been working fine for quite a while.  All
project mirrors are slow in Australia so we have been using the AWS one
since Colin brought it online to make it a better experience for our team
to update the fleet.

FWIW adjust update.FreeBSD.org to aws.update.FreeBSD.org in
/etc/freebsd-update.conf and you are good to go.

Can the project look at offering traditional mirrors for base and pkgs
rather than the current offering?  Those that want to stand up un-official
mirrors can do so by pointing rsync to a Tier 2 mirror for updating
purposes so they can provide faster, localised access.  From Melbourne to
our closest geo mirror is 240ms, this latency really drags out updates, so
having a mirror out of a Melbourne DC would be beneficial.

Cheers,

Jason.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd-update and speed

[Philip Paeps]

On 2021-04-18 08:51:05 (+0800), Philip Paeps wrote:

On 2021-04-18 03:12:35 (+0800), Rainer Duffner wrote:
I’m cc-ing clusteradm and dnsadmin, in hope that there’s somebody 
there who can either fix it or take update4 out of the srv record…


I can take update4 out of the DNS if it's misbehaving consistently.  
If at all possible though, I'd prefer to fix the actual problem rather 
than simply make it disappear from the DNS.


I've taken update4 out of the SRV record for the time being since it was 
doing more harm than good.


Philip

--
Philip Paeps
Senior Reality Engineer
Alternative Enterprises
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd-update and speed

[Philip Paeps]

On 2021-04-18 03:12:35 (+0800), Rainer Duffner wrote:

Am 16.04.2021 um 10:17 schrieb Ferdinand Goldmann 
:


On Thu, 15 Apr 2021, Rainer Duffner wrote:




It’s OK-ish most of the time here (CH).

It does *NOT* work through a proxy, due to the use of pipelined 
http-requests.


What’s your internet-connection?


The 10Gbit uplink of my university, directly connected to the 
internet, not
behind a proxy. I don't think that's the problem. When update3 was 
still online

I'd always use that and updates were really fast back then.

Now that update3 is gone all update servers seem to be in the US or 
Australia.


After waiting for nearly one hour:

..853085408550856085708580859086008610862086308640865086608670868086908700 
 done.

Applying patches... done.
Fetching 9628 files... gunzip: (stdin): unexpected end of file
0a4626107f3700cf5f87bd9c123bf427bd5a8561aadc2eca1d1605465c090935 has 
incorrect hash.


This is getting kind of tiresome. :(

Regards
Ferdinand





There seems to be a problem with update4.

I now have this, too.


I’m cc-ing clusteradm and dnsadmin, in hope that there’s somebody 
there who can either fix it or take update4 out of the srv record…


I can take update4 out of the DNS if it's misbehaving consistently.  If 
at all possible though, I'd prefer to fix the actual problem rather than 
simply make it disappear from the DNS.


It looks like there were at least experiments with pointing 
freebsd-update at AWS, similar to how portsnap currently works.  I will 
check if these experiments went anywhere and possibly point 
freebsd-update there instead.


I believe the problem with update4 is load-related.


:-(

I would rather just mirror the update server but I think this is not 
supposed to be done?


I think you can set up your own freebsd-update servers, but that won't 
fix this problem.


I'll see what can be done to fix this.

Watch this space.

Philip [hat: clusteradm firefighter]


--
Philip Paeps
Senior Reality Engineer
Alternative Enterprises
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd-update and speed

[Cejka Rudolf]

Ferdinand Goldmann wrote (2021/04/15):
> Hello,
> 
> I've noticed that ever since update3.freebsd.org is gone (which was in Czech
> republic I think), FreeBSD updates are often quite slow for me (= 
> Austria/Europe)
> Especially so for major release upgrades. In fact so slow that I have time
> to type this mail while waiting for '8778 patches'.

Hello, you are right, it was ;o)

> The other day, freebsd-update even suffered a timeout.
> 
> What are other European users experiences and is there anything to do about 
> it?

Did you try aws.update.freebsd.org also?

-- 
Rudolf Cejka  http://www.fit.vut.cz/~cejkar
Brno University of Technology, Faculty of Information Technology
Bozetechova 2, 612 66  Brno, Czech Republic
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

geli - is it better to partition then encrypt, or vice versa ?

[Freddie Cash]

On Sat., Apr. 17, 2021, 1:04 p.m. Clayton Milos,  wrote:

> I encrypt the whole disk and then add it to the pool. No need to partition
> it. If I remember correctly zfs prefers unpartitioned disks


>
ZFS on Solaris used to require the use of entire, raw disks as the cache
was disabled if the disk was partitioned, tanking performance

ZFS on FreeBSD has never had this issue, and has fully supported the use of
partitioned disks from the very first import of ZFS into 7-Stable.

No other OS that supports ZFS has this issue; it's strictly a Solaris (and
derivatives) issue.

Cheers,
Freddie

Typos due to smartphone keyboard.


-- 
Freddie Cash
fjwc...@gmail.com
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: geli - is it better to partition then encrypt, or vice versa ?

[Karl Denninger]

On 4/17/2021 15:52, Pete French wrote:
So, am building a zpool on some encrypted discs - and what I have done 
is to partition the disc with GPT add a single big partition, and 
encrypt that. So the pool is on nda1p1.eli.


But I could, of course, encrypt the disc first, and then partition the 
encrypted disc, or indded just put the zpool directly onto it.


Just wondering what the general consensus is as to the best way to go 
here ... if there is one! :-) What do other people do ?


IMHO one reason to partition first (and the reason I do it) is to 
prevent "drive attachment point hopping" from causing an unwelcome 
surprise if/when there is a failure or if, for some reason, you plug a 
drive into a different machine at some point.  If you partition and 
label, then geli init and attach at "/dev/gpt/the-label" you now can 
label the drive carrier with that and irrespective of the slot or 
adapter that gets connected to on whatever machine it will be in the 
same place.  If it fails this also means (assuming you labeled the 
carrier) you know which carrier to yank and replace. Yanking the wrong 
drive can be an unpleasant surprise.


This also makes "geli groups" trivial in /etc/rc.conf for attachment at 
boot time irrespective of whether they physically come up in the same 
place (again typically yes, but in the case of a failure or you plug it 
into a different adapter.)


--
Karl Denninger
k...@denninger.net 
/The Market Ticker/
/[S/MIME encrypted email preferred]/


smime.p7s
Description: S/MIME Cryptographic Signature

Re: geli - is it better to partition then encrypt, or vice versa ?

[Alan Somers]

On Sat, Apr 17, 2021 at 1:53 PM Pete French 
wrote:

> So, am building a zpool on some encrypted discs - and what I have done
> is to partition the disc with GPT add a single big partition, and
> encrypt that. So the pool is on nda1p1.eli.
>
> But I could, of course, encrypt the disc first, and then partition the
> encrypted disc, or indded just put the zpool directly onto it.
>
> Just wondering what the general consensus is as to the best way to go
> here ... if there is one! :-) What do other people do ?
>
> -pete.
>

The answer depends on why you want to partition in the first place.  What
do you intend to store on those disks besides ZFS?  If the answer is
nothing, then don't bother partitioning; just write ZFS over GELI over the
whole disk.

(Also, it's worth asking why you want GELI, now that FreeBSD 13 supports
ZFS native crypto.  ZFS native crypto on RAIDZ has substantially better
write performance than RAIDZ on GELI.  However, if you're paranoid, then
GELI does provide better security; ZFS native crypto is vulnerable to some
kinds of watermarking attacks.)
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: geli - is it better to partition then encrypt, or vice versa ?

[Clayton Milos]

I encrypt the whole disk and then add it to the pool. No need to partition it. 
If I remember correctly zfs prefers unpartitioned disks.

\\Clay

> On 17 Apr 2021, at 21:54, Pete French  wrote:
> 
> So, am building a zpool on some encrypted discs - and what I have done is to 
> partition the disc with GPT add a single big partition, and encrypt that. So 
> the pool is on nda1p1.eli.
> 
> But I could, of course, encrypt the disc first, and then partition the 
> encrypted disc, or indded just put the zpool directly onto it.
> 
> Just wondering what the general consensus is as to the best way to go here 
> ... if there is one! :-) What do other people do ?
> 
> -pete.
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

geli - is it better to partition then encrypt, or vice versa ?

[Pete French]

So, am building a zpool on some encrypted discs - and what I have done 
is to partition the disc with GPT add a single big partition, and 
encrypt that. So the pool is on nda1p1.eli.


But I could, of course, encrypt the disc first, and then partition the 
encrypted disc, or indded just put the zpool directly onto it.


Just wondering what the general consensus is as to the best way to go 
here ... if there is one! :-) What do other people do ?


-pete.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd-update and speed

[Rainer Duffner]

> Am 16.04.2021 um 10:17 schrieb Ferdinand Goldmann :
> 
> On Thu, 15 Apr 2021, Rainer Duffner wrote:
> 
>> 
>> 
>> It’s OK-ish most of the time here (CH).
>> 
>> It does *NOT* work through a proxy, due to the use of pipelined 
>> http-requests.
>> 
>> What’s your internet-connection?
> 
> The 10Gbit uplink of my university, directly connected to the internet, not
> behind a proxy. I don't think that's the problem. When update3 was still 
> online
> I'd always use that and updates were really fast back then.
> 
> Now that update3 is gone all update servers seem to be in the US or Australia.
> 
> After waiting for nearly one hour:
> 
> ..853085408550856085708580859086008610862086308640865086608670868086908700
>   done.
> Applying patches... done.
> Fetching 9628 files... gunzip: (stdin): unexpected end of file
> 0a4626107f3700cf5f87bd9c123bf427bd5a8561aadc2eca1d1605465c090935 has 
> incorrect hash.
> 
> This is getting kind of tiresome. :(
> 
> Regards
> Ferdinand




There seems to be a problem with update4.

I now have this, too.


I’m cc-ing clusteradm and dnsadmin, in hope that there’s somebody there who can 
either fix it or take update4 out of the srv record…


:-(

I would rather just mirror the update server but I think this is not supposed 
to be done?





___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"