Re: bsd.sites.mk: Do we prefer http or https (or both)

2017-03-13 Thread Tijl Coosemans 
On Mon, 13 Mar 2017 09:32:13 -0600 Adam Weinberger  wrote:
> On 13 Mar, 2017, at 7:32, Tijl Coosemans  wrote:
>> On Sat, 11 Mar 2017 14:25:13 -0700 Adam Weinberger 
>> wrote:
 On 11 Mar, 2017, at 12:53, Adam Weinberger  wrote:
> On 11 Mar, 2017, at 12:29, Tijl Coosemans  wrote:
> On Sat, 11 Mar 2017 10:18:18 -0700 Adam Weinberger 
> wrote:
>> On 11 Mar, 2017, at 10:13, Tijl Coosemans 
>> wrote:
>>> On Sat, 11 Mar 2017 12:18:51 + (UTC) jbe...@freebsd.org (Jan
>>> Beich) wrote:
 Tijl Coosemans  writes:
> On Sat, 11 Mar 2017 10:53:01 +0100 (CET) Gerald Pfeifer
>  wrote:
>> As some of you may have seen, I have done a bit of work on
>> bsd.sites.mk recently.
>> 
>> One question I ran into:  If a site offers both HTTPS and
>> HTTP, which of the two do we prefer?  (Or do we want to list
>> both?)
> 
> https first for people that run 'make makesum'.
 
 It was made MITM-friendly sometime ago.
 
 https://svnweb.freebsd.org/changeset/ports/324051
>>> 
>>> Ugh, can portmgr approve the attached patch?
>> 
>> If distfiles from sites with invalid certificates won't fetch for
>> end-users, they won't fetch during makesum either.
> 
> - Given that web browsers have become much less forgiving about such
>  certificates this is probably much less of a problem nowadays.
> - Possibly, many of these errors are because users forgot to install
>  ca_root_nss.  We can hold port maintainers to a higher standard and
>  expect them to have this installed.
> - Such sites should perhaps be removed from MASTER_SITES.  If
>  that's not possible FETCH_ENV can be set in the port Makefile.
 
 I don't disagree with any point. Do you want to submit a PR so that
 an exp-run of sorts can see how many distfiles we're talking about?
>>> 
>>> Antoine reminded me that this only affects makesum, so I guess there's
>>> really no way of telling what ports this would affect. Either way,
>>> your reasoning is sound and you've convinced me. I'm good with this
>>> change; as you said, worst-case scenario, ports with broken
>>> MASTER_SITES can override FETCH_ENV or a toggle can be added.
>> 
>> Committed in r436081.
>
> Can you please add a quick blurb about this to CHANGES?

Added in r436086.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: bsd.sites.mk: Do we prefer http or https (or both)

2017-03-13 Thread Adam Weinberger 
> On 13 Mar, 2017, at 7:32, Tijl Coosemans  wrote:
> 
> On Sat, 11 Mar 2017 14:25:13 -0700 Adam Weinberger 
> wrote:
>>> On 11 Mar, 2017, at 12:53, Adam Weinberger  wrote:
 On 11 Mar, 2017, at 12:29, Tijl Coosemans  wrote:
 On Sat, 11 Mar 2017 10:18:18 -0700 Adam Weinberger 
 wrote:
> On 11 Mar, 2017, at 10:13, Tijl Coosemans 
> wrote:
>> On Sat, 11 Mar 2017 12:18:51 + (UTC) jbe...@freebsd.org (Jan
>> Beich) wrote:
>>> Tijl Coosemans  writes:
 On Sat, 11 Mar 2017 10:53:01 +0100 (CET) Gerald Pfeifer
  wrote:
> As some of you may have seen, I have done a bit of work on
> bsd.sites.mk recently.
> 
> One question I ran into:  If a site offers both HTTPS and
> HTTP, which of the two do we prefer?  (Or do we want to list
> both?)
 
 https first for people that run 'make makesum'.
>>> 
>>> It was made MITM-friendly sometime ago.
>>> 
>>> https://svnweb.freebsd.org/changeset/ports/324051
>> 
>> Ugh, can portmgr approve the attached patch?
> 
> If distfiles from sites with invalid certificates won't fetch for
> end-users, they won't fetch during makesum either.
 
 - Given that web browsers have become much less forgiving about such
  certificates this is probably much less of a problem nowadays.
 - Possibly, many of these errors are because users forgot to install
  ca_root_nss.  We can hold port maintainers to a higher standard and
  expect them to have this installed.
 - Such sites should perhaps be removed from MASTER_SITES.  If
  that's not possible FETCH_ENV can be set in the port Makefile.
>>> 
>>> I don't disagree with any point. Do you want to submit a PR so that
>>> an exp-run of sorts can see how many distfiles we're talking about?
>> 
>> Antoine reminded me that this only affects makesum, so I guess there's
>> really no way of telling what ports this would affect. Either way,
>> your reasoning is sound and you've convinced me. I'm good with this
>> change; as you said, worst-case scenario, ports with broken
>> MASTER_SITES can override FETCH_ENV or a toggle can be added.
> 
> Committed in r436081.

Can you please add a quick blurb about this to CHANGES?

# Adam


-- 
Adam Weinberger
ad...@adamw.org
https://www.adamw.org


___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: bsd.sites.mk: Do we prefer http or https (or both)

2017-03-13 Thread Tijl Coosemans 
On Sat, 11 Mar 2017 14:25:13 -0700 Adam Weinberger 
wrote:
>> On 11 Mar, 2017, at 12:53, Adam Weinberger  wrote:
>>> On 11 Mar, 2017, at 12:29, Tijl Coosemans  wrote:
>>> On Sat, 11 Mar 2017 10:18:18 -0700 Adam Weinberger 
>>> wrote:
 On 11 Mar, 2017, at 10:13, Tijl Coosemans 
 wrote:
> On Sat, 11 Mar 2017 12:18:51 + (UTC) jbe...@freebsd.org (Jan
> Beich) wrote:
>> Tijl Coosemans  writes:
>>> On Sat, 11 Mar 2017 10:53:01 +0100 (CET) Gerald Pfeifer
>>>  wrote:
 As some of you may have seen, I have done a bit of work on
 bsd.sites.mk recently.
 
 One question I ran into:  If a site offers both HTTPS and
 HTTP, which of the two do we prefer?  (Or do we want to list
 both?)
>>> 
>>> https first for people that run 'make makesum'.
>> 
>> It was made MITM-friendly sometime ago.
>> 
>> https://svnweb.freebsd.org/changeset/ports/324051
> 
> Ugh, can portmgr approve the attached patch?
 
 If distfiles from sites with invalid certificates won't fetch for
 end-users, they won't fetch during makesum either.
>>> 
>>> - Given that web browsers have become much less forgiving about such
>>>   certificates this is probably much less of a problem nowadays.
>>> - Possibly, many of these errors are because users forgot to install
>>>   ca_root_nss.  We can hold port maintainers to a higher standard and
>>>   expect them to have this installed.
>>> - Such sites should perhaps be removed from MASTER_SITES.  If
>>>   that's not possible FETCH_ENV can be set in the port Makefile.
>> 
>> I don't disagree with any point. Do you want to submit a PR so that
>> an exp-run of sorts can see how many distfiles we're talking about?
>
> Antoine reminded me that this only affects makesum, so I guess there's
> really no way of telling what ports this would affect. Either way,
> your reasoning is sound and you've convinced me. I'm good with this
> change; as you said, worst-case scenario, ports with broken
> MASTER_SITES can override FETCH_ENV or a toggle can be added.

Committed in r436081.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: bsd.sites.mk: Do we prefer http or https (or both)

2017-03-13 Thread Mathieu Arnold 
Le 11/03/2017 à 19:32, Eitan Adler a écrit :
> On 11 March 2017 at 09:13, Tijl Coosemans  wrote:
>> On Sat, 11 Mar 2017 12:18:51 + (UTC) jbe...@freebsd.org (Jan Beich) 
>> wrote:
>>> Tijl Coosemans  writes:
 On Sat, 11 Mar 2017 10:53:01 +0100 (CET) Gerald Pfeifer 
  wrote:
> As some of you may have seen, I have done a bit of work on
> bsd.sites.mk recently.
>
> One question I ran into:  If a site offers both HTTPS and HTTP,
> which of the two do we prefer?  (Or do we want to list both?)
 https first for people that run 'make makesum'.
>>> It was made MITM-friendly sometime ago.
>>>
>>> https://svnweb.freebsd.org/changeset/ports/324051
>> Ugh, can portmgr approve the attached patch?
> I can't approve on behalf of portmgr but I'd like to echo this
> request on behalf of ports-secteam. Maintainers rarely verify the
> hashes that makesum generates.
>
> I wish we can go further and filter out non-HTTPS sites during makesum.

This should be pretty easy to do with the existing MASTER_SORT feature.

-- 
Mathieu Arnold




signature.asc
Description: OpenPGP digital signature

Re: bsd.sites.mk: Do we prefer http or https (or both)

2017-03-11 Thread Adam Weinberger 
> On 11 Mar, 2017, at 12:53, Adam Weinberger  wrote:
> 
>> On 11 Mar, 2017, at 12:29, Tijl Coosemans  wrote:
>> 
>> On Sat, 11 Mar 2017 10:18:18 -0700 Adam Weinberger 
>> wrote:
>>> On 11 Mar, 2017, at 10:13, Tijl Coosemans  wrote:
 On Sat, 11 Mar 2017 12:18:51 + (UTC) jbe...@freebsd.org (Jan
 Beich) wrote:  
> Tijl Coosemans  writes:  
>> On Sat, 11 Mar 2017 10:53:01 +0100 (CET) Gerald Pfeifer
>>  wrote:  
>>> As some of you may have seen, I have done a bit of work on
>>> bsd.sites.mk recently.
>>> 
>>> One question I ran into:  If a site offers both HTTPS and HTTP, 
>>> which of the two do we prefer?  (Or do we want to list both?)
>> 
>> https first for people that run 'make makesum'.
> 
> It was made MITM-friendly sometime ago.
> 
> https://svnweb.freebsd.org/changeset/ports/324051  
 
 Ugh, can portmgr approve the attached
 patch?___  
>>> 
>>> If distfiles from sites with invalid certificates won't fetch for
>>> end-users, they won't fetch during makesum either.
>> 
>> - Given that web browsers have become much less forgiving about such
>> certificates this is probably much less of a problem nowadays.
>> - Possibly, many of these errors are because users forgot to install
>> ca_root_nss.  We can hold port maintainers to a higher standard and
>> expect them to have this installed.
>> - Such sites should perhaps be removed from MASTER_SITES.  If that's not
>> possible FETCH_ENV can be set in the port Makefile.
> 
> I don't disagree with any point. Do you want to submit a PR so that an 
> exp-run of sorts can see how many distfiles we're talking about?

Antoine reminded me that this only affects makesum, so I guess there's really 
no way of telling what ports this would affect. Either way, your reasoning is 
sound and you've convinced me. I'm good with this change; as you said, 
worst-case scenario, ports with broken MASTER_SITES can override FETCH_ENV or a 
toggle can be added.

# Adam


-- 
Adam Weinberger
ad...@adamw.org
https://www.adamw.org

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: bsd.sites.mk: Do we prefer http or https (or both)

2017-03-11 Thread Adam Weinberger 
> On 11 Mar, 2017, at 12:29, Tijl Coosemans  wrote:
> 
> On Sat, 11 Mar 2017 10:18:18 -0700 Adam Weinberger 
> wrote:
>> On 11 Mar, 2017, at 10:13, Tijl Coosemans  wrote:
>>> On Sat, 11 Mar 2017 12:18:51 + (UTC) jbe...@freebsd.org (Jan
>>> Beich) wrote:  
 Tijl Coosemans  writes:  
> On Sat, 11 Mar 2017 10:53:01 +0100 (CET) Gerald Pfeifer
>  wrote:  
>> As some of you may have seen, I have done a bit of work on
>> bsd.sites.mk recently.
>> 
>> One question I ran into:  If a site offers both HTTPS and HTTP, 
>> which of the two do we prefer?  (Or do we want to list both?)
> 
> https first for people that run 'make makesum'.
 
 It was made MITM-friendly sometime ago.
 
 https://svnweb.freebsd.org/changeset/ports/324051  
>>> 
>>> Ugh, can portmgr approve the attached
>>> patch?___  
>> 
>> If distfiles from sites with invalid certificates won't fetch for
>> end-users, they won't fetch during makesum either.
> 
> - Given that web browsers have become much less forgiving about such
>  certificates this is probably much less of a problem nowadays.
> - Possibly, many of these errors are because users forgot to install
>  ca_root_nss.  We can hold port maintainers to a higher standard and
>  expect them to have this installed.
> - Such sites should perhaps be removed from MASTER_SITES.  If that's not
>  possible FETCH_ENV can be set in the port Makefile.

I don't disagree with any point. Do you want to submit a PR so that an exp-run 
of sorts can see how many distfiles we're talking about?

# Adam


-- 
Adam Weinberger
ad...@adamw.org
https://www.adamw.org

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: bsd.sites.mk: Do we prefer http or https (or both)

2017-03-11 Thread Tijl Coosemans 
On Sat, 11 Mar 2017 10:18:18 -0700 Adam Weinberger 
wrote:
> On 11 Mar, 2017, at 10:13, Tijl Coosemans  wrote:
>> On Sat, 11 Mar 2017 12:18:51 + (UTC) jbe...@freebsd.org (Jan
>> Beich) wrote:  
>>> Tijl Coosemans  writes:  
 On Sat, 11 Mar 2017 10:53:01 +0100 (CET) Gerald Pfeifer
  wrote:  
> As some of you may have seen, I have done a bit of work on
> bsd.sites.mk recently.
> 
> One question I ran into:  If a site offers both HTTPS and HTTP, 
> which of the two do we prefer?  (Or do we want to list both?)
 
 https first for people that run 'make makesum'.
>>> 
>>> It was made MITM-friendly sometime ago.
>>> 
>>> https://svnweb.freebsd.org/changeset/ports/324051  
>> 
>> Ugh, can portmgr approve the attached
>> patch?___  
> 
> If distfiles from sites with invalid certificates won't fetch for
> end-users, they won't fetch during makesum either.

- Given that web browsers have become much less forgiving about such
  certificates this is probably much less of a problem nowadays.
- Possibly, many of these errors are because users forgot to install
  ca_root_nss.  We can hold port maintainers to a higher standard and
  expect them to have this installed.
- Such sites should perhaps be removed from MASTER_SITES.  If that's not
  possible FETCH_ENV can be set in the port Makefile.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: bsd.sites.mk: Do we prefer http or https (or both)

2017-03-11 Thread Eitan Adler 
On 11 March 2017 at 09:13, Tijl Coosemans  wrote:
> On Sat, 11 Mar 2017 12:18:51 + (UTC) jbe...@freebsd.org (Jan Beich) wrote:
>> Tijl Coosemans  writes:
>>> On Sat, 11 Mar 2017 10:53:01 +0100 (CET) Gerald Pfeifer 
>>>  wrote:
 As some of you may have seen, I have done a bit of work on
 bsd.sites.mk recently.

 One question I ran into:  If a site offers both HTTPS and HTTP,
 which of the two do we prefer?  (Or do we want to list both?)
>>>
>>> https first for people that run 'make makesum'.
>>
>> It was made MITM-friendly sometime ago.
>>
>> https://svnweb.freebsd.org/changeset/ports/324051
>
> Ugh, can portmgr approve the attached patch?

I can't approve on behalf of portmgr but I'd like to echo this
request on behalf of ports-secteam. Maintainers rarely verify the
hashes that makesum generates.

I wish we can go further and filter out non-HTTPS sites during makesum.


-- 
Eitan Adler
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: bsd.sites.mk: Do we prefer http or https (or both)

2017-03-11 Thread Adam Weinberger 
> On 11 Mar, 2017, at 10:45, Gerald Pfeifer  wrote:
> 
> On Sat, 11 Mar 2017, Jan Beich wrote:
>>> https first for people that run 'make makesum'.
>> It was made MITM-friendly sometime ago.
>> 
>> https://svnweb.freebsd.org/changeset/ports/324051
> 
> With that, isn't https pretty pointless?  I guess I'll leave
> things as are, then, for that mirror that offers both.
> 
> Another question on "https first", Tijl.  Some MASTER_SITEs
> have a dozen entries or more, and I always thought that the
> infrastructure picks one of these randomly every time.  In
> some tests I did today with two sites (one https, one http)
> it _always_ picked the first, confirming your point.  Or is
> that only the case for `make makesum`?
> 
> Gerald

That's activated by RANDOMIZE_MASTER_SITES.

# Adam


-- 
Adam Weinberger
ad...@adamw.org
https://www.adamw.org


___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: bsd.sites.mk: Do we prefer http or https (or both)

2017-03-11 Thread Gerald Pfeifer 
On Sat, 11 Mar 2017, Jan Beich wrote:
>> https first for people that run 'make makesum'.
> It was made MITM-friendly sometime ago.
>
> https://svnweb.freebsd.org/changeset/ports/324051

With that, isn't https pretty pointless?  I guess I'll leave
things as are, then, for that mirror that offers both.
 
Another question on "https first", Tijl.  Some MASTER_SITEs
have a dozen entries or more, and I always thought that the
infrastructure picks one of these randomly every time.  In
some tests I did today with two sites (one https, one http)
it _always_ picked the first, confirming your point.  Or is
that only the case for `make makesum`?

Gerald
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: bsd.sites.mk: Do we prefer http or https (or both)

2017-03-11 Thread Adam Weinberger 
> On 11 Mar, 2017, at 10:13, Tijl Coosemans  wrote:
> 
> On Sat, 11 Mar 2017 12:18:51 + (UTC) jbe...@freebsd.org (Jan Beich) wrote:
>> Tijl Coosemans  writes:
>>> On Sat, 11 Mar 2017 10:53:01 +0100 (CET) Gerald Pfeifer 
>>>  wrote:
 As some of you may have seen, I have done a bit of work on
 bsd.sites.mk recently.
 
 One question I ran into:  If a site offers both HTTPS and HTTP, 
 which of the two do we prefer?  (Or do we want to list both?)  
>>> 
>>> https first for people that run 'make makesum'.  
>> 
>> It was made MITM-friendly sometime ago.
>> 
>> https://svnweb.freebsd.org/changeset/ports/324051
> 
> Ugh, can portmgr approve the attached 
> patch?___

If distfiles from sites with invalid certificates won't fetch for end-users, 
they won't fetch during makesum either.

# Adam


-- 
Adam Weinberger
ad...@adamw.org
https://www.adamw.org


___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: bsd.sites.mk: Do we prefer http or https (or both)

2017-03-11 Thread Tijl Coosemans 
On Sat, 11 Mar 2017 12:18:51 + (UTC) jbe...@freebsd.org (Jan Beich) wrote:
> Tijl Coosemans  writes:
>> On Sat, 11 Mar 2017 10:53:01 +0100 (CET) Gerald Pfeifer  
>> wrote:
>>> As some of you may have seen, I have done a bit of work on
>>> bsd.sites.mk recently.
>>> 
>>> One question I ran into:  If a site offers both HTTPS and HTTP, 
>>> which of the two do we prefer?  (Or do we want to list both?)  
>>
>> https first for people that run 'make makesum'.  
> 
> It was made MITM-friendly sometime ago.
> 
> https://svnweb.freebsd.org/changeset/ports/324051

Ugh, can portmgr approve the attached patch?Index: Mk/bsd.port.mk
===
--- Mk/bsd.port.mk	(revision 435950)
+++ Mk/bsd.port.mk	(working copy)
@@ -2007,7 +2007,9 @@ BUILD_FAIL_MESSAGE+=	Try to set MAKE_JOB
 
 .include "${PORTSDIR}/Mk/bsd.ccache.mk"
 
+.if !make(makesum)
 FETCH_ENV?=		SSL_NO_VERIFY_PEER=1 SSL_NO_VERIFY_HOSTNAME=1
+.endif
 FETCH_BINARY?=	/usr/bin/fetch
 FETCH_ARGS?=	-Fpr
 FETCH_REGET?=	1
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: bsd.sites.mk: Do we prefer http or https (or both)

2017-03-11 Thread Jan Beich 
Tijl Coosemans  writes:

> On Sat, 11 Mar 2017 10:53:01 +0100 (CET) Gerald Pfeifer  
> wrote:
>
>> As some of you may have seen, I have done a bit of work on
>> bsd.sites.mk recently.
>> 
>> One question I ran into:  If a site offers both HTTPS and HTTP, 
>> which of the two do we prefer?  (Or do we want to list both?)
>
> https first for people that run 'make makesum'.

It was made MITM-friendly sometime ago.

https://svnweb.freebsd.org/changeset/ports/324051
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: bsd.sites.mk: Do we prefer http or https (or both)

2017-03-11 Thread Tijl Coosemans 
On Sat, 11 Mar 2017 10:53:01 +0100 (CET) Gerald Pfeifer  
wrote:
> As some of you may have seen, I have done a bit of work on
> bsd.sites.mk recently.
> 
> One question I ran into:  If a site offers both HTTPS and HTTP, 
> which of the two do we prefer?  (Or do we want to list both?)

https first for people that run 'make makesum'.
http second for people that can't use https.

For pkg-descr WWW I always use https if available.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"