ZFSROOT UEFI boot
Hi, I need to get one of my machines converted over from bios GPT zfsroot boot to efi. I know you can boot freebsd under EFI with a ufs kernel but this isnt the route i want. There are patches under test for EFI zfs root. However when I read the thread it was unclear which version of these patches were needed and where to get them. Does anyone know where they are, if there are any prebuilt zfsloader etc binaries, or if the patches have made it to head yet? Also does anyone have any pointers or good experience with grub efi and zfs on root? I'm considering this option as it would make booting into specific boot environments easier ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: ZFSROOT UEFI boot
Ive literally just got this working on 10.2 after working on the code posted on the review which you can find here: https://reviews.freebsd.org/D4104 If you're happy running current then the patch file I linked in my comment should apply cleanly and just work. If you want 10.x then there's quite a bit more needed. As I said I do have this working so can post patches when I'm back in the office. Either way once applied a standard efi install just works. Essentially create efi partition and use gpart to install the efi bootcode and away you go. I've just used this with a custom mfsbsd iso to perform and 10.2-RELEASE ZFS boot install on some Intel nvme disks setup as raidz2, which only support efi boot. On 10 Dec 2015, at 12:18, krad wrote: Hi, I need to get one of my machines converted over from bios GPT zfsroot boot to efi. I know you can boot freebsd under EFI with a ufs kernel but this isnt the route i want. There are patches under test for EFI zfs root. However when I read the thread it was unclear which version of these patches were needed and where to get them. Does anyone know where they are, if there are any prebuilt zfsloader etc binaries, or if the patches have made it to head yet? Also does anyone have any pointers or good experience with grub efi and zfs on root? I'm considering this option as it would make booting into specific boot environments easier ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
RPC request sent to 127.0.0.1 becomes from other IP on machine
Hi,
Mark has reported a problem via email where the nfsuserd daemon sees
requests coming from an IP# assigned to the machine instead of 127.0.0.1.
Here's a snippet from his message:
Ok, I have Plex in a jail and when I scan the remote NFS file share the
*local* server's nfsuserd spams the logs.
Spamming the logs refers to the messages nfsuserd generates when it gets
a request from an address other than 127.0.0.1.
I think the best solution is to switch nfsuserd over to using an AF_LOCAL
socket like the gssd uses, but that will take a little coding and probably
won't be MFCable.
I've sent him the attached patch to try as a workaround.
Does anyone happen to know under what circumstances the address 127.0.0.1
gets replaced?
And do you know if it will always be replaced with the same
address?
(I'm basically wondering if the workaround needs to be a list of IP addresses
instead of a single address?)
Thanks in advance for any help with this, rick
--- nfsuserd.c.sav 2015-12-09 18:46:29.284972000 -0500
+++ nfsuserd.c 2015-12-09 18:59:33.564498000 -0500
@@ -40,6 +40,10 @@ __FBSDID("$FreeBSD: head/usr.sbin/nfsuse
#include
#include
+#include
+
+#include
+
#include
#include
@@ -94,6 +98,7 @@ gid_t defaultgid = (gid_t)32767;
int verbose = 0, im_a_slave = 0, nfsuserdcnt = -1, forcestart = 0;
int defusertimeout = DEFUSERTIMEOUT, manage_gids = 0;
pid_t slaves[MAXNFSUSERD];
+struct in_addr fromip;
int
main(int argc, char *argv[])
@@ -144,6 +149,7 @@ main(int argc, char *argv[])
}
}
}
+ fromip.s_addr = inet_addr("127.0.0.1");
nid.nid_usermax = DEFUSERMAX;
nid.nid_usertimeout = defusertimeout;
@@ -190,6 +196,15 @@ main(int argc, char *argv[])
usage();
}
nid.nid_usertimeout = defusertimeout = i * 60;
+ } else if (!strcmp(*argv, "-fromip")) {
+ if (argc == 1)
+usage();
+ argc--;
+ argv++;
+ if (inet_aton(*argv, &fromip) == 0) {
+fprintf(stderr, "Bad -fromip %s\n", *argv);
+usage();
+ }
} else if (nfsuserdcnt == -1) {
nfsuserdcnt = atoi(*argv);
if (nfsuserdcnt < 1)
@@ -458,22 +473,22 @@ nfsuserdsrv(struct svc_req *rqstp, SVCXP
u_short sport;
struct info info;
struct nfsd_idargs nid;
- u_int32_t saddr;
gid_t grps[NGROUPS];
int ngroup;
/*
- * Only handle requests from 127.0.0.1 on a reserved port number.
+ * Only handle requests from 127.0.0.1 on a reserved port number,
+ * unless the "-fromip" specified a different address.
* (Since a reserved port # at localhost implies a client with
* local root, there won't be a security breach. This is about
* the only case I can think of where a reserved port # means
* something.)
*/
sport = ntohs(transp->xp_raddr.sin_port);
- saddr = ntohl(transp->xp_raddr.sin_addr.s_addr);
if ((rqstp->rq_proc != NULLPROC && sport >= IPPORT_RESERVED) ||
- saddr != 0x7f01) {
- syslog(LOG_ERR, "req from ip=0x%x port=%d\n", saddr, sport);
+ transp->xp_raddr.sin_addr.s_addr != fromip.s_addr) {
+ syslog(LOG_ERR, "req from ip=%s port=%d\n",
+ inet_ntoa(transp->xp_raddr.sin_addr), sport);
svcerr_weakauth(transp);
return;
}
@@ -721,5 +736,5 @@ usage(void)
{
errx(1,
- "usage: nfsuserd [-usermax cache_size] [-usertimeout minutes] [-verbose] [-manage-gids] [-domain domain_name] [n]");
+ "usage: nfsuserd [-usermax cache_size] [-usertimeout minutes] [-verbose] [-manage-gids] [-domain domain_name] [-fromip xx.xx.xx.xx] [n]");
}
--- nfsuserd.8.sav 2015-12-09 19:13:48.173812000 -0500
+++ nfsuserd.8 2015-12-09 19:19:38.522516000 -0500
@@ -24,7 +24,7 @@
.\"
.\" $FreeBSD: head/usr.sbin/nfsuserd/nfsuserd.8 276258 2014-12-26 21:56:23Z joel $
.\"
-.Dd November 1, 2015
+.Dd December 9, 2015
.Dt NFSUSERD 8
.Os
.Sh NAME
@@ -37,6 +37,7 @@ services plus support manage-gids for al
.Op Fl domain Ar domain_name
.Op Fl usertimeout Ar minutes
.Op Fl usermax Ar max_cache_size
+.Op Fl fromip Ar ip_address
.Op Fl verbose
.Op Fl force
.Op Fl manage-gids
@@ -76,6 +77,21 @@ the more kernel memory is used, but the
system can afford the memory use, make this the sum of the number of
entries in your group and password databases.
The default is 200 entries.
+.It Fl fromip Ar ip_address
+This overrides the default upcall from address of 127.0.0.1.
+Although the upcall connection is done to 127.0.0.1, some network
+configurations will result in another IP address assigned to the machine
+as the from address.
+If you get syslog messages like:
+.sp
+.nf
+Dec 9 19:03:20 nfsv4-laptop nfsuserd:[506]: req from ip=131.104.48.17 port=861
+.fi
+.sp
+then you should use this option to set the from IP address to the one in
+the message.
+Only do this for IP addresses assigned to the machine this daemon is running
+on.
.It Fl verbose
When set, the server logs a bunch of information to syslog.
.It Fl force
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send
Re: RPC request sent to 127.0.0.1 becomes from other IP on machine
I think a local socket is probably the best solution long term. Using a local socket also allows using filesystem permissions to control access which is required for gssd but not necessarily for nfsuserd. On 10 December 2015 at 13:37, Rick Macklem wrote: > Hi, > > Mark has reported a problem via email where the nfsuserd daemon sees > requests coming from an IP# assigned to the machine instead of 127.0.0.1. > Here's a snippet from his message: > Ok, I have Plex in a jail and when I scan the remote NFS file share the > *local* server's nfsuserd spams the logs. > Spamming the logs refers to the messages nfsuserd generates when it gets > a request from an address other than 127.0.0.1. > > I think the best solution is to switch nfsuserd over to using an AF_LOCAL > socket like the gssd uses, but that will take a little coding and probably > won't be MFCable. > > I've sent him the attached patch to try as a workaround. > > Does anyone happen to know under what circumstances the address 127.0.0.1 > gets replaced? > > And do you know if it will always be replaced with the same > address? > (I'm basically wondering if the workaround needs to be a list of IP > addresses > instead of a single address?) > > Thanks in advance for any help with this, rick > > > ___ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RPC request sent to 127.0.0.1 becomes from other IP on machine
On Thu, Dec 10, 2015, at 07:37, Rick Macklem wrote: > Hi, > > Mark has reported a problem via email where the nfsuserd daemon sees > requests coming from an IP# assigned to the machine instead of 127.0.0.1. > Here's a snippet from his message: > Ok, I have Plex in a jail and when I scan the remote NFS file share the > *local* server's nfsuserd spams the logs. > Spamming the logs refers to the messages nfsuserd generates when it gets > a request from an address other than 127.0.0.1. > > I think the best solution is to switch nfsuserd over to using an AF_LOCAL > socket like the gssd uses, but that will take a little coding and > probably > won't be MFCable. > > I've sent him the attached patch to try as a workaround. > > Does anyone happen to know under what circumstances the address 127.0.0.1 > gets replaced? > > And do you know if it will always be replaced with the same > address? > (I'm basically wondering if the workaround needs to be a list of IP > addresses > instead of a single address?) > > Thanks in advance for any help with this, rick > I've opened a PR per your request https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=205193 -- Mark Felder ports-secteam member f...@freebsd.org ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic in arptimer in r289937
Hi,
Here is the backtrace for a reproducable panic seen with arptimer():
#0 doadump (textdump=0) at pcpu.h:221
#1 0x80385afb in db_dump (dummy=, dummy2=false,
dummy3=0, dummy4=0x0) at /usr/img/freebsd/sys/ddb/db_command.c:533
#2 0x803858ee in db_command (cmd_table=0x0) at
/usr/img/freebsd/sys/ddb/db_command.c:440
#3 0x80385684 in db_command_loop () at
/usr/img/freebsd/sys/ddb/db_command.c:493
#4 0x8038818b in db_trap (type=, code=0) at
/usr/img/freebsd/sys/ddb/db_main.c:251
#5 0x80ae0973 in kdb_trap (type=12, code=0, tf=)
at /usr/img/freebsd/sys/kern/subr_kdb.c:654
#6 0x80f276f1 in trap_fatal (frame=0xfe00f59f4950, eva=) at /usr/img/freebsd/sys/amd64/amd64/trap.c:829
#7 0x80f27924 in trap_pfault (frame=0xfe00f59f4950, usermode=) at /usr/img/freebsd/sys/amd64/amd64/trap.c:684
#8 0x80f270de in trap (frame=0xfe00f59f4950) at
/usr/img/freebsd/sys/amd64/amd64/trap.c:435
#9 0x80f0a347 in calltrap () at
/usr/img/freebsd/sys/amd64/amd64/exception.S:234
#10 0x80be9e3d in arptimer (arg=0xf8011d0fda00) at atomic.h:184
#11 0x80ab54f1 in softclock_call_cc (c=0xf8011d0fdaa8,
cc=0x81ccd600, direct=)
at /usr/img/freebsd/sys/kern/kern_timeout.c:832
#12 0x80ab5814 in softclock (arg=0x81ccd600) at
/usr/img/freebsd/sys/kern/kern_timeout.c:921
#13 0x80a5d7f6 in intr_event_execute_handlers (p=,
ie=0xf80003998b00) at /usr/img/freebsd/sys/kern/kern_intr.c:1262
#14 0x80a5de06 in ithread_loop (arg=0xf8000396cde0) at
/usr/img/freebsd/sys/kern/kern_intr.c:1275
#15 0x80a5a87c in fork_exit (callout=0x80a5dd60 ,
arg=0xf8000396cde0, frame=0xfe00f59f4c00)
at /usr/img/freebsd/sys/kern/kern_fork.c:1011
#16 0x80f0a87e in fork_trampoline () at
/usr/img/freebsd/sys/amd64/amd64/exception.S:609
#17 0x in ?? ()
(kgdb) print ((struct llentry *)arg)[0]
$5 = {
lle_next = {
le_next = 0x0,
le_prev = 0xf80069b5fa98
},
r_l3addr = {
addr4 = {
s_addr = 1563742475
},
addr6 = {
__u6_addr = {
__u6_addr8 = 0xf8011d0fda10 "\v�4]",
__u6_addr16 = 0xf8011d0fda10,
__u6_addr32 = 0xf8011d0fda10
}
}
},
ll_addr = {
mac_aligned = 121984137371108,
mac16 = 0xf8011d0fda20,
mac8 = 0xf8011d0fda20 "�\035-��n"
},
r_flags = 1,
r_skip_req = 1,
spare1 = 0,
lle_tbl = 0xf80005653300,
lle_head = 0xf80069b5fa98,
lle_free = 0x80bf2270 ,
la_hold = 0x0,
la_numheld = 0,
la_expire = 12422,
la_flags = 1,
la_asked = 0,
la_preempt = 5,
ln_state = 2,
ln_router = 0,
ln_ntick = 0,
lle_refcnt = 1,
lle_chain = {
le_next = 0x0,
le_prev = 0x0
},
lle_timer = {
c_links = {
le = {
le_next = 0x0,
le_prev = 0x81ccd718
},
sle = {
sle_next = 0x0
},
tqe = {
tqe_next = 0x0,
tqe_prev = 0x81ccd718
}
},
c_time = 53354272998546,
c_precision = 268435437,
c_arg = 0xf8011d0fda00,
c_func = 0x80be9950 ,
c_lock = 0x0,
c_flags = 16,
c_cpu = 0
},
lle_lock = {
lock_object = {
lo_name = 0x8144abf0 "lle",
lo_flags = 90374144,
lo_data = 0,
lo_witness = 0x0
},
rw_lock = 1
},
req_mtx = {
lock_object = {
lo_name = 0x8144abf4 "lle req",
lo_flags = 16973824,
lo_data = 0,
lo_witness = 0x0
},
mtx_lock = 4
}
}
(kgdb) print /x ((struct llentry *)arg)->lle_tbl[0]
$6 = {
llt_link = {
sle_next = 0xf8004be51900
},
llt_af = 0x1d243aa0,
llt_hsize = 0xf801,
lle_head = 0xf800053e5330,
llt_ifp = 0xd04,
llt_lookup = 0x0,
llt_alloc_entry = 0xfe0001a472f0,
llt_delete_entry = 0xf800504a5100,
llt_prefix_free = 0xf800503d8c88,
llt_dump_entry = 0x0,
llt_hash = 0x0,
llt_match_prefix = 0x0,
llt_free_entry = 0x0,
llt_foreach_entry = 0x0,
llt_link_entry = 0x0,
llt_unlink_entry = 0x38e425e,
llt_fill_sa_entry = 0x0,
llt_free_tbl = 0xf8011d243ab0
}
It appears arptimer() was called after lltable_unlink_entry() was
called, because la_flags does not have the LLE_LINKED bit set, which can
happen!! If arptimer() is firing exactly when we call
lltable_unlink_entry(), then arptimer() will refer to freed memory. Does
the following patch make sense?
Index: netinet/if_ether.c
===
--- netinet/if_ether.c (revision 291256)
+++ netinet/if_ether.c (working copy)
@@ -185,7 +185,13 @@
LLE_WUNLOCK(lle);
return;
}
- ifp = lle->lle_tbl->llt_ifp;
+ if (lle->la_flags & LLE_LINKED) {
+ ifp = lle->lle_tbl->llt_ifp;
+ } else {
+ /* XXX RACE entry has been freed */
+ llentry_free(lle);
+
Re: panic in arptimer in r289937
On 12/10/15 16:25, Randall Stewart wrote: For callout_stop/drain you get -1 == Callout as already gone off or is not running (usually the latter) else the caller iks not using locking properly or did not lock and check the active() value (which would have returned not active so no stop would have been needed); 0 == we could not stop the callout, it was in process.. 1 == it was stopped successfully For callout_reset() you get 0 == it was started 1 == it was restarted. There is no -1 since it is either started or restarted never left in a not-running state... Hi, Correct, though if the return values would be the same, it might simplify the callout API and manual page a bit: /* return values for all callout_xxx() functions */ #define CALLOUT_RET_DRAINING0 /* callout cannot be stopped, need drain */ #define CALLOUT_RET_CANCELLED 1 /* callout was successfully stopped */ #define CALLOUT_RET_STOPPED -1 /* callout was already stopped */ Then callout_reset() would return -1, if it was started from the stopped state. --HPS ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic in arptimer in r289937
On 12/10/15 16:35, Randall Stewart wrote: If you did that it would change the KPI a bit meaning lots of thrashing in the code. Hi, There are only 5 consumers of the callout_reset() return code in the FreeBSD 11-current kernel from what I can see: grep -r "= callout_reset" sys/ | wc -l 5 Two of these are already using > 0 checks. --HPS ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic in arptimer in r289937
Hans: Though it would not hurt to add your patch, its not possible for callout_reset() to return anything but 1 or 0. Only callout_stop(), callout_drain(), callout_async_drain() can return -1. So I don’t think that this will fix it. R On Dec 4, 2015, at 11:34 AM, Hans Petter Selasky wrote: . Randall Stewart r...@netflix.com 803-317-4952 ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic in arptimer in r289937
For callout_stop/drain you get -1 == Callout as already gone off or is not running (usually the latter) else the caller iks not using locking properly or did not lock and check the active() value (which would have returned not active so no stop would have been needed); 0 == we could not stop the callout, it was in process.. 1 == it was stopped successfully For callout_reset() you get 0 == it was started 1 == it was restarted. There is no -1 since it is either started or restarted never left in a not-running state... R On Dec 4, 2015, at 11:52 AM, Hans Petter Selasky wrote: > On 12/04/15 20:34, Hans Petter Selasky wrote: >> Hi Adrian, >> >> On 10/31/15 16:01, Alexander V. Chernikov wrote: >>> >>> >>> 31.10.2015, 16:46, "Adrian Chadd" : On 31 October 2015 at 09:34, Alexander V. Chernikov wrote: > 31.10.2015, 05:32, "Adrian Chadd" : >> Hiya, >> >> Here's a panic from arptimer: > Hi Adrian, > > As far as I see, line 205 in if_ether.c is IF_AFDATA_LOCK(ifp) > which happens after LLE_WUNLOCK(). > So, it looks like (pre-cached) ifp had been freed before locking > ifdata. > Do you have any more details on that? (e.g. was some interface > detached at that moment, is it reproducible, etc..) > > From a quick glance, potential use-after-free has been possible > for quite a long time, but I wonder why it hasn't been observed before. > Probably lltable_free() changes might have triggered that. > > I'll take a deeper look on that and reply. >> >> Observed on an idle box with projects/hps_head too: >> >>> panic: bogus refcnt 0 on lle 0xf8016508ca00 >>> cpuid = 7 >>> KDB: stack backtrace: >>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >>> 0xfe03e4e8c7e0 >>> vpanic() at vpanic+0x182/frame 0xfe03e4e8c860 >>> kassert_panic() at kassert_panic+0x126/frame 0xfe03e4e8c8d0 >>> llentry_free() at llentry_free+0x136/frame 0xfe03e4e8c900 >>> arptimer() at arptimer+0x20e/frame 0xfe03e4e8c950 >>> softclock_call_cc() at softclock_call_cc+0x170/frame 0xfe03e4e8c9c0 >>> softclock() at softclock+0x47/frame 0xfe03e4e8c9e0 >>> intr_event_execute_handlers() at >>> intr_event_execute_handlers+0x96/frame 0xfe03e4e8ca20 >>> ithread_loop() at ithread_loop+0xa6/frame 0xfe03e4e8ca70 >>> fork_exit() at fork_exit+0x84/frame 0xfe03e4e8cab0 >>> fork_trampoline() at fork_trampoline+0xe/frame 0xfe03e4e8cab0 >>> --- trap 0, rip = 0, rsp = 0, rbp = 0 --- >> >> Looks like callout_reset() must be examined too, and was missed by: >> >> https://svnweb.freebsd.org/changeset/base/290805 >> >> Can you try the attached patch? >> >> Randall: Can you fix this ASAP? >> >> --HPS >> > > Hi, > > Randall: > > I see for 11-current, callout_reset() doesn't return -1 when the callout is > stopped like with callout_stop(). Is this a bug or a feature? Why can't the > callout_reset() and callout_stop() functions use the same return values? > > In nd6_llinfo_settimer_locked() the return value of both callout_reset() and > callout_stop() is checked for positive values, but not in the other places > mentioned by my patch. > > --HPS Randall Stewart r...@netflix.com 803-317-4952 ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic in arptimer in r289937
If you did that it would change the KPI a bit meaning lots of thrashing in the code. And on top of that you now would no longer return 0.. You would get 1 it was restarted or -1 it was not running but is now started. Makes no sense to me sorry.. R On Dec 10, 2015, at 7:35 AM, Hans Petter Selasky wrote: > On 12/10/15 16:25, Randall Stewart wrote: >> For callout_stop/drain you get >> >> -1 == Callout as already gone off or is not running (usually the latter) >> else the caller iks >> not using locking properly or did not lock and check the active() >> value (which would have returned not active so no stop >> would have been needed); >> 0 == we could not stop the callout, it was in process.. >> 1 == it was stopped successfully >> >> >> For callout_reset() you get >> >> 0 == it was started >> 1 == it was restarted. >> >> There is no -1 since it is either started or restarted never left in a >> not-running state... >> > > Hi, > > Correct, though if the return values would be the same, it might simplify the > callout API and manual page a bit: > > /* return values for all callout_xxx() functions */ > #define CALLOUT_RET_DRAINING0 /* callout cannot be stopped, need drain */ > #define CALLOUT_RET_CANCELLED 1 /* callout was successfully stopped */ > #define CALLOUT_RET_STOPPED -1 /* callout was already stopped */ > > Then callout_reset() would return -1, if it was started from the stopped > state. > > --HPS Randall Stewart r...@netflix.com 803-317-4952 ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
My build work and goals
This mail is to outline my recent work, goals and motivations. This is long. This is not really a architectural review or discussion mail. I am leaving those and details for their own threads where needed, so please do not start discussions about any of the details here. We will have them later. No one has really objected to my work but many have asked what my goals are and if the churn is worth it. A lot of my work lately has been to remove hacks in the build and use normal mechanisms or framework to achieve the same thing. Some of this is to remove hacks from the META build. Some of this is just natural evolution of the framework since we haven't had a real maintainer for so long. That may be putting it wrong but I think it is fair to say we're all a little scared of touching share/mk and try to do so as little as possible. Consider bsd.files.mk/bsd.incs.mk/bsd.confs.mk(new) are all 95% the same. I have not yet combined them but plan to. Why is SCRIPTS only in bsd.prog.mk? There's many problems in these files that need fixing. The vast majority of my work and "churn" lately has been improving the META mode build which is not the default build. So it may appear that I am churning the build a lot when I am really not. More on the META build later. First an introductory and background. Who am I? I came to the FreeBSD community as a Ports committer who was primarily focused on building up the Ports framework and its build tools. I tookover upstream maintenance of Portupgrade and then got involved with Poudriere in its early stages and helped bring it to what it is today. I think it is fair to say I have been the maintainer of Poudriere for some time now. I am one of the maintainers of the pkg.FreeBSD.org package builds and oversee its automation. I've written a few FreeBSD Journal articles about that. 99% of this was not sponsored though. For my day job at Isilon I recently moved my efforts to the base build. In many people's eyes the build system, 'buildworld', mostly "just works". The problems come in when "works" to you is time and productivity. At Isilon we have a range of development efforts that span from developers only caring about the kernel to ones who care about 1 userland tool. All of us expect that we should be able to just build our 1 thing rather than everything. Some of these 1 userland tool cases though have hundreds of dependencies. Most developers instinctively try building manually rather than Jenkins as it feels like it should be quicker. This leads to grief. The problem comes down to productivity. I've been given a great opportunity to address these problems and am running with it. Isilon has a quite convoluted build. Our product has its own ELF brand/ABI/KBI and cannot run on native FreeBSD systems. The build is done from a FreeBSD system for reasons, so is entirely a cross-build by expectations. We have a buildworld, ports phase, and then we have a buildworld-type thing of stuff that depends on ports! Both the ports and ports-dependent pieces are built in a jail using a special hack of a kernel module that provides KBI compatibility to the native FreeBSD system so we can run target binaries. QEMU does not apply here as it's not an architecture problem, it's a syscall/KBI problem. Solving ports cross-building will remove the need for this. Stepping back to pure FreeBSD now. Some various problems I've observed in the build: - Building clang twice. We build clang from the source tree so we can build everything with it even if /usr/bin/cc qualifies as "new enough" and "capable of cross-building the target". We later build clang for the target as well. Try doing this for 'make universe' for N architectures and you build clang N*2 times rather than N times. - Not being highly parallel. - Requiring building everything to build anything (without being an expert on manually building dependencies). AKA, no reliable sub-directory builds. - Incremental builds don't work reliably. We have stealth dependencies that are not tracked such as csu, libgcc, compiler_rt, compiler and Makefile (CFLAGS/LDFLAGS,build command). - There is gross under-linkage in libraries and over-linkage in binaries. - No foreign OS cross-build support. You must build from FreeBSD 8.0+. This is a problem when people prefer to run OSX or Linux for their primary system. - No cross-build support in Ports. - share/mk and Makefile.inc1 has a ton of bitrot. Some various problems I've observed with maintaining the build: - Adding new libraries into the build usually results in doing it wrong in Makefile.inc1's handling of 'make libraries'. I think it is fair to say that most people don't understand how any of this works. Just yesterday I discovered more of how it works that surprised me. - Adding new libraries into the build is usually done wrong in terms of the new framework. - There are little build framework sanity checks. The ports build has grown a large array of sanity checks o
Re: My build work and goals
Firstly, nice writup - a few extra blank lines might have helped my eyes
though.
Bryan Drewery wrote:
> This mail is to outline my recent work, goals and motivations. This is
> long. This is not really a architectural review or discussion mail. I
...
> Some problems with the combined DIRDEPS_BUILD build:
>
> It is very hard to maintain, prone to massive churn in directories you
> did and did not touch, requires checking in fickle generated files, has
> a chicken-and-egg problem for adding new things, does not respect
> WITH/WITHOUT options or ARCH-dependent SUBDIR, has subtle problems with
> a static dependencies list around options, and has no 'installworld'
> support.
Dealing with all the optional cases is definitely a problem to be
solved.
I've a few ideas on that front, but haven't had time to experiment.
FWIW we (Juniper) see zero "churn" in our dependencies
a/ we use a fixed set of options
b/ we've fixed the bugs in makefiles that were causing churn.
Ignoring the WITH/WITHOUT options stuff for a sec, dependency churn is
usually a result of bugs in the leaf makefiles.
These are often easy to spot, sometimes quite hard.
The bootstrap issue and even just the need to checkin "generated" files
is a head scratcher for a surprising number of people - even after 4+
years of building this way.
> The lack of 'installworld' is a deal breaker, albeit probably trivial to
> fix. The reason for this is that the system is used to generate
> packages (not pkgng style) at Juniper. We have our own packaging effort
> occurring in projects/release-pkg though and won't likely use the same
> methods that the DIRDEPS_BUILD does (of which none of the support is
> really in the src tree).
Though AFAIK all the tools we use are, so its really just a matter of
adding some suitable makefiles and manifest - to say build a bootable vm
image - I probably would have done that by now if bhyve supported vmdk
or qcow2 ;-)
But that's likely orthogonal to your point.
Some form of "installworld" is probably needed.
> This build does not use recursive SUBDIR walking (by design). It uses a
> static list of directories to build in the
> targets/pseudo/userland/*/Makefile.depend files.
That's mostly just for the purpose of demonstration/example.
If/when FreeBSD has some form of manifest/list for what goes into its
"packages" (of whatever form they might be,) then the necessary data
to drive what needs to be built could be obtained from them - as we do
in the Junos build.
> The checked-in Makefile.depend files have a flaw (feature for static
> builds perhaps) that the invoking the linker will cause non-direct
> dependencies to show up in Makefile.depend. This leads to situations
> such as https://svnweb.freebsd.org/changeset/base/291558. There is also
> a situation where a local build may set MK_SSP=no and yet need to build
> a dependency that wants MK_SSP=yes (libssp dependency) but because of
Yes, options are a pain.
You can either hard code them via local.dirdeps.mk but it is probably
better to do something a bit like what the current intermediate
makefiles do (one of the idea I mentioned above).
Eg. if the captured dependencies had something like ${lib_libssp} rather than
lib/libssp. If MK_SSP=no it expands to nothing.
I can elaborate in separate thread if you prefer.
> than upstream, is an exhaustive chicken-and-egg problem. These problems
> make checked-in Makefile.depend infeasible. "But wait, can you just not
> check them in?" Not really.
You can, but you lose virtually all the benefits.
> recently added support to local.dirdeps.mk to make an educated guess on
> what DIRDEPS should be even if there is no Makefile.depend. This is
> based on what is being built (C and C++ objects have common
> dependencies) and what DPADD/LIBADD there are. In most cases this is
> enough. It has greatly helped when bootstrapping Makefile.depend.
You still have the issue of local depenenencies.
Sure, only a small percentage of makefiles have problems in this regard,
but humans are generally bad at expressing the dependencies that result
from using things like yacc - get it wrong about 80% of the time.
> Going further though I have written a script that just recursively walks
> the tree to generate the DIRDEPS graph from the first make process
> rather than including the Makefile.depend files. This removes the need
That's what we used to do, but was deemed too expensive.
Using that to generate Makefile.depend though for your tree, could save
you some overhead on subsequent builds, while allowing tailorying tree
to your set of options.
> P.S. Working on this stuff can be exhausting. Mistakes are easy.
Thanks for your efforts btw.
--sjg
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RPC request sent to 127.0.0.1 becomes from other IP on machine
On Thu, 10 Dec 2015, Rick Macklem wrote:
> Hi,
>
> Mark has reported a problem via email where the nfsuserd daemon sees
> requests coming from an IP# assigned to the machine instead of 127.0.0.1.
> Here's a snippet from his message:
> Ok, I have Plex in a jail and when I scan the remote NFS file share the
> *local* server's nfsuserd spams the logs.
> Spamming the logs refers to the messages nfsuserd generates when it gets
> a request from an address other than 127.0.0.1.
>
> I think the best solution is to switch nfsuserd over to using an AF_LOCAL
> socket like the gssd uses, but that will take a little coding and probably
> won't be MFCable.
>
> I've sent him the attached patch to try as a workaround.
>
> Does anyone happen to know under what circumstances the address 127.0.0.1
> gets replaced?
My memory is quite hazy on this subject, but I think that outbound traffic
from a jail is not permitted to use the system loopback address 127.0.0.1;
traffic from this address within a jail gets replace with the jail's
primary IP address. It is possible to specify an alternate loopback
address for use within the jail (e.g., 127.0.0.2) and if that alternate
address is only bound within the jail, it can be used for outgoing traffic
to the host. See jail.conf(5); I appear to have something like:
kduck {
host.hostname = "kduck.mit.edu";
ip4.addr = lo0|127.0.0.2, 18.18.0.52;
[...]
}
Note that there may be some additional magic about the primary address of
the jail being first (or last?) in the list of addresses.
-Ben
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
FreeBSD_HEAD_i386 - Build #1858 - Fixed
FreeBSD_HEAD_i386 - Build #1858 - Fixed: Build information: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_i386/1858/ Full change log: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_i386/1858/changes Full build log: https://jenkins.FreeBSD.org/job/FreeBSD_HEAD_i386/1858/console Change summaries: 292067 by cem: vm_page_replace: remove redundant radix lookup Remove redundant lookup of the old page from vm_page_replace. Verification that the old page exists is already done by vm_radix_replace. Submitted by: Ryan Libby Reviewed by:alc, kib Sponsored by: EMC / Isilon Storage Division Follow-up to: https://reviews.freebsd.org/D4326 Differential Revision: https://reviews.freebsd.org/D4471 292066 by asomers: During vdev_geom_open, require that the vdev guids match the device's label except during split, add, or create operations. This fixes a bug where the wrong disk could be returned, and higher layers of ZFS would immediately eject it again. sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c: o When opening by GUID, require both the pool and vdev GUIDs to match. While it is highly unlikely for two vdevs to have the same vdev GUIDs, the ZFS storage pool allocator only guarantees they are unique within a pool. o Modify the open behavior to: - If we are opening a vdev that hasn't previously been opened, open by path without checking GUIDs. - Otherwise, open by path and verify GUIDs. - If that fails, search all geom providers for a device with matching GUIDs. - If that fails, return ENOENT. Submitted by: gibbs, asomers Reviewed by:smh MFC after: 4 weeks Sponsored by: Spectra Logic Corp Differential Revision: https://reviews.freebsd.org/D4486 ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: RPC request sent to 127.0.0.1 becomes from other IP on machine
Ok, I had a hunch it was related to the use of jails.
I am just testing a patch that switches the nfsuserd over to
using an af_local socket, so this will be avoided.
(I think it makes more sense anyhow. I just never got around
to doing it.;-)
Thanks for the info, rick
- Original Message -
> On Thu, 10 Dec 2015, Rick Macklem wrote:
>
> > Hi,
> >
> > Mark has reported a problem via email where the nfsuserd daemon sees
> > requests coming from an IP# assigned to the machine instead of 127.0.0.1.
> > Here's a snippet from his message:
> > Ok, I have Plex in a jail and when I scan the remote NFS file share the
> > *local* server's nfsuserd spams the logs.
> > Spamming the logs refers to the messages nfsuserd generates when it gets
> > a request from an address other than 127.0.0.1.
> >
> > I think the best solution is to switch nfsuserd over to using an AF_LOCAL
> > socket like the gssd uses, but that will take a little coding and probably
> > won't be MFCable.
> >
> > I've sent him the attached patch to try as a workaround.
> >
> > Does anyone happen to know under what circumstances the address 127.0.0.1
> > gets replaced?
>
> My memory is quite hazy on this subject, but I think that outbound traffic
> from a jail is not permitted to use the system loopback address 127.0.0.1;
> traffic from this address within a jail gets replace with the jail's
> primary IP address. It is possible to specify an alternate loopback
> address for use within the jail (e.g., 127.0.0.2) and if that alternate
> address is only bound within the jail, it can be used for outgoing traffic
> to the host. See jail.conf(5); I appear to have something like:
>
> kduck {
> host.hostname = "kduck.mit.edu";
> ip4.addr = lo0|127.0.0.2, 18.18.0.52;
> [...]
> }
>
> Note that there may be some additional magic about the primary address of
> the jail being first (or last?) in the list of addresses.
>
> -Ben
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: My build work and goals
On 11/12/2015 3:56 AM, Bryan Drewery wrote: I think it is fair to say that most people don't understand how any of this works. [...] - No one really was trying to improve it head-on and focused on FreeBSD's general audience needs. The maintenance problems come down to expertise. Most developers are not experts in the build and don't have time or interest to become so. These three points are all pretty much the same thing. I agree that it definitely needs to have a fairy godfather. Some improvements I have made recently: - WITH_FAST_DEPEND: Replacing the antiquated 'make depend'/'mkdep' with compiler dependency flags to generate the .depend files as a side-effect of compiling. This saves tremendous time in buildworld and buildkernel. Mach used to have this through their entire (bsd 4.3) tree. I forget how they bootstrapped it. Gcc would generate added dep files which were included into the static files. Without the dep files, every thing was assumed to be dirty. The new dep files only kicked in on the second build. It worked very very well. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
