Re: ps_strings

2013-08-19 Thread Chris Torek
Yes, p_args caches the arguments, but not always.  Right now, kernel
does not cache arguments if the string is longer than 256 bytes.  Look
for ps_arg_cache_limit in kern_exec.c.

setproctitle() always informs the kernel with sysctl and sets the
pointers in ps_strings. kern.proc.args sysctl first tries the p_args,
and falls back to reading ps_strings and following the pointers if
p_args is NULL.

Ah, that's what I get for scanning through years of updates too fast.
:-)

This seems a bit of a worst of both worlds: there's now some
extra kernel code for poking through the ps_strings and the
pointer-vectors (this code is no longer in libkvm at all --
that was where I looked first and found the sysctl), for the no
p_args case.  It seems like perhaps there could just be a sysctl
to return the ps_strings address, and leave the follow argv
pointers code in libkvm, if there is to be code for that.

(The kernel saves a bit of time for the presumably-usual p_args
not NULL case, and finding the location of the ps_strings
structure when it *is* used is automatically correct.  So that
part is a straight-up improvement, at least.)

Not that big a deal either way, but it does seem as though there
should be documentation for ps_strings.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: ps_strings

2013-08-18 Thread Chris Torek
Despite I made a request not long ago[1], I'm looking for
documentation to create the ps_strings structure man page because
isn't covered in other man page such e.g.  execve(2). So, I'm
interested to know for what it's currently used.

Nothing.  (Well, backwards compatibility, depending on how far
backwards you go.)

I invented the ps_strings struct a long time ago, about the same
time libkvm was new.

Some background: There was code in ps that would grub around in
the top stack page of each user process and extract the argv
strings.  This code knew how execve() worked inside the kernel
(copying the argv and environment strings into the user stack,
just below the signal trampoline code, and then setting up argv
and envp pointers and invoking the libc/csu start program code
at the entry point).

We moved this grub-around-in-process-stack code to libkvm, but it
was still rather horrible code.

Meanwhile, we had programs like sendmail that would set their
process title by saving, in some secret global variable, the
space where the argv array itself and its strings lived, and
then -- in setproctitle() -- carefully overwrite it.  Of course
there was only as much room there as the kernel provided, based on
the actual strings at execve() time.

I figured this would all be much cleaner if we created a small
data structure, namely ps_strings, to hold the information that
libkvm would extract (and hence, ps would show).  It would be
simpler than the original code, because the ps_strings structure
lived at a fixed, well known user-space virtual address (the same
VA in every process).  Moreover, a user program (like sendmail)
could modify the ps_strings data to point to any other user-space
area: libkvm was smart enough to extract arbitrary data (and
verify the validity of the address too).  This removed the limit
on how large a process title could be.

FreeBSD now, however, uses a per-process p_args field in the
proc structure, with sysctl()s to set and get p_args.  (I had
nothing to do with this new code, but I approve, as if anyone
cares. :-) )  This removes the fixed-virtual-address limitation.
The cost is a bit more kernel code (for the sysctl()s) and this
per-process data, but there is no more messing-about with where
is ps_strings in this memory-layout / emulation etc.  (Meanwhile
libkvm still retrieves the arguments.  It just does it now with
sysctl().)

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: expanding amd64 past the 1TB limit

2013-07-15 Thread Chris Torek
(Durn mailing list software, eating attachments... there are just
the two so I will just send them one at a time here.  I took the
individual people off the to/cc since presumably you all got the
attachments already.)

Date: Sun, 14 Jul 2013 19:39:51 -0600
Subject: [PATCH 1/2] create_pagetables: cosmetics

Using local variables with the appropriate types,
eliminate a bunch of casts and shorten the code a bit.
---
 amd64/amd64/pmap.c | 62 +++---
 1 file changed, 31 insertions(+), 31 deletions(-)

diff --git a/amd64/amd64/pmap.c b/amd64/amd64/pmap.c
index 8dcf232..46f6940 100644
--- a/amd64/amd64/pmap.c
+++ b/amd64/amd64/pmap.c
@@ -531,6 +531,10 @@ static void
 create_pagetables(vm_paddr_t *firstaddr)
 {
int i, j, ndm1g, nkpdpe;
+   pt_entry_t *pt_p;
+   pd_entry_t *pd_p;
+   pdp_entry_t *pdp_p;
+   pml4_entry_t *p4_p;
 
/* Allocate page table pages for the direct map */
ndmpdp = (ptoa(Maxmem) + NBPDP - 1)  PDPSHIFT;
@@ -561,32 +565,26 @@ create_pagetables(vm_paddr_t *firstaddr)
KPDphys = allocpages(firstaddr, nkpdpe);
 
/* Fill in the underlying page table pages */
-   /* Read-only from zero to physfree */
+   /* Nominally read-only (but really R/W) from zero to physfree */
/* XXX not fully used, underneath 2M pages */
-   for (i = 0; (i  PAGE_SHIFT)  *firstaddr; i++) {
-   ((pt_entry_t *)KPTphys)[i] = i  PAGE_SHIFT;
-   ((pt_entry_t *)KPTphys)[i] |= PG_RW | PG_V | PG_G;
-   }
+   pt_p = (pt_entry_t *)KPTphys;
+   for (i = 0; ptoa(i)  *firstaddr; i++)
+   pt_p[i] = ptoa(i) | PG_RW | PG_V | PG_G;
 
/* Now map the page tables at their location within PTmap */
-   for (i = 0; i  nkpt; i++) {
-   ((pd_entry_t *)KPDphys)[i] = KPTphys + (i  PAGE_SHIFT);
-   ((pd_entry_t *)KPDphys)[i] |= PG_RW | PG_V;
-   }
+   pd_p = (pd_entry_t *)KPDphys;
+   for (i = 0; i  nkpt; i++)
+   pd_p[i] = (KPTphys + ptoa(i)) | PG_RW | PG_V;
 
/* Map from zero to end of allocations under 2M pages */
/* This replaces some of the KPTphys entries above */
-   for (i = 0; (i  PDRSHIFT)  *firstaddr; i++) {
-   ((pd_entry_t *)KPDphys)[i] = i  PDRSHIFT;
-   ((pd_entry_t *)KPDphys)[i] |= PG_RW | PG_V | PG_PS | PG_G;
-   }
+   for (i = 0; (i  PDRSHIFT)  *firstaddr; i++)
+   pd_p[i] = (i  PDRSHIFT) | PG_RW | PG_V | PG_PS | PG_G;
 
/* And connect up the PD to the PDP */
-   for (i = 0; i  nkpdpe; i++) {
-   ((pdp_entry_t *)KPDPphys)[i + KPDPI] = KPDphys +
-   (i  PAGE_SHIFT);
-   ((pdp_entry_t *)KPDPphys)[i + KPDPI] |= PG_RW | PG_V | PG_U;
-   }
+   pdp_p = (pdp_entry_t *)KPDPphys;
+   for (i = 0; i  nkpdpe; i++)
+   pdp_p[i + KPDPI] = (KPDphys + ptoa(i)) | PG_RW | PG_V | PG_U;
 
/*
 * Now, set up the direct map region using 2MB and/or 1GB pages.  If
@@ -596,37 +594,39 @@ create_pagetables(vm_paddr_t *firstaddr)
 * memory, pmap_change_attr() will demote any 2MB or 1GB page mappings
 * that are partially used. 
 */
+   pd_p = (pd_entry_t *)DMPDphys;
for (i = NPDEPG * ndm1g, j = 0; i  NPDEPG * ndmpdp; i++, j++) {
-   ((pd_entry_t *)DMPDphys)[j] = (vm_paddr_t)i  PDRSHIFT;
+   pd_p[j] = (vm_paddr_t)i  PDRSHIFT;
/* Preset PG_M and PG_A because demotion expects it. */
-   ((pd_entry_t *)DMPDphys)[j] |= PG_RW | PG_V | PG_PS | PG_G |
+   pd_p[j] |= PG_RW | PG_V | PG_PS | PG_G |
PG_M | PG_A;
}
+   pdp_p = (pdp_entry_t *)DMPDPphys;
for (i = 0; i  ndm1g; i++) {
-   ((pdp_entry_t *)DMPDPphys)[i] = (vm_paddr_t)i  PDPSHIFT;
+   pdp_p[i] = (vm_paddr_t)i  PDPSHIFT;
/* Preset PG_M and PG_A because demotion expects it. */
-   ((pdp_entry_t *)DMPDPphys)[i] |= PG_RW | PG_V | PG_PS | PG_G |
+   pdp_p[i] |= PG_RW | PG_V | PG_PS | PG_G |
PG_M | PG_A;
}
for (j = 0; i  ndmpdp; i++, j++) {
-   ((pdp_entry_t *)DMPDPphys)[i] = DMPDphys + (j  PAGE_SHIFT);
-   ((pdp_entry_t *)DMPDPphys)[i] |= PG_RW | PG_V | PG_U;
+   pdp_p[i] = DMPDphys + ptoa(j);
+   pdp_p[i] |= PG_RW | PG_V | PG_U;
}
 
/* And recursively map PML4 to itself in order to get PTmap */
-   ((pdp_entry_t *)KPML4phys)[PML4PML4I] = KPML4phys;
-   ((pdp_entry_t *)KPML4phys)[PML4PML4I] |= PG_RW | PG_V | PG_U;
+   p4_p = (pml4_entry_t *)KPML4phys;
+   p4_p[PML4PML4I] = KPML4phys;
+   p4_p[PML4PML4I] |= PG_RW | PG_V | PG_U;
 
/* Connect the Direct Map slot(s) up to the PML4. */
for (i = 0; i  NDMPML4E; i++) {
-   ((pdp_entry_t *)KPML4phys)[DMPML4I + i] = DMPDPphys +
-   (i  

Re: expanding amd64 past the 1TB limit

2013-07-15 Thread Chris Torek
(Durn mailing list software, eating attachments... there are just
the two so I will just send them one at a time here.  I took the
individual people off the to/cc since presumably you all got the 
attachments already.)

Date: Thu, 27 Jun 2013 18:49:29 -0600
Subject: [PATCH 2/2] increase physical and virtual memory limits

Increase kernel VM space: go from .5 TB of KVA and 1 TB of direct
map, to 8 TB of KVA and 16 TB of direct map.  However, we allocate
less direct map space for small physical-memory systems.  Also, if
Maxmem is so large that there is not enough direct map space,
reduce Maxmem to fit, so that the system can boot unassisted.
---
 amd64/amd64/pmap.c  | 44 +---
 amd64/include/pmap.h| 36 +---
 amd64/include/vmparam.h | 13 +++--
 3 files changed, 69 insertions(+), 24 deletions(-)

diff --git a/amd64/amd64/pmap.c b/amd64/amd64/pmap.c
index 46f6940..5e43c93 100644
--- a/amd64/amd64/pmap.c
+++ b/amd64/amd64/pmap.c
@@ -232,6 +232,7 @@ u_int64_t   KPML4phys;  /* phys addr of kernel 
level 4 */
 
 static u_int64_t   DMPDphys;   /* phys addr of direct mapped level 2 */
 static u_int64_t   DMPDPphys;  /* phys addr of direct mapped level 3 */
+static int ndmpdpphys; /* number of DMPDPphys pages */
 
 static struct rwlock_padalign pvh_global_lock;
 
@@ -540,7 +541,18 @@ create_pagetables(vm_paddr_t *firstaddr)
ndmpdp = (ptoa(Maxmem) + NBPDP - 1)  PDPSHIFT;
if (ndmpdp  4) /* Minimum 4GB of dirmap */
ndmpdp = 4;
-   DMPDPphys = allocpages(firstaddr, NDMPML4E);
+   ndmpdpphys = howmany(ndmpdp, NPDPEPG);
+   if (ndmpdpphys  NDMPML4E) {
+   /*
+* Each NDMPML4E allows 512 GB, so limit to that,
+* and then readjust ndmpdp and ndmpdpphys.
+*/
+   printf(NDMPML4E limits system to %d GB\n, NDMPML4E * 512);
+   Maxmem = atop(NDMPML4E * NBPML4);
+   ndmpdpphys = NDMPML4E;
+   ndmpdp = NDMPML4E * NPDEPG;
+   }
+   DMPDPphys = allocpages(firstaddr, ndmpdpphys);
ndm1g = 0;
if ((amd_feature  AMDID_PAGE1GB) != 0)
ndm1g = ptoa(Maxmem)  PDPSHIFT;
@@ -557,6 +569,10 @@ create_pagetables(vm_paddr_t *firstaddr)
 * bootstrap.  We defer this until after all memory-size dependent
 * allocations are done (e.g. direct map), so that we don't have to
 * build in too much slop in our estimate.
+*
+* Note that when NKPML4E  1, we have an empty page underneath
+* all but the KPML4I'th one, so we need NKPML4E-1 extra (zeroed)
+* pages.  (pmap_enter requires a PD page to exist for each KPML4E.)
 */
nkpt_init(*firstaddr);
nkpdpe = NKPDPE(nkpt);
@@ -581,8 +597,8 @@ create_pagetables(vm_paddr_t *firstaddr)
for (i = 0; (i  PDRSHIFT)  *firstaddr; i++)
pd_p[i] = (i  PDRSHIFT) | PG_RW | PG_V | PG_PS | PG_G;
 
-   /* And connect up the PD to the PDP */
-   pdp_p = (pdp_entry_t *)KPDPphys;
+   /* And connect up the PD to the PDP (leaving room for L4 pages) */
+   pdp_p = (pdp_entry_t *)(KPDPphys + ptoa(KPML4I - KPML4BASE));
for (i = 0; i  nkpdpe; i++)
pdp_p[i + KPDPI] = (KPDphys + ptoa(i)) | PG_RW | PG_V | PG_U;
 
@@ -619,14 +635,16 @@ create_pagetables(vm_paddr_t *firstaddr)
p4_p[PML4PML4I] |= PG_RW | PG_V | PG_U;
 
/* Connect the Direct Map slot(s) up to the PML4. */
-   for (i = 0; i  NDMPML4E; i++) {
+   for (i = 0; i  ndmpdpphys; i++) {
p4_p[DMPML4I + i] = DMPDPphys + ptoa(i);
p4_p[DMPML4I + i] |= PG_RW | PG_V | PG_U;
}
 
-   /* Connect the KVA slot up to the PML4 */
-   p4_p[KPML4I] = KPDPphys;
-   p4_p[KPML4I] |= PG_RW | PG_V | PG_U;
+   /* Connect the KVA slots up to the PML4 */
+   for (i = 0; i  NKPML4E; i++) {
+   p4_p[KPML4BASE + i] = KPDPphys + ptoa(i);
+   p4_p[KPML4BASE + i] |= PG_RW | PG_V | PG_U;
+   }
 }
 
 /*
@@ -1685,8 +1703,11 @@ pmap_pinit(pmap_t pmap)
pagezero(pmap-pm_pml4);
 
/* Wire in kernel global address entries. */
-   pmap-pm_pml4[KPML4I] = KPDPphys | PG_RW | PG_V | PG_U;
-   for (i = 0; i  NDMPML4E; i++) {
+   for (i = 0; i  NKPML4E; i++) {
+   pmap-pm_pml4[KPML4BASE + i] = (KPDPphys + (i  PAGE_SHIFT)) |
+   PG_RW | PG_V | PG_U;
+   }
+   for (i = 0; i  ndmpdpphys; i++) {
pmap-pm_pml4[DMPML4I + i] = (DMPDPphys + (i  PAGE_SHIFT)) |
PG_RW | PG_V | PG_U;
}
@@ -1941,8 +1962,9 @@ pmap_release(pmap_t pmap)
 
m = PHYS_TO_VM_PAGE(pmap-pm_pml4[PML4PML4I]  PG_FRAME);
 
-   pmap-pm_pml4[KPML4I] = 0;  /* KVA */
-   for (i = 0; i  NDMPML4E; i++)  /* Direct Map */
+   for (i = 0; i  NKPML4E; i++)   /* KVA */
+

am I abusing the UMA allocator?

2013-07-15 Thread Chris Torek
I have been experimenting with using the UMA (slab) allocator for
special-purpose physical address ranges.  (The underlying issue is
that we need zone-like and/or mbuf-like data structures to talk to
hardware that has special needs in terms of which physical pages
it can in turn use.  Each device has a limited memory window it
can access.)

For my purposes it's nice that the allocation function receives a
zone argument, even though the comment in the call says zone is
passed for legacy reasons.  However, the free function does not
get the zone argument, or anything other than a single bit -- up
to 4 if you cheat harder.  This is ... less convenient (although
in my case I can use the VA being free'd, instead).

What I'm wondering is what this single bit is really for; whether
the allocation and free might be made more flexible for special-
purpose back-end allocators; and whether this is really using
things as intended.

Details:

In the allocator, there's a per-keg uk_allocf and uk_freef
(allocation and free function) pointer, and you can set your
own allocation and free functions for any zone with:

void uma_zone_set_allocf(uma_zone_t zone, uma_alloc allocf);
void uma_zone_set_freef(uma_zone_t zone, uma_free freef);

(Aside: it seems a bit weird that you set these per *zone*
but they're stored in the *kegs*, specifically the special
first keg, but never mind... :-) )

Each allocf is called as:

/* arguments: uma_zone_t zone, int size, uint8_t *pflag, int wait */
mem = allocf(zone, nbytes, flags, wait);

where wait is made up of malloc flags (M_WAITOK, M_NOWAIT,
M_ZERO, M_USE_RESERVE).  The flags argument is not initialized
at this point, so the allocation function must fill it in.  The
filled-in value is stored in the per-slab us_flags and eventually
passed back to each freef function:

/* arguments: void *mem, int size, uint8_t flag */
freef(mem, nbytes, pflag); /* where pflag = us-us_flags */

The flags are defined in sys/vm/uma.h and are the UMA_SLAB_* flags
(BOOT, KMEM, KERNEL, PRIV, OFFP, MALLOC).  UMA_SLAB_PRIV is
described as private.  The bit is never tested though, so it
seems that a private allocator can set UMA_SLAB_PRIV, or not set
it, freely.  It appears to be the only UMA_SLAB_* bit that has no
other defined meaning in uma_core.c or elsewhere.  (Not entirely
true, there's also UMA_SLAB_OFFP which is never tested or set, and
bits 0x40 and 0x80 are unused.  There's also an unused us_pad
right after that.  It looks like OFFP is a leftover, with on vs
off page slab management controlled through UMA_ZONE_HASH and
also the PG_SLAB bit in the underlying struct vm_page.)

There's also a per-keg flag spelled UMA_ZFLAG_PRIVALLOC, along
with UMA_ZONE_NOFREE.  But UMA_ZFLAG_PRIVALLOC is never tested;
and UMA_ZONE_NOFREE is really per-keg, and you can't set it from
outside the UMA code.

When the system gets low on memory, it calls uma_reclaim(), which
does (simplified):

zone_foreach(zone_drain)
| zone_drain(zone)
  | zone_drain_wait(zone)
| bucket_cache_drain()
| zone_foreach_keg()
  | keg_drain()
| test: (UMA_ZONE_NOFREE || keg-uk_freef==NULL)
| if either is the case, return now, can't free

The issue here is that draining these special purpose, special-
physical-page-backed zones is not actually going to help the
system any (though freeing internal bucket data structures
could help slightly).  Of course I can have uk_freef == NULL,
but it is nice to keep some statistics, and maybe be able to
trade pages between various special-purpose physical spaces
(by doing my own zone_drain()s on them -- the one in uma_reclaim()
is not going to help the OS much as the physical pages cannot
be handed out to processes, and they run out against themselves,
not the VM system).

All in all, I'm now thinking that I'm abusing the slab allocator
too much here.  But I wonder if perhaps some minor changes to
uma_core might make this more useable, or if this is really within
the intent of the UMA code at all.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: expanding amd64 past the 1TB limit

2013-07-14 Thread Chris Torek
The changes associated with pt_p, pd_p and p4_p are cosmetic and IMHO
detract from the meat of the change.

My preference would be for the cosmetic changes to be committed
separately from the changes that rearrange the KVA.

I can split these out.

 -   DMPDPphys = allocpages(firstaddr, NDMPML4E);
 +   ndmpdpphys = howmany(ndmpdp, NPML4EPG);

NPDPEPG should be used here instead of NPML4EPG even though they are
numerically identical.

Ah, late-night editing thinko. :-)

 + * KERNBASE.  Similarly, if KMPL4I  (base+N), extra level 4 PDEs are

level 2 PDE's, right?

Er, yes (and L1s of course), and that should be  (base+N-1).

The idea here is that if for some reason KERNBASE were in the
middle of the NKPML4E entries, you'd have, at least initially
when only a few things are valid at VM_MIN_ADDRESS and at
KERNBASE:

 L4 tableL3 page  ... mapping:
+---+ VM_MIN_KERNEL_ADDRESS
  KPMLBASE  |   * --- [valid at 0 -- L2, then all-0s]
+---+
|   * --- [all-0s]
+---+
|   * --- [all-0s]
+---+
|  ...  |
+---+ KERNBASE
KPML4I  |   * --- [mostly-0s; the KDPDI'th -- L2]
+---+
  base+N-1  |   * --- [all-0s]
+---+ VM_MAX_KERNEL_ADDRESS

but then eventually allocating something between KERNBASE and
VM_MAX_KERNEL_ADDRESS would cause that last page of all-0s to
get filled in, etc.  Of course KPML4I should be the very last
(511'th) L4 table entry always, unless the kernel grows really
huge :-)

I'll fix the comments in the split-up patch.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: expanding amd64 past the 1TB limit

2013-07-14 Thread Chris Torek
Here's the split-up version with the additional comment
corrections.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org

Re: expanding amd64 past the 1TB limit

2013-07-08 Thread Chris Torek
Here is a final (I hope) version of the patch.  I dropped the
config option, but I added code to limit the real size of the
direct map PDEs.  The end result is that on small systems, this
ties up 14 more pages (15 from increasing NKPML4E, but one
regained because the new static variable ndmpdpphys is 1 instead
of 2).

(I fixed the comment errors I spotted earlier, too.)

Chris

 amd64/amd64/pmap.c  | 100 +---
 amd64/include/pmap.h|  36 +
 amd64/include/vmparam.h |  13 ---
 3 files changed, 97 insertions(+), 52 deletions(-)

Author: Chris Torek chris.to...@gmail.com
Date:   Thu Jun 27 18:49:29 2013 -0600

increase physical and virtual memory limits

Increase kernel VM space: go from .5 TB of KVA and 1 TB of direct
map, to 8 TB of KVA and 16 TB of direct map.  However, we allocate
less direct map space for small physical-memory systems.  Also, if
Maxmem is so large that there is not enough direct map space,
reduce Maxmem to fit, so that the system can boot unassisted.

diff --git a/amd64/amd64/pmap.c b/amd64/amd64/pmap.c
index 8dcf232..7368c96 100644
--- a/amd64/amd64/pmap.c
+++ b/amd64/amd64/pmap.c
@@ -232,6 +232,7 @@ u_int64_t   KPML4phys;  /* phys addr of kernel 
level 4 */
 
 static u_int64_t   DMPDphys;   /* phys addr of direct mapped level 2 */
 static u_int64_t   DMPDPphys;  /* phys addr of direct mapped level 3 */
+static int ndmpdpphys; /* number of DMPDPphys pages */
 
 static struct rwlock_padalign pvh_global_lock;
 
@@ -531,12 +532,27 @@ static void
 create_pagetables(vm_paddr_t *firstaddr)
 {
int i, j, ndm1g, nkpdpe;
+   pt_entry_t *pt_p;
+   pd_entry_t *pd_p;
+   pdp_entry_t *pdp_p;
+   pml4_entry_t *p4_p;
 
/* Allocate page table pages for the direct map */
ndmpdp = (ptoa(Maxmem) + NBPDP - 1)  PDPSHIFT;
if (ndmpdp  4) /* Minimum 4GB of dirmap */
ndmpdp = 4;
-   DMPDPphys = allocpages(firstaddr, NDMPML4E);
+   ndmpdpphys = howmany(ndmpdp, NPML4EPG);
+   if (ndmpdpphys  NDMPML4E) {
+   /*
+* Each NDMPML4E allows 512 GB, so limit to that,
+* and then readjust ndmpdp and ndmpdpphys.
+*/
+   printf(NDMPML4E limits system to %d GB\n, NDMPML4E * 512);
+   Maxmem = atop(NDMPML4E * NBPML4);
+   ndmpdpphys = NDMPML4E;
+   ndmpdp = NDMPML4E * NPDEPG;
+   }
+   DMPDPphys = allocpages(firstaddr, ndmpdpphys);
ndm1g = 0;
if ((amd_feature  AMDID_PAGE1GB) != 0)
ndm1g = ptoa(Maxmem)  PDPSHIFT;
@@ -553,6 +569,10 @@ create_pagetables(vm_paddr_t *firstaddr)
 * bootstrap.  We defer this until after all memory-size dependent
 * allocations are done (e.g. direct map), so that we don't have to
 * build in too much slop in our estimate.
+*
+* Note that when NKPML4E  1, we have an empty page underneath
+* all but the KPML4I'th one, so we need NKPML4E-1 extra (zeroed)
+* pages.  (pmap_enter requires a PD page to exist for each KPML4E.)
 */
nkpt_init(*firstaddr);
nkpdpe = NKPDPE(nkpt);
@@ -561,32 +581,26 @@ create_pagetables(vm_paddr_t *firstaddr)
KPDphys = allocpages(firstaddr, nkpdpe);
 
/* Fill in the underlying page table pages */
-   /* Read-only from zero to physfree */
+   /* Nominally read-only (but really R/W) from zero to physfree */
/* XXX not fully used, underneath 2M pages */
-   for (i = 0; (i  PAGE_SHIFT)  *firstaddr; i++) {
-   ((pt_entry_t *)KPTphys)[i] = i  PAGE_SHIFT;
-   ((pt_entry_t *)KPTphys)[i] |= PG_RW | PG_V | PG_G;
-   }
+   pt_p = (pt_entry_t *)KPTphys;
+   for (i = 0; ptoa(i)  *firstaddr; i++)
+   pt_p[i] = ptoa(i) | PG_RW | PG_V | PG_G;
 
/* Now map the page tables at their location within PTmap */
-   for (i = 0; i  nkpt; i++) {
-   ((pd_entry_t *)KPDphys)[i] = KPTphys + (i  PAGE_SHIFT);
-   ((pd_entry_t *)KPDphys)[i] |= PG_RW | PG_V;
-   }
+   pd_p = (pd_entry_t *)KPDphys;
+   for (i = 0; i  nkpt; i++)
+   pd_p[i] = (KPTphys + ptoa(i)) | PG_RW | PG_V;
 
/* Map from zero to end of allocations under 2M pages */
/* This replaces some of the KPTphys entries above */
-   for (i = 0; (i  PDRSHIFT)  *firstaddr; i++) {
-   ((pd_entry_t *)KPDphys)[i] = i  PDRSHIFT;
-   ((pd_entry_t *)KPDphys)[i] |= PG_RW | PG_V | PG_PS | PG_G;
-   }
+   for (i = 0; (i  PDRSHIFT)  *firstaddr; i++)
+   pd_p[i] = (i  PDRSHIFT) | PG_RW | PG_V | PG_PS | PG_G;
 
-   /* And connect up the PD to the PDP */
-   for (i = 0; i  nkpdpe; i++) {
-   ((pdp_entry_t *)KPDPphys)[i + KPDPI] = KPDphys +
-   (i  PAGE_SHIFT

Re: hw.physmem/hw.realmem question

2013-07-02 Thread Chris Torek
 by your
BIOS and architectural holes, including the large PCI hole.  Your
BIOS and motherboard etc may (or may not) allow you to remap some
of your hidden or shadowed RAM (out of the PCI hole),
increasing the boot-time value of Maxmem, and hence also
increasing both hw.realmem and actual, useable pages.

Note: the x86's architectural holes still use up some dedicated
kernel memory, even with shadowed-memory remapping: if addresses
from zero to 8.5 GB are valid (as they are on my box), the kernel
allocates enough vm_page data structures to have one for all 8.5
GB, even though there's a .5 GB PCI hole with no RAM behind it.
Those pages are marked not here, never use these -- but they
still take sizeof(struct vm_page) bytes (120 bytes) to represent.
512 MB of hole = 512*1024*1024 / 4096 = 131072 pages, which means
the kernel is using 15728640 bytes (15 MB) to track this empty
area.  So my remapping hardware gains me 512 MB and then the
kernel loses 15, for a net of 497 MB recovered.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: expanding amd64 past the 1TB limit

2013-07-01 Thread Chris Torek
I went ahead and implemented the ndmpdpphys change I
had thought about.  Here's that patch by itself.

At this point it might be reasonable to use the patch
without the AMD64_HUGE option, just increasing the KVM
area, as the direct map no longer consumes extra pages.

(There's a glitch in the comment in the original patch,
I'll fix that if/when there's any agreement on whether
it gets applied in some form :-) )

Chris

diff --git a/amd64/amd64/pmap.c b/amd64/amd64/pmap.c
index acf5af2..f1ed8b6 100644
--- a/amd64/amd64/pmap.c
+++ b/amd64/amd64/pmap.c
@@ -232,6 +232,7 @@ u_int64_t   KPML4phys;  /* phys addr of kernel 
level 4 */
 
 static u_int64_t   DMPDphys;   /* phys addr of direct mapped level 2 */
 static u_int64_t   DMPDPphys;  /* phys addr of direct mapped level 3 */
+static int ndmpdpphys; /* number of DMPDPphys pages */
 
 static struct rwlock_padalign pvh_global_lock;
 
@@ -543,7 +544,18 @@ create_pagetables(vm_paddr_t *firstaddr)
ndmpdp = (ptoa(Maxmem) + NBPDP - 1)  PDPSHIFT;
if (ndmpdp  4) /* Minimum 4GB of dirmap */
ndmpdp = 4;
-   DMPDPphys = allocpages(firstaddr, NDMPML4E);
+   ndmpdpphys = howmany(ndmpdp, NPML4EPG);
+   if (ndmpdpphys  NDMPML4E) {
+   /*
+* Each NDMPML4E allows 512 GB, so limit to that,
+* and then readjust ndmpdp and ndmpdpphys.
+*/
+   printf(NDMPML4E limits system to %d GB\n, NDMPML4E * 512);
+   Maxmem = atop(NDMPML4E * NBPML4);
+   ndmpdpphys = NDMPML4E;
+   ndmpdp = NDMPML4E * NPDEPG;
+   }
+   DMPDPphys = allocpages(firstaddr, ndmpdpphys);
ndm1g = 0;
if ((amd_feature  AMDID_PAGE1GB) != 0)
ndm1g = ptoa(Maxmem)  PDPSHIFT;
@@ -626,7 +638,7 @@ create_pagetables(vm_paddr_t *firstaddr)
p4_p[PML4PML4I] |= PG_RW | PG_V | PG_U;
 
/* Connect the Direct Map slot(s) up to the PML4. */
-   for (i = 0; i  NDMPML4E; i++) {
+   for (i = 0; i  ndmpdpphys; i++) {
p4_p[DMPML4I + i] = DMPDPphys + ptoa(i);
p4_p[DMPML4I + i] |= PG_RW | PG_V | PG_U;
}
@@ -1698,7 +1710,7 @@ pmap_pinit(pmap_t pmap)
pmap-pm_pml4[KPML4BASE + i] = (KPDPphys + (i  PAGE_SHIFT)) |
PG_RW | PG_V | PG_U;
}
-   for (i = 0; i  NDMPML4E; i++) {
+   for (i = 0; i  ndmpdpphys; i++) {
pmap-pm_pml4[DMPML4I + i] = (DMPDPphys + (i  PAGE_SHIFT)) |
PG_RW | PG_V | PG_U;
}
@@ -1955,7 +1967,7 @@ pmap_release(pmap_t pmap)
 
for (i = 0; i  NKPML4E; i++)   /* KVA */
pmap-pm_pml4[KPML4BASE + i] = 0;
-   for (i = 0; i  NDMPML4E; i++)  /* Direct Map */
+   for (i = 0; i  ndmpdpphys; i++)/* Direct Map */
pmap-pm_pml4[DMPML4I + i] = 0;
pmap-pm_pml4[PML4PML4I] = 0;   /* Recursive Mapping */
 
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: expanding amd64 past the 1TB limit

2013-06-28 Thread Chris Torek
, NDMPML4E);

new:ndmpdpphys = howmany(ndmpdp, NPML4EPG);
if (ndmpdpphys  NDMPML4E)
panic(something or other); /* or shrink to fit? */
DMPDPphys = allocpages(firstaddr, ndmpdpphys);

and then instead of connecting NDMPML4E pages, connect ndmpdpphys
of them.  Would that break anything?  The direct mapped VA range
is known in advance; if you get a bad value it will look direct-
mapped, but now not have an L3 page under it, whereas before it
would always have an L3 page, just no L2 page.  (Offhand, I think
this would only affect pmap_enter(), and calling that for an
invalid physical address would be bad anyway.)

[I'm also not sure if we might be able to tweak the KPTphys usage
slightly to eliminate whole pages full of L1 PTEs, e.g., if the
GENERIC kernel occupies about 15 MB, we can map it with 7 2MB big
page entries in KPDphys, then just one regular PTE-page and 256
regular PTEs in the first actually-used page of KPTphys.  (This
would recover another 7 pages in this particular example.)  But
this would at least affect pmap_init()'s loop over nkpt entries to
initialize the vm page array entries that describe the KPTphys
area, so I did not attempt it.]

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: expanding amd64 past the 1TB limit

2013-06-27 Thread Chris Torek
 .. (but I still needed the map I drew of the page tables...).

care to share? :-)

It's on paper (I need to get a whiteboard...).  If I can ASCIIfy it ...

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: expanding amd64 past the 1TB limit

2013-06-27 Thread Chris Torek
OK, I wasted :-) way too much time, but here's a text file that
can be comment-ified or stored somewhere alongside the code or
whatever...

(While drawing this I realized that there's at least one wasted
page if the machine has .5 TB or less: we can just leave zero
slots in the corresponding L4 direct-map entries.  But that would
require switching to the bcopy() method also mentioned below.  Or
indexing into vmspace0.vm_pmap.pm_pml4, which is basically the
same thing.)

Chris

-

There are six -- or sometimes five -- sets of pages allocated here
at boot time to map physical memory in two ways.  Note that each
page, regardless of level, stores 512 PTEs (or PDEs or PDPs, but
let's just use PTE here and prefix it with level as needed: 4,
3, 2, or 1.)

There is one page for the top level, L4, page table entries.  Each
L4 PTE maps 512 GB of space.  Unless it's marked invalid, no L4
PTE can be marked stop here: it either is marked as this
address is invalid, or it points to one physically-adressed page
full of L3 PTEs.  Eventually, those L3 PTEs will map-or-reject
half a terabyte.  512 entries, each mapping .5 TB, allow us to map
256 TB, which is as much as the hardware supports (there are, in
effect, only 48 virtual address bits: the top 16 bits must match
the 47th bit).

The L4 entry halfway down, at PML4PML4I, is set to point back to
this page itself; that's the recursive page table for user
space, which we do nothing else with at boot time.

We need (up to) NDMPML4E pages, each holding 512 L3 PTEs, for the
direct map space.  If the processor supports 1 GB pages, an L3 PTE
can be marked with stop here and these L3 PTEs each grant (or
forbid) access to 1 GB of physical space at a time.  A system
with, say, 3 GB of RAM starting at 0 can map it all with three L3
PTEs: address 0 is valid for 1GB, address 1GB is valid for
1GB, address 2GB is valid for 1GB.  The remaining L3 PTEs are
zero, making the remaining address space invalid.

If the processor does not support 1 GB pages, or if there is less
than 1 GB of RAM at the end (e.g., if the system has 4.5 GB),
the L3 PTEs may need to point to more pages holding L2 PTEs.
These L2 PTEs always support 2 MB pages.  Each page of L2 PTEs
maps 1 GB. So a machine with 4.5 GB and 1 GB mappings needs one L3
page with four valid 1 GB L3 PTEs and then one L3 PTE pointing to
one page of L2 PTEs.  That one page of L2 PTEs is half-filled,
containing 256 2MB-sized PTEs, mapping the 512 MB.  The remaining
half of that page is zero, making the remaining addresses invalid.

Pictorially, and adding the names of the physical page(s), thus
far we have this.  (Note, the L4 PTE page is drawn more than twice
as tall as the L3 and L2 pages, just to get space for arrows.)

  LEVEL 4:LEVEL 3: LEVEL 2:
  _._
  KPML4phys  v   \
 +-+  |
 |  0: |  |
 |-|  |
 |  1: |  | DMPDPphys  DMPDphys
 (   ...   )  |.- +-+ ++
 | 127:|  |   /|  0: 0GB | .- |  0: 4GB|
 |-|  |  | |  1: 1GB |/|  1: 4GB+2MB|
  PML4PML4I: | 128: *--|--/  | |  2: 2GB |   / |  2: 4GB+4MB|
 |-| | |  3: 3GB |  /  (  ...   )
 | 129:| | |  4:  *--|-/   | 255: 4.5GB-2MB |
 |   ...   | | |  5: | | 256:   |
     |-| | (   ...   ) | 257:   |
 /  DMPML4I: |  *--|-/ | 511:| (  ...   )
 NDMPML4E|-|   +-+ ++
 \   |  *--|- |   0:|
 |-|   |   1:|
 | |   |   2:|  (These are used only
 |-|   |   3:|   if the system has more
 |   ...   |   (   ...   )   than 512 GB)
   ( |-|  )| 509:|
   ( | 510: see below )| 510:|
   ( |-|  )| 511:|
   ( | 511: see below )+-+
 +-+

If the hardware supports 1GB pages, ndm1g is the number of
gigabyte entries (4 in the example above).  Otherwise it's just
zero.  Meanwhile ndmpdp is the number of gigabytes of RAM that
need to be mapped, in this case 5.  Thus, if ndmpdp  ndm1g, we
need ndmpdp-ndm1g pages to hold some L2 PTEs.

Now we get to the weirder case of the kernel itself (both its
non-direct-mapped dynamically allocated virtual memory, and its
text/data/bss).  The branch offset limitations encourage the
placement of the kernel's text, etc., in the last 2 GB of virtual
space, i.e., starting at 0x..f800..  But, we want
a reasonable amount of room for dynamic VM.  So we give the kernel
at least 512 GB of VM -- that's one L4 PTE -- while

expanding amd64 past the 1TB limit

2013-06-26 Thread Chris Torek
(Note: Last week I asked about this on the freebsd-current list.
It turned out slightly harder than I thought, as the 512GB kernel
virtual area is based on what fits into a single L4 page table
entry.)

I was asked to expand the kernel limits for amd64 systems.  While
I do not have a system with enough RAM to test this for real, the
changes below seem to boot and run OK.

I went just a little bit wild in create_pagetables(). :-)  The
lines with the casts got long (and hard to read) so I shortened
them (but I still needed the map I drew of the page tables...).
If using ptoa() like this is OK, probably there should be a few
more of those, e.g., in the changes to pmap_pinit().

Anyway, I wonder if some form of this patch (perhaps even without
the #ifdefs) might be accepted back.  I'm not sure about the KPML4BASE
name, but it clearly needs to be different from KPML4I.  (At first
I was considering moving KERNBASE too but the branch offsets seem
to be the real limiting factor here.)

Possibly dumb question: around the comment this replaces some of
the KPTphys entries above, would it be possible to reclaim a few
pages by calculating in advance where the 2MB page mappings obviate
the need for the underlying KPTphys pages, and just offset things?

Another note: one could get rid of the power of 2 requirement
for NDMPML4E.  It arises from the translation between direct
mapped virtual and physical addresses (being |= and =~), but the
same result can be achieved by adding and subtracting an offset,
which would allow the base and limit to be arbitrary, rather than
a power of two.  (Still, it did not seem worth doing here.)

Chris

diff --git a/amd64/amd64/pmap.c b/amd64/amd64/pmap.c
index 272158d..acf5af2 100644
--- a/amd64/amd64/pmap.c
+++ b/amd64/amd64/pmap.c
@@ -534,6 +534,10 @@ static void
 create_pagetables(vm_paddr_t *firstaddr)
 {
int i, j, ndm1g, nkpdpe;
+   pt_entry_t *pt_p;
+   pd_entry_t *pd_p;
+   pdp_entry_t *pdp_p;
+   pml4_entry_t *p4_p;
 
/* Allocate page table pages for the direct map */
ndmpdp = (ptoa(Maxmem) + NBPDP - 1)  PDPSHIFT;
@@ -556,6 +560,10 @@ create_pagetables(vm_paddr_t *firstaddr)
 * bootstrap.  We defer this until after all memory-size dependent
 * allocations are done (e.g. direct map), so that we don't have to
 * build in too much slop in our estimate.
+*
+* Note that when NKPML4E  1, we have an empty page underneath
+* all but the KPML4I'th one, so we need NKPML4E-1 extra (zeroed)
+* pages.  (pmap_enter requires a PD page to exist for each KPML4E.)
 */
nkpt_init(*firstaddr);
nkpdpe = NKPDPE(nkpt);
@@ -564,32 +572,26 @@ create_pagetables(vm_paddr_t *firstaddr)
KPDphys = allocpages(firstaddr, nkpdpe);
 
/* Fill in the underlying page table pages */
-   /* Read-only from zero to physfree */
+   /* Nominally read-only (but really R/W) from zero to physfree */
/* XXX not fully used, underneath 2M pages */
-   for (i = 0; (i  PAGE_SHIFT)  *firstaddr; i++) {
-   ((pt_entry_t *)KPTphys)[i] = i  PAGE_SHIFT;
-   ((pt_entry_t *)KPTphys)[i] |= PG_RW | PG_V | PG_G;
-   }
+   pt_p = (pt_entry_t *)KPTphys;
+   for (i = 0; ptoa(i)  *firstaddr; i++)
+   pt_p[i] = ptoa(i) | PG_RW | PG_V | PG_G;
 
/* Now map the page tables at their location within PTmap */
-   for (i = 0; i  nkpt; i++) {
-   ((pd_entry_t *)KPDphys)[i] = KPTphys + (i  PAGE_SHIFT);
-   ((pd_entry_t *)KPDphys)[i] |= PG_RW | PG_V;
-   }
+   pd_p = (pd_entry_t *)KPDphys;
+   for (i = 0; i  nkpt; i++)
+   pd_p[i] = (KPTphys + ptoa(i)) | PG_RW | PG_V;
 
/* Map from zero to end of allocations under 2M pages */
/* This replaces some of the KPTphys entries above */
-   for (i = 0; (i  PDRSHIFT)  *firstaddr; i++) {
-   ((pd_entry_t *)KPDphys)[i] = i  PDRSHIFT;
-   ((pd_entry_t *)KPDphys)[i] |= PG_RW | PG_V | PG_PS | PG_G;
-   }
+   for (i = 0; (i  PDRSHIFT)  *firstaddr; i++)
+   pd_p[i] = (i  PDRSHIFT) | PG_RW | PG_V | PG_PS | PG_G;
 
-   /* And connect up the PD to the PDP */
-   for (i = 0; i  nkpdpe; i++) {
-   ((pdp_entry_t *)KPDPphys)[i + KPDPI] = KPDphys +
-   (i  PAGE_SHIFT);
-   ((pdp_entry_t *)KPDPphys)[i + KPDPI] |= PG_RW | PG_V | PG_U;
-   }
+   /* And connect up the PD to the PDP (leaving room for L4 pages) */
+   pdp_p = (pdp_entry_t *)(KPDPphys + ptoa(KPML4I - KPML4BASE));
+   for (i = 0; i  nkpdpe; i++)
+   pdp_p[i + KPDPI] = (KPDphys + ptoa(i)) | PG_RW | PG_V | PG_U;
 
/*
 * Now, set up the direct map region using 2MB and/or 1GB pages.  If
@@ -599,37 +601,41 @@ create_pagetables(vm_paddr_t *firstaddr)
 * memory, pmap_change_attr() will demote any 2MB or 1GB page mappings
 * that are partially used

Re: sed query

2013-06-02 Thread Chris Rees
On 2 June 2013 11:41, Eduardo Morras emorr...@yahoo.es wrote:
 On Fri, 31 May 2013 15:01:59 +0100
 Chris Rees utis...@gmail.com wrote:

 Hi all,

 I think I've discovered a strange behaviour of sed perhaps triggered
 by the length of a regex passed to it.  I noticed that a certain
 expression I passed took a very long time, and suspected the usual
 backtracking loop, so I started trimming it... and discovered this:

 [crees@pegasus]~% time sed -ne s,^BitchX-[0-9][^|]*[\|]/usr/por,,
 /var/db/pkg/INDEX-9
 4.699u 0.007s 0:04.70 99.7% 40+2733k 0+0io 0pf+0w
 [crees@pegasus]~% time sed -ne s,^BitchX-[0-9][^|]*[\|]/usr/po,,
 /var/db/pkg/INDEX-9
 0.042u 0.000s 0:00.04 100.0% 48+3216k 0+0io 0pf+0w

 I've looked at the code, and can't from a brief glance figure out why
 a slightly longer regex makes such a difference-- does it start to
 split it?

 Perhaps second one uses memory cache data? Run both twice and show us the 
 second times.


Nope, same.

[crees@pegasus]~% time sed -ne s,^BitchX-[0-9][^|]*[\|]/usr/por,,
/var/db/pkg/INDEX-9
4.703u 0.007s 0:04.85 96.9% 40+2732k 210+0io 0pf+0w
[crees@pegasus]~% time sed -ne s,^BitchX-[0-9][^|]*[\|]/usr/por,,
/var/db/pkg/INDEX-9
4.748u 0.007s 0:04.75 99.7% 40+2732k 0+0io 0pf+0w

I also get the same on head;

[crees@medusa]~% time sed -ne s,^BitchX-[0-9][^|]*[\|]/usr/por,,
/var/db/pkg/INDEX-10
7.813u 0.015s 0:07.96 98.2% 40+183k 0+0io 0pf+0w
[crees@medusa]~% time sed -ne s,^BitchX-[0-9][^|]*[\|]/usr/po,,
/var/db/pkg/INDEX-10
0.070u 0.000s 0:00.07 100.0% 45+205k 0+0io 0pf+0w
[crees@medusa]~% uname -a
FreeBSD medusa 10.0-CURRENT FreeBSD 10.0-CURRENT #0 r250009: Thu May
30 10:11:16 BST 2013 root@medusa:/usr/obj/usr/src/sys/MEDUSA
amd64

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: sed query

2013-06-02 Thread Chris Rees
On 2 June 2013 18:15, Florent Peterschmitt flor...@peterschmitt.fr wrote:
 Le 02/06/2013 14:16, Chris Rees a écrit :
 On 2 June 2013 11:41, Eduardo Morras emorr...@yahoo.es wrote:
 On Fri, 31 May 2013 15:01:59 +0100
 Chris Rees utis...@gmail.com wrote:

 Hi all,

 I think I've discovered a strange behaviour of sed perhaps triggered
 by the length of a regex passed to it.  I noticed that a certain
 expression I passed took a very long time, and suspected the usual
 backtracking loop, so I started trimming it... and discovered this:

 [crees@pegasus]~% time sed -ne s,^BitchX-[0-9][^|]*[\|]/usr/por,,
 /var/db/pkg/INDEX-9
 4.699u 0.007s 0:04.70 99.7% 40+2733k 0+0io 0pf+0w
 [crees@pegasus]~% time sed -ne s,^BitchX-[0-9][^|]*[\|]/usr/po,,
 /var/db/pkg/INDEX-9
 0.042u 0.000s 0:00.04 100.0% 48+3216k 0+0io 0pf+0w

 I've looked at the code, and can't from a brief glance figure out why
 a slightly longer regex makes such a difference-- does it start to
 split it?

 Perhaps second one uses memory cache data? Run both twice and show us the 
 second times.


 Nope, same.

 [crees@pegasus]~% time sed -ne s,^BitchX-[0-9][^|]*[\|]/usr/por,,
 /var/db/pkg/INDEX-9
 4.703u 0.007s 0:04.85 96.9% 40+2732k 210+0io 0pf+0w
 [crees@pegasus]~% time sed -ne s,^BitchX-[0-9][^|]*[\|]/usr/por,,
 /var/db/pkg/INDEX-9
 4.748u 0.007s 0:04.75 99.7% 40+2732k 0+0io 0pf+0w

 I also get the same on head;

 [crees@medusa]~% time sed -ne s,^BitchX-[0-9][^|]*[\|]/usr/por,,
 /var/db/pkg/INDEX-10
 7.813u 0.015s 0:07.96 98.2% 40+183k 0+0io 0pf+0w
 [crees@medusa]~% time sed -ne s,^BitchX-[0-9][^|]*[\|]/usr/po,,
 /var/db/pkg/INDEX-10
 0.070u 0.000s 0:00.07 100.0% 45+205k 0+0io 0pf+0w
 [crees@medusa]~% uname -a
 FreeBSD medusa 10.0-CURRENT FreeBSD 10.0-CURRENT #0 r250009: Thu May
 30 10:11:16 BST 2013 root@medusa:/usr/obj/usr/src/sys/MEDUSA
 amd64

 Chris

 Yes I tried too on -current. And I tried also on GNU/Linux and there
 isn't this problem. Is it gnu or bsd sed ?


BSD sed, GNU sed doesn't show this;

[crees@pegasus]/usr/ports/textproc/gsed% time gsed -ne
s,^BitchX-[0-9][^|]*[\|]/usr/por,, /var/db/pkg/INDEX-9
0.019u 0.009s 0:00.04 25.0% 408+6132k 1+0io 2pf+0w

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


sed query

2013-05-31 Thread Chris Rees
Hi all,

I think I've discovered a strange behaviour of sed perhaps triggered
by the length of a regex passed to it.  I noticed that a certain
expression I passed took a very long time, and suspected the usual
backtracking loop, so I started trimming it... and discovered this:

[crees@pegasus]~% time sed -ne s,^BitchX-[0-9][^|]*[\|]/usr/por,,
/var/db/pkg/INDEX-9
4.699u 0.007s 0:04.70 99.7% 40+2733k 0+0io 0pf+0w
[crees@pegasus]~% time sed -ne s,^BitchX-[0-9][^|]*[\|]/usr/po,,
/var/db/pkg/INDEX-9
0.042u 0.000s 0:00.04 100.0% 48+3216k 0+0io 0pf+0w

I've looked at the code, and can't from a brief glance figure out why
a slightly longer regex makes such a difference-- does it start to
split it?

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: /bin/sh = STDIN functions, var scope messing

2013-05-31 Thread Chris Rees
On 31 May 2013 20:50, Dan Nelson dnel...@allantgroup.com wrote:

 In the last episode (May 31), Reid Linnemann said:
  On Fri, May 31, 2013 at 1:12 PM, Teske, Devin devin.te...@fisglobal.com
wrote:
   If you're arguing we have to change sh's behavior to be more
compliant,
   jilles already quoted XCU 2.12 (our shell is well within its right to
   run any/all lvalue/rvalue operands of a pipe in a sub-shell without
   contradicting the guidelines).
  
   But if you're arguing that it has to change to make things better or
   easier...  I don't know about that.  Might just make people lulled
into
   using a style that's non-portable.  I'd like to keep things the way
they
   are so that if you program for FreeBSD, you're inherently going to
   program in a fashion that is more portable.
 
  FWIW bash (invoked as sh) on RHEL-based linux systems exhibits the same
  behavior-
 
  sh-3.2$ var=1
  sh-3.2$ yes|var=2
  sh-3.2$ echo $var
  1
  sh-3.2$
 
  If my opinion matters at all, I would agree that for the sake of
  portability that behavior be consistent with the majority of sh
  implementations rather than right according to arbitrary ruling.

 On the other hand, zsh runs the last component of a pipeline in the parent
 shell.  The usual model is do work in pipeline, process results in final
 component, and being able to simply set variables there that can be used
in
 the rest of the script is very elegant.

Right... But it's not portable.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Order of canonical upgrade sequence

2013-05-30 Thread Chris Rees
On 30 May 2013 14:42, Steve Wills st...@mouf.net wrote:
 On 05/29/13 16:02, Chris Rees wrote:
 Hi all!

 Back in 2005, when Alexander Leidinger wrote the make delete-old
 target, he documented the order of upgrade such that it should be run
 before mergemaster [1];

 #  7.  `make installworld'
 #  8.  `make delete-old'
 #  9.  `mergemaster'


 It would be good to mention that it's wise to make check-old, and
 rebuilding any ports that depend on the old libs, before doing make
 delete-old.

make delete-old doesn't touch the libraries; it's the next steps that
do that (make delete-old-libs).

The Handbook section is far more verbose on this, and I think that
this reference in the Makefile is more as a quick reminder than a
step-by-step walkthrough.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Order of canonical upgrade sequence

2013-05-29 Thread Chris Rees
Hi all!

Back in 2005, when Alexander Leidinger wrote the make delete-old
target, he documented the order of upgrade such that it should be run
before mergemaster [1];

#  7.  `make installworld'
#  8.  `make delete-old'
#  9.  `mergemaster'

I have merged the delete-old section of the Handbook into the
upgrading chapter, and independently decided to put mergemaster first,
because I thought it would be safer, but checked here before I
committed.

I think that steps 8 and 9 should be reversed, because of the
possibility of an unbootable system being made, when an rc script
references an executable that has just been removed for example.

I cannot think of an example where the system is left
unbootable/damaged if make delete-old is run after mergemaster.

What do people think of the patch at [2]?

Chris

[1] http://svnweb.freebsd.org/base/head/Makefile?r1=148329r2=148330;

[2] http://www.bayofrum.net/~crees/patches/delete-old-order.diff
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Order of canonical upgrade sequence

2013-05-29 Thread Chris Rees
On 29 May 2013 19:57, Alexander Leidinger alexan...@leidinger.net wrote:
 On Wed, 29 May 2013 11:53:41 -0600
 Warner Losh i...@bsdimp.com wrote:

 [[ summarizing a conversation in irc ]]

 The below fragment doesn't match UPDATING. Since I don't think the
 order matters; and since we've had no reports that UPDATING is wrong;
 and since I think way more people follow updating than the Makefile;
 we should fix the makefile and make the docs match both.

 The order matters, mergemaster first, then delete-old. UPDATING is
 correct. At least regarding the order all places should be corrected
 which tell to use the reverse order.

 Warner

 On May 29, 2013, at 10:02 AM, Chris Rees wrote:

  Hi all!
 
  Back in 2005, when Alexander Leidinger wrote the make delete-old
  target, he documented the order of upgrade such that it should be
  run before mergemaster [1];
 
  #  7.  `make installworld'
  #  8.  `make delete-old'
  #  9.  `mergemaster'

 This is the wrong order.

  I have merged the delete-old section of the Handbook into the
  upgrading chapter, and independently decided to put mergemaster
  first, because I thought it would be safer, but checked here before
  I committed.
 
  I think that steps 8 and 9 should be reversed, because of the
  possibility of an unbootable system being made, when an rc script
  references an executable that has just been removed for example.

 I agree.


Committed.  Thanks!

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: FreeBSD installers and future direction

2013-05-27 Thread Chris Rees
On 27 May 2013 03:10, Daniel Eischen deisc...@freebsd.org wrote:

 On Mon, 27 May 2013, Teske, Devin wrote:


 I don't think there's any reason why we have to write it in C if we can
write
 it in sh.


 I don't really care one way or the other (C or sh), but
 I can say that I can understand(*) well structured C a lot
 better than well structured sh.  Having something more
 strongly typed certainly helps understanding.

 (*) Assuming some level of complexity (I know that's
 subjective).

I think it's all down to familiarity.  I suppose sh is more resistant to
many stupid bugs and handles strings well But it has its own troubles
too of course.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: stupid question about sendmail

2013-05-24 Thread Chris Rees
On 24 May 2013 08:34, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:

 how to redirect recipient address. i mean - if someone try to send to
x...@y.pl from serwer then it should be redirected to local account, while the
rest of mails to domain @y.pl should get out normally.

 alternatively outgoing mail to x...@y.pl should be rejected.


 tried access.db -

 To:x...@y.pl REJECT

 doesn't work


 any idea. thank you

Try a sendmail list?

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: stupid question about sendmail

2013-05-24 Thread Chris Rees
On 24 May 2013 11:05, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:


 http://www.sendmail.com/sm/open_source/docs/m4/features.html

 Maybe a line like this one will help you achieve your goal:

 j...@bar.com error:5.7.0:550 Address invalid


 I was wrong again, sorry, but I believe I got it right this time:

 1. Edit the /etc/mail/access file.

 2. Insert a line like this one:

 To:mail...@some.domain.tld REJECT


 tried too.

 doesn't work.


http://www.sendmail.com/sm/open_source/support/public_forums/

There is also an IRC channel, #sendmail on Freenode.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: ladmins that cannot type a command

2013-05-04 Thread Chris Rees
On 3 May 2013 20:01, Dieter BSD dieter...@gmail.com wrote:

  good for today and future ladmins that cannot type a command.
 
  Any USEFUL proposals that add some real functionality?
 
  Since this will enable more people to run FreeBSD that otherwise
  wouldn't give it a second glance, I would say it is VERY useful.

 Really?  How useful is FreeBSD going to be to someone who cannot type?
 Who is the target here?  Idiots?  People without fingers?

 I see lots of chatter about adding new stuff, much of which is ...
 rather questionable.  Meanwhile many of the features FreeBSD
 already has are broken.  No shortage of PRs that need fixing.
 Many of these PRs have been sitting around, unloved, for *years*.
 Some of them even contain patches (that *work*)!

 If you're building a race car, make sure the engine runs before
 spending your time polishing the wheels.

You need to spend more time writing patches rather than telling other
people what not to do.  We're (nearly) all volunteers.

If you think something needs fixing, you can have a go yourself or pay
someone, but do NOT tell someone that their contribution is not worthwhile
while pushing your agenda.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: How to sabotage 9.*-RELEASE worldbuilds?

2013-04-27 Thread Chris Rees
On 27 Apr 2013 11:09, rank1see...@gmail.com wrote:

 Just do this:
 # echo 'FORMATS=html txt'  /etc/make.conf

 buildworld fails, at the SAME point for ALL 9.*-RELEASE-es:
 ===
 === lib/libauditd (all)
 === lib/libcom_err (all)
 gzip -cn /usr/src/lib/libcom_err/../../contrib/com_err/com_err.3 
com_err.3.gz
 === lib/libcom_err/doc (all)
 Graph cycles through com_err.'html

 make: don't know how to make txt'. Stop
 *** Error code 2

 Stop in /usr/src/lib/libcom_err.
 *** Error code 1

 Stop in /usr/src/lib.
 *** Error code 1

 Stop in /usr/src.
 *** Error code 1

 Stop in /usr/src.
 *** Error code 1

 Stop in /usr/src.
 ===


 If you aren't in a mood, to wait for a buildworld to fail, you can see
errors faster:
 # cd /usr/src  make cleandir  /dev/null
 ---
 /usr/src/share/mk/bsd.info.mk, line 140: warning: duplicate script for
target txt.gz ignored
 ...
 ---


 For /usr/doc builds, in /etc/make.conf, I have:
 ===
 # For port textproc/docproj
 WITH_JADETEX=YES
 DOC_LANG=en_US.ISO8859-1
 FORMATS=html txt

And yet another example of misunderstanding how quotes in Makefiles breaks
things :)

If in doubt, leave quotes out!

(This only applies to make)

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Fwd: GSOC: Qt front-ends

2013-04-24 Thread Chris Rees
On 24 Apr 2013 05:36, Justin Edward Muniz justin.mu...@maine.edu wrote:

 
  Justin I say stick to  FreeBSD-update . My reason is, as Pkgng becomes
  more popular , a front end for ports will be less useful as binary
packages
  become more popular . Kports is a monster program , you should set a
  reasonable goal ,and target dates; which may be hard with a cleanup
project
  .   Also a update notifier for kde that handles FreeBSD update would be
  very useful .
 
  My 2cents .
  ---
  Mark saad | mark.s...@longcount.org
 


 Thank you very much Mark,

  I was definitely hoping to get community feedback on this, and I
value
 you voicing your opinion. I agree that kports is a mammoth, and also that
a
 system updater GUI should have a way to notify the user of new updates.

  Any other perspectives are welcome, as well as support for a
 freebsd-update approach. I am working to refine my proposal, which as
 you've pointed out is very important. Eventually I would like to help in
 all three mentioned areas, but for now I must focus on one application.
 Does anyone think that a custom kernel configuration and management GUI
 utility would be desirable?

  I will shape my goals to meet the needs of the community.

Our kernel is actually very easy to configure, so I'm not convinced that
it's needed; you may be thinking of Linux's menuconfig, but I think that is
because of the complexity.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Fwd: GSOC: Qt front-ends

2013-04-24 Thread Chris Rees
On 24 April 2013 18:30, Justin Edward Muniz justin.mu...@maine.edu wrote:
 Our kernel is actually very easy to configure, so I'm not convinced that
 it's needed; you may be thinking of Linux's menuconfig, but I think that is
 because of the complexity.

 Chris



 While configuring the kernel may be trivial to someone who understands the
 process and their systems needs, I am thinking of a software tool that goes
 beyond the scope of the occasional generating of a kernel configuration
 file.

 Imagine that you have a number of systems and you want to run kernels that
 are lighter weight than the generic kernel but each system has its own
 individual needs. A GUI could help manage a large number of custom kernels,
 and provide access to convenient access to features such as specifying a
 kernel to load on the next boot only for testing. You could even configure
 the custom kernel profiles to be built from separate source directories.

 That is not to say of course that everyone else using x11 couldn't benefit
 from it as well. The application could help avoid compatibility issues
 during kernel installation by comparing the kernel's version to the version
 of world. Some helpful aids would be visual categorization of options as
 well as option descriptions, caveats, and hyperlinks to more in depth
 information.

 As for its place in Google Summer of Code, you could be right, it may not be
 enough to dedicate such resources. I know however that I would use it, maybe
 others would as well? Thank you for your advice once again Chris! What do
 you think about the other utilities?

I think the interface to pkgng and freebsd-update are still
interesting; at least more worthwhile than the kernel configuration
one.

I think the pkgng one has the edge, since packages are updated far
more often than base, and it's easier to track base.

Now you are at a stage where you should make your own decision; which
one looks the most interesting to you?  Once you decide on an area of
interest, you can just start hacking :)

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: GSOC: Qt front-end for freebsd-update

2013-04-14 Thread Chris Rees
On 14 April 2013 07:11, Justin Edward Muniz justin.mu...@maine.edu wrote:
  I am excited for this year's Google Summer of Code, and I have a
 project in mind that I am working to propose.

  I am a CS major and have experience with Qt, C++ and shell scripting.
 I have been developing on FreeBSD for several years, and I am looking to
 tackle developing a new Qt front-end for the freebsd-update command.

  I am thinking a rather straight-forward user interface with more
 advanced options available (such as selecting a specific server). I am
 looking for constructive feedback, and also a developer interested in
 mentoring me this Summer.

  Thanks everyone for your time, and for your hard work.

Are you referring to the suggestion below?

https://wiki.freebsd.org/IdeasPage#KDE_front-ends_to_the_freebsd-update.288.29utility

Colin Percival (CCd) mentions he would be the technical contact for
this project, though that was five years ago-- perhaps he'd like to
comment.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: considering i386 as a tier 1 architecture

2013-04-02 Thread Chris BeHanna
Goodness gracious, did no one see the date on the original post?

What's the limit on this fishing hole?

-- 
Chris BeHanna
ch...@behanna.org
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: boot time crash in if_detach_internal()

2013-04-02 Thread Chris Torek
Can you provide a backtrace that leads to this?

Sure.  In case it's not obvious, the __rw_rlock at the top of the
trace is working on a lock that has never been initialized (the
first of the two ipv4 PCBs).

Chris

Booting...
GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.0-CURRENT #2: Sun Mar 31 01:32:38 MDT 2013
to...@dev.torek.net:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 3.2 (tags/RELEASE_32/final 170710) 20121221
WARNING: WITNESS option enabled, expect reduced performance.
CPU: Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz (3292.28-MHz K8-class CPU)
  Origin = GenuineIntel  Id = 0x306a9  Family = 0x6  Model = 0x3a  Stepping = 
9
  
Features=0x8fa3ab7fFPU,VME,DE,PSE,TSC,MSR,PAE,CX8,APIC,SEP,PGE,CMOV,PAT,PSE36,DTS,MMX,FXSR,SSE,SSE2,SS,PBE
  
Features2=0xa1bae217SSE3,PCLMULQDQ,DTES64,DS_CPL,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,F16C,HV
  AMD Features=0x28100800SYSCALL,NX,RDTSCP,LM
  AMD Features2=0x1LAHF
  TSC: P-state invariant
real memory  = 536870912 (512 MB)
avail memory = 472559616 (450 MB)
Event timer LAPIC quality 400
ACPI APIC Table: BHYVE  BVMADT  
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 2 package(s) x 1 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
random device not loaded; using insecure entropy
ioapic0: Changing APIC ID to 2
ioapic0 Version 1.1 irqs 0-16 on motherboard
module_register_init: MOD_LOAD (vesa, 0x80c38e80, 0) error 19
kbd0 at kbdmux0
acpi0: BHYVE BVXSDT on motherboard
atrtc0: AT realtime clock port 0x70-0x71,0x72-0x77 irq 8 on acpi0
Event timer RTC frequency 32768 Hz quality 0
Timecounter ACPI-fast frequency 3579545 Hz quality 900
acpi_timer0: 32-bit timer at 3.579545MHz port 0x408-0x40b on acpi0
pcib0: ACPI Host-PCI bridge port 0xcf8-0xcff on acpi0
pci0: ACPI PCI bus on pcib0
pcib0: no PRT entry for 0.31.INTA
virtio_pci0: VirtIO PCI Network adapter port 0x2000-0x201f at device 1.0 on 
pci0
vtnet0: VirtIO Networking Adapter on virtio_pci0
virtio_pci0: host features: 0x18020 Status,MrgRxBuf,MacAddress
virtio_pci0: negotiated features: 0x18020 Status,MrgRxBuf,MacAddress
vtnet0: Ethernet address: 00:a0:98:36:6d:e8
virtio_pci0: exhausted all interrupt allocation attempts
vtnet0: cannot setup virtqueue interrupts


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0x378
fault code  = supervisor read data, page not present
instruction pointer = 0x20:0x808894bb
stack pointer   = 0x28:0x818434b0
frame pointer   = 0x28:0x81843550
code segment= base 0x0, limit 0xf, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags= interrupt enabled, resume, IOPL = 0
current process = 0 (swapper)
[ thread pid 0 tid 10 ]
Stopped at  __rw_rlock+0x23b:   movl0x378(%r12),%eax
db bt
Tracing pid 0 tid 10 td 0x8150d370
__rw_rlock() at __rw_rlock+0x23b/frame 0x81843550
in_pcbpurgeif0() at in_pcbpurgeif0+0x30/frame 0x818435a0
in_ifdetach() at in_ifdetach+0x1c/frame 0x818435d0
if_detach() at if_detach+0x19b/frame 0x81843630
vtnet_attach() at vtnet_attach+0xb63/frame 0x81843760
device_attach() at device_attach+0x396/frame 0x818437b0
vtpci_probe_and_attach_child() at vtpci_probe_and_attach_child+0x91/frame 
0x818437f0
vtpci_attach() at vtpci_attach+0x23b/frame 0x81843830
device_attach() at device_attach+0x396/frame 0x81843880
bus_generic_attach() at bus_generic_attach+0x4a/frame 0x818438a0
acpi_pci_attach() at acpi_pci_attach+0x15f/frame 0x818438f0
device_attach() at device_attach+0x396/frame 0x81843940
bus_generic_attach() at bus_generic_attach+0x4a/frame 0x81843960
acpi_pcib_attach() at acpi_pcib_attach+0x24d/frame 0x818439b0
acpi_pcib_acpi_attach() at acpi_pcib_acpi_attach+0x299/frame 0x81843a00
device_attach() at device_attach+0x396/frame 0x81843a50
bus_generic_attach() at bus_generic_attach+0x4a/frame 0x81843a70
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: boot time crash in if_detach_internal()

2013-04-02 Thread Chris Torek
Not sure if the right answer is for drivers not to call ether_ifattach()
until the point-of-no-failure (lots of drivers are wrong then) or
initialize other parts earlier.

The other obvious method is to rearrange the sysinit priorities
(/sys/sys/kernel.h) so that all network domains are initialized
before invoking the device configuration code -- moving
SI_SUB_PROTO* before SI_CONFIGURE -- but presumably this idea was
tried and rejected earlier and hence the code in ether_ifattach
to check the same global variable.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


boot time crash in if_detach_internal()

2013-04-01 Thread Chris Torek
I have been poking about with the bhyve virtualization code in
FreeBSD 10-current, and managed to crash FreeBSD during its
bootstrap process due to the fact that if_detach is called
from boot time configuration code, before the internal domain
system initialization has happened.

I added the following patch to work around the problem.  As
the large comment notes, it might not be quite correct but it
does allow the boot to proceed (of course the dead network
device is soon a problem anyway...).

The fix mirrors (more or less) the code in if_attach_internal().
Feel free to accept, ignore, or modify the patch. :-)

Chris

diff --git a/sys/net/if.c b/sys/net/if.c
--- a/sys/net/if.c
+++ b/sys/net/if.c
@@ -845,6 +845,15 @@
 
if_purgeaddrs(ifp);
 
+   /*
+* torek: it's not entirely clear to me where and how this
+* should go, but if domain_init_status  2 then there should
+* be no inet, inet6, etc items, and this is where the crash
+* happens during boot, so let's try this:
+*/
+   if (domain_init_status  2)
+   return;
+
 #ifdef INET
in_ifdetach(ifp);
 #endif
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Ports: make fails, if DESTDIR path has spaces

2013-04-01 Thread Chris Rees
On 1 April 2013 17:13,  rank1see...@gmail.com wrote:
 Try DESTDIR='/usr/TZ\ ONE'; export DESTDIR.

 You need to escape the space.

 This is '#!/bin/sh' scripting.
 Doing so fixes *.mk, but breaks sh = dir simply doesn't exist anymore.

 Matthias  Chris = *.mk is at fault here, for not supporting FreeBSD's
 full range of chars for dir paths

As I explained before, Makefiles and sh have a strange relationship,
where escaping is similar but different in weird places.

Unless you are an expert, you should not use spaces in pathnames.
Stop doing it; the ports system is not designed around them.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Ports: make fails, if DESTDIR path has spaces

2013-03-31 Thread Chris Rees
On 31 March 2013 14:38,  rank1see...@gmail.com wrote:
 Under 9.0-RELEASE-p6

 --
 #!/bin/sh

 # Contains 9.0 world
 DESTDIR='/usr/TZ ONE';  export DESTDIR

 cd /usr/ports/benchmarks/unixbench
 /usr/bin/make showconfig
 --


 Errors:
 ---
 [: /usr/TZ: unexpected operator
 ===  Creating some important subdirectories
 [: /usr/TZ: unexpected operator
 === /tmp subdirectory has been successfully created
 [: /usr/TZ: unexpected operator
 === /dev subdirectory has been successfully created
 mktemp: mkdtemp failed on /usr/TZ: File exists
 === Failed to create temporary mount point
 *** Error code 9

 Stop in /usr/ports/benchmarks/unixbench.

Yeah, don't do that.

Spaces in directories will always cause problems; I suppose someone
could fix it in bsd.port.mk, but it will probably break somewhere
else; spaces in Make variables have a special meaning.

Use an underscore if you're desperate.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: snd_geode - where it is

2013-03-14 Thread Chris Rees
On 13 March 2013 13:56, Lowell Gilbert freebsd-li...@be-well.ilk.org wrote:
 Wojciech Puchar woj...@wojtek.tensor.gdynia.pl writes:

 i found it mentioned  on older mailing list archives. Unfortunately all
 links to source are dead.

 There is no such driver now in FreeBSD-9.

 Is there any reason that it was removed, or it wasn't commited at all?

 any place where i can get it?

 I'm pretty sure it was part of the base system at one time, but that was
 a long time ago. It's not like it would drop into a modern GIANT-free
 FreeBSD anyway. Try OSS from ports, would be my first guess.

Remember that if you install audio/oss, you will have to compile a
kernel without sound drivers built in.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: blocking teamviewer trojan

2013-02-21 Thread Chris Rees
On 21 Feb 2013 16:22, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:

 anyone have tested method on doing this?

 tried blocking port 5938 as well as teamviewer.com domain in squid (and
all port 80 traffic is already forwarded to squid).

 Still doesn't work.

 thanks for any help.

You would have better luck on a Windows security list.

This list is for development of FreeBSD.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: building select ports for packaging on install media

2013-02-11 Thread Chris Rees
Tinderbox will also work fine.  (Also, this belongs on ports@)

Chris
On 11 Feb 2013 19:31, Alexander Yerenkow yeren...@gmail.com wrote:

 Best way is to have poudriere :)

 Regards, Alexander Yerenkow
 11.02.2013 21:24 пользователь Kurt Lidl l...@pix.net написал:

  Greetings.
 
  I'm looking for a little guidance in building a small
  (one to two dozen) packages for inclusion on a locally
  generated install CDROM.
 
  (I'm doing this on for sparc64 machines, but I don't think
  that matters tremendously.)
 
  I have successfully generated bootable cd-rom media
  by doing:
 
  cd /usr/src/release
  make release
 
  After grinding around alot, I get a viable sparc64 bootable
  cdrom.
 
  What I'd like to do is augement that CD-ROM image with several
  binary packages, so I can just install them via 'sysinstall',
  rather than having to maintain a /usr/ports tree on every host
  and compile the same software again and again...
 
  I've found:
 
 
 http://www.freebsd.org/doc/en_US.ISO8859-1/articles/releng-packages/article.html
  and
  http://www.freebsd.org/doc/en/articles/portbuild/article.html
 
  But those seem to revolve around building *all* the ports.
  I just want to do a couple of dozen of them, but I'd like to
  end up with something that will generate binary packages that
  'pkg install' can deal with.
 
  Thanks for any tips.
 
  -Kurt
  ___
  freebsd-hackers@freebsd.org mailing list
  http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
  To unsubscribe, send any mail to 
 freebsd-hackers-unsubscr...@freebsd.org
 
 ___
 freebsd-hackers@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
 To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org

Re: ZFS regimen: scrub, scrub, scrub and scrub again.

2013-01-23 Thread Chris Rees
On 23 Jan 2013 20:23, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:

 While RAID-Z is already a king of bad performance,


 I don't believe RAID-Z is any worse than RAID5.  Do you have any actual
 measurements to back up your claim?


 it is clearly described even in ZFS papers. Both on reads and writes it
gives single drive random I/O performance.

So we have to take your word for it?

Provide a link if you're going to make assertions, or they're no more than
your own opinion.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: ZFS regimen: scrub, scrub, scrub and scrub again.

2013-01-23 Thread Chris Rees
On 23 January 2013 21:24, Wojciech Puchar
woj...@wojtek.tensor.gdynia.pl wrote:

 I've heard this same thing -- every vdev == 1 drive in performance. I've
 never seen any proof/papers on it though.

 read original ZFS papers.

No, you are making the assertion, provide a link.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: ZFS regimen: scrub, scrub, scrub and scrub again.

2013-01-23 Thread Chris Rees
On 23 Jan 2013 21:45, Michel Talon ta...@lpthe.jussieu.fr wrote:

 On Wed, 23 Jan 2013 14:26:43 -0600, Chris Rees utis...@gmail.com wrote:

 
  So we have to take your word for it?
  Provide a link if you're going to make assertions, or they're no more
  than
  your own opinion.

 I've heard this same thing -- every vdev == 1 drive in performance. I've
 never seen any proof/papers on it though.


 first google answer from request raids performance
 https://blogs.oracle.com/roch/entry/when_to_and_not_to

 Effectively,  as  a first approximation,  an  N-disk RAID-Z group will
 behave as   a single   device in  terms  of  deliveredrandom input
 IOPS. Thus  a 10-disk group of devices  each capable of 200-IOPS, will
 globally act as a 200-IOPS capable RAID-Z group.  This is the price to
 pay to achieve proper data  protection without  the 2X block  overhead
 associated with mirroring.

Thanks for the link, but I could have done that;  I am attempting to
explain to Wojciech that his habit of making bold assertions and
arrogantly refusing to back them up makes for frustrating reading.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: disadvantages of running 8.3 kernel on freebsd 8.2 system

2013-01-17 Thread Chris Rees
On 17 Jan 2013 12:43, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:


   - What are specific disadvantages that i can see clearly, running 8.3
   kernel on freebsd 8.2?


 no idea. just get latest -8 sources, compile world and kernel and install.

 all newest and in sync.

I agree;  very weird problems sometimes happen with out of sync
kernel/world; normally with new world old kernel, but the opposite is
possible.

Are you simply apprehensive over the time of buildworld?

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: disadvantages of running 8.3 kernel on freebsd 8.2 system

2013-01-17 Thread Chris Rees
On 17 Jan 2013 17:13, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:


 Are you simply apprehensive over the time of buildworld?


 no idea what you mean - my english isn't perfect.

Sorry, was asking the OP.

 I normally have latest binaries and generic kernel built for FreeBSD 8
which i use on servers (don't upgrade now as it works and there is no need
to).

 I have .tar.gz file with binaries, and compile custom kernel on given
machine but with same sys sources. so all always in sync.

 I once had strange out of sync problems - but with virtualbox kernel
module. recompiling this module solved all of them.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: cvs deprecated

2012-12-30 Thread Chris Rees
On 30 Dec 2012 12:58, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:

 do ports have to be updated this way or i can use portsnap as today? will
portsnap be continued or is too deprecated?

Portsnap is staying.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: cvs deprecated

2012-12-27 Thread Chris Rees
On 27 Dec 2012 09:47, Garrett Cooper yaneg...@gmail.com wrote:

 On Thu, Dec 27, 2012 at 1:18 AM, Wojciech Puchar
 woj...@wojtek.tensor.gdynia.pl wrote:
  but i can't find moron guide for using svn to update tree.
 
  I never used cvs or svn myself just want to
 
  1) get latest FreeBSD 9-* sources
  2) get latest HEAD sources.

 1. Checkout the sources:

 cd /usr/src # Change to something else if you don't want to checkout
 to /usr/src.
 svn co http://svn.freebsd.org/base/stable/9 . # stable/9
 svn co http://svn.freebsd.org/base/head . # CURRENT

I recall a cluster administrator advising use of svn protocol rather than
http.  Something to do with overheads.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: HEADS UP: FreeBSD git mirrors demoted to beta status, need your help

2012-12-15 Thread Chris Rees
On 15 December 2012 13:22, Ulrich Spörlein u...@freebsd.org wrote:
 Bad news everyone,

 tl;dr The git mirror of the source repository needs to be re-rolled to
 make the conversion deterministically repeatable, this will change
 pretty much all git commit hashes.

 The re-roll will be done January 15, 2013.

 Not affected are the ports and doc repositories, nor is the svn_head
 (for use with git-svn) affected.


 Background

 The converter (svn2git) was handing commits and objects to git's
 fast-import in arbitrary order, this makes merge commits have an
 arbitrary order of their parent commits and thus these merge commits
 have changing commit hashes for each converter run.

 This has been fixed, but requires us to move all the branches over to
 this deterministic scheme, which will change all their commit hashes.
 None of the contents of these commits change, so rebasing/remerging your
 work into these branches is possible without running into any merge
 conflicts.


 We need your help

 A goal of these conversions is to have them repeatable by you (yes,
 you!), so the correctness of the conversion can be verified. There are
 also no backups of the conversion runs, as they should be repeatable
 anyway.

 We need 2-3 volunteers to run these conversions themselves and verify
 that the produced commit hashes match the published ones. The necessary
 steps to do this are documented on the Wiki under

http://wiki.freebsd.org/GitWorkflow#History

 Please send me your output of git show-ref in a private mail, thanks.

Hey,

http://www.bayofrum.net/~crees/scratch/git-show-ref.txt

I hope it's what you were hoping for :)

My local svn mirror is synchronised at midnight GMT (UTC).

Need anything else?

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: old style kernel configuration

2012-11-22 Thread Chris Rees
On 22 Nov 2012 16:26, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:



 Value: ability to embed entire config (comments and all) into the kernel


 value 2: simplicity.


How is it simpler?

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: old style kernel configuration

2012-11-22 Thread Chris Rees
On 22 Nov 2012 18:56, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:


 How is it simpler?

 Chris



 strange question i thought it is obviously clear.

 $EDITOR config
 config kernel
 cd ../compile/kernel
 make depend
 make
 make install


 that's all. and no need to keep whole /usr/src, just sys

 what is wrong in it?

I was only confused because you had asserted that six commands was
simpler-- Now you point out that you don't keep a src tree, paired with
freebsd-update it is be a good combination for custom kernels :)

Thank you for clarifying.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: FreeBSD needs Git to ensure repo integrity [was: 2012 incident]

2012-11-19 Thread Chris Rees
On 19 Nov 2012 13:05, Andriy Gapon a...@freebsd.org wrote:

 on 18/11/2012 16:17 Chris Rees said the following:
  On 18 November 2012 14:04, Adrian Chadd adr...@freebsd.org wrote:
  On 18 November 2012 02:48, Andriy Gapon a...@freebsd.org wrote:
 
  What you describe is not a workflow issue, but a local development
  environment(s) setup issue.
 
  Which is a workflow issue.
 
  I mean, we could bang heads on semantics for hours on end, or we can
  realise that git isn't a magic bullet for FreeBSD development.
 
  Also... did I mention git is GPL?

 Yes, you did, twice now.
 I do not see any relevance of Git license to FreeBSD's choice of SCM
solution.
 But, of course, I am not:
 a) a zealot
 b) proposing Git (or Subversion, or Mercurial) to be bundled with a base
system

I'm disappointed that you choose to call me a zealot- this is a valid
concern, especially since we're trying to move away from GPL stuff.
Whether it's in base or not, you'd still be forcing me to install it for
development.  Why are people so dismissive of this point?

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: FreeBSD needs Git to ensure repo integrity [was: 2012 incident]

2012-11-18 Thread Chris Rees
On 18 November 2012 14:04, Adrian Chadd adr...@freebsd.org wrote:
 On 18 November 2012 02:48, Andriy Gapon a...@freebsd.org wrote:

 What you describe is not a workflow issue, but a local development
 environment(s) setup issue.

 Which is a workflow issue.

 I mean, we could bang heads on semantics for hours on end, or we can
 realise that git isn't a magic bullet for FreeBSD development.

Also... did I mention git is GPL?

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: FreeBSD needs Git to ensure repo integrity [was: 2012 incident]

2012-11-17 Thread Chris Rees
On 17 Nov 2012 21:00, Michael Ross g...@ross.cx wrote:

 On Sat, 17 Nov 2012 21:11:43 +0100, Ivan Klymenko fi...@ukr.net wrote:

 В Sat, 17 Nov 2012 15:00:06 -0500
 grarpamp grarp...@gmail.com пишет:

 http://www.freebsd.org/news/2012-compromise.html

http://it.slashdot.org/story/12/11/17/143219/freebsd-project-discloses-security-breach-via-stolen-ssh-key

 This is not about this incident, but about why major opensource
 projects need to be using a repository that has traceable, verifiable,
 built-in cryptographic authentication.


 LOL And how will this help Linux?
 http://lwn.net/Articles/457142/


 In the first comment on the article you link to, you find this:


http://www.linux.com/news/featured-blogs/171-jonathan-corbet/491001-the-cracking-of-kernelorg

 where the OPs view is susbstantiated.

Yes, but git doesn't work with our workflow.  It's been discussed several
times, and changing to a tool that doesn't work for us (and is GPL btw) is
no good at all.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org

Re: Ports cross-compiling

2012-11-14 Thread Chris Rees
On 5 September 2012 07:40, Alexander Yerenkow yeren...@gmail.com wrote:
 Hello all.
 I'm currently trying to make poudriere cross compile some ports for arm
 arch.
 I'm using such command (as example):

 env TARGET=arm TARGET_ARCH=armv6 TARGET_CPUARCH=armv6
 PATH=/usr/obj/arm.armv6/usr/src/tmp/usr/bin:${PATH} CONFIGURE_HOST=amd64
 STRIP_CMD=true make PKGNAMESUFFIX=-arm WRKDIRPREFIX=/tmp WITHOUT_CHECKS=yes
 LOCALBASE=/usr/localarm6 -C /usr/ports/A/B package

 I have problem:
 1. When I run this command from chrooted env, all builds fine; configure
 script make check if we cross-compiling, somehow it gets no and all goes
 well.
 2. When this command run by poudriere in jail, all fails as here:

 checking whether we are cross compiling... configure: error: in
 `/tmp/usr/ports/devel/pcre/work/pcre-8.31':
 configure: error: cannot run C compiled programs.
 If you meant to cross compile, use `--host'.
 See `config.log' for more details
 === Script configure failed unexpectedly.


 So, I'd like to propose small changes to ports/Mk/bsd.port.mk

 SET_LATE_CONFIGURE_ARGS= \

 .
 if [ ! -z ${CONFIGURE_HOST} ]; then \
 _LATE_CONFIGURE_ARGS=$${_LATE_CONFIGURE_ARGS} --host=${CONFIGURE_HOST} ; \
 fi ; \
 .


 And after this, builds going just fine (of course for ports that can be
 cross-built).

 Is it possible to accept this? Seems to me non-destructive patch.

Hi Alexander,

Sorry for the late reply.  You're definitely better off discussing
this on ports@-- there has been some work done on cross-building
ports, but it's fairly early days yet; the only real cross that can be
done with much degree of success is i386 on amd64 currently.

Your patch doesn't look destructive, but it could be better integrated
with TARGET_ARCH setting.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program

2012-10-28 Thread Chris Rees
On 27 October 2012 22:10, Simon J. Gerraty s...@juniper.net wrote:

 On Sat, 27 Oct 2012 19:53:56 +0100, Chris Rees writes:
I'm saying that it's unacceptable to expect people to change their
systems just to make the ports tree work after we have broken it on a
supposedly supported version.

 But there's no suggestion of that.
 The ports tree would take care of itself.

 The comment about fixing makefiles refered to the concern about things
 outside of base/ports.

From your comment, I now understand that we aren't having the same
conversation :)

Please answer me these to check we're on the same page:

Are we planning to replace /usr/bin/make with bmake in the near future?

If yes, what changes are we going to make to the ports tree to ensure
that -CURRENT can still use it?

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program

2012-10-28 Thread Chris Rees
On 28 October 2012 19:11, Simon J. Gerraty s...@juniper.net wrote:

 On Sun, 28 Oct 2012 14:06:41 +, Chris Rees writes:
Are we planning to replace /usr/bin/make with bmake in the near future?

 That was what I heard, but any such move is dependent on dealing with
 ports.  The ~sjg/ports2bmake.tar.gz on freefall is the plan I came up
 with after the above requirement was introduced at last BSDCan.

If yes, what changes are we going to make to the ports tree to ensure
that -CURRENT can still use it?

 If you mean -current (aka head); the plan is to convert ports to bmake
 syntax wrt to the 2 conflicting modifiers.  At my last test there are
 just under 300 makefiles in ports that use the old modifiers.

 Now for  head (ie. /usr/bin/make is an old version), the above ports
 tree detects that bmake is not being used, and invokes a shell script
 (bmake-sh) to do what was asked.

 That script will look for bmake and if necessary build/install it.
 To do that, it creates a temp copy of Mk/*.mk converted back to the old
 syntax so that the old make can build and install bmake, and then the
 old system is on par with -current.

 That's what I meant by ports will take care of itself.
 The main gap btw in the above, is if a user who does not have privs to
 install bmake, is the only person trying to do something with ports.

 The above plan needs to be approved by portmgr, and obviouslty a test
 run of building all ports is needed (possibly more than one).

 Does that help?

Certainly, thank you.

I didn't find it clear when inspecting the tarball (obviously I hadn't
read the README clearly enough).

I'm going to have to echo John's non-obvious comment however, and I
think it would be very helpful to have a clear public writeup, perhaps
QA style to allay any other such fears.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program

2012-10-27 Thread Chris Rees
On 27 Oct 2012 00:35, Simon J. Gerraty s...@juniper.net wrote:


 On Fri, 26 Oct 2012 22:02:00 +0100, Chris Rees writes:
 In that case we have a switch time on the order of years, not weeks; 8.3
is
 supported until May '14, and unless we get a :tl etc MFC into 8, even
 longer.  All this time the ports tree must work with pmake.

 I'm pretty sure I was told it is already in 8 and 7

Not in 8.3 at least:

svnweb.freebsd.org/base/releng/8.3/usr.bin/make/var.c?view=log

 I don't want to discourage you or belittle your excellent work here, but
 Marcel made me very nervous with his comment on the process being a few
 weeks.

 That was based on discussions at the last devsummit.

These discussions need backing up with a real roadmap, including detail on
exactly what 8.3 and 7.4 users will have to do to ensure that the ports
tree still works.

I don't see where these considerations have been made.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program

2012-10-27 Thread Chris Rees
[trim CC list a little to stop people regretting replying to this thread]

On 27 October 2012 10:15, Chris Rees utis...@gmail.com wrote:

 On 27 Oct 2012 00:35, Simon J. Gerraty s...@juniper.net wrote:


 On Fri, 26 Oct 2012 22:02:00 +0100, Chris Rees writes:
 In that case we have a switch time on the order of years, not weeks; 8.3
  is
 supported until May '14, and unless we get a :tl etc MFC into 8, even
 longer.  All this time the ports tree must work with pmake.

 I'm pretty sure I was told it is already in 8 and 7

 Not in 8.3 at least:

 svnweb.freebsd.org/base/releng/8.3/usr.bin/make/var.c?view=log

 I don't want to discourage you or belittle your excellent work here, but
 Marcel made me very nervous with his comment on the process being a few
 weeks.

 That was based on discussions at the last devsummit.

 These discussions need backing up with a real roadmap, including detail on
 exactly what 8.3 and 7.4 users will have to do to ensure that the ports tree
 still works.

 I don't see where these considerations have been made.

OK, so how about this.

We (ab)use the security update mechanism to merge the pmake changes
(:tl and :tu) into releng/7.4 and releng/8.3 (possibly the earlier
releng branches such as 7.3, 8.2, 9.0).  We could then send out a
message on ports-announce, giving a few weeks' notice that the change
to bsd.port.mk is going through and that users need the latest
'security' patches.

When we change bsd.port.mk over, include a snippet such as the one at
[1], which gives more informative error text and refers user to
documentation.

Although I still think this is less than ideal, it is the only way I
can see that we can switch before May '14, if the urgency is there.

Chris

[1] http://www.bayofrum.net/~crees/patches/bmake-pmake.diff
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program

2012-10-27 Thread Chris Rees
On 27 October 2012 15:32, Bryan Drewery br...@shatow.net wrote:
 On 10/27/2012 8:23 AM, Chris Rees wrote:
 [trim CC list a little to stop people regretting replying to this thread]

 On 27 October 2012 10:15, Chris Rees utis...@gmail.com wrote:

 On 27 Oct 2012 00:35, Simon J. Gerraty s...@juniper.net wrote:


 On Fri, 26 Oct 2012 22:02:00 +0100, Chris Rees writes:
 In that case we have a switch time on the order of years, not weeks; 8.3
 is
 supported until May '14, and unless we get a :tl etc MFC into 8, even
 longer.  All this time the ports tree must work with pmake.

 I'm pretty sure I was told it is already in 8 and 7

 Not in 8.3 at least:

 svnweb.freebsd.org/base/releng/8.3/usr.bin/make/var.c?view=log

 I don't want to discourage you or belittle your excellent work here, but
 Marcel made me very nervous with his comment on the process being a few
 weeks.

 That was based on discussions at the last devsummit.

 These discussions need backing up with a real roadmap, including detail on
 exactly what 8.3 and 7.4 users will have to do to ensure that the ports tree
 still works.

 I don't see where these considerations have been made.

 OK, so how about this.

 We (ab)use the security update mechanism to merge the pmake changes
 (:tl and :tu) into releng/7.4 and releng/8.3 (possibly the earlier
 releng branches such as 7.3, 8.2, 9.0).  We could then send out a
 message on ports-announce, giving a few weeks' notice that the change
 to bsd.port.mk is going through and that users need the latest
 'security' patches.

 This weeks is making a assumptions that users 1. reads ports@ or 2.
 Update to security/errata patches in a timely manner or 3. Read UPDATING

Quite.  This should be at least a few months, otherwise we're making
unreasonable requests of our users, and yet again annoy them by
breaking older versions-- this time with no real benefit for
end-users.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Call for review -- rc needs some love!

2012-10-27 Thread Chris Rees
Hi all,

I've tried to have a look at some of the lingering issues in our rc
[1] as well as kick up some discussion over some other patches, but
looking over the archives of the list it seems that no-one is
maintaining it or reviewing patches.

Because of this, I'm having a hard time working out how to get any of
my patches in!

Please would someone with a src bit review some of my fixes [2]?  I'm
working on a few more, but I would need approval for anything
committed.

Thanks!

Chris

[1] http://www.freebsd.org/cgi/query-pr-summary.cgi?responsible=freebsd-rc

[2] http://lists.freebsd.org/pipermail/freebsd-rc/2012-October/thread.html
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program

2012-10-27 Thread Chris Rees
On 27 October 2012 18:27, Simon J. Gerraty s...@juniper.net wrote:
These discussions need backing up with a real roadmap, including detail on
exactly what 8.3 and 7.4 users will have to do to ensure that the ports
tree still works.

 I've tested the ports tree converted to bmake - per the patch I
 mentioned on a 7.1 box.  It worked for me.  Once the ports tree has
 found or installed bmake, the system version makes no further
 difference.

 Obviously not a conclusive result, but yes this issue has been given
 consideration.

What about these?

[crees@pegasus]~% grep -n :\[LU] /usr/ports/Mk/bsd.port.mk | tee
/dev/tty | wc -l
1324:PORTVERSION=
${DISTVERSION:L:C/([a-z])[a-z]+/\1/g:C/([0-9])([a-z])/\1.\2/g:C/:(.)/\1/g:C/[^a-z0-9+]+/./g}
1451:.if (defined(USE_QT_VER)  ${USE_QT_VER:L} == 3) ||
defined(USE_KDELIBS_VER) || defined(USE_KDEBASE_VER)
1455:.if defined(USE_QT_VER)  ${USE_QT_VER:L} == 4 || defined(USE_QT4)
1674:.if ${USE_PKGCONFIG:L} == yes || ${USE_PKGCONFIG:L} == build
1677:.elif ${USE_PKGCONFIG:L} == both
1681:.elif ${USE_PKGCONFIG:L} == run
1696:${b}=  ${LOCALBASE}/bin/${b:C/PP/++/:L}
1763:_USE_OPENAL+= ${_OPENAL_${_OPENAL_SYSTEM:U}}
1783:_USE_OPENAL+=  ${_OPENAL_${component:U}}
1829:.if defined(FAM_SYSTEM_${FAM_SYSTEM:U})
1830:LIB_DEPENDS+=  ${FAM_SYSTEM_${FAM_SYSTEM:U}}
1836:.if defined(USE_RC_SUBR)  ${USE_RC_SUBR:U} != YES
1844:.if defined(USE_LDCONFIG)  ${USE_LDCONFIG:L} == yes
1847:.if defined(USE_LDCONFIG32)  ${USE_LDCONFIG32:L} == yes
1856:.  if ${USE_GETTEXT:L} == build
1858:.  elif ${USE_GETTEXT:L} == run
1860:.  elif ${USE_GETTEXT:L} == yes
1888:.  if ${USE_LINUX:L} == yes
1899:.  if ${USE_LINUX:L} == yes
1977:. if ${USE_GL:L} == yes
1994:. if ${USE_BISON:L} == build
1996:. elif ${USE_BISON:L} == run
1998:. elif ${USE_BISON:L} == both
2044:.if defined(USE_QT_VER)  ${USE_QT_VER:L} == 4 || defined(USE_QT4)
3038:_MANPAGES+=
${MAN${sect}:S%^%${MAN${sect}PREFIX}/${manlang}/man${sect:L}/%}
3043:.if defined(MAN${sect}_${manlang:S%^man/%%:U})
3044:_MANPAGES+=
${MAN${sect}_${manlang:S%^man/%%:U}:S%^%${MAN${sect}PREFIX}/${manlang}/man${sect:L}/%}
3056:_MANPAGES+=
${MAN${sect}_EN:S%^%${MAN${sect}PREFIX}/man/man${sect:L}/%}
3312:   || defined(CONFIG_DONE_${UNIQUENAME:U}) || \
3600:.if ${USE_DOS2UNIX:U}==YES
4361:${target}: ${${target:U}_COOKIE}
4364:   @cd ${.CURDIR}  ${MAKE} CONFIG_DONE_${UNIQUENAME:U}=1
${${target:U}_COOKIE}
4368:.if !exists(${${target:U}_COOKIE})
4370:.if ${UID} != 0  defined(_${target:U}_SUSEQ)  !defined(INSTALL_AS_USER)
4372:${${target:U}_COOKIE}: ${_${target:U}_DEP}
4373:   @cd ${.CURDIR}  ${MAKE} ${_${target:U}_SEQ}
4375:${${target:U}_COOKIE}: ${_${target:U}_DEP} ${_${target:U}_SEQ}
4379:   ${SU_CMD} ${MAKE} ${_${target:U}_SUSEQ}
4383:${${target:U}_COOKIE}: ${_${target:U}_DEP}
4385:   ${MAKE} ${_${target:U}_SEQ} ${_${target:U}_SUSEQ}
4388:${${target:U}_COOKIE}: ${_${target:U}_DEP} ${_${target:U}_SEQ}
${_${target:U}_SUSEQ}
4393:${${target:U}_COOKIE}::
4802:   for alg in ${CHECKSUM_ALGORITHMS:U}; do \
4825:   for alg in ${CHECKSUM_ALGORITHMS:U}; do \
4836:   for alg in ${CHECKSUM_ALGORITHMS:U}; do \
4850:   for alg in ${CHECKSUM_ALGORITHMS:U}; do \
4904:   for alg in ${CHECKSUM_ALGORITHMS:U}; do \
5032:${deptype:L}-depends:
5653:${i:S/-//:U}=  ${WRKDIR}/${SUB_FILES:M${i}*}
5700:.if defined(PLIST_REINPLACE_${reinplace:U})
5701:   @${SED} -i  -e '${PLIST_REINPLACE_${reinplace:U}}' ${TMPPLIST}
5854:.if defined(USE_RCORDER) || defined(USE_RC_SUBR) 
${USE_RC_SUBR:U} != YES
5864:.if defined(USE_RC_SUBR)  ${USE_RC_SUBR:U} != YES
  53
[crees@pegasus]~%

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program

2012-10-27 Thread Chris Rees
On 27 October 2012 19:52, Simon J. Gerraty s...@juniper.net wrote:

 On Sat, 27 Oct 2012 14:23:29 +0100, Chris Rees writes:
We (ab)use the security update mechanism to merge the pmake changes
(:tl and :tu) into releng/7.4 and releng/8.3 (possibly the earlier

 I originally provided the :tl and :tu patch for something like that
 (not planning any abuse mind ;-)

 But, if portmgr test my patch and find it works ok (for some value of
 ok) for older releases, this probably isn't necessary?

 It may still be useful though to provide an updated fmake via ports,
 which could make it easier for folk to migrate other code bases.
 The sed script to be applied to makefiles is trivial btw:

 $ cat f2bmake.sed
 /$.*:[UL][:)}]/ { s,:L,:tl,g;s,:U,:tu,g; }
 $

I know the fix is trivial :)

I'm saying that it's unacceptable to expect people to change their
systems just to make the ports tree work after we have broken it on a
supposedly supported version.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Porting patch(1) from NetBSD to FreeBSD (was Re: FreeBSD in Google Code-In 2012? You can help too!)

2012-10-27 Thread Chris Rees
On 27 October 2012 22:17, hiren panchasara hiren.panchas...@gmail.com wrote:
 [removing the CC list]

 On Wed, Oct 24, 2012 at 3:36 PM, Pedro Giffuni p...@freebsd.org wrote:

 (cc'ing -ports and cutting most of the rest)

  From: Eitan Adler
 .
 On 24 October 2012 13:24, Fernando Apesteguía wrote:
  Also related to that, what about writing a section about redports[1]
  in the porter's handbook[2]?
 
 This is a good documentation task... but we need more *coding* tasks as
 well.
 

 We do need to port and test patch (1) from NetBSD or DragonFly to replace
 GNU patch, and this shouldn't be difficult.


 Hi Pedro / List,

 I am not part of google summer of code but I've tried to port patch(1) from
 NetBSD into FreeBSD head. I hope that is okay.

 Patching was trivial and It _seems_ to be working fine.

 I would appreciate any ideas around how to test the changes and how to
 proceed further.

Have you a patch :)?  You're right, there shouldn't have been many
changes needed.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program

2012-10-26 Thread Chris Rees
On 26 Oct 2012 06:01, Konstantin Belousov kostik...@gmail.com wrote:

 On Thu, Oct 25, 2012 at 03:53:53PM -0700, Simon J. Gerraty wrote:
 
  On Thu, 25 Oct 2012 23:01:27 +0100, Chris Rees writes:
  Is there a Wiki page where the actual benefits of moving to bmake are
  made clear?  This is a major, *major* upheaval, and having two
  versions of bsd.port.mk for years is simply not an option.
 
  There is no need/plan for two versions of bsd.port.mk, the patch I just
  mentioned, deals with older systems by detecting that bmake was not
  used, and using it (installing if need be).
 
  Have you discussed this on ports@?
 
  I have not at least.
  This was discussed at the last couple of BSDCan's and dev summits.
 
  The original plan discussed at BSDCan a couple of years ago, was to
  allow bmake and the old make to cooexist for some time so that ports
  could continue to use the old make.
 
  At the last BSDCan we were told that wasn't an option - hence the patch
  to ports that was mentioned.
 
  FWIW the changes to 99.9% of the ports tree are trivial (:L - :tl etc).
  The only interesting changes are to bsd.port.mk (the diff other than the
  above is 54 lines) they cover 2 things - dealing with old make as
  mentioned above, and man pages.  The nested .for loops that deal
  with MLINKS are replaced with one line - this was safer that attempting
  to hack those .for loops to work with both makes.

 I am watching the serial for some time.  Could please, someone, describe
 why bmake cannot grow the compat features to be a drop-in replacement for
 FreeBSD make, instead of patching all the trees ?

 In particular, why cannot the ':L' and ':U' support be added ?

:U is already used by bmake for something else- I can't remember what, but
I checked the man page last night :(

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program

2012-10-26 Thread Chris Rees
On 26 Oct 2012 19:12, David O'Brien obr...@freebsd.org wrote:

 On Fri, Oct 26, 2012 at 12:12:44AM +0200, Baptiste Daroussin wrote:
  Do be able to get the ports tree working with bmake asap, I also asked
  him to MFC it to 9.1, from latest reply he got positive answer from re@
  about this, but was waiting for something I don't remember.

 :tu/:tl is in releng/9.1, so it will also be in 9.1-RELEASE.

Then we only have two supported stable branches you propose to break...

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program

2012-10-26 Thread Chris Rees
On 26 Oct 2012 20:15, Chris Rees utis...@gmail.com wrote:


 On 26 Oct 2012 19:12, David O'Brien obr...@freebsd.org wrote:
 
  On Fri, Oct 26, 2012 at 12:12:44AM +0200, Baptiste Daroussin wrote:
   Do be able to get the ports tree working with bmake asap, I also asked
   him to MFC it to 9.1, from latest reply he got positive answer from
re@
   about this, but was waiting for something I don't remember.
 
  :tu/:tl is in releng/9.1, so it will also be in 9.1-RELEASE.

 Then we only have two supported stable branches you propose to break...


OK, how about this:

:L -- seems that bmake's use for this is kinda pointless; returning the
name of the variable; we could swap that usage over directly.

:U -- with bmake has non-optional arguments, so for example:

${VAR:U} - pmake behaviour

${VAR:Uval} - make behaviour.

Would that be acceptable?  I can get a patch in if that's popular.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program

2012-10-26 Thread Chris Rees
On 26 Oct 2012 21:51, Simon J. Gerraty s...@juniper.net wrote:


 On Fri, 26 Oct 2012 21:00:26 +0100, Chris Rees writes:
 :L -- seems that bmake's use for this is kinda pointless; returning the
 name of the variable; we could swap that usage over directly.

 Acutally it is very useful.
 The debugging facilities in dirdeps.mk rely on it.
 The junos build uses it in many other places too.


 :U -- with bmake has non-optional arguments, so for example:
 
 ${VAR:U} - pmake behaviour
 
 ${VAR:Uval} - make behaviour.
 
 Would that be acceptable?  I can get a patch in if that's popular.

 No, please don't do that.
 I'm trying to reduce the divergence b/w freebsd and netbsd.

In that case we have a switch time on the order of years, not weeks; 8.3 is
supported until May '14, and unless we get a :tl etc MFC into 8, even
longer.  All this time the ports tree must work with pmake.

I don't want to discourage you or belittle your excellent work here, but
Marcel made me very nervous with his comment on the process being a few
weeks.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program

2012-10-25 Thread Chris Rees
On 25 October 2012 22:15, David O'Brien obr...@freebsd.org wrote:
 On Mon, Oct 08, 2012 at 09:11:29AM -0700, Marcel Moolenaar wrote:
 two independent efforts (ATF  bmake) and there was no indication that
 one would be greatly benefitted from the other. At least not to the
 point of creating a dependency.

 It seems we do have the situation where folks feel there is a dependency
 between the two.

 Before we can switch permanently to bmake, we need to do the following
 first:
 1.  Request an EXP ports build with bmake as make(1). This should tell
 us the damage of switching to bmake for ports.
 2.  In parallel with 1: build www  docs with bmake and assess the
 damage
 3.  Fix all the damage

 It could be a while (many weeks) before we get to 4, so the question

 Given the time this will take, I feel we need to add another knob to the
 Bmake build so that 'make world' gives one both the FreeBSD make as
 /usr/bin/make and Bmake as /usr/bin/bmake.


We really aren't going to have any luck yet...

[crees@pegasus]/usr/ports% sudo make MAKE=/usr/bin/bmake index | head
Generating INDEX-9 - please wait..bmake: /usr/ports/Mk/bsd.port.mk
line 5127: warning: duplicate script for target -depends ignored
bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous
script for -depends defined here
bmake: /usr/ports/Mk/bsd.port.mk line 5127: warning: duplicate
script for target -depends ignored
bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous
script for -depends defined here
bmake: /usr/ports/Mk/bsd.port.mk line 5127: warning: duplicate
script for target -depends ignored
bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous
script for -depends defined here
bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: duplicate
script for target -depends ignored
bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous
script for -depends defined here
bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: duplicate
script for target -depends ignored
bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous
script for -depends defined here

Looks like a few missing .if !target s, but the breakage is pretty big
even for simple things :/

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program

2012-10-25 Thread Chris Rees
On 25 October 2012 22:32, Garrett Cooper yaneg...@gmail.com wrote:
 On Thu, Oct 25, 2012 at 2:23 PM, Marcel Moolenaar mar...@xcllnt.net wrote:

 ...

 I think there are 2 reasons why not to:

 1.  The people working on ATF have not raised this concern and
 have expressed that using the WITH_BMAKE knob is but a small
 price to pay. So let's work the bmake side and be able to
 get rid of the knob as soon as possible.

 It is annoying with the magnitude of build-related errors, but I have
 a workaround.

 2.  More knobs isn't better -- we must have none of the knobs in
 the end, so the more we create, the more work we have to get
 rid of them. That's just more work spent not focusing on the
 task at hand and thus more time wasted.

 Yes, but not being able to update one's machine makes me sad panda.

 In short: this isn't a 2-knob problem by any stretch of the
 imagination.

 The real issue is that I need to take the patch Simon developed, run
 with it, and in parallel he needs to -- and hopefully already is --
 engage portmgr to get it through a number of exp- runs to make sure
 bmake does what it's supposed to do with his patch. Backwards
 compatibility will need to be maintained for ports because ports has
 to work on multiple versions of FreeBSD [where bmake isn't yet
 available/present], so maybe a fork in the road for bsd.port.mk should
 be devised in order to make everything work.

Now you've terrified me, and probably most other ports people too.

Is there a Wiki page where the actual benefits of moving to bmake are
made clear?  This is a major, *major* upheaval, and having two
versions of bsd.port.mk for years is simply not an option.

Have you discussed this on ports@?

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Providing a default graphical environment on FreeBSD

2012-09-18 Thread Chris Rees
Can you perhaps read the whole thread and organise your thoughts into just
one email?

Chris
On 18 Sep 2012 09:09, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:

 To be succinct: this is not OSX/Windows. True Unix and Unix clones can
 be decoupled from a desktop environment enough that forcing everyone
 to have one choice for desktop user experience doesn't make sense, and
 the fact that there isn't a common GUI development toolkit (GTK, QT,
 etc) encourages fragmentation of effort further (I think it's called
 the Bazaar model of development :P).


 That's all true. But do anyone understand why there is still so much
 pressure for every open source OS and specifically *BSD on default desktop
 environment or similar ideas?

 Such a pressure exist for 20 years at least, and - between other results -
 started demise of linux as trusty high performance system.

 Why people still not learned from faults and keep harming good open source
 projects that way?
 __**_
 freebsd-hackers@freebsd.org mailing list
 http://lists.freebsd.org/**mailman/listinfo/freebsd-**hackershttp://lists.freebsd.org/mailman/listinfo/freebsd-hackers
 To unsubscribe, send any mail to freebsd-hackers-unsubscribe@**
 freebsd.org freebsd-hackers-unsubscr...@freebsd.org


___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Providing a default graphical environment on FreeBSD

2012-09-18 Thread Chris Rees
On 18 Sep 2012 09:41, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:

 desktop environment or similar ideas?


 Tell you what:

 When you have at least 75% of the user population of FreeBSD agreeing
 on which window manager we should offer as the default, we can talk
 about this.


 so if 76% would decide that FreeBSD should have KDE included in system -
it means that it should?

No.  Read the thread properly.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: gpart is junk

2012-09-17 Thread Chris Rees
On 17 Sep 2012 12:58, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:

 IMHO, gpart and GEOM are fantastic.


 anyway it is MUCH easier and faster to edit disklabels with bsdlabel -e
than with gpart.

 Unfortunately since some time bsdlabel cannot edit labels if ANY of
partitions are open.

You may be more familiar with manually editing slice tables, but it is very
easy to mess up.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Providing a default graphical environment on FreeBSD

2012-09-17 Thread Chris Rees
On 17 Sep 2012 17:22, Tom Evans tevans...@googlemail.com wrote:

 On Mon, Sep 17, 2012 at 5:00 PM, Zhihao Yuan lich...@gmail.com wrote:
  I definitely agree with this. Sun has a book, UNIX Essentials
  featuring the Solaris..., and GUI takes a big part in the book. A
  default GUI is essential to a modern UNIX. FreeBSD can no longer
  regard GUI as a third-party bonus.

 This is according to *your* use cases though. There are many of us who
 do not put X - or any graphical environment - on our FreeBSD servers.

 If FreeBSD did not regard a GUI as an optional 3rd party component,
 that would mean bringing Xorg, and a specified default WM into base -
 potentially even dbus and hald as well. IMO that would be a waste of
 time and resources, as both Xorg and most WM have rapid development
 changes - just look at how many issues are brought up on x11@ when
 there are new upgrades of Xorg available.

 As well as this, Xorg versions would have to remain relatively stable
 during minor releases, meaning if you DO want X11, then you are being
 hamstrung by requiring it in base.

 Status quo for me please.

Time and time again, this comes up.

Being official does not mean it should be in base.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Why fdisk can't open root disk with MBR for writing?

2012-09-16 Thread Chris Rees
On 16 September 2012 10:23, Julian H. Stacey j...@berklix.com wrote:
 This is a resend as Benjamin Kaduk ka...@mit.edu
 dropped the
 Yuri y...@rawbw.com
 from CC line,  Yuri was the original first poster in thread who
 my patch would presumably have helped.

 Reference:
 From: Benjamin Kaduk ka...@mit.edu
 Date: Sat, 15 Sep 2012 14:49:41 -0400 (EDT)
 Message-id:   alpine.gso.1.10.1209151448200@multics.mit.edu

 Benjamin Kaduk wrote:
 On Sat, 15 Sep 2012, Julian H. Stacey wrote:

  I have been applying this diff to my man fdisk:
 
  http://berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/gen/sbin/fdisk/
 
  *** 8.0-RELEASE/src/sbin/fdisk/fdisk.8  Sat Mar 14 22:32:16 2009
  --- new-generic/src/sbin/fdisk/fdisk.8  Sat Mar 14 22:35:10 2009
  ***
  *** 462,464 
  --- 462,468 
   The
   .Xr bsdlabel 8
   command must be used for this.
  + .Pp
  + When running multi user, you cannot write unless you first run this:
  + .br
  + sysctl kern.geom.debugflags=16
 
  I never submitted it as a send-pr,
  anyone think I should submit it to help save people ?

 We have had a long discussion about kern.geom.debugflags starting here:
 http://lists.freebsd.org/pipermail/freebsd-current/2011-October/028090.html
 My understanding from that discussion is that your patch should not be
 accepted.

 Got to travel now, will read that thread later,

 A shame if we would leave fdisk crippled undocumented, when so easy
 to doc. the solution. Fdisk is known across many OSs Unix  beyond,
 our BSD tools may be better nicer, but fdisk it what many will first
 reach for,
  a shame not to help newer people  visitors used to fdisk.
 We inside FreeBSD crippled fdisk by adding these flags. Outsiders
 wont expect the weirdness  get impression FreeBSD has a bug.

I think you have a point, but at the moment fdisk really doesn't work
properly at all; I find so many people complaining on IRC about Why
doesn't fdisk work?

To be honest, I'd be happy with replacing fdisk with a huge warning
USE GPART!!; fdisk isn't really standard anyway.

The alternative of course is to fix fdisk...  *properly*.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: [RFC] Add *.orig/*.rej to svn:ignore in src?

2012-09-16 Thread Chris Rees
On 16 September 2012 10:11, Garrett Cooper yaneg...@gmail.com wrote:
 I noticed that we have a handful of patterns currently ignored in 
 svn:ignore (at least at the top-level… the lower levels don't appear to be 
 set in any particular manner):

 $ svn propget svn:ignore
 _.tinderbox.*
 _.amd64.*
 _.arm.*
 _.i386.*
 _.ia64.*
 _.mips.*
 _.pc98.*
 _.powerpc.*
 _.sparc64.*
 _.sun4v.*

 $ svn info | grep URL:
 URL: http://svn.freebsd.org/base/head

 I was wondering if *.orig and *.rej should be added to the list as 
 well to avoid checking in patch `artifact` files?

Do you have an example where this has happened?

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: FreeBSD Kernel Internals, lecture video

2012-08-26 Thread Chris Rees
On 26 Aug 2012 13:15, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:

 very expensive..

Training and education generally are.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: port devel/doxygen failing to test on -CURRENT and -STABLE

2012-08-11 Thread Chris Rees
On 12 July 2012 18:52, Chris Rees utis...@gmail.com wrote:
 On 9 July 2012 02:49, David Xu davi...@freebsd.org wrote:
 On 2012/07/08 18:21, Chris Rees wrote:

 Hi all / David,

 doxygen has been failing for a while now on -CURRENT and apparently
 -STABLE too.  The current fix is disabling one of the tests in the
 build, but obviously it points to a problem with our base system

 I've trussed [1] the failing code [2], and it looks as though it's
 hanging on a _umtx call.  I'm gratuitously ignorant of what goes on
 there... but the timings of recent commits to umtx.h [3] could
 indicate a link (hope it's not bogus...).

 Any pointers on what I should do next?

 Chris

 [1] http://www.bayofrum.net/~crees/scratch/doxygen-truss

 _umtx_op(0x8012b0280,0x16,0x0,0x0,0x0,0x1)   ERR#22 'Invalid argument'

 can you execute it in gdb and print its value ?

 print/x *(int *)0x8012b0280
 print/x *(int *)(0x8012b0280+4)

 I've been having trouble debugging it since it's threaded, and so I
 ran a binary search over the last few days of revisions from 1/Apr to
 1/May.

 Unfortunately I discovered to my horror today that all but the first
 test was useless, because the patch I committed to disable the test
 was of course readded to my ports tree, so none of the tests ran :/

 I'll hopefully have it narrowed down to the offending commit over the
 next few days.

Extremely late now, I know; these tests take ages, and I've been
dragged away on holiday.

I've narrowed it down to cvsup checkout between 00:00 6/April and
12:00 6/April, which I assume corresponds to r233932-r233945.

The only major change that I could see is melifaro's bpf changes in
r233937; not sure why this test could use bpf.

I'll keep digging, and I may owe you (David) an apology ;)



Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Published paper

2012-08-07 Thread Chris Nehren
On Tue, Aug 07, 2012 at 17:24:37 -0700 , ali mousa wrote:
 Hi,
 
 I have recently published a paper related to Operating System, can you
 please check it out http://www.ijcsi.org/papers/IJCSI-9-4-1-77-84.pdf
 and tell me your opinion, looking forward to hear from you ?

The DragonFlyBSD project may also find this interesting, because AFAIR
one of the reasons Matt forked FreeBSD was due to disagreements in the
SMP implementation. 

-- 
Thanks and best regards,
Chris Nehren


pgpf2mA8W83Os.pgp
Description: PGP signature


Re: FreeBSD ZFS source

2012-08-02 Thread Chris Nehren
On Thu, Aug 02, 2012 at 22:48:50 +0200 , Fredrik wrote:
 Hello,
 
 Excuse me for this newb question but exactly where are the current ZFS
 files located? I have been looking at the CVS on freebsd.org under
 /src/contrib/opensolaris/ but that does not seem to be the current
 ones. Is this correct?

$ find /usr/src -type d -iname zfs
/usr/src/cddl/contrib/opensolaris/cmd/zfs
/usr/src/cddl/sbin/zfs
/usr/src/lib/libprocstat/zfs
/usr/src/sys/boot/zfs
/usr/src/sys/cddl/boot/zfs
/usr/src/sys/cddl/contrib/opensolaris/common/zfs
/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs
/usr/src/sys/modules/zfs
/usr/src/tools/regression/zfs

Those are probably a good start. Some of them just contain a Makefile
pointing you elsewhere in the tree, though.

I might have missed something, and I'm sure someone will correct me if I
have.

-- 
Thanks and best regards,
Chris Nehren


pgp4qvQrT9NKB.pgp
Description: PGP signature


Re: Resistance to documentation? (was Re: Pull in upstream before 9.1 code freeze?)

2012-07-18 Thread Chris Rees
On 18 Jul 2012 09:39, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:


 I think this is unintentionally specious reasoning. No offense intended.
:)

 true


 The program itself is fairly trivial to write.


 i don't need such a tool, but if it would be separate tool, then it is
all right if someone like to write it. This would be unix way.


 So it's really not possible to write the tool and associated
documentation myself. Otherwise I would. The pointless debating is
really an attempt to get a critical mass of knowledgeable developers to
agree to participate in


 not really. the idea was to integrate it with shell and turn on it by
default, prefering newbies over norml users, and making another change that
would actually prevent getting knowledge to newbie.


 I've been using FreeBSD since the 90s. My perception (over many years of
observation) is that the FreeBSD people most able to document what exists
and how to use it seem to also have the greatest resistance to writing any
documentation.


 what do you mean? FreeBSD manual pages is what i consider great part.

 i mean manual pages. handbook is not always up to date but still fine for
a NEW USER to learn FreeBSD.

 And that's the right way.

 creators, helpers like proposed reminds me windows. A Help that
doesn't really help learning anything.


 There are already lots of stupid things that is ALREADY done in FreeBSD
without reason.

 Few examples:

 1) XML output from some sysctl variables. It isn't just stupid. It's sad.
 2) bsdlabel -e allows editing only when NONE of partitions are
open/mounted, in spite that they are not modified. In FreeBSD 6 it allowed
editing everytime and all worked.

 Preventing people doing stupid things would prevent doing clever things.

 OK there is gpart now but still bsdlabel is far easier and useful.

Bsdlabel was a monstrosity that required a calculator to hand to do
anything useful.

Gpart on the other hand is easily scriptable.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: port devel/doxygen failing to test on -CURRENT and -STABLE

2012-07-12 Thread Chris Rees
On 9 July 2012 02:49, David Xu davi...@freebsd.org wrote:
 On 2012/07/08 18:21, Chris Rees wrote:

 Hi all / David,

 doxygen has been failing for a while now on -CURRENT and apparently
 -STABLE too.  The current fix is disabling one of the tests in the
 build, but obviously it points to a problem with our base system

 I've trussed [1] the failing code [2], and it looks as though it's
 hanging on a _umtx call.  I'm gratuitously ignorant of what goes on
 there... but the timings of recent commits to umtx.h [3] could
 indicate a link (hope it's not bogus...).

 Any pointers on what I should do next?

 Chris

 [1] http://www.bayofrum.net/~crees/scratch/doxygen-truss

 _umtx_op(0x8012b0280,0x16,0x0,0x0,0x0,0x1)   ERR#22 'Invalid argument'

 can you execute it in gdb and print its value ?

 print/x *(int *)0x8012b0280
 print/x *(int *)(0x8012b0280+4)

I've been having trouble debugging it since it's threaded, and so I
ran a binary search over the last few days of revisions from 1/Apr to
1/May.

Unfortunately I discovered to my horror today that all but the first
test was useless, because the patch I committed to disable the test
was of course readded to my ports tree, so none of the tests ran :/

I'll hopefully have it narrowed down to the offending commit over the
next few days.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Better ldns docs? (Was: bikeshedding about BIND)

2012-07-09 Thread Chris Nehren
On Sun, Jul 08, 2012 at 02:21:46 -0700 , Doug Barton wrote:
 That's an implementation issue, and is easily handled with drill, or the
 host-like program we all agree is a really-nice-to-have.

About that: as I said elsewhere in one of these threads (I want my
bikeshed clear and chartreuse at the same time, thank you), I decided to
give this a try. However I'm finding the Doxygen autogenerated
documentation to be less than useful. I'd really appreciate a
task-oriented set of documentation. The design guide helps somewhat but
I'm still missing a good set of docs for how things fit together. I
suppose you could say well volunteered to that, too, but I'm not sure
if the docs will get finished in time for a good CLI client for
10-RELEASE.

-- 
Thanks and best regards,
Chris Nehren


pgpjqnyBdJSbw.pgp
Description: PGP signature


port devel/doxygen failing to test on -CURRENT and -STABLE

2012-07-08 Thread Chris Rees
Hi all / David,

doxygen has been failing for a while now on -CURRENT and apparently
-STABLE too.  The current fix is disabling one of the tests in the
build, but obviously it points to a problem with our base system

I've trussed [1] the failing code [2], and it looks as though it's
hanging on a _umtx call.  I'm gratuitously ignorant of what goes on
there... but the timings of recent commits to umtx.h [3] could
indicate a link (hope it's not bogus...).

Any pointers on what I should do next?

Chris

[1] http://www.bayofrum.net/~crees/scratch/doxygen-truss

[2] 
http://www.bayofrum.net/tb/index.php?action=display_markup_logbuild=10-localid=1037

[3] http://svnweb.freebsd.org/base/head/sys/sys/umtx.h?view=log
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Better error messages for command not found (was Re: Pull in upstream before 9.1 code freeze?)

2012-07-08 Thread Chris Rees
On 5 July 2012 01:30, Tim Kientzle t...@kientzle.com wrote:
 On Jul 4, 2012, at 4:41 PM, Jason Hellenthal wrote:

 On Wed, Jul 04, 2012 at 03:59:29PM -0700, Doug Barton wrote:
 On 07/04/2012 15:55, Jason Hellenthal wrote:
 Seeing as sudo plays a big part of this

 No ... not only is sudo not a necessary component, it shouldn't be
 involved at all. The feature works on debian/ubuntu for regular
 userspace commands.


 What are they using to authenticate for the install ? do you know ?

 Huh?  What install?  Who's talking about install?

 The version of this I've seen looks like this:

 $ svn co https://some.url/
 svn: Command not found.
   To use this command, install one of the following packages:
   devel/subversion
   devel/subversion-freebsd
   devel/subversion16

 That's all it does:  It just prints out a more informative error message.
 It does not install anything, it requires no special permissions,
 and does not (as far as I can see) introduce any security or
 performance problems.

 The implementation is pretty simple:
  * A tool for building a database that maps command names
to package names.  (This would run against a ports tree or
package repository.  Conceptually, it's pretty similar to
how port/package indexes get built today.)
  * Some way to distribute that database (Probably as part of ISO
releases, maybe extend 'portsnap' or 'pkg_add' to update it?)
  * A program to look up command names in that database
and print out the results.
  * A shell hook to run said program whenever a command not found
error occurs.

 As a first prototype, the database could just be a text file
 and the look up program could be a shell script that uses
 grep and sed.

Anyone looking to implement something like this could also talk to
Sulev-Madis Silber (ketas on EFNet) about his code for making a
universal list of files provided by all ports-- he has written a
conflicts checker recently and may know some shortcuts.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Training wheels for commandline (was Re: Pull in upstream before 9.1 code freeze?)

2012-07-07 Thread Chris Rees
On Jul 7, 2012 10:25 AM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:

 something they probably don't even know about, than to skilled users to
 turn it off.

 If this feature is going to prints quite a few extra lines, let's just
 add one more line saying:

 To disable this message run: echo set 31337mode  ~/.tcshrc

 --

 should i - from now, understand that this way of extending OS is
considered right (i mean going down to newbies instead of going up) by
FreeBSD developers?

 Please answer it is important for me, and many other people for a future.

This is not 'going down'. This is adding features to help newcomers.  You
are free to disable them.  It will not remove anything from FreeBSD.

I point to Doug's statement on elitism.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Training wheels for commandline (was Re: Pull in upstream before 9.1 code freeze?)

2012-07-07 Thread Chris Rees
On Jul 7, 2012 10:46 AM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:

 This is not 'going down'. This is adding features to help newcomers.
You are free to disable them.  It will not remove anything

 this doesn't help newcomers. Just like easy installers, desktop
environments and so on.

 this only generate herds of morons that know FreeBSD and dissolve real
user base.

What is the 'real user base?' People who insult newcomers and call them
morons?  People who consider it cool to use a OS that is unnecessarily
difficult to learn?

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Training wheels for commandline (was Re: Pull in upstream before 9.1 code freeze?)

2012-07-05 Thread Chris Rees
On Jul 5, 2012 11:16 AM, Jonathan McKeown j.mcke...@ru.ac.za wrote:

 On Thursday 05 July 2012 11:03:32 Doug Barton wrote:
  On 07/05/2012 01:28, Peter Jeremy wrote:
   On 2012-Jul-05 09:22:25 +0200, Jonathan McKeown
  
   j.mcke...@ru.ac.za wrote:
   As for the idea that Linux refugees need extra help to migrate,
   that's the sort of thinking that led to things like:
  
   alias dir=ls
  
   Whilst we're on the subject, can we please also have #define BEGIN
   { #define END } wired into gcc to help people migrating from Algol
   and Pascal.
 
  Um, this kind of elitist crap really isn't helpful.

 It was intended to be a slightly humorous response to your original
question:

  why would you *not* want a feature that tells you what to
  install if you type a command that doesn't exist on the system?

 rather than ``elitist crap'' (as was the deliberately the over-the-top
 comparison to Clippy). I don't think suggesting that someone who wants to
use
 a system learn how it works is elitist; and I don't object to optional
tools
 to help  them ``settle in'' (but see below).

 You might also notice that I made a suggestion that might help people
 migrating - namely some adaptation of the Unix Rosetta Stone in the
Handbook
 so that people who know how to do something in Linux are quickly guided to
 the best way to do it in FreeBSD (and perhaps vice versa).

  If the new feature gets created, and you don't want to use it, turn it
  off. No problem.

 No. I think this is entirely the wrong way round. If the new feature is
 created and you want it, turn it on. Don't make me turn off something I
 didn't want in the first place. Given the choice between a system in
which I
 switch on whatever I need, versus one which has absolutely everything
 switched on where I spend ages switching it all off/deinstalling it all, I
 know which I prefer - and others have made similar comments.

That's crazy- this is the logic that led to our sh having tab completion
and history disabled by default for years.  How many people honestly knew
it was there?  The people who would benefit from this feature are the ones
who wouldn't know it was there.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Gentoo Solution to Nanny Terminal Problem

2012-07-05 Thread Chris Rees
On Jul 5, 2012 4:00 PM, Sean s...@gothic.net.au wrote:


 On 05/07/2012, at 10:02 PM, Richard Yao wrote:
 
  The second is the e-file command, which will query that database for
  whatever follows it. For example, if I want to find out which package
  installs repoman, I can do `e-file repoman`. I can also do `e-file
  /usr/bin/repoman`.
 
  if FreeBSD had an equivalent to this command, this command, then I
  imagine that calls for Ubuntu/Fedora features should cease. Gentoo users
  seem to be happy with e-file.
 


 0:55 Fri 06-Jul sean@queen [~] pkg_info -W bash
 /usr/local/bin/bash was installed by package bash-4.2.28

 0:57 Fri 06-Jul sean@queen [~] pkg_info -W /usr/local/sbin/sendmail
 /usr/local/sbin/sendmail was installed by package postfix-2.9.3,1

This isn't the same at all; it only reports for installed packages.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Better error messages for command not found (was Re: Pull in upstream before 9.1 code freeze?)

2012-07-05 Thread Chris Rees
On Jul 5, 2012 5:37 PM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:


 are you serious that linux distros have such a think now?
 I didn't use linux for a long time and no plan to use it, but you are
joking isn't it?


 They do, and it's actually very useful in two cases:


 no it isn't. unless it would be extra keypress for that.

 i don't want to be treated as a moron just as when i use google search
with javascript active.


You'd fall into the category of 'I would disable that feature' then.

Since that is the case, you should stop commenting, now, and simply disable
it if/when it comes out.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Pull in upstream before 9.1 code freeze?

2012-07-04 Thread Chris Nehren
On Wed, Jul 04, 2012 at 17:51:52 +0100 , Simon L. B. Nielsen wrote:
 On Tue, Jul 3, 2012 at 9:39 PM, Doug Barton do...@freebsd.org wrote:
  also be to enable drill so that we have a command-line dns lookup tool
  in the base, but that's trivial once you've got ldns imported.
 
 Does that means loosing host(1) ? That would be somewhat annoying.

You seem to have missed another message Doug posted to this thread
(message-id 4ff36174.2000...@freebsd.org, dated Tue, 03 Jul 2012
14:17:40 -0700) wherein he wrote:

 Whether we do the above or not, ldns/drill should be imported into the
 base so that we have at least one command line DNS resolution tool. A
 good junior hacker project would be to make a host(1) clone using
 ldns. If users want the regular bind tools, ports/dns/bind-tools
 already exists.

I was curious and started poking at ldns to create such a tool. It
shouldn't be difficult for anyone familiar with C and DNS who has the
tuits to spare.

-- 
Thanks and best regards,
Chris Nehren


pgpR1jUEqDRxU.pgp
Description: PGP signature


Re: sysctl filesystem ?

2012-06-26 Thread Chris Rees
On Jun 26, 2012 7:07 AM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:

 as well as we don't depend of /proc for normal operation we shouldn't for
say /proc/sysctl

 improvements are welcome, better documentation is welcome, changes to
what is OK - isn't.

/proc/sysctl might be useful.  Just because Linux uses it doesn't make it a
bad idea.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: MAGIC with HP KVM - someone will help?

2012-06-23 Thread Chris Rees
On Jun 23, 2012 7:04 AM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:

 100% HIT!

 yes i have on of those that have PS/2 reset issues 

Unfortunately even good adaptors have their share of problems. I refuse to
move from the keyboard I have currently, and occasionally keys get
'stuck' but it's so obvious when it happens it's no big hardship. I
have a Belkin adaptor one by the way, and it works very well with FreeBSD
apart from that issue, if that's any help.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Replacing rc(8) (Was: FreeBSD Boot Times)

2012-06-22 Thread Chris Rees
On Jun 22, 2012 7:33 AM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:

 separate and optional program.
 that's acceptable, except i have no idea why this fscadm enable/disable.
 editing config is enough.


 I don't think I understand
 i have no idea why this fscadm enable/disable.  editing config is
enough.
 and would ask you elaborate for me.  Thanks,


 why adding solaris style command to add a line in text file. just edit a
text file.


Scripting?

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Replacing rc(8) (Was: FreeBSD Boot Times)

2012-06-21 Thread Chris Rees
On Jun 20, 2012 11:03 PM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:

 scripts, i deleted most of it and put startup sequence in single file.

 It was plain horror.


 You would weep if you saw Solaris's SMF, then.  Everything is


 i don't really know what i've seen. i've installed latest solaris demo
downloaded from oracle. After nearly an hour of installation over which i
had really no control and no real clue what's going on i finally after over
5 minutes got the system booted, logged as root and tried to understand the
/etc tree, boot process and gave up really quickly.
 just seeing a files and directories (both numerous) of just a kernel
binaries was truly enough.

 Now... lets go back to FreeBSD which is controllable and understandable
by a human.

Your arrogance is astounding.

Did you read man hier? Unfamiliarity does not make it incomprehensible.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Replacing rc(8) (Was: FreeBSD Boot Times)

2012-06-21 Thread Chris Rees
On Jun 21, 2012 5:15 PM, Jason Hellenthal jhellent...@dataix.net wrote:



 On Thu, Jun 21, 2012 at 12:22:08PM +0200, Wojciech Puchar wrote:
  Lets make a summary.
 
  What functionality would be good to have in FreeBSD that doesn't exist:
 
  1) runlevels with arbitrary names. runlevel change would start and
stop
  right services.
  2) exploit startup parallelism.
 
 
  What we do not want to change:
 
  - file structure which is simple. one file in rc.d/ per service and one
  global config file (rc.conf)
  - anything else that would make things more complicated.
 
 
  As for
 
  1) i propose in rc.conf an option to put NO, YES (or ALL) or
runlevel
  list for each service or runlevel exclusion list for service.
 
 
  examples:
 
  service1_enable=YES
  service2_enable=NO
  service3_enable=foolevel maintenance
  service4_enable=YES -foolevel (or ALL -funkyrunlevel)
 
  name of default runlevel may be full or multiuser
 
  service 1 will always work, service 2 never, service 3 only at runlevels
  foolevel and maintenance, service4 with any runlevel except
  foolevel.
 
  still single rc.conf, not much bigger in practice.
 
  2) no change in rc.d/* scripts and rc.conf, but change in scripts.
 
 
  If everyone agree i think i may write this new scripts.

 Sorry but I completely disagree here. Why ?

 Because do one thing and do one thing well. What do you mean ?

He means the UNIX philosophy.

http://www.faqs.org/docs/artu/ch01s06.html

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: BIO_DELETE equivalent for file on FFS filesystem

2012-06-16 Thread Chris Rees
On Jun 14, 2012 5:49 AM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:

 file to take 900MB or... can i call some system function to punch
 holes?


 I think you can only truncate the file at this time, pretty much like
 brk() works for memory.



 BAD. suppose i keep windoze VM image on filesystem which takes 10GB but
uses 5GB.

 i could write simple program to find out what blocks are unused and
then...do nothing.


What if you cp it?

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: BIO_DELETE equivalent for file on FFS filesystem

2012-06-16 Thread Chris Rees
On Jun 16, 2012 8:37 PM, Xin LI delp...@gmail.com wrote:

 On Sat, Jun 16, 2012 at 12:01 PM, Chris Rees utis...@gmail.com wrote:
  On Jun 14, 2012 5:49 AM, Wojciech Puchar 
woj...@wojtek.tensor.gdynia.pl
  wrote:
 
  file to take 900MB or... can i call some system function to punch
  holes?
 
 
  I think you can only truncate the file at this time, pretty much like
  brk() works for memory.
 
 
 
  BAD. suppose i keep windoze VM image on filesystem which takes 10GB but
  uses 5GB.
 
  i could write simple program to find out what blocks are unused and
  then...do nothing.
 
 
  What if you cp it?

 That would be a dd(1) unless we teach cp(1) how to do sparse.  I think
 what he wanted is to tell the OS I don't need block XX - YY anymore
 and the OS creates a sparse hole, which is not available at this time.

Sorry, I must have misread.  I take it cp would take a file with holes and
only copy the data part? i.e. take a 10G file of which 5G is a hole, you'd
end up with a 5G file?

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: proper newfs options for SSD disk

2012-06-16 Thread Chris Rees
On Jun 16, 2012 8:26 PM, rank1see...@gmail.com wrote:

   And PLEASE DO NOT make this stupid MSDOS style slices. It is not just
   unneeded but introduces mess and only mess.
  
   just have /dev/ad0a not /dev/ad0s1a


 What FreeBSD devs have to say about this?
 ;)

I say we have GPT this decade :)

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Upcoming release schedule - 8.4 ?

2012-06-15 Thread Chris Rees
On Jun 15, 2012 9:39 AM, Peter Jeremy pe...@rulingia.com wrote:

 On 2012-Jun-14 08:09:30 +0100, Chris Rees utis...@gmail.com wrote:
 Except STABLE is no good for production, and the problem is EoL- updates
 and support stop.

 There's nothing stopping you from from running -stable in production.
 Obviously, you will need to do more extensive testing than you might
 need to for a -release.

 As for EoL, all software goes EoL and support for that software stops.
 FreeBSD releases are typically supported for 4 years - IMO, that's
 excellent value for money.

 On 2012-Jun-14 12:48:19 +0100, Chris Rees utis...@gmail.com wrote:
 On Jun 14, 2012 9:30 AM, Damien Fleuriot m...@my.gd wrote:
  I've moved us to 8.3-STABLE recently and am quite happy with it, so
far.
 
 Too strong wording perhaps; but you can't claim that an EOL stable branch
 will have the level of support afforded to live branches.  That was
 supposed to be my point, as Mark has also explained.

 You are the only person that is claiming that 8.x is EOL.  I have not
 seen any official announcement to that effect.  The absence of an
 announcement of 8.4-release does not make it EOL.

Of course not. The fear that this whole thread is about the possibility of
EoL this year or soon; the OP wants support for longer.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Upcoming release schedule - 8.4 ?

2012-06-14 Thread Chris Rees
On Jun 14, 2012 5:52 AM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl
wrote:

 Friends,

 I am looking at the upcoming release schedule, and I only see 9.1 listed
- can anyone confirm or deny 8.4 ?


 does it matter. cvsup RELENG_8 and you see updates are done constantly.
 just sometime somebody decide to change number :)

Except STABLE is no good for production, and the problem is EoL- updates
and support stop.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: Upcoming release schedule - 8.4 ?

2012-06-14 Thread Chris Rees
On Jun 14, 2012 9:30 AM, Damien Fleuriot m...@my.gd wrote:

 On 6/14/12 9:09 AM, Chris Rees wrote:
  On Jun 14, 2012 5:52 AM, Wojciech Puchar 
woj...@wojtek.tensor.gdynia.pl
  wrote:
 
  Friends,
 
  I am looking at the upcoming release schedule, and I only see 9.1
listed
  - can anyone confirm or deny 8.4 ?
 
 
  does it matter. cvsup RELENG_8 and you see updates are done constantly.
  just sometime somebody decide to change number :)
 
  Except STABLE is no good for production, and the problem is EoL- updates
  and support stop.
 
  Chris

 Whoever said STABLE is no good for production ?

 I used to make us stick to 8.2-RELEASE here at work, but some bugfixes
 are just too important to skip (we're running firewalls and had a
 problem with a CARP bug).


 I've moved us to 8.3-STABLE recently and am quite happy with it, so far.

Too strong wording perhaps; but you can't claim that an EOL stable branch
will have the level of support afforded to live branches.  That was
supposed to be my point, as Mark has also explained.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


Re: CD bootcode

2012-06-07 Thread Chris Rees
On Jun 7, 2012 2:58 PM, rank1see...@gmail.com wrote:

 - Original Message -
 From: John Baldwin j...@freebsd.org
 To: freebsd-hackers@freebsd.org
 Cc: rank1see...@gmail.com
 Date: Thu, 7 Jun 2012 08:21:39 -0400
 Subject: Re: CD bootcode

  On Wednesday, June 06, 2012 12:15:14 pm rank1see...@gmail.com wrote:
   FreeBSD's CD bootcode '/boot/cdboot' is targeting stage 3 boot -
loader
   For example, stage 2 boot '/boot/boot' is attempting the same.
  
   In my case of a '/boot/loader', '/boot' is a symlink!
  
   So if '/boot/boot' works with symlinks, why '/boot/cdboot' doesn't!
   Yes, I did use Rock-Ridge extensions, upon creation of 'cd.iso'
 
  Because I didn't make cdboot's lookup routine handle symlinks.  It
  also doesn't look at Rock-Ridge extensions and only uses the base
  ISO-9660 directory entries.  That was enough fun to write in assembly.
  OTOH, CD sectors are 2k, so you do have that much room to work with
  and can probably fit a more advanced directory lookup into cdboot.
 
  I'm happy to review any patches you come up with.
 
  --
  John Baldwin


 I need to correct myself.
 BOTH stage 2 boot '/boot/boot' AND '/boot/cdboot' don't work with
symlinks!

 '/boot/boot' on UFS sees symlink as file, same as '/boot/cdboot' does so
on Rock-Ridge.

 I can handle '/boot/boot' by providing full path to loader via
'boot.config'
 '/boot/cdboot' doesn't have such a file, so in
'/usr/src/sys/boot/i386/cdboot/cdboot.s':
 --
 loader_paths:   .asciz  /BOOT/LOADER
.asciz  /boot/loader
.byte 0
 --

 I injected third '.asciz' line with full path to 'loader' and recompiled
it.
 And what is first line /BOOT/LOADER doing in there ?!?


Because strict ISO9660 8.3 is case insensitive and uses caps?

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org


  1   2   3   4   5   6   7   >