Re: [CFT] Autofs.

[Lev Serebryakov]

Hello, Edward.
You wrote 4 сентября 2014 г., 16:43:30:

ETN> It's a bug.  Or rather, a missing feature.  The problem here is that
ETN> the "/" export "shadows" the rest.  To handle this correctly, automountd(8)
ETN> would need to mount the "/" share, then mount autofs on "/usr" etc, and
ETN> then call it done.  This part is easy.  The problem is: how to expire
ETN> (automatically unmount) it?  Because of autofs mounts, the "/" share
ETN> will always be busy, and thus won't ever get automatically unmounted.
ETN> So, for now, we don't even try to handle this situation.
  I have same problem with /usr + /usr/home, which make autofs pretty
useless for me :(

  Looks like, "autofs" mounts should not be considered "busy" for other
automounted filesystems and should be processed separately...

-- 
// Black Lion AKA Lev Serebryakov 

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: ZFS-related panic: "possible" spa->spa_errlog_lock deadlock

[Alexander Motin]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 07.09.2014 18:56, Xin Li wrote:
> On 9/7/14 11:23 PM, Fabian Keil wrote:
>> Xin Li  wrote:
> 
>>> On 9/7/14 9:02 PM, Fabian Keil wrote:
 Using a kernel built from FreeBSD 11.0-CURRENT r271182 I got 
 the following panic yesterday:
 
 [...] Unread portion of the kernel message buffer: [6880] 
 panic: deadlkres: possible deadlock detected for 
 0xf80015289490, blocked for 1800503 ticks
>>> 
>>> Any chance to get all backtraces (e.g. thread apply all bt
>>> full 16)? I think a different thread that held the lock have
>>> been blocked, probably related to your disconnected vdev.
> 
>> Output of "thread apply all bt full 16" is available at: 
>> http://www.fabiankeil.de/tmp/freebsd/kgdb-output-spa_errlog_lock-deadlock.txt
>
>>  A lot of the backtraces prematurely end with "Cannot access
>> memory at address", therefore I also added "thread apply all bt"
>> output.
> 
>> Apparently there are at least two additional threads blocking
>> below spa_get_stats():
> 
>> Thread 1182 (Thread 101989): #0  sched_switch 
>> (td=0xf800628cc490, newtd=,
>> flags=) at
>> /usr/src/sys/kern/sched_ule.c:1932 #1 0x805a23c1 in
>> mi_switch (flags=260, newtd=0x0) at 
>> /usr/src/sys/kern/kern_synch.c:493 #2  0x805e4bca in 
>> sleepq_wait (wchan=0x0, pri=0) at 
>> /usr/src/sys/kern/subr_sleepqueue.c:631 #3  0x80539f10
>> in _cv_wait (cvp=0xf80025534a50, lock=0xf80025534a30) at 
>> /usr/src/sys/kern/kern_condvar.c:139 #4  0x811721db in 
>> zio_wait (zio=) at 
>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1442
>>  #5  0x81102111 in dbuf_read (db=, 
>> zio=, flags=) at 
>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:649
>>  #6  0x81108e6d in dmu_buf_hold (os=> out>, object=, offset=> out>, tag=0x0, dbp=0xfe00955c6648, flags=> out>) at 
>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:172
>>  #7  0x81163986 in zap_lockdir (os=0xf8002b7ab000, 
>> obj=92, tx=0x0, lti=RW_READER, fatreader=1, adding=0,
>> zapp=) at 
>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c:467
>
>> 
> 
> #8  0x811644ad in zap_count (os=0x0, zapobj=0, 
> count=0xfe00955c66d8) at 
> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c:712
>>
> 
#9  0x8114a6dc in spa_get_errlog_size
>> (spa=0xf800062ed000) at 
>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_errlog.c:149
>
>> 
> 
> ---Type  to continue, or q  to quit---
>> #10 0x8113f549 in spa_get_stats (name=0xfe0044cac000 
>> "spaceloop", config=0xfe00955c68e8,
>> altroot=0xfe0044cac430 "", buflen=2048) at 
>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:3287
>>  #11 0x81189a45 in zfs_ioc_pool_stats 
>> (zc=0xfe0044cac000) at 
>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:1656
>
>> 
> 
> #12 0x81187290 in zfsdev_ioctl (dev=, 
> zcmd=, arg=, flag= optimized out>, td=)
>> at 
>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:6136
>
>> 
> 
> #13 0x80464a55 in devfs_ioctl_f (fp=0xf80038bd00a0, 
> com=3222821381, data=0xf800067b80a0, cred= out>, td=0xf800628cc490) at
> /usr/src/sys/fs/devfs/devfs_vnops.c:757
>> #14 0x805f3c3d in kern_ioctl (td=0xf800628cc490, 
>> fd=, com=0) at file.h:311 #15 
>> 0x805f381c in sys_ioctl (td=0xf800628cc490, 
>> uap=0xfe00955c6b80) at /usr/src/sys/kern/sys_generic.c:702
>> #16 0x8085c2db in amd64_syscall (td=0xf800628cc490, 
>> traced=0) at subr_syscall.c:133 #17 0x8083f90b in 
>> Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:390 #18 
>> 0x0008019fc3da in ?? () Previous frame inner to this frame 
>> (corrupt stack?)
> 
> Yes, thread 1182 owned the lock and is waiting for the zio be
> done. Other threads that wanted the lock would have to wait.
> 
> I don't have much clue why the system entered this state, however,
> as the operations should have errored out (the GELI device is gone
> on 21:44:56 based on your log, which suggests all references were
> closed) instead of waiting.
> 
> Adding mav@ as he may have some idea.

Some time ago I experimented with ZFS behavior in situation of
disappearing disks. I fixed several most easily reproducible
scenarios, but finally I've got to conclusion that ZFS in many places
written in a way that simply does not expect errors. In such cases it
just stucks, waiting for disk to reappear and I/O to complete. In some
situations after disk reconnect it can now be recovered with `zpool
clear ...`, but still not all, since sometimes code may stuck holding
some lock required for recovery.

- -- 
Alexander Motin
-BEGIN PGP SIGNATURE-
Version: GnuPG v2
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQF8BAEBCgBmBQJUDWMaXxSAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w
ZW5wZ3AuZmlmd

Re: [RFC] Patch to improve TSO limitation formula in general

[Rick Macklem]

Hans Petter Selasky wrote:
> On 09/06/14 00:09, Rick Macklem wrote:
> > Hans Petter Selesky wrote:
> >> On 09/05/14 23:19, Eric Joyner wrote:
> >>> There are some concerns if we use this with devices that ixl
> >>> supports:
> >>>
> >>> - The maximum fragment size is 16KB-1, which isn't a power of 2.
> >>>
> >>
> >> Hi Eric,
> >>
> >> Multiplying by powers of two are more fast, than non-powers of
> >> two.
> >> So
> >> in this case you would have to use 8KB as a maximum.
> >>
> > Well, I'm no architecture expert, but I really doubt the CPU delay
> > of a
> > non-power of 2 multiply/divide is significant related to doing
> > smaller
> > TSO segments. Long ago (as in 1970s) I did work on machines where
> > shifts
> > for power of 2 multiply/divide was preferable, but these days I
> > doubt it
> > is going to matter??
> >
> >>> - You can't get the maximum TSO size for ixl devices by
> >>> multiplying
> >>> the
> >>> maximum number of fragments by the maximum size.
> >>> Instead the number of fragments is AFAIK unlimited, but a segment
> >>> can only
> >>> span 8 mbufs (including the [up to 3] mbufs containing the
> >>> header),
> >>> and the
> >>> maximum TSO size is 256KB.
> 
> Hi,
> 
> Maybe that can be a separate parameter?
> 
> I see that your patch assumes that a segment can be any-length. That
> is
> not always the case. Remember there are JUMBO mbufs too.
> 
I thought JUMBO mbufs were only going to be used on the receive side,
however I suppose if a packet is received into a JUMBO mbuf and then
forwarded on another interface...

> With my patch, the maximum segment size is a separate parameter. The
> total number of TSO bytes is then not so useful.
> 
Well, if you are referring to if_hw_tsomax, I'm not sure it was the
best plan to begin with. It was implemented for xen and I'm not sure
that any other driver uses it as of now.

However...
I'm not a network/hardware guy, but it seems some devices do have
the IP_MAXPACKET limit (use the ip_len field in the ip header to
know how large the TSO segment is). There is also at least one device
(82598 chip for "ix" driver) that can handle more that IP_MAXPACKET,
so it seems appropriate to have a value that the driver can set.

Since the maximum size of the gather list for transmit does seem to
vary a lot between devices, with several handling less than 35, it
does seem appropriate to allow drivers to specify that.

If devices can't handle a single gather entry over a certain size,
I think that does need to be specified along with the max size of
the gather list, since the driver will need to use multiple gather
entries for a single mbuf and tcp_output() should take that into
account.

In summary, yep, I basically agree.

rick
ps: Who will look at your patch soon.

> --HPS
> 
> ___
> freebsd-current@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to
> "freebsd-current-unsubscr...@freebsd.org"
> 
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: UEFI display frozen on Retina MacBook Pro

[Anders Bolt Evensen]

On 05.09.14 19:37, John Nielsen wrote:

On Sep 5, 2014, at 11:30 AM, Glen Barber  wrote:


On Fri, Sep 05, 2014 at 11:20:21AM -0600, John Nielsen wrote:

I have a "MacBook Pro Retina, Mid 2012" (MacBookPro10,1) on which I'd like to 
be able to boot FreeBSD from an external USB drive. For testing I've been using the 
mini-memstick images from the -CURRENT snapshots, most recently the one from 20140903.

I am able to select "EFI Boot" on the USB device from the Mac's boot menu, and 
it does _something_, but the screen never changes--the image of the boot menu is 
displayed indefinitely. I think it is actually booting since there is drive activity and 
the caps lock key indicator starts working a few seconds in, but the screen just stays 
the same. Thinking the resolution of the Retina display may have been an issue, I tried 
booting with it disabled (lid closed) and an external monitor and keyboard. The result 
was the same--Mac boot menu frozen on the external display.

Is there anything I should try to troubleshoot or debug this issue? Anything 
else I should include in a PR? I can test patches if needed (probably after 
building an image including the patch from a VM).


To be clear, which boot menu do you see?  If you see the FreeBSD loader
menu, escape to the loader prompt and try:

set kern.vty=vt
set hw.vga.textmode=1
boot

I am a bit unclear under which conditions 'hw.vga.textmode=1' is
required, though.

No, I don't ever see the FreeBSD loader. I see the menu you get on a Mac when you hold down the 
option (alt) key while booting--big disk icons representing the bootable disks/partitions in the 
system. In my case it was the "Macintosh HD" volume (Mac OS Mavericks), my Windows 
partition, and the USB stick with the FreeBSD memstick image on it, which the Mac just called 
"EFI Boot" (and the icon was that of a USB disk). There is also a little section at the 
bottom that allows wifi network booting (if you've done all the black magic (not PXE) to get that 
to happen). It shows a circular activity animation while it scans for wireless networks. That 
animation stops when I select the USB EFI icon and press enter (and that is the only visual 
indication I get that I made a selection).

JN

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


To see the FreeBSD (U)EFI boot loader on the Mac, you need to install an 
EFI shell like rEFIt on either your hard drive or a HFS formatted memory 
stick:
1) Download the rEFIt installer from here: 
http://downloads.sourceforge.net/project/refit/rEFIt/0.14/rEFIt-0.14.dmg?r=http%3A%2F%2Frefit.sourceforge.net%2F&ts=1410181876&use_mirror=optimate

2) Open the downloaded file
3) Run the following command from the terminal: sudo installer -pkg 
/Volumes/rEFIt/rEFIt.mpkg -tgt /Volumes/memstick (in this example, I'm 
using an HFS formatted memory stick).

4) Run the command "sudo /Volumes/memstick/efi/enable.sh"
5) When you reboot your Mac, when you hold down the alt key, choose 
rEFIt on the startup menu. Then, choose the "BOOTx64.efi from …" option
If everything now goes as it should, you should see the FreeBSD loader. 
When the "Press enter to boot or any other key to go to loader in X 
seconds" (or whatever it says), press a random key. Then try to type the 
commands suggested by John Nielsen.


Hope this helps. :)


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: make -DNO_ROOT to create chroot, problem installing into chroot with pkg

[Brooks Davis]

On Sun, Sep 07, 2014 at 04:03:59PM -0700, Craig Rodrigues wrote:
> Hi,
> 
> I am using pkg 1.3.7.
> 
> I did the following as a regular user, not root:
> 
> rm -fr /tmp/package
> cd /usr/src
> make buildworld
> make buildkernel
> make  -DNO_ROOT -DDB_FROM_SRC  installworld DESTDIR=/tmp/package
> make  -DNO_ROOT -DDB_FROM_SRC  installkernel DESTDIR=/tmp/package
> make  -DNO_ROOT -DDB_FROM_SRC  distribution DESTDIR=/tmp/package
> 
> This created an installed world under /tmp/package
> 
> Then I did:
> 
> pkg -c /tmp/package install -y devel/kyua
> 
> I got:
> 
> pkg: chroot failed!
> 
> Then I tried the same command under sudo:
> 
> sudo pkg -c /tmp/package install -y devel/kyua
> 
> I got:
> 
> pkg: /var/db/pkg wrong user or group ownership (expected 0/0 versus actual
> 818/0)
> 
> Is there a way to install packages into chroot without
> being root?

If you don't mind the ownership being wrong and there being a few extra
+FOO files tar works.  It would be great for someone to teach package to
install without root and to update a METALOG file.  That's not 100% of
the solution, but it's a solid 80-90% solution.

-- Brooks


pgpfgGdFgQ2eM.pgp
Description: PGP signature

Re: panic m_demote: m_nextpkt not NULL

[Mark Atkinson]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 09/05/2014 12:24, Gleb Smirnoff wrote:
> On Fri, Sep 05, 2014 at 08:38:10AM -0700, Mark Atkinson wrote: M>
> r271093 GENERIC amd64.   Received this panic in the tcp reassembly
> code: M> Unread portion of the kernel message buffer: M> panic:
> m_demote: m_nextpkt not NULL M> cpuid = 0
> 
> Sorry for that. Was fixed in r271123.

Thank you and Jean-Sébastien for the pointer, I was able rebuild the
kernel quickly on Fri with just this change and it's working fine.

-BEGIN PGP SIGNATURE-
Version: GnuPG v2
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlQN2jgACgkQrDN5kXnx8yZjlgCfcIyo87hclmT7YPVws/X0DAap
PK8AoKA7G9IQGO7Om0s0fXkwvzGtKZaR
=HSx2
-END PGP SIGNATURE-

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: make -DNO_ROOT to create chroot, problem installing into chroot with pkg

[Julio Merino]

On Mon, Sep 8, 2014 at 11:48 AM, Brooks Davis  wrote:
>
> If you don't mind the ownership being wrong and there being a few extra
> +FOO files tar works.  It would be great for someone to teach package to
> install without root and to update a METALOG file.  That's not 100% of
> the solution, but it's a solid 80-90% solution.

(This is just my completely uninformed opinion as I don't know the
internals of pkg.) There are other issues to be addressed.

Teaching pkg to install without root means changing pkg to not use
chroot: i.e. to make pkg be able to deal with the concept of a
"destdir" for package installations. That is probably easy: just
prefixing a bunch of destdir to the locations being touched.

However, the tricky part here is dealing with any package-specific
post-install scripts to ensure they understand destdir, which in turn
means that any tools executed by the scripts must also be capable of
dealing with a destdir. Also, the scripts (being potentially-untrusted
code) cannot be guaranteed to behave correctly on the
outside-of-the-chroot system.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

network hang using intel em nic

[Steve Wills]

Hi,

I've got some new hardware and have been experiencing lockups using the em
driver. They seem to only happen on large downloads, smaller things like ssh
and web browsing work OK. The hardware is:

em0@pci0:0:25:0:class=0x02 card=0x309f17aa chip=0x153a8086 rev=0x04 
hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Connection I217-LM'
class  = network
subclass   = ethernet
bar   [10] = type Memory, range 32, base 0xf7c0, size 131072, enabled
bar   [14] = type Memory, range 32, base 0xf7c3d000, size 4096, enabled
bar   [18] = type I/O Port, range 32, base 0xf080, size 32, enabled

I did have the vboxnet driver loaded, but tried unloading that and still saw
the issue. One thing I notice is some garbage at the top of the screen when the
hang happens, perhaps there's a conflict with the vesa video. (This system is
Haswell, using the vga/vesa driver.)

Any suggestions on how to debug?

Thanks,
Steve
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: network hang using intel em nic

[Allan Jude]

On 2014-09-08 13:14, Steve Wills wrote:
> Hi,
> 
> I've got some new hardware and have been experiencing lockups using the em
> driver. They seem to only happen on large downloads, smaller things like ssh
> and web browsing work OK. The hardware is:
> 
> em0@pci0:0:25:0:class=0x02 card=0x309f17aa chip=0x153a8086 
> rev=0x04 hdr=0x00
> vendor = 'Intel Corporation'
> device = 'Ethernet Connection I217-LM'
> class  = network
> subclass   = ethernet
> bar   [10] = type Memory, range 32, base 0xf7c0, size 131072, enabled
> bar   [14] = type Memory, range 32, base 0xf7c3d000, size 4096, enabled
> bar   [18] = type I/O Port, range 32, base 0xf080, size 32, enabled
> 
> I did have the vboxnet driver loaded, but tried unloading that and still saw
> the issue. One thing I notice is some garbage at the top of the screen when 
> the
> hang happens, perhaps there's a conflict with the vesa video. (This system is
> Haswell, using the vga/vesa driver.)
> 
> Any suggestions on how to debug?
> 
> Thanks,
> Steve
> ___
> freebsd-current@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
> 

Try periodically setting dev.em.0.debug=1

it will dump a bunch of stats to syslog then set it self back to -1

there are also a bunch of useful stats under:
dev.em.0 including things like:
dev.em.0.mbuf_alloc_fail
dev.em.0.watchdog_timeouts
dev.em.0.mac_stats.collision_count

etc

that might provide some insight.

I have a box with 4x of the i210 (but that shows up as igb(4))

I have one of the i217LM nics in this machine, but it is used for video
production so it isn't running BSD at the moment.



-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature

Re: network hang using intel em nic

[Steve Wills]

On Mon, Sep 08, 2014 at 01:29:31PM -0400, Allan Jude wrote:
> 
> Try periodically setting dev.em.0.debug=1
> 
> it will dump a bunch of stats to syslog then set it self back to -1
> 
> there are also a bunch of useful stats under:
> dev.em.0 including things like:
> dev.em.0.mbuf_alloc_fail
> dev.em.0.watchdog_timeouts
> dev.em.0.mac_stats.collision_count
> 
> etc
> 
> that might provide some insight.
> 
> I have a box with 4x of the i210 (but that shows up as igb(4))
> 
> I have one of the i217LM nics in this machine, but it is used for video
> production so it isn't running BSD at the moment.

Thanks for the suggestions, I think I may have solved it just by ensuring my
kernel modules for vbox and cuse were in sync with the kernel.

Steve
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: make -DNO_ROOT to create chroot, problem installing into chroot with pkg

[Brooks Davis]

On Mon, Sep 08, 2014 at 01:06:38PM -0400, Julio Merino wrote:
> On Mon, Sep 8, 2014 at 11:48 AM, Brooks Davis  wrote:
> >
> > If you don't mind the ownership being wrong and there being a few extra
> > +FOO files tar works.  It would be great for someone to teach package to
> > install without root and to update a METALOG file.  That's not 100% of
> > the solution, but it's a solid 80-90% solution.
> 
> (This is just my completely uninformed opinion as I don't know the
> internals of pkg.) There are other issues to be addressed.
> 
> Teaching pkg to install without root means changing pkg to not use
> chroot: i.e. to make pkg be able to deal with the concept of a
> "destdir" for package installations. That is probably easy: just
> prefixing a bunch of destdir to the locations being touched.

This should be pretty easy given that most of the process is extracting
files from the tarball.

> However, the tricky part here is dealing with any package-specific
> post-install scripts to ensure they understand destdir, which in turn
> means that any tools executed by the scripts must also be capable of
> dealing with a destdir. Also, the scripts (being potentially-untrusted
> code) cannot be guaranteed to behave correctly on the
> outside-of-the-chroot system.

I believe the majority of packages don't suffer from post-install
scripts hence the suggestion that extracting in the right place without
root would solve 80-90% of the problem (and probably take no more than
10% of the work).  I could live with the pain of not having scripts run
during install.  The correct long term fix as proposed by bapt is the do
anyway with most scripts in favor of common actions and if any
significant scripts remain add the ability to run them on first boot.

-- Brooks


pgpsjCORfwSXs.pgp
Description: PGP signature

Re: make -DNO_ROOT to create chroot, problem installing into chroot with pkg

[Craig Rodrigues]

On Mon, Sep 8, 2014 at 8:48 AM, Brooks Davis  wrote:

>
> If you don't mind the ownership being wrong and there being a few extra
> +FOO files tar works.  It would be great for someone to teach package to
> install without root and to update a METALOG file.  That's not 100% of
> the solution, but it's a solid 80-90% solution.
>
>
I've opened this feature request against pkg:

https://github.com/freebsd/pkg/issues/1002

Please add any comments/suggestions there, since you have very valuable
input.
--
Craig
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

_ftello() modification requires additional capsicum rights, breaking tcpdump and dhclient

[Patrick Kelsey]

In r268997, _ftello() was modified to use _fcntl(F_GETFL) in the
non-append, write-only path.  Consequently, programs that use _ftello()
(via ftell, fgetpos, fsetpos, fseek, rewind...) on non-append, write-only
files and that use capsicum to restrict capabilities on the associated fds
to [CAP_SEEK, CAP_WRITE] broke as all ftell() (and friends) calls on those
files fail with ENOTCAPABLE due to lack of CAP_FCNTL rights.  There appear
to be only two affected programs in the tree - tcpdump and dhclient.  This
affects both CURRENT and 10-STABLE (including 10.1-PRERELEASE)

tcpdump, when configured to write to capture files rotated by size, fails
to rotate and captures indefinitely to the first file in the series.  This
can be reproduced by a command such as: tcpdump -i  -C 1 -W 2 -w
packets -v

By inspection, dhclient will fail to trim old data from its client leases
file when rewriting that file with a lesser amount of data than it
currently contains.  See the ftruncate() call in
dhclient.c:rewrite_client_leases().

The attached patch adds CAP_FCNTL to the limited rights established for
non-append, write-only files used by tcpdump and dhclient.  It also
restricts the fcntl rights to CAP_FCNTL_GETFL.

The current need to have CAP_FCNTL rights in order to get or set the file
position on non-append, write-only files is subtle.  Perhaps part of the
answer is to define a CAP_FSEEK right in sys/capability.h that resolves to
CAP_SEEK|CAP_FCNTL, or to modify the CAP_SEEK description in rights(4) to
note the need for CAP_FCNTL when using ftell() and friends.

-Patrick


ftell_cap_rights.patch
Description: Binary data
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: _ftello() modification requires additional capsicum rights, breaking tcpdump and dhclient

[Andrey Chernov]

On 09.09.2014 0:28, Patrick Kelsey wrote:
> In r268997, _ftello() was modified to use _fcntl(F_GETFL) in the
> non-append, write-only path.  Consequently, programs that use _ftello()
> (via ftell, fgetpos, fsetpos, fseek, rewind...) on non-append,
> write-only files and that use capsicum to restrict capabilities on the
> associated fds to [CAP_SEEK, CAP_WRITE] broke as all ftell() (and
> friends) calls on those files fail with ENOTCAPABLE due to lack of
> CAP_FCNTL rights.  There appear to be only two affected programs in the
> tree - tcpdump and dhclient.  This affects both CURRENT and 10-STABLE
> (including 10.1-PRERELEASE)
> 
> tcpdump, when configured to write to capture files rotated by size,
> fails to rotate and captures indefinitely to the first file in the
> series.  This can be reproduced by a command such as: tcpdump -i
>  -C 1 -W 2 -w packets -v
> 
> By inspection, dhclient will fail to trim old data from its client
> leases file when rewriting that file with a lesser amount of data than
> it currently contains.  See the ftruncate() call in
> dhclient.c:rewrite_client_leases().
> 
> The attached patch adds CAP_FCNTL to the limited rights established for
> non-append, write-only files used by tcpdump and dhclient.  It also
> restricts the fcntl rights to CAP_FCNTL_GETFL.
> 
> The current need to have CAP_FCNTL rights in order to get or set the
> file position on non-append, write-only files is subtle.  Perhaps part
> of the answer is to define a CAP_FSEEK right in sys/capability.h that
> resolves to CAP_SEEK|CAP_FCNTL, or to modify the CAP_SEEK description in
> rights(4) to note the need for CAP_FCNTL when using ftell() and friends.
> 
> -Patrick

Stdio code use fcntl(F_GETFL) already in many places, f.e. fdopen(),
freopen(). libc code in general use it in rpc code. According to your
note, all that places are currently broken in anyway.

I don't think that this read-only fcntl(F_GETFL) which doesn not modify
anything deserves any special rights at all (i.e. can be just enabled by
default in contrast to F_SETFL), but I am not capsicum expert.

-- 
http://ache.vniz.net/
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: _ftello() modification requires additional capsicum rights, breaking tcpdump and dhclient

[Patrick Kelsey]

On Mon, Sep 8, 2014 at 4:42 PM, Andrey Chernov  wrote:

> On 09.09.2014 0:28, Patrick Kelsey wrote:
> > In r268997, _ftello() was modified to use _fcntl(F_GETFL) in the
> > non-append, write-only path.  Consequently, programs that use _ftello()
> > (via ftell, fgetpos, fsetpos, fseek, rewind...) on non-append,
> > write-only files and that use capsicum to restrict capabilities on the
> > associated fds to [CAP_SEEK, CAP_WRITE] broke as all ftell() (and
> > friends) calls on those files fail with ENOTCAPABLE due to lack of
> > CAP_FCNTL rights.  There appear to be only two affected programs in the
> > tree - tcpdump and dhclient.  This affects both CURRENT and 10-STABLE
> > (including 10.1-PRERELEASE)
> >
> > tcpdump, when configured to write to capture files rotated by size,
> > fails to rotate and captures indefinitely to the first file in the
> > series.  This can be reproduced by a command such as: tcpdump -i
> >  -C 1 -W 2 -w packets -v
> >
> > By inspection, dhclient will fail to trim old data from its client
> > leases file when rewriting that file with a lesser amount of data than
> > it currently contains.  See the ftruncate() call in
> > dhclient.c:rewrite_client_leases().
> >
> > The attached patch adds CAP_FCNTL to the limited rights established for
> > non-append, write-only files used by tcpdump and dhclient.  It also
> > restricts the fcntl rights to CAP_FCNTL_GETFL.
> >
> > The current need to have CAP_FCNTL rights in order to get or set the
> > file position on non-append, write-only files is subtle.  Perhaps part
> > of the answer is to define a CAP_FSEEK right in sys/capability.h that
> > resolves to CAP_SEEK|CAP_FCNTL, or to modify the CAP_SEEK description in
> > rights(4) to note the need for CAP_FCNTL when using ftell() and friends.
> >
> > -Patrick
>
> Stdio code use fcntl(F_GETFL) already in many places, f.e. fdopen(),
> freopen(). libc code in general use it in rpc code. According to your
> note, all that places are currently broken in anyway.
>

You make a godo point about the wider use of fcntl() in libc - aside from
the rpc code, by my count there are 14 other entry points in libc that use
fcntl in their implementation.  To experience breakage, programs that use
those entry points would also have to be supplying them fds with restricted
rights that do not include CAP_FCNTL.  By my count, there are currently
only 12 programs in -current that call cap_rights_limit().  I don't think
these counts inform us very well as to the presence and extent of any
capsicum+libc issues similar to the one that I've raised.  Those 12
programs mentioned above would have to be audited to determine if any of
the 15 libc entry points (including fcntl) that use fcntl are being used on
those restricted fds without being granted CAP_FCNTL rights, and whether
there are overt or potential failures occurring as a result.  Consider that
the failure mode in tcpdump that I found requires that you be using
multiple capture files with size-based rotation, otherwise all works fine.
Also consider that the failure mode in dhclient only occurs when a
rewritten client lease file is smaller than its predecessor.


>
> I don't think that this read-only fcntl(F_GETFL) which doesn not modify
> anything deserves any special rights at all (i.e. can be just enabled by
> default in contrast to F_SETFL), but I am not capsicum expert.
>

I don't think I am in a position to comment on the implications of
permanent F_GETFL rights either.  I do think that the point about wider use
of fcntl(F_GETFL) in libc does argue against making a CAP_FSEEK right in
sys/capability.h, as it would appear users of capsicum and libc are more in
need of a map of capsicum rights required by libc entry points than they
are of convenience #defines.

-Patrick
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Build failed in Jenkins: FreeBSD_HEAD #1411

[jenkins-admin]

See 

--
Started by an SCM change
Building remotely on jenkins-10.freebsd.org (FreeBSD-10) in workspace 

Updating svn://svn.freebsd.org/base/head at revision '2014-09-08T21:28:38.587 
+'
U contrib/libc-vis/vis.c
 Ucontrib/libc-vis
A 
contrib/llvm/patches/patch-r271282-clang-r200797-r200798-r200805-debug-info-crash.diff
U contrib/llvm/tools/clang/lib/CodeGen/CGDebugInfo.cpp
U sys/boot/arm/uboot/help.uboot
U sys/boot/uboot/common/main.c
U sys/dev/ixgbe/ixgbe.c
U sys/dev/ixgbe/ixv.c
U crypto/heimdal/tools/krb5-config.in
At revision 271289
hudson.util.IOException2: revision check failed on 
svn://svn.freebsd.org/base/head
at 
hudson.scm.SubversionChangeLogBuilder.buildModule(SubversionChangeLogBuilder.java:178)
at 
hudson.scm.SubversionChangeLogBuilder.run(SubversionChangeLogBuilder.java:113)
at hudson.scm.SubversionSCM.calcChangeLog(SubversionSCM.java:654)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:815)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1252)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:624)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:530)
at hudson.model.Run.execute(Run.java:1732)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:234)
Caused by: org.tmatesoft.svn.core.SVNException: svn: E210003: connection 
refused by the server
at 
org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:85)
at 
org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:69)
at 
org.tmatesoft.svn.core.internal.io.svn.SVNPlainConnector.open(SVNPlainConnector.java:62)
at 
org.tmatesoft.svn.core.internal.io.svn.SVNConnection.open(SVNConnection.java:77)
at 
org.tmatesoft.svn.core.internal.io.svn.SVNRepositoryImpl.openConnection(SVNRepositoryImpl.java:1252)
at 
org.tmatesoft.svn.core.internal.io.svn.SVNRepositoryImpl.getLatestRevision(SVNRepositoryImpl.java:168)
at 
org.tmatesoft.svn.core.internal.wc2.ng.SvnNgRepositoryAccess.getRevisionNumber(SvnNgRepositoryAccess.java:118)
at 
org.tmatesoft.svn.core.internal.wc2.SvnRepositoryAccess.getLocations(SvnRepositoryAccess.java:148)
at 
org.tmatesoft.svn.core.internal.wc2.ng.SvnNgRepositoryAccess.createRepositoryFor(SvnNgRepositoryAccess.java:45)
at 
org.tmatesoft.svn.core.internal.wc2.remote.SvnRemoteLog.run(SvnRemoteLog.java:160)
at 
org.tmatesoft.svn.core.internal.wc2.remote.SvnRemoteLog.run(SvnRemoteLog.java:35)
at 
org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:20)
at 
org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1238)
at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
at org.tmatesoft.svn.core.wc.SVNLogClient.doLog(SVNLogClient.java:967)
at org.tmatesoft.svn.core.wc.SVNLogClient.doLog(SVNLogClient.java:872)
at 
hudson.scm.SubversionChangeLogBuilder.buildModule(SubversionChangeLogBuilder.java:166)
... 11 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:385)
at java.net.Socket.connect(Socket.java:546)
at 
org.tmatesoft.svn.core.internal.util.SVNSocketFactory.connect(SVNSocketFactory.java:146)
at 
org.tmatesoft.svn.core.internal.util.SVNSocketFactory.createPlainSocket(SVNSocketFactory.java:73)
at 
org.tmatesoft.svn.core.internal.io.svn.SVNPlainConnector.open(SVNPlainConnector.java:53)
... 25 more
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: _ftello() modification requires additional capsicum rights, breaking tcpdump and dhclient

[Andrey Chernov]

On 09.09.2014 1:13, Patrick Kelsey wrote:
> You make a godo point about the wider use of fcntl() in libc - aside
> from the rpc code, by my count there are 14 other entry points in libc
> that use fcntl in their implementation.  To experience breakage,
> programs that use those entry points would also have to be supplying
> them fds with restricted rights that do not include CAP_FCNTL.  By my
> count, there are currently only 12 programs in -current that call
> cap_rights_limit().  I don't think these counts inform us very well as
> to the presence and extent of any capsicum+libc issues similar to the
> one that I've raised.  Those 12 programs mentioned above would have to
> be audited to determine if any of the 15 libc entry points (including
> fcntl) that use fcntl are being used on those restricted fds without
> being granted CAP_FCNTL rights, and whether there are overt or potential
> failures occurring as a result.  Consider that the failure mode in
> tcpdump that I found requires that you be using multiple capture files
> with size-based rotation, otherwise all works fine.  Also consider that
> the failure mode in dhclient only occurs when a rewritten client lease
> file is smaller than its predecessor.

Just to note by quick glance:
tcpdump use fdopen(), so in some cases probably already broken without
F_GETFL rights.
openssh use fdopen(), so suspicious about F_GETFL too, but I don't
traverse the order in which fdopen() and cap_rights_* there are applied.

> I don't think that this read-only fcntl(F_GETFL) which doesn not modify
> anything deserves any special rights at all (i.e. can be just enabled by
> default in contrast to F_SETFL), but I am not capsicum expert.
> 
> I don't think I am in a position to comment on the implications of
> permanent F_GETFL rights either.  I do think that the point about wider
> use of fcntl(F_GETFL) in libc does argue against making a CAP_FSEEK
> right in sys/capability.h, as it would appear users of capsicum and libc
> are more in need of a map of capsicum rights required by libc entry
> points than they are of convenience #defines.

Theoretically it will be possible to get rid of fcntl(F_GETFL) in
fseek(), but O_APPEND flag need to be stored somewhere in that case, and
stdio _flags already have all bit occupied for 16bit short. So the price
will be changing size of the main stdio structure __sFILE to add new
space for flags, which is undesirable I think.

-- 
http://ache.vniz.net/
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [RFC] Patch to improve TSO limitation formula in general

[Eric Joyner]

Let me remove my concerns earlier in the thread -- this patch won't
negatively affect any of our drivers; and the problem I mentioned with ixl
would require a change somewhere further up the stack.

---
- Eric Joyner

On Mon, Sep 8, 2014 at 5:05 AM, Rick Macklem  wrote:

> Hans Petter Selasky wrote:
> > On 09/06/14 00:09, Rick Macklem wrote:
> > > Hans Petter Selesky wrote:
> > >> On 09/05/14 23:19, Eric Joyner wrote:
> > >>> There are some concerns if we use this with devices that ixl
> > >>> supports:
> > >>>
> > >>> - The maximum fragment size is 16KB-1, which isn't a power of 2.
> > >>>
> > >>
> > >> Hi Eric,
> > >>
> > >> Multiplying by powers of two are more fast, than non-powers of
> > >> two.
> > >> So
> > >> in this case you would have to use 8KB as a maximum.
> > >>
> > > Well, I'm no architecture expert, but I really doubt the CPU delay
> > > of a
> > > non-power of 2 multiply/divide is significant related to doing
> > > smaller
> > > TSO segments. Long ago (as in 1970s) I did work on machines where
> > > shifts
> > > for power of 2 multiply/divide was preferable, but these days I
> > > doubt it
> > > is going to matter??
> > >
> > >>> - You can't get the maximum TSO size for ixl devices by
> > >>> multiplying
> > >>> the
> > >>> maximum number of fragments by the maximum size.
> > >>> Instead the number of fragments is AFAIK unlimited, but a segment
> > >>> can only
> > >>> span 8 mbufs (including the [up to 3] mbufs containing the
> > >>> header),
> > >>> and the
> > >>> maximum TSO size is 256KB.
> >
> > Hi,
> >
> > Maybe that can be a separate parameter?
> >
> > I see that your patch assumes that a segment can be any-length. That
> > is
> > not always the case. Remember there are JUMBO mbufs too.
> >
> I thought JUMBO mbufs were only going to be used on the receive side,
> however I suppose if a packet is received into a JUMBO mbuf and then
> forwarded on another interface...
>
> > With my patch, the maximum segment size is a separate parameter. The
> > total number of TSO bytes is then not so useful.
> >
> Well, if you are referring to if_hw_tsomax, I'm not sure it was the
> best plan to begin with. It was implemented for xen and I'm not sure
> that any other driver uses it as of now.
>
> However...
> I'm not a network/hardware guy, but it seems some devices do have
> the IP_MAXPACKET limit (use the ip_len field in the ip header to
> know how large the TSO segment is). There is also at least one device
> (82598 chip for "ix" driver) that can handle more that IP_MAXPACKET,
> so it seems appropriate to have a value that the driver can set.
>
> Since the maximum size of the gather list for transmit does seem to
> vary a lot between devices, with several handling less than 35, it
> does seem appropriate to allow drivers to specify that.
>
> If devices can't handle a single gather entry over a certain size,
> I think that does need to be specified along with the max size of
> the gather list, since the driver will need to use multiple gather
> entries for a single mbuf and tcp_output() should take that into
> account.
>
> In summary, yep, I basically agree.
>
> rick
> ps: Who will look at your patch soon.
>
> > --HPS
> >
> > ___
> > freebsd-current@freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to
> > "freebsd-current-unsubscr...@freebsd.org"
> >
>
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Any recent spam that states it is from me is not.

[Joe Nosay]

Please check the headers and then ask me. This is for those who may assume
that  I do such things.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Jenkins build is back to normal : FreeBSD_HEAD #1412

[jenkins-admin]

See 

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"