Outdated IX intel driver in /usr/src tree
ix-3.3.32 is on intel website since back in July. https://www.intel.com/content/www/us/en/download/14303/intel-network-adapters-driver-for-pcie-10-gigabit-network-connections-under-freebsd.html /usr/src/sys/dev/ixgbe seems to be behind. Currently having issues with Intel 520 DA card hitting 10 gigabit speeds, last night booted same machine with Rocky Linux 9 on USB stick, no problem hitting 10gigabit with iperf3. Tried booting even a 13 release FreeBSD same issues, so nothing related to CURRENT. I've played with /boot/loader.conf and /etc/sysctl.conf tuning parameters to no avail, nothing seems to work, perhaps the updated driver may fix this? I notice there are new sysctl parameters under newest driver. router:/usr/src/sys/modules # uname -a FreeBSD router.sunsaturn.com 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n259557-4fa1e855be60: Sun Dec 4 22:28:09 CST 2022 d...@router.sunsaturn.com:/usr/obj/usr/src/amd64.amd64/sys/MYKERNEL amd64 router:/usr/src/sys/modules # Standard Dell r720 server. Dan.
Re: vfs.zfs.vol.recursive hang makes it impossible to mount zvol
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=267769 I'm just going to submit bug this way, I do like fact I can mount ZFS guests now, and I love being able to pass through multiple disks through with one line to bhyve such as : -s 4:0,virtio-scsi,/dev/cam/ctl1.0 very handy, and very organized especially if someone wanted to pass 10 devices through to a guest with a 1 liner to bhyve. I am not sure what is happening after resume, would be disheartening to have to go back to virtio-blk :( Think virtio-blk being phased out to virtio-scsi from what I have read. Dan. -- Dan The Man CEO & Founder Websites, Domains and Everything else http://www.SunSaturn.com/aboutus.php Email: d...@sunsaturn.com PGP Key: https://SunSaturn.com/pgp.txt A1A7 6E84 FB0B 8994 C3B5 A1BA FF6F 4997 7311 C386 On Mon, 7 Nov 2022, Alexander Motin wrote: On 07.11.2022 18:53, Dan The Man wrote: router:~ # sysctl vfs.zfs.vol.recursive=1 vfs.zfs.vol.recursive: 0 -> 1 router:~ # zpool import pool: testing id: 8013833172609421701 state: ONLINE action: The pool can be imported using its name or numeric identifier. config: testing ONLINE zvol/zroot/asterisk2p3 ONLINE router:~ # zpool import -fR /mnt testing This hangs forever The only way to import that pool from the zvol that I know of. Mounting ZFS from ZVOLs is blocked for a reason. It causes deadlocks due to lock recursion. I don't know what you are trying to achieve, but as alternatives, the ZVOL can be passed inside VM, it can be shared via iSCSI (even inside the host itself), etc. -- Alexander Motin
Re: vfs.zfs.vol.recursive hang makes it impossible to mount zvol
hold files when guest
suspended
SERIAL="/dev/nmdm_asteriskA" # For "screen /dev/nmdm_asteriskB"
- pkg install screen
TAP="tap0"
CPU="8"
RAM="8G"
#For testing virtio-scsi
STORAGE="/dev/cam/ctl1.0" # port from /etc/ctl.conf(port
ioctl/1) - core dumping on resume
DEVICE="virtio-scsi"
#for testing virtio-blk # Comment out above 2 lines if
using these
#DEVICE="virtio-blk"
#STORAGE="/dev/zvol/zroot/asterisk" # Standard zvol
#STORAGE="/dev/da1" # Block device created from
iscsictl
#
usage() {
echo "Usage: $1 start(Start the guest: $VMNAME)";
echo "Usage: $1 stop (Stop the guest: $VMNAME)";
echo "Usage: $1 resume (Resume the guest from last suspend:
$VMNAME)";
echo "Usage: $1 suspend (Suspend the guest: $VMNAME)";
echo "Usage: $1 install (Install new guest: $VMNAME)";
exit
}
if [ ! -d "$DIR" ]; then
mkdir -p $DIR
fi
#if [ -z "$2" ]; then
# usage
#else
# VMNAME=$2
#fi
if [ "$1" == "install" ]; then
#Kill it before starting it
echo "Execute: screen $SERIAL"
bhyvectl --destroy --vm=$VMNAME
bhyve -c $CPU -m $RAM -w -H -A \
-s 0:0,hostbridge \
-s 3:0,ahci-cd,$ISO \
-s 4:0,virtio-scsi,$STORAGE \
-s 5:0,virtio-net,$TAP \
-s 29,fbuf,tcp=$HOST:$PORT,w=$WIDTH,h=$HEIGHT \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
$VMNAME
#kill it after
bhyvectl --destroy --vm=$VMNAME
elif [ "$1" == "start" ]; then
while true
do
echo "Starting $VMNAME -s
29,fbuf,tcp=$HOST:$PORT,w=$WIDTH,h=$HEIGHT"
#Kill it before starting it
bhyvectl --destroy --vm=$VMNAME > /dev/null 2>&1
if [ -f "$DIR/default.ckp" ]; then
bhyve -c $CPU -m $RAM -w -H -A \
-s 0:0,hostbridge \
-s 4:0,virtio-scsi,$STORAGE \
-s 5:0,virtio-net,$TAP \
-s 29,fbuf,tcp=$HOST:$PORT,w=$WIDTH,h=$HEIGHT \
-s 30,xhci,tablet \
-s 31,lpc -l com1,$SERIAL \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
$VMNAME
fi
#DISABLING REBOOT LOOP AS SUSPEND RETURNS ERROR CODE 0 AS WELL
#if [ "$?" != 0 ];
#then
# echo "The exit code was not reboot code 0!: $?"
# exit
#fi
echo "The exit code was : $?"
exit
done
elif [ "$1" == "resume" ]; then
while true
do
echo "Starting $VMNAME -s
29,fbuf,tcp=$HOST:$PORT,w=$WIDTH,h=$HEIGHT"
#Kill it before starting it
bhyvectl --destroy --vm=$VMNAME > /dev/null 2>&1
if [ -f "$DIR/default.ckp" ]; then
bhyve -c $CPU -m $RAM -w -H -A \
-s 0:0,hostbridge \
-s 4:0,virtio-scsi,$STORAGE \
-s 5:0,virtio-net,$TAP \
-s 29,fbuf,tcp=$HOST:$PORT,w=$WIDTH,h=$HEIGHT \
-s 30,xhci,tablet \
-s 31,lpc -l com1,$SERIAL \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
-r $DIR/default.ckp \
$VMNAME
fi
#DISABLING REBOOT LOOP AS SUSPEND RETURNS ERROR CODE 0 AS WELL
#if [ "$?" != 0 ];
#then
# echo "The exit code was not reboot code 0!: $?"
# exit
#fi
echo "The exit code was : $?"
exit
done
elif [ "$1" == "suspend" ];
then
bhyvectl --suspend $DIR/default.ckp --vm=$VMNAME
elif [ "$1" == "stop" ]; then
bhyvectl --destroy --vm=$VMNAME
else
usage
fi
Dan.
--
Dan The Man
CEO & Founder
Websites, Domains and Everything else
http://www.SunSaturn.com/aboutus.php
Email: d...@sunsaturn.com
PGP Key: https://SunSaturn.com/pgp.txt
A1A7 6E84 FB0B 8994 C3B5 A1BA FF6F 4997 7311 C386
On Mon, 7 Nov 2022, Alexander Motin wrote:
On 07.11.2022 18:53, Dan The Man wrote:
router:~ # sysctl vfs.zfs.vol.recursive=1
vfs.zfs.vol.recursive: 0 -> 1
router:~ # zpool import
pool: testing
id: 8013833172609421701
state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:
testing ONLINE
zvol/zroot/asterisk2p3 ONLINE
router:~ # zpool import -fR /mnt testing
This hangs forever
The only way to import that pool from the zvol that I know of.
Mounting ZFS from ZVOLs is blocked for a reason. It causes deadlocks due to
lock recursion. I don't know what you are trying to achieve, but as
alternatives, the ZVOL can be passed inside VM, it can be shared via iSCSI
(even inside the host itself), etc.
--
Alexander Motin
vfs.zfs.vol.recursive hang makes it impossible to mount zvol
router:~ # uname -a FreeBSD router.sunsaturn.com 14.0-CURRENT FreeBSD 14.0-CURRENT #1 main-n259058-105019e0d6c: Sat Nov 5 05:37:28 CDT 2022 d...@router.sunsaturn.com:/usr/obj/usr/src/amd64.amd64/sys/MYKERNEL amd64 router:~ # zfs create -V30G -o volmode=full zroot/asterisk2 (install a ZFS guest on it and call zroot testing) router:~ # gpart show /dev/zvol/zroot/asterisk2 => 40 62914480 zvol/zroot/asterisk2 GPT (30G) 40532480 1 efi (260M) 532520 2008- free - (1.0M) 534528 16777216 2 freebsd-swap (8.0G) 17311744 45600768 3 freebsd-zfs (22G) 62912512 2008- free - (1.0M) router:~ # gdisk -l /dev/zvol/zroot/asterisk2 GPT fdisk (gdisk) version 1.0.9 Partition table scan: MBR: protective BSD: not present APM: not present GPT: present Found valid GPT with protective MBR; using GPT. Disk /dev/zvol/zroot/asterisk2: 62914560 sectors, 30.0 GiB Sector size (logical): 512 bytes Disk identifier (GUID): 8ACD2112-5EBF-11ED-8F56-00A098E3C14E Partition table holds up to 128 entries Main partition table begins at sector 2 and ends at sector 33 First usable sector is 40, last usable sector is 62914519 Partitions will be aligned on 8-sector boundaries Total free space is 4016 sectors (2.0 MiB) Number Start (sector)End (sector) Size Code Name 1 40 532519 260.0 MiB EF00 efiboot0 2 53452817311743 8.0 GiB A502 swap0 31731174462912511 21.7 GiBA504 zfs0 router:~ # zpool import pool: testing id: 8013833172609421701 state: UNAVAIL status: One or more devices are missing from the system. action: The pool cannot be imported. Attach the missing devices and try again. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-3C config: testing UNAVAIL insufficient replicas zvol/zroot/asterisk2p3 UNAVAIL cannot open router:~ # sysctl vfs.zfs.vol.recursive=1 vfs.zfs.vol.recursive: 0 -> 1 router:~ # zpool import pool: testing id: 8013833172609421701 state: ONLINE action: The pool can be imported using its name or numeric identifier. config: testing ONLINE zvol/zroot/asterisk2p3 ONLINE router:~ # zpool import -fR /mnt testing This hangs forever The only way to import that pool from the zvol that I know of..... Dan -- Dan The Man CEO & Founder Websites, Domains and Everything else http://www.SunSaturn.com/aboutus.php Email: d...@sunsaturn.com PGP Key: https://SunSaturn.com/pgp.txt A1A7 6E84 FB0B 8994 C3B5 A1BA FF6F 4997 7311 C386
Options for production testing under current(samba slow)
cd /usr/src/sys/amd64/conf cp GENERIC-NODEBUG MYKERNEL; (add: options BHYVE_SNAPSHOT) cd /usr/src make -j12 buildworld -DWITH_BHYVE_SNAPSHOT -DWITH_MALLOC_PRODUCTION make -j12 buildkernel KERNCONF=MYKERNEL make installkernel KERNCONF=MYKERNEL etc Been playing with current lately mostly to use bhyve suspend/resume(which works fine btw, other than core dumps if I left a cd device in there with an ISO etc). I have noticed on samba I am loosing 100-200MB/s transfer speeds now over 10gigabit ix0 device, I am wondering if I need more options than -DWITH_MALLOC_PRODUCTION? Dan. -- Dan The Man CEO & Founder Websites, Domains and Everything else http://www.SunSaturn.com/aboutus.php Email: d...@sunsaturn.com PGP Key: https://SunSaturn.com/pgp.txt A1A7 6E84 FB0B 8994 C3B5 A1BA FF6F 4997 7311 C386
sysctl kern.ipc.somaxconn limit 65535 why?
sunsaturn:~# sysctl -w kern.ipc.somaxconn=20 kern.ipc.somaxconn: 4096 sysctl: kern.ipc.somaxconn: Invalid argument sunsaturn:~# sysctl -w kern.ipc.somaxconn=65536 kern.ipc.somaxconn: 4096 sysctl: kern.ipc.somaxconn: Invalid argument sunsaturn:~# sysctl -w kern.ipc.somaxconn=65535 kern.ipc.somaxconn: 4096 - 65535 sunsaturn:~# Trying to stress test a framework here that tosses 100k of connections into a listen queue before doing anything, I realize I'll have to use multiple local IPs get get around port limitations, but why is this backlog using a limit? Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: sysctl kern.ipc.somaxconn limit 65535 why?
On Wed, 4 Jan 2012, Chuck Swiger wrote: On Jan 4, 2012, at 12:22 PM, Dan The Man wrote: Trying to stress test a framework here that tosses 100k of connections into a listen queue before doing anything, I realize I'll have to use multiple local IPs get get around port limitations, but why is this backlog using a limit? Even a backlog of a 1000 is large compared to the default listen queue size of around 50 or 128. And if you can drain 1000 connections per second, a 65K backlog is big enough that plenty of clients (I'm thinking web-browsers here in particular) will have given up and maybe retried rather than waiting for 60+ seconds just to exchange data. For web browsers makes sense, but if your coding your own server application its only a matter of increasing the read and write timeouts to fill queue that high and still process them. Of course wouldn't need anything that high, but for benchmarking how much can toss in that listen queue then write something to socket on each one after connection established to see how fast application can finish them all, I think its relevant. This linux box I have no issues: cappy:~# /sbin/sysctl -w net.core.somaxconn=20 net.core.somaxconn = 20 cappy:~# sysctl -w net.ipv4.tcp_max_syn_backlog=2 net.ipv4.tcp_max_syn_backlog = 20 cappy:~# Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: sysctl kern.ipc.somaxconn limit 65535 why?
On Wed, 4 Jan 2012, Chuck Swiger wrote: On Jan 4, 2012, at 12:44 PM, Dan The Man wrote: Even a backlog of a 1000 is large compared to the default listen queue size of around 50 or 128. And if you can drain 1000 connections per second, a 65K backlog is big enough that plenty of clients (I'm thinking web-browsers here in particular) will have given up and maybe retried rather than waiting for 60+ seconds just to exchange data. For web browsers makes sense, but if your coding your own server application its only a matter of increasing the read and write timeouts to fill queue that high and still process them. Sure, agreed. Of course wouldn't need anything that high, but for benchmarking how much can toss in that listen queue then write something to socket on each one after connection established to see how fast application can finish them all, I think its relevant. This linux box I have no issues: cappy:~# /sbin/sysctl -w net.core.somaxconn=20 net.core.somaxconn = 20 cappy:~# sysctl -w net.ipv4.tcp_max_syn_backlog=2 net.ipv4.tcp_max_syn_backlog = 20 cappy:~# However, I'm not convinced that it is useful to do this. At some point, you are better off timing out and retrying via exponential backoff than you are queuing hundreds of thousands of connections in the hopes that they will eventually be serviced by something sometime considerably later. I agree completely, in practical application this makes sense, but why should the OS dictate not being able to temporarily set that setting higher in order to fully benchmark the application at 100k+ in the listen queue if the developer so chooses? I think that alone should be a good reason, to make freebsd developer friendly. Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: sysctl kern.ipc.somaxconn limit 65535 why?
On Wed, 4 Jan 2012, Dan The Man wrote: On Wed, 4 Jan 2012, Chuck Swiger wrote: On Jan 4, 2012, at 12:44 PM, Dan The Man wrote: Even a backlog of a 1000 is large compared to the default listen queue size of around 50 or 128. And if you can drain 1000 connections per second, a 65K backlog is big enough that plenty of clients (I'm thinking web-browsers here in particular) will have given up and maybe retried rather than waiting for 60+ seconds just to exchange data. For web browsers makes sense, but if your coding your own server application its only a matter of increasing the read and write timeouts to fill queue that high and still process them. Sure, agreed. Of course wouldn't need anything that high, but for benchmarking how much can toss in that listen queue then write something to socket on each one after connection established to see how fast application can finish them all, I think its relevant. This linux box I have no issues: cappy:~# /sbin/sysctl -w net.core.somaxconn=20 net.core.somaxconn = 20 cappy:~# sysctl -w net.ipv4.tcp_max_syn_backlog=2 net.ipv4.tcp_max_syn_backlog = 20 cappy:~# However, I'm not convinced that it is useful to do this. At some point, you are better off timing out and retrying via exponential backoff than you are queuing hundreds of thousands of connections in the hopes that they will eventually be serviced by something sometime considerably later. I agree completely, in practical application this makes sense, but why should the OS dictate not being able to temporarily set that setting higher in order to fully benchmark the application at 100k+ in the listen queue if the developer so chooses? I think that alone should be a good reason, to make freebsd developer friendly. Anyways its not a big deal I can live with a 60k or so benchmark, I just really wanted to see how freebsd's kqueue would perform processing that many connections right out of the listen queue. Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: sysctl kern.ipc.somaxconn limit 65535 why?
On Wed, 4 Jan 2012, Chuck Swiger wrote: On Jan 4, 2012, at 1:49 PM, Arnaud Lacombe wrote: Hi, On Wed, Jan 4, 2012 at 4:42 PM, Chuck Swiger cswi...@mac.com wrote: On Jan 4, 2012, at 1:03 PM, Dan The Man wrote: However, I'm not convinced that it is useful to do this. At some point, you are better off timing out and retrying via exponential backoff than you are queuing hundreds of thousands of connections in the hopes that they will eventually be serviced by something sometime considerably later. I agree completely, in practical application this makes sense, but why should the OS dictate not being able to temporarily set that setting higher in order to fully benchmark the application at 100k+ in the listen queue if the developer so chooses? I think that alone should be a good reason, to make freebsd developer friendly. The job of the OS is to manage resources on behalf of the users and processes using the system. No. The job of the OS is to service the user with the resource available, not constrict the user within some arbitrary predefined wall when there is still plenty of room available. If resource become scarce, then take action. It is not arbitrary. Systems ought to provide sensible limits, which can be adjusted if needed and appropriate. The fact that a system might have 50,000 file descriptors globally available does not mean that it would be OK for any random process to consume half of them, even if there is still adequate room left for other tasks. It's common for ulimit -n to be set to 256 or 1024. Sensibly limits means a sensible stock default, not imposing an OS limit on what admin/developer can set on his own hardware. With the new IBM developments underway of 16 core atom processors and hundreds of gigabytes of memory, surely a backlog of 100k is manageable. Or what about the future of 500 core systems with a terrabyte of memory, 100k listen queue could be processed instantly. Some developers feel that VM means that the system should always claim have more memory available, but always saying yes isn't managing resources. I'd rather have the OS return a null pointer and set ENOMEM when someone tries to malloc() more memory than the system (including swap, VM overcommit, etc) has, and I expect developers to code well enough to handle malloc() failures. this is merely a policy issue, not yours to impose. If we're speaking of machines which I administer, it is a policy issue that would be mine to impose. If we're speaking of someone else's machines, then they can set their own policies as they please. Setting the listen queue to an arbitrarily high value isn't useful, and developers would be better advised to pay attention to best practices in the face of a massive connection backlog. Stress-testing isn't about best practice. It is about shaking enough the system to highlight weak point. Yes. If the system doesn't handle connectivity problems via something like exponential backoff, then the weak point is poor software design and not FreeBSD being unwilling to set the socket listen queue to a value in the hundreds of thousands. I think what me and Arnaud are trying to say here, is let freebsd use a sensible default value, but let the admin dictate the actual policy if he so chooses to change it for stress testing, future proofing or anything else. Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: sysctl kern.ipc.somaxconn limit 65535 why?
On Wed, 4 Jan 2012, Dan The Man wrote: On Wed, 4 Jan 2012, Chuck Swiger wrote: On Jan 4, 2012, at 1:49 PM, Arnaud Lacombe wrote: Hi, On Wed, Jan 4, 2012 at 4:42 PM, Chuck Swiger cswi...@mac.com wrote: On Jan 4, 2012, at 1:03 PM, Dan The Man wrote: However, I'm not convinced that it is useful to do this. At some point, you are better off timing out and retrying via exponential backoff than you are queuing hundreds of thousands of connections in the hopes that they will eventually be serviced by something sometime considerably later. I agree completely, in practical application this makes sense, but why should the OS dictate not being able to temporarily set that setting higher in order to fully benchmark the application at 100k+ in the listen queue if the developer so chooses? I think that alone should be a good reason, to make freebsd developer friendly. The job of the OS is to manage resources on behalf of the users and processes using the system. No. The job of the OS is to service the user with the resource available, not constrict the user within some arbitrary predefined wall when there is still plenty of room available. If resource become scarce, then take action. It is not arbitrary. Systems ought to provide sensible limits, which can be adjusted if needed and appropriate. The fact that a system might have 50,000 file descriptors globally available does not mean that it would be OK for any random process to consume half of them, even if there is still adequate room left for other tasks. It's common for ulimit -n to be set to 256 or 1024. Sensibly limits means a sensible stock default, not imposing an OS limit on what admin/developer can set on his own hardware. With the new IBM developments underway of 16 core atom processors and hundreds of gigabytes of memory, surely a backlog of 100k is manageable. Or what about the future of 500 core systems with a terrabyte of memory, 100k listen queue could be processed instantly. Some developers feel that VM means that the system should always claim have more memory available, but always saying yes isn't managing resources. I'd rather have the OS return a null pointer and set ENOMEM when someone tries to malloc() more memory than the system (including swap, VM overcommit, etc) has, and I expect developers to code well enough to handle malloc() failures. this is merely a policy issue, not yours to impose. If we're speaking of machines which I administer, it is a policy issue that would be mine to impose. If we're speaking of someone else's machines, then they can set their own policies as they please. Setting the listen queue to an arbitrarily high value isn't useful, and developers would be better advised to pay attention to best practices in the face of a massive connection backlog. Stress-testing isn't about best practice. It is about shaking enough the system to highlight weak point. Yes. If the system doesn't handle connectivity problems via something like exponential backoff, then the weak point is poor software design and not FreeBSD being unwilling to set the socket listen queue to a value in the hundreds of thousands. I think what me and Arnaud are trying to say here, is let freebsd use a sensible default value, but let the admin dictate the actual policy if he so chooses to change it for stress testing, future proofing or anything else. How about a sensible solution? I think everyone has been making valid points here, about sensible limits for all programs on system and per application limit changes. How about changing the hard limit high, and an application can make the soft limit higher as it sees fit, its a win win, like ulimit does with openfiles and such. Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: sysctl kern.ipc.somaxconn limit 65535 why?
On Wed, 4 Jan 2012, Chuck Swiger wrote: Hi-- On Jan 4, 2012, at 2:23 PM, Dan The Man wrote: It is not arbitrary. Systems ought to provide sensible limits, which can be adjusted if needed and appropriate. The fact that a system might have 50,000 file descriptors globally available does not mean that it would be OK for any random process to consume half of them, even if there is still adequate room left for other tasks. It's common for ulimit -n to be set to 256 or 1024. Sensibly limits means a sensible stock default, not imposing an OS limit on what admin/developer can set on his own hardware. In point of fact, protocols like TCP/IP impose limits on what is possible. It is in fact the job of the OS to say no when a developer asks for a TTL of a million via setsockopt(), because RFC-791 limits the maximum value of the time to live field to 255. With the new IBM developments underway of 16 core atom processors and hundreds of gigabytes of memory, surely a backlog of 100k is manageable. Or what about the future of 500 core systems with a terrabyte of memory, 100k listen queue could be processed instantly. Um. I gather you don't have much background in operating system design or massively parallelized systems? Due to locking constraints imposed by whatever synchronization mechanism and communications topology is employed between cores, you simply cannot just add more processors to a system and expect it to go faster in a linear fashion. Having 500 cores contending over a single queue is almost certain to result in horrible performance. Even though the problem of a bunch of independent requests is embarrassingly parallelizeable, you do that by partitioning the queue into multiple pieces that are fed to different groups or pools of processors to minimize contention over a single data structure. I guess your calling me out to talk about what I'm doing based on that statement: First framework I was working on a few weeks back just had a parent bind socket, then spawn a certain amount of children to do the accept on the socket, so parent could just focus on dealing with SIGCHLD and what not. I had issues with this design for some reason, all the sockets were set to non-blocking etc, and using kqueue to monitor the socket, but randomly I would have a 1-2 second delay at times from a child doing an accept, I was horrified and changed design quickly. New design, parent does all the accepts and passes blocking work to children via socketpairs it created when forking. Now you talk about scaling on multiple cores, well each child could have its own core to do its blocking I/O on and each have its own processor time, which isn't parallism , but I never said it was doing that. The better part of this design is you have 1 process utilizing a processor efficiently instead of paging the system with useless processes. Also could could have other machines connect in to parent and it could do same thing it does with children via a socket, so in my opinion its more scalable and can centralize everything in one spot. Obviously some cons to this design, you are passing data via socket pairs instead of child writing directly to client. To stress test this new design I simply wrote an asycronouse client counterpart to create 100k of connections to parents listen queue, then it would go off writing to each socket, of course soon as I reached 60k or so client would get numerous failures due to OS limits. So my intention was to see how long it would take children to process request and send response back to client, starting from listen queue with 100k of fd's ready to go I thought would have been really nice test not only for testing applications speed but also testing cpu usage, I/O usage etc with parent processing a client trying to talk to it 100k times at once to really see how kqueue does. Without being able to increase simple limits like these how ever going to find where we can burn down the system and make it outperform epoll() one day. What it so bad to see how many fd's I could toss at kqueue before it croaked? @60k was still handling like a champ with about 50 children getting handed work in my tests. Yes. If the system doesn't handle connectivity problems via something like exponential backoff, then the weak point is poor software design and not FreeBSD being unwilling to set the socket listen queue to a value in the hundreds of thousands. I think what me and Arnaud are trying to say here, is let freebsd use a sensible default value, but let the admin dictate the actual policy if he so chooses to change it for stress testing, future proofing or anything else. FreeBSD does provide a sensible default value for the listen queue size. It's tunable to a factor of about 1000 times larger, and is a value which is sufficiently large to hold a backlog of several minutes worth of connections, assuming you can process the requests
jexec -h hostname option
http://www.freebsd.org/relnotes/CURRENT/relnotes/new.html#USERLAND jexec(8) now supports -h hostname option to specify the jail where the command will be executed. When was this added? I don't see it functioning: sunsaturn:~# jexec -h jexec: illegal option -- h usage: jexec [-u username | -U username] jail command ... sunsaturn:~# uname -a FreeBSD sunsaturn.com 9.0-PRERELEASE FreeBSD 9.0-PRERELEASE #0: Sun Dec 11 19:20:46 CST 2011 dr...@sunsaturn.com:/usr/obj/usr/src/sys/MYKERNEL amd64 sunsaturn:~# Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: jexec -h hostname option
On Thu, 22 Dec 2011, Bjoern A. Zeeb wrote: On 22. Dec 2011, at 14:03 , Dan The Man wrote: http://www.freebsd.org/relnotes/CURRENT/relnotes/new.html#USERLAND jexec(8) now supports -h hostname option to specify the jail where the command will be executed. Oh wow. That's all but current. When was this added? I don't see it functioning: 3 years 6 months ago and it was shortly afterwards removed again as neither a) the hostname not b) the ip addresses needed to be unique anymore with multi-IP jails (a) not even before that). The suggested replacement was -n to name the jails yourself. I think the uniqueness limit has since been removed on that as well but the option has stayed and by default is the jail ID these days and it's name=.. in the modern syntax. /bz -- Bjoern A. Zeeb You have to have visions! Stop bit received. Insert coin for new address family. Yeah, seems problematic, from what I have seen so far everytime you stop and restart the jail it gets a different jail ID, which would make it difficult to cron anything to execute in the jail. I can't seem to get jexec to take anything but jail id. Came up with a temporary type solution assuming you have only 1 jail: JAILID=`/usr/sbin/jls -n name|cut -d '=' -f 2`; /usr/sbin/jexec $JAILID command I can see this being problematic for a long term/portable solution. Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
pf.conf + IPV6 to IPV4 port rdr
pfctl -v -s nat rdr inet6 proto tcp from any to 2001:49f0:4004::/48 port = 9191 - :::67.159.46.238 [ Evaluations: 512 Packets: 3 Bytes: 228 States: 1 ] [ Inserted: uid 0 pid 80940 State Creations: 2 ] I can see here that after i tried on another host to telnet to 2001:49f0:4004::2 9191 , that a state was in fact created for the rdr, but it doesn't appear to be actually forwarding: My rule: rdr inet6 proto tcp to 2001:49f0:4004::/48 port 9191 - :::67.159.46.238 Am I missing something here? I have checked on ipv6 forwarding and redirects set to 1, net.inet6.ip6.v6only=0 to allow the mapping... I can even telnet to :::67.159.46.238 9191 from any host yet it will not forward the 2001:49f0:4004:: addresses, and yes inet6 is allowing the port to pass, so this makes no sense to me Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
rc.conf changes IPV6
I notice we have changed way IPV6 is done in rc.conf now. I assume someone will update: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/network-ipv6.html My question now concerns aliases, and what the norm will end up being. Here is example below: Here we have the new layout with IPV6, the below works fine, however since the ifconfig lines for IPV4 and IPV6 are essentially the same other than the actual inet and inet6, will it be ok to start with alias0 for both IPV4 and IPV6, or should I in this example be starting at alias4 for IPV6? I would like idea to keep it way it is each protocal starting at alias0. #GATEWAY defaultrouter=67.159.46.233 hostname=sunsaturn.com #IPV4 ifconfig_em1=inet 67.159.46.238 netmask 255.255.255.248 ifconfig_em1_alias0=inet 67.159.46.234 netmask 255.255.255.248 ifconfig_em1_alias1=inet 67.159.46.235 netmask 255.255.255.248 ifconfig_em1_alias2=inet 67.159.46.236 netmask 255.255.255.248 ifconfig_em1_alias3=inet 67.159.46.237 netmask 255.255.255.248 #IPV6 ipv6_activate_all_interfaces=YES ipv6_network_interfaces=em1 ipv6_defaultrouter=2001:49f0:4004:::::0001 ifconfig_em1_ipv6=inet6 2001:49f0:4004:::::0002 prefixlen 48 ifconfig_em1_alias0=inet6 2001:49f0:4004:::::0003 prefixlen 48 ifconfig_em1_alias1=inet6 2001:49f0:4004:::::0004 prefixlen 48 Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: rc.conf changes IPV6
On Mon, 21 Nov 2011, Stefan Bethke wrote: Am 21.11.2011 um 20:25 schrieb Dan The Man: I notice we have changed way IPV6 is done in rc.conf now. I assume someone will update: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/network-ipv6.html My question now concerns aliases, and what the norm will end up being. Here is example below: Here we have the new layout with IPV6, the below works fine, however since the ifconfig lines for IPV4 and IPV6 are essentially the same other than the actual inet and inet6, will it be ok to start with alias0 for both IPV4 and IPV6, or should I in this example be starting at alias4 for IPV6? I would like idea to keep it way it is each protocal starting at alias0. #GATEWAY defaultrouter=67.159.46.233 hostname=sunsaturn.com #IPV4 ifconfig_em1=inet 67.159.46.238 netmask 255.255.255.248 ifconfig_em1_alias0=inet 67.159.46.234 netmask 255.255.255.248 ifconfig_em1_alias1=inet 67.159.46.235 netmask 255.255.255.248 ifconfig_em1_alias2=inet 67.159.46.236 netmask 255.255.255.248 ifconfig_em1_alias3=inet 67.159.46.237 netmask 255.255.255.248 #IPV6 ipv6_activate_all_interfaces=YES ipv6_network_interfaces=em1 ipv6_defaultrouter=2001:49f0:4004:::::0001 ifconfig_em1_ipv6=inet6 2001:49f0:4004:::::0002 prefixlen 48 ifconfig_em1_alias0=inet6 2001:49f0:4004:::::0003 prefixlen 48 ifconfig_em1_alias1=inet6 2001:49f0:4004:::::0004 prefixlen 48 Remember that rc.conf follows shell syntax and sematics, so the second _alias0 and _alias1 will overwrite the previous ones. In 9.0 you can use the ipv4_addrs_ifname variable to set both the IPv4 main address as well as alias addresses, see rc.conf(5). There doesn't seem to be an equivalent IPv6 option, as best as I can tell. Stefan -- Stefan Bethke s...@lassitu.de Fon +49 151 14070811 You would assume so Stefan, that the duplicate alias0 would overwrite, but it seems since ifconfig separates the namespace for IPV4/IPV6 it actually works. Which is good news I would assume, as I have time to time added a new IPV4 alias, and would have forgotten to update IPV6 alias numbers many times unless I revise my config to put ipv4 and ipv6 aliases together in rc.conf instead of separating the protocals in there. I looked at man page, thats a very nice config option ipv4_addrs_ifname, I agree it would be nice to have an IPV6 equivalent considering how big the IPV6 range is. Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: rc.conf changes IPV6
Good point, I did switch to new config, let me find a box I can take down and I'll report back. I assume you'd be right considering how shell scripting works, that would not make much sense... Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com On Mon, 21 Nov 2011, Doug Barton wrote: On 11/21/2011 17:39, Dan The Man wrote: On Mon, 21 Nov 2011, Stefan Bethke wrote: Am 21.11.2011 um 20:25 schrieb Dan The Man: I notice we have changed way IPV6 is done in rc.conf now. I assume someone will update: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/network-ipv6.html My question now concerns aliases, and what the norm will end up being. Here is example below: Here we have the new layout with IPV6, the below works fine, however since the ifconfig lines for IPV4 and IPV6 are essentially the same other than the actual inet and inet6, will it be ok to start with alias0 for both IPV4 and IPV6, or should I in this example be starting at alias4 for IPV6? I would like idea to keep it way it is each protocal starting at alias0. #GATEWAY defaultrouter=67.159.46.233 hostname=sunsaturn.com #IPV4 ifconfig_em1=inet 67.159.46.238 netmask 255.255.255.248 ifconfig_em1_alias0=inet 67.159.46.234 netmask 255.255.255.248 ifconfig_em1_alias1=inet 67.159.46.235 netmask 255.255.255.248 ifconfig_em1_alias2=inet 67.159.46.236 netmask 255.255.255.248 ifconfig_em1_alias3=inet 67.159.46.237 netmask 255.255.255.248 #IPV6 ipv6_activate_all_interfaces=YES ipv6_network_interfaces=em1 ipv6_defaultrouter=2001:49f0:4004:::::0001 ifconfig_em1_ipv6=inet6 2001:49f0:4004:::::0002 prefixlen 48 ifconfig_em1_alias0=inet6 2001:49f0:4004:::::0003 prefixlen 48 ifconfig_em1_alias1=inet6 2001:49f0:4004:::::0004 prefixlen 48 Remember that rc.conf follows shell syntax and sematics, so the second _alias0 and _alias1 will overwrite the previous ones. In 9.0 you can use the ipv4_addrs_ifname variable to set both the IPv4 main address as well as alias addresses, see rc.conf(5). There doesn't seem to be an equivalent IPv6 option, as best as I can tell. You would assume so Stefan, that the duplicate alias0 would overwrite, but it seems since ifconfig separates the namespace for IPV4/IPV6 it actually works. I don't see how that could possibly be true, how have you tested it? Doug -- We could put the whole Internet into a book. Too practical. Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: rc.conf changes IPV6
Your right Doug, didn't bother taking box down after changing to new format, it works as expected, ignoring duplicate aliases regardless of protocal. I've regrouped my configs into interfaces instead of protocals now for the change so ipv4/6 aliases can be seen together without making mistakes. Definately would be nice to see ipv6_addrs_ifname added. Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com On Mon, 21 Nov 2011, Dan The Man wrote: Good point, I did switch to new config, let me find a box I can take down and I'll report back. I assume you'd be right considering how shell scripting works, that would not make much sense... Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com On Mon, 21 Nov 2011, Doug Barton wrote: On 11/21/2011 17:39, Dan The Man wrote: On Mon, 21 Nov 2011, Stefan Bethke wrote: Am 21.11.2011 um 20:25 schrieb Dan The Man: I notice we have changed way IPV6 is done in rc.conf now. I assume someone will update: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/network-ipv6.html My question now concerns aliases, and what the norm will end up being. Here is example below: Here we have the new layout with IPV6, the below works fine, however since the ifconfig lines for IPV4 and IPV6 are essentially the same other than the actual inet and inet6, will it be ok to start with alias0 for both IPV4 and IPV6, or should I in this example be starting at alias4 for IPV6? I would like idea to keep it way it is each protocal starting at alias0. #GATEWAY defaultrouter=67.159.46.233 hostname=sunsaturn.com #IPV4 ifconfig_em1=inet 67.159.46.238 netmask 255.255.255.248 ifconfig_em1_alias0=inet 67.159.46.234 netmask 255.255.255.248 ifconfig_em1_alias1=inet 67.159.46.235 netmask 255.255.255.248 ifconfig_em1_alias2=inet 67.159.46.236 netmask 255.255.255.248 ifconfig_em1_alias3=inet 67.159.46.237 netmask 255.255.255.248 #IPV6 ipv6_activate_all_interfaces=YES ipv6_network_interfaces=em1 ipv6_defaultrouter=2001:49f0:4004:::::0001 ifconfig_em1_ipv6=inet6 2001:49f0:4004:::::0002 prefixlen 48 ifconfig_em1_alias0=inet6 2001:49f0:4004:::::0003 prefixlen 48 ifconfig_em1_alias1=inet6 2001:49f0:4004:::::0004 prefixlen 48 Remember that rc.conf follows shell syntax and sematics, so the second _alias0 and _alias1 will overwrite the previous ones. In 9.0 you can use the ipv4_addrs_ifname variable to set both the IPv4 main address as well as alias addresses, see rc.conf(5). There doesn't seem to be an equivalent IPv6 option, as best as I can tell. You would assume so Stefan, that the duplicate alias0 would overwrite, but it seems since ifconfig separates the namespace for IPV4/IPV6 it actually works. I don't see how that could possibly be true, how have you tested it? Doug -- We could put the whole Internet into a book. Too practical. Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
spamcop abuse of power
Today I had an unhappy unix student try to submit an assignment to me and could not. Spamcop has decided to go off blacklisting all yahoo/shaw etc servers worldwide. Example Solution Postfix: remove: reject_rbl_client bl.spamcop.net from your smtpd_recipient_restrictions line until they fix their abuse issues. Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
NFSV4 readlink_stat
) rsync: readlink_stat(/asterisk/public/mp3/Kass Tunes/Z Too Small to Seperate/Z Favorite WMA - Songs/Heavy Metal/Motely Crue/M\#366tley Cr\#374e - Live Wire [Kick Ass '91 Remix].wma) failed: Invalid argument (22) rsync: readlink_stat(/asterisk/public/mp3/Kass Tunes/Z Too Small to Seperate/Z Favorite WMA - Songs/Heavy Metal/Motely Crue/M\#366tley Cr\#374e - Black Widow.wma) failed: Invalid argument (22) rsync: readlink_stat(/asterisk/public/mp3/Kass Tunes/Z Too Small to Seperate/Z Favorite WMA - Songs/Heavy Metal/Motely Crue/M\#366tley Cr\#374e - Rock 'N' Roll Junkie.wma) failed: Invalid argument (22) rsync: readlink_stat(/asterisk/public/mp3/Kass Tunes/Z Too Small to Seperate/Z Favorite WMA - Songs/Heavy Metal/Motely Crue/Motley\#240Crue - Loveshine.wma) failed: Invalid argument (22) rsync: readlink_stat(/asterisk/public/mp3/Kass Tunes/Z Too Small to Seperate/Z Favorite WMA - Songs/Heavy Metal/Motely Crue/M\#366tley Cr\#374e - Dragstrip Superstar.wma) failed: Invalid argument (22) rsync: readlink_stat(/asterisk/public/mp3/Kass Tunes/Z Too Small to Seperate/Z Favorite WMA - Songs/Heavy Metal/Motely Crue/M\#366tley Cr\#374e - Hammered - 07.wma) failed: Invalid argument (22) rsync: readlink_stat(/asterisk/public/mp3/Kass Tunes/Z Too Small to Seperate/Z Favorite WMA - Songs/Heavy Metal/Motely Crue/M\#366tley Cr\#374e - Poison Apples.wma) failed: Invalid argument (22) rsync: readlink_stat(/asterisk/public/mp3/Kass Tunes/Z Too Small to Seperate/Z Favorite WMA - Songs/Heavy Metal/Motely Crue/M\#366tley Cr\#374e - Afraid.wma) failed: Invalid argument (22) rsync: readlink_stat(/asterisk/public/mp3/Kass Tunes/Z Too Small to Seperate/Z Favorite WMA - Songs/Heavy Metal/Motely Crue/M\#366tley Cr\#374e - Kiss the Sky.wma) failed: Invalid argument (22) rsync: readlink_stat(/asterisk/public/mp3/Kass Tunes/Z Too Small to Seperate/Z Favorite WMA - Songs/Heavy Metal/Motely Crue/M\#366tley Cr\#374e - Glitter - Generation Swine - 07.wma) failed: Invalid argument (22) rsync: readlink_stat(/asterisk/public/mp3/Kass Tunes/Z Too Small to Seperate/Z Favorite WMA - Songs/Original Soundtrack/Various Artists/Various Artists/Pure Dance 1998/U2 Discoth\#350que [DM Radio Mix] 07 Pure Dance 1998 Rock.mp3) failed: Invalid argument (22) rsync: readlink_stat(/asterisk/public/mp3/Kass Tunes/Z Too Small to Seperate/Z Favorite WMA - Songs/Original Soundtrack/Various Artists/Various Artists/Pure Dance 1998/Tony! Toni! Ton\#351! Let's Get Down [Fitch Bros. Club Radio Edit] 06 Pure Dance 1998 Rock.mp3) failed: Invalid argument (22) rsync: readlink_stat(/asterisk/public/mp3/Kass Tunes/Z Too Small to Seperate/Z Favorite WMA - Songs/Rock n Roll/Q-S/Q/Queensryche/Queensr\#377che - Best I Can - 01.wma) failed: Invalid argument (22) Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: samba+zfs
Running tcpdump to trace what samba is doing so maybe someone can give some insight, lan interface is sk0. 02:52:34.347357 IP (tos 0x0, ttl 64, id 56121, offset 0, flags [none], proto TCP (6), length 1500) asterisk.microsoft-ds desktop.58858: Flags [.], cksum 0x5e3f (correct), seq 105687574:105689034, ack 63894978, win 256, length 1460SMB-over-TCP packet:(raw data or continuation?) 02:52:34.347361 IP (tos 0x0, ttl 64, id 56122, offset 0, flags [none], proto TCP (6), length 1500) asterisk.microsoft-ds desktop.58858: Flags [.], cksum 0xbe99 (correct), seq 105689034:105690494, ack 63894978, win 256, length 1460SMB-over-TCP packet:(raw data or continuation?) 02:52:34.347365 IP (tos 0x0, ttl 64, id 56123, offset 0, flags [none], proto TCP (6), length 1500) asterisk.microsoft-ds desktop.58858: Flags [.], cksum 0x0b31 (correct), seq 105690494:105691954, ack 63894978, win 256, length 1460SMB-over-TCP packet:(raw data or continuation?) 02:52:34.347369 IP (tos 0x0, ttl 64, id 56124, offset 0, flags [none], proto TCP (6), length 1500) asterisk.microsoft-ds desktop.58858: Flags [.], cksum 0xaee6 (correct), seq 105691954:105693414, ack 63894978, win 256, length 1460SMB-over-TCP packet:(raw data or continuation?) 02:52:34.347372 IP (tos 0x0, ttl 64, id 56125, offset 0, flags [none], proto TCP (6), length 1500) asterisk.microsoft-ds desktop.58858: Flags [.], cksum 0xcff5 (correct), seq 105693414:105694874, ack 63894978, win 256, length 1460SMB-over-TCP packet:(raw data or continuation?) 02:52:34.347376 IP (tos 0x0, ttl 64, id 56126, offset 0, flags [none], proto TCP (6), length 1500) asterisk.microsoft-ds desktop.58858: Flags [.], cksum 0xd893 (correct), seq 105694874:105696334, ack 63894978, win 256, length 1460SMB-over-TCP packet:(raw data or continuation?) We get a ton of these, my mapped samba drive on Z: becomes nearly unresponsive after i start transferring things through it, yet I can jump on Y: drive which is NFS mount to same interface on machine and everything is fine and responsive Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com On Sat, 12 Nov 2011, Dan The Man wrote: Well been running a week now and problems again. 3 3 terrabyte drives are @85% with compression enabled, i have to wonder if that is part of the problem. Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com On Wed, 9 Nov 2011, Kurt Touet wrote: On Wed, Nov 9, 2011 at 1:07 AM, Daniel O'Connor docon...@gsoft.com.au wrote: On 09/11/2011, at 17:32, Garrett Cooper wrote dd's of large files (spooled backups going to tape) to /dev/null are as slow as Samba. - Dedupe? Nope. - Compression? On the mail spool ports, but not on the tape spool. - How much RAM? 8GB. - What debug options do you have enabled in the kernel? It is 8.2-GENERIC so.. no WITNESS (for example) I've been noticing a slowdown in some respects with NFS/SMB, but I suspected it was because I have an re(4) based NIC. ZFS has also wired down a lot of my system memory for the L2ARC… re isn't great but I wouldn't expect it to slow down over time.. Unless bounce buffers got used more and more or something. I have an em0 card in this system - but in any case it is slow locally (i.e. dd a large file with 64k block size). -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au The nice thing about standards is that there are so many of them to choose from. -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C Right now (while experience slow writes via samba+zfs) this is general read speed off a 4 x 1.5TB sata2 raidz1: # dd if=test.file of=/dev/null 13753502+1 records in 13753502+1 records out 7041793036 bytes transferred in 100.020897 secs (70403218 bytes/sec) That's not in the same ball park of slow writes, but it is below what I expect for reads. My setup is a little odd: 4x1.5tb raidz sata2 on mobo + 2 x 2tb mirror on sata1 pci controller, zfs v28, stable/9 r227357, amd x4 810 2.6ghz, 4gb ram, no dedupe, no compression, daily snapshots saved for 7 days The above file read was stored before the 2 x 2tb mirror addition, so it was a solely read off the sata2 mobo ports. Reading off of something more recent (and split amongst both raidz1 and mirror vdevs): # dd if=test2.file of=/dev/null 9154715+1 records in 9154715+1 records out 4687214153 bytes transferred in 82.963181 secs (56497522 bytes/sec) This is, again, seems slower than usual, but not as terrible as the write speeds that I've been seeing via samba. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd
Re: samba+zfs
(0x7fffd170,0x2,0x7fffd18c,0x7fffd180,0x0,0x0) = 0 (0x0) It actually seems to be running some timed event smbd_idle literally holding up process for many seconds all the time Dan, -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: samba+zfs
Well been running a week now and problems again. 3 3 terrabyte drives are @85% with compression enabled, i have to wonder if that is part of the problem. Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com On Wed, 9 Nov 2011, Kurt Touet wrote: On Wed, Nov 9, 2011 at 1:07 AM, Daniel O'Connor docon...@gsoft.com.au wrote: On 09/11/2011, at 17:32, Garrett Cooper wrote dd's of large files (spooled backups going to tape) to /dev/null are as slow as Samba. - Dedupe? Nope. - Compression? On the mail spool ports, but not on the tape spool. - How much RAM? 8GB. - What debug options do you have enabled in the kernel? It is 8.2-GENERIC so.. no WITNESS (for example) I've been noticing a slowdown in some respects with NFS/SMB, but I suspected it was because I have an re(4) based NIC. ZFS has also wired down a lot of my system memory for the L2ARC… re isn't great but I wouldn't expect it to slow down over time.. Unless bounce buffers got used more and more or something. I have an em0 card in this system - but in any case it is slow locally (i.e. dd a large file with 64k block size). -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au The nice thing about standards is that there are so many of them to choose from. -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C Right now (while experience slow writes via samba+zfs) this is general read speed off a 4 x 1.5TB sata2 raidz1: # dd if=test.file of=/dev/null 13753502+1 records in 13753502+1 records out 7041793036 bytes transferred in 100.020897 secs (70403218 bytes/sec) That's not in the same ball park of slow writes, but it is below what I expect for reads. My setup is a little odd: 4x1.5tb raidz sata2 on mobo + 2 x 2tb mirror on sata1 pci controller, zfs v28, stable/9 r227357, amd x4 810 2.6ghz, 4gb ram, no dedupe, no compression, daily snapshots saved for 7 days The above file read was stored before the 2 x 2tb mirror addition, so it was a solely read off the sata2 mobo ports. Reading off of something more recent (and split amongst both raidz1 and mirror vdevs): # dd if=test2.file of=/dev/null 9154715+1 records in 9154715+1 records out 4687214153 bytes transferred in 82.963181 secs (56497522 bytes/sec) This is, again, seems slower than usual, but not as terrible as the write speeds that I've been seeing via samba. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: samba+zfs
on this Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com On Fri, 28 Oct 2011, Garrett Cooper wrote: On Thu, Oct 27, 2011 at 6:42 PM, Dan d...@sunsaturn.com wrote: Updated from 9.0 beta3 to RC1 and using mkvmerge over samba/zfs its taking over an hour to just mux in things like DTS english, where it was 15 minutes on beta3. Hi Dan, - Can you do more deterministic / scientific benchmarks? - Did you upgrade Samba? - What is your system's operating hardware profile? Thanks! -Garrett ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: samba+zfs
Sorry I meant it was running fine on beta3 and 8.2 stable, and NOT RC1: asterisk:~# uname -a FreeBSD asterisk.sunsaturn.com 9.0-RC1 FreeBSD 9.0-RC1 #0: Mon Oct 31 19:46:53 CDT 2011 dr...@asterisk.sunsaturn.com:/usr/obj/usr/src/sys/MYKERNEL amd64 asterisk:~# Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com On Tue, 8 Nov 2011, Dan The Man wrote: Ok here is some specs: this been running fine on 8.2 stable and i was sure it was running fine on RC1 as well. I did some testing against samba 34 35 and 36 in the ports collection all with the same slow write problems. I did further testing mounting drive in question with NFS and it did not suffer the same problem, so it seems just samba related here, where samba would actually outperform my NFS mount before, now its taking 10x as long to write anything. This is really most simplistic setup I have, all I want to do is map a network drive at the house read/write so my laptop, desktop etc all have access to it. I have played with all the smb.conf options, and can't seem to find where the issue is, further research suggests others are experiencing same problems with beta3 from following forum post: http://forums.freebsd.org/showthread.php?t=27300 Hardware this is running on: I beleive a 4 year old amd chip and board, with 2 gigs of ram, this is a home PC that serves as a NAS, it has 1 SSD as UFS boot OS filesystem, and uses ZFS in raid0 with 3 3terrabyte cheap hitachi drives for my storage space,which is mirrored nightly with rsync with another duplicate machine(cause I know someone is going to say why not use raid5-raidz) Network specs: machine currently has dedicated IPV4 and gif0 tunneled IPV6 addresses to he.net. I've played with nearly every option in smb.conf disabling, enabling etc and can't seem to find the issue: here are my current config file settings on machine that could apply to samba: asterisk:~# cat /boot/loader.conf autoboot_delay=5 accf_data_load=YES aio_load=YES zfs_load=YES kern.maxbcache=64M kern.ipc.maxpipekva=4M vfs.zfs.prefetch_disable=1 vm.kmem_size=1844M vfs.zfs.arc_min=1024M vfs.zfs.arc_max=1536M vfs.zfs.vdev.min_pending=2 vfs.zfs.vdev.max_pending=8 vfs.zfs.txg.timeout=5 vfs.zfs.zil_disable=1 ahci_load=YES asterisk:~# asterisk:~# cat /usr/local/etc/smb.conf # Global parameters [global] workgroup = HOME netbios name = ASTERISK server string = Primary backups interfaces = sk0 #smb ports = 139 #security = USER security = SHARE encrypt passwords = Yes #socket options = TCP_NODELAY SO_RCVBUF=65536 SO_SNDBUF=65536 domain master = no wins support = yes guest account = root socket options=SO_RCVBUF=131072 SO_SNDBUF=131072 TCP_NODELAY use sendfile = no level2 oplocks = True read raw = no write cache size = 262144 min receivefile size = 16384 aio read size = 16384 aio write size = 16384 aio write behind = yes dns proxy = no max log size = 50 #log file = /dev/null log file = /var/log/samba.log debug level = 1 syslog = 0 [data] comment = Primary backups path = /data/public read only = No guest ok = Yes valid users = root asterisk:~# asterisk:~# cat /etc/sysctl.conf # $FreeBSD: src/etc/sysctl.conf,v 1.8.34.1 2009/08/03 08:13:06 kensmith Exp $ # # This file is read when going to multi-user and its contents piped thru # ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for details. # # Uncomment this to prevent users from seeing information about processes that # are being run under another UID. #security.bsd.see_other_uids=0 #raise file descriptors on the system kern.maxfiles=204916 kern.maxfilesperproc=204916 #raise sockets we can accept kern.ipc.somaxconn=32768 #http://www-didc.lbl.gov/TCP-tuning/FreeBSD.html kern.ipc.maxsockbuf=16777216 net.inet.tcp.rfc1323=1 net.inet.tcp.sendbuf_max=16777216 net.inet.tcp.recvbuf_max=16777216 net.inet.tcp.sendspace=65536 net.inet.tcp.recvspace=131072 #NGINX webserver http://wiki.nginx.org/FreeBSDOptimizations net.inet.icmp.icmplim=500 kern.ipc.nmbjumbop=192000 kern.ipc.nmbclusters=229376 kern.ipc.maxsockets=204800 net.inet.tcp.maxtcptw=163840 #also add following to /boot/loader.conf #vm.kmem_size=1844M #kern.maxbcache=64M #kern.ipc.maxpipekva=4M #default setting of net.inet.ip.portrange.first is to low, causing us problems with bind net.inet.ip.portrange.last=65535 net.inet.ip.portrange.first=1024 #DOS protection net.inet.tcp.msl=7500 net.inet.tcp.blackhole=2 net.inet.udp.blackhole=1 net.inet.icmp.icmplim=50 net.inet.ip.accept_sourceroute=0 net.inet.ip.sourceroute=0 #some stuff for samba kern.ipc.nmbclusters=32768 kern.maxvnodes=80 net.inet.tcp.delayed_ack=0 net.inet.tcp.inflight.enable=0 net.inet.tcp.path_mtu_discovery=0
Proftpd + Freebsd 9 + mod_mysql
Not sure if anyone else is having problem with proftpd on freebsd 9, but here is a patch to stop it terminating, should be included in next release, courtesy of TJ saunders working with me on it. Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com freebsd-mysql-signal.patch Description: Binary data ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
/etc/group chkgrp invalid character @
With libnss-mysql we are able to login using email addresses etc. In the daily cron Daily run output email always get the following: Verifying group file syntax: chkgrp: /etc/group: line 3: '@' invalid character Could we modify system to support email addresses as usernames. From my testing it works fine, even with Daily run output complaining I can still su to user i added in wheel group. We'd need to fix ckkgrp source, adduser source, and making move to: #define MAXLOGNAME 256 in /usr/src/sys/sys/param.h I beleive OS's like macOSX etc when I read over their source is already setting this to 256. I beleive param.h is only place need to define this, in 8.2 and previous UT_NAMESIZE needed to be set in /usr/src/include/utmp.h as 255 and /usr/src/sys/sys/param.h needed MAXLOGNAME set to UT_NAMESIZE+1, but seems we did away with utmp.h in freebsd 9.0 only needing to set param.h now. Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com On Tue, 8 Nov 2011, Kurt Touet wrote: On Tue, Nov 8, 2011 at 3:32 AM, Dan The Man d...@sunsaturn.com wrote: Sorry I meant it was running fine on beta3 and 8.2 stable, and NOT RC1: asterisk:~# uname -a FreeBSD asterisk.sunsaturn.com 9.0-RC1 FreeBSD 9.0-RC1 #0: Mon Oct 31 19:46:53 CDT 2011 dr...@asterisk.sunsaturn.com:/usr/obj/usr/src/sys/MYKERNEL amd64 asterisk:~# Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com On Tue, 8 Nov 2011, Dan The Man wrote: Ok here is some specs: this been running fine on 8.2 stable and i was sure it was running fine on RC1 as well. I did some testing against samba 34 35 and 36 in the ports collection all with the same slow write problems. I did further testing mounting drive in question with NFS and it did not suffer the same problem, so it seems just samba related here, where samba would actually outperform my NFS mount before, now its taking 10x as long to write anything. This is really most simplistic setup I have, all I want to do is map a network drive at the house read/write so my laptop, desktop etc all have access to it. I have played with all the smb.conf options, and can't seem to find where the issue is, further research suggests others are experiencing same problems with beta3 from following forum post: http://forums.freebsd.org/showthread.php?t=27300 Hardware this is running on: I beleive a 4 year old amd chip and board, with 2 gigs of ram, this is a home PC that serves as a NAS, it has 1 SSD as UFS boot OS filesystem, and uses ZFS in raid0 with 3 3terrabyte cheap hitachi drives for my storage space,which is mirrored nightly with rsync with another duplicate machine(cause I know someone is going to say why not use raid5-raidz) Network specs: machine currently has dedicated IPV4 and gif0 tunneled IPV6 addresses to he.net. I've played with nearly every option in smb.conf disabling, enabling etc and can't seem to find the issue: here are my current config file settings on machine that could apply to samba: asterisk:~# cat /boot/loader.conf autoboot_delay=5 accf_data_load=YES aio_load=YES zfs_load=YES kern.maxbcache=64M kern.ipc.maxpipekva=4M vfs.zfs.prefetch_disable=1 vm.kmem_size=1844M vfs.zfs.arc_min=1024M vfs.zfs.arc_max=1536M vfs.zfs.vdev.min_pending=2 vfs.zfs.vdev.max_pending=8 vfs.zfs.txg.timeout=5 vfs.zfs.zil_disable=1 ahci_load=YES asterisk:~# asterisk:~# cat /usr/local/etc/smb.conf # Global parameters [global] workgroup = HOME netbios name = ASTERISK server string = Primary backups interfaces = sk0 #smb ports = 139 #security = USER security = SHARE encrypt passwords = Yes #socket options = TCP_NODELAY SO_RCVBUF=65536 SO_SNDBUF=65536 domain master = no wins support = yes guest account = root socket options=SO_RCVBUF=131072 SO_SNDBUF=131072 TCP_NODELAY use sendfile = no level2 oplocks = True read raw = no write cache size = 262144 min receivefile size = 16384 aio read size = 16384 aio write size = 16384 aio write behind = yes dns proxy = no max log size = 50 #log file = /dev/null log file = /var/log/samba.log debug level = 1 syslog = 0 [data] comment = Primary backups path = /data/public read only = No guest ok = Yes valid users = root asterisk:~# asterisk:~# cat /etc/sysctl.conf # $FreeBSD: src/etc/sysctl.conf,v 1.8.34.1 2009/08/03 08:13:06 kensmith Exp $ # # This file is read when going to multi-user and its contents piped thru # ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for details. # # Uncomment this to prevent users from seeing information about processes that # are being run under another UID. #security.bsd.see_other_uids=0 #raise file descriptors on the system kern.maxfiles=204916
Re: /etc/group chkgrp invalid character @
Understood, will resubmit, I didn't think of the email headers. Thank-you, Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com On Tue, 8 Nov 2011, Doug Barton wrote: If you want to start a new thread please don't reply to an old one and change the subject line, please create a whole new message instead. That way those of us who read our mail threaded won't miss your new topic because it is hidden under an old one. -- We could put the whole Internet into a book. Too practical. Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
MAXLOGNAME + /etc/group + chkgrp invalid character @
With libnss-mysql we are able to login using email addresses etc. In the daily cron Daily run output email always get the following: Verifying group file syntax: chkgrp: /etc/group: line 3: '@' invalid character Could we modify system to support email addresses as usernames. From my testing it works fine, even with Daily run output complaining I can still su to user i added in wheel group. We'd need to fix ckkgrp source, adduser source, and making move to: #define MAXLOGNAME 256 in /usr/src/sys/sys/param.h I beleive OS's like macOSX etc when I read over their source is already setting this to 256. I beleive param.h is only place need to define this, in 8.2 and previous UT_NAMESIZE needed to be set in /usr/src/include/utmp.h as 255 and /usr/src/sys/sys/param.h needed MAXLOGNAME set to UT_NAMESIZE+1, but seems we did away with utmp.h in freebsd 9.0 only needing to set param.h now. Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: MAXLOGNAME + /etc/group + chkgrp invalid character @
On Tue, 8 Nov 2011, Chuck Swiger wrote: On Nov 8, 2011, at 3:47 PM, Dan The Man wrote: In the daily cron Daily run output email always get the following: Verifying group file syntax: chkgrp: /etc/group: line 3: '@' invalid character chkgrp expects group names to consist of characters in isalnum(). K so thats a simple fix where it does that check. Could we modify system to support email addresses as usernames. Sure, that's why FreeBSD comes with source code. You can modify anything you like. :-) However, if you want to use a domain-aware login mechanism, Kerberos is in the base system, and SASL and LDAP are available in ports. You're not going to break anything allowing @ into the list of characters which pw(8) likes, but the flatfile passwd and group files are not hierarchical the way domain-aware network identity systems are. A secondary issue is that there is rarely a one-to-one relationship between email addresses and users; many email addresses are aliases which expand either to a different username, or even to multiple users. Wish you would elaborate abit more here, what I have found is email addresses tend to make the best usernames, people can remember them :) They are unique, and you solve 2 problems right away: a) they can actually remember their username b) they aren't having to pick through a million different taken usernames they have to pick on their own, which is frusterating way people often do signups. From my testing it works fine, even with Daily run output complaining I can still su to user i added in wheel group. We'd need to fix ckkgrp source, adduser source, and making move to: #define MAXLOGNAME 256 in /usr/src/sys/sys/param.h You can do that also, but I think you'll break compatibility with NIS/YP. Well with nss-mysql its as simple as modifying the /etc/nsswitch.conf on any machine to just point to same db server, works just fine. You might not care, but don't be surprised if you find that folks aren't willing to adopt this change back into FreeBSD-- I've seen a few people wanting to increase MAXLOGNAME since 2003 or so. I've talked to many sys admins as well, that are all modifying the code to the kernel for a decade now on every new make buildworld, would be nice to see it mainstream. Only issue doing this I have seen so far, is having to nuke the wtmp/utx* files from /var/log on new installs to get them into new format, but that would be solved mainstream as well. I just find the benefits far outweight the cons, sure when we were all back in our computer science classes in 80s/90s it made sense. We all had accounts on the system for those 3-4 years, and generic usernames made sense, but now moving to webhosting environments and providing sftp/ssh type access to people on a larger scale, I think the email address as usernames make alot more sense now. I still teach unix at the university time to time and we still use the old putty/securecrt to sshd daemon way of learning from the command line, in that environment I find its about people forgetting passwords, take it up a notch to webhosting environment, and i find people forget their usernames to, and why I think it would be a good move... Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com Regards, -- -Chuck ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: MAXLOGNAME + /etc/group + chkgrp invalid character @
On Tue, 8 Nov 2011, Chuck Swiger wrote:
Hi--
On Nov 8, 2011, at 4:32 PM, Dan The Man wrote:
On Tue, 8 Nov 2011, Chuck Swiger wrote:
On Nov 8, 2011, at 3:47 PM, Dan The Man wrote:
In the daily cron Daily run output email always get the following:
Verifying group file syntax:
chkgrp: /etc/group: line 3: '@' invalid character
chkgrp expects group names to consist of characters in isalnum().
K so thats a simple fix where it does that check.
usr.sbin/chkgrp/chkgrp.c, line ~117:
for (cp = f[0] ; *cp ; cp++) {
if (!isalnum(*cp) *cp != '.' *cp != '_' *cp != '-'
(cp f[0] || *cp != '+')) {
warnx(%s: line %d: '%c' invalid character, gfn, n, *cp);
e++;
}
}
Add a *cp != '@' clause to the if statement.
It doesn't bother me enough to have to modify the if statement on every
new buildworld(which I seem to be doing alot of lately), honestly that one
email login in the group file is only
email login that will ever touch an /etc file, rest is in database. I
wouldn't personally add any email type logins to a base system directly.
Could we modify system to support email addresses as usernames.
Sure, that's why FreeBSD comes with source code.
You can modify anything you like. :-)
However, if you want to use a domain-aware login mechanism, Kerberos is in the base
system, and SASL and LDAP are available in ports. You're not going to break anything
allowing @ into the list of characters which pw(8) likes, but the flatfile
passwd and group files are not hierarchical the way domain-aware network identity systems
are.
A secondary issue is that there is rarely a one-to-one relationship between
email addresses and users; many email addresses are aliases which expand either
to a different username, or even to multiple users.
Wish you would elaborate abit more here, what I have found is email addresses
tend to make the best usernames, people can remember them :)
They are unique, and you solve 2 problems right away:
a) they can actually remember their username
b) they aren't having to pick through a million different taken usernames
they have to pick on their own, which is frusterating way people often do
signups.
If you've got a database of millions of users, you're definitely functioning in
a different realm than what /etc/passwd and /etc/group were designed for. :-)
Well thats the idea right, unlimited growth but a central place to store
everything for scalability purposes. Of course if you actually tried to
login 1 million people to same machine, I am sure you would kill the I/O
instantly :)
Basically approach I've been taking is central database that stores all
users, any machine can be configured to not allow logins from anyone in
database by just removing them from /etc/nsswitch.conf. You can control
which of those million users can login to a certain machine by simply
modifying the select statement to the database in the nss-database config
file...all in all these changes take 2 seconds to change and makes things
quite easy.
Anyway, the idea is that you should be able to define multiple hierarchy levels for your identity database,
which NIS+, NetInfo, Kerberos, and LDAP (kinda-sorta) can support. This lets you define an identity either
at the root level, which is visible everywhere, or in subdomains from root, which means the identity is valid
only within that subdomain but not in other subdomains-- and johndoe in one subdomain can be
entirely different than johndoe in some other domain. (If you want johndoe the same
everywhere, you'd define it at root instead.)
That's just a bare-bones explanation, but a more complete one would likely
approach book-length. :-)
I'm sure it would make an interesting read, I try not to concern myself to
much with mechanisms that would likely to take to long to respond if there
were millions of users, working directly with mysql/postgres/oracle from
getgo i think is safe.
Think its almost like an analogy to where you started off building your
network code using something simple like select() then you kicked yourself
in the ass after for not having used epoll() on linux or kqueue() on
freebsd in an eventloop as your application got to many connections.
I do like your idea in a small office type setting though, definately
wouldn't mind trying something like that here at home so I had a central
way for all these development machines I got lieing around to have some
central login place.
Found this an interesting read last night:
http://www.mrp3.com/windows-to-unix-samba.html
might be abit dated...
Dan.
--
Dan The Man
CTO/ Senior System Administrator
Websites, Domains and Everything else
http://www.SunSaturn.com
Email: d...@sunsaturn.com
You might not care, but don't be surprised if you find that folks aren't
willing to adopt this change back into FreeBSD-- I've seen a few people wanting
to increase MAXLOGNAME since
Re: samba+zfs
On Tue, 8 Nov 2011, Kurt Touet wrote: On Tue, Nov 8, 2011 at 3:32 AM, Dan The Man d...@sunsaturn.com wrote: Sorry I meant it was running fine on beta3 and 8.2 stable, and NOT RC1: asterisk:~# uname -a FreeBSD asterisk.sunsaturn.com 9.0-RC1 FreeBSD 9.0-RC1 #0: Mon Oct 31 19:46:53 CDT 2011 dr...@asterisk.sunsaturn.com:/usr/obj/usr/src/sys/MYKERNEL amd64 asterisk:~# Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com On Tue, 8 Nov 2011, Dan The Man wrote: Ok here is some specs: this been running fine on 8.2 stable and i was sure it was running fine on RC1 as well. I did some testing against samba 34 35 and 36 in the ports collection all with the same slow write problems. I did further testing mounting drive in question with NFS and it did not suffer the same problem, so it seems just samba related here, where samba would actually outperform my NFS mount before, now its taking 10x as long to write anything. This is really most simplistic setup I have, all I want to do is map a network drive at the house read/write so my laptop, desktop etc all have access to it. I have played with all the smb.conf options, and can't seem to find where the issue is, further research suggests others are experiencing same problems with beta3 from following forum post: http://forums.freebsd.org/showthread.php?t=27300 Hardware this is running on: I beleive a 4 year old amd chip and board, with 2 gigs of ram, this is a home PC that serves as a NAS, it has 1 SSD as UFS boot OS filesystem, and uses ZFS in raid0 with 3 3terrabyte cheap hitachi drives for my storage space,which is mirrored nightly with rsync with another duplicate machine(cause I know someone is going to say why not use raid5-raidz) Network specs: machine currently has dedicated IPV4 and gif0 tunneled IPV6 addresses to he.net. I've played with nearly every option in smb.conf disabling, enabling etc and can't seem to find the issue: here are my current config file settings on machine that could apply to samba: asterisk:~# cat /boot/loader.conf autoboot_delay=5 accf_data_load=YES aio_load=YES zfs_load=YES kern.maxbcache=64M kern.ipc.maxpipekva=4M vfs.zfs.prefetch_disable=1 vm.kmem_size=1844M vfs.zfs.arc_min=1024M vfs.zfs.arc_max=1536M vfs.zfs.vdev.min_pending=2 vfs.zfs.vdev.max_pending=8 vfs.zfs.txg.timeout=5 vfs.zfs.zil_disable=1 ahci_load=YES asterisk:~# asterisk:~# cat /usr/local/etc/smb.conf # Global parameters [global] workgroup = HOME netbios name = ASTERISK server string = Primary backups interfaces = sk0 #smb ports = 139 #security = USER security = SHARE encrypt passwords = Yes #socket options = TCP_NODELAY SO_RCVBUF=65536 SO_SNDBUF=65536 domain master = no wins support = yes guest account = root socket options=SO_RCVBUF=131072 SO_SNDBUF=131072 TCP_NODELAY use sendfile = no level2 oplocks = True read raw = no write cache size = 262144 min receivefile size = 16384 aio read size = 16384 aio write size = 16384 aio write behind = yes dns proxy = no max log size = 50 #log file = /dev/null log file = /var/log/samba.log debug level = 1 syslog = 0 [data] comment = Primary backups path = /data/public read only = No guest ok = Yes valid users = root asterisk:~# asterisk:~# cat /etc/sysctl.conf # $FreeBSD: src/etc/sysctl.conf,v 1.8.34.1 2009/08/03 08:13:06 kensmith Exp $ # # This file is read when going to multi-user and its contents piped thru # ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for details. # # Uncomment this to prevent users from seeing information about processes that # are being run under another UID. #security.bsd.see_other_uids=0 #raise file descriptors on the system kern.maxfiles=204916 kern.maxfilesperproc=204916 #raise sockets we can accept kern.ipc.somaxconn=32768 #http://www-didc.lbl.gov/TCP-tuning/FreeBSD.html kern.ipc.maxsockbuf=16777216 net.inet.tcp.rfc1323=1 net.inet.tcp.sendbuf_max=16777216 net.inet.tcp.recvbuf_max=16777216 net.inet.tcp.sendspace=65536 net.inet.tcp.recvspace=131072 #NGINX webserver http://wiki.nginx.org/FreeBSDOptimizations net.inet.icmp.icmplim=500 kern.ipc.nmbjumbop=192000 kern.ipc.nmbclusters=229376 kern.ipc.maxsockets=204800 net.inet.tcp.maxtcptw=163840 #also add following to /boot/loader.conf #vm.kmem_size=1844M #kern.maxbcache=64M #kern.ipc.maxpipekva=4M #default setting of net.inet.ip.portrange.first is to low, causing us problems with bind net.inet.ip.portrange.last=65535 net.inet.ip.portrange.first=1024 #DOS protection net.inet.tcp.msl=7500 net.inet.tcp.blackhole=2 net.inet.udp.blackhole=1 net.inet.icmp.icmplim=50 net.inet.ip.accept_sourceroute=0 net.inet.ip.sourceroute=0 #some stuff for samba kern.ipc.nmbclusters=32768 kern.maxvnodes=80 net.inet.tcp.delayed_ack=0
