[pfSense] KVM virtualization: Fatal trap 9: general protection fault while in kernel mode
Hi. I'm reposting here a question I asked on the forum, hoping for a different audience. I'm running pfSense for some time now, since 2.0.something it has always been running without issues. With the latest 2.1 releases I'm very rarely running into crashes. Today I managet do catch one: Fatal trap 9: general protection fault while in kernel mode [...] Stopped at rn_match+0x25: cmpw $0,0x10(%r13) Here's a full screenshot of dump caught on console: https://dl.dropboxusercontent.com/u/706934/pfsense_crash.png The VM is configured with VirtIO disks, emulated e1000 network cards. Any hint on what it could be? Where to look at? thanks -- Lorenzo Milesi - lorenzo.mil...@yetopen.it YetOpen S.r.l. - http://www.yetopen.it/ ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] KVM virtualization: Fatal trap 9: general protection fault while in kernel mode
The VM is configured with VirtIO disks, emulated e1000 network cards. I use kvm and have had no problems running any of the 2.1 releases. I'm building a VM server right now that will run pfSense and one other guest OS. I have used the virtio drivers for nics, storage, and memory ballooning, but because of the steps you have to take to switch to virtio, I'm using e1000 and IDE emulation on this one to keep it simple. What host OS are you using, and what hardware is it running on? (real cpu, ram, and storage) Is it possible to see the results of virsh dumpxml for the guest? ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] ZFS warning message on local console during boot
On Jul 30, 2014, at 5:37 AM, Stefan Baur newsgroups.ma...@stefanbaur.de wrote: Hi list, I'm seeing the following warning on my pfsense 2.1.4-RELEASE (i386): ZFS WARNING: Recommended minimim kmem_size is 512MB; expect unstable behavior. Consider tuning vm.kmem_size and vm.kmem_size_max in /boot/loader.conf Currently, the values are: vm.kmem_size=525544320 vm.kmem_size_max=535544320 Given this machine has 1 Gigabyte of RAM, which values should I enter? Personally, I think ZFS on i386 has become a losing proposition as of late. I ran a ZFS-on-root FreeBSD/i386 10-STABLE system with 2 GB of RAM and it appeared to become very flaky with ZFS in its latter months (I eventually switched it out for a FreeBSD/amd64 system). I had to be careful with what values for vm.kmem_size, vm.kmem_size_max, and vfs.zfs.arc_max I put in /boot/loader.conf because often certain combinations would panic the system on boot. Also, to use quite a bit of the available RAM for ARC required me to build a custom kernel with KVA_PAGES=512 set in the kernel config file. I believe the days when FreeBSD/i386 was considered the primary, tried-and-tested distribution and FreeBSD/amd64 the less-tested version are long behind us. If you can run FreeBSD/amd64 then you should. If you can only run FreeBSD/i386 then I wouldn't recommend using ZFS with it. I just don't think it gets adequate testing any more. (YMMV.) Cheers, Paul. ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] ZFS warning message on local console during boot
Faster caching when using squid and/or some of the other packages? But, yes, it would be a bit silly, regardless. -Adam On July 30, 2014 9:43:01 AM CDT, Vick Khera vi...@khera.org wrote: On Wed, Jul 30, 2014 at 9:50 AM, Paul Mather p...@gromit.dlib.vt.edu wrote: Personally, I think ZFS on i386 has become a losing proposition as of late. I ran a ZFS-on-root FreeBSD/i386 10-STABLE system with 2 GB of RAM and it appeared to become very flaky with ZFS in its latter months (I eventually switched it out for a FreeBSD/amd64 system). I cannot fathom a sensible use case for using ZFS on pfSense at all. ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list -- Sent from my Android device with K-9 Mail. Please excuse my brevity.___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
[pfSense] LDAP PAM auth with Local Database accounts?
At our organisation we have a central LDAP database that contains administrative information. For Unix purposes, it's only useful for PAM auth, as its schema does not contain the requisite Posix attributes required by Unix accounts. Nevertheless, it is still very useful for password authentication because the 24/7 service our organisation provides for password reset and management can be leveraged when authenticating against this LDAP source. On my FreeBSD and Linux servers, this means I can have the PAM auth component for services in pam.d work to do password authentication using the user's organisation password, yet all the account data still comes from local accounts on the system. The upshot is that if the user forgets his or her password, they don't come to me, they go to the organisational 4HELP. :-) Is it possible to use this kind of setup on pfSense 2? It almost seems to work for me, but maybe I am doing something wrong. The authentication part works, but, because there are no Group attributes in our central LDAP, the user seems to become a member of no groups when logging in. This appears to throw pfSense for a loop. :-) It would be nice if pfSense would fall back to Local Database attributes when LDAP doesn't provide them, or, maybe better still, if a new blended authentication method of LDAP auth + Local Database Attributes was available that used LDAP for auth but the Local Database for account information such as real name, groups, etc. This latter approach is how applications such as Redmine use LDAP authentication. Cheers, Paul. ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] ZFS warning message on local console during boot
Am 30.07.2014 um 16:43 schrieb Vick Khera: On Wed, Jul 30, 2014 at 9:50 AM, Paul Mather p...@gromit.dlib.vt.edu wrote: Personally, I think ZFS on i386 has become a losing proposition as of late. I ran a ZFS-on-root FreeBSD/i386 10-STABLE system with 2 GB of RAM and it appeared to become very flaky with ZFS in its latter months (I eventually switched it out for a FreeBSD/amd64 system). I cannot fathom a sensible use case for using ZFS on pfSense at all. I'm not consciously using ZFS for anything on pfSense, I *think* I performed the default install, but it could be using ntfs or vfat for all that I care. ;-) So I don't know why it's trying to use that - is it normal for a default pfSense install or not? I just saw the warning message and was wondering what to do about it. -Stefan ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] ZFS warning message on local console during boot
ZFS = FS+LVM. Its efficient in many ways. Its highly resillient to things like silent data corruption ( disk FW bugs, power spikes). It has on the fly checking and repair. Copy on write, snapshoting, NFSv4 native acls and a few more nice things. I dont understand the bashing? -lsf 30. juli 2014 21:44 skrev Stefan Baur newsgroups.ma...@stefanbaur.de følgende: Am 30.07.2014 um 16:43 schrieb Vick Khera: On Wed, Jul 30, 2014 at 9:50 AM, Paul Mather p...@gromit.dlib.vt.edu wrote: Personally, I think ZFS on i386 has become a losing proposition as of late. I ran a ZFS-on-root FreeBSD/i386 10-STABLE system with 2 GB of RAM and it appeared to become very flaky with ZFS in its latter months (I eventually switched it out for a FreeBSD/amd64 system). I cannot fathom a sensible use case for using ZFS on pfSense at all. I'm not consciously using ZFS for anything on pfSense, I *think* I performed the default install, but it could be using ntfs or vfat for all that I care. ;-) So I don't know why it's trying to use that - is it normal for a default pfSense install or not? I just saw the warning message and was wondering what to do about it. -Stefan ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] ZFS warning message on local console during boot
Am 30.07.2014 um 22:09 schrieb Espen Johansen: ZFS = FS+LVM. Its efficient in many ways. Its highly resillient to things like silent data corruption ( disk FW bugs, power spikes). It has on the fly checking and repair. Copy on write, snapshoting, NFSv4 native acls and a few more nice things. I dont understand the bashing? This is a firewall, not a fileserver, where such features do indeed make sense. And no bashing, just saying I don't care what filesystem pfSense uses under the hood, as long as it works. The fact that it spits out a warning seems to indicate that it does not work and there's something wrong, so I came here to ask. -Stefan ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] ZFS warning message on local console during boot
On Jul 30, 2014, at 4:09 PM, Espen Johansen pfse...@gmail.com wrote: ZFS = FS+LVM. Its efficient in many ways. Its highly resillient to things like silent data corruption ( disk FW bugs, power spikes). It has on the fly checking and repair. Copy on write, snapshoting, NFSv4 native acls and a few more nice things. I dont understand the bashing? I swear by ZFS on my regular FreeBSD systems (though I was having trouble with it on FreeBSD/i386 latterly). I don't think there's any bashing of ZFS per se, just a wondering why you'd use it on a firewall appliance that's basically a nanobsd setup at heart... Cheers, Paul. -lsf 30. juli 2014 21:44 skrev Stefan Baur newsgroups.ma...@stefanbaur.de følgende: Am 30.07.2014 um 16:43 schrieb Vick Khera: On Wed, Jul 30, 2014 at 9:50 AM, Paul Mather p...@gromit.dlib.vt.edu wrote: Personally, I think ZFS on i386 has become a losing proposition as of late. I ran a ZFS-on-root FreeBSD/i386 10-STABLE system with 2 GB of RAM and it appeared to become very flaky with ZFS in its latter months (I eventually switched it out for a FreeBSD/amd64 system). I cannot fathom a sensible use case for using ZFS on pfSense at all. I'm not consciously using ZFS for anything on pfSense, I *think* I performed the default install, but it could be using ntfs or vfat for all that I care. ;-) So I don't know why it's trying to use that - is it normal for a default pfSense install or not? I just saw the warning message and was wondering what to do about it. -Stefan ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] ZFS warning message on local console during boot
Also remeber that pfsense has had packages like freenas (for some the Ultimate all in one home device). -lsf 30. juli 2014 22:24 skrev Paul Mather p...@gromit.dlib.vt.edu følgende: On Jul 30, 2014, at 4:09 PM, Espen Johansen pfse...@gmail.com wrote: ZFS = FS+LVM. Its efficient in many ways. Its highly resillient to things like silent data corruption ( disk FW bugs, power spikes). It has on the fly checking and repair. Copy on write, snapshoting, NFSv4 native acls and a few more nice things. I dont understand the bashing? I swear by ZFS on my regular FreeBSD systems (though I was having trouble with it on FreeBSD/i386 latterly). I don't think there's any bashing of ZFS per se, just a wondering why you'd use it on a firewall appliance that's basically a nanobsd setup at heart... Cheers, Paul. -lsf 30. juli 2014 21:44 skrev Stefan Baur newsgroups.ma...@stefanbaur.de følgende: Am 30.07.2014 um 16:43 schrieb Vick Khera: On Wed, Jul 30, 2014 at 9:50 AM, Paul Mather p...@gromit.dlib.vt.edu wrote: Personally, I think ZFS on i386 has become a losing proposition as of late. I ran a ZFS-on-root FreeBSD/i386 10-STABLE system with 2 GB of RAM and it appeared to become very flaky with ZFS in its latter months (I eventually switched it out for a FreeBSD/amd64 system). I cannot fathom a sensible use case for using ZFS on pfSense at all. I'm not consciously using ZFS for anything on pfSense, I *think* I performed the default install, but it could be using ntfs or vfat for all that I care. ;-) So I don't know why it's trying to use that - is it normal for a default pfSense install or not? I just saw the warning message and was wondering what to do about it. -Stefan ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] ZFS warning message on local console during boot
On Jul 30, 2014, at 3:21 PM, Stefan Baur newsgroups.ma...@stefanbaur.de wrote: Am 30.07.2014 um 22:09 schrieb Espen Johansen: ZFS = FS+LVM. Its efficient in many ways. Its highly resillient to things like silent data corruption ( disk FW bugs, power spikes). It has on the fly checking and repair. Copy on write, snapshoting, NFSv4 native acls and a few more nice things. I dont understand the bashing? This is a firewall, not a fileserver, where such features do indeed make sense. And no bashing, just saying I don't care what filesystem pfSense uses under the hood, as long as it works. The fact that it spits out a warning seems to indicate that it does not work and there's something wrong, so I came here to ask. tl;dr: I wouldn’t run ZFS… yet. I didn’t see the error message, you’re barking up a tree attempting to use it right now. That said, there are certain advantages to ZFS, and there are internal experiments underway looking to use it for a future (64-bit only) release of pfSense. The data integrity and resiliency (due to COW semantics checksumming) (etc) is one thing. I’ve had pretty good results turning on LZJB compression and ‘copies=2”, which is nearly as good as a nanobsd image with 2 separate slices, and, since you have a live filesystem, has NONE of the drawbacks of the nanobsd approach. One could even ‘checkpoint’ (snapshot) the zvol prior to any change (pkg install, config change, etc), and, of course zfs send | ssh foo; zfs receive” makes it entirely trivial to keep your entire firewall backed up, rather than (just) the config file. People who say, “I can’t fathom a sensible use care for using ZFS on pfSense” or “why use it to replace nanobsd?” are (likely) stuck in a system admin mindset/mentality(*). I get the same pushback about bhyve (“why would you use that on a firewall?”) from people stuck in the same headspace. I’m not going to reveal everything here, because it’s going to be post-2.2 before any of this comes about, and I’m keeping the focus on 2.2. In short: ZFS is not just about building a NAS. Jim (*) If there isn’t an O’Reilly book out about it, it seems to not exist to these people. ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] ZFS warning message on local console during boot
Sounds like the mikrotik metarouter feature. Josh Reynolds, CIO SPITwSPOTS www.spitwspots.com On 07/30/2014 01:34 PM, Jim Thompson wrote: On Jul 30, 2014, at 3:21 PM, Stefan Baur newsgroups.ma...@stefanbaur.de wrote: Am 30.07.2014 um 22:09 schrieb Espen Johansen: ZFS = FS+LVM. Its efficient in many ways. Its highly resillient to things like silent data corruption ( disk FW bugs, power spikes). It has on the fly checking and repair. Copy on write, snapshoting, NFSv4 native acls and a few more nice things. I dont understand the bashing? This is a firewall, not a fileserver, where such features do indeed make sense. And no bashing, just saying I don't care what filesystem pfSense uses under the hood, as long as it works. The fact that it spits out a warning seems to indicate that it does not work and there's something wrong, so I came here to ask. tl;dr: I wouldn’t run ZFS… yet. I didn’t see the error message, you’re barking up a tree attempting to use it right now. That said, there are certain advantages to ZFS, and there are internal experiments underway looking to use it for a future (64-bit only) release of pfSense. The data integrity and resiliency (due to COW semantics checksumming) (etc) is one thing. I’ve had pretty good results turning on LZJB compression and ‘copies=2”, which is nearly as good as a nanobsd image with 2 separate slices, and, since you have a live filesystem, has NONE of the drawbacks of the nanobsd approach. One could even ‘checkpoint’ (snapshot) the zvol prior to any change (pkg install, config change, etc), and, of course zfs send | ssh foo; zfs receive” makes it entirely trivial to keep your entire firewall backed up, rather than (just) the config file. People who say, “I can’t fathom a sensible use care for using ZFS on pfSense” or “why use it to replace nanobsd?” are (likely) stuck in a system admin mindset/mentality(*). I get the same pushback about bhyve (“why would you use that on a firewall?”) from people stuck in the same headspace. I’m not going to reveal everything here, because it’s going to be post-2.2 before any of this comes about, and I’m keeping the focus on 2.2. In short: ZFS is not just about building a NAS. Jim (*) If there isn’t an O’Reilly book out about it, it seems to not exist to these people. ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] ZFS warning message on local console during boot
Am 30.07.2014 um 23:34 schrieb Jim Thompson: tl;dr: I wouldn’t run ZFS… yet. I didn’t see the error message, you’re barking up a tree attempting to use it right now. Again, I don't care what FS pfSense uses under the hood as long as it works. I didn't make a conscious decision to install/run ZFS, I firmly believe I picked the default options during the pfSense install and now I'm seeing this warning. I don't insist on using ZFS at all. If I can and should get rid of ZFS to get rid of the warning, just tell me how. -Stefan ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] ZFS warning message on local console during boot
On Jul 30, 2014, at 4:40 PM, Stefan Baur newsgroups.ma...@stefanbaur.de wrote: Am 30.07.2014 um 23:34 schrieb Jim Thompson: tl;dr: I wouldn’t run ZFS… yet. I didn’t see the error message, you’re barking up a tree attempting to use it right now. Again, I don't care what FS pfSense uses under the hood as long as it works. I didn't make a conscious decision to install/run ZFS, I firmly believe I picked the default options during the pfSense install and now I'm seeing this warning. I don't insist on using ZFS at all. If I can and should get rid of ZFS to get rid of the warning, just tell me how. no pfSense we produce has an installer that will make a zfs filesystem. Try again? ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] ZFS warning message on local console during boot
Well, you could use it for that (pfSense on pfSense), but there will be unnecessary overhead. On Jul 30, 2014, at 4:38 PM, Josh Reynolds j...@spitwspots.com wrote: Sounds like the mikrotik metarouter feature. Josh Reynolds, CIO SPITwSPOTS www.spitwspots.com On 07/30/2014 01:34 PM, Jim Thompson wrote: On Jul 30, 2014, at 3:21 PM, Stefan Baur newsgroups.ma...@stefanbaur.de wrote: Am 30.07.2014 um 22:09 schrieb Espen Johansen: ZFS = FS+LVM. Its efficient in many ways. Its highly resillient to things like silent data corruption ( disk FW bugs, power spikes). It has on the fly checking and repair. Copy on write, snapshoting, NFSv4 native acls and a few more nice things. I dont understand the bashing? This is a firewall, not a fileserver, where such features do indeed make sense. And no bashing, just saying I don't care what filesystem pfSense uses under the hood, as long as it works. The fact that it spits out a warning seems to indicate that it does not work and there's something wrong, so I came here to ask. tl;dr: I wouldn’t run ZFS… yet. I didn’t see the error message, you’re barking up a tree attempting to use it right now. That said, there are certain advantages to ZFS, and there are internal experiments underway looking to use it for a future (64-bit only) release of pfSense. The data integrity and resiliency (due to COW semantics checksumming) (etc) is one thing. I’ve had pretty good results turning on LZJB compression and ‘copies=2”, which is nearly as good as a nanobsd image with 2 separate slices, and, since you have a live filesystem, has NONE of the drawbacks of the nanobsd approach. One could even ‘checkpoint’ (snapshot) the zvol prior to any change (pkg install, config change, etc), and, of course zfs send | ssh foo; zfs receive” makes it entirely trivial to keep your entire firewall backed up, rather than (just) the config file. People who say, “I can’t fathom a sensible use care for using ZFS on pfSense” or “why use it to replace nanobsd?” are (likely) stuck in a system admin mindset/mentality(*). I get the same pushback about bhyve (“why would you use that on a firewall?”) from people stuck in the same headspace. I’m not going to reveal everything here, because it’s going to be post-2.2 before any of this comes about, and I’m keeping the focus on 2.2. In short: ZFS is not just about building a NAS. Jim (*) If there isn’t an O’Reilly book out about it, it seems to not exist to these people. ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] ZFS warning message on local console during boot
Am 30.07.2014 um 23:47 schrieb Jim Thompson: JT no pfSense we produce has an installer that will make a zfs filesystem. JT JT Try again? Well, mount doesn't show any mounted zfs filesystems (only ufs, devfs, and msdosfs - the latter's where the config file is stored) which makes this error message even more confusing - or actually, made it more confusing until Adam Thompson's message, which just cleared things up: AT Stefan: just ignore the message. AT It's there because ZFS is in the pfSense kernel, even though it isn't used today. AT If you don't mount any ZFS file systems, and you don't tweak any of the values, all it does is use up a bit of memory. AT -Adam So, I guess the issue is a non-issue. -Stefan ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] ZFS warning message on local console during boot
On Jul 30, 2014, at 7:20 PM, Paul Mather p...@gromit.dlib.vt.edu wrote: Despite all that FreeBSD ZFS love, I still would not recommend it on FreeBSD/i386-based installations (as the OP said he was using). It is much more of a headache to use in that milieu, and, IMHO, doesn't get the testing and general care and feeding that the FreeBSD/amd64 version gets. Note that I said any use we make would be amd64 only. Also, ZFS would not be a good fit on low-memory embedded hardware. There are enough problems getting ARC to play nicely on high-memory systems under memory pressure... :-) What do you consider ‘low-memory’? It’s getting difficult to put less than 4GB in some systems. ZFS works really well on a 4GB system with around 100GB of ssd/m-sata. auto-tuned ARC maximum is physical RAM less 1GB, or 1/2 of available RAM. on a 2GB system, this is 1GB, on a 4GB system, its 2GB. Have you looked at memory usage in pfSense lately? Most of the ‘tuning guides’ consider fileserver/webserver/db applications. pfSense is none of these. There are several applications that would like to reliably write logfiles / rrd files, etc., however. ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] ZFS warning message on local console during boot
On 2014-07-30 14:47, Jim Thompson wrote: no pfSense we produce has an installer that will make a zfs filesystem. I also get some zfs warnings during boot, and I absolutely guarantee you that I have not created or changed any partitions at all from pfSense's defaults. Based on other messages in this thread, it appears that it's harmless and can be ignored since no zfs partitions are actually mounted, but the error still appears. -- Dave Warren http://www.hireahit.com/ http://ca.linkedin.com/in/davejwarren ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] ZFS warning message on local console during boot
On 2014-07-30 13:23, Paul Mather wrote: I swear by ZFS on my regular FreeBSD systems (though I was having trouble with it on FreeBSD/i386 latterly). I don't think there's any bashing of ZFS per se, just a wondering why you'd use it on a firewall appliance that's basically a nanobsd setup at heart... Maybe it's just me, but I want my firewall to just work after power failures, on failing drives, etc is a big plus. Having a self-repairing, snapshotting file system sounds like a huge benefit, but I don't know what the drawbacks are in this context, so I can't make an actual recommendation. Imagine having snapshots before updates or major changes so that things can be reverted to a working state, rather than relying on the piecemeal XML backups which, at best, brings you a moderately similar to the previous state configuration. Being immune to corruption due to power-failures would be nice too; when I was running squid on pfSense, an unexpected power failure virtually always resulted in file system corruption being repaired, still resulting in a broken squid cache -- I have the impression that zfs would give me a lot more resiliency here (but possibly not, perhaps squid simply can't ever recover gracefully) -- Dave Warren http://www.hireahit.com/ http://ca.linkedin.com/in/davejwarren ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
