If NFS and jails are known to misbehave, I can look at alternatives.
That said, if there is interest in troubleshooting this further, I am
set-up to play guinea pig.
To test this scenario without jails, I set jail_enable="NO" and moved
the nullfs lines from my "jail" fstab into /etc/fstab. After a
half-dozen cycles of "reboot, generate activity on the mount, umount",
I have been unable to reproduce the behavior. FWIW, I did have to set
the nullfs mounts "late" in this configuration to get NFS mounted
before creating the nullfs.
On Sat, Jul 9, 2016 at 9:22 AM, James Gritton <ja...@gritton.org> wrote:
> On 2016-07-08 12:28, Thomas Johnson wrote:
>>
>> I am working on developing a clustered application utilizing jails and
>> running into problems that seem to be NFS-related. I'm hoping that
>> someone can point out my error.
>>
>> The jail images and my application data are served via NFS. The host
>> mounts NFS at boot, and then uses nullfs mounts to assemble the jail
>> tree when the jail is created (fstab files and jail.conf are below).
>> This seems to work fine, the jail starts and is usable. The problem
>> comes when I remove/restart the jail. Frequently (but not
>> consistently), the jail gets stuck in a dying state, causing the
>> unmount of the jail root (nullfs) to fail with a "device busy" error.
>>
>> # jail -f /var/local/jail.conf -r wds1-1a
>> Stopping cron.
>> Waiting for PIDS: 1361.
>> .
>> Terminated
>> wds1-1a: removed
>> umount: unmount of /var/jail/wds1-1a failed: Device busy
>> # jls -av
>> JID Hostname Path
>> Name State
>> CPUSetID
>> IP Address(es)
>> 1 wds1-1a /var/jail/wds1-1a
>> wds1-1a DYING
>> 2
>> 2620:1:1:1:1a::1
>>
>> Through trial-and-error I have determined that forcing an unmount of
>> the root works, but subsequent mounts to that mount point will fail to
>> unmount with the same error. Deleting and recreating the mountpoint
>> fixes the mounting issue, but the dying jail remains permanently.
>>
>> I have also found that if I copy the jail root to local storage and
>> update the jail's fstab to nullfs mount this, the problem seems to go
>> away. This leads me to believe that the issue is related to the NFS
>> source for the nullfs mount. statd and lockd are both running on the
>> host.
>>
>> My relevant configurations are below. I can provide any other
>> information desired.
>>
>> # Host fstab line for jail root.
>> #
>> 10.219.212.1:/vol/dev/wds/jail_base /jail/base nfs ro 0 0
>>
>>
>> # Jail fstab file (mount.fstab)
>> #
>> /jail/base /var/jail/wds1-1a nullfs ro 0 0
>> # writable (UFS-backed) /var
>> /var/jail-vars/wds1-1a /var/jail/wds1-1a/var nullfs rw 0 0
>>
>>
>> # jail.conf file
>> #
>> * {
>> devfs_ruleset = "4";
>> mount.devfs;
>> exec.start = "/bin/sh /etc/rc";
>> exec.stop = "/bin/sh /etc/rc.shutdown";
>> interface = "vmx1";
>> allow.dying = 1;
>> exec.prestart = "/usr/local/bin/rsync -avC --delete
>> /jail/${image}/var/ /var/jail-vars/${host.hostname}/";
>> }
>>
>> # JMANAGE wds1-1a
>> wds1-1a {
>> path = "/var/jail/wds1-1a";
>> ip6.addr = "2620:1:1:1:1a::1";
>> host.hostname = "wds1-1a";
>> host.domainname = "dev";
>> mount.fstab = "/var/local/fstab.wds1-1a";
>> $image = "base";
>> }
>
>
> What happens if you take jails out of the equation? I know this isn't
> entirely a non-jail issue, but I wonder if a jail is required for the mount
> point to be un-re-mountable. I've heard before of NFS-related problems
> where a jail remains dying forever, but this has been more of an annoyance
> than a real problem.
>
> It's not so much that I want to absolve jails, as I want to see where the
> main fight exists. It's tricky enough fixing an interface between two
> systems, but we've got three here.
>
> - Jamie
_______________________________________________
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
