OK, a couple of updates. First, I tracked down line ptp4l call that starts this off, it's the ioctl(fd, SIOCSHWTSTAMP, &ifreq); line 88 in sk.c. I can see if a standalone program that just does that ioctl has the same affect.
Second, I was able to reproduce this using the mainline 5.0-rc8 kernel. -Paul