Re: readdir/telldir/seekdir problem (i think)
On 4/23/15 9:54 PM, John Baldwin wrote: On Thursday, April 23, 2015 05:02:08 PM Julian Elischer wrote: On 4/23/15 11:20 AM, Julian Elischer wrote: I'm debugging a problem being seen with samba 3.6. basically telldir/seekdir/readdir don't seem to work as advertised.. ok so it looks like readdir() (and friends) is totally broken in the face of deletes unless you read the entire directory at once or reset to the the first file before the deletes, or earlier. I'm not sure that Samba isn't assuming non-portable behavior. For example: From http://pubs.opengroup.org/onlinepubs/009695399/functions/readdir_r.html If a file is removed from or added to the directory after the most recent call to opendir() or rewinddir(), whether a subsequent call to readdir() returns an entry for that file is unspecified. While this doesn't speak directly to your case, it does note that you will get inconsistencies if you scan a directory concurrent with add and remove. UFS might kind of work actually since deletes do not compact the backing directory, but I suspect NFS and ZFS would not work. looking in ufs_getdirent (or whatever it's called) even UFS is packing the list of files and removing deleted entries. it's also what I'm seeing in practice. if I do a seekdir back to the location of the first deleted file, I find a new file in that slot, that has been shuffled down. In addition, our current NFS support for seekdir is pretty flaky and can't be fixed without changes to return the seek offset for each directory entry (I believe that the projects/ino64 patches include this since they are breaking the ABI of the relevant structures already). The ABI breakage makes this a very non-trivial task. However, even if you have that per-item cookie, it is likely meaningless in the face of filesystems that use any sort of more advanced structure than an array (such as trees, etc.) to store directory entries. POSIX specifically mentions this in the rationale for seekdir: http://pubs.opengroup.org/onlinepubs/009695399/functions/seekdir.html One of the perceived problems of implementation is that returning to a given point in a directory is quite difficult to describe formally, in spite of its intuitive appeal, when systems that use B-trees, hashing functions, or other similar mechanisms to order their directories are considered. The definition of seekdir() and telldir() does not specify whether, when using these interfaces, a given directory entry will be seen at all, or more than once. In fact, given that quote, I would argue that what Samba is doing is non-portable. This would seem to indicate that a conforming seekdir could just change readdir to immediately return EOF until you call rewinddir. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: readdir/telldir/seekdir problem (i think)
John Baldwin wrote: On Thursday, April 23, 2015 05:02:08 PM Julian Elischer wrote: On 4/23/15 11:20 AM, Julian Elischer wrote: I'm debugging a problem being seen with samba 3.6. basically telldir/seekdir/readdir don't seem to work as advertised.. ok so it looks like readdir() (and friends) is totally broken in the face of deletes unless you read the entire directory at once or reset to the the first file before the deletes, or earlier. I'm not sure that Samba isn't assuming non-portable behavior. For example: From http://pubs.opengroup.org/onlinepubs/009695399/functions/readdir_r.html If a file is removed from or added to the directory after the most recent call to opendir() or rewinddir(), whether a subsequent call to readdir() returns an entry for that file is unspecified. While this doesn't speak directly to your case, it does note that you will get inconsistencies if you scan a directory concurrent with add and remove. UFS might kind of work actually since deletes do not compact the backing directory, but I suspect NFS and ZFS would not work. In addition, our current NFS support for seekdir is pretty flaky and can't be fixed without changes to return the seek offset for each directory entry (I believe that the projects/ino64 patches include this since they are breaking the ABI of the relevant structures already). The ABI breakage makes this a very non-trivial task. However, even if you have that per-item cookie, it is likely meaningless in the face of filesystems that use any sort of more advanced structure than an array (such as trees, etc.) to store directory entries. POSIX specifically mentions this in the rationale for seekdir: http://pubs.opengroup.org/onlinepubs/009695399/functions/seekdir.html One of the perceived problems of implementation is that returning to a given point in a directory is quite difficult to describe formally, in spite of its intuitive appeal, when systems that use B-trees, hashing functions, or other similar mechanisms to order their directories are considered. The definition of seekdir() and telldir() does not specify whether, when using these interfaces, a given directory entry will be seen at all, or more than once. In fact, given that quote, I would argue that what Samba is doing is non-portable. This would seem to indicate that a conforming seekdir could just change readdir to immediately return EOF until you call rewinddir. Loosely related to this, I have been tempted to modify readdir() to read the entire directory on the first readdir() for NFS, to avoid the readdir()/unlink() problem. My concern was doing this for a very large directory. Maybe it could be done for directories up to some size? rick -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: readdir/telldir/seekdir problem (i think)
John Baldwin wrote: On Thursday, April 23, 2015 05:02:08 PM Julian Elischer wrote: On 4/23/15 11:20 AM, Julian Elischer wrote: I'm debugging a problem being seen with samba 3.6. basically telldir/seekdir/readdir don't seem to work as advertised.. ok so it looks like readdir() (and friends) is totally broken in the face of deletes unless you read the entire directory at once or reset to the the first file before the deletes, or earlier. I'm not sure that Samba isn't assuming non-portable behavior. For example: From http://pubs.opengroup.org/onlinepubs/009695399/functions/readdir_r.html If a file is removed from or added to the directory after the most recent call to opendir() or rewinddir(), whether a subsequent call to readdir() returns an entry for that file is unspecified. While this doesn't speak directly to your case, it does note that you will get inconsistencies if you scan a directory concurrent with add and remove. UFS might kind of work actually since deletes do not compact the backing directory, but I suspect NFS and ZFS would not work. In addition, our current NFS support for seekdir is pretty flaky and can't be fixed without changes to return the seek offset for each directory entry (I believe that the projects/ino64 patches include this since they are breaking the ABI of the relevant structures already). The ABI breakage makes this a very non-trivial task. However, even if you have that per-item cookie, it is likely meaningless in the face of filesystems that use any sort of more advanced structure than an array (such as trees, etc.) to store directory entries. POSIX specifically mentions this in the rationale for seekdir: http://pubs.opengroup.org/onlinepubs/009695399/functions/seekdir.html One of the perceived problems of implementation is that returning to a given point in a directory is quite difficult to describe formally, in spite of its intuitive appeal, when systems that use B-trees, hashing functions, or other similar mechanisms to order their directories are considered. The definition of seekdir() and telldir() does not specify whether, when using these interfaces, a given directory entry will be seen at all, or more than once. In fact, given that quote, I would argue that what Samba is doing is non-portable. This would seem to indicate that a conforming seekdir could just change readdir to immediately return EOF until you call rewinddir. Btw, Linux somehow makes readdir()/unlink() work for NFS. I haven't looked, but I strongly suspect that it reads the entire directory upon either opendir() or the first readdir(). Oh, and I hate to say it, but I suspect Linux defines the standard on this and not POSIX. (In other words, if it works on Linux, it isn't broken;-) rick -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: readdir/telldir/seekdir problem (i think)
On Thursday, April 23, 2015 05:02:08 PM Julian Elischer wrote: On 4/23/15 11:20 AM, Julian Elischer wrote: I'm debugging a problem being seen with samba 3.6. basically telldir/seekdir/readdir don't seem to work as advertised.. ok so it looks like readdir() (and friends) is totally broken in the face of deletes unless you read the entire directory at once or reset to the the first file before the deletes, or earlier. I'm not sure that Samba isn't assuming non-portable behavior. For example: From http://pubs.opengroup.org/onlinepubs/009695399/functions/readdir_r.html If a file is removed from or added to the directory after the most recent call to opendir() or rewinddir(), whether a subsequent call to readdir() returns an entry for that file is unspecified. While this doesn't speak directly to your case, it does note that you will get inconsistencies if you scan a directory concurrent with add and remove. UFS might kind of work actually since deletes do not compact the backing directory, but I suspect NFS and ZFS would not work. In addition, our current NFS support for seekdir is pretty flaky and can't be fixed without changes to return the seek offset for each directory entry (I believe that the projects/ino64 patches include this since they are breaking the ABI of the relevant structures already). The ABI breakage makes this a very non-trivial task. However, even if you have that per-item cookie, it is likely meaningless in the face of filesystems that use any sort of more advanced structure than an array (such as trees, etc.) to store directory entries. POSIX specifically mentions this in the rationale for seekdir: http://pubs.opengroup.org/onlinepubs/009695399/functions/seekdir.html One of the perceived problems of implementation is that returning to a given point in a directory is quite difficult to describe formally, in spite of its intuitive appeal, when systems that use B-trees, hashing functions, or other similar mechanisms to order their directories are considered. The definition of seekdir() and telldir() does not specify whether, when using these interfaces, a given directory entry will be seen at all, or more than once. In fact, given that quote, I would argue that what Samba is doing is non-portable. This would seem to indicate that a conforming seekdir could just change readdir to immediately return EOF until you call rewinddir. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: help with sandybridge/ivybridge hwpmc NUMA DRAM counters
On Thu, Apr 23, 2015 at 12:24:06AM -0700, Adrian Chadd wrote: On 23 April 2015 at 00:22, Slawa Olhovchenkov s...@zxy.spb.ru wrote: On Thu, Apr 23, 2015 at 12:20:14AM -0700, Adrian Chadd wrote: Yeah, on stable/10. But on -HEAD it's different. There's two entries - one for d3_01 and one for d3_03. What CPU model? CPU: Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz (2600.06-MHz K8-class CPU) Origin=GenuineIntel Id=0x306e4 Family=0x6 Model=0x3e Stepping=4 Same with me? ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: help with sandybridge/ivybridge hwpmc NUMA DRAM counters
On Wed, Apr 22, 2015 at 11:38:22PM -0700, Adrian Chadd wrote: hi all, I'm having a spot of problem trying to get the local/remote dram counters working on the NUMA sandybridge and ivybridge systems here. Things work fine using intel-pcm, but those same counters don't work with hwpmc. Sandybridge - there's apparently an MSR that needs to be fiddled if the counters are active. Ivybridge - the v1 and v2 chips have different local/remote dram counters, and on my v2 setup there's actually /two/ LOCAL_DRAM: # pmccontrol -L | grep DRAM MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM MEM_LOAD_UOPS_LLC_MISS_RETIRED.REMOTE_DRAM CPU: Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz (2600.05-MHz K8-class CPU) adrian@testbox1:~/git/github/erikarn/freebsd/sys/dev/hwpmc % pmccontrol -L | grep DRAM MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM MEM_LOAD_UOPS_LLC_MISS_RETIRED.REMOTE_DRAM Now, i may be able to get to figuring this out at some point in the distant future, but I'd really appreciate any help I can get now. I'm now at the point with the NUMA affinity API stuff where I'm now chasing down when things are correctly working with local/remote RAM, and I'd really like to use hwpmc in sampling mode to work on it. Thanks for any help! -adrian ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: help with sandybridge/ivybridge hwpmc NUMA DRAM counters
On 23 April 2015 at 00:22, Slawa Olhovchenkov s...@zxy.spb.ru wrote: On Thu, Apr 23, 2015 at 12:20:14AM -0700, Adrian Chadd wrote: Yeah, on stable/10. But on -HEAD it's different. There's two entries - one for d3_01 and one for d3_03. What CPU model? CPU: Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz (2600.06-MHz K8-class CPU) Origin=GenuineIntel Id=0x306e4 Family=0x6 Model=0x3e Stepping=4 -adrian ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: help with sandybridge/ivybridge hwpmc NUMA DRAM counters
On Thu, Apr 23, 2015 at 12:20:14AM -0700, Adrian Chadd wrote: Yeah, on stable/10. But on -HEAD it's different. There's two entries - one for d3_01 and one for d3_03. What CPU model? ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
help with sandybridge/ivybridge hwpmc NUMA DRAM counters
hi all, I'm having a spot of problem trying to get the local/remote dram counters working on the NUMA sandybridge and ivybridge systems here. Things work fine using intel-pcm, but those same counters don't work with hwpmc. Sandybridge - there's apparently an MSR that needs to be fiddled if the counters are active. Ivybridge - the v1 and v2 chips have different local/remote dram counters, and on my v2 setup there's actually /two/ LOCAL_DRAM: adrian@testbox1:~/git/github/erikarn/freebsd/sys/dev/hwpmc % pmccontrol -L | grep DRAM MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM MEM_LOAD_UOPS_LLC_MISS_RETIRED.REMOTE_DRAM Now, i may be able to get to figuring this out at some point in the distant future, but I'd really appreciate any help I can get now. I'm now at the point with the NUMA affinity API stuff where I'm now chasing down when things are correctly working with local/remote RAM, and I'd really like to use hwpmc in sampling mode to work on it. Thanks for any help! -adrian ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: help with sandybridge/ivybridge hwpmc NUMA DRAM counters
Yeah, on stable/10. But on -HEAD it's different. There's two entries - one for d3_01 and one for d3_03. -adrian ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: CM6206 USB Audio is mute.
2015-04-22 9:40 GMT+02:00 Hans Petter Selasky h...@selasky.org:
On 04/21/15 21:30, Ranjan1018 . wrote:
Connecting the USB cable the item is recognized:
ugen0.5: vendor 0x0d8c at usbus0
uaudio0: vendor 0x0d8c USB Sound Device, class 0/0, rev 1.10/0.10, addr
5
on usbus0
uaudio0: Play: 48000 Hz, 4 ch, 16-bit S-LE PCM format, 2x8ms buffer.
uaudio0: Play: 44100 Hz, 4 ch, 16-bit S-LE PCM format, 2x8ms buffer.
uaudio0: Record: 48000 Hz, 2 ch, 16-bit S-LE PCM format, 2x8ms buffer.
uaudio0: Record: 44100 Hz, 2 ch, 16-bit S-LE PCM format, 2x8ms buffer.
uaudio0: No MIDI sequencer.
pcm1: USB audio on uaudio0
uaudio0: HID volume keys found.
Hi,
Did you check:
mixer -f /dev/mixerX
that sound volume is OK?
Setting it as default unit
# sysctl hw.snd.default_unit=1
and playing a song the interrupts count increase:
Some volume knobs might be hidden. Try to get all:
sysctl -a | grep pcm
But the output is mute.
In Debian Linux it works:
# lsusb
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 014: ID 0d8c:0102 C-Media Electronics, Inc. CM106 Like
Sound
Device
# aplay -l
List of PLAYBACK Hardware Devices
card 1: Device [USB Sound Device], device 0: USB Audio [USB Audio]
Subdevices: 0/1
Subdevice #0: subdevice #0
Looking at http://lxr.free-electrons.com/source/sound/usb/quirks.c I have
found this code:
/*
632 * C-Media CM106/CM106+ have four 16-bit internal registers that are
nicely
633 * documented in the device's data sheet.
634 */
635 static int snd_usb_cm106_write_int_reg(struct usb_device *dev, int
reg,
u16 value)
636 {
637 u8 buf[4];
638 buf[0] = 0x20;
639 buf[1] = value 0xff;
640 buf[2] = (value 8) 0xff;
641 buf[3] = reg;
642 return snd_usb_ctl_msg(dev, usb_sndctrlpipe(dev, 0),
USB_REQ_SET_CONFIGURATION,
643USB_DIR_OUT | USB_TYPE_CLASS |
USB_RECIP_ENDPOINT,
6440, 0, buf, 4);
645 }
This USB control request can be issued from userspace like this (you need
to convert all variables below into constants):
usbconfig -d X.Y do_request USB_DIR_OUT | USB_TYPE_CLASS |
USB_RECIP_ENDPOINT USB_REQ_SET_CONFIGURATION 0 0 4 0x20 (value
0xff) (value 8) 0xff reg
Such a command can easily be added like a devd script to run automatically
during device attach.
646
647 static int snd_usb_cm106_boot_quirk(struct usb_device *dev)
648 {
649 /*
650 * Enable line-out driver mode, set headphone source to front
651 * channels, enable stereo mic.
652 */
653 return snd_usb_cm106_write_int_reg(dev, 2, 0x8004);
654 }
655
656 /*
657 * C-Media CM6206 is based on CM106 with two additional
658 * registers that are not documented in the data sheet.
659 * Values here are chosen based on sniffing USB traffic
660 * under Windows.
661 */
662 static int snd_usb_cm6206_boot_quirk(struct usb_device *dev)
663 {
664 int err = 0, reg;
665 int val[] = {0x2004, 0x3000, 0xf800, 0x143f, 0x, 0x3000};
666
667 for (reg = 0; reg ARRAY_SIZE(val); reg++) {
668 err = snd_usb_cm106_write_int_reg(dev, reg, val[reg]);
669 if (err 0)
670 return err;
671 }
672
673 return err;
674 }
675
How can I test this code in FreeBSD ?
See answer above.
Please also use -stable or -current when testing.
--HPS
Thank you Hans, it works !!!
This is only the first step in supporting the C-Media CM6206 in FreeBSD,
but works.
I have converted the C code in a bash script: snd-cm6206.sh available at
http://pastebin.com/dLtJAqE7
Instructions:
- Connect the USB cable of the CM6206
- With the command
# dmesg | tail
look at the the line : ugenX.Y: vendor 0x0d8c at usbus
and the line : pcmZ: USB audio on uaudio
- Run the script: # snd-cm6206.sh X.Y
- Run the command: # sysctl hw.snd.default_unit=Z
- Reduce the volume ;-) : # mixer -f /dev/mixerZ 10
On my laptop running 11-CURRENT, after connecting the CM6206, I have :
# dmesg | tail
pcm3: detached
ugen1.4: vendor 0x0d8c at usbus1
uaudio0: vendor 0x0d8c USB Sound Device, class 0/0, rev 1.10/0.10, addr 4
on usbus1
uaudio0: Play: 48000 Hz, 4 ch, 16-bit S-LE PCM format, 2x8ms buffer.
uaudio0: Play: 44100 Hz, 4 ch, 16-bit S-LE PCM format, 2x8ms buffer.
uaudio0: Record: 48000 Hz, 2 ch, 16-bit S-LE PCM format, 2x8ms buffer.
uaudio0: Record: 44100 Hz, 2 ch, 16-bit S-LE PCM format, 2x8ms buffer.
uaudio0: No MIDI sequencer.
pcm3: USB audio on uaudio0
uaudio0: HID volume keys found.
# snd-cm6206.sh 1.4
Writing register 0
REQUEST = OK
Writing register 1
REQUEST = OK
Writing register 2
REQUEST = OK
Writing register 3
REQUEST = OK
Writing register 4
REQUEST = OK
Writing register 5
REQUEST = OK
# sysctl hw.snd.default_unit=3
hw.snd.default_unit: 0 - 3
# mixer -f
Re: readdir/telldir/seekdir problem (i think)
On 4/23/15 11:20 AM, Julian Elischer wrote:
I'm debugging a problem being seen with samba 3.6.
basically telldir/seekdir/readdir don't seem to work as advertised..
ok so it looks like readdir() (and friends) is totally broken in the face
of deletes unless you read the entire directory at once or reset to the
the first file before the deletes, or earlier.
So if you use readdir() to enumerate files in order to delete them,
and you do it in sections in order to not have to load a potentially
huge list of files all at once, you can not do it unless you rewind to
0 after each set of unlinks. (or you do all the unlinks in one hit).
this appears to be because getdirentries (or maybe the filesystem behind)
will 'compact' the returned data.
At least this appears to be the case looking from the outside.
Next step is to look at the getdirentries code..
here's a little test program
#include stdio.h
#include sys/types.h
#include dirent.h
#include fcntl.h
#include errno.h
#include string.h
#include stdlib.h
#include unistd.h
#include sysexits.h
#include err.h
#define CHUNKSIZE 5
#define TOTALFILES 40
static void
SeekDir(DIR *dirp, long loc)
{
printf(Seeking back to location %ld\n, loc);
seekdir(dirp, loc);
}
static long
TellDir(DIR *dirp)
{
long loc;
loc = telldir(dirp);
printf(telldir assigned location %ld\n, loc);
return (loc);
}
int
main(int argc, char *argv[])
{
DIR*dirp;
inti;
intj;
longoffset = 0, prev_offset = 0;
char *files[100];
charfilename[100];
intfd;
struct dirent *dp = NULL;
if (chdir(./test2) != 0) {
err(EX_OSERR, chdir);
}
/*/
/* Put a set of sample files in the target directory */
/*/
for (i=1; i TOTALFILES ; i++)
{
sprintf(filename, file-%d, i);
fd = open(filename, O_CREAT, 0666);
if (fd == -1) {
err(EX_OSERR, open);
}
close(fd);
}
dirp = opendir(.);
offset = TellDir(dirp);
for (i = 0; i 20; i++)
files[i] = malloc(20);
/***/
/* enumerate and delete small sets of files, one group */
/* at a time. */
/***/
do {
/*/
/* Read in up to CHUNKSIZE file names*/
/* i will be the number of files we hold */
/*/
for (i = 0; i CHUNKSIZE; i++) {
if ((dp = readdir(dirp)) != NULL) {
strcpy(files[i], dp-d_name);
printf(readdir (%ld) returned file %s\n,
offset, files[i]);
prev_offset = offset;
offset = TellDir(dirp);
} else {
printf(readdir returned null\n);
break;
}
}
//
/* Simuate the last entry not fitting into our (samba's)
buffer */
/* If we read someting in on the last slot, push it
back*/
/* Pretend it didn't fit. This is approximately what SAMBA
does.*/
//
if (dp != NULL) {
/* Step back */
SeekDir(dirp, prev_offset);
offset = TellDir(dirp);
i--;
printf(file %s returned\n, files[i]);
}
/*/
/* i is the number of names we have left.*/
/* Delete them. */
/*/
for (j = 0; j i; j++) {
if (*files[j] == '.') {
printf (skipping %s\n, files[j]);
} else {
printf(Unlinking file %s\n, files[j]);
if (unlink(files[j]) != 0) {
err(EX_OSERR, unlink);
}
}
}
} while (dp != NULL);
closedir(dirp);
//chdir(..);
}
The program is simulating what Samba does when fails. (doing a
recursive delete of a directory)
What it does is reads a chunk of names using readdir() until it's
(small) buffer s full,
then it uses seekdir() to seek back before the last entry it read,
(which fails to fit),
theortically leaving it for the next read.
It then deletes the entries it found and repeats the cycle.
Eventually it should have found all the files in the directory and
deleted them.
Except that it doesn't.
What actually happens is that some files are not enumerated, even
though
the seekdir() should have made the readdir() find them.
for added fun. the FIRST seekdir appears to work. but all subsequent
ones don't.
It
Re: AUTOFS and NFS not playing together nicely
On 0419T1700, David Boyd wrote: At the risk of being dubbed insane, here goes: On five FreeBSD 10.1-RELEASE-p9 production servers autofs(5) is enabled and working as advertised. On the same servers nfs v3 clients are also fat, dumb and happy. On a test server where autofs(5) is also enabled and working well, I am testing nfs v3 (later v4) server. Strange things are happening. When nfs mountd(8) is running, the autofs(5) auto-mount (via automountd) function seems to work, but the autofs(5) auto-unmount (via autounmountd) never occurs. Without nfs mountd(8), when the filesystem /disc is auto-mounted (via autoumountd), the mount(8) command shows status of (ufs, local, journaled soft-updates, auto-mounted) for the auto-mounted filesystem and after the autofs(5) timeout period (600 seconds) the filesystem is auto-unmounted (via autounmountd). No problem. With nfs mountd(8) the mount(8) command shows (ufs, local, journaled soft-updates). The auto-mounted filesystem is never (a long, long time) unmounted. Big problem. Without nfs mountd(8) running, the mount(8) command mount -o automounted /dev/ada0p8 /disc mounts the filesystem and the mount(8) command shows (ufs, local, journaled soft-updates, automounted) for the manually mounted filesystem and after the autofs(5) timeout period (600 seconds) the filesystem is auto-unmounted even though it was not mounted automatically. No problem. With nfs mountd(8), the mount(8) command mount -o automounted /dev/ada0p8 /disc mounts the filesystem but the mount(8) command shows (ufs, local, journaled soft-updates) and after the timeout period (600 seconds) the filesystem is remains mounted. Big problem. It appears that nfs mountd(8) is interferring with the mount(8) command's -o option processing but admittedly that is just a very weak SWAG. That's an interesting bug. I can reproduce it, but hadn't tracked down the cause just yet. I'll work on that shortly. Thanks for reporting! ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: help with sandybridge/ivybridge hwpmc NUMA DRAM counters
On Thu, Apr 23, 2015 at 10:24:55AM +0300, Slawa Olhovchenkov wrote: On Thu, Apr 23, 2015 at 12:24:06AM -0700, Adrian Chadd wrote: On 23 April 2015 at 00:22, Slawa Olhovchenkov s...@zxy.spb.ru wrote: On Thu, Apr 23, 2015 at 12:20:14AM -0700, Adrian Chadd wrote: Yeah, on stable/10. But on -HEAD it's different. There's two entries - one for d3_01 and one for d3_03. What CPU model? CPU: Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz (2600.06-MHz K8-class CPU) Origin=GenuineIntel Id=0x306e4 Family=0x6 Model=0x3e Stepping=4 Same with me? May be in you case E5-269x? ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Newer yacc needed for building world
On 22/04/2015 23:37, Ed Maste wrote: On 22 April 2015 at 15:55, Willem Jan Withagen w...@digiware.nl wrote: Yes: https://lists.freebsd.org/pipermail/freebsd-current/2015-February/054740.html But this is not enough to make yacc part of the build tools?? yacc is unconditionally built during bootstrap-tools as of r281615. What SVN rev is your tree? # svn info Path: . Working Copy Root Path: /usr/src URL: svn://svn.freebsd.org/base/stable/10 Relative URL: ^/stable/10 Repository Root: svn://svn.freebsd.org/base Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 281853 Node Kind: directory Schedule: normal Last Changed Author: kib Last Changed Rev: 281849 Last Changed Date: 2015-04-22 12:59:05 +0200 (Wed, 22 Apr 2015) Then I removed /usr/obj/* to get a fresh start. Removing yacc just gets me into trouble even earlier: # make -j 32 buildworld . . . --- _bootstrap-tools-usr.bin/compile_et --- --- parse.c --- yacc -d -o parse.c /usr/src/usr.bin/compile_et/../../contrib/com_err/parse.y yacc: not found --- _bootstrap-tools-usr.sbin/bsnmpd/gensnmptree --- /usr/obj/usr/src/tmp/usr/src/usr.sbin/bsnmpd/gensnmptree created for /usr/src/usr.sbin/bsnmpd/gensnmptree --- _bootstrap-tools-usr.bin/compile_et --- *** [parse.c] Error code 127 So I have relatively little further to test. Perhaps the '-j 32' was a bit aggressive, but it gets fast where the error is. --WjW ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: readdir/telldir/seekdir problem (i think)
On 4/24/15 6:12 AM, Rick Macklem wrote: John Baldwin wrote: On Thursday, April 23, 2015 05:02:08 PM Julian Elischer wrote: On 4/23/15 11:20 AM, Julian Elischer wrote: I'm debugging a problem being seen with samba 3.6. basically telldir/seekdir/readdir don't seem to work as advertised.. ok so it looks like readdir() (and friends) is totally broken in the face of deletes unless you read the entire directory at once or reset to the the first file before the deletes, or earlier. I'm not sure that Samba isn't assuming non-portable behavior. For example: From http://pubs.opengroup.org/onlinepubs/009695399/functions/readdir_r.html If a file is removed from or added to the directory after the most recent call to opendir() or rewinddir(), whether a subsequent call to readdir() returns an entry for that file is unspecified. While this doesn't speak directly to your case, it does note that you will get inconsistencies if you scan a directory concurrent with add and remove. UFS might kind of work actually since deletes do not compact the backing directory, but I suspect NFS and ZFS would not work. In addition, our current NFS support for seekdir is pretty flaky and can't be fixed without changes to return the seek offset for each directory entry (I believe that the projects/ino64 patches include this since they are breaking the ABI of the relevant structures already). The ABI breakage makes this a very non-trivial task. However, even if you have that per-item cookie, it is likely meaningless in the face of filesystems that use any sort of more advanced structure than an array (such as trees, etc.) to store directory entries. POSIX specifically mentions this in the rationale for seekdir: http://pubs.opengroup.org/onlinepubs/009695399/functions/seekdir.html One of the perceived problems of implementation is that returning to a given point in a directory is quite difficult to describe formally, in spite of its intuitive appeal, when systems that use B-trees, hashing functions, or other similar mechanisms to order their directories are considered. The definition of seekdir() and telldir() does not specify whether, when using these interfaces, a given directory entry will be seen at all, or more than once. In fact, given that quote, I would argue that what Samba is doing is non-portable. This would seem to indicate that a conforming seekdir could just change readdir to immediately return EOF until you call rewinddir. Btw, Linux somehow makes readdir()/unlink() work for NFS. I haven't looked, but I strongly suspect that it reads the entire directory upon either opendir() or the first readdir(). they make it work for everything apparently.. which is quite a trick. Oh, and I hate to say it, but I suspect Linux defines the standard on this and not POSIX. (In other words, if it works on Linux, it isn't broken;-) rick -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: readdir/telldir/seekdir problem (i think)
On 4/23/15 9:54 PM, John Baldwin wrote: On Thursday, April 23, 2015 05:02:08 PM Julian Elischer wrote: On 4/23/15 11:20 AM, Julian Elischer wrote: I'm debugging a problem being seen with samba 3.6. basically telldir/seekdir/readdir don't seem to work as advertised.. ok so it looks like readdir() (and friends) is totally broken in the face of deletes unless you read the entire directory at once or reset to the the first file before the deletes, or earlier. I'm not sure that Samba isn't assuming non-portable behavior. For example: From http://pubs.opengroup.org/onlinepubs/009695399/functions/readdir_r.html If a file is removed from or added to the directory after the most recent call to opendir() or rewinddir(), whether a subsequent call to readdir() returns an entry for that file is unspecified. While this doesn't speak directly to your case, it does note that you will get inconsistencies if you scan a directory concurrent with add and remove. UFS might kind of work actually since deletes do not compact the backing directory, but I suspect NFS and ZFS would not work. In addition, our current NFS support for seekdir is pretty flaky and can't be fixed without changes to return the seek offset for each directory entry (I believe that the projects/ino64 patches include this since they are breaking the ABI of the relevant structures already). The ABI breakage makes this a very non-trivial task. However, even if you have that per-item cookie, it is likely meaningless in the face of filesystems that use any sort of more advanced structure than an array (such as trees, etc.) to store directory entries. POSIX specifically mentions this in the rationale for seekdir: One of the perceived problems of implementation is that returning to a given point in a directory is quite difficult to describe formally, in spite of its intuitive appeal, when systems that use B-trees, hashing functions, or other similar mechanisms to order their directories are considered. The definition of seekdir() and telldir() does not specify whether, when using these interfaces, a given directory entry will be seen at all, or more than once. In fact, given that quote, I would argue that what Samba is doing is non-portable. This would seem to indicate that a conforming seekdir could just change readdir to immediately return EOF until you call rewinddir. how does readdir know that the directory has been changed? fstat? ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: readdir/telldir/seekdir problem (i think)
On 4/24/15 5:50 AM, Rick Macklem wrote: John Baldwin wrote: On Thursday, April 23, 2015 05:02:08 PM Julian Elischer wrote: On 4/23/15 11:20 AM, Julian Elischer wrote: I'm debugging a problem being seen with samba 3.6. basically telldir/seekdir/readdir don't seem to work as advertised.. ok so it looks like readdir() (and friends) is totally broken in the face of deletes unless you read the entire directory at once or reset to the the first file before the deletes, or earlier. I'm not sure that Samba isn't assuming non-portable behavior. For example: From http://pubs.opengroup.org/onlinepubs/009695399/functions/readdir_r.html If a file is removed from or added to the directory after the most recent call to opendir() or rewinddir(), whether a subsequent call to readdir() returns an entry for that file is unspecified. While this doesn't speak directly to your case, it does note that you will get inconsistencies if you scan a directory concurrent with add and remove. UFS might kind of work actually since deletes do not compact the backing directory, but I suspect NFS and ZFS would not work. In addition, our current NFS support for seekdir is pretty flaky and can't be fixed without changes to return the seek offset for each directory entry (I believe that the projects/ino64 patches include this since they are breaking the ABI of the relevant structures already). The ABI breakage makes this a very non-trivial task. However, even if you have that per-item cookie, it is likely meaningless in the face of filesystems that use any sort of more advanced structure than an array (such as trees, etc.) to store directory entries. POSIX specifically mentions this in the rationale for seekdir: http://pubs.opengroup.org/onlinepubs/009695399/functions/seekdir.html One of the perceived problems of implementation is that returning to a given point in a directory is quite difficult to describe formally, in spite of its intuitive appeal, when systems that use B-trees, hashing functions, or other similar mechanisms to order their directories are considered. The definition of seekdir() and telldir() does not specify whether, when using these interfaces, a given directory entry will be seen at all, or more than once. In fact, given that quote, I would argue that what Samba is doing is non-portable. This would seem to indicate that a conforming seekdir could just change readdir to immediately return EOF until you call rewinddir. Loosely related to this, I have been tempted to modify readdir() to read the entire directory on the first readdir() for NFS, to avoid the readdir()/unlink() problem. I did find a bug in our readdir/seekdir that makes it a lot worse... We reload the kernel's idea of the directory every time we do a seekdir() back, even if it's within the same block, which makes us a lot more susceptible to the problem.. making it not do that unless the new location is in another block made it work on directories with up to several thousand files (with 32k blocksize) before failing. With that bug it's possible do make every seekdir() produce failures even in a directory of just 3 files.. The downside is that the client continues to see the old contents of the block even though he has done a seekdir() within it. My concern was doing this for a very large directory. Maybe it could be done for directories up to some size? rick -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Jenkins build became unstable: FreeBSD_HEAD-tests2 #969
See https://jenkins.freebsd.org/job/FreeBSD_HEAD-tests2/969/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
