PROBLEM: Failed open() with O_DIRECT creates file

[Hallvard Breien Furuseth]

If the filesystem does not support O_DIRECT, then
open(...O_CREAT|O_DIRECT..) fails but creates the file anyway.

Eric Sandeen@RedHat thought a fix would need a lot of vfs restructuring.
(I reported this in 2013 to RedHat (bug#1008073), but just realized he
was talking about "upstream" so maybe the report didn't get further.)

Linux 3.10.0-327.28.3.el7.x86_64, RHEL 7.2 (Maipo).
The bug existed at least since 2.6.18-348.6.1.el5.

To reproduce, run this as:

  ./a.out /dev/test-direct.dat

(Originally I wrote to /dev/shm/, but tmpfs now accepts O_DIRECT.)

#define _GNU_SOURCE
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <sys/stat.h>
#include <unistd.h>

int main(int argc, char *argv[]) {
    char *fname = argc > 1 ? argv[1] : "test-direct.dat";
    int fd = open(fname, O_WRONLY|O_CREAT|O_EXCL|O_DIRECT, 0666);
    int e = fd < 0 ? errno : 0;

    if (fd >= 0)
        puts("open() succeeded");
    else
        perror("open() error");
    if ((access(fname, F_OK) == 0) != (fd >= 0 || e == EEXIST))
        puts(fd < 0 ? "Created file anyway!" : "File disappeared!");

    if (e != EEXIST)
        unlink(fname);
    return 0;
}

-- 
Hallvard