[Unicon-group] An error in the open() function

[Frank J. Lhota]

While testing a port of Unicon to the Cygwin environment, I discovered an error in the Unicon open() function. This error occurs on POSIX platform only. When creating a new file in bi-directional mode (i. e., open for both reading and writing), open() will fail even when it is possible to create the file in bi-directional mode. For example,       f := open( "NewFile.dat", "bcu" )   fails if the file "NewFile.dat" did not exist before the call to open(), even when the corresponding C code       {     FILE* f;       f = fopen( "NewFile.dat", "w+b" );     }   would succeed.   The problem was not hard to trace. The code for the open() function can be found in the source file ./src/runtime/fsys.r. This one function is used to open a wide variety of external data sources and destinations. For POSIX systems, it is possible to open up a directory for reading. For example, if there is a directory "/home/AmyTan/novels", then the Unicon call       d := open( "/home/AmyTan/novels", "r" )   will create a readable Unicon file object for that directory. The _expression_ !d will generate the names of the files and subdirectories in the "/home/AmyTan/novels" directory.   To implement this functionality, the open() function needs to know whether a given path name is the name of a directory. Assume that we are not opening a window, socket, database, etc. and that we are therefore dealing with opening a path name. We need to know if the path name refers to a directory. The POSIX function stat() is used for this purpose: if stat() succeeds and reports that the file is an existing directory, then the Unicon runtime opens and processes that directory using the POSIX opendir() / readdir() / closedir() functions. If stat() succeeds but reports that the path is not a directory, then the Unicon open() tries to open it as a regular file using fopen().   But what happens if stat() fails? The most common cause of stat() failure is an ENOENT error, that is, there is no file or directory with the given name. Given that the Unicon open() function can be used to create new files, some ENOENT errors are to be expected. If the Unicon open() encounters an ENOENT error, it could simple try to open the file as a regular file using fopen(). The current version of open(), however, does the following optimization: if stat() reports an ENOENT error and the open() function is attempting to open the file for reading, then we fail immediately, without even attempting an fopen() call. Here is the code that does this optimization:     else if (stat(fnamestr, &st) < 0) {      if (errno == ENOENT && (status & Fs_Read))         fail;      else         f = fopen(fnamestr, mode);   }   else {      /*       * check and see if the file was actually a directory       */      if (S_ISDIR(st.st_mode)) {         ... One could imagine the reasoning behind this optimization: how can one read from a file that does not exist? This hypothethical question, however, has an answer: one could be creating a new file in bi-directional mode! This optimization is precisely the cause of this problem. When opening a new file in bi-directional mode, the optization skips the fopen() call even though it would succeed and produce a file object that meets the specifications.   It may be possible to fix the optimization with a smarter check of the status bits, but there would be a concern that some other cases might exist where the optimization would skip an fopen() call that might succeed. Given that it is more important to be correct than to be fast, I cut the optimization completely. I modified the code so that fopen() is always attemped if stat() fails:     else if (stat(fnamestr, &st) < 0) {      f = fopen(fnamestr, mode);   }   else {      /*       * check and see if the file was actually a directory       */      if (S_ISDIR(st.st_mode)) {         ... Once I did this, the problem went away.   The IPL has a program "farb2.icn" that demonstrates this problem. You may wish to use it for testing purposes. You may also wish to run this program for your own amusement. Its output is quite entertaining!