> I have narrowed down the issue with a small program, and concluded from > running that the problem showed up whenever the position indicator wasn’t at > SEEK_END on a file that is writable.
If you can tell us how it behaves on DMTCP-1.2.4, that will be nice. I also have one further question. At the time of checkpoint, was the SEEK_END at the end of the file? If it was, then I believe DMTCP will assume that the file is being written in streaming access. In that case, on restart, DMTCP should truncate the file back down to the original length. (If this is not happening in 1.2.4 for you, that's a bug, and we'd appreciate hearing about it.) On the other hand, if the SEEK_END was not at the end of the file at the time of checkpoint, then the proper heuristic is not clear. This could be a file for which the application is writing to random access locations. In that case, it might be better to use: dmtcp_checkpoint --checkpoint-open-files [ Checkpoint open files and restore old working dir. (Default: do neither) ] dmtcp_checkpoint --help will show you the various options. I hope these comments help, - Gene On Thu, Feb 23, 2012 at 03:57:07PM -0500, Kapil Arya wrote: > Hi Joshua, > > First of all, I want to apologize for not sending this reply earlier. > The email got lost somehow, and I saw it today only. > > Do you still have the problem? If so, I would be very happy to work on > it to fix it. Please let me know. > > BTW, we released 1.2.4 a few weeks ago and so you might want to try this. > > Thanks and once again, sorry for not replying in sooner. > Kapil > > On Tue, Dec 6, 2011 at 12:49 PM, Louie, Joshua D > <[email protected]> wrote: > > Hello, > > > > > > > > While working on a larger project using DMTCP, I ran into an issue regarding > > IO file when doing the dmtcp_restart_script.sh. The error generally looks > > like: > > > > [2815] ERROR at connection.cpp:1054 in restore; > > REASON='JASSERT(jalib::Filesystem::FileExists(_path) == false) failed' > > > > _path = some_path/some_file > > > > > > > > I have narrowed down the issue with a small program, and concluded from > > running that the problem showed up whenever the position indicator wasn’t at > > SEEK_END on a file that is writable. > > > > I’m using DMTCP version 1.2.3, configured without any special options. > > > > > > > > Sorry the below program is so large, originally I had thought the problem > > might be due to open instead of fopen, but that didn’t turn out to be true. > > > > Anytime we use write or read/write and use SEEK_SET (not SEEK_END). I also > > tested (not shown below) that if I simply offset from SEEK_END minus one, it > > still had the problem, so I think it only works for writing if it’s at > > SEEK_END specifically. > > > > To expose the problem, I ran the below program (named test_io) as > > “dmtcp_checkpoint test_io -read -write -use_open -position SEEK_SET” > > > > > > > > #include <stdio.h> > > > > #include <stdlib.h> > > > > #include <sys/types.h> > > > > #include <sys/stat.h> > > > > #include <fcntl.h> > > > > > > > > #include <dmtcpaware.h> > > > > > > > > #define SOME_FILE "some_file" > > > > > > > > int main(int argc, char *argv[]) > > > > { > > > > int i, show_usage = 0; > > > > int use_fopen = 1; > > > > int is_read = 0, is_write = 0; > > > > int origin = SEEK_CUR; > > > > char *position_str = "not moved"; > > > > > > > > /* Touch file to be written/read to be safe */ > > > > system("touch " SOME_FILE); > > > > > > > > for (i = 1; i < argc; i++) { > > > > if (!strcmp(argv[i], "-use_open")) { > > > > use_fopen = 0; > > > > } else if (!strcmp(argv[i], "-read")) { > > > > is_read = 1; > > > > } else if (!strcmp(argv[i], "-write")) { > > > > is_write = 1; > > > > } else if (!strcmp(argv[i], "-h")) { > > > > show_usage = 1; > > > > } else if (!strcmp(argv[i], "-position")) { > > > > i++; > > > > if (!strcmp(argv[i], "SEEK_SET")) { > > > > origin = SEEK_SET; > > > > position_str = "moved to beginning"; > > > > } else if (!strcmp(argv[i], "SEEK_END")) { > > > > origin = SEEK_END; > > > > position_str = "moved to end"; > > > > } else { > > > > show_usage = 1; > > > > } > > > > } else { > > > > show_usage = 1; > > > > } > > > > } > > > > > > > > if (!is_read && !is_write) { > > > > show_usage = 1; > > > > } > > > > > > > > if (show_usage) { > > > > printf("Usage: %s [-h] [-use_open] [-read] [-write] " > > > > "<-position <SEEK_SET|SEEK_END>>\n"); > > > > printf(" -h : Display usage\n"); > > > > printf(" -use_open : Use open function instead of fopen\n"); > > > > printf(" -read : Open file with read flag.\n"); > > > > printf(" -write : Open file with write flag.\n"); > > > > printf(" -position : Position indicator moved to start or " > > > > "end of file\n"); > > > > printf("\nMust use -read and/or -write. If both are given, then " > > > > "open the file in both read/write\n"); > > > > return 0; > > > > } > > > > > > > > /* Open file */ > > > > if (use_fopen) { > > > > FILE *fp; > > > > > > > > if (is_read && is_write) { > > > > fp = fopen(SOME_FILE, "r+"); > > > > } else if (is_read) { > > > > fp = fopen(SOME_FILE, "r"); > > > > } else if (is_write) { > > > > fp = fopen(SOME_FILE, "w"); > > > > } > > > > > > > > /* Write something in the file if given write */ > > > > if (is_write) { > > > > fprintf(fp, "Garbage\n"); > > > > fflush(fp); > > > > } > > > > > > > > /* Move to requested position */ > > > > fseek(fp, 0, origin); > > > > } else { > > > > int fd; > > > > > > > > if (is_read && is_write) { > > > > fd = open(SOME_FILE, O_RDWR); > > > > } else if (is_read) { > > > > fd = open(SOME_FILE, O_RDONLY); > > > > } else if (is_write) { > > > > fd = open(SOME_FILE, O_WRONLY); > > > > } > > > > > > > > /* Write something in the file if given write */ > > > > if (is_write) { > > > > write(fd, "Garbage\n", 8); > > > > } > > > > > > > > /* Move to requested position */ > > > > lseek(fd, 0, origin); > > > > } > > > > > > > > sleep(1); > > > > if (dmtcpIsEnabled()) { > > > > printf("Pre-checkpoint\n"); > > > > printf("Accessing file with: %s\n", use_fopen ? "fopen" : "open"); > > > > printf("Position indicator %s\n", position_str); > > > > printf("File opened in the following modes:%s%s\n", > > > > (is_read) ? " read" : "", > > > > (is_write) ? " write" : ""); > > > > dmtcpCheckpoint(); > > > > sleep(1); > > > > printf("Post-checkpoint\n"); > > > > } > > > > printf("Accessing file with: %s\n", use_fopen ? "fopen" : "open"); > > > > printf("Position indicator %s\n", position_str); > > > > printf("File opened in the following modes:%s%s\n", > > > > (is_read) ? " read" : "", > > > > (is_write) ? " write" : ""); > > > > return 0; > > > > } > > > > > > > > Thanks, > > > > Joshua Louie > > > > > > ------------------------------------------------------------------------------ > > Cloud Services Checklist: Pricing and Packaging Optimization > > This white paper is intended to serve as a reference, checklist and point of > > discussion for anyone considering optimizing the pricing and packaging model > > of a cloud services business. Read Now! > > http://www.accelacomm.com/jaw/sfnl/114/51491232/ > > _______________________________________________ > > Dmtcp-forum mailing list > > [email protected] > > https://lists.sourceforge.net/lists/listinfo/dmtcp-forum > > > > ------------------------------------------------------------------------------ > Virtualization & Cloud Management Using Capacity Planning > Cloud computing makes use of virtualization - but cloud computing > also focuses on allowing computing to be delivered as a service. > http://www.accelacomm.com/jaw/sfnl/114/51521223/ > _______________________________________________ > Dmtcp-forum mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/dmtcp-forum ------------------------------------------------------------------------------ Virtualization & Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ _______________________________________________ Dmtcp-forum mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/dmtcp-forum
