Not misleading - I was just talking about the standard file objects, not these channels. So this might be your answer then.
On Dec 26, 6:36 pm, Eitan <eitan...@gmail.com> wrote: > So do you think that the FileChannel documentation is misleading? > > "The view of a file provided by an instance of this class is > guaranteed > to be consistent with other views of the same file provided by other > instances in the same program" > > This is taken from the javadoc of FileChannel. > Thanks > > On Dec 26, 3:09 am, Christian Catchpole <christ...@catchpole.net> > wrote: > > > > > Well, as you are finding, this "simple" approach is causing you > > problems. The only thing I can add is to just be cautious about > > relying on side-effects or presumptions about these multiple readers. > > Because they could change at any time. I've never considered such an > > approach because I have always thought of a File Stream / Reader and > > the actual file as a 1:1 relationship. Anything else (except for 100% > > read only) sounds dodgy. You mention you sometimes going into append > > mode. But this would invalidate all the other read locks. > > > I agree that caches are not magic - they only work when on average, > > access tends to cause clumps of common data. > > > If you were to pull apart the guts of database i wonder if you would > > find multiple file readers. My guess is no. > > > On Dec 26, 1:07 am, Eitan <eitan...@gmail.com> wrote: > > > > Because it's a simple feature that I need and I don't want an overkill > > > solution that I do not control its thread count, its memory > > > consumption and cpu usage. > > > It is just a simple component inside my highly performance server that > > > needs to keep a single list of redo log in the disk if it takes to > > > much memory which will be very light and won't impact the server > > > performance. This occurs when the backup server it replicates to is > > > temporarily down and the redo log starts to accumulate. There are many > > > consideration which I can't really explain, but I do not wish to use > > > any embedded or remote third party overkill product for this very > > > simplified capabilities, like I've described above, unlimited single > > > list of elements that can only be added at the end of the list, > > > removed from the first and iterated over the list in read only mode. > > > > Any other generic product addresses a much more complex scenario and > > > probably will perform worse because of that. > > > > I already implemented it and it works fine, but I am not sure about > > > the multi reader over a file regarding the visibility issue and I was > > > hoping that someone knows what kind of guarantee is there for the > > > visibility of multiple RandomAccessFile inside the same process over > > > the same file. (only one in rw mode and the rest in r mode) > > > > Thanks > > > > On Dec 25, 12:41 pm, Viktor Klang <viktor.kl...@gmail.com> wrote: > > > > > Why a file mate? > > > > Why not Redis, Cassandra, MongoDB, SQLite whatnot? > > > > > On Fri, Dec 25, 2009 at 10:20 AM, Eitan <eitan...@gmail.com> wrote: > > > > > What about what I said above, using multiple FileChannel/ > > > > > RandomAccessFile to read from the same file (not writing, there's a > > > > > single writer multi reader mutual exclusion protection over these > > > > > FileChannel/RandomAccessFile access), all I am worried about is the > > > > > visibility that updates the writer will do will be visible to the > > > > > readers once done, FileChannel guarantee this and I suspect it must > > > > > rely on this kind of guarantee from the RandomAccessFile that creates > > > > > it though it is not documented, But according to the simple analysis I > > > > > of the RandomAccessFile getChannel method I did, it seems that it does > > > > > need to guarantee it in order for FileChannel to guarantee it. > > > > > > Since my implementation here is just an "unlimited" single list of > > > > > objects that starts in the memory and once there's no room continue in > > > > > the disk, I went for this solution and not using an over kill embedded > > > > > data base. I only need to support appending objects at the end of the > > > > > list, removing from the start and iterating over the list in read only > > > > > mode. (Either from the start to the end or from the end to the start). > > > > > > Regarding the caching you suggested, I created caching mechanism and > > > > > there are a few layers of abstraction over this file, but, I am > > > > > talking here about the lowest level of the abstraction which access > > > > > the file and that might occur concurrently and I want to remove the > > > > > contention there as well. (for instance 10 threads currently iterating > > > > > over the list all at a different location) besides the whole issue > > > > > here is to move the data from the memory to the disk because I am out > > > > > of memory space, so caching the 10g file is not an option, I only > > > > > cache the end and the start of the list since they are accesses most > > > > > frequently. > > > > > The time spending ratio here is not the problem, there can be tens of > > > > > threads concurrently iterating over the list and I want them to be > > > > > able to iterate concurrently and not wait for each other. According to > > > > > amdal's law even if the time spent on reading is just 10%, I can only > > > > > accelerate this process by at most 10 so 100 threads iterating will > > > > > work no faster than 10. > > > > > > Basically if I get the visibility guarantee then my multi > > > > > RandomAccessFile in read mode over the file is good enough solution, > > > > > let the disk and the OS do the caching of accessing the same file > > > > > sections and keep my program more concurrent. > > > > > > Thanks > > > > > > On Dec 24, 11:45 pm, Christian Catchpole <christ...@catchpole.net> > > > > > wrote: > > > > > > Is not the simplest solution to allow each thread to get a reference > > > > > > to the *single* RandomAccessFile instance and simply synchronize on > > > > > > it > > > > > > while reading (not forgetting to reset the offset into the file). > > > > > > It's not going to be super performant, but how is it ever going to > > > > > > be > > > > > > with multiple threads thrashing on one file? If anything it might > > > > > > reduce thrashing as you are controlling the points of file > > > > > > contention. What is the ratio of time spent reading the file to > > > > > > time > > > > > > spent operating on the data? > > > > > > > If all these threads will in-fact thrash on the file it might only > > > > > > be > > > > > > solved with: > > > > > > > 1. Better scheduling than "free for all" threads. > > > > > > 2. a large OS level disk cache > > > > > > 3. an in memory disk cache > > > > > > 4. some kind of abstract layer over the file which does caching and > > > > > > returns you "model objects" rather than being seen as a byte stream. > > > > > > > On Dec 25, 1:38 am, Eitan <eitan...@gmail.com> wrote: > > > > > > > > Looking at FileChannel it has the following part in its > > > > > > > documentation: > > > > > > > > The view of a file provided by an instance of this class is > > > > > > > guaranteed > > > > > > > to be consistent with other views of the same file provided by > > > > > > > other > > > > > > > instances in the same program. The view provided by an instance of > > > > > > > this > > > > > > > class may or may not, however, be consistent with the views seen > > > > > > > by > > > > > > > other > > > > > > > concurrently-running programs due to caching performed by the > > > > > > > underlying > > > > > > > operating system and delays induced by network-filesystem > > > > > > > protocols. > > > > > > > This > > > > > > > is true regardless of the language in which these other programs > > > > > > > are > > > > > > > written, and whether they are running on the same machine or on > > > > > > > some > > > > > > > other > > > > > > > machine. The exact nature of any such inconsistencies are system- > > > > > > > dependent > > > > > > > and are therefore unspecified. > > > > > > > > This looks exactly like the guarantee I need, now I've looked into > > > > > > > RandomAccessFile.getChannel() implementation and it holds a single > > > > > > > member of FileChannel which is initialized on demand and the next > > > > > > > calls will return the same instance. > > > > > > > > So the only way I see to create multiple FileChannel instances > > > > > > > over > > > > > > > the same file is to open multiple RandomAccessFile instances over > > > > > > > that > > > > > > > file and use getChannel() on each of these instances. Since > > > > > > > FileChannel does provide this guarantee I tend to believe it > > > > > > > relies > > > > > > > upon a guarantee that RandomAccessFile provide, otherwise it would > > > > > > > need some mechanism to identify that different FileChannel > > > > > > > instances > > > > > > > over different RandomAccessFile instances are actually over the > > > > > > > same > > > > > > > file in order to provide the specified guarantee. > > > > > > > > What do you think? > > > > > > > > On Dec 21, 9:33 am, Ben Schulz <ya...@gmx.net> wrote: > > > > > > > > > My guess is that Java IO takes advantage of any OS optimizations > > > > > there > > > > > > > > are, on another OS there may be a per-file-pointer cache or > > > > > something. > > > > > > > > That would obviously invalidate your assumptions. I suggest you > > > > > > > > look > > > > > > > > into using asynchronous I/O which will get rid of any > > > > > multiple-reader- > > > > > > > > bottlenecks. This should also rule out third parties tampering > > > > > > > > with > > > > > > > > the file (since you already hold the write lock). > > > > > > > > > With kind regards > > > > > > > > Ben > > > > > > > > > On 21 Dez., 07:50, Eitan <eitan...@gmail.com> wrote: > > > > > > > > > > I am trying to avoid locking the access to file for multiple > > > > > readers > > > > > > > > > can be multi concurrent readers and it will be a bottleneck. > > > > > > > > > I've created a multi threaded test for it and it seems to work > > > > > fine, > > > > > > > > > but I am not still not confident about it. I was hoping to > > > > > > > > > find > > > > > some > > > > > > > > > documentation about it which specifies the behavior. > > > > > > > > > > On Dec 20, 11:40 pm, Christian Catchpole > > > > > > > > > <christ...@catchpole.net> > > > > > > > > > wrote: > > > > > > > > > > > Can you even get a second lock on a file that way? Sounds > > > > > dangerous. > > > > > > > > > > I would just create a some kind of service that each Thread > > > > > > > > > > can > > > > > access > > > > > > > > > > that will synchronise disk > > ... > > read more » -- You received this message because you are subscribed to the Google Groups "The Java Posse" group. To post to this group, send email to javapo...@googlegroups.com. To unsubscribe from this group, send email to javaposse+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/javaposse?hl=en.
