If you use an immutable type and replace whole records, then you can mmap it:
julia> immutable Foo bar::Int baz::Float64 end julia> foos = Array(Foo,10) 10-element Array{Foo,1}: Foo(0,0.0) Foo(0,0.0) Foo(0,0.0) Foo(0,0.0) Foo(0,0.0) Foo(0,0.0) Foo(0,0.0) Foo(0,0.0) Foo(0,0.0) Foo(0,0.0) julia> foos[1] = Foo(1,2) Foo(1,2.0) julia> p = convert(Ptr{Int},pointer(foos)) Ptr{Int64} @0x00007ff36d436af0 julia> unsafe_load(p,1) 1 julia> unsafe_load(p,2) 4611686018427387904 julia> reinterpret(Float64,unsafe_load(p,2)) 2.0 Not as convenient as mutating these in place, but maybe ok. On Tue, Mar 25, 2014 at 5:35 PM, Keith Mason <desc...@gmail.com> wrote: > This is a database application, with a large number of rows per table. So > probably not, for two reasons. > > One, in the event the application crashes, I would like the data to > be updated to disk on a per transaction basis. I can deal with the problem > of when to call flush() separately. > > Two, I would still need to keep track of which data had been updated, so I > only have to write the data that had been changed. Same problem applies. > > Keith > > On Mar 25, 2014, at 4:14 PM, Jameson Nash <vtjn...@gmail.com> wrote: > > Could you do all of the work in a finalizer ? > > On Tuesday, March 25, 2014, Keith Mason <desc...@gmail.com> wrote: > >> I definitely understand that. I would honestly prefer to be able to >> memory map directly to the disk file and let the OS do the writes behind >> the scenes. Less potential for performance penalties that way. But I >> haven't found a way in Julia to do that. mmap_array obviously only takes >> arrays and not singular objects. I could, of course, mmap_array a type >> into an array of length 1. But the real problem is that composite types >> are handled as pointers, so mmap_array on a composite type results in the >> pointers being written to disk and not the actual data. Obvious problems >> abound with that. >> >> I have had success in using mmap_array on immutable types, but the >> problem with that is that they are immutable. I need to be able to modify >> individual fields. >> >> In C, I can declare a struct, and then cast the pointer of the mmap >> function to the struct pointer. That doesn't seem to be an option in Julia. >> >> If there is another solution I haven't thought of yet, I'm all ears! >> >> On Tuesday, March 25, 2014 4:02:09 PM UTC-5, Patrick O'Leary wrote: >>> >>> On Tuesday, March 25, 2014 3:48:22 PM UTC-5, Keith Mason wrote: >>>> >>>> I have a number of variables based on composite types. I want the >>>> fields of these variables synced to disk, so that every time I modify a >>>> field, the data is written to disk. Is there any way to do this with >>>> assignment notation? >>>> >>> >>> As a more general note, hiding something as expensive as disk access >>> behind an overloaded `setfield!()` may make performance characteristics >>> less intuitive, and make tuning that performance--say, by batching >>> operations together--more difficult. Something to think about. >>> >>