If you use an immutable type and replace whole records, then you can mmap
it:
julia> immutable Foo
bar::Int
baz::Float64
end
julia> foos = Array(Foo,10)
10-element Array{Foo,1}:
Foo(0,0.0)
Foo(0,0.0)
Foo(0,0.0)
Foo(0,0.0)
Foo(0,0.0)
Foo(0,0.0)
Foo(0,0.0)
Foo(0,0.0)
Foo(0,0.0)
Foo(0,0.0)
julia> foos[1] = Foo(1,2)
Foo(1,2.0)
julia> p = convert(Ptr{Int},pointer(foos))
Ptr{Int64} @0x00007ff36d436af0
julia> unsafe_load(p,1)
1
julia> unsafe_load(p,2)
4611686018427387904
julia> reinterpret(Float64,unsafe_load(p,2))
2.0
Not as convenient as mutating these in place, but maybe ok.
On Tue, Mar 25, 2014 at 5:35 PM, Keith Mason <desc...@gmail.com> wrote:
> This is a database application, with a large number of rows per table. So
> probably not, for two reasons.
>
> One, in the event the application crashes, I would like the data to
> be updated to disk on a per transaction basis. I can deal with the problem
> of when to call flush() separately.
>
> Two, I would still need to keep track of which data had been updated, so I
> only have to write the data that had been changed. Same problem applies.
>
> Keith
>
> On Mar 25, 2014, at 4:14 PM, Jameson Nash <vtjn...@gmail.com> wrote:
>
> Could you do all of the work in a finalizer ?
>
> On Tuesday, March 25, 2014, Keith Mason <desc...@gmail.com> wrote:
>
>> I definitely understand that. I would honestly prefer to be able to
>> memory map directly to the disk file and let the OS do the writes behind
>> the scenes. Less potential for performance penalties that way. But I
>> haven't found a way in Julia to do that. mmap_array obviously only takes
>> arrays and not singular objects. I could, of course, mmap_array a type
>> into an array of length 1. But the real problem is that composite types
>> are handled as pointers, so mmap_array on a composite type results in the
>> pointers being written to disk and not the actual data. Obvious problems
>> abound with that.
>>
>> I have had success in using mmap_array on immutable types, but the
>> problem with that is that they are immutable. I need to be able to modify
>> individual fields.
>>
>> In C, I can declare a struct, and then cast the pointer of the mmap
>> function to the struct pointer. That doesn't seem to be an option in Julia.
>>
>> If there is another solution I haven't thought of yet, I'm all ears!
>>
>> On Tuesday, March 25, 2014 4:02:09 PM UTC-5, Patrick O'Leary wrote:
>>>
>>> On Tuesday, March 25, 2014 3:48:22 PM UTC-5, Keith Mason wrote:
>>>>
>>>> I have a number of variables based on composite types. I want the
>>>> fields of these variables synced to disk, so that every time I modify a
>>>> field, the data is written to disk. Is there any way to do this with
>>>> assignment notation?
>>>>
>>>
>>> As a more general note, hiding something as expensive as disk access
>>> behind an overloaded `setfield!()` may make performance characteristics
>>> less intuitive, and make tuning that performance--say, by batching
>>> operations together--more difficult. Something to think about.
>>>
>>
