Hi there,
I think I can shed some light on this. I don't think this is related to
waitForSync or transaction, but has a different root cause.
All documents, when inserted, updated or replaced are saved in ArangoDB's
write-ahead log (WAL) first.
When retrieving a document from the WAL using db.<collection>.document(key)
then a copy of the document will be returned to Foxx/JavaScript as a
regular JavaScript object:
/* fetch document with key postId from database. will return a
JavaScript object */
db.posts.insert({ _key: postId, stats: { likeCount: 0 } });
var post = db.posts.document(postId);
This object can be modified regularly and saved again, using update/replace:
post.stats.likeCount = db.likes.byExample({ _to: 'posts/' + postId
}).count();
db.posts.replace({ _id: post._id }, post);
However, at some point the documents are moved from the WAL into the
datafiles of the respective collections.
When they are then retrieved from the datafiles, they are returned as
light-weight JavaScript objects which contain only pointers to the
underlying document data.
Accessing a property of such light-weight object will trigger a property
handler, which will build the result value and return it as a *temporary
object*
That means when accessing `post.stats` in the following code, `post.stats`
will return a temporary object, for which the property `likeCount` will be
updated:
/* returns light-weight object with property handlers */
var post = db.posts.document(postId);
/* post.stats will produce a tempory object, and post.state.likeCount
will access a property of the temporary object */
post.stats.likeCount = db.likes.byExample({ _to: 'posts/' + postId
}).count();
The modification will only happen in the temporary object, but not in the
`post` object.
When then the post object is written back to the database, it won't contain
the changes.
Following is some example code that demonstrates this for a document that
is stored in the WAL and for a document from the collection datafiles.
In one case the update/replace will do what's expected and in the other it
won't:
```
db._drop("posts");
db._drop("likes");
db._create("posts");
db._createEdgeCollection("likes");
var ShapedJson = require("internal").ShapedJson;
function updateCount(issuerId, postId) {
var post = db.posts.document(postId);
db.likes.insert('users/' + issuerId, 'posts/' + postId, {}, true);
// the post object was fetched before
post.stats.likeCount = db.likes.byExample({ _to: 'posts/' + postId
}).count();
db.posts.replace({ _id: post._id }, post);
}
// insert a new post document. it will be located in the write-ahead log
first
db.posts.insert({ _key: "post1", stats: { likeCount: 0 } });
require("internal").print("the document:", db.posts.document("post1"));
require("internal").print("is ShapedJson:", db.posts.document("post1")
instanceof ShapedJson);
// update the document's count. this will work fine
updateCount("test1", "post1");
// and print results. count should have been updated
require("internal").print("count for post1 after update is:",
db.posts.document("post1"));
// insert another post document. it will be located in the write-ahead log
first
db.posts.insert({ _key: "post2", stats: { likeCount: 0 } });
// now flush the write-ahead log manually and wait for a few seconds
// note: flushing the write-ahead log may happen automatically from time to
time while
// the server is running
require("internal").wal.flush(true, true);
require("internal").wait(5, false);
require("internal").print("the document:", db.posts.document("post2"));
require("internal").print("is ShapedJson:", db.posts.document("post2")
instanceof ShapedJson);
// now update the document's count. this will update the count value of a
ShapedJson
// (which is a temporary object when retrieved from the database)
updateCount("test2", "post2");
// and print results. count will not have been updated
require("internal").print("count for post2 is now:",
db.posts.document("post2"));
```
Note that in the above example for both documents there is a check whether
they are an instance of *ShapedJson*.
In the one case (WAL case) the document won't be a ShapedJson instance but
a regular JS object, but in the other case it will be.
In the latter case, updating a property of the document which itself is an
array or object won't work, because of the temporaries produced by the
property handler.
What helps in this case is to convert the ShapedJson instance of the
document into a regular JS object, e.g. via
var _ = require("underscore");
...
var post = _.clone(db.posts.document(postId));
Or, combined with the ShapedJson test:
var ShapedJson = require("internal").ShapedJson;
var _ = require("underscore");
...
var post = db.posts.document(postId);
if (post instance of ShapedJson) {
// ShapedJson object
post = _.clone(post);
}
Note that this applies to the 1.x and 2.x branches of ArangoDB.
We have simplified this a lot in 3.0 so both the ShapedJson testing and
cloning can be omitted there. All objects returned in 3.0 will be regular
JS objects free of side effects.
I hope this helps.
Best regards
Jan
2016-06-14 5:18 GMT+02:00 Thomas Weiss <[email protected]>:
> As a follow-up, I've just seen the same behavior on my dev machine, it
> just happens less frequently (probably because it has better perfs than the
> server used for staging).
> Any comment will be welcome!
>
> Thanks,
> Thomas
>
>
> On Sunday, June 12, 2016 at 3:57:04 PM UTC+8, Thomas Weiss wrote:
>>
>> Hi there,
>>
>> I'm using a Foxx app to execute transactions. To boost query efficiency
>> (the system is read-heavy), I try to cache the count of edges directly in
>> the docs, and do this like that:
>> db.likes.insert('users/' + issuerId, 'posts/' + postId, {}, true);
>> // the post object was fetched before
>> post.stats.likeCount = db.likes.byExample({ _to: 'posts/' + postId }).
>> count();
>> db.posts.replace({ _id: post._id }, post);
>> Notice the last *true* parameter passed to *insert* to make sure that we
>> wait for the write to be flushed. This code is executed within a
>> transaction by the way.
>>
>> This works well on my dev machine (Windows 10) *but* it seems that the
>> count is not systematically updated correctly on the staging server
>> (Ubuntu). What's weird is that sometimes it works and sometimes it doesn't,
>> which makes me think about a threading/concurrency issue (and that's why I
>> added the waitForSync param).
>>
>> Any help or suggestion would be greatly appreciated here.
>>
>> Thanks,
>> Thomas
>>
> --
> You received this message because you are subscribed to the Google Groups
> "ArangoDB" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"ArangoDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
