Hi Jan,
Thank you so much for this very extensive reply. I know you guys are
working hard on the 3.0 release so I really appreciate the time you took to
write this answer!
If I understand correctly what you explained, am I right that the following
code would also solve my issue:
post.stats = {
likeCount: db.likes.byExample({ _to: 'posts/' + postId }).count(),
commentCount: post.stats.commentCount,
shareCount: post.stats.shareCount
};
db.posts.replace({ _id: post._id }, post);
Cheers,
Thomas
On Tuesday, June 14, 2016 at 3:59:57 PM UTC+8, Jan Steemann wrote:
>
> Hi there,
>
> I think I can shed some light on this. I don't think this is related to
> waitForSync or transaction, but has a different root cause.
>
> All documents, when inserted, updated or replaced are saved in ArangoDB's
> write-ahead log (WAL) first.
> When retrieving a document from the WAL using
> db.<collection>.document(key) then a copy of the document will be returned
> to Foxx/JavaScript as a regular JavaScript object:
>
> /* fetch document with key postId from database. will return a
> JavaScript object */
> db.posts.insert({ _key: postId, stats: { likeCount: 0 } });
> var post = db.posts.document(postId);
>
> This object can be modified regularly and saved again, using
> update/replace:
>
> post.stats.likeCount = db.likes.byExample({ _to: 'posts/' + postId
> }).count();
> db.posts.replace({ _id: post._id }, post);
>
> However, at some point the documents are moved from the WAL into the
> datafiles of the respective collections.
> When they are then retrieved from the datafiles, they are returned as
> light-weight JavaScript objects which contain only pointers to the
> underlying document data.
> Accessing a property of such light-weight object will trigger a property
> handler, which will build the result value and return it as a *temporary
> object*
> That means when accessing `post.stats` in the following code, `post.stats`
> will return a temporary object, for which the property `likeCount` will be
> updated:
>
> /* returns light-weight object with property handlers */
> var post = db.posts.document(postId);
> /* post.stats will produce a tempory object, and post.state.likeCount
> will access a property of the temporary object */
> post.stats.likeCount = db.likes.byExample({ _to: 'posts/' + postId
> }).count();
>
> The modification will only happen in the temporary object, but not in the
> `post` object.
> When then the post object is written back to the database, it won't
> contain the changes.
>
> Following is some example code that demonstrates this for a document that
> is stored in the WAL and for a document from the collection datafiles.
> In one case the update/replace will do what's expected and in the other it
> won't:
>
> ```
> db._drop("posts");
> db._drop("likes");
>
> db._create("posts");
> db._createEdgeCollection("likes");
>
> var ShapedJson = require("internal").ShapedJson;
>
> function updateCount(issuerId, postId) {
> var post = db.posts.document(postId);
> db.likes.insert('users/' + issuerId, 'posts/' + postId, {}, true);
> // the post object was fetched before
> post.stats.likeCount = db.likes.byExample({ _to: 'posts/' + postId
> }).count();
> db.posts.replace({ _id: post._id }, post);
> }
>
> // insert a new post document. it will be located in the write-ahead log
> first
> db.posts.insert({ _key: "post1", stats: { likeCount: 0 } });
> require("internal").print("the document:", db.posts.document("post1"));
> require("internal").print("is ShapedJson:", db.posts.document("post1")
> instanceof ShapedJson);
>
> // update the document's count. this will work fine
> updateCount("test1", "post1");
>
> // and print results. count should have been updated
> require("internal").print("count for post1 after update is:",
> db.posts.document("post1"));
>
>
>
> // insert another post document. it will be located in the write-ahead log
> first
> db.posts.insert({ _key: "post2", stats: { likeCount: 0 } });
> // now flush the write-ahead log manually and wait for a few seconds
> // note: flushing the write-ahead log may happen automatically from time
> to time while
> // the server is running
> require("internal").wal.flush(true, true);
> require("internal").wait(5, false);
>
> require("internal").print("the document:", db.posts.document("post2"));
> require("internal").print("is ShapedJson:", db.posts.document("post2")
> instanceof ShapedJson);
>
> // now update the document's count. this will update the count value of a
> ShapedJson
> // (which is a temporary object when retrieved from the database)
> updateCount("test2", "post2");
>
> // and print results. count will not have been updated
> require("internal").print("count for post2 is now:",
> db.posts.document("post2"));
> ```
>
> Note that in the above example for both documents there is a check whether
> they are an instance of *ShapedJson*.
> In the one case (WAL case) the document won't be a ShapedJson instance but
> a regular JS object, but in the other case it will be.
> In the latter case, updating a property of the document which itself is an
> array or object won't work, because of the temporaries produced by the
> property handler.
>
> What helps in this case is to convert the ShapedJson instance of the
> document into a regular JS object, e.g. via
>
> var _ = require("underscore");
> ...
> var post = _.clone(db.posts.document(postId));
>
> Or, combined with the ShapedJson test:
>
> var ShapedJson = require("internal").ShapedJson;
> var _ = require("underscore");
> ...
> var post = db.posts.document(postId);
>
>
> if (post instance of ShapedJson) {
> // ShapedJson object
> post = _.clone(post);
> }
>
> Note that this applies to the 1.x and 2.x branches of ArangoDB.
> We have simplified this a lot in 3.0 so both the ShapedJson testing and
> cloning can be omitted there. All objects returned in 3.0 will be regular
> JS objects free of side effects.
>
> I hope this helps.
> Best regards
> Jan
>
>
>
> 2016-06-14 5:18 GMT+02:00 Thomas Weiss <[email protected]
> <javascript:>>:
>
>> As a follow-up, I've just seen the same behavior on my dev machine, it
>> just happens less frequently (probably because it has better perfs than the
>> server used for staging).
>> Any comment will be welcome!
>>
>> Thanks,
>> Thomas
>>
>>
>> On Sunday, June 12, 2016 at 3:57:04 PM UTC+8, Thomas Weiss wrote:
>>>
>>> Hi there,
>>>
>>> I'm using a Foxx app to execute transactions. To boost query efficiency
>>> (the system is read-heavy), I try to cache the count of edges directly in
>>> the docs, and do this like that:
>>> db.likes.insert('users/' + issuerId, 'posts/' + postId, {}, true);
>>> // the post object was fetched before
>>> post.stats.likeCount = db.likes.byExample({ _to: 'posts/' + postId }).
>>> count();
>>> db.posts.replace({ _id: post._id }, post);
>>> Notice the last *true* parameter passed to *insert* to make sure that
>>> we wait for the write to be flushed. This code is executed within a
>>> transaction by the way.
>>>
>>> This works well on my dev machine (Windows 10) *but* it seems that the
>>> count is not systematically updated correctly on the staging server
>>> (Ubuntu). What's weird is that sometimes it works and sometimes it doesn't,
>>> which makes me think about a threading/concurrency issue (and that's why I
>>> added the waitForSync param).
>>>
>>> Any help or suggestion would be greatly appreciated here.
>>>
>>> Thanks,
>>> Thomas
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "ArangoDB" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"ArangoDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
