Beware that if you don't handle serialisation failures as rejection - then
you can easily get into a state whereby another actor keeps sending a
message that simply will not serialise when persisted. Your only awareness
of this will be the log errors that might occur in your JournalImpl and
your persistent actor will keep restarting because of the failure coming
from the journal.
This can end up being experienced by your read requests to the datastore
going through the roof and if like me you use a store which throttles based
on reads per second then you can end up denying service to the rest of your
app.
On the flip side I've seen journal implementations which rely implicitly on
gapless sequences for reading from the DB - for instance the dynamodb
journal impl (last time I looked at least) reads messages in batches of 50
until it finds no messages in the batch - despite this implicit requirement
it does support rejections. If you have a continuous series of 50
rejections then your in really deep trouble.
My best suggestion would be to set up a simple test which initialises a
persistent actor with a journal which rejects everything. Take a look at
messages like aroundPreStart and onPersistRejected and onPersistFailure and
see what behaviour you want your application to have.
As a final comment - if your implementing your own Journal beware that
EventAdapters are not free and if you plan on allowing the feature in
future (And it is incredibly useful whenever you plan to change your beans
from whatever might be stored to some new version) then you must implement
this in your Journal. We implemented EventAdapter on the read side only (EG
You can read an old version of a Bean and auto-convert it into a newer
version) and the code for that was as simple as:
(Config is com.typesafe.config.Config)
EventAdapters eventAdapters = EventAdapters.apply(actorSystem, actorConfig);
private List<PersistentRepr> persistentFromByteBuffer(ByteBuffer b) throws
NotSerializableException {
PersistentRepr originalRepr =
serialization.deserialize(ByteString.fromByteBuffer(b).toArray(),
PersistentRepr.class).get();
EventSeq adapted =
config.eventAdapters.get(originalRepr.payload().getClass()).fromJournal(originalRepr.payload(),
originalRepr.manifest());
return
JavaConversions.asJavaCollection(adapted.events()).stream().map(originalRepr::withPayload).collect(Collectors.toList());
}
On Tuesday, 11 October 2016 15:42:14 UTC+1, Patrik Nordwall wrote:
>
>
>
> On Tue, Oct 11, 2016 at 3:55 PM, Ian Grima <iang...@gmail.com
> <javascript:>> wrote:
>
>> Thanks for your reply :). If I were not to support rejections, that would
>> imply that if a serialization error occurs i would simply fail the entire
>> Future correct?
>>
>
> yes, right
>
>
>> Gapless sequence numbers does indeed sound like a worthy consideration.
>>
>> Thanks Again,
>> Ian Grima
>>
>> On Tuesday, 11 October 2016 12:21:38 UTC+2, Patrik Nordwall wrote:
>>>
>>>
>>>
>>> On Tue, Oct 11, 2016 at 11:24 AM, Ian Grima <iang...@gmail.com> wrote:
>>>
>>>> I am still looking for an opinion on this matter if anyone could help :)
>>>>
>>>>
>>>>
>>>> On Saturday, 8 October 2016 15:31:32 UTC+2, Ian Grima wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I was reading up on the API of implementing a custom journal plugin
>>>>> in Java. Could you please tell me if I have understood this section
>>>>> correctly.
>>>>>
>>>>> With regards to the following API:
>>>>>
>>>>> Future<Iterable<Optional<Exception>>> doAsyncWriteMessages(Iterable<
>>>>> AtomicWrite> messages);
>>>>>
>>>>> whose Java docs can be found here:
>>>>>
>>>>> http://doc.akka.io/docs/akka/snapshot/java/persistence.html#Journal_plugin_API
>>>>>
>>>>> <http://doc.akka.io/docs/akka/snapshot/java/persistence.html#Journal_plugin_API>
>>>>>
>>>>>
>>>>> Specifically this line : "If there are failures when storing any of
>>>>> the messages in the batch the returned `Future` must be completed
>>>>> with failure."
>>>>>
>>>>> and this section: "The journal can also signal that it rejects
>>>>> individual messages(`AtomicWrite`) by the returned
>>>>> `Iterable<Optional<Exception>>`. The returned `Iterable` must
>>>>> have as many elements as the input `messages` `Iterable`. Each
>>>>> `Optional` element signals if the corresponding `AtomicWrite` is
>>>>> rejected or not, with an exception describing
>>>>> the problem. Rejecting a message means it was not stored, i.e. it
>>>>> must not be included in a later replay. Rejecting a message is
>>>>> typically done before attempting to store it,
>>>>> e.g. because of serialization error."
>>>>>
>>>>>
>>>>> I had difficulty understanding how it would be possible to reject an
>>>>> individual atomic write when the first line states that the future must
>>>>> be
>>>>> completed with failure if storing a single message fails.
>>>>>
>>>>
>>> Rejections are not failure to store them, but it can be seen as a
>>> validation error, i.e. something that is performed before the database is
>>> involved, e.g. serialization error.
>>>
>>>
>>>> Returning the Iterable<Optional<Exception>>> result would imply that
>>>>> the future would have to be completed successfully. If I ended up
>>>>> implementing it in this manner would it be correct? :
>>>>>
>>>>> - Iterate over all messages validating and serializing each one,
>>>>> during this process I would be building an Iterable<Optional<Exception
>>>>> >>> with an Optional.empty for each successfully validated
>>>>> and serialized message and an Optional containing an exception if one
>>>>> of the messages fails validation and serialization.
>>>>>
>>>>> - I would then attempt to persist the successfully serialized batch of
>>>>> messages in a single shot (given it is more optimal to do it in batch for
>>>>> my data store).
>>>>> - If the persist succeed i would complete the future successfully by
>>>>> returning the previously constructed Iterable<Optional<Exception>>>
>>>>> containing
>>>>> possible rejections.
>>>>>
>>>>> - If the persist fails i would simply fail the future with an
>>>>> exception indicating why the persist for the entire set of AtomicWrites
>>>>> failed. We would be loosing information with regards to if any of the
>>>>> AtomicWrites were rejected but I imagine this is somewhat irrelevant
>>>>> since none of them got persisted anyway.
>>>>>
>>>>>
>>> That looks correct.
>>>
>>> You can also consider to not support rejections at all. It's an optional
>>> feature, and many of the existing implementations doesn't support it.
>>> Rejections will result in gaps in the sequence numbers. Gapless sequence
>>> numbers is a very nice feature, much more important than rejections in my
>>> opinion.
>>>
>>> /Patrik
>>>
>>>
>>>>
>>>>> Thanks,
>>>>> Ian Grima
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>>>> >>>>>>>>>> Check the FAQ:
>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>> >>>>>>>>>> Search the archives:
>>>> https://groups.google.com/group/akka-user
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Akka User List" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to akka-user+...@googlegroups.com.
>>>> To post to this group, send email to akka...@googlegroups.com.
>>>> Visit this group at https://groups.google.com/group/akka-user.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> Patrik Nordwall
>>> Akka Tech Lead
>>> Lightbend <http://www.lightbend.com/> - Reactive apps on the JVM
>>> Twitter: @patriknw
>>>
>>> --
>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>> >>>>>>>>>> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to akka-user+...@googlegroups.com <javascript:>.
>> To post to this group, send email to akka...@googlegroups.com
>> <javascript:>.
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
>
> Patrik Nordwall
> Akka Tech Lead
> Lightbend <http://www.lightbend.com/> - Reactive apps on the JVM
> Twitter: @patriknw
>
>
--
Notice: This email is confidential and may contain copyright material of
members of the Ocado Group. Opinions and views expressed in this message
may not necessarily reflect the opinions and views of the members of the
Ocado Group.
If you are not the intended recipient, please notify us immediately and
delete all copies of this message. Please note that it is your
responsibility to scan this message for viruses.
Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled
is a trading name of Marie Claire Beauty Limited, both members of the Ocado
Group.
References to the “Ocado Group” are to Ocado Group plc (registered in
England and Wales with number 7098618) and its subsidiary undertakings (as
that expression is defined in the Companies Act 2006) from time to time.
The registered office of Ocado Group plc is Titan Court, 3 Bishops Square,
Hatfield Business Park, Hatfield, Herts. AL10 9NE.
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
