Re: [akka-user] Akka Persistence journal size
+1 and to add to this... When you look at your data acquisition you need to remember that technology moves forward as well. How does drawing your curve look next to Kryder's law? http://en.wikipedia.org/wiki/Mark_Kryder On Friday, October 31, 2014 11:20:08 AM UTC+2, Akka Team wrote: Hi Dan, On Thu, Oct 30, 2014 at 6:36 PM, Dan Ellis d...@danellis.me javascript: wrote: You can delete them by reacting to a snapshot succcess by issuing an deleteMessages(toSequenceNr = lastseqnr). If I only care about recovery and not auditing, are there any disadvantages to doing that? This is also about recovery. If you have the full event log you can recover *corrupted* storage much easier, since you can delete corrupted events, modify events, etc. if necessary, which is way harder with a corrupted blob. There is a reason why many filesystems use a journal, and many databases provide an oplog, too. If you are worried about growing journal size and you really want to delete, I recommend first going without delete, seeing how quickly the journal grows and then implement a deletion policy that preferably keeps logs not just after the latest snapshot, but maybe after the 3rd latest snapshot or so, depending on your disk space requirement. -Endre Would it be considered an anti-pattern? I'm just evaluating and considering my options at the moment -- I don't know that space will be an issue. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com javascript:. To post to this group, send email to akka...@googlegroups.com javascript:. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Akka Team Typesafe - The software stack for applications that scale Blog: letitcrash.com Twitter: @akkateam -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Akka Persistence journal size
Hi Dan, On Thu, Oct 30, 2014 at 6:36 PM, Dan Ellis d...@danellis.me wrote: You can delete them by reacting to a snapshot succcess by issuing an deleteMessages(toSequenceNr = lastseqnr). If I only care about recovery and not auditing, are there any disadvantages to doing that? This is also about recovery. If you have the full event log you can recover *corrupted* storage much easier, since you can delete corrupted events, modify events, etc. if necessary, which is way harder with a corrupted blob. There is a reason why many filesystems use a journal, and many databases provide an oplog, too. If you are worried about growing journal size and you really want to delete, I recommend first going without delete, seeing how quickly the journal grows and then implement a deletion policy that preferably keeps logs not just after the latest snapshot, but maybe after the 3rd latest snapshot or so, depending on your disk space requirement. -Endre Would it be considered an anti-pattern? I'm just evaluating and considering my options at the moment -- I don't know that space will be an issue. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Akka Team Typesafe - The software stack for applications that scale Blog: letitcrash.com Twitter: @akkateam -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
[akka-user] Akka Persistence journal size
Does a journal continue to grow in size forever, even if you're using snapshots? Presumably, once a snapshot is stored, the events preceding it are no longer needed. Are they kept? If so, is it possible to purge them? -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Akka Persistence journal size
Hi Dan! Event sourcing systems like to think in terms of never delete anything”. It’s basically auditing for free as well as the power to replay and analyse the entire history of your application. In such systems snapshots are only used ot make recovery faster - not to drop the proceeding data, which is why we don’t automatically delete preceding events when a snapshot succeeds. You can delete them by reacting to a snapshot succcess by issuing an deleteMessages(toSequenceNr = lastseqnr). Hope this helps! — Konrad On 30 October 2014 at 09:23:44, Dan Ellis (d...@danellis.me) wrote: Does a journal continue to grow in size forever, even if you're using snapshots? Presumably, once a snapshot is stored, the events preceding it are no longer needed. Are they kept? If so, is it possible to purge them? -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout. -- Konrad 'ktoso' Malawski hAkker @ typesafe http://akka.io -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
Re: [akka-user] Akka Persistence journal size
You can delete them by reacting to a snapshot succcess by issuing an deleteMessages(toSequenceNr = lastseqnr). If I only care about recovery and not auditing, are there any disadvantages to doing that? Would it be considered an anti-pattern? I'm just evaluating and considering my options at the moment -- I don't know that space will be an issue. -- Read the docs: http://akka.io/docs/ Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups Akka User List group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
