Re: Trying to understand soft vs hard commit vs transaction log

2013-02-08 Thread Mark Miller
A soft commit is just like a hard commit but doesn't do things like resolve deletes or call fsync on all the files that were written to disk. It will flush to disk however. Hard commits are for durability if you are not using the update log. If you are using the update log, hard commits are abo

Re: Trying to understand soft vs hard commit vs transaction log

2013-02-08 Thread Jack Krupansky
grain of salt. -- Jack Krupansky -Original Message- From: Alexandre Rafalovitch Sent: Friday, February 08, 2013 6:11 PM To: solr-user@lucene.apache.org Subject: Re: Trying to understand soft vs hard commit vs transaction log Sorry Shawn, Somehow I am still not quite grasping it. I w

Re: Trying to understand soft vs hard commit vs transaction log

2013-02-08 Thread Shawn Heisey
On 2/8/2013 4:11 PM, Alexandre Rafalovitch wrote: Sorry Shawn, Somehow I am still not quite grasping it. I would really appreciate if somebody (or even you) could have another go at very small part of this. Maybe it will clear it up: Similarly, in performance section of Wiki, it says: "A commi

Re: Trying to understand soft vs hard commit vs transaction log

2013-02-08 Thread Alexandre Rafalovitch
Sorry Shawn, Somehow I am still not quite grasping it. I would really appreciate if somebody (or even you) could have another go at very small part of this. Maybe it will clear it up: > Similarly, in performance section of Wiki, it says: "A commit (including a soft commit) will free up almost all

Re: Trying to understand soft vs hard commit vs transaction log

2013-02-08 Thread Shawn Heisey
On 2/8/2013 3:12 AM, Isaac Hebsh wrote: Shawn, what about 'flush to disk' behaviour on MMapDirectoryFactory? MMapDirectoryFactory should flush everything to disk on a hard commit and not keep anything in RAM. I *think* that soft commits still end up in RAM with this implementation, but you'l

Re: Trying to understand soft vs hard commit vs transaction log

2013-02-08 Thread Isaac Hebsh
Shawn, what about 'flush to disk' behaviour on MMapDirectoryFactory? On Fri, Feb 8, 2013 at 11:12 AM, Prakhar Birla wrote: > Great explanation Shawn! BTW soft commited documents will be not be > recovered on JVM crash. > > On 8 February 2013 13:27, Shawn Heisey wrote: > > > On 2/7/2013 9:29 PM,

Re: Trying to understand soft vs hard commit vs transaction log

2013-02-08 Thread Prakhar Birla
Great explanation Shawn! BTW soft commited documents will be not be recovered on JVM crash. On 8 February 2013 13:27, Shawn Heisey wrote: > On 2/7/2013 9:29 PM, Alexandre Rafalovitch wrote: > >> Hello, >> >> What actually happens when using soft (as opposed to hard) commit? >> >> I understand so

Re: Trying to understand soft vs hard commit vs transaction log

2013-02-07 Thread Shawn Heisey
On 2/7/2013 9:29 PM, Alexandre Rafalovitch wrote: Hello, What actually happens when using soft (as opposed to hard) commit? I understand somewhat very high-level picture (documents become available faster, but you may loose them on power loss). I don't care about low-level implementation detail