Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-21 Thread Brian Burkhalter
Hi Uwe, You’re welcome! Brian On May 21, 2015, at 4:45 PM, Uwe Schindler wrote: > Hi Brian, > > the patch looks fine to me! Many thanks. > > Uwe > > - > Uwe Schindler > uschind...@apache.org > ASF Member, Apache Lucene PMC / Committer > Bremen, Germany > http://lucene.apache.org/

RE: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-21 Thread Uwe Schindler
PM To: nio-dev Cc: dev@lucene.apache.org; rory.odonn...@oracle.com; Balchandra Vaidya; Robert Muir Subject: Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory Status update … On May 18, 2015, at 12:04 PM, Brian Burkhalter wrote: 2. Work on a fix for <ht

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-21 Thread Dawid Weiss
On this note it'd be great if you guys could at least revisit the subject of some kind of open feedback from the bug tracker -- just a way to subscribe to updates ("watch") on a particular issue would be awesome. Dawid On Thu, May 21, 2015 at 9:55 PM, Brian Burkhalter wrote: > > On May 21, 2015,

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-21 Thread Brian Burkhalter
On May 21, 2015, at 12:54 PM, Robert Muir wrote: >> Please note that this patch has been pushed: >> >> http://hg.openjdk.java.net/jdk9/dev/jdk/rev/e08b856baa26 >> > > I just tested this with lucene's test suite without any issues. Fantastic! Thanks for the testing corroboration: that is very

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-21 Thread Robert Muir
On Thu, May 21, 2015 at 1:31 PM, Brian Burkhalter wrote: > > 2. Work on a fix for https://bugs.openjdk.java.net/browse/JDK-8080589, (fc) > FileChannel.force should use fcntl(F_FULLFSYNC) instead of fsync on OS X. > > > Please note that this patch has been pushed: > > http://hg.openjdk.java.net/jdk

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-21 Thread Brian Burkhalter
Status update … On May 18, 2015, at 12:04 PM, Brian Burkhalter wrote: > 2. Work on a fix for https://bugs.openjdk.java.net/browse/JDK-8080589, (fc) > FileChannel.force should use fcntl(F_FULLFSYNC) instead of fsync on OS X. Please note that this patch has been pushed: http://hg.openjdk.java.

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-19 Thread Brian Burkhalter
Status update ,,, On May 18, 2015, at 12:04 PM, Brian Burkhalter wrote: > We have revised our approach to this (pair or trio of) problem(s). The > following sequence of actions is proposed. > > 1. Revert the patch which fixed > https://bugs.openjdk.java.net/browse/JDK-8066915, (fs) Files.new

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-18 Thread Brian Burkhalter
On May 18, 2015, at 6:34 PM, Robert Muir wrote: >> Hopefully the foregoing plan of record shall be to everyone’s satisfaction >> and will converge to an eventual mutually acceptable, logical solution. >> > > Thank you very much for your consideration here! We aim to please when possible!

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-18 Thread Robert Muir
On Mon, May 18, 2015 at 3:04 PM, Brian Burkhalter wrote: > We have revised our approach to this (pair or trio of) problem(s). The > following sequence of actions is proposed. > > 1. Revert the patch which fixed > https://bugs.openjdk.java.net/browse/JDK-8066915, (fs) Files.newByteChannel > opens d

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-18 Thread Brian Burkhalter
Hi Uwe, I am happy to hear that you think this plan reasonable. On May 18, 2015, at 12:43 PM, Uwe Schindler wrote: > the plan looks fine from my perspective! Many thanks! > > I have a few comments on the items: > 1. We were using FileChannel.open(Path,…) to open directories, we never >

RE: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-18 Thread Uwe Schindler
v-boun...@openjdk.java.net] On Behalf Of Brian Burkhalter Sent: Monday, May 18, 2015 9:05 PM To: nio-dev Cc: rory.odonn...@oracle.com; dev@lucene.apache.org; Balchandra Vaidya Subject: Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory We have revised our approach to this (pair or trio of) pro

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-18 Thread Brian Burkhalter
We have revised our approach to this (pair or trio of) problem(s). The following sequence of actions is proposed. 1. Revert the patch which fixed https://bugs.openjdk.java.net/browse/JDK-8066915, (fs) Files.newByteChannel opens directories for cases where subsequent reads may fail, which instig

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-16 Thread Alan Bateman
On 16/05/2015 18:26, Robert Muir wrote: : Code that is responsible for synchronizing data to disk is really important, its the last place you want this kind of complexity :( Yes, we know this. Let's try to come up with a solution for this in the coming weeks. -Alan. -

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-16 Thread Robert Muir
On Sat, May 16, 2015 at 1:30 PM, Francis Galiegue wrote: > But at the core of the matter I disagree with your basic premise. To > me, and to 100-epsilon% of users for that matter, it is perfectly > logical that opening a stream to a directory should fail before you > even attempt to read(). I'm

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-16 Thread Francis Galiegue
Hello, On Sat, May 16, 2015 at 6:48 PM, Robert Muir wrote: [...] > > I don't understand the tradeoff being made here. The JDK will deliver > ... a little bit earlier exception to morons trying to read bytes from > a directory, at the cost of damaging impact to user's data for > everyone else? Wha

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-16 Thread Francis Galiegue
On Sat, May 16, 2015 at 7:30 PM, Francis Galiegue wrote: [...] > > As to a new API, well, why not StandardOpenOption.DIRECTORY? > O_DIRECTORY has existed for open(2) since POSIX.1-2008 (says open(2)), > and it is even specified that if the path is not a directory the > failure is ENOTDIR. Excellen

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-16 Thread Robert Muir
On Sat, May 16, 2015 at 12:58 PM, Alan Bateman wrote: >> > JDK 9 is not planned to ship until 2016 so if there is a new API then I > would hope this is time enough for Lucene to add support. For now then I > think we need to come up with the right API for this. We can see later if we > want to get

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-16 Thread Alan Bateman
On 16/05/2015 17:48, Robert Muir wrote: : Separately, the addition of an explicit api for this is a fantastic idea, but its no solution for users on the existing api. JDK 9 is not planned to ship until 2016 so if there is a new API then I would hope this is time enough for Lucene to add supp

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-16 Thread Robert Muir
Is it really not possible to revert the change that broke all this? I urge you to reconsider the impacts of this. At the very least, we can't change already-released versions of lucene code: we can't add retroactively add "two screens of fsync code" to them to do the right thing. That code is alre

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-15 Thread Brian Burkhalter
On May 13, 2015, at 7:51 AM, Brian Burkhalter wrote: >> I agree with Alan that adding an OpenOption would be a good possibility. In >> any case, as Files only contains static methods, we could still add a >> “utility” method that forces file/directory buffers to disk, that just uses >> the n

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-13 Thread Brian Burkhalter
Hi Uwe, On May 13, 2015, at 2:27 AM, Uwe Schindler wrote: > many thanks for opening this issue! You’re welcome! > I agree with Alan that adding an OpenOption would be a good possibility. In > any case, as Files only contains static methods, we could still add a > “utility” method that forces

RE: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-13 Thread Uwe Schindler
Burkhalter Sent: Wednesday, May 13, 2015 12:26 AM To: nio-dev Cc: rory.odonn...@oracle.com; dev@lucene.apache.org; Balchandra Vaidya Subject: Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory I have created an enhancement issue here: https://bugs.openjdk.java.net/browse/JDK-8080235

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-12 Thread Brian Burkhalter
I have created an enhancement issue here: https://bugs.openjdk.java.net/browse/JDK-8080235 Brian On May 12, 2015, at 3:10 PM, Brian Burkhalter wrote: > I will create an issue now and post the ID.

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-12 Thread Brian Burkhalter
On May 12, 2015, at 2:11 PM, Alan Bateman wrote: > I've been too busy with other JDK 9 work to spend on time on this issue > recently. I do think we should introduce some support for this use-case as > it's clearly important. > > The issue with adding a method to Files is that it requires rev

Re: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-12 Thread Alan Bateman
On 12/05/2015 19:59, Uwe Schindler wrote: Hallo Alan, I just wanted to come back to this issue, because there was no further communication recently regarding the behavior of Java 9 with opening a FileChannel on a directory to fsync the directory metadata. Unfortunately, this would break the im

RE: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory

2015-05-12 Thread Uwe Schindler
gt; Uwe Schindler > Sent: Friday, January 09, 2015 7:56 PM > To: 'Alan Bateman'; nio-...@openjdk.java.net > Cc: rory.odonn...@oracle.com; 'Balchandra Vaidya' > Subject: RE: Recent Java 9 commit (e5b66323ae45) breaks fsync on directory > > Hi Alan, > > Th