Re: Request/discussion: BufferedReader reading using async API while providing sync API

Hey! Is this forgotten? On 03/11/2016 14:35, Brunoais wrote: Any information you can give on this? On 29/10/2016 19:10, Brunoais wrote: Here's my try on going async. On my tests on my local windows 10 machine (I don't have access to the linux one without a VM ATM) , now with 1GB file, I no

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Any information you can give on this? On 29/10/2016 19:10, Brunoais wrote: Here's my try on going async. On my tests on my local windows 10 machine (I don't have access to the linux one without a VM ATM) , now with 1GB file, I noticed: ~2s of speed improvement from BufferedInputStream to B

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Here's my try on going async. On my tests on my local windows 10 machine (I don't have access to the linux one without a VM ATM) , now with 1GB file, I noticed: ~2s of speed improvement from BufferedInputStream to BufferedNonBlockStream when BufferedNonBlockStream is at its most advantageous

Re: Request/discussion: BufferedReader reading using async API while providing sync API

I'll try going back to a previous version I worked on which used the java7's AsynchronousFileChannel and work from there. My small research shows it can also work with AsynchronousFileChannel mostly without changes. For now, 1 question: Is Thread.sleep() a possible way of dealing the block requ

Re: Request/discussion: BufferedReader reading using async API while providing sync API

On 27/10/2016 22:45, Vitaly Davidovich wrote: On Thursday, October 27, 2016, Brunoais > wrote: You are right. Even in windows it does not set the flags for async reads. It seems like it is windows itself that does the decision to buffer the contents b

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Sorry nio-dev ... On 28/10/2016 7:09 AM, David Holmes wrote: You might try discussing on net-dev rather than core-libs-dev, to get additional historical info related to the io and nio file APIs. David On 28/10/2016 5:08 AM, Brunoais wrote: You are right. Even in windows it does not set the fl

Re: Request/discussion: BufferedReader reading using async API while providing sync API

On Thursday, October 27, 2016, Brunoais wrote: > You are right. Even in windows it does not set the flags for async reads. > It seems like it is windows itself that does the decision to buffer the > contents based on its own heuristics. > You mean nonblocking, not async, right? Two different thin

Re: Request/discussion: BufferedReader reading using async API while providing sync API

You might try discussing on net-dev rather than core-libs-dev, to get additional historical info related to the io and nio file APIs. David On 28/10/2016 5:08 AM, Brunoais wrote: You are right. Even in windows it does not set the flags for async reads. It seems like it is windows itself that d

Re: Request/discussion: BufferedReader reading using async API while providing sync API

You are right. Even in windows it does not set the flags for async reads. It seems like it is windows itself that does the decision to buffer the contents based on its own heuristics. But... Why? Why won't it be? Why is there no API for it? How am I getting 100% HDD use and faster times when I

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Here's my current code for benchmarking. Does this seem correct to you? On 27/10/2016 14:55, Peter Levart wrote: On 10/27/2016 03:48 PM, Peter Levart wrote: On 10/27/2016 02:20 PM, Brunoais wrote: Hey. Any idea how to skip tests? When testing for BufferedInputStream, the directBufferSiz

Re: Request/discussion: BufferedReader reading using async API while providing sync API

On 10/27/2016 03:48 PM, Peter Levart wrote: On 10/27/2016 02:20 PM, Brunoais wrote: Hey. Any idea how to skip tests? When testing for BufferedInputStream, the directBufferSize is not used so testing with different directBufferSize makes no sense. I already tried "return 0" on the benchm

Re: Request/discussion: BufferedReader reading using async API while providing sync API

On 10/27/2016 02:20 PM, Brunoais wrote: Hey. Any idea how to skip tests? When testing for BufferedInputStream, the directBufferSize is not used so testing with different directBufferSize makes no sense. I already tried "return 0" on the benchmarked test but jmh fills the output with " (*i

Re: Request/discussion: BufferedReader reading using async API while providing sync API

I don't know about Windows specifically, but generally file systems across major OS's will implement readahead in their IO scheduler when they detect sequential scans. On Linux, you can also strace your test to confirm which syscalls are emitted (you should be seeing plain read()'s there, with Fil

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Thanks for the heads up. I'll try that later. These tests are still useful then. Meanwhile, I'll end up also checking how FileChannel queries the OS on windows. I'm getting 100% HDD reads... Could it be that the OS reads the file ahead on its own?... Anyway, I'll look into it. Thanks for the h

Re: Request/discussion: BufferedReader reading using async API while providing sync API

On Thu, Oct 27, 2016 at 8:34 AM, Brunoais wrote: > Oh... I see. In that case, it means something is terribly wrong. It can be > my initial tests, though. > > I'm testing on both linux and windows and I'm getting performance gains > from using the FileChannel compared to using FileInputStream... T

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Oh... I see. In that case, it means something is terribly wrong. It can be my initial tests, though. I'm testing on both linux and windows and I'm getting performance gains from using the FileChannel compared to using FileInputStream... The tests also make sense based on my predictions O_O...

Re: Request/discussion: BufferedReader reading using async API while providing sync API

illing-wraps-around case and not issue two requests, that might remove some additional latency. Gruss Bernd -- http://bernd.eckenfels.net _________ From: Brunoais mailto:brunoa...@gmail.com>> Sent: Montag, Oktober 24, 2016 6:30 PM Subject: Re: Request/discussion: BufferedRe

Re: Request/discussion: BufferedReader reading using async API while providing sync API

On Thursday, October 27, 2016, Brunoais wrote: > Did you read the C code? I looked at the Linux code in the JDK. > Have you got any idea how many functions Windows or Linux (nearly all > flavors) have for the read operation towards a file? I do. > > I have already done that homework myself. I

Re: Request/discussion: BufferedReader reading using async API while providing sync API

On 27/10/2016 10:28, Bernd Eckenfels wrote: : AsyncFileChannel uses Java threads in the background for the completion handling (and last time I checked they are not signaled by the OS). Performance with that was not very good for sequential processing in my tests. It does use completion ports

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Hello, the NIO2 code for files is pretty disappointing as it does not use file completion ports on Windows or AIO on Linux. You use the file channel, in that case it is blocking. It does not implement SelectableChannel. AsyncFileChannel uses Java threads in the background for the completion hand

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Did you read the C code? Have you got any idea how many functions Windows or Linux (nearly all flavors) have for the read operation towards a file? I have already done that homework myself. I may not have read JVM's source code but I know well that there's functions on both Windows and Linux

Re: Request/discussion: BufferedReader reading using async API while providing sync API

On Wednesday, October 26, 2016, Brunoais wrote: > It is actually based on the premise that: > > 1. The first call to ReadableByteChannel.read(ByteBuffer) sets the OS >buffer size to fill in as the same size as ByteBuffer. Why do you say that? AFAICT, it issues a read syscall and that will bl

Re: Request/discussion: BufferedReader reading using async API while providing sync API

to only read till the end of the buffer in the backfilling-wraps-around case and not issue two requests, that might remove some additional latency. Gruss Bernd -- http://bernd.eckenfels.net _________ From: Brunoais mailto:brunoa...@gmail.com>> Sent: Montag, Ok

Re: Request/discussion: BufferedReader reading using async API while providing sync API

case and not issue two requests, that might remove some additional latency. Gruss Bernd -- http://bernd.eckenfels.net _____________ From: Brunoais mailto:brunoa...@gmail.com>> Sent: Montag, Oktober 24, 2016 6:30 PM Subject: Re: Request/discussion: BufferedReader reading using async API while providing

Re: Request/discussion: BufferedReader reading using async API while providing sync API

. I thought back in the time this was mostly die >>>> to the fact that it NOT wraps to real AsyncFIO on most platforms. >>>> But maybe I just measured it wrong, so I will have a closer look on >>>> your impl. >>>> >>>> Generally I would recom

Re: Request/discussion: BufferedReader reading using async API while providing sync API

he buffer in the backfilling-wraps-around case and not issue two requests, that might remove some additional latency. Gruss Bernd -- http://bernd.eckenfels.net _____ From: Brunoais mailto:brunoa...@gmail.com>> Sent: Montag, Oktober 24, 2016 6:30 PM Subject: Re: R

Re: Request/discussion: BufferedReader reading using async API while providing sync API

buffer in the backfilling-wraps-around case and not issue two requests, that might remove some additional latency. Gruss Bernd -- http://bernd.eckenfels.net _ From: Brunoais mailto:brunoa...@gmail.com>> Sent: Montag, Oktober 24, 2016 6:30 PM Subject: Re: Request/d

Re: Request/discussion: BufferedReader reading using async API while providing sync API

It is actually based on the premise that: 1. The first call to ReadableByteChannel.read(ByteBuffer) sets the OS buffer size to fill in as the same size as ByteBuffer. 2. The consecutive calls to ReadableByteChannel.read(ByteBuffer) orders the JVM to order the OS to execute memcpy() to copy

Re: Request/discussion: BufferedReader reading using async API while providing sync API

I believe I see where you coming from. Please correct me if I'm wrong. Your implementation is based on the premise that a call to ReadableByteChannel.read() _initiates_ the operation and returns immediately. The OS then continues to fill the buffer while there's a free space in the buffer and th

Re: Request/discussion: BufferedReader reading using async API while providing sync API

ernd -- http://bernd.eckenfels.net _ From: Brunoais mailto:brunoa...@gmail.com>> Sent: Montag, Oktober 24, 2016 6:30 PM Subject: Re: Request/discussion: BufferedReader reading using async API while providing sync API To: Pavel Rappo mailto:pavel.ra...@oracle.com&

Re: Request/discussion: BufferedReader reading using async API while providing sync API

30 PM Subject: Re: Request/discussion: BufferedReader reading using async API while providing sync API To: Pavel Rappo mailto:pavel.ra...@oracle.com>> Cc: <mailto:core-libs-dev@openjdk.java.net>> Attached and sending! On 24/10/2016 13:48, Pavel Rappo wrote: > Could you p

Re: Request/discussion: BufferedReader reading using async API while providing sync API

/discussion: BufferedReader reading using async API while providing sync API To: Pavel Rappo Cc: Attached and sending! On 24/10/2016 13:48, Pavel Rappo wrote: > Could you please send a new email on this list with the source attached as a > text file? > >> On 23 Oct 2016, at 19:14,

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Thank you for your time. I'll try to explain it. I hope I can clear it up. First of it, I made a meaning mistake between asynchronous and non-blocking. This implementation uses a non-blocking algorithm internally while providing a blocking-like algorithm on the surface. It is single-threaded a

Re: Request/discussion: BufferedReader reading using async API while providing sync API

I've skimmed through the code and I'm not sure I can see any asynchronicity (you were pointing at the lack of it in BufferedReader). And the mechanics of this is very puzzling to me, to be honest: void blockingFill(boolean forced) throws IOException { fill(forced); while (read

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Attached and sending! On 24/10/2016 13:48, Pavel Rappo wrote: Could you please send a new email on this list with the source attached as a text file? On 23 Oct 2016, at 19:14, Brunoais wrote: Here's my poc/prototype: http://pastebin.com/WRpYWDJF I've implemented the bare minimum of the cla

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Could you please send a new email on this list with the source attached as a text file? > On 23 Oct 2016, at 19:14, Brunoais wrote: > > Here's my poc/prototype: > http://pastebin.com/WRpYWDJF > > I've implemented the bare minimum of the class that follows the same contract > of BufferedReader

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Here's my poc/prototype: http://pastebin.com/WRpYWDJF I've implemented the bare minimum of the class that follows the same contract of BufferedReader while signaling all issues I think it may have or has in comments. I also wrote some javadoc to help guiding through the class. I could have us

Re: Request/discussion: BufferedReader reading using async API while providing sync API

OK. Will do. On 21/10/2016 19:21, Pavel Rappo wrote: Just to append to my previous email. BufferedReader wraps any Reader out there. Not specifically FileReader. While you're talking about the case of effective reading from a file. I guess there's one existing possibility to provide exactly wh

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Just to append to my previous email. BufferedReader wraps any Reader out there. Not specifically FileReader. While you're talking about the case of effective reading from a file. I guess there's one existing possibility to provide exactly what you need (as I understand it) under this method: /**

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Pavel is right. In reality, I was expecting such BufferedReader to use only a single buffer and have that Buffer being filled asynchronously, not in a different Thread. Additionally, I don't have the intention of having a larger buffer than before unless stated through the API (the constructor

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Hi Pavel, I think Brunoais asking for a double buffering scheme in which the implementation of BufferReader fills (a second buffer) in parallel with the application reading from the 1st buffer and managing the swaps and async reads transparently. It would not change the API but would change th

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Off the top of my head, I would say it's not possible to change the design of an _extensible_ type that has been out there for 20 or so years. All these I/O streams from java.io were designed for simple synchronous use case. It's not that their design is flawed in some way, it's that they doesn't

Re: Request/discussion: BufferedReader reading using async API while providing sync API

Any feedback on this? I'm really interested in implementing such BufferedReader/BufferedStreamReader to allow speeding up my applications without having to think in an asynchronous way or multi-threading while programming with it. That's why I'm asking this here. On 13/10/2016 14:45, Brunoai

Request/discussion: BufferedReader reading using async API while providing sync API

Hi, I'm new here, so if this is belongs to a different mailing list please point me in the right direction. I looked at BufferedReader source code for java 9 long with the source code of the channels/streams used. I noticed that, like in java 7, BufferedReader does not use an Async API to load d

Request/discussion: BufferedReader reading using async API while providing sync API

Hi, I looked at BufferedReader source code for java 9 long with the source code of the channels/streams used. I noticed that, like in java 7, BufferedReader does not use an Async API to load data from files, instead, the data loading is all done synchronously even when the OS allows requestin