RE: 8193034: Optimize URL.toExternalForm

[Langer, Christoph]

Hi Martin,

I think the new code is more compact and readable which makes it nice. I 
reviewed it to check that the result is still the same as before. However, I 
can’t assess if it is acceptable from a performance point of view and would 
defer this assessment to some other reviewer. But apart from performance 
constraints the change seems good.

Best regards
Christoph

From: net-dev [mailto:net-dev-boun...@openjdk.java.net] On Behalf Of Martin 
Buchholz
Sent: Dienstag, 5. Dezember 2017 05:01
To: net-dev 
Subject: RFR: 8193034: Optimize URL.toExternalForm

http://cr.openjdk.java.net/~martin/webrevs/jdk/URL.toExternalForm/

RFR: 8193034: Optimize URL.toExternalForm

[Martin Buchholz]

http://cr.openjdk.java.net/~martin/webrevs/jdk/URL.toExternalForm/

Re: JEP 321: HTTP Client (Standard)

[David Lloyd]

On Mon, Dec 4, 2017 at 5:01 PM, Chris Hegarty  wrote:
> On 4 Dec 2017, at 22:03, David Lloyd  wrote:
>
> ...
>
> You mention general-purpose concepts such as ByteBufferReference and
> ByteBufferPool. Note that these are tiny implementation classes (150 lines
> in total) and not exposed in the API.
>
>
> Yes they are, currently - at least ByteBufferReference is at the heart of
> it:
>
> http://hg.openjdk.java.net/jdk/jdk/file/6dcbdc9f99fc/src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/AsyncConnection.java#l61
>
>
> I see my error, this is a non-public interface.  Nonetheless, I'm not
> sure that it's really safe to say that this is ready.  Has there been
> _any_ external feedback on this API?
>
>
> [ You can probably ignore my previous email, I sent it before receiving your
> reply. The confusion seems to have been resolved now. ]
>
> Yes, there has been external feedback on the API. I’ll dig it up, I need to
> trawl the net-dev archives and JIRA issues.
>
> For your, and others, reference, here is a snapshot of the latest API, built
> from the ‘http-client-branch’ of the sandbox:
>
>
> http://cr.openjdk.java.net/~chegar/httpclient/javadoc/api/jdk/incubator/http/package-summary.html

Thanks.  If you don't mind answering one more question: is there any
possibility to intercept authentication completely?  There does not
seem to be a lot of documentation about authentication in this API, or
what happens (for example) if there is no Authenticator.

-- 
- DML

Re: JEP 321: HTTP Client (Standard)

[Chris Hegarty]

> On 4 Dec 2017, at 22:03, David Lloyd  wrote:
>> ...
>> 
>>> You mention general-purpose concepts such as ByteBufferReference and
>>> ByteBufferPool. Note that these are tiny implementation classes (150 lines
>>> in total) and not exposed in the API.
>> 
>> Yes they are, currently - at least ByteBufferReference is at the heart of it:
>> 
>> http://hg.openjdk.java.net/jdk/jdk/file/6dcbdc9f99fc/src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/AsyncConnection.java#l61
> 
> I see my error, this is a non-public interface.  Nonetheless, I'm not
> sure that it's really safe to say that this is ready.  Has there been
> _any_ external feedback on this API?


[ You can probably ignore my previous email, I sent it before receiving your 
reply. The confusion seems to have been resolved now. ]

Yes, there has been external feedback on the API. I’ll dig it up, I need to 
trawl the net-dev archives and JIRA issues.

For your, and others, reference, here is a snapshot of the latest API, built 
from the ‘http-client-branch’ of the sandbox:

  
http://cr.openjdk.java.net/~chegar/httpclient/javadoc/api/jdk/incubator/http/package-summary.html
 


-Chris.

Re: JEP 321: HTTP Client (Standard)

[Chris Hegarty]

> On 4 Dec 2017, at 21:56, David Lloyd  wrote:
> ...
>> You mention general-purpose concepts such as ByteBufferReference and
>> ByteBufferPool. Note that these are tiny implementation classes (150 lines
>> in total) and not exposed in the API.
> 
> Yes they are, currently - at least ByteBufferReference is at the heart of it:
> 
> http://hg.openjdk.java.net/jdk/jdk/file/6dcbdc9f99fc/src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/AsyncConnection.java#l61
>  
> 
> 
> And that class relies directly on ByteBufferPool in its own API.
> 
> Unless the jdk/jdk branch does not reflect the latest incarnation of
> this JEP, in which case I definitely haven't have been up to date with
> it, despite following this list.


Sorry, before replying further, there is clearly some confusion here. As Alan 
already said, the above types are NOT part of the API, they are non-public 
implementation.

The ‘http-client-branch’ of the JDK sandbox [1] contains the very latest code.

-Chris.

[1] http://hg.openjdk.java.net/jdk/sandbox/branches

Re: JEP 321: HTTP Client (Standard)

[David Lloyd]

On Mon, Dec 4, 2017 at 3:56 PM, David Lloyd  wrote:
> On Mon, Dec 4, 2017 at 2:01 PM, Alan Bateman  wrote:
>> On 04/12/2017 18:41, David Lloyd wrote:
>>>
>>> :
>>> Speaking *solely* in the interests of platform quality and integrity,
>>> I think that before _any_ high-level non-blocking/asynchronous
>>> protocol API is ever introduced into the platform, it would be an
>>> incredible waste to not have some kind of design consultation with
>>> other industry experts.  Now I'm not suggesting that a JDK API would
>>> have to be _agreeable_ to every expert, as we all know that is
>>> basically impossible; but at the very minimum, I am very confident
>>> that we can tell you what _doesn't_ work and the pitfalls we've found
>>> along the way, as well as what each of us would consider to be an
>>> ideal API, and that is information that has incredible value.
>>>
>> The HTTP client API has been an ongoing effort for several years, the
>> original JEP goes back to 2014. It was initially available in the OpenJDK
>> sandbox and in the JDK main-line before moving to an incubating module in
>> 2016. I can't tell if you've been following this project or not but there
>> has been lots of opportunity to try it out and provide feedback on both the
>> API and implementation.
>
> I've had opportunity to give feedback, perhaps, though the API always
> seemed incomplete.  At least nobody (that I saw) sent out a message
> saying "Here it is, it's all done, what do you think?".  I've
> certainly never had opportunity to try it out: given its status as an
> incubating module present only in OpenJDK, the only people who are
> really in a position to try it out are those using OpenJDK (as opposed
> to other JDKs) with the flexibility to rewrite their use case if and
> when the API changes status (being integrated or disappearing) or form
> (evolving, presumably as a response to feedback), or people writing
> throwaway applications for the sole purpose of testing this particular
> API.  But those who are best able to make this kind of determination
> are those who need to be able to immediately use the API, and rely
> upon it indefinitely (for good or bad), which is definitely not the
> case for anything incubated in the OpenJDK project.  Why take the risk
> when you can use the Apache HTTP client instead?
>
> The lack of feedback on a proposed standard should not be considered a
> tacit endorsement of it - quite the opposite in fact.  It should be
> considered overwhelming disinterest (i.e. there's nothing particularly
> compelling about it), or at absolute best, insufficient information to
> make a determination.  The burden should be on the proposer to
> evangelize their idea and seek out feedback, rather than waiting for
> interest to appear and feedback to come to them.
>
>> You mention general-purpose concepts such as ByteBufferReference and
>> ByteBufferPool. Note that these are tiny implementation classes (150 lines
>> in total) and not exposed in the API.
>
> Yes they are, currently - at least ByteBufferReference is at the heart of it:
>
> http://hg.openjdk.java.net/jdk/jdk/file/6dcbdc9f99fc/src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/AsyncConnection.java#l61

I see my error, this is a non-public interface.  Nonetheless, I'm not
sure that it's really safe to say that this is ready.  Has there been
_any_ external feedback on this API?

-- 
- DML

Re: JEP 321: HTTP Client (Standard)

[David Lloyd]

On Mon, Dec 4, 2017 at 2:01 PM, Alan Bateman  wrote:
> On 04/12/2017 18:41, David Lloyd wrote:
>>
>> :
>> Speaking *solely* in the interests of platform quality and integrity,
>> I think that before _any_ high-level non-blocking/asynchronous
>> protocol API is ever introduced into the platform, it would be an
>> incredible waste to not have some kind of design consultation with
>> other industry experts.  Now I'm not suggesting that a JDK API would
>> have to be _agreeable_ to every expert, as we all know that is
>> basically impossible; but at the very minimum, I am very confident
>> that we can tell you what _doesn't_ work and the pitfalls we've found
>> along the way, as well as what each of us would consider to be an
>> ideal API, and that is information that has incredible value.
>>
> The HTTP client API has been an ongoing effort for several years, the
> original JEP goes back to 2014. It was initially available in the OpenJDK
> sandbox and in the JDK main-line before moving to an incubating module in
> 2016. I can't tell if you've been following this project or not but there
> has been lots of opportunity to try it out and provide feedback on both the
> API and implementation.

I've had opportunity to give feedback, perhaps, though the API always
seemed incomplete.  At least nobody (that I saw) sent out a message
saying "Here it is, it's all done, what do you think?".  I've
certainly never had opportunity to try it out: given its status as an
incubating module present only in OpenJDK, the only people who are
really in a position to try it out are those using OpenJDK (as opposed
to other JDKs) with the flexibility to rewrite their use case if and
when the API changes status (being integrated or disappearing) or form
(evolving, presumably as a response to feedback), or people writing
throwaway applications for the sole purpose of testing this particular
API.  But those who are best able to make this kind of determination
are those who need to be able to immediately use the API, and rely
upon it indefinitely (for good or bad), which is definitely not the
case for anything incubated in the OpenJDK project.  Why take the risk
when you can use the Apache HTTP client instead?

The lack of feedback on a proposed standard should not be considered a
tacit endorsement of it - quite the opposite in fact.  It should be
considered overwhelming disinterest (i.e. there's nothing particularly
compelling about it), or at absolute best, insufficient information to
make a determination.  The burden should be on the proposer to
evangelize their idea and seek out feedback, rather than waiting for
interest to appear and feedback to come to them.

> You mention general-purpose concepts such as ByteBufferReference and
> ByteBufferPool. Note that these are tiny implementation classes (150 lines
> in total) and not exposed in the API.

Yes they are, currently - at least ByteBufferReference is at the heart of it:

http://hg.openjdk.java.net/jdk/jdk/file/6dcbdc9f99fc/src/jdk.incubator.httpclient/share/classes/jdk/incubator/http/AsyncConnection.java#l61

And that class relies directly on ByteBufferPool in its own API.

Unless the jdk/jdk branch does not reflect the latest incarnation of
this JEP, in which case I definitely haven't have been up to date with
it, despite following this list.

> Promoting these to java.nio would be
> outside the scope of this API. If the buffer API were to add such concepts
> in the future then there is nothing to stop the HTTP client implementation
> making use in its implementation.

See above.


-- 
- DML

Re: JEP 321: HTTP Client (Standard)

[Alan Bateman]

On 04/12/2017 18:41, David Lloyd wrote:

:
Speaking *solely* in the interests of platform quality and integrity,
I think that before _any_ high-level non-blocking/asynchronous
protocol API is ever introduced into the platform, it would be an
incredible waste to not have some kind of design consultation with
other industry experts.  Now I'm not suggesting that a JDK API would
have to be _agreeable_ to every expert, as we all know that is
basically impossible; but at the very minimum, I am very confident
that we can tell you what _doesn't_ work and the pitfalls we've found
along the way, as well as what each of us would consider to be an
ideal API, and that is information that has incredible value.

The HTTP client API has been an ongoing effort for several years, the 
original JEP goes back to 2014. It was initially available in the 
OpenJDK sandbox and in the JDK main-line before moving to an incubating 
module in 2016. I can't tell if you've been following this project or 
not but there has been lots of opportunity to try it out and provide 
feedback on both the API and implementation.


You mention general-purpose concepts such as ByteBufferReference and 
ByteBufferPool. Note that these are tiny implementation classes (150 
lines in total) and not exposed in the API. Promoting these to java.nio 
would be outside the scope of this API. If the buffer API were to add 
such concepts in the future then there is nothing to stop the HTTP 
client implementation making use in its implementation.


-Alan

Re: JEP 321: HTTP Client (Standard)

[Norman Maurer]

Well put David,

I couldn’t agree more  I would even go this far and say this is not 
something I would include in the platform itself at all.

Bye
Norman


> Am 04.12.2017 um 19:41 schrieb David Lloyd :
> 
>> On Mon, Dec 4, 2017 at 10:17 AM,   wrote:
>> New JEP Candidate: http://openjdk.java.net/jeps/321
> 
> I have concerns.
> 
> This will be the first public, NIO-based, asynchronous/non-blocking
> network protocol API introduced into the JDK proper, _ever_.
> 
> First, I want to note that the API seems to bear no resemblance
> whatsoever with the asynchronous NIO.2 API.  Now I'm no fan of that
> API as it stands for a couple of reasons, but nevertheless it calls
> into question either the validity of the HTTP API as it stands, or the
> scope of reusability of the NIO.2 API.
> 
> With that items put aside: there are a wide variety of mature,
> non-blocking network protocol implementations out there, with
> literally thousands of years of experience distributed amongst their
> authors, maintainers, supporters, and communities, none of which were
> used as a model.  There was, as far as I can see, no kind of study of
> existing non-blocking approaches in Java, and their strengths and
> weaknesses; there was no round table of engineers with experience in
> this field, talking about what works and what doesn't work.
> 
> Some new, seemingly general-purpose, concepts are introduced by the
> code base, for example: ByteBufferReference, and ByteBufferPool.  Are
> these strategies that should be promoted to NIO proper?  If not, then
> are they _really_ right for this particular use case, particularly if,
> for some reason, a _second_ non-blocking network protocol API might be
> introduced some day, probably duplicating these concepts?
> 
> Making this thing be the first real platform NIO-based asynchronous
> network protocol API, yet being completely removed from the previous
> NIO.2 asynchronous APIs and any other existing stable, mature API,
> should be done very carefully and deliberately, and perhaps most
> importantly, incrementally: first, establish a non-blocking byte
> stream API that makes sense generally, and bring that into NIO
> (NIO.3?); then, perhaps, enhancements to byte buffers to better
> support efficient pooling.  By the time that point is reached, it is
> hopefully rapidly becoming obvious that this is not something that
> should be taken lightly.
> 
> I believe that most third-party implementations are taken less lightly
> than this seems to have been.  I and my team have been developing an
> asynchronous/non-blocking NIO library for ten years now, and while I'm
> proud of our work and the discoveries we've made, I am realistic about
> the fact that it's still pretty far from as good as it could be (not
> in the least part due to existing platform limitations), certainly far
> from something I'd say "hey let's standardize this as is".  I think
> that to standardize something of this type that was just written over
> the past 18-odd months reflects, to put it kindly, some pretty
> incredible confidence that I wish I shared.
> 
> Speaking *solely* in the interests of platform quality and integrity,
> I think that before _any_ high-level non-blocking/asynchronous
> protocol API is ever introduced into the platform, it would be an
> incredible waste to not have some kind of design consultation with
> other industry experts.  Now I'm not suggesting that a JDK API would
> have to be _agreeable_ to every expert, as we all know that is
> basically impossible; but at the very minimum, I am very confident
> that we can tell you what _doesn't_ work and the pitfalls we've found
> along the way, as well as what each of us would consider to be an
> ideal API, and that is information that has incredible value.
> 
> Talking about introducing the first-ever non-blocking protocol API
> into the platform, at _this_ stage, seems premature and needlessly
> risky.  I would suggest that maybe it's best for the API to stick to
> blocking I/O, at least for now.  Or else, take it outside of the
> platform, and let it mature in the wild where it can evolve without an
> overbearing concern for compatibility for a decade or so (no, I'm not
> kidding).  As long as this thing lives in the JDK, but isn't
> standardized, it's probably not going to be used heavily enough to
> really feel out its weak points.  And once it's in, its ability to
> evolve is severely hampered by compatibility constraints.
> 
> I feel like it is impossible to over-emphasize the difficulty of the
> problem of non-blocking I/O when it comes to interactions with user
> programs.  Though the fruits of such an effort are probably small in
> terms of API surface, the complexity is hard: hard enough that it is,
> in my mind, a project of a larger scale, maybe JSR scale.  And the
> benefit is potentially large: large enough that it could change the
> landscape of other specifications and network applications.  Or at
> least, I think so, which is why I

Re: JEP 321: HTTP Client (Standard)

[David Lloyd]

On Mon, Dec 4, 2017 at 10:17 AM,   wrote:
> New JEP Candidate: http://openjdk.java.net/jeps/321

I have concerns.

This will be the first public, NIO-based, asynchronous/non-blocking
network protocol API introduced into the JDK proper, _ever_.

First, I want to note that the API seems to bear no resemblance
whatsoever with the asynchronous NIO.2 API.  Now I'm no fan of that
API as it stands for a couple of reasons, but nevertheless it calls
into question either the validity of the HTTP API as it stands, or the
scope of reusability of the NIO.2 API.

With that items put aside: there are a wide variety of mature,
non-blocking network protocol implementations out there, with
literally thousands of years of experience distributed amongst their
authors, maintainers, supporters, and communities, none of which were
used as a model.  There was, as far as I can see, no kind of study of
existing non-blocking approaches in Java, and their strengths and
weaknesses; there was no round table of engineers with experience in
this field, talking about what works and what doesn't work.

Some new, seemingly general-purpose, concepts are introduced by the
code base, for example: ByteBufferReference, and ByteBufferPool.  Are
these strategies that should be promoted to NIO proper?  If not, then
are they _really_ right for this particular use case, particularly if,
for some reason, a _second_ non-blocking network protocol API might be
introduced some day, probably duplicating these concepts?

Making this thing be the first real platform NIO-based asynchronous
network protocol API, yet being completely removed from the previous
NIO.2 asynchronous APIs and any other existing stable, mature API,
should be done very carefully and deliberately, and perhaps most
importantly, incrementally: first, establish a non-blocking byte
stream API that makes sense generally, and bring that into NIO
(NIO.3?); then, perhaps, enhancements to byte buffers to better
support efficient pooling.  By the time that point is reached, it is
hopefully rapidly becoming obvious that this is not something that
should be taken lightly.

I believe that most third-party implementations are taken less lightly
than this seems to have been.  I and my team have been developing an
asynchronous/non-blocking NIO library for ten years now, and while I'm
proud of our work and the discoveries we've made, I am realistic about
the fact that it's still pretty far from as good as it could be (not
in the least part due to existing platform limitations), certainly far
from something I'd say "hey let's standardize this as is".  I think
that to standardize something of this type that was just written over
the past 18-odd months reflects, to put it kindly, some pretty
incredible confidence that I wish I shared.

Speaking *solely* in the interests of platform quality and integrity,
I think that before _any_ high-level non-blocking/asynchronous
protocol API is ever introduced into the platform, it would be an
incredible waste to not have some kind of design consultation with
other industry experts.  Now I'm not suggesting that a JDK API would
have to be _agreeable_ to every expert, as we all know that is
basically impossible; but at the very minimum, I am very confident
that we can tell you what _doesn't_ work and the pitfalls we've found
along the way, as well as what each of us would consider to be an
ideal API, and that is information that has incredible value.

Talking about introducing the first-ever non-blocking protocol API
into the platform, at _this_ stage, seems premature and needlessly
risky.  I would suggest that maybe it's best for the API to stick to
blocking I/O, at least for now.  Or else, take it outside of the
platform, and let it mature in the wild where it can evolve without an
overbearing concern for compatibility for a decade or so (no, I'm not
kidding).  As long as this thing lives in the JDK, but isn't
standardized, it's probably not going to be used heavily enough to
really feel out its weak points.  And once it's in, its ability to
evolve is severely hampered by compatibility constraints.

I feel like it is impossible to over-emphasize the difficulty of the
problem of non-blocking I/O when it comes to interactions with user
programs.  Though the fruits of such an effort are probably small in
terms of API surface, the complexity is hard: hard enough that it is,
in my mind, a project of a larger scale, maybe JSR scale.  And the
benefit is potentially large: large enough that it could change the
landscape of other specifications and network applications.  Or at
least, I think so, which is why I've spent so many years of my life in
pursuit of such a thing.

-- 
- DML

JEP 321: HTTP Client (Standard)

[mark . reinhold]

New JEP Candidate: http://openjdk.java.net/jeps/321

- Mark