[jira] [Commented] (FLINK-8000) Sort REST handler URLs in RestServerEndpoint

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245441#comment-16245441
 ] 

ASF GitHub Bot commented on FLINK-8000:
---

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/4958


> Sort REST handler URLs in RestServerEndpoint
> 
>
> Key: FLINK-8000
> URL: https://issues.apache.org/jira/browse/FLINK-8000
> Project: Flink
>  Issue Type: Improvement
>  Components: REST
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
>  Labels: flip-6
> Fix For: 1.5.0
>
>
> In order to make the {{RestServerEndpoint}} more easily extendable, we should 
> automatically sort the returned list of rest handler when calling 
> {{RestServerEndpoint#initializeHandlers}}. That way the order in which the 
> handlers are added to the list is independent of the actual registration 
> order. This is, for example, important for the static file server which 
> always needs to be registered last.
> I propose to add a special {{String}} {{Comparator}} which considers the 
> charactor {{':'}} to be the character with the largest value. That way we 
> should get always the following sort order:
> - URLs without path parameters have precedence over similar URLs where parts 
> are replaced by path parameters (e.g. {{/jobs/overview}}, {{/jobs/:jobid}} 
> and {{/jobs/:jobid/config}}, {{/jobs/:jobid/vertices/:vertexId}})
> - Prefixes are sorted before URLs containing the prefix (e.g. {{/jobs}}, 
> {{/jobs/overview}})



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-8000) Sort REST handler URLs in RestServerEndpoint

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245425#comment-16245425
 ] 

ASF GitHub Bot commented on FLINK-8000:
---

Github user tillrohrmann commented on the issue:

https://github.com/apache/flink/pull/4958
  
Will remove the comment and then merge the PR. Thanks for your review 
@zentol.


> Sort REST handler URLs in RestServerEndpoint
> 
>
> Key: FLINK-8000
> URL: https://issues.apache.org/jira/browse/FLINK-8000
> Project: Flink
>  Issue Type: Improvement
>  Components: REST
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
>  Labels: flip-6
>
> In order to make the {{RestServerEndpoint}} more easily extendable, we should 
> automatically sort the returned list of rest handler when calling 
> {{RestServerEndpoint#initializeHandlers}}. That way the order in which the 
> handlers are added to the list is independent of the actual registration 
> order. This is, for example, important for the static file server which 
> always needs to be registered last.
> I propose to add a special {{String}} {{Comparator}} which considers the 
> charactor {{':'}} to be the character with the largest value. That way we 
> should get always the following sort order:
> - URLs without path parameters have precedence over similar URLs where parts 
> are replaced by path parameters (e.g. {{/jobs/overview}}, {{/jobs/:jobid}} 
> and {{/jobs/:jobid/config}}, {{/jobs/:jobid/vertices/:vertexId}})
> - Prefixes are sorted before URLs containing the prefix (e.g. {{/jobs}}, 
> {{/jobs/overview}})



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-8000) Sort REST handler URLs in RestServerEndpoint

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16242192#comment-16242192
 ] 

ASF GitHub Bot commented on FLINK-8000:
---

Github user zentol commented on the issue:

https://github.com/apache/flink/pull/4958
  
you can also remove the comment in the DispatcherRestEndpoint that says to 
register the stat file handler last.


> Sort REST handler URLs in RestServerEndpoint
> 
>
> Key: FLINK-8000
> URL: https://issues.apache.org/jira/browse/FLINK-8000
> Project: Flink
>  Issue Type: Improvement
>  Components: REST
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
>  Labels: flip-6
>
> In order to make the {{RestServerEndpoint}} more easily extendable, we should 
> automatically sort the returned list of rest handler when calling 
> {{RestServerEndpoint#initializeHandlers}}. That way the order in which the 
> handlers are added to the list is independent of the actual registration 
> order. This is, for example, important for the static file server which 
> always needs to be registered last.
> I propose to add a special {{String}} {{Comparator}} which considers the 
> charactor {{':'}} to be the character with the largest value. That way we 
> should get always the following sort order:
> - URLs without path parameters have precedence over similar URLs where parts 
> are replaced by path parameters (e.g. {{/jobs/overview}}, {{/jobs/:jobid}} 
> and {{/jobs/:jobid/config}}, {{/jobs/:jobid/vertices/:vertexId}})
> - Prefixes are sorted before URLs containing the prefix (e.g. {{/jobs}}, 
> {{/jobs/overview}})



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-8000) Sort REST handler URLs in RestServerEndpoint

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16242098#comment-16242098
 ] 

ASF GitHub Bot commented on FLINK-8000:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4958#discussion_r149382049
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java
 ---
@@ -268,4 +287,69 @@ private static void registerHandler(Router router, 
Tuple2The comparator orders the Rest URLs such that URLs with path 
parameters are ordered behind
+* those without parameters. E.g.:
+* /jobs
+* /jobs/overview
+* /jobs/:jobid
+* /jobs/:jobid/config
+* /:*
+*
+* IMPORTANT: This comparator is highly specific to how Netty path 
parameter are encoded. Namely
+* via a preceding ':' character.
+*/
+   static final class RestHandlerUrlComparator implements 
Comparator>, 
Serializable {
+
+   private static final long serialVersionUID = 
2388466767835547926L;
+
+   private static final Comparator CASE_INSENSITIVE_ORDER 
= new CaseInsensitiveOrderComparator();
+
+   static final RestHandlerUrlComparator INSTANCE = new 
RestHandlerUrlComparator();
+
+   @Override
+   public int compare(
+   Tuple2 o1,
+   Tuple2 o2) {
+   return 
CASE_INSENSITIVE_ORDER.compare(o1.f0.getTargetRestEndpointURL(), 
o2.f0.getTargetRestEndpointURL());
+   }
+
+   static final class CaseInsensitiveOrderComparator implements 
Comparator, Serializable {
+   private static final long serialVersionUID = 
8550835445193437027L;
+
+   @Override
+   public int compare(String s1, String s2) {
+   int n1 = s1.length();
+   int n2 = s2.length();
+   int min = Math.min(n1, n2);
+   for (int i = 0; i < min; i++) {
+   char c1 = s1.charAt(i);
+   char c2 = s2.charAt(i);
+   if (c1 != c2) {
+   c1 = Character.toUpperCase(c1);
+   c2 = Character.toUpperCase(c2);
+   if (c1 != c2) {
+   c1 = 
Character.toLowerCase(c1);
--- End diff --

Found another note in `String#regionMatches`:
```
// Unfortunately, conversion to uppercase does not work properly
// for the Georgian alphabet, which has strange rules about case
// conversion.  So we need to make one last check before
// exiting.
```

TIL, +1 to merge.


> Sort REST handler URLs in RestServerEndpoint
> 
>
> Key: FLINK-8000
> URL: https://issues.apache.org/jira/browse/FLINK-8000
> Project: Flink
>  Issue Type: Improvement
>  Components: REST
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
>  Labels: flip-6
>
> In order to make the {{RestServerEndpoint}} more easily extendable, we should 
> automatically sort the returned list of rest handler when calling 
> {{RestServerEndpoint#initializeHandlers}}. That way the order in which the 
> handlers are added to the list is independent of the actual registration 
> order. This is, for example, important for the static file server which 
> always needs to be registered last.
> I propose to add a special {{String}} {{Comparator}} which considers the 
> charactor {{':'}} to be the character with the largest value. That way we 
> should get always the following sort order:
> - URLs without path parameters have precedence over similar URLs where parts 
> are replaced by path parameters (e.g. {{/jobs/overview}}, {{/jobs/:jobid}} 
> and {{/jobs/:jobid/config}}, {{/jobs/:jobid/vertices/:vertexId}})
> - Prefixes are sorted before URLs containing the prefix (e.g. {{/jobs}}, 
> {{/jobs/overview}})



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-8000) Sort REST handler URLs in RestServerEndpoint

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16242081#comment-16242081
 ] 

ASF GitHub Bot commented on FLINK-8000:
---

Github user tillrohrmann commented on a diff in the pull request:

https://github.com/apache/flink/pull/4958#discussion_r149379485
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java
 ---
@@ -268,4 +287,69 @@ private static void registerHandler(Router router, 
Tuple2The comparator orders the Rest URLs such that URLs with path 
parameters are ordered behind
+* those without parameters. E.g.:
+* /jobs
+* /jobs/overview
+* /jobs/:jobid
+* /jobs/:jobid/config
+* /:*
+*
+* IMPORTANT: This comparator is highly specific to how Netty path 
parameter are encoded. Namely
+* via a preceding ':' character.
+*/
+   static final class RestHandlerUrlComparator implements 
Comparator>, 
Serializable {
+
+   private static final long serialVersionUID = 
2388466767835547926L;
+
+   private static final Comparator CASE_INSENSITIVE_ORDER 
= new CaseInsensitiveOrderComparator();
+
+   static final RestHandlerUrlComparator INSTANCE = new 
RestHandlerUrlComparator();
+
+   @Override
+   public int compare(
+   Tuple2 o1,
+   Tuple2 o2) {
+   return 
CASE_INSENSITIVE_ORDER.compare(o1.f0.getTargetRestEndpointURL(), 
o2.f0.getTargetRestEndpointURL());
+   }
+
+   static final class CaseInsensitiveOrderComparator implements 
Comparator, Serializable {
+   private static final long serialVersionUID = 
8550835445193437027L;
+
+   @Override
+   public int compare(String s1, String s2) {
+   int n1 = s1.length();
+   int n2 = s2.length();
+   int min = Math.min(n1, n2);
+   for (int i = 0; i < min; i++) {
+   char c1 = s1.charAt(i);
+   char c2 = s2.charAt(i);
+   if (c1 != c2) {
+   c1 = Character.toUpperCase(c1);
+   c2 = Character.toUpperCase(c2);
+   if (c1 != c2) {
+   c1 = 
Character.toLowerCase(c1);
--- End diff --

That's how the `String#compareToIgnoreCase` is implemented.

I assume that this is due to a note in `Character.toUpperCase` and 
`Character.toLowerCase`
```
 *Note that
 * {@code Character.isUpperCase(Character.toUpperCase(codePoint))}
 * does not always return {@code true} for some ranges of
 * characters, particularly those that are symbols or ideographs.
```


> Sort REST handler URLs in RestServerEndpoint
> 
>
> Key: FLINK-8000
> URL: https://issues.apache.org/jira/browse/FLINK-8000
> Project: Flink
>  Issue Type: Improvement
>  Components: REST
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
>  Labels: flip-6
>
> In order to make the {{RestServerEndpoint}} more easily extendable, we should 
> automatically sort the returned list of rest handler when calling 
> {{RestServerEndpoint#initializeHandlers}}. That way the order in which the 
> handlers are added to the list is independent of the actual registration 
> order. This is, for example, important for the static file server which 
> always needs to be registered last.
> I propose to add a special {{String}} {{Comparator}} which considers the 
> charactor {{':'}} to be the character with the largest value. That way we 
> should get always the following sort order:
> - URLs without path parameters have precedence over similar URLs where parts 
> are replaced by path parameters (e.g. {{/jobs/overview}}, {{/jobs/:jobid}} 
> and {{/jobs/:jobid/config}}, {{/jobs/:jobid/vertices/:vertexId}})
> - Prefixes are sorted before URLs containing the prefix (e.g. {{/jobs}}, 
> {{/jobs/overview}})



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-8000) Sort REST handler URLs in RestServerEndpoint

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16241794#comment-16241794
 ] 

ASF GitHub Bot commented on FLINK-8000:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4958#discussion_r149325305
  
--- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java
 ---
@@ -268,4 +287,69 @@ private static void registerHandler(Router router, 
Tuple2The comparator orders the Rest URLs such that URLs with path 
parameters are ordered behind
+* those without parameters. E.g.:
+* /jobs
+* /jobs/overview
+* /jobs/:jobid
+* /jobs/:jobid/config
+* /:*
+*
+* IMPORTANT: This comparator is highly specific to how Netty path 
parameter are encoded. Namely
+* via a preceding ':' character.
+*/
+   static final class RestHandlerUrlComparator implements 
Comparator>, 
Serializable {
+
+   private static final long serialVersionUID = 
2388466767835547926L;
+
+   private static final Comparator CASE_INSENSITIVE_ORDER 
= new CaseInsensitiveOrderComparator();
+
+   static final RestHandlerUrlComparator INSTANCE = new 
RestHandlerUrlComparator();
+
+   @Override
+   public int compare(
+   Tuple2 o1,
+   Tuple2 o2) {
+   return 
CASE_INSENSITIVE_ORDER.compare(o1.f0.getTargetRestEndpointURL(), 
o2.f0.getTargetRestEndpointURL());
+   }
+
+   static final class CaseInsensitiveOrderComparator implements 
Comparator, Serializable {
+   private static final long serialVersionUID = 
8550835445193437027L;
+
+   @Override
+   public int compare(String s1, String s2) {
+   int n1 = s1.length();
+   int n2 = s2.length();
+   int min = Math.min(n1, n2);
+   for (int i = 0; i < min; i++) {
+   char c1 = s1.charAt(i);
+   char c2 = s2.charAt(i);
+   if (c1 != c2) {
+   c1 = Character.toUpperCase(c1);
+   c2 = Character.toUpperCase(c2);
+   if (c1 != c2) {
+   c1 = 
Character.toLowerCase(c1);
--- End diff --

why are we checking both lower case (example edge case)?


> Sort REST handler URLs in RestServerEndpoint
> 
>
> Key: FLINK-8000
> URL: https://issues.apache.org/jira/browse/FLINK-8000
> Project: Flink
>  Issue Type: Improvement
>  Components: REST
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
>  Labels: flip-6
>
> In order to make the {{RestServerEndpoint}} more easily extendable, we should 
> automatically sort the returned list of rest handler when calling 
> {{RestServerEndpoint#initializeHandlers}}. That way the order in which the 
> handlers are added to the list is independent of the actual registration 
> order. This is, for example, important for the static file server which 
> always needs to be registered last.
> I propose to add a special {{String}} {{Comparator}} which considers the 
> charactor {{':'}} to be the character with the largest value. That way we 
> should get always the following sort order:
> - URLs without path parameters have precedence over similar URLs where parts 
> are replaced by path parameters (e.g. {{/jobs/overview}}, {{/jobs/:jobid}} 
> and {{/jobs/:jobid/config}}, {{/jobs/:jobid/vertices/:vertexId}})
> - Prefixes are sorted before URLs containing the prefix (e.g. {{/jobs}}, 
> {{/jobs/overview}})



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-8000) Sort REST handler URLs in RestServerEndpoint

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16240558#comment-16240558
 ] 

ASF GitHub Bot commented on FLINK-8000:
---

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4958

[FLINK-8000] Sort Rest handler URLS in RestServerEndpoint

## What is the purpose of the change

Introduce special `RestHandlerUrlComparator` to sort REST URLs such that
URLs with path parameters are sorted after those without or fewer.

E.g. the following order would be established

```
/jobs
/jobs/overview
/jobs/:jobid
/jobs/:jobid/config
/:*
```

## Brief change log

- Introduce `RestHandlerUrlComparator`
- Sort list of returned rest handlers in `RestServerEndpoint` before 
registering them

## Verifying this change

- `RestServerEndpointTest#testRestHandlerUrlSorting`

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
  - The S3 file system connector: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)
  - If yes, how is the feature documented? (not applicable)


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink sortRestHandlerUrls

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4958.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4958


commit d1c436ef0e9e446004e21e3c6be6173d15aba359
Author: Till Rohrmann 
Date:   2017-11-04T13:56:11Z

[FLINK-8000] Sort Rest handler URLS in RestServerEndpoint

Introduce special RestHandlerUrlComparator to sort REST URLs such that
URLs with path parameters are sorted after those without or fewer.

E.g. the following order would be established

/jobs
/jobs/overview
/jobs/:jobid
/jobs/:jobid/config
/:*




> Sort REST handler URLs in RestServerEndpoint
> 
>
> Key: FLINK-8000
> URL: https://issues.apache.org/jira/browse/FLINK-8000
> Project: Flink
>  Issue Type: Improvement
>  Components: REST
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
>  Labels: flip-6
>
> In order to make the {{RestServerEndpoint}} more easily extendable, we should 
> automatically sort the returned list of rest handler when calling 
> {{RestServerEndpoint#initializeHandlers}}. That way the order in which the 
> handlers are added to the list is independent of the actual registration 
> order. This is, for example, important for the static file server which 
> always needs to be registered last.
> I propose to add a special {{String}} {{Comparator}} which considers the 
> charactor {{':'}} to be the character with the largest value. That way we 
> should get always the following sort order:
> - URLs without path parameters have precedence over similar URLs where parts 
> are replaced by path parameters (e.g. {{/jobs/overview}}, {{/jobs/:jobid}} 
> and {{/jobs/:jobid/config}}, {{/jobs/:jobid/vertices/:vertexId}})
> - Prefixes are sorted before URLs containing the prefix (e.g. {{/jobs}}, 
> {{/jobs/overview}})



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)