Re: Git migration - Timing

[Michael Osipov]

> All,
> 
> I'd like to propose that we make the move from svn to git for Tomcat
> 7.0.x, 8.5.x and 9.0.x as soon as the next 7.0.x release is complete.
> 
> The proposed approach is documented here:
> https://cwiki.apache.org/confluence/display/TOMCAT/Git+migration
> 
> I anticipate that the repositories will be read only for a couple of hours.
> 
> I also anticipate that the CI systems - particularly Gump - will take up
> to a day to switch over and iron out any problems.
> 
> Any objections?
> 
> If not, I'll start a formal VOTE.

Can you leave me a few more days? I'd like to review the wiki page and add some 
propsals.

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Re: Git migration - Timing

[Michael Osipov]

> On 11/02/2019 19:57, Michael Osipov wrote:
> >> All,
> >>
> >> I'd like to propose that we make the move from svn to git for Tomcat
> >> 7.0.x, 8.5.x and 9.0.x as soon as the next 7.0.x release is complete.
> >>
> >> The proposed approach is documented here:
> >> https://cwiki.apache.org/confluence/display/TOMCAT/Git+migration
> >>
> >> I anticipate that the repositories will be read only for a couple of hours.
> >>
> >> I also anticipate that the CI systems - particularly Gump - will take up
> >> to a day to switch over and iron out any problems.
> >>
> >> Any objections?
> >>
> >> If not, I'll start a formal VOTE.
> > 
> > Can you leave me a few more days? I'd like to review the wiki page and add 
> > some propsals.
> 
> What sort of proposals?
> 
> Each of the issues has been discussed and the approach agreed within the
> community - on this list - over a period of many months.
> 
> Unless there is a show-stopper issue that has been missed in the
> discussions to date, I really do think it is time to move forward with
> the actual migration.

I completely missed that discussion, truly my fault.
It maybe nothing special I'd object or maybe nothing at all.
At Maven TLP we have migrated tens of repos, so we gained some experience here.

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Git migration - Timing

[Michael Osipov]

Am 2019-02-11 um 15:51 schrieb Mark Thomas:

All,

I'd like to propose that we make the move from svn to git for Tomcat
7.0.x, 8.5.x and 9.0.x as soon as the next 7.0.x release is complete.

The proposed approach is documented here:
https://cwiki.apache.org/confluence/display/TOMCAT/Git+migration

I anticipate that the repositories will be read only for a couple of hours.

I also anticipate that the CI systems - particularly Gump - will take up
to a day to switch over and iron out any problems.

Any objections?


Hi Mark,

thanks for bearing with me. I was now finally able to look at the 
proposal. As far as I understand there wil be a single repo for the 
entire code base and everything will happen in branches.


Here are my comments:

1. You have set up branches: master, tomcat70, tomcat85, etc, but the 
document talks about "Branch names. master, tc8.5, tc8.0, tc7.0 etc" 
that seems to be inconsistent.
2. What will happens with Tomcat JDBC Pool? Will in remain in the 
Subversion repo or will it move to tomcat-jdbc-pool.git? The code does 
not change that offen that justifies a tandem release with Tomcat.
3. Why do you bother to import Tomcat 8.0? It is EOL, leave it in 
Subversion. Do you plan to perform any futher merge here?
4. Can we *please* have a readible tag name scheme?! I simply don't 
understand why people uppercase everything and replace dots with 
underscores. That's just ugly. Consistent tag names would be 
"{branch-name}-{version}" for the mono repo. E.g, tomcat85-8.5.40, or 
tomcat70-7.0.95. This would heavily reduce sed(1) magic for package 
maintainers and improve readability.
5. I assume that mod_jk, native and friends will remain in the 
Subversion repo?
6. I don't see in the wiki page an agreement on a good/wellknown Git 
commit message scheme: "BZ #123: \n\n"


Regards,

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Git migration - Timing

[Michael Osipov]

Am 2019-02-16 um 14:46 schrieb Mark Thomas:

On 16/02/2019 13:39, Michael Osipov wrote:

Am 2019-02-11 um 15:51 schrieb Mark Thomas:

All,

I'd like to propose that we make the move from svn to git for Tomcat
7.0.x, 8.5.x and 9.0.x as soon as the next 7.0.x release is complete.

The proposed approach is documented here:
https://cwiki.apache.org/confluence/display/TOMCAT/Git+migration

I anticipate that the repositories will be read only for a couple of
hours.

I also anticipate that the CI systems - particularly Gump - will take up
to a day to switch over and iron out any problems.

Any objections?


Hi Mark,

thanks for bearing with me. I was now finally able to look at the
proposal. As far as I understand there wil be a single repo for the
entire code base and everything will happen in branches.

Here are my comments:

1. You have set up branches: master, tomcat70, tomcat85, etc, but the
document talks about "Branch names. master, tc8.5, tc8.0, tc7.0 etc"
that seems to be inconsistent.


The demo was set up before the branch names were agreed.


2. What will happens with Tomcat JDBC Pool? Will in remain in the
Subversion repo or will it move to tomcat-jdbc-pool.git? The code does
not change that offen that justifies a tandem release with Tomcat.


It will remain part of the repo for each Tomcat version - as it is now.


3. Why do you bother to import Tomcat 8.0? It is EOL, leave it in
Subversion. Do you plan to perform any futher merge here?


The demo was set up before 8.0.x was made EOL. There are no plans to
move 8.0.c to git.


4. Can we *please* have a readible tag name scheme?! I simply don't
understand why people uppercase everything and replace dots with
underscores. That's just ugly. Consistent tag names would be
"{branch-name}-{version}" for the mono repo. E.g, tomcat85-8.5.40, or
tomcat70-7.0.95. This would heavily reduce sed(1) magic for package
maintainers and improve readability.


The community has not expressed a desire to change the naming scheme.
You are, of course, free to start such a discussion on dev@.


5. I assume that mod_jk, native and friends will remain in the
Subversion repo?


Yes.


6. I don't see in the wiki page an agreement on a good/wellknown Git
commit message scheme: "BZ #123: \n\n"


No such convention exists for subversion. The community has not
expressed a desire to introduce one with the move to git. Again, you are
free to start a discussion on dev@


Thanks for the quick response. I will start the approapriate discussions!

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Git migration: new branch/tag naming scheme

[Michael Osipov]

Folks,

given that we are currently in the process of migrating to Git I'd like 
to propose a more readible and with the branch names consistent tag 
naming scheme.


The given approach, for whatsoever reason, performs an uppercase and 
replaces dots with underscores. This reduces readability, but also 
requires people (esp. package maintainers) to perform sed(1) magic to 
convert back and forth.


There are bascially two approaches I'd like to discuss:

Approch 1: It will reuse the branch name of the current major version, 
excluding master. Thus, we will have the following prefixes: tomcat90-, 
tomcat85-, and tomcat70-. Since JDBC Pool remains in place and if it is 
released separately the prefix would be jdbc-pool-. The version itself 
would remain as-is and simply appended, e.g., 8.5.40, 9.0.25, etc.


Finally it would be tomcat85-8.5.40, tomcat90-9.0.25. Another benefit 
would be autocompletion in Git CLI. I could simply say: "git checkout 
tomcat85" or "git checkout tomcat85-" and grab the specific version 
I want.


Approach 2: A simpler, less redundant approach would be naming the 
branches master, 7.0.x, 8.5.x, etc. and get rid of the "tomcat" prefix 
at all. The tags would simply be the versions as-is: 8.5.40, 9.0.25, 
etc. The Git autocompletion will work here too.


I personally opt for approach 2 because it is consistent, concise and 
removes redundancy on always used prefixes.


Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Git migration: consistent commit message

[Michael Osipov]

Folks,

most of you know that there is a common convention of how to write 
decent Git commit messages. The scheme is: \n\n".


Where as title (proposed) is: BZ #: 
If there is no corresponding BZ issue (which almost always should be), 
just an abtract.

The abstract shouldn't contain more than 80 chars (sometimes not possible).

This will highly improve commit message readablity in Git CLI, GitHub as 
well as Gitbox. Everything is trimmed on "\n\n" in the Git 
world.


Personally, I sometimes have trouble to follow the commit log in 
Subversion to diff two versions between deployments. Therefore, I'd 
appreciate a better approach for the entire user base.


Here are three good examples of a readable log:
* https://github.com/apache/maven/commits/master
* https://github.com/curl/curl/commits/master
* https://github.com/apache/httpcomponents-client/commits/master

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Git migration: new branch/tag naming scheme

[Michael Osipov]

Am 2019-02-18 um 11:03 schrieb Mark Thomas:

On 18/02/2019 09:13, Rémy Maucherat wrote:

On Sat, Feb 16, 2019 at 4:09 PM Michael Osipov  wrote:


Folks,

given that we are currently in the process of migrating to Git I'd like
to propose a more readible and with the branch names consistent tag
naming scheme.

The given approach, for whatsoever reason, performs an uppercase and
replaces dots with underscores. This reduces readability, but also
requires people (esp. package maintainers) to perform sed(1) magic to
convert back and forth.

There are bascially two approaches I'd like to discuss:

Approch 1: It will reuse the branch name of the current major version,
excluding master. Thus, we will have the following prefixes: tomcat90-,
tomcat85-, and tomcat70-. Since JDBC Pool remains in place and if it is
released separately the prefix would be jdbc-pool-. The version itself
would remain as-is and simply appended, e.g., 8.5.40, 9.0.25, etc.

Finally it would be tomcat85-8.5.40, tomcat90-9.0.25. Another benefit
would be autocompletion in Git CLI. I could simply say: "git checkout
tomcat85" or "git checkout tomcat85-" and grab the specific version
I want.

Approach 2: A simpler, less redundant approach would be naming the
branches master, 7.0.x, 8.5.x, etc. and get rid of the "tomcat" prefix
at all. The tags would simply be the versions as-is: 8.5.40, 9.0.25,
etc. The Git autocompletion will work here too.

I personally opt for approach 2 because it is consistent, concise and
removes redundancy on always used prefixes.



I guess it's hard to argue against option 2.

The main downside is that it comes late and Mark already did the work and
lots of testing for his proposed plan.


The current, community agreed proposal for branch naming is "master,
tc8.5 and tc7.0"

There were strong views on the branch naming but "master, 8.5.x and
7.0.x" would be consistent with those views. I'm not sure I see much
difference between either approach. If there is a strong preference for
one over the other - or a good reason to choose one over the other -
please make those views known in the next few days.


The current proposal, community agreed proposal for tags is to continue
with the current naming scheme. Switching to using the version as-is
(9.0.1, 9.0.2, etc) is doable. It is just a little more work during the
migration. If the as-is naming scheme makes it easier for downstream
users then that strikes me as a good reason to change it. Are there any
objections to doing so?


I'd be happy if we could clean that up..


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Git migration: new branch/tag naming scheme

[Michael Osipov]

Am 2019-02-18 um 15:19 schrieb Igal Sapir:

On Mon, Feb 18, 2019 at 2:03 AM Mark Thomas  wrote:


On 18/02/2019 09:13, Rémy Maucherat wrote:

On Sat, Feb 16, 2019 at 4:09 PM Michael Osipov 

wrote:



Folks,

given that we are currently in the process of migrating to Git I'd like
to propose a more readible and with the branch names consistent tag
naming scheme.

The given approach, for whatsoever reason, performs an uppercase and
replaces dots with underscores. This reduces readability, but also
requires people (esp. package maintainers) to perform sed(1) magic to
convert back and forth.

There are bascially two approaches I'd like to discuss:

Approch 1: It will reuse the branch name of the current major version,
excluding master. Thus, we will have the following prefixes: tomcat90-,
tomcat85-, and tomcat70-. Since JDBC Pool remains in place and if it is
released separately the prefix would be jdbc-pool-. The version itself
would remain as-is and simply appended, e.g., 8.5.40, 9.0.25, etc.

Finally it would be tomcat85-8.5.40, tomcat90-9.0.25. Another benefit
would be autocompletion in Git CLI. I could simply say: "git checkout
tomcat85" or "git checkout tomcat85-" and grab the specific version
I want.

Approach 2: A simpler, less redundant approach would be naming the
branches master, 7.0.x, 8.5.x, etc. and get rid of the "tomcat" prefix
at all. The tags would simply be the versions as-is: 8.5.40, 9.0.25,
etc. The Git autocompletion will work here too.

I personally opt for approach 2 because it is consistent, concise and
removes redundancy on always used prefixes.



I guess it's hard to argue against option 2.

The main downside is that it comes late and Mark already did the work and
lots of testing for his proposed plan.


The current, community agreed proposal for branch naming is "master,
tc8.5 and tc7.0"



That's fine by me.




There were strong views on the branch naming but "master, 8.5.x and
7.0.x" would be consistent with those views. I'm not sure I see much
difference between either approach. If there is a strong preference for
one over the other - or a good reason to choose one over the other -
please make those views known in the next few days.



No good reason.  Based on the OP I thought that we are comparing to a more
verbose scheme like "tomcat-90-xxx" (without the standard "master" branch).

I actually prefer "tc8.5" and "tc7.0" for the branches (over "8.5.x" and
"7.0.x").  If tags will only use the numeric versions then this will make
it easy to differentiate between branches and tags.


tc8.5 could also misread as 8.5 release. 8.5.x implies that this is in 
development. This a common scheme in many many repos.
Where is the benefit keeping the prefix? Git autocompletion will stop at 
"8.5." and you choose either 'x' or a patch release.


Michael


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Git migration: new branch/tag naming scheme

[Michael Osipov]

Am 2019-02-20 um 17:44 schrieb Mark Thomas:

On 20/02/2019 16:14, Igal Sapir wrote:

Michael,

On Mon, Feb 18, 2019 at 11:53 AM Michael Osipov  wrote:


Am 2019-02-18 um 15:19 schrieb Igal Sapir:



I actually prefer "tc8.5" and "tc7.0" for the branches (over "8.5.x" and
"7.0.x").  If tags will only use the numeric versions then this will make
it easy to differentiate between branches and tags.


tc8.5 could also misread as 8.5 release. 8.5.x implies that this is in
development. This a common scheme in many many repos.
Where is the benefit keeping the prefix? Git autocompletion will stop at
"8.5." and you choose either 'x' or a patch release.



If I want to switch branches, which is more often than switching to a tag
in my case, I would simply start with 't' without having to go through the
tags.

Having to go through the tag options when I actually want to switch to a
branch is inefficient.

Don't get me wrong, I don't care much which ones have a 'tc' prefix
(branches) and which do not (tags), I just like that there is a difference
between branches and tags.


This got me thinking about how many key pressed would actually be
required for branches

Case A:
Branches: 7.0.x, 8.5.x, master
Tags: 7.0.11, 8.5.20, 9.0.12 etc.

git checkout [7|8|m]
master then auto-completes
7.0.x and 8.5.x get to "7.0." and "8.5." respectively and need a further
"x" to complete.

So that makes (ignoring the "git checkout ") 2 key presses for master
and 3 for 7.0.x or 8.5.x

Case B:
Branches: tc7.0.x, tc8.5.x, master
Tags: 7.0.11, 8.5.20, 9.0.12 etc.

git checkout [t|m]
master then auto-completes
7.0.x and 8.5.x both get to "tc" and need a further "7" or "8" and 
to complete.

So that makes (ignoring the "git checkout ") 2 key presses for master
and 4 for 7.0.x or 8.5.x


Of course there are complications to this:
- I suspect most developers will be using worktrees
- keypresses are probably less important than how the GUIs of our
   preferred tools handles this


In a GUI it will show how clearly what is a branch or a tag. There 
shouldn't be any  ambiguity.



I think this comes down to whether it is worth using a prefix (tc) to
(more clearly) differentiate release branches and tags in those places
where the two appear together.

To put it another way:

Is "9.0.5 vs tc9.0.x" clearer than "9.0.5 vs 9.0.x"? And if it is, do we
want that additional clarity?


Fo me, it looks inconsistent. "9.0.5 vs 9.0.x" clearly says that the 
former is a fixed version whereas the latter is a moving target in 9.0.


I'd clearly opt for simplicity.

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Better integration of RemoteIpValve, AuthenticatorBase and reverse proxies

[Michael Osipov]

Hi folks,

I have some improvement ideas for several components where I think 
others would benefit from too.


We intend to run a set of apps on Tomcat 8.5 behind Apache 2.4.x for a 
possible future load balacing scenario. While evaluating this task I 
have stumbled on the issue that it isn't that trivial to tell HTTPd that 
Tomcat is peforming the authentication and here is remoe user + auth 
type, use that for your access logs too. I have a partially working idea 
I'd like to get upstream (partially reported as BZ 62496):


1. RemoteIpValve sets requestAttributesEnabled by default in [1], I'd 
like to add here:
> org.apache.coyote.Constants.FORWARDED_REQUEST_ATTRIBUTE: 
"org.apache.tomcat.forwardedRequest"

> request.setAttribute(Constants.FORWARDED_REQUEST_ATTRIBUTE, "true");
2. AuthenticatorBase would pick this up in [2] by checking this 
attribute and doing:

> response.setHeader(remoteUserHeaderName, request.getRemoteUser());
> response.setHeader(authtTypeHeaderName, request.getAuthType());

where default header names are: X-Remote-User, X-Auth-Type.
I am not yet certain whether it should require just 
FORWARDED_REQUEST_ATTRIBUTE, but also another attribute (e.g., boolean 
respondAuthInfoOnForwardedRequests) also. FORWARDED_REQUEST_ATTRIBUTE is 
nice because any internal component will know that this request is not 
an original request, but a forwarded one.


On HTTPd I have:

Header note X-Remote-User REMOTE_USER
Header note X-Auth-Type AUTH_TYPE
Header unset X-Remote-User
Header unset Auth-Type
LuaHookLog /usr/local/etc/apache24/register_remote_user.lua register_remote_user


Access logs now look fine for me on both Tomcat and Apache HTTPd.

WDYT?

Michael

[1] 
https://github.com/apache/tomcat85/blob/trunk/java/org/apache/catalina/valves/RemoteIpValve.java#L666-L676
[2] 
https://github.com/apache/tomcat85/blob/trunk/java/org/apache/catalina/authenticator/AuthenticatorBase.java#L999-L1001


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Migrate to git

[Michael Osipov]

Am 2019-02-21 um 17:13 schrieb Mark Thomas:

This is a VOTE to migrate the primary source code repository for Apache
Tomcat 9.0.x, 8.5.x and 7.0.x from svn to git.

The migration will be performed as per:
https://cwiki.apache.org/confluence/display/TOMCAT/Git+migration

with the following changes:
- 8.0.x will not be migrated
- the tag name format will be changed from "TOMCAT_9_0_5" to "9.0.5"
- the branches will be named master, 8.5.x and 7.0.x

The proposed date (subject to Infra agreement) for the migration is 26
Feb 2018.

The migration process will be:
- Make svn read only for trunk, 8.5.x and 7.0.x
- Turn off the svn->git replication for trunk, 8.5.x and 7.0.x
- Make git://git.apache.org/tomcat.git read/write for me only
- Perform the migration as set out in the wiki with the modifications
   described above
- Check the migration
- Make git://git.apache.org/tomcat.git read/write for all committers
   (Note: This automatically makes https://github.com/apache/tomcat
read/write as well)

The critical work is done at this point. The following tasks are more
clean-up and may end up being spread over several days.

- Confirm there are no open PRs for https://github.com/apache/tomcat85
   and then delete it and git://git.apache.org/tomcat85.git
- Confirm there are no open PRs for https://github.com/apache/tomcat70
   and then delete it and git://git.apache.org/tomcat70.git
- Update the CI systems to pull the source from git
- Create /source.html and replace /svn.html with a redirect to
   /source.html
- Update migration guide to pull diffs from gitweb
- Update Tomcat Native to pull in source from git hash
- Fix anything else we have forgotten about.

If anything goes wrong and we can't fix is easily, the fallback is to
make svn read-write and go back to using svn while we clean up the git
side of things, figure out what went wrong and come up with a better
migration plan.

[ ] +1 Go ahead with the migration
[ ] -1 Postpone the migration because...


+1

I hope we can accommodate the proposed changes, e.g., branches.

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Git migration read for testing

[Michael Osipov]

Am 2019-02-26 um 14:54 schrieb Rémy Maucherat:

On Tue, Feb 26, 2019 at 1:33 PM Mark Thomas  wrote:


All,

https://github.com/apache/tomcat

is now ready for testing.

It should contain:
branches
- master (9.0.x)
- 8.5.x
- 7.0.x

Tags:
- one for each 7.0.x, 8.5.x and 9.0.x release

Tags have all been renamed to follow a a.b.c-MODIFIERn format for
version number where modifier is RC or M.

The repository is probably read/write for all committers now but please
refrain from making any changes until we confirm that all is well.

If you have some time available now, please test this new repository and
report and questions or concerns to this thread.

Assuming no issues are discovered, I'd like to formally move over to git
later today.



That looks quite good :)

Should we plan to all work with PRs for large changes ?


I'd love to have this, giving some time for the community to watch over 
the change...and assigning reviewers to it.



-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Git migration read for testing

[Michael Osipov]

Am 2019-02-26 um 13:33 schrieb Mark Thomas:

All,

https://github.com/apache/tomcat

is now ready for testing.

It should contain:
branches
- master (9.0.x)
- 8.5.x
- 7.0.x

Tags:
- one for each 7.0.x, 8.5.x and 9.0.x release

Tags have all been renamed to follow a a.b.c-MODIFIERn format for
version number where modifier is RC or M.


That looks so sweet and tidy. I cannot clone the repo at the moment, 
Gitbox seems to be down in EU.


Thank you for your effort!

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Git migration read for testing

[Michael Osipov]

Am 2019-03-01 um 21:50 schrieb Mark Thomas:

On 01/03/2019 19:54, Mark Thomas wrote:

On 01/03/2019 19:00, Coty Sutherland wrote:

The email notifications work for when we push commits to the repository,
but it looks like we're missing emails when PRs are opened.


ACK. I'll talk to infra.


Fixed.

FYI we have the option to route commits to one location and PRs +
comments to another. At the moment, everything goes to dev@

Some projects have separate commits@ and notifications@ lists (and
possibly issues@ as well).

Personally, I like it all in one place and tend to filter everything
into a single folder anyway. But I wanted to mention that the option was
there.


I'd opt for commits to commits@ and PRs to dev@. For those who'd like 
see commits will need to subscribe to it. Commits on dev@ produce a lot 
of traffic not necessarily related to dev discusssions.


Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] 01/01: Fix wrong protocol version usage

[Michael Osipov]

Am 2019-03-31 um 14:50 schrieb Konstantin Kolinko:

-1 (veto).
This was discussed several years ago, and the decision was to use "HTTP/2.0"
https://bz.apache.org/bugzilla/show_bug.cgi?id=58605


I tripped over this when comparing access logs between Tomcat and 
Apache. Also, as you have noted, according to the RFC, it is HTTP/2, not 
HTTP/2.0. Only the PRI request uses HTTP/2.0 for compat reasons.


I don't see a reason to keep an incorrect proto version. Just because 
Mozilla makes it wrong, it doens't mean we have to.


M


вс, 31 мар. 2019 г. в 11:04, :



This is an automated email from the ASF dual-hosted git repository.

michaelo pushed a commit to branch wrong-http2-version
in repository https://gitbox.apache.org/repos/asf/tomcat.git

commit 0e1de0d34302cdea6b3c2a47b03dcca4c7e2f9b7
Author: Michael Osipov 
AuthorDate: Sun Mar 31 10:03:29 2019 +0200

 Fix wrong protocol version usage

 When serving a HTTP/2 request the protocol version was set as "HTTP/2.0"
 which does not exist.
---
  java/org/apache/coyote/http2/Stream.java | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/java/org/apache/coyote/http2/Stream.java 
b/java/org/apache/coyote/http2/Stream.java
index 3e64329..437279a 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -126,7 +126,7 @@ class Stream extends AbstractStream implements 
HeaderEmitter {
  this.coyoteRequest.setSendfile(handler.hasAsyncIO() && 
handler.getProtocol().getUseSendfile());
  this.coyoteResponse.setOutputBuffer(http2OutputBuffer);
  this.coyoteRequest.setResponse(coyoteResponse);
-this.coyoteRequest.protocol().setString("HTTP/2.0");
+this.coyoteRequest.protocol().setString("HTTP/2");
  if (this.coyoteRequest.getStartTime() < 0) {
  this.coyoteRequest.setStartTime(System.currentTimeMillis());
  }


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org






-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] 01/01: Fix wrong protocol version usage

[Michael Osipov]

Am 2019-04-01 um 09:36 schrieb Rémy Maucherat:

On Sun, Mar 31, 2019 at 7:38 PM Michael Osipov  wrote:


Am 2019-03-31 um 14:50 schrieb Konstantin Kolinko:

-1 (veto).
This was discussed several years ago, and the decision was to use

"HTTP/2.0"

https://bz.apache.org/bugzilla/show_bug.cgi?id=58605


I tripped over this when comparing access logs between Tomcat and
Apache. Also, as you have noted, according to the RFC, it is HTTP/2, not
HTTP/2.0. Only the PRI request uses HTTP/2.0 for compat reasons.

I don't see a reason to keep an incorrect proto version. Just because
Mozilla makes it wrong, it doens't mean we have to.



The commit was vetoed with a reason given, so it should be reverted for
now, and then discussion may continue on the merits.


The commit is not part of the master branch. I have intentionally 
created a PR. I don't like the CTR approach because it procudes mess in 
the first place on the commit log.


I am happy if we can have that discussion on the mailing list or on the 
PR. For now, only Konstantin has objected, I'd like to hear other 
comitter's opinion on.


Michael


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch master updated: https://bz.apache.org/bugzilla/show_bug.cgi?id=63286 access log formats

[Michael Osipov]

Am 2019-04-01 um 13:10 schrieb ma...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
  new 924d15a  https://bz.apache.org/bugzilla/show_bug.cgi?id=63286 access 
log formats
924d15a is described below

commit 924d15aa09c590f6d1b932a90df8297d333d2b2e
Author: Mark Thomas 
AuthorDate: Mon Apr 1 12:08:42 2019 +0100

 https://bz.apache.org/bugzilla/show_bug.cgi?id=63286 access log formats
 
 Document the differences in behaviour between the LogFormat directive in

 httpd and the pattern attribute in the AccessLogValve for %D and %T.
---
  webapps/docs/changelog.xml| 6 ++
  webapps/docs/config/valve.xml | 9 +++--
  2 files changed, 13 insertions(+), 2 deletions(-)

+%T - Time taken to process the request, in seconds. Note: This
+value has millisecond resolution whereas in httpd it has
+second resolution. Behaviour will be align to httpd
+in Tomcat 10 onwards.


One more nit, %T in Tomcat isn't just writing seconds, but seconds with 
a fraction:



} else {
// second
buf.append(Long.toString(time / 1000));
buf.append('.');
int remains = (int) (time % 1000);
buf.append(Long.toString(remains / 100));
remains = remains % 100;
buf.append(Long.toString(remains / 10));
buf.append(Long.toString(remains % 10));
}
}


That's not really obvious, I guess.

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch master updated: https://bz.apache.org/bugzilla/show_bug.cgi?id=63286 access log formats

[Michael Osipov]

Am 2019-04-01 um 13:42 schrieb Mark Thomas:

On 01/04/2019 12:41, Michael Osipov wrote:

Am 2019-04-01 um 13:10 schrieb ma...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
   new 924d15a
https://bz.apache.org/bugzilla/show_bug.cgi?id=63286 access log formats
924d15a is described below

commit 924d15aa09c590f6d1b932a90df8297d333d2b2e
Author: Mark Thomas 
AuthorDate: Mon Apr 1 12:08:42 2019 +0100

  https://bz.apache.org/bugzilla/show_bug.cgi?id=63286 access log
formats
   Document the differences in behaviour between the LogFormat
directive in
  httpd and the pattern attribute in the AccessLogValve for %D and %T.
---
   webapps/docs/changelog.xml    | 6 ++
   webapps/docs/config/valve.xml | 9 +++--
   2 files changed, 13 insertions(+), 2 deletions(-)

+    %T - Time taken to process the request, in seconds.
Note: This
+    value has millisecond resolution whereas in httpd
it has
+    second resolution. Behaviour will be align to httpd
+    in Tomcat 10 onwards.


One more nit, %T in Tomcat isn't just writing seconds, but seconds with
a fraction:


Which is covered in the text above. The value is in seconds with
millisecond resolution.


Ok, right, that's doesn't really popup unambiguously for the non-native 
English reader.


Maybe precision and/or integer/double would really clarify.

Michael


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] 01/01: Fix wrong protocol version usage

[Michael Osipov]

Am 2019-04-02 um 04:53 schrieb Christopher Schultz:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Michael,

On 3/31/19 04:03, micha...@apache.org wrote:

This is an automated email from the ASF dual-hosted git
repository.

michaelo pushed a commit to branch wrong-http2-version in
repository https://gitbox.apache.org/repos/asf/tomcat.git

commit 0e1de0d34302cdea6b3c2a47b03dcca4c7e2f9b7 Author: Michael
Osipov  AuthorDate: Sun Mar 31 10:03:29 2019
+0200

Fix wrong protocol version usage

When serving a HTTP/2 request the protocol version was set as
"HTTP/2.0" which does not exist. ---
java/org/apache/coyote/http2/Stream.java | 2 +- 1 file changed, 1
insertion(+), 1 deletion(-)

diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java index 3e64329..437279a
100644 --- a/java/org/apache/coyote/http2/Stream.java +++
b/java/org/apache/coyote/http2/Stream.java @@ -126,7 +126,7 @@
class Stream extends AbstractStream implements HeaderEmitter {
this.coyoteRequest.setSendfile(handler.hasAsyncIO() &&
handler.getProtocol().getUseSendfile());
this.coyoteResponse.setOutputBuffer(http2OutputBuffer);
this.coyoteRequest.setResponse(coyoteResponse); -
this.coyoteRequest.protocol().setString("HTTP/2.0"); +
this.coyoteRequest.protocol().setString("HTTP/2"); if
(this.coyoteRequest.getStartTime() < 0) {
this.coyoteRequest.setStartTime(System.currentTimeMillis()); }


Does this all come down to what is shown in the log files and what is
returned by request.getProtocol() for servlets to consume? It isn't an
issue with an HTTP/2 response at all.


Correct,


Looks like it should be "HTTP/2.0", based upon these references:

RFC 7540 (HTTP/2)
Status line contains neither protocol nor status text: only the
numeric code.
https://tools.ietf.org/html/rfc7540#section-3.1

JavaEE javadoc for HttpServletRequest#getProtocol:

"
Returns the name and version of the protocol the request uses in the
form protocol/majorVersion.minorVersion, for example, HTTP/1.1. For
HTTP servlets, the value returned is the same as the value of the CGI
variable SERVER_PROTOCOL.
"

If we want to change what HttpServletRequest.getProtocol returns, I
think we need to get agreement from the Servlet EG about an exception
for HTTP/2 because anyone reading the documentation for that method
should expect a ".0" on  the end of "HTTP/2".


This is of course an argumentation I cannot resist to agree, though the 
documentatin of %r is wrong:

%r - First line of the request (method and request URI)

It misses the protocol. Also for the sake of consistency I'd like to see:
coyoteRequest.protocol().setString(Constants.HTTP_20);

I'll drop the PR.

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch master updated: Update ServerInfo to reflect actual information instead of placeholders when running development builds

[Michael Osipov]

Am 2019-04-11 um 21:49 schrieb csuth...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

csutherl pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
  new c8748aa  Update ServerInfo to reflect actual information instead of 
placeholders when running development builds
c8748aa is described below

commit c8748aaf9f3f7bc9f38c5805ed80e1a333696216
Author: Coty Sutherland 
AuthorDate: Thu Apr 11 15:45:55 2019 -0400

 Update ServerInfo to reflect actual information instead of placeholders 
when running development builds
---
  java/org/apache/catalina/util/ServerInfo.java | 8 
  1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/java/org/apache/catalina/util/ServerInfo.java 
b/java/org/apache/catalina/util/ServerInfo.java
index 020d926..a70b5bf 100644
--- a/java/org/apache/catalina/util/ServerInfo.java
+++ b/java/org/apache/catalina/util/ServerInfo.java
@@ -68,11 +68,11 @@ public class ServerInfo {
  } catch (Throwable t) {
  ExceptionUtils.handleThrowable(t);
  }
-if (info == null)
-info = "Apache Tomcat 9.0.x-dev";
-if (built == null)
+if (info == null || info.equals("Apache Tomcat/@VERSION@"))
+info = "Apache Tomcat/9.0.x-dev";
+if (built == null || built.equals("@VERSION_BUILT@"))
  built = "unknown";
-if (number == null)
+if (number == null || number.equals("@VERSION_NUMBER@"))
  number = "9.0.x";


I don't like such an approach. This isn't Java, this is the 
C-define-from-external approach.


The better approach is to read a build.properties at runtime.

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Maven uploads and hashes

[Michael Osipov]

I have just released Maven Resolver Ant Tasks 1.2.1. Should be soon on 
Central. SHA-2 is up next.


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Java library bug in JCEKS keystore loader

[Michael Osipov]

Am 2020-06-12 um 23:54 schrieb Christopher Schultz:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

All,

I've been writing a Java-based certification-expiration checking
utility that can handle all kinds of file formats like PEM and the
various keystore formats supported by the JVM.

Since it's not possible to tell what type of keystore is being loaded
without writing a bunch of magic-checking code or implementing an
ASN.1 parser (no thank you), I simply try all keystore types until I
find one that works. I'm using a rewindable InputStream which works well
.


there is no need to, use Apache Directory Kerby ASN.1 module. I use it 
too and it works very well for my certiticate tasks at work.


Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Building mod_jk for Windows

[Michael Osipov]

Am 2020-06-13 um 19:56 schrieb Christopher Schultz:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

All,

I see Mladen has gone crazy updating mod_jk for IIS. The build process
looks fairly straightforward in a way that isn't so straightforward
for e.g. libtcnative.

I suspect most of it is the work that has gone into his "Custom
Microsoft Compiler Toolkit Compilation" to make sure it has everything
it needs.

I wonder if this could be Dockerized to make it even easier for just
about anyone to perform a Windows build of mod_jk. Even better,
perhaps a similar toolchain could be used to build libtcnative as well.

Does anyone have extensive Docker experience? I certainly do not...


I prefer CMake for truly portable builds. Due to my recent work on 
libtcnative I have noticed that both the autoconf files als well as 
Makefiles have a lot of ancient cruft. I'd be happy to invoke just CMake 
and just make on any platform. That is a long-term goal for me with 
tcnative.


M

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Changing the name of the default branch in our git repos

[Michael Osipov]

Am 2020-06-16 um 10:02 schrieb Mark Thomas:

All,

You may have seen the recent discussions both inside and outside the ASF
about the user of "master" as the name of the default git branch. If you
haven't, the short version is that the name can be traced back to
master/slave and its associations with human slavery.

I'd like to propose that the Apache Tomcat project renames the master
branch in all of the project repositories.

I think there are two front runners for the new name:

- main - this looks to be the name GitHub and a number of OSS projects
  will be switching to

- trunk - reflects the Subversion heritage of both the project and the
   ASF

Other options I have seen suggested include "default", "dev", "develop".
Other suggestions welcome.

Personally, I am leaning towards main as that looks to be the choice of
the majority and using the majority choice will make it (a little bit)
easier for new community members to find their way around the project.

In terms of impact, changing the name is going to break stuff. It is
really creating a new branch and deleting the old one.

Deleting a branch triggers the automatic closure of github PRs against
that branch. However if we create "$new_branch" we can edit the PRs to
use "$new_branch" before we delete master. Given the small number of
open PRs that is easily done.

CI systems will need to be updated (buildbot, gump). That should be
relatively simple.

Docs will need to be updated (relatively simple).

Committers and contributors will rebase any local branches to $new_branch

Having thought about what is involved, renaming the default branch
doesn't look as problematic as I thought it might be. This looks like
something that could be done in around an hour for all our repos.

Thoughts?


Although on the Git ML this has been discussed that master comes from 
Master Copy (music, recoding, etc) and is not related to slavery, I 
prefer the term "main" as many other projects now do.


M

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Changing the name of the default branch in our git repos

[Michael Osipov]

Am 2020-06-16 um 12:09 schrieb Mark Thomas:

On 16/06/2020 10:25, Michael Osipov wrote:

Am 2020-06-16 um 10:02 schrieb Mark Thomas:

All,

You may have seen the recent discussions both inside and outside the ASF
about the user of "master" as the name of the default git branch. If you
haven't, the short version is that the name can be traced back to
master/slave and its associations with human slavery.

I'd like to propose that the Apache Tomcat project renames the master
branch in all of the project repositories.

I think there are two front runners for the new name:

- main - this looks to be the name GitHub and a number of OSS projects
   will be switching to

- trunk - reflects the Subversion heritage of both the project and the
    ASF

Other options I have seen suggested include "default", "dev", "develop".
Other suggestions welcome.

Personally, I am leaning towards main as that looks to be the choice of
the majority and using the majority choice will make it (a little bit)
easier for new community members to find their way around the project.

In terms of impact, changing the name is going to break stuff. It is
really creating a new branch and deleting the old one.

Deleting a branch triggers the automatic closure of github PRs against
that branch. However if we create "$new_branch" we can edit the PRs to
use "$new_branch" before we delete master. Given the small number of
open PRs that is easily done.

CI systems will need to be updated (buildbot, gump). That should be
relatively simple.

Docs will need to be updated (relatively simple).

Committers and contributors will rebase any local branches to $new_branch

Having thought about what is involved, renaming the default branch
doesn't look as problematic as I thought it might be. This looks like
something that could be done in around an hour for all our repos.

Thoughts?


Although on the Git ML this has been discussed that master comes from
Master Copy (music, recoding, etc) and is not related to slavery, I
prefer the term "main" as many other projects now do.


It isn't that clear cut. If you trace it back there are places where
branches are referred to as master and slave as well as places where the
master record idea is used. Various references can be found in this thread:

https://twitter.com/tobie/status/1270290278029631489


Indeed, this one is the worst: 
https://github.com/bitkeeper-scm/bitkeeper/blob/master/doc/HOWTO.ask#L290-L291



-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 8.5.57

[Michael Osipov]

Am 2020-07-01 um 00:14 schrieb Mark Thomas:

The proposed Apache Tomcat 8.5.57 release is now available for voting.

The notable changes compared to the 8.5.56 release are:

- Implement a significant portion of the TLS environment variables
   for the rewrite valve.

- Reduce memory footprint of closed HTTP/2 streams

- Improve parsing of RFC 2109 cookies

Along with lots of other bug fixes and improvements.

For full details, see the changelog:
https://ci.apache.org/projects/tomcat/tomcat85/docs/changelog.html

It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.5.57/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1274/

The tag is:
https://github.com/apache/tomcat/tree/8.5.57
9c649984ef92c2534a734c6584220a9a0c0c3462

The proposed 8.5.57 release is:
[ ] Broken - do not release
[X] Stable - go ahead and release as 8.5.57


Passes smoothly on AdoptOpenJDK 8u252 on FreeBSD 12-STABLE.

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [PROPOSAL] Remove the functional specs from docs webapp

[Michael Osipov]

Am 2020-08-10 um 17:46 schrieb Mark Thomas:

Hi all,

I'd like to propose removing all the functional spec pages from the
documentation web application.

My reasoning for this proposal is, in short, that we aren't using or
maintaining these pages.

I don't recall any discussion of these docs on the dev list, proposals
to change them, proposals for additions etc.

There have been changes but going back over the changes from the last 10
years (and there are very few of them) they each appear to be part of a
wider global change that is updating something or removing references to
a feature that has been removed.

Should someone want to revive these pages, or more likely a subset of
them, they'll always be in the history.

Thoughts?


+1

Can you list them specifically? I am a bit lost which you exactly mean.

M

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch master updated: Improve entity tag handling

[Michael Osipov]

Am 2020-08-11 um 16:52 schrieb ma...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
  new bef507e  Improve entity tag handling
bef507e is described below

commit bef507e1b7ac2eb0ff012d0d40035e218a5839cc
Author: Mark Thomas 
AuthorDate: Tue Aug 11 15:27:45 2020 +0100

 Improve entity tag handling
---
  conf/web.xml   |   8 ++
  .../apache/catalina/servlets/DefaultServlet.java   | 116 +
  .../apache/tomcat/util/http/parser/EntityTag.java  |  82 
  .../TestDefaultServletIfMatchRequests.java | 143 +
  webapps/docs/changelog.xml |   7 +
  webapps/docs/default-servlet.xml   |   8 +-
  6 files changed, 284 insertions(+), 80 deletions(-)

diff --git a/conf/web.xml b/conf/web.xml
index a685947..b5e0b30 100644
--- a/conf/web.xml
+++ b/conf/web.xml
@@ -114,6 +114,14 @@



+  
+  
+  
+  
+  
+  
+  
+  

...


@@ -279,6 +280,8 @@ public class DefaultServlet extends HttpServlet {
   */
  private boolean allowPartialPut = true;
  
+protected boolean useWeakComparisonWithIfMatch = true;


I really must object this. It clearly violates RFC 7232, section 3.1:

   An origin server MUST use the strong comparison function when
   comparing entity-tags for If-Match...


Even an option for this is wrong. I agree that we cannto produce strong 
ETags by default, but it is now better decoupled and a subclass can 
handle this. Please retain the semantics as described in RFC 7232.


Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch master updated: Improve entity tag handling

[Michael Osipov]

Am 2020-08-11 um 18:53 schrieb Mark Thomas:

On 11/08/2020 17:29, Michael Osipov wrote:

Am 2020-08-11 um 16:52 schrieb ma...@apache.org:





commit bef507e1b7ac2eb0ff012d0d40035e218a5839cc
Author: Mark Thomas 
AuthorDate: Tue Aug 11 15:27:45 2020 +0100

  Improve entity tag handling





@@ -279,6 +280,8 @@ public class DefaultServlet extends HttpServlet {
    */
   private boolean allowPartialPut = true;
   +    protected boolean useWeakComparisonWithIfMatch = true;


I really must object this. It clearly violates RFC 7232, section 3.1:


Prior to this commit the code did not use a strong comparison for
If-Match contrary to the requirements of RFC 7232.

This commit does not change this behaviour (by default)

This commit adds an option so that RFC 7232 compliant behaviour can be
enabled by those who want it without breaking backwards compatibility
for any users that are reliant on the current, non-compliant behaviour.


    An origin server MUST use the strong comparison function when
    comparing entity-tags for If-Match...


Even an option for this is wrong. I agree that we cannto produce strong
ETags by default, but it is now better decoupled and a subclass can
handle this. Please retain the semantics as described in RFC 7232.


It isn't possible to retain the semantics of RFC 7232 because Tomcat
prior to this commit did not implement them.

If you look at the code prior to this commit, any "W/" was stripped from
the resource ETag and the ETag values in the If-Match header before
comparison. The result of doing that is that the comparison is
effectively a weak one rather than a strong one.

The option is required to preserve backwards compatibility.


Granted. This ultimately means that Tomcat 10 should remove this cruft 
and be RFC-compliant. For previous versions a comment about behavioral 
change/deprecation should be added too.


Can we agree on this?


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [PROPOSAL] Remove the functional specs from docs webapp

[Michael Osipov]

Am 2020-08-11 um 21:04 schrieb Mark Thomas:

On 11/08/2020 17:30, Michael Osipov wrote:

Am 2020-08-10 um 17:46 schrieb Mark Thomas:

Hi all,

I'd like to propose removing all the functional spec pages from the
documentation web application.





+1

Can you list them specifically? I am a bit lost which you exactly mean.


Everything under:

https://tomcat.apache.org/tomcat-10.0-doc/funcspecs/


Thanks, I hav never seen these pages before.

A lot of information would be gone, but I prefer rather no information 
than outdated -- even worse wrong information.


M

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch master updated: Extracted CSS styles to external file for better code mainenance

[Michael Osipov]

Am 2020-08-16 um 20:05 schrieb isa...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

isapir pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
  new 9c5d2e3  Extracted CSS styles to external file for better code 
mainenance
9c5d2e3 is described below

commit 9c5d2e3b633fdb651bc9f11db4aac97ad3ad4df2
Author: Igal Sapir 
AuthorDate: Sun Aug 16 11:05:05 2020 -0700

 Extracted CSS styles to external file for better code mainenance
 
 Also replaced gif logo with svg

---
  java/org/apache/catalina/manager/Constants.java|  79 +-


There is a slight problem to that: TOMCAT_CSS is shared throughout the 
codebase. Do you want to break that up in CSS modules or duplicate code?


M


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch master updated: Extracted CSS styles to external file for better code mainenance

[Michael Osipov]

Am 2020-08-16 um 21:58 schrieb Igal Sapir:

Michael,

On Sun, Aug 16, 2020 at 11:37 AM Michael Osipov  wrote:


Am 2020-08-16 um 20:05 schrieb isa...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

isapir pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
   new 9c5d2e3  Extracted CSS styles to external file for better code

mainenance

9c5d2e3 is described below

commit 9c5d2e3b633fdb651bc9f11db4aac97ad3ad4df2
Author: Igal Sapir 
AuthorDate: Sun Aug 16 11:05:05 2020 -0700

  Extracted CSS styles to external file for better code mainenance

  Also replaced gif logo with svg
---
   java/org/apache/catalina/manager/Constants.java|  79 +-


There is a slight problem to that: TOMCAT_CSS is shared throughout the
codebase. Do you want to break that up in CSS modules or duplicate code?



The new file "manager.css" includes the code from TOMCAT_CSS, which is
really just a few lines of CSS [1].

We can certainly add another file with just that but IMO it would be an
overkill and would result in an additional unnecessary http request.


Accepted!

Toda raba!

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Time for Tomcat Native 1.2.25

[Michael Osipov]

Am 2020-08-20 um 18:30 schrieb Mark Thomas:

Hi,

It has been a while since 1.2.24 and there are a few fixes in the
changelog (mainly for LibreSSL and better support for a range of
platforms). With this in mind, I'm currently intending to tag 1.2.25 in
~24 hours


Please go ahead. I have started at some point in the parst to go through 
ifdefs and identify all LibreSSL versions which implement the OpenSSL 
counterparts. I hope I can pick that up next months.


Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat Native 1.2.25

[Michael Osipov]

Am 2020-08-21 um 20:22 schrieb Mark Thomas:

Version 1.2.25 includes the following changes compared to 1.2.24

- Improvements to LibreSSL support

- Improvements to HP_UX support

Various other fixes and improvements. See the changelog for details.

The proposed release artefacts can be found at [1],
and the build was done using tag [2].


A bit late, but here are my findings. I am not sure whether they are in 
Tomcat or libtcnative.


I see reliable failures in tests with LibreSSL 2.9.0 and 3.2.1 (master) 
with libtcnative master and Tomcat master on FreeBSD 12-STABLE and 
OpenJDK 1.8.0_265.


Please see: 
http://home.apache.org/~michaelo/tomcat-master-tcnative-master-libressl-2.9.0/ 
as well as 
http://home.apache.org/~michaelo/tomcat-master-tcnative-master-libressl-master/.


Let me know if you need further information for analysis/fixing.

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat Native 1.2.25

[Michael Osipov]

Am 2020-08-24 um 15:42 schrieb Mark Thomas:



On 24/08/2020 12:58, Michael Osipov wrote:

Am 2020-08-21 um 20:22 schrieb Mark Thomas:

Version 1.2.25 includes the following changes compared to 1.2.24

- Improvements to LibreSSL support

- Improvements to HP_UX support

Various other fixes and improvements. See the changelog for details.

The proposed release artefacts can be found at [1],
and the build was done using tag [2].


A bit late, but here are my findings. I am not sure whether they are in
Tomcat or libtcnative.

I see reliable failures in tests with LibreSSL 2.9.0 and 3.2.1 (master)
with libtcnative master and Tomcat master on FreeBSD 12-STABLE and
OpenJDK 1.8.0_265.


Is that any different with 1.2.24?


1.2.24 does not compile at all with LibreSSL unless you perform these 
steps: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246373#c0


I did that, compiled against 2.9.2 and 3.2.1 (master): Same behavior!

This isn't a blocker (critical) for you now?

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat Native 1.2.25

[Michael Osipov]

Am 2020-08-24 um 15:42 schrieb Mark Thomas:



On 24/08/2020 12:58, Michael Osipov wrote:

Am 2020-08-21 um 20:22 schrieb Mark Thomas:

Version 1.2.25 includes the following changes compared to 1.2.24

- Improvements to LibreSSL support

- Improvements to HP_UX support

Various other fixes and improvements. See the changelog for details.

The proposed release artefacts can be found at [1],
and the build was done using tag [2].


A bit late, but here are my findings. I am not sure whether they are in
Tomcat or libtcnative.

I see reliable failures in tests with LibreSSL 2.9.0 and 3.2.1 (master)
with libtcnative master and Tomcat master on FreeBSD 12-STABLE and
OpenJDK 1.8.0_265.


Is that any different with 1.2.24?


FWIW, I have tried now 2.9.x, 3.0.x, 3.1.x and 3.2.x (master). All of 
them crash (JVM crash) 
TEST-org.apache.tomcat.util.net.TestSSLHostConfigCompat.*.txt. It must 
be a general problem somewhere.



frame #10: 0x000a6e3df807 
libtcnative-1.so.0.2.24`Java_org_apache_tomcat_jni_SSLContext_setCertificateRaw(e=0x0008054421e0,
 o=, ctx=44874834080, javaCert=, 
javaKey=, idx=0) at sslcontext.c:1192:9
frame #11: 0x000805618587

as well as

Stack: [0x7fffdfefe000,0x7fffdfffe000],  sp=0x7fffdfffb7c0,  free 
space=1013k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libssl.so.48+0x42b59]  SSL_CTX_use_certificate+0x9
C  [libtcnative-1.so.0.2.24+0x24807]  
Java_org_apache_tomcat_jni_SSLContext_setCertificateRaw+0x1d7
j  org.apache.tomcat.jni.SSLContext.setCertificateRaw(J[B[BI)Z+0
j  
org.apache.tomcat.util.net.openssl.OpenSSLContext.addCertificate(Lorg/apache/tomcat/util/net/SSLHostConfigCertificate;)V+214
j  
org.apache.tomcat.util.net.AprEndpoint.createSSLContext(Lorg/apache/tomcat/util/net/SSLHostConfig;)V+146
j  org.apache.tomcat.util.net.AprEndpoint.bind()V+461
j  org.apache.tomcat.util.net.AbstractEndpoint.bindWithCleanup()V+1


M

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE][RESULT] Release Apache Tomcat Native 1.2.25

[Michael Osipov]

Am 2020-09-03 um 16:34 schrieb Mark Thomas:

The following votes were cast:

Binding:
+1: markt, mgrigorov, fschumacher

+0: schultz

The vote therefore passes.

I think it is worth noting that there were crashes / unit test failures
reported with LibreSSL. This isn't unexpected. There is more work to do
for LibreSSL support.


Back then, when I discovered the first flaws with LibreSSL, I started 
going thorugh all ifdefs which disable features with LibreSSL and 
searching in the Git repo for the proper LibreSSL version. I will 
publish my findings in an ubrella ticket soon.


Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 7.0.95

[Michael Osipov]

Am 2019-07-11 um 20:39 schrieb Violeta Georgieva:

The proposed Apache Tomcat 7.0.95 release is now available for voting.

For full details, see the changelog:
https://ci.apache.org/projects/tomcat/tomcat7/docs/changelog.html


That's confusing. I do not see my entry for BZ 63556 from 
633950059c04f869d645553540b8ffe825891608.


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch 7.0.x updated: Fix broken test

[Michael Osipov]

> Gesendet: Donnerstag, 01. August 2019 um 12:16 Uhr
> Von: ma...@apache.org
> An: "dev@tomcat.apache.org" 
> Betreff: [tomcat] branch 7.0.x updated: Fix broken test
>
> This is an automated email from the ASF dual-hosted git repository.
>
> markt pushed a commit to branch 7.0.x
> in repository https://gitbox.apache.org/repos/asf/tomcat.git
>
>
> The following commit(s) were added to refs/heads/7.0.x by this push:
>  new bb618dc  Fix broken test
> bb618dc is described below
>
> commit bb618dceba4c37e167e12aef3e852fe571ff5190
> Author: Mark Thomas 
> AuthorDate: Thu Aug 1 11:16:33 2019 +0100
>
> Fix broken test
> ---
>  .../apache/catalina/authenticator/TestAuthInfoResponseHeaders.java| 4 
> ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git 
> a/test/org/apache/catalina/authenticator/TestAuthInfoResponseHeaders.java 
> b/test/org/apache/catalina/authenticator/TestAuthInfoResponseHeaders.java
> index 9004744..6ce7fd8 100644
> --- a/test/org/apache/catalina/authenticator/TestAuthInfoResponseHeaders.java
> +++ b/test/org/apache/catalina/authenticator/TestAuthInfoResponseHeaders.java
> @@ -16,7 +16,6 @@
>   */
>  package org.apache.catalina.authenticator;
>
> -import java.nio.charset.StandardCharsets;
>  import java.util.ArrayList;
>  import java.util.HashMap;
>  import java.util.List;
> @@ -36,6 +35,7 @@ import org.apache.catalina.startup.TesterServlet;
>  import org.apache.catalina.startup.Tomcat;
>  import org.apache.catalina.startup.TomcatBaseTest;
>  import org.apache.catalina.valves.RemoteIpValve;
> +import org.apache.tomcat.util.buf.B2CConverter;
>  import org.apache.tomcat.util.buf.ByteChunk;
>  import org.apache.tomcat.util.codec.binary.Base64;
>
> @@ -67,7 +67,7 @@ public class TestAuthInfoResponseHeaders extends 
> TomcatBaseTest {
>  password = aPassword;
>  String userCredentials = username + ":" + password;
>  byte[] credentialsBytes =
> -userCredentials.getBytes(StandardCharsets.ISO_8859_1);
> +userCredentials.getBytes(B2CConverter.ISO_8859_1);
>  String base64auth = Base64.encodeBase64String(credentialsBytes);
>  credentials= method + " " + base64auth;
>  }


Thanks for the fix, but I do not understand why this has happened.
I ran "ant clean" and "ant test". Why didn't it fail for me?

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 8.5.45

[Michael Osipov]

Am 2019-08-15 um 00:48 schrieb Mark Thomas:

The proposed Apache Tomcat 8.5.45 release is now available for voting.

The major changes compared to the 8.5.43 release are:

- Expand the HTTP/2 excessive overhead protection to cover various forms
   of abusive client behaviour and close the connection if any such
   behaviour is detected.

- Security improvements to the Windows installer including a change in
   the default user from Local System to Local Service.

- Improve handling of invalid requests so that 400 responses are
   returned to the client rather than 500 responses.

Along with lots of other bug fixes and improvements.

For full details, see the changelog:
https://ci.apache.org/projects/tomcat/tomcat85/docs/changelog.html

It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.5.45/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1228/

The tag is:
https://github.com/apache/tomcat/tree/8.5.45
46d444a14cdac3e7e1f011a02cbdac9e5a80631c

The proposed 8.5.45 release is:
[ ] Broken - do not release
[ ] Stable - go ahead and release as 8.5.45


+1, works for me!

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 8.5.46

[Michael Osipov]

Am 2019-09-16 um 20:46 schrieb Mark Thomas:

The proposed Apache Tomcat 8.5.46 release is now available for voting.

The major changes compared to the 8.5.45 release are:

- Update to Commons Daemon 1.2.1 to pick up fixes for regressions in
   Commons Daemon 1.2.0, most notably a failure to start when using
   a 32-bit JVM on Windows.

- Avoid an NPE when accessing an https port using http.

- Fix a potential hang when using HTTP/2 with the asynchronous Servlet
   API.

Along with lots of other bug fixes and improvements.

For full details, see the changelog:
https://ci.apache.org/projects/tomcat/tomcat85/docs/changelog.html

It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.5.46/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1231/

The tag is:
https://github.com/apache/tomcat/tree/8.5.46
914f68b45127207170dff894e03ec31732cac898


+1

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [PROPOSAL] Tomcat 10: Remove WebDAV

[Michael Osipov]

Am 2019-10-07 um 16:54 schrieb Christopher Schultz:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

All,

I recently gave a presentation on locking-down Apache Tomcat[1] and I
briefly discussed the "sharp edges" present in Tomcat. Some of them
are unnecessarily sharp and may be actually unnecessary. I'm going to
make a few proposals to remove functions from Tomcat.

Proposal: Remove WebDAV

Justification:

WebDAV is a protocol that never really took off[2].


From where do you take this? We, at work, use it all the time. Either 
from Sharepoint, or a new project with mod_dav.


Another great example is mod_dav_svn. You can access you repo with any 
DAV client (except crappy Windows Explorer).



Read-only WebDAV
can practically be replaced by standard HTTP GET 


No, it can't. you can't list collections with multistatus w/o WebDAV.


and read-write WebDAV
has a host of security problems. There are better solutions to
supporting WebDAV than using the Tomcat module.


Which are? Milton.io?

The only drawback I see with the current servlet is that I cannot have 
arbitrary paths of my context served by this servlet. It serves either 
the entire app or nothing. That's why I have resorted to mod_dav.



A recent search of the users mailing list shows only 10 threads
regarding WebDAV in the past 6 years.


Maybe people are just happy with the servlet?

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [PROPOSAL] Tomcat 10: Drop APR Connector

[Michael Osipov]

Am 2019-10-07 um 16:39 schrieb Christopher Schultz:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

All,

I recently gave a presentation on locking-down Apache Tomcat[1] and I
briefly discussed the "sharp edges" present in Tomcat. Some of them
are unnecessarily sharp and may be actually unnecessary. I'm going to
make a few proposals to remove functions from Tomcat.

Proposal: Remove APR connector

Justification:

The APR connector was once used to provide superior I/O when compared
to the only other available I/O mechanism available in Java: blocking
I/O. Specifically, the APR connector allowed Tomcat to wait for
keepalive requests on a connection to in a non-blocking fashion which
was not possible with Java BIO-based connectors.

The introduction of NIO into Java back in Java 1.4 (!!) changed
things, and NIO support was added to Tomcat in 6.0. Now that it has
had time to mature, the NIO connector is superior to the APR connector
in several ways:

1. NIO connector allows non-blocking TLS handshakes
2. NIO connector uses less (Tomcat-owned) native code

The first item improves performance and availability and the second
item improves stability (and thus availability).

The last advantage which (until recently) made the APR connector still
very useful was the ability to use the OpenSSL cryptographic library
for all cryptographic operations which is measurably
higher-performance than those typically provided by the JVM.

This last advantage no longer exists since we have a JSSE provider
available for OpenSSL using libtcnative.

Notes:

This proposal does not recommend the removal of libtcnative. Only the
removal of the APR connector, the APR lifecycle listener, and the
associated native code required to support those components.


Though, I have no opion for or against. It has worked very well for me 
for the last 10+ years on HP-UX for our software.


Do we have any numbers comparing performance of both for different 
loads? Are there any drawbacks not using the APR connector?


OpenSSL must stay, it always works very well.

Michael


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 8.5.47

[Michael Osipov]

Am 2019-10-07 um 15:58 schrieb Mark Thomas:

The proposed Apache Tomcat 8.5.47 release is now available for voting.

The major changes compared to the 8.5.46 release are:

- Update to Commons Daemon 1.2.2 to pick up the fix for a regression in
   Commons Daemon 1.2.0 and 1.2.1 that triggered a crash on startup when
   running on a Windows OS that had not been fully updated.

- Fix some edge cases with NIO2 and TLS that could has a request to
   hang.


Along with lots of other bug fixes and improvements.

For full details, see the changelog:
https://ci.apache.org/projects/tomcat/tomcat85/docs/changelog.html

It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.5.47/

The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1234/

The tag is:
https://github.com/apache/tomcat/tree/8.5.47
14bdacea996993a3b94ec0972cea92370e42ae4d


+1

Though I see some minor failures I have not investigated yet


$ java -version
openjdk version "1.8.0_222"
OpenJDK Runtime Environment (build 1.8.0_222-b10)
OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)
$ uname -a
FreeBSD deblndw011x.ad001.siemens.net 12.0-STABLE FreeBSD 12.0-STABLE #0 
r351324: Wed Aug 21 15:47:48 CEST 2019 
r...@deblndw011x.ad001.siemens.net:/usr/obj/usr/src/amd64.amd64/sys/DEBLNDW011X 
 amd64


in

/var/osipovmi/Projekte/tomcat/output/build/logs/TEST-org.apache.catalina.nonblocking.TestNonBlockingAPI.NIO.txt
/var/osipovmi/Projekte/tomcat/output/build/logs/TEST-org.apache.catalina.nonblocking.TestNonBlockingAPI.NIO2.txt
/var/osipovmi/Projekte/tomcat/output/build/logs/TEST-org.apache.coyote.http2.TestHttp2InitialConnection.*.txt
/var/osipovmi/Projekte/tomcat/output/build/logs/TEST-org.apache.coyote.http2.TestHttp2Limits.*.txt


with


Testcase: testNonBlockingWriteError took 33,493 sec
FAILED
Uri: /, Status: 500, Time: 33456 duration is not < 30600
junit.framework.AssertionFailedError: Uri: /, Status: 500, Time: 33456 duration is 
not < 30600
at 
org.apache.catalina.valves.TesterAccessLogValve.validateAccessLog(TesterAccessLogValve.java:92)
at 
org.apache.catalina.nonblocking.TestNonBlockingAPI.testNonBlockingWriteError(TestNonBlockingAPI.java:380)


and


Testcase: testMultipleHostHeaders took 1,385 sec
FAILED
expected:<...content-length]-[112[7]
1-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
1-HeadersEnd
1-Body-1127]
1-EndOfStream

but was:<...content-length]-[112[8]

1-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
1-HeadersEnd
1-Body-1128]
1-EndOfStream



junit.framework.AssertionFailedError: expected:<...content-length]-[112[7]
1-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
1-HeadersEnd
1-Body-1127]
1-EndOfStream

but was:<...content-length]-[112[8]

1-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
1-HeadersEnd
1-Body-1128]
1-EndOfStream



at 
org.apache.coyote.http2.Http2TestBase.validateHttp2InitialResponse(Http2TestBase.java:129)
at 
org.apache.coyote.http2.Http2TestBase.http2Connect(Http2TestBase.java:111)
at 
org.apache.coyote.http2.TestHttp2InitialConnection.testMultipleHostHeaders(TestHttp2InitialConnection.java:55)

Testcase: testValidHostHeader took 0,143 sec
Testcase: testNoHostHeader took 0,185 sec
FAILED
expected:<...content-length]-[112[7]
1-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
1-HeadersEnd
1-Body-1127]
1-EndOfStream

but was:<...content-length]-[112[8]

1-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
1-HeadersEnd
1-Body-1128]
1-EndOfStream



> ...

and the last one is a locale issue with the resource bundles:


08-Oct-2019 11:24:51.852 INFORMATION [main] org.apache.coyote.AbstractProtocol.destroy 
Destroying ProtocolHandler ["http-apr-127.0.0.1-auto-24-26939"]
-  ---

Testcase: testHeaderLimits100x32 took 1,391 sec
Testcase: testHeaderLimits101x32 took 0,411 sec
Testcase: testHeaderLimits1x128k took 0,416 sec
FAILED

Expected: match to regular expression pattern 
[0-Goaway-\[1\]-\[11\]-\[Connection \[\d++\], Stream \[3\], .*]
 but: was "0-Goaway-[1]-[11]-[Verbindung [2], Stream [3], Gesamt-Header-Größe zu 
groß]"
junit.framework.AssertionFailedError:
Expected: match to regular expression pattern 
[0-Goaway-\[1\]-\[11\]-\[Connection \[\d++\], Stream \[3\], .*]
 but: was "0-Goaway-[1]-[11]-[Verbindung [2], Stream [3], Gesamt-Header-Größe zu 
groß]"
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
at 
org.apache.coyote.http2.TestHttp2Limits.doTestHeaderLimits(TestHttp2Limits.java:259)
at 
org.apache.coyote.http2.TestHttp2Limits.doTestHeaderLimits(TestHttp2Limits.java:170)
at 
org.apache.coyote.http2.TestHttp2Limits.doTestHeaderLimits(TestHttp2Limits.java:164)
at 
org.apache.coyote.http2.TestHttp2Limits.doTestHeaderLimits(TestHttp2Limits.java:158)
at 
org.apache.coyote.http2.TestHttp2Limits.testHeaderLimits1x128k(TestHttp2Limits.java:138)


Michael


-
To unsubscribe

Possible bugs in Http11Processor

[Michael Osipov]

Folks,

while working on an improvement for Http11Processor I have noticed there 
constructs:



if ((contentEncodingMB != null)
   (contentEncodingMB.indexOf("gzip") != -1))




if (connectionValue != null)
foundUpgrade = 
connectionValue.toLowerCase(Locale.ENGLISH).contains("upgrade");



if (findBytes(connectionValueBC, Constants.CLOSE_BYTES) != -1) {
keepAlive = false;
} else if (findBytes(connectionValueBC,
Constants.KEEPALIVE_BYTES) != -1) {


and on likely other spots. I believe they are wrong. A simple curl test:

$ curl http://md11gxtc:8080/ -I --verbose   -H "Connection: close2"
* Uses proxy env variable NO_PROXY == 'localhost .siemens.net .siemens.com 
.siemens.de'
*   Trying 147.54.67.8:8080...
* TCP_NODELAY set
* Connected to md11gxtc (147.54.67.8) port 8080 (#0)

HEAD / HTTP/1.1
Host: md11gxtc:8080
User-Agent: curl/7.66.0
Accept: */*
Connection: close2


* Mark bundle as not supporting multiuse
< HTTP/1.1 404
HTTP/1.1 404
< Content-Type: text/html;charset=utf-8
Content-Type: text/html;charset=utf-8
< Content-Language: en
Content-Language: en
< Transfer-Encoding: chunked
Transfer-Encoding: chunked
< Date: Wed, 09 Oct 2019 15:54:49 GMT
Date: Wed, 09 Oct 2019 15:54:49 GMT
< Connection: close
Connection: close

<
* Closing connection 0


I don't expect the server to respond with "Connection: close" when I 
send a non-sense value "close2".


Here w/o the header:

$ curl http://md11gxtc:8080/ -I --verbose
* Uses proxy env variable NO_PROXY == 'localhost .siemens.net .siemens.com 
.siemens.de'
*   Trying 147.54.67.8:8080...
* TCP_NODELAY set
* Connected to md11gxtc (147.54.67.8) port 8080 (#0)

HEAD / HTTP/1.1
Host: md11gxtc:8080
User-Agent: curl/7.66.0
Accept: */*


* Mark bundle as not supporting multiuse
< HTTP/1.1 404
HTTP/1.1 404
< Content-Type: text/html;charset=utf-8
Content-Type: text/html;charset=utf-8
< Content-Language: en
Content-Language: en
< Transfer-Encoding: chunked
Transfer-Encoding: chunked
< Date: Wed, 09 Oct 2019 15:56:31 GMT
Date: Wed, 09 Oct 2019 15:56:31 GMT

<
* Connection #0 to host md11gxtc left intact


Tests performed with 8.5.x/4a9f854a67bc5cece8fa83278ac5449c4b1f54d9.

WDYT?

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Unbundling Commons Daemon and tcnative

[Michael Osipov]

Guys,

I have been wondering recently why are we bundling 
commons-daemon-native.tar.gz and tomcat-native.tar.gz every time with a 
new release at all?


There are two scenarios where people don't require to use it from the 
tarball:


1. Tomcat Native and Commons Daemon are installed through OS package or 
port. Thus making the bundling redundant. E.g., on RHEL, Fedora, *BSD, 
etc. => OS managed
2. Both are compiled from source and survive Tomcat updates because 
there are installed outside of Tomcat, e.g., in /usr/local or 
/opt/ports, etc. These people don't need those tarballs too because they 
get them from dist.apache.org. => manually managed


I do both approaches on different operating systems.

We can say/document that Tomcat release T requires libtcnative x.y.z and 
Commons Daemon a.b.c to run, get your own from dist.apache.org if you do 
it manually.


More confusing is that apache-tomcat-8.5.48-dev-windows-*.zip bundles 
tcnative-1.dll (which is huge in size, OpenSSL statically linked?) *and* 
the source tarball, same for Commons Daemon. Why?


Opinions?

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Possible bugs in Http11Processor

[Michael Osipov]

Just found another bug:

private static boolean isConnectionClose(MimeHeaders headers) {
MessageBytes connection = headers.getValue(Constants.CONNECTION);
if (connection == null) {
return false;
}
return connection.equals(Constants.CLOSE);
}



RFC 7230, section 6.1. says:

Connection options are case-insensitive.


Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Unbundling Commons Daemon and tcnative

[Michael Osipov]

Am 2019-10-09 um 19:14 schrieb Mark Thomas:

On 09/10/2019 17:36, Michael Osipov wrote:

I have been wondering recently why are we bundling
commons-daemon-native.tar.gz and tomcat-native.tar.gz every time with a
new release at all?


Because users find it convenient to have the latest versions available.


I already expected this, but any other reason?


There are two scenarios where people don't require to use it from the
tarball:

1. Tomcat Native and Commons Daemon are installed through OS package or
port. Thus making the bundling redundant. E.g., on RHEL, Fedora, *BSD,
etc. => OS managed


In that case I'd expect Tomcat to be installed via an OS package or port
which makes any redundancy the packager's responsibility. If users want
to mix packaged and direct download then I think it is fair to expect
them to deal with a little redundancy - especially when they just have
to ignore/delete a couple of files.


Most OSes won't deliver an up to date Tomcat, so expect mixing. Yes, 
redundancy is OK here.



2. Both are compiled from source and survive Tomcat updates because
there are installed outside of Tomcat, e.g., in /usr/local or
/opt/ports, etc. These people don't need those tarballs too because they
get them from dist.apache.org. => manually managed

I do both approaches on different operating systems.

We can say/document that Tomcat release T requires libtcnative x.y.z and
Commons Daemon a.b.c to run, get your own from dist.apache.org if you do
it manually.


I think that is less convenient for users.


That is true, but would also reduce the binary artifacts.


*and* the source tarball, same for Commons Daemon. Why?


On Windows I think there is a case for not shipping the native sources
since the binaries are already present. Asking a user who wants them to
download them seems reasonable in that instance.


Shall I create an enhancement request for this?

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 8.5.47

[Michael Osipov]

Am 2019-10-09 um 18:55 schrieb Mark Thomas:

On 09/10/2019 16:48, Michael Osipov wrote:

and


Testcase: testMultipleHostHeaders took 1,385 sec
     FAILED
expected:<...content-length]-[112[7]
1-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
1-HeadersEnd
1-Body-1127]
1-EndOfStream

but was:<...content-length]-[112[8]

1-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
1-HeadersEnd
1-Body-1128]
1-EndOfStream



junit.framework.AssertionFailedError:
expected:<...content-length]-[112[7]
1-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
1-HeadersEnd
1-Body-1127]
1-EndOfStream

but was:<...content-length]-[112[8]

1-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
1-HeadersEnd
1-Body-1128]
1-EndOfStream



     at
org.apache.coyote.http2.Http2TestBase.validateHttp2InitialResponse(Http2TestBase.java:129)

     at
org.apache.coyote.http2.Http2TestBase.http2Connect(Http2TestBase.java:111)

     at
org.apache.coyote.http2.TestHttp2InitialConnection.testMultipleHostHeaders(TestHttp2InitialConnection.java:55)


Testcase: testValidHostHeader took 0,143 sec
Testcase: testNoHostHeader took 0,185 sec
     FAILED
expected:<...content-length]-[112[7]
1-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
1-HeadersEnd
1-Body-1127]
1-EndOfStream

but was:<...content-length]-[112[8]

1-Header-[date]-[Wed, 11 Nov 2015 19:18:42 GMT]
1-HeadersEnd
1-Body-1128]
1-EndOfStream


Hmm. Those are odd. I wonder why the content length is off. Some sort of
line ending issue maybe?


I assume so. I will look into it.


and the last one is a locale issue with the resource bundles:


08-Oct-2019 11:24:51.852 INFORMATION [main]
org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler
["http-apr-127.0.0.1-auto-24-26939"]
-  ---

Testcase: testHeaderLimits100x32 took 1,391 sec
Testcase: testHeaderLimits101x32 took 0,411 sec
Testcase: testHeaderLimits1x128k took 0,416 sec
     FAILED

Expected: match to regular expression pattern
[0-Goaway-\[1\]-\[11\]-\[Connection \[\d++\], Stream \[3\], .*]
  but: was "0-Goaway-[1]-[11]-[Verbindung [2], Stream [3],
Gesamt-Header-Größe zu groß]"
junit.framework.AssertionFailedError:
Expected: match to regular expression pattern
[0-Goaway-\[1\]-\[11\]-\[Connection \[\d++\], Stream \[3\], .*]
  but: was "0-Goaway-[1]-[11]-[Verbindung [2], Stream [3],
Gesamt-Header-Größe zu groß]"
     at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
     at
org.apache.coyote.http2.TestHttp2Limits.doTestHeaderLimits(TestHttp2Limits.java:259)

     at
org.apache.coyote.http2.TestHttp2Limits.doTestHeaderLimits(TestHttp2Limits.java:170)

     at
org.apache.coyote.http2.TestHttp2Limits.doTestHeaderLimits(TestHttp2Limits.java:164)

     at
org.apache.coyote.http2.TestHttp2Limits.doTestHeaderLimits(TestHttp2Limits.java:158)

     at
org.apache.coyote.http2.TestHttp2Limits.testHeaderLimits1x128k(TestHttp2Limits.java:138)


We should find a way to fix that one.


Either exclude the resource JARs from the classpath or change the code.
Shall I file an issue here?

Michael


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [PROPOSAL] Tomcat 10: Remove WebDAV

[Michael Osipov]

Am 2019-10-09 um 21:35 schrieb Christopher Schultz:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Michael,

On 10/9/19 11:36, Michael Osipov wrote:

Am 2019-10-07 um 16:54 schrieb Christopher Schultz:

-BEGIN PGP SIGNED MESSAGE- Hash: SHA256

Or, since svn is HTTP, you can just use plain-old HTTP. Besides,
mod_dav_svn doesn't work with Tomcat.


Again, plain HTTP != WebDAV.


Read-only WebDAV can practically be replaced by standard HTTP GET



No, it can't. you can't list collections with multistatus w/o
WebDAV.


Meh.


and read-write WebDAV has a host of security problems. There are
better solutions to supporting WebDAV than using the Tomcat
module.


Which are? Milton.io?


How about mod_dav and friends?


I was thinking about Java-based solution in Tomcat, at best with Spring 
to fully reuse my authnz code. I don't run HTTPd if it is not strictly 
necessary. Tomcat just performs perfectly well for dynamic, static and 
transport-encrypted content.



The only drawback I see with the current servlet is that I cannot
have arbitrary paths of my context served by this servlet. It
serves either the entire app or nothing. That's why I have resorted
to mod_dav.


Okay, so someone who really wants to make DAV work has decided that
Tomcat's implementation won't cut it. I fee that as further evidence
that Tomcat's implementation can just die.


As you might know, people will only complain when something is 
gone/broken and not when it is working well.



A recent search of the users mailing list shows only 10 threads
regarding WebDAV in the past 6 years.


Maybe people are just happy with the servlet?


People are super happy with the TLS implementation and ask about it
all the time.


Because encryption is complex...

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Possible bugs in Http11Processor

[Michael Osipov]

Am 2019-10-09 um 19:08 schrieb Mark Thomas:

On 09/10/2019 16:58, Michael Osipov wrote:

Folks,

while working on an improvement for Http11Processor I have noticed there
constructs:


if ((contentEncodingMB != null)
    (contentEncodingMB.indexOf("gzip") != -1))


The parsing of this is much tighter on input. For output I think this is
a reasonable trade-off. The worst that will happen is that Tomcat won't
compress something it might have been able to.


I agree on output, but there is at least on counter example:


// Check if browser support gzip encoding
MessageBytes acceptEncodingMB =
request.getMimeHeaders().getValue("accept-encoding");

if ((acceptEncodingMB == null)
|| (acceptEncodingMB.indexOf("gzip") == -1)) {
return false;
}


It would even accept "Accept-Encoding: figzip" as valid input.


if (findBytes(connectionValueBC, Constants.CLOSE_BYTES) != -1) {
     keepAlive = false;
} else if (findBytes(connectionValueBC,
     Constants.KEEPALIVE_BYTES) != -1) {


In this specific case I said via curl: "Connection: close2" and it still 
matched.



and on likely other spots. I believe they are wrong.


Yes, but. Is the cost of parsing that header (and any similar headers)
fully worth the benefit? The header parser is reasonably efficient so it
might be OK.

I'd suggest creating a BZ issue for this.


Will do!


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Possible bugs in Http11Processor

[Michael Osipov]

Am 2019-10-09 um 23:23 schrieb Mark Thomas:

On 09/10/2019 22:03, Michael Osipov wrote:

Am 2019-10-09 um 19:08 schrieb Mark Thomas:

On 09/10/2019 16:58, Michael Osipov wrote:

Folks,

while working on an improvement for Http11Processor I have noticed there
constructs:


if ((contentEncodingMB != null)
     (contentEncodingMB.indexOf("gzip") != -1))


The parsing of this is much tighter on input. For output I think this is
a reasonable trade-off. The worst that will happen is that Tomcat won't
compress something it might have been able to.


I agree on output, but there is at least on counter example:


     // Check if browser support gzip encoding
     MessageBytes acceptEncodingMB =
     request.getMimeHeaders().getValue("accept-encoding");

     if ((acceptEncodingMB == null)
     || (acceptEncodingMB.indexOf("gzip") == -1)) {
     return false;
     }


It would even accept "Accept-Encoding: figzip" as valid input.


That isn't there in 9.0.x. I wonder why the new CompressionConfig class
isn't being used in 8.5.x (that parses this correctly). I have a vague
recollection of backwards compatibility issues with the back-port but it
is worth revisiting that.


Checked CompressionConfig in master:


if (contentEncodingMB != null &&
(contentEncodingMB.indexOf("gzip") != -1 ||
contentEncodingMB.indexOf("br") != -1)) {
return false;
}



if ((acceptEncodingMB == null) || (acceptEncodingMB.indexOf("gzip") == 
-1)) {
return false;
}


don't look any better, do they?

Issue?


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Possible bugs in Http11Processor

[Michael Osipov]

Am 2019-10-09 um 23:46 schrieb Michael Osipov:

Am 2019-10-09 um 23:23 schrieb Mark Thomas:

On 09/10/2019 22:03, Michael Osipov wrote:

Am 2019-10-09 um 19:08 schrieb Mark Thomas:

On 09/10/2019 16:58, Michael Osipov wrote:

Folks,

while working on an improvement for Http11Processor I have noticed 
there

constructs:


if ((contentEncodingMB != null)
 (contentEncodingMB.indexOf("gzip") != -1))


The parsing of this is much tighter on input. For output I think 
this is

a reasonable trade-off. The worst that will happen is that Tomcat won't
compress something it might have been able to.


I agree on output, but there is at least on counter example:


 // Check if browser support gzip encoding
 MessageBytes acceptEncodingMB =
 request.getMimeHeaders().getValue("accept-encoding");

 if ((acceptEncodingMB == null)
 || (acceptEncodingMB.indexOf("gzip") == -1)) {
 return false;
 }


It would even accept "Accept-Encoding: figzip" as valid input.


That isn't there in 9.0.x. I wonder why the new CompressionConfig class
isn't being used in 8.5.x (that parses this correctly). I have a vague
recollection of backwards compatibility issues with the back-port but it
is worth revisiting that.


Checked CompressionConfig in master:


if (contentEncodingMB != null &&
    (contentEncodingMB.indexOf("gzip") != -1 ||
    contentEncodingMB.indexOf("br") != -1)) {
    return false;
    }


    if ((acceptEncodingMB == null) || 
(acceptEncodingMB.indexOf("gzip") == -1)) {

    return false;
    }


don't look any better, do they?

Issue?


I guess, I need to spawn an issue anyway because of 
https://tools.ietf.org/html/rfc7231#section-3.1.2.1:

"All content-coding values are case-insensitive ..."



-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] 01/01: First draft

[Michael Osipov]

Am 2019-10-11 um 11:51 schrieb Rémy Maucherat:

On Fri, Oct 11, 2019 at 10:30 AM  wrote:


This is an automated email from the ASF dual-hosted git repository.

michaelo pushed a commit to branch BZ-63835/8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git

commit 6ff2233cbbd27c9c2c649208a21931e5f3e132a6
Author: Michael Osipov 
AuthorDate: Fri Oct 11 10:30:08 2019 +0200

 First draft



+if (keepAliveTimeout > 0) {
+String value = "timeout=" +
TimeUnit.MILLISECONDS.toSeconds(keepAliveTimeout);
+
+if (maxKeepAliveRequests > 0) {
+value += ", max=" + maxKeepAliveRequests;
+}
StringBuilder ?


StringBuilder does not make any sense because:

* The compiler will replace the static code automatically with a 
StringBuilder
* StringBuilder gains benefit when you concat strings in a for/while/do 
loop. The above code is purely static.



Can you add a new flag controlling the feature ? The information may not be
very useful in many cases such as when proxying, so it would be better to
skip generating it by default.


This is -- as documented -- a first draft.

As mentioned on the ticket. This is hop-by-hop and writetn only if the 
client requests this piece of information. We can surely discuss a flag 
for this on the connector.



-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch BZ-63835/8.5.x created (now 6ff2233)

[Michael Osipov]

Am 2019-10-11 um 11:32 schrieb Rémy Maucherat:

On Fri, Oct 11, 2019 at 10:43 AM Mark Thomas  wrote:


On 11/10/2019 09:30, micha...@apache.org wrote:

This is an automated email from the ASF dual-hosted git repository.

michaelo pushed a change to branch BZ-63835/8.5.x


New features should be implemented against master and then back-ported
(assuming the community accepts them).



I also (still) prefer either PRs (it allows comments) or in master.


This is a first draft, nothing PR worthy. As soon as I see the code 
working, I will create the PR by then and when approved in general I 
will add tests and re-request review. So please be patient.


Michael


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Possible bug in Http11Processor/SocketWrapperBase

[Michael Osipov]

Folks,

while working on BZ-63835 I have noticed an odd thing and I'd like 
someone to review whether my code/understanding is wrong or the one 
already present in Tomcat.


Note: The same code path in HTTPd behaves corectly.

Consider this output received by curl:


HTTP/1.1 302

Transfer-Encoding: chunked
Keep-Alive: timeout=300, max=100
Connection: keep-alive

HTTP/1.1 302

Transfer-Encoding: chunked
Keep-Alive: timeout=300, max=99
Connection: keep-alive

HTTP/1.1 302

...

>

HTTP/1.1 302

Transfer-Encoding: chunked
Keep-Alive: timeout=300, max=2
Connection: keep-alive

HTTP/1.1 302

Transfer-Encoding: chunked
Connection: close


we have a total of 100 requests (HTTP/1.1 302). I'd assume the 
connection to sustain 100 requests and on the n+1 requests to be closed.


This is caused by

} else if (maxKeepAliveRequests > 0 &&
socketWrapper.decrementKeepAlive() <= 0) {
keepAlive = false;
}


while decrementKeepAlive() being a predecrement, chopping off the 
current request.

Making this postdecrement the outcome is as expected:


HTTP/1.1 302

Transfer-Encoding: chunked
Keep-Alive: timeout=20, max=100
Connection: keep-alive

HTTP/1.1 302

Transfer-Encoding: chunked
Keep-Alive: timeout=20, max=99
Connection: keep-alive

...

HTTP/1.1 302

Transfer-Encoding: chunked
Keep-Alive: timeout=20, max=2
Connection: keep-alive

HTTP/1.1 302

Transfer-Encoding: chunked
Keep-Alive: timeout=20, max=1
Connection: keep-alive

HTTP/1.1 302

Transfer-Encoding: chunked
Connection: close


having 101 requests in total, matching HTTPd behavior.

The expired header proposal says:

   The value of the "max" parameter counts the number of requests since
   the connection was created.


My code is here:
https://github.com/apache/tomcat/commit/a5e3e1d7a498a3156350ae8bbed36706b2600e64

Am I wrong?

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] 01/01: First draft

[Michael Osipov]

Am 2019-10-11 um 14:35 schrieb Rémy Maucherat:

On Fri, Oct 11, 2019 at 1:49 PM Michael Osipov  wrote:


Am 2019-10-11 um 11:51 schrieb Rémy Maucherat:

On Fri, Oct 11, 2019 at 10:30 AM  wrote:


This is an automated email from the ASF dual-hosted git repository.

michaelo pushed a commit to branch BZ-63835/8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git

commit 6ff2233cbbd27c9c2c649208a21931e5f3e132a6
Author: Michael Osipov 
AuthorDate: Fri Oct 11 10:30:08 2019 +0200

  First draft



+if (keepAliveTimeout > 0) {
+String value = "timeout=" +
TimeUnit.MILLISECONDS.toSeconds(keepAliveTimeout);
+
+if (maxKeepAliveRequests > 0) {
+value += ", max=" + maxKeepAliveRequests;
+}
StringBuilder ?


StringBuilder does not make any sense because:

* The compiler will replace the static code automatically with a
StringBuilder
* StringBuilder gains benefit when you concat strings in a for/while/do
loop. The above code is purely static.



I don't understand how this can work, or how it is static, but if you're
100% certain it's fine.


Please look here: 
https://dzone.com/articles/string-concatenation-performacne-improvement-in-ja


Exactly the same case.


Can you add a new flag controlling the feature ? The information may not

be

very useful in many cases such as when proxying, so it would be better to
skip generating it by default.


This is -- as documented -- a first draft.

As mentioned on the ticket. This is hop-by-hop and writetn only if the
client requests this piece of information. We can surely discuss a flag
for this on the connector.



Ok indeed. I never understood why some clients kept sending Connection:
keep-alive since it was also not needed.


To be honest, I didn't know either until I started digging for the 
client problem a colleague had.


Try against HTTPd and you'll see even with HTTP 1.1 client.


Michael


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch BZ-63835/8.5.x created (now 6ff2233)

[Michael Osipov]

Am 2019-10-11 um 15:10 schrieb Rémy Maucherat:

On Fri, Oct 11, 2019 at 1:51 PM Michael Osipov  wrote:


Am 2019-10-11 um 11:32 schrieb Rémy Maucherat:

On Fri, Oct 11, 2019 at 10:43 AM Mark Thomas  wrote:


On 11/10/2019 09:30, micha...@apache.org wrote:

This is an automated email from the ASF dual-hosted git repository.

michaelo pushed a change to branch BZ-63835/8.5.x


New features should be implemented against master and then back-ported
(assuming the community accepts them).



I also (still) prefer either PRs (it allows comments) or in master.


This is a first draft, nothing PR worthy. As soon as I see the code
working, I will create the PR by then and when approved in general I
will add tests and re-request review. So please be patient.



For this use I'd recommend using your own forked repository then. Using a
branch for that doesn't work for me, it sends way too many emails that are
not actually useful to dev to the dev mailing list. I tried for a while, it
didn't work, sorry :(


I am not really convinced by this and I will tell you why:

* Other TLPs have a commits@ for this
* The purpose of branches is exactly to work on something
* Having it in a fork would take the ability away that people instantly 
notice my intermediate changes in a canonical location and leave 
valuable comments as you did guys. No one will look at my fork as long 
as I don't request it. I rather fix an issue early than two weeks later 
in the PR.



-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch BZ-63835/8.5.x created (now 6ff2233)

[Michael Osipov]

Am 2019-10-11 um 15:20 schrieb Mark Thomas:

On 11/10/2019 14:10, Rémy Maucherat wrote:

On Fri, Oct 11, 2019 at 1:51 PM Michael Osipov mailto:micha...@apache.org>> wrote:

 Am 2019-10-11 um 11:32 schrieb Rémy Maucherat:
 > On Fri, Oct 11, 2019 at 10:43 AM Mark Thomas mailto:ma...@apache.org>> wrote:
 >
 >> On 11/10/2019 09:30, micha...@apache.org
 <mailto:micha...@apache.org> wrote:
 >>> This is an automated email from the ASF dual-hosted git repository.
 >>>
 >>> michaelo pushed a change to branch BZ-63835/8.5.x
 >>
 >> New features should be implemented against master and then
 back-ported
 >> (assuming the community accepts them).
 >>
 >
 > I also (still) prefer either PRs (it allows comments) or in master.

 This is a first draft, nothing PR worthy. As soon as I see the code
 working, I will create the PR by then and when approved in general I
 will add tests and re-request review. So please be patient.


For this use I'd recommend using your own forked repository then. Using
a branch for that doesn't work for me, it sends way too many emails that
are not actually useful to dev to the dev mailing list. I tried for a
while, it didn't work, sorry :(


I use upstream so much I should probably reconfigure the names of the
remotes in my checkout:
origin -> fork
upstream -> origin


This doesn't feel right. Origin always denotes the repo you cloned from, 
your fork. So keeping origin and upstream seems fine. I call upstream 
simply 'apache' in my forked ones and point to gitbox.apache.org.


Michael


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch BZ-63835/8.5.x created (now 6ff2233)

[Michael Osipov]

Am 2019-10-11 um 16:07 schrieb Rémy Maucherat:

On Fri, Oct 11, 2019 at 3:46 PM Michael Osipov  wrote:


Am 2019-10-11 um 15:10 schrieb Rémy Maucherat:

On Fri, Oct 11, 2019 at 1:51 PM Michael Osipov 

wrote:



Am 2019-10-11 um 11:32 schrieb Rémy Maucherat:

On Fri, Oct 11, 2019 at 10:43 AM Mark Thomas  wrote:


On 11/10/2019 09:30, micha...@apache.org wrote:

This is an automated email from the ASF dual-hosted git repository.

michaelo pushed a change to branch BZ-63835/8.5.x


New features should be implemented against master and then back-ported
(assuming the community accepts them).



I also (still) prefer either PRs (it allows comments) or in master.


This is a first draft, nothing PR worthy. As soon as I see the code
working, I will create the PR by then and when approved in general I
will add tests and re-request review. So please be patient.



For this use I'd recommend using your own forked repository then. Using a
branch for that doesn't work for me, it sends way too many emails that

are

not actually useful to dev to the dev mailing list. I tried for a while,

it

didn't work, sorry :(


I am not really convinced by this and I will tell you why:

* Other TLPs have a commits@ for this
* The purpose of branches is exactly to work on something
* Having it in a fork would take the ability away that people instantly
notice my intermediate changes in a canonical location and leave
valuable comments as you did guys. No one will look at my fork as long
as I don't request it. I rather fix an issue early than two weeks later
in the PR.



I disagree with this so I will draft and hold a formal vote on this to
clarify the rules regarding the use of private branches.


Sounds reasonable.


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Private branches in the official Tomcat git repository

[Michael Osipov]

Am 2019-10-11 um 16:20 schrieb Rémy Maucherat:

Hi,

This vote is to regulate the use of branches in the official Tomcat
repository beyond branches that are approved by the community such as 8.5.x
and 7.0.x. It is possible to do development in private branches directly in
the official Tomcat repository, as an alternative to using forks and pull
requests.

Should private branches be allowed in the official Tomcat git repository ?
[ ] Yes
[ ] No


I don't like the term 'private' because everytihing I add to the 
canonical repo is intended to merged into upstream sooner or later. 
Purely private stuff must be in a fork anyway.


Please redefine.

In that case as depicted by me:
Yes!


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Private branches in the official Tomcat git repository

[Michael Osipov]

Am 2019-10-12 um 15:57 schrieb Konstantin Kolinko:

пт, 11 окт. 2019 г. в 17:21, Rémy Maucherat :


Hi,

This vote is to regulate the use of branches in the official Tomcat repository 
beyond branches that are approved by the community such as 8.5.x and 7.0.x. It 
is possible to do development in private branches directly in the official 
Tomcat repository, as an alternative to using forks and pull requests.

Should private branches be allowed in the official Tomcat git repository ?
[x] Yes
[ ] No


Certainly Yes. We must be able to conduct our business without relying
on GitHub and without relying on its pull requests.


+1


If we call them not "private" branches, but "feature" branches - your
email concerns will be the same, but many projects are using feature
branches.


I agree that titling a commit as "First draft" is a bad naming. I
would like to see more elaborated commit message, with more context in
it.

If you remember when we were using Subversion, feature branches were
used several times. E.g. I used them to backport testing framework to
Tomcat 6. It generated a lot of emails, but their Subject line
contained the root path of the commit, and in Subversion such path
includes a branch name.


It is the very same approach. I am working same as before on features 
under a Bugzilla issue id. This is not intended to be a playground.



As a big github user, i expect main repo to not have unofficially supported 
code.


Technically, the only supported code are the official releases that
have passed a vote. Anything else is a work in progress.


+1


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Possible bug in Http11Processor/SocketWrapperBase

[Michael Osipov]

Am 2019-10-11 um 17:06 schrieb Mark Thomas:

On 11/10/2019 13:21, Michael Osipov wrote:

Folks,

while working on BZ-63835 I have noticed an odd thing and I'd like
someone to review whether my code/understanding is wrong or the one
already present in Tomcat.





we have a total of 100 requests (HTTP/1.1 302).


That is consistent with the docs that state there will be
maxKeepAliveRequests before the connection is closed.


I'd assume the
connection to sustain 100 requests and on the n+1 requests to be closed.


That assumption is not consistent with the documentation (which has been
the same for as long as I can remember).


I have just reread the documentation in Tomcat and in HTTPd. They both 
sound similiar, but why are both differntly implemented? Would you say 
that bouth have a different intepretation and neither is wrong?


I tend to agree with your that if 100 is send 100 is max, and not 100 + 1.

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Tomcat 8.5.x and repeatable builds

[Michael Osipov]

On 2023/11/14 16:53:38 Mark Thomas wrote:
> All,
> 
> We are currently unable to produce cross-platform repeatable builds for 
> Tomcat 8.5.x with Java 11 due to https://bugs.openjdk.org/browse/JDK-8320082
> 
> We have several options:
> 
> 1. Do nothing. Build remains repeatable on the same OS. Wait and see if 
> OpenJDK fix the bug.

I think that this is more than enough for 99% of the people especially because 
four months are left until EOL.

What I wouldn want to see that Tomcat 9 still stays testable with Java 8. So 
for Tomcat 10 with Java 11.

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Backporting patch for CVE-2023-46589 to Tomcat 8.0.14

[Michael Osipov]

On 2023/12/18 17:00:43 Mark Thomas wrote:
> On 17/12/2023 16:32, Sean Whitton wrote:
> > Hello,
> > 
> > I am working to backport the fix for CVE-2023-46589 to Tomcat version
> > 8.0.14, which is what we have in Debian "jessie".  This is under the
> > Extended LTS project for older Debian releases, run by Freexian SARL.
> >  
> 
> Sean,
> 
> Am I understanding this request correctly?
> 
> Freexian has sold at least one customer - probably multiple customers - 
> long term support for Tomcat 8.0.x and has now found that it is unable 
> to provide that support.
> 
> Feexian's solution to this dilemma is to ask the Tomcat community - who 
> stopped supporting Tomcat 8.0.x over five years ago in June 2018 - to 
> provide free support to fill this gap in Freexian's capability to 
> support Tomcat.
> 
> There are several things that don't seem right about the above so I am 
> looking forward to you correcting my understanding of the circumstances 
> of this request.

SCNR: https://unixsheikh.com/articles/the-delusions-of-debian.html

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Moving to Tomcat Native 1.3.x

[Michael Osipov]

On 2024/02/04 19:54:25 Mark Thomas wrote:
> Hi all,
> 
> AS you have probably noticed I am working on another round of Tomcat 
> Native releases.
> 
> We are overdue on switching to 1.3.x so I would like to propose the 
> following with this release round:
> 
> - create a new 1.3.x branch from the current 1.2.x HEAD
> - update minimum OpenSSL to 1.1.1
> - update minimum APR to 1.6.3
> - remove code supporting OpenSSL < 1.1.1

Good, you have basically picked up 
https://bz.apache.org/bugzilla/show_bug.cgi?id=67683.

You have missed at least two spots:
* 
https://github.com/apache/tomcat-native/blob/f6e1474c6b05e9cab0ad308647a3bde533c1cbce/native/build/tcnative.m4#L41-L45
* 
https://github.com/apache/tomcat-native/blob/f6e1474c6b05e9cab0ad308647a3bde533c1cbce/native/src/jnilib.c#L77-L81

> The next 8.5.x and 9.0.x releases would then ship with Tomcat Native 
> 1.3.0 but minimum required/recommended Tomcat Native versions would not 
> change.

I wouldn't bother with 8.5 and 1.3, I'd use 1.2.x until end of 8.5 and the put 
1.2.x EOL.

Overall sounds like a good plan. I will do also testing here on my side.

M

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Moving to Tomcat Native 1.3.x

[Michael Osipov]

> >> The next 8.5.x and 9.0.x releases would then ship with Tomcat Native
> >> 1.3.0 but minimum required/recommended Tomcat Native versions would not
> >> change.
> > 
> > I wouldn't bother with 8.5 and 1.3, I'd use 1.2.x until end of 8.5 and the 
> > put 1.2.x EOL.
> 
> I'm still leaning towards switching 8.5.x to 1.3.0 but if the consensus 
> is to stcik with the current 1.2.x release I'm fine with that. I'm 
> mainly trying to avoid another 1.2.x release on top of all the other 
> releases I am juggling at the moment.

I'd produce a last 1.2.x release to clean up/deliver the last changes from that 
branch to production, then the branch can be "closed".

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Moving to Tomcat Native 1.3.x

[Michael Osipov]

On 2024/02/04 19:54:25 Mark Thomas wrote:
> Hi all,
> 
> AS you have probably noticed I am working on another round of Tomcat 
> Native releases.
> 
> We are overdue on switching to 1.3.x so I would like to propose the 
> following with this release round:
> 
> - create a new 1.3.x branch from the current 1.2.x HEAD
> - update minimum OpenSSL to 1.1.1
> - update minimum APR to 1.6.3
> - remove code supporting OpenSSL < 1.1.1
> 
> The next 8.5.x and 9.0.x releases would then ship with Tomcat Native 
> 1.3.0 but minimum required/recommended Tomcat Native versions would not 
> change.

I have just tested Tomcat 9.0.x from Git repo against:
FreeBSD 13-STABLE:
OpenSSL 1.1.1w-freebsd  11 Sep 2023
Tomcat Native library [1.3.1-dev] using APR version [1.7.3]


HP-UX 11.31:
OpenSSL 1.1.1w  11 Sep 2023
Tomcat Native library [1.3.1-dev] using APR version [1.7.4]

I will try with OpenSSL 3.0.x soon. It is very unfortunate that 9.0.x requires 
Java 17 to build, it is not available on HP-UX and will never be by HPE. I had 
to downgrade BND to 6.4.0 to make it run. I still consider this a wrong move 
for at least Tomcat 9.0.x, Java 11 should have stayed the minimum.

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Moving to Tomcat Native 1.3.x

[Michael Osipov]

On 2024/02/07 18:19:24 Christopher Schultz wrote:
> Michael,
> 
> On 2/7/24 11:05, Michael Osipov wrote:
> > On 2024/02/04 19:54:25 Mark Thomas wrote:
> >> Hi all,
> >>
> >> AS you have probably noticed I am working on another round of Tomcat
> >> Native releases.
> >>
> >> We are overdue on switching to 1.3.x so I would like to propose the
> >> following with this release round:
> >>
> >> - create a new 1.3.x branch from the current 1.2.x HEAD
> >> - update minimum OpenSSL to 1.1.1
> >> - update minimum APR to 1.6.3
> >> - remove code supporting OpenSSL < 1.1.1
> >>
> >> The next 8.5.x and 9.0.x releases would then ship with Tomcat Native
> >> 1.3.0 but minimum required/recommended Tomcat Native versions would not
> >> change.
> > 
> > I have just tested Tomcat 9.0.x from Git repo against:
> > FreeBSD 13-STABLE:
> > OpenSSL 1.1.1w-freebsd  11 Sep 2023
> > Tomcat Native library [1.3.1-dev] using APR version [1.7.3]
> > 
> > 
> > HP-UX 11.31:
> > OpenSSL 1.1.1w  11 Sep 2023
> > Tomcat Native library [1.3.1-dev] using APR version [1.7.4]
> > 
> > I will try with OpenSSL 3.0.x soon. It is very unfortunate that 9.0.x 
> > requires Java 17 to build, it is not available on HP-UX and will never be 
> > by HPE. I had to downgrade BND to 6.4.0 to make it run. I still consider 
> > this a wrong move for at least Tomcat 9.0.x, Java 11 should have stayed the 
> > minimum.
> 
> I think it's actually possible to build with Java 11, but the release 
> builds require Java 17 for  #reasons.
> 
> Try just hacking the build files to allow Java 11 and see if you can build.

It does work:
diff --git a/build.properties.default b/build.properties.default
index 2ec1dbfb16..82aec7debb 100644
--- a/build.properties.default
+++ b/build.properties.default
@@ -307 +307 @@ 
spotbugs.loc=${base-maven.loc}/com/github/spotbugs/spotbugs/${spotbugs.version}/
-bnd.version=7.0.0
+bnd.version=6.4.0
diff --git a/build.xml b/build.xml
index 94e80620e2..83852a889f 100644
--- a/build.xml
+++ b/build.xml
@@ -110 +110 @@
-  
+  

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Moving to Tomcat Native 1.3.x

[Michael Osipov]

On 2024/02/07 16:05:17 Michael Osipov wrote:
> On 2024/02/04 19:54:25 Mark Thomas wrote:
> > Hi all,
> > 
> > AS you have probably noticed I am working on another round of Tomcat 
> > Native releases.
> > 
> > We are overdue on switching to 1.3.x so I would like to propose the 
> > following with this release round:
> > 
> > - create a new 1.3.x branch from the current 1.2.x HEAD
> > - update minimum OpenSSL to 1.1.1
> > - update minimum APR to 1.6.3
> > - remove code supporting OpenSSL < 1.1.1
> > 
> > The next 8.5.x and 9.0.x releases would then ship with Tomcat Native 
> > 1.3.0 but minimum required/recommended Tomcat Native versions would not 
> > change.
> 
> I have just tested Tomcat 9.0.x from Git repo against:
> FreeBSD 13-STABLE:
> OpenSSL 1.1.1w-freebsd  11 Sep 2023
> Tomcat Native library [1.3.1-dev] using APR version [1.7.3]
> 
> 
> HP-UX 11.31:
> OpenSSL 1.1.1w  11 Sep 2023
> Tomcat Native library [1.3.1-dev] using APR version [1.7.4]
> 
> I will try with OpenSSL 3.0.x soon. It is very unfortunate that 9.0.x 
> requires Java 17 to build, it is not available on HP-UX and will never be by 
> HPE. I had to downgrade BND to 6.4.0 to make it run. I still consider this a 
> wrong move for at least Tomcat 9.0.x, Java 11 should have stayed the minimum.

Both setups do also work with OpenSSL 3.0.13 30 Jan 2024

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Required build version(s)

[Michael Osipov]

I have complained about this many times...

On 2024/02/12 20:58:14 Christopher Schultz wrote:
> All,
> 
> The release managers have bumped-up their tool chains to Java 17 or 
> later for all supported releases. Tomcat 11 requires Java 21. Tomcat 
> 8.5.x only requires Java 11.
> 
> But the documented Java versions for each release are actually:
> 
> Tomcat 11 - Java 21
> Tomcat 10 - Java 11
> Tomcat  9 - Java 8
> Tomcat  8 - Java 7
> 
> Theoretically, anyone ought to be able to build Tomcat with the 
> minimum-required Java version for that branch. There are some practical 
> reasons why you can't build Tomcat 8.5.x with Java 7 but those have more 
> to do with supporting versions of Java *after* 7 than anything else.
> 
> I'd like to be able to allow anyone to build their Tomcat from source 
> using our tooling (e.g. ant-based build) and the minimum Java version 
> (with one exception: No support for Java 7 for builds).
> 
> Take Tomcat 9.0.x for example. The ant-based build will complain if you 
> are using Java 11 to build, because the build tools demand the use of 
> Java 17. If you simply change the "required version" from 17 to 11, the 
> build works perfectly fine.

No, it won't. I have written exactly this last week. BND is now at version 7.0 
which requires Java 17at runtime. I have downgraded manually to 6.4.0 to test 
on HP-UX with Tomcat Native 1.3.x.
For the sake of latest BND I don't see a strong reason to force Java 17, 
especially for those who want to test...

My overall opinion is to impose the oldest required version to build and test 
it. (not talking about release).

Michael

PS: Thank for raising, Chris!

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Release Managers wanted

[Michael Osipov]

Am 2021-05-13 um 00:10 schrieb Mark Thomas:

All,

Assuming 7.0.109 is the last Tomcat 7 release I am the current release 
manager for 10.0.x, 9.0.x, 8.5.x, migration tool, native and mod_jk. I'd 
like to share the load and the knowledge a little.


The step-by-step release process was documented for Tomcat 7:
https://cwiki.apache.org/confluence/display/TOMCAT/ReleaseProcess


Looking at the cwiki it does not apply to tcnative..Do you have 
instructions?
My long-term goal would be to provide a CMake-based generator, 
cross-platform.


M

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch main updated: Remove code deprecated in 10.1.x apart from the APR Endpoint

[Michael Osipov]

Am 2021-05-24 um 20:55 schrieb Mark Thomas:

On 24/05/2021 18:43, Rémy Maucherat wrote:

On Mon, May 24, 2021 at 6:34 PM  wrote:


This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/main by this push:
  new 620e06c  Remove code deprecated in 10.1.x apart from the APR
Endpoint
620e06c is described below

commit 620e06c468a02ca98dc4beacf556dd12d2440877
Author: Mark Thomas 
AuthorDate: Mon May 24 17:34:09 2021 +0100

 Remove code deprecated in 10.1.x apart from the APR Endpoint


Ah, it's *that* APR time again :)


Indeed. My plan was to give it a few days now folks have been reminded 
that this is going to happen and then do the APR removal in a separate 
commit.


I have an important fix for APR on Windows which affects also Tomcat as 
well. A very subtile one. Can you hold off for a moment? I will prepare 
a decent decription for dev@tomcat.a.o today and already have a suitable 
fix for it.


Michael


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch main updated: Remove code deprecated in 10.1.x apart from the APR Endpoint

[Michael Osipov]

Am 2021-05-25 um 09:44 schrieb Mark Thomas:

On 25/05/2021 08:27, Michael Osipov wrote:

Am 2021-05-24 um 20:55 schrieb Mark Thomas:

On 24/05/2021 18:43, Rémy Maucherat wrote:

On Mon, May 24, 2021 at 6:34 PM  wrote:


This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/main by this push:
  new 620e06c  Remove code deprecated in 10.1.x apart from the APR
Endpoint
620e06c is described below

commit 620e06c468a02ca98dc4beacf556dd12d2440877
Author: Mark Thomas 
AuthorDate: Mon May 24 17:34:09 2021 +0100

 Remove code deprecated in 10.1.x apart from the APR Endpoint


Ah, it's *that* APR time again :)


Indeed. My plan was to give it a few days now folks have been 
reminded that this is going to happen and then do the APR removal in 
a separate commit.


I have an important fix for APR on Windows which affects also Tomcat 
as well. A very subtile one. Can you hold off for a moment? I will 
prepare a decent decription for dev@tomcat.a.o today and already have 
a suitable fix for it.


What is the benefit of holding off on the removal of the APR endpoint 
from 10.1.x?


Just for clarification, 10.x on main will move to 10.1.x and APR will 
only run on 9 and 8.5?


M


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch main updated: Remove code deprecated in 10.1.x apart from the APR Endpoint

[Michael Osipov]

Am 2021-05-25 um 10:06 schrieb Mark Thomas:

On 25/05/2021 09:03, Michael Osipov wrote:

Am 2021-05-25 um 09:44 schrieb Mark Thomas:

On 25/05/2021 08:27, Michael Osipov wrote:

Am 2021-05-24 um 20:55 schrieb Mark Thomas:

On 24/05/2021 18:43, Rémy Maucherat wrote:

On Mon, May 24, 2021 at 6:34 PM  wrote:


This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/main by this push:
  new 620e06c  Remove code deprecated in 10.1.x apart from 
the APR

Endpoint
620e06c is described below

commit 620e06c468a02ca98dc4beacf556dd12d2440877
Author: Mark Thomas 
AuthorDate: Mon May 24 17:34:09 2021 +0100

 Remove code deprecated in 10.1.x apart from the APR Endpoint


Ah, it's *that* APR time again :)


Indeed. My plan was to give it a few days now folks have been 
reminded that this is going to happen and then do the APR removal 
in a separate commit.


I have an important fix for APR on Windows which affects also Tomcat 
as well. A very subtile one. Can you hold off for a moment? I will 
prepare a decent decription for dev@tomcat.a.o today and already 
have a suitable fix for it.


What is the benefit of holding off on the removal of the APR endpoint 
from 10.1.x?


Just for clarification, 10.x on main will move to 10.1.x and APR will 
only run on 9 and 8.5?


Correct.

10.1.x will continue to support the use of the Tomcat Native Connector 
(i.e. the native library) to provide OpenSSL support for NIO and NIO2. 
The plan is that version 2 of that library will be significantly reduced 
in scope to only provide the functionality necessary to hook into OpenSSL.


Alright, please decide yourself whether this should go into 10.x [1] for 
those who want branch off/retain APR here. Otherwise I will provide a 
polished patch for < 10.


Michael

[1] 
https://github.com/michael-o/tomcat/commit/5393bcb991b924a18169ba34264ec2bc5c009b22



-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

APR connector on Windows 8+ fails to listen on all addresses

[Michael Osipov]

Folks,

we needed to deploy Tomcat 9.0.x on a Windows server (no jokes, please), 
but the contractor wasn't able to configure the APR connector to accept 
on external interfaces even after a day.
After my analysis it turned out be a subtile bug in libapr which affects 
Windows users only. I am also surprised why no one complained before.


Setup:
* Windows 8+ or Windows Server 2016/2019
* Have at least IPv6 available, no IP addresses necessary, ::1 is sufficient
* Any Tomcat with libtcnative 1.2.28 with the DLL compiled by Mark Thomas.
* Start Tomcat with the AprLifecycleListener and make sure that no 
address (hostname) is set.


To make a long investigation story short:
libapr, thus libtcnative suffer from a very subtile bug only visible
on dual-stack systems. Since on INET6 sockets IPV6_V6ONLY is 1 by 
default on Windows, no IPv4 addresses are bound. In the case above, 
Tomcat is only accessible on ::1. APR is supposed to set IPV6_V6ONLY to 
0 by default, but this fails because APR 1.7.x does not recognise 
anything above Windows 7 and assumes it to be Windows XP by default. As 
you might know Vista was the first Windows with true IPv6 an 
dual-sockets. When setsockopt is invoked APR gives you 70023, not 
implemented.


I was able, according to Mark's instructions, to compile OpenSSL, APR 
and Tomcat Native on Windows 10 and deploy on Windows Server 2019.

I'd like to push
* https://github.com/michael-o/tomcat/compare/main...clean-bind
* https://github.com/michael-o/tomcat-native/compare/main...clean-bind
as well as the real fix in APR 1.7.x: 
https://github.com/michael-o/apr/compare/1.7.x...1.7.x-windows


I ran all unit tests (main) with those modifications on these platforms:
* Windows 10, APR 1.7.0, 1.7.1-dev
* Windows Server 2019, APR 1.7.0, 1.7.1-dev
* FreeBSD 12-STABLE, APR 1.7.0, 1.7.1-dev
* RHEL 7, APR 1.4.8
* HP-UX 11i, APR 1.6.6

Some hosts are dual-stack, some IPv4 only. Moreover, I wrote a simple 
program which binds the socket for tracing only: 
https://gist.github.com/michael-o/dfb86df472f62d2b2dff6ef12ee3758e
It runs as expected on the above platforms, even with zone id on 
link-local addresses.


If no one objects, I'll merge soon.

Mark, I don't know when the next APR release will happen, but I consider 
this to be very annoying. Maybe it makes sense to push 1.2.29 with APR 
1.7.1-dev to please Windows users?


Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Renamings with APR removal in 10.1.x

[Michael Osipov]

Mark,

since you are going to remove all bits soon, will you rename Java items 
with still carry the APR substring? E.g., AprLifecycleListener will be a 
misleading name for obvious reasons.


Michael

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Renamings with APR removal in 10.1.x

[Michael Osipov]

Am 2021-05-26 um 15:29 schrieb Mark Thomas:

On 25/05/2021 17:27, Michael Osipov wrote:

Mark,

since you are going to remove all bits soon, will you rename Java 
items with still carry the APR substring? E.g., AprLifecycleListener 
will be a misleading name for obvious reasons.


I hadn't considered that but it seems like a good opportunity to remove 
some of the confusion that that naming has caused. I think a switch to 
TomcatNativeLifecylcleListener would work.


My expectation is that in the early milestone releases for 10.1.x that 
the listener looks for Tomcat Native 1.2.x but at some point before the 
first stable release it switches to Tomcat Native 2.0.x. That assumes we 
find the time to create Tomcat Native 2.0.x. Given that it will mostly 
be deleting code, that should be doable.


Have also a look at build.xml: test.apr.loc should also be renamed to 
test.tomcat-native.loc.


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: APR connector on Windows 8+ fails to listen on all addresses

[Michael Osipov]

Am 2021-05-26 um 15:25 schrieb Mark Thomas:

On 25/05/2021 17:23, Michael Osipov wrote:



Nice research.

Mark, I don't know when the next APR release will happen, but I 
consider this to be very annoying. Maybe it makes sense to push 1.2.29 
with APR 1.7.1-dev to please Windows users?


I'd prefer to use a released version of APR but failing that we can 
provide a patch against 1.7.0. If you can add the appropriate patch here:

https://github.com/apache/tomcat-native/tree/main/native/srclib/apr

I can include it in the next release.


The change has been committed to apr-1.7.x, here [1] is the changeset, 
compiled for me nicely several times. Would be good if you could also 
check at your end.

I will inquire on dev@apr.a.o if someone is willing to perform 1.7.1 soon.

We are close to starting the June releases so if we move quickly on a 
Tomcat-Native release now, we can included a fixed version of Tomcat 
Native in the next set of Tomcat releases.


I have also applied by changes (simplifcations) to Tomcat Native, but 
Mladen objected [2] and asked me to move to a PR to discuss [3], but I 
am still waiting for an explanation of his objection.
The purpose of the change was to simplify and have consistency with the 
Java code where we don't do such things.

Hopefully we can setttle this before 1.2.29.

M

[1] 
https://github.com/apache/apr/commit/2bcd4b3ddb108d16f1c758c00a45de9aef57aa3a
[2] 
https://github.com/apache/tomcat-native/commit/420cd1c159e4f27bb5d2a873dbd1fb7ea5d3473c

[3] https://github.com/apache/tomcat-native/pull/9

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: APR connector on Windows 8+ fails to listen on all addresses

[Michael Osipov]

Am 2021-05-27 um 07:39 schrieb Mladen Turk:

APR has to be compiled with -D_WIN32_WINNT=0x0600 -DWINVER=0x0600
Same with Tomcat Native

Use apr-1.6.x


Both don't work out for the following reasons:
* Your setenv.bat sets WINVER=0x0601 which fails for the NMakefile which 
in turn requires a symbolic name instead of a hex value, I have to pass 
WINVER=WIN7 anyway, then the rest works.
* Even if I do, the check in APR is performed at runtime for the OS 
version, therefore setting the socket option fails.
* How is 1.6.x going me to help here if it does not now anything about 
Windows 8, 10?


This [1] is what you need to make it work.

Kindly respond to my PR.

M

[1] 
https://github.com/apache/apr/commit/2bcd4b3ddb108d16f1c758c00a45de9aef57aa3a



On 25/05/2021 18:23, Michael Osipov wrote:

Folks,

we needed to deploy Tomcat 9.0.x on a Windows server (no jokes, 
please), but the contractor wasn't able to configure the APR connector 
to accept on external interfaces even after a day.
After my analysis it turned out be a subtile bug in libapr which 
affects Windows users only. I am also surprised why no one complained 
before.


Setup:
* Windows 8+ or Windows Server 2016/2019
* Have at least IPv6 available, no IP addresses necessary, ::1 is 
sufficient
* Any Tomcat with libtcnative 1.2.28 with the DLL compiled by Mark 
Thomas.
* Start Tomcat with the AprLifecycleListener and make sure that no 
address (hostname) is set.


To make a long investigation story 
short:http://svn.apache.org/viewvc?rev=1889037&view=rev

libapr, thus libtcnative suffer from a very subtile bug only visible
on dual-stack systems. Since on INET6 sockets IPV6_V6ONLY is 1 by 
default on Windows, no IPv4 addresses are bound. In the case above, 
Tomcat is only accessible on ::1. APR is supposed to set 
IPV6_V6ONLhttp://svn.apache.org/viewvc?rev=1889037&view=revY to 0 by 
default, but this fails because APR 1.7.x does not recognise anything 
above Windows 7 and assumes it to be Windows XP by default. As you 
might know Vista was the first Windows with true IPv6 an dual-sockets. 
When setsockopt is invoked APR gives you 70023, not implemented.


I was able, according to Mark's instructions, to compile OpenSSL, APR 
and Tomcat Native on Windows 10 and deploy on Windows Server 2019.

I'd like to push
* https://github.com/michael-o/tomcat/compare/main...clean-bind
* https://github.com/michael-o/tomcat-native/compare/main...clean-bind
as well as the real fix in APR 1.7.x: 
https://github.com/michael-o/apr/compare/1.7.x...1.7.x-windows


I ran all unit tests (main) with those modifications on these platforms:
* Windows 10, APR 1.7.0, 1.7.1-dev
* Windows Server 2019, APR 1.7.0, 1.7.1-dev
* FreeBSD 12-STABLE, APR 1.7.0, 1.7.1-dev
* RHEL 7, APR 1.4.8
* HP-UX 11i, APR 1.6.6

Some hosts are dual-stack, some IPv4 only. Moreover, I wrote a simple 
program which binds the socket for tracing only: 
https://gist.github.com/michael-o/dfb86df472f62d2b2dff6ef12ee3758e
It runs as expected on the above platforms, even with zone id on 
link-local addresses.


If no one objects, I'll merge soon.

Mark, I don't know when the next APR release will happen, but I 
consider this to be very annoying. Maybe it makes sense to push 1.2.29 
with APR 1.7.1-dev to please Windows users?


Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org






-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: APR connector on Windows 8+ fails to listen on all addresses

[Michael Osipov]

Am 2021-05-27 um 11:52 schrieb Mark Thomas:

Michael,

I think we need to step back a bit.

I am unable to recreate the issue you describe. I am using:

- Windows Server 2019
- IPv6 enabled (and IPv4 and both have addresses)
- Apache Tomcat 9.0.46
- Apache Tomcat Native 1.2.28
- AprLifecycleListener is enabled
- No address configured for Connector
- Logs confirm http-apr-8080

I can access the default home page via both IPv4 and IPv6 from both the 
local machine and remotely.




On 25/05/2021 18:23, Michael Osipov wrote: >>> * Windows 8+ or 
Windows Server 2016/2019
* Have at least IPv6 available, no IP addresses necessary, ::1 is 
sufficient
* Any Tomcat with libtcnative 1.2.28 with the DLL compiled by Mark 
Thomas.
* Start Tomcat with the AprLifecycleListener and make sure that no 
address (hostname) is set.


I think I have replicated the above but obviously I have missed 
something. What do I need to do to recreate your results?


Please try my simple program from this gist: 
https://gist.github.com/michael-o/dfb86df472f62d2b2dff6ef12ee3758e


Vanilla DLL on Windows 10:

PS Z:\> java "-cp" 
".;C:\Entwicklung\Projekte\tomcat-native\dist\tomcat-native-1.2.29-dev.jar" 
"-Djava.library.path=C:\Entwicklung\Programme\apache-tomcat-9.0.46\bin" AprTest
libtcnative version: 1.2.28, libapr version 1.7.0
hostname: null, family: 2, next: 522994096
setsockopt IPV6_V6ONLY status: 70023
socket bind status: 0
socket listen status: 0


An IPv6 only socket is bound because default is 1 on the socket option 
and one cannot set to 0:

PS C:\Users\osipovmi> netstat -a  | Select-String -Pattern 

  TCP[::]:  md2pcvtc:0 ABHÖREN



then compiled with patches:

PS Z:\> java "-cp" 
".;C:\Entwicklung\Projekte\tomcat-native\dist\tomcat-native-1.2.29-dev.jar" 
"-Djava.library.path=C:\Entwicklung\Projekte\tomcat-native\native\WIN7_X64_DLL_RELEASE" AprTest
libtcnative version: 1.2.29-dev, libapr version 1.7.1-dev
hostname: null, family: 2, next: 523321776
setsockopt IPV6_V6ONLY status: 0
socket bind status: 0
socket listen status: 0


An IPv6 socket with dual-stack has been bound:

PS C:\Users\osipovmi> netstat -a  | Select-String -Pattern 

  TCP0.0.0.0:   md2pcvtc:0 ABHÖREN
  TCP[::]:  md2pcvtc:0 ABHÖREN



Please note that I have IPv6 enabled, but no IP addresses configured. By 
default only ::1 is present.


Addresses:

PS C:\Users\osipovmi> Get-NetIPAddress | Format-Table

ifIndex IPAddress   PrefixLength 
PrefixOrigin SuffixOrigin AddressState PolicyStore
--- -    
   ---
1   ::1  128 WellKnown  
  WellKnownPreferredActiveStore
20  169.254.228.218   16 WellKnown  
  Link TentativeActiveStore
32  192.168.58.19328 Manual 
  Manual   PreferredActiveStore
15  169.254.26.74 16 WellKnown  
  Link TentativeActiveStore
18  169.254.76.82 16 WellKnown  
  Link TentativeActiveStore
22  169.254.155.142   16 WellKnown  
  Link TentativeActiveStore
21  192.168.1.23  24 Dhcp   
  Dhcp PreferredActiveStore
1   127.0.0.1  8 WellKnown  
  WellKnownPreferredActiveStore


Does this help?

M


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: APR connector on Windows 8+ fails to listen on all addresses

[Michael Osipov]

Prefences in the OS:

PS C:\Users\osipovmi> netsh interface ipv6 show prefixpolicies
Der aktive Status wird abgefragt...

Vorgänger   Label  Präfix
--  -  
50  0  :::0:0/96
40  1  ::1/128
30  2  ::/0
20  3  2002::/16
 5  5  2001::/32
 3 13  fc00::/7
 1 11  fec0::/10
 1 12  3ffe::/16
 1  4  ::/96


Taken from https://superuser.com/a/436944/222550



-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: APR connector on Windows 8+ fails to listen on all addresses

[Michael Osipov]

Am 2021-05-27 um 15:41 schrieb Mark Thomas:

On 27/05/2021 13:24, Mark Thomas wrote:

On 27/05/2021 11:12, Michael Osipov wrote:

Am 2021-05-27 um 11:52 schrieb Mark Thomas:

Michael,

I think we need to step back a bit.

I am unable to recreate the issue you describe. I am using:

- Windows Server 2019
- IPv6 enabled (and IPv4 and both have addresses)
- Apache Tomcat 9.0.46
- Apache Tomcat Native 1.2.28
- AprLifecycleListener is enabled
- No address configured for Connector
- Logs confirm http-apr-8080

I can access the default home page via both IPv4 and IPv6 from both 
the local machine and remotely.




Please try my simple program from this gist: 
https://gist.github.com/michael-o/dfb86df472f62d2b2dff6ef12ee3758e





Does this help?


Thanks. That helped me narrow down where the differences where. If you 
run Tomcat as a service it binds both 0.0.0.0:8080 and [::]:8080. If 
you run it from the command line, it only binds [::]:8080.


I'll take a look at the proposed APR patch next.


I can confirm that the patch to APR fixes the issue for me too.

With the original 1.2.28 DLL, starting Tomcat on the command line only 
binds to IPv6.


With a patched 1.2.29-dev DLL, starting Tomcat on the command line binds 
to IPv6 and IPv4.


Pefect!


I'll tag and start the release process for Tomcat Native shortly.


Before you do, I'd like to settle 
https://github.com/apache/tomcat-native/pull/9


This aligns with the NIO connectors for obtaining addresses by relying 
on the OS. Please have a look and share your opinion.


M

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat Native 1.2.30

[Michael Osipov]

Am 2021-06-01 um 11:53 schrieb Mark Thomas:

Resending with correct subject line...

Version 1.2.30 includes the following changes compared to 1.2.28

- Fix an issue where some Windows systems in some configurations would
   only listen on IPv6 addresses on dual stack systems even though
   configured to listen on both IPv6 and IPv4 addresses.

- Complete the fix for BZ 65181

- Correct constants used for Windows versions. Expand versions
   supported.

The proposed release artefacts can be found at [1],
and the build was done using tag [2].

The Apache Tomcat Native 1.2.30 release is
  [ ] Stable, go ahead and release
  [ ] Broken because of ...



+1


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch main updated: Portable temp path code

[Michael Osipov]

Am 2021-06-30 um 17:45 schrieb r...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/main by this push:
  new d2ef158  Portable temp path code
d2ef158 is described below

commit d2ef158945a4aca00dc06900c65c059cbb66c410
Author: remm 
AuthorDate: Wed Jun 30 17:45:20 2021 +0200

 Portable temp path code
---
  test/org/apache/tomcat/util/net/TestXxxEndpoint.java | 4 +++-
  1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/test/org/apache/tomcat/util/net/TestXxxEndpoint.java 
b/test/org/apache/tomcat/util/net/TestXxxEndpoint.java
index 653e751..efc4dd1 100644
--- a/test/org/apache/tomcat/util/net/TestXxxEndpoint.java
+++ b/test/org/apache/tomcat/util/net/TestXxxEndpoint.java
@@ -213,7 +213,9 @@ public class TestXxxEndpoint extends TomcatBaseTest {
  c.getProtocolHandlerClassName().contains("NioProtocol")
  && JreCompat.isJre16Available());
  
-final String unixDomainSocketPath = "/tmp/testUnixDomainSocket";

+File tempPath = File.createTempFile("tomcat", ".uds");


The common extension for UDS is actually '.sock'

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] 01/04: Add connection pool to JNDI realm

[Michael Osipov]

Am 2020-10-07 um 22:34 schrieb r...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git

commit 50de36b7874da98591345e40b374a1e2dd52c188
Author: remm 
AuthorDate: Thu Jan 30 17:22:51 2020 +0100

 Add connection pool to JNDI realm
 
 This implements a TODO from the class javadoc header.

 As described in the javadoc, the idea is to use a pool to avoid blocking
 on a single connection, which could possibly become a bottleneck in some
 cases. The message formats need to be kept along with the connection
 since they are not thread safe.
 Preserve the default behavior: sync without pooling (using a Lock object
 which is more flexible).
 I may backport this since this is limited to the JNDI realm, but only
 once it is confirmed to be regression free. Tested with ApacheDS but my
 LDAP skills are very limited.
---
  java/org/apache/catalina/realm/JNDIRealm.java  | 442 -
  .../apache/catalina/realm/LocalStrings.properties  |   1 +
  test/org/apache/catalina/realm/TestJNDIRealm.java  |   7 +-
  webapps/docs/changelog.xml |   3 +
  webapps/docs/config/realm.xml  |   7 +
  5 files changed, 276 insertions(+), 184 deletions(-)


Salut Rémy,

this is a very very nice improvement to the matter and I gave your idea 
a spin my public Active Directory realm. Based on my tests with AD I see 
room for improvement: (Note that I don't use the JNDIRealm because 
because is it too generic and usable for me)


* You might want to consider to introduce a maxIdleTime to avoid stale 
connections, e.g., AD has default value of 15 minutes. Yep, I had had 
several RSTs resulting in 401s
* Validating connections optionally might be a good option to detect 
stale connections or broken ones ny networks issues. This works for me 
very fast:

protected boolean validate(DirContextConnection connection) {
if (logger.isDebugEnabled())

logger.debug(sm.getString("activeDirectoryRealm.validate"));

SearchControls controls = new SearchControls();
controls.setSearchScope(SearchControls.OBJECT_SCOPE);
controls.setCountLimit(1);
controls.setReturningAttributes(new String[] { "objectClass" });
controls.setTimeLimit(500);

try {
NamingEnumeration results = 
connection.context.search("", "objectclass=*",
controls);

if (results.hasMore()) {
close(results);
return true;
}
} catch (NamingException e) {

logger.error(sm.getString("activeDirectoryRealm.validate.namingException"), e);

return false;
}

return false;
}

I do this in the acquire() method

* The acquire(), authenticate(), fail, close(), authenticate(), 
release() is brittle and proved to fail here. I have a curl script which 
does 4 requests in parallel and 1200 requests in total. Pool size is 8. 
4 connections in the pool. Rest for 16 minutes, connections are broken 
now. Start requests. authenticate() will fail twice because top two 
acquired connections from pool are broken. Requests end with 401. I'd 
prefer to to modify acquire() is such a fashion:

protected DirContextConnection acquire() {
if (logger.isDebugEnabled())

logger.debug(sm.getString("activeDirectoryRealm.acquire"));

DirContextConnection connection = null;

while (connection == null) {
connection = connectionPool.pop();

if (connection != null) {
long idleTime = System.currentTimeMillis() - 
connection.lastBorrowTime;
if (idleTime > maxIdleTime) {
if (logger.isDebugEnabled())

logger.debug(sm.getString("activeDirectoryRealm.exceedMaxIdleTime"));
close(connection);
connection = null;
} else {
boolean valid = validate(connection);
if (valid) {
if (logger.isDebugEnabled())

logger.debug(sm.getString("activeDirectoryRealm.reuse"));
} else {
close(connection);
connection = null

[DISCUSS] Deprecate and remove RealmBase#stripRealmForGss

[Michael Osipov]

Folks,

I'd like to propose to get rid of that config option in 10 and deprecate 
in previous versions for the following reasons:


* It suffers from abstraction: It assumes that the GSS name is always 
email style w/o checking its OID
* The realm part, if any, is an integeral part of the principal. Much 
like with an email address' domain. You wouldn't stip here too, would you?
* It is a surprise for clients having the princippal mutilated by 
default. I trip over and over again this when I set up 
UserDatabaseRealms for testing purposes I wonder why 
michae...@example.com does not work.
* In a multi realm environment, it is perfectly fine and valid to have 
user1@REALMA and user1@REALMB. These are distinct principals, but 
treated by RealmBase equally, this has implications.
* Finally, when doing cert-based auth in an AD envinronment, is it 
pretty common to extract the msUPN name from the cert's SAN which is 
almost always email address (enteprise principal) which would end up in 
michael.osipov, but where is the rest?!


Thoughts?

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] 01/04: Add connection pool to JNDI realm

[Michael Osipov]

Am 2020-10-13 um 13:49 schrieb Rémy Maucherat:

On Tue, Oct 13, 2020 at 11:33 AM Michael Osipov  wrote:


Am 2020-10-07 um 22:34 schrieb r...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git

commit 50de36b7874da98591345e40b374a1e2dd52c188
Author: remm 
AuthorDate: Thu Jan 30 17:22:51 2020 +0100

  Add connection pool to JNDI realm

  This implements a TODO from the class javadoc header.
  As described in the javadoc, the idea is to use a pool to avoid

blocking

  on a single connection, which could possibly become a bottleneck in

some

  cases. The message formats need to be kept along with the connection
  since they are not thread safe.
  Preserve the default behavior: sync without pooling (using a Lock

object

  which is more flexible).
  I may backport this since this is limited to the JNDI realm, but

only

  once it is confirmed to be regression free. Tested with ApacheDS

but my

  LDAP skills are very limited.
---
   java/org/apache/catalina/realm/JNDIRealm.java  | 442

-

   .../apache/catalina/realm/LocalStrings.properties  |   1 +
   test/org/apache/catalina/realm/TestJNDIRealm.java  |   7 +-
   webapps/docs/changelog.xml |   3 +
   webapps/docs/config/realm.xml  |   7 +
   5 files changed, 276 insertions(+), 184 deletions(-)


Salut Rémy,

this is a very very nice improvement to the matter and I gave your idea
a spin my public Active Directory realm. Based on my tests with AD I see
room for improvement: (Note that I don't use the JNDIRealm because
because is it too generic and usable for me)



Ok, I only tested this with a local ldap, so not much. I verified it was
pooling connections.




* You might want to consider to introduce a maxIdleTime to avoid stale
connections, e.g., AD has default value of 15 minutes. Yep, I had had
several RSTs resulting in 401s



Oops. So I don't think adding something like this helps since there could
be plenty of issues besides a timeout. Basically it's supposed to retry
instead when something fails. Can you give me a stack trace to show what's
wrong ?


I consider it cheaper to test for a timeout and close actively rather 
than recover from a resetted connection. Here is a strack trace for 
concurrency of 4 (with a pool of max 8). Compare the timestamp of the 
first line and the subsequent ones:



2020-10-12T22:55:47 FEIN [https-openssl-apr-8444-exec-8] 
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.release Releasing directory 
server connection to pool
2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-5] 
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Acquiring directory 
server connection from pool
2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-5] 
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.validate Validating directory 
server connection from pool
2020-10-12T23:13:06 SCHWERWIEGEND [https-openssl-apr-8444-exec-5] 
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.validate Exception validating 
directory server connection
javax.naming.CommunicationException: Connection reset [Root exception is 
java.net.SocketException: Connection reset]; remaining name ''
at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:2030)
at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1872)
at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1797)
at 
com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(ComponentDirContext.java:392)
at 
com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:358)
at 
com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:341)
at 
javax.naming.directory.InitialDirContext.search(InitialDirContext.java:296)
at 
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.validate(ActiveDirectoryRealm.java:316)
at 
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire(ActiveDirectoryRealm.java:285)
at 
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.getPrincipal(ActiveDirectoryRealm.java:244)
at org.apache.catalina.realm.RealmBase.authenticate(RealmBase.java:501)
at 
net.sf.michaelo.tomcat.authenticator.SpnegoAuthenticator.doAuthenticate(SpnegoAuthenticator.java:165)
at 
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:633)
at 
org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:571)
at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at 
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
at 
org.apache.catalina.c

Re: [tomcat] branch 8.5.x updated: Always retry on a new connection, even when pooling

[Michael Osipov]

Am 2020-10-13 um 16:05 schrieb r...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/8.5.x by this push:
  new 883600b  Always retry on a new connection, even when pooling
883600b is described below

commit 883600b8a77c0be93b3a8dc2404e8d1110970ee7
Author: remm 
AuthorDate: Tue Oct 13 14:19:54 2020 +0200

 Always retry on a new connection, even when pooling
 
 This keeps the same very simple design as for the single connection

 scenario, for now.
---
  java/org/apache/catalina/realm/JNDIRealm.java | 22 +++---
  webapps/docs/changelog.xml|  5 +
  2 files changed, 24 insertions(+), 3 deletions(-)

diff --git a/java/org/apache/catalina/realm/JNDIRealm.java 
b/java/org/apache/catalina/realm/JNDIRealm.java
index 72087ab..98007f7 100644
--- a/java/org/apache/catalina/realm/JNDIRealm.java
+++ b/java/org/apache/catalina/realm/JNDIRealm.java
@@ -1311,7 +1311,7 @@ public class JNDIRealm extends RealmBase {
  close(connection);
  
  // open a new directory context.

-connection = get();
+connection = get(true);
  
  // Try the authentication again.

  principal = authenticate(connection, username, credentials);
@@ -2389,12 +2389,28 @@ public class JNDIRealm extends RealmBase {
   * @exception NamingException if a directory server error occurs
   */
  protected JNDIConnection get() throws NamingException {
+return get(false);
+}
+
+/**
+ * Open (if necessary) and return a connection to the configured
+ * directory server for this Realm.
+ * @param create when pooling, this forces creation of a new connection,
+ *   for example after an error
+ * @return the connection
+ * @exception NamingException if a directory server error occurs
+ */
+protected JNDIConnection get(boolean create) throws NamingException {
  JNDIConnection connection = null;
  // Use the pool if available, otherwise use the single connection
  if (connectionPool != null) {
-connection = connectionPool.pop();
-if (connection == null) {
+if (create) {
  connection = create();
+} else {
+connection = connectionPool.pop();
+if (connection == null) {
+connection = create();
+}
  }
  } else {
  singleConnectionLock.lock();


That suitable and simple approach.

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [DISCUSS] Deprecate and remove RealmBase#stripRealmForGss

[Michael Osipov]

Am 2020-10-13 um 12:32 schrieb Mark Thomas:

On 13/10/2020 10:48, Michael Osipov wrote:

Folks,

I'd like to propose to get rid of that config option in 10 and deprecate
in previous versions for the following reasons:

* It suffers from abstraction: It assumes that the GSS name is always
email style w/o checking its OID
* The realm part, if any, is an integeral part of the principal. Much
like with an email address' domain. You wouldn't stip here too, would you?
* It is a surprise for clients having the princippal mutilated by
default. I trip over and over again this when I set up
UserDatabaseRealms for testing purposes I wonder why
michae...@example.com does not work.
* In a multi realm environment, it is perfectly fine and valid to have
user1@REALMA and user1@REALMB. These are distinct principals, but
treated by RealmBase equally, this has implications.
* Finally, when doing cert-based auth in an AD envinronment, is it
pretty common to extract the msUPN name from the cert's SAN which is
almost always email address (enteprise principal) which would end up in
michael.osipov, but where is the rest?!

Thoughts?


No objections to changing the default to false in 10.0.x but I think
removal/deprecation is a step too far.


OK


Not everyone uses the full u...@bigco.com format as the "primary key" in
their internal user database. I see lots of variation depending on which
system is viewed as the source of truth to which other systems are
synchronised.


So you will never be able to satisfy all, why try?


We have X509UsernameRetriever for certificate authentication. That came
out of bug 52500 where the aim was to support a wide variety of formats.
The root of that requirement was essentially the same - integration with
a variety of systems where the user name could be in in a range of
formats. The mapping of that format to an X509 cert just added another
layer of complexity.


I don't think that this compares (well( because a certificate is a 
complex object explicitly containing SAN which contain wellknown forms 
for an entity whereas a GSS name is merely a string with an identifier 
for its format. Your argumentation could also be applied to an LDAP 
bind. Active Directory, for instance, accepts a various amounts of input 
formats for the same entity, yet we don't provide a logic to normalize 
them in the JNDI Realm. That seems to be unfair to me.

It lacks a bit of consistency.


If there was user demand for it (I haven't seen any) I'd support a
similar interface for SPNEGO. With such an interface in place,
deprecation and removal of stripRealmForGss would make sense.


See above.

Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch 8.5.x updated: Always retry on a new connection, even when pooling

[Michael Osipov]

Am 2020-10-14 um 12:32 schrieb Rémy Maucherat:

On Tue, Oct 13, 2020 at 8:27 PM Michael Osipov  wrote:


Am 2020-10-13 um 16:05 schrieb r...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/8.5.x by this push:
   new 883600b  Always retry on a new connection, even when pooling
883600b is described below

commit 883600b8a77c0be93b3a8dc2404e8d1110970ee7
Author: remm 
AuthorDate: Tue Oct 13 14:19:54 2020 +0200

  Always retry on a new connection, even when pooling

  This keeps the same very simple design as for the single connection
  scenario, for now.
---
   java/org/apache/catalina/realm/JNDIRealm.java | 22

+++---

   webapps/docs/changelog.xml|  5 +
   2 files changed, 24 insertions(+), 3 deletions(-)

diff --git a/java/org/apache/catalina/realm/JNDIRealm.java

b/java/org/apache/catalina/realm/JNDIRealm.java

index 72087ab..98007f7 100644
--- a/java/org/apache/catalina/realm/JNDIRealm.java
+++ b/java/org/apache/catalina/realm/JNDIRealm.java
@@ -1311,7 +1311,7 @@ public class JNDIRealm extends RealmBase {
   close(connection);

   // open a new directory context.
-connection = get();
+connection = get(true);

   // Try the authentication again.
   principal = authenticate(connection, username,

credentials);

@@ -2389,12 +2389,28 @@ public class JNDIRealm extends RealmBase {
* @exception NamingException if a directory server error occurs
*/
   protected JNDIConnection get() throws NamingException {
+return get(false);
+}
+
+/**
+ * Open (if necessary) and return a connection to the configured
+ * directory server for this Realm.
+ * @param create when pooling, this forces creation of a new

connection,

+ *   for example after an error
+ * @return the connection
+ * @exception NamingException if a directory server error occurs
+ */
+protected JNDIConnection get(boolean create) throws NamingException

{

   JNDIConnection connection = null;
   // Use the pool if available, otherwise use the single

connection

   if (connectionPool != null) {
-connection = connectionPool.pop();
-if (connection == null) {
+if (create) {
   connection = create();
+} else {
+connection = connectionPool.pop();
+if (connection == null) {
+connection = create();
+}
   }
   } else {
   singleConnectionLock.lock();


That suitable and simple approach.



If you have the code for adding a max idle check on hand and tested, you
should add it IMO, it will be more efficient.


I will need to give this a couple more weeks of testing. This is what I 
have observed today:

2020-10-14T16:01:47.039 147.54.155.198 w99sezx0... "GET 
/x2tc-proxy-bln/rest/info/targetSystem HTTP/1.1" 200 8 92132

20609 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166] 
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Acquiring directory 
server connection from pool
20610 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166] 
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Directory server 
connection from pool exceeds max idle time, closing it
20611 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166] 
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.close Closing directory 
server connection
20612 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166] 
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.open Opening new directory 
server connection
20613 2020-10-14T16:01:47 FEIN [https-openssl-apr-8444-exec-166] 
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.getUser Searching for 
username 'w99sezx0' in base ...


As you can see it took 90 seconds to server the request because the 
connection has expired and close took way too long. In average the 
request takes:

2020-10-14T13:57:06.730 10.81.50.232 osipovmi@... "GET 
/x2tc-proxy-bln/rest/info/targetSystem HTTP/1.1" 200 8 70


when the connection is healthy.

Michael


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch master updated: Add extended ErrorReportValve that returns response as JSON instead of HTML

[Michael Osipov]

Am 2020-10-14 um 16:55 schrieb kfuj...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

kfujino pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
  new 42a0841  Add extended ErrorReportValve that returns response as JSON 
instead of HTML
42a0841 is described below

commit 42a0841d289a35117da842201622454a9f387cd7
Author: KeiichiFujino 
AuthorDate: Wed Oct 14 23:54:22 2020 +0900

 Add extended ErrorReportValve that returns response as JSON instead of HTML


Why not properly negotiate the content type? It does not seems that you 
aren't returning the same content the HTML version does. Moreover, why 
don't you use the print writer from the response?


Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: Re: [tomcat] branch 8.5.x updated: Always retry on a new connection, even when pooling

[Michael Osipov]

> On Wed, Oct 14, 2020 at 6:32 PM Michael Osipov  wrote:
>
> > Am 2020-10-14 um 12:32 schrieb R=C3=A9my Maucherat:
> > > On Tue, Oct 13, 2020 at 8:27 PM Michael Osipov 
> > wrote:
> > >
> > >> Am 2020-10-13 um 16:05 schrieb r...@apache.org:
> > >>> This is an automated email from the ASF dual-hosted git repository.
> > >>>
> > >>> remm pushed a commit to branch 8.5.x
> > >>> in repository https://gitbox.apache.org/repos/asf/tomcat.git
> > >>>
> > >>>
> > >>> The following commit(s) were added to refs/heads/8.5.x by this push:
> > >>>new 883600b  Always retry on a new connection, even when pooli=
> ng
> > >>> 883600b is described below
> > >>>
> > >>> commit 883600b8a77c0be93b3a8dc2404e8d1110970ee7
> > >>> Author: remm 
> > >>> AuthorDate: Tue Oct 13 14:19:54 2020 +0200
> > >>>
> > >>>   Always retry on a new connection, even when pooling
> > >>>
> > >>>   This keeps the same very simple design as for the single
> > connection
> > >>>   scenario, for now.
> > >>> ---
> > >>>java/org/apache/catalina/realm/JNDIRealm.java | 22
> > >> +++---
> > >>>webapps/docs/changelog.xml|  5 +
> > >>>2 files changed, 24 insertions(+), 3 deletions(-)
> > >>>
> > >>> diff --git a/java/org/apache/catalina/realm/JNDIRealm.java
> > >> b/java/org/apache/catalina/realm/JNDIRealm.java
> > >>> index 72087ab..98007f7 100644
> > >>> --- a/java/org/apache/catalina/realm/JNDIRealm.java
> > >>> +++ b/java/org/apache/catalina/realm/JNDIRealm.java
> > >>> @@ -1311,7 +1311,7 @@ public class JNDIRealm extends RealmBase {
> > >>>close(connection);
> > >>>
> > >>>// open a new directory context.
> > >>> -connection =3D get();
> > >>> +connection =3D get(true);
> > >>>
> > >>>// Try the authentication again.
> > >>>principal =3D authenticate(connection, username,
> > >> credentials);
> > >>> @@ -2389,12 +2389,28 @@ public class JNDIRealm extends RealmBase {
> > >>> * @exception NamingException if a directory server error occu=
> rs
> > >>> */
> > >>>protected JNDIConnection get() throws NamingException {
> > >>> +return get(false);
> > >>> +}
> > >>> +
> > >>> +/**
> > >>> + * Open (if necessary) and return a connection to the configured
> > >>> + * directory server for this Realm.
> > >>> + * @param create when pooling, this forces creation of a new
> > >> connection,
> > >>> + *   for example after an error
> > >>> + * @return the connection
> > >>> + * @exception NamingException if a directory server error occurs
> > >>> + */
> > >>> +protected JNDIConnection get(boolean create) throws
> > NamingException
> > >> {
> > >>>JNDIConnection connection =3D null;
> > >>>// Use the pool if available, otherwise use the single
> > >> connection
> > >>>if (connectionPool !=3D null) {
> > >>> -connection =3D connectionPool.pop();
> > >>> -if (connection =3D=3D null) {
> > >>> +if (create) {
> > >>>connection =3D create();
> > >>> +} else {
> > >>> +connection =3D connectionPool.pop();
> > >>> +if (connection =3D=3D null) {
> > >>> +connection =3D create();
> > >>> +}
> > >>>}
> > >>>} else {
> > >>>singleConnectionLock.lock();
> > >>
> > >> That suitable and simple approach.
> > >>
> > >
> > > If you have the code for adding a max idle check on hand and tested, yo=
> u
> > > should add it IMO, it will be more efficient.
> >
> > I will need to give this a couple more weeks of testing. This is 

Re: [tomcat] branch 8.5.x updated: Always retry on a new connection, even when pooling

[Michael Osipov]

Am 2020-10-15 um 16:48 schrieb Rémy Maucherat:

On Thu, Oct 15, 2020 at 2:31 PM Michael Osipov <1983-01...@gmx.net> wrote:


On Wed, Oct 14, 2020 at 6:32 PM Michael Osipov 

wrote:



Am 2020-10-14 um 12:32 schrieb R=C3=A9my Maucherat:

On Tue, Oct 13, 2020 at 8:27 PM Michael Osipov 

wrote:



Am 2020-10-13 um 16:05 schrieb r...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/8.5.x by this

push:

new 883600b  Always retry on a new connection, even when

pooli=

ng

883600b is described below

commit 883600b8a77c0be93b3a8dc2404e8d1110970ee7
Author: remm 
AuthorDate: Tue Oct 13 14:19:54 2020 +0200

   Always retry on a new connection, even when pooling

   This keeps the same very simple design as for the single

connection

   scenario, for now.
---
java/org/apache/catalina/realm/JNDIRealm.java | 22

+++---

webapps/docs/changelog.xml|  5 +
2 files changed, 24 insertions(+), 3 deletions(-)

diff --git a/java/org/apache/catalina/realm/JNDIRealm.java

b/java/org/apache/catalina/realm/JNDIRealm.java

index 72087ab..98007f7 100644
--- a/java/org/apache/catalina/realm/JNDIRealm.java
+++ b/java/org/apache/catalina/realm/JNDIRealm.java
@@ -1311,7 +1311,7 @@ public class JNDIRealm extends RealmBase {
close(connection);

// open a new directory context.
-connection =3D get();
+connection =3D get(true);

// Try the authentication again.
principal =3D authenticate(connection, username,

credentials);

@@ -2389,12 +2389,28 @@ public class JNDIRealm extends RealmBase {
 * @exception NamingException if a directory server error

occu=

rs

 */
protected JNDIConnection get() throws NamingException {
+return get(false);
+}
+
+/**
+ * Open (if necessary) and return a connection to the

configured

+ * directory server for this Realm.
+ * @param create when pooling, this forces creation of a new

connection,

+ *   for example after an error
+ * @return the connection
+ * @exception NamingException if a directory server error

occurs

+ */
+protected JNDIConnection get(boolean create) throws

NamingException

{

JNDIConnection connection =3D null;
// Use the pool if available, otherwise use the single

connection

if (connectionPool !=3D null) {
-connection =3D connectionPool.pop();
-if (connection =3D=3D null) {
+if (create) {
connection =3D create();
+} else {
+connection =3D connectionPool.pop();
+if (connection =3D=3D null) {
+connection =3D create();
+}
}
} else {
singleConnectionLock.lock();


That suitable and simple approach.



If you have the code for adding a max idle check on hand and tested,

yo=

u

should add it IMO, it will be more efficient.


I will need to give this a couple more weeks of testing. This is what I
have observed today:

2020-10-14T16:01:47.039 147.54.155.198 w99sezx0... "GET

/x2tc-proxy-bln/rest/info/targetSystem HTTP/1.1" 200 8 92132


20609 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]

net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Acquiring
directory server connection from pool

20610 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]

net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Directory

serve=

r

connection from pool exceeds max idle time, closing it

20611 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]

net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.close Closing

directory

server connection

20612 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]

net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.open Opening new
directory server connection

20613 2020-10-14T16:01:47 FEIN [https-openssl-apr-8444-exec-166]

net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.getUser Searching for
username 'w99sezx0' in base ...

As you can see it took 90 seconds to server the request because the
connection has expired and close took way too long. In average the
request takes:

2020-10-14T13:57:06.730 10.81.50.232 osipovmi@... "GET

/x2tc-proxy-bln/rest/info/targetSystem HTTP/1.1" 200 8 70

when the connection is healthy.



Ok, so there's some real incentive to avoid reusing a connection that was
idle for too long.


I made further analysis. I was partially wrong about my statement. The
cause
for the 90 s was a faulty KDC which did not respond in time for a service
ticket. Java Kerberos does 3 retries with 30 s timeout. I have set to 1
retry
an

Re: [DISCUSS] Deprecate and remove RealmBase#stripRealmForGss

[Michael Osipov]

Am 2020-10-13 um 21:08 schrieb Michael Osipov:

Am 2020-10-13 um 12:32 schrieb Mark Thomas:

On 13/10/2020 10:48, Michael Osipov wrote:

Folks,

I'd like to propose to get rid of that config option in 10 and deprecate
in previous versions for the following reasons:

* It suffers from abstraction: It assumes that the GSS name is always
email style w/o checking its OID
* The realm part, if any, is an integeral part of the principal. Much
like with an email address' domain. You wouldn't stip here too, would 
you?

* It is a surprise for clients having the princippal mutilated by
default. I trip over and over again this when I set up
UserDatabaseRealms for testing purposes I wonder why
michae...@example.com does not work.
* In a multi realm environment, it is perfectly fine and valid to have
user1@REALMA and user1@REALMB. These are distinct principals, but
treated by RealmBase equally, this has implications.
* Finally, when doing cert-based auth in an AD envinronment, is it
pretty common to extract the msUPN name from the cert's SAN which is
almost always email address (enteprise principal) which would end up in
michael.osipov, but where is the rest?!

Thoughts?


No objections to changing the default to false in 10.0.x but I think
removal/deprecation is a step too far.


OK


Not everyone uses the full u...@bigco.com format as the "primary key" in
their internal user database. I see lots of variation depending on which
system is viewed as the source of truth to which other systems are
synchronised.


So you will never be able to satisfy all, why try?


We have X509UsernameRetriever for certificate authentication. That came
out of bug 52500 where the aim was to support a wide variety of formats.
The root of that requirement was essentially the same - integration with
a variety of systems where the user name could be in in a range of
formats. The mapping of that format to an X509 cert just added another
layer of complexity.


I don't think that this compares (well( because a certificate is a 
complex object explicitly containing SAN which contain wellknown forms 
for an entity whereas a GSS name is merely a string with an identifier 
for its format. Your argumentation could also be applied to an LDAP 
bind. Active Directory, for instance, accepts a various amounts of input 
formats for the same entity, yet we don't provide a logic to normalize 
them in the JNDI Realm. That seems to be unfair to me.

It lacks a bit of consistency.


If there was user demand for it (I haven't seen any) I'd support a
similar interface for SPNEGO. With such an interface in place,
deprecation and removal of stripRealmForGss would make sense.


See above.


What we basically miss is a Java counterpart of GSS-API's gss_localname().

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat] branch 8.5.x updated: Always retry on a new connection, even when pooling

[Michael Osipov]

Am 2020-10-15 um 16:48 schrieb Rémy Maucherat:

On Thu, Oct 15, 2020 at 2:31 PM Michael Osipov <1983-01...@gmx.net> wrote:


On Wed, Oct 14, 2020 at 6:32 PM Michael Osipov 

wrote:



Am 2020-10-14 um 12:32 schrieb R=C3=A9my Maucherat:

On Tue, Oct 13, 2020 at 8:27 PM Michael Osipov 

wrote:



Am 2020-10-13 um 16:05 schrieb r...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/8.5.x by this

push:

new 883600b  Always retry on a new connection, even when

pooli=

ng

883600b is described below

commit 883600b8a77c0be93b3a8dc2404e8d1110970ee7
Author: remm 
AuthorDate: Tue Oct 13 14:19:54 2020 +0200

   Always retry on a new connection, even when pooling

   This keeps the same very simple design as for the single

connection

   scenario, for now.
---
java/org/apache/catalina/realm/JNDIRealm.java | 22

+++---

webapps/docs/changelog.xml|  5 +
2 files changed, 24 insertions(+), 3 deletions(-)

diff --git a/java/org/apache/catalina/realm/JNDIRealm.java

b/java/org/apache/catalina/realm/JNDIRealm.java

index 72087ab..98007f7 100644
--- a/java/org/apache/catalina/realm/JNDIRealm.java
+++ b/java/org/apache/catalina/realm/JNDIRealm.java
@@ -1311,7 +1311,7 @@ public class JNDIRealm extends RealmBase {
close(connection);

// open a new directory context.
-connection =3D get();
+connection =3D get(true);

// Try the authentication again.
principal =3D authenticate(connection, username,

credentials);

@@ -2389,12 +2389,28 @@ public class JNDIRealm extends RealmBase {
 * @exception NamingException if a directory server error

occu=

rs

 */
protected JNDIConnection get() throws NamingException {
+return get(false);
+}
+
+/**
+ * Open (if necessary) and return a connection to the

configured

+ * directory server for this Realm.
+ * @param create when pooling, this forces creation of a new

connection,

+ *   for example after an error
+ * @return the connection
+ * @exception NamingException if a directory server error

occurs

+ */
+protected JNDIConnection get(boolean create) throws

NamingException

{

JNDIConnection connection =3D null;
// Use the pool if available, otherwise use the single

connection

if (connectionPool !=3D null) {
-connection =3D connectionPool.pop();
-if (connection =3D=3D null) {
+if (create) {
connection =3D create();
+} else {
+connection =3D connectionPool.pop();
+if (connection =3D=3D null) {
+connection =3D create();
+}
}
} else {
singleConnectionLock.lock();


That suitable and simple approach.



If you have the code for adding a max idle check on hand and tested,

yo=

u

should add it IMO, it will be more efficient.


I will need to give this a couple more weeks of testing. This is what I
have observed today:

2020-10-14T16:01:47.039 147.54.155.198 w99sezx0... "GET

/x2tc-proxy-bln/rest/info/targetSystem HTTP/1.1" 200 8 92132


20609 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]

net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Acquiring
directory server connection from pool

20610 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]

net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Directory

serve=

r

connection from pool exceeds max idle time, closing it

20611 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]

net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.close Closing

directory

server connection

20612 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]

net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.open Opening new
directory server connection

20613 2020-10-14T16:01:47 FEIN [https-openssl-apr-8444-exec-166]

net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.getUser Searching for
username 'w99sezx0' in base ...

As you can see it took 90 seconds to server the request because the
connection has expired and close took way too long. In average the
request takes:

2020-10-14T13:57:06.730 10.81.50.232 osipovmi@... "GET

/x2tc-proxy-bln/rest/info/targetSystem HTTP/1.1" 200 8 70

when the connection is healthy.



Ok, so there's some real incentive to avoid reusing a connection that was
idle for too long.


I made further analysis. I was partially wrong about my statement. The
cause
for the 90 s was a faulty KDC which did not respond in time for a service
ticket. Java Kerberos does 3 retries with 30 s timeout. I have set to 1
retry
an

Re: [tomcat] branch 8.5.x updated: Always retry on a new connection, even when pooling

[Michael Osipov]

Am 2020-10-19 um 16:34 schrieb Rémy Maucherat:

On Mon, Oct 19, 2020 at 3:11 PM Michael Osipov  wrote:


Am 2020-10-15 um 16:48 schrieb Rémy Maucherat:

On Thu, Oct 15, 2020 at 2:31 PM Michael Osipov <1983-01...@gmx.net>

wrote:



On Wed, Oct 14, 2020 at 6:32 PM Michael Osipov 

wrote:



Am 2020-10-14 um 12:32 schrieb R=C3=A9my Maucherat:

On Tue, Oct 13, 2020 at 8:27 PM Michael Osipov 

wrote:



Am 2020-10-13 um 16:05 schrieb r...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/8.5.x by this

push:

 new 883600b  Always retry on a new connection, even when

pooli=

ng

883600b is described below

commit 883600b8a77c0be93b3a8dc2404e8d1110970ee7
Author: remm 
AuthorDate: Tue Oct 13 14:19:54 2020 +0200

Always retry on a new connection, even when pooling

This keeps the same very simple design as for the single

connection

scenario, for now.
---
 java/org/apache/catalina/realm/JNDIRealm.java | 22

+++---

 webapps/docs/changelog.xml|  5 +
 2 files changed, 24 insertions(+), 3 deletions(-)

diff --git a/java/org/apache/catalina/realm/JNDIRealm.java

b/java/org/apache/catalina/realm/JNDIRealm.java

index 72087ab..98007f7 100644
--- a/java/org/apache/catalina/realm/JNDIRealm.java
+++ b/java/org/apache/catalina/realm/JNDIRealm.java
@@ -1311,7 +1311,7 @@ public class JNDIRealm extends RealmBase {
 close(connection);

 // open a new directory context.
-connection =3D get();
+connection =3D get(true);

 // Try the authentication again.
 principal =3D authenticate(connection,

username,

credentials);

@@ -2389,12 +2389,28 @@ public class JNDIRealm extends RealmBase {
  * @exception NamingException if a directory server error

occu=

rs

  */
 protected JNDIConnection get() throws NamingException {
+return get(false);
+}
+
+/**
+ * Open (if necessary) and return a connection to the

configured

+ * directory server for this Realm.
+ * @param create when pooling, this forces creation of a new

connection,

+ *   for example after an error
+ * @return the connection
+ * @exception NamingException if a directory server error

occurs

+ */
+protected JNDIConnection get(boolean create) throws

NamingException

{

 JNDIConnection connection =3D null;
 // Use the pool if available, otherwise use the single

connection

 if (connectionPool !=3D null) {
-connection =3D connectionPool.pop();
-if (connection =3D=3D null) {
+if (create) {
 connection =3D create();
+} else {
+connection =3D connectionPool.pop();
+if (connection =3D=3D null) {
+connection =3D create();
+}
 }
 } else {
 singleConnectionLock.lock();


That suitable and simple approach.



If you have the code for adding a max idle check on hand and tested,

yo=

u

should add it IMO, it will be more efficient.


I will need to give this a couple more weeks of testing. This is what

I

have observed today:

2020-10-14T16:01:47.039 147.54.155.198 w99sezx0... "GET

/x2tc-proxy-bln/rest/info/targetSystem HTTP/1.1" 200 8 92132


20609 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]

net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Acquiring
directory server connection from pool

20610 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]

net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Directory

serve=

r

connection from pool exceeds max idle time, closing it

20611 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]

net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.close Closing

directory

server connection

20612 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]

net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.open Opening new
directory server connection

20613 2020-10-14T16:01:47 FEIN [https-openssl-apr-8444-exec-166]

net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.getUser Searching

for

username 'w99sezx0' in base ...

As you can see it took 90 seconds to server the request because the
connection has expired and close took way too long. In average the
request takes:

2020-10-14T13:57:06.730 10.81.50.232 osipovmi@... "GET

/x2tc-proxy-bln/rest/info/targetSystem HTTP/1.1" 200 8 70

when the connection is healthy.



Ok, so there's some real incentive to avoid reusing a connection that

was

idle for too long.


I made further analysis. I was partially wrong about my statement. The
cause
for the 90 s was a

Re: Objection to the deprecation of the tomcat-native/APR connector

[Michael Osipov]

Am 2020-12-01 um 13:09 schrieb Mark Thomas:

On 01/12/2020 11:05, Graham Leggett wrote:

Hi all,

I object to the deprecation of the tomcat-native/APR connector.

Most specifically, I am -1 on the following:

https://marc.info/?l=tomcat-dev&m=160681846808019&w=2

Looking at past discussion on this, the justification has been:

"It is inherently less stable. If we get the NIO code wrong, you might
see a NullPointerException. If we get the APR code wrong you might see a
JVM crash.”

Both a NullPointerException and a crash result in the same outcome - a non 
working server.


No it isn't. The difference is a single failed request compared to the
entire server failing.


Tomcat-native has releases in the 
https://archive.apache.org/dist/tomcat/tomcat-connectors/native/ going back 15 
years to 2005, a claim of a lack of stability needs to be quantifiable.


See the long list of bugs raised against Tomcat and the Tomcat Native
Connector that reported a JVM crash. The reports have slowly been
getting less frequent over the years and are at a much lower level now
than they were but the risk remains.


I also object to the removal of OpenSSL code, for the same reason.


It isn't being removed. The APR/Native library will be retained along
with OpenSSL support for the NIO and NIO2.

I expect the scope of the APR/Native library for Tomcat 10.1.x onwards
will be reduced to just those native methods required to interact with
OpenSSL which may mean removal of the APR dependency. If we can use
OpenSSL without any native code of our own (e.g. via project Panama or
similar) then better still.


We are in the middle of a global pandemic. Our users do not have the resources 
to suddenly divert to reengineering what is to them a perfectly working system, 
to replace what exists with something else that just works differently.


Upgrading to Tomcat 10 already requires significant re-engineering work
due to the java package change for all the specification APIs.

Switching an HTTP or AJP connector from APR/Native to NIO or NIO2 with
OpenSSL requires a change of three/four characters in one configuration
file. We have deliberately made it very easy to switch between connectors.

No-one is being forced to upgrade. Tomcat 8.5.x and 9.0.x will continue
to support the APR/Native connector for AJP and HTTP. Based on the
typical 10 year support lifetime of a major Tomcat release users have at
least five to six years before they would be forced to migrate away from
using an APR/Native HTTP or AJP connector.

I'll note that Tomcat supports at least 3 major versions in parallel
with each major version being supported for ~10 years. That is a very
generous support offering.


Mark, while I always appreciate your professional answers, I do agree 
with Graham from a different opinion:
As most of you have noticed I worked to some extend on libtcnative 
because it simply works for me and just has failed only once many many 
years go. I tried to remove some light APR dependencies for many OSes. 
Now, we all know pre-10 Tomcat versions won't go away soon we will 
continue support libtcnative for those versions anyway.
With or without the deprecation, we can always say if APR does not work 
for you, take NIO2.

A suitable roadmap for libtcnative would be:
* Tag current patch version
* Move to 1.3.0 and remove everything non-TLS/networking related out
* Move to 1.4.0 drop OpenSSL support for < 1.1.1 because it requires 
thread locks from APR which aren't necessary with 1.1.1
* Likely split code between OpenSSL to Java and APR to Java with that we 
could satisfy both sides.


I am now nicely acquiant with the code, I could at least remove 
everything for 1.3.0 and have at least three completely different OSes 
to test.


Another side note is that building on Windows is a pain, I do not and 
will not install Visual Studio to compile a few hundred lines of C code. 
I would highly favorize a CMake-based build which works on Windows too, 
but that is a different discussion, of course.


Michael


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [VOTE] Release Apache Tomcat 10.0.0

[Michael Osipov]

Am 2020-12-03 um 11:43 schrieb Mark Thomas:

The proposed Apache Tomcat 10.0.0 release is now available for
voting.

Apache Tomcat 10.x implements Jakarta EE 9 and, as such, the primary
package for all the specification APIs has changed from javax.* to jakarta.*
Applications that run on Tomcat 9 will not run on Tomcat 10 without changes.

The notable changes compared to 10.0.0-M10 are:

- Specs are now final. Tomcat passes the TCKs apart from a number of
   expected failures that don't impact spec compliance.

- The APR/Native AJP and HTTP connectors have been deprecated.
   Tomcat Native will continue to be used to support OpenSSL use with NIO
   and NIO2.

- Align the behaviour of ServletContext.getRealPath(String path) with
   the recent clarification from the Servlet specification project. If
   the path parameter does not start with / then Tomcat processes the
   call as if / is appended to the beginning of the provided path.

Along with lots of other bug fixes and improvements.


For full details, see the changelog:
https://ci.apache.org/projects/tomcat/tomcat10/docs/changelog.html

It can be obtained from:
https://dist.apache.org/repos/dist/dev/tomcat/tomcat-10/v10.0.0/
The Maven staging repo is:
https://repository.apache.org/content/repositories/orgapachetomcat-1287/
The tag is:
https://github.com/apache/tomcat/tree/10.0.0
4c8b650437e2464c1c31c6598a263b3805b7a81f

The proposed 10.0.0 release is:
[ ] Broken - do not release
[ ] Beta   - go ahead and release as 10.0.0 (beta)
[ ] Stable - go ahead and release as 10.0.0 (stable)


This is confusing. You are casting a vote von a GA version: 10.0.0, but 
provide to mark this as beta? Huh? Shouldn't this be a binary vote? 
Either we can go GA and it is stable or this is broken and needs fixing 
and should remain on a milestone (which does not imply any alpha, beta, 
or RC quality).


Michael

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org

Re: [tomcat-native] branch master updated: Add more information around minimum versions

[Michael Osipov]

Am 2020-12-10 um 13:21 schrieb ma...@apache.org:

This is an automated email from the ASF dual-hosted git repository.

markt pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat-native.git


The following commit(s) were added to refs/heads/master by this push:
  new 51468b2  Add more information around minimum versions
51468b2 is described below

commit 51468b2357ae46ecd7363aa05285e5c48a0888b2
Author: Mark Thomas 
AuthorDate: Thu Dec 10 12:21:05 2020 +

 Add more information around minimum versions
---
  native/srclib/VERSIONS | 35 ++-
  1 file changed, 34 insertions(+), 1 deletion(-)

diff --git a/native/srclib/VERSIONS b/native/srclib/VERSIONS
index f6b09de..e52a291 100644
--- a/native/srclib/VERSIONS
+++ b/native/srclib/VERSIONS
@@ -1,7 +1,40 @@
+The current minimum versions are:
+- OpenSSL 1.0.2
+- APR 1.4.3
+
  The following version of the libraries are recommended:
  
  - APR 1.7.0 or later, http://apr.apache.org

  - OpenSSL 1.1.1g or later, http://www.openssl.org
  
  Older versions should also work but are not as thoroughly tested by the Tomcat

-Native team
\ No newline at end of file
+Native team
+
+It is current anticipated that Tomcat Native releases will transition to 1.3.x
+after April 2021 when the minimum version will become OpenSSL 1.1.0 and
+APR 1.5.2.
+
+The minimum version of OpenSSL is driven by the version of OpenSSL used by
+downstream distributions.
+
+The current state of OpenSSL in Debian is:
+- OpenSSL 1.1.0l in Debian 9 (EOL in June 2022)
+- OpenSSL 1.1.1d in Debian 10
+
+And in Ubuntu:
+- OpenSSL 1.0.2g in Ubuntu 16.04 LTS (EOL in April 2021)
+- OpenSSL 1.1.1  in Ubuntu 18.04 LTS
+- OpenSSL 1.1.1f in Ubuntu 20.04 LTS
+
+
+The minimum version of APR is driven by the version of APR used by
+downstream distributions.
+
+The current state of APR in Debian is:
+- APR 1.5.2 in Debian 9 (EOL in June 2022)
+- APR 1.6.5 in Debian 10
+
+And in Ubuntu:
+- APR 1.5.2 in Ubuntu 16.04 LTS (EOL in April 2021)
+- APR 1.6.3 in Ubuntu 18.04 LTS
+- APR 1.6.5 in Ubuntu 20.04 LTS


I consider this to be wrong. We should support only what upstream 
supports, with some grace period of course. Everything else is a OS 
vendor problem, not ours. Especially for Mark Shuttleworth's project.


M

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org