buildbot success in on tomcat-7-trunk
The Buildbot has detected a restored build on builder tomcat-7-trunk while building tomcat. Full details are available at: https://ci.apache.org/builders/tomcat-7-trunk/builds/1788 Buildbot URL: https://ci.apache.org/ Buildslave for this Build: asf946_ubuntu Build Reason: The AnyBranchScheduler scheduler named 'on-tomcat-7-commit' triggered this build Build Source Stamp: [branch 7.0.x] 34504ba97bef854671e6114414055b84026da6f3 Blamelist: Mark Thomas Build succeeded! Sincerely, -The Buildbot - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] branch 7.0.x updated: Trivial commit to trigger CI. Align with 8.5.x
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 7.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/7.0.x by this push:
new 34504ba Trivial commit to trigger CI. Align with 8.5.x
34504ba is described below
commit 34504ba97bef854671e6114414055b84026da6f3
Author: Mark Thomas
AuthorDate: Wed Sep 30 18:23:52 2020 +0100
Trivial commit to trigger CI. Align with 8.5.x
---
java/org/apache/catalina/loader/JdbcLeakPrevention.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/java/org/apache/catalina/loader/JdbcLeakPrevention.java
b/java/org/apache/catalina/loader/JdbcLeakPrevention.java
index c6862de..6459dba 100644
--- a/java/org/apache/catalina/loader/JdbcLeakPrevention.java
+++ b/java/org/apache/catalina/loader/JdbcLeakPrevention.java
@@ -34,7 +34,7 @@ import java.util.List;
* version is do not just create a new instance of this class with the new
* keyword.
*
- * Since this class is loaded by {@link WebappClassLoaderBase}, it can not
refer
+ * Since this class is loaded by {@link WebappClassLoaderBase}, it cannot refer
* to any internal Tomcat classes as that will cause the security manager to
* complain.
*/
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: buildbot failure in on tomcat-7-trunk
On 30/09/2020 18:12, build...@apache.org wrote: > The Buildbot has detected a new failure on builder tomcat-7-trunk while > building tomcat. Full details are available at: > https://ci.apache.org/builders/tomcat-7-trunk/builds/1787 > > Buildbot URL: https://ci.apache.org/ > > Buildslave for this Build: asf946_ubuntu > > Build Reason: The AnyBranchScheduler scheduler named 'on-tomcat-7-commit' > triggered this build > Build Source Stamp: [branch 7.0.x] c4b1559d6e7f131be804373719fe41c26969df54 > Blamelist: Kyle Stiemann ,Mark Thomas > > > BUILD FAILED: failed compile_1 Looks like a TLS / download issue. I'll fix it on the build node. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
buildbot failure in on tomcat-7-trunk
The Buildbot has detected a new failure on builder tomcat-7-trunk while building tomcat. Full details are available at: https://ci.apache.org/builders/tomcat-7-trunk/builds/1787 Buildbot URL: https://ci.apache.org/ Buildslave for this Build: asf946_ubuntu Build Reason: The AnyBranchScheduler scheduler named 'on-tomcat-7-commit' triggered this build Build Source Stamp: [branch 7.0.x] c4b1559d6e7f131be804373719fe41c26969df54 Blamelist: Kyle Stiemann ,Mark Thomas BUILD FAILED: failed compile_1 Sincerely, -The Buildbot - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Tagging 10.0.x, 9.0.x and 8.5.x.
Hi, It looks like Martin has found a way to reproduce the buffer overflow reported in bug 64710. Therefore I plan to hold of on the next round of tags until I have been ablw to investigate (and hopefully) fix that issue. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 64778] org.apache.el.lang.ELSupport: methods removed
https://bz.apache.org/bugzilla/show_bug.cgi?id=64778 Mark Thomas changed: What|Removed |Added Resolution|--- |WONTFIX Status|NEW |RESOLVED --- Comment #5 from Mark Thomas --- Ack. I'll resolve this as WONTFIX then. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[GitHub] [tomcat] markt-asf closed pull request #360: Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
markt-asf closed pull request #360: URL: https://github.com/apache/tomcat/pull/360 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[GitHub] [tomcat] markt-asf closed pull request #359: Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
markt-asf closed pull request #359: URL: https://github.com/apache/tomcat/pull/359 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[GitHub] [tomcat] markt-asf commented on pull request #359: Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
markt-asf commented on pull request #359: URL: https://github.com/apache/tomcat/pull/359#issuecomment-701522483 Thanks. Applied manually so I could make a few minor tweaks (mainly for consistency with existing tests). Also back-ported to 8.5.x and 7.0.x, This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[GitHub] [tomcat] markt-asf commented on pull request #360: Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
markt-asf commented on pull request #360: URL: https://github.com/apache/tomcat/pull/360#issuecomment-701522275 Thanks. Applied manually so I could make a few minor tweaks (mainly for consistency with existing tests). Also back-ported to 8.5.x and 7.0.x, This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 64735] ServletContext.addJspFile() always fails with SecurityManager
https://bz.apache.org/bugzilla/show_bug.cgi?id=64735 Mark Thomas changed: What|Removed |Added Resolution|--- |FIXED Status|NEW |RESOLVED --- Comment #6 from Mark Thomas --- Thanks again for the patch. I back-ported it (with a few changes due to minimum Java versions) all the way back to 7.0.x where there was one method missing. Fixed in: - master for 10.0.0-M9 onwards - 9.0.x for 9.0.39 onwards - 8.5.x for 8.5.59 onwards - 7.0.x for 7.0.107 onwards -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] branch 7.0.x updated (e487719 -> c4b1559)
This is an automated email from the ASF dual-hosted git repository. markt pushed a change to branch 7.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git. from e487719 Use a link to TestTomcat in Git instead of SVN new b649d6c Align with 10.0.x and 9.0.x new c4b1559 Fix 64735 ServletContext.addJspFile() always fails with SecurityManager The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: build.properties.default | 24 ++-- build.xml | 17 ++- .../catalina/core/ApplicationContextFacade.java| 1 + ...estApplicationContextFacadeSecurityManager.java | 153 + .../util/security/SecurityManagerBaseTest.java | 50 +++ webapps/docs/changelog.xml | 5 + 6 files changed, 235 insertions(+), 15 deletions(-) create mode 100644 test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java create mode 100644 test/org/apache/tomcat/util/security/SecurityManagerBaseTest.java - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 01/02: Align with 10.0.x and 9.0.x
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 7.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit b649d6c195e67c8f9b515ffd15ac16edc63f2661 Author: Mark Thomas AuthorDate: Wed Sep 30 13:49:00 2020 +0100 Align with 10.0.x and 9.0.x --- build.xml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/build.xml b/build.xml index 3c0cf4a..707e630 100644 --- a/build.xml +++ b/build.xml @@ -1557,7 +1557,7 @@ /> - + @@ -1569,8 +1569,6 @@ - - - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 02/02: Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 7.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit c4b1559d6e7f131be804373719fe41c26969df54
Author: Kyle Stiemann
AuthorDate: Thu Sep 10 16:47:21 2020 -0400
Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
---
build.properties.default | 24 ++--
build.xml | 13 ++
.../catalina/core/ApplicationContextFacade.java| 1 +
...estApplicationContextFacadeSecurityManager.java | 153 +
.../util/security/SecurityManagerBaseTest.java | 50 +++
webapps/docs/changelog.xml | 5 +
6 files changed, 234 insertions(+), 12 deletions(-)
diff --git a/build.properties.default b/build.properties.default
index 96637e5..c4ab2ba 100644
--- a/build.properties.default
+++ b/build.properties.default
@@ -257,29 +257,29 @@ hamcrest.home=${base.path}/hamcrest-${hamcrest.version}
hamcrest.jar=${hamcrest.home}/hamcrest-core-${hamcrest.version}.jar
hamcrest.loc=${base-maven.loc}/org/hamcrest/hamcrest-core/${hamcrest.version}/hamcrest-core-${hamcrest.version}.jar
-# - EasyMock, version 3.2 or later -
-easymock.version=3.2
+# - EasyMock, version 3.6 or later -
+easymock.version=3.6
easymock.checksum.enabled=true
-easymock.checksum.algorithm=MD5|SHA-1
-easymock.checksum.value=0da4291328e18798621c36fdf8bc4c3a|00c82f7fa3ef377d8954b1db25123944b5af2ba4
+easymock.checksum.algorithm=SHA-512
+easymock.checksum.value=b49673e7872aa8b006d377a4ca56eed8f6e98e45ea4efe26d24cafe81ea569b557f5a326edcd387288267db8c4b59f2c3c61010f3ad01c4d1067a35430533794
easymock.home=${base.path}/easymock-${easymock.version}
easymock.jar=${easymock.home}/easymock-${easymock.version}.jar
easymock.loc=${base-maven.loc}/org/easymock/easymock/${easymock.version}/easymock-${easymock.version}.jar
-# - cglib, used by EasyMock, version 2.2 or later -
-cglib.version=2.2.3
+# - cglib, used by EasyMock, version 3.3 or later -
+cglib.version=3.3.0
cglib.checksum.enabled=true
-cglib.checksum.algorithm=MD5|SHA-1
-cglib.checksum.value=694815351007f966c14ea093ec838323|6a4af5d9112066a5baf235fd55d5876969bc813c
+cglib.checksum.algorithm=SHA-512
+cglib.checksum.value=faa1d2121e87ae69e179e3aae217accd0834e0da716b91a029fd526e192612e71675f2740bedf48e23ef1edc45f672a2be1b3e78bbfb1ad59c96dd3d2feeedba
cglib.home=${base.path}/cglib-${cglib.version}
cglib.jar=${cglib.home}/cglib-nodep-${cglib.version}.jar
cglib.loc=${base-sf.loc}/cglib/cglib-nodep-${cglib.version}.jar
-# - objenesis, used by EasyMock, version 1.2 or later -
-objenesis.version=1.2
+# - objenesis, used by EasyMock, version 2.6 or later -
+objenesis.version=2.6
objenesis.checksum.enabled=true
-objenesis.checksum.algorithm=MD5|SHA-1
-objenesis.checksum.value=bee117291d50b41b8e8cf0ac5435df1d|bfcb0539a071a4c5a30690388903ac48c0667f2a
+objenesis.checksum.algorithm=SHA-512
+objenesis.checksum.value=23a593bded8cb43236faad2018b008da47bf4e29cc60c2e98fd4f2ed578fe2baddd3a98547dc14273017c82cb19ce8eaaab71d49273411856a2ba1a5d51015fc
objenesis.home=${base.path}/objenesis-${objenesis.version}
objenesis.jar=${objenesis.home}/objenesis-${objenesis.version}.jar
objenesis.loc=${base-maven.loc}/org/objenesis/objenesis/${objenesis.version}/objenesis-${objenesis.version}.jar
diff --git a/build.xml b/build.xml
index 707e630..5bd5a7c 100644
--- a/build.xml
+++ b/build.xml
@@ -1569,8 +1569,21 @@
+
+
+
+
+
+
diff --git a/java/org/apache/catalina/core/ApplicationContextFacade.java
b/java/org/apache/catalina/core/ApplicationContextFacade.java
index 2c39404..544b219 100644
--- a/java/org/apache/catalina/core/ApplicationContextFacade.java
+++ b/java/org/apache/catalina/core/ApplicationContextFacade.java
@@ -117,6 +117,7 @@ public class ApplicationContextFacade implements
ServletContext {
classCache.put("getAttribute", clazz);
classCache.put("log", clazz);
classCache.put("setSessionTrackingModes", new Class[]{Set.class} );
+classCache.put("declareRoles", new Class[]{String[].class});
}
diff --git
a/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
new file mode 100644
index 000..442e1e7
--- /dev/null
+++
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
@@ -0,0 +1,153 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not
[tomcat] 09/13: Refactor the pruning so more stream info is retained for longer
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 3d3060b741f8b67fc597cf9665b48b82d4cb9a12
Author: Mark Thomas
AuthorDate: Fri Sep 25 14:14:06 2020 +0100
Refactor the pruning so more stream info is retained for longer
The memory impact of this is mitigated by the previous changes to
replace closed Stream instances with RecycledStream instances.
---
.../apache/coyote/http2/Http2UpgradeHandler.java | 89 --
1 file changed, 49 insertions(+), 40 deletions(-)
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index 295e9aa..fe60f32 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -30,6 +30,7 @@ import java.util.TreeSet;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentSkipListMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
@@ -126,8 +127,7 @@ public class Http2UpgradeHandler extends AbstractStream
implements InternalHttpU
private long readTimeout = Http2Protocol.DEFAULT_READ_TIMEOUT;
private long keepAliveTimeout = Http2Protocol.DEFAULT_KEEP_ALIVE_TIMEOUT;
private long writeTimeout = Http2Protocol.DEFAULT_WRITE_TIMEOUT;
-
-private final ConcurrentMap streams = new
ConcurrentHashMap<>();
+private final ConcurrentMap streams = new
ConcurrentSkipListMap<>();
protected final AtomicInteger activeRemoteStreamCount = new
AtomicInteger(0);
// Start at -1 so the 'add 2' logic in closeIdleStreams() works
private volatile int maxActiveRemoteStreamId = -1;
@@ -1197,17 +1197,16 @@ public class Http2UpgradeHandler extends AbstractStream
implements InternalHttpU
// 2. Completed streams used for a request with children
// 3. Closed final streams
//
-// Steps 1 and 2 will always be completed.
-// Step 3 will be completed to the minimum extent necessary to bring
the
-// total number of streams under the limit.
+// The pruning halts as soon as enough streams have been pruned.
// Use these sets to track the different classes of streams
-TreeSet candidatesStepOne = new TreeSet<>();
TreeSet candidatesStepTwo = new TreeSet<>();
TreeSet candidatesStepThree = new TreeSet<>();
-for (Entry entry : streams.entrySet())
{
-AbstractNonZeroStream stream = entry.getValue();
+// Step 1
+// Iterator is in key order so we automatically have the oldest streams
+// first
+for (AbstractNonZeroStream stream : streams.values()) {
// Never remove active streams
if (stream instanceof Stream && ((Stream) stream).isActive()) {
continue;
@@ -1216,58 +1215,68 @@ public class Http2UpgradeHandler extends AbstractStream
implements InternalHttpU
if (stream.isClosedFinal()) {
// This stream went from IDLE to CLOSED and is likely to have
// been created by the client as part of the priority tree.
-candidatesStepThree.add(entry.getKey());
+// Candidate for steo 3.
+candidatesStepThree.add(stream.getIdentifier());
} else if (stream.getChildStreams().size() == 0) {
-// Closed, no children
-candidatesStepOne.add(entry.getKey());
-} else {
-// Closed, with children
-candidatesStepTwo.add(entry.getKey());
-}
-}
-
-// Process the step one list
-for (Integer streamIdToRemove : candidatesStepOne) {
-// Remove this childless stream
-AbstractNonZeroStream removedStream =
streams.remove(streamIdToRemove);
-removedStream.detachFromParent();
-toClose--;
-if (log.isDebugEnabled()) {
-log.debug(sm.getString("upgradeHandler.pruned", connectionId,
streamIdToRemove));
-}
-
-// Did this make the parent childless?
-AbstractStream parent = removedStream.getParentStream();
-while (parent instanceof Stream && !((Stream) parent).isActive() &&
-!((Stream) parent).isClosedFinal() &&
parent.getChildStreams().size() == 0) {
-streams.remove(parent.getIdentifier());
-parent.detachFromParent();
-toClose--;
+// Prune it
+AbstractStream parent = stream.getParentStream();
+streams.remove(stream.getIdentifier());
+
[tomcat] 13/13: Fix back-port of new unit test
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 3e5dd5c3121fa9763b3c0e0ae1875453f9d87f0f
Author: Mark Thomas
AuthorDate: Wed Sep 30 17:00:48 2020 +0100
Fix back-port of new unit test
---
build.properties.default | 6 +--
...estApplicationContextFacadeSecurityManager.java | 49 --
2 files changed, 30 insertions(+), 25 deletions(-)
diff --git a/build.properties.default b/build.properties.default
index e506f8e..761e161 100644
--- a/build.properties.default
+++ b/build.properties.default
@@ -225,7 +225,7 @@ hamcrest.home=${base.path}/hamcrest-${hamcrest.version}
hamcrest.jar=${hamcrest.home}/hamcrest-${hamcrest.version}.jar
hamcrest.loc=${base-maven.loc}/org/hamcrest/hamcrest/${hamcrest.version}/hamcrest-${hamcrest.version}.jar
-# - EasyMock, version 3.2 or later -
+# - EasyMock, version 3.6 or later -
easymock.version=3.6
easymock.checksum.enabled=true
easymock.checksum.algorithm=SHA-512
@@ -234,7 +234,7 @@ easymock.home=${base.path}/easymock-${easymock.version}
easymock.jar=${easymock.home}/easymock-${easymock.version}.jar
easymock.loc=${base-maven.loc}/org/easymock/easymock/${easymock.version}/easymock-${easymock.version}.jar
-# - cglib, used by EasyMock, version 2.2 or later -
+# - cglib, used by EasyMock, version 3.3 or later -
cglib.version=3.3.0
cglib.checksum.enabled=true
cglib.checksum.algorithm=SHA-512
@@ -243,7 +243,7 @@ cglib.home=${base.path}/cglib-${cglib.version}
cglib.jar=${cglib.home}/cglib-nodep-${cglib.version}.jar
cglib.loc=${base-maven.loc}/cglib/cglib-nodep/${cglib.version}/cglib-nodep-${cglib.version}.jar
-# - objenesis, used by EasyMock, version 1.2 or later -
+# - objenesis, used by EasyMock, version 2.6 or later -
objenesis.version=2.6
objenesis.checksum.enabled=true
objenesis.checksum.algorithm=SHA-512
diff --git
a/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
index fc5a5d5..a89b558 100644
---
a/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
+++
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
@@ -20,9 +20,9 @@ import java.lang.reflect.Array;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
-import java.util.Arrays;
+import java.util.ArrayList;
import java.util.Collection;
-import java.util.stream.Collectors;
+import java.util.List;
import org.junit.Assert;
import org.junit.Test;
@@ -33,6 +33,7 @@ import org.junit.runners.Parameterized.Parameter;
import org.apache.catalina.security.SecurityUtil;
import org.apache.tomcat.util.security.SecurityManagerBaseTest;
import org.easymock.EasyMock;
+import org.easymock.IAnswer;
import org.easymock.IExpectationSetters;
import org.easymock.internal.LastControl;
@@ -46,24 +47,24 @@ public final class
TestApplicationContextFacadeSecurityManager extends SecurityM
*/
@Parameterized.Parameters(name = "{index}: method={0}")
public static Collection publicApplicationContextFacadeMethods() {
-return Arrays.stream(ApplicationContextFacade.class.getMethods())
-.filter(method -> !Modifier.isStatic(method.getModifiers()))
-.filter(method -> {
-try {
-Object.class.getMethod(method.getName(),
method.getParameterTypes());
-return false;
-} catch (final NoSuchMethodException e) {
-return true;
-}
-})
-.collect(Collectors.toList());
+List result = new ArrayList<>();
+for (Method m : ApplicationContextFacade.class.getMethods()) {
+if (!Modifier.isStatic(m.getModifiers())) {
+try {
+Object.class.getMethod(m.getName(), m.getParameterTypes());
+// Skip;
+} catch (final NoSuchMethodException e) {
+result.add(m);
+}
+}
+}
+return result;
}
private static Object[] getDefaultParams(final Method method) {
-final int paramsCount = method.getParameterCount();
-final Object[] params = new Object[paramsCount];
final Class[] paramTypes = method.getParameterTypes();
+final Object[] params = new Object[paramTypes.length];
for (int i = 0; i < params.length; i++) {
params[i] = getDefaultValue(paramTypes[i]);
}
@@ -118,13 +119,17 @@ public final class
TestApplicationContextFacadeSecurityManager extends SecurityM
EasyMock.expect(expectedAppContextMethod.invoke(
[tomcat] 12/13: Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 55b8e013d5bff2a11065537a0c304681d08c5f73
Author: Kyle Stiemann
AuthorDate: Thu Sep 10 16:47:21 2020 -0400
Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
---
build.xml | 13 ++
.../catalina/core/ApplicationContextFacade.java| 5 +
...estApplicationContextFacadeSecurityManager.java | 148 +
.../util/security/SecurityManagerBaseTest.java | 50 +++
webapps/docs/changelog.xml | 5 +
5 files changed, 221 insertions(+)
diff --git a/build.xml b/build.xml
index 9aa56ee..5db4081 100644
--- a/build.xml
+++ b/build.xml
@@ -1564,8 +1564,21 @@
+
+
+
+
+
+
diff --git a/java/org/apache/catalina/core/ApplicationContextFacade.java
b/java/org/apache/catalina/core/ApplicationContextFacade.java
index 3992c7e..157b34f 100644
--- a/java/org/apache/catalina/core/ApplicationContextFacade.java
+++ b/java/org/apache/catalina/core/ApplicationContextFacade.java
@@ -117,6 +117,11 @@ public class ApplicationContextFacade implements
ServletContext {
classCache.put("getAttribute", clazz);
classCache.put("log", clazz);
classCache.put("setSessionTrackingModes", new Class[]{Set.class} );
+classCache.put("addJspFile", new Class[]{String.class, String.class});
+classCache.put("declareRoles", new Class[]{String[].class});
+classCache.put("setSessionTimeout", new Class[]{int.class});
+classCache.put("setRequestCharacterEncoding", new
Class[]{String.class});
+classCache.put("setResponseCharacterEncoding", new
Class[]{String.class});
}
diff --git
a/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
new file mode 100644
index 000..fc5a5d5
--- /dev/null
+++
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.catalina.core;
+
+import java.lang.reflect.Array;
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.lang.reflect.Modifier;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.stream.Collectors;
+
+import org.junit.Assert;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameter;
+
+import org.apache.catalina.security.SecurityUtil;
+import org.apache.tomcat.util.security.SecurityManagerBaseTest;
+import org.easymock.EasyMock;
+import org.easymock.IExpectationSetters;
+import org.easymock.internal.LastControl;
+
+@RunWith(Parameterized.class)
+public final class TestApplicationContextFacadeSecurityManager extends
SecurityManagerBaseTest {
+
+/**
+ * @return {@link Collection} of non-static, non-object, public {@link
+ * Method}s in {@link ApplicationContextFacade} to be run with the the Java
+ * 2 {@link SecurityManager} been enabled.
+ */
+@Parameterized.Parameters(name = "{index}: method={0}")
+public static Collection publicApplicationContextFacadeMethods() {
+return Arrays.stream(ApplicationContextFacade.class.getMethods())
+.filter(method -> !Modifier.isStatic(method.getModifiers()))
+.filter(method -> {
+try {
+Object.class.getMethod(method.getName(),
method.getParameterTypes());
+return false;
+} catch (final NoSuchMethodException e) {
+return true;
+}
+})
+.collect(Collectors.toList());
+}
+
+
+private static Object[] getDefaultParams(final Method method) {
+final int paramsCount = method.getParameterCount();
+final
[tomcat] 06/13: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit d6ddfaa1b3729feb78afd9093eaff4a3d2a2b859
Author: Mark Thomas
AuthorDate: Thu Sep 24 21:36:19 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Refactor StreamStateMachine to remove reference to Stream
---
java/org/apache/coyote/http2/Stream.java| 2 +-
.../org/apache/coyote/http2/StreamStateMachine.java | 21 -
2 files changed, 13 insertions(+), 10 deletions(-)
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index 78a288c..260ce54 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -94,7 +94,7 @@ public class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
this.handler = handler;
handler.addChild(this);
setWindowSize(handler.getRemoteSettings().getInitialWindowSize());
-state = new StreamStateMachine(this);
+state = new StreamStateMachine(getConnectionId(), getIdAsString());
if (coyoteRequest == null) {
// HTTP/2 new request
this.coyoteRequest = new Request();
diff --git a/java/org/apache/coyote/http2/StreamStateMachine.java
b/java/org/apache/coyote/http2/StreamStateMachine.java
index 66aee45..fdc06a4 100644
--- a/java/org/apache/coyote/http2/StreamStateMachine.java
+++ b/java/org/apache/coyote/http2/StreamStateMachine.java
@@ -39,12 +39,15 @@ public class StreamStateMachine {
private static final Log log = LogFactory.getLog(StreamStateMachine.class);
private static final StringManager sm =
StringManager.getManager(StreamStateMachine.class);
-private final Stream stream;
+private final String connectionId;
+private final String streamId;
+
private State state;
-public StreamStateMachine(Stream stream) {
-this.stream = stream;
+public StreamStateMachine(String connectionId, String streamId) {
+this.connectionId = connectionId;
+this.streamId = streamId;
stateChange(null, State.IDLE);
}
@@ -97,7 +100,7 @@ public class StreamStateMachine {
public synchronized void sendReset() {
if (state == State.IDLE) {
throw new
IllegalStateException(sm.getString("streamStateMachine.debug.change",
-stream.getConnectionId(), stream.getIdAsString(), state));
+connectionId, streamId, state));
}
if (state.canReset()) {
stateChange(state, State.CLOSED_RST_TX);
@@ -114,8 +117,8 @@ public class StreamStateMachine {
if (state == oldState) {
state = newState;
if (log.isDebugEnabled()) {
-log.debug(sm.getString("streamStateMachine.debug.change",
stream.getConnectionId(),
-stream.getIdAsString(), oldState, newState));
+log.debug(sm.getString("streamStateMachine.debug.change",
connectionId,
+streamId, oldState, newState));
}
}
}
@@ -127,12 +130,12 @@ public class StreamStateMachine {
if (!isFrameTypePermitted(frameType)) {
if (state.connectionErrorForInvalidFrame) {
throw new
ConnectionException(sm.getString("streamStateMachine.invalidFrame",
-stream.getConnectionId(), stream.getIdAsString(),
state, frameType),
+connectionId, streamId, state, frameType),
state.errorCodeForInvalidFrame);
} else {
throw new
StreamException(sm.getString("streamStateMachine.invalidFrame",
-stream.getConnectionId(), stream.getIdAsString(),
state, frameType),
-state.errorCodeForInvalidFrame, stream.getIdAsInt());
+connectionId, streamId, state, frameType),
+state.errorCodeForInvalidFrame,
Integer.parseInt(streamId));
}
}
}
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 07/13: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit b69d0c7fdf2e6c40d0d77819daffeff75dfcc296
Author: Mark Thomas
AuthorDate: Thu Sep 24 21:47:09 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Pull up state
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 21 -
java/org/apache/coyote/http2/RecycledStream.java| 16 +---
java/org/apache/coyote/http2/Stream.java| 20 +++-
3 files changed, 20 insertions(+), 37 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index e1764a1..bad3b76 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -31,17 +31,22 @@ abstract class AbstractNonZeroStream extends AbstractStream
{
private static final Log log =
LogFactory.getLog(AbstractNonZeroStream.class);
private static final StringManager sm =
StringManager.getManager(AbstractNonZeroStream.class);
+protected final StreamStateMachine state;
+
private volatile int weight;
-AbstractNonZeroStream(Integer identifier) {
-this(identifier, Constants.DEFAULT_WEIGHT);
+AbstractNonZeroStream(String connectionId, Integer identifier) {
+super(identifier);
+this.weight = Constants.DEFAULT_WEIGHT;
+this.state = new StreamStateMachine(connectionId, getIdAsString());
}
-AbstractNonZeroStream(Integer identifier, int weight) {
+AbstractNonZeroStream(Integer identifier, int weight, StreamStateMachine
state) {
super(identifier);
this.weight = weight;
+this.state = state;
}
@@ -97,7 +102,13 @@ abstract class AbstractNonZeroStream extends AbstractStream
{
this.weight = weight;
}
-abstract boolean isClosedFinal();
-abstract void checkState(FrameType frameType) throws Http2Exception;
+final boolean isClosedFinal() {
+return state.isClosedFinal();
+}
+
+
+final void checkState(FrameType frameType) throws Http2Exception {
+state.checkFrameType(frameType);
+}
}
diff --git a/java/org/apache/coyote/http2/RecycledStream.java
b/java/org/apache/coyote/http2/RecycledStream.java
index 7d408ea..e6f90f0 100644
--- a/java/org/apache/coyote/http2/RecycledStream.java
+++ b/java/org/apache/coyote/http2/RecycledStream.java
@@ -23,12 +23,10 @@ package org.apache.coyote.http2;
class RecycledStream extends AbstractNonZeroStream {
private final String connectionId;
-private final StreamStateMachine state;
RecycledStream(String connectionId, Integer identifier, int weight,
StreamStateMachine state) {
-super(identifier, weight);
+super(identifier, weight, state);
this.connectionId = connectionId;
-this.state = state;
}
@@ -38,18 +36,6 @@ class RecycledStream extends AbstractNonZeroStream {
}
-@Override
-boolean isClosedFinal() {
-return state.isClosedFinal();
-}
-
-
-@Override
-final void checkState(FrameType frameType) throws Http2Exception {
-state.checkFrameType(frameType);
-}
-
-
@SuppressWarnings("sync-override")
@Override
protected void incrementWindowSize(int increment) throws Http2Exception {
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index 260ce54..22d62ed 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -66,7 +66,6 @@ public class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
private volatile long contentLengthReceived = 0;
private final Http2UpgradeHandler handler;
-private final StreamStateMachine state;
private final WindowAllocationManager allocationManager = new
WindowAllocationManager(this);
// State machine would be too much overhead
@@ -90,11 +89,10 @@ public class Stream extends AbstractNonZeroStream
implements HeaderEmitter {
public Stream(Integer identifier, Http2UpgradeHandler handler, Request
coyoteRequest) {
-super(identifier);
+super(handler.getConnectionId(), identifier);
this.handler = handler;
handler.addChild(this);
setWindowSize(handler.getRemoteSettings().getInitialWindowSize());
-state = new StreamStateMachine(getConnectionId(), getIdAsString());
if (coyoteRequest == null) {
// HTTP/2 new request
this.coyoteRequest = new Request();
@@ -192,13 +190,7 @@ public class Stream extends AbstractNonZeroStream
implements HeaderEmitter {
@Override
-final
[tomcat] 05/13: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit fe31cb697a030f8fb703262082dc983150e684f5
Author: Mark Thomas
AuthorDate: Thu Sep 24 18:11:37 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Refactoring getStream to handle differences between Stream and
RecycledStream
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 2 +
.../apache/coyote/http2/Http2UpgradeHandler.java | 131 ++---
java/org/apache/coyote/http2/RecycledStream.java | 26 +++-
java/org/apache/coyote/http2/Stream.java | 22 ++--
4 files changed, 116 insertions(+), 65 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index d1eb38a..e1764a1 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -98,4 +98,6 @@ abstract class AbstractNonZeroStream extends AbstractStream {
}
abstract boolean isClosedFinal();
+
+abstract void checkState(FrameType frameType) throws Http2Exception;
}
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index 4e15e9d..751637c 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -23,7 +23,6 @@ import java.nio.charset.StandardCharsets;
import java.util.Collections;
import java.util.HashSet;
import java.util.Iterator;
-import java.util.Map;
import java.util.Map.Entry;
import java.util.Queue;
import java.util.Set;
@@ -127,8 +126,8 @@ public class Http2UpgradeHandler extends AbstractStream
implements InternalHttpU
private long keepAliveTimeout = Http2Protocol.DEFAULT_KEEP_ALIVE_TIMEOUT;
private long writeTimeout = Http2Protocol.DEFAULT_WRITE_TIMEOUT;
-private final Map streams = new ConcurrentHashMap<>();
-private final AtomicInteger activeRemoteStreamCount = new AtomicInteger(0);
+private final ConcurrentMap streams = new
ConcurrentHashMap<>();
+protected final AtomicInteger activeRemoteStreamCount = new
AtomicInteger(0);
// Start at -1 so the 'add 2' logic in closeIdleStreams() works
private volatile int maxActiveRemoteStreamId = -1;
private volatile int maxProcessedStreamId;
@@ -1082,7 +1081,22 @@ public class Http2UpgradeHandler extends AbstractStream
implements InternalHttpU
private Stream getStream(int streamId, boolean unknownIsError) throws
ConnectionException {
Integer key = Integer.valueOf(streamId);
-Stream result = streams.get(key);
+AbstractStream result = streams.get(key);
+if (result instanceof Stream) {
+return (Stream) result;
+}
+if (unknownIsError) {
+// Stream has been closed and removed from the map
+throw new
ConnectionException(sm.getString("upgradeHandler.stream.closed",
key.toString()),
+Http2Error.PROTOCOL_ERROR);
+}
+return null;
+}
+
+
+private AbstractNonZeroStream getStreamMayBeClosed(int streamId, boolean
unknownIsError) throws ConnectionException {
+Integer key = Integer.valueOf(streamId);
+AbstractNonZeroStream result = streams.get(key);
if (result == null && unknownIsError) {
// Stream has been closed and removed from the map
throw new
ConnectionException(sm.getString("upgradeHandler.stream.closed",
key.toString()),
@@ -1126,10 +1140,12 @@ public class Http2UpgradeHandler extends AbstractStream
implements InternalHttpU
return;
}
-for (Stream stream : streams.values()) {
-// The connection is closing. Close the associated streams as no
-// longer required (also notifies any threads waiting for
allocations).
-stream.receiveReset(Http2Error.CANCEL.getCode());
+for (AbstractNonZeroStream stream : streams.values()) {
+if (stream instanceof Stream) {
+// The connection is closing. Close the associated streams as
no
+// longer required (also notifies any threads waiting for
allocations).
+((Stream) stream).receiveReset(Http2Error.CANCEL.getCode());
+}
}
try {
socketWrapper.close();
@@ -1185,10 +1201,10 @@ public class Http2UpgradeHandler extends AbstractStream
implements InternalHttpU
TreeSet candidatesStepTwo = new TreeSet<>();
TreeSet candidatesStepThree = new TreeSet<>();
-for (Entry entry : streams.entrySet()) {
-Stream stream = entry.getValue();
+for (Entry
[tomcat] 04/13: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 5295fbc7cf0a3b487f92194d5184810fe9557292
Author: Mark Thomas
AuthorDate: Thu Sep 24 17:50:43 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Pull-up isClosedFinal()
---
java/org/apache/coyote/http2/AbstractNonZeroStream.java | 1 +
java/org/apache/coyote/http2/RecycledStream.java| 8
java/org/apache/coyote/http2/Stream.java| 3 ++-
3 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index e76496d..d1eb38a 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -97,4 +97,5 @@ abstract class AbstractNonZeroStream extends AbstractStream {
this.weight = weight;
}
+abstract boolean isClosedFinal();
}
diff --git a/java/org/apache/coyote/http2/RecycledStream.java
b/java/org/apache/coyote/http2/RecycledStream.java
index 17a02cb..c17ae02 100644
--- a/java/org/apache/coyote/http2/RecycledStream.java
+++ b/java/org/apache/coyote/http2/RecycledStream.java
@@ -23,10 +23,12 @@ package org.apache.coyote.http2;
class RecycledStream extends AbstractNonZeroStream {
private final String connectionId;
+private final boolean closedFinal;
RecycledStream(Stream stream) {
super(stream.getIdentifier(), stream.getWeight());
connectionId = stream.getConnectionId();
+closedFinal = stream.isClosedFinal();
}
@@ -37,6 +39,12 @@ class RecycledStream extends AbstractNonZeroStream {
@Override
+boolean isClosedFinal() {
+return closedFinal;
+}
+
+
+@Override
@Deprecated
protected synchronized void doNotifyAll() {
// NO-OP. Unused.
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index be33904..5f3543f 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -612,7 +612,8 @@ public class Stream extends AbstractNonZeroStream
implements HeaderEmitter {
}
-boolean isClosedFinal() {
+@Override
+final boolean isClosedFinal() {
return state.isClosedFinal();
}
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 02/13: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 07bbf4a6f410965a088bb909509e98c0e9ea6679
Author: Mark Thomas
AuthorDate: Wed Sep 23 19:59:13 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Initial changes to object hierarchy in preparation for further
changes.
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 29
java/org/apache/coyote/http2/AbstractStream.java | 3 +-
java/org/apache/coyote/http2/RecycledStream.java | 52 ++
java/org/apache/coyote/http2/Stream.java | 2 +-
4 files changed, 84 insertions(+), 2 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
new file mode 100644
index 000..63fb5e7
--- /dev/null
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.coyote.http2;
+
+
+/**
+ * Base class for all streams other than stream 0, the connection. Primarily
+ * provides functionality shared between full Stream and RecycledStream.
+ */
+abstract class AbstractNonZeroStream extends AbstractStream {
+
+AbstractNonZeroStream(Integer identifier) {
+super(identifier);
+}
+}
diff --git a/java/org/apache/coyote/http2/AbstractStream.java
b/java/org/apache/coyote/http2/AbstractStream.java
index 3e4ec73..9c6054e 100644
--- a/java/org/apache/coyote/http2/AbstractStream.java
+++ b/java/org/apache/coyote/http2/AbstractStream.java
@@ -25,7 +25,8 @@ import org.apache.juli.logging.LogFactory;
import org.apache.tomcat.util.res.StringManager;
/**
- * Used to managed prioritisation.
+ * Base class for all streams including the connection (referred to as Stream
0)
+ * and is used primarily when managing prioritization.
*/
abstract class AbstractStream {
diff --git a/java/org/apache/coyote/http2/RecycledStream.java
b/java/org/apache/coyote/http2/RecycledStream.java
new file mode 100644
index 000..4cd3e79
--- /dev/null
+++ b/java/org/apache/coyote/http2/RecycledStream.java
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.coyote.http2;
+
+/**
+ * Represents a closed stream in the priority tree. Used in preference to the
+ * full {@link Stream} as has much lower memory usage.
+ */
+class RecycledStream extends AbstractNonZeroStream {
+
+private final String connectionId;
+private int weight;
+
+RecycledStream(Stream stream) {
+super(stream.getIdentifier());
+connectionId = stream.getConnectionId();
+weight = stream.getWeight();
+}
+
+
+@Override
+protected String getConnectionId() {
+return connectionId;
+}
+
+
+@Override
+protected int getWeight() {
+return weight;
+}
+
+
+@Override
+@Deprecated
+protected synchronized void doNotifyAll() {
+// NO-OP. Unused.
+}
+}
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index b78d55a..d4cfa0d 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@
[tomcat] 08/13: Fully replace Stream with RecycledStream
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 8a0ab84ea67025ba75e545e5456fc28f051ace70
Author: Mark Thomas
AuthorDate: Fri Sep 25 12:26:40 2020 +0100
Fully replace Stream with RecycledStream
Profiler (YourKit) suggests retain memory for Stream ~60k whereas
RecycledStream ins ~300 bytes.
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 37 +++--
.../apache/coyote/http2/Http2UpgradeHandler.java | 64 +-
java/org/apache/coyote/http2/RecycledStream.java | 4 +-
java/org/apache/coyote/http2/Stream.java | 2 +-
4 files changed, 74 insertions(+), 33 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index bad3b76..d85840d 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -33,19 +33,17 @@ abstract class AbstractNonZeroStream extends AbstractStream
{
protected final StreamStateMachine state;
-private volatile int weight;
+private volatile int weight = Constants.DEFAULT_WEIGHT;
AbstractNonZeroStream(String connectionId, Integer identifier) {
super(identifier);
-this.weight = Constants.DEFAULT_WEIGHT;
this.state = new StreamStateMachine(connectionId, getIdAsString());
}
-AbstractNonZeroStream(Integer identifier, int weight, StreamStateMachine
state) {
+AbstractNonZeroStream(Integer identifier, StreamStateMachine state) {
super(identifier);
-this.weight = weight;
this.state = state;
}
@@ -56,6 +54,13 @@ abstract class AbstractNonZeroStream extends AbstractStream {
}
+/*
+ * General method used when reprioritising a stream and care needs to be
+ * taken not to create circular references.
+ *
+ * Changes to the priority tree need to be sychronized at the connection
+ * level. This is the caller's responsibility.
+ */
final void rePrioritise(AbstractStream parent, boolean exclusive, int
weight) {
if (log.isDebugEnabled()) {
log.debug(sm.getString("stream.reprioritisation.debug",
@@ -90,6 +95,9 @@ abstract class AbstractNonZeroStream extends AbstractStream {
/*
* Used when removing closed streams from the tree and we know there is no
* need to check for circular references.
+ *
+ * Changes to the priority tree need to be sychronized at the connection
+ * level. This is the caller's responsibility.
*/
final void rePrioritise(AbstractStream parent, int weight) {
if (log.isDebugEnabled()) {
@@ -103,6 +111,27 @@ abstract class AbstractNonZeroStream extends
AbstractStream {
}
+/*
+ * Used when "recycling" a stream and replacing a Stream instance with a
+ * RecycledStream instance.
+ *
+ * Replace this stream with the provided stream in the parent/child
+ * hierarchy.
+ *
+ * Changes to the priority tree need to be sychronized at the connection
+ * level. This is the caller's responsibility.
+ */
+void replaceStream(AbstractNonZeroStream replacement) {
+replacement.setParentStream(getParentStream());
+detachFromParent();
+for (AbstractNonZeroStream child : getChildStreams()) {
+replacement.addChild(child);
+}
+getChildStreams().clear();
+replacement.weight = weight;
+}
+
+
final boolean isClosedFinal() {
return state.isClosedFinal();
}
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index 751637c..295e9aa 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -93,6 +93,8 @@ public class Http2UpgradeHandler extends AbstractStream
implements InternalHttpU
private static final HeaderSink HEADER_SINK = new HeaderSink();
+private final Object priorityTreeLock = new Object();
+
private final String connectionId;
private final Http2Protocol protocol;
@@ -103,8 +105,7 @@ public class Http2UpgradeHandler extends AbstractStream
implements InternalHttpU
private volatile Http2Parser parser;
// Simple state machine (sequence of states)
-private AtomicReference connectionState =
-new AtomicReference<>(ConnectionState.NEW);
+private AtomicReference connectionState = new
AtomicReference<>(ConnectionState.NEW);
private volatile long pausedNanoTime = Long.MAX_VALUE;
/**
@@ -1166,7 +1167,7 @@ public class Http2UpgradeHandler extends AbstractStream
implements InternalHttpU
newStreamsSinceLastPrune = 0;
// RFC 7540, 5.3.4 endpoints should maintain state for at least the
-// maximum
[tomcat] 01/13: Align with 10.0.x and 9.0.x
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 8.5.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit 6daba8e92e359ab72ae45aa04caf21d376668f6c Author: Mark Thomas AuthorDate: Wed Sep 30 13:49:00 2020 +0100 Align with 10.0.x and 9.0.x --- build.xml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/build.xml b/build.xml index 327ce0b..9aa56ee 100644 --- a/build.xml +++ b/build.xml @@ -1555,7 +1555,7 @@ /> - + @@ -1564,8 +1564,6 @@ - - - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 03/13: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 50a931abd48271d80ad1d32536ae773d87dfddce
Author: Mark Thomas
AuthorDate: Thu Sep 24 17:32:49 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Pull up re-prioritisation
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 71 ++
java/org/apache/coyote/http2/AbstractStream.java | 8 +--
.../apache/coyote/http2/Http2UpgradeHandler.java | 10 +--
java/org/apache/coyote/http2/RecycledStream.java | 10 +--
java/org/apache/coyote/http2/Stream.java | 59 +-
5 files changed, 83 insertions(+), 75 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 63fb5e7..e76496d 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -16,6 +16,11 @@
*/
package org.apache.coyote.http2;
+import java.util.Iterator;
+
+import org.apache.juli.logging.Log;
+import org.apache.juli.logging.LogFactory;
+import org.apache.tomcat.util.res.StringManager;
/**
* Base class for all streams other than stream 0, the connection. Primarily
@@ -23,7 +28,73 @@ package org.apache.coyote.http2;
*/
abstract class AbstractNonZeroStream extends AbstractStream {
+private static final Log log =
LogFactory.getLog(AbstractNonZeroStream.class);
+private static final StringManager sm =
StringManager.getManager(AbstractNonZeroStream.class);
+
+private volatile int weight;
+
+
AbstractNonZeroStream(Integer identifier) {
+this(identifier, Constants.DEFAULT_WEIGHT);
+}
+
+
+AbstractNonZeroStream(Integer identifier, int weight) {
super(identifier);
+this.weight = weight;
+}
+
+
+@Override
+protected final int getWeight() {
+return weight;
}
+
+
+final void rePrioritise(AbstractStream parent, boolean exclusive, int
weight) {
+if (log.isDebugEnabled()) {
+log.debug(sm.getString("stream.reprioritisation.debug",
+getConnectionId(), getIdAsString(),
Boolean.toString(exclusive),
+parent.getIdAsString(), Integer.toString(weight)));
+}
+
+// Check if new parent is a descendant of this stream
+if (isDescendant(parent)) {
+parent.detachFromParent();
+// Cast is always safe since any descendant of this stream must be
+// an instance of Stream
+getParentStream().addChild((Stream) parent);
+}
+
+if (exclusive) {
+// Need to move children of the new parent to be children of this
+// stream. Slightly convoluted to avoid concurrent modification.
+Iterator parentsChildren =
parent.getChildStreams().iterator();
+while (parentsChildren.hasNext()) {
+AbstractNonZeroStream parentsChild = parentsChildren.next();
+parentsChildren.remove();
+this.addChild(parentsChild);
+}
+}
+detachFromParent();
+parent.addChild(this);
+this.weight = weight;
+}
+
+
+/*
+ * Used when removing closed streams from the tree and we know there is no
+ * need to check for circular references.
+ */
+final void rePrioritise(AbstractStream parent, int weight) {
+if (log.isDebugEnabled()) {
+log.debug(sm.getString("stream.reprioritisation.debug",
+getConnectionId(), getIdAsString(), Boolean.FALSE,
+parent.getIdAsString(), Integer.toString(weight)));
+}
+
+parent.addChild(this);
+this.weight = weight;
+}
+
}
diff --git a/java/org/apache/coyote/http2/AbstractStream.java
b/java/org/apache/coyote/http2/AbstractStream.java
index 9c6054e..43b8cde 100644
--- a/java/org/apache/coyote/http2/AbstractStream.java
+++ b/java/org/apache/coyote/http2/AbstractStream.java
@@ -37,8 +37,8 @@ abstract class AbstractStream {
private final String idAsString;
private volatile AbstractStream parentStream = null;
-private final Set childStreams =
-Collections.newSetFromMap(new ConcurrentHashMap());
+private final Set childStreams =
+Collections.newSetFromMap(new
ConcurrentHashMap());
private long windowSize =
ConnectionSettingsBase.DEFAULT_INITIAL_WINDOW_SIZE;
@@ -71,7 +71,7 @@ abstract class AbstractStream {
}
-final void addChild(Stream child) {
+final void addChild(AbstractNonZeroStream child) {
child.setParentStream(this);
childStreams.add(child);
}
@@ -98,7 +98,7 @@ abstract class
[tomcat] branch 8.5.x updated (c08d2d4 -> 3e5dd5c)
This is an automated email from the ASF dual-hosted git repository.
markt pushed a change to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git.
from c08d2d4 Use a link to TestTomcat in Git instead of SVN
new 6daba8e Align with 10.0.x and 9.0.x
new 07bbf4a Reduce memory footprint of closed http/2 streams
new 50a931a Reduce memory footprint of closed http/2 streams
new 5295fbc Reduce memory footprint of closed http/2 streams
new fe31cb6 Reduce memory footprint of closed http/2 streams
new d6ddfaa Reduce memory footprint of closed http/2 streams
new b69d0c7 Reduce memory footprint of closed http/2 streams
new 8a0ab84 Fully replace Stream with RecycledStream
new 3d3060b Refactor the pruning so more stream info is retained for
longer
new d64f3d5 Update changelog
new de7f52b Optimize the iteration when closing idle streams
new 55b8e01 Fix 64735 ServletContext.addJspFile() always fails with
SecurityManager
new 3e5dd5c Fix back-port of new unit test
The 13 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
build.properties.default | 6 +-
build.xml | 17 +-
.../catalina/core/ApplicationContextFacade.java| 5 +
.../apache/coyote/http2/AbstractNonZeroStream.java | 143 +++
java/org/apache/coyote/http2/AbstractStream.java | 11 +-
.../apache/coyote/http2/Http2UpgradeHandler.java | 273 +
.../http2/{HeaderSink.java => RecycledStream.java} | 34 ++-
java/org/apache/coyote/http2/Stream.java | 98 +---
.../apache/coyote/http2/StreamStateMachine.java| 21 +-
...estApplicationContextFacadeSecurityManager.java | 153
.../util/security/SecurityManagerBaseTest.java | 50
webapps/docs/changelog.xml | 10 +
12 files changed, 597 insertions(+), 224 deletions(-)
create mode 100644 java/org/apache/coyote/http2/AbstractNonZeroStream.java
copy java/org/apache/coyote/http2/{HeaderSink.java => RecycledStream.java}
(58%)
create mode 100644
test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
create mode 100644
test/org/apache/tomcat/util/security/SecurityManagerBaseTest.java
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 11/13: Optimize the iteration when closing idle streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit de7f52b53e57079a44701edc0932c4735c7d77c1
Author: Martin Tzvetanov Grigorov
AuthorDate: Mon Sep 28 14:53:25 2020 +0300
Optimize the iteration when closing idle streams
---
java/org/apache/coyote/http2/Http2UpgradeHandler.java | 15 ---
1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index fe60f32..2a2fee5 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -30,6 +30,7 @@ import java.util.TreeSet;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentNavigableMap;
import java.util.concurrent.ConcurrentSkipListMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
@@ -127,7 +128,7 @@ public class Http2UpgradeHandler extends AbstractStream
implements InternalHttpU
private long readTimeout = Http2Protocol.DEFAULT_READ_TIMEOUT;
private long keepAliveTimeout = Http2Protocol.DEFAULT_KEEP_ALIVE_TIMEOUT;
private long writeTimeout = Http2Protocol.DEFAULT_WRITE_TIMEOUT;
-private final ConcurrentMap streams = new
ConcurrentSkipListMap<>();
+private final ConcurrentNavigableMap
streams = new ConcurrentSkipListMap<>();
protected final AtomicInteger activeRemoteStreamCount = new
AtomicInteger(0);
// Start at -1 so the 'add 2' logic in closeIdleStreams() works
private volatile int maxActiveRemoteStreamId = -1;
@@ -1635,12 +1636,12 @@ public class Http2UpgradeHandler extends AbstractStream
implements InternalHttpU
private void closeIdleStreams(int newMaxActiveRemoteStreamId) {
-for (Entry entry : streams.entrySet()) {
-int id = entry.getKey().intValue();
-if (id > maxActiveRemoteStreamId && id <
newMaxActiveRemoteStreamId) {
-if (entry.getValue() instanceof Stream) {
-((Stream) entry.getValue()).closeIfIdle();
-}
+final ConcurrentNavigableMap subMap =
streams.subMap(
+Integer.valueOf(maxActiveRemoteStreamId), false,
+Integer.valueOf(newMaxActiveRemoteStreamId), false);
+for (AbstractNonZeroStream stream : subMap.values()) {
+if (stream instanceof Stream) {
+((Stream)stream).closeIfIdle();
}
}
maxActiveRemoteStreamId = newMaxActiveRemoteStreamId;
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 10/13: Update changelog
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 8.5.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit d64f3d5d1b34c3286b33c63023828a99f1e371d7 Author: Mark Thomas AuthorDate: Fri Sep 25 14:33:45 2020 +0100 Update changelog --- webapps/docs/changelog.xml | 5 + 1 file changed, 5 insertions(+) diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml index 3cfd81f..a8f955b 100644 --- a/webapps/docs/changelog.xml +++ b/webapps/docs/changelog.xml @@ -101,6 +101,11 @@ Don't send the Keep-Alive response header if the connection has been explicitly closed. (markt) + +Refactor the handling of closed HTTP/2 streams to reduce the heap usage +associated with used streams and to retain information for more streams +in the priority tree. (markt) + - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
buildbot success in on tomcat-9-trunk
The Buildbot has detected a restored build on builder tomcat-9-trunk while building tomcat. Full details are available at: https://ci.apache.org/builders/tomcat-9-trunk/builds/471 Buildbot URL: https://ci.apache.org/ Buildslave for this Build: asf946_ubuntu Build Reason: The AnyBranchScheduler scheduler named 'on-tomcat-9-commit' triggered this build Build Source Stamp: [branch 9.0.x] e44fd931075f63acd0405603fa47e9ee335d667a Blamelist: Kyle Stiemann ,Mark Thomas ,Martin Tzvetanov Grigorov Build succeeded! Sincerely, -The Buildbot - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 57129] Regression. Load WEB-INF/lib jarfiles in alphabetical order
https://bz.apache.org/bugzilla/show_bug.cgi?id=57129 --- Comment #44 from Mateusz Matela --- (In reply to mgrigorov from comment #42) > As Mark mentioned since recently Tomcat added functionality to scan the jars > in parallel. No matter whether the jars are sorted or not the results from > the parallel scanning will be shuffled. My app doesn't even use annotation scanning. A lot of legacy apps don't. Look, you can't save everyone who may have strange ideas about resource loading in the future. What we have now doesn't even save anyone, it only potentially makes them suffer earlier in hopes it would be less severe. But in order to achieve that, it makes lots of people suffer a lot right now. Mostly people, who just try to run their legacy app in a new Tomcat, because Tomcat 7 EOL approaches. That's mostly apps that won't use any fancy new features and may potentially never bump into this problem otherwise. > They really need to know about this problem and start fixing their > application! OK, if this is such a serious and prevalent problem, then the solution should not be "oh, let's have the system behave non-deterministically. But only if the underlying file system is also non-deterministic." Do you see how that sounds? Maybe Tomcat should scan the jars at context loading, detect all the duplicated resource paths and list them in logs with a big warning. What we have now definitely causes more pain than it saves. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 57129] Regression. Load WEB-INF/lib jarfiles in alphabetical order
https://bz.apache.org/bugzilla/show_bug.cgi?id=57129 --- Comment #43 from Bernhard Seebass --- (In reply to mgrigorov from comment #42) > > As Mark mentioned since recently Tomcat added functionality to scan the jars > in parallel. No matter whether the jars are sorted or not the results from > the parallel scanning will be shuffled. Now, this will be real fun. Instead of platform dependency, which can be handled by proper testing in staging environment, we will get race conditions and total unpredictability! Seems to be the time to have a closer look at https://github.com/basepom/duplicate-finder-maven-plugin to clean up conflicting maven dependencies. I just hope that PreResources (see Comment 3) stay supported and won't be loaded in parallel to allow overriding buggy classes in included jar-files. Regards Bernhard -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 05/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit a6e84280ac145ab14f86a2cbe37238f9acda31d6
Author: Mark Thomas
AuthorDate: Thu Sep 24 21:36:19 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Refactor StreamStateMachine to remove reference to Stream
---
java/org/apache/coyote/http2/Stream.java| 2 +-
.../org/apache/coyote/http2/StreamStateMachine.java | 21 -
2 files changed, 13 insertions(+), 10 deletions(-)
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index 8aa4a7b..f6210d7 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -97,7 +97,7 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
this.handler = handler;
handler.addChild(this);
setWindowSize(handler.getRemoteSettings().getInitialWindowSize());
-state = new StreamStateMachine(this);
+state = new StreamStateMachine(getConnectionId(), getIdAsString());
if (coyoteRequest == null) {
// HTTP/2 new request
this.coyoteRequest = new Request();
diff --git a/java/org/apache/coyote/http2/StreamStateMachine.java
b/java/org/apache/coyote/http2/StreamStateMachine.java
index 851e1a8..72028fd 100644
--- a/java/org/apache/coyote/http2/StreamStateMachine.java
+++ b/java/org/apache/coyote/http2/StreamStateMachine.java
@@ -39,12 +39,15 @@ class StreamStateMachine {
private static final Log log = LogFactory.getLog(StreamStateMachine.class);
private static final StringManager sm =
StringManager.getManager(StreamStateMachine.class);
-private final Stream stream;
+private final String connectionId;
+private final String streamId;
+
private State state;
-StreamStateMachine(Stream stream) {
-this.stream = stream;
+StreamStateMachine(String connectionId, String streamId) {
+this.connectionId = connectionId;
+this.streamId = streamId;
stateChange(null, State.IDLE);
}
@@ -92,7 +95,7 @@ class StreamStateMachine {
public synchronized void sendReset() {
if (state == State.IDLE) {
throw new
IllegalStateException(sm.getString("streamStateMachine.debug.change",
-stream.getConnectionId(), stream.getIdAsString(), state));
+connectionId, streamId, state));
}
if (state.canReset()) {
stateChange(state, State.CLOSED_RST_TX);
@@ -109,8 +112,8 @@ class StreamStateMachine {
if (state == oldState) {
state = newState;
if (log.isDebugEnabled()) {
-log.debug(sm.getString("streamStateMachine.debug.change",
stream.getConnectionId(),
-stream.getIdAsString(), oldState, newState));
+log.debug(sm.getString("streamStateMachine.debug.change",
connectionId,
+streamId, oldState, newState));
}
}
}
@@ -122,12 +125,12 @@ class StreamStateMachine {
if (!isFrameTypePermitted(frameType)) {
if (state.connectionErrorForInvalidFrame) {
throw new
ConnectionException(sm.getString("streamStateMachine.invalidFrame",
-stream.getConnectionId(), stream.getIdAsString(),
state, frameType),
+connectionId, streamId, state, frameType),
state.errorCodeForInvalidFrame);
} else {
throw new
StreamException(sm.getString("streamStateMachine.invalidFrame",
-stream.getConnectionId(), stream.getIdAsString(),
state, frameType),
-state.errorCodeForInvalidFrame, stream.getIdAsInt());
+connectionId, streamId, state, frameType),
+state.errorCodeForInvalidFrame,
Integer.parseInt(streamId));
}
}
}
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] branch 9.0.x updated (f258efe -> e44fd93)
This is an automated email from the ASF dual-hosted git repository. markt pushed a change to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git. from f258efe Optimize the iteration when closing idle streams new 5e7c765 Align with 10.0.x new e44fd93 Fix 64735 ServletContext.addJspFile() always fails with SecurityManager The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: build.xml | 17 ++- .../catalina/core/ApplicationContextFacade.java| 5 + ...estApplicationContextFacadeSecurityManager.java | 148 + .../util/security/SecurityManagerBaseTest.java | 50 +++ webapps/docs/changelog.xml | 5 + 5 files changed, 222 insertions(+), 3 deletions(-) create mode 100644 test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java create mode 100644 test/org/apache/tomcat/util/security/SecurityManagerBaseTest.java - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] branch 9.0.x updated (f258efe -> e44fd93)
This is an automated email from the ASF dual-hosted git repository. markt pushed a change to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git. from f258efe Optimize the iteration when closing idle streams new 5e7c765 Align with 10.0.x new e44fd93 Fix 64735 ServletContext.addJspFile() always fails with SecurityManager The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: build.xml | 17 ++- .../catalina/core/ApplicationContextFacade.java| 5 + ...estApplicationContextFacadeSecurityManager.java | 148 + .../util/security/SecurityManagerBaseTest.java | 50 +++ webapps/docs/changelog.xml | 5 + 5 files changed, 222 insertions(+), 3 deletions(-) create mode 100644 test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java create mode 100644 test/org/apache/tomcat/util/security/SecurityManagerBaseTest.java - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 07/10: Fully replace Stream with RecycledStream
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit d1f204ee12ed3aeb7713b95661a53ddaadd7e9b7
Author: Mark Thomas
AuthorDate: Fri Sep 25 12:26:40 2020 +0100
Fully replace Stream with RecycledStream
Profiler (YourKit) suggests retain memory for Stream ~60k whereas
RecycledStream ins ~300 bytes.
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 37 +++--
.../apache/coyote/http2/Http2UpgradeHandler.java | 64 +-
java/org/apache/coyote/http2/RecycledStream.java | 4 +-
java/org/apache/coyote/http2/Stream.java | 2 +-
4 files changed, 74 insertions(+), 33 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 084de33..bbdfa33 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -33,19 +33,17 @@ abstract class AbstractNonZeroStream extends AbstractStream
{
protected final StreamStateMachine state;
-private volatile int weight;
+private volatile int weight = Constants.DEFAULT_WEIGHT;
AbstractNonZeroStream(String connectionId, Integer identifier) {
super(identifier);
-this.weight = Constants.DEFAULT_WEIGHT;
this.state = new StreamStateMachine(connectionId, getIdAsString());
}
-AbstractNonZeroStream(Integer identifier, int weight, StreamStateMachine
state) {
+AbstractNonZeroStream(Integer identifier, StreamStateMachine state) {
super(identifier);
-this.weight = weight;
this.state = state;
}
@@ -56,6 +54,13 @@ abstract class AbstractNonZeroStream extends AbstractStream {
}
+/*
+ * General method used when reprioritising a stream and care needs to be
+ * taken not to create circular references.
+ *
+ * Changes to the priority tree need to be sychronized at the connection
+ * level. This is the caller's responsibility.
+ */
final void rePrioritise(AbstractStream parent, boolean exclusive, int
weight) {
if (log.isDebugEnabled()) {
log.debug(sm.getString("stream.reprioritisation.debug",
@@ -90,6 +95,9 @@ abstract class AbstractNonZeroStream extends AbstractStream {
/*
* Used when removing closed streams from the tree and we know there is no
* need to check for circular references.
+ *
+ * Changes to the priority tree need to be sychronized at the connection
+ * level. This is the caller's responsibility.
*/
final void rePrioritise(AbstractStream parent, int weight) {
if (log.isDebugEnabled()) {
@@ -103,6 +111,27 @@ abstract class AbstractNonZeroStream extends
AbstractStream {
}
+/*
+ * Used when "recycling" a stream and replacing a Stream instance with a
+ * RecycledStream instance.
+ *
+ * Replace this stream with the provided stream in the parent/child
+ * hierarchy.
+ *
+ * Changes to the priority tree need to be sychronized at the connection
+ * level. This is the caller's responsibility.
+ */
+void replaceStream(AbstractNonZeroStream replacement) {
+replacement.setParentStream(getParentStream());
+detachFromParent();
+for (AbstractNonZeroStream child : getChildStreams()) {
+replacement.addChild(child);
+}
+getChildStreams().clear();
+replacement.weight = weight;
+}
+
+
final boolean isClosedFinal() {
return state.isClosedFinal();
}
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index c51efd9..aebb70b 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -94,6 +94,8 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private static final HeaderSink HEADER_SINK = new HeaderSink();
+private final Object priorityTreeLock = new Object();
+
protected final String connectionId;
protected final Http2Protocol protocol;
@@ -104,8 +106,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private volatile Http2Parser parser;
// Simple state machine (sequence of states)
-private AtomicReference connectionState =
-new AtomicReference<>(ConnectionState.NEW);
+private AtomicReference connectionState = new
AtomicReference<>(ConnectionState.NEW);
private volatile long pausedNanoTime = Long.MAX_VALUE;
/**
@@ -1175,7 +1176,7 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
newStreamsSinceLastPrune = 0;
// RFC 7540, 5.3.4 endpoints should maintain state for at least the
-//
[tomcat] 08/10: Refactor the pruning so more stream info is retained for longer
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 83f5e03b52c54a9b213e979c9bb47e0967530cbe
Author: Mark Thomas
AuthorDate: Fri Sep 25 14:14:06 2020 +0100
Refactor the pruning so more stream info is retained for longer
The memory impact of this is mitigated by the previous changes to
replace closed Stream instances with RecycledStream instances.
---
.../apache/coyote/http2/Http2UpgradeHandler.java | 89 --
1 file changed, 49 insertions(+), 40 deletions(-)
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index aebb70b..9f71af9 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -30,6 +30,7 @@ import java.util.TreeSet;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentSkipListMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
@@ -123,7 +124,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private HpackDecoder hpackDecoder;
private HpackEncoder hpackEncoder;
-private final ConcurrentMap streams = new
ConcurrentHashMap<>();
+private final ConcurrentMap streams = new
ConcurrentSkipListMap<>();
protected final AtomicInteger activeRemoteStreamCount = new
AtomicInteger(0);
// Start at -1 so the 'add 2' logic in closeIdleStreams() works
private volatile int maxActiveRemoteStreamId = -1;
@@ -1207,78 +1208,86 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
// 2. Completed streams used for a request with children
// 3. Closed final streams
//
-// Steps 1 and 2 will always be completed.
-// Step 3 will be completed to the minimum extent necessary to bring
the
-// total number of streams under the limit.
+// The pruning halts as soon as enough streams have been pruned.
// Use these sets to track the different classes of streams
-TreeSet candidatesStepOne = new TreeSet<>();
TreeSet candidatesStepTwo = new TreeSet<>();
TreeSet candidatesStepThree = new TreeSet<>();
-for (Entry entry : streams.entrySet())
{
-AbstractNonZeroStream stream = entry.getValue();
+// Step 1
+// Iterator is in key order so we automatically have the oldest streams
+// first
+for (AbstractNonZeroStream stream : streams.values()) {
// Never remove active streams
if (stream instanceof Stream && ((Stream) stream).isActive()) {
continue;
}
-final Integer streamIdentifier = entry.getKey();
if (stream.isClosedFinal()) {
// This stream went from IDLE to CLOSED and is likely to have
// been created by the client as part of the priority tree.
-candidatesStepThree.add(streamIdentifier);
+// Candidate for steo 3.
+candidatesStepThree.add(stream.getIdentifier());
} else if (stream.getChildStreams().size() == 0) {
-// Closed, no children
-candidatesStepOne.add(streamIdentifier);
-} else {
-// Closed, with children
-candidatesStepTwo.add(streamIdentifier);
-}
-}
-
-// Process the step one list
-for (Integer streamIdToRemove : candidatesStepOne) {
-// Remove this childless stream
-AbstractNonZeroStream removedStream =
streams.remove(streamIdToRemove);
-removedStream.detachFromParent();
-toClose--;
-if (log.isDebugEnabled()) {
-log.debug(sm.getString("upgradeHandler.pruned", connectionId,
streamIdToRemove));
-}
-
-// Did this make the parent childless?
-AbstractStream parent = removedStream.getParentStream();
-while (parent instanceof Stream && !((Stream) parent).isActive() &&
-!((Stream) parent).isClosedFinal() &&
parent.getChildStreams().size() == 0) {
-streams.remove(parent.getIdentifier());
-parent.detachFromParent();
-toClose--;
+// Prune it
+AbstractStream parent = stream.getParentStream();
+streams.remove(stream.getIdentifier());
+stream.detachFromParent();
if (log.isDebugEnabled()) {
-log.debug(sm.getString("upgradeHandler.pruned",
connectionId, streamIdToRemove));
+
[Bug 64751] Incomplete module info descriptor
https://bz.apache.org/bugzilla/show_bug.cgi?id=64751 --- Comment #4 from rotty3000 --- Nice fix markt! -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 01/02: Align with 10.0.x
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit 5e7c765448993975a575af836ad900c13b29960c Author: Mark Thomas AuthorDate: Wed Sep 30 13:48:47 2020 +0100 Align with 10.0.x --- build.xml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/build.xml b/build.xml index f09f209..6a1d141 100644 --- a/build.xml +++ b/build.xml @@ -1958,7 +1958,7 @@ /> - + @@ -1969,8 +1969,6 @@ - - - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 01/02: Align with 10.0.x
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit 5e7c765448993975a575af836ad900c13b29960c Author: Mark Thomas AuthorDate: Wed Sep 30 13:48:47 2020 +0100 Align with 10.0.x --- build.xml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/build.xml b/build.xml index f09f209..6a1d141 100644 --- a/build.xml +++ b/build.xml @@ -1958,7 +1958,7 @@ /> - + @@ -1969,8 +1969,6 @@ - - - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 05/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit a6e84280ac145ab14f86a2cbe37238f9acda31d6
Author: Mark Thomas
AuthorDate: Thu Sep 24 21:36:19 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Refactor StreamStateMachine to remove reference to Stream
---
java/org/apache/coyote/http2/Stream.java| 2 +-
.../org/apache/coyote/http2/StreamStateMachine.java | 21 -
2 files changed, 13 insertions(+), 10 deletions(-)
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index 8aa4a7b..f6210d7 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -97,7 +97,7 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
this.handler = handler;
handler.addChild(this);
setWindowSize(handler.getRemoteSettings().getInitialWindowSize());
-state = new StreamStateMachine(this);
+state = new StreamStateMachine(getConnectionId(), getIdAsString());
if (coyoteRequest == null) {
// HTTP/2 new request
this.coyoteRequest = new Request();
diff --git a/java/org/apache/coyote/http2/StreamStateMachine.java
b/java/org/apache/coyote/http2/StreamStateMachine.java
index 851e1a8..72028fd 100644
--- a/java/org/apache/coyote/http2/StreamStateMachine.java
+++ b/java/org/apache/coyote/http2/StreamStateMachine.java
@@ -39,12 +39,15 @@ class StreamStateMachine {
private static final Log log = LogFactory.getLog(StreamStateMachine.class);
private static final StringManager sm =
StringManager.getManager(StreamStateMachine.class);
-private final Stream stream;
+private final String connectionId;
+private final String streamId;
+
private State state;
-StreamStateMachine(Stream stream) {
-this.stream = stream;
+StreamStateMachine(String connectionId, String streamId) {
+this.connectionId = connectionId;
+this.streamId = streamId;
stateChange(null, State.IDLE);
}
@@ -92,7 +95,7 @@ class StreamStateMachine {
public synchronized void sendReset() {
if (state == State.IDLE) {
throw new
IllegalStateException(sm.getString("streamStateMachine.debug.change",
-stream.getConnectionId(), stream.getIdAsString(), state));
+connectionId, streamId, state));
}
if (state.canReset()) {
stateChange(state, State.CLOSED_RST_TX);
@@ -109,8 +112,8 @@ class StreamStateMachine {
if (state == oldState) {
state = newState;
if (log.isDebugEnabled()) {
-log.debug(sm.getString("streamStateMachine.debug.change",
stream.getConnectionId(),
-stream.getIdAsString(), oldState, newState));
+log.debug(sm.getString("streamStateMachine.debug.change",
connectionId,
+streamId, oldState, newState));
}
}
}
@@ -122,12 +125,12 @@ class StreamStateMachine {
if (!isFrameTypePermitted(frameType)) {
if (state.connectionErrorForInvalidFrame) {
throw new
ConnectionException(sm.getString("streamStateMachine.invalidFrame",
-stream.getConnectionId(), stream.getIdAsString(),
state, frameType),
+connectionId, streamId, state, frameType),
state.errorCodeForInvalidFrame);
} else {
throw new
StreamException(sm.getString("streamStateMachine.invalidFrame",
-stream.getConnectionId(), stream.getIdAsString(),
state, frameType),
-state.errorCodeForInvalidFrame, stream.getIdAsInt());
+connectionId, streamId, state, frameType),
+state.errorCodeForInvalidFrame,
Integer.parseInt(streamId));
}
}
}
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 09/10: Update changelog
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit a4054eb3f27ac0d242d5f2a6c4599e3f6cb9a519 Author: Mark Thomas AuthorDate: Fri Sep 25 14:33:45 2020 +0100 Update changelog --- webapps/docs/changelog.xml | 5 + 1 file changed, 5 insertions(+) diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml index 0be3539..a82ec7b 100644 --- a/webapps/docs/changelog.xml +++ b/webapps/docs/changelog.xml @@ -127,6 +127,11 @@ (markt) +Refactor the handling of closed HTTP/2 streams to reduce the heap usage +associated with used streams and to retain information for more streams +in the priority tree. (markt) + + Don't send the Keep-Alive response header if the connection has been explicitly closed. (markt) - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 06/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 2b2ac4f3ce99027d8589595e921570495f7b08c0
Author: Mark Thomas
AuthorDate: Thu Sep 24 21:47:09 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Pull up state
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 21 -
java/org/apache/coyote/http2/RecycledStream.java| 16 +---
java/org/apache/coyote/http2/Stream.java| 16 +---
3 files changed, 18 insertions(+), 35 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 582ab1e..084de33 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -31,17 +31,22 @@ abstract class AbstractNonZeroStream extends AbstractStream
{
private static final Log log =
LogFactory.getLog(AbstractNonZeroStream.class);
private static final StringManager sm =
StringManager.getManager(AbstractNonZeroStream.class);
+protected final StreamStateMachine state;
+
private volatile int weight;
-AbstractNonZeroStream(Integer identifier) {
-this(identifier, Constants.DEFAULT_WEIGHT);
+AbstractNonZeroStream(String connectionId, Integer identifier) {
+super(identifier);
+this.weight = Constants.DEFAULT_WEIGHT;
+this.state = new StreamStateMachine(connectionId, getIdAsString());
}
-AbstractNonZeroStream(Integer identifier, int weight) {
+AbstractNonZeroStream(Integer identifier, int weight, StreamStateMachine
state) {
super(identifier);
this.weight = weight;
+this.state = state;
}
@@ -97,7 +102,13 @@ abstract class AbstractNonZeroStream extends AbstractStream
{
this.weight = weight;
}
-abstract boolean isClosedFinal();
-abstract void checkState(FrameType frameType) throws Http2Exception;
+final boolean isClosedFinal() {
+return state.isClosedFinal();
+}
+
+
+final void checkState(FrameType frameType) throws Http2Exception {
+state.checkFrameType(frameType);
+}
}
diff --git a/java/org/apache/coyote/http2/RecycledStream.java
b/java/org/apache/coyote/http2/RecycledStream.java
index 1915dff..9d6177c 100644
--- a/java/org/apache/coyote/http2/RecycledStream.java
+++ b/java/org/apache/coyote/http2/RecycledStream.java
@@ -23,12 +23,10 @@ package org.apache.coyote.http2;
class RecycledStream extends AbstractNonZeroStream {
private final String connectionId;
-private final StreamStateMachine state;
RecycledStream(String connectionId, Integer identifier, int weight,
StreamStateMachine state) {
-super(identifier, weight);
+super(identifier, weight, state);
this.connectionId = connectionId;
-this.state = state;
}
@@ -38,18 +36,6 @@ class RecycledStream extends AbstractNonZeroStream {
}
-@Override
-boolean isClosedFinal() {
-return state.isClosedFinal();
-}
-
-
-@Override
-final void checkState(FrameType frameType) throws Http2Exception {
-state.checkFrameType(frameType);
-}
-
-
@SuppressWarnings("sync-override")
@Override
void incrementWindowSize(int increment) throws Http2Exception {
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index f6210d7..140a76d 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -69,7 +69,6 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
private volatile long contentLengthReceived = 0;
private final Http2UpgradeHandler handler;
-private final StreamStateMachine state;
private final WindowAllocationManager allocationManager = new
WindowAllocationManager(this);
// State machine would be too much overhead
@@ -93,11 +92,10 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
Stream(Integer identifier, Http2UpgradeHandler handler, Request
coyoteRequest) {
-super(identifier);
+super(handler.getConnectionId(), identifier);
this.handler = handler;
handler.addChild(this);
setWindowSize(handler.getRemoteSettings().getInitialWindowSize());
-state = new StreamStateMachine(getConnectionId(), getIdAsString());
if (coyoteRequest == null) {
// HTTP/2 new request
this.coyoteRequest = new Request();
@@ -194,12 +192,6 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
@Override
-final void checkState(FrameType frameType) throws
[tomcat] branch 9.0.x updated (8dcf599 -> f258efe)
This is an automated email from the ASF dual-hosted git repository.
markt pushed a change to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git.
from 8dcf599 HTTP2 log statement fixes:
new 0d79ae5 Reduce memory footprint of closed http/2 streams
new e1a53fb Reduce memory footprint of closed http/2 streams
new 3783dc8 Reduce memory footprint of closed http/2 streams
new 496b3cf Reduce memory footprint of closed http/2 streams
new a6e8428 Reduce memory footprint of closed http/2 streams
new 2b2ac4f Reduce memory footprint of closed http/2 streams
new d1f204e Fully replace Stream with RecycledStream
new 83f5e03 Refactor the pruning so more stream info is retained for
longer
new a4054eb Update changelog
new f258efe Optimize the iteration when closing idle streams
The 10 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../apache/coyote/http2/AbstractNonZeroStream.java | 143 +++
java/org/apache/coyote/http2/AbstractStream.java | 9 +-
.../apache/coyote/http2/Http2UpgradeHandler.java | 264 +
...tionSettingsRemote.java => RecycledStream.java} | 23 +-
java/org/apache/coyote/http2/Stream.java | 90 +--
.../apache/coyote/http2/StreamStateMachine.java| 21 +-
webapps/docs/changelog.xml | 5 +
7 files changed, 348 insertions(+), 207 deletions(-)
create mode 100644 java/org/apache/coyote/http2/AbstractNonZeroStream.java
copy java/org/apache/coyote/http2/{ConnectionSettingsRemote.java =>
RecycledStream.java} (61%)
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 02/02: Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit e44fd931075f63acd0405603fa47e9ee335d667a
Author: Kyle Stiemann
AuthorDate: Thu Sep 10 16:47:21 2020 -0400
Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
---
build.xml | 13 ++
.../catalina/core/ApplicationContextFacade.java| 5 +
...estApplicationContextFacadeSecurityManager.java | 148 +
.../util/security/SecurityManagerBaseTest.java | 50 +++
webapps/docs/changelog.xml | 5 +
5 files changed, 221 insertions(+)
diff --git a/build.xml b/build.xml
index 6a1d141..56a9701 100644
--- a/build.xml
+++ b/build.xml
@@ -1969,8 +1969,21 @@
+
+
+
+
+
+
diff --git a/java/org/apache/catalina/core/ApplicationContextFacade.java
b/java/org/apache/catalina/core/ApplicationContextFacade.java
index 8d506b9..67065b0 100644
--- a/java/org/apache/catalina/core/ApplicationContextFacade.java
+++ b/java/org/apache/catalina/core/ApplicationContextFacade.java
@@ -117,6 +117,11 @@ public class ApplicationContextFacade implements
ServletContext {
classCache.put("getAttribute", clazz);
classCache.put("log", clazz);
classCache.put("setSessionTrackingModes", new Class[]{Set.class} );
+classCache.put("addJspFile", new Class[]{String.class, String.class});
+classCache.put("declareRoles", new Class[]{String[].class});
+classCache.put("setSessionTimeout", new Class[]{int.class});
+classCache.put("setRequestCharacterEncoding", new
Class[]{String.class});
+classCache.put("setResponseCharacterEncoding", new
Class[]{String.class});
}
diff --git
a/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
new file mode 100644
index 000..fc5a5d5
--- /dev/null
+++
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.catalina.core;
+
+import java.lang.reflect.Array;
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.lang.reflect.Modifier;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.stream.Collectors;
+
+import org.junit.Assert;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameter;
+
+import org.apache.catalina.security.SecurityUtil;
+import org.apache.tomcat.util.security.SecurityManagerBaseTest;
+import org.easymock.EasyMock;
+import org.easymock.IExpectationSetters;
+import org.easymock.internal.LastControl;
+
+@RunWith(Parameterized.class)
+public final class TestApplicationContextFacadeSecurityManager extends
SecurityManagerBaseTest {
+
+/**
+ * @return {@link Collection} of non-static, non-object, public {@link
+ * Method}s in {@link ApplicationContextFacade} to be run with the the Java
+ * 2 {@link SecurityManager} been enabled.
+ */
+@Parameterized.Parameters(name = "{index}: method={0}")
+public static Collection publicApplicationContextFacadeMethods() {
+return Arrays.stream(ApplicationContextFacade.class.getMethods())
+.filter(method -> !Modifier.isStatic(method.getModifiers()))
+.filter(method -> {
+try {
+Object.class.getMethod(method.getName(),
method.getParameterTypes());
+return false;
+} catch (final NoSuchMethodException e) {
+return true;
+}
+})
+.collect(Collectors.toList());
+}
+
+
+private static Object[] getDefaultParams(final Method method) {
+final int paramsCount = method.getParameterCount();
+final
[tomcat] 04/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 496b3cf5dd99179cd5460c728819bd799f662f99
Author: Mark Thomas
AuthorDate: Thu Sep 24 18:11:37 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Refactoring getStream to handle differences between Stream and
RecycledStream
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 2 +
.../apache/coyote/http2/Http2UpgradeHandler.java | 125 ++---
java/org/apache/coyote/http2/RecycledStream.java | 26 -
java/org/apache/coyote/http2/Stream.java | 20 ++--
4 files changed, 111 insertions(+), 62 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 70cfe35..582ab1e 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -98,4 +98,6 @@ abstract class AbstractNonZeroStream extends AbstractStream {
}
abstract boolean isClosedFinal();
+
+abstract void checkState(FrameType frameType) throws Http2Exception;
}
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index ad5e0e4..c51efd9 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -29,6 +29,7 @@ import java.util.Set;
import java.util.TreeSet;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
@@ -121,7 +122,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private HpackDecoder hpackDecoder;
private HpackEncoder hpackEncoder;
-private final Map streams = new ConcurrentHashMap<>();
+private final ConcurrentMap streams = new
ConcurrentHashMap<>();
protected final AtomicInteger activeRemoteStreamCount = new
AtomicInteger(0);
// Start at -1 so the 'add 2' logic in closeIdleStreams() works
private volatile int maxActiveRemoteStreamId = -1;
@@ -1089,7 +1090,22 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
private Stream getStream(int streamId, boolean unknownIsError) throws
ConnectionException {
Integer key = Integer.valueOf(streamId);
-Stream result = streams.get(key);
+AbstractStream result = streams.get(key);
+if (result instanceof Stream) {
+return (Stream) result;
+}
+if (unknownIsError) {
+// Stream has been closed and removed from the map
+throw new
ConnectionException(sm.getString("upgradeHandler.stream.closed",
key.toString()),
+Http2Error.PROTOCOL_ERROR);
+}
+return null;
+}
+
+
+private AbstractNonZeroStream getStreamMayBeClosed(int streamId, boolean
unknownIsError) throws ConnectionException {
+Integer key = Integer.valueOf(streamId);
+AbstractNonZeroStream result = streams.get(key);
if (result == null && unknownIsError) {
// Stream has been closed and removed from the map
throw new
ConnectionException(sm.getString("upgradeHandler.stream.closed",
key.toString()),
@@ -1133,10 +1149,12 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
return;
}
-for (Stream stream : streams.values()) {
-// The connection is closing. Close the associated streams as no
-// longer required (also notifies any threads waiting for
allocations).
-stream.receiveReset(Http2Error.CANCEL.getCode());
+for (AbstractNonZeroStream stream : streams.values()) {
+if (stream instanceof Stream) {
+// The connection is closing. Close the associated streams as
no
+// longer required (also notifies any threads waiting for
allocations).
+((Stream) stream).receiveReset(Http2Error.CANCEL.getCode());
+}
}
try {
socketWrapper.close();
@@ -1193,10 +1211,10 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
TreeSet candidatesStepTwo = new TreeSet<>();
TreeSet candidatesStepThree = new TreeSet<>();
-for (Entry entry : streams.entrySet()) {
-Stream stream = entry.getValue();
+for (Entry entry : streams.entrySet())
{
+AbstractNonZeroStream stream =
[tomcat] 01/02: Align with 10.0.x
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit 5e7c765448993975a575af836ad900c13b29960c Author: Mark Thomas AuthorDate: Wed Sep 30 13:48:47 2020 +0100 Align with 10.0.x --- build.xml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/build.xml b/build.xml index f09f209..6a1d141 100644 --- a/build.xml +++ b/build.xml @@ -1958,7 +1958,7 @@ /> - + @@ -1969,8 +1969,6 @@ - - - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 09/10: Update changelog
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit a4054eb3f27ac0d242d5f2a6c4599e3f6cb9a519 Author: Mark Thomas AuthorDate: Fri Sep 25 14:33:45 2020 +0100 Update changelog --- webapps/docs/changelog.xml | 5 + 1 file changed, 5 insertions(+) diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml index 0be3539..a82ec7b 100644 --- a/webapps/docs/changelog.xml +++ b/webapps/docs/changelog.xml @@ -127,6 +127,11 @@ (markt) +Refactor the handling of closed HTTP/2 streams to reduce the heap usage +associated with used streams and to retain information for more streams +in the priority tree. (markt) + + Don't send the Keep-Alive response header if the connection has been explicitly closed. (markt) - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 03/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 3783dc83065414cd85179e9a9014da44780e14b6
Author: Mark Thomas
AuthorDate: Thu Sep 24 17:50:43 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Pull-up isClosedFinal()
---
java/org/apache/coyote/http2/AbstractNonZeroStream.java | 1 +
java/org/apache/coyote/http2/RecycledStream.java| 8
java/org/apache/coyote/http2/Stream.java| 1 +
3 files changed, 10 insertions(+)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index ce08cc5..70cfe35 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -97,4 +97,5 @@ abstract class AbstractNonZeroStream extends AbstractStream {
this.weight = weight;
}
+abstract boolean isClosedFinal();
}
diff --git a/java/org/apache/coyote/http2/RecycledStream.java
b/java/org/apache/coyote/http2/RecycledStream.java
index 1e630df..dbbdc10 100644
--- a/java/org/apache/coyote/http2/RecycledStream.java
+++ b/java/org/apache/coyote/http2/RecycledStream.java
@@ -23,10 +23,12 @@ package org.apache.coyote.http2;
class RecycledStream extends AbstractNonZeroStream {
private final String connectionId;
+private final boolean closedFinal;
RecycledStream(Stream stream) {
super(stream.getIdentifier(), stream.getWeight());
connectionId = stream.getConnectionId();
+closedFinal = stream.isClosedFinal();
}
@@ -34,4 +36,10 @@ class RecycledStream extends AbstractNonZeroStream {
String getConnectionId() {
return connectionId;
}
+
+
+@Override
+boolean isClosedFinal() {
+return closedFinal;
+}
}
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index ef7ed29..489ce53 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -646,6 +646,7 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
}
+@Override
final boolean isClosedFinal() {
return state.isClosedFinal();
}
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 02/02: Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit e44fd931075f63acd0405603fa47e9ee335d667a
Author: Kyle Stiemann
AuthorDate: Thu Sep 10 16:47:21 2020 -0400
Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
---
build.xml | 13 ++
.../catalina/core/ApplicationContextFacade.java| 5 +
...estApplicationContextFacadeSecurityManager.java | 148 +
.../util/security/SecurityManagerBaseTest.java | 50 +++
webapps/docs/changelog.xml | 5 +
5 files changed, 221 insertions(+)
diff --git a/build.xml b/build.xml
index 6a1d141..56a9701 100644
--- a/build.xml
+++ b/build.xml
@@ -1969,8 +1969,21 @@
+
+
+
+
+
+
diff --git a/java/org/apache/catalina/core/ApplicationContextFacade.java
b/java/org/apache/catalina/core/ApplicationContextFacade.java
index 8d506b9..67065b0 100644
--- a/java/org/apache/catalina/core/ApplicationContextFacade.java
+++ b/java/org/apache/catalina/core/ApplicationContextFacade.java
@@ -117,6 +117,11 @@ public class ApplicationContextFacade implements
ServletContext {
classCache.put("getAttribute", clazz);
classCache.put("log", clazz);
classCache.put("setSessionTrackingModes", new Class[]{Set.class} );
+classCache.put("addJspFile", new Class[]{String.class, String.class});
+classCache.put("declareRoles", new Class[]{String[].class});
+classCache.put("setSessionTimeout", new Class[]{int.class});
+classCache.put("setRequestCharacterEncoding", new
Class[]{String.class});
+classCache.put("setResponseCharacterEncoding", new
Class[]{String.class});
}
diff --git
a/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
new file mode 100644
index 000..fc5a5d5
--- /dev/null
+++
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.catalina.core;
+
+import java.lang.reflect.Array;
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.lang.reflect.Modifier;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.stream.Collectors;
+
+import org.junit.Assert;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameter;
+
+import org.apache.catalina.security.SecurityUtil;
+import org.apache.tomcat.util.security.SecurityManagerBaseTest;
+import org.easymock.EasyMock;
+import org.easymock.IExpectationSetters;
+import org.easymock.internal.LastControl;
+
+@RunWith(Parameterized.class)
+public final class TestApplicationContextFacadeSecurityManager extends
SecurityManagerBaseTest {
+
+/**
+ * @return {@link Collection} of non-static, non-object, public {@link
+ * Method}s in {@link ApplicationContextFacade} to be run with the the Java
+ * 2 {@link SecurityManager} been enabled.
+ */
+@Parameterized.Parameters(name = "{index}: method={0}")
+public static Collection publicApplicationContextFacadeMethods() {
+return Arrays.stream(ApplicationContextFacade.class.getMethods())
+.filter(method -> !Modifier.isStatic(method.getModifiers()))
+.filter(method -> {
+try {
+Object.class.getMethod(method.getName(),
method.getParameterTypes());
+return false;
+} catch (final NoSuchMethodException e) {
+return true;
+}
+})
+.collect(Collectors.toList());
+}
+
+
+private static Object[] getDefaultParams(final Method method) {
+final int paramsCount = method.getParameterCount();
+final
[tomcat] 02/02: Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit e44fd931075f63acd0405603fa47e9ee335d667a
Author: Kyle Stiemann
AuthorDate: Thu Sep 10 16:47:21 2020 -0400
Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
---
build.xml | 13 ++
.../catalina/core/ApplicationContextFacade.java| 5 +
...estApplicationContextFacadeSecurityManager.java | 148 +
.../util/security/SecurityManagerBaseTest.java | 50 +++
webapps/docs/changelog.xml | 5 +
5 files changed, 221 insertions(+)
diff --git a/build.xml b/build.xml
index 6a1d141..56a9701 100644
--- a/build.xml
+++ b/build.xml
@@ -1969,8 +1969,21 @@
+
+
+
+
+
+
diff --git a/java/org/apache/catalina/core/ApplicationContextFacade.java
b/java/org/apache/catalina/core/ApplicationContextFacade.java
index 8d506b9..67065b0 100644
--- a/java/org/apache/catalina/core/ApplicationContextFacade.java
+++ b/java/org/apache/catalina/core/ApplicationContextFacade.java
@@ -117,6 +117,11 @@ public class ApplicationContextFacade implements
ServletContext {
classCache.put("getAttribute", clazz);
classCache.put("log", clazz);
classCache.put("setSessionTrackingModes", new Class[]{Set.class} );
+classCache.put("addJspFile", new Class[]{String.class, String.class});
+classCache.put("declareRoles", new Class[]{String[].class});
+classCache.put("setSessionTimeout", new Class[]{int.class});
+classCache.put("setRequestCharacterEncoding", new
Class[]{String.class});
+classCache.put("setResponseCharacterEncoding", new
Class[]{String.class});
}
diff --git
a/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
new file mode 100644
index 000..fc5a5d5
--- /dev/null
+++
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.catalina.core;
+
+import java.lang.reflect.Array;
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.lang.reflect.Modifier;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.stream.Collectors;
+
+import org.junit.Assert;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameter;
+
+import org.apache.catalina.security.SecurityUtil;
+import org.apache.tomcat.util.security.SecurityManagerBaseTest;
+import org.easymock.EasyMock;
+import org.easymock.IExpectationSetters;
+import org.easymock.internal.LastControl;
+
+@RunWith(Parameterized.class)
+public final class TestApplicationContextFacadeSecurityManager extends
SecurityManagerBaseTest {
+
+/**
+ * @return {@link Collection} of non-static, non-object, public {@link
+ * Method}s in {@link ApplicationContextFacade} to be run with the the Java
+ * 2 {@link SecurityManager} been enabled.
+ */
+@Parameterized.Parameters(name = "{index}: method={0}")
+public static Collection publicApplicationContextFacadeMethods() {
+return Arrays.stream(ApplicationContextFacade.class.getMethods())
+.filter(method -> !Modifier.isStatic(method.getModifiers()))
+.filter(method -> {
+try {
+Object.class.getMethod(method.getName(),
method.getParameterTypes());
+return false;
+} catch (final NoSuchMethodException e) {
+return true;
+}
+})
+.collect(Collectors.toList());
+}
+
+
+private static Object[] getDefaultParams(final Method method) {
+final int paramsCount = method.getParameterCount();
+final
[tomcat] 04/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 496b3cf5dd99179cd5460c728819bd799f662f99
Author: Mark Thomas
AuthorDate: Thu Sep 24 18:11:37 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Refactoring getStream to handle differences between Stream and
RecycledStream
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 2 +
.../apache/coyote/http2/Http2UpgradeHandler.java | 125 ++---
java/org/apache/coyote/http2/RecycledStream.java | 26 -
java/org/apache/coyote/http2/Stream.java | 20 ++--
4 files changed, 111 insertions(+), 62 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 70cfe35..582ab1e 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -98,4 +98,6 @@ abstract class AbstractNonZeroStream extends AbstractStream {
}
abstract boolean isClosedFinal();
+
+abstract void checkState(FrameType frameType) throws Http2Exception;
}
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index ad5e0e4..c51efd9 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -29,6 +29,7 @@ import java.util.Set;
import java.util.TreeSet;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
@@ -121,7 +122,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private HpackDecoder hpackDecoder;
private HpackEncoder hpackEncoder;
-private final Map streams = new ConcurrentHashMap<>();
+private final ConcurrentMap streams = new
ConcurrentHashMap<>();
protected final AtomicInteger activeRemoteStreamCount = new
AtomicInteger(0);
// Start at -1 so the 'add 2' logic in closeIdleStreams() works
private volatile int maxActiveRemoteStreamId = -1;
@@ -1089,7 +1090,22 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
private Stream getStream(int streamId, boolean unknownIsError) throws
ConnectionException {
Integer key = Integer.valueOf(streamId);
-Stream result = streams.get(key);
+AbstractStream result = streams.get(key);
+if (result instanceof Stream) {
+return (Stream) result;
+}
+if (unknownIsError) {
+// Stream has been closed and removed from the map
+throw new
ConnectionException(sm.getString("upgradeHandler.stream.closed",
key.toString()),
+Http2Error.PROTOCOL_ERROR);
+}
+return null;
+}
+
+
+private AbstractNonZeroStream getStreamMayBeClosed(int streamId, boolean
unknownIsError) throws ConnectionException {
+Integer key = Integer.valueOf(streamId);
+AbstractNonZeroStream result = streams.get(key);
if (result == null && unknownIsError) {
// Stream has been closed and removed from the map
throw new
ConnectionException(sm.getString("upgradeHandler.stream.closed",
key.toString()),
@@ -1133,10 +1149,12 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
return;
}
-for (Stream stream : streams.values()) {
-// The connection is closing. Close the associated streams as no
-// longer required (also notifies any threads waiting for
allocations).
-stream.receiveReset(Http2Error.CANCEL.getCode());
+for (AbstractNonZeroStream stream : streams.values()) {
+if (stream instanceof Stream) {
+// The connection is closing. Close the associated streams as
no
+// longer required (also notifies any threads waiting for
allocations).
+((Stream) stream).receiveReset(Http2Error.CANCEL.getCode());
+}
}
try {
socketWrapper.close();
@@ -1193,10 +1211,10 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
TreeSet candidatesStepTwo = new TreeSet<>();
TreeSet candidatesStepThree = new TreeSet<>();
-for (Entry entry : streams.entrySet()) {
-Stream stream = entry.getValue();
+for (Entry entry : streams.entrySet())
{
+AbstractNonZeroStream stream =
[tomcat] 07/10: Fully replace Stream with RecycledStream
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit d1f204ee12ed3aeb7713b95661a53ddaadd7e9b7
Author: Mark Thomas
AuthorDate: Fri Sep 25 12:26:40 2020 +0100
Fully replace Stream with RecycledStream
Profiler (YourKit) suggests retain memory for Stream ~60k whereas
RecycledStream ins ~300 bytes.
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 37 +++--
.../apache/coyote/http2/Http2UpgradeHandler.java | 64 +-
java/org/apache/coyote/http2/RecycledStream.java | 4 +-
java/org/apache/coyote/http2/Stream.java | 2 +-
4 files changed, 74 insertions(+), 33 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 084de33..bbdfa33 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -33,19 +33,17 @@ abstract class AbstractNonZeroStream extends AbstractStream
{
protected final StreamStateMachine state;
-private volatile int weight;
+private volatile int weight = Constants.DEFAULT_WEIGHT;
AbstractNonZeroStream(String connectionId, Integer identifier) {
super(identifier);
-this.weight = Constants.DEFAULT_WEIGHT;
this.state = new StreamStateMachine(connectionId, getIdAsString());
}
-AbstractNonZeroStream(Integer identifier, int weight, StreamStateMachine
state) {
+AbstractNonZeroStream(Integer identifier, StreamStateMachine state) {
super(identifier);
-this.weight = weight;
this.state = state;
}
@@ -56,6 +54,13 @@ abstract class AbstractNonZeroStream extends AbstractStream {
}
+/*
+ * General method used when reprioritising a stream and care needs to be
+ * taken not to create circular references.
+ *
+ * Changes to the priority tree need to be sychronized at the connection
+ * level. This is the caller's responsibility.
+ */
final void rePrioritise(AbstractStream parent, boolean exclusive, int
weight) {
if (log.isDebugEnabled()) {
log.debug(sm.getString("stream.reprioritisation.debug",
@@ -90,6 +95,9 @@ abstract class AbstractNonZeroStream extends AbstractStream {
/*
* Used when removing closed streams from the tree and we know there is no
* need to check for circular references.
+ *
+ * Changes to the priority tree need to be sychronized at the connection
+ * level. This is the caller's responsibility.
*/
final void rePrioritise(AbstractStream parent, int weight) {
if (log.isDebugEnabled()) {
@@ -103,6 +111,27 @@ abstract class AbstractNonZeroStream extends
AbstractStream {
}
+/*
+ * Used when "recycling" a stream and replacing a Stream instance with a
+ * RecycledStream instance.
+ *
+ * Replace this stream with the provided stream in the parent/child
+ * hierarchy.
+ *
+ * Changes to the priority tree need to be sychronized at the connection
+ * level. This is the caller's responsibility.
+ */
+void replaceStream(AbstractNonZeroStream replacement) {
+replacement.setParentStream(getParentStream());
+detachFromParent();
+for (AbstractNonZeroStream child : getChildStreams()) {
+replacement.addChild(child);
+}
+getChildStreams().clear();
+replacement.weight = weight;
+}
+
+
final boolean isClosedFinal() {
return state.isClosedFinal();
}
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index c51efd9..aebb70b 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -94,6 +94,8 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private static final HeaderSink HEADER_SINK = new HeaderSink();
+private final Object priorityTreeLock = new Object();
+
protected final String connectionId;
protected final Http2Protocol protocol;
@@ -104,8 +106,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private volatile Http2Parser parser;
// Simple state machine (sequence of states)
-private AtomicReference connectionState =
-new AtomicReference<>(ConnectionState.NEW);
+private AtomicReference connectionState = new
AtomicReference<>(ConnectionState.NEW);
private volatile long pausedNanoTime = Long.MAX_VALUE;
/**
@@ -1175,7 +1176,7 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
newStreamsSinceLastPrune = 0;
// RFC 7540, 5.3.4 endpoints should maintain state for at least the
-//
[tomcat] 03/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 3783dc83065414cd85179e9a9014da44780e14b6
Author: Mark Thomas
AuthorDate: Thu Sep 24 17:50:43 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Pull-up isClosedFinal()
---
java/org/apache/coyote/http2/AbstractNonZeroStream.java | 1 +
java/org/apache/coyote/http2/RecycledStream.java| 8
java/org/apache/coyote/http2/Stream.java| 1 +
3 files changed, 10 insertions(+)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index ce08cc5..70cfe35 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -97,4 +97,5 @@ abstract class AbstractNonZeroStream extends AbstractStream {
this.weight = weight;
}
+abstract boolean isClosedFinal();
}
diff --git a/java/org/apache/coyote/http2/RecycledStream.java
b/java/org/apache/coyote/http2/RecycledStream.java
index 1e630df..dbbdc10 100644
--- a/java/org/apache/coyote/http2/RecycledStream.java
+++ b/java/org/apache/coyote/http2/RecycledStream.java
@@ -23,10 +23,12 @@ package org.apache.coyote.http2;
class RecycledStream extends AbstractNonZeroStream {
private final String connectionId;
+private final boolean closedFinal;
RecycledStream(Stream stream) {
super(stream.getIdentifier(), stream.getWeight());
connectionId = stream.getConnectionId();
+closedFinal = stream.isClosedFinal();
}
@@ -34,4 +36,10 @@ class RecycledStream extends AbstractNonZeroStream {
String getConnectionId() {
return connectionId;
}
+
+
+@Override
+boolean isClosedFinal() {
+return closedFinal;
+}
}
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index ef7ed29..489ce53 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -646,6 +646,7 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
}
+@Override
final boolean isClosedFinal() {
return state.isClosedFinal();
}
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] branch 9.0.x updated (8dcf599 -> f258efe)
This is an automated email from the ASF dual-hosted git repository.
markt pushed a change to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git.
from 8dcf599 HTTP2 log statement fixes:
new 0d79ae5 Reduce memory footprint of closed http/2 streams
new e1a53fb Reduce memory footprint of closed http/2 streams
new 3783dc8 Reduce memory footprint of closed http/2 streams
new 496b3cf Reduce memory footprint of closed http/2 streams
new a6e8428 Reduce memory footprint of closed http/2 streams
new 2b2ac4f Reduce memory footprint of closed http/2 streams
new d1f204e Fully replace Stream with RecycledStream
new 83f5e03 Refactor the pruning so more stream info is retained for
longer
new a4054eb Update changelog
new f258efe Optimize the iteration when closing idle streams
The 10 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../apache/coyote/http2/AbstractNonZeroStream.java | 143 +++
java/org/apache/coyote/http2/AbstractStream.java | 9 +-
.../apache/coyote/http2/Http2UpgradeHandler.java | 264 +
...tionSettingsRemote.java => RecycledStream.java} | 23 +-
java/org/apache/coyote/http2/Stream.java | 90 +--
.../apache/coyote/http2/StreamStateMachine.java| 21 +-
webapps/docs/changelog.xml | 5 +
7 files changed, 348 insertions(+), 207 deletions(-)
create mode 100644 java/org/apache/coyote/http2/AbstractNonZeroStream.java
copy java/org/apache/coyote/http2/{ConnectionSettingsRemote.java =>
RecycledStream.java} (61%)
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] branch 9.0.x updated (f258efe -> e44fd93)
This is an automated email from the ASF dual-hosted git repository. markt pushed a change to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git. from f258efe Optimize the iteration when closing idle streams new 5e7c765 Align with 10.0.x new e44fd93 Fix 64735 ServletContext.addJspFile() always fails with SecurityManager The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: build.xml | 17 ++- .../catalina/core/ApplicationContextFacade.java| 5 + ...estApplicationContextFacadeSecurityManager.java | 148 + .../util/security/SecurityManagerBaseTest.java | 50 +++ webapps/docs/changelog.xml | 5 + 5 files changed, 222 insertions(+), 3 deletions(-) create mode 100644 test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java create mode 100644 test/org/apache/tomcat/util/security/SecurityManagerBaseTest.java - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 10/10: Optimize the iteration when closing idle streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit f258efef6c1e9c71ceaf1f50e1dc354f2514ab3a
Author: Martin Tzvetanov Grigorov
AuthorDate: Mon Sep 28 14:53:25 2020 +0300
Optimize the iteration when closing idle streams
---
java/org/apache/coyote/http2/Http2UpgradeHandler.java | 16
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index 9f71af9..a1002c1 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -29,7 +29,7 @@ import java.util.Set;
import java.util.TreeSet;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
-import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentNavigableMap;
import java.util.concurrent.ConcurrentSkipListMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
@@ -124,7 +124,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private HpackDecoder hpackDecoder;
private HpackEncoder hpackEncoder;
-private final ConcurrentMap streams = new
ConcurrentSkipListMap<>();
+private final ConcurrentNavigableMap
streams = new ConcurrentSkipListMap<>();
protected final AtomicInteger activeRemoteStreamCount = new
AtomicInteger(0);
// Start at -1 so the 'add 2' logic in closeIdleStreams() works
private volatile int maxActiveRemoteStreamId = -1;
@@ -1530,12 +1530,12 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
private void closeIdleStreams(int newMaxActiveRemoteStreamId) {
-for (Entry entry : streams.entrySet()) {
-int id = entry.getKey().intValue();
-if (id > maxActiveRemoteStreamId && id <
newMaxActiveRemoteStreamId) {
-if (entry.getValue() instanceof Stream) {
-((Stream) entry.getValue()).closeIfIdle();
-}
+final ConcurrentNavigableMap subMap =
streams.subMap(
+Integer.valueOf(maxActiveRemoteStreamId), false,
+Integer.valueOf(newMaxActiveRemoteStreamId), false);
+for (AbstractNonZeroStream stream : subMap.values()) {
+if (stream instanceof Stream) {
+((Stream)stream).closeIfIdle();
}
}
maxActiveRemoteStreamId = newMaxActiveRemoteStreamId;
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] branch master updated: Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/master by this push:
new e54267a Fix 64735 ServletContext.addJspFile() always fails with
SecurityManager
e54267a is described below
commit e54267a3f71d29f9f850ed9966e7be9612f219d4
Author: Kyle Stiemann
AuthorDate: Thu Sep 10 16:47:21 2020 -0400
Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
---
build.xml | 13 ++
.../catalina/core/ApplicationContextFacade.java| 5 +
...estApplicationContextFacadeSecurityManager.java | 148 +
.../util/security/SecurityManagerBaseTest.java | 50 +++
webapps/docs/changelog.xml | 5 +
5 files changed, 221 insertions(+)
diff --git a/build.xml b/build.xml
index c5c79cb..1ab4af5 100644
--- a/build.xml
+++ b/build.xml
@@ -1978,8 +1978,21 @@
+
+
+
+
+
+
diff --git a/java/org/apache/catalina/core/ApplicationContextFacade.java
b/java/org/apache/catalina/core/ApplicationContextFacade.java
index 5dd3a37..ef004c6 100644
--- a/java/org/apache/catalina/core/ApplicationContextFacade.java
+++ b/java/org/apache/catalina/core/ApplicationContextFacade.java
@@ -117,6 +117,11 @@ public class ApplicationContextFacade implements
ServletContext {
classCache.put("getAttribute", clazz);
classCache.put("log", clazz);
classCache.put("setSessionTrackingModes", new Class[]{Set.class} );
+classCache.put("addJspFile", new Class[]{String.class, String.class});
+classCache.put("declareRoles", new Class[]{String[].class});
+classCache.put("setSessionTimeout", new Class[]{int.class});
+classCache.put("setRequestCharacterEncoding", new
Class[]{String.class});
+classCache.put("setResponseCharacterEncoding", new
Class[]{String.class});
}
diff --git
a/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
new file mode 100644
index 000..fc5a5d5
--- /dev/null
+++
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.catalina.core;
+
+import java.lang.reflect.Array;
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.lang.reflect.Modifier;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.stream.Collectors;
+
+import org.junit.Assert;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameter;
+
+import org.apache.catalina.security.SecurityUtil;
+import org.apache.tomcat.util.security.SecurityManagerBaseTest;
+import org.easymock.EasyMock;
+import org.easymock.IExpectationSetters;
+import org.easymock.internal.LastControl;
+
+@RunWith(Parameterized.class)
+public final class TestApplicationContextFacadeSecurityManager extends
SecurityManagerBaseTest {
+
+/**
+ * @return {@link Collection} of non-static, non-object, public {@link
+ * Method}s in {@link ApplicationContextFacade} to be run with the the Java
+ * 2 {@link SecurityManager} been enabled.
+ */
+@Parameterized.Parameters(name = "{index}: method={0}")
+public static Collection publicApplicationContextFacadeMethods() {
+return Arrays.stream(ApplicationContextFacade.class.getMethods())
+.filter(method -> !Modifier.isStatic(method.getModifiers()))
+.filter(method -> {
+try {
+Object.class.getMethod(method.getName(),
method.getParameterTypes());
+return false;
+} catch (final NoSuchMethodException e) {
+return true;
+}
+})
+
[tomcat] 02/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit e1a53fb2d567b81cfaa4fcfa94ef870f15a6e289
Author: Mark Thomas
AuthorDate: Thu Sep 24 17:32:49 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Pull up re-prioritisation
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 71 ++
java/org/apache/coyote/http2/AbstractStream.java | 6 +-
.../apache/coyote/http2/Http2UpgradeHandler.java | 6 +-
java/org/apache/coyote/http2/RecycledStream.java | 10 +--
java/org/apache/coyote/http2/Stream.java | 55 -
5 files changed, 78 insertions(+), 70 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 63fb5e7..ce08cc5 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -16,6 +16,11 @@
*/
package org.apache.coyote.http2;
+import java.util.Iterator;
+
+import org.apache.juli.logging.Log;
+import org.apache.juli.logging.LogFactory;
+import org.apache.tomcat.util.res.StringManager;
/**
* Base class for all streams other than stream 0, the connection. Primarily
@@ -23,7 +28,73 @@ package org.apache.coyote.http2;
*/
abstract class AbstractNonZeroStream extends AbstractStream {
+private static final Log log =
LogFactory.getLog(AbstractNonZeroStream.class);
+private static final StringManager sm =
StringManager.getManager(AbstractNonZeroStream.class);
+
+private volatile int weight;
+
+
AbstractNonZeroStream(Integer identifier) {
+this(identifier, Constants.DEFAULT_WEIGHT);
+}
+
+
+AbstractNonZeroStream(Integer identifier, int weight) {
super(identifier);
+this.weight = weight;
+}
+
+
+@Override
+final int getWeight() {
+return weight;
}
+
+
+final void rePrioritise(AbstractStream parent, boolean exclusive, int
weight) {
+if (log.isDebugEnabled()) {
+log.debug(sm.getString("stream.reprioritisation.debug",
+getConnectionId(), getIdAsString(),
Boolean.toString(exclusive),
+parent.getIdAsString(), Integer.toString(weight)));
+}
+
+// Check if new parent is a descendant of this stream
+if (isDescendant(parent)) {
+parent.detachFromParent();
+// Cast is always safe since any descendant of this stream must be
+// an instance of Stream
+getParentStream().addChild((Stream) parent);
+}
+
+if (exclusive) {
+// Need to move children of the new parent to be children of this
+// stream. Slightly convoluted to avoid concurrent modification.
+Iterator parentsChildren =
parent.getChildStreams().iterator();
+while (parentsChildren.hasNext()) {
+AbstractNonZeroStream parentsChild = parentsChildren.next();
+parentsChildren.remove();
+this.addChild(parentsChild);
+}
+}
+detachFromParent();
+parent.addChild(this);
+this.weight = weight;
+}
+
+
+/*
+ * Used when removing closed streams from the tree and we know there is no
+ * need to check for circular references.
+ */
+final void rePrioritise(AbstractStream parent, int weight) {
+if (log.isDebugEnabled()) {
+log.debug(sm.getString("stream.reprioritisation.debug",
+getConnectionId(), getIdAsString(), Boolean.FALSE,
+parent.getIdAsString(), Integer.toString(weight)));
+}
+
+parent.addChild(this);
+this.weight = weight;
+}
+
}
diff --git a/java/org/apache/coyote/http2/AbstractStream.java
b/java/org/apache/coyote/http2/AbstractStream.java
index 21963ee..c7374b6 100644
--- a/java/org/apache/coyote/http2/AbstractStream.java
+++ b/java/org/apache/coyote/http2/AbstractStream.java
@@ -37,7 +37,7 @@ abstract class AbstractStream {
private final String idAsString;
private volatile AbstractStream parentStream = null;
-private final Set childStreams = Collections.newSetFromMap(new
ConcurrentHashMap<>());
+private final Set childStreams =
Collections.newSetFromMap(new ConcurrentHashMap<>());
private long windowSize =
ConnectionSettingsBase.DEFAULT_INITIAL_WINDOW_SIZE;
@@ -70,7 +70,7 @@ abstract class AbstractStream {
}
-final void addChild(Stream child) {
+final void addChild(AbstractNonZeroStream child) {
child.setParentStream(this);
childStreams.add(child);
}
@@ -97,7 +97,7 @@ abstract class AbstractStream {
}
-final Set
[tomcat] 01/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 0d79ae569f3fd2646ff76d395f0f56e98b798a98
Author: Mark Thomas
AuthorDate: Wed Sep 23 19:59:13 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Initial changes to object hierarchy in preparation for further
changes.
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 29 ++
java/org/apache/coyote/http2/AbstractStream.java | 3 +-
java/org/apache/coyote/http2/RecycledStream.java | 45 ++
java/org/apache/coyote/http2/Stream.java | 2 +-
4 files changed, 77 insertions(+), 2 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
new file mode 100644
index 000..63fb5e7
--- /dev/null
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.coyote.http2;
+
+
+/**
+ * Base class for all streams other than stream 0, the connection. Primarily
+ * provides functionality shared between full Stream and RecycledStream.
+ */
+abstract class AbstractNonZeroStream extends AbstractStream {
+
+AbstractNonZeroStream(Integer identifier) {
+super(identifier);
+}
+}
diff --git a/java/org/apache/coyote/http2/AbstractStream.java
b/java/org/apache/coyote/http2/AbstractStream.java
index 088d5b0..21963ee 100644
--- a/java/org/apache/coyote/http2/AbstractStream.java
+++ b/java/org/apache/coyote/http2/AbstractStream.java
@@ -25,7 +25,8 @@ import org.apache.juli.logging.LogFactory;
import org.apache.tomcat.util.res.StringManager;
/**
- * Used to managed prioritisation.
+ * Base class for all streams including the connection (referred to as Stream
0)
+ * and is used primarily when managing prioritization.
*/
abstract class AbstractStream {
diff --git a/java/org/apache/coyote/http2/RecycledStream.java
b/java/org/apache/coyote/http2/RecycledStream.java
new file mode 100644
index 000..f500646
--- /dev/null
+++ b/java/org/apache/coyote/http2/RecycledStream.java
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.coyote.http2;
+
+/**
+ * Represents a closed stream in the priority tree. Used in preference to the
+ * full {@link Stream} as has much lower memory usage.
+ */
+class RecycledStream extends AbstractNonZeroStream {
+
+private final String connectionId;
+private int weight;
+
+RecycledStream(Stream stream) {
+super(stream.getIdentifier());
+connectionId = stream.getConnectionId();
+weight = stream.getWeight();
+}
+
+
+@Override
+String getConnectionId() {
+return connectionId;
+}
+
+
+@Override
+int getWeight() {
+return weight;
+}
+}
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index af777ab..56bc129 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -46,7 +46,7 @@ import org.apache.tomcat.util.net.ApplicationBufferHandler;
import org.apache.tomcat.util.net.WriteBuffer;
import
[tomcat] 01/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 0d79ae569f3fd2646ff76d395f0f56e98b798a98
Author: Mark Thomas
AuthorDate: Wed Sep 23 19:59:13 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Initial changes to object hierarchy in preparation for further
changes.
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 29 ++
java/org/apache/coyote/http2/AbstractStream.java | 3 +-
java/org/apache/coyote/http2/RecycledStream.java | 45 ++
java/org/apache/coyote/http2/Stream.java | 2 +-
4 files changed, 77 insertions(+), 2 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
new file mode 100644
index 000..63fb5e7
--- /dev/null
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.coyote.http2;
+
+
+/**
+ * Base class for all streams other than stream 0, the connection. Primarily
+ * provides functionality shared between full Stream and RecycledStream.
+ */
+abstract class AbstractNonZeroStream extends AbstractStream {
+
+AbstractNonZeroStream(Integer identifier) {
+super(identifier);
+}
+}
diff --git a/java/org/apache/coyote/http2/AbstractStream.java
b/java/org/apache/coyote/http2/AbstractStream.java
index 088d5b0..21963ee 100644
--- a/java/org/apache/coyote/http2/AbstractStream.java
+++ b/java/org/apache/coyote/http2/AbstractStream.java
@@ -25,7 +25,8 @@ import org.apache.juli.logging.LogFactory;
import org.apache.tomcat.util.res.StringManager;
/**
- * Used to managed prioritisation.
+ * Base class for all streams including the connection (referred to as Stream
0)
+ * and is used primarily when managing prioritization.
*/
abstract class AbstractStream {
diff --git a/java/org/apache/coyote/http2/RecycledStream.java
b/java/org/apache/coyote/http2/RecycledStream.java
new file mode 100644
index 000..f500646
--- /dev/null
+++ b/java/org/apache/coyote/http2/RecycledStream.java
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.coyote.http2;
+
+/**
+ * Represents a closed stream in the priority tree. Used in preference to the
+ * full {@link Stream} as has much lower memory usage.
+ */
+class RecycledStream extends AbstractNonZeroStream {
+
+private final String connectionId;
+private int weight;
+
+RecycledStream(Stream stream) {
+super(stream.getIdentifier());
+connectionId = stream.getConnectionId();
+weight = stream.getWeight();
+}
+
+
+@Override
+String getConnectionId() {
+return connectionId;
+}
+
+
+@Override
+int getWeight() {
+return weight;
+}
+}
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index af777ab..56bc129 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -46,7 +46,7 @@ import org.apache.tomcat.util.net.ApplicationBufferHandler;
import org.apache.tomcat.util.net.WriteBuffer;
import
[tomcat] 10/10: Optimize the iteration when closing idle streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit f258efef6c1e9c71ceaf1f50e1dc354f2514ab3a
Author: Martin Tzvetanov Grigorov
AuthorDate: Mon Sep 28 14:53:25 2020 +0300
Optimize the iteration when closing idle streams
---
java/org/apache/coyote/http2/Http2UpgradeHandler.java | 16
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index 9f71af9..a1002c1 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -29,7 +29,7 @@ import java.util.Set;
import java.util.TreeSet;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
-import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentNavigableMap;
import java.util.concurrent.ConcurrentSkipListMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
@@ -124,7 +124,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private HpackDecoder hpackDecoder;
private HpackEncoder hpackEncoder;
-private final ConcurrentMap streams = new
ConcurrentSkipListMap<>();
+private final ConcurrentNavigableMap
streams = new ConcurrentSkipListMap<>();
protected final AtomicInteger activeRemoteStreamCount = new
AtomicInteger(0);
// Start at -1 so the 'add 2' logic in closeIdleStreams() works
private volatile int maxActiveRemoteStreamId = -1;
@@ -1530,12 +1530,12 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
private void closeIdleStreams(int newMaxActiveRemoteStreamId) {
-for (Entry entry : streams.entrySet()) {
-int id = entry.getKey().intValue();
-if (id > maxActiveRemoteStreamId && id <
newMaxActiveRemoteStreamId) {
-if (entry.getValue() instanceof Stream) {
-((Stream) entry.getValue()).closeIfIdle();
-}
+final ConcurrentNavigableMap subMap =
streams.subMap(
+Integer.valueOf(maxActiveRemoteStreamId), false,
+Integer.valueOf(newMaxActiveRemoteStreamId), false);
+for (AbstractNonZeroStream stream : subMap.values()) {
+if (stream instanceof Stream) {
+((Stream)stream).closeIfIdle();
}
}
maxActiveRemoteStreamId = newMaxActiveRemoteStreamId;
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 08/10: Refactor the pruning so more stream info is retained for longer
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 83f5e03b52c54a9b213e979c9bb47e0967530cbe
Author: Mark Thomas
AuthorDate: Fri Sep 25 14:14:06 2020 +0100
Refactor the pruning so more stream info is retained for longer
The memory impact of this is mitigated by the previous changes to
replace closed Stream instances with RecycledStream instances.
---
.../apache/coyote/http2/Http2UpgradeHandler.java | 89 --
1 file changed, 49 insertions(+), 40 deletions(-)
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index aebb70b..9f71af9 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -30,6 +30,7 @@ import java.util.TreeSet;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentSkipListMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
@@ -123,7 +124,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private HpackDecoder hpackDecoder;
private HpackEncoder hpackEncoder;
-private final ConcurrentMap streams = new
ConcurrentHashMap<>();
+private final ConcurrentMap streams = new
ConcurrentSkipListMap<>();
protected final AtomicInteger activeRemoteStreamCount = new
AtomicInteger(0);
// Start at -1 so the 'add 2' logic in closeIdleStreams() works
private volatile int maxActiveRemoteStreamId = -1;
@@ -1207,78 +1208,86 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
// 2. Completed streams used for a request with children
// 3. Closed final streams
//
-// Steps 1 and 2 will always be completed.
-// Step 3 will be completed to the minimum extent necessary to bring
the
-// total number of streams under the limit.
+// The pruning halts as soon as enough streams have been pruned.
// Use these sets to track the different classes of streams
-TreeSet candidatesStepOne = new TreeSet<>();
TreeSet candidatesStepTwo = new TreeSet<>();
TreeSet candidatesStepThree = new TreeSet<>();
-for (Entry entry : streams.entrySet())
{
-AbstractNonZeroStream stream = entry.getValue();
+// Step 1
+// Iterator is in key order so we automatically have the oldest streams
+// first
+for (AbstractNonZeroStream stream : streams.values()) {
// Never remove active streams
if (stream instanceof Stream && ((Stream) stream).isActive()) {
continue;
}
-final Integer streamIdentifier = entry.getKey();
if (stream.isClosedFinal()) {
// This stream went from IDLE to CLOSED and is likely to have
// been created by the client as part of the priority tree.
-candidatesStepThree.add(streamIdentifier);
+// Candidate for steo 3.
+candidatesStepThree.add(stream.getIdentifier());
} else if (stream.getChildStreams().size() == 0) {
-// Closed, no children
-candidatesStepOne.add(streamIdentifier);
-} else {
-// Closed, with children
-candidatesStepTwo.add(streamIdentifier);
-}
-}
-
-// Process the step one list
-for (Integer streamIdToRemove : candidatesStepOne) {
-// Remove this childless stream
-AbstractNonZeroStream removedStream =
streams.remove(streamIdToRemove);
-removedStream.detachFromParent();
-toClose--;
-if (log.isDebugEnabled()) {
-log.debug(sm.getString("upgradeHandler.pruned", connectionId,
streamIdToRemove));
-}
-
-// Did this make the parent childless?
-AbstractStream parent = removedStream.getParentStream();
-while (parent instanceof Stream && !((Stream) parent).isActive() &&
-!((Stream) parent).isClosedFinal() &&
parent.getChildStreams().size() == 0) {
-streams.remove(parent.getIdentifier());
-parent.detachFromParent();
-toClose--;
+// Prune it
+AbstractStream parent = stream.getParentStream();
+streams.remove(stream.getIdentifier());
+stream.detachFromParent();
if (log.isDebugEnabled()) {
-log.debug(sm.getString("upgradeHandler.pruned",
connectionId, streamIdToRemove));
+
[tomcat] 06/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 2b2ac4f3ce99027d8589595e921570495f7b08c0
Author: Mark Thomas
AuthorDate: Thu Sep 24 21:47:09 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Pull up state
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 21 -
java/org/apache/coyote/http2/RecycledStream.java| 16 +---
java/org/apache/coyote/http2/Stream.java| 16 +---
3 files changed, 18 insertions(+), 35 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 582ab1e..084de33 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -31,17 +31,22 @@ abstract class AbstractNonZeroStream extends AbstractStream
{
private static final Log log =
LogFactory.getLog(AbstractNonZeroStream.class);
private static final StringManager sm =
StringManager.getManager(AbstractNonZeroStream.class);
+protected final StreamStateMachine state;
+
private volatile int weight;
-AbstractNonZeroStream(Integer identifier) {
-this(identifier, Constants.DEFAULT_WEIGHT);
+AbstractNonZeroStream(String connectionId, Integer identifier) {
+super(identifier);
+this.weight = Constants.DEFAULT_WEIGHT;
+this.state = new StreamStateMachine(connectionId, getIdAsString());
}
-AbstractNonZeroStream(Integer identifier, int weight) {
+AbstractNonZeroStream(Integer identifier, int weight, StreamStateMachine
state) {
super(identifier);
this.weight = weight;
+this.state = state;
}
@@ -97,7 +102,13 @@ abstract class AbstractNonZeroStream extends AbstractStream
{
this.weight = weight;
}
-abstract boolean isClosedFinal();
-abstract void checkState(FrameType frameType) throws Http2Exception;
+final boolean isClosedFinal() {
+return state.isClosedFinal();
+}
+
+
+final void checkState(FrameType frameType) throws Http2Exception {
+state.checkFrameType(frameType);
+}
}
diff --git a/java/org/apache/coyote/http2/RecycledStream.java
b/java/org/apache/coyote/http2/RecycledStream.java
index 1915dff..9d6177c 100644
--- a/java/org/apache/coyote/http2/RecycledStream.java
+++ b/java/org/apache/coyote/http2/RecycledStream.java
@@ -23,12 +23,10 @@ package org.apache.coyote.http2;
class RecycledStream extends AbstractNonZeroStream {
private final String connectionId;
-private final StreamStateMachine state;
RecycledStream(String connectionId, Integer identifier, int weight,
StreamStateMachine state) {
-super(identifier, weight);
+super(identifier, weight, state);
this.connectionId = connectionId;
-this.state = state;
}
@@ -38,18 +36,6 @@ class RecycledStream extends AbstractNonZeroStream {
}
-@Override
-boolean isClosedFinal() {
-return state.isClosedFinal();
-}
-
-
-@Override
-final void checkState(FrameType frameType) throws Http2Exception {
-state.checkFrameType(frameType);
-}
-
-
@SuppressWarnings("sync-override")
@Override
void incrementWindowSize(int increment) throws Http2Exception {
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index f6210d7..140a76d 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -69,7 +69,6 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
private volatile long contentLengthReceived = 0;
private final Http2UpgradeHandler handler;
-private final StreamStateMachine state;
private final WindowAllocationManager allocationManager = new
WindowAllocationManager(this);
// State machine would be too much overhead
@@ -93,11 +92,10 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
Stream(Integer identifier, Http2UpgradeHandler handler, Request
coyoteRequest) {
-super(identifier);
+super(handler.getConnectionId(), identifier);
this.handler = handler;
handler.addChild(this);
setWindowSize(handler.getRemoteSettings().getInitialWindowSize());
-state = new StreamStateMachine(getConnectionId(), getIdAsString());
if (coyoteRequest == null) {
// HTTP/2 new request
this.coyoteRequest = new Request();
@@ -194,12 +192,6 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
@Override
-final void checkState(FrameType frameType) throws
[tomcat] 02/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit e1a53fb2d567b81cfaa4fcfa94ef870f15a6e289
Author: Mark Thomas
AuthorDate: Thu Sep 24 17:32:49 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Pull up re-prioritisation
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 71 ++
java/org/apache/coyote/http2/AbstractStream.java | 6 +-
.../apache/coyote/http2/Http2UpgradeHandler.java | 6 +-
java/org/apache/coyote/http2/RecycledStream.java | 10 +--
java/org/apache/coyote/http2/Stream.java | 55 -
5 files changed, 78 insertions(+), 70 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 63fb5e7..ce08cc5 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -16,6 +16,11 @@
*/
package org.apache.coyote.http2;
+import java.util.Iterator;
+
+import org.apache.juli.logging.Log;
+import org.apache.juli.logging.LogFactory;
+import org.apache.tomcat.util.res.StringManager;
/**
* Base class for all streams other than stream 0, the connection. Primarily
@@ -23,7 +28,73 @@ package org.apache.coyote.http2;
*/
abstract class AbstractNonZeroStream extends AbstractStream {
+private static final Log log =
LogFactory.getLog(AbstractNonZeroStream.class);
+private static final StringManager sm =
StringManager.getManager(AbstractNonZeroStream.class);
+
+private volatile int weight;
+
+
AbstractNonZeroStream(Integer identifier) {
+this(identifier, Constants.DEFAULT_WEIGHT);
+}
+
+
+AbstractNonZeroStream(Integer identifier, int weight) {
super(identifier);
+this.weight = weight;
+}
+
+
+@Override
+final int getWeight() {
+return weight;
}
+
+
+final void rePrioritise(AbstractStream parent, boolean exclusive, int
weight) {
+if (log.isDebugEnabled()) {
+log.debug(sm.getString("stream.reprioritisation.debug",
+getConnectionId(), getIdAsString(),
Boolean.toString(exclusive),
+parent.getIdAsString(), Integer.toString(weight)));
+}
+
+// Check if new parent is a descendant of this stream
+if (isDescendant(parent)) {
+parent.detachFromParent();
+// Cast is always safe since any descendant of this stream must be
+// an instance of Stream
+getParentStream().addChild((Stream) parent);
+}
+
+if (exclusive) {
+// Need to move children of the new parent to be children of this
+// stream. Slightly convoluted to avoid concurrent modification.
+Iterator parentsChildren =
parent.getChildStreams().iterator();
+while (parentsChildren.hasNext()) {
+AbstractNonZeroStream parentsChild = parentsChildren.next();
+parentsChildren.remove();
+this.addChild(parentsChild);
+}
+}
+detachFromParent();
+parent.addChild(this);
+this.weight = weight;
+}
+
+
+/*
+ * Used when removing closed streams from the tree and we know there is no
+ * need to check for circular references.
+ */
+final void rePrioritise(AbstractStream parent, int weight) {
+if (log.isDebugEnabled()) {
+log.debug(sm.getString("stream.reprioritisation.debug",
+getConnectionId(), getIdAsString(), Boolean.FALSE,
+parent.getIdAsString(), Integer.toString(weight)));
+}
+
+parent.addChild(this);
+this.weight = weight;
+}
+
}
diff --git a/java/org/apache/coyote/http2/AbstractStream.java
b/java/org/apache/coyote/http2/AbstractStream.java
index 21963ee..c7374b6 100644
--- a/java/org/apache/coyote/http2/AbstractStream.java
+++ b/java/org/apache/coyote/http2/AbstractStream.java
@@ -37,7 +37,7 @@ abstract class AbstractStream {
private final String idAsString;
private volatile AbstractStream parentStream = null;
-private final Set childStreams = Collections.newSetFromMap(new
ConcurrentHashMap<>());
+private final Set childStreams =
Collections.newSetFromMap(new ConcurrentHashMap<>());
private long windowSize =
ConnectionSettingsBase.DEFAULT_INITIAL_WINDOW_SIZE;
@@ -70,7 +70,7 @@ abstract class AbstractStream {
}
-final void addChild(Stream child) {
+final void addChild(AbstractNonZeroStream child) {
child.setParentStream(this);
childStreams.add(child);
}
@@ -97,7 +97,7 @@ abstract class AbstractStream {
}
-final Set
[tomcat] 09/10: Update changelog
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit a4054eb3f27ac0d242d5f2a6c4599e3f6cb9a519 Author: Mark Thomas AuthorDate: Fri Sep 25 14:33:45 2020 +0100 Update changelog --- webapps/docs/changelog.xml | 5 + 1 file changed, 5 insertions(+) diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml index 0be3539..a82ec7b 100644 --- a/webapps/docs/changelog.xml +++ b/webapps/docs/changelog.xml @@ -127,6 +127,11 @@ (markt) +Refactor the handling of closed HTTP/2 streams to reduce the heap usage +associated with used streams and to retain information for more streams +in the priority tree. (markt) + + Don't send the Keep-Alive response header if the connection has been explicitly closed. (markt) - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 10/10: Optimize the iteration when closing idle streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit f258efef6c1e9c71ceaf1f50e1dc354f2514ab3a
Author: Martin Tzvetanov Grigorov
AuthorDate: Mon Sep 28 14:53:25 2020 +0300
Optimize the iteration when closing idle streams
---
java/org/apache/coyote/http2/Http2UpgradeHandler.java | 16
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index 9f71af9..a1002c1 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -29,7 +29,7 @@ import java.util.Set;
import java.util.TreeSet;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
-import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentNavigableMap;
import java.util.concurrent.ConcurrentSkipListMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
@@ -124,7 +124,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private HpackDecoder hpackDecoder;
private HpackEncoder hpackEncoder;
-private final ConcurrentMap streams = new
ConcurrentSkipListMap<>();
+private final ConcurrentNavigableMap
streams = new ConcurrentSkipListMap<>();
protected final AtomicInteger activeRemoteStreamCount = new
AtomicInteger(0);
// Start at -1 so the 'add 2' logic in closeIdleStreams() works
private volatile int maxActiveRemoteStreamId = -1;
@@ -1530,12 +1530,12 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
private void closeIdleStreams(int newMaxActiveRemoteStreamId) {
-for (Entry entry : streams.entrySet()) {
-int id = entry.getKey().intValue();
-if (id > maxActiveRemoteStreamId && id <
newMaxActiveRemoteStreamId) {
-if (entry.getValue() instanceof Stream) {
-((Stream) entry.getValue()).closeIfIdle();
-}
+final ConcurrentNavigableMap subMap =
streams.subMap(
+Integer.valueOf(maxActiveRemoteStreamId), false,
+Integer.valueOf(newMaxActiveRemoteStreamId), false);
+for (AbstractNonZeroStream stream : subMap.values()) {
+if (stream instanceof Stream) {
+((Stream)stream).closeIfIdle();
}
}
maxActiveRemoteStreamId = newMaxActiveRemoteStreamId;
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 08/10: Refactor the pruning so more stream info is retained for longer
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 83f5e03b52c54a9b213e979c9bb47e0967530cbe
Author: Mark Thomas
AuthorDate: Fri Sep 25 14:14:06 2020 +0100
Refactor the pruning so more stream info is retained for longer
The memory impact of this is mitigated by the previous changes to
replace closed Stream instances with RecycledStream instances.
---
.../apache/coyote/http2/Http2UpgradeHandler.java | 89 --
1 file changed, 49 insertions(+), 40 deletions(-)
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index aebb70b..9f71af9 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -30,6 +30,7 @@ import java.util.TreeSet;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentSkipListMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
@@ -123,7 +124,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private HpackDecoder hpackDecoder;
private HpackEncoder hpackEncoder;
-private final ConcurrentMap streams = new
ConcurrentHashMap<>();
+private final ConcurrentMap streams = new
ConcurrentSkipListMap<>();
protected final AtomicInteger activeRemoteStreamCount = new
AtomicInteger(0);
// Start at -1 so the 'add 2' logic in closeIdleStreams() works
private volatile int maxActiveRemoteStreamId = -1;
@@ -1207,78 +1208,86 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
// 2. Completed streams used for a request with children
// 3. Closed final streams
//
-// Steps 1 and 2 will always be completed.
-// Step 3 will be completed to the minimum extent necessary to bring
the
-// total number of streams under the limit.
+// The pruning halts as soon as enough streams have been pruned.
// Use these sets to track the different classes of streams
-TreeSet candidatesStepOne = new TreeSet<>();
TreeSet candidatesStepTwo = new TreeSet<>();
TreeSet candidatesStepThree = new TreeSet<>();
-for (Entry entry : streams.entrySet())
{
-AbstractNonZeroStream stream = entry.getValue();
+// Step 1
+// Iterator is in key order so we automatically have the oldest streams
+// first
+for (AbstractNonZeroStream stream : streams.values()) {
// Never remove active streams
if (stream instanceof Stream && ((Stream) stream).isActive()) {
continue;
}
-final Integer streamIdentifier = entry.getKey();
if (stream.isClosedFinal()) {
// This stream went from IDLE to CLOSED and is likely to have
// been created by the client as part of the priority tree.
-candidatesStepThree.add(streamIdentifier);
+// Candidate for steo 3.
+candidatesStepThree.add(stream.getIdentifier());
} else if (stream.getChildStreams().size() == 0) {
-// Closed, no children
-candidatesStepOne.add(streamIdentifier);
-} else {
-// Closed, with children
-candidatesStepTwo.add(streamIdentifier);
-}
-}
-
-// Process the step one list
-for (Integer streamIdToRemove : candidatesStepOne) {
-// Remove this childless stream
-AbstractNonZeroStream removedStream =
streams.remove(streamIdToRemove);
-removedStream.detachFromParent();
-toClose--;
-if (log.isDebugEnabled()) {
-log.debug(sm.getString("upgradeHandler.pruned", connectionId,
streamIdToRemove));
-}
-
-// Did this make the parent childless?
-AbstractStream parent = removedStream.getParentStream();
-while (parent instanceof Stream && !((Stream) parent).isActive() &&
-!((Stream) parent).isClosedFinal() &&
parent.getChildStreams().size() == 0) {
-streams.remove(parent.getIdentifier());
-parent.detachFromParent();
-toClose--;
+// Prune it
+AbstractStream parent = stream.getParentStream();
+streams.remove(stream.getIdentifier());
+stream.detachFromParent();
if (log.isDebugEnabled()) {
-log.debug(sm.getString("upgradeHandler.pruned",
connectionId, streamIdToRemove));
+
[tomcat] branch 9.0.x updated (8dcf599 -> f258efe)
This is an automated email from the ASF dual-hosted git repository.
markt pushed a change to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git.
from 8dcf599 HTTP2 log statement fixes:
new 0d79ae5 Reduce memory footprint of closed http/2 streams
new e1a53fb Reduce memory footprint of closed http/2 streams
new 3783dc8 Reduce memory footprint of closed http/2 streams
new 496b3cf Reduce memory footprint of closed http/2 streams
new a6e8428 Reduce memory footprint of closed http/2 streams
new 2b2ac4f Reduce memory footprint of closed http/2 streams
new d1f204e Fully replace Stream with RecycledStream
new 83f5e03 Refactor the pruning so more stream info is retained for
longer
new a4054eb Update changelog
new f258efe Optimize the iteration when closing idle streams
The 10 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../apache/coyote/http2/AbstractNonZeroStream.java | 143 +++
java/org/apache/coyote/http2/AbstractStream.java | 9 +-
.../apache/coyote/http2/Http2UpgradeHandler.java | 264 +
...tionSettingsRemote.java => RecycledStream.java} | 23 +-
java/org/apache/coyote/http2/Stream.java | 90 +--
.../apache/coyote/http2/StreamStateMachine.java| 21 +-
webapps/docs/changelog.xml | 5 +
7 files changed, 348 insertions(+), 207 deletions(-)
create mode 100644 java/org/apache/coyote/http2/AbstractNonZeroStream.java
copy java/org/apache/coyote/http2/{ConnectionSettingsRemote.java =>
RecycledStream.java} (61%)
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 01/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 0d79ae569f3fd2646ff76d395f0f56e98b798a98
Author: Mark Thomas
AuthorDate: Wed Sep 23 19:59:13 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Initial changes to object hierarchy in preparation for further
changes.
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 29 ++
java/org/apache/coyote/http2/AbstractStream.java | 3 +-
java/org/apache/coyote/http2/RecycledStream.java | 45 ++
java/org/apache/coyote/http2/Stream.java | 2 +-
4 files changed, 77 insertions(+), 2 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
new file mode 100644
index 000..63fb5e7
--- /dev/null
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.coyote.http2;
+
+
+/**
+ * Base class for all streams other than stream 0, the connection. Primarily
+ * provides functionality shared between full Stream and RecycledStream.
+ */
+abstract class AbstractNonZeroStream extends AbstractStream {
+
+AbstractNonZeroStream(Integer identifier) {
+super(identifier);
+}
+}
diff --git a/java/org/apache/coyote/http2/AbstractStream.java
b/java/org/apache/coyote/http2/AbstractStream.java
index 088d5b0..21963ee 100644
--- a/java/org/apache/coyote/http2/AbstractStream.java
+++ b/java/org/apache/coyote/http2/AbstractStream.java
@@ -25,7 +25,8 @@ import org.apache.juli.logging.LogFactory;
import org.apache.tomcat.util.res.StringManager;
/**
- * Used to managed prioritisation.
+ * Base class for all streams including the connection (referred to as Stream
0)
+ * and is used primarily when managing prioritization.
*/
abstract class AbstractStream {
diff --git a/java/org/apache/coyote/http2/RecycledStream.java
b/java/org/apache/coyote/http2/RecycledStream.java
new file mode 100644
index 000..f500646
--- /dev/null
+++ b/java/org/apache/coyote/http2/RecycledStream.java
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.coyote.http2;
+
+/**
+ * Represents a closed stream in the priority tree. Used in preference to the
+ * full {@link Stream} as has much lower memory usage.
+ */
+class RecycledStream extends AbstractNonZeroStream {
+
+private final String connectionId;
+private int weight;
+
+RecycledStream(Stream stream) {
+super(stream.getIdentifier());
+connectionId = stream.getConnectionId();
+weight = stream.getWeight();
+}
+
+
+@Override
+String getConnectionId() {
+return connectionId;
+}
+
+
+@Override
+int getWeight() {
+return weight;
+}
+}
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index af777ab..56bc129 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -46,7 +46,7 @@ import org.apache.tomcat.util.net.ApplicationBufferHandler;
import org.apache.tomcat.util.net.WriteBuffer;
import
[tomcat] 03/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 3783dc83065414cd85179e9a9014da44780e14b6
Author: Mark Thomas
AuthorDate: Thu Sep 24 17:50:43 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Pull-up isClosedFinal()
---
java/org/apache/coyote/http2/AbstractNonZeroStream.java | 1 +
java/org/apache/coyote/http2/RecycledStream.java| 8
java/org/apache/coyote/http2/Stream.java| 1 +
3 files changed, 10 insertions(+)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index ce08cc5..70cfe35 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -97,4 +97,5 @@ abstract class AbstractNonZeroStream extends AbstractStream {
this.weight = weight;
}
+abstract boolean isClosedFinal();
}
diff --git a/java/org/apache/coyote/http2/RecycledStream.java
b/java/org/apache/coyote/http2/RecycledStream.java
index 1e630df..dbbdc10 100644
--- a/java/org/apache/coyote/http2/RecycledStream.java
+++ b/java/org/apache/coyote/http2/RecycledStream.java
@@ -23,10 +23,12 @@ package org.apache.coyote.http2;
class RecycledStream extends AbstractNonZeroStream {
private final String connectionId;
+private final boolean closedFinal;
RecycledStream(Stream stream) {
super(stream.getIdentifier(), stream.getWeight());
connectionId = stream.getConnectionId();
+closedFinal = stream.isClosedFinal();
}
@@ -34,4 +36,10 @@ class RecycledStream extends AbstractNonZeroStream {
String getConnectionId() {
return connectionId;
}
+
+
+@Override
+boolean isClosedFinal() {
+return closedFinal;
+}
}
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index ef7ed29..489ce53 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -646,6 +646,7 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
}
+@Override
final boolean isClosedFinal() {
return state.isClosedFinal();
}
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 04/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 496b3cf5dd99179cd5460c728819bd799f662f99
Author: Mark Thomas
AuthorDate: Thu Sep 24 18:11:37 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Refactoring getStream to handle differences between Stream and
RecycledStream
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 2 +
.../apache/coyote/http2/Http2UpgradeHandler.java | 125 ++---
java/org/apache/coyote/http2/RecycledStream.java | 26 -
java/org/apache/coyote/http2/Stream.java | 20 ++--
4 files changed, 111 insertions(+), 62 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 70cfe35..582ab1e 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -98,4 +98,6 @@ abstract class AbstractNonZeroStream extends AbstractStream {
}
abstract boolean isClosedFinal();
+
+abstract void checkState(FrameType frameType) throws Http2Exception;
}
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index ad5e0e4..c51efd9 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -29,6 +29,7 @@ import java.util.Set;
import java.util.TreeSet;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
@@ -121,7 +122,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private HpackDecoder hpackDecoder;
private HpackEncoder hpackEncoder;
-private final Map streams = new ConcurrentHashMap<>();
+private final ConcurrentMap streams = new
ConcurrentHashMap<>();
protected final AtomicInteger activeRemoteStreamCount = new
AtomicInteger(0);
// Start at -1 so the 'add 2' logic in closeIdleStreams() works
private volatile int maxActiveRemoteStreamId = -1;
@@ -1089,7 +1090,22 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
private Stream getStream(int streamId, boolean unknownIsError) throws
ConnectionException {
Integer key = Integer.valueOf(streamId);
-Stream result = streams.get(key);
+AbstractStream result = streams.get(key);
+if (result instanceof Stream) {
+return (Stream) result;
+}
+if (unknownIsError) {
+// Stream has been closed and removed from the map
+throw new
ConnectionException(sm.getString("upgradeHandler.stream.closed",
key.toString()),
+Http2Error.PROTOCOL_ERROR);
+}
+return null;
+}
+
+
+private AbstractNonZeroStream getStreamMayBeClosed(int streamId, boolean
unknownIsError) throws ConnectionException {
+Integer key = Integer.valueOf(streamId);
+AbstractNonZeroStream result = streams.get(key);
if (result == null && unknownIsError) {
// Stream has been closed and removed from the map
throw new
ConnectionException(sm.getString("upgradeHandler.stream.closed",
key.toString()),
@@ -1133,10 +1149,12 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
return;
}
-for (Stream stream : streams.values()) {
-// The connection is closing. Close the associated streams as no
-// longer required (also notifies any threads waiting for
allocations).
-stream.receiveReset(Http2Error.CANCEL.getCode());
+for (AbstractNonZeroStream stream : streams.values()) {
+if (stream instanceof Stream) {
+// The connection is closing. Close the associated streams as
no
+// longer required (also notifies any threads waiting for
allocations).
+((Stream) stream).receiveReset(Http2Error.CANCEL.getCode());
+}
}
try {
socketWrapper.close();
@@ -1193,10 +1211,10 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
TreeSet candidatesStepTwo = new TreeSet<>();
TreeSet candidatesStepThree = new TreeSet<>();
-for (Entry entry : streams.entrySet()) {
-Stream stream = entry.getValue();
+for (Entry entry : streams.entrySet())
{
+AbstractNonZeroStream stream =
[tomcat] 06/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 2b2ac4f3ce99027d8589595e921570495f7b08c0
Author: Mark Thomas
AuthorDate: Thu Sep 24 21:47:09 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Pull up state
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 21 -
java/org/apache/coyote/http2/RecycledStream.java| 16 +---
java/org/apache/coyote/http2/Stream.java| 16 +---
3 files changed, 18 insertions(+), 35 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 582ab1e..084de33 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -31,17 +31,22 @@ abstract class AbstractNonZeroStream extends AbstractStream
{
private static final Log log =
LogFactory.getLog(AbstractNonZeroStream.class);
private static final StringManager sm =
StringManager.getManager(AbstractNonZeroStream.class);
+protected final StreamStateMachine state;
+
private volatile int weight;
-AbstractNonZeroStream(Integer identifier) {
-this(identifier, Constants.DEFAULT_WEIGHT);
+AbstractNonZeroStream(String connectionId, Integer identifier) {
+super(identifier);
+this.weight = Constants.DEFAULT_WEIGHT;
+this.state = new StreamStateMachine(connectionId, getIdAsString());
}
-AbstractNonZeroStream(Integer identifier, int weight) {
+AbstractNonZeroStream(Integer identifier, int weight, StreamStateMachine
state) {
super(identifier);
this.weight = weight;
+this.state = state;
}
@@ -97,7 +102,13 @@ abstract class AbstractNonZeroStream extends AbstractStream
{
this.weight = weight;
}
-abstract boolean isClosedFinal();
-abstract void checkState(FrameType frameType) throws Http2Exception;
+final boolean isClosedFinal() {
+return state.isClosedFinal();
+}
+
+
+final void checkState(FrameType frameType) throws Http2Exception {
+state.checkFrameType(frameType);
+}
}
diff --git a/java/org/apache/coyote/http2/RecycledStream.java
b/java/org/apache/coyote/http2/RecycledStream.java
index 1915dff..9d6177c 100644
--- a/java/org/apache/coyote/http2/RecycledStream.java
+++ b/java/org/apache/coyote/http2/RecycledStream.java
@@ -23,12 +23,10 @@ package org.apache.coyote.http2;
class RecycledStream extends AbstractNonZeroStream {
private final String connectionId;
-private final StreamStateMachine state;
RecycledStream(String connectionId, Integer identifier, int weight,
StreamStateMachine state) {
-super(identifier, weight);
+super(identifier, weight, state);
this.connectionId = connectionId;
-this.state = state;
}
@@ -38,18 +36,6 @@ class RecycledStream extends AbstractNonZeroStream {
}
-@Override
-boolean isClosedFinal() {
-return state.isClosedFinal();
-}
-
-
-@Override
-final void checkState(FrameType frameType) throws Http2Exception {
-state.checkFrameType(frameType);
-}
-
-
@SuppressWarnings("sync-override")
@Override
void incrementWindowSize(int increment) throws Http2Exception {
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index f6210d7..140a76d 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -69,7 +69,6 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
private volatile long contentLengthReceived = 0;
private final Http2UpgradeHandler handler;
-private final StreamStateMachine state;
private final WindowAllocationManager allocationManager = new
WindowAllocationManager(this);
// State machine would be too much overhead
@@ -93,11 +92,10 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
Stream(Integer identifier, Http2UpgradeHandler handler, Request
coyoteRequest) {
-super(identifier);
+super(handler.getConnectionId(), identifier);
this.handler = handler;
handler.addChild(this);
setWindowSize(handler.getRemoteSettings().getInitialWindowSize());
-state = new StreamStateMachine(getConnectionId(), getIdAsString());
if (coyoteRequest == null) {
// HTTP/2 new request
this.coyoteRequest = new Request();
@@ -194,12 +192,6 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
@Override
-final void checkState(FrameType frameType) throws
[tomcat] 05/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit a6e84280ac145ab14f86a2cbe37238f9acda31d6
Author: Mark Thomas
AuthorDate: Thu Sep 24 21:36:19 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Refactor StreamStateMachine to remove reference to Stream
---
java/org/apache/coyote/http2/Stream.java| 2 +-
.../org/apache/coyote/http2/StreamStateMachine.java | 21 -
2 files changed, 13 insertions(+), 10 deletions(-)
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index 8aa4a7b..f6210d7 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -97,7 +97,7 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
this.handler = handler;
handler.addChild(this);
setWindowSize(handler.getRemoteSettings().getInitialWindowSize());
-state = new StreamStateMachine(this);
+state = new StreamStateMachine(getConnectionId(), getIdAsString());
if (coyoteRequest == null) {
// HTTP/2 new request
this.coyoteRequest = new Request();
diff --git a/java/org/apache/coyote/http2/StreamStateMachine.java
b/java/org/apache/coyote/http2/StreamStateMachine.java
index 851e1a8..72028fd 100644
--- a/java/org/apache/coyote/http2/StreamStateMachine.java
+++ b/java/org/apache/coyote/http2/StreamStateMachine.java
@@ -39,12 +39,15 @@ class StreamStateMachine {
private static final Log log = LogFactory.getLog(StreamStateMachine.class);
private static final StringManager sm =
StringManager.getManager(StreamStateMachine.class);
-private final Stream stream;
+private final String connectionId;
+private final String streamId;
+
private State state;
-StreamStateMachine(Stream stream) {
-this.stream = stream;
+StreamStateMachine(String connectionId, String streamId) {
+this.connectionId = connectionId;
+this.streamId = streamId;
stateChange(null, State.IDLE);
}
@@ -92,7 +95,7 @@ class StreamStateMachine {
public synchronized void sendReset() {
if (state == State.IDLE) {
throw new
IllegalStateException(sm.getString("streamStateMachine.debug.change",
-stream.getConnectionId(), stream.getIdAsString(), state));
+connectionId, streamId, state));
}
if (state.canReset()) {
stateChange(state, State.CLOSED_RST_TX);
@@ -109,8 +112,8 @@ class StreamStateMachine {
if (state == oldState) {
state = newState;
if (log.isDebugEnabled()) {
-log.debug(sm.getString("streamStateMachine.debug.change",
stream.getConnectionId(),
-stream.getIdAsString(), oldState, newState));
+log.debug(sm.getString("streamStateMachine.debug.change",
connectionId,
+streamId, oldState, newState));
}
}
}
@@ -122,12 +125,12 @@ class StreamStateMachine {
if (!isFrameTypePermitted(frameType)) {
if (state.connectionErrorForInvalidFrame) {
throw new
ConnectionException(sm.getString("streamStateMachine.invalidFrame",
-stream.getConnectionId(), stream.getIdAsString(),
state, frameType),
+connectionId, streamId, state, frameType),
state.errorCodeForInvalidFrame);
} else {
throw new
StreamException(sm.getString("streamStateMachine.invalidFrame",
-stream.getConnectionId(), stream.getIdAsString(),
state, frameType),
-state.errorCodeForInvalidFrame, stream.getIdAsInt());
+connectionId, streamId, state, frameType),
+state.errorCodeForInvalidFrame,
Integer.parseInt(streamId));
}
}
}
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] branch master updated: Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/master by this push:
new e54267a Fix 64735 ServletContext.addJspFile() always fails with
SecurityManager
e54267a is described below
commit e54267a3f71d29f9f850ed9966e7be9612f219d4
Author: Kyle Stiemann
AuthorDate: Thu Sep 10 16:47:21 2020 -0400
Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
---
build.xml | 13 ++
.../catalina/core/ApplicationContextFacade.java| 5 +
...estApplicationContextFacadeSecurityManager.java | 148 +
.../util/security/SecurityManagerBaseTest.java | 50 +++
webapps/docs/changelog.xml | 5 +
5 files changed, 221 insertions(+)
diff --git a/build.xml b/build.xml
index c5c79cb..1ab4af5 100644
--- a/build.xml
+++ b/build.xml
@@ -1978,8 +1978,21 @@
+
+
+
+
+
+
diff --git a/java/org/apache/catalina/core/ApplicationContextFacade.java
b/java/org/apache/catalina/core/ApplicationContextFacade.java
index 5dd3a37..ef004c6 100644
--- a/java/org/apache/catalina/core/ApplicationContextFacade.java
+++ b/java/org/apache/catalina/core/ApplicationContextFacade.java
@@ -117,6 +117,11 @@ public class ApplicationContextFacade implements
ServletContext {
classCache.put("getAttribute", clazz);
classCache.put("log", clazz);
classCache.put("setSessionTrackingModes", new Class[]{Set.class} );
+classCache.put("addJspFile", new Class[]{String.class, String.class});
+classCache.put("declareRoles", new Class[]{String[].class});
+classCache.put("setSessionTimeout", new Class[]{int.class});
+classCache.put("setRequestCharacterEncoding", new
Class[]{String.class});
+classCache.put("setResponseCharacterEncoding", new
Class[]{String.class});
}
diff --git
a/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
new file mode 100644
index 000..fc5a5d5
--- /dev/null
+++
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.catalina.core;
+
+import java.lang.reflect.Array;
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.lang.reflect.Modifier;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.stream.Collectors;
+
+import org.junit.Assert;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameter;
+
+import org.apache.catalina.security.SecurityUtil;
+import org.apache.tomcat.util.security.SecurityManagerBaseTest;
+import org.easymock.EasyMock;
+import org.easymock.IExpectationSetters;
+import org.easymock.internal.LastControl;
+
+@RunWith(Parameterized.class)
+public final class TestApplicationContextFacadeSecurityManager extends
SecurityManagerBaseTest {
+
+/**
+ * @return {@link Collection} of non-static, non-object, public {@link
+ * Method}s in {@link ApplicationContextFacade} to be run with the the Java
+ * 2 {@link SecurityManager} been enabled.
+ */
+@Parameterized.Parameters(name = "{index}: method={0}")
+public static Collection publicApplicationContextFacadeMethods() {
+return Arrays.stream(ApplicationContextFacade.class.getMethods())
+.filter(method -> !Modifier.isStatic(method.getModifiers()))
+.filter(method -> {
+try {
+Object.class.getMethod(method.getName(),
method.getParameterTypes());
+return false;
+} catch (final NoSuchMethodException e) {
+return true;
+}
+})
+
[tomcat] 02/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit e1a53fb2d567b81cfaa4fcfa94ef870f15a6e289
Author: Mark Thomas
AuthorDate: Thu Sep 24 17:32:49 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Pull up re-prioritisation
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 71 ++
java/org/apache/coyote/http2/AbstractStream.java | 6 +-
.../apache/coyote/http2/Http2UpgradeHandler.java | 6 +-
java/org/apache/coyote/http2/RecycledStream.java | 10 +--
java/org/apache/coyote/http2/Stream.java | 55 -
5 files changed, 78 insertions(+), 70 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 63fb5e7..ce08cc5 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -16,6 +16,11 @@
*/
package org.apache.coyote.http2;
+import java.util.Iterator;
+
+import org.apache.juli.logging.Log;
+import org.apache.juli.logging.LogFactory;
+import org.apache.tomcat.util.res.StringManager;
/**
* Base class for all streams other than stream 0, the connection. Primarily
@@ -23,7 +28,73 @@ package org.apache.coyote.http2;
*/
abstract class AbstractNonZeroStream extends AbstractStream {
+private static final Log log =
LogFactory.getLog(AbstractNonZeroStream.class);
+private static final StringManager sm =
StringManager.getManager(AbstractNonZeroStream.class);
+
+private volatile int weight;
+
+
AbstractNonZeroStream(Integer identifier) {
+this(identifier, Constants.DEFAULT_WEIGHT);
+}
+
+
+AbstractNonZeroStream(Integer identifier, int weight) {
super(identifier);
+this.weight = weight;
+}
+
+
+@Override
+final int getWeight() {
+return weight;
}
+
+
+final void rePrioritise(AbstractStream parent, boolean exclusive, int
weight) {
+if (log.isDebugEnabled()) {
+log.debug(sm.getString("stream.reprioritisation.debug",
+getConnectionId(), getIdAsString(),
Boolean.toString(exclusive),
+parent.getIdAsString(), Integer.toString(weight)));
+}
+
+// Check if new parent is a descendant of this stream
+if (isDescendant(parent)) {
+parent.detachFromParent();
+// Cast is always safe since any descendant of this stream must be
+// an instance of Stream
+getParentStream().addChild((Stream) parent);
+}
+
+if (exclusive) {
+// Need to move children of the new parent to be children of this
+// stream. Slightly convoluted to avoid concurrent modification.
+Iterator parentsChildren =
parent.getChildStreams().iterator();
+while (parentsChildren.hasNext()) {
+AbstractNonZeroStream parentsChild = parentsChildren.next();
+parentsChildren.remove();
+this.addChild(parentsChild);
+}
+}
+detachFromParent();
+parent.addChild(this);
+this.weight = weight;
+}
+
+
+/*
+ * Used when removing closed streams from the tree and we know there is no
+ * need to check for circular references.
+ */
+final void rePrioritise(AbstractStream parent, int weight) {
+if (log.isDebugEnabled()) {
+log.debug(sm.getString("stream.reprioritisation.debug",
+getConnectionId(), getIdAsString(), Boolean.FALSE,
+parent.getIdAsString(), Integer.toString(weight)));
+}
+
+parent.addChild(this);
+this.weight = weight;
+}
+
}
diff --git a/java/org/apache/coyote/http2/AbstractStream.java
b/java/org/apache/coyote/http2/AbstractStream.java
index 21963ee..c7374b6 100644
--- a/java/org/apache/coyote/http2/AbstractStream.java
+++ b/java/org/apache/coyote/http2/AbstractStream.java
@@ -37,7 +37,7 @@ abstract class AbstractStream {
private final String idAsString;
private volatile AbstractStream parentStream = null;
-private final Set childStreams = Collections.newSetFromMap(new
ConcurrentHashMap<>());
+private final Set childStreams =
Collections.newSetFromMap(new ConcurrentHashMap<>());
private long windowSize =
ConnectionSettingsBase.DEFAULT_INITIAL_WINDOW_SIZE;
@@ -70,7 +70,7 @@ abstract class AbstractStream {
}
-final void addChild(Stream child) {
+final void addChild(AbstractNonZeroStream child) {
child.setParentStream(this);
childStreams.add(child);
}
@@ -97,7 +97,7 @@ abstract class AbstractStream {
}
-final Set
[tomcat] 07/10: Fully replace Stream with RecycledStream
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit d1f204ee12ed3aeb7713b95661a53ddaadd7e9b7
Author: Mark Thomas
AuthorDate: Fri Sep 25 12:26:40 2020 +0100
Fully replace Stream with RecycledStream
Profiler (YourKit) suggests retain memory for Stream ~60k whereas
RecycledStream ins ~300 bytes.
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 37 +++--
.../apache/coyote/http2/Http2UpgradeHandler.java | 64 +-
java/org/apache/coyote/http2/RecycledStream.java | 4 +-
java/org/apache/coyote/http2/Stream.java | 2 +-
4 files changed, 74 insertions(+), 33 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 084de33..bbdfa33 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -33,19 +33,17 @@ abstract class AbstractNonZeroStream extends AbstractStream
{
protected final StreamStateMachine state;
-private volatile int weight;
+private volatile int weight = Constants.DEFAULT_WEIGHT;
AbstractNonZeroStream(String connectionId, Integer identifier) {
super(identifier);
-this.weight = Constants.DEFAULT_WEIGHT;
this.state = new StreamStateMachine(connectionId, getIdAsString());
}
-AbstractNonZeroStream(Integer identifier, int weight, StreamStateMachine
state) {
+AbstractNonZeroStream(Integer identifier, StreamStateMachine state) {
super(identifier);
-this.weight = weight;
this.state = state;
}
@@ -56,6 +54,13 @@ abstract class AbstractNonZeroStream extends AbstractStream {
}
+/*
+ * General method used when reprioritising a stream and care needs to be
+ * taken not to create circular references.
+ *
+ * Changes to the priority tree need to be sychronized at the connection
+ * level. This is the caller's responsibility.
+ */
final void rePrioritise(AbstractStream parent, boolean exclusive, int
weight) {
if (log.isDebugEnabled()) {
log.debug(sm.getString("stream.reprioritisation.debug",
@@ -90,6 +95,9 @@ abstract class AbstractNonZeroStream extends AbstractStream {
/*
* Used when removing closed streams from the tree and we know there is no
* need to check for circular references.
+ *
+ * Changes to the priority tree need to be sychronized at the connection
+ * level. This is the caller's responsibility.
*/
final void rePrioritise(AbstractStream parent, int weight) {
if (log.isDebugEnabled()) {
@@ -103,6 +111,27 @@ abstract class AbstractNonZeroStream extends
AbstractStream {
}
+/*
+ * Used when "recycling" a stream and replacing a Stream instance with a
+ * RecycledStream instance.
+ *
+ * Replace this stream with the provided stream in the parent/child
+ * hierarchy.
+ *
+ * Changes to the priority tree need to be sychronized at the connection
+ * level. This is the caller's responsibility.
+ */
+void replaceStream(AbstractNonZeroStream replacement) {
+replacement.setParentStream(getParentStream());
+detachFromParent();
+for (AbstractNonZeroStream child : getChildStreams()) {
+replacement.addChild(child);
+}
+getChildStreams().clear();
+replacement.weight = weight;
+}
+
+
final boolean isClosedFinal() {
return state.isClosedFinal();
}
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index c51efd9..aebb70b 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -94,6 +94,8 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private static final HeaderSink HEADER_SINK = new HeaderSink();
+private final Object priorityTreeLock = new Object();
+
protected final String connectionId;
protected final Http2Protocol protocol;
@@ -104,8 +106,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private volatile Http2Parser parser;
// Simple state machine (sequence of states)
-private AtomicReference connectionState =
-new AtomicReference<>(ConnectionState.NEW);
+private AtomicReference connectionState = new
AtomicReference<>(ConnectionState.NEW);
private volatile long pausedNanoTime = Long.MAX_VALUE;
/**
@@ -1175,7 +1176,7 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
newStreamsSinceLastPrune = 0;
// RFC 7540, 5.3.4 endpoints should maintain state for at least the
-//
[tomcat] branch 9.0.x updated (8dcf599 -> f258efe)
This is an automated email from the ASF dual-hosted git repository.
markt pushed a change to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git.
from 8dcf599 HTTP2 log statement fixes:
new 0d79ae5 Reduce memory footprint of closed http/2 streams
new e1a53fb Reduce memory footprint of closed http/2 streams
new 3783dc8 Reduce memory footprint of closed http/2 streams
new 496b3cf Reduce memory footprint of closed http/2 streams
new a6e8428 Reduce memory footprint of closed http/2 streams
new 2b2ac4f Reduce memory footprint of closed http/2 streams
new d1f204e Fully replace Stream with RecycledStream
new 83f5e03 Refactor the pruning so more stream info is retained for
longer
new a4054eb Update changelog
new f258efe Optimize the iteration when closing idle streams
The 10 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../apache/coyote/http2/AbstractNonZeroStream.java | 143 +++
java/org/apache/coyote/http2/AbstractStream.java | 9 +-
.../apache/coyote/http2/Http2UpgradeHandler.java | 264 +
...tionSettingsRemote.java => RecycledStream.java} | 23 +-
java/org/apache/coyote/http2/Stream.java | 90 +--
.../apache/coyote/http2/StreamStateMachine.java| 21 +-
webapps/docs/changelog.xml | 5 +
7 files changed, 348 insertions(+), 207 deletions(-)
create mode 100644 java/org/apache/coyote/http2/AbstractNonZeroStream.java
copy java/org/apache/coyote/http2/{ConnectionSettingsRemote.java =>
RecycledStream.java} (61%)
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 04/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 496b3cf5dd99179cd5460c728819bd799f662f99
Author: Mark Thomas
AuthorDate: Thu Sep 24 18:11:37 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Refactoring getStream to handle differences between Stream and
RecycledStream
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 2 +
.../apache/coyote/http2/Http2UpgradeHandler.java | 125 ++---
java/org/apache/coyote/http2/RecycledStream.java | 26 -
java/org/apache/coyote/http2/Stream.java | 20 ++--
4 files changed, 111 insertions(+), 62 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 70cfe35..582ab1e 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -98,4 +98,6 @@ abstract class AbstractNonZeroStream extends AbstractStream {
}
abstract boolean isClosedFinal();
+
+abstract void checkState(FrameType frameType) throws Http2Exception;
}
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index ad5e0e4..c51efd9 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -29,6 +29,7 @@ import java.util.Set;
import java.util.TreeSet;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
@@ -121,7 +122,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private HpackDecoder hpackDecoder;
private HpackEncoder hpackEncoder;
-private final Map streams = new ConcurrentHashMap<>();
+private final ConcurrentMap streams = new
ConcurrentHashMap<>();
protected final AtomicInteger activeRemoteStreamCount = new
AtomicInteger(0);
// Start at -1 so the 'add 2' logic in closeIdleStreams() works
private volatile int maxActiveRemoteStreamId = -1;
@@ -1089,7 +1090,22 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
private Stream getStream(int streamId, boolean unknownIsError) throws
ConnectionException {
Integer key = Integer.valueOf(streamId);
-Stream result = streams.get(key);
+AbstractStream result = streams.get(key);
+if (result instanceof Stream) {
+return (Stream) result;
+}
+if (unknownIsError) {
+// Stream has been closed and removed from the map
+throw new
ConnectionException(sm.getString("upgradeHandler.stream.closed",
key.toString()),
+Http2Error.PROTOCOL_ERROR);
+}
+return null;
+}
+
+
+private AbstractNonZeroStream getStreamMayBeClosed(int streamId, boolean
unknownIsError) throws ConnectionException {
+Integer key = Integer.valueOf(streamId);
+AbstractNonZeroStream result = streams.get(key);
if (result == null && unknownIsError) {
// Stream has been closed and removed from the map
throw new
ConnectionException(sm.getString("upgradeHandler.stream.closed",
key.toString()),
@@ -1133,10 +1149,12 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
return;
}
-for (Stream stream : streams.values()) {
-// The connection is closing. Close the associated streams as no
-// longer required (also notifies any threads waiting for
allocations).
-stream.receiveReset(Http2Error.CANCEL.getCode());
+for (AbstractNonZeroStream stream : streams.values()) {
+if (stream instanceof Stream) {
+// The connection is closing. Close the associated streams as
no
+// longer required (also notifies any threads waiting for
allocations).
+((Stream) stream).receiveReset(Http2Error.CANCEL.getCode());
+}
}
try {
socketWrapper.close();
@@ -1193,10 +1211,10 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
TreeSet candidatesStepTwo = new TreeSet<>();
TreeSet candidatesStepThree = new TreeSet<>();
-for (Entry entry : streams.entrySet()) {
-Stream stream = entry.getValue();
+for (Entry entry : streams.entrySet())
{
+AbstractNonZeroStream stream =
[tomcat] 07/10: Fully replace Stream with RecycledStream
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit d1f204ee12ed3aeb7713b95661a53ddaadd7e9b7
Author: Mark Thomas
AuthorDate: Fri Sep 25 12:26:40 2020 +0100
Fully replace Stream with RecycledStream
Profiler (YourKit) suggests retain memory for Stream ~60k whereas
RecycledStream ins ~300 bytes.
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 37 +++--
.../apache/coyote/http2/Http2UpgradeHandler.java | 64 +-
java/org/apache/coyote/http2/RecycledStream.java | 4 +-
java/org/apache/coyote/http2/Stream.java | 2 +-
4 files changed, 74 insertions(+), 33 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 084de33..bbdfa33 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -33,19 +33,17 @@ abstract class AbstractNonZeroStream extends AbstractStream
{
protected final StreamStateMachine state;
-private volatile int weight;
+private volatile int weight = Constants.DEFAULT_WEIGHT;
AbstractNonZeroStream(String connectionId, Integer identifier) {
super(identifier);
-this.weight = Constants.DEFAULT_WEIGHT;
this.state = new StreamStateMachine(connectionId, getIdAsString());
}
-AbstractNonZeroStream(Integer identifier, int weight, StreamStateMachine
state) {
+AbstractNonZeroStream(Integer identifier, StreamStateMachine state) {
super(identifier);
-this.weight = weight;
this.state = state;
}
@@ -56,6 +54,13 @@ abstract class AbstractNonZeroStream extends AbstractStream {
}
+/*
+ * General method used when reprioritising a stream and care needs to be
+ * taken not to create circular references.
+ *
+ * Changes to the priority tree need to be sychronized at the connection
+ * level. This is the caller's responsibility.
+ */
final void rePrioritise(AbstractStream parent, boolean exclusive, int
weight) {
if (log.isDebugEnabled()) {
log.debug(sm.getString("stream.reprioritisation.debug",
@@ -90,6 +95,9 @@ abstract class AbstractNonZeroStream extends AbstractStream {
/*
* Used when removing closed streams from the tree and we know there is no
* need to check for circular references.
+ *
+ * Changes to the priority tree need to be sychronized at the connection
+ * level. This is the caller's responsibility.
*/
final void rePrioritise(AbstractStream parent, int weight) {
if (log.isDebugEnabled()) {
@@ -103,6 +111,27 @@ abstract class AbstractNonZeroStream extends
AbstractStream {
}
+/*
+ * Used when "recycling" a stream and replacing a Stream instance with a
+ * RecycledStream instance.
+ *
+ * Replace this stream with the provided stream in the parent/child
+ * hierarchy.
+ *
+ * Changes to the priority tree need to be sychronized at the connection
+ * level. This is the caller's responsibility.
+ */
+void replaceStream(AbstractNonZeroStream replacement) {
+replacement.setParentStream(getParentStream());
+detachFromParent();
+for (AbstractNonZeroStream child : getChildStreams()) {
+replacement.addChild(child);
+}
+getChildStreams().clear();
+replacement.weight = weight;
+}
+
+
final boolean isClosedFinal() {
return state.isClosedFinal();
}
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index c51efd9..aebb70b 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -94,6 +94,8 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private static final HeaderSink HEADER_SINK = new HeaderSink();
+private final Object priorityTreeLock = new Object();
+
protected final String connectionId;
protected final Http2Protocol protocol;
@@ -104,8 +106,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private volatile Http2Parser parser;
// Simple state machine (sequence of states)
-private AtomicReference connectionState =
-new AtomicReference<>(ConnectionState.NEW);
+private AtomicReference connectionState = new
AtomicReference<>(ConnectionState.NEW);
private volatile long pausedNanoTime = Long.MAX_VALUE;
/**
@@ -1175,7 +1176,7 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
newStreamsSinceLastPrune = 0;
// RFC 7540, 5.3.4 endpoints should maintain state for at least the
-//
[tomcat] branch master updated: Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/master by this push:
new e54267a Fix 64735 ServletContext.addJspFile() always fails with
SecurityManager
e54267a is described below
commit e54267a3f71d29f9f850ed9966e7be9612f219d4
Author: Kyle Stiemann
AuthorDate: Thu Sep 10 16:47:21 2020 -0400
Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
---
build.xml | 13 ++
.../catalina/core/ApplicationContextFacade.java| 5 +
...estApplicationContextFacadeSecurityManager.java | 148 +
.../util/security/SecurityManagerBaseTest.java | 50 +++
webapps/docs/changelog.xml | 5 +
5 files changed, 221 insertions(+)
diff --git a/build.xml b/build.xml
index c5c79cb..1ab4af5 100644
--- a/build.xml
+++ b/build.xml
@@ -1978,8 +1978,21 @@
+
+
+
+
+
+
diff --git a/java/org/apache/catalina/core/ApplicationContextFacade.java
b/java/org/apache/catalina/core/ApplicationContextFacade.java
index 5dd3a37..ef004c6 100644
--- a/java/org/apache/catalina/core/ApplicationContextFacade.java
+++ b/java/org/apache/catalina/core/ApplicationContextFacade.java
@@ -117,6 +117,11 @@ public class ApplicationContextFacade implements
ServletContext {
classCache.put("getAttribute", clazz);
classCache.put("log", clazz);
classCache.put("setSessionTrackingModes", new Class[]{Set.class} );
+classCache.put("addJspFile", new Class[]{String.class, String.class});
+classCache.put("declareRoles", new Class[]{String[].class});
+classCache.put("setSessionTimeout", new Class[]{int.class});
+classCache.put("setRequestCharacterEncoding", new
Class[]{String.class});
+classCache.put("setResponseCharacterEncoding", new
Class[]{String.class});
}
diff --git
a/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
new file mode 100644
index 000..fc5a5d5
--- /dev/null
+++
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.catalina.core;
+
+import java.lang.reflect.Array;
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.lang.reflect.Modifier;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.stream.Collectors;
+
+import org.junit.Assert;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameter;
+
+import org.apache.catalina.security.SecurityUtil;
+import org.apache.tomcat.util.security.SecurityManagerBaseTest;
+import org.easymock.EasyMock;
+import org.easymock.IExpectationSetters;
+import org.easymock.internal.LastControl;
+
+@RunWith(Parameterized.class)
+public final class TestApplicationContextFacadeSecurityManager extends
SecurityManagerBaseTest {
+
+/**
+ * @return {@link Collection} of non-static, non-object, public {@link
+ * Method}s in {@link ApplicationContextFacade} to be run with the the Java
+ * 2 {@link SecurityManager} been enabled.
+ */
+@Parameterized.Parameters(name = "{index}: method={0}")
+public static Collection publicApplicationContextFacadeMethods() {
+return Arrays.stream(ApplicationContextFacade.class.getMethods())
+.filter(method -> !Modifier.isStatic(method.getModifiers()))
+.filter(method -> {
+try {
+Object.class.getMethod(method.getName(),
method.getParameterTypes());
+return false;
+} catch (final NoSuchMethodException e) {
+return true;
+}
+})
+
[tomcat] 01/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 0d79ae569f3fd2646ff76d395f0f56e98b798a98
Author: Mark Thomas
AuthorDate: Wed Sep 23 19:59:13 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Initial changes to object hierarchy in preparation for further
changes.
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 29 ++
java/org/apache/coyote/http2/AbstractStream.java | 3 +-
java/org/apache/coyote/http2/RecycledStream.java | 45 ++
java/org/apache/coyote/http2/Stream.java | 2 +-
4 files changed, 77 insertions(+), 2 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
new file mode 100644
index 000..63fb5e7
--- /dev/null
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.coyote.http2;
+
+
+/**
+ * Base class for all streams other than stream 0, the connection. Primarily
+ * provides functionality shared between full Stream and RecycledStream.
+ */
+abstract class AbstractNonZeroStream extends AbstractStream {
+
+AbstractNonZeroStream(Integer identifier) {
+super(identifier);
+}
+}
diff --git a/java/org/apache/coyote/http2/AbstractStream.java
b/java/org/apache/coyote/http2/AbstractStream.java
index 088d5b0..21963ee 100644
--- a/java/org/apache/coyote/http2/AbstractStream.java
+++ b/java/org/apache/coyote/http2/AbstractStream.java
@@ -25,7 +25,8 @@ import org.apache.juli.logging.LogFactory;
import org.apache.tomcat.util.res.StringManager;
/**
- * Used to managed prioritisation.
+ * Base class for all streams including the connection (referred to as Stream
0)
+ * and is used primarily when managing prioritization.
*/
abstract class AbstractStream {
diff --git a/java/org/apache/coyote/http2/RecycledStream.java
b/java/org/apache/coyote/http2/RecycledStream.java
new file mode 100644
index 000..f500646
--- /dev/null
+++ b/java/org/apache/coyote/http2/RecycledStream.java
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.coyote.http2;
+
+/**
+ * Represents a closed stream in the priority tree. Used in preference to the
+ * full {@link Stream} as has much lower memory usage.
+ */
+class RecycledStream extends AbstractNonZeroStream {
+
+private final String connectionId;
+private int weight;
+
+RecycledStream(Stream stream) {
+super(stream.getIdentifier());
+connectionId = stream.getConnectionId();
+weight = stream.getWeight();
+}
+
+
+@Override
+String getConnectionId() {
+return connectionId;
+}
+
+
+@Override
+int getWeight() {
+return weight;
+}
+}
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index af777ab..56bc129 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -46,7 +46,7 @@ import org.apache.tomcat.util.net.ApplicationBufferHandler;
import org.apache.tomcat.util.net.WriteBuffer;
import
[tomcat] 05/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit a6e84280ac145ab14f86a2cbe37238f9acda31d6
Author: Mark Thomas
AuthorDate: Thu Sep 24 21:36:19 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Refactor StreamStateMachine to remove reference to Stream
---
java/org/apache/coyote/http2/Stream.java| 2 +-
.../org/apache/coyote/http2/StreamStateMachine.java | 21 -
2 files changed, 13 insertions(+), 10 deletions(-)
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index 8aa4a7b..f6210d7 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -97,7 +97,7 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
this.handler = handler;
handler.addChild(this);
setWindowSize(handler.getRemoteSettings().getInitialWindowSize());
-state = new StreamStateMachine(this);
+state = new StreamStateMachine(getConnectionId(), getIdAsString());
if (coyoteRequest == null) {
// HTTP/2 new request
this.coyoteRequest = new Request();
diff --git a/java/org/apache/coyote/http2/StreamStateMachine.java
b/java/org/apache/coyote/http2/StreamStateMachine.java
index 851e1a8..72028fd 100644
--- a/java/org/apache/coyote/http2/StreamStateMachine.java
+++ b/java/org/apache/coyote/http2/StreamStateMachine.java
@@ -39,12 +39,15 @@ class StreamStateMachine {
private static final Log log = LogFactory.getLog(StreamStateMachine.class);
private static final StringManager sm =
StringManager.getManager(StreamStateMachine.class);
-private final Stream stream;
+private final String connectionId;
+private final String streamId;
+
private State state;
-StreamStateMachine(Stream stream) {
-this.stream = stream;
+StreamStateMachine(String connectionId, String streamId) {
+this.connectionId = connectionId;
+this.streamId = streamId;
stateChange(null, State.IDLE);
}
@@ -92,7 +95,7 @@ class StreamStateMachine {
public synchronized void sendReset() {
if (state == State.IDLE) {
throw new
IllegalStateException(sm.getString("streamStateMachine.debug.change",
-stream.getConnectionId(), stream.getIdAsString(), state));
+connectionId, streamId, state));
}
if (state.canReset()) {
stateChange(state, State.CLOSED_RST_TX);
@@ -109,8 +112,8 @@ class StreamStateMachine {
if (state == oldState) {
state = newState;
if (log.isDebugEnabled()) {
-log.debug(sm.getString("streamStateMachine.debug.change",
stream.getConnectionId(),
-stream.getIdAsString(), oldState, newState));
+log.debug(sm.getString("streamStateMachine.debug.change",
connectionId,
+streamId, oldState, newState));
}
}
}
@@ -122,12 +125,12 @@ class StreamStateMachine {
if (!isFrameTypePermitted(frameType)) {
if (state.connectionErrorForInvalidFrame) {
throw new
ConnectionException(sm.getString("streamStateMachine.invalidFrame",
-stream.getConnectionId(), stream.getIdAsString(),
state, frameType),
+connectionId, streamId, state, frameType),
state.errorCodeForInvalidFrame);
} else {
throw new
StreamException(sm.getString("streamStateMachine.invalidFrame",
-stream.getConnectionId(), stream.getIdAsString(),
state, frameType),
-state.errorCodeForInvalidFrame, stream.getIdAsInt());
+connectionId, streamId, state, frameType),
+state.errorCodeForInvalidFrame,
Integer.parseInt(streamId));
}
}
}
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 08/10: Refactor the pruning so more stream info is retained for longer
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 83f5e03b52c54a9b213e979c9bb47e0967530cbe
Author: Mark Thomas
AuthorDate: Fri Sep 25 14:14:06 2020 +0100
Refactor the pruning so more stream info is retained for longer
The memory impact of this is mitigated by the previous changes to
replace closed Stream instances with RecycledStream instances.
---
.../apache/coyote/http2/Http2UpgradeHandler.java | 89 --
1 file changed, 49 insertions(+), 40 deletions(-)
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index aebb70b..9f71af9 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -30,6 +30,7 @@ import java.util.TreeSet;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentSkipListMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
@@ -123,7 +124,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private HpackDecoder hpackDecoder;
private HpackEncoder hpackEncoder;
-private final ConcurrentMap streams = new
ConcurrentHashMap<>();
+private final ConcurrentMap streams = new
ConcurrentSkipListMap<>();
protected final AtomicInteger activeRemoteStreamCount = new
AtomicInteger(0);
// Start at -1 so the 'add 2' logic in closeIdleStreams() works
private volatile int maxActiveRemoteStreamId = -1;
@@ -1207,78 +1208,86 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
// 2. Completed streams used for a request with children
// 3. Closed final streams
//
-// Steps 1 and 2 will always be completed.
-// Step 3 will be completed to the minimum extent necessary to bring
the
-// total number of streams under the limit.
+// The pruning halts as soon as enough streams have been pruned.
// Use these sets to track the different classes of streams
-TreeSet candidatesStepOne = new TreeSet<>();
TreeSet candidatesStepTwo = new TreeSet<>();
TreeSet candidatesStepThree = new TreeSet<>();
-for (Entry entry : streams.entrySet())
{
-AbstractNonZeroStream stream = entry.getValue();
+// Step 1
+// Iterator is in key order so we automatically have the oldest streams
+// first
+for (AbstractNonZeroStream stream : streams.values()) {
// Never remove active streams
if (stream instanceof Stream && ((Stream) stream).isActive()) {
continue;
}
-final Integer streamIdentifier = entry.getKey();
if (stream.isClosedFinal()) {
// This stream went from IDLE to CLOSED and is likely to have
// been created by the client as part of the priority tree.
-candidatesStepThree.add(streamIdentifier);
+// Candidate for steo 3.
+candidatesStepThree.add(stream.getIdentifier());
} else if (stream.getChildStreams().size() == 0) {
-// Closed, no children
-candidatesStepOne.add(streamIdentifier);
-} else {
-// Closed, with children
-candidatesStepTwo.add(streamIdentifier);
-}
-}
-
-// Process the step one list
-for (Integer streamIdToRemove : candidatesStepOne) {
-// Remove this childless stream
-AbstractNonZeroStream removedStream =
streams.remove(streamIdToRemove);
-removedStream.detachFromParent();
-toClose--;
-if (log.isDebugEnabled()) {
-log.debug(sm.getString("upgradeHandler.pruned", connectionId,
streamIdToRemove));
-}
-
-// Did this make the parent childless?
-AbstractStream parent = removedStream.getParentStream();
-while (parent instanceof Stream && !((Stream) parent).isActive() &&
-!((Stream) parent).isClosedFinal() &&
parent.getChildStreams().size() == 0) {
-streams.remove(parent.getIdentifier());
-parent.detachFromParent();
-toClose--;
+// Prune it
+AbstractStream parent = stream.getParentStream();
+streams.remove(stream.getIdentifier());
+stream.detachFromParent();
if (log.isDebugEnabled()) {
-log.debug(sm.getString("upgradeHandler.pruned",
connectionId, streamIdToRemove));
+
[tomcat] 02/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit e1a53fb2d567b81cfaa4fcfa94ef870f15a6e289
Author: Mark Thomas
AuthorDate: Thu Sep 24 17:32:49 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Pull up re-prioritisation
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 71 ++
java/org/apache/coyote/http2/AbstractStream.java | 6 +-
.../apache/coyote/http2/Http2UpgradeHandler.java | 6 +-
java/org/apache/coyote/http2/RecycledStream.java | 10 +--
java/org/apache/coyote/http2/Stream.java | 55 -
5 files changed, 78 insertions(+), 70 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 63fb5e7..ce08cc5 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -16,6 +16,11 @@
*/
package org.apache.coyote.http2;
+import java.util.Iterator;
+
+import org.apache.juli.logging.Log;
+import org.apache.juli.logging.LogFactory;
+import org.apache.tomcat.util.res.StringManager;
/**
* Base class for all streams other than stream 0, the connection. Primarily
@@ -23,7 +28,73 @@ package org.apache.coyote.http2;
*/
abstract class AbstractNonZeroStream extends AbstractStream {
+private static final Log log =
LogFactory.getLog(AbstractNonZeroStream.class);
+private static final StringManager sm =
StringManager.getManager(AbstractNonZeroStream.class);
+
+private volatile int weight;
+
+
AbstractNonZeroStream(Integer identifier) {
+this(identifier, Constants.DEFAULT_WEIGHT);
+}
+
+
+AbstractNonZeroStream(Integer identifier, int weight) {
super(identifier);
+this.weight = weight;
+}
+
+
+@Override
+final int getWeight() {
+return weight;
}
+
+
+final void rePrioritise(AbstractStream parent, boolean exclusive, int
weight) {
+if (log.isDebugEnabled()) {
+log.debug(sm.getString("stream.reprioritisation.debug",
+getConnectionId(), getIdAsString(),
Boolean.toString(exclusive),
+parent.getIdAsString(), Integer.toString(weight)));
+}
+
+// Check if new parent is a descendant of this stream
+if (isDescendant(parent)) {
+parent.detachFromParent();
+// Cast is always safe since any descendant of this stream must be
+// an instance of Stream
+getParentStream().addChild((Stream) parent);
+}
+
+if (exclusive) {
+// Need to move children of the new parent to be children of this
+// stream. Slightly convoluted to avoid concurrent modification.
+Iterator parentsChildren =
parent.getChildStreams().iterator();
+while (parentsChildren.hasNext()) {
+AbstractNonZeroStream parentsChild = parentsChildren.next();
+parentsChildren.remove();
+this.addChild(parentsChild);
+}
+}
+detachFromParent();
+parent.addChild(this);
+this.weight = weight;
+}
+
+
+/*
+ * Used when removing closed streams from the tree and we know there is no
+ * need to check for circular references.
+ */
+final void rePrioritise(AbstractStream parent, int weight) {
+if (log.isDebugEnabled()) {
+log.debug(sm.getString("stream.reprioritisation.debug",
+getConnectionId(), getIdAsString(), Boolean.FALSE,
+parent.getIdAsString(), Integer.toString(weight)));
+}
+
+parent.addChild(this);
+this.weight = weight;
+}
+
}
diff --git a/java/org/apache/coyote/http2/AbstractStream.java
b/java/org/apache/coyote/http2/AbstractStream.java
index 21963ee..c7374b6 100644
--- a/java/org/apache/coyote/http2/AbstractStream.java
+++ b/java/org/apache/coyote/http2/AbstractStream.java
@@ -37,7 +37,7 @@ abstract class AbstractStream {
private final String idAsString;
private volatile AbstractStream parentStream = null;
-private final Set childStreams = Collections.newSetFromMap(new
ConcurrentHashMap<>());
+private final Set childStreams =
Collections.newSetFromMap(new ConcurrentHashMap<>());
private long windowSize =
ConnectionSettingsBase.DEFAULT_INITIAL_WINDOW_SIZE;
@@ -70,7 +70,7 @@ abstract class AbstractStream {
}
-final void addChild(Stream child) {
+final void addChild(AbstractNonZeroStream child) {
child.setParentStream(this);
childStreams.add(child);
}
@@ -97,7 +97,7 @@ abstract class AbstractStream {
}
-final Set
[tomcat] 09/10: Update changelog
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit a4054eb3f27ac0d242d5f2a6c4599e3f6cb9a519 Author: Mark Thomas AuthorDate: Fri Sep 25 14:33:45 2020 +0100 Update changelog --- webapps/docs/changelog.xml | 5 + 1 file changed, 5 insertions(+) diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml index 0be3539..a82ec7b 100644 --- a/webapps/docs/changelog.xml +++ b/webapps/docs/changelog.xml @@ -127,6 +127,11 @@ (markt) +Refactor the handling of closed HTTP/2 streams to reduce the heap usage +associated with used streams and to retain information for more streams +in the priority tree. (markt) + + Don't send the Keep-Alive response header if the connection has been explicitly closed. (markt) - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 06/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 2b2ac4f3ce99027d8589595e921570495f7b08c0
Author: Mark Thomas
AuthorDate: Thu Sep 24 21:47:09 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Pull up state
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 21 -
java/org/apache/coyote/http2/RecycledStream.java| 16 +---
java/org/apache/coyote/http2/Stream.java| 16 +---
3 files changed, 18 insertions(+), 35 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 582ab1e..084de33 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -31,17 +31,22 @@ abstract class AbstractNonZeroStream extends AbstractStream
{
private static final Log log =
LogFactory.getLog(AbstractNonZeroStream.class);
private static final StringManager sm =
StringManager.getManager(AbstractNonZeroStream.class);
+protected final StreamStateMachine state;
+
private volatile int weight;
-AbstractNonZeroStream(Integer identifier) {
-this(identifier, Constants.DEFAULT_WEIGHT);
+AbstractNonZeroStream(String connectionId, Integer identifier) {
+super(identifier);
+this.weight = Constants.DEFAULT_WEIGHT;
+this.state = new StreamStateMachine(connectionId, getIdAsString());
}
-AbstractNonZeroStream(Integer identifier, int weight) {
+AbstractNonZeroStream(Integer identifier, int weight, StreamStateMachine
state) {
super(identifier);
this.weight = weight;
+this.state = state;
}
@@ -97,7 +102,13 @@ abstract class AbstractNonZeroStream extends AbstractStream
{
this.weight = weight;
}
-abstract boolean isClosedFinal();
-abstract void checkState(FrameType frameType) throws Http2Exception;
+final boolean isClosedFinal() {
+return state.isClosedFinal();
+}
+
+
+final void checkState(FrameType frameType) throws Http2Exception {
+state.checkFrameType(frameType);
+}
}
diff --git a/java/org/apache/coyote/http2/RecycledStream.java
b/java/org/apache/coyote/http2/RecycledStream.java
index 1915dff..9d6177c 100644
--- a/java/org/apache/coyote/http2/RecycledStream.java
+++ b/java/org/apache/coyote/http2/RecycledStream.java
@@ -23,12 +23,10 @@ package org.apache.coyote.http2;
class RecycledStream extends AbstractNonZeroStream {
private final String connectionId;
-private final StreamStateMachine state;
RecycledStream(String connectionId, Integer identifier, int weight,
StreamStateMachine state) {
-super(identifier, weight);
+super(identifier, weight, state);
this.connectionId = connectionId;
-this.state = state;
}
@@ -38,18 +36,6 @@ class RecycledStream extends AbstractNonZeroStream {
}
-@Override
-boolean isClosedFinal() {
-return state.isClosedFinal();
-}
-
-
-@Override
-final void checkState(FrameType frameType) throws Http2Exception {
-state.checkFrameType(frameType);
-}
-
-
@SuppressWarnings("sync-override")
@Override
void incrementWindowSize(int increment) throws Http2Exception {
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index f6210d7..140a76d 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -69,7 +69,6 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
private volatile long contentLengthReceived = 0;
private final Http2UpgradeHandler handler;
-private final StreamStateMachine state;
private final WindowAllocationManager allocationManager = new
WindowAllocationManager(this);
// State machine would be too much overhead
@@ -93,11 +92,10 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
Stream(Integer identifier, Http2UpgradeHandler handler, Request
coyoteRequest) {
-super(identifier);
+super(handler.getConnectionId(), identifier);
this.handler = handler;
handler.addChild(this);
setWindowSize(handler.getRemoteSettings().getInitialWindowSize());
-state = new StreamStateMachine(getConnectionId(), getIdAsString());
if (coyoteRequest == null) {
// HTTP/2 new request
this.coyoteRequest = new Request();
@@ -194,12 +192,6 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
@Override
-final void checkState(FrameType frameType) throws
[tomcat] 10/10: Optimize the iteration when closing idle streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit f258efef6c1e9c71ceaf1f50e1dc354f2514ab3a
Author: Martin Tzvetanov Grigorov
AuthorDate: Mon Sep 28 14:53:25 2020 +0300
Optimize the iteration when closing idle streams
---
java/org/apache/coyote/http2/Http2UpgradeHandler.java | 16
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index 9f71af9..a1002c1 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -29,7 +29,7 @@ import java.util.Set;
import java.util.TreeSet;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
-import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentNavigableMap;
import java.util.concurrent.ConcurrentSkipListMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
@@ -124,7 +124,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private HpackDecoder hpackDecoder;
private HpackEncoder hpackEncoder;
-private final ConcurrentMap streams = new
ConcurrentSkipListMap<>();
+private final ConcurrentNavigableMap
streams = new ConcurrentSkipListMap<>();
protected final AtomicInteger activeRemoteStreamCount = new
AtomicInteger(0);
// Start at -1 so the 'add 2' logic in closeIdleStreams() works
private volatile int maxActiveRemoteStreamId = -1;
@@ -1530,12 +1530,12 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
private void closeIdleStreams(int newMaxActiveRemoteStreamId) {
-for (Entry entry : streams.entrySet()) {
-int id = entry.getKey().intValue();
-if (id > maxActiveRemoteStreamId && id <
newMaxActiveRemoteStreamId) {
-if (entry.getValue() instanceof Stream) {
-((Stream) entry.getValue()).closeIfIdle();
-}
+final ConcurrentNavigableMap subMap =
streams.subMap(
+Integer.valueOf(maxActiveRemoteStreamId), false,
+Integer.valueOf(newMaxActiveRemoteStreamId), false);
+for (AbstractNonZeroStream stream : subMap.values()) {
+if (stream instanceof Stream) {
+((Stream)stream).closeIfIdle();
}
}
maxActiveRemoteStreamId = newMaxActiveRemoteStreamId;
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 03/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 3783dc83065414cd85179e9a9014da44780e14b6
Author: Mark Thomas
AuthorDate: Thu Sep 24 17:50:43 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Pull-up isClosedFinal()
---
java/org/apache/coyote/http2/AbstractNonZeroStream.java | 1 +
java/org/apache/coyote/http2/RecycledStream.java| 8
java/org/apache/coyote/http2/Stream.java| 1 +
3 files changed, 10 insertions(+)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index ce08cc5..70cfe35 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -97,4 +97,5 @@ abstract class AbstractNonZeroStream extends AbstractStream {
this.weight = weight;
}
+abstract boolean isClosedFinal();
}
diff --git a/java/org/apache/coyote/http2/RecycledStream.java
b/java/org/apache/coyote/http2/RecycledStream.java
index 1e630df..dbbdc10 100644
--- a/java/org/apache/coyote/http2/RecycledStream.java
+++ b/java/org/apache/coyote/http2/RecycledStream.java
@@ -23,10 +23,12 @@ package org.apache.coyote.http2;
class RecycledStream extends AbstractNonZeroStream {
private final String connectionId;
+private final boolean closedFinal;
RecycledStream(Stream stream) {
super(stream.getIdentifier(), stream.getWeight());
connectionId = stream.getConnectionId();
+closedFinal = stream.isClosedFinal();
}
@@ -34,4 +36,10 @@ class RecycledStream extends AbstractNonZeroStream {
String getConnectionId() {
return connectionId;
}
+
+
+@Override
+boolean isClosedFinal() {
+return closedFinal;
+}
}
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index ef7ed29..489ce53 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -646,6 +646,7 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
}
+@Override
final boolean isClosedFinal() {
return state.isClosedFinal();
}
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 04/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 496b3cf5dd99179cd5460c728819bd799f662f99
Author: Mark Thomas
AuthorDate: Thu Sep 24 18:11:37 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Refactoring getStream to handle differences between Stream and
RecycledStream
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 2 +
.../apache/coyote/http2/Http2UpgradeHandler.java | 125 ++---
java/org/apache/coyote/http2/RecycledStream.java | 26 -
java/org/apache/coyote/http2/Stream.java | 20 ++--
4 files changed, 111 insertions(+), 62 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 70cfe35..582ab1e 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -98,4 +98,6 @@ abstract class AbstractNonZeroStream extends AbstractStream {
}
abstract boolean isClosedFinal();
+
+abstract void checkState(FrameType frameType) throws Http2Exception;
}
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index ad5e0e4..c51efd9 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -29,6 +29,7 @@ import java.util.Set;
import java.util.TreeSet;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
@@ -121,7 +122,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private HpackDecoder hpackDecoder;
private HpackEncoder hpackEncoder;
-private final Map streams = new ConcurrentHashMap<>();
+private final ConcurrentMap streams = new
ConcurrentHashMap<>();
protected final AtomicInteger activeRemoteStreamCount = new
AtomicInteger(0);
// Start at -1 so the 'add 2' logic in closeIdleStreams() works
private volatile int maxActiveRemoteStreamId = -1;
@@ -1089,7 +1090,22 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
private Stream getStream(int streamId, boolean unknownIsError) throws
ConnectionException {
Integer key = Integer.valueOf(streamId);
-Stream result = streams.get(key);
+AbstractStream result = streams.get(key);
+if (result instanceof Stream) {
+return (Stream) result;
+}
+if (unknownIsError) {
+// Stream has been closed and removed from the map
+throw new
ConnectionException(sm.getString("upgradeHandler.stream.closed",
key.toString()),
+Http2Error.PROTOCOL_ERROR);
+}
+return null;
+}
+
+
+private AbstractNonZeroStream getStreamMayBeClosed(int streamId, boolean
unknownIsError) throws ConnectionException {
+Integer key = Integer.valueOf(streamId);
+AbstractNonZeroStream result = streams.get(key);
if (result == null && unknownIsError) {
// Stream has been closed and removed from the map
throw new
ConnectionException(sm.getString("upgradeHandler.stream.closed",
key.toString()),
@@ -1133,10 +1149,12 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
return;
}
-for (Stream stream : streams.values()) {
-// The connection is closing. Close the associated streams as no
-// longer required (also notifies any threads waiting for
allocations).
-stream.receiveReset(Http2Error.CANCEL.getCode());
+for (AbstractNonZeroStream stream : streams.values()) {
+if (stream instanceof Stream) {
+// The connection is closing. Close the associated streams as
no
+// longer required (also notifies any threads waiting for
allocations).
+((Stream) stream).receiveReset(Http2Error.CANCEL.getCode());
+}
}
try {
socketWrapper.close();
@@ -1193,10 +1211,10 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
TreeSet candidatesStepTwo = new TreeSet<>();
TreeSet candidatesStepThree = new TreeSet<>();
-for (Entry entry : streams.entrySet()) {
-Stream stream = entry.getValue();
+for (Entry entry : streams.entrySet())
{
+AbstractNonZeroStream stream =
[tomcat] 08/10: Refactor the pruning so more stream info is retained for longer
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 83f5e03b52c54a9b213e979c9bb47e0967530cbe
Author: Mark Thomas
AuthorDate: Fri Sep 25 14:14:06 2020 +0100
Refactor the pruning so more stream info is retained for longer
The memory impact of this is mitigated by the previous changes to
replace closed Stream instances with RecycledStream instances.
---
.../apache/coyote/http2/Http2UpgradeHandler.java | 89 --
1 file changed, 49 insertions(+), 40 deletions(-)
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index aebb70b..9f71af9 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -30,6 +30,7 @@ import java.util.TreeSet;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentSkipListMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
@@ -123,7 +124,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private HpackDecoder hpackDecoder;
private HpackEncoder hpackEncoder;
-private final ConcurrentMap streams = new
ConcurrentHashMap<>();
+private final ConcurrentMap streams = new
ConcurrentSkipListMap<>();
protected final AtomicInteger activeRemoteStreamCount = new
AtomicInteger(0);
// Start at -1 so the 'add 2' logic in closeIdleStreams() works
private volatile int maxActiveRemoteStreamId = -1;
@@ -1207,78 +1208,86 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
// 2. Completed streams used for a request with children
// 3. Closed final streams
//
-// Steps 1 and 2 will always be completed.
-// Step 3 will be completed to the minimum extent necessary to bring
the
-// total number of streams under the limit.
+// The pruning halts as soon as enough streams have been pruned.
// Use these sets to track the different classes of streams
-TreeSet candidatesStepOne = new TreeSet<>();
TreeSet candidatesStepTwo = new TreeSet<>();
TreeSet candidatesStepThree = new TreeSet<>();
-for (Entry entry : streams.entrySet())
{
-AbstractNonZeroStream stream = entry.getValue();
+// Step 1
+// Iterator is in key order so we automatically have the oldest streams
+// first
+for (AbstractNonZeroStream stream : streams.values()) {
// Never remove active streams
if (stream instanceof Stream && ((Stream) stream).isActive()) {
continue;
}
-final Integer streamIdentifier = entry.getKey();
if (stream.isClosedFinal()) {
// This stream went from IDLE to CLOSED and is likely to have
// been created by the client as part of the priority tree.
-candidatesStepThree.add(streamIdentifier);
+// Candidate for steo 3.
+candidatesStepThree.add(stream.getIdentifier());
} else if (stream.getChildStreams().size() == 0) {
-// Closed, no children
-candidatesStepOne.add(streamIdentifier);
-} else {
-// Closed, with children
-candidatesStepTwo.add(streamIdentifier);
-}
-}
-
-// Process the step one list
-for (Integer streamIdToRemove : candidatesStepOne) {
-// Remove this childless stream
-AbstractNonZeroStream removedStream =
streams.remove(streamIdToRemove);
-removedStream.detachFromParent();
-toClose--;
-if (log.isDebugEnabled()) {
-log.debug(sm.getString("upgradeHandler.pruned", connectionId,
streamIdToRemove));
-}
-
-// Did this make the parent childless?
-AbstractStream parent = removedStream.getParentStream();
-while (parent instanceof Stream && !((Stream) parent).isActive() &&
-!((Stream) parent).isClosedFinal() &&
parent.getChildStreams().size() == 0) {
-streams.remove(parent.getIdentifier());
-parent.detachFromParent();
-toClose--;
+// Prune it
+AbstractStream parent = stream.getParentStream();
+streams.remove(stream.getIdentifier());
+stream.detachFromParent();
if (log.isDebugEnabled()) {
-log.debug(sm.getString("upgradeHandler.pruned",
connectionId, streamIdToRemove));
+
[tomcat] 07/10: Fully replace Stream with RecycledStream
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit d1f204ee12ed3aeb7713b95661a53ddaadd7e9b7
Author: Mark Thomas
AuthorDate: Fri Sep 25 12:26:40 2020 +0100
Fully replace Stream with RecycledStream
Profiler (YourKit) suggests retain memory for Stream ~60k whereas
RecycledStream ins ~300 bytes.
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 37 +++--
.../apache/coyote/http2/Http2UpgradeHandler.java | 64 +-
java/org/apache/coyote/http2/RecycledStream.java | 4 +-
java/org/apache/coyote/http2/Stream.java | 2 +-
4 files changed, 74 insertions(+), 33 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 084de33..bbdfa33 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -33,19 +33,17 @@ abstract class AbstractNonZeroStream extends AbstractStream
{
protected final StreamStateMachine state;
-private volatile int weight;
+private volatile int weight = Constants.DEFAULT_WEIGHT;
AbstractNonZeroStream(String connectionId, Integer identifier) {
super(identifier);
-this.weight = Constants.DEFAULT_WEIGHT;
this.state = new StreamStateMachine(connectionId, getIdAsString());
}
-AbstractNonZeroStream(Integer identifier, int weight, StreamStateMachine
state) {
+AbstractNonZeroStream(Integer identifier, StreamStateMachine state) {
super(identifier);
-this.weight = weight;
this.state = state;
}
@@ -56,6 +54,13 @@ abstract class AbstractNonZeroStream extends AbstractStream {
}
+/*
+ * General method used when reprioritising a stream and care needs to be
+ * taken not to create circular references.
+ *
+ * Changes to the priority tree need to be sychronized at the connection
+ * level. This is the caller's responsibility.
+ */
final void rePrioritise(AbstractStream parent, boolean exclusive, int
weight) {
if (log.isDebugEnabled()) {
log.debug(sm.getString("stream.reprioritisation.debug",
@@ -90,6 +95,9 @@ abstract class AbstractNonZeroStream extends AbstractStream {
/*
* Used when removing closed streams from the tree and we know there is no
* need to check for circular references.
+ *
+ * Changes to the priority tree need to be sychronized at the connection
+ * level. This is the caller's responsibility.
*/
final void rePrioritise(AbstractStream parent, int weight) {
if (log.isDebugEnabled()) {
@@ -103,6 +111,27 @@ abstract class AbstractNonZeroStream extends
AbstractStream {
}
+/*
+ * Used when "recycling" a stream and replacing a Stream instance with a
+ * RecycledStream instance.
+ *
+ * Replace this stream with the provided stream in the parent/child
+ * hierarchy.
+ *
+ * Changes to the priority tree need to be sychronized at the connection
+ * level. This is the caller's responsibility.
+ */
+void replaceStream(AbstractNonZeroStream replacement) {
+replacement.setParentStream(getParentStream());
+detachFromParent();
+for (AbstractNonZeroStream child : getChildStreams()) {
+replacement.addChild(child);
+}
+getChildStreams().clear();
+replacement.weight = weight;
+}
+
+
final boolean isClosedFinal() {
return state.isClosedFinal();
}
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index c51efd9..aebb70b 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -94,6 +94,8 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private static final HeaderSink HEADER_SINK = new HeaderSink();
+private final Object priorityTreeLock = new Object();
+
protected final String connectionId;
protected final Http2Protocol protocol;
@@ -104,8 +106,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private volatile Http2Parser parser;
// Simple state machine (sequence of states)
-private AtomicReference connectionState =
-new AtomicReference<>(ConnectionState.NEW);
+private AtomicReference connectionState = new
AtomicReference<>(ConnectionState.NEW);
private volatile long pausedNanoTime = Long.MAX_VALUE;
/**
@@ -1175,7 +1176,7 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
newStreamsSinceLastPrune = 0;
// RFC 7540, 5.3.4 endpoints should maintain state for at least the
-//
[tomcat] 09/10: Update changelog
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit a4054eb3f27ac0d242d5f2a6c4599e3f6cb9a519 Author: Mark Thomas AuthorDate: Fri Sep 25 14:33:45 2020 +0100 Update changelog --- webapps/docs/changelog.xml | 5 + 1 file changed, 5 insertions(+) diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml index 0be3539..a82ec7b 100644 --- a/webapps/docs/changelog.xml +++ b/webapps/docs/changelog.xml @@ -127,6 +127,11 @@ (markt) +Refactor the handling of closed HTTP/2 streams to reduce the heap usage +associated with used streams and to retain information for more streams +in the priority tree. (markt) + + Don't send the Keep-Alive response header if the connection has been explicitly closed. (markt) - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] branch master updated: Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/master by this push:
new e54267a Fix 64735 ServletContext.addJspFile() always fails with
SecurityManager
e54267a is described below
commit e54267a3f71d29f9f850ed9966e7be9612f219d4
Author: Kyle Stiemann
AuthorDate: Thu Sep 10 16:47:21 2020 -0400
Fix 64735 ServletContext.addJspFile() always fails with SecurityManager
---
build.xml | 13 ++
.../catalina/core/ApplicationContextFacade.java| 5 +
...estApplicationContextFacadeSecurityManager.java | 148 +
.../util/security/SecurityManagerBaseTest.java | 50 +++
webapps/docs/changelog.xml | 5 +
5 files changed, 221 insertions(+)
diff --git a/build.xml b/build.xml
index c5c79cb..1ab4af5 100644
--- a/build.xml
+++ b/build.xml
@@ -1978,8 +1978,21 @@
+
+
+
+
+
+
diff --git a/java/org/apache/catalina/core/ApplicationContextFacade.java
b/java/org/apache/catalina/core/ApplicationContextFacade.java
index 5dd3a37..ef004c6 100644
--- a/java/org/apache/catalina/core/ApplicationContextFacade.java
+++ b/java/org/apache/catalina/core/ApplicationContextFacade.java
@@ -117,6 +117,11 @@ public class ApplicationContextFacade implements
ServletContext {
classCache.put("getAttribute", clazz);
classCache.put("log", clazz);
classCache.put("setSessionTrackingModes", new Class[]{Set.class} );
+classCache.put("addJspFile", new Class[]{String.class, String.class});
+classCache.put("declareRoles", new Class[]{String[].class});
+classCache.put("setSessionTimeout", new Class[]{int.class});
+classCache.put("setRequestCharacterEncoding", new
Class[]{String.class});
+classCache.put("setResponseCharacterEncoding", new
Class[]{String.class});
}
diff --git
a/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
new file mode 100644
index 000..fc5a5d5
--- /dev/null
+++
b/test/org/apache/catalina/core/TestApplicationContextFacadeSecurityManager.java
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.catalina.core;
+
+import java.lang.reflect.Array;
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.lang.reflect.Modifier;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.stream.Collectors;
+
+import org.junit.Assert;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameter;
+
+import org.apache.catalina.security.SecurityUtil;
+import org.apache.tomcat.util.security.SecurityManagerBaseTest;
+import org.easymock.EasyMock;
+import org.easymock.IExpectationSetters;
+import org.easymock.internal.LastControl;
+
+@RunWith(Parameterized.class)
+public final class TestApplicationContextFacadeSecurityManager extends
SecurityManagerBaseTest {
+
+/**
+ * @return {@link Collection} of non-static, non-object, public {@link
+ * Method}s in {@link ApplicationContextFacade} to be run with the the Java
+ * 2 {@link SecurityManager} been enabled.
+ */
+@Parameterized.Parameters(name = "{index}: method={0}")
+public static Collection publicApplicationContextFacadeMethods() {
+return Arrays.stream(ApplicationContextFacade.class.getMethods())
+.filter(method -> !Modifier.isStatic(method.getModifiers()))
+.filter(method -> {
+try {
+Object.class.getMethod(method.getName(),
method.getParameterTypes());
+return false;
+} catch (final NoSuchMethodException e) {
+return true;
+}
+})
+
[tomcat] 02/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit e1a53fb2d567b81cfaa4fcfa94ef870f15a6e289
Author: Mark Thomas
AuthorDate: Thu Sep 24 17:32:49 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Pull up re-prioritisation
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 71 ++
java/org/apache/coyote/http2/AbstractStream.java | 6 +-
.../apache/coyote/http2/Http2UpgradeHandler.java | 6 +-
java/org/apache/coyote/http2/RecycledStream.java | 10 +--
java/org/apache/coyote/http2/Stream.java | 55 -
5 files changed, 78 insertions(+), 70 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 63fb5e7..ce08cc5 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -16,6 +16,11 @@
*/
package org.apache.coyote.http2;
+import java.util.Iterator;
+
+import org.apache.juli.logging.Log;
+import org.apache.juli.logging.LogFactory;
+import org.apache.tomcat.util.res.StringManager;
/**
* Base class for all streams other than stream 0, the connection. Primarily
@@ -23,7 +28,73 @@ package org.apache.coyote.http2;
*/
abstract class AbstractNonZeroStream extends AbstractStream {
+private static final Log log =
LogFactory.getLog(AbstractNonZeroStream.class);
+private static final StringManager sm =
StringManager.getManager(AbstractNonZeroStream.class);
+
+private volatile int weight;
+
+
AbstractNonZeroStream(Integer identifier) {
+this(identifier, Constants.DEFAULT_WEIGHT);
+}
+
+
+AbstractNonZeroStream(Integer identifier, int weight) {
super(identifier);
+this.weight = weight;
+}
+
+
+@Override
+final int getWeight() {
+return weight;
}
+
+
+final void rePrioritise(AbstractStream parent, boolean exclusive, int
weight) {
+if (log.isDebugEnabled()) {
+log.debug(sm.getString("stream.reprioritisation.debug",
+getConnectionId(), getIdAsString(),
Boolean.toString(exclusive),
+parent.getIdAsString(), Integer.toString(weight)));
+}
+
+// Check if new parent is a descendant of this stream
+if (isDescendant(parent)) {
+parent.detachFromParent();
+// Cast is always safe since any descendant of this stream must be
+// an instance of Stream
+getParentStream().addChild((Stream) parent);
+}
+
+if (exclusive) {
+// Need to move children of the new parent to be children of this
+// stream. Slightly convoluted to avoid concurrent modification.
+Iterator parentsChildren =
parent.getChildStreams().iterator();
+while (parentsChildren.hasNext()) {
+AbstractNonZeroStream parentsChild = parentsChildren.next();
+parentsChildren.remove();
+this.addChild(parentsChild);
+}
+}
+detachFromParent();
+parent.addChild(this);
+this.weight = weight;
+}
+
+
+/*
+ * Used when removing closed streams from the tree and we know there is no
+ * need to check for circular references.
+ */
+final void rePrioritise(AbstractStream parent, int weight) {
+if (log.isDebugEnabled()) {
+log.debug(sm.getString("stream.reprioritisation.debug",
+getConnectionId(), getIdAsString(), Boolean.FALSE,
+parent.getIdAsString(), Integer.toString(weight)));
+}
+
+parent.addChild(this);
+this.weight = weight;
+}
+
}
diff --git a/java/org/apache/coyote/http2/AbstractStream.java
b/java/org/apache/coyote/http2/AbstractStream.java
index 21963ee..c7374b6 100644
--- a/java/org/apache/coyote/http2/AbstractStream.java
+++ b/java/org/apache/coyote/http2/AbstractStream.java
@@ -37,7 +37,7 @@ abstract class AbstractStream {
private final String idAsString;
private volatile AbstractStream parentStream = null;
-private final Set childStreams = Collections.newSetFromMap(new
ConcurrentHashMap<>());
+private final Set childStreams =
Collections.newSetFromMap(new ConcurrentHashMap<>());
private long windowSize =
ConnectionSettingsBase.DEFAULT_INITIAL_WINDOW_SIZE;
@@ -70,7 +70,7 @@ abstract class AbstractStream {
}
-final void addChild(Stream child) {
+final void addChild(AbstractNonZeroStream child) {
child.setParentStream(this);
childStreams.add(child);
}
@@ -97,7 +97,7 @@ abstract class AbstractStream {
}
-final Set
[tomcat] 10/10: Optimize the iteration when closing idle streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit f258efef6c1e9c71ceaf1f50e1dc354f2514ab3a
Author: Martin Tzvetanov Grigorov
AuthorDate: Mon Sep 28 14:53:25 2020 +0300
Optimize the iteration when closing idle streams
---
java/org/apache/coyote/http2/Http2UpgradeHandler.java | 16
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index 9f71af9..a1002c1 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -29,7 +29,7 @@ import java.util.Set;
import java.util.TreeSet;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
-import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentNavigableMap;
import java.util.concurrent.ConcurrentSkipListMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
@@ -124,7 +124,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
private HpackDecoder hpackDecoder;
private HpackEncoder hpackEncoder;
-private final ConcurrentMap streams = new
ConcurrentSkipListMap<>();
+private final ConcurrentNavigableMap
streams = new ConcurrentSkipListMap<>();
protected final AtomicInteger activeRemoteStreamCount = new
AtomicInteger(0);
// Start at -1 so the 'add 2' logic in closeIdleStreams() works
private volatile int maxActiveRemoteStreamId = -1;
@@ -1530,12 +1530,12 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
private void closeIdleStreams(int newMaxActiveRemoteStreamId) {
-for (Entry entry : streams.entrySet()) {
-int id = entry.getKey().intValue();
-if (id > maxActiveRemoteStreamId && id <
newMaxActiveRemoteStreamId) {
-if (entry.getValue() instanceof Stream) {
-((Stream) entry.getValue()).closeIfIdle();
-}
+final ConcurrentNavigableMap subMap =
streams.subMap(
+Integer.valueOf(maxActiveRemoteStreamId), false,
+Integer.valueOf(newMaxActiveRemoteStreamId), false);
+for (AbstractNonZeroStream stream : subMap.values()) {
+if (stream instanceof Stream) {
+((Stream)stream).closeIfIdle();
}
}
maxActiveRemoteStreamId = newMaxActiveRemoteStreamId;
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 06/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 2b2ac4f3ce99027d8589595e921570495f7b08c0
Author: Mark Thomas
AuthorDate: Thu Sep 24 21:47:09 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Pull up state
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 21 -
java/org/apache/coyote/http2/RecycledStream.java| 16 +---
java/org/apache/coyote/http2/Stream.java| 16 +---
3 files changed, 18 insertions(+), 35 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index 582ab1e..084de33 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -31,17 +31,22 @@ abstract class AbstractNonZeroStream extends AbstractStream
{
private static final Log log =
LogFactory.getLog(AbstractNonZeroStream.class);
private static final StringManager sm =
StringManager.getManager(AbstractNonZeroStream.class);
+protected final StreamStateMachine state;
+
private volatile int weight;
-AbstractNonZeroStream(Integer identifier) {
-this(identifier, Constants.DEFAULT_WEIGHT);
+AbstractNonZeroStream(String connectionId, Integer identifier) {
+super(identifier);
+this.weight = Constants.DEFAULT_WEIGHT;
+this.state = new StreamStateMachine(connectionId, getIdAsString());
}
-AbstractNonZeroStream(Integer identifier, int weight) {
+AbstractNonZeroStream(Integer identifier, int weight, StreamStateMachine
state) {
super(identifier);
this.weight = weight;
+this.state = state;
}
@@ -97,7 +102,13 @@ abstract class AbstractNonZeroStream extends AbstractStream
{
this.weight = weight;
}
-abstract boolean isClosedFinal();
-abstract void checkState(FrameType frameType) throws Http2Exception;
+final boolean isClosedFinal() {
+return state.isClosedFinal();
+}
+
+
+final void checkState(FrameType frameType) throws Http2Exception {
+state.checkFrameType(frameType);
+}
}
diff --git a/java/org/apache/coyote/http2/RecycledStream.java
b/java/org/apache/coyote/http2/RecycledStream.java
index 1915dff..9d6177c 100644
--- a/java/org/apache/coyote/http2/RecycledStream.java
+++ b/java/org/apache/coyote/http2/RecycledStream.java
@@ -23,12 +23,10 @@ package org.apache.coyote.http2;
class RecycledStream extends AbstractNonZeroStream {
private final String connectionId;
-private final StreamStateMachine state;
RecycledStream(String connectionId, Integer identifier, int weight,
StreamStateMachine state) {
-super(identifier, weight);
+super(identifier, weight, state);
this.connectionId = connectionId;
-this.state = state;
}
@@ -38,18 +36,6 @@ class RecycledStream extends AbstractNonZeroStream {
}
-@Override
-boolean isClosedFinal() {
-return state.isClosedFinal();
-}
-
-
-@Override
-final void checkState(FrameType frameType) throws Http2Exception {
-state.checkFrameType(frameType);
-}
-
-
@SuppressWarnings("sync-override")
@Override
void incrementWindowSize(int increment) throws Http2Exception {
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index f6210d7..140a76d 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -69,7 +69,6 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
private volatile long contentLengthReceived = 0;
private final Http2UpgradeHandler handler;
-private final StreamStateMachine state;
private final WindowAllocationManager allocationManager = new
WindowAllocationManager(this);
// State machine would be too much overhead
@@ -93,11 +92,10 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
Stream(Integer identifier, Http2UpgradeHandler handler, Request
coyoteRequest) {
-super(identifier);
+super(handler.getConnectionId(), identifier);
this.handler = handler;
handler.addChild(this);
setWindowSize(handler.getRemoteSettings().getInitialWindowSize());
-state = new StreamStateMachine(getConnectionId(), getIdAsString());
if (coyoteRequest == null) {
// HTTP/2 new request
this.coyoteRequest = new Request();
@@ -194,12 +192,6 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
@Override
-final void checkState(FrameType frameType) throws
[tomcat] 03/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 3783dc83065414cd85179e9a9014da44780e14b6
Author: Mark Thomas
AuthorDate: Thu Sep 24 17:50:43 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Pull-up isClosedFinal()
---
java/org/apache/coyote/http2/AbstractNonZeroStream.java | 1 +
java/org/apache/coyote/http2/RecycledStream.java| 8
java/org/apache/coyote/http2/Stream.java| 1 +
3 files changed, 10 insertions(+)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
index ce08cc5..70cfe35 100644
--- a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -97,4 +97,5 @@ abstract class AbstractNonZeroStream extends AbstractStream {
this.weight = weight;
}
+abstract boolean isClosedFinal();
}
diff --git a/java/org/apache/coyote/http2/RecycledStream.java
b/java/org/apache/coyote/http2/RecycledStream.java
index 1e630df..dbbdc10 100644
--- a/java/org/apache/coyote/http2/RecycledStream.java
+++ b/java/org/apache/coyote/http2/RecycledStream.java
@@ -23,10 +23,12 @@ package org.apache.coyote.http2;
class RecycledStream extends AbstractNonZeroStream {
private final String connectionId;
+private final boolean closedFinal;
RecycledStream(Stream stream) {
super(stream.getIdentifier(), stream.getWeight());
connectionId = stream.getConnectionId();
+closedFinal = stream.isClosedFinal();
}
@@ -34,4 +36,10 @@ class RecycledStream extends AbstractNonZeroStream {
String getConnectionId() {
return connectionId;
}
+
+
+@Override
+boolean isClosedFinal() {
+return closedFinal;
+}
}
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index ef7ed29..489ce53 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -646,6 +646,7 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
}
+@Override
final boolean isClosedFinal() {
return state.isClosedFinal();
}
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] branch 9.0.x updated (8dcf599 -> f258efe)
This is an automated email from the ASF dual-hosted git repository.
markt pushed a change to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git.
from 8dcf599 HTTP2 log statement fixes:
new 0d79ae5 Reduce memory footprint of closed http/2 streams
new e1a53fb Reduce memory footprint of closed http/2 streams
new 3783dc8 Reduce memory footprint of closed http/2 streams
new 496b3cf Reduce memory footprint of closed http/2 streams
new a6e8428 Reduce memory footprint of closed http/2 streams
new 2b2ac4f Reduce memory footprint of closed http/2 streams
new d1f204e Fully replace Stream with RecycledStream
new 83f5e03 Refactor the pruning so more stream info is retained for
longer
new a4054eb Update changelog
new f258efe Optimize the iteration when closing idle streams
The 10 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../apache/coyote/http2/AbstractNonZeroStream.java | 143 +++
java/org/apache/coyote/http2/AbstractStream.java | 9 +-
.../apache/coyote/http2/Http2UpgradeHandler.java | 264 +
...tionSettingsRemote.java => RecycledStream.java} | 23 +-
java/org/apache/coyote/http2/Stream.java | 90 +--
.../apache/coyote/http2/StreamStateMachine.java| 21 +-
webapps/docs/changelog.xml | 5 +
7 files changed, 348 insertions(+), 207 deletions(-)
create mode 100644 java/org/apache/coyote/http2/AbstractNonZeroStream.java
copy java/org/apache/coyote/http2/{ConnectionSettingsRemote.java =>
RecycledStream.java} (61%)
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 05/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit a6e84280ac145ab14f86a2cbe37238f9acda31d6
Author: Mark Thomas
AuthorDate: Thu Sep 24 21:36:19 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Refactor StreamStateMachine to remove reference to Stream
---
java/org/apache/coyote/http2/Stream.java| 2 +-
.../org/apache/coyote/http2/StreamStateMachine.java | 21 -
2 files changed, 13 insertions(+), 10 deletions(-)
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index 8aa4a7b..f6210d7 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -97,7 +97,7 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
this.handler = handler;
handler.addChild(this);
setWindowSize(handler.getRemoteSettings().getInitialWindowSize());
-state = new StreamStateMachine(this);
+state = new StreamStateMachine(getConnectionId(), getIdAsString());
if (coyoteRequest == null) {
// HTTP/2 new request
this.coyoteRequest = new Request();
diff --git a/java/org/apache/coyote/http2/StreamStateMachine.java
b/java/org/apache/coyote/http2/StreamStateMachine.java
index 851e1a8..72028fd 100644
--- a/java/org/apache/coyote/http2/StreamStateMachine.java
+++ b/java/org/apache/coyote/http2/StreamStateMachine.java
@@ -39,12 +39,15 @@ class StreamStateMachine {
private static final Log log = LogFactory.getLog(StreamStateMachine.class);
private static final StringManager sm =
StringManager.getManager(StreamStateMachine.class);
-private final Stream stream;
+private final String connectionId;
+private final String streamId;
+
private State state;
-StreamStateMachine(Stream stream) {
-this.stream = stream;
+StreamStateMachine(String connectionId, String streamId) {
+this.connectionId = connectionId;
+this.streamId = streamId;
stateChange(null, State.IDLE);
}
@@ -92,7 +95,7 @@ class StreamStateMachine {
public synchronized void sendReset() {
if (state == State.IDLE) {
throw new
IllegalStateException(sm.getString("streamStateMachine.debug.change",
-stream.getConnectionId(), stream.getIdAsString(), state));
+connectionId, streamId, state));
}
if (state.canReset()) {
stateChange(state, State.CLOSED_RST_TX);
@@ -109,8 +112,8 @@ class StreamStateMachine {
if (state == oldState) {
state = newState;
if (log.isDebugEnabled()) {
-log.debug(sm.getString("streamStateMachine.debug.change",
stream.getConnectionId(),
-stream.getIdAsString(), oldState, newState));
+log.debug(sm.getString("streamStateMachine.debug.change",
connectionId,
+streamId, oldState, newState));
}
}
}
@@ -122,12 +125,12 @@ class StreamStateMachine {
if (!isFrameTypePermitted(frameType)) {
if (state.connectionErrorForInvalidFrame) {
throw new
ConnectionException(sm.getString("streamStateMachine.invalidFrame",
-stream.getConnectionId(), stream.getIdAsString(),
state, frameType),
+connectionId, streamId, state, frameType),
state.errorCodeForInvalidFrame);
} else {
throw new
StreamException(sm.getString("streamStateMachine.invalidFrame",
-stream.getConnectionId(), stream.getIdAsString(),
state, frameType),
-state.errorCodeForInvalidFrame, stream.getIdAsInt());
+connectionId, streamId, state, frameType),
+state.errorCodeForInvalidFrame,
Integer.parseInt(streamId));
}
}
}
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] 01/10: Reduce memory footprint of closed http/2 streams
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 0d79ae569f3fd2646ff76d395f0f56e98b798a98
Author: Mark Thomas
AuthorDate: Wed Sep 23 19:59:13 2020 +0100
Reduce memory footprint of closed http/2 streams
This refactoring replaces closed streams with a new RecycledStream
object and changes the mechanism used to look up known streams.
Initial changes to object hierarchy in preparation for further
changes.
---
.../apache/coyote/http2/AbstractNonZeroStream.java | 29 ++
java/org/apache/coyote/http2/AbstractStream.java | 3 +-
java/org/apache/coyote/http2/RecycledStream.java | 45 ++
java/org/apache/coyote/http2/Stream.java | 2 +-
4 files changed, 77 insertions(+), 2 deletions(-)
diff --git a/java/org/apache/coyote/http2/AbstractNonZeroStream.java
b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
new file mode 100644
index 000..63fb5e7
--- /dev/null
+++ b/java/org/apache/coyote/http2/AbstractNonZeroStream.java
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.coyote.http2;
+
+
+/**
+ * Base class for all streams other than stream 0, the connection. Primarily
+ * provides functionality shared between full Stream and RecycledStream.
+ */
+abstract class AbstractNonZeroStream extends AbstractStream {
+
+AbstractNonZeroStream(Integer identifier) {
+super(identifier);
+}
+}
diff --git a/java/org/apache/coyote/http2/AbstractStream.java
b/java/org/apache/coyote/http2/AbstractStream.java
index 088d5b0..21963ee 100644
--- a/java/org/apache/coyote/http2/AbstractStream.java
+++ b/java/org/apache/coyote/http2/AbstractStream.java
@@ -25,7 +25,8 @@ import org.apache.juli.logging.LogFactory;
import org.apache.tomcat.util.res.StringManager;
/**
- * Used to managed prioritisation.
+ * Base class for all streams including the connection (referred to as Stream
0)
+ * and is used primarily when managing prioritization.
*/
abstract class AbstractStream {
diff --git a/java/org/apache/coyote/http2/RecycledStream.java
b/java/org/apache/coyote/http2/RecycledStream.java
new file mode 100644
index 000..f500646
--- /dev/null
+++ b/java/org/apache/coyote/http2/RecycledStream.java
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.coyote.http2;
+
+/**
+ * Represents a closed stream in the priority tree. Used in preference to the
+ * full {@link Stream} as has much lower memory usage.
+ */
+class RecycledStream extends AbstractNonZeroStream {
+
+private final String connectionId;
+private int weight;
+
+RecycledStream(Stream stream) {
+super(stream.getIdentifier());
+connectionId = stream.getConnectionId();
+weight = stream.getWeight();
+}
+
+
+@Override
+String getConnectionId() {
+return connectionId;
+}
+
+
+@Override
+int getWeight() {
+return weight;
+}
+}
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index af777ab..56bc129 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -46,7 +46,7 @@ import org.apache.tomcat.util.net.ApplicationBufferHandler;
import org.apache.tomcat.util.net.WriteBuffer;
import
