[
https://issues.apache.org/jira/browse/ODE-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695520#action_12695520
]
Karthick Sankarachary commented on ODE-561:
-------------------------------------------
Hi Jim,
The agents module got left out of the patch - sorry about that. You may want to
just update to the revision shown below. I couldn't check it in yesterday due
to technical difficulties, but its in now.
Revision: 761744
Author: karthick
Date: 11:06:37 AM, Friday, April 03, 2009
Message:
ODE-561 Improve Process Cache Design
----
Modified : /ode/branches/APACHE_ODE_1.X/Rakefile
Added : /ode/branches/APACHE_ODE_1.X/agents
Added : /ode/branches/APACHE_ODE_1.X/agents/src
Added : /ode/branches/APACHE_ODE_1.X/agents/src/main
Added : /ode/branches/APACHE_ODE_1.X/agents/src/main/java
Added : /ode/branches/APACHE_ODE_1.X/agents/src/main/java/org
Added : /ode/branches/APACHE_ODE_1.X/agents/src/main/java/org/apache
Added : /ode/branches/APACHE_ODE_1.X/agents/src/main/java/org/apache/ode
Added : /ode/branches/APACHE_ODE_1.X/agents/src/main/java/org/apache/ode/agents
Added :
/ode/branches/APACHE_ODE_1.X/agents/src/main/java/org/apache/ode/agents/memory
Added :
/ode/branches/APACHE_ODE_1.X/agents/src/main/java/org/apache/ode/agents/memory/SizingAgent.java
Added : /ode/branches/APACHE_ODE_1.X/agents/src/main/resources
Added : /ode/branches/APACHE_ODE_1.X/agents/src/main/resources/META-INF
Added :
/ode/branches/APACHE_ODE_1.X/agents/src/main/resources/META-INF/MANIFEST.MF
Modified :
/ode/branches/APACHE_ODE_1.X/axis2/src/main/java/org/apache/ode/axis2/BindingContextImpl.java
Modified :
/ode/branches/APACHE_ODE_1.X/axis2/src/main/java/org/apache/ode/axis2/ODEServer.java
Modified :
/ode/branches/APACHE_ODE_1.X/axis2/src/main/java/org/apache/ode/axis2/ODEService.java
Modified :
/ode/branches/APACHE_ODE_1.X/axis2/src/main/java/org/apache/ode/axis2/deploy/DeploymentPoller.java
Modified :
/ode/branches/APACHE_ODE_1.X/axis2/src/main/java/org/apache/ode/axis2/util/SoapMessageConverter.java
Modified :
/ode/branches/APACHE_ODE_1.X/axis2-war/src/test/java/org/apache/ode/axis2/JettyWrapper.java
Added :
/ode/branches/APACHE_ODE_1.X/axis2-war/src/test/java/org/apache/ode/axis2/hydration
Added :
/ode/branches/APACHE_ODE_1.X/axis2-war/src/test/java/org/apache/ode/axis2/hydration/InstanceCountTest.java
Added :
/ode/branches/APACHE_ODE_1.X/axis2-war/src/test/java/org/apache/ode/axis2/hydration/ProcessCountTest.java
Added :
/ode/branches/APACHE_ODE_1.X/axis2-war/src/test/java/org/apache/ode/axis2/hydration/ProcessSizeTest.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-api/src/main/java/org/apache/ode/bpel/iapi/BindingContext.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-api/src/main/java/org/apache/ode/bpel/iapi/BpelEngine.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-api/src/main/java/org/apache/ode/bpel/iapi/ProcessConf.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-compiler/src/main/java/org/apache/ode/bpel/compiler/BpelCompiler.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-epr/src/main/java/org/apache/ode/bpel/epr/EndpointFactory.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-epr/src/main/java/org/apache/ode/il/config/OdeConfigProperties.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-obj/src/main/java/org/apache/ode/bpel/o/OConstants.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/main/java/org/apache/ode/bpel/engine/BpelEngineImpl.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/main/java/org/apache/ode/bpel/engine/BpelProcess.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/main/java/org/apache/ode/bpel/engine/BpelRuntimeContextImpl.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/main/java/org/apache/ode/bpel/engine/BpelServerImpl.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/main/java/org/apache/ode/bpel/engine/Contexts.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/main/java/org/apache/ode/bpel/engine/InterceptorContextImpl.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/main/java/org/apache/ode/bpel/engine/MessageExchangeImpl.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/main/java/org/apache/ode/bpel/engine/MyRoleMessageExchangeImpl.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/main/java/org/apache/ode/bpel/engine/NStateLatch.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/main/java/org/apache/ode/bpel/engine/PartnerLinkMyRoleImpl.java
Added :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/main/java/org/apache/ode/bpel/intercept/InstanceCountThrottler.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/main/java/org/apache/ode/bpel/intercept/InterceptorInvoker.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/main/java/org/apache/ode/bpel/intercept/MessageExchangeInterceptor.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/main/java/org/apache/ode/bpel/intercept/NoOpInterceptor.java
Added :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/main/java/org/apache/ode/bpel/intercept/ProcessCountThrottler.java
Added :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/main/java/org/apache/ode/bpel/intercept/ProcessSizeThrottler.java
Deleted :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/main/java/org/apache/ode/bpel/intercept/ThrottlingInterceptor.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/main/java/org/apache/ode/bpel/runtime/InvalidProcessException.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-runtime/src/test/java/org/apache/ode/bpel/runtime/MockBpelServer.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-store/src/main/java/org/apache/ode/store/ProcessConfImpl.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-test/src/main/java/org/apache/ode/test/BindingContextImpl.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-test/src/main/java/org/apache/ode/test/MessageExchangeContextImpl.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-test/src/test/java/org/apache/ode/test/BasicActivities20Test.java
Modified :
/ode/branches/APACHE_ODE_1.X/bpel-test/src/test/java/org/apache/ode/test/StructuredActivities20Test.java
Modified :
/ode/branches/APACHE_ODE_1.X/dao-jpa/src/main/java/org/apache/ode/dao/jpa/ActivityRecoveryDAOImpl.java
Modified :
/ode/branches/APACHE_ODE_1.X/dao-jpa/src/main/java/org/apache/ode/dao/jpa/ProcessDAOImpl.java
Modified :
/ode/branches/APACHE_ODE_1.X/dao-jpa/src/main/java/org/apache/ode/dao/jpa/ProcessInstanceDAOImpl.java
Modified :
/ode/branches/APACHE_ODE_1.X/jbi/src/main/java/org/apache/ode/jbi/BindingContextImpl.java
Modified :
/ode/branches/APACHE_ODE_1.X/jbi/src/main/java/org/apache/ode/jbi/OdeContext.java
> Process hydration/dehydration improvements to better control memory footprint
> -----------------------------------------------------------------------------
>
> Key: ODE-561
> URL: https://issues.apache.org/jira/browse/ODE-561
> Project: ODE
> Issue Type: Improvement
> Components: Axis2 Integration, BPEL Compilation/Parsing, BPEL
> Runtime, JBI Integration
> Affects Versions: 1.2
> Reporter: Karthick Sankarachary
> Assignee: Karthick Sankarachary
> Fix For: 1.3
>
> Attachments: bounded-process-cache.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> At present, the BPEL server stashes the metadata of deployed processes in an
> in-memory, unbounded cache. Given the unbounded nature of the cache, the
> server is bound to eventually run out of memory when presented with a
> sufficiently large process set. The graceful way to handle this situation
> would be to place bounds on the cache, either in terms of the total size or
> number of processes that it may contain at any given point in time. While
> throttling the server this way may effectively reduce its throughput, it is
> certainly the far lesser evil compared to the alternative.
> To that end, we define the following configurable properties for the benefit
> of the administrator:
> a) "process.hydration.throttled.maximum.size": The value of this property
> specifies the maximum size of the metadata of all processes that are
> currently in-use and may be cached in-memory. If the process metadata was
> hydrated at least once by the server instance, then its size is calculated
> once by traversing its object model. If not, then we estimate its in-memory
> size based on the size of the (.cbp) file where its serialized bytes are
> persisted.
> b) "process.hydration.throttled.maximum.count": The value of this property
> specifies the maximum number of processes that are currently in-use and whose
> metadata may be cached in-memory.
> A process that is stored on disk but is not loaded in-memory is said to be
> "unhydrated". When the server receives a message that is targeted at an
> unhydrated process, we must decide whether or not there is sufficient free
> memory in the cache for it. If not, then we select the least recently used
> process as our victim and evict (or dehydrate) it. This check and balance is
> performed until there is sufficient memory, in which case we may hydrate the
> process. On the other hand, if the cache capacity is exceeded, then we
> process the message based on its exchange pattern, as described below:
> i) If the message is one-way (asynchronous), then we queue it back with the
> job scheduler so that it can be retried later. The hope is that, given a
> sufficient amount of delay and number of retries, the cache will have enough
> room for the targeted process.
> ii) If the message is two-way (synchronous), then we cannot simply
> re-schedule it back, because the client will no doubt timeout. The logical
> thing to do here is to just return a SOAP fault message that forces the
> client to handle retries itself, if it so desires.
> Furthermore, the administrator may use the following properties to enable and
> control lazy-loading:
> c) "process.hydration.lazy": The value of this property specifies whether or
> not a process metadata is to be lazy-loaded. If so, then the server will not
> immediately load process metadata upon startup or deployment. In fact, the
> process is not hydrated until the server receives a message that demands that
> that process be loaded in-memory. This is a global property that may be
> overriden on a process-by-process basis by specifying a value for its
> namesake in the process' deployment descriptor (deploy.xml) file.
> d) "process.hydration.lazy.minimum.size": The value of this property
> specifies the approximate minimum size of the process metadata for which
> lazy-loading should be enabled. If the server calculates the approximate size
> of the process metadata (based on its serialized file size) to be less than
> the value specified herein, then it will load the process eagerly. This way,
> you don't suffer the time delay on hydration for messages that don't deal
> with large-sized processes.
> Lastly, we introduce the following property to throttle the number of
> instances that a process may handle:
> e) "process.instance.throttled.maximum.count": The value of this property
> specifies the maximum number of instances that may be simultaneously active
> for any given process. This is a global property that may be overriden on a
> process-by-process basis by specifying a value for its namesake in the
> process' deployment descriptor (deploy.xml) file. Note that we assume that
> the process is already hydrated at the time we perform this check and as such
> this property is not directly related to caching. However, it may have an
> indirect effect on the cache by virtue of the fact that the it will most
> likely cause the process to dehydrate sooner rather than later (through
> dehydration policies).
> PS: Note that the the in-memory representation of process metadata is made up
> of WSDL and BPEL object models. The cache that we refer to here deals
> specifically with BPEL objects. For the purposes of this issue, caching of
> WSDL objects is considered to be out of scope.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
