Author: rwesten
Date: Tue Jan 31 08:00:27 2012
New Revision: 1238222
URL: http://svn.apache.org/viewvc?rev=1238222&view=rev
Log:
updated execution metadata documetnation based on decisions in the upcomming
implementation
Modified:
incubator/stanbol/site/trunk/content/stanbol/docs/trunk/enhancer/executionmetadata.mdtext
Modified:
incubator/stanbol/site/trunk/content/stanbol/docs/trunk/enhancer/executionmetadata.mdtext
URL:
http://svn.apache.org/viewvc/incubator/stanbol/site/trunk/content/stanbol/docs/trunk/enhancer/executionmetadata.mdtext?rev=1238222&r1=1238221&r2=1238222&view=diff
==============================================================================
---
incubator/stanbol/site/trunk/content/stanbol/docs/trunk/enhancer/executionmetadata.mdtext
(original)
+++
incubator/stanbol/site/trunk/content/stanbol/docs/trunk/enhancer/executionmetadata.mdtext
Tue Jan 31 08:00:27 2012
@@ -1,10 +1,13 @@
Title: Execution Metadata
-The execution metadata are added by the
[EnhancementJobManager](enhancementjobmanager.html) to the metadata of the
[ContentItem](contentitem.html). This metadata provide information about the
execution of the [ExecutionPlan](chains/executionplan.html) provided by the
[Chain](chains/enhancementchain.html) and can be used by clients to get
detailed information about the enhancement process of a content item.
+The execution metadata provide detailed information about an ongoing/completed
enhancement process. Basically they describe how the
[ExecutionPlan](chains/executionplan.html) provided by the
[Chain](chains.enhancementchain.html) was executed by the
[EnhancementJobManager](enhancementjobmanager.html). Both the ExecutionMetadata
and the ExecutionPlan are provided with the ContentItem as an own content part
of the type MGraph with the URI
"http://stanbol.apache.org/ontology/enhancer/executionMetadata#ChainExecution";.
For users of the Stanbol Enhacer the Execution Metadata are of interest to:
-In the case of asynchronous calls to the enhancers RESTful interface (requests
that immediately return and do not wait for the enhancement process to
complete) this information might also be useful to provide information about
the current state of the enhancement process.
+* check progress of asynchronously started Enhancement Processes: Metadata for
all planed executions of engines are created as soon as an ContentItem is
parsed to the EnhancementJobManager and are updated as soon as the execution of
engines start/complete/fail.
+* Monitor the performance of different EnhancementEngines: The Execution
Metadata provide detailed information about starting/completion time points for
engine executions.
+* Inspect the Enhancement Process: Check if optional EnhancementEngines where
successfully executed or where skipped/failed; Validate the configured
EnhancementChain by checking the actual execution order of the
EnhancementEngines.
-### Exection Metadata Ontology
+
+## Exection Metadata Ontology
The RDFS schema used for the execution plan is defined as follows.
@@ -25,6 +28,7 @@ The RDFS schema used for the execution p
* __em:ExecutionStatus__ : Class describing the status of an EngineExecution
* __em:StatusSheduled__ : ExecutionStatis instance that described that an
execution is scheduled but has not yet started
* __em:StatusInProgress__ : ExecutuinStatus instance that describes that
the execution of the linked EngineExecution is in progress
+ * __em:StatusSkipped__ : ExecutionStatus instance describing that the
execution was skipped. This can only be the case for optional ep:ExecutionNodess
* __em:StatusCompleted__ : ExecutionStatus instance describing that the
execution has already completed successfully
* __em:StatusFailed__ : ExecutionStatus indicating that the execution has
failed. Typically a em:statusMessage describing the reason for the failed
execution is provided for em:Executions with that state.
* __em:StatusSkiped__ : ExecutionStatus indicating that the execution if
an sp:ExecutionNode was skipped. This is only allowed for execution nodes that
are marked as optional. Typically also a em:statusMessage with the reason
should be provided.
@@ -118,4 +122,124 @@ The Execution Plan: (copy from the examp
ep:engine zemanta
ep:optional "true"^^xsd:boolean
-Note that both the Execution Metadata AND the Execution Plan need to be
contained within the metadata of the [ContentItem](contentitem.html)
+
+## Creation/Management of ExecutionMetadata
+
+This section is primarily intended for implementors of EnhancementJobManager.
However it might also provide insights for users that want/need to monitor the
state of enhancement processes as it describes what information are added when
to the Execution Metadata.
+
+When the [EnhancementJobManager](enhancementjobmanager.html) starts the
Enhancement of an ContentItem he needs check if the
[ContentItem](contentitem.html) already contains ExecutionMetadata in the
ContentPart with the URI
"http://stanbol.apache.org/ontology/enhancer/executionMetadata#ChainExecution";.
If this is the case it needs to initialize itself based on the pre-existing
information. If no ExecutionMetadata are present a new EnhancementProcess needs
to be created based on the parsed Chain. Difreencies between this two cases are
explained in the rolling two sub sections
+
+### Initialization
+
+If no ExecutionMetadata are present within a parsed ContentItem a new
EnhancementProcess needs to be set up. This includes the following steps:
+
+1. Get the [ExecutionPlan](chains/executionplan.html) for the parsed
enhancement [Chain](chains/enhancementchain.html). If no chain is parsed the
default chain need to be acquired by using the
[ChainManager](chains/chainmanager.html).
+2. Create the content part for the ExecutionMetadata with the
[ContentItem](contentitem.html) and add the information of the
[ExecutionPlan](chains/executionplan.html) to it.
+3. Create the initial ExecutionMetadata. This includes the 'em:ChainExecution'
instance for the 'ep:ExecutionPlan' as well as 'em:EngineExecution' instances
for all 'ep:ExecutionNode's defined by the execution plan. All such
'em:Execution' instances MUST BE created with the 'em:ExecutionStatus'
'em:StatusSheduled'.
+
+The ExecutionMetadataHelper utility of the
"org.apache.stanbol.enhancer.servicesapi" module contains utility methods for
initializing execution metadata.
+
+### Continuation
+
+If the parsed ContentItem does already contain ExecutionMetadata in the
content part with the URI
"http://stanbol.apache.org/ontology/enhancer/executionMetadata#ChainExecution";
the EnhancementJobManager MUST follow the following steps to continue an
EnhancementProcess.
+
+1. Check if the contained ExecutionMetadata are valid
+ * If a 'em:ChainExecution' node is present that 'em:enhances' the parsed
ContentItem
+ * If the ExecutionPlan is included and if the value of the 'ep:chain'
property for the 'ep:ExecutionPlan' resource corresponds to the name of the
Chain parsed in the request.
+2. Check the status of all 'em:Execution' instances
+ * reset the status of 'em:Execution's that are in-progress to scheduled.
+ * TODO: here we could also retry the execution of failed 'em:Execution's
+
+Note that with an Continuation the ExecutionPlan MUST NOT be updated. It MUST
BE also NOT checked if a Chain with the name as stored in the ExecutionMetadata
is still present. Note also that configuration changes of EnhancementEngine
will affect the continuation of the enhancement process.
+
+The ExecutionMetadataHelper utility of the
"org.apache.stanbol.enhancer.servicesapi" module contains utility methods for
reading and validating pre-existing execution metadata.
+
+### Execution State Management
+
+The following metadata need to be updated by the EnhancementJobManager when:
+
+* Enhancement process starts
+ * set the 'em:status' of the 'em:ChainExecution' to 'em:StatusInProgress'
+ * set the 'em:started' to the current date time
+* EnhancementEngine execution starts:
+ * set the 'em:status' of the 'em:EngineExecution' to 'em:StatusInProgress'
+ * set the 'em:started' to the current date time
+* EnhancementEngine completes
+ * set the 'em:status' of the 'em:EngineExecution' to 'em:StatusCompleted'
+ * set the 'em:completed' to the current date time
+* Optional EnhancementEngine not available
+ * set the 'em:status' of the 'em:EngineExecution' to 'em:StatusSkipped'
+ * set both 'em:started' and 'em:completed' to the current date time
+* Optional EnhancementEngine failed
+ * set the 'em:status' of the 'em:EngineExecution' to 'em:StatusFailed'
+ * set the 'em:completed' to the current date time
+* Required EnhancementEngine failed or not available
+ * set the 'em:status' of the 'em:EngineExecution' to 'em:StatusFailed'
+ * set the 'em:status' of the 'em:ChainExecution' to 'em:StatusFailed'
+ * set the 'em:completed' of both the engine and the chain execution to the
current date time
+* Enhancement process completes
+ * set the 'em:status' of the 'em:ChainExecution' to 'em:StatusCompleted'
+ * set the 'em:completed' to the current date time
+* Internal error in the EnhancementJobManager implementation
+ * set the 'em:status' of the 'em:ChainExecution' to 'em:StatusFailed'
+ * do not set any 'em:EngineExecution' to failed.
+ * set the 'em:completed' value of the 'em:ChainExecution' to the current
date time
+
+The ExecutionMetadataHelper utility of the
"org.apache.stanbol.enhancer.servicesapi" module contains utility methods to
preform state transitions on 'em:Execution' instances.
+
+## Using ExecutionMetadata
+
+This Section provides some Examples on how to access and retrieve information
from the ExecutionMetadata.
+
+### Accessing ExecutionMetadata
+
+The ExecutionMetadata and the [ExecutionPlan](chains/executionplan.html) are
stored in a content part with with URI
"http://stanbol.apache.org/ontology/enhancer/executionMetadata#ChainExecution";
with the [ContentItem](contentitem.html). The following code segment can be
used to retrieve the RDF graph with the ExecutionMetadata:
+
+ ContentItem ci; //the ContentItem
+ //the URI is available as constant of the ExecutionMetadata class
+ UriRef contentPartURI = ExecutionMetadata.CHAIN_EXECUTION;
+
+ MGraph executionMetadata = ci.getPart(contentPartURI,MGraph.class);
+
+The ExecutionMetadata are stored as read-/writeable RDF graph. To parse an
read-only version to other components one can use the "getGraph()" method
defined by MGraph.
+
+### Getting details about the em:ChainExecution
+
+The following code segments show how to access information about the execution
of the enhancement process for a [ContentItem](contentitem.html). All directly
accessed Methods in the examples below are static imports from one of the
following two utility classes part of the
"org.apache.stanbol.enhancer.servicesapi" module.
+
+* ExecutionPlanHelper: Utility class that provides methods for reading and
creating [ExecutionPlan](chains/executionplan.html).
+* ExecutionMetadataHelper: Utility class for reading and manipulating the
ExecutionMetadata
+* EnhancementEngineHelper: Utility that contains general purpose RDF utilities.
+
+ ContentItem ci; //the ContentItem
+ MGraph em; //the ExecutionMetadata
+
+ //get the ChainExecution, ExecutionPlan and the name of the Chain
+ NonLiteral ce = getChainExecution(em,ci.getUri());
+ if(ce != null){
+ NonLiteral ep = getExecutionPlan(em,ce);
+ String chainName = getString(em,ep,ExecutionPlan.CHAIN);
+ } else {
+ log.warn("ExecutionMetadata of not contain information for "
+ + "ContentItem {}!",ci.getUri());
+ }
+
+ //get the EngineExecutions and the name of the Engines
+ Set<NonLiteral> executions = getExecutions(em,ce);
+ for(NonLiteral ex : executions){
+ NonLiteral en = getExecutionNode(em,ex);
+ if(en != null){
+ String engineName = getEngine(em,en);
+ boolean optional = isOptional(em,en);
+ } else { //maybe a sub-chain execution
+ //currently not supported, but might
+ //added in future versions
+ }
+ UriRef status = getStatus(em,ex);
+ Date started = getStarted(em,ex);
+ Date completed = getCompleted(em,ex);
+ }
+
+
+
+
