TINKERPOP-1612 Re-write server docs given new plugin model.
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/979458b6 Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/979458b6 Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/979458b6 Branch: refs/heads/TINKERPOP-1612 Commit: 979458b6da0bd1c12add6af5a0f28613611d4e10 Parents: 692f09c Author: Stephen Mallette <[email protected]> Authored: Mon Feb 27 19:20:42 2017 -0500 Committer: Stephen Mallette <[email protected]> Committed: Tue Feb 28 06:32:40 2017 -0500 ---------------------------------------------------------------------- .../src/reference/gremlin-applications.asciidoc | 90 ++++++++++---------- 1 file changed, 43 insertions(+), 47 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/979458b6/docs/src/reference/gremlin-applications.asciidoc ---------------------------------------------------------------------- diff --git a/docs/src/reference/gremlin-applications.asciidoc b/docs/src/reference/gremlin-applications.asciidoc index 8270087..5d543eb 100644 --- a/docs/src/reference/gremlin-applications.asciidoc +++ b/docs/src/reference/gremlin-applications.asciidoc @@ -406,12 +406,9 @@ $ bin/gremlin-server.sh conf/gremlin-server-modern.yaml [INFO] MetricManager - Configured Metrics Slf4jReporter configured with interval=180000ms and loggerName=org.apache.tinkerpop.gremlin.server.Settings$Slf4jReporterMetrics [INFO] GraphManager - Graph [graph] was successfully configured via [conf/tinkergraph-empty.properties]. [INFO] ServerGremlinExecutor - Initialized Gremlin thread pool. Threads in pool named with pattern gremlin-* -[INFO] ScriptEngines - Loaded gremlin-groovy ScriptEngine -[INFO] GremlinExecutor - Initialized gremlin-groovy ScriptEngine with scripts/generate-modern.groovy [INFO] ServerGremlinExecutor - Initialized GremlinExecutor and configured ScriptEngines. [INFO] ServerGremlinExecutor - A GraphTraversalSource is now bound to [g] with graphtraversalsource[tinkergraph[vertices:0 edges:0], standard] [INFO] OpLoader - Adding the standard OpProcessor. -[INFO] OpLoader - Adding the control OpProcessor. [INFO] OpLoader - Adding the session OpProcessor. [INFO] OpLoader - Adding the traversal OpProcessor. [INFO] TraversalOpProcessor - Initialized cache for TraversalOpProcessor with size 1000 and expiration time of 600000 ms @@ -420,7 +417,11 @@ $ bin/gremlin-server.sh conf/gremlin-server-modern.yaml [INFO] AbstractChannelizer - Configured application/vnd.gremlin-v1.0+gryo with org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0 [INFO] AbstractChannelizer - Configured application/vnd.gremlin-v1.0+gryo-lite with org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0 [INFO] AbstractChannelizer - Configured application/vnd.gremlin-v1.0+gryo-stringd with org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0 -[INFO] GremlinServer$1 - Gremlin Server configured with worker thread pool of 1, gremlin pool of 8 and boss thread pool of 1. +[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v1.0+json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0 +[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v2.0+json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0 +[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v3.0+json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0 +[INFO] AbstractChannelizer - Configured application/json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0 +[INFO] GremlinServer$1 - Gremlin Server configured with worker thread pool of 1, gremlin pool of 4 and boss thread pool of 1. [INFO] GremlinServer$1 - Channel started at port 8182. ---- @@ -1067,7 +1068,6 @@ The following table describes the various YAML configuration options that Gremli |metrics.jmxReporter.enabled |Turns on JMX reporting of metrics. |false |metrics.slf4jReporter.enabled |Turns on SLF4j reporting of metrics. |false |metrics.slf4jReporter.interval |Time in milliseconds between reports of metrics to SLF4j. |180000 -|plugins |A list of plugins that should be activated on server startup in the available script engines. It assumes that the plugins are in Gremlin Server's classpath. |_none_ |port |The port to bind the server to. |8182 |processors |A `List` of `Map` settings, where each `Map` represents a `OpProcessor` implementation to use along with its configuration. |_none_ |processors[X].className |The full class name of the `OpProcessor` implementation. |_none_ @@ -1330,7 +1330,7 @@ export JAVA_OPTIONS="$JAVA_OPTIONS -Djava.security.auth.login.config=conf/gremli Protecting Script Execution +++++++++++++++++++++++++++ -It is important to remember that Gremlin Server exposes a `ScriptEngine` instance that allows for remote execution +It is important to remember that Gremlin Server exposes `GremlinScriptEngine` instances that allows for remote execution of arbitrary code on the server. Obviously, this situation can represent a security risk or, more minimally, provide ways for "bad" scripts to be inadvertently executed. A simple example of a "valid" Gremlin script that would cause some problems would be, `while(true) {}`, which would consume a thread in the Gremlin pool indefinitely, thus @@ -1338,30 +1338,33 @@ preventing it from serving other requests. Sending enough of these kinds of scr available threads and Gremlin Server would stop responding. Gremlin Server (more specifically the `GremlinGroovyScriptEngine`) provides methods to protect itself from these -kinds of troublesome scripts. A user can configure the script engine with different `CompilerCustomizerProvider` -implementations. Consider the basic configuration from the Gremlin Server YAML file: +kinds of troublesome scripts. A user can configure the script engine with a `GroovyCompilerGremlinPlugin` +implementation. Consider the basic configuration from the Gremlin Server YAML file: [source,yaml] +---- scriptEngines: { gremlin-groovy: { - imports: [java.lang.Math], - staticImports: [java.lang.Math.PI], - scripts: [scripts/empty-sample.groovy]}} + plugins: { org.apache.tinkerpop.gremlin.server.jsr223.GremlinServerGremlinPlugin: {}, + org.apache.tinkerpop.gremlin.tinkergraph.jsr223.TinkerGraphGremlinPlugin: {}, + org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {classImports: [java.lang.Math], methodImports: [java.lang.Math#*]}, + org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: [scripts/empty-sample.groovy]}}}} +---- -This configuration can be extended to include a `config` key as follows: +This configuration can be expaded to include a the `GroovyCompilerGremlinPlugin`: [source,yaml] +---- scriptEngines: { gremlin-groovy: { - imports: [java.lang.Math], - staticImports: [java.lang.Math.PI], - scripts: [scripts/empty-sample.groovy], - config: { - compilerCustomizerProviders: { - "org.apache.tinkerpop.gremlin.groovy.jsr223.customizer.TimedInterruptCustomizerProvider":[10000] }}} - -This configuration sets up the script engine with a `CompilerCustomizerProvider` implementation. The -`TimedInterruptCustomizerProvider` injects checks that ensure that loops (like `while`) can only execute for `10000` + plugins: { org.apache.tinkerpop.gremlin.server.jsr223.GremlinServerGremlinPlugin: {}, + org.apache.tinkerpop.gremlin.tinkergraph.jsr223.TinkerGraphGremlinPlugin: {} + org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {classImports: [java.lang.Math], methodImports: [java.lang.Math#*]}, + org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: [scripts/empty-sample-secure.groovy]}, + org.apache.tinkerpop.gremlin.groovy.jsr223.GroovyCompilerGremlinPlugin: {timedInterrupt: 10000}}}} +---- + +This configuration sets up the script engine with to ensure that loops (like `while`) can only execute for `10000` milliseconds. With this configuration in place, a remote execution as follows, now times out rather than consuming the thread continuously: @@ -1371,41 +1374,34 @@ gremlin> :remote connect tinkerpop.server conf/remote.yaml gremlin> :> while(true) { } Execution timed out after 10000 units. Start time: Fri Jul 24 11:04:52 EDT 2015 -There are a number of pre-packaged `CustomizerProvider` implementations: +The `GroovyCompilerGremlinPlugin` has a number of configuration options: [width="100%",cols="3,10a",options="header"] |========================================================= |Customizer |Description -|`CompileStaticCustomizerProvider` |Applies `CompileStatic` annotations to incoming scripts thus removing dynamic dispatch. More information about static compilation can be found in the link:http://docs.groovy-lang.org/latest/html/documentation/#_static_compilation[Groovy Documentation]. It is possible to configure this `CustomizerProvider` by specifying a comma separated list of link:http://docs.groovy-lang.org/latest/html/documentation/#Typecheckingextensions-Workingwithextensions[type checking extensions] that can have the effect of securing calls to various methods. -|`ConfigurationCustomizerProvider` |Allows configuration of the Groovy `CompilerConfiguration` object by taking a `Map` of key/value pairs where the "key" is a property to set on the `CompilerConfiguration`. -|`ThreadInterruptCustomizerProvider` |Injects checks for thread interruption, thus allowing the thread to potentially respect calls to `Thread.interrupt()` -|`TimedInterruptCustomizerProvider` |Injects checks into loops to interrupt them if they exceed the configured timeout in milliseconds. -|`TypeCheckedCustomizerProvider` |Similar to the above mentioned, `CompileStaticCustomizerProvider`, the `TypeCheckedCustomizerProvider` injects `TypeChecked` annotations to incoming scripts. More information on the nature of this annotation can be found in the link:http://docs.groovy-lang.org/latest/html/documentation/#_the_code_typechecked_code_annotation[Groovy Documentation]. It too takes a comma separated list of link:http://docs.groovy-lang.org/latest/html/documentation/#Typecheckingextensions-Workingwithextensions[type checking extensions]. +|`compilation` |Allows for three configurations: `COMPILE_STATIC`, `TYPE_CHECKED` or `NONE` (default). When configured with `COMPILE_STATIC` or `TYPE_CHECKED` it applies `CompileStatic` or `TypeChecked` annotations (respectively) to incoming scripts thus removing dynamic dispatch. More information about static compilation can be found link:http://docs.groovy-lang.org/latest/html/documentation/#_static_compilation[here] and additional information on `TypeChecked` usage can be found link:http://docs.groovy-lang.org/latest/html/documentation/#_the_code_typechecked_code_annotation[here]. +|`compilerConfigurationOptions` |Allows configuration of the Groovy `CompilerConfiguration` object by taking a `Map` of key/value pairs where the "key" is a property to set on the `CompilerConfiguration`. +|`enableThreadInterrupt` |Injects checks for thread interruption, thus allowing the thread to potentially respect calls to `Thread.interrupt()` +|`extensions` | This setting is for use when `compilation` is configured with `COMPILE_STATIC` or `TYPE_CHECKED` and accepts a comma separated list of link:http://docs.groovy-lang.org/latest/html/documentation/#Typecheckingextensions-Workingwithextensions[type checking extensions] that can have the effect of securing calls to various methods. +|`timedInterrupt` |Injects checks into loops to interrupt them if they exceed the configured timeout in milliseconds. |========================================================= +NOTE: Consult the latest link:http://docs.groovy-lang.org/latest/html/documentation/#_typing[Groovy Documentation] +for information on the differences. It is important to understand the impact that these configuration will have on +submitted scripts before enabling this feature. + To provide some basic out-of-the-box protections against troublesome scripts, the following configuration can be used: [source,yaml] +---- scriptEngines: { gremlin-groovy: { - imports: [java.lang.Math], - staticImports: [java.lang.Math.PI], - scripts: [scripts/empty-sample.groovy], - config: { - compilerCustomizerProviders: { - "org.apache.tinkerpop.gremlin.groovy.jsr223.customizer.ThreadInterruptCustomizerProvider":[], - "org.apache.tinkerpop.gremlin.groovy.jsr223.customizer.TimedInterruptCustomizerProvider":[10000], - "org.apache.tinkerpop.gremlin.groovy.jsr223.customizer.CompileStaticCustomizerProvider":["org.apache.tinkerpop.gremlin.groovy.jsr223.customizer.SimpleSandboxExtension"]}}}} - -NOTE: The above configuration could also use the `TypeCheckedCustomizerProvider` in place of the -`CompileStaticCustomizerProvider`. The differences between `TypeChecked` and `CompileStatic` are beyond the scope of -this documentation. Consult the latest link:http://docs.groovy-lang.org/latest/html/documentation/#_typing[Groovy Documentation] -for information on the differences. It is important to understand the impact that these configuration will have on -submitted scripts before enabling this feature. - -NOTE: The import of classes to the script engine is handled by the `ImportCustomizerProvider`. As the concept of -"imports" is a first-class citizen (i.e. has its own configuration options), it is not recommended that the -`ImportCustomizerProvider` be used as a configuration option to `compilerCustomizerProviders`. + plugins: { org.apache.tinkerpop.gremlin.server.jsr223.GremlinServerGremlinPlugin: {}, + org.apache.tinkerpop.gremlin.tinkergraph.jsr223.TinkerGraphGremlinPlugin: {} + org.apache.tinkerpop.gremlin.groovy.jsr223.GroovyCompilerGremlinPlugin: {enableThreadInterrupt: true, timedInterrupt: 10000, compilation: COMPILE_STATIC, extensions: org.apache.tinkerpop.gremlin.groovy.jsr223.customizer.SimpleSandboxExtension}, + org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {classImports: [java.lang.Math], methodImports: [java.lang.Math#*]}, + org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: [scripts/empty-sample-secure.groovy]}}}} +---- This configuration uses the `SimpleSandboxExtension`, which blacklists calls to methods on the `System` class, thereby preventing someone from remotely killing the server: @@ -1487,8 +1483,8 @@ The `FileSandboxExtension` provides for a basic configurable security function i sandboxing implementations can be developed by using this white listing model and extending from the `AbstractSandboxExtension`. -A final thought on the topic of `CompilerCustomizerProvider` implementations is that they are not just for -"security" (though they are demonstrated in that capacity here). They can be used for a variety of features that +A final thought on the topic of `GroovyCompilerGremlinPlugin` implementation is that it is not just for +"security" (though is is demonstrated in that capacity here). It can be used for a variety of features that can fine tune the Groovy compilation process. Read more about compilation customization in the link:http://docs.groovy-lang.org/latest/html/documentation/#compilation-customizers[Groovy Documentation].
