http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/main/resources/docs/org.apache.nifi.processors.script.ExecuteScript/index.html ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/main/resources/docs/org.apache.nifi.processors.script.ExecuteScript/index.html b/nar-bundles/execute-script-bundle/execute-script-processors/src/main/resources/docs/org.apache.nifi.processors.script.ExecuteScript/index.html new file mode 100644 index 0000000..acb47c5 --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/main/resources/docs/org.apache.nifi.processors.script.ExecuteScript/index.html @@ -0,0 +1,264 @@ +<!DOCTYPE html> +<html lang="en"> + <!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + <head> + <meta charset="utf-8" /> + <title>ExecuteScript</title> + + <link rel="stylesheet" href="../../css/component-usage.css" type="text/css" /> + </head> + + <body> + <!-- Processor Documentation ================================================== --> + <h2>Description:</h2> + <p> + This processor provides the capability to execute scripts in various scripting languages, and passes into the scripts + the input stream and output stream(s) representing an incoming flow file and any created flow files. The processor is designed to be + thread safe, so multiple concurrent tasks may execute against a single script. The processor provides a framework which enables + script writers to implement 3 different types of scripts: + <ul> + ReaderScript - which enables stream-based reading of a FlowFile's content</br> + WriterScript - which enables stream-based reading and writing/modifying of a FlowFile's content</br> + ConverterScript - which enables stream-based reading a FlowFile's content and stream-based writing to newly created FlowFiles</br> + </ul> + Presently, the processor supports 3 scripting languages: Ruby, Python, and JavaScript. The processor is built on the + javax.script API which enables ScriptEngine discovery, thread management, and encapsulates much of the low level bridging-code that + enables Java to Script language integration. Thus, it is designed to be easily extended to other scripting languages. </br> + The attributes of a FlowFile and properties of the Processor are exposed to the script by either a variable in the base class or + a getter method. A script may declare new Processor Properties and different Relationships via overriding the getPropertyDescriptors + and getRelationships methods, respectively. + </p> + The processor provides some boilerplate script to aid in the creation of the three different types of scripts. For example, + the processor provides import statements for classes commonly used within a processor. + <pre> + 'org.apache.nifi.components.PropertyDescriptor' + 'org.apache.nifi.components.Validator' + 'org.apache.nifi.processor.util.StandardValidators' + 'org.apache.nifi.processor.Relationship' + 'org.apache.nifi.logging.ProcessorLog' + 'org.apache.nifi.scripting.ReaderScript' + 'org.apache.nifi.scripting.WriterScript' + 'org.apache.nifi.scripting.ConverterScript' + </pre> + The processor appends to the script's execution path the parent directory of the specified script file and a sub-directory + called 'lib', which may be useful for supporting scripts. </p> +<p> + <strong>Shared Variables</strong> +</p> +The following variables are provided as shared variables for the scripts: +<ul> + <li>logger + <ul> + <li> The processor's logger </li> + <li> Scope is GLOBAL, thus in Ruby the syntax is $logger</li> + </ul> + </li> + <li>properties + <ul> + <li> A Map of the processor's configuration properties; key and value are strings</li> + <li> Scope is INSTANCE, thus in Ruby the syntax is @properties</li> + </ul> + </li> +</ul> +<p> + <strong>Properties:</strong> +</p> +<p> + In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered + optional. If a property has a default value, it is indicated. If a property supports the use of the NiFi Expression Language + (or simply, "expression language"), that is also indicated. Of particular note: This processor allows scripts to define additional + Processor properties, which will not be initially visible. Once the processor's configuration is validated, script defined properties + will become visible, and may affect the validity of the processor. +</p> +<ul> + <li> + <strong>Script File Name</strong> + <ul> + <li>Script location, can be relative or absolute path.</li> + <li>Default value: no default</li> + <li>Supports expression language: false</li> + </ul> + </li> + <li> + <strong>Script Check Interval</strong> + <ul> + <li>The time period between checking for updates to a script.</li> + <li>Default value: 15 sec</li> + <li>Supports expression language: false</li> + </ul> + </li> +</ul> + +<p> + <strong>Relationships:</strong> +</p> +<p> + The initial 'out of the box' relationships are below. Of particular note is the ability of a script to change the set of + relationships. However, any relationships defined by the script will not be visible until the processor's configuration has been + validated. Once done, new relationships will become visible. +</p> +<ul> + <li> + success + <ul> + <li>Used when a file is successfully processed by a script.</li> + </ul> + </li> + <li> + failure + <ul> + <li>Used when an error occurs while processing a file with a script.</li> + </ul> + </li> +</ul> + +<p> + <strong>Example Scripts:</strong> +</p> +<ul> + JavaScript example - the 'with' statement imports packages defined in the framework and limits the importing to the local scope, + rather than global. The 'Scripting' variable uses the JavaImporter class within JavaScript. Since the 'instance' variable is intended to + be local scope (not global), it must be named 'instance' as it it not passed back to the processor upon script evaluation and must be + fetched. If you make it global, you can name it whatever you'd like...but this is intended to be multi-threaded so do so at your own + risk.</p> +Presently, there are issues with the JavaScript scripting engine that prevent sub-classing the base classes in the Processor's Java +framework. So, what is actually happening is an instance of the ReaderScript is created with a provided callback object. When we are able +to move to a more competent scripting engine (supposedly in Java 8), the code below will remain the same, but the 'instance' variable +will actually be a sub-class of ReaderScript. +<pre> + with (Scripting) { + var instance = new ReaderScript({ + route : function(input) { + var str = IOUtils.toString(input); + var expr = instance.getProperty("expr"); + filename = instance.attributes.get("filename"); + instance.setAttribute("filename", filename + ".modified"); + if (str.match(expr)) { + return Script.FAIL_RELATIONSHIP; + } else { + return Script.SUCCESS_RELATIONSHIP; + } + } + }); + } +</pre> +Ruby example - the 'OutputStreamHandler' is an interface which is called when creating flow files. +<pre> + java_import 'org.apache.nifi.scripting.OutputStreamHandler' + class SimpleConverter < ConverterScript + field_reader :FAIL_RELATIONSHIP, :SUCCESS_RELATIONSHIP, :logger, :attributes + + def convert(input) + in_io = input.to_io + createFlowFile("firstLine", FAIL_RELATIONSHIP, OutputStreamHandler.impl do |method, out| + out_io = out.to_io + out_io << in_io.readline.to_java_bytes + out_io.close + logger.debug("Wrote data to failure...this message logged with logger from super class") + end) + + createFlowFile("otherLines", SUCCESS_RELATIONSHIP, OutputStreamHandler.impl do |method, out| + out_io = out.to_io + in_io.each_line { |line| + out_io << line + } + out_io.close + logger.debug("Wrote data to success...this message logged with logger from super class") + end) + in_io.close + end + + end + + $logger.debug("Creating SimpleConverter...this message logged with logger from shared variables") + SimpleConverter.new +</pre> +Python example - The difficulty with Python is that it does not return objects upon script evaluation, so the instance of the Script +class must be fetched by name. Thus, you must define a variable called 'instance'. +<pre> + import re + + class RoutingReader(ReaderScript): + A = Relationship.Builder().name("a").description("some good stuff").build() + B = Relationship.Builder().name("b").description("some other stuff").build() + C = Relationship.Builder().name("c").description("some bad stuff").build() + + def getRelationships(self): + return [self.A,self.B,self.C] + + def getExceptionRoute(self): + return self.C + + def route( self, input ): + logger.info("Executing route") + for line in FileUtil.wrap(input): + if re.match("^bad", line, re.IGNORECASE): + return self.B + if re.match("^sed", line): + raise RuntimeError("That's no good!") + + return self.A + logger.debug("Constructing instance") + instance = RoutingReader() + +</pre> +</ul> +<p> + <strong>Script API:</strong> +</p> +<ul> + <li>getAttribute(String) : String</li> + <li>getAttributes() : Map(String,String)</li> + <li>getExceptionRoute() : Relationship</li> + <li>getFileName() : String</li> + <li>getFlowFileEntryDate() : Calendar</li> + <li>getFlowFileSize() : long</li> + <li>getProperties() : Map(String, String)</li> + <li>getProperty(String) : String</li> + <li>getPropertyDescriptors() : List(PropertyDescriptor)</li> + <li>getRelationships() : Collection(Relationship)</li> + <li>getRoute() : Relationship</li> + <li>setRoute(Relationship)</li> + <li>setAttribute(String, String)</li> + <li>validate() : Collection(String)</li> +</ul> +<p> + <strong>ReaderScript API:</strong> +</p> +<ul> + <li>route(InputStream) : Relationship</li> +</ul> +<p> + <strong>WriterScript API:</strong> +</p> +<ul> + <li>process(InputStream, OutputStream)</li> +</ul> +<p> + <strong>ConverterScript API:</strong> +</p> +<ul> + <li>convert(InputStream)</li> + <li>createFlowFile(String, Relationship, OutputStreamHandler)</li> +</ul> +<p> + <strong>OutputStreamHandler API:</strong> +</p> +<ul> + <li>write(OutputStream)</li> +</ul> +</body> +</html>
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/java/org/apache/nifi/processors/script/TestExecuteScript.java ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/java/org/apache/nifi/processors/script/TestExecuteScript.java b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/java/org/apache/nifi/processors/script/TestExecuteScript.java new file mode 100644 index 0000000..3a34769 --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/java/org/apache/nifi/processors/script/TestExecuteScript.java @@ -0,0 +1,939 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.processors.script; + +import org.apache.nifi.processors.script.ExecuteScript; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +import java.io.File; +import java.io.IOException; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Set; + +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.util.MockFlowFile; +import org.apache.nifi.util.TestRunner; +import org.apache.nifi.util.TestRunners; + +import org.apache.commons.io.FileUtils; +import org.junit.Before; +import org.junit.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * @author unattributed + * + */ +public class TestExecuteScript { + + static Logger LOG; + + static { + System.setProperty("org.slf4j.simpleLogger.defaultLogLevel", "info"); + System.setProperty("org.slf4j.simpleLogger.showDateTime", "true"); + System.setProperty("org.slf4j.simpleLogger.log.nifi.processors.script.ExecuteScript", "trace"); + System.setProperty("org.slf4j.simpleLogger.log.nifi.processors.script.TestExecuteScript", "debug"); + System.setProperty("org.slf4j.simpleLogger.log.nifi.processors.AbstractProcessor", "debug"); + LOG = LoggerFactory.getLogger(TestExecuteScript.class); + } + + private TestRunner controller; + + private final String multiline = "Lorem ipsum dolor sit amet,\n" + + "consectetur adipisicing elit,\n" + + "sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.\n" + + "Ut enim ad minim veniam,\n" + + "quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.\n" + + "Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.\n" + + "Excepteur sint occaecat cupidatat non proident,\n" + + "sunt in culpa qui officia deserunt mollit anim id est laborum."; + + /** + * Create a mock SingleProcessorController using our processor and pass data + * to it via byte array. Returns the Sink that provides access to any files + * that pass out of the processor + */ + @Before + public void setupEach() throws IOException { + controller = TestRunners.newTestRunner(ExecuteScript.class); + controller.setValidateExpressionUsage(false); + + // copy all scripts to target directory and run from there. some python + // scripts create .class files that end up in src/test/resources. + FileUtils.copyDirectory(new File("src/test/resources"), new File("target/test-scripts")); + } + + // Fail if the specified relationship does not contain exactly one file + // with the expected value + private void assertRelationshipContents(String expected, String relationship) { + controller.assertTransferCount(relationship, 1); + MockFlowFile ff = controller.getFlowFilesForRelationship(relationship).get(0); + ff.assertContentEquals(expected); + } + + // Fail if the specified relationship does not contain specified number of files + // with the expected value + private void assertRelationshipContents(String expected, String relationship, int count) { + controller.assertTransferCount(relationship, count); + MockFlowFile ff = controller.getFlowFilesForRelationship(relationship).get(count - 1); + ff.assertContentEquals(expected); + } + + // //////////////////////////////////// + // General tests + @Test(expected = IllegalArgumentException.class) + public void failOnBadName() { + LOG.info("Supplying bad script file names"); + + // None of these should result in actually setting the property, because they're non-existent / bad files + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "not/really.rb"); + controller.assertNotValid(); + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "fakey/fake.js"); + controller.assertNotValid(); + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "pom.xml"); + controller.assertNotValid(); + } + + // //////////////////////////////////// + // Ruby script tests + @Test + public void testSimpleReadR() { + LOG.info("Ruby script: fail file based on reading contents"); + + controller.enqueue("This stuff is fine".getBytes()); + controller.enqueue(multiline.getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/readTest.rb"); + controller.setThreadCount(2); + controller.run(2); + + assertRelationshipContents(multiline, "failure"); + assertRelationshipContents("This stuff is fine", "success"); + + controller.getFlowFilesForRelationship("success").get(0).assertAttributeEquals("filename", "NewFileNameFromReadTest"); + } + + @Test + public void testParamReadR() { + LOG.info("Ruby script: Failing file based on reading contents"); + + Map<String, String> attrs1 = new HashMap<>(); + attrs1.put("filename", "StuffIsFine.txt"); + Map<String, String> attrs2 = new HashMap<>(); + attrs2.put("filename", "multiline.txt"); + controller.enqueue("This stuff is fine".getBytes(), attrs1); + controller.enqueue(multiline.getBytes(), attrs2); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/readWithParams.rb"); + controller.setProperty("expr", "rehenderit"); + controller.run(2); + + assertRelationshipContents(multiline, "failure"); + assertRelationshipContents("This stuff is fine", "success"); + } + + @Test + public void testWriteLastLineR() { + LOG.info("Running Ruby script to output last line of file"); + + controller.enqueue(multiline.getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/writeTest.rb"); + controller.run(); + + List<MockFlowFile> files = controller.getFlowFilesForRelationship("success"); + + assertEquals("Process did not generate an output file", 1, files.size()); + + byte[] blob = files.get(0).toByteArray(); + String[] lines = new String(blob).split("\n"); + + assertEquals("File had more than one line", 1, lines.length); + assertEquals("sunt in culpa qui officia deserunt mollit anim id est laborum.", lines[0]); + } + + @Test + public void testWriteOptionalParametersR() { + LOG.info("Ruby script that uses optional parameters"); + + controller.enqueue(multiline.getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/paramTest.rb"); + controller.setProperty("repeat", "3"); + controller.run(); + + List<MockFlowFile> files = controller.getFlowFilesForRelationship("success"); + + assertEquals("Process did not generate an output file", 1, files.size()); + + byte[] blob = files.get(0).toByteArray(); + String[] lines = new String(blob).split("\n"); + + assertEquals("File did not have 3 lines", 3, lines.length); + assertEquals("sunt in culpa qui officia deserunt mollit anim id est laborum.", lines[0]); + } + + @Test + public void testSetupOptionalValidationR() { + LOG.info("Ruby script creating validators for optional properties"); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/optionalValidators.rb"); + controller.assertNotValid(); + controller.setProperty("int", "abc"); + controller.assertNotValid(); + controller.setProperty("url", "not@valid"); + controller.assertNotValid(); + controller.setProperty("nonEmpty", ""); + controller.assertNotValid(); + + controller.setProperty("int", "123"); + controller.setProperty("url", "http://localhost"); + controller.setProperty("nonEmpty", "abc123"); + controller.assertValid(); + } + + @Test + public void testTwoScriptsSameThreadSameClassName() { + LOG.info("Test 2 different scripts with the same ruby class name"); + + Map<String, String> attrs1 = new HashMap<>(); + attrs1.put("filename", "StuffIsFine.txt"); + Map<String, String> attrs2 = new HashMap<>(); + attrs2.put("filename", "multiline.txt"); + + controller.enqueue("This stuff is fine".getBytes(), attrs1); + controller.enqueue(multiline.getBytes(), attrs2); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/readWithParams.rb"); + controller.setProperty("expr", "rehenderit"); + controller.run(2); + + assertRelationshipContents(multiline, "failure"); + assertRelationshipContents("This stuff is fine", "success"); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/optionalValidators.rb"); + controller.assertNotValid(); + controller.setProperty("int", "abc"); + controller.assertNotValid(); + controller.setProperty("url", "not@valid"); + controller.assertNotValid(); + controller.setProperty("nonEmpty", ""); + controller.assertNotValid(); + + controller.setProperty("int", "123"); + controller.setProperty("url", "http://localhost"); + controller.setProperty("nonEmpty", "abc123"); + controller.assertValid(); + } + + @Test + public void testUpdateScriptR() throws Exception { + LOG.info("Test one script with updated class"); + + File testFile = File.createTempFile("script", ".rb"); + File original = new File("target/test-scripts/readWithParams.rb"); + FileUtils.copyFile(original, testFile); + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, testFile.getPath()); + controller.assertValid(); + original = new File("target/test-scripts/optionalValidators.rb"); + FileUtils.copyFile(original, testFile); + controller.setProperty(ExecuteScript.SCRIPT_CHECK_INTERVAL, "5 secs"); + Thread.sleep(6000); + + controller.assertNotValid(); + controller.setProperty("int", "abc"); + controller.assertNotValid(); + controller.setProperty("url", "not@valid"); + controller.assertNotValid(); + controller.setProperty("nonEmpty", ""); + controller.assertNotValid(); + + controller.setProperty("int", "123"); + controller.setProperty("url", "http://localhost"); + controller.setProperty("nonEmpty", "abc123"); + controller.assertValid(); + FileUtils.deleteQuietly(testFile); + } + + @Test + public void testMultiThreadExecR() { + LOG.info("Ruby script 20 threads: Failing file based on reading contents"); + + Map<String, String> attrs1 = new HashMap<>(); + attrs1.put("filename", "StuffIsFine.txt"); + Map<String, String> attrs2 = new HashMap<>(); + attrs2.put("filename", "multiline.txt"); + controller.setThreadCount(20); + for (int i = 0; i < 10; i++) { + controller.enqueue("This stuff is fine".getBytes(), attrs1); + controller.enqueue(multiline.getBytes(), attrs2); + } + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/readWithParams.rb"); + controller.setProperty("expr", "rehenderit"); + controller.run(20); + + controller.assertTransferCount("failure", 10); + controller.assertTransferCount("success", 10); + for (int i = 0; i < 10; i++) { + MockFlowFile ff = controller.getFlowFilesForRelationship("failure").get(i); + ff.assertContentEquals(multiline); + assertTrue(ff.getAttribute("filename").endsWith("modified")); + ff = controller.getFlowFilesForRelationship("success").get(i); + ff.assertContentEquals("This stuff is fine"); + assertTrue(ff.getAttribute("filename").endsWith("modified")); + } + + } + + @Test + public void testManualValidationR() { + LOG.info("Ruby script defining manual validator"); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/alwaysFail.rb"); + controller.assertNotValid(); + } + + @Test + public void testGetRelationshipsR() { + LOG.info("Ruby script: getRelationships"); + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/routeTest.rb"); + // at this point, the script has not been instantiated so the processor simply returns an empty set + Set<Relationship> rels = controller.getProcessor().getRelationships(); + assertEquals(0, rels.size()); + // this will instantiate the script + controller.assertValid(); + // this will call the script + rels = controller.getProcessor().getRelationships(); + assertEquals(3, rels.size()); + } + + @Test + public void testGetExceptionRouteR() { + LOG.info("Ruby script defining route taken in event of exception"); + + controller.enqueue("This stuff is fine".getBytes()); + controller.enqueue("Bad things go to 'b'.".getBytes()); + controller.enqueue(multiline.getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/routeTest.rb"); + + controller.run(3); + + assertRelationshipContents("This stuff is fine", "a"); + assertRelationshipContents("Bad things go to 'b'.", "b"); + assertRelationshipContents(multiline, "c"); + + } + + @Test + public void testSimpleConverterR() { + LOG.info("Running Ruby converter script"); + + for (int i = 0; i < 20; i++) { + controller.enqueue(multiline.getBytes()); + } + + controller.setThreadCount(20); + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/simpleConverter.rb"); + controller.run(20); + + List<MockFlowFile> successFiles = controller.getFlowFilesForRelationship("success"); + List<MockFlowFile> failFiles = controller.getFlowFilesForRelationship("failure"); + + assertEquals("Process did not generate 20 SUCCESS files", 20, successFiles.size()); + assertEquals("Process did not generate 20 FAILURE files", 20, failFiles.size()); + + MockFlowFile sFile = successFiles.get(19); + MockFlowFile fFile = failFiles.get(19); + + byte[] blob = fFile.toByteArray(); + String[] lines = new String(blob).split("\n"); + + assertEquals("File had more than one line", 1, lines.length); + assertEquals("Lorem ipsum dolor sit amet,", lines[0]); + + blob = sFile.toByteArray(); + lines = new String(blob).split("\n"); + + assertEquals("SUCCESS had wrong number of lines", 7, lines.length); + assertEquals("consectetur adipisicing elit,", lines[0]); + } + + @Test + public void testLoadLocalR() { + LOG.info("Ruby: load another script file"); + + controller.enqueue("This stuff is fine".getBytes()); + controller.enqueue(multiline.getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/loadLocal.rb"); + controller.run(2); + + assertRelationshipContents(multiline, "failure"); + assertRelationshipContents("This stuff is fine", "success"); + } + + @Test + public void testFlowFileR() { + LOG.info("Ruby: get FlowFile properties"); + + controller.enqueue(multiline.getBytes()); + HashMap<String, String> meta = new HashMap<String, String>(); + meta.put("evict", "yup"); + controller.enqueue("This would be plenty long but it's also evicted.".getBytes(), meta); + controller.enqueue("This is too short".getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/ffTest.rb"); + controller.run(3); + + assertRelationshipContents(multiline, "success"); + assertRelationshipContents("This is too short", "failure"); + assertRelationshipContents("This would be plenty long but it's also evicted.", "evict"); + } + + // //////////////////////////////////// // JS tests + @Test + public void testSimpleReadJS() { + LOG.info("Javascript: fail file based on reading contents"); + + controller.enqueue("This stuff is fine".getBytes()); + controller.enqueue(multiline.getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/readTest.js"); + controller.run(2); + + assertRelationshipContents(multiline, "failure"); + assertRelationshipContents("This stuff is fine", "success"); + } + + @Test + public void testParamReadJS() { + LOG.info("Javascript: read contents and fail based on parameter"); + + controller.enqueue("This stuff is fine".getBytes()); + controller.enqueue(multiline.getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/readWithParams.js"); + controller.setProperty("expr", "sed do"); + controller.run(2); + + assertRelationshipContents(multiline, "failure"); + assertRelationshipContents("This stuff is fine", "success"); + } + + @Test + public void testWriteLastLineJS() { + LOG.info("Running Javascript to output last line of file"); + + controller.enqueue(multiline.getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/writeTest.js"); + controller.run(); + + List<MockFlowFile> sunkFiles = controller.getFlowFilesForRelationship("success"); + + assertEquals("Process did not generate an output file", 1, sunkFiles.size()); + + MockFlowFile sunkFile = sunkFiles.iterator().next(); + byte[] blob = sunkFile.toByteArray(); + String[] lines = new String(blob).split("\n"); + + assertEquals("File had more than one line", 1, lines.length); + assertEquals("sunt in culpa qui officia deserunt mollit anim id est laborum.", lines[0]); + } + + @Test + public void testWriteOptionalParametersJS() { + LOG.info("Javascript processCallback that uses optional parameters"); + + controller.enqueue(multiline.getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/paramTest.js"); + controller.setProperty("repeat", "3"); + controller.run(); + + List<MockFlowFile> sunkFiles = controller.getFlowFilesForRelationship("success"); + + assertEquals("Process did not generate an output file", 1, sunkFiles.size()); + + MockFlowFile sunkFile = sunkFiles.iterator().next(); + byte[] blob = sunkFile.toByteArray(); + String[] lines = new String(blob).split("\n"); + + assertEquals("File did not have 3 lines", 3, lines.length); + assertEquals("sunt in culpa qui officia deserunt mollit anim id est laborum.", lines[0]); + } + + @Test + public void testSetupOptionalValidationJS() { + LOG.info("Javascript creating validators for optional properties"); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/optionalValidators.js"); + controller.setProperty("int", "abc"); + controller.setProperty("url", "not@valid"); + controller.setProperty("nonEmpty", ""); + assertEquals(2, controller.getProcessor().getPropertyDescriptors().size()); + controller.assertNotValid(); // due to invalid values above + assertEquals(5, controller.getProcessor().getPropertyDescriptors().size()); + + controller.setProperty("int", "123"); + controller.setProperty("url", "http://localhost"); + controller.setProperty("nonEmpty", "abc123"); + assertEquals(5, controller.getProcessor().getPropertyDescriptors().size()); + controller.assertValid(); + } + + @Test + public void testManualValidationJS() { + LOG.info("Javascript defining manual validator"); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/alwaysFail.js"); + controller.assertNotValid(); + } + + @Test + public void testGetExceptionRouteJS() { + LOG.info("Javascript defining route taken in event of exception"); + + controller.enqueue("This stuff is fine".getBytes()); + controller.enqueue("Bad things go to 'b'.".getBytes()); + controller.enqueue(multiline.getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/routeTest.js"); + + controller.run(3); + + assertRelationshipContents("This stuff is fine", "a"); + assertRelationshipContents("Bad things go to 'b'.", "b"); + assertRelationshipContents(multiline, "c"); + + } + + @Test + public void testSimpleConverterJS() { + LOG.info("Running Javascript converter script"); + + for (int i = 0; i < 20; i++) { + controller.enqueue(multiline.getBytes()); + } + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/simpleConverter.js"); + controller.run(20); + + List<MockFlowFile> successFiles = controller.getFlowFilesForRelationship("success"); + List<MockFlowFile> failFiles = controller.getFlowFilesForRelationship("failure"); + + assertEquals("Process did not generate 20 SUCCESS files", 20, successFiles.size()); + assertEquals("Process did not generate 20 FAILURE file", 20, failFiles.size()); + + MockFlowFile sFile = successFiles.get(19); + MockFlowFile fFile = failFiles.get(0); + + byte[] blob = sFile.toByteArray(); + String[] lines = new String(blob).split("\n"); + + assertEquals("SUCCESS had wrong number of lines", 7, lines.length); + assertTrue(lines[0].startsWith("consectetur adipisicing elit,")); + + blob = fFile.toByteArray(); + lines = new String(blob).split("\n"); + + assertEquals("File had more than one line", 1, lines.length); + assertTrue(lines[0].startsWith("Lorem ipsum dolor sit amet,")); + } + + @Test + public void testLoadLocalJS() { + LOG.info("Javascript: load another script file"); + + controller.enqueue("This stuff is fine".getBytes()); + controller.enqueue(multiline.getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/loadLocal.js"); + controller.run(2); + + assertRelationshipContents(multiline, "failure"); + assertRelationshipContents("This stuff is fine", "success"); + } + + @Test + public void testXMLJS() { + LOG.info("Javascript: native XML parser"); + + controller.enqueue("<a><b foo='bar'>Bad</b><b good='true'>Good</b><b good='false'>Bad</b></a>".getBytes()); + controller.enqueue("<a><b>Hello</b><b>world</b></a>".getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/parseXml.js"); + controller.run(2); + + assertRelationshipContents("Good", "success"); + assertRelationshipContents("<a><b>Hello</b><b>world</b></a>", "failure"); + } + + @Test + public void testFlowFileJS() { + LOG.info("JavaScript: get FlowFile properties"); + + controller.enqueue("This is too short".getBytes()); + controller.enqueue(multiline.getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/ffTest.js"); + controller.run(2); + + assertRelationshipContents(multiline, "success"); + assertRelationshipContents("This is too short", "failure"); + } + + @Test + public void testMultiThreadExecJS() { + LOG.info("JavaScript script 20 threads: Failing file based on reading contents"); + + Map<String, String> attrs1 = new HashMap<>(); + attrs1.put("filename", "StuffIsFine.txt"); + Map<String, String> attrs2 = new HashMap<>(); + attrs2.put("filename", "multiline.txt"); + controller.setThreadCount(20); + for (int i = 0; i < 10; i++) { + controller.enqueue("This stuff is fine".getBytes(), attrs1); + controller.enqueue(multiline.getBytes(), attrs2); + } + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/readWithParams.js"); + controller.setProperty("expr", "rehenderit"); + controller.run(20); + + controller.assertTransferCount("failure", 10); + controller.assertTransferCount("success", 10); + for (int i = 0; i < 10; i++) { + MockFlowFile ff = controller.getFlowFilesForRelationship("failure").get(i); + ff.assertContentEquals(multiline); + assertTrue(ff.getAttribute("filename").endsWith("modified")); + ff = controller.getFlowFilesForRelationship("success").get(i); + ff.assertContentEquals("This stuff is fine"); + assertTrue(ff.getAttribute("filename").endsWith("modified")); + } + } + + @Test + public void testUpdateScriptJS() throws Exception { + LOG.info("Test one script with updated class"); + + File testFile = File.createTempFile("script", ".js"); + File original = new File("target/test-scripts/readWithParams.js"); + FileUtils.copyFile(original, testFile); + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, testFile.getPath()); + controller.assertValid(); + original = new File("target/test-scripts/optionalValidators.js"); + FileUtils.copyFile(original, testFile); + controller.setProperty(ExecuteScript.SCRIPT_CHECK_INTERVAL, "5 secs"); + Thread.sleep(6000); + + controller.assertNotValid(); + controller.setProperty("int", "abc"); + controller.assertNotValid(); + controller.setProperty("url", "not@valid"); + controller.assertNotValid(); + controller.setProperty("nonEmpty", ""); + controller.assertNotValid(); + + controller.setProperty("int", "123"); + controller.setProperty("url", "http://localhost"); + controller.setProperty("nonEmpty", "abc123"); + controller.assertValid(); + FileUtils.deleteQuietly(testFile); + } + + // ////////////////////////////////// // Python script tests + @Test + public void testSimpleReadP() { + LOG.info("Python script: fail file based on reading contents"); + + for (int i = 0; i < 20; i++) { + Map<String, String> attr1 = new HashMap<>(); + attr1.put("filename", "FineStuff"); + attr1.put("counter", Integer.toString(i)); + Map<String, String> attr2 = new HashMap<>(); + attr2.put("filename", "MultiLine"); + attr2.put("counter", Integer.toString(i)); + controller.enqueue("This stuff is fine".getBytes(), attr1); + controller.enqueue(multiline.getBytes(), attr2); + } + + controller.setThreadCount(40); + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/readTest.py"); + controller.run(40); + + assertRelationshipContents(multiline, "failure", 20); + assertRelationshipContents("This stuff is fine", "success", 20); + + List<MockFlowFile> fails = controller.getFlowFilesForRelationship("failure"); + List<MockFlowFile> successes = controller.getFlowFilesForRelationship("success"); + for (int i = 0; i < 20; i++) { + assertTrue(fails.get(i).getAttribute("filename").matches("^.*\\d+$")); + assertTrue(successes.get(i).getAttribute("filename").matches("^.*\\d+$")); + } + } + + @Test + public void testParamReadP() { + LOG.info("Python script: read contents and fail based on parameter"); + + controller.enqueue("This stuff is fine".getBytes()); + controller.enqueue(multiline.getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/readWithParams.py"); + controller.setProperty("expr", "sed do"); + controller.run(2); + + assertRelationshipContents(multiline, "failure"); + assertRelationshipContents("This stuff is fine", "success"); + } + + @Test + public void testWriteLastLineP() { + LOG.info("Running Python script to output last line of file"); + + controller.enqueue(multiline.getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/writeTest.py"); + controller.run(); + + List<MockFlowFile> sunkFiles = controller.getFlowFilesForRelationship("success"); + + assertEquals("Process did not generate an output file", 1, sunkFiles.size()); + + MockFlowFile sunkFile = sunkFiles.iterator().next(); + byte[] blob = sunkFile.toByteArray(); + String[] lines = new String(blob).split("\n"); + + assertEquals("File had more than one line", 1, lines.length); + assertEquals("sunt in culpa qui officia deserunt mollit anim id est laborum.", lines[0]); + } + + @Test + public void testWriteOptionalParametersP() { + LOG.info("Python script processCallback that uses optional parameters"); + + controller.enqueue(multiline.getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/paramTest.py"); + controller.setProperty("repeat", "3"); + controller.run(); + + List<MockFlowFile> sunkFiles = controller.getFlowFilesForRelationship("success"); + + assertEquals("Process did not generate an output file", 1, sunkFiles.size()); + + MockFlowFile sunkFile = sunkFiles.iterator().next(); + byte[] blob = sunkFile.toByteArray(); + String[] lines = new String(blob).split("\n"); + + assertEquals("File did not have 3 lines", 3, lines.length); + assertTrue(lines[2].startsWith("sunt in culpa qui officia deserunt mollit anim id est laborum.")); + } + + @Test + public void testManualValidationP() { + LOG.info("Python defining manual validator"); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/alwaysFail.py"); + controller.assertNotValid(); + } + + @Test + public void testSetupOptionalValidationP() { + LOG.info("Python script creating validators for optional properties"); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/optionalValidators.py"); + controller.setProperty("int", "abc"); + controller.setProperty("url", "not@valid"); + controller.setProperty("nonEmpty", ""); + assertEquals(2, controller.getProcessor().getPropertyDescriptors().size()); + controller.assertNotValid(); + + controller.setProperty("int", "123"); + controller.setProperty("url", "http://localhost"); + controller.setProperty("nonEmpty", "abc123"); + assertEquals(5, controller.getProcessor().getPropertyDescriptors().size()); + controller.assertValid(); + } + + @Test + public void testGetExceptionRouteP() { + LOG.info("Python script defining route taken in event of exception"); + + controller.enqueue("This stuff is fine".getBytes()); + controller.enqueue("Bad things go to 'b'.".getBytes()); + controller.enqueue(multiline.getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/routeTest.py"); + + // Don't put the error in the logs + // TestableAppender ta = new TestableAppender(); + // ta.attach(Logger.getLogger(ExecuteScript.class)); + controller.run(3); + // ta.detach(); + + assertRelationshipContents("This stuff is fine", "a"); + assertRelationshipContents("Bad things go to 'b'.", "b"); + assertRelationshipContents(multiline, "c"); + + // ta.assertFound("threw exception"); + } + + @Test + public void testLoadLocalP() throws Exception { + + final Thread t = new Thread(new Runnable() { + @Override + public void run() { + try { + testGetExceptionRouteP(); + setupEach(); + } catch (Exception e) { + + } + } + }); + + t.start(); + t.join(); + + LOG.info("Python: load another script file"); + + controller.enqueue("This stuff is fine".getBytes()); + controller.enqueue(multiline.getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/loadLocal.py"); + controller.run(2); + + assertRelationshipContents(multiline, "failure"); + assertRelationshipContents("This stuff is fine", "success"); + } + + @Test + public void testSimpleConverterP() { + LOG.info("Running Python converter script"); + + for (int i = 0; i < 20; i++) { + controller.enqueue(multiline.getBytes()); + } + + controller.setThreadCount(20); + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/simpleConverter.py"); + controller.run(20); + + List<MockFlowFile> successFiles = controller.getFlowFilesForRelationship("success"); + List<MockFlowFile> failFiles = controller.getFlowFilesForRelationship("failure"); + + assertEquals("Process did not generate 20 SUCCESS files", 20, successFiles.size()); + assertEquals("Process did not generate 20 FAILURE files", 20, failFiles.size()); + + MockFlowFile sFile = successFiles.iterator().next(); + MockFlowFile fFile = failFiles.iterator().next(); + + byte[] blob = sFile.toByteArray(); + String[] lines = new String(blob).split("\n"); + + assertEquals("SUCCESS had wrong number of lines", 7, lines.length); + assertTrue(lines[0].startsWith("consectetur adipisicing elit,")); + + blob = fFile.toByteArray(); + lines = new String(blob).split("\n"); + + assertEquals("File had more than one line", 1, lines.length); + assertTrue(lines[0].startsWith("Lorem ipsum dolor sit amet,")); + } + + @Test + public void testFlowFileP() { + LOG.info("Python: get FlowFile properties"); + + controller.enqueue("This is too short".getBytes()); + controller.enqueue(multiline.getBytes()); + + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/ffTest.py"); + controller.run(2); + + assertRelationshipContents(multiline, "success"); + assertRelationshipContents("This is too short", "failure"); + } + + @Test + public void testMultiThreadExecP() { + LOG.info("Pthon script 20 threads: Failing file based on reading contents"); + + Map<String, String> attrs1 = new HashMap<>(); + attrs1.put("filename", "StuffIsFine.txt"); + Map<String, String> attrs2 = new HashMap<>(); + attrs2.put("filename", "multiline.txt"); + for (int i = 0; i < 10; i++) { + controller.enqueue("This stuff is fine".getBytes(), attrs1); + controller.enqueue(multiline.getBytes(), attrs2); + } + + controller.setThreadCount(20); + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, "target/test-scripts/readWithParams.py"); + controller.setProperty("expr", "sed do"); + controller.run(20); + + controller.assertTransferCount("failure", 10); + controller.assertTransferCount("success", 10); + for (int i = 0; i < 10; i++) { + MockFlowFile ff = controller.getFlowFilesForRelationship("failure").get(i); + ff.assertContentEquals(multiline); + assertTrue(ff.getAttribute("filename").endsWith("modified")); + ff = controller.getFlowFilesForRelationship("success").get(i); + ff.assertContentEquals("This stuff is fine"); + assertTrue(ff.getAttribute("filename").endsWith("modified")); + } + } + + @Test + public void testUpdateScriptP() throws Exception { + LOG.info("Test one script with updated class"); + + File testFile = File.createTempFile("script", ".py"); + File original = new File("target/test-scripts/readTest.py"); + FileUtils.copyFile(original, testFile); + controller.setProperty(ExecuteScript.SCRIPT_FILE_NAME, testFile.getPath()); + controller.assertValid(); + original = new File("target/test-scripts/readWithParams.py"); + FileUtils.copyFile(original, testFile); + controller.setProperty(ExecuteScript.SCRIPT_CHECK_INTERVAL, "5 secs"); + Thread.sleep(6000); + + controller.assertNotValid(); // need to set 'expr' + controller.setProperty("int", "abc"); + controller.assertNotValid(); + controller.setProperty("url", "not@valid"); + controller.assertNotValid(); + controller.setProperty("nonEmpty", ""); + controller.assertNotValid(); + + controller.setProperty("expr", "sed do"); + controller.assertValid(); + assertEquals(6, controller.getProcessContext().getProperties().size()); + FileUtils.deleteQuietly(testFile); + } + +} http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/alwaysFail.js ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/alwaysFail.js b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/alwaysFail.js new file mode 100644 index 0000000..40f038d --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/alwaysFail.js @@ -0,0 +1,24 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +with (Scripting) { + var instance = new ReaderScript({ + validate: function () { + return ["This will never work."]; + } + }); +} + http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/alwaysFail.py ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/alwaysFail.py b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/alwaysFail.py new file mode 100644 index 0000000..4b744ab --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/alwaysFail.py @@ -0,0 +1,19 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# type(name, superclass_tuple, member_dict) is a shorthand for defining an +# anonymous class. Note the trailing parens (), because scriptBuilder must +# return an *instance* of the class. +instance = type("FailingReader", (ReaderScript, object),\ + {"validate": lambda self : ["I am broken"]})() \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/alwaysFail.rb ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/alwaysFail.rb b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/alwaysFail.rb new file mode 100644 index 0000000..076cdfa --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/alwaysFail.rb @@ -0,0 +1,21 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +class NeverSatisfied < ReaderScript + def validate + return ["This is supposed to fail"] + end +end + +NeverSatisfied.new http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/ffTest.js ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/ffTest.js b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/ffTest.js new file mode 100644 index 0000000..d40623e --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/ffTest.js @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +with (Scripting) { + var instance = new ReaderScript({ + route: function (input) { + if (instance.getFlowFileSize() < 20) { + return Script.FAIL_RELATIONSHIP; + } else { + return Script.SUCCESS_RELATIONSHIP; + } + } + }); + +} http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/ffTest.py ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/ffTest.py b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/ffTest.py new file mode 100644 index 0000000..f4deb22 --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/ffTest.py @@ -0,0 +1,22 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +class SimpleJythonReader(ReaderScript): + def route(self, input): + if self.getFlowFileSize() < 20 : return self.FAIL_RELATIONSHIP + + return self.SUCCESS_RELATIONSHIP + + +instance = SimpleJythonReader() http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/ffTest.rb ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/ffTest.rb b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/ffTest.rb new file mode 100644 index 0000000..0435ea7 --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/ffTest.rb @@ -0,0 +1,30 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +class SimpleJRubyReader < ReaderScript + field_reader :FAIL_RELATIONSHIP, :SUCCESS_RELATIONSHIP + + def getRelationships + @@evict = Relationship::Builder.new.name("evict").description("some evicted stuff").build() + [FAIL_RELATIONSHIP, SUCCESS_RELATIONSHIP, @@evict] + end + + def route( input ) + return FAIL_RELATIONSHIP if getFlowFileSize < 20 + return @@evict if !getAttribute("evict").nil? + return SUCCESS_RELATIONSHIP + end +end + +SimpleJRubyReader.new http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/lib/Sub.py ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/lib/Sub.py b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/lib/Sub.py new file mode 100644 index 0000000..827816c --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/lib/Sub.py @@ -0,0 +1,18 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +class TestFilter: + def notAllowed(self): + return "^sed" + \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/lib/sub.js ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/lib/sub.js b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/lib/sub.js new file mode 100644 index 0000000..489036a --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/lib/sub.js @@ -0,0 +1,22 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +function notAllowed() { // Works for eval(readFile(...)) + return /sed do/i; +} + +exports.notAllowed = notAllowed; // Works for require(...) + http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/lib/sub.rb ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/lib/sub.rb b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/lib/sub.rb new file mode 100644 index 0000000..775aad4 --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/lib/sub.rb @@ -0,0 +1,17 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +def notAllowed + return /^sed/i +end \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/loadLocal.js ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/loadLocal.js b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/loadLocal.js new file mode 100644 index 0000000..39396ab --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/loadLocal.js @@ -0,0 +1,30 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +var sub = require("lib/sub.js"); + +with (Scripting) { + var instance = new ReaderScript({ + route: function (input) { + var str = IOUtils.toString(input); + if (str.match(sub.notAllowed())) { + return Script.FAIL_RELATIONSHIP; + } else { + return Script.SUCCESS_RELATIONSHIP; + } + } + }); +} http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/loadLocal.py ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/loadLocal.py b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/loadLocal.py new file mode 100644 index 0000000..2429ea2 --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/loadLocal.py @@ -0,0 +1,26 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +import re +from Sub import TestFilter +class SimpleJythonReader(ReaderScript): + def route(self, input): + tf = TestFilter() + for line in FileUtil.wrap(input): + if re.match(tf.notAllowed(),line): return self.FAIL_RELATIONSHIP + + return self.SUCCESS_RELATIONSHIP + + +instance = SimpleJythonReader() http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/loadLocal.rb ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/loadLocal.rb b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/loadLocal.rb new file mode 100644 index 0000000..f1b3ec5 --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/loadLocal.rb @@ -0,0 +1,29 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +require 'sub' + +class SimpleJRubyReader < ReaderScript + field_reader :FAIL_RELATIONSHIP, :SUCCESS_RELATIONSHIP + + def route( input ) + input.to_io.each_line do |line| + return FAIL_RELATIONSHIP if line.match notAllowed + end + + return SUCCESS_RELATIONSHIP + end +end + +SimpleJRubyReader.new http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/log4j.xml ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/log4j.xml b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/log4j.xml new file mode 100644 index 0000000..d984dff --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/log4j.xml @@ -0,0 +1,54 @@ +<?xml version="1.0" encoding="UTF-8" ?> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> +<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd"> + +<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/"> + <appender name="CONSOLE" class="org.apache.log4j.ConsoleAppender"> + <param name="Target" value="System.out"/> + <param name="Threshold" value="DEBUG"/> + + <layout class="org.apache.log4j.PatternLayout"> + <param name="ConversionPattern" value="%d{MM-dd-yy HH:mm:ss.SSS} %-5p %c{2} %x - %m%n"/> + </layout> + + </appender> + + <logger name="org.nifi.model.processor.FlowFileProcessor" additivity="false"> + <level value="INFO"/> + <appender-ref ref="CONSOLE"/> + </logger> + + <logger name="junit.TestableAppender" additivity="false"> + <level value="TRACE"/> + <appender-ref ref="CONSOLE"/> + </logger> + + <logger name="org.nifi.model.processor.impl.ScriptRunnerProcessor" additivity="false"> + <level value="WARN"/> + <appender-ref ref="CONSOLE"/> + </logger> + + <logger name="org.nifi.model.processor.impl.ScriptRunnerProcessorTest" additivity="false"> + <level value="INFO"/> + <appender-ref ref="CONSOLE"/> + </logger> + + <root> + <level value="INFO"/> + <appender-ref ref="CONSOLE" /> + </root> + +</log4j:configuration> http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/optionalValidators.js ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/optionalValidators.js b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/optionalValidators.js new file mode 100644 index 0000000..cf7c551 --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/optionalValidators.js @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +with (Scripting) { + var instance = new WriterScript({ + getPropertyDescriptors: function () { + i = new PropertyDescriptor.Builder().name("int").description("an int").required(true).addValidator(StandardValidators.POSITIVE_INTEGER_VALIDATOR).build(); + u = new PropertyDescriptor.Builder().name("url").description("a url").required(true).addValidator(StandardValidators.URL_VALIDATOR).build(); + s = new PropertyDescriptor.Builder().name("nonEmpty").description("a non empty property").required(true).addValidator(StandardValidators.NON_EMPTY_VALIDATOR).build(); + return [i, u, s]; + } + }); + +} + http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/optionalValidators.py ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/optionalValidators.py b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/optionalValidators.py new file mode 100644 index 0000000..9f1a43d --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/optionalValidators.py @@ -0,0 +1,22 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +class SimpleJythonReader(ReaderScript): + def getPropertyDescriptors( self ): + nev = PropertyDescriptor.Builder().name("nonEmpty").required(1).addValidator(StandardValidators.NON_EMPTY_VALIDATOR).build() + iv = PropertyDescriptor.Builder().name("int").required(1).addValidator(StandardValidators.POSITIVE_INTEGER_VALIDATOR).build() + uv = PropertyDescriptor.Builder().name("url").required(1).addValidator(StandardValidators.URL_VALIDATOR).build() + return [nev, iv, uv] + +instance = SimpleJythonReader() http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/optionalValidators.rb ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/optionalValidators.rb b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/optionalValidators.rb new file mode 100644 index 0000000..3fa53b4 --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/optionalValidators.rb @@ -0,0 +1,39 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +class SimpleJRubyReader < ReaderScript + field_reader :FAIL_RELATIONSHIP, :SUCCESS_RELATIONSHIP, :logger + + def getPropertyDescriptors + logger.debug("Defining descriptors"); + i = StandardValidators::INTEGER_VALIDATOR + u = StandardValidators::URL_VALIDATOR + s = StandardValidators::NON_EMPTY_VALIDATOR + intPropDesc = PropertyDescriptor::Builder.new().name("int").required(true).addValidator(i).build() + urlPropDesc = PropertyDescriptor::Builder.new().name("url").required(true).addValidator(u).build() + nonEmptyPropDesc = PropertyDescriptor::Builder.new().name("nonEmpty").addValidator(s).build() + return [intPropDesc, urlPropDesc, nonEmptyPropDesc] + end + + def route( input ) + logger.debug("Routing input"); + input.to_io.each_line do |line| + return FAIL_RELATIONSHIP if line.match /^sed/i + end + + return SUCCESS_RELATIONSHIP + end +end +$logger.debug("Creating SimpleJRubyReader with props" + @properties.to_s) +SimpleJRubyReader.new \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/paramTest.js ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/paramTest.js b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/paramTest.js new file mode 100644 index 0000000..c00537a --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/paramTest.js @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +with (Scripting) { + var instance = new WriterScript({ + process: function (input, output) { + var str = IOUtils.toString(input); + var last = str.split("\n").pop() + "\n"; + for (var i = 0; i < instance.getProperty("repeat"); i++) { + IOUtils.write(last, output); + } + output.flush(); + } + }); +} http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/paramTest.py ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/paramTest.py b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/paramTest.py new file mode 100644 index 0000000..3495e4a --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/paramTest.py @@ -0,0 +1,26 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +class ParamUsingWriter(WriterScript): + def process ( self, input, output ): + last = FileUtil.wrap(input).readlines()[-1] + '\n' + writer = FileUtil.wrap(output) + times = int(self.getProperty("repeat")) + lines = [last] * times + + writer.writelines(lines) + + writer.close() + +instance = ParamUsingWriter() http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/paramTest.rb ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/paramTest.rb b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/paramTest.rb new file mode 100644 index 0000000..7c2eb23 --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/paramTest.rb @@ -0,0 +1,31 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +class ParamUsingWriter < WriterScript + def process ( input, output ) + reader = input.to_io + writer = output.to_io + + last = reader.readlines.last + + getProperty("repeat").to_i.times do + writer << last + "\n" + end + + writer.close + reader.close + end +end + +ParamUsingWriter.new http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/4d998c12/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/parseXml.js ---------------------------------------------------------------------- diff --git a/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/parseXml.js b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/parseXml.js new file mode 100644 index 0000000..901e158 --- /dev/null +++ b/nar-bundles/execute-script-bundle/execute-script-processors/src/test/resources/parseXml.js @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +with (Scripting) { + var instance = new WriterScript({ + process: function (istream, ostream) { + var str = IOUtils.toString(istream); + var obj = new XML(str); + print(obj) + var good = obj.b.(@good == "true"); + if (good.length() == 0) { + instance.setRoute(Script.FAIL_RELATIONSHIP); + IOUtils.write(str, ostream); + } else { + instance.setRoute(Script.SUCCESS_RELATIONSHIP); + for each (var goodStr in good) { + IOUtils.write(goodStr, ostream); + } + } + ostream.flush(); + } + }); +} \ No newline at end of file