[jira] [Commented] (MINIFI-350) Create integration tests for cpp docker image
[ https://issues.apache.org/jira/browse/MINIFI-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163363#comment-16163363 ] ASF subversion and git services commented on MINIFI-350: Commit 84f50b5173b6d3cb88ddd8a8f366b80483081da2 in nifi-minifi-cpp's branch refs/heads/master from Andrew I. Christianson [ https://git-wip-us.apache.org/repos/asf?p=nifi-minifi-cpp.git;h=84f50b5 ] MINIFI-350 Added pytest-based system integration test framework and initial test cases This closes #126. Signed-off-by: Aldrin Piri> Create integration tests for cpp docker image > - > > Key: MINIFI-350 > URL: https://issues.apache.org/jira/browse/MINIFI-350 > Project: Apache NiFi MiNiFi > Issue Type: Bug > Components: C++ >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Create a unit test suite which uses compose or docker commands to set up > minifi using the docker image and send data through test flows. Consider > looking into existing test frameworks/harnesses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFI-350) Create integration tests for cpp docker image
[ https://issues.apache.org/jira/browse/MINIFI-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163352#comment-16163352 ] ASF subversion and git services commented on MINIFI-350: Commit 17fe045b48f74ba84035cffb77053ac7dbb50c94 in nifi-minifi-cpp's branch refs/heads/master from Andrew I. Christianson [ https://git-wip-us.apache.org/repos/asf?p=nifi-minifi-cpp.git;h=17fe045 ] MINIFI-350 Added pytest-based system integration test framework and initial test cases This closes #126. Signed-off-by: Aldrin Piri> Create integration tests for cpp docker image > - > > Key: MINIFI-350 > URL: https://issues.apache.org/jira/browse/MINIFI-350 > Project: Apache NiFi MiNiFi > Issue Type: Bug > Components: C++ >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Create a unit test suite which uses compose or docker commands to set up > minifi using the docker image and send data through test flows. Consider > looking into existing test frameworks/harnesses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFI-350) Create integration tests for cpp docker image
[ https://issues.apache.org/jira/browse/MINIFI-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163285#comment-16163285 ] ASF GitHub Bot commented on MINIFI-350: --- Github user achristianson commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/126 The rope files should not have been committed at all. > Create integration tests for cpp docker image > - > > Key: MINIFI-350 > URL: https://issues.apache.org/jira/browse/MINIFI-350 > Project: Apache NiFi MiNiFi > Issue Type: Bug > Components: C++ >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Create a unit test suite which uses compose or docker commands to set up > minifi using the docker image and send data through test flows. Consider > looking into existing test frameworks/harnesses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFI-350) Create integration tests for cpp docker image
[ https://issues.apache.org/jira/browse/MINIFI-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163276#comment-16163276 ] ASF GitHub Bot commented on MINIFI-350: --- Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/126 Hey @achristianson, Everything looks good. Was unsure about the rope file that was created. Let me know how we should handle that. > Create integration tests for cpp docker image > - > > Key: MINIFI-350 > URL: https://issues.apache.org/jira/browse/MINIFI-350 > Project: Apache NiFi MiNiFi > Issue Type: Bug > Components: C++ >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Create a unit test suite which uses compose or docker commands to set up > minifi using the docker image and send data through test flows. Consider > looking into existing test frameworks/harnesses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFI-350) Create integration tests for cpp docker image
[ https://issues.apache.org/jira/browse/MINIFI-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163273#comment-16163273 ] ASF GitHub Bot commented on MINIFI-350: --- Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/126#discussion_r138406101 --- Diff: docker/test/integration/.ropeproject/config.py --- @@ -0,0 +1,112 @@ +# The default ``config.py`` --- End diff -- Does this file make sense to include? Not familiar with rope. If it has purpose, it does need a license header. > Create integration tests for cpp docker image > - > > Key: MINIFI-350 > URL: https://issues.apache.org/jira/browse/MINIFI-350 > Project: Apache NiFi MiNiFi > Issue Type: Bug > Components: C++ >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Create a unit test suite which uses compose or docker commands to set up > minifi using the docker image and send data through test flows. Consider > looking into existing test frameworks/harnesses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFI-350) Create integration tests for cpp docker image
[ https://issues.apache.org/jira/browse/MINIFI-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163201#comment-16163201 ] ASF GitHub Bot commented on MINIFI-350: --- Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/126 reviewing updates > Create integration tests for cpp docker image > - > > Key: MINIFI-350 > URL: https://issues.apache.org/jira/browse/MINIFI-350 > Project: Apache NiFi MiNiFi > Issue Type: Bug > Components: C++ >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Create a unit test suite which uses compose or docker commands to set up > minifi using the docker image and send data through test flows. Consider > looking into existing test frameworks/harnesses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFI-350) Create integration tests for cpp docker image
[ https://issues.apache.org/jira/browse/MINIFI-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153662#comment-16153662 ] ASF GitHub Bot commented on MINIFI-350: --- Github user achristianson commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/126 Added docs to docker/test/integration/README.md. > Create integration tests for cpp docker image > - > > Key: MINIFI-350 > URL: https://issues.apache.org/jira/browse/MINIFI-350 > Project: Apache NiFi MiNiFi > Issue Type: Bug > Components: C++ >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Create a unit test suite which uses compose or docker commands to set up > minifi using the docker image and send data through test flows. Consider > looking into existing test frameworks/harnesses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFI-350) Create integration tests for cpp docker image
[ https://issues.apache.org/jira/browse/MINIFI-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153659#comment-16153659 ] ASF GitHub Bot commented on MINIFI-350: --- Github user achristianson commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/126#discussion_r136992525 --- Diff: README.md --- @@ -124,6 +133,8 @@ $ brew install cmake \ doxygen $ brew install curl $ brew link curl --force +$ # (Optional) for building docker image/running system integration tests +$ # Install docker using instructions at https://docs.docker.com/docker-for-mac/install/ --- End diff -- Fixed. Added instruction to pip install virtualenv. > Create integration tests for cpp docker image > - > > Key: MINIFI-350 > URL: https://issues.apache.org/jira/browse/MINIFI-350 > Project: Apache NiFi MiNiFi > Issue Type: Bug > Components: C++ >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Create a unit test suite which uses compose or docker commands to set up > minifi using the docker image and send data through test flows. Consider > looking into existing test frameworks/harnesses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFI-350) Create integration tests for cpp docker image
[ https://issues.apache.org/jira/browse/MINIFI-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16143261#comment-16143261 ] ASF GitHub Bot commented on MINIFI-350: --- Github user achristianson commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/126 Thanks for the feedback. Adding more documentation is a good idea. I'll get on that and update the PR. > Create integration tests for cpp docker image > - > > Key: MINIFI-350 > URL: https://issues.apache.org/jira/browse/MINIFI-350 > Project: Apache NiFi MiNiFi > Issue Type: Bug > Components: C++ >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Create a unit test suite which uses compose or docker commands to set up > minifi using the docker image and send data through test flows. Consider > looking into existing test frameworks/harnesses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFI-350) Create integration tests for cpp docker image
[ https://issues.apache.org/jira/browse/MINIFI-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16143258#comment-16143258 ] ASF GitHub Bot commented on MINIFI-350: --- Github user achristianson commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/126#discussion_r135424099 --- Diff: docker/test/integration/test_http.py --- @@ -0,0 +1,36 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the \"License\"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an \"AS IS\" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from minifi import * +from minifi.test import * + + +def test_invoke_listen(): +""" +Verify sending using InvokeHTTP to a receiver using ListenHTTP. +""" + +invoke_flow = (GetFile('/tmp/input') >> LogAttribute() >> + InvokeHTTP('http://minifi-listen:8080/contentListener', method='POST')) + +listen_flow = ListenHTTP(8080) >> LogAttribute() >> PutFile('/tmp/output') + +with DockerTestCluster(SingleFileOutputValidator('test')) as cluster: + +cluster.put_test_data('test') +cluster.deploy_flow(listen_flow, name='minifi-listen') +cluster.deploy_flow(invoke_flow, name='minifi-invoke') + +assert cluster.check_output() --- End diff -- The http tests are dependent on the phrocker/MINIFI-339 branch to pass, and possibly some of the other PRs I put in. There is unfortunately a bit of inter-dependence going on right now between the various branches. > Create integration tests for cpp docker image > - > > Key: MINIFI-350 > URL: https://issues.apache.org/jira/browse/MINIFI-350 > Project: Apache NiFi MiNiFi > Issue Type: Bug > Components: C++ >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Create a unit test suite which uses compose or docker commands to set up > minifi using the docker image and send data through test flows. Consider > looking into existing test frameworks/harnesses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFI-350) Create integration tests for cpp docker image
[ https://issues.apache.org/jira/browse/MINIFI-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16143254#comment-16143254 ] ASF GitHub Bot commented on MINIFI-350: --- Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/126#discussion_r135423728 --- Diff: docker/test/integration/test_http.py --- @@ -0,0 +1,36 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the \"License\"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an \"AS IS\" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from minifi import * +from minifi.test import * + + +def test_invoke_listen(): +""" +Verify sending using InvokeHTTP to a receiver using ListenHTTP. +""" + +invoke_flow = (GetFile('/tmp/input') >> LogAttribute() >> + InvokeHTTP('http://minifi-listen:8080/contentListener', method='POST')) + +listen_flow = ListenHTTP(8080) >> LogAttribute() >> PutFile('/tmp/output') + +with DockerTestCluster(SingleFileOutputValidator('test')) as cluster: + +cluster.put_test_data('test') +cluster.deploy_flow(listen_flow, name='minifi-listen') +cluster.deploy_flow(invoke_flow, name='minifi-invoke') + +assert cluster.check_output() --- End diff -- This seems to be failing for me in OS X with Docker Mac. Output below: ``` DEBUG:urllib3.connectionpool:http://localhost:None "DELETE /v1.30/networks/5f6cae20d1721e4ba01039fddabedc627532b5ec698897598cf6f18d2eed21d4 HTTP/1.1" 204 0 FAILED FAILURES ___ test_invoke_listen ___ def test_invoke_listen(): """ Verify sending using InvokeHTTP to a receiver using ListenHTTP. """ invoke_flow = (GetFile('/tmp/input') >> LogAttribute() >> InvokeHTTP('http://minifi-listen:8080/contentListener', method='POST')) listen_flow = ListenHTTP(8080) >> LogAttribute() >> PutFile('/tmp/output') with DockerTestCluster(SingleFileOutputValidator('test')) as cluster: cluster.put_test_data('test') cluster.deploy_flow(listen_flow, name='minifi-listen') cluster.deploy_flow(invoke_flow, name='minifi-invoke') > assert cluster.check_output() E assert False E+ where False = >() E+where > = .check_output ../docker/test/integration/test_http.py:36: AssertionError == 1 failed, 2 passed in 22.31 seconds === make[3]: *** [CMakeFiles/docker-verify] Error 1 make[2]: *** [CMakeFiles/docker-verify.dir/all] Error 2 make[1]: *** [CMakeFiles/docker-verify.dir/rule] Error 2 make: *** [docker-verify] Error 2 ``` > Create integration tests for cpp docker image > - > > Key: MINIFI-350 > URL: https://issues.apache.org/jira/browse/MINIFI-350 > Project: Apache NiFi MiNiFi > Issue Type: Bug > Components: C++ >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Create a unit test suite which uses compose or docker commands to set up > minifi using the docker image and send data through test flows. Consider > looking into existing test frameworks/harnesses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFI-350) Create integration tests for cpp docker image
[ https://issues.apache.org/jira/browse/MINIFI-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16143255#comment-16143255 ] ASF GitHub Bot commented on MINIFI-350: --- Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/126#discussion_r135423290 --- Diff: README.md --- @@ -124,6 +133,8 @@ $ brew install cmake \ doxygen $ brew install curl $ brew link curl --force +$ # (Optional) for building docker image/running system integration tests +$ # Install docker using instructions at https://docs.docker.com/docker-for-mac/install/ --- End diff -- Should include an OS X option for virtualenv. The pip route worked for me. Homebrew had issues. > Create integration tests for cpp docker image > - > > Key: MINIFI-350 > URL: https://issues.apache.org/jira/browse/MINIFI-350 > Project: Apache NiFi MiNiFi > Issue Type: Bug > Components: C++ >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Create a unit test suite which uses compose or docker commands to set up > minifi using the docker image and send data through test flows. Consider > looking into existing test frameworks/harnesses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFI-350) Create integration tests for cpp docker image
[ https://issues.apache.org/jira/browse/MINIFI-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16143152#comment-16143152 ] ASF GitHub Bot commented on MINIFI-350: --- Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/126 reviewing > Create integration tests for cpp docker image > - > > Key: MINIFI-350 > URL: https://issues.apache.org/jira/browse/MINIFI-350 > Project: Apache NiFi MiNiFi > Issue Type: Bug > Components: C++ >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Create a unit test suite which uses compose or docker commands to set up > minifi using the docker image and send data through test flows. Consider > looking into existing test frameworks/harnesses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFI-350) Create integration tests for cpp docker image
[ https://issues.apache.org/jira/browse/MINIFI-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120677#comment-16120677 ] ASF GitHub Bot commented on MINIFI-350: --- GitHub user achristianson opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/126 MINIFI-350 added pytest-based system integration test framework and initial test cases Added a test framework which enables automated tests suits for the docker image as well as simulation/system integration testing of multiple-host flows and/or integration with other non-MiNiFi components. Currently, the HTTP tests are failing due to https://issues.apache.org/jira/browse/MINIFI-369, but once the fix for that is merged, the test suite will pass. You can merge this pull request into a Git repository by running: $ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFI-350 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/126.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #126 commit a9b4be93cf2cec21b314854776c412f7d0a2282c Author: Andrew ChristiansonDate: 2017-07-13T14:42:35Z MINIFI-350 added pytest-based system integration test framework and initial test cases > Create integration tests for cpp docker image > - > > Key: MINIFI-350 > URL: https://issues.apache.org/jira/browse/MINIFI-350 > Project: Apache NiFi MiNiFi > Issue Type: Bug > Components: C++ >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Create a unit test suite which uses compose or docker commands to set up > minifi using the docker image and send data through test flows. Consider > looking into existing test frameworks/harnesses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFI-350) Create integration tests for cpp docker image
[ https://issues.apache.org/jira/browse/MINIFI-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120596#comment-16120596 ] Andrew Christianson commented on MINIFI-350: The HTTP integration tests are currently failing, but will pass as soon as MINIFI-369 is merged in. > Create integration tests for cpp docker image > - > > Key: MINIFI-350 > URL: https://issues.apache.org/jira/browse/MINIFI-350 > Project: Apache NiFi MiNiFi > Issue Type: Bug > Components: C++ >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Create a unit test suite which uses compose or docker commands to set up > minifi using the docker image and send data through test flows. Consider > looking into existing test frameworks/harnesses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFI-350) Create integration tests for cpp docker image
[ https://issues.apache.org/jira/browse/MINIFI-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111065#comment-16111065 ] Andrew Christianson commented on MINIFI-350: Initial pytest tests show much better portability and flexibility. Currently migrating from using HTTP to input/output test data to using volumes/FS events because communicating from host to container requires special firewall rules and also introduces race conditions (waiting for HTTP server to start up). > Create integration tests for cpp docker image > - > > Key: MINIFI-350 > URL: https://issues.apache.org/jira/browse/MINIFI-350 > Project: Apache NiFi MiNiFi > Issue Type: Bug > Components: C++ >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Create a unit test suite which uses compose or docker commands to set up > minifi using the docker image and send data through test flows. Consider > looking into existing test frameworks/harnesses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFI-350) Create integration tests for cpp docker image
[ https://issues.apache.org/jira/browse/MINIFI-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109125#comment-16109125 ] Andrew Christianson commented on MINIFI-350: Implemented an initial system in shell/bats. When switching between mac and linux environments, it is clear that there are substantial differences in the shell environments/tools available. For this reason, I think it's worth writing an implementation in python. Python is one of two languages that has an official/full docker SDK, with the other being go, and with virtual environments, it should meet portability and repeatability requirements for a test system. > Create integration tests for cpp docker image > - > > Key: MINIFI-350 > URL: https://issues.apache.org/jira/browse/MINIFI-350 > Project: Apache NiFi MiNiFi > Issue Type: Bug > Components: C++ >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Create a unit test suite which uses compose or docker commands to set up > minifi using the docker image and send data through test flows. Consider > looking into existing test frameworks/harnesses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)