[GitHub] nifi issue #2872: NIFI-5318 Implement NiFi test harness: initial commit of n...
Github user peter-gergely-horvath commented on the issue: https://github.com/apache/nifi/pull/2872 Hi @MikeThomsen sorry for the delay, I have been just very busy. It turns out there have been some changes in the flow XML format, which I have fixed against the latest GA version: now all of the tests are executed properly. I have also introduced strict version checking: the NiFi version referenced in the flow file is checked against the actual version of the NiFi distribution used so that we can avoid confusion in the future. If a user tries to run the test harness with a flow created using a different version of NiFi than the one actually used, this will fail fast, preventing a non functioning installation leaving the user being confused why it does not work properly. ---
[jira] [Commented] (NIFI-5318) Implement NiFi test harness
[ https://issues.apache.org/jira/browse/NIFI-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16657991#comment-16657991 ] ASF GitHub Bot commented on NIFI-5318: -- Github user peter-gergely-horvath commented on the issue: https://github.com/apache/nifi/pull/2872 Hi @MikeThomsen sorry for the delay, I have been just very busy. It turns out there have been some changes in the flow XML format, which I have fixed against the latest GA version: now all of the tests are executed properly. I have also introduced strict version checking: the NiFi version referenced in the flow file is checked against the actual version of the NiFi distribution used so that we can avoid confusion in the future. If a user tries to run the test harness with a flow created using a different version of NiFi than the one actually used, this will fail fast, preventing a non functioning installation leaving the user being confused why it does not work properly. > Implement NiFi test harness > --- > > Key: NIFI-5318 > URL: https://issues.apache.org/jira/browse/NIFI-5318 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Peter Horvath >Priority: Major > > Currently, it is not really possible to automatically test the behaviour of a > specific NiFi flow and make unit test type asserts if it works as expected. > For example, if the expected behaviour of a NiFi flow is that a file placed > to a specific directory will trigger some operation after which some output > file will appear at another directory, once currently can only do one thing: > test the NiFi flow manually. > Manual testing is especially hard to manage if a NiFi flow is being actively > developed: any change to a complex, existing NiFi flow might require a lot of > manual testing just to ensure there are no regressions introduced. > Some kind of Java API that allows managing a NiFi instance and manipulating > flow deployments like for example, [Codehaus > Cargo|]https://codehaus-cargo.github.io/] would be of great help. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4558) Populate default keystore/truststore types in SSLContextService
[
https://issues.apache.org/jira/browse/NIFI-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658003#comment-16658003
]
ASF subversion and git services commented on NIFI-4558:
---
Commit 08c54bc48d14d559ce38970ecde0b9583ecc96a5 in nifi's branch
refs/heads/master from thenatog
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=08c54bc ]
Revert "NIFI-4558 - Set JKS as the default keystore type and truststore type."
This reverts commit 2b9fdf8415eb2b033bfc63a6c5254bba3e69ecdd.
This closes #3097.
> Populate default keystore/truststore types in SSLContextService
> ---
>
> Key: NIFI-4558
> URL: https://issues.apache.org/jira/browse/NIFI-4558
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.4.0
>Reporter: Andy LoPresto
>Assignee: Nathan Gough
>Priority: Trivial
> Labels: controller_services, jks, keystore, pkcs12, security,
> ssl, tls, truststore, ux
> Fix For: 1.8.0
>
>
> The keystore and truststore type is almost always JKS as opposed to PKCS12
> when creating SSL controller services. Both {{StandardSSLContextService}} and
> {{StandardRestrictedSSLContextService}} should have those fields
> autopopulated to JKS, saving 2-4 clicks per instantiation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #3097: Revert "NIFI-4558 - Set JKS as the default keystore...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/3097 ---
[GitHub] nifi issue #3097: Revert "NIFI-4558 - Set JKS as the default keystore type a...
Github user jtstorck commented on the issue: https://github.com/apache/nifi/pull/3097 +1, merged to master. I verified that the changes have been reverted, and that the full build with tests and contrib-check is successful. ---
[jira] [Commented] (NIFI-4558) Populate default keystore/truststore types in SSLContextService
[
https://issues.apache.org/jira/browse/NIFI-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658005#comment-16658005
]
ASF GitHub Bot commented on NIFI-4558:
--
Github user jtstorck commented on the issue:
https://github.com/apache/nifi/pull/3097
+1, merged to master.
I verified that the changes have been reverted, and that the full build
with tests and contrib-check is successful.
> Populate default keystore/truststore types in SSLContextService
> ---
>
> Key: NIFI-4558
> URL: https://issues.apache.org/jira/browse/NIFI-4558
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.4.0
>Reporter: Andy LoPresto
>Assignee: Nathan Gough
>Priority: Trivial
> Labels: controller_services, jks, keystore, pkcs12, security,
> ssl, tls, truststore, ux
> Fix For: 1.8.0
>
>
> The keystore and truststore type is almost always JKS as opposed to PKCS12
> when creating SSL controller services. Both {{StandardSSLContextService}} and
> {{StandardRestrictedSSLContextService}} should have those fields
> autopopulated to JKS, saving 2-4 clicks per instantiation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4558) Populate default keystore/truststore types in SSLContextService
[
https://issues.apache.org/jira/browse/NIFI-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658004#comment-16658004
]
ASF GitHub Bot commented on NIFI-4558:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/3097
> Populate default keystore/truststore types in SSLContextService
> ---
>
> Key: NIFI-4558
> URL: https://issues.apache.org/jira/browse/NIFI-4558
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.4.0
>Reporter: Andy LoPresto
>Assignee: Nathan Gough
>Priority: Trivial
> Labels: controller_services, jks, keystore, pkcs12, security,
> ssl, tls, truststore, ux
> Fix For: 1.8.0
>
>
> The keystore and truststore type is almost always JKS as opposed to PKCS12
> when creating SSL controller services. Both {{StandardSSLContextService}} and
> {{StandardRestrictedSSLContextService}} should have those fields
> autopopulated to JKS, saving 2-4 clicks per instantiation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-4558) Populate default keystore/truststore types in SSLContextService
[
https://issues.apache.org/jira/browse/NIFI-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Storck updated NIFI-4558:
--
Fix Version/s: (was: 1.8.0)
> Populate default keystore/truststore types in SSLContextService
> ---
>
> Key: NIFI-4558
> URL: https://issues.apache.org/jira/browse/NIFI-4558
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.4.0
>Reporter: Andy LoPresto
>Assignee: Nathan Gough
>Priority: Trivial
> Labels: controller_services, jks, keystore, pkcs12, security,
> ssl, tls, truststore, ux
>
> The keystore and truststore type is almost always JKS as opposed to PKCS12
> when creating SSL controller services. Both {{StandardSSLContextService}} and
> {{StandardRestrictedSSLContextService}} should have those fields
> autopopulated to JKS, saving 2-4 clicks per instantiation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4558) Populate default keystore/truststore types in SSLContextService
[
https://issues.apache.org/jira/browse/NIFI-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658006#comment-16658006
]
Jeff Storck commented on NIFI-4558:
---
Removed fix version of 1.8.0, with the changes for this JIRA having been
reverted in master.
> Populate default keystore/truststore types in SSLContextService
> ---
>
> Key: NIFI-4558
> URL: https://issues.apache.org/jira/browse/NIFI-4558
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.4.0
>Reporter: Andy LoPresto
>Assignee: Nathan Gough
>Priority: Trivial
> Labels: controller_services, jks, keystore, pkcs12, security,
> ssl, tls, truststore, ux
>
> The keystore and truststore type is almost always JKS as opposed to PKCS12
> when creating SSL controller services. Both {{StandardSSLContextService}} and
> {{StandardRestrictedSSLContextService}} should have those fields
> autopopulated to JKS, saving 2-4 clicks per instantiation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5719) FetchFile can fail to move original file on completion but still route to success
[ https://issues.apache.org/jira/browse/NIFI-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Storck updated NIFI-5719: -- Fix Version/s: 1.8.0 > FetchFile can fail to move original file on completion but still route to > success > - > > Key: NIFI-5719 > URL: https://issues.apache.org/jira/browse/NIFI-5719 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.7.1 >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Fix For: 1.8.0 > > > Currently before fetching the file there are some checks to ensure that the > completion strategy will succeed. The check for the move strategy ensures > that if the target directory exists that it is a directory and is writable. > The issue is that if the directory does not exist, then later during the > actual move we try to create it, but this could fail, primarily due to the > parent directory not being writable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-5446) Specify the file encoding option to UTF-8 in pom.xml
[
https://issues.apache.org/jira/browse/NIFI-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Storck updated NIFI-5446:
--
Fix Version/s: 1.8.0
> Specify the file encoding option to UTF-8 in pom.xml
>
>
> Key: NIFI-5446
> URL: https://issues.apache.org/jira/browse/NIFI-5446
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Takanobu Asanuma
>Assignee: Takanobu Asanuma
>Priority: Major
> Fix For: 1.8.0
>
>
> {{TestJacksonCSVRecordReader.testUTF8}} failed in my environment (CentOS 7).
> {noformat}
> [INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO] Running org.apache.nifi.csv.TestJacksonCSVRecordReader
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.272
> s <<< FAILURE! - in org.apache.nifi.csv.TestJacksonCSVRecordReader
> [ERROR] testUTF8(org.apache.nifi.csv.TestJacksonCSVRecordReader) Time
> elapsed: 1.007 s <<< FAILURE!
> org.junit.ComparisonFailure: expected:<[???]> but was:<[???]>
> at
> org.apache.nifi.csv.TestJacksonCSVRecordReader.testUTF8(TestJacksonCSVRecordReader.java:81)
> {noformat}
> After a bit of research, the problem seems to be same as
> [here|https://stackoverflow.com/questions/17656475/maven-source-encoding-in-utf-8-not-working].
> I specified the file-encoding option of UTF-8 in the command line arguments
> and the unit test succeeded.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5720) Release Apache NiFi 1.8.0
[ https://issues.apache.org/jira/browse/NIFI-5720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658039#comment-16658039 ] ASF subversion and git services commented on NIFI-5720: --- Commit 19bdd375c32c97e2b7dfd41e5ffe65f5e1eb2435 in nifi's branch refs/heads/NIFI-5720-RC2 from [~jtstorck] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=19bdd37 ] NIFI-5720-RC2 prepare release nifi-1.8.0-RC2 > Release Apache NiFi 1.8.0 > - > > Key: NIFI-5720 > URL: https://issues.apache.org/jira/browse/NIFI-5720 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Affects Versions: 1.8.0 >Reporter: Jeff Storck >Assignee: Jeff Storck >Priority: Blocker > Labels: release > Fix For: 1.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5720) Release Apache NiFi 1.8.0
[ https://issues.apache.org/jira/browse/NIFI-5720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658040#comment-16658040 ] ASF subversion and git services commented on NIFI-5720: --- Commit c96fa490acb98a9dfd228847d85cc14201dcb850 in nifi's branch refs/heads/NIFI-5720-RC2 from [~jtstorck] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=c96fa49 ] NIFI-5720-RC2 prepare for next development iteration > Release Apache NiFi 1.8.0 > - > > Key: NIFI-5720 > URL: https://issues.apache.org/jira/browse/NIFI-5720 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Affects Versions: 1.8.0 >Reporter: Jeff Storck >Assignee: Jeff Storck >Priority: Blocker > Labels: release > Fix For: 1.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3098: NIFI-5728 XML Writer to populate record tag name pr...
GitHub user bdesert opened a pull request: https://github.com/apache/nifi/pull/3098 NIFI-5728 XML Writer to populate record tag name properly All the changes are tested against - "Use Schema Text" (regression) - "AvroSchemaRegistry" Controller Service - "Hortonworks Schema Registry" Controller Service (docker). Also added Test Case to handle scenario when schema name in xxxRegistry Controller Service doesn't mach "name" field of the root record in schema text. --- Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/bdesert/nifi NIFI-5728_XMLWriter Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3098.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3098 commit 268ae74deeaf0228fb07cf50c3dd343ff9fb7e9f Author: Ed B Date: 2018-10-21T01:42:21Z NIFI-5728 XML Writer to populate record tag name properly ---
[jira] [Commented] (NIFI-5728) Inconsistent behavior in XMLRecordSetWriter for Root Record Tag
[
https://issues.apache.org/jira/browse/NIFI-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658047#comment-16658047
]
ASF GitHub Bot commented on NIFI-5728:
--
GitHub user bdesert opened a pull request:
https://github.com/apache/nifi/pull/3098
NIFI-5728 XML Writer to populate record tag name properly
All the changes are tested against
- "Use Schema Text" (regression)
- "AvroSchemaRegistry" Controller Service
- "Hortonworks Schema Registry" Controller Service (docker).
Also added Test Case to handle scenario when schema name in xxxRegistry
Controller Service doesn't mach "name" field of the root record in schema text.
---
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with NIFI- where is the JIRA number
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [x] Is your initial contribution a single, squashed commit?
### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn
-Pcontrib-check clean install at the root nifi folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file, including the main
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in
which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bdesert/nifi NIFI-5728_XMLWriter
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/3098.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3098
commit 268ae74deeaf0228fb07cf50c3dd343ff9fb7e9f
Author: Ed B
Date: 2018-10-21T01:42:21Z
NIFI-5728 XML Writer to populate record tag name properly
> Inconsistent behavior in XMLRecordSetWriter for Root Record Tag
> ---
>
> Key: NIFI-5728
> URL: https://issues.apache.org/jira/browse/NIFI-5728
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.7.1
>Reporter: Ed Berezitsky
>Assignee: Ed Berezitsky
>Priority: Major
>
> In XMLRecordSetWriter:
> When used with "Use Schema Text" and "Name of Record Tag" is empty (so record
> name should be used to wrapping XML tag) - works correctly.
> When used with Schema Registry and "Name of Record Tag" is empty - it doesn't
> write record name, but uses schema name instead. I believe this inconsistency
> is coming from the fact that when schema in a writer defined by "Use Schema
> Text" - it doesn't have a name and then name is taken from a record. But when
> it comes from registry - it simply uses schema idetifier:
> {code:java}
> recordSchema.getIdentifier().getName();
> {code}
> IMO root record name should be used in this case instead of schema name.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
