[GitHub] [nifi] dependabot[bot] commented on pull request #7183: Bump jetty-server from 9.4.51.v20230217 to 10.0.14
dependabot[bot] commented on PR #7183: URL: https://github.com/apache/nifi/pull/7183#issuecomment-1513994425 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting `@dependabot ignore this major version` or `@dependabot ignore this minor version`. If you change your mind, just re-open this PR and I'll resolve any conflicts on it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory closed pull request #7183: Bump jetty-server from 9.4.51.v20230217 to 10.0.14
exceptionfactory closed pull request #7183: Bump jetty-server from 9.4.51.v20230217 to 10.0.14 URL: https://github.com/apache/nifi/pull/7183 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] dependabot[bot] opened a new pull request, #7183: Bump jetty-server from 9.4.51.v20230217 to 10.0.14
dependabot[bot] opened a new pull request, #7183: URL: https://github.com/apache/nifi/pull/7183 Bumps [jetty-server](https://github.com/eclipse/jetty.project) from 9.4.51.v20230217 to 10.0.14. Release notes Sourced from https://github.com/eclipse/jetty.project/releases";>jetty-server's releases. 10.0.14 Special Thanks to the following Eclipse Jetty community members https://github.com/pzygielo";>@pzygielo (Piotrek Żygieło) https://github.com/jluehe";>@jluehe (jluehe) https://github.com/dzoech";>@dzoech (Dominik Zöchbauer) Changelog https://redirect.github.com/eclipse/jetty.project/issues/9344";>#9344 - Cleanup Multipart handling for CVE-2023-26048 https://redirect.github.com/eclipse/jetty.project/issues/9343";>#9343 - URI Host Mismatch with optional Compliance modes https://redirect.github.com/eclipse/jetty.project/issues/9339";>#9339 - Cleanup Cookie Cutter handling for CVE-2023-26049 https://redirect.github.com/eclipse/jetty.project/issues/9337";>#9337 - LowResourceMonitor.getReasons should include detailed reason instead of hard-coded message (https://github.com/jluehe";>@jluehe) https://redirect.github.com/eclipse/jetty.project/issues/9334";>#9334 - Better support for Cookie RFC 2965 compliance https://redirect.github.com/eclipse/jetty.project/issues/9285";>#9285 - ContextHandler sends redirect on BaseResponse instead of Wrapped Response object from Handler chain https://redirect.github.com/eclipse/jetty.project/issues/9283";>#9283 - Configurable Unsafe Host Header Behaviors https://redirect.github.com/eclipse/jetty.project/issues/9188";>#9188 - Log as info exceptions from server after sending stop with StopMojo. https://redirect.github.com/eclipse/jetty.project/issues/9183";>#9183 - ConnectHandler may close the connection instead of sending 200 OK https://redirect.github.com/eclipse/jetty.project/issues/9128";>#9128 - Do not execute any phase for maven plugin :start (https://github.com/pzygielo";>@pzygielo) https://redirect.github.com/eclipse/jetty.project/issues/9119";>#9119 - Wrong value of javax.servlet.forward.context_path attribute https://redirect.github.com/eclipse/jetty.project/issues/9092";>#9092 - Use ASM Bom https://redirect.github.com/eclipse/jetty.project/issues/9059";>#9059 - IteratingCallback not serializing close() and failed() https://redirect.github.com/eclipse/jetty.project/issues/9055";>#9055 - PathMappings optimizations https://redirect.github.com/eclipse/jetty.project/issues/7650";>#7650 - QueuedThreadPool: Stopped without executing or closing null (https://github.com/dzoech";>@dzoech) Dependencies https://redirect.github.com/eclipse/jetty.project/issues/9242";>#9242 - Bump infinispan-bom to 11.0.17.Final https://redirect.github.com/eclipse/jetty.project/issues/9359";>#9359 - Bump maven.version to 3.9.0 https://redirect.github.com/eclipse/jetty.project/issues/9102";>#9102 - Bump org.apache.aries.spifly.dynamic.bundle to 1.3.6 https://redirect.github.com/eclipse/jetty.project/issues/9098";>#9098 - Bump org.eclipse.osgi to 3.18.200 https://redirect.github.com/eclipse/jetty.project/issues/9106";>#9106 - Bump org.eclipse.osgi.services to 3.11.100 https://redirect.github.com/eclipse/jetty.project/issues/9097";>#9097 - Bump protostream to 4.6.0.Final https://redirect.github.com/eclipse/jetty.project/issues/9367";>#9367 - Bump tycho-p2-repository-plugin to 3.0.2 10.0.13 Special Thanks to the following Eclipse Jetty community members https://github.com/janvojt";>@janvojt (Jan Vojt) https://github.com/joschi";>@joschi (Jochen Schalanda) https://github.com/leonchen83";>@leonchen83 (Baoyi Chen) https://github.com/cowwoc";>@cowwoc (Gili Tzabari) https://github.com/Vlatombe";>@Vlatombe (Vincent Latombe) Changelog https://redirect.github.com/eclipse/jetty.project/issues/9006";>#9006 - WebSocket Message InputStream read() returns signed byte https://redirect.github.com/eclipse/jetty.project/issues/8913";>#8913 - Review Jetty XML syntax to allow calling JDK methods ... (truncated) Commits https://github.com/eclipse/jetty.project/commit/976721d0f3e903a243584d47870ad2f2c1bf9e55";>976721d Updating to version 10.0.14 https://github.com/eclipse/jetty.project/commit/b7075161d015ddce23fbf3db873d5f6b539f6a6b";>b707516 Fix osgi dependencies for update to org.eclipse.osgi.services. https://github.com/eclipse/jetty.project/commit/4d146412c8feac05c25d171b15c4f6ab4d42719b";>4d14641 Fix https://redirect.github.com/eclipse/jetty.project/issues/9334";>#9334 Cookie Compliance (https://redirect.github.com/eclipse/jetty.project/issues/9402";>#9402) https://github.com/eclipse/jetty.project/commit/f01d53895f8930e1ebc52c9d89944df14fe5d6f2";>f01d538 Merge pull request https://redirect.github.com/eclipse/jetty.project/issues/9380";>#9380 from eclipse/depen
[GitHub] [nifi] davis-anthony commented on pull request #6903: NIFI-11111 add option to output Elasticsearch error responses as FlowFile to PutElasticsearchJson and PutElasticsearchRecord
davis-anthony commented on PR #6903: URL: https://github.com/apache/nifi/pull/6903#issuecomment-1513872398 I've been struggling with the various PutElasticsearch processors and their different handling of responses/errors for a while now, and nothing really does what I would expect it to do. The only one that kind of works is PutElasticsearchHttp because at least that one places the "reason" attribute on the flowfile, but it doesn't handle all types of errors and it makes a bunch of assumptions as to what is retryable. What I really need is for the original documents that fail on a bulk index/create to be routed to failure or error and have the error message and status code added as flowfile attributes. This would enable custom routing logic based on status code and/or error message. I can't tell if that's what this PR is attempting to do. Is that what this PR does or should I post a new feature request for this? If it's just sending the error response documents as separate flowfiles I can see how some developers might find that useful, but it doesn't really help with rerouting the original document and providing error information. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] bbende commented on a diff in pull request #7092: NIFI-11327 Add Export/Import All - NiFi CLI - NiFi Registry
bbende commented on code in PR #7092: URL: https://github.com/apache/nifi/pull/7092#discussion_r1170134766 ## nifi-docs/src/main/asciidoc/toolkit-guide.adoc: ## @@ -768,6 +770,325 @@ This command migrates nifi.properties back from AWS_KMS to AES_GCM protection sc -v +[export_import_all_flows] +=== Export All Flows +You can use the `export-all-flows` to perform the following tasks: + +* List all the buckets +* For each bucket, list all flows +* For each flow, list all versions +* Export each version into a provided directory + +Running the command requires an `--outputDirectory` parameter. + +=== Import All Flows +You can use the `import-all-flows` to perform the following tasks: + +* List all files, representing a flow version, from a directory created by export-all-flows +* Create all the corresponding buckets +* Create all the corresponding flows +* Import all the corresponding flow versions + +Running the command requires 2 parameters: + +* `--input` parameter represents a directory to read files from +* `--skipExisting` parameter, configuring how to handle existing flow and flow version creation. +If true the flow and flow version creation will be skipped regardless of there are missing flow versions. +If false the missing flow versions will be created. The default value is true, skip creation. + +=== Usage +The input source for an import-all-flows command must be created by an export-all-flows command. +To avoid migration conflicts, no modification should be performed in the NiFi Registry during this activity. +Buckets and flows with the same name are considered equal. + +* Export all flow versions: + + ./bin/cli.sh registry export-all-flows -u http://localhost:18080 --outputDirectory "/my_dir/flow_exports" + +* Import all flow versions: + + ./bin/cli.sh registry import-all-flows -u http://localhost:18080 --input "/my_dir/flow_exports" --skipExisting false + +=== Expected behaviour +=== Use case 1: reconfiguring an existing NiFi Registry + +NiFi is connecting to NiFi Registry, the NiFi Registry does not change, only its configuration. +All the data will be created. + +1. Export versions: + + ./bin/cli.sh registry export-all-flows -u http://localhost:18080 --outputDirectory "/my_dir/flow_exports" + +2. Stop registry + +3. Switch provider + +4. Start registry + +5. Import versions + + ./bin/cli.sh registry import-all-flows -u http://localhost:18080 --input "/my_dir/flow_exports" --skipExisting true + + +=== Use case 2: data replication + +NiFi_1 is connecting to NiFi Registry_1 and NiFi_2 is connecting to NiFi Registry_2. + +For disaster recovery purposes the data from NiFi Registry_1 needs to be periodically replicated to NiFi Registry_2 via a scheduled job. + +The initial version of Nifi Registry_2 needs to be created by this tool. + +The missing buckets, flows and versions will be created. If bucket and flow exist the missing versions will be created. + +1. Export versions: + + ./bin/cli.sh registry export-all-flows -u http://nifi-registry-1:18080 --outputDirectory "/my_dir/flow_exports" + +2. Import versions: + + ./bin/cli.sh registry import-all-flows -u http://nifi-registry-2:18080 --input "/my_dir/flow_exports" --skipExisting false + + Review Comment: I think this is adding back a bunch of sections that were removed on main. Need to update so that this is only adding your new section, and everything afterwards starting with `File Manager` down to `tls_toolkit` is removed. ## nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/command/registry/flow/ImportAllFlows.java: ## @@ -0,0 +1,331 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.toolkit.cli.impl.command.registry.flow; + +import com.fasterxml.jackson.databind.ObjectMapper; +import org.apache.commons.cli.MissingOptionException; +import org.apache.commons.cli.ParseException; +import org.apache.commons.lang3.tuple.ImmutablePair; +import org.apache.commons.lang3.tuple.Pair; +import org.apache.curator.shaded.com.google.common.collect.ComparisonChain; +import org.apache.nifi.flow.VersionedFlowCoordinates; +import org.apache.nifi.flow.VersionedProcessGroup; +import org.apach
[jira] [Resolved] (MINIFICPP-2105) Throw on invalid output format in ConsumeWindowsEventLog
[ https://issues.apache.org/jira/browse/MINIFICPP-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink resolved MINIFICPP-2105. Resolution: Fixed https://github.com/apache/nifi-minifi-cpp/commit/d23e59615ffb64d1e09cd95c896939a814a011cd > Throw on invalid output format in ConsumeWindowsEventLog > > > Key: MINIFICPP-2105 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2105 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Adam Debreceni >Assignee: Adam Debreceni >Priority: Major > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2093) Update default features/extensions
[ https://issues.apache.org/jira/browse/MINIFICPP-2093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink resolved MINIFICPP-2093. Resolution: Fixed https://github.com/apache/nifi-minifi-cpp/commit/8f64ed4edd182dad9e590ebe0e906fe847049351 > Update default features/extensions > -- > > Key: MINIFICPP-2093 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2093 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > ...and disable kubernetes on windows, to avoid a cmake error -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] martinzink closed pull request #1559: MINIFICPP-2105 - Validate ConsumeWindowsEventLogs output/json format
martinzink closed pull request #1559: MINIFICPP-2105 - Validate ConsumeWindowsEventLogs output/json format URL: https://github.com/apache/nifi-minifi-cpp/pull/1559 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink closed pull request #1549: MINIFICPP-2093 Update default features/extensions
martinzink closed pull request #1549: MINIFICPP-2093 Update default features/extensions URL: https://github.com/apache/nifi-minifi-cpp/pull/1549 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7181: NIFI-11461 Improve User and Group Tenants Search
exceptionfactory commented on code in PR #7181:
URL: https://github.com/apache/nifi/pull/7181#discussion_r1170473571
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java:
##
@@ -4461,6 +4465,62 @@ public Set getUsers() {
.collect(Collectors.toSet());
}
+/**
+ * Search for User and Group Tenants with optimized conversion from
specific objects to Tenant objects
+ *
+ * @param query Search query where null or empty returns unfiltered results
+ * @return Tenants Entity containing zero or more matching Users and Groups
+ */
+@Override
+public TenantsEntity searchTenants(final String query) {
+final PermissionsDTO permissions =
dtoFactory.createPermissionsDto(authorizableLookup.getTenant());
+
+final Set usersFound = userDAO.getUsers()
+.stream()
+.filter(user -> isMatched(user.getIdentity(), query))
+.map(user -> createTenantEntity(user, permissions))
+.collect(Collectors.toSet());
+
+final Set userGroupsFound = userGroupDAO.getUserGroups()
+.stream()
+.filter(userGroup -> isMatched(userGroup.getName(), query))
+.map(userGroup -> createTenantEntity(userGroup, permissions))
+.collect(Collectors.toSet());
+
+final TenantsEntity tenantsEntity = new TenantsEntity();
+tenantsEntity.setUsers(usersFound);
+tenantsEntity.setUserGroups(userGroupsFound);
+return tenantsEntity;
+}
+
+private boolean isMatched(final String label, final String query) {
+return StringUtils.isEmpty(query) || containsIgnoreCase(label, query);
+}
+
+private TenantEntity createTenantEntity(final User user, final
PermissionsDTO permissions) {
+final TenantDTO tenant = dtoFactory.createTenantDTO(user);
+return createTenantEntity(tenant, permissions);
+}
+
+private TenantEntity createTenantEntity(final Group userGroup, final
PermissionsDTO permissions) {
Review Comment:
Thanks for pointer to `EntityFactory`, I pushed an update removing the
private methods and adjusting the stream function to make use of
`EntityFactory.createTenantEntity()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-11468) Attribute values are converted to sensitive values after NiFi upgrades, when custom nars are used
John Wise created NIFI-11468: Summary: Attribute values are converted to sensitive values after NiFi upgrades, when custom nars are used Key: NIFI-11468 URL: https://issues.apache.org/jira/browse/NIFI-11468 Project: Apache NiFi Issue Type: Bug Components: Core Framework Reporter: John Wise We've recently started experiencing an issue where the attribute values in ~1/4 to 1/3 of the processors in a given flow are changed from plain text to sensitive values. Those values are then non-recoverable, and must be manually deleted and re-created. We narrowed this down to some dependencies on previous NiFi versions in our custom nars. If I exclude the custom nars until after the upgrade is done & the NiFi service is run for the first time after the upgrade, then the all of the standard NiFi processor upgrade versions as expected. Once I re-add the custom nars, those with previous NiFi dependencies will convert attribute values from plain text to sensitive values. This seems to be a bug in the processor version upgrade logic. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (NIFI-11464) importing a versioned flow with a nested versioned flow shows nested versioned flow with invalid controller services.
[
https://issues.apache.org/jira/browse/NIFI-11464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713686#comment-17713686
]
Patrick A. Mol edited comment on NIFI-11464 at 4/18/23 5:35 PM:
(your comment above came in while I was writing the below)
I am using a single NiFi instance with a development PG and a production PG.
The problem is not that the import into a process group in the production PG
itself does not work,
the problem is that the result is a *tenant_flow_T* flow with an invalid
{*}reusable_flow_R{*}.
Caused by the nested versioned flow getting the controller service identifiers
from the snapshot in the registry.
There is no post-import resolution process after import of the *tenant_flow_T*
to fix this.
And imho it should work irrespective of the NiFi instance the *tenant_flow_T*
gets imported in with the instances sharing the same registry.
Maybe a bad analogy but here goes.
A.
You have a developed a reusable door and applied yellow paint, and stored it in
the "registry", so whenever you need a door, you can get one out of the
registry.
B.
You have a developed a blue car and you get a door from the registry.
When fitted to the car the door either automagically (the name-based
resolution) changes color to blue or you paint it blue.
According to regulation, the car should be one color to be "valid".
You like the completely blue car the way it is and store the "valid" car in the
"registry", so whenever you need a car, you can get one out of the registry.
C.
You found a customer, so you pull a car out of the registry.
It turns out to be "invalid". The car is blue, but the door is yellow.
was (Author: JIRAUSER299819):
I am using a single NiFi instance with a development PG and a production PG.
The problem is not that the import into a process group in the production PG
itself does not work,
the problem is that the result is a *tenant_flow_T* flow with an invalid
{*}reusable_flow_R{*}.
Caused by the nested versioned flow getting the controller service identifiers
from the snapshot in the registry.
There is no post-import resolution process after import of the *tenant_flow_T*
to fix this.
And imho it should work irrespective of the NiFi instance the *tenant_flow_T*
gets imported in with the instances sharing the same registry.
Maybe a bad analogy but here goes.
A.
You have a developed a reusable door and applied yellow paint, and stored it in
the "registry", so whenever you need a door, you can get one out of the
registry.
B.
You have a developed a blue car and you get a door from the registry.
When fitted to the car the door either automagically (the name-based
resolution) changes color to blue or you paint it blue.
According to regulation, the car should be one color to be "valid".
You like the completely blue car the way it is and store the "valid" car in the
"registry", so whenever you need a car, you can get one out of the registry.
C.
You found a customer, so you pull a car out of the registry.
It turns out to be "invalid". The car is blue, but the door is yellow.
> importing a versioned flow with a nested versioned flow shows nested
> versioned flow with invalid controller services.
> -
>
> Key: NIFI-11464
> URL: https://issues.apache.org/jira/browse/NIFI-11464
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Affects Versions: 1.21.0
> Environment: nifi 1.21.0 and nifi registry 1.21.0 (on ubuntu 20.04)
>Reporter: Patrick A. Mol
>Priority: Major
> Attachments: exported_flow_versions_pretty.zip,
> image-2023-04-17-17-34-08-898.png, image-2023-04-17-18-06-16-102.png,
> nested_versioned_flow_issue.xml, screenprints_reproduction_steps.zip
>
>
> When a flow (reusable_flow_Q) has controller services inherited from the
> hierarchy (process group reusables) and a version of the flow is stored, the
> flow version contains the references to these external controller services
> (as seen in an exported flow version [see below]).
> When this versioned flow is imported in another flow (tenant_flows) the
> controller services need to be reset to the controller services in the new
> hierarchy.
> When we have a working flow with the nested versioned flow ready in
> development we can check this flow into version control.
> When we then deploy the flow in production, the nested versioned flow shows
> up with invalid components. It shows the external controller service
> identifiers as stored in the flow version.
> When we then go back to development version of tenant_flows and make a minor
> change to the nested versioned flow reusable_flow_Q and commit this change to
> version control.
> Due to this version change,
[jira] [Commented] (NIFI-11464) importing a versioned flow with a nested versioned flow shows nested versioned flow with invalid controller services.
[
https://issues.apache.org/jira/browse/NIFI-11464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713686#comment-17713686
]
Patrick A. Mol commented on NIFI-11464:
---
I am using a single NiFi instance with a development PG and a production PG.
The problem is not that the import into a process group in the production PG
itself does not work,
the problem is that the result is a *tenant_flow_T* flow with an invalid
{*}reusable_flow_R{*}.
Caused by the nested versioned flow getting the controller service identifiers
from the snapshot in the registry.
There is no post-import resolution process after import of the *tenant_flow_T*
to fix this.
And imho it should work irrespective of the NiFi instance the *tenant_flow_T*
gets imported in with the instances sharing the same registry.
Maybe a bad analogy but here goes.
A.
You have a developed a reusable door and applied yellow paint, and stored it in
the "registry", so whenever you need a door, you can get one out of the
registry.
B.
You have a developed a blue car and you get a door from the registry.
When fitted to the car the door either automagically (the name-based
resolution) changes color to blue or you paint it blue.
According to regulation, the car should be one color to be "valid".
You like the completely blue car the way it is and store the "valid" car in the
"registry", so whenever you need a car, you can get one out of the registry.
C.
You found a customer, so you pull a car out of the registry.
It turns out to be "invalid". The car is blue, but the door is yellow.
> importing a versioned flow with a nested versioned flow shows nested
> versioned flow with invalid controller services.
> -
>
> Key: NIFI-11464
> URL: https://issues.apache.org/jira/browse/NIFI-11464
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Affects Versions: 1.21.0
> Environment: nifi 1.21.0 and nifi registry 1.21.0 (on ubuntu 20.04)
>Reporter: Patrick A. Mol
>Priority: Major
> Attachments: exported_flow_versions_pretty.zip,
> image-2023-04-17-17-34-08-898.png, image-2023-04-17-18-06-16-102.png,
> nested_versioned_flow_issue.xml, screenprints_reproduction_steps.zip
>
>
> When a flow (reusable_flow_Q) has controller services inherited from the
> hierarchy (process group reusables) and a version of the flow is stored, the
> flow version contains the references to these external controller services
> (as seen in an exported flow version [see below]).
> When this versioned flow is imported in another flow (tenant_flows) the
> controller services need to be reset to the controller services in the new
> hierarchy.
> When we have a working flow with the nested versioned flow ready in
> development we can check this flow into version control.
> When we then deploy the flow in production, the nested versioned flow shows
> up with invalid components. It shows the external controller service
> identifiers as stored in the flow version.
> When we then go back to development version of tenant_flows and make a minor
> change to the nested versioned flow reusable_flow_Q and commit this change to
> version control.
> Due to this version change, we need to also commit the changes for the
> tenant_flows process group.
> When we now go back to production, and import this new version of
> tenant_flows, the nested versioned flow reusable_flow_Q does not have invalid
> controller services.
> If you have several flows under development using the same reusable
> components,
> you will likely end up with invalid components after import.
> Depending on the amount of versioned flows used, it could be a lot of work.
> It could also lead to issues when using the ExecuteStateless processor.
> Please see attached template nested_version_flow_issue.xml for a starting
> point to reproduce the issue. It contains the steps.
> Screenprints are attached in a zip file show the process and diagnosis.
> Controller services identifiers in version 2.
> {code:java}
> $ fgrep -C 4 reusables_avro reusable_flow_Q.json.pretty
> "controllerServices": [
> {
> "identifier": "dc884171-4d75-3854-8604-afab91bd0e60",
> "instanceIdentifier": "8f647d06-0187-1000-4be9-14a61f55d904",
> "name": "reusables_avro_reader",
> "comments": "",
> "type": "org.apache.nifi.avro.AvroReader",
> "bundle": {
> "group": "org.apache.nifi",
> --
> },
> {
> "identifier": "b512b238-cdee-3642-b5cb-0c98d30dd133",
> "instanceIdentifier": "8f64f2c6-0187-1000-7557-ca63c88054dd",
> "name": "reusables_avro_writer",
> "comments": "",
> "type": "org.apache.nifi.avro.AvroRecordSetWriter",
>
[GitHub] [nifi] mtien-apache commented on pull request #7174: [NIFIF-11433] update angular, d3, moment, slickgrid, and jquery depen…
mtien-apache commented on PR #7174: URL: https://github.com/apache/nifi/pull/7174#issuecomment-1513535831 Reviewing... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11464) importing a versioned flow with a nested versioned flow shows nested versioned flow with invalid controller services.
[
https://issues.apache.org/jira/browse/NIFI-11464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713665#comment-17713665
]
Bryan Bende commented on NIFI-11464:
I think I do see one problem in the logic...
The logic for resolving the external services happens before processing nested
versioned flows. So say you are importing "Parent PG" with inner "Child PG"
that is also under version control, at the time it resolves the external
services, it only has "Parent PG" with the placeholder for "Child PG", so
anything that needs to be resolved inside "Child PG" is not happening.
> importing a versioned flow with a nested versioned flow shows nested
> versioned flow with invalid controller services.
> -
>
> Key: NIFI-11464
> URL: https://issues.apache.org/jira/browse/NIFI-11464
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Affects Versions: 1.21.0
> Environment: nifi 1.21.0 and nifi registry 1.21.0 (on ubuntu 20.04)
>Reporter: Patrick A. Mol
>Priority: Major
> Attachments: exported_flow_versions_pretty.zip,
> image-2023-04-17-17-34-08-898.png, image-2023-04-17-18-06-16-102.png,
> nested_versioned_flow_issue.xml, screenprints_reproduction_steps.zip
>
>
> When a flow (reusable_flow_Q) has controller services inherited from the
> hierarchy (process group reusables) and a version of the flow is stored, the
> flow version contains the references to these external controller services
> (as seen in an exported flow version [see below]).
> When this versioned flow is imported in another flow (tenant_flows) the
> controller services need to be reset to the controller services in the new
> hierarchy.
> When we have a working flow with the nested versioned flow ready in
> development we can check this flow into version control.
> When we then deploy the flow in production, the nested versioned flow shows
> up with invalid components. It shows the external controller service
> identifiers as stored in the flow version.
> When we then go back to development version of tenant_flows and make a minor
> change to the nested versioned flow reusable_flow_Q and commit this change to
> version control.
> Due to this version change, we need to also commit the changes for the
> tenant_flows process group.
> When we now go back to production, and import this new version of
> tenant_flows, the nested versioned flow reusable_flow_Q does not have invalid
> controller services.
> If you have several flows under development using the same reusable
> components,
> you will likely end up with invalid components after import.
> Depending on the amount of versioned flows used, it could be a lot of work.
> It could also lead to issues when using the ExecuteStateless processor.
> Please see attached template nested_version_flow_issue.xml for a starting
> point to reproduce the issue. It contains the steps.
> Screenprints are attached in a zip file show the process and diagnosis.
> Controller services identifiers in version 2.
> {code:java}
> $ fgrep -C 4 reusables_avro reusable_flow_Q.json.pretty
> "controllerServices": [
> {
> "identifier": "dc884171-4d75-3854-8604-afab91bd0e60",
> "instanceIdentifier": "8f647d06-0187-1000-4be9-14a61f55d904",
> "name": "reusables_avro_reader",
> "comments": "",
> "type": "org.apache.nifi.avro.AvroReader",
> "bundle": {
> "group": "org.apache.nifi",
> --
> },
> {
> "identifier": "b512b238-cdee-3642-b5cb-0c98d30dd133",
> "instanceIdentifier": "8f64f2c6-0187-1000-7557-ca63c88054dd",
> "name": "reusables_avro_writer",
> "comments": "",
> "type": "org.apache.nifi.avro.AvroRecordSetWriter",
> "bundle": {
> "group": "org.apache.nifi",
> $ head -15 reusable_flow_Q-version-2.json.pretty
> {
> "externalControllerServices": {
> "dc884171-4d75-3854-8604-afab91bd0e60": {
> "identifier": "dc884171-4d75-3854-8604-afab91bd0e60",
> "name": "reusables_avro_reader"
> },
> "b512b238-cdee-3642-b5cb-0c98d30dd133": {
> "identifier": "b512b238-cdee-3642-b5cb-0c98d30dd133",
> "name": "reusables_avro_writer"
> }
> },
> "flowContents": {
> "comments": "used to perform Q ...",
> "componentType": "PROCESS_GROUP",
> "connections": [
> {code}
> Controller services identifiers with version 3 committed in process group
> "tenant_flows".
> {code:java}
> pmo@hpmo:~/Documents.local/nested_versioned_flows_controller_issue$ fgrep -C
> 4 tenant_flow_avro tenant_flows-version-1.json.pretty
> ],
> "groupIdentifier": "a984831b-8587-3e17-bbbc-ef4b85c3898d",
> "identifier": "5d9d
[GitHub] [nifi] scottyaslan commented on a diff in pull request #7174: [NIFIF-11433] update angular, d3, moment, slickgrid, and jquery depen…
scottyaslan commented on code in PR #7174:
URL: https://github.com/apache/nifi/pull/7174#discussion_r1170338240
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-component-version.js:
##
@@ -96,9 +96,9 @@
*/
var resetDialog = function () {
// clear the versions
-var versions = versionMap.keys();
+var versions = Array.from(versionMap.keys());
$.each(versions, function (_, version) {
-versionMap.remove(version);
+versionMap['delete'](version);
Review Comment:
The yui compressor does not like `.delete()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] scottyaslan commented on a diff in pull request #7174: [NIFIF-11433] update angular, d3, moment, slickgrid, and jquery depen…
scottyaslan commented on code in PR #7174: URL: https://github.com/apache/nifi/pull/7174#discussion_r1170335400 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-processor.js: ## @@ -69,7 +72,7 @@ // --- // cache for components that are added/removed from the canvas -// --- +// r--- Review Comment: Yes, lol. Shane found where it goes! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] scottyaslan commented on a diff in pull request #7174: [NIFIF-11433] update angular, d3, moment, slickgrid, and jquery depen…
scottyaslan commented on code in PR #7174:
URL: https://github.com/apache/nifi/pull/7174#discussion_r1170334993
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-processor.js:
##
@@ -1140,9 +1188,9 @@
*/
get: function (id) {
if (nfCommon.isUndefined(id)) {
-return processorMap.values();
+return Array.from(processorMap.values());
} else {
-return processorMap.get(id);
+return processoMap.get(id);
Review Comment:
You found the misplaced 'r'!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] mcgilman commented on a diff in pull request #7181: NIFI-11461 Improve User and Group Tenants Search
mcgilman commented on code in PR #7181:
URL: https://github.com/apache/nifi/pull/7181#discussion_r1170313272
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java:
##
@@ -4461,6 +4465,62 @@ public Set getUsers() {
.collect(Collectors.toSet());
}
+/**
+ * Search for User and Group Tenants with optimized conversion from
specific objects to Tenant objects
+ *
+ * @param query Search query where null or empty returns unfiltered results
+ * @return Tenants Entity containing zero or more matching Users and Groups
+ */
+@Override
+public TenantsEntity searchTenants(final String query) {
+final PermissionsDTO permissions =
dtoFactory.createPermissionsDto(authorizableLookup.getTenant());
+
+final Set usersFound = userDAO.getUsers()
+.stream()
+.filter(user -> isMatched(user.getIdentity(), query))
+.map(user -> createTenantEntity(user, permissions))
+.collect(Collectors.toSet());
+
+final Set userGroupsFound = userGroupDAO.getUserGroups()
+.stream()
+.filter(userGroup -> isMatched(userGroup.getName(), query))
+.map(userGroup -> createTenantEntity(userGroup, permissions))
+.collect(Collectors.toSet());
+
+final TenantsEntity tenantsEntity = new TenantsEntity();
+tenantsEntity.setUsers(usersFound);
+tenantsEntity.setUserGroups(userGroupsFound);
+return tenantsEntity;
+}
+
+private boolean isMatched(final String label, final String query) {
+return StringUtils.isEmpty(query) || containsIgnoreCase(label, query);
+}
+
+private TenantEntity createTenantEntity(final User user, final
PermissionsDTO permissions) {
+final TenantDTO tenant = dtoFactory.createTenantDTO(user);
+return createTenantEntity(tenant, permissions);
+}
+
+private TenantEntity createTenantEntity(final Group userGroup, final
PermissionsDTO permissions) {
Review Comment:
Our `EntityFactory` already contains methods for creating a `TenantEntity`
given a `tenant`, `revision`, and `permissions`. Can we use that in favor of
these new methods?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (MINIFICPP-2102) Compile failure with -ENABLE_LUA_SCRIPTING on aarch64
[ https://issues.apache.org/jira/browse/MINIFICPP-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink resolved MINIFICPP-2102. Resolution: Workaround It seems the problem only exists due to the configuration of lua-devel yum package. It defines LUA_COMPAT_5_2 which tries to load LuaBitOp. We can workaround this be undefing the symbol or simply commenting out the #define LUA_COMPAT_5_2 in /usr/include/luaconf-aarch64.h > Compile failure with -ENABLE_LUA_SCRIPTING on aarch64 > - > > Key: MINIFICPP-2102 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2102 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > > Due to missing luaopen_bit32 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11464) importing a versioned flow with a nested versioned flow shows nested versioned flow with invalid controller services.
[
https://issues.apache.org/jira/browse/NIFI-11464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713655#comment-17713655
]
Bryan Bende commented on NIFI-11464:
{code:java}
The information that reusable_flow_R was configured to use controller service
with identifier T123 is lost at this point, i.e is not stored in the registry.
{code}
I don't see why this is a problem. The identifier of the service is really
irrelevant because it is a local id to the current NiFi instance that will
never exist in another nifi instance. We could actually replace it with some
fake placeholder id on save to registry if we wanted. For example we could have
saved the snapshot to registry like:
{code:java}
"properties": {
"record-writer": "my-fake-id"
}
...
externalControllerServices": {
"my-fake-id": {
"identifier": "my-fake-id",
"name": "reusables_avro_writer"
}
}{code}
This would import fine because all it is doing is looking for a service in the
parent hierarchy that matches the name of "resuables_avro_writer" and a
matching type/bundle of the required service type.
> importing a versioned flow with a nested versioned flow shows nested
> versioned flow with invalid controller services.
> -
>
> Key: NIFI-11464
> URL: https://issues.apache.org/jira/browse/NIFI-11464
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Affects Versions: 1.21.0
> Environment: nifi 1.21.0 and nifi registry 1.21.0 (on ubuntu 20.04)
>Reporter: Patrick A. Mol
>Priority: Major
> Attachments: exported_flow_versions_pretty.zip,
> image-2023-04-17-17-34-08-898.png, image-2023-04-17-18-06-16-102.png,
> nested_versioned_flow_issue.xml, screenprints_reproduction_steps.zip
>
>
> When a flow (reusable_flow_Q) has controller services inherited from the
> hierarchy (process group reusables) and a version of the flow is stored, the
> flow version contains the references to these external controller services
> (as seen in an exported flow version [see below]).
> When this versioned flow is imported in another flow (tenant_flows) the
> controller services need to be reset to the controller services in the new
> hierarchy.
> When we have a working flow with the nested versioned flow ready in
> development we can check this flow into version control.
> When we then deploy the flow in production, the nested versioned flow shows
> up with invalid components. It shows the external controller service
> identifiers as stored in the flow version.
> When we then go back to development version of tenant_flows and make a minor
> change to the nested versioned flow reusable_flow_Q and commit this change to
> version control.
> Due to this version change, we need to also commit the changes for the
> tenant_flows process group.
> When we now go back to production, and import this new version of
> tenant_flows, the nested versioned flow reusable_flow_Q does not have invalid
> controller services.
> If you have several flows under development using the same reusable
> components,
> you will likely end up with invalid components after import.
> Depending on the amount of versioned flows used, it could be a lot of work.
> It could also lead to issues when using the ExecuteStateless processor.
> Please see attached template nested_version_flow_issue.xml for a starting
> point to reproduce the issue. It contains the steps.
> Screenprints are attached in a zip file show the process and diagnosis.
> Controller services identifiers in version 2.
> {code:java}
> $ fgrep -C 4 reusables_avro reusable_flow_Q.json.pretty
> "controllerServices": [
> {
> "identifier": "dc884171-4d75-3854-8604-afab91bd0e60",
> "instanceIdentifier": "8f647d06-0187-1000-4be9-14a61f55d904",
> "name": "reusables_avro_reader",
> "comments": "",
> "type": "org.apache.nifi.avro.AvroReader",
> "bundle": {
> "group": "org.apache.nifi",
> --
> },
> {
> "identifier": "b512b238-cdee-3642-b5cb-0c98d30dd133",
> "instanceIdentifier": "8f64f2c6-0187-1000-7557-ca63c88054dd",
> "name": "reusables_avro_writer",
> "comments": "",
> "type": "org.apache.nifi.avro.AvroRecordSetWriter",
> "bundle": {
> "group": "org.apache.nifi",
> $ head -15 reusable_flow_Q-version-2.json.pretty
> {
> "externalControllerServices": {
> "dc884171-4d75-3854-8604-afab91bd0e60": {
> "identifier": "dc884171-4d75-3854-8604-afab91bd0e60",
> "name": "reusables_avro_reader"
> },
> "b512b238-cdee-3642-b5cb-0c98d30dd133": {
> "identifier": "b512b238-cdee-3642-b5cb-0c98d30dd133",
> "name": "reusables_avro_writ
[jira] [Comment Edited] (NIFI-11464) importing a versioned flow with a nested versioned flow shows nested versioned flow with invalid controller services.
[
https://issues.apache.org/jira/browse/NIFI-11464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713651#comment-17713651
]
Patrick A. Mol edited comment on NIFI-11464 at 4/18/23 4:33 PM:
The core of the issue is {color:#ffab00}highlighted{color} below.
My parent PG is called *reusables* with a controller service
_same_name_avro_reader_ with identifier *R123.*
Within this parent I create flow {*}reusable_flow_R{*}, using
_same_name_avro_reader_ controller service.
I commit the flow, and the flow is stored with an external controller service
information: identifier "{*}R123{*}", name {_}same_name_avro_reader{_}.
I create another parent PG called *tenant_flow_T* and create a controller
service _same_name_avro_reader_ that has identifier *T123.*
I then import the *reusable_flow_R* into {*}tenant_flow_T{*}.
The _same_name_avro_reader_ gets resolved to the controller service as defined
in tenant_flows with identifier {*}T123{*}.
The name-based resolution works as intended. =>> +that is not the issue+
<<=
I then proceed to store *tenant_flow_T* in version control.
The stored version of *tenant_flow_T* includes afaik:
* the controller service same_name_avro_reader with identifier *T123*
* a reference to where nested versioned flow can be found in the registry.
{color:#ffab00}The information that *reusable_flow_R* was configured to use
controller service with identifier *T123* is lost at this point, i.e is not
stored in the registry.{color}
Now I want to run *tenant_flows_T* in production, so I import the flow from
version control.
The flow gets imported, including the controller service definition for
_same_name_avro_reader_ with identifier {*}T123{*}.
{color:#ffab00}The nested versioned flow *reusable_flow_R* is pulled from the
registry as part of the checkout
and the controller services property is set to the identifiers it was stored
with in the registry, i.e. {*}R123{*},
which is of course invalid because *tenant_flow_T* does not know about a
controller service with identifier {*}R123{*}. {color}
{color:#ffab00}Unlike when you import a versioned flow in the UI, the
name-based resolution switching *R123* to *T123* does not occur this
time.{color}
{color:#ffab00}In theory this could be fixed by applying the same logic, but
you would be confined to using controller services with the same name.{color}
{color:#ffab00}*While a flow is valid in development, it might not be valid
after doing a fresh import. It is not guaranteed.*{color}
Important side-effect.
+One would not be able to use the flow with the ExecuteStateless processor if a
fresh checkout results in a flow with invalid components.+
Workarounds
* Workaround is to change the references after import,
but I would need to repeat this every time I checkout the a new instance of the
flow.
* Develop and commit *reusable_flow_R* in the confines of {*}tenant_flow_T{*},
so the version of *reusable_flow_R* is stored with *T123* instead of {*}R123{*}.
{color:#ffab00}This means I am no longer developing reusables independently or
separately.{color}
Implications for development.
When you import a reusable for the first time, you see a UUID, not a name, when
the controller service cannot be resolved.
You would need enforced strict naming policies across the board,
or find out what names are used before setting up controller services for the
name-based resolution to work.
I think this is a huge burden.
was (Author: JIRAUSER299819):
The core of the issue is {color:#ffab00}highlighted{color} below.
My parent PG is called *reusables* with a controller service
_same_name_avro_reader_ with identifier *R123.*
Within this parent I create flow {*}reusable_flow_R{*}, using
_same_name_avro_reader_ controller service.
I commit the flow, and the flow is stored with an external controller service
information: identifier "{*}R123{*}", name {_}same_name_avro_reader{_}.
I create another parent PG called *tenant_flow_T* and create a controller
service _same_name_avro_reader_ that has identifier *T123.*
I then import the *reusable_flow_R* into {*}tenant_flow_T{*}.
The _same_name_avro_reader_ gets resolved to the controller service as defined
in tenant_flows with identifier {*}T123{*}.
The name-based resolution works as intended. --> +that is not the issue+
<--
I then proceed to store *tenant_flow_T* in version control.
The stored version of *tenant_flow_T* includes afaik:
* the controller service same_name_avro_reader with identifier *T123*
* a reference to where nested versioned flow can be found in the registry.
{color:#ffab00}The information that *reusable_flow_R* was configured to use
controller service with identifier *T123* is lost at this point, i.e is not
stored in the registry.
{color}
Now I want to run *tenant_flows_T* in production, so I import the flow from
version control.
The flow ge
[jira] [Commented] (NIFI-11464) importing a versioned flow with a nested versioned flow shows nested versioned flow with invalid controller services.
[
https://issues.apache.org/jira/browse/NIFI-11464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713651#comment-17713651
]
Patrick A. Mol commented on NIFI-11464:
---
The core of the issue is {color:#ffab00}highlighted{color} below.
My parent PG is called *reusables* with a controller service
_same_name_avro_reader_ with identifier *R123.*
Within this parent I create flow {*}reusable_flow_R{*}, using
_same_name_avro_reader_ controller service.
I commit the flow, and the flow is stored with an external controller service
information: identifier "{*}R123{*}", name {_}same_name_avro_reader{_}.
I create another parent PG called *tenant_flow_T* and create a controller
service _same_name_avro_reader_ that has identifier *T123.*
I then import the *reusable_flow_R* into {*}tenant_flow_T{*}.
The _same_name_avro_reader_ gets resolved to the controller service as defined
in tenant_flows with identifier {*}T123{*}.
The name-based resolution works as intended. --> +that is not the issue+
<--
I then proceed to store *tenant_flow_T* in version control.
The stored version of *tenant_flow_T* includes afaik:
* the controller service same_name_avro_reader with identifier *T123*
* a reference to where nested versioned flow can be found in the registry.
{color:#ffab00}The information that *reusable_flow_R* was configured to use
controller service with identifier *T123* is lost at this point, i.e is not
stored in the registry.
{color}
Now I want to run *tenant_flows_T* in production, so I import the flow from
version control.
The flow gets imported, including the controller service definition for
_same_name_avro_reader_ with identifier {*}T123{*}.
{color:#ffab00}The nested versioned flow *reusable_flow_R* is pulled from the
registry as part of the checkout
and the controller services property is set to the identifiers it was stored
with in the registry, i.e. {*}R123{*},
which is of course invalid because *tenant_flow_T* does not know about a
controller service with identifier {*}R123{*}. {color}
{color:#ffab00}Unlike when you import a versioned flow in the UI, the
name-based resolution switching *R123* to *T123* does not occur this
time.{color}
{color:#ffab00}In theory this could be fixed by applying the same logic, but
you would be confined to using controller services with the same name.{color}
{color:#ffab00}*While a flow is valid in development, it might not be valid
after doing a fresh import. It is not guaranteed.*{color}
Important side-effect.
+One would not be able to use the flow with the ExecuteStateless processor if a
fresh checkout results in a flow with invalid components.+
Workarounds
* Workaround is to change the references after import,
but I would need to repeat this every time I checkout the a new instance of the
flow.
* Develop and commit *reusable_flow_R* in the confines of {*}tenant_flow_T{*},
so the version of *reusable_flow_R* is stored with *T123* instead of {*}R123{*}.
{color:#ffab00}This means I am no longer developing reusables independently or
separately.{color}
Implications for development.
When you import a reusable for the first time, you see a UUID, not a name, when
the controller service cannot be resolved.
You would need enforced strict naming policies across the board,
or find out what names are used before setting up controller services for the
name-based resolution to work.
I think this is a huge burden.
> importing a versioned flow with a nested versioned flow shows nested
> versioned flow with invalid controller services.
> -
>
> Key: NIFI-11464
> URL: https://issues.apache.org/jira/browse/NIFI-11464
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Affects Versions: 1.21.0
> Environment: nifi 1.21.0 and nifi registry 1.21.0 (on ubuntu 20.04)
>Reporter: Patrick A. Mol
>Priority: Major
> Attachments: exported_flow_versions_pretty.zip,
> image-2023-04-17-17-34-08-898.png, image-2023-04-17-18-06-16-102.png,
> nested_versioned_flow_issue.xml, screenprints_reproduction_steps.zip
>
>
> When a flow (reusable_flow_Q) has controller services inherited from the
> hierarchy (process group reusables) and a version of the flow is stored, the
> flow version contains the references to these external controller services
> (as seen in an exported flow version [see below]).
> When this versioned flow is imported in another flow (tenant_flows) the
> controller services need to be reset to the controller services in the new
> hierarchy.
> When we have a working flow with the nested versioned flow ready in
> development we can check this flow into version control.
> When we then deploy the flow in production, the nested versio
[jira] [Commented] (NIFI-11255) Add allowable value for “Use ‘s3.region’" attribute
[ https://issues.apache.org/jira/browse/NIFI-11255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713649#comment-17713649 ] ASF subversion and git services commented on NIFI-11255: Commit bfa66de729ff3a57129d6b7c52aafb0811642a24 in nifi's branch refs/heads/support/nifi-1.x from krisztina-zsihovszki [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=bfa66de729 ] NIFI-11255 Allowable value for 'Use s3.region Attribute' This closes #7051. Signed-off-by: Peter Turcsanyi > Add allowable value for “Use ‘s3.region’" attribute > --- > > Key: NIFI-11255 > URL: https://issues.apache.org/jira/browse/NIFI-11255 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Zsihovszki Krisztina >Assignee: Zsihovszki Krisztina >Priority: Major > Fix For: 1.latest, 2.latest > > Time Spent: 1h 50m > Remaining Estimate: 0h > > With the S3 processors, there’s no way to make the “Region” come from > FlowFile attributes. > It would be useful to offer a possibility for the user to take the region > value from FlowFile attribute, just like S3 Key and Bucket values. > We should have an allowable value for “Use ‘s3.region’ attribute”. In case > user selects "Use ‘s3.region’ attribute” from the region drop-down list, the > region value will be taken from the FlowFile attribute. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-11255) Add allowable value for “Use ‘s3.region’" attribute
[ https://issues.apache.org/jira/browse/NIFI-11255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi resolved NIFI-11255. Fix Version/s: 2.0.0 1.22.0 (was: 1.latest) (was: 2.latest) Resolution: Fixed > Add allowable value for “Use ‘s3.region’" attribute > --- > > Key: NIFI-11255 > URL: https://issues.apache.org/jira/browse/NIFI-11255 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Zsihovszki Krisztina >Assignee: Zsihovszki Krisztina >Priority: Major > Fix For: 2.0.0, 1.22.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > With the S3 processors, there’s no way to make the “Region” come from > FlowFile attributes. > It would be useful to offer a possibility for the user to take the region > value from FlowFile attribute, just like S3 Key and Bucket values. > We should have an allowable value for “Use ‘s3.region’ attribute”. In case > user selects "Use ‘s3.region’ attribute” from the region drop-down list, the > region value will be taken from the FlowFile attribute. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11255) Add allowable value for “Use ‘s3.region’" attribute
[ https://issues.apache.org/jira/browse/NIFI-11255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713642#comment-17713642 ] ASF subversion and git services commented on NIFI-11255: Commit 9a002d9a43b79528cb0a514e1cbcd62d35d76370 in nifi's branch refs/heads/main from krisztina-zsihovszki [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=9a002d9a43 ] NIFI-11255 Allowable value for 'Use s3.region Attribute' This closes #7051. Signed-off-by: Peter Turcsanyi > Add allowable value for “Use ‘s3.region’" attribute > --- > > Key: NIFI-11255 > URL: https://issues.apache.org/jira/browse/NIFI-11255 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Zsihovszki Krisztina >Assignee: Zsihovszki Krisztina >Priority: Major > Fix For: 1.latest, 2.latest > > Time Spent: 1h 40m > Remaining Estimate: 0h > > With the S3 processors, there’s no way to make the “Region” come from > FlowFile attributes. > It would be useful to offer a possibility for the user to take the region > value from FlowFile attribute, just like S3 Key and Bucket values. > We should have an allowable value for “Use ‘s3.region’ attribute”. In case > user selects "Use ‘s3.region’ attribute” from the region drop-down list, the > region value will be taken from the FlowFile attribute. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] asfgit closed pull request #7051: NIFI-11255 Allowable value for 'Use s3.region Attribute'
asfgit closed pull request #7051: NIFI-11255 Allowable value for 'Use s3.region Attribute' URL: https://github.com/apache/nifi/pull/7051 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11464) importing a versioned flow with a nested versioned flow shows nested versioned flow with invalid controller services.
[
https://issues.apache.org/jira/browse/NIFI-11464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713621#comment-17713621
]
Bryan Bende commented on NIFI-11464:
I'm still not following this part:
{code:java}
But the way nested versioned flows are handled, I would not be able to
develop/test reusables separately. {code}
You can develop/test reusables separately, as long as everywhere you use it,
you have a parent PG with services that have the same names. If you had a dev
parent PG with services named "resuables reader" and then tenants parent PG
also with "reusables reader", then what is the issue?
> importing a versioned flow with a nested versioned flow shows nested
> versioned flow with invalid controller services.
> -
>
> Key: NIFI-11464
> URL: https://issues.apache.org/jira/browse/NIFI-11464
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Affects Versions: 1.21.0
> Environment: nifi 1.21.0 and nifi registry 1.21.0 (on ubuntu 20.04)
>Reporter: Patrick A. Mol
>Priority: Major
> Attachments: exported_flow_versions_pretty.zip,
> image-2023-04-17-17-34-08-898.png, image-2023-04-17-18-06-16-102.png,
> nested_versioned_flow_issue.xml, screenprints_reproduction_steps.zip
>
>
> When a flow (reusable_flow_Q) has controller services inherited from the
> hierarchy (process group reusables) and a version of the flow is stored, the
> flow version contains the references to these external controller services
> (as seen in an exported flow version [see below]).
> When this versioned flow is imported in another flow (tenant_flows) the
> controller services need to be reset to the controller services in the new
> hierarchy.
> When we have a working flow with the nested versioned flow ready in
> development we can check this flow into version control.
> When we then deploy the flow in production, the nested versioned flow shows
> up with invalid components. It shows the external controller service
> identifiers as stored in the flow version.
> When we then go back to development version of tenant_flows and make a minor
> change to the nested versioned flow reusable_flow_Q and commit this change to
> version control.
> Due to this version change, we need to also commit the changes for the
> tenant_flows process group.
> When we now go back to production, and import this new version of
> tenant_flows, the nested versioned flow reusable_flow_Q does not have invalid
> controller services.
> If you have several flows under development using the same reusable
> components,
> you will likely end up with invalid components after import.
> Depending on the amount of versioned flows used, it could be a lot of work.
> It could also lead to issues when using the ExecuteStateless processor.
> Please see attached template nested_version_flow_issue.xml for a starting
> point to reproduce the issue. It contains the steps.
> Screenprints are attached in a zip file show the process and diagnosis.
> Controller services identifiers in version 2.
> {code:java}
> $ fgrep -C 4 reusables_avro reusable_flow_Q.json.pretty
> "controllerServices": [
> {
> "identifier": "dc884171-4d75-3854-8604-afab91bd0e60",
> "instanceIdentifier": "8f647d06-0187-1000-4be9-14a61f55d904",
> "name": "reusables_avro_reader",
> "comments": "",
> "type": "org.apache.nifi.avro.AvroReader",
> "bundle": {
> "group": "org.apache.nifi",
> --
> },
> {
> "identifier": "b512b238-cdee-3642-b5cb-0c98d30dd133",
> "instanceIdentifier": "8f64f2c6-0187-1000-7557-ca63c88054dd",
> "name": "reusables_avro_writer",
> "comments": "",
> "type": "org.apache.nifi.avro.AvroRecordSetWriter",
> "bundle": {
> "group": "org.apache.nifi",
> $ head -15 reusable_flow_Q-version-2.json.pretty
> {
> "externalControllerServices": {
> "dc884171-4d75-3854-8604-afab91bd0e60": {
> "identifier": "dc884171-4d75-3854-8604-afab91bd0e60",
> "name": "reusables_avro_reader"
> },
> "b512b238-cdee-3642-b5cb-0c98d30dd133": {
> "identifier": "b512b238-cdee-3642-b5cb-0c98d30dd133",
> "name": "reusables_avro_writer"
> }
> },
> "flowContents": {
> "comments": "used to perform Q ...",
> "componentType": "PROCESS_GROUP",
> "connections": [
> {code}
> Controller services identifiers with version 3 committed in process group
> "tenant_flows".
> {code:java}
> pmo@hpmo:~/Documents.local/nested_versioned_flows_controller_issue$ fgrep -C
> 4 tenant_flow_avro tenant_flows-version-1.json.pretty
> ],
> "groupIdentifier": "a984831b-8587-3e17-bbbc-ef4b85c3898d",
>
[GitHub] [nifi] koccs commented on a diff in pull request #7174: [NIFIF-11433] update angular, d3, moment, slickgrid, and jquery depen…
koccs commented on code in PR #7174:
URL: https://github.com/apache/nifi/pull/7174#discussion_r1169910360
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-component-version.js:
##
@@ -96,9 +96,9 @@
*/
var resetDialog = function () {
// clear the versions
-var versions = versionMap.keys();
+var versions = Array.from(versionMap.keys());
$.each(versions, function (_, version) {
-versionMap.remove(version);
+versionMap['delete'](version);
Review Comment:
Why did you use string indexer? It should be simply
[`.delete()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map/delete)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] koccs commented on a diff in pull request #7174: [NIFIF-11433] update angular, d3, moment, slickgrid, and jquery depen…
koccs commented on code in PR #7174:
URL: https://github.com/apache/nifi/pull/7174#discussion_r1169924327
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-connection.js:
##
@@ -1089,37 +1117,42 @@
if (!nfCommon.isBlank(connectionNameValue)) {
// see if the connection name label is already
rendered
if (connectionName.empty()) {
-connectionName =
connectionLabelContainer.append('g')
-.attrs({
+connectionName = d3Helpers.multiAttr(
Review Comment:
Nit picking, but there is only one attribute here, you could use `.attr()`
instead of the helper method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] koccs commented on a diff in pull request #7174: [NIFIF-11433] update angular, d3, moment, slickgrid, and jquery depen…
koccs commented on code in PR #7174:
URL: https://github.com/apache/nifi/pull/7174#discussion_r1170162373
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/nf-status-history.js:
##
@@ -356,8 +356,8 @@
// add status history details
var detailsContainer = buildDetailsContainer('Status History');
-d3.map(statusHistory.details).each(function (value, label) {
-addDetailItem(detailsContainer, label, value);
+Object.keys(statusHistory.details).forEach(function(key) {
+addDetailItem(detailsContainer, key,
statusHistory.details[key]);
Review Comment:
[`Object.entries()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/entries)
would provide a `[key, value]` pair, so you won't need the accessor for the
value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] koccs commented on a diff in pull request #7174: [NIFIF-11433] update angular, d3, moment, slickgrid, and jquery depen…
koccs commented on code in PR #7174: URL: https://github.com/apache/nifi/pull/7174#discussion_r1170164370 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-processor.js: ## @@ -69,7 +72,7 @@ // --- // cache for components that are added/removed from the canvas -// --- +// r--- Review Comment: I assume this change was unintentional. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11464) importing a versioned flow with a nested versioned flow shows nested versioned flow with invalid controller services.
[ https://issues.apache.org/jira/browse/NIFI-11464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713610#comment-17713610 ] Patrick A. Mol commented on NIFI-11464: --- I am sorry, but you are still missing the point. The "reusables" flows are process groups that you would develop and test separately (following FDLC) and use in other larger flows as basic building blocks, comparable to a standard processor. The other flows, like tenant_flows, are versioned as well at some point in time, and then deployed to production. When you first import a reusable process group, you might need to adjust the controller services, but only if you are using different names. I get that. That is not up for debate. But the way nested versioned flows are handled, I would not be able to develop/test reusables separately. I would need to do this in tenant_flows and commit while inside tenant_flows for deployment of tenant_flows to work properly. If I then decide to develop "customer_flows", I cannot use any of the "reusables" that were committed in "tenant_flows", or I would need re-commit a new version of each in customer_flows. My logs will then get flooded with messages that reusable_flow_Q is not the latest version, because all uses in tenant_flows are now one version behind. Alternatively I need to clone/branch the reusables for each development. That is not really DRY and fun either. While you could setup avro controller services at the root PG, this is not the case with a DBCP controller service, when each different tenant_flows deployment would need access to a different database or use different credentials. So I think the design needs tweaking, and a nested versioned flow should retain an indicator that it is a versioned flow and where to find the original in the registry, but a copy of the nested flow with the settings it got as part of the larger flow should be stored with the version of the larger flow. If a larger versioned flow is checked out, and the original nested flow is no longer available in the registry, then it could mark the nested flow with a "?", as is already working currently. The larger flow would still work, and you can proceed to fix the issue with the missing version. So while it looks like the last screenshot is a completely different issue, there is overlap. While you are asked for confirmation when you want to delete a flow version from the registry, there is no "referential integrity" telling you the version is in use. So it is easy to topple the whole setup on purpose, but also when you have an issue with the registry. > importing a versioned flow with a nested versioned flow shows nested > versioned flow with invalid controller services. > - > > Key: NIFI-11464 > URL: https://issues.apache.org/jira/browse/NIFI-11464 > Project: Apache NiFi > Issue Type: Bug > Components: Flow Versioning >Affects Versions: 1.21.0 > Environment: nifi 1.21.0 and nifi registry 1.21.0 (on ubuntu 20.04) >Reporter: Patrick A. Mol >Priority: Major > Attachments: exported_flow_versions_pretty.zip, > image-2023-04-17-17-34-08-898.png, image-2023-04-17-18-06-16-102.png, > nested_versioned_flow_issue.xml, screenprints_reproduction_steps.zip > > > When a flow (reusable_flow_Q) has controller services inherited from the > hierarchy (process group reusables) and a version of the flow is stored, the > flow version contains the references to these external controller services > (as seen in an exported flow version [see below]). > When this versioned flow is imported in another flow (tenant_flows) the > controller services need to be reset to the controller services in the new > hierarchy. > When we have a working flow with the nested versioned flow ready in > development we can check this flow into version control. > When we then deploy the flow in production, the nested versioned flow shows > up with invalid components. It shows the external controller service > identifiers as stored in the flow version. > When we then go back to development version of tenant_flows and make a minor > change to the nested versioned flow reusable_flow_Q and commit this change to > version control. > Due to this version change, we need to also commit the changes for the > tenant_flows process group. > When we now go back to production, and import this new version of > tenant_flows, the nested versioned flow reusable_flow_Q does not have invalid > controller services. > If you have several flows under development using the same reusable > components, > you will likely end up with invalid components after import. > Depending on the amount of versioned flows used, it could be a lot of work. > It co
[jira] [Commented] (NIFI-11208) Remove Hortonworks Schema Registry
[
https://issues.apache.org/jira/browse/NIFI-11208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713609#comment-17713609
]
David Handermann commented on NIFI-11208:
-
As mentioned on the pull request discussion, the [Hortonworks
Registry|https://github.com/hortonworks/registry] repository includes a more
recent release of version
[1.0.0|https://github.com/hortonworks/registry/releases/tag/1.0.0-rc2]. It is
probably worth some additional conversation to evaluate the roadmap for the
project and potential integration options. The NiFi 1 NAR files should be
initially compatible with NiFi 2, so that may be another option for this
service and other removed components that integrate with previous versions of
various services.
> Remove Hortonworks Schema Registry
> --
>
> Key: NIFI-11208
> URL: https://issues.apache.org/jira/browse/NIFI-11208
> Project: Apache NiFi
> Issue Type: Task
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The {{HortonworksSchemaRegistry}} was deprecated for removal in NiFi 1.20.0
> and should be removed along with the {{nifi-hwx-schema-registry-bundle}}
> modules.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] sardell commented on a diff in pull request #7174: [NIFIF-11433] update angular, d3, moment, slickgrid, and jquery depen…
sardell commented on code in PR #7174:
URL: https://github.com/apache/nifi/pull/7174#discussion_r1170122858
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-processor.js:
##
@@ -1140,9 +1188,9 @@
*/
get: function (id) {
if (nfCommon.isUndefined(id)) {
-return processorMap.values();
+return Array.from(processorMap.values());
} else {
-return processorMap.get(id);
+return processoMap.get(id);
Review Comment:
```suggestion
return processorMap.get(id);
```
This typo currently breaks updating processor configurations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] bbende merged pull request #7165: NiFi CLI - add possibility to set 'Maximum Timer Driven Thread Count'
bbende merged PR #7165: URL: https://github.com/apache/nifi/pull/7165 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11435) NiFi CLI - add possibility to set "Maximum Timer Driven Thread Count"
[ https://issues.apache.org/jira/browse/NIFI-11435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-11435: --- Fix Version/s: 1.latest > NiFi CLI - add possibility to set "Maximum Timer Driven Thread Count" > - > > Key: NIFI-11435 > URL: https://issues.apache.org/jira/browse/NIFI-11435 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Timea Barna >Assignee: Timea Barna >Priority: Major > Fix For: 2.0.0, 1.latest > > > Add a capability in the NiFi CLI to set the value for "Maximum Timer Driven > Thread Count". -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11435) NiFi CLI - add possibility to set "Maximum Timer Driven Thread Count"
[ https://issues.apache.org/jira/browse/NIFI-11435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-11435: --- Fix Version/s: 1.22.0 (was: 1.latest) > NiFi CLI - add possibility to set "Maximum Timer Driven Thread Count" > - > > Key: NIFI-11435 > URL: https://issues.apache.org/jira/browse/NIFI-11435 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Timea Barna >Assignee: Timea Barna >Priority: Major > Fix For: 2.0.0, 1.22.0 > > > Add a capability in the NiFi CLI to set the value for "Maximum Timer Driven > Thread Count". -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11435) NiFi CLI - add possibility to set "Maximum Timer Driven Thread Count"
[ https://issues.apache.org/jira/browse/NIFI-11435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-11435: --- Fix Version/s: 2.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) > NiFi CLI - add possibility to set "Maximum Timer Driven Thread Count" > - > > Key: NIFI-11435 > URL: https://issues.apache.org/jira/browse/NIFI-11435 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Timea Barna >Assignee: Timea Barna >Priority: Major > Fix For: 2.0.0 > > > Add a capability in the NiFi CLI to set the value for "Maximum Timer Driven > Thread Count". -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11464) importing a versioned flow with a nested versioned flow shows nested versioned flow with invalid controller services.
[
https://issues.apache.org/jira/browse/NIFI-11464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17713590#comment-17713590
]
Bryan Bende commented on NIFI-11464:
I still think the controller service ids part is working correctly based on the
current design (using name + type). The externalServices section of a flow is
populated at the time the the snapshot is saved to the registry. So when saving
resuables v1 to registry, if reusables is inside of tenants and the services
being used are the tenants services, then the externalServices section of
reusables v1 will have the ids and names of the tenants services (the ids are
really meaningless). When resuables v1 is imported inside some other parent PG,
it will look for services with the same name and type as the tenants services,
if they are there they will be selected, if not they won't be.
Your last screenshot is a completely different issue... if a nested process
group that is under version control is missing, what would you expect to happen
when importing the parent? if we let it proceed, there would be a whole part of
the parent flow completely missing, and it could have had connections going
into it which couldn't be made. It is basically unusable if the entire thing
can't be resolved.
> importing a versioned flow with a nested versioned flow shows nested
> versioned flow with invalid controller services.
> -
>
> Key: NIFI-11464
> URL: https://issues.apache.org/jira/browse/NIFI-11464
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Affects Versions: 1.21.0
> Environment: nifi 1.21.0 and nifi registry 1.21.0 (on ubuntu 20.04)
>Reporter: Patrick A. Mol
>Priority: Major
> Attachments: exported_flow_versions_pretty.zip,
> image-2023-04-17-17-34-08-898.png, image-2023-04-17-18-06-16-102.png,
> nested_versioned_flow_issue.xml, screenprints_reproduction_steps.zip
>
>
> When a flow (reusable_flow_Q) has controller services inherited from the
> hierarchy (process group reusables) and a version of the flow is stored, the
> flow version contains the references to these external controller services
> (as seen in an exported flow version [see below]).
> When this versioned flow is imported in another flow (tenant_flows) the
> controller services need to be reset to the controller services in the new
> hierarchy.
> When we have a working flow with the nested versioned flow ready in
> development we can check this flow into version control.
> When we then deploy the flow in production, the nested versioned flow shows
> up with invalid components. It shows the external controller service
> identifiers as stored in the flow version.
> When we then go back to development version of tenant_flows and make a minor
> change to the nested versioned flow reusable_flow_Q and commit this change to
> version control.
> Due to this version change, we need to also commit the changes for the
> tenant_flows process group.
> When we now go back to production, and import this new version of
> tenant_flows, the nested versioned flow reusable_flow_Q does not have invalid
> controller services.
> If you have several flows under development using the same reusable
> components,
> you will likely end up with invalid components after import.
> Depending on the amount of versioned flows used, it could be a lot of work.
> It could also lead to issues when using the ExecuteStateless processor.
> Please see attached template nested_version_flow_issue.xml for a starting
> point to reproduce the issue. It contains the steps.
> Screenprints are attached in a zip file show the process and diagnosis.
> Controller services identifiers in version 2.
> {code:java}
> $ fgrep -C 4 reusables_avro reusable_flow_Q.json.pretty
> "controllerServices": [
> {
> "identifier": "dc884171-4d75-3854-8604-afab91bd0e60",
> "instanceIdentifier": "8f647d06-0187-1000-4be9-14a61f55d904",
> "name": "reusables_avro_reader",
> "comments": "",
> "type": "org.apache.nifi.avro.AvroReader",
> "bundle": {
> "group": "org.apache.nifi",
> --
> },
> {
> "identifier": "b512b238-cdee-3642-b5cb-0c98d30dd133",
> "instanceIdentifier": "8f64f2c6-0187-1000-7557-ca63c88054dd",
> "name": "reusables_avro_writer",
> "comments": "",
> "type": "org.apache.nifi.avro.AvroRecordSetWriter",
> "bundle": {
> "group": "org.apache.nifi",
> $ head -15 reusable_flow_Q-version-2.json.pretty
> {
> "externalControllerServices": {
> "dc884171-4d75-3854-8604-afab91bd0e60": {
> "identifier": "dc884171-4d75-3854-8604-afab91bd0e60",
> "name": "reus
[GitHub] [nifi] sardell commented on pull request #7174: [NIFIF-11433] update angular, d3, moment, slickgrid, and jquery depen…
sardell commented on PR #7174: URL: https://github.com/apache/nifi/pull/7174#issuecomment-1513082058 Reviewing now... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-11467) Update jjwt to 0.11.5
Mike R created NIFI-11467: - Summary: Update jjwt to 0.11.5 Key: NIFI-11467 URL: https://issues.apache.org/jira/browse/NIFI-11467 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.21.0 Reporter: Mike R Update jjwt to 0.11.5 to resolve CVE -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1552: MINIFICPP-2089 prefix EventData in flat JSON output so it doesnt need t…
szaszm commented on code in PR #1552:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1552#discussion_r1169959599
##
extensions/windows-event-log/tests/CWELCustomProviderTests.cpp:
##
@@ -172,9 +172,10 @@ TEST_CASE("ConsumeWindowsEventLog prints events in
JSON::Flattened correctly cus
{
"Name": ")" + CUSTOM_PROVIDER_NAME + R"(",
"Channel": ")" + CUSTOM_CHANNEL /* Channel is not overwritten by data
named "Channel" */ + R"(",
- "EventData": )" + EVENT_DATA_JSON /* EventData is not discarded */ + R"(,
- "param1": "Actual event",
- "param2": "Second"
+ "EventData.param1.Data": "Actual event",
+ "EventData.param2.Data": "Second",
+ "EventData.Channel.Data": "Third",
+ "EventData.Binary": "0901"
Review Comment:
I think we should drop the type info from here, if there is a name. It
doesn't seem to add any value.
```suggestion
"EventData.param1": "Actual event",
"EventData.param2": "Second",
"EventData.Channel": "Third",
"EventData.Binary": "0901"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (NIFI-11463) Add option for IdentifyMimeType custom mime definitions to be in addition to the default tika definitions
[ https://issues.apache.org/jira/browse/NIFI-11463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nissim Shiman reassigned NIFI-11463: Assignee: Nissim Shiman > Add option for IdentifyMimeType custom mime definitions to be in addition to > the default tika definitions > - > > Key: NIFI-11463 > URL: https://issues.apache.org/jira/browse/NIFI-11463 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Nissim Shiman >Assignee: Nissim Shiman >Priority: Major > > IdentifyMimeType has options for users to configure custom mime types > definitions. > These user defined definitions override the default tika mime type > definitions. > Add option to allow custom mime type definitions to be in addition to the > default tika mime type definitions. > Users will be able to decide if custom definitions should be used instead of > or in addition to default tika definitions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] martinzink commented on pull request #1560: MINIFICPP-2101 Compilation fix in PutSFTPTests (libc++)
martinzink commented on PR #1560: URL: https://github.com/apache/nifi-minifi-cpp/pull/1560#issuecomment-1512894436 The windows test failure seems related, I'll investigate -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] timeabarna commented on pull request #7092: NIFI-11327 Add Export/Import All - NiFi CLI - NiFi Registry
timeabarna commented on PR #7092: URL: https://github.com/apache/nifi/pull/7092#issuecomment-1512843623 Thanks @bbende and @exceptionfactory for your help, I've updated the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a diff in pull request #1560: MINIFICPP-2101 Compilation fix in PutSFTPTests (libc++)
fgerlits commented on code in PR #1560:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1560#discussion_r1169721997
##
libminifi/src/utils/file/FileUtils.cpp:
##
@@ -74,13 +74,23 @@ time_t to_time_t(std::filesystem::file_time_type file_time)
{
#endif
}
-std::chrono::time_point
to_sys(std::filesystem::file_time_type file_time) {
-#if defined(WIN32) || defined(_LIBCPP_VERSION) && _LIBCPP_VERSION >= 14000
- return
std::chrono::time_point_cast(file_time -
std::filesystem::file_time_type::clock::now() +
std::chrono::system_clock::now());
-#elif defined(_LIBCPP_VERSION)
+std::chrono::system_clock::time_point to_sys(std::filesystem::file_time_type
file_time) {
+#if defined(WIN32)
+ return std::chrono::clock_cast(file_time);
+#elif defined(_LIBCPP_VERSION) && (_LIBCPP_VERSION < 14000)
return
std::chrono::system_clock::from_time_t(std::chrono::file_clock::to_time_t(file_time));
#else
- return std::chrono::file_clock::to_sys(file_time);
+ return
std::chrono::time_point_cast(std::chrono::file_clock::to_sys(file_time));
+#endif
+}
+
+std::filesystem::file_time_type from_sys(std::chrono::system_clock::time_point
sys_time) {
+#if defined(WIN32)
+ return std::chrono::clock_cast(sys_time);
+#elif defined(_LIBCPP_VERSION) && (_LIBCPP_VERSION < 14000)
+ return
std::chrono::file_clock::from_time_t(std::chrono::system_clock::to_time_t(sys_time));
+#else
+ return std::chrono::file_clock::from_sys(sys_time);
Review Comment:
Yes, I think adding the `time_point_cast` in the unnecessary direction would
help to make the code less confusing to humans. Hopefully, the compiler will
optimize it out. Even more hopefullier, we can soon remove all this mess and
simply use `clock_cast`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1560: MINIFICPP-2101 Compilation fix in PutSFTPTests (libc++)
martinzink commented on code in PR #1560:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1560#discussion_r1169706602
##
libminifi/src/utils/file/FileUtils.cpp:
##
@@ -74,13 +74,23 @@ time_t to_time_t(std::filesystem::file_time_type file_time)
{
#endif
}
-std::chrono::time_point
to_sys(std::filesystem::file_time_type file_time) {
-#if defined(WIN32) || defined(_LIBCPP_VERSION) && _LIBCPP_VERSION >= 14000
- return
std::chrono::time_point_cast(file_time -
std::filesystem::file_time_type::clock::now() +
std::chrono::system_clock::now());
-#elif defined(_LIBCPP_VERSION)
+std::chrono::system_clock::time_point to_sys(std::filesystem::file_time_type
file_time) {
+#if defined(WIN32)
+ return std::chrono::clock_cast(file_time);
+#elif defined(_LIBCPP_VERSION) && (_LIBCPP_VERSION < 14000)
return
std::chrono::system_clock::from_time_t(std::chrono::file_clock::to_time_t(file_time));
#else
- return std::chrono::file_clock::to_sys(file_time);
+ return
std::chrono::time_point_cast(std::chrono::file_clock::to_sys(file_time));
+#endif
+}
+
+std::filesystem::file_time_type from_sys(std::chrono::system_clock::time_point
sys_time) {
+#if defined(WIN32)
+ return std::chrono::clock_cast(sys_time);
+#elif defined(_LIBCPP_VERSION) && (_LIBCPP_VERSION < 14000)
+ return
std::chrono::file_clock::from_time_t(std::chrono::system_clock::to_time_t(sys_time));
+#else
+ return std::chrono::file_clock::from_sys(sys_time);
Review Comment:
You only need to cast it where it can lose precision otherwise the cast is
implicit.
So from hours to minutes you dont need duration_cast while you would need it
in the other direction.
Of course we could add the explicit cast to other function aswell, if we
expect that on some platfrom system_clock will be more precise than file_clock
(or simply to improve readability) What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1560: MINIFICPP-2101 Compilation fix in PutSFTPTests (libc++)
martinzink commented on code in PR #1560:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1560#discussion_r1169706602
##
libminifi/src/utils/file/FileUtils.cpp:
##
@@ -74,13 +74,23 @@ time_t to_time_t(std::filesystem::file_time_type file_time)
{
#endif
}
-std::chrono::time_point
to_sys(std::filesystem::file_time_type file_time) {
-#if defined(WIN32) || defined(_LIBCPP_VERSION) && _LIBCPP_VERSION >= 14000
- return
std::chrono::time_point_cast(file_time -
std::filesystem::file_time_type::clock::now() +
std::chrono::system_clock::now());
-#elif defined(_LIBCPP_VERSION)
+std::chrono::system_clock::time_point to_sys(std::filesystem::file_time_type
file_time) {
+#if defined(WIN32)
+ return std::chrono::clock_cast(file_time);
+#elif defined(_LIBCPP_VERSION) && (_LIBCPP_VERSION < 14000)
return
std::chrono::system_clock::from_time_t(std::chrono::file_clock::to_time_t(file_time));
#else
- return std::chrono::file_clock::to_sys(file_time);
+ return
std::chrono::time_point_cast(std::chrono::file_clock::to_sys(file_time));
+#endif
+}
+
+std::filesystem::file_time_type from_sys(std::chrono::system_clock::time_point
sys_time) {
+#if defined(WIN32)
+ return std::chrono::clock_cast(sys_time);
+#elif defined(_LIBCPP_VERSION) && (_LIBCPP_VERSION < 14000)
+ return
std::chrono::file_clock::from_time_t(std::chrono::system_clock::to_time_t(sys_time));
+#else
+ return std::chrono::file_clock::from_sys(sys_time);
Review Comment:
You only need to cast it where it can lose precision otherwise the cast is
implicit.
So from hours to minutes you dont need duration_cast while you would need in
the other direction.
Of course we could add the explicit cast to other function aswell, if we
expect that on some platfrom system_clock will be more precise than file_clock
(or simply to improve readability) What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a diff in pull request #1560: MINIFICPP-2101 Compilation fix in PutSFTPTests (libc++)
fgerlits commented on code in PR #1560:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1560#discussion_r1169704112
##
libminifi/src/utils/file/FileUtils.cpp:
##
@@ -74,13 +74,23 @@ time_t to_time_t(std::filesystem::file_time_type file_time)
{
#endif
}
-std::chrono::time_point
to_sys(std::filesystem::file_time_type file_time) {
-#if defined(WIN32) || defined(_LIBCPP_VERSION) && _LIBCPP_VERSION >= 14000
- return
std::chrono::time_point_cast(file_time -
std::filesystem::file_time_type::clock::now() +
std::chrono::system_clock::now());
-#elif defined(_LIBCPP_VERSION)
+std::chrono::system_clock::time_point to_sys(std::filesystem::file_time_type
file_time) {
+#if defined(WIN32)
+ return std::chrono::clock_cast(file_time);
+#elif defined(_LIBCPP_VERSION) && (_LIBCPP_VERSION < 14000)
return
std::chrono::system_clock::from_time_t(std::chrono::file_clock::to_time_t(file_time));
#else
- return std::chrono::file_clock::to_sys(file_time);
+ return
std::chrono::time_point_cast(std::chrono::file_clock::to_sys(file_time));
+#endif
+}
+
+std::filesystem::file_time_type from_sys(std::chrono::system_clock::time_point
sys_time) {
+#if defined(WIN32)
+ return std::chrono::clock_cast(sys_time);
+#elif defined(_LIBCPP_VERSION) && (_LIBCPP_VERSION < 14000)
+ return
std::chrono::file_clock::from_time_t(std::chrono::system_clock::to_time_t(sys_time));
+#else
+ return std::chrono::file_clock::from_sys(sys_time);
Review Comment:
Ah, got it. The duration conversion happens automatically in the `return`
statement, but we need the `time_point_cast` when converting from a 128-bit
underlying type to a 64-bit one (but not the other way round).
That's a bit nasty -- I hope `clock_cast` will fix this problem when it's
available on all compilers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a diff in pull request #1560: MINIFICPP-2101 Compilation fix in PutSFTPTests (libc++)
fgerlits commented on code in PR #1560:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1560#discussion_r1169687229
##
libminifi/src/utils/file/FileUtils.cpp:
##
@@ -74,13 +74,23 @@ time_t to_time_t(std::filesystem::file_time_type file_time)
{
#endif
}
-std::chrono::time_point
to_sys(std::filesystem::file_time_type file_time) {
-#if defined(WIN32) || defined(_LIBCPP_VERSION) && _LIBCPP_VERSION >= 14000
- return
std::chrono::time_point_cast(file_time -
std::filesystem::file_time_type::clock::now() +
std::chrono::system_clock::now());
-#elif defined(_LIBCPP_VERSION)
+std::chrono::system_clock::time_point to_sys(std::filesystem::file_time_type
file_time) {
+#if defined(WIN32)
+ return std::chrono::clock_cast(file_time);
+#elif defined(_LIBCPP_VERSION) && (_LIBCPP_VERSION < 14000)
return
std::chrono::system_clock::from_time_t(std::chrono::file_clock::to_time_t(file_time));
#else
- return std::chrono::file_clock::to_sys(file_time);
+ return
std::chrono::time_point_cast(std::chrono::file_clock::to_sys(file_time));
+#endif
+}
+
+std::filesystem::file_time_type from_sys(std::chrono::system_clock::time_point
sys_time) {
+#if defined(WIN32)
+ return std::chrono::clock_cast(sys_time);
+#elif defined(_LIBCPP_VERSION) && (_LIBCPP_VERSION < 14000)
+ return
std::chrono::file_clock::from_time_t(std::chrono::system_clock::to_time_t(sys_time));
+#else
+ return std::chrono::file_clock::from_sys(sys_time);
Review Comment:
I don't get it. If `std::chrono::file_clock::to_sys(file_time)` returns
nanoseconds and needs to be duration-cast to microseconds, then doesn't
`std::chrono::file_clock::from_sys(sys_time)` return microseconds and needs to
be duration-cast to nanoseconds?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a diff in pull request #1560: MINIFICPP-2101 Compilation fix in PutSFTPTests (libc++)
fgerlits commented on code in PR #1560:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1560#discussion_r1169687229
##
libminifi/src/utils/file/FileUtils.cpp:
##
@@ -74,13 +74,23 @@ time_t to_time_t(std::filesystem::file_time_type file_time)
{
#endif
}
-std::chrono::time_point
to_sys(std::filesystem::file_time_type file_time) {
-#if defined(WIN32) || defined(_LIBCPP_VERSION) && _LIBCPP_VERSION >= 14000
- return
std::chrono::time_point_cast(file_time -
std::filesystem::file_time_type::clock::now() +
std::chrono::system_clock::now());
-#elif defined(_LIBCPP_VERSION)
+std::chrono::system_clock::time_point to_sys(std::filesystem::file_time_type
file_time) {
+#if defined(WIN32)
+ return std::chrono::clock_cast(file_time);
+#elif defined(_LIBCPP_VERSION) && (_LIBCPP_VERSION < 14000)
return
std::chrono::system_clock::from_time_t(std::chrono::file_clock::to_time_t(file_time));
#else
- return std::chrono::file_clock::to_sys(file_time);
+ return
std::chrono::time_point_cast(std::chrono::file_clock::to_sys(file_time));
+#endif
+}
+
+std::filesystem::file_time_type from_sys(std::chrono::system_clock::time_point
sys_time) {
+#if defined(WIN32)
+ return std::chrono::clock_cast(sys_time);
+#elif defined(_LIBCPP_VERSION) && (_LIBCPP_VERSION < 14000)
+ return
std::chrono::file_clock::from_time_t(std::chrono::system_clock::to_time_t(sys_time));
+#else
+ return std::chrono::file_clock::from_sys(sys_time);
Review Comment:
I don't get it. If `std::chrono::file_clock::to_sys(file_time)` returns 128
bits and needs to be duration-cast to 64 bits, then doesn't
`std::chrono::file_clock::from_sys(sys_time)` return 64 bits and needs to be
duration-cast to 128 bits?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] mr1716 commented on pull request #7174: [NIFIF-11433] update angular, d3, moment, slickgrid, and jquery depen…
mr1716 commented on PR #7174: URL: https://github.com/apache/nifi/pull/7174#issuecomment-1512659140 Think this would take care or not of the owasp dependency check findings -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1560: MINIFICPP-2101 Compilation fix in PutSFTPTests (libc++)
martinzink commented on code in PR #1560:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1560#discussion_r1169655064
##
libminifi/src/utils/file/FileUtils.cpp:
##
@@ -74,13 +74,23 @@ time_t to_time_t(std::filesystem::file_time_type file_time)
{
#endif
}
-std::chrono::time_point
to_sys(std::filesystem::file_time_type file_time) {
-#if defined(WIN32) || defined(_LIBCPP_VERSION) && _LIBCPP_VERSION >= 14000
- return
std::chrono::time_point_cast(file_time -
std::filesystem::file_time_type::clock::now() +
std::chrono::system_clock::now());
-#elif defined(_LIBCPP_VERSION)
+std::chrono::system_clock::time_point to_sys(std::filesystem::file_time_type
file_time) {
+#if defined(WIN32)
+ return std::chrono::clock_cast(file_time);
+#elif defined(_LIBCPP_VERSION) && (_LIBCPP_VERSION < 14000)
return
std::chrono::system_clock::from_time_t(std::chrono::file_clock::to_time_t(file_time));
#else
- return std::chrono::file_clock::to_sys(file_time);
+ return
std::chrono::time_point_cast(std::chrono::file_clock::to_sys(file_time));
+#endif
+}
+
+std::filesystem::file_time_type from_sys(std::chrono::system_clock::time_point
sys_time) {
+#if defined(WIN32)
+ return std::chrono::clock_cast(sys_time);
+#elif defined(_LIBCPP_VERSION) && (_LIBCPP_VERSION < 14000)
+ return
std::chrono::file_clock::from_time_t(std::chrono::system_clock::to_time_t(sys_time));
+#else
+ return std::chrono::file_clock::from_sys(sys_time);
Review Comment:
On libc++ (if the _LIBCPP_HAS_NO_INT128 is not defined) file_clock uses a
128 bit resolution while the system_clock only uses 64 bit, due to this we only
need the cast in from_sys.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a diff in pull request #1561: MINIFICPP-2103 JNI extension compile fix (libc++)
fgerlits commented on code in PR #1561:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1561#discussion_r1169650365
##
extensions/jni/jvm/JniReferenceObjects.h:
##
@@ -128,8 +128,8 @@ class JniByteInputStream {
return read;
}
- int64_t read(uint8_t &arr) {
-return stream_->read(arr);
+ int64_t read(uint8_t &arr) const {
Review Comment:
also, nitpicking and old code, but it's weird to call a reference-to-byte
parameter "arr"
##
extensions/jni/jvm/JniReferenceObjects.h:
##
@@ -128,8 +128,8 @@ class JniByteInputStream {
return read;
}
- int64_t read(uint8_t &arr) {
-return stream_->read(arr);
+ int64_t read(uint8_t &arr) const {
+return gsl::narrow(stream_->read(arr));
Review Comment:
If `stream_` is a `minifi::io::InputStream`, then `read()` can return
`static_cast(-1)`, which will cause this to terminate (or return an
incorrect value if `size_t` is smaller than 64 bits). We need to check if the
return value `io::isError()`, and return `-1` explicitly in that case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a diff in pull request #1560: MINIFICPP-2101 Compilation fix in PutSFTPTests (libc++)
fgerlits commented on code in PR #1560:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1560#discussion_r1169627160
##
libminifi/src/utils/file/FileUtils.cpp:
##
Review Comment:
some unit tests for these conversion functions would be useful
##
libminifi/src/utils/file/FileUtils.cpp:
##
@@ -74,13 +74,23 @@ time_t to_time_t(std::filesystem::file_time_type file_time)
{
#endif
}
-std::chrono::time_point
to_sys(std::filesystem::file_time_type file_time) {
-#if defined(WIN32) || defined(_LIBCPP_VERSION) && _LIBCPP_VERSION >= 14000
- return
std::chrono::time_point_cast(file_time -
std::filesystem::file_time_type::clock::now() +
std::chrono::system_clock::now());
-#elif defined(_LIBCPP_VERSION)
+std::chrono::system_clock::time_point to_sys(std::filesystem::file_time_type
file_time) {
+#if defined(WIN32)
+ return std::chrono::clock_cast(file_time);
+#elif defined(_LIBCPP_VERSION) && (_LIBCPP_VERSION < 14000)
return
std::chrono::system_clock::from_time_t(std::chrono::file_clock::to_time_t(file_time));
#else
- return std::chrono::file_clock::to_sys(file_time);
+ return
std::chrono::time_point_cast(std::chrono::file_clock::to_sys(file_time));
+#endif
+}
+
+std::filesystem::file_time_type from_sys(std::chrono::system_clock::time_point
sys_time) {
+#if defined(WIN32)
+ return std::chrono::clock_cast(sys_time);
+#elif defined(_LIBCPP_VERSION) && (_LIBCPP_VERSION < 14000)
+ return
std::chrono::file_clock::from_time_t(std::chrono::system_clock::to_time_t(sys_time));
+#else
+ return std::chrono::file_clock::from_sys(sys_time);
Review Comment:
For which compiler is the duration cast needed in `to_sys`? Does it need to
be added for that compiler here in `from_sys`, as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (MINIFICPP-2103) JNI extension fails to compile with (libc++)
[ https://issues.apache.org/jira/browse/MINIFICPP-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-2103: --- Summary: JNI extension fails to compile with (libc++) (was: JNI extension fails to compile with clang) > JNI extension fails to compile with (libc++) > > > Key: MINIFICPP-2103 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2103 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > extensions/jni/jvm/JniReferenceObjects.h:112:86: error: no matching function > for call to 'min' > int actual = > static_cast(stream_->read(gsl::make_span(buffer_).subspan(0, > std::min(remaining, buffer_.size(); > > > ^~~~ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2103) JNI extension fails to compile (libc++)
[ https://issues.apache.org/jira/browse/MINIFICPP-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-2103: --- Summary: JNI extension fails to compile (libc++) (was: JNI extension fails to compile with (libc++)) > JNI extension fails to compile (libc++) > --- > > Key: MINIFICPP-2103 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2103 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > extensions/jni/jvm/JniReferenceObjects.h:112:86: error: no matching function > for call to 'min' > int actual = > static_cast(stream_->read(gsl::make_span(buffer_).subspan(0, > std::min(remaining, buffer_.size(); > > > ^~~~ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2103) JNI extension fails to compile with clang
[ https://issues.apache.org/jira/browse/MINIFICPP-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-2103: --- Status: Patch Available (was: In Progress) https://github.com/apache/nifi-minifi-cpp/pull/1561 > JNI extension fails to compile with clang > - > > Key: MINIFICPP-2103 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2103 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > extensions/jni/jvm/JniReferenceObjects.h:112:86: error: no matching function > for call to 'min' > int actual = > static_cast(stream_->read(gsl::make_span(buffer_).subspan(0, > std::min(remaining, buffer_.size(); > > > ^~~~ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2101) Compilation failure in PutSFTPTests (libc++)
[ https://issues.apache.org/jira/browse/MINIFICPP-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-2101: --- Status: Patch Available (was: In Progress) https://github.com/apache/nifi-minifi-cpp/pull/1560 > Compilation failure in PutSFTPTests (libc++) > > > Key: MINIFICPP-2101 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2101 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > file_clock::from_sys(modification_time) used in PutSFTPTests is not available > on every supported platform -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2101) Compilation failure in PutSFTPTests (libc++)
[ https://issues.apache.org/jira/browse/MINIFICPP-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink updated MINIFICPP-2101: --- Summary: Compilation failure in PutSFTPTests (libc++) (was: Compilation failure in PutSFTPTests on MacOS) > Compilation failure in PutSFTPTests (libc++) > > > Key: MINIFICPP-2101 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2101 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > file_clock::from_sys(modification_time) used in PutSFTPTests is not available > on every supported platform -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] martinzink opened a new pull request, #1561: MINIFICPP-2103 JNI extension compile fix (libc++)
martinzink opened a new pull request, #1561: URL: https://github.com/apache/nifi-minifi-cpp/pull/1561 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically main)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Reopened] (MINIFICPP-1825) Create Properties at compile time
[
https://issues.apache.org/jira/browse/MINIFICPP-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits reopened MINIFICPP-1825:
---
> Create Properties at compile time
> -
>
> Key: MINIFICPP-1825
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1825
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Ferenc Gerlits
>Assignee: Ferenc Gerlits
>Priority: Minor
>
> Properties and Relationships of Processors and Controller Services should be
> {{{}constexpr{}}}, so they get created at compile time. This would speed up
> the startup, as well as avoid the problem of the unpredictable order of
> initialization of these Properties.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-1825) Create Properties at compile time
[
https://issues.apache.org/jira/browse/MINIFICPP-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits resolved MINIFICPP-1825.
---
Resolution: Fixed
> Create Properties at compile time
> -
>
> Key: MINIFICPP-1825
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1825
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Ferenc Gerlits
>Assignee: Ferenc Gerlits
>Priority: Minor
>
> Properties and Relationships of Processors and Controller Services should be
> {{{}constexpr{}}}, so they get created at compile time. This would speed up
> the startup, as well as avoid the problem of the unpredictable order of
> initialization of these Properties.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] martinzink opened a new pull request, #1560: MINIFICPP-2101 Compilation fix in PutSFTPTests (libc++)
martinzink opened a new pull request, #1560: URL: https://github.com/apache/nifi-minifi-cpp/pull/1560 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically main)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
