[jira] [Created] (NIFI-12786) NiFi Data Provenance search date filter not working properly
macdoor615 created NIFI-12786: - Summary: NiFi Data Provenance search date filter not working properly Key: NIFI-12786 URL: https://issues.apache.org/jira/browse/NIFI-12786 Project: Apache NiFi Issue Type: Bug Components: Core UI Affects Versions: 2.0.0-M2 Environment: AlmaLinux 8.9 Kernel 4.18.0-513.5.1.el8_9.x86_64 Apache NiFi 2.0.0-M2 Reporter: macdoor615 Attachments: NiFi Data Provenance event list.png, Search with Start Date 2 days in advance.png, Search with default Start Date.png Select a FlowFile Uuid that just occurred from the NiFi Data Provenance event list, and then search. The result is empty. If you modify the Start Date 2 days in advance, the correct results will be returned. For comparison, 2.0.0-M1 can get correct search results with the default Start Date. pls refer uploaded pictures -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12785) InvokeHTTP handler should not urlencode HTTP URL
macdoor615 created NIFI-12785: - Summary: InvokeHTTP handler should not urlencode HTTP URL Key: NIFI-12785 URL: https://issues.apache.org/jira/browse/NIFI-12785 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 2.0.0-M2 Environment: AlmaLinux 8.9 Kernel 4.18.0-513.5.1.el8_9.x86_64 Apache NiFi 2.0.0-M2 Reporter: macdoor615 Attachments: M1-output.png, M2-output.png InvokeHTTP processor call HTTP URL [http://hb3-ifz-gitlab-000:8100/gitlab/api/v4/projects/318/repository/files/ftp%2Fstage%2F15m%2Fheshangwuzhibo.yaml/raw?ref=main] output attribute invokehttp.request.url: [http://hb3-ifz-gitlab-000:8100/gitlab/api/v4/projects/318/repository/files/ftp%252Fstage%252F15m%252Fheshangwuzhibo.yaml/raw?ref=main] invokehttp.status.code: 404 The situation is different for version 2.0.0-M1, output attribute invokehttp.request.url: [http://hb3-ifz-gitlab-000:8100/gitlab/api/v4/projects/318/repository/files/ftp%2Fstage%2F15m%2Fheshangwuzhibo.yaml/raw?ref=main] invokehttp.status.code: 200 I found that in the M2 version % symbol was urlencoded to %25, M1 version. The M1 version does not urlencode pls refer to the uploaded pictures -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-4385) Adjust the QueryDatabaseTable processor for handling big tables.
[ https://issues.apache.org/jira/browse/NIFI-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817192#comment-17817192 ] Matt Burgess commented on NIFI-4385: For large tables, GenerateTableFetch is the way to go. If you have a standalone NiFi instance and want to fetch a ton of rows, it's going to be slow. > Adjust the QueryDatabaseTable processor for handling big tables. > > > Key: NIFI-4385 > URL: https://issues.apache.org/jira/browse/NIFI-4385 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.3.0 >Reporter: Tim Späth >Priority: Major > > When querying large database tables, the *QueryDatabaseTable* processor does > not perform very well. > The processor will always perform the full query and then transfer all > flowfiles as a list instead of > transferring them particularly after the *ResultSet* is fetching the next > rows(If a fetch size is given). > If you want to query a billion rows from a table, > the processor will add all flowfiles in an ArrayList in memory > before transferring the whole list after the last row is fetched by the > ResultSet. > I've checked the code in > *org.apache.nifi.processors.standard.QueryDatabaseTable.java* > and in my opinion, it would be no big deal to move the session.transfer to a > proper position in the code (into the while loop where the flowfile is added > to the list) to > achieve a real _stream support_. There was also a bug report for this problem > which resulted in adding the new property *Maximum Number of Fragments*, > but this property will just limit the results. > Now you have to multiply *Max Rows Per Flow File* with *Maximum Number of > Fragments* to get your limit, > which is not really a solution for the original problem imho. > Also the workaround with GenerateTableFetch and/or ExecuteSQL processors is > much slower than using a database cursor or a ResultSet > and stream the rows in flowfiles directly in the queue. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12784) Only display 'Path Not Found Behavior' property in EvaluateJsonPath when 'Destination' property value is flowfile-attributes
Daniel Stieglitz created NIFI-12784: --- Summary: Only display 'Path Not Found Behavior' property in EvaluateJsonPath when 'Destination' property value is flowfile-attributes Key: NIFI-12784 URL: https://issues.apache.org/jira/browse/NIFI-12784 Project: Apache NiFi Issue Type: Improvement Reporter: Daniel Stieglitz Assignee: Daniel Stieglitz Currently in EvaluateJsonPath the 'Path Not Found Behavior' property which defines behavior for when the 'Destination' property value is is "flowfile-attributes" is even visible when the 'Destination' property value is is "flowfile-content". This is not necessary and could be confusing. This ticket aims to only display the "Path Not Found Behavior" property when the "Destination" property value is flowfile-attributes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12783) Kafka Producer Processors do not route in failure queue on timeout
[ https://issues.apache.org/jira/browse/NIFI-12783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817180#comment-17817180 ] Pierre Villard commented on NIFI-12783: --- Some observations regarding this discussion. * dropping flow files is a rather unusual use case to be honest, this is something that not a lot of users are doing AFAIK * routing the failure relationship back to the processor is not really a good practice anymore, the retry should be directly set on the relationship (this has been added recently in the framework and can be set directly on the relationships tab of the processor's configuration view) * Kafka processors are used very extensively by the community so any change should preserve the current behavior. We may consider a breaking change only for NiFi 2.x at this point but I'm not sure I'd be ok with it Some processors like PutKudu are specifying a property "Failure strategy" where behavior on failure can be specified: rolling back versus routing to failure. IMO the best approach would be to expose a similar property with a default of "rolling back / yielding" to preserve current behavior. One of the use cases where rolling back is absolutely key is when ordering must be preserved which is a very valid use case when sending data to Kafka (CDC use cases for example). Thoughts? > Kafka Producer Processors do not route in failure queue on timeout > -- > > Key: NIFI-12783 > URL: https://issues.apache.org/jira/browse/NIFI-12783 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.23.2 >Reporter: Denis Jakupovic >Priority: Major > > Hi, > the Kafka producer processors do not route the FlowFiles on a timeout e.g. > into the failure connection. They are yielded in the incomming connection. > You can see the behaviour here e.g.: > [https://stackoverflow.com/questions/71460008/apache-nifi-publishkafka-timeout-exception] > I think this is a design flaw. I have a use case where messages should be > dropped after a specific configurable time. If the messages are yielded in > the incomming queue they are always published when the kafka broker are > available again. I know I can set the expiration time in secs or mins in the > incomming queue but it is not dynamically configurable because no attributes > are allowed. > Best > Denis -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10428) Allow for loading Avro record schema from files
[ https://issues.apache.org/jira/browse/NIFI-10428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Stieglitz updated NIFI-10428: Resolution: Won't Fix Status: Resolved (was: Patch Available) > Allow for loading Avro record schema from files > --- > > Key: NIFI-10428 > URL: https://issues.apache.org/jira/browse/NIFI-10428 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.18.0 >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Labels: Record, avro, registry, schema > Time Spent: 0.5h > Remaining Estimate: 0h > > The various record reader and writer controllers XML, CSV, JSON and AVRO, etc > can load schemas from a registry service. The AvroSchemaRegistry can only > load its schema from a text property. It would be useful to have a file > based Avro schema registry, with a configurable refresh interval to allow for > adding, removing or updates to schema files. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12767: Error handling in Provenance and Lineage [nifi]
mcgilman commented on code in PR #8386:
URL: https://github.com/apache/nifi/pull/8386#discussion_r1488649389
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/provenance/state/provenance-event-listing/provenance-event-listing.effects.ts:
##
@@ -58,10 +63,14 @@ export class ProvenanceEventListingEffects {
response
})
),
-catchError((error) =>
+catchError(() =>
of(
-ProvenanceEventListingActions.provenanceApiError({
-error: error.error
+
ProvenanceEventListingActions.loadProvenanceOptionsSuccess({
+response: {
+provenanceOptions: {
+searchableFields: []
+}
+}
Review Comment:
This behavior was from the current NiFi UI. The searchable fields are not
loaded when the dialog opens, but rather once when the page loads. I'll update
the dialog to indicate when there are no searchable fields available.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Comment Edited] (NIFI-12783) Kafka Producer Processors do not route in failure queue on timeout
[ https://issues.apache.org/jira/browse/NIFI-12783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817161#comment-17817161 ] Denis Jakupovic edited comment on NIFI-12783 at 2/13/24 9:23 PM: - "Because other users expect the existing behavior of the PublishKafka processor, we would want to make any changes in a way that would preserve this. Connectivity failures should, by default, yield as they do currently. But a reasonable specialization might involve a new property that specified this alternate routing, perhaps along with a new relationship that could optionally receive relevant FlowFiles on encountering this situation." I think that most people do not expect this behavior. I think NiFi per Design should route any failure into the failure relationship or really yield if the user specifies this in the properties of the processor. In most cases people reroute the failure queue into the processor again and penalize the FF accordingly. This is a standard approach. But if the yield in the incoming queue there are no possibilities to react e.g. route into another processor e.g. do not store into kafka but store into s3. Yes a separate relationship/queue for connectivity issues would be great. Only a relationship lets me decide who to react on connectivity issues. Thank you [~pgrey] was (Author: djakupovic): "Because other users expect the existing behavior of the PublishKafka processor, we would want to make any changes in a way that would preserve this. Connectivity failures should, by default, yield as they do currently. But a reasonable specialization might involve a new property that specified this alternate routing, perhaps along with a new relationship that could optionally receive relevant FlowFiles on encountering this situation." I think that most people do not expect this behavior. I think NiFi per Design should route any failure into the failure relationship or really yield if the user specifies this in the properties of the processor. In most cases people reroute the failure queue into the processor again and penalize the FF accordingly. This is a standard approach. But if the yield in the incoming queue there are no possibilities to react e.g. route into another processor e.g. do not store into kafka put store into s3. Yes a separate relationship/queue for connectivity issues would be great. Only a relationship lets me decide who to react on connectivity issues. Thank you [~pgrey] > Kafka Producer Processors do not route in failure queue on timeout > -- > > Key: NIFI-12783 > URL: https://issues.apache.org/jira/browse/NIFI-12783 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.23.2 >Reporter: Denis Jakupovic >Priority: Major > > Hi, > the Kafka producer processors do not route the FlowFiles on a timeout e.g. > into the failure connection. They are yielded in the incomming connection. > You can see the behaviour here e.g.: > [https://stackoverflow.com/questions/71460008/apache-nifi-publishkafka-timeout-exception] > I think this is a design flaw. I have a use case where messages should be > dropped after a specific configurable time. If the messages are yielded in > the incomming queue they are always published when the kafka broker are > available again. I know I can set the expiration time in secs or mins in the > incomming queue but it is not dynamically configurable because no attributes > are allowed. > Best > Denis -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12783) Kafka Producer Processors do not route in failure queue on timeout
[ https://issues.apache.org/jira/browse/NIFI-12783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817161#comment-17817161 ] Denis Jakupovic commented on NIFI-12783: "Because other users expect the existing behavior of the PublishKafka processor, we would want to make any changes in a way that would preserve this. Connectivity failures should, by default, yield as they do currently. But a reasonable specialization might involve a new property that specified this alternate routing, perhaps along with a new relationship that could optionally receive relevant FlowFiles on encountering this situation." I think that most people do not expect this behavior. I think NiFi per Design should route any failure into the failure relationship or really yield if the user specifies this in the properties of the processor. In most cases people reroute the failure queue into the processor again and penalize the FF accordingly. This is a standard approach. But if the yield in the incoming queue there are no possibilities to react e.g. route into another processor e.g. do not store into kafka put store into s3. Yes a separate relationship/queue for connectivity issues would be great. Only a relationship lets me decide who to react on connectivity issues. Thank you [~pgrey] > Kafka Producer Processors do not route in failure queue on timeout > -- > > Key: NIFI-12783 > URL: https://issues.apache.org/jira/browse/NIFI-12783 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.23.2 >Reporter: Denis Jakupovic >Priority: Major > > Hi, > the Kafka producer processors do not route the FlowFiles on a timeout e.g. > into the failure connection. They are yielded in the incomming connection. > You can see the behaviour here e.g.: > [https://stackoverflow.com/questions/71460008/apache-nifi-publishkafka-timeout-exception] > I think this is a design flaw. I have a use case where messages should be > dropped after a specific configurable time. If the messages are yielded in > the incomming queue they are always published when the kafka broker are > available again. I know I can set the expiration time in secs or mins in the > incomming queue but it is not dynamically configurable because no attributes > are allowed. > Best > Denis -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12783) Kafka Producer Processors do not route in failure queue on timeout
[ https://issues.apache.org/jira/browse/NIFI-12783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817156#comment-17817156 ] Paul Grey commented on NIFI-12783: -- Thanks; I think I understand better. The InvokeHTTP processor on the main branch behaves in a way similar to what you are suggesting. The "Retry" and "No Retry" relationships allow routing of FlowFiles based on a successful HTTP request/response pair, based on the HTTP status code in the response. The "Failure" relationship, on the other hand, is used to route FlowFiles on connection failures / connection timeouts. Because other users expect the existing behavior of the PublishKafka processor, we would want to make any changes in a way that would preserve this. Connectivity failures should, by default, yield as they do currently. But a reasonable specialization might involve a new property that specified this alternate routing, perhaps along with a new relationship that could optionally receive relevant FlowFiles on encountering this situation. A final approach might differ somewhat from this proposed solution, but might something along these lines work for you? > Kafka Producer Processors do not route in failure queue on timeout > -- > > Key: NIFI-12783 > URL: https://issues.apache.org/jira/browse/NIFI-12783 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.23.2 >Reporter: Denis Jakupovic >Priority: Major > > Hi, > the Kafka producer processors do not route the FlowFiles on a timeout e.g. > into the failure connection. They are yielded in the incomming connection. > You can see the behaviour here e.g.: > [https://stackoverflow.com/questions/71460008/apache-nifi-publishkafka-timeout-exception] > I think this is a design flaw. I have a use case where messages should be > dropped after a specific configurable time. If the messages are yielded in > the incomming queue they are always published when the kafka broker are > available again. I know I can set the expiration time in secs or mins in the > incomming queue but it is not dynamically configurable because no attributes > are allowed. > Best > Denis -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12767: Error handling in Provenance and Lineage [nifi]
mcgilman commented on code in PR #8386:
URL: https://github.com/apache/nifi/pull/8386#discussion_r1488532914
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/provenance/state/provenance-event-listing/provenance-event-listing.effects.ts:
##
@@ -311,18 +321,31 @@ export class ProvenanceEventListingEffects {
dialogReference.componentInstance.replay
.pipe(takeUntil(dialogReference.afterClosed()))
.subscribe(() => {
-
this.provenanceService.replay(request.id).subscribe(() => {
-this.store.dispatch(
-
ProvenanceEventListingActions.showOkDialog({
-title: 'Provenance',
-message: 'Successfully
submitted replay request.'
-})
-);
+
this.provenanceService.replay(request.id).subscribe({
+next: () => {
+dialogReference.close();
+
+this.store.dispatch(
+
ProvenanceEventListingActions.showOkDialog({
+title: 'Provenance',
+message: 'Successfully
submitted replay request.'
+})
+);
+},
+error: (errorResponse:
HttpErrorResponse) => {
+this.store.dispatch(
+ErrorActions.snackBarError({
error: errorResponse.error })
+);
Review Comment:
I will also update to close the dialog so it error is shown more clearly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12767: Error handling in Provenance and Lineage [nifi]
mcgilman commented on code in PR #8386:
URL: https://github.com/apache/nifi/pull/8386#discussion_r1488506126
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/provenance/state/lineage/index.ts:
##
@@ -66,7 +66,7 @@ export interface Lineage {
}
export interface LineageState {
-lineage: Lineage | null;
-error: string | null;
+activeLineage: Lineage | null;
+completedLineage: Lineage;
status: 'pending' | 'loading' | 'error' | 'success';
Review Comment:
I think so. It was needed in an earlier iteration of this solution. I'll
remove it here and in the provenance state.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12767: Error handling in Provenance and Lineage [nifi]
mcgilman commented on code in PR #8386:
URL: https://github.com/apache/nifi/pull/8386#discussion_r1488499673
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/queue/state/queue-listing/index.ts:
##
@@ -102,7 +102,7 @@ export interface FlowFileDialogRequest {
export interface QueueListingState {
activeListingRequest: ListingRequest | null;
-completedListingRequest: ListingRequest | null;
+completedListingRequest: ListingRequest;
Review Comment:
Let me spend some time re-evaluating. Rather than being null, I provided an
empty state in the provenance and lineage reducers similar to how the canvas
state is initialized in the flow reducer. I should be able to make this change
but may need further adjustments where this state is consumed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12767: Error handling in Provenance and Lineage [nifi]
mcgilman commented on code in PR #8386: URL: https://github.com/apache/nifi/pull/8386#discussion_r1488493933 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/ui/common/provenance-event-dialog/provenance-event-dialog.component.html: ## @@ -17,6 +17,7 @@ Provenance Event + Review Comment: I meant to delete this `error-banner` because there was no scenario where errors could surface from the dialog directly. If I do, we could punt the contextual banner errors down the road until we hit it again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12767: Error handling in Provenance and Lineage [nifi]
mcgilman commented on code in PR #8386:
URL: https://github.com/apache/nifi/pull/8386#discussion_r1488490343
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/provenance/state/provenance-event-listing/provenance-event-listing.effects.ts:
##
@@ -311,18 +321,31 @@ export class ProvenanceEventListingEffects {
dialogReference.componentInstance.replay
.pipe(takeUntil(dialogReference.afterClosed()))
.subscribe(() => {
-
this.provenanceService.replay(request.id).subscribe(() => {
-this.store.dispatch(
-
ProvenanceEventListingActions.showOkDialog({
-title: 'Provenance',
-message: 'Successfully
submitted replay request.'
-})
-);
+
this.provenanceService.replay(request.id).subscribe({
+next: () => {
+dialogReference.close();
+
+this.store.dispatch(
+
ProvenanceEventListingActions.showOkDialog({
+title: 'Provenance',
+message: 'Successfully
submitted replay request.'
+})
+);
+},
+error: (errorResponse:
HttpErrorResponse) => {
+this.store.dispatch(
+ErrorActions.snackBarError({
error: errorResponse.error })
+);
Review Comment:
I made this choice because the error isn't based on user input (there's
nothing they can change and resubmit) in the dialog and a snackbar error was
used in similar situations like when loading descriptors for new services and
parameters.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-12725) Update json-schema-validator to 1.3.0
[ https://issues.apache.org/jira/browse/NIFI-12725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817122#comment-17817122 ] David Handermann commented on NIFI-12725: - Thanks [~dstiegli1], that would be great! I have assigned it to you. > Update json-schema-validator to 1.3.0 > - > > Key: NIFI-12725 > URL: https://issues.apache.org/jira/browse/NIFI-12725 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M2 >Reporter: Mike R >Assignee: Daniel Stieglitz >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > Update json-schema-validator to 1.3.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-12725) Update json-schema-validator to 1.3.0
[ https://issues.apache.org/jira/browse/NIFI-12725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann reassigned NIFI-12725: --- Assignee: Daniel Stieglitz (was: David Handermann) > Update json-schema-validator to 1.3.0 > - > > Key: NIFI-12725 > URL: https://issues.apache.org/jira/browse/NIFI-12725 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M2 >Reporter: Mike R >Assignee: Daniel Stieglitz >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > Update json-schema-validator to 1.3.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12725) Update json-schema-validator to 1.3.0
[ https://issues.apache.org/jira/browse/NIFI-12725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817120#comment-17817120 ] Daniel Stieglitz commented on NIFI-12725: - [~exceptionfactory] Do you mind if I take this? > Update json-schema-validator to 1.3.0 > - > > Key: NIFI-12725 > URL: https://issues.apache.org/jira/browse/NIFI-12725 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M2 >Reporter: Mike R >Assignee: David Handermann >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > Update json-schema-validator to 1.3.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (NIFI-12725) Update json-schema-validator to 1.3.0
[ https://issues.apache.org/jira/browse/NIFI-12725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817120#comment-17817120 ] Daniel Stieglitz edited comment on NIFI-12725 at 2/13/24 7:25 PM: -- [~exceptionfactory] Do you mind if I take this? I believe I see what has to be done to fix this. was (Author: JIRAUSER294662): [~exceptionfactory] Do you mind if I take this? > Update json-schema-validator to 1.3.0 > - > > Key: NIFI-12725 > URL: https://issues.apache.org/jira/browse/NIFI-12725 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M2 >Reporter: Mike R >Assignee: David Handermann >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > Update json-schema-validator to 1.3.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [NIFI-12774] configure RPG [nifi]
scottyaslan commented on code in PR #8398:
URL: https://github.com/apache/nifi/pull/8398#discussion_r1488429184
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-designer/ui/canvas/items/remote-process-group/edit-remote-process-group/edit-remote-process-group.component.html:
##
@@ -0,0 +1,123 @@
+
+
+Edit Remote Process Group
+
+
+
+
+Name
+{{ request.entity.component.name }}
+
+
+Id
+{{ request.entity.component.id }}
+
+
+
+
+URLs
+https://remotehost:8443/nifi"; />
+
+
+
+
+
+
+Transport Protocol
+
+
+RAW
+
+
+HTTP
+
Review Comment:
ok I can remove these. I just saw they were present in current nifi so I
added them here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12767: Error handling in Provenance and Lineage [nifi]
rfellows commented on code in PR #8386:
URL: https://github.com/apache/nifi/pull/8386#discussion_r1488335062
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/queue/state/queue-listing/index.ts:
##
@@ -102,7 +102,7 @@ export interface FlowFileDialogRequest {
export interface QueueListingState {
activeListingRequest: ListingRequest | null;
-completedListingRequest: ListingRequest | null;
+completedListingRequest: ListingRequest;
Review Comment:
It seems peculiar that the `completedListingRequest` would never be null. I
would assume it would remain null until there is a request that has completed.
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/provenance/ui/provenance-event-listing/provenance-event-table/provenance-event-table.component.spec.ts:
##
@@ -20,14 +20,29 @@ import { ComponentFixture, TestBed } from
'@angular/core/testing';
import { ProvenanceEventTable } from './provenance-event-table.component';
import { MatTableModule } from '@angular/material/table';
import { BrowserAnimationsModule } from '@angular/platform-browser/animations';
+import { provideMockStore } from '@ngrx/store/testing';
+import { initialState } from '../../../../../state/error/error.reducer';
+import { Component } from '@angular/core';
describe('ProvenanceEventTable', () => {
let component: ProvenanceEventTable;
let fixture: ComponentFixture;
+@Component({
+selector: 'error-banner',
+standalone: true,
+template: ''
+})
+class MockErrorBanner {}
+
beforeEach(() => {
TestBed.configureTestingModule({
-imports: [ProvenanceEventTable, MatTableModule,
BrowserAnimationsModule]
+imports: [ProvenanceEventTable, MockErrorBanner, MatTableModule,
BrowserAnimationsModule],
Review Comment:
In general, we should be using `NoopAnimationsModule` rather than
`BrowserAnimationsModule` in tests to make them as efficient as possible.
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/provenance/state/lineage/index.ts:
##
@@ -66,7 +66,7 @@ export interface Lineage {
}
export interface LineageState {
-lineage: Lineage | null;
-error: string | null;
+activeLineage: Lineage | null;
+completedLineage: Lineage;
status: 'pending' | 'loading' | 'error' | 'success';
Review Comment:
Can we remove `error`?
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/ui/common/provenance-event-dialog/provenance-event-dialog.component.html:
##
@@ -17,6 +17,7 @@
Provenance Event
+
Review Comment:
We might need to devise some way of giving these banners some context. I
found a scenario where I had a banner error while on the lineage screen then
opened the View Details option of one of the nodes and now the error is in 2
places at the same time. Dismissing one, dismisses them both.
https://github.com/apache/nifi/assets/713866/edd94d1f-d187-4a7d-900b-b32ee7fac53d";>
Maybe we need to specify the source of the banner error in the store
(optionally???) when firing them, and then the banner elements can take an
input of the context they should be interested in?
```
```
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/provenance/ui/provenance-event-listing/provenance-event-table/provenance-event-table.component.html:
##
@@ -15,7 +15,8 @@
~ limitations under the License.
-->
-
+
+
Review Comment:
As this banner is used for both provenance and lineage issues, we might need
to clear them when we switch from one view to the other.
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/provenance/state/provenance-event-listing/provenance-event-listing.effects.ts:
##
@@ -205,18 +211,26 @@ export class ProvenanceEventListingEffects {
)
);
-deleteProvenanceQuery$ = createEffect(
-() =>
-this.actions$.pipe(
-ofType(ProvenanceEventListingActions.deleteProvenanceQuery),
-concatLatestFrom(() => [this.store.select(selectProvenanceId),
this.store.select(selectClusterNodeId)]),
-tap(([, id, clusterNodeId]) => {
-if (id) {
-this.provenanceService.deleteProvenanceQuery(id,
clusterNodeId).subscribe();
-}
-})
-),
-{ dispatch: false }
+deleteProvenanceQuery$ = createEffect(() =>
+this.actions$.pipe(
+ofType(ProvenanceEventListingActions.deleteProvenanceQuery),
+concatLatestFrom(() => [
+
[jira] [Updated] (NIFI-12777) QueryRecord does not support UUID field type
[
https://issues.apache.org/jira/browse/NIFI-12777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-12777:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> QueryRecord does not support UUID field type
>
>
> Key: NIFI-12777
> URL: https://issues.apache.org/jira/browse/NIFI-12777
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.25.0
> Environment: windows
>Reporter: Nikolas Falco
>Priority: Major
> Fix For: 1.26.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> I have a QueryRecord processor that takes a flowfile as input. This flowfile
> contains records with some fields of type UUID.
> {noformat}
> Objavro.schema„N{"type":"record","name":"nifiRecord","namespace":"org.apache.nifi","fields":[{"name":"obbligato_cognome","type":["string","null"]},{"name":"data_inserimento","type":[{"type":"long","logicalType":"timestamp-millis"},"null"]},{"name":"obbligato_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"trasgressore_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"id_verbale","type":["string","null"]},{"name":"stato_verbale","type":["string","null"]},{"name":"obbligato_comune_sede_legale_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"tipo_verbale","type":["string","null"]},{"name":"obbligato_indirizzo_residenza","type":["string","null"]},{"name":"trasgressore_tipo_anagrafica","type":["string","null"]},{"name":"tipo_accertamento_nome","type":["string","null"]},{"name":"importo_scontato","type":[{"type":"bytes","logicalType":"decimal","precision":15,"scale":6},"null"]},{"name":"obbligato_comune_residenza_id","type":["string","null"]},{"name":"luogo_comune_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"obbligato_data_nascita","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"data_spedizione_verbale","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"trasgressore_esito_spedizione","type":["string","null"]},{"name":"obbligato_nome","type":["string","null"]},{"name":"luogo_strada","type":["string","null"]},{"name":"gruppo_nome_comune","type":["string","null"]},{"name":"tipo_accertamento_descrizione","type":["string","null"]},{"name":"agente_accertatore","type":["string","null"]},{"name":"data_decorrenza_verbale","type":[{"type":"long","logicalType":"timestamp-millis"},"null"]},{"name":"trasgressore_civico_residenza","type":["string","null"]},{"name":"trasgressore_cognome","type":["string","null"]},{"name":"tipo_accertamento_codice_veloce","type":["string","null"]},{"name":"trasgressore_rappresentante_legale","type":["string","null"]},{"name":"aggId","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"trasgressore_comune_residenza_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"obbligato_tipo_anagrafica","type":["string","null"]},{"name":"obbligato_sesso","type":["string","null"]},{"name":"trasgressore_data_spedizione","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"obbligato_ragione_sociale","type":["string","null"]},{"name":"obbligato_indirizzo_sede_legale","type":["string","null"]},{"name":"obbligato_data_notifica","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"importo_minimo_edittale","type":[{"type":"bytes","logicalType":"decimal","precision":15,"scale":6},"null"]},{"name":"trasgressore_cf_pi","type":["string","null"]},{"name":"veicolo_stato_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"trasgressore_nome","type":["string","null"]},{"name":"trasgressore_cap_comune_residenza","type":["string","null"]},{"name":"trasgressore_comune_sede_legale_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"obbligato_cf_pi","type":["string","null"]},{"name":"trasgressore_ragione_sociale","type":["string","null"]},{"name":"stazione_descrizione","type":["string","null"]},{"name":"obbligato_rappresentante_legale","type":["string","null"]},{"name":"gruppo_descizione","type":["string","null"]},{"name":"obbligato_data_spedizione","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"trasgressore_civico_sede_legale","type":["string","null"]},{"name":"data_notifica_verbale","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"obbligato_cap_comune_residenza","type":["string","null"]},{"name":"trasgressore_data_notifica","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"trasgressore_data_nascita","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"trasgressore_indirizzo_residenza","type":["string","null"]},{"name":"gruppo_comando_id","type":["string","null"]},{"name":"obbligato_esito_spedizione","type":["string","null"]},{"name":"data_verbale","type":[{"type":"long","logicalType":"
[jira] [Commented] (NIFI-12777) QueryRecord does not support UUID field type
[
https://issues.apache.org/jira/browse/NIFI-12777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817106#comment-17817106
]
ASF subversion and git services commented on NIFI-12777:
Commit b262f96b5671802f775034db4d46c0848c2a246f in nifi's branch
refs/heads/support/nifi-1.x from Nikolas Falco
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b262f96b56 ]
NIFI-12777 Add support for UUID record field type in QueryRecord processor
Signed-off-by: Matt Burgess
This closes #8395
> QueryRecord does not support UUID field type
>
>
> Key: NIFI-12777
> URL: https://issues.apache.org/jira/browse/NIFI-12777
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.25.0
> Environment: windows
>Reporter: Nikolas Falco
>Priority: Major
> Fix For: 1.26.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I have a QueryRecord processor that takes a flowfile as input. This flowfile
> contains records with some fields of type UUID.
> {noformat}
> Objavro.schema„N{"type":"record","name":"nifiRecord","namespace":"org.apache.nifi","fields":[{"name":"obbligato_cognome","type":["string","null"]},{"name":"data_inserimento","type":[{"type":"long","logicalType":"timestamp-millis"},"null"]},{"name":"obbligato_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"trasgressore_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"id_verbale","type":["string","null"]},{"name":"stato_verbale","type":["string","null"]},{"name":"obbligato_comune_sede_legale_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"tipo_verbale","type":["string","null"]},{"name":"obbligato_indirizzo_residenza","type":["string","null"]},{"name":"trasgressore_tipo_anagrafica","type":["string","null"]},{"name":"tipo_accertamento_nome","type":["string","null"]},{"name":"importo_scontato","type":[{"type":"bytes","logicalType":"decimal","precision":15,"scale":6},"null"]},{"name":"obbligato_comune_residenza_id","type":["string","null"]},{"name":"luogo_comune_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"obbligato_data_nascita","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"data_spedizione_verbale","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"trasgressore_esito_spedizione","type":["string","null"]},{"name":"obbligato_nome","type":["string","null"]},{"name":"luogo_strada","type":["string","null"]},{"name":"gruppo_nome_comune","type":["string","null"]},{"name":"tipo_accertamento_descrizione","type":["string","null"]},{"name":"agente_accertatore","type":["string","null"]},{"name":"data_decorrenza_verbale","type":[{"type":"long","logicalType":"timestamp-millis"},"null"]},{"name":"trasgressore_civico_residenza","type":["string","null"]},{"name":"trasgressore_cognome","type":["string","null"]},{"name":"tipo_accertamento_codice_veloce","type":["string","null"]},{"name":"trasgressore_rappresentante_legale","type":["string","null"]},{"name":"aggId","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"trasgressore_comune_residenza_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"obbligato_tipo_anagrafica","type":["string","null"]},{"name":"obbligato_sesso","type":["string","null"]},{"name":"trasgressore_data_spedizione","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"obbligato_ragione_sociale","type":["string","null"]},{"name":"obbligato_indirizzo_sede_legale","type":["string","null"]},{"name":"obbligato_data_notifica","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"importo_minimo_edittale","type":[{"type":"bytes","logicalType":"decimal","precision":15,"scale":6},"null"]},{"name":"trasgressore_cf_pi","type":["string","null"]},{"name":"veicolo_stato_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"trasgressore_nome","type":["string","null"]},{"name":"trasgressore_cap_comune_residenza","type":["string","null"]},{"name":"trasgressore_comune_sede_legale_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"obbligato_cf_pi","type":["string","null"]},{"name":"trasgressore_ragione_sociale","type":["string","null"]},{"name":"stazione_descrizione","type":["string","null"]},{"name":"obbligato_rappresentante_legale","type":["string","null"]},{"name":"gruppo_descizione","type":["string","null"]},{"name":"obbligato_data_spedizione","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"trasgressore_civico_sede_legale","type":["string","null"]},{"name":"data_notifica_verbale","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"obbligato_cap_comune_residenza","type":["string","null"]},{"name":"trasgressore_data_notifica","type":[{"type":"int","logicalType":"date"},"null"]},{"name":
Re: [PR] NIFI-12777 Add support for UUID record field type in QueryRecord processor [nifi]
mattyb149 closed pull request #8395: NIFI-12777 Add support for UUID record field type in QueryRecord processor URL: https://github.com/apache/nifi/pull/8395 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12777 Add support for UUID record field type in QueryRecord processor [nifi]
mattyb149 commented on PR #8395: URL: https://github.com/apache/nifi/pull/8395#issuecomment-1942142991 +1 LGTM, thanks for the fix! Merging to support/nifi-1.x -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12777) QueryRecord does not support UUID field type
[
https://issues.apache.org/jira/browse/NIFI-12777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-12777:
Fix Version/s: 1.26.0
> QueryRecord does not support UUID field type
>
>
> Key: NIFI-12777
> URL: https://issues.apache.org/jira/browse/NIFI-12777
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.25.0
> Environment: windows
>Reporter: Nikolas Falco
>Priority: Major
> Fix For: 1.26.0
>
>
> I have a QueryRecord processor that takes a flowfile as input. This flowfile
> contains records with some fields of type UUID.
> {noformat}
> Objavro.schema„N{"type":"record","name":"nifiRecord","namespace":"org.apache.nifi","fields":[{"name":"obbligato_cognome","type":["string","null"]},{"name":"data_inserimento","type":[{"type":"long","logicalType":"timestamp-millis"},"null"]},{"name":"obbligato_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"trasgressore_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"id_verbale","type":["string","null"]},{"name":"stato_verbale","type":["string","null"]},{"name":"obbligato_comune_sede_legale_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"tipo_verbale","type":["string","null"]},{"name":"obbligato_indirizzo_residenza","type":["string","null"]},{"name":"trasgressore_tipo_anagrafica","type":["string","null"]},{"name":"tipo_accertamento_nome","type":["string","null"]},{"name":"importo_scontato","type":[{"type":"bytes","logicalType":"decimal","precision":15,"scale":6},"null"]},{"name":"obbligato_comune_residenza_id","type":["string","null"]},{"name":"luogo_comune_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"obbligato_data_nascita","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"data_spedizione_verbale","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"trasgressore_esito_spedizione","type":["string","null"]},{"name":"obbligato_nome","type":["string","null"]},{"name":"luogo_strada","type":["string","null"]},{"name":"gruppo_nome_comune","type":["string","null"]},{"name":"tipo_accertamento_descrizione","type":["string","null"]},{"name":"agente_accertatore","type":["string","null"]},{"name":"data_decorrenza_verbale","type":[{"type":"long","logicalType":"timestamp-millis"},"null"]},{"name":"trasgressore_civico_residenza","type":["string","null"]},{"name":"trasgressore_cognome","type":["string","null"]},{"name":"tipo_accertamento_codice_veloce","type":["string","null"]},{"name":"trasgressore_rappresentante_legale","type":["string","null"]},{"name":"aggId","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"trasgressore_comune_residenza_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"obbligato_tipo_anagrafica","type":["string","null"]},{"name":"obbligato_sesso","type":["string","null"]},{"name":"trasgressore_data_spedizione","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"obbligato_ragione_sociale","type":["string","null"]},{"name":"obbligato_indirizzo_sede_legale","type":["string","null"]},{"name":"obbligato_data_notifica","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"importo_minimo_edittale","type":[{"type":"bytes","logicalType":"decimal","precision":15,"scale":6},"null"]},{"name":"trasgressore_cf_pi","type":["string","null"]},{"name":"veicolo_stato_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"trasgressore_nome","type":["string","null"]},{"name":"trasgressore_cap_comune_residenza","type":["string","null"]},{"name":"trasgressore_comune_sede_legale_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"obbligato_cf_pi","type":["string","null"]},{"name":"trasgressore_ragione_sociale","type":["string","null"]},{"name":"stazione_descrizione","type":["string","null"]},{"name":"obbligato_rappresentante_legale","type":["string","null"]},{"name":"gruppo_descizione","type":["string","null"]},{"name":"obbligato_data_spedizione","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"trasgressore_civico_sede_legale","type":["string","null"]},{"name":"data_notifica_verbale","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"obbligato_cap_comune_residenza","type":["string","null"]},{"name":"trasgressore_data_notifica","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"trasgressore_data_nascita","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"trasgressore_indirizzo_residenza","type":["string","null"]},{"name":"gruppo_comando_id","type":["string","null"]},{"name":"obbligato_esito_spedizione","type":["string","null"]},{"name":"data_verbale","type":[{"type":"long","logicalType":"timestamp-millis"},"null"]},{"name":"agente_proprietario","type":["string","null"]},{"name":"trasg
[jira] [Updated] (NIFI-12777) QueryRecord does not support UUID field type
[
https://issues.apache.org/jira/browse/NIFI-12777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-12777:
Status: Patch Available (was: Open)
> QueryRecord does not support UUID field type
>
>
> Key: NIFI-12777
> URL: https://issues.apache.org/jira/browse/NIFI-12777
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.25.0
> Environment: windows
>Reporter: Nikolas Falco
>Priority: Major
>
> I have a QueryRecord processor that takes a flowfile as input. This flowfile
> contains records with some fields of type UUID.
> {noformat}
> Objavro.schema„N{"type":"record","name":"nifiRecord","namespace":"org.apache.nifi","fields":[{"name":"obbligato_cognome","type":["string","null"]},{"name":"data_inserimento","type":[{"type":"long","logicalType":"timestamp-millis"},"null"]},{"name":"obbligato_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"trasgressore_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"id_verbale","type":["string","null"]},{"name":"stato_verbale","type":["string","null"]},{"name":"obbligato_comune_sede_legale_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"tipo_verbale","type":["string","null"]},{"name":"obbligato_indirizzo_residenza","type":["string","null"]},{"name":"trasgressore_tipo_anagrafica","type":["string","null"]},{"name":"tipo_accertamento_nome","type":["string","null"]},{"name":"importo_scontato","type":[{"type":"bytes","logicalType":"decimal","precision":15,"scale":6},"null"]},{"name":"obbligato_comune_residenza_id","type":["string","null"]},{"name":"luogo_comune_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"obbligato_data_nascita","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"data_spedizione_verbale","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"trasgressore_esito_spedizione","type":["string","null"]},{"name":"obbligato_nome","type":["string","null"]},{"name":"luogo_strada","type":["string","null"]},{"name":"gruppo_nome_comune","type":["string","null"]},{"name":"tipo_accertamento_descrizione","type":["string","null"]},{"name":"agente_accertatore","type":["string","null"]},{"name":"data_decorrenza_verbale","type":[{"type":"long","logicalType":"timestamp-millis"},"null"]},{"name":"trasgressore_civico_residenza","type":["string","null"]},{"name":"trasgressore_cognome","type":["string","null"]},{"name":"tipo_accertamento_codice_veloce","type":["string","null"]},{"name":"trasgressore_rappresentante_legale","type":["string","null"]},{"name":"aggId","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"trasgressore_comune_residenza_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"obbligato_tipo_anagrafica","type":["string","null"]},{"name":"obbligato_sesso","type":["string","null"]},{"name":"trasgressore_data_spedizione","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"obbligato_ragione_sociale","type":["string","null"]},{"name":"obbligato_indirizzo_sede_legale","type":["string","null"]},{"name":"obbligato_data_notifica","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"importo_minimo_edittale","type":[{"type":"bytes","logicalType":"decimal","precision":15,"scale":6},"null"]},{"name":"trasgressore_cf_pi","type":["string","null"]},{"name":"veicolo_stato_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"trasgressore_nome","type":["string","null"]},{"name":"trasgressore_cap_comune_residenza","type":["string","null"]},{"name":"trasgressore_comune_sede_legale_id","type":[{"type":"string","logicalType":"uuid"},"null"]},{"name":"obbligato_cf_pi","type":["string","null"]},{"name":"trasgressore_ragione_sociale","type":["string","null"]},{"name":"stazione_descrizione","type":["string","null"]},{"name":"obbligato_rappresentante_legale","type":["string","null"]},{"name":"gruppo_descizione","type":["string","null"]},{"name":"obbligato_data_spedizione","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"trasgressore_civico_sede_legale","type":["string","null"]},{"name":"data_notifica_verbale","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"obbligato_cap_comune_residenza","type":["string","null"]},{"name":"trasgressore_data_notifica","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"trasgressore_data_nascita","type":[{"type":"int","logicalType":"date"},"null"]},{"name":"trasgressore_indirizzo_residenza","type":["string","null"]},{"name":"gruppo_comando_id","type":["string","null"]},{"name":"obbligato_esito_spedizione","type":["string","null"]},{"name":"data_verbale","type":[{"type":"long","logicalType":"timestamp-millis"},"null"]},{"name":"agente_proprietario","type":["string","null"]},{"name":"trasgressore_indirizzo
[jira] [Commented] (NIFI-12783) Kafka Producer Processors do not route in failure queue on timeout
[ https://issues.apache.org/jira/browse/NIFI-12783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817098#comment-17817098 ] Denis Jakupovic commented on NIFI-12783: Hi [~pgrey] thank you for your response. - Are we talking about PublishKafka_2_6 or PublishKafkaRecord_2_6? -- both - Is there a network timeout connecting to Kafka? -- yes or even a wrong bootstrap address - Is there a network timeout sending to Kafka? -- kafka is down and NiFi cannot write into kafka - Do you have any stack traces from your NiFi app log at the point in time where you see a yield to an incoming connection? -- No but it is easy to reproduce, just put a wrong bootstrap address and try to publish the content of a flowfile. The FF will yield in the incoming connection of the Kafka Publish Processor - What do you mean about needing dynamically configurable expiration times in the incoming queue? -- If the processor cannot send the data into kafka, and if on failure the data are routed into the failure queue, I can determine for myself what I want to do with the data not successfully written into Kafka. Now the data is yielding before the processor and I cannot react on the kafka timeout - If a queue could expire FlowFiles after an interval based on FlowFile attributes, would that meet your need? -- Yes this is my current solution. I expire FF if they are yielded in the incoming Kafka queue. > Kafka Producer Processors do not route in failure queue on timeout > -- > > Key: NIFI-12783 > URL: https://issues.apache.org/jira/browse/NIFI-12783 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.23.2 >Reporter: Denis Jakupovic >Priority: Major > > Hi, > the Kafka producer processors do not route the FlowFiles on a timeout e.g. > into the failure connection. They are yielded in the incomming connection. > You can see the behaviour here e.g.: > [https://stackoverflow.com/questions/71460008/apache-nifi-publishkafka-timeout-exception] > I think this is a design flaw. I have a use case where messages should be > dropped after a specific configurable time. If the messages are yielded in > the incomming queue they are always published when the kafka broker are > available again. I know I can set the expiration time in secs or mins in the > incomming queue but it is not dynamically configurable because no attributes > are allowed. > Best > Denis -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12783) Kafka Producer Processors do not route in failure queue on timeout
[ https://issues.apache.org/jira/browse/NIFI-12783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17817092#comment-17817092 ] Paul Grey commented on NIFI-12783: -- Can you share more detail about your observations? - Are we talking about PublishKafka_2_6 or PublishKafkaRecord_2_6? - Is there a network timeout connecting to Kafka? - Is there a network timeout sending to Kafka? - Do you have any stack traces from your NiFi app log at the point in time where you see a yield to an incoming connection? - What do you mean about needing dynamically configurable expiration times in the incoming queue? - If a queue could expire FlowFiles after an interval based on FlowFile attributes, would that meet your need? >From a brief scan of the processor code, it looks like the yield() is reserved >for situations where connectivity to Kafka cannot be established. Are you >seeing something different? > Kafka Producer Processors do not route in failure queue on timeout > -- > > Key: NIFI-12783 > URL: https://issues.apache.org/jira/browse/NIFI-12783 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.23.2 >Reporter: Denis Jakupovic >Priority: Major > > Hi, > the Kafka producer processors do not route the FlowFiles on a timeout e.g. > into the failure connection. They are yielded in the incomming connection. > You can see the behaviour here e.g.: > [https://stackoverflow.com/questions/71460008/apache-nifi-publishkafka-timeout-exception] > I think this is a design flaw. I have a use case where messages should be > dropped after a specific configurable time. If the messages are yielded in > the incomming queue they are always published when the kafka broker are > available again. I know I can set the expiration time in secs or mins in the > incomming queue but it is not dynamically configurable because no attributes > are allowed. > Best > Denis -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] MINIFICPP-2278 Add dynamic relationship support for python processors [nifi-minifi-cpp]
lordgamez commented on code in PR #1722: URL: https://github.com/apache/nifi-minifi-cpp/pull/1722#discussion_r1488278314 ## docker/test/integration/resources/python/RotatingForwarder.py: ## Review Comment: The goal of this PR is to support the same dynamic relationships that NiFi has in their python processors. Previously python processors only supported the predefined "success", "failure" and "original" relationships predefined in the ExecutePythonProcessor processor and bound to the REL_SUCCESS, REL_FAILURE and REL_ORIGINAL python objects. In NiFi the result of the `transform` method is a `FlowFileTransformResult` object where the relationship can be any custom string that represents a relationship. This is the dynamic relationship we would like to support, which may differ from the dynamic relationship concept previously used in MiNiFi C++. Not sure if we should use the "supports dynamic relationships" attribute here as well, or that should be reverted to false. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2278 Add dynamic relationship support for python processors [nifi-minifi-cpp]
lordgamez commented on code in PR #1722: URL: https://github.com/apache/nifi-minifi-cpp/pull/1722#discussion_r1488278314 ## docker/test/integration/resources/python/RotatingForwarder.py: ## Review Comment: The goal of this PR is to support the same dynamic relationships that NiFi has in their python processors. Previously python processors only supported the predefined "success", "failure" and "original" relationships predefined in the ExecutePythonProcessor processor and bound to the REL_SUCCESS, REL_FAILURE and REL_ORIGINAL python objects. In NiFi the result of the `transform` method is a `FlowFileTransformResult` object where the relationship can be any custom string that represents a relationship. This is the dynamic relationship we would like to support, which may differ from the dynamic relationship concept previously used in MiNiFi C++. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINIFICPP-2278 Add dynamic relationship support for python processors [nifi-minifi-cpp]
fgerlits commented on code in PR #1722: URL: https://github.com/apache/nifi-minifi-cpp/pull/1722#discussion_r1488262406 ## docker/test/integration/resources/python/RotatingForwarder.py: ## Review Comment: This is not the best example of dynamic relationships, since all the relationships are hard-coded into the processor. I understand that even this was not supported until now, but in the rest of the code (eg. RouteText, RouteOnAttribute), "supports dynamic relationships" means the user can add (outgoing) relationships by configuring the processor's properties. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [NIFI-12754] - Flow Configuration History [nifi]
mcgilman commented on code in PR #8399:
URL: https://github.com/apache/nifi/pull/8399#discussion_r1488211642
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-configuration-history/feature/flow-configuration-history-routing.module.ts:
##
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { RouterModule, Routes } from '@angular/router';
+import { FlowConfigurationHistory } from
'./flow-configuration-history.component';
+import { NgModule } from '@angular/core';
+
+const routes: Routes = [
+{
+path: '',
+component: FlowConfigurationHistory,
+children: [
+{
+path: ':id',
Review Comment:
I agree with the comparison to provenance. Removing the action specific
route sounds good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [NIFI-12774] configure RPG [nifi]
mcgilman commented on code in PR #8398:
URL: https://github.com/apache/nifi/pull/8398#discussion_r1488195104
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-designer/state/flow/flow.effects.ts:
##
@@ -1250,6 +1255,54 @@ export class FlowEffects {
{ dispatch: false }
);
+openEditRemoteProcessGroupDialog$ = createEffect(
+() =>
+this.actions$.pipe(
+ofType(FlowActions.openEditRemoteProcessGroupDialog),
+map((action) => action.request),
+tap((request) => {
+const editDialogReference =
this.dialog.open(EditRemoteProcessGroup, {
+data: request,
+panelClass: 'large-dialog'
+});
+
+editDialogReference.componentInstance.saving$ =
this.store.select(selectSaving);
+
+
editDialogReference.componentInstance.editRemoteProcessGroup
+.pipe(takeUntil(editDialogReference.afterClosed()))
+.subscribe((payload: any) => {
+this.store.dispatch(
+FlowActions.updateComponent({
+request: {
+id: request.entity.id,
+uri: request.uri,
+type: request.type,
+payload
+}
+})
+);
+});
+
+editDialogReference.afterClosed().subscribe(() => {
+this.store.dispatch(FlowActions.clearFlowApiError());
+this.store.dispatch(
+FlowActions.selectComponents({
+request: {
+components: [
+{
+id: request.entity.id,
+componentType: request.type
+}
+]
+}
+})
+);
Review Comment:
We should conditionally dispatch this action like we do in other `EditXyz`
dialogs if the user isn't routed elsewhere. We have an interceptor that may
forward to an error page. If that happens we have a fallback to ensure that all
dialogs are closed and the close payload will be `ROUTED`. If this were to
occur we wouldn't want `selectComponents` to be dispatched.
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-designer/ui/canvas/items/remote-process-group/edit-remote-process-group/edit-remote-process-group.component.html:
##
@@ -0,0 +1,123 @@
+
+
+Edit Remote Process Group
+
+
+
+
+Name
+{{ request.entity.component.name }}
+
+
+Id
+{{ request.entity.component.id }}
+
+
+
+
+URLs
+https://remotehost:8443/nifi"; />
+
+
+
+
+
+
+Transport Protocol
+
+
+RAW
+
+
+HTTP
+
Review Comment:
Thus far we've only used a tooltip on `mat-option` when they have additional
description beyond the option text.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [NIFI-12774] configure RPG [nifi]
mcgilman commented on PR #8398: URL: https://github.com/apache/nifi/pull/8398#issuecomment-1941910849 Will review... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [NIFI-12761] refresh RPG [nifi]
scottyaslan commented on code in PR #8401:
URL: https://github.com/apache/nifi/pull/8401#discussion_r1488041318
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-designer/state/flow/flow.effects.ts:
##
@@ -348,14 +352,78 @@ export class FlowEffects {
this.flowService.goToRemoteProcessGroup(request);
} else {
this.store.dispatch(
-showOkDialog({ title: 'Remote Process Group',
message: 'No target URI defined.' })
+FlowActions.showOkDialog({
+title: 'Remote Process Group',
+message: 'No target URI defined.'
+})
);
}
})
),
{ dispatch: false }
);
+startRemoteProcessGroupPolling$ = createEffect(() =>
+this.actions$.pipe(
+ofType(FlowActions.startRemoteProcessGroupPolling),
+switchMap((request) => {
+return interval(3000, asyncScheduler).pipe(
+
takeUntil(this.actions$.pipe(ofType(FlowActions.stopRemoteProcessGroupPolling)))
+);
+}),
+switchMap(() => {
+return of(FlowActions.refreshRemoteProcessGroup());
+})
+)
+);
+
+requestRemoteProcessGroupPolling$ = createEffect(() =>
+this.actions$.pipe(
+ofType(FlowActions.requestRemoteProcessGroupPolling),
+switchMap(() => {
+return of(FlowActions.startRemoteProcessGroupPolling());
+})
+)
+);
+
+refreshRemoteProcessGroup$ = createEffect(() =>
+this.actions$.pipe(
+ofType(FlowActions.refreshRemoteProcessGroup),
+concatLatestFrom(() => this.store.select(selectRpgToPoll)),
+map(([, rpgToPoll]) => {
+return rpgToPoll;
+}),
+isDefinedAndNotNull(),
+switchMap((rpgToPoll) =>
+from(
+this.flowService.getRemoteProcessGroup(rpgToPoll.id).pipe(
+map((response: any) => {
+const entity = response;
+
+if (rpgToPoll.refreshTimestamp !==
response.component.flowRefreshed) {
+
this.store.dispatch(FlowActions.stopRemoteProcessGroupPolling());
+} else {
+entity.component.flowRefreshed =
'Refreshing...';
+}
+
+return FlowActions.loadRemoteProcessGroupSuccess({
+response: {
+id: entity.id,
+remoteProcessGroup: entity
+}
+});
+}),
+catchError((errorResponse: HttpErrorResponse) => {
+
this.store.dispatch(FlowActions.stopRemoteProcessGroupPolling());
+
+return
of(this.errorHelper.handleLoadingError(status, errorResponse));
Review Comment:
I think this is a copy past error...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [NIFI-12761] refresh RPG [nifi]
scottyaslan commented on code in PR #8401:
URL: https://github.com/apache/nifi/pull/8401#discussion_r1488040708
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-designer/service/canvas-context-menu.service.ts:
##
@@ -740,8 +741,15 @@ export class CanvasContextMenu implements
ContextMenuDefinitionProvider {
},
clazz: 'fa fa-refresh',
text: 'Refresh remote',
-action: () => {
-// TODO - refreshRemoteFlow
+action: (selection: any) => {
+const d = selection.datum();
Review Comment:
Ok... does this mean that the 'Manage Remote Ports' action will need to
check `this.canvasUtils.canRead(selection) &&
this.canvasUtils.canModify(selection)`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [NIFI-12761] refresh RPG [nifi]
mcgilman commented on code in PR #8401:
URL: https://github.com/apache/nifi/pull/8401#discussion_r1487935809
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-designer/service/canvas-context-menu.service.ts:
##
@@ -740,8 +741,15 @@ export class CanvasContextMenu implements
ContextMenuDefinitionProvider {
},
clazz: 'fa fa-refresh',
text: 'Refresh remote',
-action: () => {
-// TODO - refreshRemoteFlow
+action: (selection: any) => {
+const d = selection.datum();
Review Comment:
The condition for both this action and the condition for `Go To` need to be
updated to also check `this.canvasUtils.canRead(selection)`.
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-designer/state/flow/flow.effects.ts:
##
@@ -348,14 +352,78 @@ export class FlowEffects {
this.flowService.goToRemoteProcessGroup(request);
} else {
this.store.dispatch(
-showOkDialog({ title: 'Remote Process Group',
message: 'No target URI defined.' })
+FlowActions.showOkDialog({
+title: 'Remote Process Group',
+message: 'No target URI defined.'
+})
);
}
})
),
{ dispatch: false }
);
+startRemoteProcessGroupPolling$ = createEffect(() =>
+this.actions$.pipe(
+ofType(FlowActions.startRemoteProcessGroupPolling),
+switchMap((request) => {
+return interval(3000, asyncScheduler).pipe(
+
takeUntil(this.actions$.pipe(ofType(FlowActions.stopRemoteProcessGroupPolling)))
+);
+}),
+switchMap(() => {
+return of(FlowActions.refreshRemoteProcessGroup());
+})
+)
+);
+
+requestRemoteProcessGroupPolling$ = createEffect(() =>
+this.actions$.pipe(
+ofType(FlowActions.requestRemoteProcessGroupPolling),
+switchMap(() => {
+return of(FlowActions.startRemoteProcessGroupPolling());
+})
+)
+);
+
+refreshRemoteProcessGroup$ = createEffect(() =>
+this.actions$.pipe(
+ofType(FlowActions.refreshRemoteProcessGroup),
+concatLatestFrom(() => this.store.select(selectRpgToPoll)),
+map(([, rpgToPoll]) => {
+return rpgToPoll;
+}),
+isDefinedAndNotNull(),
+switchMap((rpgToPoll) =>
+from(
+this.flowService.getRemoteProcessGroup(rpgToPoll.id).pipe(
+map((response: any) => {
+const entity = response;
+
+if (rpgToPoll.refreshTimestamp !==
response.component.flowRefreshed) {
+
this.store.dispatch(FlowActions.stopRemoteProcessGroupPolling());
+} else {
+entity.component.flowRefreshed =
'Refreshing...';
+}
+
+return FlowActions.loadRemoteProcessGroupSuccess({
+response: {
+id: entity.id,
+remoteProcessGroup: entity
+}
+});
+}),
+catchError((errorResponse: HttpErrorResponse) => {
+
this.store.dispatch(FlowActions.stopRemoteProcessGroupPolling());
+
+return
of(this.errorHelper.handleLoadingError(status, errorResponse));
Review Comment:
`status` isn't defined here. I understand that error handling has not been
introduced in this `FlowEffects` yet (there is a line item for it in
NIFI-12400). But the `hanldingLoadingError` method was meant to handle errors
loading the initial state. In addition to stopping polling, I suspect that here
we would want to check if the error is one that should be shown in context and
let that drive whether we want a snackbar or full screen error.
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-designer/state/flow/flow.effects.ts:
##
@@ -348,14 +352,78 @@ export class FlowEffects {
this.flowService.goToRemoteProcessGroup(request);
} else {
this.store.dispatch(
-showOkDialog({ title: 'Remote Process Group',
message: 'No target URI de
Re: [PR] MINIFICPP-1797 Python bootstrap (part 1) [nifi-minifi-cpp]
fgerlits commented on code in PR #1681:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1681#discussion_r1487979767
##
bootstrap/py_bootstrap.sh:
##
Review Comment:
To accomodate users who don't read instructions, we should first check if
`python` and `python-venv` are installed, and tell the user to install them if
they aren't.
Also, if `python` is installed but `python-venv` isn't, the script creates a
broken `venv` directory, so it won't work even after `python-venv` is installed
later. The `venv` directory should be removed (or not created) in this case.
##
bootstrap/cli.py:
##
Review Comment:
After a step has finished in `Step by step` mode, is it possible to return
to the `Step by step` menu instead of the main menu?
##
bootstrap/package_manager.py:
##
@@ -0,0 +1,259 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import os
+import platform
+import subprocess
+import sys
+from typing import Dict, Set
+
+from distro import distro
+
+
+def _query_yes_no(question: str, no_confirm: bool) -> bool:
+valid = {"yes": True, "y": True, "ye": True, "no": False, "n": False}
+
+if no_confirm:
+print("Running {} with noconfirm".format(question))
+return True
+while True:
+print("{} [y/n]".format(question))
Review Comment:
prompts requiring a response should be printed without a newline:
```suggestion
print("{} [y/n]".format(question), end=' ', flush=True)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [NIFI-12761] refresh RPG [nifi]
mcgilman commented on PR #8401: URL: https://github.com/apache/nifi/pull/8401#issuecomment-1941597842 Will review... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (MINIFICPP-2300) Kubernetes tests transiently fail in CI
Gábor Gyimesi created MINIFICPP-2300:
Summary: Kubernetes tests transiently fail in CI
Key: MINIFICPP-2300
URL: https://issues.apache.org/jira/browse/MINIFICPP-2300
Project: Apache NiFi MiNiFi C++
Issue Type: Bug
Reporter: Gábor Gyimesi
Attachments: behavex_output (3).zip
Kubernetes tests fail with the following error in CI:
* 404 Client Error for
http+docker://localhost/v1.43/containers/kind-control-plane/json: Not Found
("No such container: kind-control-plane")
Locally the tests pass, more information in the attached logs
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (MINIFICPP-2299) SQL tests transiently fail on MacOS in CI
Gábor Gyimesi created MINIFICPP-2299: Summary: SQL tests transiently fail on MacOS in CI Key: MINIFICPP-2299 URL: https://issues.apache.org/jira/browse/MINIFICPP-2299 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Gábor Gyimesi SQL tests often fail with the following issue on MacOS in github actions: /Users/runner/work/nifi-minifi-cpp/nifi-minifi-cpp/extensions/sql/tests/QueryDatabaseTableTests.cpp:176: FAILED: due to unexpected exception with message: Errorconnectingtodatabase: [iODBC][Driver Manager]dlopen(libsqlite3odbc. so,0x0006): tried: 'libsqlite3odbc.so' (nosuchfile), '/System/Volumes/ Preboot/Cryptexes/OSlibsqlite3odbc.so' (no such file), '/Users/runner/work/ nifi-minifi-cpp/nifi-minifi-cpp/build/bin/libsqlite3odbc.so' ( (SQL state 0) The runner machines seem to be a bit different when the failure occurs, the difference I found was that the installed SQLite version is 3.45.1 when the tests fail and 3.44.2 when the tests succeed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [NIFI-12754] - Flow Configuration History [nifi]
rfellows commented on code in PR #8399:
URL: https://github.com/apache/nifi/pull/8399#discussion_r1487815597
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-configuration-history/feature/flow-configuration-history-routing.module.ts:
##
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { RouterModule, Routes } from '@angular/router';
+import { FlowConfigurationHistory } from
'./flow-configuration-history.component';
+import { NgModule } from '@angular/core';
+
+const routes: Routes = [
+{
+path: '',
+component: FlowConfigurationHistory,
+children: [
+{
+path: ':id',
Review Comment:
Good catch. I'm not sure on how we should approach this. One option would be
to not add the selection to the route, similar to provenance. Could be an
acceptable solution here as there is no reason to link to a specific selection
in the history table. Thoughts, @mcgilman?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (NIFI-12780) Stateless NiFi properties missing in stateless.properties base config
[ https://issues.apache.org/jira/browse/NIFI-12780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephanie Ambrose resolved NIFI-12780. -- Resolution: Won't Fix Ignore, we have a way to support camelCase. > Stateless NiFi properties missing in stateless.properties base config > - > > Key: NIFI-12780 > URL: https://issues.apache.org/jira/browse/NIFI-12780 > Project: Apache NiFi > Issue Type: Bug >Reporter: Stephanie Ambrose >Priority: Major > > These new properties from NIFI-11471 need to be added to the base > stateless.properties file otherwise you won't be able to override them with > environment variables in the replace docker scripts: > > nifi.stateless.component.enableTimeout > nifi.stateless.processor.startTimeout > > Alternatively, make the properties not case-sensitive. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [NIFI-12754] - Flow Configuration History [nifi]
rfellows commented on code in PR #8399:
URL: https://github.com/apache/nifi/pull/8399#discussion_r1487790655
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-configuration-history/ui/flow-configuration-history-listing/flow-configuration-history-table/flow-configuration-history-table.component.ts:
##
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { Component, EventEmitter, Input, Output } from '@angular/core';
+import { CommonModule } from '@angular/common';
+import { MatTableDataSource, MatTableModule } from '@angular/material/table';
+import { MatSortModule, Sort } from '@angular/material/sort';
+import { ActionEntity } from
'../../../state/flow-configuration-history-listing';
+import { NiFiCommon } from '../../../../../service/nifi-common.service';
+
+@Component({
+selector: 'flow-configuration-history-table',
+standalone: true,
+imports: [CommonModule, MatTableModule, MatSortModule],
+templateUrl: './flow-configuration-history-table.component.html',
+styleUrls: ['./flow-configuration-history-table.component.scss']
+})
+export class FlowConfigurationHistoryTable {
+@Input() selectedHistoryActionId: string | null = null;
+@Input() initialSortColumn: 'timestamp' | 'sourceName' | 'sourceType' |
'operation' | 'userIdentity' = 'timestamp';
+@Input() initialSortDirection: 'asc' | 'desc' = 'desc';
+
+@Input() set historyActions(historyActions: ActionEntity[]) {
+if (historyActions) {
+this.dataSource.data = historyActions;
+}
+}
+
+@Output() selectionChanged: EventEmitter = new
EventEmitter();
+@Output() sortChanged: EventEmitter = new EventEmitter();
+@Output() moreDetailsClicked: EventEmitter = new
EventEmitter();
+
+activeSort: Sort = {
+active: this.initialSortColumn,
+direction: this.initialSortDirection
+};
+
+displayedColumns: string[] = ['moreDetails', 'timestamp', 'sourceName',
'sourceType', 'operation', 'userIdentity'];
+dataSource: MatTableDataSource = new
MatTableDataSource();
+
+constructor(private nifiCommon: NiFiCommon) {}
+
+sortData(sort: Sort) {
+this.sortChanged.next(sort);
+}
+
+canRead(item: ActionEntity): boolean {
+return item.canRead;
+}
+
+private format(item: ActionEntity, property: string): string {
+if (this.canRead(item) && Object.hasOwn(item.action, property)) {
+const value = (item.action as any)[property];
+if (!value) {
+return 'Empty String Set';
+}
+return value;
+}
+return 'Not Authorized';
+}
+
+formatTimestamp(item: ActionEntity): string {
+return this.format(item, 'timestamp');
+}
+
+formatName(item: ActionEntity): string {
+return this.format(item, 'sourceName');
+}
+
+formatType(item: ActionEntity): string {
+return this.format(item, 'sourceType');
+}
+
+formatOperation(item: ActionEntity): string {
+return this.format(item, 'operation');
+}
+
+formatUser(item: ActionEntity): string {
+return this.format(item, 'userIdentity');
+}
+
+select(item: ActionEntity) {
+this.selectionChanged.next(item);
+}
+
+isSelected(item: ActionEntity): boolean {
+if (this.selectedHistoryActionId) {
+return `${item.id}` === this.selectedHistoryActionId;
Review Comment:
Yes. the `selectedHistoryActionId` is from the route, which is a string.
However, the `ActionEntity.id` is a `number`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [NIFI-12754] - Flow Configuration History [nifi]
rfellows commented on code in PR #8399: URL: https://github.com/apache/nifi/pull/8399#discussion_r1487773990 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/flow-configuration-history/feature/flow-configuration-history.component.html: ## @@ -0,0 +1,26 @@ + + + + + + + Review Comment: The closest analog to this is the bulletins listing. Looks like it has a touch more padding around the form. I'll update it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12783) Kafka Producer Processors do not route in failure queue on timeout
[ https://issues.apache.org/jira/browse/NIFI-12783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis Jakupovic updated NIFI-12783: --- Description: Hi, the Kafka producer processors do not route the FlowFiles on a timeout e.g. into the failure connection. They are yielded in the incomming connection. You can see the behaviour here e.g.: [https://stackoverflow.com/questions/71460008/apache-nifi-publishkafka-timeout-exception] I think this is a design flaw. I have a use case where messages should be dropped after a specific configurable time. If the messages are yielded in the incomming queue they are always published when the kafka broker are available again. I know I can set the expiration time in secs or mins in the incomming queue but it is not dynamically configurable because no attributes are allowed. Best Denis was: Hi, the Kafka producer processors do not route the FlowFiles on a timeout e.g. into the failure connection. They are yielded in the incomming connection. You can see the behaviour here e.g.: [https://stackoverflow.com/questions/71460008/apache-nifi-publishkafka-timeout-exception] I think this is a design flaw. I have a use case where messages should be dropped after a specific time. If the messages are yielded in the incomming queue they are always published when the kafka broker are available again. Best Denis > Kafka Producer Processors do not route in failure queue on timeout > -- > > Key: NIFI-12783 > URL: https://issues.apache.org/jira/browse/NIFI-12783 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.23.2 >Reporter: Denis Jakupovic >Priority: Major > > Hi, > the Kafka producer processors do not route the FlowFiles on a timeout e.g. > into the failure connection. They are yielded in the incomming connection. > You can see the behaviour here e.g.: > [https://stackoverflow.com/questions/71460008/apache-nifi-publishkafka-timeout-exception] > I think this is a design flaw. I have a use case where messages should be > dropped after a specific configurable time. If the messages are yielded in > the incomming queue they are always published when the kafka broker are > available again. I know I can set the expiration time in secs or mins in the > incomming queue but it is not dynamically configurable because no attributes > are allowed. > Best > Denis -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12783) Kafka Producer Processors do not route in failure queue on timeout
Denis Jakupovic created NIFI-12783: -- Summary: Kafka Producer Processors do not route in failure queue on timeout Key: NIFI-12783 URL: https://issues.apache.org/jira/browse/NIFI-12783 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.23.2 Reporter: Denis Jakupovic Hi, the Kafka producer processors do not route the FlowFiles on a timeout e.g. into the failure connection. They are yielded in the incomming connection. You can see the behaviour here e.g.: [https://stackoverflow.com/questions/71460008/apache-nifi-publishkafka-timeout-exception] I think this is a design flaw. I have a use case where messages should be dropped after a specific time. If the messages are yielded in the incomming queue they are always published when the kafka broker are available again. Best Denis -- This message was sent by Atlassian Jira (v8.20.10#820010)
