[jira] [Created] (NIFI-12497) Upgrade snowflake-jdbc to 3.14.4
Peter Turcsanyi created NIFI-12497:
--
Summary: Upgrade snowflake-jdbc to 3.14.4
Key: NIFI-12497
URL: https://issues.apache.org/jira/browse/NIFI-12497
Project: Apache NiFi
Issue Type: Improvement
Reporter: Peter Turcsanyi
Assignee: Peter Turcsanyi
{{snowflake-jdbc:3.14.4}} is supposed to fix issue with Java 21
([SNOW-917458|https://github.com/snowflakedb/snowflake-jdbc/issues/1512],
NIFI-10883).
Compatibility with {{snowflake-ingest-sdk}} has to be checked also.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] NIFI-12496 - move python extensions unpacking to common.xml [nifi]
pvillard31 opened a new pull request, #8148: URL: https://github.com/apache/nifi/pull/8148 # Summary [NIFI-12496](https://issues.apache.org/jira/browse/NIFI-12496) - move python extensions unpacking to common.xml # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12496) Building with -DallProfiles does not include python extensions
[ https://issues.apache.org/jira/browse/NIFI-12496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12496: -- Status: Patch Available (was: Open) > Building with -DallProfiles does not include python extensions > -- > > Key: NIFI-12496 > URL: https://issues.apache.org/jira/browse/NIFI-12496 > Project: Apache NiFi > Issue Type: Bug > Components: Tools and Build >Affects Versions: 2.0.0-M1 >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > > When using -DallProfiles for building NiFi, the Ranger profile gets activated > and it redeclares the assembly plugin and uses ranger.xml instead of > dependencies.xml. > Since we already have some python parts defined in common.xml, I'll file a PR > moving this part to common.xml as well so it gets included even with the > Ranger profile. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12496) Building with -DallProfiles does not include python extensions
Pierre Villard created NIFI-12496: - Summary: Building with -DallProfiles does not include python extensions Key: NIFI-12496 URL: https://issues.apache.org/jira/browse/NIFI-12496 Project: Apache NiFi Issue Type: Bug Components: Tools and Build Affects Versions: 2.0.0-M1 Reporter: Pierre Villard Assignee: Pierre Villard When using -DallProfiles for building NiFi, the Ranger profile gets activated and it redeclares the assembly plugin and uses ranger.xml instead of dependencies.xml. Since we already have some python parts defined in common.xml, I'll file a PR moving this part to common.xml as well so it gets included even with the Ranger profile. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12495) Update the NiFi website to a new design
James Elliott created NIFI-12495: Summary: Update the NiFi website to a new design Key: NIFI-12495 URL: https://issues.apache.org/jira/browse/NIFI-12495 Project: Apache NiFi Issue Type: Improvement Components: Documentation & Website Reporter: James Elliott Continuation of work started by [~exceptionfactory] to update the design of the NiFi website. * Replace Foundation with UIkit * Remove fontawesome * Simplify navigation between NiFi and MiNiFi subproject * Major styling updates -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (MINIFICPP-2272) Support NiFi Python processors
Gábor Gyimesi created MINIFICPP-2272: Summary: Support NiFi Python processors Key: MINIFICPP-2272 URL: https://issues.apache.org/jira/browse/MINIFICPP-2272 Project: Apache NiFi MiNiFi C++ Issue Type: New Feature Reporter: Gábor Gyimesi Assignee: Gábor Gyimesi NiFi 2.0 introduced Python processor support, which requires Python classes to be inherited from NiFi specific classes in the nifiapi Python package. In MiNiFi C++ python support is done in a different way: native python processors are each separate python files with specific global functions that need to be defined. We should update our Python binding library and adapt our code to the NiFi's python extension so that NiFi's Python processors could be used with MiNiFi C++ as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12479) Add pg-export to toolkit CLI
[ https://issues.apache.org/jira/browse/NIFI-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12479: -- Fix Version/s: 2.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Add pg-export to toolkit CLI > > > Key: NIFI-12479 > URL: https://issues.apache.org/jira/browse/NIFI-12479 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Timea Barna >Assignee: Timea Barna >Priority: Major > Fix For: 2.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > > Add a CLI command allowing to save a process group in NiFi as a JSON file. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12479) Add pg-export to toolkit CLI
[ https://issues.apache.org/jira/browse/NIFI-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794810#comment-17794810 ] ASF subversion and git services commented on NIFI-12479: Commit 71d99aff5d355d2dde9de298288e4fcc7bbc521a in nifi's branch refs/heads/main from Timea Barna [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=71d99aff5d ] NIFI-12479 Add pg-export to toolkit CLI Signed-off-by: Pierre Villard This closes #8137. > Add pg-export to toolkit CLI > > > Key: NIFI-12479 > URL: https://issues.apache.org/jira/browse/NIFI-12479 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Timea Barna >Assignee: Timea Barna >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > Add a CLI command allowing to save a process group in NiFi as a JSON file. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12479 Add pg-export to toolkit CLI [nifi]
asfgit closed pull request #8137: NIFI-12479 Add pg-export to toolkit CLI URL: https://github.com/apache/nifi/pull/8137 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-11627 Added an in memory JSON schema registry which ValidationJson can now use to retrieve JSON schemas from for validation. [nifi]
dan-s1 commented on PR #8005: URL: https://github.com/apache/nifi/pull/8005#issuecomment-1847516537 @exceptionfactory Sorry I had a Checkstyle violation as I had not removed an unused import. I have fixed that and pushed the change. Can you please at your earliest convenience restart the builds? Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12445: Provenance Event Listing [nifi]
mcgilman commented on code in PR #8133: URL: https://github.com/apache/nifi/pull/8133#discussion_r1420752026 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/ui/common/provenance-event-dialog/provenance-event-dialog.component.html: ## @@ -0,0 +1,447 @@ + + +Provenance Event + + + + + + + Review Comment: My general strategy for spacing is to use flex gap. However, it's not always possible if some of the children are conditional. For instance, in the attributes tab in this dialog. I can however use it here. I tried `gap-y-2` as you suggested but it felt a little tight. I'll make it 3 for now and we can look to adjust more later if needed. As for the consistency of `4` vs `5`, we should look to settle on something. What makes it a little tricky is that when used in a form with other `mat-form-fields` (which have their own padding/spacing) more or less margin may be needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-8134 allow unescapeJson Record Path function to recursively convert Maps to Records [nifi]
ChrisSamo632 commented on code in PR #7745:
URL: https://github.com/apache/nifi/pull/7745#discussion_r1420713574
##
nifi-commons/nifi-record-path/src/test/java/org/apache/nifi/record/path/TestRecordPath.java:
##
@@ -1837,7 +1838,41 @@ public void testUnescapeJson() {
put("firstName", "John");
put("age", 30);
}}, "json_str"),
-RecordPath.compile("unescapeJson(/json_str,
'true')").evaluate(recordFromMap).getSelectedFields().findFirst().orElseThrow(IllegalStateException::new).getValue()
+RecordPath.compile("unescapeJson(/json_str,
'true')").evaluate(recordFromMap).getSelectedFields().findFirst().orElseThrow(AssertionError::new).getValue()
+);
+
+// test nested Record converted from Map Object
+final Record nestedRecordFromMap = new MapRecord(schema,
+Collections.singletonMap(
+"json_str",
+
"{\"firstName\":\"John\",\"age\":30,\"addresses\":[{\"address_1\":\"123 Fake
Street\"}]}")
+);
+// recursively convert Maps to Records (addresses becomes and ARRAY or
RECORDs)
+assertEquals(
+DataTypeUtils.toRecord(new LinkedHashMap(){{
+put("firstName", "John");
+put("age", 30);
+put("addresses", new Object[]
{DataTypeUtils.toRecord(Collections.singletonMap("address_1", "123 Fake
Street"), "addresses")});
+}}, "json_str"),
Review Comment:
Absolutely. The only problem is that `Map.of` doesn't provide deterministic
ordering, so we do need to use `LinkedHasMap`s in some places, which I've
changed to use the approach suggested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12445: Provenance Event Listing [nifi]
mcgilman commented on code in PR #8133:
URL: https://github.com/apache/nifi/pull/8133#discussion_r1420686953
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-frontend/src/main/nifi/src/app/pages/provenance/state/provenance-event-listing/provenance-event-listing.effects.ts:
##
@@ -0,0 +1,335 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { Injectable } from '@angular/core';
+import { Actions, createEffect, ofType } from '@ngrx/effects';
+import * as ProvenanceEventListingActions from
'./provenance-event-listing.actions';
+import {
+asyncScheduler,
+catchError,
+from,
+interval,
+map,
+NEVER,
+of,
+switchMap,
+take,
+takeUntil,
+tap,
+withLatestFrom
+} from 'rxjs';
+import { MatDialog } from '@angular/material/dialog';
+import { Store } from '@ngrx/store';
+import { NiFiState } from '../../../../state';
+import { Router } from '@angular/router';
+import { OkDialog } from '../../../../ui/common/ok-dialog/ok-dialog.component';
+import { ProvenanceService } from '../../service/provenance.service';
+import {
+selectClusterNodeId,
+selectProvenanceId,
+selectProvenanceOptions,
+selectProvenanceRequest,
+selectTimeOffset
+} from './provenance-event-listing.selectors';
+import { Provenance, ProvenanceRequest } from './index';
+import { ProvenanceSearchDialog } from
'../../ui/provenance-event-listing/provenance-search-dialog/provenance-search-dialog.component';
+import { selectAbout } from '../../../../state/about/about.selectors';
+import { ProvenanceEventDialog } from
'../../../../ui/common/provenance-event-dialog/provenance-event-dialog.component';
+
+@Injectable()
+export class ProvenanceEventListingEffects {
+constructor(
+private actions$: Actions,
+private store: Store,
+private provenanceService: ProvenanceService,
+private dialog: MatDialog,
+private router: Router
+) {}
+
+loadProvenanceOptions$ = createEffect(() =>
+this.actions$.pipe(
+ofType(ProvenanceEventListingActions.loadProvenanceOptions),
+switchMap((request) =>
+from(this.provenanceService.getSearchOptions()).pipe(
+map((response) =>
+
ProvenanceEventListingActions.loadProvenanceOptionsSuccess({
+response
+})
+),
+catchError((error) =>
+of(
+ProvenanceEventListingActions.provenanceApiError({
+error: error.error
+})
+)
+)
+)
+)
+)
+);
+
+submitProvenanceQuery$ = createEffect(() =>
+this.actions$.pipe(
+ofType(ProvenanceEventListingActions.submitProvenanceQuery),
+map((action) => action.request),
+switchMap((request) =>
+
from(this.provenanceService.submitProvenanceQuery(request)).pipe(
+map((response) =>
+
ProvenanceEventListingActions.submitProvenanceQuerySuccess({
+response: {
+provenance: response.provenance
+}
+})
+),
+catchError((error) => {
+this.store.dispatch(
+ProvenanceEventListingActions.showOkDialog({
+title: 'Error',
+message: error.error
+})
+);
+
+return of(
+ProvenanceEventListingActions.provenanceApiError({
+error: error.error
+})
+);
+})
+)
+)
+)
+);
+
+submitProvenanceQuerySuccess$ = createEffect(() =>
+this.actions$.pipe(
+ofType(ProvenanceEventListingActions.submitProvenanceQuerySucce
Re: [PR] NIFI-11627 Added an in memory JSON schema registry which ValidationJson can now use to retrieve JSON schemas from for validation. [nifi]
dan-s1 commented on code in PR #8005:
URL: https://github.com/apache/nifi/pull/8005#discussion_r1420661560
##
nifi-nar-bundles/nifi-registry-bundle/nifi-registry-service/src/main/java/org/apache/nifi/schemaregistry/services/InMemoryJsonSchemaRegistry.java:
##
@@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.schemaregistry.services;
+
+import com.networknt.schema.JsonSchemaFactory;
+import com.networknt.schema.SpecVersion;
+import org.apache.nifi.annotation.behavior.DynamicProperty;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.JsonSchema;
+import org.apache.nifi.schema.access.JsonSchemaRegistryComponent;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.schema.access.SchemaVersion;
+
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import static org.apache.commons.lang3.StringUtils.isBlank;
+import static org.apache.commons.lang3.StringUtils.isNotBlank;
+
+@Tags({"schema", "registry", "json"})
+@CapabilityDescription("Provides a service for registering and accessing
schemas. One can register a schema "
++ "as a dynamic property where 'name' represents the schema name and
'value' represents the textual "
++ "representation of the actual schema following the syntax and
semantics of JSON's Schema format. "
++ "Empty schemas and schemas only consisting of whitespace are not
acceptable schemas."
++ "The registry is heterogeneous registry as it can store schemas of
multiple schema draft versions. "
++ "By default the registry is configured to store schemas of Draft
2020-12. When a schema is added, the version "
++ "which is currently is set, is what the schema is saved as.")
+@DynamicProperty(name = "Schema name", value = "Schema Content",
+description = "Adds a named schema using the JSON string
representation of a JSON schema",
+expressionLanguageScope = ExpressionLanguageScope.NONE)
+public class InMemoryJsonSchemaRegistry extends AbstractControllerService
implements JsonSchemaRegistry, JsonSchemaRegistryComponent {
+
+private static final List PROPERTY_DESCRIPTORS =
Collections.singletonList(SCHEMA_VERSION);
+
+private final ConcurrentMap jsonSchemas;
+private final ConcurrentMap
schemaFactories;
+private volatile SchemaVersion schemaVersion;
+
+public InMemoryJsonSchemaRegistry() {
+jsonSchemas = new ConcurrentHashMap<>();
+schemaFactories = Arrays.stream(SchemaVersion.values())
+.collect(Collectors.toConcurrentMap(Function.identity(),
+schemaDraftVersion ->
JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.fromId(schemaDraftVersion.getUri()).get(;
+schemaVersion =
SchemaVersion.valueOf(SCHEMA_VERSION.getDefaultValue());
+}
+
+@Override
+public void onPropertyModified(final PropertyDescriptor descriptor, final
String oldValue, final String newValue) {
+if (SCHEMA_VERSION.getName().equals(descriptor.getName()) &&
!newValue.equals(oldValue)) {
+schemaVersion = SchemaVersion.valueOf(newValue);
+} else if(descriptor.isDynamic() && isBlank(newValue)) {
+jsonSchemas.remove(descriptor.getName());
+} else if (descriptor.isDynamic() && isNotBlank(newValue)) {
+try {
+final String schemaName = descriptor.getName();
+final JsonSchemaFactory jsonSchemaFactory =
schemaF
[jira] [Commented] (NIFI-12494) Broken CLI - jakarta.ws.rs.client.ClientBuilder cannot be found
[
https://issues.apache.org/jira/browse/NIFI-12494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794769#comment-17794769
]
Pierre Villard commented on NIFI-12494:
---
Thanks David, it does not seem to be enough.
With this change:
[https://github.com/apache/nifi/pull/8137/files#diff-1de3983793fcb77680415b49a038bc327e8bac67e5bebbe595c1cdfe120d51ba]
I see the below stacktraces:
{code:java}
Dec 08, 2023 4:12:06 PM org.glassfish.jersey.internal.Errors logErrors
WARNING: The following warnings have been detected: WARNING: HK2 failure has
been detected in a code that does not run in an active Jersey Error scope.
WARNING: HK2 service reification failed for
[org.glassfish.jersey.jaxb.internal.XmlRootElementJaxbProvider$Text] with an
exception:
MultiException stack 1 of 2
java.lang.NoClassDefFoundError: jakarta/xml/bind/JAXBException
at java.base/java.lang.Class.getDeclaredConstructors0(Native Method)
at java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3549)
at java.base/java.lang.Class.getDeclaredConstructors(Class.java:2727)
at org.jvnet.hk2.internal.Utilities$3.run(Utilities.java:1355)
at org.jvnet.hk2.internal.Utilities$3.run(Utilities.java:1351)
at
java.base/java.security.AccessController.doPrivileged(AccessController.java:319)
at org.jvnet.hk2.internal.Utilities.getAllConstructors(Utilities.java:1351)
at
org.jvnet.hk2.internal.Utilities.findProducerConstructor(Utilities.java:1294)
at
org.jvnet.hk2.internal.DefaultClassAnalyzer.getConstructor(DefaultClassAnalyzer.java:60)
at
org.glassfish.jersey.inject.hk2.JerseyClassAnalyzer.getConstructor(JerseyClassAnalyzer.java:124)
at org.jvnet.hk2.internal.Utilities.getConstructor(Utilities.java:157)
at org.jvnet.hk2.internal.ClazzCreator.initialize(ClazzCreator.java:106)
at org.jvnet.hk2.internal.ClazzCreator.initialize(ClazzCreator.java:165)
at
org.jvnet.hk2.internal.SystemDescriptor.internalReify(SystemDescriptor.java:719)
at org.jvnet.hk2.internal.SystemDescriptor.reify(SystemDescriptor.java:673)
at
org.jvnet.hk2.internal.ServiceLocatorImpl.reifyDescriptor(ServiceLocatorImpl.java:438)
at
org.jvnet.hk2.internal.ServiceLocatorImpl.narrow(ServiceLocatorImpl.java:2316)
at
org.jvnet.hk2.internal.ServiceLocatorImpl$10.compute(ServiceLocatorImpl.java:1401)
at
org.jvnet.hk2.internal.ServiceLocatorImpl$10.compute(ServiceLocatorImpl.java:1396)
at
org.glassfish.hk2.utilities.cache.internal.WeakCARCacheImpl.compute(WeakCARCacheImpl.java:105)
at
org.jvnet.hk2.internal.ServiceLocatorImpl.internalGetAllServiceHandles(ServiceLocatorImpl.java:1458)
at
org.jvnet.hk2.internal.ServiceLocatorImpl.getAllServiceHandles(ServiceLocatorImpl.java:1384)
at
org.jvnet.hk2.internal.ServiceLocatorImpl.getAllServiceHandles(ServiceLocatorImpl.java:1372)
at
org.glassfish.jersey.inject.hk2.AbstractHk2InjectionManager.getAllServiceHolders(AbstractHk2InjectionManager.java:134)
at
org.glassfish.jersey.inject.hk2.ImmediateHk2InjectionManager.getAllServiceHolders(ImmediateHk2InjectionManager.java:30)
at
org.glassfish.jersey.internal.inject.Providers.getServiceHolders(Providers.java:299)
at
org.glassfish.jersey.internal.inject.Providers.getProviders(Providers.java:133)
at
org.glassfish.jersey.message.internal.MessageBodyFactory.initialize(MessageBodyFactory.java:246)
at
org.glassfish.jersey.client.ClientMessageBodyFactory$MessageBodyWorkersConfigurator.postInit(ClientMessageBodyFactory.java:75)
at
org.glassfish.jersey.client.ClientConfig$State.lambda$initRuntime$2(ClientConfig.java:468)
at java.base/java.util.Arrays$ArrayList.forEach(Arrays.java:4305)
at
org.glassfish.jersey.client.ClientConfig$State.initRuntime(ClientConfig.java:468)
at
org.glassfish.jersey.internal.util.collection.Values$LazyValueImpl.get(Values.java:317)
at
org.glassfish.jersey.client.ClientConfig.getRuntime(ClientConfig.java:834)
at
org.glassfish.jersey.client.ClientRequest.getClientRuntime(ClientRequest.java:181)
at
org.glassfish.jersey.client.JerseyWebTarget.onBuilder(JerseyWebTarget.java:371)
at
org.glassfish.jersey.client.JerseyWebTarget.request(JerseyWebTarget.java:199)
at
org.glassfish.jersey.client.JerseyWebTarget.request(JerseyWebTarget.java:38)
at
org.apache.nifi.toolkit.cli.impl.client.nifi.impl.AbstractJerseyClient.getRequestBuilder(AbstractJerseyClient.java:56)
at
org.apache.nifi.toolkit.cli.impl.client.nifi.impl.JerseyFlowClient.lambda$getProcessGroup$1(JerseyFlowClient.java:94)
at
org.apache.nifi.toolkit.cli.impl.client.nifi.impl.AbstractJerseyClient.executeAction(AbstractJerseyClient.java:76)
at
org.apache.nifi.toolkit.cli.impl.client.nifi.impl.JerseyFlowClient.getProcessGroup(JerseyFlowClient.java:89)
at
org.apache.nifi.toolkit.cli.impl.client.nifi.impl.JerseyFlowClient.getRootGroupId(JerseyFlowClient.java:79)
Re: [PR] NIFI-11627 Added an in memory JSON schema registry which ValidationJson can now use to retrieve JSON schemas from for validation. [nifi]
exceptionfactory commented on code in PR #8005:
URL: https://github.com/apache/nifi/pull/8005#discussion_r1420621232
##
nifi-nar-bundles/nifi-registry-bundle/nifi-registry-service/src/main/java/org/apache/nifi/schemaregistry/services/InMemoryJsonSchemaRegistry.java:
##
@@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.schemaregistry.services;
+
+import com.networknt.schema.JsonSchemaFactory;
+import com.networknt.schema.SpecVersion;
+import org.apache.nifi.annotation.behavior.DynamicProperty;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.JsonSchema;
+import org.apache.nifi.schema.access.JsonSchemaRegistryComponent;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.schema.access.SchemaVersion;
+
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import static org.apache.commons.lang3.StringUtils.isBlank;
+import static org.apache.commons.lang3.StringUtils.isNotBlank;
+
+@Tags({"schema", "registry", "json"})
+@CapabilityDescription("Provides a service for registering and accessing
schemas. One can register a schema "
++ "as a dynamic property where 'name' represents the schema name and
'value' represents the textual "
++ "representation of the actual schema following the syntax and
semantics of JSON's Schema format. "
++ "Empty schemas and schemas only consisting of whitespace are not
acceptable schemas."
++ "The registry is heterogeneous registry as it can store schemas of
multiple schema draft versions. "
++ "By default the registry is configured to store schemas of Draft
2020-12. When a schema is added, the version "
++ "which is currently is set, is what the schema is saved as.")
+@DynamicProperty(name = "Schema name", value = "Schema Content",
+description = "Adds a named schema using the JSON string
representation of a JSON schema",
+expressionLanguageScope = ExpressionLanguageScope.NONE)
+public class InMemoryJsonSchemaRegistry extends AbstractControllerService
implements JsonSchemaRegistry, JsonSchemaRegistryComponent {
+
+private static final List PROPERTY_DESCRIPTORS =
Collections.singletonList(SCHEMA_VERSION);
+
+private final ConcurrentMap jsonSchemas;
+private final ConcurrentMap
schemaFactories;
+private volatile SchemaVersion schemaVersion;
+
+public InMemoryJsonSchemaRegistry() {
+jsonSchemas = new ConcurrentHashMap<>();
+schemaFactories = Arrays.stream(SchemaVersion.values())
+.collect(Collectors.toConcurrentMap(Function.identity(),
+schemaDraftVersion ->
JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.fromId(schemaDraftVersion.getUri()).get(;
+schemaVersion =
SchemaVersion.valueOf(SCHEMA_VERSION.getDefaultValue());
+}
+
+@Override
+public void onPropertyModified(final PropertyDescriptor descriptor, final
String oldValue, final String newValue) {
+if (SCHEMA_VERSION.getName().equals(descriptor.getName()) &&
!newValue.equals(oldValue)) {
+schemaVersion = SchemaVersion.valueOf(newValue);
+} else if(descriptor.isDynamic() && isBlank(newValue)) {
+jsonSchemas.remove(descriptor.getName());
+} else if (descriptor.isDynamic() && isNotBlank(newValue)) {
+try {
+final String schemaName = descriptor.getName();
+final JsonSchemaFactory jsonSchemaFactory
Re: [PR] NIFI-11627 Added an in memory JSON schema registry which ValidationJson can now use to retrieve JSON schemas from for validation. [nifi]
dan-s1 commented on code in PR #8005:
URL: https://github.com/apache/nifi/pull/8005#discussion_r1420614969
##
nifi-nar-bundles/nifi-registry-bundle/nifi-registry-service/src/main/java/org/apache/nifi/schemaregistry/services/InMemoryJsonSchemaRegistry.java:
##
@@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.schemaregistry.services;
+
+import com.networknt.schema.JsonSchemaFactory;
+import com.networknt.schema.SpecVersion;
+import org.apache.nifi.annotation.behavior.DynamicProperty;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.JsonSchema;
+import org.apache.nifi.schema.access.JsonSchemaRegistryComponent;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.schema.access.SchemaVersion;
+
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import static org.apache.commons.lang3.StringUtils.isBlank;
+import static org.apache.commons.lang3.StringUtils.isNotBlank;
+
+@Tags({"schema", "registry", "json"})
+@CapabilityDescription("Provides a service for registering and accessing
schemas. One can register a schema "
++ "as a dynamic property where 'name' represents the schema name and
'value' represents the textual "
++ "representation of the actual schema following the syntax and
semantics of JSON's Schema format. "
++ "Empty schemas and schemas only consisting of whitespace are not
acceptable schemas."
++ "The registry is heterogeneous registry as it can store schemas of
multiple schema draft versions. "
++ "By default the registry is configured to store schemas of Draft
2020-12. When a schema is added, the version "
++ "which is currently is set, is what the schema is saved as.")
+@DynamicProperty(name = "Schema name", value = "Schema Content",
+description = "Adds a named schema using the JSON string
representation of a JSON schema",
+expressionLanguageScope = ExpressionLanguageScope.NONE)
+public class InMemoryJsonSchemaRegistry extends AbstractControllerService
implements JsonSchemaRegistry, JsonSchemaRegistryComponent {
+
+private static final List PROPERTY_DESCRIPTORS =
Collections.singletonList(SCHEMA_VERSION);
+
+private final ConcurrentMap jsonSchemas;
+private final ConcurrentMap
schemaFactories;
+private volatile SchemaVersion schemaVersion;
+
+public InMemoryJsonSchemaRegistry() {
+jsonSchemas = new ConcurrentHashMap<>();
+schemaFactories = Arrays.stream(SchemaVersion.values())
+.collect(Collectors.toConcurrentMap(Function.identity(),
+schemaDraftVersion ->
JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.fromId(schemaDraftVersion.getUri()).get(;
+schemaVersion =
SchemaVersion.valueOf(SCHEMA_VERSION.getDefaultValue());
+}
+
+@Override
+public void onPropertyModified(final PropertyDescriptor descriptor, final
String oldValue, final String newValue) {
+if (SCHEMA_VERSION.getName().equals(descriptor.getName()) &&
!newValue.equals(oldValue)) {
+schemaVersion = SchemaVersion.valueOf(newValue);
+} else if(descriptor.isDynamic() && isBlank(newValue)) {
+jsonSchemas.remove(descriptor.getName());
+} else if (descriptor.isDynamic() && isNotBlank(newValue)) {
+try {
+final String schemaName = descriptor.getName();
+final JsonSchemaFactory jsonSchemaFactory =
schemaF
Re: [PR] NIFI-12479 Add pg-export to toolkit CLI [nifi]
timeabarna commented on code in PR #8137: URL: https://github.com/apache/nifi/pull/8137#discussion_r1420535121 ## nifi-docs/src/main/asciidoc/toolkit-guide.adoc: ## @@ -290,21 +291,21 @@ For example, typing tab at an empty prompt should display possible commands for Typing "nifi " and then a tab will show the sub-commands for NiFi: #> nifi - cluster-summary export-param-contextlist-users pg-set-param-context - connect-nodeget-nodemerge-param-context pg-set-var - create-param-contextget-nodes offload-node pg-start - create-reg-client get-param-context pg-change-version pg-status - create-reporting-task get-policy pg-create-service pg-stop - create-service get-reg-client-id pg-create-service set-param - create-user get-reporting-task pg-disable-services start-reporting-tasks - create-user-group get-reporting-tasks pg-enable-services stop-reporting-tasks - current-userget-root-id pg-get-all-versions update-policy - delete-node get-service pg-get-param-context update-reg-client - delete-paramget-servicespg-get-services update-user-group - delete-param-contextimport-param-contextpg-get-vars delete-reporting-task - disable-serviceslist-param-contexts pg-get-version + cluster-summary export-param-contextlist-users pg-list + connect-nodeget-nodemerge-param-context pg-set-param-context + create-param-contextget-nodes offload-node pg-set-var + create-reg-client get-param-context pg-change-version pg-start + create-reporting-task get-policy pg-create-service pg-status + create-service get-reg-client-id pg-create-service pg-stop + create-user get-reporting-task pg-disable-services set-param + create-user-group get-reporting-tasks pg-enable-services start-reporting-tasks + current-userget-root-id pg-get-all-versions stop-reporting-tasks + delete-node get-service pg-get-param-context update-policy + delete-paramget-servicespg-get-services update-reg-client + delete-param-contextimport-param-contextpg-get-vars update-user-group + disable-serviceslist-param-contexts pg-get-version delete-reporting-task disconnect-node list-reg-clientspg-import - enable-services list-user-groupspg-list + enable-services list-user-groupspg-export Review Comment: Thanks @pvillard31 for your review, I've updated the documentation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-11627 Added an in memory JSON schema registry which ValidationJson can now use to retrieve JSON schemas from for validation. [nifi]
exceptionfactory commented on code in PR #8005:
URL: https://github.com/apache/nifi/pull/8005#discussion_r1420507222
##
nifi-nar-bundles/nifi-registry-bundle/nifi-registry-service/src/main/java/org/apache/nifi/schemaregistry/services/InMemoryJsonSchemaRegistry.java:
##
@@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.schemaregistry.services;
+
+import com.networknt.schema.JsonSchemaFactory;
+import com.networknt.schema.SpecVersion;
+import org.apache.nifi.annotation.behavior.DynamicProperty;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.JsonSchema;
+import org.apache.nifi.schema.access.JsonSchemaRegistryComponent;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.schema.access.SchemaVersion;
+
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import static org.apache.commons.lang3.StringUtils.isBlank;
+import static org.apache.commons.lang3.StringUtils.isNotBlank;
+
+@Tags({"schema", "registry", "json"})
+@CapabilityDescription("Provides a service for registering and accessing
schemas. One can register a schema "
++ "as a dynamic property where 'name' represents the schema name and
'value' represents the textual "
++ "representation of the actual schema following the syntax and
semantics of JSON's Schema format. "
++ "Empty schemas and schemas only consisting of whitespace are not
acceptable schemas."
++ "The registry is heterogeneous registry as it can store schemas of
multiple schema draft versions. "
++ "By default the registry is configured to store schemas of Draft
2020-12. When a schema is added, the version "
++ "which is currently is set, is what the schema is saved as.")
+@DynamicProperty(name = "Schema name", value = "Schema Content",
+description = "Adds a named schema using the JSON string
representation of a JSON schema",
+expressionLanguageScope = ExpressionLanguageScope.NONE)
+public class InMemoryJsonSchemaRegistry extends AbstractControllerService
implements JsonSchemaRegistry, JsonSchemaRegistryComponent {
+
+private static final List PROPERTY_DESCRIPTORS =
Collections.singletonList(SCHEMA_VERSION);
+
+private final ConcurrentMap jsonSchemas;
+private final ConcurrentMap
schemaFactories;
+private volatile SchemaVersion schemaVersion;
+
+public InMemoryJsonSchemaRegistry() {
+jsonSchemas = new ConcurrentHashMap<>();
+schemaFactories = Arrays.stream(SchemaVersion.values())
+.collect(Collectors.toConcurrentMap(Function.identity(),
+schemaDraftVersion ->
JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.fromId(schemaDraftVersion.getUri()).get(;
+schemaVersion =
SchemaVersion.valueOf(SCHEMA_VERSION.getDefaultValue());
+}
+
+@Override
+public void onPropertyModified(final PropertyDescriptor descriptor, final
String oldValue, final String newValue) {
+if (SCHEMA_VERSION.getName().equals(descriptor.getName()) &&
!newValue.equals(oldValue)) {
+schemaVersion = SchemaVersion.valueOf(newValue);
+} else if(descriptor.isDynamic() && isBlank(newValue)) {
+jsonSchemas.remove(descriptor.getName());
+} else if (descriptor.isDynamic() && isNotBlank(newValue)) {
+try {
+final String schemaName = descriptor.getName();
+final JsonSchemaFactory jsonSchemaFactory
[jira] [Updated] (NIFI-12481) UI error when listing registry clients and not authorized
[
https://issues.apache.org/jira/browse/NIFI-12481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Gilman updated NIFI-12481:
---
Status: Patch Available (was: In Progress)
> UI error when listing registry clients and not authorized
> -
>
> Key: NIFI-12481
> URL: https://issues.apache.org/jira/browse/NIFI-12481
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.24.0, 2.0.0-M1
>Reporter: Bryan Bende
>Assignee: Matt Gilman
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently the authorization for registry clients is based on READ to
> /controller (this is a separate issue that should be addressed).
> Steps:
> * Run a secure instance locally
> * Use initial admin to create a registry client
> * Remove initial admin from controller policies
> * Create a new PG and choose Import from Registry
> * Notice nothing happens
> UI Error in dev tools:
> {code:java}
> nf-canvas-all.js?2.0.0-SNAPSHOT:47 Uncaught TypeError: Cannot read properties
> of undefined (reading 'name')
> at Object. (nf-canvas-all.js?2.0.0-SNAPSHOT:47:3678)
> at Function.each (jquery.min.js:2:3003)
> at b.each (jquery.each.js:1:96)
> at Object. (nf-canvas-all.js?2.0.0-SNAPSHOT:47:3610)
> at c (jquery.min.js:2:28447)
> at Object.fireWith [as resolveWith] (jquery.min.js:2:29192)
> at l (jquery.min.js:2:80176)
> at XMLHttpRequest. (jquery.min.js:2:82630) {code}
> The issue is that the listing of registry clients will optionally fill in the
> DTO in the entity based on the user's permissions for the entity, but the
> permissions are always based on /controller, so if they don't have
> /controller the DTO will be null.
> The UI should still be able to load a screen with an empty list of registry
> clients.
> cc [~mcgilman]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-11627 Added an in memory JSON schema registry which ValidationJson can now use to retrieve JSON schemas from for validation. [nifi]
dan-s1 commented on code in PR #8005:
URL: https://github.com/apache/nifi/pull/8005#discussion_r1420491373
##
nifi-nar-bundles/nifi-registry-bundle/nifi-registry-service/src/main/java/org/apache/nifi/schemaregistry/services/InMemoryJsonSchemaRegistry.java:
##
@@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.schemaregistry.services;
+
+import com.networknt.schema.JsonSchemaFactory;
+import com.networknt.schema.SpecVersion;
+import org.apache.nifi.annotation.behavior.DynamicProperty;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.JsonSchema;
+import org.apache.nifi.schema.access.JsonSchemaRegistryComponent;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.schema.access.SchemaVersion;
+
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import static org.apache.commons.lang3.StringUtils.isBlank;
+import static org.apache.commons.lang3.StringUtils.isNotBlank;
+
+@Tags({"schema", "registry", "json"})
+@CapabilityDescription("Provides a service for registering and accessing
schemas. One can register a schema "
++ "as a dynamic property where 'name' represents the schema name and
'value' represents the textual "
++ "representation of the actual schema following the syntax and
semantics of JSON's Schema format. "
++ "Empty schemas and schemas only consisting of whitespace are not
acceptable schemas."
++ "The registry is heterogeneous registry as it can store schemas of
multiple schema draft versions. "
++ "By default the registry is configured to store schemas of Draft
2020-12. When a schema is added, the version "
++ "which is currently is set, is what the schema is saved as.")
+@DynamicProperty(name = "Schema name", value = "Schema Content",
+description = "Adds a named schema using the JSON string
representation of a JSON schema",
+expressionLanguageScope = ExpressionLanguageScope.NONE)
+public class InMemoryJsonSchemaRegistry extends AbstractControllerService
implements JsonSchemaRegistry, JsonSchemaRegistryComponent {
+
+private static final List PROPERTY_DESCRIPTORS =
Collections.singletonList(SCHEMA_VERSION);
+
+private final ConcurrentMap jsonSchemas;
+private final ConcurrentMap
schemaFactories;
+private volatile SchemaVersion schemaVersion;
+
+public InMemoryJsonSchemaRegistry() {
+jsonSchemas = new ConcurrentHashMap<>();
+schemaFactories = Arrays.stream(SchemaVersion.values())
+.collect(Collectors.toConcurrentMap(Function.identity(),
+schemaDraftVersion ->
JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.fromId(schemaDraftVersion.getUri()).get(;
+schemaVersion =
SchemaVersion.valueOf(SCHEMA_VERSION.getDefaultValue());
+}
+
+@Override
+public void onPropertyModified(final PropertyDescriptor descriptor, final
String oldValue, final String newValue) {
+if (SCHEMA_VERSION.getName().equals(descriptor.getName()) &&
!newValue.equals(oldValue)) {
+schemaVersion = SchemaVersion.valueOf(newValue);
+} else if(descriptor.isDynamic() && isBlank(newValue)) {
+jsonSchemas.remove(descriptor.getName());
+} else if (descriptor.isDynamic() && isNotBlank(newValue)) {
+try {
+final String schemaName = descriptor.getName();
+final JsonSchemaFactory jsonSchemaFactory =
schemaF
[PR] NIFI-12481: Filtering out unauthorized registry clients to avoid unhandled error [nifi]
mcgilman opened a new pull request, #8147: URL: https://github.com/apache/nifi/pull/8147 NIFI-12481: - Filtering out unauthorized registry clients to avoid unhandled error. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12442) Adding support for RocksDB
[ https://issues.apache.org/jira/browse/NIFI-12442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794728#comment-17794728 ] David Handermann commented on NIFI-12442: - Thanks for the link to the repository [~kommpn], it is helpful to see the implementation. On initial evaluation, it seems like the Reader and Writer Processors would not be necessary if you used RocksDB to implement the [DistributedMapCacheClient|https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-standard-services/nifi-distributed-cache-client-service-api/src/main/java/org/apache/nifi/distributed/cache/client/DistributedMapCacheClient.java] interface. That interface can be used with existing Processors to add and remove elements, which appears to be the primary purpose of the custom Processors, if I am following the implementation. It is worth noting that the DistributedMapCacheClient, by its name, aims to support services that are in fact distributed, which RocksDB is not. However, for a proof-of-concept, the interface seems like a good fit, and it would enable integration with existing components. In terms of Apache NiFi maintenance, there is still the concern about the RocksDB JNI JAR containing 60 MB of native libraries, but as a generally available component through Git, you could also publish it to Maven Central for others to evaluate. > Adding support for RocksDB > -- > > Key: NIFI-12442 > URL: https://issues.apache.org/jira/browse/NIFI-12442 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.23.2 >Reporter: Giovanni >Priority: Minor > Original Estimate: 336h > Remaining Estimate: 336h > > I would like to suggest the creation of 3 new components: > The first one is a service, which opens an existing RocksDb or, eventually, > using RocksOptions, create it from scratch. It will manage all the open > options (classic, read/write, only read, secondary). > The second one, is a RocksDbReader, that uses the service to communicate with > the RocksDb in order to retrieve informations through a lookup. It can save > the searched content inside an attribute or inside the flowFile content. It > will be capable of using both APIs, such as simple "db.get" and via > RocksIterator. > The last one, is a RocksDbWriter, that uses the same service as the reader, > but can write values inside the RocksDb, both from flowFile attribute or > flowFile content, using properties to determine the key to use. > > Feel free to express your opinions, if you think this will be useful or > useless. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-8134 allow unescapeJson Record Path function to recursively convert Maps to Records [nifi]
exceptionfactory commented on code in PR #7745:
URL: https://github.com/apache/nifi/pull/7745#discussion_r1420465749
##
nifi-commons/nifi-record-path/src/test/java/org/apache/nifi/record/path/TestRecordPath.java:
##
@@ -1837,7 +1838,41 @@ public void testUnescapeJson() {
put("firstName", "John");
put("age", 30);
}}, "json_str"),
-RecordPath.compile("unescapeJson(/json_str,
'true')").evaluate(recordFromMap).getSelectedFields().findFirst().orElseThrow(IllegalStateException::new).getValue()
+RecordPath.compile("unescapeJson(/json_str,
'true')").evaluate(recordFromMap).getSelectedFields().findFirst().orElseThrow(AssertionError::new).getValue()
+);
+
+// test nested Record converted from Map Object
+final Record nestedRecordFromMap = new MapRecord(schema,
+Collections.singletonMap(
+"json_str",
+
"{\"firstName\":\"John\",\"age\":30,\"addresses\":[{\"address_1\":\"123 Fake
Street\"}]}")
+);
+// recursively convert Maps to Records (addresses becomes and ARRAY or
RECORDs)
+assertEquals(
+DataTypeUtils.toRecord(new LinkedHashMap(){{
+put("firstName", "John");
+put("age", 30);
+put("addresses", new Object[]
{DataTypeUtils.toRecord(Collections.singletonMap("address_1", "123 Fake
Street"), "addresses")});
+}}, "json_str"),
Review Comment:
Yes, Map.of() is much better in many ways, and it is unfortunate that some
portions of test code use the initialization approach out of convenience.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-11627 Added an in memory JSON schema registry which ValidationJson can now use to retrieve JSON schemas from for validation. [nifi]
exceptionfactory commented on code in PR #8005:
URL: https://github.com/apache/nifi/pull/8005#discussion_r1420462605
##
nifi-nar-bundles/nifi-registry-bundle/nifi-registry-service/src/main/java/org/apache/nifi/schemaregistry/services/InMemoryJsonSchemaRegistry.java:
##
@@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.schemaregistry.services;
+
+import com.networknt.schema.JsonSchemaFactory;
+import com.networknt.schema.SpecVersion;
+import org.apache.nifi.annotation.behavior.DynamicProperty;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.JsonSchema;
+import org.apache.nifi.schema.access.JsonSchemaRegistryComponent;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.schema.access.SchemaVersion;
+
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import static org.apache.commons.lang3.StringUtils.isBlank;
+import static org.apache.commons.lang3.StringUtils.isNotBlank;
+
+@Tags({"schema", "registry", "json"})
+@CapabilityDescription("Provides a service for registering and accessing
schemas. One can register a schema "
++ "as a dynamic property where 'name' represents the schema name and
'value' represents the textual "
++ "representation of the actual schema following the syntax and
semantics of JSON's Schema format. "
++ "Empty schemas and schemas only consisting of whitespace are not
acceptable schemas."
++ "The registry is heterogeneous registry as it can store schemas of
multiple schema draft versions. "
++ "By default the registry is configured to store schemas of Draft
2020-12. When a schema is added, the version "
++ "which is currently is set, is what the schema is saved as.")
+@DynamicProperty(name = "Schema name", value = "Schema Content",
+description = "Adds a named schema using the JSON string
representation of a JSON schema",
+expressionLanguageScope = ExpressionLanguageScope.NONE)
+public class InMemoryJsonSchemaRegistry extends AbstractControllerService
implements JsonSchemaRegistry, JsonSchemaRegistryComponent {
+
+private static final List PROPERTY_DESCRIPTORS =
Collections.singletonList(SCHEMA_VERSION);
+
+private final ConcurrentMap jsonSchemas;
+private final ConcurrentMap
schemaFactories;
+private volatile SchemaVersion schemaVersion;
+
+public InMemoryJsonSchemaRegistry() {
+jsonSchemas = new ConcurrentHashMap<>();
+schemaFactories = Arrays.stream(SchemaVersion.values())
+.collect(Collectors.toConcurrentMap(Function.identity(),
+schemaDraftVersion ->
JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.fromId(schemaDraftVersion.getUri()).get(;
+schemaVersion =
SchemaVersion.valueOf(SCHEMA_VERSION.getDefaultValue());
+}
+
+@Override
+public void onPropertyModified(final PropertyDescriptor descriptor, final
String oldValue, final String newValue) {
+if (SCHEMA_VERSION.getName().equals(descriptor.getName()) &&
!newValue.equals(oldValue)) {
+schemaVersion = SchemaVersion.valueOf(newValue);
+} else if(descriptor.isDynamic() && isBlank(newValue)) {
+jsonSchemas.remove(descriptor.getName());
+} else if (descriptor.isDynamic() && isNotBlank(newValue)) {
+try {
+final String schemaName = descriptor.getName();
+final JsonSchemaFactory jsonSchemaFactory
Re: [PR] NIFI-11627 Added an in memory JSON schema registry which ValidationJson can now use to retrieve JSON schemas from for validation. [nifi]
dan-s1 commented on code in PR #8005:
URL: https://github.com/apache/nifi/pull/8005#discussion_r1420455910
##
nifi-nar-bundles/nifi-registry-bundle/nifi-registry-service/src/main/java/org/apache/nifi/schemaregistry/services/InMemoryJsonSchemaRegistry.java:
##
@@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.schemaregistry.services;
+
+import com.networknt.schema.JsonSchemaFactory;
+import com.networknt.schema.SpecVersion;
+import org.apache.nifi.annotation.behavior.DynamicProperty;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.JsonSchema;
+import org.apache.nifi.schema.access.JsonSchemaRegistryComponent;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.schema.access.SchemaVersion;
+
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import static org.apache.commons.lang3.StringUtils.isBlank;
+import static org.apache.commons.lang3.StringUtils.isNotBlank;
+
+@Tags({"schema", "registry", "json"})
+@CapabilityDescription("Provides a service for registering and accessing
schemas. One can register a schema "
++ "as a dynamic property where 'name' represents the schema name and
'value' represents the textual "
++ "representation of the actual schema following the syntax and
semantics of JSON's Schema format. "
++ "Empty schemas and schemas only consisting of whitespace are not
acceptable schemas."
++ "The registry is heterogeneous registry as it can store schemas of
multiple schema draft versions. "
++ "By default the registry is configured to store schemas of Draft
2020-12. When a schema is added, the version "
++ "which is currently is set, is what the schema is saved as.")
+@DynamicProperty(name = "Schema name", value = "Schema Content",
+description = "Adds a named schema using the JSON string
representation of a JSON schema",
+expressionLanguageScope = ExpressionLanguageScope.NONE)
+public class InMemoryJsonSchemaRegistry extends AbstractControllerService
implements JsonSchemaRegistry, JsonSchemaRegistryComponent {
+
+private static final List PROPERTY_DESCRIPTORS =
Collections.singletonList(SCHEMA_VERSION);
+
+private final ConcurrentMap jsonSchemas;
+private final ConcurrentMap
schemaFactories;
+private volatile SchemaVersion schemaVersion;
+
+public InMemoryJsonSchemaRegistry() {
+jsonSchemas = new ConcurrentHashMap<>();
+schemaFactories = Arrays.stream(SchemaVersion.values())
+.collect(Collectors.toConcurrentMap(Function.identity(),
+schemaDraftVersion ->
JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.fromId(schemaDraftVersion.getUri()).get(;
+schemaVersion =
SchemaVersion.valueOf(SCHEMA_VERSION.getDefaultValue());
+}
+
+@Override
+public void onPropertyModified(final PropertyDescriptor descriptor, final
String oldValue, final String newValue) {
+if (SCHEMA_VERSION.getName().equals(descriptor.getName()) &&
!newValue.equals(oldValue)) {
+schemaVersion = SchemaVersion.valueOf(newValue);
+} else if(descriptor.isDynamic() && isBlank(newValue)) {
+jsonSchemas.remove(descriptor.getName());
+} else if (descriptor.isDynamic() && isNotBlank(newValue)) {
+try {
+final String schemaName = descriptor.getName();
+final JsonSchemaFactory jsonSchemaFactory =
schemaF
Re: [PR] NIFI-11627 Added an in memory JSON schema registry which ValidationJson can now use to retrieve JSON schemas from for validation. [nifi]
dan-s1 commented on code in PR #8005:
URL: https://github.com/apache/nifi/pull/8005#discussion_r1420455910
##
nifi-nar-bundles/nifi-registry-bundle/nifi-registry-service/src/main/java/org/apache/nifi/schemaregistry/services/InMemoryJsonSchemaRegistry.java:
##
@@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.schemaregistry.services;
+
+import com.networknt.schema.JsonSchemaFactory;
+import com.networknt.schema.SpecVersion;
+import org.apache.nifi.annotation.behavior.DynamicProperty;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.JsonSchema;
+import org.apache.nifi.schema.access.JsonSchemaRegistryComponent;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.schema.access.SchemaVersion;
+
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import static org.apache.commons.lang3.StringUtils.isBlank;
+import static org.apache.commons.lang3.StringUtils.isNotBlank;
+
+@Tags({"schema", "registry", "json"})
+@CapabilityDescription("Provides a service for registering and accessing
schemas. One can register a schema "
++ "as a dynamic property where 'name' represents the schema name and
'value' represents the textual "
++ "representation of the actual schema following the syntax and
semantics of JSON's Schema format. "
++ "Empty schemas and schemas only consisting of whitespace are not
acceptable schemas."
++ "The registry is heterogeneous registry as it can store schemas of
multiple schema draft versions. "
++ "By default the registry is configured to store schemas of Draft
2020-12. When a schema is added, the version "
++ "which is currently is set, is what the schema is saved as.")
+@DynamicProperty(name = "Schema name", value = "Schema Content",
+description = "Adds a named schema using the JSON string
representation of a JSON schema",
+expressionLanguageScope = ExpressionLanguageScope.NONE)
+public class InMemoryJsonSchemaRegistry extends AbstractControllerService
implements JsonSchemaRegistry, JsonSchemaRegistryComponent {
+
+private static final List PROPERTY_DESCRIPTORS =
Collections.singletonList(SCHEMA_VERSION);
+
+private final ConcurrentMap jsonSchemas;
+private final ConcurrentMap
schemaFactories;
+private volatile SchemaVersion schemaVersion;
+
+public InMemoryJsonSchemaRegistry() {
+jsonSchemas = new ConcurrentHashMap<>();
+schemaFactories = Arrays.stream(SchemaVersion.values())
+.collect(Collectors.toConcurrentMap(Function.identity(),
+schemaDraftVersion ->
JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.fromId(schemaDraftVersion.getUri()).get(;
+schemaVersion =
SchemaVersion.valueOf(SCHEMA_VERSION.getDefaultValue());
+}
+
+@Override
+public void onPropertyModified(final PropertyDescriptor descriptor, final
String oldValue, final String newValue) {
+if (SCHEMA_VERSION.getName().equals(descriptor.getName()) &&
!newValue.equals(oldValue)) {
+schemaVersion = SchemaVersion.valueOf(newValue);
+} else if(descriptor.isDynamic() && isBlank(newValue)) {
+jsonSchemas.remove(descriptor.getName());
+} else if (descriptor.isDynamic() && isNotBlank(newValue)) {
+try {
+final String schemaName = descriptor.getName();
+final JsonSchemaFactory jsonSchemaFactory =
schemaF
Re: [PR] NIFI-12493 Update Documentation References to Java 21 [nifi]
EndzeitBegins commented on PR #8144: URL: https://github.com/apache/nifi/pull/8144#issuecomment-1847173795 Looks good. Thanks for incorporating the additional adjustments. 👍🏻 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12494) Broken CLI - jakarta.ws.rs.client.ClientBuilder cannot be found
[
https://issues.apache.org/jira/browse/NIFI-12494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794713#comment-17794713
]
David Handermann commented on NIFI-12494:
-
The problem is due to the nifi-toolkit-assembly referencing the older version
of Jersey as mentioned. Unfortunately the property-based update in
nifi-toolkit-cli does not carry over to nifi-toolkit-assembly at build time.
This can be resolved in the interim by setting the jersey.bom.version property
in the nifi-toolkit-assembly.
> Broken CLI - jakarta.ws.rs.client.ClientBuilder cannot be found
> ---
>
> Key: NIFI-12494
> URL: https://issues.apache.org/jira/browse/NIFI-12494
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 2.0.0-M1
>Reporter: Pierre Villard
>Priority: Blocker
> Fix For: 2.0.0
>
>
> When using the CLI, it currently throws the below error:
> {code:java}
> org.apache.nifi.toolkit.cli.api.CommandException: Error executing command
> 'pg-export' : java.lang.ClassNotFoundException: Provider for
> jakarta.ws.rs.client.ClientBuilder cannot be found
> at
> org.apache.nifi.toolkit.cli.impl.command.nifi.AbstractNiFiCommand.doExecute(AbstractNiFiCommand.java:65)
> at
> org.apache.nifi.toolkit.cli.impl.command.AbstractPropertyCommand.execute(AbstractPropertyCommand.java:74)
> at
> org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.processCommand(CommandProcessor.java:248)
> at
> org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.processGroupCommand(CommandProcessor.java:229)
> at
> org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.process(CommandProcessor.java:184)
> at org.apache.nifi.toolkit.cli.CLIMain.runSingleCommand(CLIMain.java:145)
> at org.apache.nifi.toolkit.cli.CLIMain.main(CLIMain.java:72)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Provider for jakarta.ws.rs.client.ClientBuilder cannot be found
> at jakarta.ws.rs.client.ClientBuilder.newBuilder(ClientBuilder.java:75)
> at
> org.apache.nifi.toolkit.cli.impl.client.nifi.impl.JerseyNiFiClient.(JerseyNiFiClient.java:98)
> at
> org.apache.nifi.toolkit.cli.impl.client.nifi.impl.JerseyNiFiClient$Builder.build(JerseyNiFiClient.java:345)
> at
> org.apache.nifi.toolkit.cli.impl.client.NiFiClientFactory.createClient(NiFiClientFactory.java:169)
> at
> org.apache.nifi.toolkit.cli.impl.client.NiFiClientFactory.createClient(NiFiClientFactory.java:59)
> at
> org.apache.nifi.toolkit.cli.impl.command.nifi.AbstractNiFiCommand.doExecute(AbstractNiFiCommand.java:62)
> ... 6 more
> Caused by: java.lang.ClassNotFoundException: Provider for
> jakarta.ws.rs.client.ClientBuilder cannot be found
> at jakarta.ws.rs.client.FactoryFinder.find(FactoryFinder.java:163)
> at jakarta.ws.rs.client.ClientBuilder.newBuilder(ClientBuilder.java:62)
> ... 11 more{code}
> Not sure which exact change caused this but it's likely related to some
> dependency management improvements we did.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-8134 allow unescapeJson Record Path function to recursively convert Maps to Records [nifi]
ChrisSamo632 commented on code in PR #7745:
URL: https://github.com/apache/nifi/pull/7745#discussion_r1420434506
##
nifi-commons/nifi-record-path/src/test/java/org/apache/nifi/record/path/TestRecordPath.java:
##
@@ -1837,7 +1838,41 @@ public void testUnescapeJson() {
put("firstName", "John");
put("age", 30);
}}, "json_str"),
-RecordPath.compile("unescapeJson(/json_str,
'true')").evaluate(recordFromMap).getSelectedFields().findFirst().orElseThrow(IllegalStateException::new).getValue()
+RecordPath.compile("unescapeJson(/json_str,
'true')").evaluate(recordFromMap).getSelectedFields().findFirst().orElseThrow(AssertionError::new).getValue()
+);
+
+// test nested Record converted from Map Object
+final Record nestedRecordFromMap = new MapRecord(schema,
+Collections.singletonMap(
+"json_str",
+
"{\"firstName\":\"John\",\"age\":30,\"addresses\":[{\"address_1\":\"123 Fake
Street\"}]}")
+);
+// recursively convert Maps to Records (addresses becomes and ARRAY or
RECORDs)
+assertEquals(
+DataTypeUtils.toRecord(new LinkedHashMap(){{
+put("firstName", "John");
+put("age", 30);
+put("addresses", new Object[]
{DataTypeUtils.toRecord(Collections.singletonMap("address_1", "123 Fake
Street"), "addresses")});
+}}, "json_str"),
Review Comment:
I've never been a particular fan of the `{{ ... }}` constructor approach,
but I've seen it throughout lots of NiFi modules, so had just been copying the
style.
With the advent of `Map.of`, that's a better approach anyway I think
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-12435) Update QuestDB to 7.3.7
[ https://issues.apache.org/jira/browse/NIFI-12435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-12435: -- Fix Version/s: 2.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Update QuestDB to 7.3.7 > --- > > Key: NIFI-12435 > URL: https://issues.apache.org/jira/browse/NIFI-12435 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: David Handermann >Priority: Minor > Fix For: 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Update QuestDB to 7.3.5 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12492) Refactor QuestDB Status Repository to Separate NAR
[
https://issues.apache.org/jira/browse/NIFI-12492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-12492.
---
Fix Version/s: 2.0.0
Resolution: Fixed
> Refactor QuestDB Status Repository to Separate NAR
> --
>
> Key: NIFI-12492
> URL: https://issues.apache.org/jira/browse/NIFI-12492
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Extensions
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Fix For: 2.0.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The Embedded QuestDB Status History Repository provides an implementation
> that supports persistent storage of component metrics. The current
> implementation is packaged together with the default implementation in
> {{{}nifi-framework-core{}}}.
> Based on recent discussions regarding maintenance and improvements to
> QuestDB, moving the implementation to a separate NAR would streamline the
> standard framework NAR and also make it easier to improve the QuestDB
> implementation. There are several shared components that should be moved to a
> common JAR module, which both the QuestDB and volatile implementations can
> use as needed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12494) Broken CLI - jakarta.ws.rs.client.ClientBuilder cannot be found
[
https://issues.apache.org/jira/browse/NIFI-12494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794706#comment-17794706
]
David Handermann commented on NIFI-12494:
-
Thanks for highlighting this issue [~pvillard]. It appears to be related to
NIFI-12384, which included targeted upgrades for Jersey and JAX RS to Registry
and Toolkit components. I am working on the more general upgrade to Spring 6
and related libraries in NIFI-11703, which will align these versions at a
higher level, which will help resolve this problem if it is not addressed more
narrowly before then.
> Broken CLI - jakarta.ws.rs.client.ClientBuilder cannot be found
> ---
>
> Key: NIFI-12494
> URL: https://issues.apache.org/jira/browse/NIFI-12494
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 2.0.0-M1
>Reporter: Pierre Villard
>Priority: Blocker
> Fix For: 2.0.0
>
>
> When using the CLI, it currently throws the below error:
> {code:java}
> org.apache.nifi.toolkit.cli.api.CommandException: Error executing command
> 'pg-export' : java.lang.ClassNotFoundException: Provider for
> jakarta.ws.rs.client.ClientBuilder cannot be found
> at
> org.apache.nifi.toolkit.cli.impl.command.nifi.AbstractNiFiCommand.doExecute(AbstractNiFiCommand.java:65)
> at
> org.apache.nifi.toolkit.cli.impl.command.AbstractPropertyCommand.execute(AbstractPropertyCommand.java:74)
> at
> org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.processCommand(CommandProcessor.java:248)
> at
> org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.processGroupCommand(CommandProcessor.java:229)
> at
> org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.process(CommandProcessor.java:184)
> at org.apache.nifi.toolkit.cli.CLIMain.runSingleCommand(CLIMain.java:145)
> at org.apache.nifi.toolkit.cli.CLIMain.main(CLIMain.java:72)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Provider for jakarta.ws.rs.client.ClientBuilder cannot be found
> at jakarta.ws.rs.client.ClientBuilder.newBuilder(ClientBuilder.java:75)
> at
> org.apache.nifi.toolkit.cli.impl.client.nifi.impl.JerseyNiFiClient.(JerseyNiFiClient.java:98)
> at
> org.apache.nifi.toolkit.cli.impl.client.nifi.impl.JerseyNiFiClient$Builder.build(JerseyNiFiClient.java:345)
> at
> org.apache.nifi.toolkit.cli.impl.client.NiFiClientFactory.createClient(NiFiClientFactory.java:169)
> at
> org.apache.nifi.toolkit.cli.impl.client.NiFiClientFactory.createClient(NiFiClientFactory.java:59)
> at
> org.apache.nifi.toolkit.cli.impl.command.nifi.AbstractNiFiCommand.doExecute(AbstractNiFiCommand.java:62)
> ... 6 more
> Caused by: java.lang.ClassNotFoundException: Provider for
> jakarta.ws.rs.client.ClientBuilder cannot be found
> at jakarta.ws.rs.client.FactoryFinder.find(FactoryFinder.java:163)
> at jakarta.ws.rs.client.ClientBuilder.newBuilder(ClientBuilder.java:62)
> ... 11 more{code}
> Not sure which exact change caused this but it's likely related to some
> dependency management improvements we did.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12492) Refactor QuestDB Status Repository to Separate NAR
[
https://issues.apache.org/jira/browse/NIFI-12492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794702#comment-17794702
]
ASF subversion and git services commented on NIFI-12492:
Commit a126f6a7dff64f2d6a1db1832890d696df11e67b in nifi's branch
refs/heads/main from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a126f6a7df ]
NIFI-12492 Moved QuestDB Status Repository to Separate NAR
- Added nifi-framework-status-history-shared module for common components
- Added include-questdb build profile for including the QuestDB NAR
- Upgraded QuestDB from 7.2 to 7.3.7 for NIFI-12435
Signed-off-by: Bence Simon
This closes #8141
> Refactor QuestDB Status Repository to Separate NAR
> --
>
> Key: NIFI-12492
> URL: https://issues.apache.org/jira/browse/NIFI-12492
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Extensions
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The Embedded QuestDB Status History Repository provides an implementation
> that supports persistent storage of component metrics. The current
> implementation is packaged together with the default implementation in
> {{{}nifi-framework-core{}}}.
> Based on recent discussions regarding maintenance and improvements to
> QuestDB, moving the implementation to a separate NAR would streamline the
> standard framework NAR and also make it easier to improve the QuestDB
> implementation. There are several shared components that should be moved to a
> common JAR module, which both the QuestDB and volatile implementations can
> use as needed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12435) Update QuestDB to 7.3.7
[ https://issues.apache.org/jira/browse/NIFI-12435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794703#comment-17794703 ] ASF subversion and git services commented on NIFI-12435: Commit a126f6a7dff64f2d6a1db1832890d696df11e67b in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a126f6a7df ] NIFI-12492 Moved QuestDB Status Repository to Separate NAR - Added nifi-framework-status-history-shared module for common components - Added include-questdb build profile for including the QuestDB NAR - Upgraded QuestDB from 7.2 to 7.3.7 for NIFI-12435 Signed-off-by: Bence Simon This closes #8141 > Update QuestDB to 7.3.7 > --- > > Key: NIFI-12435 > URL: https://issues.apache.org/jira/browse/NIFI-12435 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 2.0.0-M1 >Reporter: Mike R >Assignee: David Handermann >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > Update QuestDB to 7.3.5 -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] NIFI-12492 Move QuestDB Status Repository to Separate NAR [nifi]
simonbence closed pull request #8141: NIFI-12492 Move QuestDB Status Repository to Separate NAR URL: https://github.com/apache/nifi/pull/8141 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12475) Disable Bypass Validation by Default in PutMongoRecord
[
https://issues.apache.org/jira/browse/NIFI-12475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-12475:
--
Fix Version/s: 2.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Disable Bypass Validation by Default in PutMongoRecord
> --
>
> Key: NIFI-12475
> URL: https://issues.apache.org/jira/browse/NIFI-12475
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 2.0.0-M1, 1.16.1, 1.24.0
> Environment: based on standard container apache/nifi from docker hub
> no customer processors.
>Reporter: Patrick A. Mol
>Assignee: David Handermann
>Priority: Minor
> Fix For: 2.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Across a few versions of NiFi and Mongo,
> the use of PutMongoRecord suddenly stopped working and returned an error.
> Using a flowfile with one valid record, and on PutMongoRecord failure,
> sending the flowfile to SplitRecord and feeding it to PutMongo, using the
> same standard mongodb controller service, the insert would work without error.
> Turns out that the default setting for the PutMongoRecord property Bypass
> Validation is {_}True{_}, which requires elevated privileges in Mongo.
> Changing the property to False allows insert without error.
> The error text is
> {noformat}
> PutMongoRecord[id=018b1026-a670-1590-7941-b6978c972dc6] PutMongoRecord failed
> with error:: com.mongodb.MongoCommandException: Command failed with error 13
> (Unauthorized): 'not authorized on MONGO_DATABASE_NAME to execute command {
> insert: "COLLECTION_NAME", ordered: false, bypassDocumentValidation: true,
> txnNumber: 1, $db: "MONGO_DATABASE_NAME", $clusterTime: { clusterTime:
> Timestamp(1701736623, 1), signature: { hash: BinData(0,
> 62B24A36869A7FAF07C7798019F072CC764E8A9D), keyId: 7264286828646105928 } },
> lsid: { id: UUID("722347df-2349-4ec5-88f2-867a946d9614") } }' on server
> MONGODB_URI_WITH_PORTNUMBER. The full response is {"ok": 0.0, "errmsg": "not
> authorized on MONGO_DATABASE_NAME to execute command { insert:
> \"COLLECTION_NAME\", ordered: false, bypassDocumentValidation: true,
> txnNumber: 1, $db: \"MONGO_DATABASE_NAME\", $clusterTime: { clusterTime:
> Timestamp(1701736623, 1), signature: { hash: BinData(0,
> 62B24A36869A7FAF07C7798019F072CC764E8A9D), keyId: 7264286828646105928 } },
> lsid: { id: UUID(\"722347df-2349-4ec5-88f2-867a946d9614\") } }", "code": 13,
> "codeName": "Unauthorized", "$clusterTime": {"clusterTime": {"$timestamp":
> {"t": 1701736623, "i": 1}}, "signature": {"hash": {"$binary": {"base64":
> "YrJKNoaaf68Hx3mAGfByzHZOip0=", "subType": "00"}}, "keyId":
> 7264286828646105928}}, "operationTime": {"$timestamp": {"t": 1701736623, "i":
> 1}}}
> {noformat}
> Apparently, PutMongo does not use the same setting for the bypass document
> validation flag, so there is an inconsistency.
> Other libraries/tools, e.g. pymongo insert_many(), also default to False.
> Details regarding the privilege in MongoDB are here
> https://www.mongodb.com/docs/manual/reference/privilege-actions/#mongodb-authaction-bypassDocumentValidation
> With the privilege requiring a custom role in MongoDB, it is debatable
> whether the default setting to True is a bug or changing it to False is an
> improvement.
> At least the error and resolution is recorded.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12475) Disable Bypass Validation by Default in PutMongoRecord
[
https://issues.apache.org/jira/browse/NIFI-12475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794699#comment-17794699
]
ASF subversion and git services commented on NIFI-12475:
Commit ec3da7144c1b8d5825ff12f9c307dc1f11a2d7e9 in nifi's branch
refs/heads/main from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ec3da7144c ]
NIFI-12475 Disabled Bypass Validation by Default in PutMongoRecord
- Updated property description to indicate that bypass validation is a
Privilege Action that can cause authorization errors.
Signed-off-by: Pierre Villard
This closes #8139.
> Disable Bypass Validation by Default in PutMongoRecord
> --
>
> Key: NIFI-12475
> URL: https://issues.apache.org/jira/browse/NIFI-12475
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 2.0.0-M1, 1.16.1, 1.24.0
> Environment: based on standard container apache/nifi from docker hub
> no customer processors.
>Reporter: Patrick A. Mol
>Assignee: David Handermann
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Across a few versions of NiFi and Mongo,
> the use of PutMongoRecord suddenly stopped working and returned an error.
> Using a flowfile with one valid record, and on PutMongoRecord failure,
> sending the flowfile to SplitRecord and feeding it to PutMongo, using the
> same standard mongodb controller service, the insert would work without error.
> Turns out that the default setting for the PutMongoRecord property Bypass
> Validation is {_}True{_}, which requires elevated privileges in Mongo.
> Changing the property to False allows insert without error.
> The error text is
> {noformat}
> PutMongoRecord[id=018b1026-a670-1590-7941-b6978c972dc6] PutMongoRecord failed
> with error:: com.mongodb.MongoCommandException: Command failed with error 13
> (Unauthorized): 'not authorized on MONGO_DATABASE_NAME to execute command {
> insert: "COLLECTION_NAME", ordered: false, bypassDocumentValidation: true,
> txnNumber: 1, $db: "MONGO_DATABASE_NAME", $clusterTime: { clusterTime:
> Timestamp(1701736623, 1), signature: { hash: BinData(0,
> 62B24A36869A7FAF07C7798019F072CC764E8A9D), keyId: 7264286828646105928 } },
> lsid: { id: UUID("722347df-2349-4ec5-88f2-867a946d9614") } }' on server
> MONGODB_URI_WITH_PORTNUMBER. The full response is {"ok": 0.0, "errmsg": "not
> authorized on MONGO_DATABASE_NAME to execute command { insert:
> \"COLLECTION_NAME\", ordered: false, bypassDocumentValidation: true,
> txnNumber: 1, $db: \"MONGO_DATABASE_NAME\", $clusterTime: { clusterTime:
> Timestamp(1701736623, 1), signature: { hash: BinData(0,
> 62B24A36869A7FAF07C7798019F072CC764E8A9D), keyId: 7264286828646105928 } },
> lsid: { id: UUID(\"722347df-2349-4ec5-88f2-867a946d9614\") } }", "code": 13,
> "codeName": "Unauthorized", "$clusterTime": {"clusterTime": {"$timestamp":
> {"t": 1701736623, "i": 1}}, "signature": {"hash": {"$binary": {"base64":
> "YrJKNoaaf68Hx3mAGfByzHZOip0=", "subType": "00"}}, "keyId":
> 7264286828646105928}}, "operationTime": {"$timestamp": {"t": 1701736623, "i":
> 1}}}
> {noformat}
> Apparently, PutMongo does not use the same setting for the bypass document
> validation flag, so there is an inconsistency.
> Other libraries/tools, e.g. pymongo insert_many(), also default to False.
> Details regarding the privilege in MongoDB are here
> https://www.mongodb.com/docs/manual/reference/privilege-actions/#mongodb-authaction-bypassDocumentValidation
> With the privilege requiring a custom role in MongoDB, it is debatable
> whether the default setting to True is a bug or changing it to False is an
> improvement.
> At least the error and resolution is recorded.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-12475 Disable Bypass Validation by Default in PutMongoRecord [nifi]
asfgit closed pull request #8139: NIFI-12475 Disable Bypass Validation by Default in PutMongoRecord URL: https://github.com/apache/nifi/pull/8139 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12493 Update Documentation References to Java 21 [nifi]
exceptionfactory commented on PR #8144: URL: https://github.com/apache/nifi/pull/8144#issuecomment-1847141493 Thanks for the helpful pointers to additional references @EndzeitBegins! I pushed a change updating references to Java 21, and in a couple comments, removed version numbers as it was not necessary. That's also a good point regarding those class references. There are probably other similar types of things, and it would be useful to address those in a separate pull request that covers general minor changes for workarounds that are no longer necessary. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12494) Broken CLI - jakarta.ws.rs.client.ClientBuilder cannot be found
[
https://issues.apache.org/jira/browse/NIFI-12494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-12494:
--
Priority: Blocker (was: Major)
> Broken CLI - jakarta.ws.rs.client.ClientBuilder cannot be found
> ---
>
> Key: NIFI-12494
> URL: https://issues.apache.org/jira/browse/NIFI-12494
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 2.0.0-M1
>Reporter: Pierre Villard
>Priority: Blocker
> Fix For: 2.0.0
>
>
> When using the CLI, it currently throws the below error:
> {code:java}
> org.apache.nifi.toolkit.cli.api.CommandException: Error executing command
> 'pg-export' : java.lang.ClassNotFoundException: Provider for
> jakarta.ws.rs.client.ClientBuilder cannot be found
> at
> org.apache.nifi.toolkit.cli.impl.command.nifi.AbstractNiFiCommand.doExecute(AbstractNiFiCommand.java:65)
> at
> org.apache.nifi.toolkit.cli.impl.command.AbstractPropertyCommand.execute(AbstractPropertyCommand.java:74)
> at
> org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.processCommand(CommandProcessor.java:248)
> at
> org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.processGroupCommand(CommandProcessor.java:229)
> at
> org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.process(CommandProcessor.java:184)
> at org.apache.nifi.toolkit.cli.CLIMain.runSingleCommand(CLIMain.java:145)
> at org.apache.nifi.toolkit.cli.CLIMain.main(CLIMain.java:72)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Provider for jakarta.ws.rs.client.ClientBuilder cannot be found
> at jakarta.ws.rs.client.ClientBuilder.newBuilder(ClientBuilder.java:75)
> at
> org.apache.nifi.toolkit.cli.impl.client.nifi.impl.JerseyNiFiClient.(JerseyNiFiClient.java:98)
> at
> org.apache.nifi.toolkit.cli.impl.client.nifi.impl.JerseyNiFiClient$Builder.build(JerseyNiFiClient.java:345)
> at
> org.apache.nifi.toolkit.cli.impl.client.NiFiClientFactory.createClient(NiFiClientFactory.java:169)
> at
> org.apache.nifi.toolkit.cli.impl.client.NiFiClientFactory.createClient(NiFiClientFactory.java:59)
> at
> org.apache.nifi.toolkit.cli.impl.command.nifi.AbstractNiFiCommand.doExecute(AbstractNiFiCommand.java:62)
> ... 6 more
> Caused by: java.lang.ClassNotFoundException: Provider for
> jakarta.ws.rs.client.ClientBuilder cannot be found
> at jakarta.ws.rs.client.FactoryFinder.find(FactoryFinder.java:163)
> at jakarta.ws.rs.client.ClientBuilder.newBuilder(ClientBuilder.java:62)
> ... 11 more{code}
> Not sure which exact change caused this but it's likely related to some
> dependency management improvements we did.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12494) Broken CLI - jakarta.ws.rs.client.ClientBuilder cannot be found
[
https://issues.apache.org/jira/browse/NIFI-12494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-12494:
--
Description:
When using the CLI, it currently throws the below error:
{code:java}
org.apache.nifi.toolkit.cli.api.CommandException: Error executing command
'pg-export' : java.lang.ClassNotFoundException: Provider for
jakarta.ws.rs.client.ClientBuilder cannot be found
at
org.apache.nifi.toolkit.cli.impl.command.nifi.AbstractNiFiCommand.doExecute(AbstractNiFiCommand.java:65)
at
org.apache.nifi.toolkit.cli.impl.command.AbstractPropertyCommand.execute(AbstractPropertyCommand.java:74)
at
org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.processCommand(CommandProcessor.java:248)
at
org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.processGroupCommand(CommandProcessor.java:229)
at
org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.process(CommandProcessor.java:184)
at org.apache.nifi.toolkit.cli.CLIMain.runSingleCommand(CLIMain.java:145)
at org.apache.nifi.toolkit.cli.CLIMain.main(CLIMain.java:72)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
Provider for jakarta.ws.rs.client.ClientBuilder cannot be found
at jakarta.ws.rs.client.ClientBuilder.newBuilder(ClientBuilder.java:75)
at
org.apache.nifi.toolkit.cli.impl.client.nifi.impl.JerseyNiFiClient.(JerseyNiFiClient.java:98)
at
org.apache.nifi.toolkit.cli.impl.client.nifi.impl.JerseyNiFiClient$Builder.build(JerseyNiFiClient.java:345)
at
org.apache.nifi.toolkit.cli.impl.client.NiFiClientFactory.createClient(NiFiClientFactory.java:169)
at
org.apache.nifi.toolkit.cli.impl.client.NiFiClientFactory.createClient(NiFiClientFactory.java:59)
at
org.apache.nifi.toolkit.cli.impl.command.nifi.AbstractNiFiCommand.doExecute(AbstractNiFiCommand.java:62)
... 6 more
Caused by: java.lang.ClassNotFoundException: Provider for
jakarta.ws.rs.client.ClientBuilder cannot be found
at jakarta.ws.rs.client.FactoryFinder.find(FactoryFinder.java:163)
at jakarta.ws.rs.client.ClientBuilder.newBuilder(ClientBuilder.java:62)
... 11 more{code}
Not sure which exact change caused this but it's likely related to some
dependency management improvements we did.
was:
When using the CLI, it currently throws the below error:
{code:java}
ERROR: Error executing command 'pg-export' : java.lang.ClassNotFoundException:
Provider for jakarta.ws.rs.client.ClientBuilder cannot be found
org.apache.nifi.toolkit.cli.api.CommandException: Error executing command
'pg-export' : java.lang.ClassNotFoundException: Provider for
jakarta.ws.rs.client.ClientBuilder cannot be found at
org.apache.nifi.toolkit.cli.impl.command.nifi.AbstractNiFiCommand.doExecute(AbstractNiFiCommand.java:65)
at
org.apache.nifi.toolkit.cli.impl.command.AbstractPropertyCommand.execute(AbstractPropertyCommand.java:74)
at
org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.processCommand(CommandProcessor.java:248)
at
org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.processGroupCommand(CommandProcessor.java:229)
at
org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.process(CommandProcessor.java:184)
at org.apache.nifi.toolkit.cli.CLIMain.runSingleCommand(CLIMain.java:145)
at org.apache.nifi.toolkit.cli.CLIMain.main(CLIMain.java:72)Caused by:
java.lang.RuntimeException: java.lang.ClassNotFoundException: Provider for
jakarta.ws.rs.client.ClientBuilder cannot be foundat
jakarta.ws.rs.client.ClientBuilder.newBuilder(ClientBuilder.java:75) at
org.apache.nifi.toolkit.cli.impl.client.nifi.impl.JerseyNiFiClient.(JerseyNiFiClient.java:98)
at
org.apache.nifi.toolkit.cli.impl.client.nifi.impl.JerseyNiFiClient$Builder.build(JerseyNiFiClient.java:345)
at
org.apache.nifi.toolkit.cli.impl.client.NiFiClientFactory.createClient(NiFiClientFactory.java:169)
at
org.apache.nifi.toolkit.cli.impl.client.NiFiClientFactory.createClient(NiFiClientFactory.java:59)
at
org.apache.nifi.toolkit.cli.impl.command.nifi.AbstractNiFiCommand.doExecute(AbstractNiFiCommand.java:62)
... 6 moreCaused by: java.lang.ClassNotFoundException: Provider for
jakarta.ws.rs.client.ClientBuilder cannot be found at
jakarta.ws.rs.client.FactoryFinder.find(FactoryFinder.java:163) at
jakarta.ws.rs.client.ClientBuilder.newBuilder(ClientBuilder.java:62) ... 11
more {code}
Not sure which exact change caused this but it's likely related to some
dependency management improvements we did.
> Broken CLI - jakarta.ws.rs.client.ClientBuilder cannot be found
> ---
>
> Key: NIFI-12494
> URL: https://issues.apache.org/jira/browse/NIFI-12494
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 2.0.0-M1
>Reporter: Pierre
[jira] [Created] (NIFI-12494) Broken CLI - jakarta.ws.rs.client.ClientBuilder cannot be found
Pierre Villard created NIFI-12494:
-
Summary: Broken CLI - jakarta.ws.rs.client.ClientBuilder cannot be
found
Key: NIFI-12494
URL: https://issues.apache.org/jira/browse/NIFI-12494
Project: Apache NiFi
Issue Type: Bug
Components: Tools and Build
Affects Versions: 2.0.0-M1
Reporter: Pierre Villard
Fix For: 2.0.0
When using the CLI, it currently throws the below error:
{code:java}
ERROR: Error executing command 'pg-export' : java.lang.ClassNotFoundException:
Provider for jakarta.ws.rs.client.ClientBuilder cannot be found
org.apache.nifi.toolkit.cli.api.CommandException: Error executing command
'pg-export' : java.lang.ClassNotFoundException: Provider for
jakarta.ws.rs.client.ClientBuilder cannot be found at
org.apache.nifi.toolkit.cli.impl.command.nifi.AbstractNiFiCommand.doExecute(AbstractNiFiCommand.java:65)
at
org.apache.nifi.toolkit.cli.impl.command.AbstractPropertyCommand.execute(AbstractPropertyCommand.java:74)
at
org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.processCommand(CommandProcessor.java:248)
at
org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.processGroupCommand(CommandProcessor.java:229)
at
org.apache.nifi.toolkit.cli.impl.command.CommandProcessor.process(CommandProcessor.java:184)
at org.apache.nifi.toolkit.cli.CLIMain.runSingleCommand(CLIMain.java:145)
at org.apache.nifi.toolkit.cli.CLIMain.main(CLIMain.java:72)Caused by:
java.lang.RuntimeException: java.lang.ClassNotFoundException: Provider for
jakarta.ws.rs.client.ClientBuilder cannot be foundat
jakarta.ws.rs.client.ClientBuilder.newBuilder(ClientBuilder.java:75) at
org.apache.nifi.toolkit.cli.impl.client.nifi.impl.JerseyNiFiClient.(JerseyNiFiClient.java:98)
at
org.apache.nifi.toolkit.cli.impl.client.nifi.impl.JerseyNiFiClient$Builder.build(JerseyNiFiClient.java:345)
at
org.apache.nifi.toolkit.cli.impl.client.NiFiClientFactory.createClient(NiFiClientFactory.java:169)
at
org.apache.nifi.toolkit.cli.impl.client.NiFiClientFactory.createClient(NiFiClientFactory.java:59)
at
org.apache.nifi.toolkit.cli.impl.command.nifi.AbstractNiFiCommand.doExecute(AbstractNiFiCommand.java:62)
... 6 moreCaused by: java.lang.ClassNotFoundException: Provider for
jakarta.ws.rs.client.ClientBuilder cannot be found at
jakarta.ws.rs.client.FactoryFinder.find(FactoryFinder.java:163) at
jakarta.ws.rs.client.ClientBuilder.newBuilder(ClientBuilder.java:62) ... 11
more {code}
Not sure which exact change caused this but it's likely related to some
dependency management improvements we did.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] NIFI-8134 allow unescapeJson Record Path function to recursively convert Maps to Records [nifi]
ChrisSamo632 commented on code in PR #7745:
URL: https://github.com/apache/nifi/pull/7745#discussion_r1420402511
##
nifi-commons/nifi-record-path/src/test/java/org/apache/nifi/record/path/TestRecordPath.java:
##
@@ -1837,7 +1838,41 @@ public void testUnescapeJson() {
put("firstName", "John");
put("age", 30);
}}, "json_str"),
-RecordPath.compile("unescapeJson(/json_str,
'true')").evaluate(recordFromMap).getSelectedFields().findFirst().orElseThrow(IllegalStateException::new).getValue()
+RecordPath.compile("unescapeJson(/json_str,
'true')").evaluate(recordFromMap).getSelectedFields().findFirst().orElseThrow(AssertionError::new).getValue()
+);
+
+// test nested Record converted from Map Object
+final Record nestedRecordFromMap = new MapRecord(schema,
+Collections.singletonMap(
+"json_str",
+
"{\"firstName\":\"John\",\"age\":30,\"addresses\":[{\"address_1\":\"123 Fake
Street\"}]}")
Review Comment:
I'd been sticking with Java 8 compatible code to get this onto the
`support/1.x` branch, but if we're happy with it being only NiFi 2.x now, I'll
update these tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12492 Move QuestDB Status Repository to Separate NAR [nifi]
exceptionfactory commented on PR #8141: URL: https://github.com/apache/nifi/pull/8141#issuecomment-1847122208 > I reviewed the code and tested the changes. It looks good to me and I am willing to merge it. The only question I have is in this case what is the benefit of using a "nar" instead of a "jar" for `nifi-framework-questdb-status-history`? It contains no pluggable services at this point and based on my tests it should work as a simple jar. I see that the other profiles are refering to nars as well but I assume they contain some processors and controller services. What is the resason for that in this case? Thanks for the feedback @simonbence. The extension point class is `org.apache.nifi.controller.status.history.EmbeddedQuestDbStatusHistoryRepository`, which implements the `StatusHistoryRepository` interfaces, as enumerated in the file under `META-INF/services`. This follows the same pattern as Processors and Controller Services, and is also similar to other framework extension points like the Kubernetes State Manager and Leader Election implementations. With the NAR packaging, this has the standard benefits of ClassLoader isolation and discrete packaging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-8134 allow unescapeJson Record Path function to recursively convert Maps to Records [nifi]
ChrisSamo632 commented on code in PR #7745:
URL: https://github.com/apache/nifi/pull/7745#discussion_r1420401479
##
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java:
##
@@ -469,21 +473,51 @@ public static RecordSchema inferSchema(final Map values, final S
final RecordField recordField = new RecordField(key,
inferredDataType, true);
inferredFieldTypes.add(recordField);
-final Object coercedValue = convertType(rawValue,
inferredDataType, fieldName, charset);
-coercedValues.put(key, coercedValue);
+convertType(rawValue, inferredDataType, fieldName, charset);
}
-final RecordSchema inferredSchema = new
SimpleRecordSchema(inferredFieldTypes);
-return inferredSchema;
+return new SimpleRecordSchema(inferredFieldTypes);
}
public static Record toRecord(final Object value, final String fieldName,
final Charset charset) {
+return toRecord(value, fieldName, charset, false);
+}
+
+private static Object covertObjectToRecord(final Object rawValue, final
String key, final Charset charset) {
+final Object coercedValue;
+if (rawValue instanceof Map) {
+coercedValue = toRecord(rawValue, key, charset, true);
+} else if (rawValue instanceof Object[]) {
+final Object[] objArray = (Object[]) rawValue;
+coercedValue = Arrays.stream(objArray).noneMatch(o -> o instanceof
Map)
+? objArray
+: Arrays.stream(objArray).map(o -> toRecord(o, key,
charset, true)).toArray();
+} else if (rawValue instanceof Collection) {
+final Collection objCollection = (Collection) rawValue;
+// Records have ARRAY DataTypes, so convert any Collections
+coercedValue = objCollection.stream().noneMatch(o -> o instanceof
Map)
+? objCollection.toArray()
+: objCollection.stream().map(o -> toRecord(o, key,
charset, true)).toArray();
Review Comment:
That's fair, I've reverted this and found a coupleof other Streams within
`DataTypeUtils` that should probably be changed too, will include in this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[PR] NIFI-6730 AMQP QoS support [nifi]
MooseTheBrown opened a new pull request, #8146: URL: https://github.com/apache/nifi/pull/8146 # Summary [NIFI-6730](https://issues.apache.org/jira/browse/NIFI-6730) Implement basicQos (prefetch count) support in ConsumeAMQP processor. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ x] Pull Request based on current revision of the `main` branch - [ x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [x ] Build completed using `mvn clean install -P contrib-check` - [x ] JDK 21 ### Licensing - [ x] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ x] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ x] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] NIFI-12490 MiNiFi docker graceful shutdown [nifi]
briansolo1985 opened a new pull request, #8145: URL: https://github.com/apache/nifi/pull/8145 # Summary [NIFI-12490](https://issues.apache.org/jira/browse/NIFI-12490) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [x] Build completed using `mvn clean install -P contrib-check` - [x] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
