[jira] [Created] (NIFIREG-448) Using the BucketFlows Diff endpoint generates an NPE
Daniel Chaffelson created NIFIREG-448:
-
Summary: Using the BucketFlows Diff endpoint generates an NPE
Key: NIFIREG-448
URL: https://issues.apache.org/jira/browse/NIFIREG-448
Project: NiFi Registry
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Daniel Chaffelson
Calling the Bucket Flows Diff endpoint generates an NPE
It might be this courtesy of [~bbende] :
{noformat}
final FlowComparator flowComparator = new
StandardFlowComparator(comparableFlowA, comparableFlowB,
null, new ConciseEvolvingDifferenceDescriptor());{noformat}
{noformat}
An unexpected error has occurred: java.lang.NullPointerException. Returning
Internal Server Error response.java.lang.NullPointerException: nullat
org.apache.nifi.registry.flow.diff.StandardFlowComparator.lambda$compareProperties$2(StandardFlowComparator.java:218)
at java.util.LinkedHashMap.forEach(LinkedHashMap.java:684)at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.compareProperties(StandardFlowComparator.java:182)
at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.compare(StandardFlowComparator.java:156)
at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.lambda$compareComponents$0(StandardFlowComparator.java:89)
at java.util.HashMap.forEach(HashMap.java:1289)at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.compareComponents(StandardFlowComparator.java:87)
at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.compare(StandardFlowComparator.java:341)
at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.lambda$compare$4(StandardFlowComparator.java:347)
at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.lambda$compareComponents$0(StandardFlowComparator.java:89)
at java.util.HashMap.forEach(HashMap.java:1289)at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.compareComponents(StandardFlowComparator.java:87)
at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.compare(StandardFlowComparator.java:347)
at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.lambda$compare$4(StandardFlowComparator.java:347)
at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.lambda$compareComponents$0(StandardFlowComparator.java:89)
at java.util.HashMap.forEach(HashMap.java:1289)at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.compareComponents(StandardFlowComparator.java:87)
at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.compare(StandardFlowComparator.java:347)
at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.lambda$compare$4(StandardFlowComparator.java:347)
at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.lambda$compareComponents$0(StandardFlowComparator.java:89)
at java.util.HashMap.forEach(HashMap.java:1289)at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.compareComponents(StandardFlowComparator.java:87)
at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.compare(StandardFlowComparator.java:347)
at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.lambda$compare$4(StandardFlowComparator.java:347)
at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.lambda$compareComponents$0(StandardFlowComparator.java:89)
at java.util.HashMap.forEach(HashMap.java:1289)at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.compareComponents(StandardFlowComparator.java:87)
at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.compare(StandardFlowComparator.java:347)
at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.compare(StandardFlowComparator.java:76)
at
org.apache.nifi.registry.flow.diff.StandardFlowComparator.compare(StandardFlowComparator.java:67)
at
org.apache.nifi.registry.service.RegistryService.getFlowDiff(RegistryService.java:1055)
at
org.apache.nifi.registry.service.RegistryService$$FastClassBySpringCGLIB$$274de08.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at
org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:366)
at
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
[jira] [Commented] (NIFI-7995) Requesting create Parameter Context via API with Parameters null generates an NPE
[ https://issues.apache.org/jira/browse/NIFI-7995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17230062#comment-17230062 ] Daniel Chaffelson commented on NIFI-7995: - Use the Next branch of NiPyAPI: [https://github.com/Chaffelson/nipyapi/blob/27002493d6f4fbe85d2c0f62d4d69f2825883ac9/nipyapi/parameters.py#L91] Modify this line to be 'else None' to submit the json null instead of empty list, and it should produce the NPE. > Requesting create Parameter Context via API with Parameters null generates an > NPE > - > > Key: NIFI-7995 > URL: https://issues.apache.org/jira/browse/NIFI-7995 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.10.0, 1.12.1 >Reporter: Daniel Chaffelson >Assignee: Otto Fowler >Priority: Minor > > org.apache.nifi.web.api.ParameterContextResource.validateParameterNames(ParameterContextResource.java:403) > gets very upset if you submit a JSON with null for the Parameter listing > instead of an empty or populated list. It is a minor thing, but probably > worth tidying up in the validator. > Discovered because NiPyAPI defaults to Python 'None' for unpopulated > properties. > > Full logged error is: > 2020-11-09 13:20:28,612 ERROR [NiFi Web Server-22] > o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: > java.lang.NullPointerException. Returning Internal Server Error response. > java.lang.NullPointerException: null > at > org.apache.nifi.web.api.ParameterContextResource.validateParameterNames(ParameterContextResource.java:403) > at > org.apache.nifi.web.api.ParameterContextResource.createParameterContext(ParameterContextResource.java:199) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) > at > org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) > at > org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) > at > org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) > at > org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) > at > org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) > at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) > at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) > at org.glassfish.jersey.internal.Errors.process(Errors.java:316) > at org.glassfish.jersey.internal.Errors.process(Errors.java:298) > at org.glassfish.jersey.internal.Errors.process(Errors.java:268) > at > org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) > at > org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) > at > org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) > at > org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) > at > org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) > at > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) > at > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) > at > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) > at > org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1395) > at > org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:755) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1617) > at > org.apache.nifi.web.filter.RequestLogger.doFilter(RequestLogger.java:66) > at >
[jira] [Created] (NIFI-7995) Requesting create Parameter Context via API with Parameters null generates an NPE
Daniel Chaffelson created NIFI-7995: --- Summary: Requesting create Parameter Context via API with Parameters null generates an NPE Key: NIFI-7995 URL: https://issues.apache.org/jira/browse/NIFI-7995 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.12.1, 1.10.0 Reporter: Daniel Chaffelson org.apache.nifi.web.api.ParameterContextResource.validateParameterNames(ParameterContextResource.java:403) gets very upset if you submit a JSON with null for the Parameter listing instead of an empty or populated list. It is a minor thing, but probably worth tidying up in the validator. Discovered because NiPyAPI defaults to Python 'None' for unpopulated properties. Full logged error is: 2020-11-09 13:20:28,612 ERROR [NiFi Web Server-22] o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: java.lang.NullPointerException. Returning Internal Server Error response. java.lang.NullPointerException: null at org.apache.nifi.web.api.ParameterContextResource.validateParameterNames(ParameterContextResource.java:403) at org.apache.nifi.web.api.ParameterContextResource.createParameterContext(ParameterContextResource.java:199) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1395) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:755) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1617) at org.apache.nifi.web.filter.RequestLogger.doFilter(RequestLogger.java:66) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) at
[jira] [Created] (NIFI-5715) Swagger def missing new Processor Validating status
Daniel Chaffelson created NIFI-5715: --- Summary: Swagger def missing new Processor Validating status Key: NIFI-5715 URL: https://issues.apache.org/jira/browse/NIFI-5715 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.7.1 Reporter: Daniel Chaffelson Building the Swagger Json from the rel/nifi-1.7.1 tag produces a definition that is missing the 'Validating' status from relevant Processor Status DTO and Processor Status Snapshot DTO introduced in recent versions. The workaround is to manually add 'Validating' to the list of status entries before compiling any NiFi client against this definition. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4900) nifi Swagger definition - FlowApi ScheduleComponents returns empty component list
[ https://issues.apache.org/jira/browse/NIFI-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375059#comment-16375059 ] Daniel Chaffelson commented on NIFI-4900: - Thanks for the detailed explanation [~mcgilman] This makes sense, in that the field is essentially for submission control only. It is not difficult to implement a wait loop for the components to achieve the submitted state or timeout, which is ultimately what I've done in my Python Client SDK. I think, from your description, that this isn't a bug and also isn't really worth being a feature either, so unless someone else feels invested in the idea I'll close it. > nifi Swagger definition - FlowApi ScheduleComponents returns empty component > list > - > > Key: NIFI-4900 > URL: https://issues.apache.org/jira/browse/NIFI-4900 > Project: Apache NiFi > Issue Type: Bug > Components: SDLC >Affects Versions: 1.5.0 >Reporter: Daniel Chaffelson >Priority: Minor > > When issuing a command to Schedule Components with the 'components' attribute > set, the command returns an empty set back. This seems incongruous. I would > expect to receive back an object listing out the states of each component > that the scheduling command was issued against so I can tell which components > where operated upon. > This can be reproduced by creating two processors within a process group, and > issuing a schedule component command with a component attribute referencing > only one of the processors, which will then change scheduled state leaving > the other as-was. It evidences that the individual component request is being > honoured, but the return from the API is the same as if all components in the > process group had been scheduled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-4900) nifi Swagger definition - FlowApi ScheduleComponents returns empty component list
Daniel Chaffelson created NIFI-4900: --- Summary: nifi Swagger definition - FlowApi ScheduleComponents returns empty component list Key: NIFI-4900 URL: https://issues.apache.org/jira/browse/NIFI-4900 Project: Apache NiFi Issue Type: Bug Components: SDLC Affects Versions: 1.5.0 Reporter: Daniel Chaffelson When issuing a command to Schedule Components with the 'components' attribute set, the command returns an empty set back. This seems incongruous. I would expect to receive back an object listing out the states of each component that the scheduling command was issued against so I can tell which components where operated upon. This can be reproduced by creating two processors within a process group, and issuing a schedule component command with a component attribute referencing only one of the processors, which will then change scheduled state leaving the other as-was. It evidences that the individual component request is being honoured, but the return from the API is the same as if all components in the process group had been scheduled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4859) Swagger Spec VersionControlInformationDTO missing SYNC_FAILURE state
[
https://issues.apache.org/jira/browse/NIFI-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365571#comment-16365571
]
Daniel Chaffelson edited comment on NIFI-4859 at 2/15/18 2:16 PM:
--
This also applies to ProcessGroupStatusSnapshotDTO, specifically the
VersionedFlowState
And the ProcessGroupEntity, and the FlowBreadCrumbEntity
was (Author: chaffelson):
This also applies to ProcessGroupStatusSnapshotDTO, specifically the
VersionedFlowState
> Swagger Spec VersionControlInformationDTO missing SYNC_FAILURE state
>
>
> Key: NIFI-4859
> URL: https://issues.apache.org/jira/browse/NIFI-4859
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Affects Versions: 1.5.0
>Reporter: Daniel Chaffelson
>Assignee: Kevin Doran
>Priority: Minor
> Fix For: 1.6.0
>
>
> It is possible to get a Versioned Process Group into a SYNC_FAILURE state,
> but this is not an allowable state in the code generated from the swagger.json
> This prevents versioned objects from being manipulated via the API in some
> use cases.
> {noformat}
> @state.setter
> def state(self, state):
> """
> Sets the state of this VersionControlInformationDTO.
> The current state of the Process Group, as it relates to the Versioned Flow
> :param state: The state of this VersionControlInformationDTO.
> :type: str
> """
> allowed_values = ["LOCALLY_MODIFIED_DESCENDANT", "LOCALLY_MODIFIED", "STALE",
> "LOCALLY_MODIFIED_AND_STALE", "UP_TO_DATE"]
> if state not in allowed_values:
> raise ValueError(
> "Invalid value for `state` ({0}), must be one of {1}"
> > .format(state, allowed_values)
> )
> E ValueError: Invalid value for `state` (SYNC_FAILURE), must be one of
> ['LOCALLY_MODIFIED_DESCENDANT', 'LOCALLY_MODIFIED', 'STALE',
> 'LOCALLY_MODIFIED_AND_STALE', 'UP_TO_DATE']
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4859) Swagger Spec VersionControlInformationDTO missing SYNC_FAILURE state
[
https://issues.apache.org/jira/browse/NIFI-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365571#comment-16365571
]
Daniel Chaffelson commented on NIFI-4859:
-
This also applies to ProcessGroupStatusSnapshotDTO, specifically the
VersionedFlowState
> Swagger Spec VersionControlInformationDTO missing SYNC_FAILURE state
>
>
> Key: NIFI-4859
> URL: https://issues.apache.org/jira/browse/NIFI-4859
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Affects Versions: 1.5.0
>Reporter: Daniel Chaffelson
>Assignee: Kevin Doran
>Priority: Minor
> Fix For: 1.6.0
>
>
> It is possible to get a Versioned Process Group into a SYNC_FAILURE state,
> but this is not an allowable state in the code generated from the swagger.json
> This prevents versioned objects from being manipulated via the API in some
> use cases.
> {noformat}
> @state.setter
> def state(self, state):
> """
> Sets the state of this VersionControlInformationDTO.
> The current state of the Process Group, as it relates to the Versioned Flow
> :param state: The state of this VersionControlInformationDTO.
> :type: str
> """
> allowed_values = ["LOCALLY_MODIFIED_DESCENDANT", "LOCALLY_MODIFIED", "STALE",
> "LOCALLY_MODIFIED_AND_STALE", "UP_TO_DATE"]
> if state not in allowed_values:
> raise ValueError(
> "Invalid value for `state` ({0}), must be one of {1}"
> > .format(state, allowed_values)
> )
> E ValueError: Invalid value for `state` (SYNC_FAILURE), must be one of
> ['LOCALLY_MODIFIED_DESCENDANT', 'LOCALLY_MODIFIED', 'STALE',
> 'LOCALLY_MODIFIED_AND_STALE', 'UP_TO_DATE']
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFI-4865) Registry API command Create Version can produce a VersionedProcessGroup that causes NiFi to NPE on Version change
Daniel Chaffelson created NIFI-4865:
---
Summary: Registry API command Create Version can produce a
VersionedProcessGroup that causes NiFi to NPE on Version change
Key: NIFI-4865
URL: https://issues.apache.org/jira/browse/NIFI-4865
Project: Apache NiFi
Issue Type: Bug
Components: Flow Versioning
Affects Versions: 1.5.0
Environment: NiFi Docker 1.5.0 / NiFi-Registry Docker 0.1.0
Reporter: Daniel Chaffelson
I imported a complex (300kb) xml template into NiFi and placed it under version
control using the typical GUI commands.
Then using the Registry API command 'create version', I instructed it to make a
zero change new version, resulting in ver1 and ver2 being in the same bucket
and flow object.
Returning to the GUI, I instructed it to update to the new version, which
causes the below NPE.
I have retested this with a fresh environment and am able to reproduce it at
will, though as the template belongs to a client and is very detailed I cannot
in good faith share it directly on a public service - I am happy to supply it
directly in the interests of locking down this bug.
{noformat}
2018-02-09 21:45:40,900 INFO [Version Control Update Thread-1]
o.a.nifi.groups.StandardProcessGroup Updating
StandardProcessGroup[identifier=420d499d-2b1d-3af8-6cc3-2722817f4d49] to
VersionedFlowSnapshot[flowId=e799c617-0dd9-4685-b471-2987b19ad121,
flowName=test1, version=2, comments=null]; there are 0 differences to take into
account:
2018-02-09 21:45:40,903 INFO [Version Control Update Thread-1]
org.apache.nifi.web.api.VersionsResource Successfully updated flow; re-enabling
0 Controller Services
2018-02-09 21:45:40,903 INFO [Version Control Update Thread-1]
org.apache.nifi.web.api.VersionsResource Restarting 0 Processors
2018-02-09 21:45:40,903 ERROR [Version Control Update Thread-1]
org.apache.nifi.web.api.VersionsResource Failed to update flow to new version
java.lang.NullPointerException: null
at
org.apache.nifi.groups.StandardProcessGroup.updateConnection(StandardProcessGroup.java:3784)
at
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3656)
at
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3488)
at
org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3295)
at
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:279)
at
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
at
org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:258)
at sun.reflect.GeneratedMethodAccessor644.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
at
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
at
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$3bf4fdb8.updateProcessGroupFlow()
at
org.apache.nifi.web.StandardNiFiServiceFacade$13.update(StandardNiFiServiceFacade.java:4304)
at
org.apache.nifi.web.revision.NaiveRevisionManager.updateRevision(NaiveRevisionManager.java:120)
at
org.apache.nifi.web.StandardNiFiServiceFacade.updateProcessGroupContents(StandardNiFiServiceFacade.java:4300)
at
org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at
[jira] [Created] (NIFI-4859) Swagger Spec VersionControlInformationDTO missing SYNC_FAILURE state
Daniel Chaffelson created NIFI-4859:
---
Summary: Swagger Spec VersionControlInformationDTO missing
SYNC_FAILURE state
Key: NIFI-4859
URL: https://issues.apache.org/jira/browse/NIFI-4859
Project: Apache NiFi
Issue Type: Bug
Components: Flow Versioning
Affects Versions: 1.5.0, 1.6.0
Reporter: Daniel Chaffelson
It is possible to get a Versioned Process Group into a SYNC_FAILURE state, but
this is not an allowable state in the code generated from the swagger.json
This prevents versioned objects from being manipulated via the API in some use
cases.
{noformat}
@state.setter
def state(self, state):
"""
Sets the state of this VersionControlInformationDTO.
The current state of the Process Group, as it relates to the Versioned Flow
:param state: The state of this VersionControlInformationDTO.
:type: str
"""
allowed_values = ["LOCALLY_MODIFIED_DESCENDANT", "LOCALLY_MODIFIED", "STALE",
"LOCALLY_MODIFIED_AND_STALE", "UP_TO_DATE"]
if state not in allowed_values:
raise ValueError(
"Invalid value for `state` ({0}), must be one of {1}"
> .format(state, allowed_values)
)
E ValueError: Invalid value for `state` (SYNC_FAILURE), must be one of
['LOCALLY_MODIFIED_DESCENDANT', 'LOCALLY_MODIFIED', 'STALE',
'LOCALLY_MODIFIED_AND_STALE', 'UP_TO_DATE']
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFIREG-135) BucketFlowsApi.create_flow_version returns an incorrect version count
[ https://issues.apache.org/jira/browse/NIFIREG-135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349209#comment-16349209 ] Daniel Chaffelson commented on NIFIREG-135: --- I was hoping it'd be in the swagger spec so we can quickly fix it, but as it's in the backend I'll mark the test as pending NIFIREG-135 and move on. Thanks [~kdoran] > BucketFlowsApi.create_flow_version returns an incorrect version count > - > > Key: NIFIREG-135 > URL: https://issues.apache.org/jira/browse/NIFIREG-135 > Project: NiFi Registry > Issue Type: Bug >Affects Versions: 0.1.0, 0.2.0 >Reporter: Daniel Chaffelson >Assignee: Kevin Doran >Priority: Major > > Using the Python version of the Swagger defined client at: > [https://github.com/Chaffelson/nipyapi/blob/master/nipyapi/registry/apis/bucket_flows_api.py] > The VersionedFlow sub-object in the VersionedFlowSnapshot returned by the > create_flow_version call does not correctly increment the version_count > variable. > If the Flow is subsequently retrieved using get_flow_version, the > version_count is correct. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFIREG-135) BucketFlowsApi.create_flow_version returns an incorrect version count
Daniel Chaffelson created NIFIREG-135: - Summary: BucketFlowsApi.create_flow_version returns an incorrect version count Key: NIFIREG-135 URL: https://issues.apache.org/jira/browse/NIFIREG-135 Project: NiFi Registry Issue Type: Bug Affects Versions: 0.1.0, 0.2.0 Reporter: Daniel Chaffelson Using the Python version of the Swagger defined client at: [https://github.com/Chaffelson/nipyapi/blob/master/nipyapi/registry/apis/bucket_flows_api.py] The VersionedFlow sub-object in the VersionedFlowSnapshot returned by the create_flow_version call does not correctly increment the version_count variable. If the Flow is subsequently retrieved using get_flow_version, the version_count is correct. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFIREG-132) Various REST API endpoints missing swagger annotations for body parameters.
[
https://issues.apache.org/jira/browse/NIFIREG-132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344655#comment-16344655
]
Daniel Chaffelson commented on NIFIREG-132:
---
Looking good, Python code output available at :
[https://github.com/Chaffelson/nipyapi/blob/master/nipyapi/registry/apis/bucket_flows_api.py]
> Various REST API endpoints missing swagger annotations for body parameters.
> ---
>
> Key: NIFIREG-132
> URL: https://issues.apache.org/jira/browse/NIFIREG-132
> Project: NiFi Registry
> Issue Type: Bug
>Affects Versions: 0.1.0
>Reporter: Kevin Doran
>Assignee: Kevin Doran
>Priority: Major
> Fix For: 0.2.0
>
>
> The following endpoints are missing the swagger {{@ApiParam}} annotation for
> the body parameter, which creates incorrect documentation/spec in the
> {{swagger.json}} file in the build output:
> * create flow
> * update flow
> * create flow version
> Thanks to [~chaffelson] for help identifying and testing this issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (NIFI-3612) Add support for Parquet to Nifi-Registry-Bundle
[ https://issues.apache.org/jira/browse/NIFI-3612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Chaffelson closed NIFI-3612. --- This requirement has been resolved through the Parquet support in the Record based processors > Add support for Parquet to Nifi-Registry-Bundle > --- > > Key: NIFI-3612 > URL: https://issues.apache.org/jira/browse/NIFI-3612 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Daniel Chaffelson >Assignee: Oleg Zhurakousky >Priority: Major > Fix For: 1.2.0 > > > This bundle could potentially be extended to include a Parquet transform by > leveraging the Apache 2.0 licensed parquet-mr/avro libraries: > https://github.com/apache/parquet-mr/tree/master/parquet-avro > This would provide coverage of this popular format to complement the ORC > support in the Hive Bundle and the other schema-dependent formats already in > this bundle. > Existing NiFi Parquet support in the kite bundle can only write to a > non-kerberised Kite Dataset, which prevents usage on secured environments or > writing to a FlowFile. > As the main competitor to ORC, providing more generic Parquet Transform > support will greatly widen the pool of potential NiFi adopters, particularly > in the Spark community. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-4824) Dockerhub Image - allow user to specific host ports on startup
Daniel Chaffelson created NIFI-4824: --- Summary: Dockerhub Image - allow user to specific host ports on startup Key: NIFI-4824 URL: https://issues.apache.org/jira/browse/NIFI-4824 Project: Apache NiFi Issue Type: Improvement Components: Docker Affects Versions: 1.5.0, 1.4.0, 1.3.0, 1.2.0, 2.0.0, 1.6.0 Reporter: Daniel Chaffelson As a user of the official apache/nifi Docker image, I would like to be able to easily pass in the ports I want NiFi to start on. This is particularly important since the 1.5.0 release, where remapping ports outside the container cause NiFi to throw a security error. This will be particularly useful in NiFI-1.5.0 and newer, where users are often running multiple instances of the same version in order to test Flow Versioning and other new features around the NiFi-Registry project. This can be done by slightly modifying the startup scripts included in the package to use an environment variable if available, or a default if it is not provided. It is then very easy to pass an environment variable to the Docker image on startup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFIREG-129) swagger spec missing snapshot_metadata param for bucket_flows_api
[ https://issues.apache.org/jira/browse/NIFIREG-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339253#comment-16339253 ] Daniel Chaffelson commented on NIFIREG-129: --- The generated client is available here for you to look at: https://github.com/Chaffelson/nipyapi/blob/master/nipyapi/registry/apis/bucket_flows_api.py > swagger spec missing snapshot_metadata param for bucket_flows_api > - > > Key: NIFIREG-129 > URL: https://issues.apache.org/jira/browse/NIFIREG-129 > Project: NiFi Registry > Issue Type: Bug >Affects Versions: 0.1.0, 0.2.0 >Reporter: Daniel Chaffelson >Assignee: Kevin Doran >Priority: Major > > When calling methods to retrieve versioned flows from the bucket_flows_api ( > get_flow, get_flow_version, get_latest_flow_version, etc) the spec calls for > a bucket_id and a flow_id, however the endpoint returns the error "Invalid > value for `snapshot_metadata`, must not be `None`" > Presumably there is a parameter missing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFIREG-129) swagger spec missing snapshot_metadata param for bucket_flows_api
[
https://issues.apache.org/jira/browse/NIFIREG-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339249#comment-16339249
]
Daniel Chaffelson commented on NIFIREG-129:
---
Here's the Python output - there's nothing in the nifi-registry-app.log because
it's not getting out of the client.
{noformat}
def get_latest_flow_ver(bucket_id, flow_id):
try:
return registry.BucketFlowsApi().get_latest_flow_version(
bucket_id, flow_id
)
except ApiExceptionR as e:
raise ValueError(e.body)
>>versioning.get_latest_flow_ver(bucket.identifier, flow.identifier)
Traceback (most recent call last):
File "", line 1, in
File "/Users/dchaffey/PycharmProjects/Nipyapi/nipyapi/versioning.py", line 230,
in get_latest_flow_ver
bucket_id, flow_id
File
"/Users/dchaffey/PycharmProjects/Nipyapi/nipyapi/registry/apis/bucket_flows_api.py",
line 853, in get_latest_flow_version
(data) = self.get_latest_flow_version_with_http_info(bucket_id, flow_id,
**kwargs)
File
"/Users/dchaffey/PycharmProjects/Nipyapi/nipyapi/registry/apis/bucket_flows_api.py",
line 940, in get_latest_flow_version_with_http_info
collection_formats=collection_formats)
File "/Users/dchaffey/PycharmProjects/Nipyapi/nipyapi/registry/api_client.py",
line 326, in call_api
_return_http_data_only, collection_formats, _preload_content, _request_timeout)
File "/Users/dchaffey/PycharmProjects/Nipyapi/nipyapi/registry/api_client.py",
line 161, in __call_api
return_data = self.deserialize(response_data, response_type)
File "/Users/dchaffey/PycharmProjects/Nipyapi/nipyapi/registry/api_client.py",
line 239, in deserialize
return self.__deserialize(data, response_type)
File "/Users/dchaffey/PycharmProjects/Nipyapi/nipyapi/registry/api_client.py",
line 279, in __deserialize
return self.__deserialize_model(data, klass)
File "/Users/dchaffey/PycharmProjects/Nipyapi/nipyapi/registry/api_client.py",
line 629, in __deserialize_model
kwargs[attr] = self.__deserialize(value, attr_type)
File "/Users/dchaffey/PycharmProjects/Nipyapi/nipyapi/registry/api_client.py",
line 279, in __deserialize
return self.__deserialize_model(data, klass)
File "/Users/dchaffey/PycharmProjects/Nipyapi/nipyapi/registry/api_client.py",
line 631, in __deserialize_model
instance = klass(**kwargs)
File
"/Users/dchaffey/PycharmProjects/Nipyapi/nipyapi/registry/models/versioned_flow_snapshot.py",
line 60, in __init__
self.snapshot_metadata = snapshot_metadata
File
"/Users/dchaffey/PycharmProjects/Nipyapi/nipyapi/registry/models/versioned_flow_snapshot.py",
line 90, in snapshot_metadata
raise ValueError("Invalid value for `snapshot_metadata`, must not be `None`")
ValueError: Invalid value for `snapshot_metadata`, must not be `None`{noformat}
> swagger spec missing snapshot_metadata param for bucket_flows_api
> -
>
> Key: NIFIREG-129
> URL: https://issues.apache.org/jira/browse/NIFIREG-129
> Project: NiFi Registry
> Issue Type: Bug
>Affects Versions: 0.1.0, 0.2.0
>Reporter: Daniel Chaffelson
>Assignee: Kevin Doran
>Priority: Major
>
> When calling methods to retrieve versioned flows from the bucket_flows_api (
> get_flow, get_flow_version, get_latest_flow_version, etc) the spec calls for
> a bucket_id and a flow_id, however the endpoint returns the error "Invalid
> value for `snapshot_metadata`, must not be `None`"
> Presumably there is a parameter missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFIREG-129) swagger spec missing snapshot_metadata param for bucket_flows_api
Daniel Chaffelson created NIFIREG-129: - Summary: swagger spec missing snapshot_metadata param for bucket_flows_api Key: NIFIREG-129 URL: https://issues.apache.org/jira/browse/NIFIREG-129 Project: NiFi Registry Issue Type: Bug Affects Versions: 0.1.0, 0.2.0 Reporter: Daniel Chaffelson When calling methods to retrieve versioned flows from the bucket_flows_api ( get_flow, get_flow_version, get_latest_flow_version, etc) the spec calls for a bucket_id and a flow_id, however the endpoint returns the error "Invalid value for `snapshot_metadata`, must not be `None`" Presumably there is a parameter missing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFIREG-128) Swagger spec missing body param for create/update bucket
[
https://issues.apache.org/jira/browse/NIFIREG-128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338877#comment-16338877
]
Daniel Chaffelson commented on NIFIREG-128:
---
I have tested the proposed changes and it looks good in Python.
Client generated from the new spec is available [NiPyAPI on
Github|https://github.com/Chaffelson/nipyapi/tree/master/nipyapi/registry]
> Swagger spec missing body param for create/update bucket
>
>
> Key: NIFIREG-128
> URL: https://issues.apache.org/jira/browse/NIFIREG-128
> Project: NiFi Registry
> Issue Type: Bug
>Affects Versions: 0.1.0
>Reporter: Daniel Chaffelson
>Assignee: Kevin Doran
>Priority: Major
> Fix For: 0.2.0
>
>
> The swagger definition doesn't allow the create_bucket call to specify a
> name, which is required to create a bucket
>
> {code:java}
> "/buckets" : {
> "post" : {
> "tags" : [ "buckets" ],
> "summary" : "Creates a bucket",
> "description" : "",
> "operationId" : "createBucket",
> "consumes" : [ "application/json" ],
> "produces" : [ "application/json" ],
> "responses" : {
> "200" : {
> "description" : "successful operation",
> "schema" : {
> "$ref" : "#/definitions/bucket"
> }
> },
> "400" : {
> "description" : "NiFi Registry was unable to complete the request because it
> was invalid. The request should not be retried without modification."
> },
> "401" : {
> "description" : "Client could not be authenticated."
> },
> "403" : {
> "description" : "Client is not authorized to make this request."
> }
> }
> }
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFIREG-128) swagger_client generated create_bucket requires but does not accept a 'name' parameter
Daniel Chaffelson created NIFIREG-128:
-
Summary: swagger_client generated create_bucket requires but does
not accept a 'name' parameter
Key: NIFIREG-128
URL: https://issues.apache.org/jira/browse/NIFIREG-128
Project: NiFi Registry
Issue Type: Bug
Affects Versions: 0.1.0, 0.2.0
Reporter: Daniel Chaffelson
The swagger definition doesn't allow the create_bucket call to specify a name,
which is required to create a bucket
{code:java}
"/buckets" : {
"post" : {
"tags" : [ "buckets" ],
"summary" : "Creates a bucket",
"description" : "",
"operationId" : "createBucket",
"consumes" : [ "application/json" ],
"produces" : [ "application/json" ],
"responses" : {
"200" : {
"description" : "successful operation",
"schema" : {
"$ref" : "#/definitions/bucket"
}
},
"400" : {
"description" : "NiFi Registry was unable to complete the request because it
was invalid. The request should not be retried without modification."
},
"401" : {
"description" : "Client could not be authenticated."
},
"403" : {
"description" : "Client is not authorized to make this request."
}
}
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-4812) delete_registry_client call only accepts ID, but endpoint also expects Revision
[
https://issues.apache.org/jira/browse/NIFI-4812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Chaffelson updated NIFI-4812:
Description:
Using the Swagger Generated Python Client.
When the delete_registry_client command is called from the ControllerApi(), the
arguments expected by NiFi (id, version) and the arguments allowed by swagger
(id only) do not match, resulting in a failure.
{code:java}
ControllerApi().delete_registry_client(test_client.id)
HTTP response body: The revision must be specified.
ControllerApi().delete_registry_client(test_client.id, test_client.revision)
TypeError: delete_registry_client() takes exactly 2 arguments (3 given)
{code}
was:
Using the Swagger Generated Python Client.
When the delete_registry_client command is called from the ControllerApi(), the
arguments expected by NiFi (id, revision) and the arguments allowed by swagger
(id only) do not match, resulting in a failure.
{code:java}
ControllerApi().delete_registry_client(test_client.id)
HTTP response body: The revision must be specified.
ControllerApi().delete_registry_client(test_client.id, test_client.revision)
TypeError: delete_registry_client() takes exactly 2 arguments (3 given)
{code}
> delete_registry_client call only accepts ID, but endpoint also expects
> Revision
> ---
>
> Key: NIFI-4812
> URL: https://issues.apache.org/jira/browse/NIFI-4812
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Flow Versioning
>Affects Versions: 1.5.0, 1.6.0
>Reporter: Daniel Chaffelson
>Priority: Minor
>
> Using the Swagger Generated Python Client.
> When the delete_registry_client command is called from the ControllerApi(),
> the arguments expected by NiFi (id, version) and the arguments allowed by
> swagger (id only) do not match, resulting in a failure.
> {code:java}
> ControllerApi().delete_registry_client(test_client.id)
> HTTP response body: The revision must be specified.
> ControllerApi().delete_registry_client(test_client.id, test_client.revision)
> TypeError: delete_registry_client() takes exactly 2 arguments (3 given)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4812) delete_registry_client call only accepts ID, but endpoint also expects Revision
[
https://issues.apache.org/jira/browse/NIFI-4812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16337449#comment-16337449
]
Daniel Chaffelson commented on NIFI-4812:
-
It works despite this if you deconstruct the json message via browser developer
tools and present the exact format expected:
{code:java}
ControllerApi().delete_registry_client(
id=client.id,
version=client.revision.version
)
{code}
However this is completely counter to the message provided by the errors.
> delete_registry_client call only accepts ID, but endpoint also expects
> Revision
> ---
>
> Key: NIFI-4812
> URL: https://issues.apache.org/jira/browse/NIFI-4812
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Flow Versioning
>Affects Versions: 1.5.0, 1.6.0
>Reporter: Daniel Chaffelson
>Priority: Minor
>
> Using the Swagger Generated Python Client.
> When the delete_registry_client command is called from the ControllerApi(),
> the arguments expected by NiFi (id, revision) and the arguments allowed by
> swagger (id only) do not match, resulting in a failure.
> {code:java}
> ControllerApi().delete_registry_client(test_client.id)
> HTTP response body: The revision must be specified.
> ControllerApi().delete_registry_client(test_client.id, test_client.revision)
> TypeError: delete_registry_client() takes exactly 2 arguments (3 given)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-4812) delete_registry_client call only accepts ID, but endpoint also expects Revision
[
https://issues.apache.org/jira/browse/NIFI-4812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Chaffelson updated NIFI-4812:
Priority: Minor (was: Major)
> delete_registry_client call only accepts ID, but endpoint also expects
> Revision
> ---
>
> Key: NIFI-4812
> URL: https://issues.apache.org/jira/browse/NIFI-4812
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Flow Versioning
>Affects Versions: 1.5.0, 1.6.0
>Reporter: Daniel Chaffelson
>Priority: Minor
>
> Using the Swagger Generated Python Client.
> When the delete_registry_client command is called from the ControllerApi(),
> the arguments expected by NiFi (id, revision) and the arguments allowed by
> swagger (id only) do not match, resulting in a failure.
> {code:java}
> ControllerApi().delete_registry_client(test_client.id)
> HTTP response body: The revision must be specified.
> ControllerApi().delete_registry_client(test_client.id, test_client.revision)
> TypeError: delete_registry_client() takes exactly 2 arguments (3 given)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFI-4812) delete_registry_client call only accepts ID, but endpoint also expects Revision
Daniel Chaffelson created NIFI-4812:
---
Summary: delete_registry_client call only accepts ID, but endpoint
also expects Revision
Key: NIFI-4812
URL: https://issues.apache.org/jira/browse/NIFI-4812
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework, Flow Versioning
Affects Versions: 1.5.0, 1.6.0
Reporter: Daniel Chaffelson
Using the Swagger Generated Python Client.
When the delete_registry_client command is called from the ControllerApi(), the
arguments expected by NiFi (id, revision) and the arguments allowed by swagger
(id only) do not match, resulting in a failure.
{code:java}
ControllerApi().delete_registry_client(test_client.id)
HTTP response body: The revision must be specified.
ControllerApi().delete_registry_client(test_client.id, test_client.revision)
TypeError: delete_registry_client() takes exactly 2 arguments (3 given)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4801) Rest-api swagger definition produces non-functional template import in Python
[ https://issues.apache.org/jira/browse/NIFI-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335986#comment-16335986 ] Daniel Chaffelson commented on NIFI-4801: - [~kdoran] [~mcgilman] Changes tested and I confirm it now works with NiPyAPI as per previous versions - I have integrated these modifications and pushed them to NiPyApi if you want to check for yourselves. > Rest-api swagger definition produces non-functional template import in Python > - > > Key: NIFI-4801 > URL: https://issues.apache.org/jira/browse/NIFI-4801 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0 > Environment: Python 2.7/3.6 >Reporter: Daniel Chaffelson >Assignee: Kevin Doran >Priority: Major > Fix For: 1.6.0 > > Attachments: corrected-swagger.json > > > The swagger.json produced when compiling NiFi-1.5.0 results in a > ProcessgroupsApi().upload_template function that only accepts the id of the > Process Group to receive the template, and no option to specify the template > itself. > It would appear that the underlying API call expects the template to be the > body of the request, but the produced function does not allow it to be > specified. This is changed from NiFi-1.2.0 where a 'template' keyword > argument was included. > It may also be related to how the TemplatesApi().export_template function > used to produce a TemplateDTO and now produces a string. > I am unsure in which version since 1.2.0 this changed, it may not > specifically be just 1.5.0 code. > An example of the procedurally generated code can be found at: > [https://github.com/Chaffelson/nifi-python-swagger-client/blob/master/swagger_client/apis/processgroups_api.py] > And documentation at: > [http://nifi-python-swagger-client.readthedocs.io/en/latest/ProcessgroupsApi/#upload_template] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4801) Rest-api swagger definition produces non-functional template import in Python
[ https://issues.apache.org/jira/browse/NIFI-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335859#comment-16335859 ] Daniel Chaffelson commented on NIFI-4801: - [~mcgilman] Fair enough, but can we return to the 'formData' approach from previous versions instead of moving to a 'body' approach? I'm a fan of consistency. > Rest-api swagger definition produces non-functional template import in Python > - > > Key: NIFI-4801 > URL: https://issues.apache.org/jira/browse/NIFI-4801 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0, 1.6.0 > Environment: Python 2.7/3.6 >Reporter: Daniel Chaffelson >Assignee: Kevin Doran >Priority: Major > > The swagger.json produced when compiling NiFi-1.5.0 results in a > ProcessgroupsApi().upload_template function that only accepts the id of the > Process Group to receive the template, and no option to specify the template > itself. > It would appear that the underlying API call expects the template to be the > body of the request, but the produced function does not allow it to be > specified. This is changed from NiFi-1.2.0 where a 'template' keyword > argument was included. > It may also be related to how the TemplatesApi().export_template function > used to produce a TemplateDTO and now produces a string. > I am unsure in which version since 1.2.0 this changed, it may not > specifically be just 1.5.0 code. > An example of the procedurally generated code can be found at: > [https://github.com/Chaffelson/nifi-python-swagger-client/blob/master/swagger_client/apis/processgroups_api.py] > And documentation at: > [http://nifi-python-swagger-client.readthedocs.io/en/latest/ProcessgroupsApi/#upload_template] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4801) Rest-api swagger definition produces non-functional template import in Python
[
https://issues.apache.org/jira/browse/NIFI-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335387#comment-16335387
]
Daniel Chaffelson commented on NIFI-4801:
-
In 1.2/1.3/1.4 we had:
{code:java}
"parameters" : [ {
"name" : "id",
"in" : "path",
"description" : "The process group id.",
"required" : true,
"type" : "string"
}, {
"name" : "template",
"in" : "formData",
"required" : false,
"type" : "file"
} ]
{code}
Would it not make more sense to revert to the previous method than introducing
a new method? Unless of course this is to bring it into consistency with other
methods in the platform, it would make more sense to me to revert.
I can easily test the resulting swagger.json, just let me know which version is
preferred or provide me a sample compile.
> Rest-api swagger definition produces non-functional template import in Python
> -
>
> Key: NIFI-4801
> URL: https://issues.apache.org/jira/browse/NIFI-4801
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0
> Environment: Python 2.7/3.6
>Reporter: Daniel Chaffelson
>Assignee: Kevin Doran
>Priority: Major
>
> The swagger.json produced when compiling NiFi-1.5.0 results in a
> ProcessgroupsApi().upload_template function that only accepts the id of the
> Process Group to receive the template, and no option to specify the template
> itself.
> It would appear that the underlying API call expects the template to be the
> body of the request, but the produced function does not allow it to be
> specified. This is changed from NiFi-1.2.0 where a 'template' keyword
> argument was included.
> It may also be related to how the TemplatesApi().export_template function
> used to produce a TemplateDTO and now produces a string.
> I am unsure in which version since 1.2.0 this changed, it may not
> specifically be just 1.5.0 code.
> An example of the procedurally generated code can be found at:
> [https://github.com/Chaffelson/nifi-python-swagger-client/blob/master/swagger_client/apis/processgroups_api.py]
> And documentation at:
> [http://nifi-python-swagger-client.readthedocs.io/en/latest/ProcessgroupsApi/#upload_template]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-4801) Rest-api swagger definition produces non-functional template import in Python
[ https://issues.apache.org/jira/browse/NIFI-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Chaffelson updated NIFI-4801: Priority: Major (was: Minor) > Rest-api swagger definition produces non-functional template import in Python > - > > Key: NIFI-4801 > URL: https://issues.apache.org/jira/browse/NIFI-4801 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0, 1.6.0 > Environment: Python 2.7/3.6 >Reporter: Daniel Chaffelson >Assignee: Kevin Doran >Priority: Major > > The swagger.json produced when compiling NiFi-1.5.0 results in a > ProcessgroupsApi().upload_template function that only accepts the id of the > Process Group to receive the template, and no option to specify the template > itself. > It would appear that the underlying API call expects the template to be the > body of the request, but the produced function does not allow it to be > specified. This is changed from NiFi-1.2.0 where a 'template' keyword > argument was included. > It may also be related to how the TemplatesApi().export_template function > used to produce a TemplateDTO and now produces a string. > I am unsure in which version since 1.2.0 this changed, it may not > specifically be just 1.5.0 code. > An example of the procedurally generated code can be found at: > [https://github.com/Chaffelson/nifi-python-swagger-client/blob/master/swagger_client/apis/processgroups_api.py] > And documentation at: > [http://nifi-python-swagger-client.readthedocs.io/en/latest/ProcessgroupsApi/#upload_template] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4801) Rest-api swagger definition produces non-functional template import in Python
[ https://issues.apache.org/jira/browse/NIFI-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Chaffelson updated NIFI-4801: Affects Version/s: 1.6.0 > Rest-api swagger definition produces non-functional template import in Python > - > > Key: NIFI-4801 > URL: https://issues.apache.org/jira/browse/NIFI-4801 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0, 1.6.0 > Environment: Python 2.7/3.6 >Reporter: Daniel Chaffelson >Assignee: Kevin Doran >Priority: Major > > The swagger.json produced when compiling NiFi-1.5.0 results in a > ProcessgroupsApi().upload_template function that only accepts the id of the > Process Group to receive the template, and no option to specify the template > itself. > It would appear that the underlying API call expects the template to be the > body of the request, but the produced function does not allow it to be > specified. This is changed from NiFi-1.2.0 where a 'template' keyword > argument was included. > It may also be related to how the TemplatesApi().export_template function > used to produce a TemplateDTO and now produces a string. > I am unsure in which version since 1.2.0 this changed, it may not > specifically be just 1.5.0 code. > An example of the procedurally generated code can be found at: > [https://github.com/Chaffelson/nifi-python-swagger-client/blob/master/swagger_client/apis/processgroups_api.py] > And documentation at: > [http://nifi-python-swagger-client.readthedocs.io/en/latest/ProcessgroupsApi/#upload_template] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4801) Rest-api swagger definition produces non-functional template import in Python
[ https://issues.apache.org/jira/browse/NIFI-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334583#comment-16334583 ] Daniel Chaffelson commented on NIFI-4801: - [~mcgilman] Yeah this duplicates that issue, though perhaps with a little more detail and test code/documentation to look at. Up to you which to move forward. I would also suggest that you look at the way Export Template works to make sure it's producing a consistent experience perhaps? > Rest-api swagger definition produces non-functional template import in Python > - > > Key: NIFI-4801 > URL: https://issues.apache.org/jira/browse/NIFI-4801 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0 > Environment: Python 2.7/3.6 >Reporter: Daniel Chaffelson >Assignee: Kevin Doran >Priority: Minor > > The swagger.json produced when compiling NiFi-1.5.0 results in a > ProcessgroupsApi().upload_template function that only accepts the id of the > Process Group to receive the template, and no option to specify the template > itself. > It would appear that the underlying API call expects the template to be the > body of the request, but the produced function does not allow it to be > specified. This is changed from NiFi-1.2.0 where a 'template' keyword > argument was included. > It may also be related to how the TemplatesApi().export_template function > used to produce a TemplateDTO and now produces a string. > I am unsure in which version since 1.2.0 this changed, it may not > specifically be just 1.5.0 code. > An example of the procedurally generated code can be found at: > [https://github.com/Chaffelson/nifi-python-swagger-client/blob/master/swagger_client/apis/processgroups_api.py] > And documentation at: > [http://nifi-python-swagger-client.readthedocs.io/en/latest/ProcessgroupsApi/#upload_template] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4801) Rest-api swagger definition produces non-functional template import in Python
[ https://issues.apache.org/jira/browse/NIFI-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Chaffelson updated NIFI-4801: Affects Version/s: (was: 1.2.0) > Rest-api swagger definition produces non-functional template import in Python > - > > Key: NIFI-4801 > URL: https://issues.apache.org/jira/browse/NIFI-4801 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0 > Environment: Python 2.7/3.6 >Reporter: Daniel Chaffelson >Priority: Minor > > The swagger.json produced when compiling NiFi-1.5.0 results in a > ProcessgroupsApi().upload_template function that only accepts the id of the > Process Group to receive the template, and no option to specify the template > itself. > It would appear that the underlying API call expects the template to be the > body of the request, but the produced function does not allow it to be > specified. This is changed from NiFi-1.2.0 where a 'template' keyword > argument was included. > It may also be related to how the TemplatesApi().export_template function > used to produce a TemplateDTO and now produces a string. > I am unsure in which version since 1.2.0 this changed, it may not > specifically be just 1.5.0 code. > An example of the procedurally generated code can be found at: > [https://github.com/Chaffelson/nifi-python-swagger-client/blob/master/swagger_client/apis/processgroups_api.py] > And documentation at: > [http://nifi-python-swagger-client.readthedocs.io/en/latest/ProcessgroupsApi/#upload_template] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-4801) Rest-api swagger definition produces non-functional template import in Python
Daniel Chaffelson created NIFI-4801: --- Summary: Rest-api swagger definition produces non-functional template import in Python Key: NIFI-4801 URL: https://issues.apache.org/jira/browse/NIFI-4801 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.5.0, 1.2.0 Environment: Python 2.7/3.6 Reporter: Daniel Chaffelson The swagger.json produced when compiling NiFi-1.5.0 results in a ProcessgroupsApi().upload_template function that only accepts the id of the Process Group to receive the template, and no option to specify the template itself. It would appear that the underlying API call expects the template to be the body of the request, but the produced function does not allow it to be specified. This is changed from NiFi-1.2.0 where a 'template' keyword argument was included. It may also be related to how the TemplatesApi().export_template function used to produce a TemplateDTO and now produces a string. I am unsure in which version since 1.2.0 this changed, it may not specifically be just 1.5.0 code. An example of the procedurally generated code can be found at: [https://github.com/Chaffelson/nifi-python-swagger-client/blob/master/swagger_client/apis/processgroups_api.py] And documentation at: [http://nifi-python-swagger-client.readthedocs.io/en/latest/ProcessgroupsApi/#upload_template] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFIREG-120) Basic Docker Image
[ https://issues.apache.org/jira/browse/NIFIREG-120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Chaffelson updated NIFIREG-120: -- Fix Version/s: 0.1.1 > Basic Docker Image > -- > > Key: NIFIREG-120 > URL: https://issues.apache.org/jira/browse/NIFIREG-120 > Project: NiFi Registry > Issue Type: Improvement >Affects Versions: 0.1.0 >Reporter: Daniel Chaffelson >Priority: Minor > Fix For: 0.1.1 > > > It would be convenient if NiFi Registry had an integrated Docker image ready > for uploading to Dockerhub, similar to the main NiFi Project, for ease of > integration testing. > This could probably be ported, with some changes, from the same approach used > in the main NiFi project for continuity. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFIREG-120) Basic Docker Image
[ https://issues.apache.org/jira/browse/NIFIREG-120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Chaffelson updated NIFIREG-120: -- Fix Version/s: (was: 0.1.1) > Basic Docker Image > -- > > Key: NIFIREG-120 > URL: https://issues.apache.org/jira/browse/NIFIREG-120 > Project: NiFi Registry > Issue Type: Improvement >Affects Versions: 0.1.0 >Reporter: Daniel Chaffelson >Priority: Minor > > It would be convenient if NiFi Registry had an integrated Docker image ready > for uploading to Dockerhub, similar to the main NiFi Project, for ease of > integration testing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFIREG-120) Basic Docker Image
[ https://issues.apache.org/jira/browse/NIFIREG-120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Chaffelson updated NIFIREG-120: -- Description: It would be convenient if NiFi Registry had an integrated Docker image ready for uploading to Dockerhub, similar to the main NiFi Project, for ease of integration testing. This could probably be ported, with some changes, from the same approach used in the main NiFi project for continuity. was:It would be convenient if NiFi Registry had an integrated Docker image ready for uploading to Dockerhub, similar to the main NiFi Project, for ease of integration testing > Basic Docker Image > -- > > Key: NIFIREG-120 > URL: https://issues.apache.org/jira/browse/NIFIREG-120 > Project: NiFi Registry > Issue Type: Improvement >Affects Versions: 0.1.0 >Reporter: Daniel Chaffelson >Priority: Minor > > It would be convenient if NiFi Registry had an integrated Docker image ready > for uploading to Dockerhub, similar to the main NiFi Project, for ease of > integration testing. > This could probably be ported, with some changes, from the same approach used > in the main NiFi project for continuity. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFIREG-120) Basic Docker Image
[ https://issues.apache.org/jira/browse/NIFIREG-120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Chaffelson updated NIFIREG-120: -- Fix Version/s: 0.1.1 > Basic Docker Image > -- > > Key: NIFIREG-120 > URL: https://issues.apache.org/jira/browse/NIFIREG-120 > Project: NiFi Registry > Issue Type: Improvement >Affects Versions: 0.1.0 >Reporter: Daniel Chaffelson >Priority: Minor > Fix For: 0.1.1 > > > It would be convenient if NiFi Registry had an integrated Docker image ready > for uploading to Dockerhub, similar to the main NiFi Project, for ease of > integration testing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFIREG-120) Basic Docker Image
[ https://issues.apache.org/jira/browse/NIFIREG-120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Chaffelson updated NIFIREG-120: -- Affects Version/s: 0.1.0 > Basic Docker Image > -- > > Key: NIFIREG-120 > URL: https://issues.apache.org/jira/browse/NIFIREG-120 > Project: NiFi Registry > Issue Type: Improvement >Affects Versions: 0.1.0 >Reporter: Daniel Chaffelson >Priority: Minor > Fix For: 0.1.1 > > > It would be convenient if NiFi Registry had an integrated Docker image ready > for uploading to Dockerhub, similar to the main NiFi Project, for ease of > integration testing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFIREG-120) Basic Docker Image
[ https://issues.apache.org/jira/browse/NIFIREG-120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Chaffelson updated NIFIREG-120: -- Description: It would be convenient if NiFi Registry had an integrated Docker image ready for uploading to Dockerhub, similar to the main NiFi Project, for ease of integration testing > Basic Docker Image > -- > > Key: NIFIREG-120 > URL: https://issues.apache.org/jira/browse/NIFIREG-120 > Project: NiFi Registry > Issue Type: Improvement >Affects Versions: 0.1.0 >Reporter: Daniel Chaffelson >Priority: Major > Fix For: 0.1.1 > > > It would be convenient if NiFi Registry had an integrated Docker image ready > for uploading to Dockerhub, similar to the main NiFi Project, for ease of > integration testing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFIREG-120) Basic Docker Image
[ https://issues.apache.org/jira/browse/NIFIREG-120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Chaffelson updated NIFIREG-120: -- Priority: Minor (was: Major) > Basic Docker Image > -- > > Key: NIFIREG-120 > URL: https://issues.apache.org/jira/browse/NIFIREG-120 > Project: NiFi Registry > Issue Type: Improvement >Affects Versions: 0.1.0 >Reporter: Daniel Chaffelson >Priority: Minor > Fix For: 0.1.1 > > > It would be convenient if NiFi Registry had an integrated Docker image ready > for uploading to Dockerhub, similar to the main NiFi Project, for ease of > integration testing -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFIREG-120) Basic Docker Image
Daniel Chaffelson created NIFIREG-120: - Summary: Basic Docker Image Key: NIFIREG-120 URL: https://issues.apache.org/jira/browse/NIFIREG-120 Project: NiFi Registry Issue Type: Improvement Reporter: Daniel Chaffelson -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4761) Allow whitelisting expected Host values
[ https://issues.apache.org/jira/browse/NIFI-4761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328273#comment-16328273 ] Daniel Chaffelson commented on NIFI-4761: - If running multiple NiFi instances in local Docker containers, it is typical to map the default port (8080) to something else (e.g. 1), this change breaks that working out of the box. Tested and working fine in NiFi 1.2-1.4, 'broken' in 1.5 > Allow whitelisting expected Host values > --- > > Key: NIFI-4761 > URL: https://issues.apache.org/jira/browse/NIFI-4761 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Matt Gilman >Assignee: Andy LoPresto >Priority: Major > > NiFi has been updated to only accept requests where the Host header contains > an expected value. Currently, the expected values are driven by the .host > properties in nifi.properties. When running behind a proxy, the value may be > the proxy host if the headers simply pass through. In this scenario, we > should offer the ability to whitelist values in case updating the proxy > configuration isn't possible. > Also, the proxy documentation in the admin guide should be updated to include > details regarding the Host name whitelisting. Also, should verify the context > path whitelisting is documented there. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4332) Add NiFi Shell for interacting with NiFi REST API
[ https://issues.apache.org/jira/browse/NIFI-4332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159239#comment-16159239 ] Daniel Chaffelson commented on NIFI-4332: - So the current community method that I have seen is to take the swagger.json produced by the build, and run it through the swagger-codegen for your preferred language to give a (admittedly, very messy) boilerplate client which has all of the DTO etc integrated. It is then up to the developer to arrange the calls into the higher level functions - this has formed the basis of nifi-config that I linked above and I duplicated it easily in NiPyApi. It provides a very useful abstraction between business-defined high-level calls like 'get the names and IDs of all Process Groups' and 'instantiate this template in this PG with these parameters', and all the necessary underlying steps in the API. Essentially I feel like you're already 50%+ of the way there with the language independent OpenAPI definition, it would perhaps be fruitful to enhance that? Or pick a language to create a client from it in (say, Java or Python as there are already efforts available to adopt) but maintain the generative method via OpenAPI so other clients are possible. The problem of course will be that as soon as you produce an official client, people will be on to you to add X feature to it even if you specify that it's only to enable a few basic functions in a centralised manner. But as you always say, 'we accept PRs'. > Add NiFi Shell for interacting with NiFi REST API > - > > Key: NIFI-4332 > URL: https://issues.apache.org/jira/browse/NIFI-4332 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer > > There are several permutations of nifi shells floating around on Github. The > fact that so many of these exists tells me its a feature people want. I > propose we add a NiFi shell to the official project that people can use for > official interaction with the NiFi REST API. While shells are typically not > written in Java I feel quite strongly in our case using Java would be the > best fit. Using Java would allow us to use reflection on the "nifi-web-api" > layer to reflected expected types, paths, responses, etc with minimal coding > effort. > I expect there will be many more features that can be added to this shell but > as a minimal starting point the shell should allow an end user to interact > with all of the NiFi REST API endpoints defined at > https://nifi.apache.org/docs/nifi-docs/rest-api/index.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (NIFI-4332) Add NiFi Shell for interacting with NiFi REST API
[ https://issues.apache.org/jira/browse/NIFI-4332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157728#comment-16157728 ] Daniel Chaffelson edited comment on NIFI-4332 at 9/7/17 9:55 PM: - In the interest of -resolving- adding to this issue I have started ernest work on a Python wrapper for the Rest API by leveraging the Swagger definition produced at build time to create a base API interface in native Python, and then a wrapper module to provide users with much-requested high level functions. The use of the Swagger definition provides the full Rest API as callable python methods and properties with minimal additional coding, with easy integration into more abstract commands like 'deploy template'. https://github.com/Chaffelson/nipyapi This is mainly driven by several large users wanting NiFi as a layer in their data-movement-as-a-service platform, where a user portal drives instantiation of templates to complete movement tasks which leverage NiFi's other desirable framework features, but make little initial use of the GUI apart from administrative monitoring tasks. Contributions and advice on how this can integrate with current development efforts very welcome. I am also aware of a project in Java focussed around Template deployment, but with good potential for expansion https://github.com/hermannpencole/nifi-config was (Author: chaffelson): In the interest of -resolving- adding to this issue I have started ernest work on a Python wrapper for the Rest API by leveraging the Swagger definition produced at build time to create a base API interface in native Python, and then a wrapper module to provide users with much-requested high level functions. The use of the Swagger definition provides the full Rest API as callable python methods and properties with minimal additional coding, with easy integration into more abstract commands like 'deploy template'. https://github.com/Chaffelson/nipyapi This is mainly driven by several large users wanting NiFi as a layer in their data-movement-as-a-service platform, where a user portal drives instantiation of templates to complete movement tasks which leverage NiFi's other desirable framework features, but make little initial use of the GUI apart from administrative monitoring tasks. I am also aware of a project in Java focussed around Template deployment, but with good potential for expansion https://github.com/hermannpencole/nifi-config > Add NiFi Shell for interacting with NiFi REST API > - > > Key: NIFI-4332 > URL: https://issues.apache.org/jira/browse/NIFI-4332 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer > > There are several permutations of nifi shells floating around on Github. The > fact that so many of these exists tells me its a feature people want. I > propose we add a NiFi shell to the official project that people can use for > official interaction with the NiFi REST API. While shells are typically not > written in Java I feel quite strongly in our case using Java would be the > best fit. Using Java would allow us to use reflection on the "nifi-web-api" > layer to reflected expected types, paths, responses, etc with minimal coding > effort. > I expect there will be many more features that can be added to this shell but > as a minimal starting point the shell should allow an end user to interact > with all of the NiFi REST API endpoints defined at > https://nifi.apache.org/docs/nifi-docs/rest-api/index.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4332) Add NiFi Shell for interacting with NiFi REST API
[ https://issues.apache.org/jira/browse/NIFI-4332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157728#comment-16157728 ] Daniel Chaffelson commented on NIFI-4332: - In the interest of -resolving- adding to this issue I have started ernest work on a Python wrapper for the Rest API by leveraging the Swagger definition produced at build time to create a base API interface in native Python, and then a wrapper module to provide users with much-requested high level functions. The use of the Swagger definition provides the full Rest API as callable python methods and properties with minimal additional coding, with easy integration into more abstract commands like 'deploy template'. https://github.com/Chaffelson/nipyapi This is mainly driven by several large users wanting NiFi as a layer in their data-movement-as-a-service platform, where a user portal drives instantiation of templates to complete movement tasks which leverage NiFi's other desirable framework features, but make little initial use of the GUI apart from administrative monitoring tasks. I am also aware of a project in Java focussed around Template deployment, but with good potential for expansion https://github.com/hermannpencole/nifi-config > Add NiFi Shell for interacting with NiFi REST API > - > > Key: NIFI-4332 > URL: https://issues.apache.org/jira/browse/NIFI-4332 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer > > There are several permutations of nifi shells floating around on Github. The > fact that so many of these exists tells me its a feature people want. I > propose we add a NiFi shell to the official project that people can use for > official interaction with the NiFi REST API. While shells are typically not > written in Java I feel quite strongly in our case using Java would be the > best fit. Using Java would allow us to use reflection on the "nifi-web-api" > layer to reflected expected types, paths, responses, etc with minimal coding > effort. > I expect there will be many more features that can be added to this shell but > as a minimal starting point the shell should allow an end user to interact > with all of the NiFi REST API endpoints defined at > https://nifi.apache.org/docs/nifi-docs/rest-api/index.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-3612) Add support for Parquet to Nifi-Registry-Bundle
[ https://issues.apache.org/jira/browse/NIFI-3612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16131220#comment-16131220 ] Daniel Chaffelson commented on NIFI-3612: - Sounds perfectly reasonable to me. > Add support for Parquet to Nifi-Registry-Bundle > --- > > Key: NIFI-3612 > URL: https://issues.apache.org/jira/browse/NIFI-3612 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Daniel Chaffelson >Assignee: Oleg Zhurakousky > > This bundle could potentially be extended to include a Parquet transform by > leveraging the Apache 2.0 licensed parquet-mr/avro libraries: > https://github.com/apache/parquet-mr/tree/master/parquet-avro > This would provide coverage of this popular format to complement the ORC > support in the Hive Bundle and the other schema-dependent formats already in > this bundle. > Existing NiFi Parquet support in the kite bundle can only write to a > non-kerberised Kite Dataset, which prevents usage on secured environments or > writing to a FlowFile. > As the main competitor to ORC, providing more generic Parquet Transform > support will greatly widen the pool of potential NiFi adopters, particularly > in the Spark community. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (NIFI-4230) Use a better default location when pasting via keystokes outside the context of the originally selected components
Daniel Chaffelson created NIFI-4230: --- Summary: Use a better default location when pasting via keystokes outside the context of the originally selected components Key: NIFI-4230 URL: https://issues.apache.org/jira/browse/NIFI-4230 Project: Apache NiFi Issue Type: Improvement Components: Core UI Affects Versions: 1.3.0 Reporter: Daniel Chaffelson Priority: Minor When copy/pasting components between different Process Groups, the keystroke behavior differs from the right-click behavior. Specifically, it does not paste the components on the current focus area of the canvas, but appears to paste them relative to their original position in the original context, which effectively puts them somewhere random on the focus canvas, requiring the user to hunt them down. It would be optimal to select a better default location somewhere in the focus area of the canvas for all paste behaviors consistently. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-3612) Add support for Parquet to Nifi-Registry-Bundle
[ https://issues.apache.org/jira/browse/NIFI-3612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Chaffelson updated NIFI-3612: Issue Type: New Feature (was: Improvement) > Add support for Parquet to Nifi-Registry-Bundle > --- > > Key: NIFI-3612 > URL: https://issues.apache.org/jira/browse/NIFI-3612 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Daniel Chaffelson >Assignee: Oleg Zhurakousky > > This bundle could potentially be extended to include a Parquet transform by > leveraging the Apache 2.0 licensed parquet-mr/avro libraries: > https://github.com/apache/parquet-mr/tree/master/parquet-avro > This would provide coverage of this popular format to complement the ORC > support in the Hive Bundle and the other schema-dependent formats already in > this bundle. > Existing NiFi Parquet support in the kite bundle can only write to a > non-kerberised Kite Dataset, which prevents usage on secured environments or > writing to a FlowFile. > As the main competitor to ORC, providing more generic Parquet Transform > support will greatly widen the pool of potential NiFi adopters, particularly > in the Spark community. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3612) Add support for Parquet to Nifi-Registry-Bundle
[ https://issues.apache.org/jira/browse/NIFI-3612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Chaffelson updated NIFI-3612: Description: This bundle could potentially be extended to include a Parquet transform by leveraging the Apache 2.0 licensed parquet-mr/avro libraries: https://github.com/apache/parquet-mr/tree/master/parquet-avro This would provide coverage of this popular format to complement the ORC support in the Hive Bundle and the other schema-dependent formats already in this bundle. Existing NiFi Parquet support in the kite bundle can only write to a non-kerberised Kite Dataset, which prevents usage on secured environments or writing to a FlowFile. As the main competitor to ORC, providing more generic Parquet Transform support will greatly widen the pool of potential NiFi adopters, particularly in the Spark community. was: This bundle could potentially be extended to include a Parquet transform by leveraging the Apache 2.0 licenses parquet-mr/avro libraries: https://github.com/apache/parquet-mr/tree/master/parquet-avro This would provide coverage of this popular format to complement the ORC support in the Hive Bundle and the other schema-dependent formats already in this bundle. > Add support for Parquet to Nifi-Registry-Bundle > --- > > Key: NIFI-3612 > URL: https://issues.apache.org/jira/browse/NIFI-3612 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Chaffelson > > This bundle could potentially be extended to include a Parquet transform by > leveraging the Apache 2.0 licensed parquet-mr/avro libraries: > https://github.com/apache/parquet-mr/tree/master/parquet-avro > This would provide coverage of this popular format to complement the ORC > support in the Hive Bundle and the other schema-dependent formats already in > this bundle. > Existing NiFi Parquet support in the kite bundle can only write to a > non-kerberised Kite Dataset, which prevents usage on secured environments or > writing to a FlowFile. > As the main competitor to ORC, providing more generic Parquet Transform > support will greatly widen the pool of potential NiFi adopters, particularly > in the Spark community. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (NIFI-3612) Add support for Parquet to Nifi-Registry-Bundle
Daniel Chaffelson created NIFI-3612: --- Summary: Add support for Parquet to Nifi-Registry-Bundle Key: NIFI-3612 URL: https://issues.apache.org/jira/browse/NIFI-3612 Project: Apache NiFi Issue Type: Improvement Reporter: Daniel Chaffelson This bundle could potentially be extended to include a Parquet transform by leveraging the Apache 2.0 licenses parquet-mr/avro libraries: https://github.com/apache/parquet-mr/tree/master/parquet-avro This would provide coverage of this popular format to complement the ORC support in the Hive Bundle and the other schema-dependent formats already in this bundle. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
