[jira] [Assigned] (MINIFI-434) Preserve security properties when pulling a new config
[ https://issues.apache.org/jira/browse/MINIFI-434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard reassigned MINIFI-434: - Assignee: Pierre Villard > Preserve security properties when pulling a new config > -- > > Key: MINIFI-434 > URL: https://issues.apache.org/jira/browse/MINIFI-434 > Project: Apache NiFi MiNiFi > Issue Type: Improvement >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > > Situation (secured instances) > MiNiFi == (pull) ==> C2 server == (pull template) ==> NiFi cluster > Template to pull contains a RPG to perform S2S between MiNiFi and NiFi > cluster over HTTPS. > When a new version of the template is pulled, security properties are removed > and S2S cannot be performed until config is manually fixed and agent > restarted. > Security properties should probably be excluded in this config update process. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFI-434) Preserve security properties when pulling a new config
[
https://issues.apache.org/jira/browse/MINIFI-434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16343947#comment-16343947
]
Pierre Villard commented on MINIFI-434:
---
I was looking into it and I suggest to make a change in
{{handleChange(InputStream)}} of {{MiNiFiConfigurationChangeListener}}. I guess
this concerns only the {{PullHttpChangeIngestor}} implementation and we could
add a property for this ingestor to let users decide if security properties
should be preserved or not. Thoughts?
> Preserve security properties when pulling a new config
> --
>
> Key: MINIFI-434
> URL: https://issues.apache.org/jira/browse/MINIFI-434
> Project: Apache NiFi MiNiFi
> Issue Type: Improvement
>Reporter: Pierre Villard
>Priority: Major
>
> Situation (secured instances)
> MiNiFi == (pull) ==> C2 server == (pull template) ==> NiFi cluster
> Template to pull contains a RPG to perform S2S between MiNiFi and NiFi
> cluster over HTTPS.
> When a new version of the template is pulled, security properties are removed
> and S2S cannot be performed until config is manually fixed and agent
> restarted.
> Security properties should probably be excluded in this config update process.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (MINIFI-436) Consider common dependency management with NiFi
Pierre Villard created MINIFI-436: - Summary: Consider common dependency management with NiFi Key: MINIFI-436 URL: https://issues.apache.org/jira/browse/MINIFI-436 Project: Apache NiFi MiNiFi Issue Type: Improvement Components: Build Reporter: Pierre Villard In order to avoid problems raised in MINIFI-431 and MINIFI-435 it could be interesting to consider a common dependency management between MiNiFi Java and NiFi projects. This would prevent dependency conflicts when common dependencies are upgraded on only one side. Not sure what's the best solution is while ensuring proper CI builds. An option could be to use the main pom of NiFi project as parent pom of the MiNiFi project with the version of NiFi that is used to build MiNiFi (not the SNAPSHOT version though). This way, common dependencies could be defined only on NiFi's side. The release process would need to be changed a bit to ensure version of parent pom is correctly updated to the least released version of NiFi. To be discussed here. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFI-435) MiNiFi - Cannot perform secured S2S with NiFi 1.5.0
[
https://issues.apache.org/jira/browse/MINIFI-435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16342528#comment-16342528
]
Pierre Villard commented on MINIFI-435:
---
I confirm that once I updated the versions to match the ones used in NiFi the
problem is resolved an secured S2S over HTTPS can be performed with a NiFi
1.5.0 instance.
> MiNiFi - Cannot perform secured S2S with NiFi 1.5.0
> ---
>
> Key: MINIFI-435
> URL: https://issues.apache.org/jira/browse/MINIFI-435
> Project: Apache NiFi MiNiFi
> Issue Type: Bug
> Components: Data Transmission
>Affects Versions: 0.4.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Blocker
> Fix For: 0.5.0
>
>
> I think that, similarly to MINIFI-431, there is a dependency issue preventing
> the possibility to perform S2S over HTTPS between MiNiFi 0.4.0 and NiFi
> 1.5.0. When trying to perform secured S2S, I get the following exception:
> {noformat}
> 2018-01-27 22:41:57,581 WARN [Timer-Driven Process Thread-4]
> o.a.n.c.t.ContinuallyRunConnectableTask
> RemoteGroupPort[name=IOT_Input,targets=https://...:9091/nifi]
> Administratively Pausing for 10 seconds due to processing failure:
> java.lang.RuntimeException: java.lang.NoSuchMethodError:
> org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
> java.lang.RuntimeException: java.lang.NoSuchMethodError:
> org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
> at
> org.apache.nifi.controller.AbstractPort.onTrigger(AbstractPort.java:257)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunConnectableTask.call(ContinuallyRunConnectableTask.java:81)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunConnectableTask.call(ContinuallyRunConnectableTask.java:40)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NoSuchMethodError:
> org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
> at
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.setupClient(SiteToSiteRestApiClient.java:278)
> at
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.getHttpClient(SiteToSiteRestApiClient.java:219)
> at
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.execute(SiteToSiteRestApiClient.java:1189)
> at
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.execute(SiteToSiteRestApiClient.java:1237)
> at
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.fetchController(SiteToSiteRestApiClient.java:419)
> at
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.getController(SiteToSiteRestApiClient.java:394)
> at
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.getController(SiteToSiteRestApiClient.java:361)
> at
> org.apache.nifi.remote.client.SiteInfoProvider.refreshRemoteInfo(SiteInfoProvider.java:69)
> at
> org.apache.nifi.remote.client.SiteInfoProvider.getSiteToSiteHttpPort(SiteInfoProvider.java:160)
> at
> org.apache.nifi.remote.client.http.HttpClient.getBootstrapPeerDescription(HttpClient.java:90)
> at
> org.apache.nifi.remote.client.PeerSelector.fetchRemotePeerStatuses(PeerSelector.java:379)
> at
> org.apache.nifi.remote.client.PeerSelector.refreshPeers(PeerSelector.java:352)
> at
> org.apache.nifi.remote.client.PeerSelector.createPeerStatusList(PeerSelector.java:316)
> at
> org.apache.nifi.remote.client.PeerSelector.getNextPeerStatus(PeerSelector.java:275)
> at
> org.apache.nifi.remote.client.http.HttpClient.createTransaction(HttpClient.java:129)
> at
> org.apache.nifi.remote.StandardRemoteGroupPort.onTrigger(StandardRemoteGroupPort.java:238)
> at
> org.apache.nifi.controller.AbstractPort.onTrigger(AbstractPort.java:250)
> ... 10 common frames omitted{noformat}
> When doing the same
[jira] [Assigned] (MINIFI-435) MiNiFi - Cannot perform secured S2S with NiFi 1.5.0
[
https://issues.apache.org/jira/browse/MINIFI-435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard reassigned MINIFI-435:
-
Assignee: Pierre Villard
> MiNiFi - Cannot perform secured S2S with NiFi 1.5.0
> ---
>
> Key: MINIFI-435
> URL: https://issues.apache.org/jira/browse/MINIFI-435
> Project: Apache NiFi MiNiFi
> Issue Type: Bug
> Components: Data Transmission
>Affects Versions: 0.4.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Blocker
> Fix For: 0.5.0
>
>
> I think that, similarly to MINIFI-431, there is a dependency issue preventing
> the possibility to perform S2S over HTTPS between MiNiFi 0.4.0 and NiFi
> 1.5.0. When trying to perform secured S2S, I get the following exception:
> {noformat}
> 2018-01-27 22:41:57,581 WARN [Timer-Driven Process Thread-4]
> o.a.n.c.t.ContinuallyRunConnectableTask
> RemoteGroupPort[name=IOT_Input,targets=https://...:9091/nifi]
> Administratively Pausing for 10 seconds due to processing failure:
> java.lang.RuntimeException: java.lang.NoSuchMethodError:
> org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
> java.lang.RuntimeException: java.lang.NoSuchMethodError:
> org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
> at
> org.apache.nifi.controller.AbstractPort.onTrigger(AbstractPort.java:257)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunConnectableTask.call(ContinuallyRunConnectableTask.java:81)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunConnectableTask.call(ContinuallyRunConnectableTask.java:40)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NoSuchMethodError:
> org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
> at
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.setupClient(SiteToSiteRestApiClient.java:278)
> at
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.getHttpClient(SiteToSiteRestApiClient.java:219)
> at
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.execute(SiteToSiteRestApiClient.java:1189)
> at
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.execute(SiteToSiteRestApiClient.java:1237)
> at
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.fetchController(SiteToSiteRestApiClient.java:419)
> at
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.getController(SiteToSiteRestApiClient.java:394)
> at
> org.apache.nifi.remote.util.SiteToSiteRestApiClient.getController(SiteToSiteRestApiClient.java:361)
> at
> org.apache.nifi.remote.client.SiteInfoProvider.refreshRemoteInfo(SiteInfoProvider.java:69)
> at
> org.apache.nifi.remote.client.SiteInfoProvider.getSiteToSiteHttpPort(SiteInfoProvider.java:160)
> at
> org.apache.nifi.remote.client.http.HttpClient.getBootstrapPeerDescription(HttpClient.java:90)
> at
> org.apache.nifi.remote.client.PeerSelector.fetchRemotePeerStatuses(PeerSelector.java:379)
> at
> org.apache.nifi.remote.client.PeerSelector.refreshPeers(PeerSelector.java:352)
> at
> org.apache.nifi.remote.client.PeerSelector.createPeerStatusList(PeerSelector.java:316)
> at
> org.apache.nifi.remote.client.PeerSelector.getNextPeerStatus(PeerSelector.java:275)
> at
> org.apache.nifi.remote.client.http.HttpClient.createTransaction(HttpClient.java:129)
> at
> org.apache.nifi.remote.StandardRemoteGroupPort.onTrigger(StandardRemoteGroupPort.java:238)
> at
> org.apache.nifi.controller.AbstractPort.onTrigger(AbstractPort.java:250)
> ... 10 common frames omitted{noformat}
> When doing the same between MiNiFi 0.3.0 and NiFi 1.5.0, it works as expected.
> I believe this could be related to NIFI-3954 /
> [https://github.com/apache/nifi/pull/1841]
> I'll try to update
[jira] [Created] (MINIFI-435) MiNiFi - Cannot perform secured S2S with NiFi 1.5.0
Pierre Villard created MINIFI-435:
-
Summary: MiNiFi - Cannot perform secured S2S with NiFi 1.5.0
Key: MINIFI-435
URL: https://issues.apache.org/jira/browse/MINIFI-435
Project: Apache NiFi MiNiFi
Issue Type: Bug
Components: Data Transmission
Affects Versions: 0.4.0
Reporter: Pierre Villard
I think that, similarly to MINIFI-431, there is a dependency issue preventing
the possibility to perform S2S over HTTPS between MiNiFi 0.4.0 and NiFi 1.5.0.
When trying to perform secured S2S, I get the following exception:
{noformat}
2018-01-27 22:41:57,581 WARN [Timer-Driven Process Thread-4]
o.a.n.c.t.ContinuallyRunConnectableTask
RemoteGroupPort[name=IOT_Input,targets=https://...:9091/nifi] Administratively
Pausing for 10 seconds due to processing failure: java.lang.RuntimeException:
java.lang.NoSuchMethodError:
org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
java.lang.RuntimeException: java.lang.NoSuchMethodError:
org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
at
org.apache.nifi.controller.AbstractPort.onTrigger(AbstractPort.java:257)
at
org.apache.nifi.controller.tasks.ContinuallyRunConnectableTask.call(ContinuallyRunConnectableTask.java:81)
at
org.apache.nifi.controller.tasks.ContinuallyRunConnectableTask.call(ContinuallyRunConnectableTask.java:40)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoSuchMethodError:
org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
at
org.apache.nifi.remote.util.SiteToSiteRestApiClient.setupClient(SiteToSiteRestApiClient.java:278)
at
org.apache.nifi.remote.util.SiteToSiteRestApiClient.getHttpClient(SiteToSiteRestApiClient.java:219)
at
org.apache.nifi.remote.util.SiteToSiteRestApiClient.execute(SiteToSiteRestApiClient.java:1189)
at
org.apache.nifi.remote.util.SiteToSiteRestApiClient.execute(SiteToSiteRestApiClient.java:1237)
at
org.apache.nifi.remote.util.SiteToSiteRestApiClient.fetchController(SiteToSiteRestApiClient.java:419)
at
org.apache.nifi.remote.util.SiteToSiteRestApiClient.getController(SiteToSiteRestApiClient.java:394)
at
org.apache.nifi.remote.util.SiteToSiteRestApiClient.getController(SiteToSiteRestApiClient.java:361)
at
org.apache.nifi.remote.client.SiteInfoProvider.refreshRemoteInfo(SiteInfoProvider.java:69)
at
org.apache.nifi.remote.client.SiteInfoProvider.getSiteToSiteHttpPort(SiteInfoProvider.java:160)
at
org.apache.nifi.remote.client.http.HttpClient.getBootstrapPeerDescription(HttpClient.java:90)
at
org.apache.nifi.remote.client.PeerSelector.fetchRemotePeerStatuses(PeerSelector.java:379)
at
org.apache.nifi.remote.client.PeerSelector.refreshPeers(PeerSelector.java:352)
at
org.apache.nifi.remote.client.PeerSelector.createPeerStatusList(PeerSelector.java:316)
at
org.apache.nifi.remote.client.PeerSelector.getNextPeerStatus(PeerSelector.java:275)
at
org.apache.nifi.remote.client.http.HttpClient.createTransaction(HttpClient.java:129)
at
org.apache.nifi.remote.StandardRemoteGroupPort.onTrigger(StandardRemoteGroupPort.java:238)
at
org.apache.nifi.controller.AbstractPort.onTrigger(AbstractPort.java:250)
... 10 common frames omitted{noformat}
When doing the same between MiNiFi 0.3.0 and NiFi 1.5.0, it works as expected.
I believe this could be related to NIFI-3954 /
[https://github.com/apache/nifi/pull/1841]
I'll try to update the dependency to the same versions on MiNiFi side and try
secured S2S.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (MINIFI-434) Preserve security properties when pulling a new config
Pierre Villard created MINIFI-434: - Summary: Preserve security properties when pulling a new config Key: MINIFI-434 URL: https://issues.apache.org/jira/browse/MINIFI-434 Project: Apache NiFi MiNiFi Issue Type: Improvement Reporter: Pierre Villard Situation (secured instances) MiNiFi == (pull) ==> C2 server == (pull template) ==> NiFi cluster Template to pull contains a RPG to perform S2S between MiNiFi and NiFi cluster over HTTPS. When a new version of the template is pulled, security properties are removed and S2S cannot be performed until config is manually fixed and agent restarted. Security properties should probably be excluded in this config update process. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MINIFI-433) PG Variable Registry support in templates
Pierre Villard created MINIFI-433:
-
Summary: PG Variable Registry support in templates
Key: MINIFI-433
URL: https://issues.apache.org/jira/browse/MINIFI-433
Project: Apache NiFi MiNiFi
Issue Type: Improvement
Reporter: Pierre Villard
Right now if a NiFi template contains a {{}} block to define a
variable registry at process group level, this is not reflected into the yaml
generated template used by MiNiFi agents. This would be very useful when
templates are deployed to MiNiFi agents via a NiFi registry/C2 approach.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (MINIFI-431) C2 - Exception while agent tries to pull the config
[
https://issues.apache.org/jira/browse/MINIFI-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338961#comment-16338961
]
Pierre Villard edited comment on MINIFI-431 at 1/25/18 9:44 AM:
I just had another quick look. I believe this is because we bumped the jersey
version to 2.26 in NiFi with NIFI-
([https://github.com/apache/nifi/pull/2206)] but we still have the jersey 1.19
version in MiNiFi causing the conflict.
Will try to submit a PR later today to fix the dependency and update L
{color:#FF}EDIT{color} - in the end, it does not seem to be an easy change
(updating versions is not enough). I don't have cycles right now to look into
it. If someone has, feel free to jump in.
was (Author: pvillard):
I just had another quick look. I believe this is because we bumped the jersey
version to 2.26 in NiFi with NIFI-
([https://github.com/apache/nifi/pull/2206)] but we still have the jersey 1.19
version in MiNiFi causing the conflict.
Will try to submit a PR later today to fix the dependency and update L
> C2 - Exception while agent tries to pull the config
> ---
>
> Key: MINIFI-431
> URL: https://issues.apache.org/jira/browse/MINIFI-431
> Project: Apache NiFi MiNiFi
> Issue Type: Bug
> Components: Command and Control
>Affects Versions: 0.4.0
>Reporter: Pierre Villard
>Priority: Major
> Attachments: bootstrap.conf, minifi-c2-context.xml
>
>
> My setup:
> MiNiFi Java agent - 0.4.0
> C2 server - 0.4.0
> On agent's side:
> {code:java}
> nifi.minifi.notifier.ingestors=org.apache.nifi.minifi.bootstrap.configuration.ingestors.PullHttpChangeIngestor
> # Hostname on which to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.hostname=localhost
> # Port on which to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.port=10080
> # Path to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.path=/c2/config
> # Query string to pull configurations with
> nifi.minifi.notifier.ingestors.pull.http.query=class=hadoop-agents
> # Period on which to pull configurations from, defaults to 5 minutes if
> commented out
> nifi.minifi.notifier.ingestors.pull.http.period.ms=6
> {code}
>
> I kept everything default on C2's side. I just put my workflow into:
> {code:java}
> ./files/hadoop-agents/config.text.yml.v1
> {code}
>
> Here is the exception I get when the agent tries to pull the config:
> {code:java}
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
> INFO: Using default applicationContext
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.spi.spring.container.SpringComponentProviderFactory
> registerSpringBeans
> INFO: Registering Spring bean, configService, of type
> org.apache.nifi.minifi.c2.service.ConfigService as a root resource class
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
> INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25
> AM'
> 2018-01-24 20:45:04,397 DEBUG [qtp1018298342-12]
> o.a.n.m.c.s.a.C2AnonymousAuthenticationFilter Populated SecurityContextHolder
> with anonymous token:
> 'org.apache.nifi.minifi.c2.security.authentication.C2AuthenticationToken@52a3d06d:
> Principal: anonymous; Credentials: [PROTECTED]; Authenticated: true;
> Details: null; Granted Authorities: ROLE_ANONYMOUS'
> 2018-01-24 20:45:04,403 WARN [qtp1018298342-12]
> org.eclipse.jetty.server.HttpChannel /c2/config
> java.lang.AbstractMethodError:
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:120)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:669)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:316)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:122)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
>
[jira] [Commented] (MINIFI-431) C2 - Exception while agent tries to pull the config
[
https://issues.apache.org/jira/browse/MINIFI-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338961#comment-16338961
]
Pierre Villard commented on MINIFI-431:
---
I just had another quick look. I believe this is because we bumped the jersey
version to 2.26 in NiFi with NIFI-
([https://github.com/apache/nifi/pull/2206)] but we still have the jersey 1.19
version in MiNiFi causing the conflict.
Will try to submit a PR later today to fix the dependency and update L
> C2 - Exception while agent tries to pull the config
> ---
>
> Key: MINIFI-431
> URL: https://issues.apache.org/jira/browse/MINIFI-431
> Project: Apache NiFi MiNiFi
> Issue Type: Bug
> Components: Command and Control
>Affects Versions: 0.4.0
>Reporter: Pierre Villard
>Priority: Major
> Attachments: bootstrap.conf, minifi-c2-context.xml
>
>
> My setup:
> MiNiFi Java agent - 0.4.0
> C2 server - 0.4.0
> On agent's side:
> {code:java}
> nifi.minifi.notifier.ingestors=org.apache.nifi.minifi.bootstrap.configuration.ingestors.PullHttpChangeIngestor
> # Hostname on which to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.hostname=localhost
> # Port on which to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.port=10080
> # Path to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.path=/c2/config
> # Query string to pull configurations with
> nifi.minifi.notifier.ingestors.pull.http.query=class=hadoop-agents
> # Period on which to pull configurations from, defaults to 5 minutes if
> commented out
> nifi.minifi.notifier.ingestors.pull.http.period.ms=6
> {code}
>
> I kept everything default on C2's side. I just put my workflow into:
> {code:java}
> ./files/hadoop-agents/config.text.yml.v1
> {code}
>
> Here is the exception I get when the agent tries to pull the config:
> {code:java}
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
> INFO: Using default applicationContext
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.spi.spring.container.SpringComponentProviderFactory
> registerSpringBeans
> INFO: Registering Spring bean, configService, of type
> org.apache.nifi.minifi.c2.service.ConfigService as a root resource class
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
> INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25
> AM'
> 2018-01-24 20:45:04,397 DEBUG [qtp1018298342-12]
> o.a.n.m.c.s.a.C2AnonymousAuthenticationFilter Populated SecurityContextHolder
> with anonymous token:
> 'org.apache.nifi.minifi.c2.security.authentication.C2AuthenticationToken@52a3d06d:
> Principal: anonymous; Credentials: [PROTECTED]; Authenticated: true;
> Details: null; Granted Authorities: ROLE_ANONYMOUS'
> 2018-01-24 20:45:04,403 WARN [qtp1018298342-12]
> org.eclipse.jetty.server.HttpChannel /c2/config
> java.lang.AbstractMethodError:
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:120)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:669)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:316)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:122)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.apache.nifi.minifi.c2.security.authentication.X509AuthenticationFilter.doFilter(X509AuthenticationFilter.java:43)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:213)
> at
>
[jira] [Updated] (MINIFI-431) C2 - Exception while agent tries to pull the config
[
https://issues.apache.org/jira/browse/MINIFI-431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated MINIFI-431:
--
Attachment: minifi-c2-context.xml
> C2 - Exception while agent tries to pull the config
> ---
>
> Key: MINIFI-431
> URL: https://issues.apache.org/jira/browse/MINIFI-431
> Project: Apache NiFi MiNiFi
> Issue Type: Bug
> Components: Command and Control
>Affects Versions: 0.4.0
>Reporter: Pierre Villard
>Priority: Major
> Attachments: bootstrap.conf, minifi-c2-context.xml
>
>
> My setup:
> MiNiFi Java agent - 0.4.0
> C2 server - 0.4.0
> On agent's side:
> {code:java}
> nifi.minifi.notifier.ingestors=org.apache.nifi.minifi.bootstrap.configuration.ingestors.PullHttpChangeIngestor
> # Hostname on which to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.hostname=localhost
> # Port on which to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.port=10080
> # Path to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.path=/c2/config
> # Query string to pull configurations with
> nifi.minifi.notifier.ingestors.pull.http.query=class=hadoop-agents
> # Period on which to pull configurations from, defaults to 5 minutes if
> commented out
> nifi.minifi.notifier.ingestors.pull.http.period.ms=6
> {code}
>
> I kept everything default on C2's side. I just put my workflow into:
> {code:java}
> ./files/hadoop-agents/config.text.yml.v1
> {code}
>
> Here is the exception I get when the agent tries to pull the config:
> {code:java}
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
> INFO: Using default applicationContext
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.spi.spring.container.SpringComponentProviderFactory
> registerSpringBeans
> INFO: Registering Spring bean, configService, of type
> org.apache.nifi.minifi.c2.service.ConfigService as a root resource class
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
> INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25
> AM'
> 2018-01-24 20:45:04,397 DEBUG [qtp1018298342-12]
> o.a.n.m.c.s.a.C2AnonymousAuthenticationFilter Populated SecurityContextHolder
> with anonymous token:
> 'org.apache.nifi.minifi.c2.security.authentication.C2AuthenticationToken@52a3d06d:
> Principal: anonymous; Credentials: [PROTECTED]; Authenticated: true;
> Details: null; Granted Authorities: ROLE_ANONYMOUS'
> 2018-01-24 20:45:04,403 WARN [qtp1018298342-12]
> org.eclipse.jetty.server.HttpChannel /c2/config
> java.lang.AbstractMethodError:
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:120)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:669)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:316)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:122)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.apache.nifi.minifi.c2.security.authentication.X509AuthenticationFilter.doFilter(X509AuthenticationFilter.java:43)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:213)
> at
> org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:176)
> at
> org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
> at
> org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
> at
>
[jira] [Updated] (MINIFI-431) C2 - Exception while agent tries to pull the config
[
https://issues.apache.org/jira/browse/MINIFI-431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated MINIFI-431:
--
Attachment: bootstrap.conf
> C2 - Exception while agent tries to pull the config
> ---
>
> Key: MINIFI-431
> URL: https://issues.apache.org/jira/browse/MINIFI-431
> Project: Apache NiFi MiNiFi
> Issue Type: Bug
> Components: Command and Control
>Affects Versions: 0.4.0
>Reporter: Pierre Villard
>Priority: Major
> Attachments: bootstrap.conf, minifi-c2-context.xml
>
>
> My setup:
> MiNiFi Java agent - 0.4.0
> C2 server - 0.4.0
> On agent's side:
> {code:java}
> nifi.minifi.notifier.ingestors=org.apache.nifi.minifi.bootstrap.configuration.ingestors.PullHttpChangeIngestor
> # Hostname on which to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.hostname=localhost
> # Port on which to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.port=10080
> # Path to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.path=/c2/config
> # Query string to pull configurations with
> nifi.minifi.notifier.ingestors.pull.http.query=class=hadoop-agents
> # Period on which to pull configurations from, defaults to 5 minutes if
> commented out
> nifi.minifi.notifier.ingestors.pull.http.period.ms=6
> {code}
>
> I kept everything default on C2's side. I just put my workflow into:
> {code:java}
> ./files/hadoop-agents/config.text.yml.v1
> {code}
>
> Here is the exception I get when the agent tries to pull the config:
> {code:java}
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
> INFO: Using default applicationContext
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.spi.spring.container.SpringComponentProviderFactory
> registerSpringBeans
> INFO: Registering Spring bean, configService, of type
> org.apache.nifi.minifi.c2.service.ConfigService as a root resource class
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
> INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25
> AM'
> 2018-01-24 20:45:04,397 DEBUG [qtp1018298342-12]
> o.a.n.m.c.s.a.C2AnonymousAuthenticationFilter Populated SecurityContextHolder
> with anonymous token:
> 'org.apache.nifi.minifi.c2.security.authentication.C2AuthenticationToken@52a3d06d:
> Principal: anonymous; Credentials: [PROTECTED]; Authenticated: true;
> Details: null; Granted Authorities: ROLE_ANONYMOUS'
> 2018-01-24 20:45:04,403 WARN [qtp1018298342-12]
> org.eclipse.jetty.server.HttpChannel /c2/config
> java.lang.AbstractMethodError:
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:120)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:669)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:316)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:122)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.apache.nifi.minifi.c2.security.authentication.X509AuthenticationFilter.doFilter(X509AuthenticationFilter.java:43)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:213)
> at
> org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:176)
> at
> org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
> at
> org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
> at
>
[jira] [Comment Edited] (MINIFI-431) C2 - Exception while agent tries to pull the config
[
https://issues.apache.org/jira/browse/MINIFI-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338209#comment-16338209
]
Pierre Villard edited comment on MINIFI-431 at 1/24/18 8:59 PM:
Here are the notable differences I see between the two versions in ./lib
directory (jars that are unique to the version):
|*0.3.0 version*|*0.4.0 version*|
|h2-1.3.176.jar|aopalliance-repackaged-2.5.0-b42.jar|
|jackson-core-asl-1.9.13.jar|hk2-api-2.5.0-b42.jar|
|jackson-jaxrs-1.9.2.jar|hk2-locator-2.5.0-b42.jar|
|jackson-mapper-asl-1.9.13.jar|hk2-utils-2.5.0-b42.jar|
|jackson-xc-1.9.2.jar|jackson-module-jaxb-annotations-2.9.1.jar|
|jaxb-api-2.2.2.jar|javassist-3.22.0-CR2.jar|
|jaxb-impl-2.2.3-1.jar|javax.annotation-api-1.2.jar|
|jersey-client-1.19.jar|javax.inject-1.jar|
|jersey-json-1.19.jar|javax.inject-2.5.0-b42.jar|
|jettison-1.1.jar|javax.ws.rs-api-2.1.jar|
|stax-api-1.0-2.jar|jersey-client-2.26.jar|
|swagger-annotations-1.5.3-M1.jar|jersey-common-2.26.jar|
| |jersey-entity-filtering-2.26.jar|
| |jersey-hk2-2.26.jar|
| |jersey-media-json-jackson-2.26.jar|
| |swagger-annotations-1.5.16.jar|
| |validation-api-2.0.0.Final.jar|
By removing the file javax.ws.rs-api-2.1.jar from lib directory in 0.4.0
version, it works as expected.
was (Author: pvillard):
I believe problem comes from here:
{noformat}
[INFO] | +- org.apache.nifi.registry:nifi-registry-client:jar:0.1.0:compile
[INFO] | | +-
org.apache.nifi.registry:nifi-registry-security-utils:jar:0.1.0:compile
[INFO] | | +-
org.glassfish.jersey.media:jersey-media-json-jackson:jar:2.26:compile
[INFO] | | | \-
org.glassfish.jersey.ext:jersey-entity-filtering:jar:2.26:compile
[INFO] | | \- org.glassfish.jersey.inject:jersey-hk2:jar:2.26:compile
[INFO] | | \- org.glassfish.hk2:hk2-locator:jar:2.5.0-b42:compile
[INFO] | | +-
org.glassfish.hk2.external:aopalliance-repackaged:jar:2.5.0-b42:compile
[INFO] | | +- org.glassfish.hk2:hk2-api:jar:2.5.0-b42:compile
[INFO] | | | \- javax.inject:javax.inject:jar:1:compile
[INFO] | | +- org.glassfish.hk2:hk2-utils:jar:2.5.0-b42:compile
[INFO] | | \- org.javassist:javassist:jar:3.22.0-CR2:compile{noformat}
Here are the notable differences I see between the two versions in ./lib
directory (jars that are unique to the version):
|*0.3.0 version*|*0.4.0 version*|
|h2-1.3.176.jar|aopalliance-repackaged-2.5.0-b42.jar|
|jackson-core-asl-1.9.13.jar|hk2-api-2.5.0-b42.jar|
|jackson-jaxrs-1.9.2.jar|hk2-locator-2.5.0-b42.jar|
|jackson-mapper-asl-1.9.13.jar|hk2-utils-2.5.0-b42.jar|
|jackson-xc-1.9.2.jar|jackson-module-jaxb-annotations-2.9.1.jar|
|jaxb-api-2.2.2.jar|javassist-3.22.0-CR2.jar|
|jaxb-impl-2.2.3-1.jar|javax.annotation-api-1.2.jar|
|jersey-client-1.19.jar|javax.inject-1.jar|
|jersey-json-1.19.jar|javax.inject-2.5.0-b42.jar|
|jettison-1.1.jar|javax.ws.rs-api-2.1.jar|
|stax-api-1.0-2.jar|jersey-client-2.26.jar|
|swagger-annotations-1.5.3-M1.jar|jersey-common-2.26.jar|
| |jersey-entity-filtering-2.26.jar|
| |jersey-hk2-2.26.jar|
| |jersey-media-json-jackson-2.26.jar|
| |swagger-annotations-1.5.16.jar|
| |validation-api-2.0.0.Final.jar|
By removing the file javax.ws.rs-api-2.1.jar from lib directory in 0.4.0
version, it works as expected.
> C2 - Exception while agent tries to pull the config
> ---
>
> Key: MINIFI-431
> URL: https://issues.apache.org/jira/browse/MINIFI-431
> Project: Apache NiFi MiNiFi
> Issue Type: Bug
> Components: Command and Control
>Affects Versions: 0.4.0
>Reporter: Pierre Villard
>Priority: Major
>
> My setup:
> MiNiFi Java agent - 0.4.0
> C2 server - 0.4.0
> On agent's side:
> {code:java}
> nifi.minifi.notifier.ingestors=org.apache.nifi.minifi.bootstrap.configuration.ingestors.PullHttpChangeIngestor
> # Hostname on which to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.hostname=localhost
> # Port on which to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.port=10080
> # Path to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.path=/c2/config
> # Query string to pull configurations with
> nifi.minifi.notifier.ingestors.pull.http.query=class=hadoop-agents
> # Period on which to pull configurations from, defaults to 5 minutes if
> commented out
> nifi.minifi.notifier.ingestors.pull.http.period.ms=6
> {code}
>
> I kept everything default on C2's side. I just put my workflow into:
> {code:java}
> ./files/hadoop-agents/config.text.yml.v1
> {code}
>
> Here is the exception I get when the agent tries to pull the config:
> {code:java}
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
> INFO: Using default applicationContext
> Jan 24, 2018 8:45:03 PM
>
[jira] [Commented] (MINIFI-431) C2 - Exception while agent tries to pull the config
[
https://issues.apache.org/jira/browse/MINIFI-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338209#comment-16338209
]
Pierre Villard commented on MINIFI-431:
---
I believe problem comes from here:
{noformat}
[INFO] | +- org.apache.nifi.registry:nifi-registry-client:jar:0.1.0:compile
[INFO] | | +-
org.apache.nifi.registry:nifi-registry-security-utils:jar:0.1.0:compile
[INFO] | | +-
org.glassfish.jersey.media:jersey-media-json-jackson:jar:2.26:compile
[INFO] | | | \-
org.glassfish.jersey.ext:jersey-entity-filtering:jar:2.26:compile
[INFO] | | \- org.glassfish.jersey.inject:jersey-hk2:jar:2.26:compile
[INFO] | | \- org.glassfish.hk2:hk2-locator:jar:2.5.0-b42:compile
[INFO] | | +-
org.glassfish.hk2.external:aopalliance-repackaged:jar:2.5.0-b42:compile
[INFO] | | +- org.glassfish.hk2:hk2-api:jar:2.5.0-b42:compile
[INFO] | | | \- javax.inject:javax.inject:jar:1:compile
[INFO] | | +- org.glassfish.hk2:hk2-utils:jar:2.5.0-b42:compile
[INFO] | | \- org.javassist:javassist:jar:3.22.0-CR2:compile{noformat}
Here are the notable differences I see between the two versions in ./lib
directory (jars that are unique to the version):
|*0.3.0 version*|*0.4.0 version*|
|h2-1.3.176.jar|aopalliance-repackaged-2.5.0-b42.jar|
|jackson-core-asl-1.9.13.jar|hk2-api-2.5.0-b42.jar|
|jackson-jaxrs-1.9.2.jar|hk2-locator-2.5.0-b42.jar|
|jackson-mapper-asl-1.9.13.jar|hk2-utils-2.5.0-b42.jar|
|jackson-xc-1.9.2.jar|jackson-module-jaxb-annotations-2.9.1.jar|
|jaxb-api-2.2.2.jar|javassist-3.22.0-CR2.jar|
|jaxb-impl-2.2.3-1.jar|javax.annotation-api-1.2.jar|
|jersey-client-1.19.jar|javax.inject-1.jar|
|jersey-json-1.19.jar|javax.inject-2.5.0-b42.jar|
|jettison-1.1.jar|javax.ws.rs-api-2.1.jar|
|stax-api-1.0-2.jar|jersey-client-2.26.jar|
|swagger-annotations-1.5.3-M1.jar|jersey-common-2.26.jar|
| |jersey-entity-filtering-2.26.jar|
| |jersey-hk2-2.26.jar|
| |jersey-media-json-jackson-2.26.jar|
| |swagger-annotations-1.5.16.jar|
| |validation-api-2.0.0.Final.jar|
By removing the file javax.ws.rs-api-2.1.jar from lib directory in 0.4.0
version, it works as expected.
> C2 - Exception while agent tries to pull the config
> ---
>
> Key: MINIFI-431
> URL: https://issues.apache.org/jira/browse/MINIFI-431
> Project: Apache NiFi MiNiFi
> Issue Type: Bug
> Components: Command and Control
>Affects Versions: 0.4.0
>Reporter: Pierre Villard
>Priority: Major
>
> My setup:
> MiNiFi Java agent - 0.4.0
> C2 server - 0.4.0
> On agent's side:
> {code:java}
> nifi.minifi.notifier.ingestors=org.apache.nifi.minifi.bootstrap.configuration.ingestors.PullHttpChangeIngestor
> # Hostname on which to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.hostname=localhost
> # Port on which to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.port=10080
> # Path to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.path=/c2/config
> # Query string to pull configurations with
> nifi.minifi.notifier.ingestors.pull.http.query=class=hadoop-agents
> # Period on which to pull configurations from, defaults to 5 minutes if
> commented out
> nifi.minifi.notifier.ingestors.pull.http.period.ms=6
> {code}
>
> I kept everything default on C2's side. I just put my workflow into:
> {code:java}
> ./files/hadoop-agents/config.text.yml.v1
> {code}
>
> Here is the exception I get when the agent tries to pull the config:
> {code:java}
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
> INFO: Using default applicationContext
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.spi.spring.container.SpringComponentProviderFactory
> registerSpringBeans
> INFO: Registering Spring bean, configService, of type
> org.apache.nifi.minifi.c2.service.ConfigService as a root resource class
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
> INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25
> AM'
> 2018-01-24 20:45:04,397 DEBUG [qtp1018298342-12]
> o.a.n.m.c.s.a.C2AnonymousAuthenticationFilter Populated SecurityContextHolder
> with anonymous token:
> 'org.apache.nifi.minifi.c2.security.authentication.C2AuthenticationToken@52a3d06d:
> Principal: anonymous; Credentials: [PROTECTED]; Authenticated: true;
> Details: null; Granted Authorities: ROLE_ANONYMOUS'
> 2018-01-24 20:45:04,403 WARN [qtp1018298342-12]
> org.eclipse.jetty.server.HttpChannel /c2/config
> java.lang.AbstractMethodError:
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:120)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:669)
> at
[jira] [Commented] (MINIFI-431) C2 - Exception while agent tries to pull the config
[
https://issues.apache.org/jira/browse/MINIFI-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338166#comment-16338166
]
Pierre Villard commented on MINIFI-431:
---
I confirm exact same configuration with C2 v0.3.0 is working. Will compare
dependencies to track down the problem.
> C2 - Exception while agent tries to pull the config
> ---
>
> Key: MINIFI-431
> URL: https://issues.apache.org/jira/browse/MINIFI-431
> Project: Apache NiFi MiNiFi
> Issue Type: Bug
> Components: Command and Control
>Affects Versions: 0.4.0
>Reporter: Pierre Villard
>Priority: Major
>
> My setup:
> MiNiFi Java agent - 0.4.0
> C2 server - 0.4.0
> On agent's side:
> {code:java}
> nifi.minifi.notifier.ingestors=org.apache.nifi.minifi.bootstrap.configuration.ingestors.PullHttpChangeIngestor
> # Hostname on which to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.hostname=localhost
> # Port on which to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.port=10080
> # Path to pull configurations from
> nifi.minifi.notifier.ingestors.pull.http.path=/c2/config
> # Query string to pull configurations with
> nifi.minifi.notifier.ingestors.pull.http.query=class=hadoop-agents
> # Period on which to pull configurations from, defaults to 5 minutes if
> commented out
> nifi.minifi.notifier.ingestors.pull.http.period.ms=6
> {code}
>
> I kept everything default on C2's side. I just put my workflow into:
> {code:java}
> ./files/hadoop-agents/config.text.yml.v1
> {code}
>
> Here is the exception I get when the agent tries to pull the config:
> {code:java}
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
> INFO: Using default applicationContext
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.spi.spring.container.SpringComponentProviderFactory
> registerSpringBeans
> INFO: Registering Spring bean, configService, of type
> org.apache.nifi.minifi.c2.service.ConfigService as a root resource class
> Jan 24, 2018 8:45:03 PM
> com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
> INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25
> AM'
> 2018-01-24 20:45:04,397 DEBUG [qtp1018298342-12]
> o.a.n.m.c.s.a.C2AnonymousAuthenticationFilter Populated SecurityContextHolder
> with anonymous token:
> 'org.apache.nifi.minifi.c2.security.authentication.C2AuthenticationToken@52a3d06d:
> Principal: anonymous; Credentials: [PROTECTED]; Authenticated: true;
> Details: null; Granted Authorities: ROLE_ANONYMOUS'
> 2018-01-24 20:45:04,403 WARN [qtp1018298342-12]
> org.eclipse.jetty.server.HttpChannel /c2/config
> java.lang.AbstractMethodError:
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:120)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:669)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:316)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:122)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.apache.nifi.minifi.c2.security.authentication.X509AuthenticationFilter.doFilter(X509AuthenticationFilter.java:43)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:213)
> at
> org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:176)
> at
> org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
> at
>
[jira] [Created] (MINIFI-431) C2 - Exception while agent tries to pull the config
Pierre Villard created MINIFI-431:
-
Summary: C2 - Exception while agent tries to pull the config
Key: MINIFI-431
URL: https://issues.apache.org/jira/browse/MINIFI-431
Project: Apache NiFi MiNiFi
Issue Type: Bug
Components: Command and Control
Affects Versions: 0.4.0
Reporter: Pierre Villard
My setup:
MiNiFi Java agent - 0.4.0
C2 server - 0.4.0
On agent's side:
{code:java}
nifi.minifi.notifier.ingestors=org.apache.nifi.minifi.bootstrap.configuration.ingestors.PullHttpChangeIngestor
# Hostname on which to pull configurations from
nifi.minifi.notifier.ingestors.pull.http.hostname=localhost
# Port on which to pull configurations from
nifi.minifi.notifier.ingestors.pull.http.port=10080
# Path to pull configurations from
nifi.minifi.notifier.ingestors.pull.http.path=/c2/config
# Query string to pull configurations with
nifi.minifi.notifier.ingestors.pull.http.query=class=hadoop-agents
# Period on which to pull configurations from, defaults to 5 minutes if
commented out
nifi.minifi.notifier.ingestors.pull.http.period.ms=6
{code}
I kept everything default on C2's side. I just put my workflow into:
{code:java}
./files/hadoop-agents/config.text.yml.v1
{code}
Here is the exception I get when the agent tries to pull the config:
{code:java}
Jan 24, 2018 8:45:03 PM
com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jan 24, 2018 8:45:03 PM
com.sun.jersey.spi.spring.container.SpringComponentProviderFactory
registerSpringBeans
INFO: Registering Spring bean, configService, of type
org.apache.nifi.minifi.c2.service.ConfigService as a root resource class
Jan 24, 2018 8:45:03 PM
com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
2018-01-24 20:45:04,397 DEBUG [qtp1018298342-12]
o.a.n.m.c.s.a.C2AnonymousAuthenticationFilter Populated SecurityContextHolder
with anonymous token:
'org.apache.nifi.minifi.c2.security.authentication.C2AuthenticationToken@52a3d06d:
Principal: anonymous; Credentials: [PROTECTED]; Authenticated: true; Details:
null; Granted Authorities: ROLE_ANONYMOUS'
2018-01-24 20:45:04,403 WARN [qtp1018298342-12]
org.eclipse.jetty.server.HttpChannel /c2/config
java.lang.AbstractMethodError:
javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:120)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:669)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:316)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:122)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at
org.apache.nifi.minifi.c2.security.authentication.X509AuthenticationFilter.doFilter(X509AuthenticationFilter.java:43)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:213)
at
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:176)
at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1613)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:541)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at
[jira] [Created] (MINIFI-97) Error when trying to stop already stopped MiNiFi
Pierre Villard created MINIFI-97:
Summary: Error when trying to stop already stopped MiNiFi
Key: MINIFI-97
URL: https://issues.apache.org/jira/browse/MINIFI-97
Project: Apache NiFi MiNiFi
Issue Type: Bug
Affects Versions: cpp-0.0.1
Environment: CentOS 7
Reporter: Pierre Villard
Priority: Minor
If calling
{noformat}
./minifi.sh stop
{noformat}
when MiNiFi is already stopped, the process will kill PID -1. When connected to
the host by SSH, I get disconnected from the host.
I'd suggest replacing:
{noformat}
if [ $(active_pid ${saved_pid}) -ne 0 ]; then
{noformat}
by
{noformat}
if [ $(active_pid ${saved_pid}) -le 0 ]; then
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MINIFI-83) Externalize configuration of logs
[ https://issues.apache.org/jira/browse/MINIFI-83?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447143#comment-15447143 ] Pierre Villard commented on MINIFI-83: -- It would also be nice to be able to configure the log file location. At the moment, it seems that the log file is located in the directory from where MiNiFi is launched. > Externalize configuration of logs > - > > Key: MINIFI-83 > URL: https://issues.apache.org/jira/browse/MINIFI-83 > Project: Apache NiFi MiNiFi > Issue Type: Improvement > Components: C++, Core Framework >Reporter: Aldrin Piri > > spdlog has a number of options that provide configuration similar to logback > in terms of rotating by date and size. These are not inherently configurable > through an existing file however. It would be a nice improvement to expose > these options in a friendly way such that a user can dictate what their > preferences are in terms of size vs time as well as the amount of disk > resources they are willing to allocate to such logs. > https://github.com/gabime/spdlog/wiki/2.-Creating-loggers -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (NIFI-2178) ReplaceTextWithMapping ignores escaped '\' character
[
https://issues.apache.org/jira/browse/NIFI-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard reopened NIFI-2178:
--
> ReplaceTextWithMapping ignores escaped '\' character
>
>
> Key: NIFI-2178
> URL: https://issues.apache.org/jira/browse/NIFI-2178
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0
>Reporter: Cole Mackenzie
>Priority: Minor
> Labels: easyfix
> Fix For: 1.0.0
>
>
> When using the ReplaceTextWithMapping processor I get a result that does not
> follow the mapping.
> Ex.
> Incoming Flow:
> {noformat}
> 9900
> {noformat}
> Mappings file
> {noformat}
> 9900 Latte\\ Macchiato
> {noformat}
> Outgoing Flow:
> {noformat}
> Latte Macchiato
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (NIFI-2178) ReplaceTextWithMapping ignores escaped '\' character
[
https://issues.apache.org/jira/browse/NIFI-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-2178.
--
Resolution: Not A Bug
> ReplaceTextWithMapping ignores escaped '\' character
>
>
> Key: NIFI-2178
> URL: https://issues.apache.org/jira/browse/NIFI-2178
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0
>Reporter: Cole Mackenzie
>Priority: Minor
> Labels: easyfix
> Fix For: 1.0.0
>
>
> When using the ReplaceTextWithMapping processor I get a result that does not
> follow the mapping.
> Ex.
> Incoming Flow:
> {noformat}
> 9900
> {noformat}
> Mappings file
> {noformat}
> 9900 Latte\\ Macchiato
> {noformat}
> Outgoing Flow:
> {noformat}
> Latte Macchiato
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-2074) Add checkstyle rule to normalize imports order
[
https://issues.apache.org/jira/browse/NIFI-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-2074:
-
Attachment: checkstyle_eclipse.txt
Just had a look.
Eclipse oriented:
{code:xml}
https://issues.apache.org/jira/browse/NIFI-2074
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Documentation & Website, Tools and Build
>Reporter: Pierre Villard
>Priority: Minor
> Labels: build, maven
> Attachments: checkstyle_eclipse.txt
>
>
> In order to avoid PR where imports order is uselessly modified because of the
> IDE formatter, it could be interesting to add a checkstyle rule to normalize
> the imports order so that when running -Pcontrib-check such a PR is not
> accepted.
> Since default formatting seems to be varying depending of the IDE, this
> should be documented
> (https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-CodeStyle).
> Checkstyle options are documented here:
> http://checkstyle.sourceforge.net/config_imports.html#CustomImportOrder
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-2074) Add checkstyle rule to normalize imports order
Pierre Villard created NIFI-2074: Summary: Add checkstyle rule to normalize imports order Key: NIFI-2074 URL: https://issues.apache.org/jira/browse/NIFI-2074 Project: Apache NiFi Issue Type: Improvement Components: Documentation & Website, Tools and Build Reporter: Pierre Villard Priority: Minor In order to avoid PR where imports order is uselessly modified because of the IDE formatter, it could be interesting to add a checkstyle rule to normalize the imports order so that when running -Pcontrib-check such a PR is not accepted. Since default formatting seems to be varying depending of the IDE, this should be documented (https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-CodeStyle). Checkstyle options are documented here: http://checkstyle.sourceforge.net/config_imports.html#CustomImportOrder -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2066) Fix concurrency in SNMP processor unit tests
[ https://issues.apache.org/jira/browse/NIFI-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15342230#comment-15342230 ] Pierre Villard commented on NIFI-2066: -- Thanks [~ozhurakousky], I used your example! > Fix concurrency in SNMP processor unit tests > > > Key: NIFI-2066 > URL: https://issues.apache.org/jira/browse/NIFI-2066 > Project: Apache NiFi > Issue Type: Test > Components: Extensions >Affects Versions: 1.0.0, 0.7.0 >Reporter: Pierre Villard > > Travis build is currently failing because of unit testing in SNMP processors. > It seems to be explained by concurrent executions of the unit tests and the > unavailability of ports already in use. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2066) Fix concurrency in SNMP processor unit tests
Pierre Villard created NIFI-2066: Summary: Fix concurrency in SNMP processor unit tests Key: NIFI-2066 URL: https://issues.apache.org/jira/browse/NIFI-2066 Project: Apache NiFi Issue Type: Test Components: Extensions Affects Versions: 1.0.0, 0.7.0 Reporter: Pierre Villard Travis build is currently failing because of unit testing in SNMP processors. It seems to be explained by concurrent executions of the unit tests and the unavailability of ports already in use. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2060) HDFS Inotify processor: unit tests failing with hadoop 2.7.x
Pierre Villard created NIFI-2060: Summary: HDFS Inotify processor: unit tests failing with hadoop 2.7.x Key: NIFI-2060 URL: https://issues.apache.org/jira/browse/NIFI-2060 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 0.7.0 Reporter: Pierre Villard Assignee: Pierre Villard Both $ mvn clean install -Dhadoop.version=2.7.0 and $ mvn clean install -DskipTests -Pmapr -Dhadoop.version=2.7.0-mapr-1506 result in: [ERROR] COMPILATION ERROR : [INFO] - [ERROR] /home/amiranda/development/nifi/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/test/java/org/apache/nifi/processors/hadoop/inotify/TestGetHDFSEvents.java:[198,38] error: incompatible types: String cannot be converted to Builder [ERROR] /home/amiranda/development/nifi/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/test/java/org/apache/nifi/processors/hadoop/inotify/TestGetHDFSEvents.java:[199,38] error: incompatible types: String cannot be converted to Builder [ERROR] /home/amiranda/development/nifi/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/test/java/org/apache/nifi/processors/hadoop/inotify/TestGetHDFSEvents.java:[200,38] error: incompatible types: String cannot be converted to Builder [ERROR] /home/amiranda/development/nifi/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/test/java/org/apache/nifi/processors/hadoop/inotify/util/EventTestUtils.java:[51,37] error: incompatible types: String cannot be converted to Builder [ERROR] /home/amiranda/development/nifi/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/test/java/org/apache/nifi/processors/hadoop/inotify/util/EventTestUtils.java:[55,15] error: constructor RenameEvent in class RenameEvent cannot be applied to given types; [ERROR] required: Builder found: String,String,long reason: actual and formal argument lists differ in length /home/amiranda/development/nifi/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/test/java/org/apache/nifi/processors/hadoop/inotify/util/EventTestUtils.java:[75,15] error: constructor UnlinkEvent in class UnlinkEvent cannot be applied to given types; [INFO] 6 errors -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1537) Add SNMP processors
[ https://issues.apache.org/jira/browse/NIFI-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339115#comment-15339115 ] Pierre Villard commented on NIFI-1537: -- Thanks [~ozhurakousky] for taking care of this! > Add SNMP processors > --- > > Key: NIFI-1537 > URL: https://issues.apache.org/jira/browse/NIFI-1537 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 0.5.0 >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Minor > Labels: snmp > Fix For: 1.0.0, 0.7.0 > > > Add SNMP processors: > - GetSNMP to allow "get"/"walk" > - SetSNMP to allow "set" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-1779) SNMP
[ https://issues.apache.org/jira/browse/NIFI-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-1779. -- Resolution: Fixed > SNMP > - > > Key: NIFI-1779 > URL: https://issues.apache.org/jira/browse/NIFI-1779 > Project: Apache NiFi > Issue Type: Wish > Components: Core Framework >Affects Versions: 0.6.0 > Environment: n/a >Reporter: Manoj Kalikodanveetil > > Hi, > In one of NiFi 0.6.0 release notes it was mentioned that SNMP input handler > being added. But I don't see that in 0.6. Is that backed out? > Or is the plan to make use of Camel? > thanks > Manoj -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-1037) Hdfs Inotify processor
[ https://issues.apache.org/jira/browse/NIFI-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-1037. -- Resolution: Fixed Fix Version/s: 0.7.0 1.0.0 > Hdfs Inotify processor > -- > > Key: NIFI-1037 > URL: https://issues.apache.org/jira/browse/NIFI-1037 > Project: Apache NiFi > Issue Type: New Feature >Reporter: nicolas maillard >Assignee: Josh Meyer >Priority: Minor > Fix For: 1.0.0, 0.7.0 > > > HDFS has an Inotify interface that enables to access the HDFS edit stream. > https://issues.apache.org/jira/browse/HDFS-6634 > Creating a processor to listen in and get notifications either for select > directories or select actions would have many applications. > - Stream to a search engine the activity on HDFS > - Wait for specific actions or files to trigger workflows, like duplication > to other clusters > - Validate ingestion processes > etc.. > probably more I don't think of. > I have a first working beta version that needs to evolve > it reuses the Hadoop-nar-bundle > Needs a HDFS 2.7 dependency currently done through editing the Hadop-lib > bundle > let me know if this idea makes sense and would be of interest to the community > would love to contribute the idea -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1832) Testing EL properties with AllowableValues
[
https://issues.apache.org/jira/browse/NIFI-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15335516#comment-15335516
]
Pierre Villard commented on NIFI-1832:
--
Hey [~aldrin],
I don't know if [~simonellistonball] is OK regarding the JIRA itself and last
comments from [~joewitt]. In any case the PR alone won't be working (from a UI
point of view) so I'll close it right now and if there are new developments on
this JIRA, I'll have another look. Thanks!
> Testing EL properties with AllowableValues
> --
>
> Key: NIFI-1832
> URL: https://issues.apache.org/jira/browse/NIFI-1832
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 0.6.1
> Environment: Testing
>Reporter: Simon Elliston Ball
>Assignee: Pierre Villard
>Priority: Minor
>
> I’ve come across an interesting problem with MockFlowFile while testing a
> custom processor. My property has an AllowableValue list, and supports
> expression language. The test uses:
> runner.setProperty(PROPERTY_REF, "${attribute.name}”);
> However, the test fails on validation of in the MockFlowFile with the
> unevaluated version of the EL invalid against the allowed values list.
> 'Property' validated against '${attribute.name}' is invalid because Given
> value is not found in allowed set ...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (NIFI-1975) Processor to Parse .evtx files
[ https://issues.apache.org/jira/browse/NIFI-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-1975. -- Resolution: Fixed Fix Version/s: 1.0.0 > Processor to Parse .evtx files > -- > > Key: NIFI-1975 > URL: https://issues.apache.org/jira/browse/NIFI-1975 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Bryan Rosander > Fix For: 1.0.0 > > > Windows event logs are stored in .evtx format as-of Windows Vista. If we > port the pure python implementation of an evtx parser at > https://github.com/williballenthin/python-evtx to Java, we should be able to > ingest those files in NiFi on any operating system > These files are located in C:\Windows\System32\winevt\Logs unless exported > elsewhere. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1620) Allow empty Content-Type in InvokeHTTP processor
[
https://issues.apache.org/jira/browse/NIFI-1620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1504#comment-1504
]
Pierre Villard commented on NIFI-1620:
--
Thanks for the review [~adamtaft]! I'll update the PR to take into account your
comments.
Regarding the proposition to not transmit Content-Type when there is no
content, I think this could possibly break backward compatibility in case some
applications are expecting content type in any case (even with an empty
content), no?
I don't care if this is postponed, there is no blocking issue on my side but,
just as a remark, some flows I have worked on seem to be impossible without
this change (the main one is probably the one dealing with the Dropbox API).
> Allow empty Content-Type in InvokeHTTP processor
>
>
> Key: NIFI-1620
> URL: https://issues.apache.org/jira/browse/NIFI-1620
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 0.5.1
>Reporter: Pierre Villard
>Assignee: Pierre Villard
> Fix For: 1.0.0, 0.7.0
>
>
> External API could expect a POST request without a Content-Type property or
> at least with this property empty:
> *Error example:*
> {quote}
> You provided a non-empty HTTP "Content-Type" header ("application/json").
> This API function requires that the header be missing or empty.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1152) Mock Framework allows processor to route to Relationships that the Processor does not support
[ https://issues.apache.org/jira/browse/NIFI-1152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332103#comment-15332103 ] Pierre Villard commented on NIFI-1152: -- [~puspendu.baner...@gmail.com] Yes you can re-submit it and I'll review it to check if this can be merged for 0.7 release. > Mock Framework allows processor to route to Relationships that the Processor > does not support > - > > Key: NIFI-1152 > URL: https://issues.apache.org/jira/browse/NIFI-1152 > Project: Apache NiFi > Issue Type: Bug > Components: Tools and Build >Reporter: Mark Payne > Labels: beginner, newbie > Fix For: 1.0.0 > > Attachments: > 0001-Fix-for-NIFI-1838-NIFI-1152-Code-modification-for-ty.patch > > > If a processor calls ProcessSession.transfer(flowFile, > NON_EXISTENT_RELATIONSHIP) the NiFi framework will throw a > FlowFileHandlingException. However, the Mock Framework simply allows it and > does not throw any sort of Exception. This needs to be addressed so that the > Mock framework functions the same way as the normal NiFi framework. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1152) Mock Framework allows processor to route to Relationships that the Processor does not support
[ https://issues.apache.org/jira/browse/NIFI-1152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331312#comment-15331312 ] Pierre Villard commented on NIFI-1152: -- [~puspendu.baner...@gmail.com] PR has been closed/deleted. Consequently, I move it to 1.0.0 version. > Mock Framework allows processor to route to Relationships that the Processor > does not support > - > > Key: NIFI-1152 > URL: https://issues.apache.org/jira/browse/NIFI-1152 > Project: Apache NiFi > Issue Type: Bug > Components: Tools and Build >Reporter: Mark Payne > Labels: beginner, newbie > Fix For: 1.0.0 > > Attachments: > 0001-Fix-for-NIFI-1838-NIFI-1152-Code-modification-for-ty.patch > > > If a processor calls ProcessSession.transfer(flowFile, > NON_EXISTENT_RELATIONSHIP) the NiFi framework will throw a > FlowFileHandlingException. However, the Mock Framework simply allows it and > does not throw any sort of Exception. This needs to be addressed so that the > Mock framework functions the same way as the normal NiFi framework. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1998) Upgrade Cassandra driver to 3.x
[ https://issues.apache.org/jira/browse/NIFI-1998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-1998: - Affects Version/s: 0.6.1 > Upgrade Cassandra driver to 3.x > --- > > Key: NIFI-1998 > URL: https://issues.apache.org/jira/browse/NIFI-1998 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 0.6.1 >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.0.0, 0.7.0 > > > The Cassandra processors use the 2.1.9 driver, which is backwards-compatible > to 1.x, but not forward-compatible to 3.x. The latest driver at the time of > this writing is 3.0.2, which is fully backwards-compatible. > Upgrading the driver, although it will involve API changes, will enable NiFi > users to interact with any Cassandra cluster of version 1.x, 2.x, or 3.x. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (NIFI-1895) PutHBaseJSON processor treats all Values as Strings
[ https://issues.apache.org/jira/browse/NIFI-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard reopened NIFI-1895: -- > PutHBaseJSON processor treats all Values as Strings > --- > > Key: NIFI-1895 > URL: https://issues.apache.org/jira/browse/NIFI-1895 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 0.6.1 >Reporter: Ryan Templeton > > line 184 of PutHBaseJSON.java treats all JsonNode values as strings by > calling the .asText() method. We are working with using this processor to > load IoT time series data and this causes issues in HBase with > timestamps/numerics not getting sorted correctly. > The operator should inspect the node value to determine type and convert as > such. > Numeric integral - Long (assumes widest type) > Numeric not integral - Double (assumes widest type) > Logical - Boolean > everything else (including current Complex Type logic) - String -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1816) HandleHTTPResponse Processor Provenance
[ https://issues.apache.org/jira/browse/NIFI-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318701#comment-15318701 ] Pierre Villard commented on NIFI-1816: -- No problem! > HandleHTTPResponse Processor Provenance > --- > > Key: NIFI-1816 > URL: https://issues.apache.org/jira/browse/NIFI-1816 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.6.1 > Environment: Centos6 >Reporter: Sam Hjelmfelt >Assignee: Pierre Villard >Priority: Minor > Fix For: 1.0.0 > > > The HandleHTTPResponse processor only creates provenance events for > auto-terminated relationships (i.e. drop events). It should also create send > events. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1816) HandleHTTPResponse Processor Provenance
[ https://issues.apache.org/jira/browse/NIFI-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318550#comment-15318550 ] Pierre Villard commented on NIFI-1816: -- [~JPercivall] It seems this has not been merged in 0.x, I don't think it is a problem but just want to confirm with you in case this should be in 0.7.0. > HandleHTTPResponse Processor Provenance > --- > > Key: NIFI-1816 > URL: https://issues.apache.org/jira/browse/NIFI-1816 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.6.1 > Environment: Centos6 >Reporter: Sam Hjelmfelt >Assignee: Pierre Villard >Priority: Minor > Fix For: 1.0.0 > > > The HandleHTTPResponse processor only creates provenance events for > auto-terminated relationships (i.e. drop events). It should also create send > events. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1959) TailFile not ingesting data when tailed file is moved with no rolling pattern
Pierre Villard created NIFI-1959: Summary: TailFile not ingesting data when tailed file is moved with no rolling pattern Key: NIFI-1959 URL: https://issues.apache.org/jira/browse/NIFI-1959 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 0.6.1 Reporter: Pierre Villard Assignee: Pierre Villard Priority: Minor In case "Rolling Filename Pattern" is not set by user and if the tailed file is moved, then the new file will not be tailed. Besides, in such case, the processor will be endlessly triggered without ingesting data: it creates a lot of tasks and consumes CPU. The reason is it never goes in if statement L448. A solution is to look at size() and lastUpdated() of the tailed file to detect a "rollover". However it won't allow the processor to ingest the potential data added in the tailed file just before being moved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1945) org.apache.nifi.processors.standard.HashContent has incorrect Attribute description
[
https://issues.apache.org/jira/browse/NIFI-1945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310947#comment-15310947
]
Pierre Villard commented on NIFI-1945:
--
Merged in both 0.x and master.
Thanks!
> org.apache.nifi.processors.standard.HashContent has incorrect Attribute
> description
> ---
>
> Key: NIFI-1945
> URL: https://issues.apache.org/jira/browse/NIFI-1945
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 0.6.1
>Reporter: Andre
>Priority: Trivial
> Fix For: 1.0.0, 0.7.0
>
>
> org.apache.nifi.processors.standard.HashContent help contents states:
> {code}
> This Processor adds an attribute whose value is the
> result of Hashing the existing FlowFile attributes. The name of this
> attribute is specified by the property
> {code}
> Shouldn't the correct be {{Hashing the existing FlowFile contents}} ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-1945) org.apache.nifi.processors.standard.HashContent has incorrect Attribute description
[
https://issues.apache.org/jira/browse/NIFI-1945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-1945:
-
Fix Version/s: 0.7.0
1.0.0
> org.apache.nifi.processors.standard.HashContent has incorrect Attribute
> description
> ---
>
> Key: NIFI-1945
> URL: https://issues.apache.org/jira/browse/NIFI-1945
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 0.6.1
>Reporter: Andre
>Priority: Trivial
> Fix For: 1.0.0, 0.7.0
>
>
> org.apache.nifi.processors.standard.HashContent help contents states:
> {code}
> This Processor adds an attribute whose value is the
> result of Hashing the existing FlowFile attributes. The name of this
> attribute is specified by the property
> {code}
> Shouldn't the correct be {{Hashing the existing FlowFile contents}} ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-1945) org.apache.nifi.processors.standard.HashContent has incorrect Attribute description
[
https://issues.apache.org/jira/browse/NIFI-1945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-1945:
-
Affects Version/s: (was: 1.0.0)
0.6.1
> org.apache.nifi.processors.standard.HashContent has incorrect Attribute
> description
> ---
>
> Key: NIFI-1945
> URL: https://issues.apache.org/jira/browse/NIFI-1945
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 0.6.1
>Reporter: Andre
>Priority: Trivial
> Fix For: 1.0.0, 0.7.0
>
>
> org.apache.nifi.processors.standard.HashContent help contents states:
> {code}
> This Processor adds an attribute whose value is the
> result of Hashing the existing FlowFile attributes. The name of this
> attribute is specified by the property
> {code}
> Shouldn't the correct be {{Hashing the existing FlowFile contents}} ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (NIFI-1945) org.apache.nifi.processors.standard.HashContent has incorrect Attribute description
[
https://issues.apache.org/jira/browse/NIFI-1945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-1945.
--
Resolution: Fixed
> org.apache.nifi.processors.standard.HashContent has incorrect Attribute
> description
> ---
>
> Key: NIFI-1945
> URL: https://issues.apache.org/jira/browse/NIFI-1945
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 0.6.1
>Reporter: Andre
>Priority: Trivial
> Fix For: 1.0.0, 0.7.0
>
>
> org.apache.nifi.processors.standard.HashContent help contents states:
> {code}
> This Processor adds an attribute whose value is the
> result of Hashing the existing FlowFile attributes. The name of this
> attribute is specified by the property
> {code}
> Shouldn't the correct be {{Hashing the existing FlowFile contents}} ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-1942) Create a processor to validate CSV against a user-supplied schema
Pierre Villard created NIFI-1942: Summary: Create a processor to validate CSV against a user-supplied schema Key: NIFI-1942 URL: https://issues.apache.org/jira/browse/NIFI-1942 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Pierre Villard Assignee: Pierre Villard Priority: Minor In order to extend the set of "quality control" processors, it would be interesting to have a processor validating CSV formatted flow files against a user-specified schema. Flow file validated against schema would be routed to "valid" relationship although flow file not validated against schema would be routed to "invalid" relationship. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1213) Allow Mock Framework to register "FlowFile Assertions"
[
https://issues.apache.org/jira/browse/NIFI-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286230#comment-15286230
]
Pierre Villard commented on NIFI-1213:
--
[~ozhurakousky] Done. Thanks a lot!
> Allow Mock Framework to register "FlowFile Assertions"
> --
>
> Key: NIFI-1213
> URL: https://issues.apache.org/jira/browse/NIFI-1213
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Tools and Build
>Reporter: Mark Payne
>Assignee: Pierre Villard
> Fix For: 1.0.0, 0.7.0
>
>
> Often times, unit tests will invoke TestRunner.run, and then iterate through
> the output FlowFiles, ensuring that a particular attribute exists, or that an
> attribute is equal to some value.
> As a convenience, we should provide a mechanism to indicate that all output
> FlowFiles (or all FlowFiles routed to a given relationship) meet some
> criteria.
> For example:
> {code}
> TestRunner.assertAllFlowFilesContainAttribute( String attributeName );
> {code}
> {code}
> TestRunner.assertAllFlowFilesContainAttribute( Relationship relationship,
> String attributeName );
> {code}
> Additionally, we should consider allowing a "callback" mechanism:
> {code}
> TestRunner.assertAllFlowFiles( FlowFileValidator validator );
> {code}
> {code}
> TestRunner.assertAllFlowFiles( Relationship relationship, FlowFileValidator
> validator );
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (NIFI-1806) Add UTF-8 for file enconding in unit tests during maven build
[
https://issues.apache.org/jira/browse/NIFI-1806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-1806.
--
Resolution: Not A Bug
> Add UTF-8 for file enconding in unit tests during maven build
> -
>
> Key: NIFI-1806
> URL: https://issues.apache.org/jira/browse/NIFI-1806
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 0.6.1
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
> Labels: build, maven, test
>
> Following discussion here :
> https://github.com/apache/nifi/pull/366#discussion_r60829511
> For some tests, it seems to be necessary to add the following in pom file:
> {code:title=pom.xml|borderStyle=solid}
>
>
>
> org.apache.maven.plugins
> maven-surefire-plugin
>
> -Dfile.encoding=UTF-8
>
>
>
>
> {code}
> Ideally this could be added in the parent pom but it must be ensured that
> this would not affect other components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1806) Add UTF-8 for file enconding in unit tests during maven build
[
https://issues.apache.org/jira/browse/NIFI-1806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15269350#comment-15269350
]
Pierre Villard commented on NIFI-1806:
--
[~ozhurakousky] [~joewitt] I agree, I am sure we will find a way to get the
related unit test working.
[~ozhurakousky] I saw that you updated the Kafka PR, I will give it a try
tonight or tomorrow.
> Add UTF-8 for file enconding in unit tests during maven build
> -
>
> Key: NIFI-1806
> URL: https://issues.apache.org/jira/browse/NIFI-1806
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 0.6.1
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
> Labels: build, maven, test
>
> Following discussion here :
> https://github.com/apache/nifi/pull/366#discussion_r60829511
> For some tests, it seems to be necessary to add the following in pom file:
> {code:title=pom.xml|borderStyle=solid}
>
>
>
> org.apache.maven.plugins
> maven-surefire-plugin
>
> -Dfile.encoding=UTF-8
>
>
>
>
> {code}
> Ideally this could be added in the parent pom but it must be ensured that
> this would not affect other components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-1840) Add compression type property in Kite processors
Pierre Villard created NIFI-1840: Summary: Add compression type property in Kite processors Key: NIFI-1840 URL: https://issues.apache.org/jira/browse/NIFI-1840 Project: Apache NiFi Issue Type: Improvement Components: Extensions Affects Versions: 0.6.1 Reporter: Pierre Villard Following https://community.hortonworks.com/questions/31017/unable-to-turn-off-snappy-compression-in-nifi-putk.html By default Kite uses Snappy compression. It would be interesting to have the ability to change this compression. http://kitesdk.org/docs/1.0.0/API-Overview.html http://kitesdk.org/docs/1.0.0/apidocs/org/kitesdk/data/DatasetDescriptor.Builder.html#compressionType(java.lang.String) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1789) TestPutUDP testUnkownHostname fails because all hostnames resolve
[
https://issues.apache.org/jira/browse/NIFI-1789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-1789:
-
Fix Version/s: 1.0.0
> TestPutUDP testUnkownHostname fails because all hostnames resolve
> -
>
> Key: NIFI-1789
> URL: https://issues.apache.org/jira/browse/NIFI-1789
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 0.6.1
>Reporter: Andy LoPresto
> Fix For: 1.0.0, 0.7.0
>
>
> On some systems, arbitrary hostnames (e.g. {{sgfgfdgs}}) resolves to a valid
> IP address depending on the DNS resolution scheme. Therefore, the test, which
> assumes that the host does not resolve, fails.
> {code}
> hw12203:/Users/alopresto (master) alopresto
> 0s @ 20:10:32 $ nslookup asdfsdfsdf
> Server: 192.168.1.1
> Address: 192.168.1.1#53
> Non-authoritative answer:
> Name: asdfsdfsdf
> Address: 198.105.254.228
> Name: asdfsdfsdf
> Address: 198.105.244.228
> hw12203:/Users/alopresto (master) alopresto
> 39s @ 20:12:24 $ dig asdfasdf
> ; <<>> DiG 9.8.3-P1 <<>> asdfasdf
> ;; global options: +cmd
> ;; Got answer:
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31295
> ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
> ;; QUESTION SECTION:
> ;asdfasdf.IN A
> ;; ANSWER SECTION:
> asdfasdf. 10 IN A 198.105.254.228
> asdfasdf. 10 IN A 198.105.244.228
> ;; Query time: 81 msec
> ;; SERVER: 192.168.1.1#53(192.168.1.1)
> ;; WHEN: Tue Apr 19 20:12:30 2016
> ;; MSG SIZE rcvd: 58
> hw12203:/Users/alopresto (master) alopresto
> 0s @ 20:12:30 $
> {code}
> Reported on Mac OS X 10.11.4
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (NIFI-1774) AbstractControllerService: use of deprecated Annotation
[ https://issues.apache.org/jira/browse/NIFI-1774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard reassigned NIFI-1774: Assignee: Pierre Villard > AbstractControllerService: use of deprecated Annotation > --- > > Key: NIFI-1774 > URL: https://issues.apache.org/jira/browse/NIFI-1774 > Project: Apache NiFi > Issue Type: Bug >Reporter: Brandon DeVries >Assignee: Pierre Villard >Priority: Trivial > > AbstractControllerService is using the deprecated annotation \@OnConfigured. > Possibly should be org.apache.nifi.annotation.lifecycle.OnEnabled. > https://github.com/apache/nifi/blob/e4b7e47836edf47042973e604005058c28eed23b/nifi-api/src/main/java/org/apache/nifi/controller/AbstractControllerService.java#L52 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-1773) Typo in AbstractControllerService
[ https://issues.apache.org/jira/browse/NIFI-1773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard reassigned NIFI-1773: Assignee: Pierre Villard > Typo in AbstractControllerService > - > > Key: NIFI-1773 > URL: https://issues.apache.org/jira/browse/NIFI-1773 > Project: Apache NiFi > Issue Type: Bug >Reporter: Brandon DeVries >Assignee: Pierre Villard >Priority: Trivial > > On line 84 of AbstractControllerService, the comment refers to "Reporting > Task" instead of "Controller Service": > https://github.com/apache/nifi/blob/e4b7e47836edf47042973e604005058c28eed23b/nifi-api/src/main/java/org/apache/nifi/controller/AbstractControllerService.java#L84 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1774) AbstractControllerService: use of deprecated Annotation
[ https://issues.apache.org/jira/browse/NIFI-1774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268427#comment-15268427 ] Pierre Villard commented on NIFI-1774: -- As a reference: https://github.com/apache/nifi/pull/406 > AbstractControllerService: use of deprecated Annotation > --- > > Key: NIFI-1774 > URL: https://issues.apache.org/jira/browse/NIFI-1774 > Project: Apache NiFi > Issue Type: Bug >Reporter: Brandon DeVries >Assignee: Pierre Villard >Priority: Trivial > > AbstractControllerService is using the deprecated annotation \@OnConfigured. > Possibly should be org.apache.nifi.annotation.lifecycle.OnEnabled. > https://github.com/apache/nifi/blob/e4b7e47836edf47042973e604005058c28eed23b/nifi-api/src/main/java/org/apache/nifi/controller/AbstractControllerService.java#L52 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-1228) StandardRootGroupPort warning message incomplete
[
https://issues.apache.org/jira/browse/NIFI-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-1228.
--
Resolution: Fixed
Fix Version/s: 1.0.0
Fixed by NIFI-1551
> StandardRootGroupPort warning message incomplete
>
>
> Key: NIFI-1228
> URL: https://issues.apache.org/jira/browse/NIFI-1228
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Brandon DeVries
>Priority: Minor
> Fix For: 1.0.0
>
>
> In StandardGroupRootPort\[1\], warning messages created on lines 409 and 414
> are missing a "%s". E.g.:
> {code}
> final String message = String.format("%s authorization failed for user %s
> because ", this, dn, ae);
> {code}
> should be:
> {code}
> final String message = String.format("%s authorization failed for user %s
> because %s", this, dn, ae);
> {code}
> \[1\]
> https://github.com/apache/nifi/blob/31fba6b3332978ca2f6a1d693f6053d719fb9daa/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-site-to-site/src/main/java/org/apache/nifi/remote/StandardRootGroupPort.java#L409#L414
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (NIFI-1213) Allow Mock Framework to register "FlowFile Assertions"
[
https://issues.apache.org/jira/browse/NIFI-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard reassigned NIFI-1213:
Assignee: Pierre Villard
> Allow Mock Framework to register "FlowFile Assertions"
> --
>
> Key: NIFI-1213
> URL: https://issues.apache.org/jira/browse/NIFI-1213
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Tools and Build
>Reporter: Mark Payne
>Assignee: Pierre Villard
> Fix For: 1.0.0
>
>
> Often times, unit tests will invoke TestRunner.run, and then iterate through
> the output FlowFiles, ensuring that a particular attribute exists, or that an
> attribute is equal to some value.
> As a convenience, we should provide a mechanism to indicate that all output
> FlowFiles (or all FlowFiles routed to a given relationship) meet some
> criteria.
> For example:
> {code}
> TestRunner.assertAllFlowFilesContainAttribute( String attributeName );
> {code}
> {code}
> TestRunner.assertAllFlowFilesContainAttribute( Relationship relationship,
> String attributeName );
> {code}
> Additionally, we should consider allowing a "callback" mechanism:
> {code}
> TestRunner.assertAllFlowFiles( FlowFileValidator validator );
> {code}
> {code}
> TestRunner.assertAllFlowFiles( Relationship relationship, FlowFileValidator
> validator );
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (NIFI-1811) Remove ProcessorLog and update dependent interfaces.
[
https://issues.apache.org/jira/browse/NIFI-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard reassigned NIFI-1811:
Assignee: Pierre Villard
> Remove ProcessorLog and update dependent interfaces.
>
>
> Key: NIFI-1811
> URL: https://issues.apache.org/jira/browse/NIFI-1811
> Project: Apache NiFi
> Issue Type: Task
> Components: Core Framework
>Affects Versions: 0.6.1
>Reporter: Aldrin Piri
>Assignee: Pierre Villard
>Priority: Minor
> Fix For: 1.0.0
>
>
> From the comments:
> {quote}
> /**
> * The ProcessorLog is an extension of ComponentLog but provides no additional
> * functionality. It exists because ProcessorLog was created first, but when
> * Controller Services and Reporting Tasks began to be used more heavily
> loggers
> * were needed for them as well. We did not want to return a ProcessorLog to a
> * ControllerService or a ReportingTask, so all of the methods were moved to a
> * higher interface named ComponentLog. However, we kept the ProcessorLog
> * interface around in order to maintain backward compatibility.
> */
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1678) Nodes in cluster should use ZooKeeper to store heartbeat messages instead of sending to NCM
[
https://issues.apache.org/jira/browse/NIFI-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15266595#comment-15266595
]
Pierre Villard commented on NIFI-1678:
--
I am not sure if I should open a new JIRA but when building with:
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5;
2015-11-10T17:41:47+01:00)
Maven home: D:\Dev\apache-maven-3.3.9\bin\..
Java version: 1.8.0_74, vendor: Oracle Corporation
Java home: C:\Program Files\Java\jdk1.8.0_74\jre
Default locale: fr_FR, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "dos"
I have the following test errors:
{noformat}
---
T E S T S
---
Running TestSuite
Tests run: 45, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 19.919 sec <<<
FAILURE! - in TestSuite
testDisconnectedHeartbeatOnStartup on
testDisconnectedHeartbeatOnStartup(org.apache.nifi.cluster.coordination.heartbeat.TestAbstractHeartbeatMonitor)(org.apache.nifi.cluster.coordi
nation.heartbeat.TestAbstractHeartbeatMonitor) Time elapsed: 2.117 sec <<<
FAILURE!
java.lang.NullPointerException: null
at
org.apache.zookeeper.server.ZooKeeperServerMain.shutdown(ZooKeeperServerMain.java:132)
at
org.apache.curator.test.TestingZooKeeperMain.close(TestingZooKeeperMain.java:122)
at
org.apache.curator.test.TestingZooKeeperServer.stop(TestingZooKeeperServer.java:110)
at org.apache.curator.test.TestingServer.stop(TestingServer.java:161)
at
org.apache.nifi.cluster.coordination.heartbeat.TestAbstractHeartbeatMonitor.clear(TestAbstractHeartbeatMonitor.java:62)
testConnectingNodeMarkedConnectedWhenHeartbeatReceived on
testConnectingNodeMarkedConnectedWhenHeartbeatReceived(org.apache.nifi.cluster.coordination.heartbeat.TestAbstractHeartbea
tMonitor)(org.apache.nifi.cluster.coordination.heartbeat.TestAbstractHeartbeatMonitor)
Time elapsed: 2.013 sec <<< FAILURE!
java.lang.NullPointerException: null
at
org.apache.zookeeper.server.ZooKeeperServerMain.shutdown(ZooKeeperServerMain.java:132)
at
org.apache.curator.test.TestingZooKeeperMain.close(TestingZooKeeperMain.java:122)
at
org.apache.curator.test.TestingZooKeeperServer.stop(TestingZooKeeperServer.java:110)
at org.apache.curator.test.TestingServer.stop(TestingServer.java:161)
at
org.apache.nifi.cluster.coordination.heartbeat.TestAbstractHeartbeatMonitor.clear(TestAbstractHeartbeatMonitor.java:62)
Results :
Failed tests:
TestAbstractHeartbeatMonitor.clear:62 » NullPointer
TestAbstractHeartbeatMonitor.clear:62 » NullPointer
{noformat}
Not sure why this is happening. When running in Eclipse, tests are OK.
> Nodes in cluster should use ZooKeeper to store heartbeat messages instead of
> sending to NCM
> ---
>
> Key: NIFI-1678
> URL: https://issues.apache.org/jira/browse/NIFI-1678
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.0.0
>
>
> Currently, nodes send heartbeats to the NCM periodically in order to indicate
> that they are actively participating in the cluster. As we move away from
> using an NCM, we need these heartbeats to go somewhere else. ZooKeeper is a
> reasonable location to push the heartbeats to, as it provides the HA that we
> need
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1832) Testing EL properties with AllowableValues
[
https://issues.apache.org/jira/browse/NIFI-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15266393#comment-15266393
]
Pierre Villard commented on NIFI-1832:
--
In fact I just realized that it won't work since, on GUI, properties with a set
of allowable values are represented using a panel list (it is not possible to
manually set a value). So I guess that if we want a property with expression
language enabled, it must be a classic property or use the first solution I
mentioned.
> Testing EL properties with AllowableValues
> --
>
> Key: NIFI-1832
> URL: https://issues.apache.org/jira/browse/NIFI-1832
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 0.6.1
> Environment: Testing
>Reporter: Simon Elliston Ball
>Assignee: Pierre Villard
>Priority: Minor
>
> I’ve come across an interesting problem with MockFlowFile while testing a
> custom processor. My property has an AllowableValue list, and supports
> expression language. The test uses:
> runner.setProperty(PROPERTY_REF, "${attribute.name}”);
> However, the test fails on validation of in the MockFlowFile with the
> unevaluated version of the EL invalid against the allowed values list.
> 'Property' validated against '${attribute.name}' is invalid because Given
> value is not found in allowed set ...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (NIFI-1832) Testing EL properties with AllowableValues
[
https://issues.apache.org/jira/browse/NIFI-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard reassigned NIFI-1832:
Assignee: Pierre Villard
> Testing EL properties with AllowableValues
> --
>
> Key: NIFI-1832
> URL: https://issues.apache.org/jira/browse/NIFI-1832
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 0.6.1
> Environment: Testing
>Reporter: Simon Elliston Ball
>Assignee: Pierre Villard
>Priority: Minor
>
> I’ve come across an interesting problem with MockFlowFile while testing a
> custom processor. My property has an AllowableValue list, and supports
> expression language. The test uses:
> runner.setProperty(PROPERTY_REF, "${attribute.name}”);
> However, the test fails on validation of in the MockFlowFile with the
> unevaluated version of the EL invalid against the allowed values list.
> 'Property' validated against '${attribute.name}' is invalid because Given
> value is not found in allowed set ...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1751) Update HTTP processors to allow a proxy authentication strategy
[
https://issues.apache.org/jira/browse/NIFI-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15265843#comment-15265843
]
Pierre Villard commented on NIFI-1751:
--
I just noticed in a unit test the following comment:
{noformat}
// Currently InvokeHttp does not support Proxy via Https
{noformat}
So it seems to be a general problem (not related to proxy authentication): we
can't use InvokeHttp behind a proxy to access HTTPS targets.
> Update HTTP processors to allow a proxy authentication strategy
> ---
>
> Key: NIFI-1751
> URL: https://issues.apache.org/jira/browse/NIFI-1751
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: David A. Wynne
>Priority: Minor
>
> If trying to use the HTTP processors from behind a proxy server, the
> following error occurs:
> Received error HTTP_ERROR: HTTP/1.1 407 Proxy Authentication Required. Will
> attempt to reconnect
> The library NiFi uses for HTTP processors has the ability to set a proxy
> authentication strategy but currently doesn't use it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1751) Update HTTP processors to allow a proxy authentication strategy
[
https://issues.apache.org/jira/browse/NIFI-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15265834#comment-15265834
]
Pierre Villard commented on NIFI-1751:
--
[~dawynne] Could you confirm you are talking about InvokeHttp processor?
For GetHttp et PostHttp, if you use username and password properties, it works
with a proxy requiring authentication.
Regarding InvokeHttp, I had a look and I am able to propose a PR, but OkHttp
dependency seems to be a little unstable regarding proxy... During my testings,
if the user provides a wrong username/password for proxy authentication, the
error will be
{noformat}
java.net.ProtocolException: Too many follow-up requests: 21
{noformat}
because OkHttp client will keep trying to authenticate against the proxy even
if the proxy replied that credentials are wrong
(https://github.com/square/okhttp/issues/2464).
Another bigger issue is that when trying, with a local proxy, to access HTTPS
pages, it always fail with the following error:
{noformat}
java.io.IOException: unexpected end of stream on null
at
com.squareup.okhttp.internal.http.Http1xStream.readResponse(Http1xStream.java:201)
at
com.squareup.okhttp.internal.io.RealConnection.createTunnel(RealConnection.java:245)
at
com.squareup.okhttp.internal.io.RealConnection.connectTls(RealConnection.java:168)
at
com.squareup.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:145)
at
com.squareup.okhttp.internal.io.RealConnection.connect(RealConnection.java:108)
at
com.squareup.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:184)
at
com.squareup.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:126)
at
com.squareup.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:95)
at
com.squareup.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:283)
at
com.squareup.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:224)
at com.squareup.okhttp.Call.getResponse(Call.java:286)
at
com.squareup.okhttp.Call$ApplicationInterceptorChain.proceed(Call.java:243)
at
com.squareup.okhttp.Call.getResponseWithInterceptorChain(Call.java:205)
at com.squareup.okhttp.Call.execute(Call.java:80)
at
org.apache.nifi.processors.standard.InvokeHTTP.onTrigger(InvokeHTTP.java:611)
...
Caused by: java.io.EOFException: \n not found: size=0 content=...
at
okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:201)
at
com.squareup.okhttp.internal.http.Http1xStream.readResponse(Http1xStream.java:186)
{noformat}
It seems to be an issue which depends of the proxy behavior
(https://github.com/square/okhttp/issues/2453)... so it could only be because
of the proxy I use. But still...
I you think that would be better than nothing, let me know and I'll submit the
PR.
Any thoughts?
> Update HTTP processors to allow a proxy authentication strategy
> ---
>
> Key: NIFI-1751
> URL: https://issues.apache.org/jira/browse/NIFI-1751
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: David A. Wynne
>Priority: Minor
>
> If trying to use the HTTP processors from behind a proxy server, the
> following error occurs:
> Received error HTTP_ERROR: HTTP/1.1 407 Proxy Authentication Required. Will
> attempt to reconnect
> The library NiFi uses for HTTP processors has the ability to set a proxy
> authentication strategy but currently doesn't use it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (NIFI-1826) Expression Language: add function to check enumerator
[
https://issues.apache.org/jira/browse/NIFI-1826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard reassigned NIFI-1826:
Assignee: Pierre Villard
> Expression Language: add function to check enumerator
> -
>
> Key: NIFI-1826
> URL: https://issues.apache.org/jira/browse/NIFI-1826
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 0.6.1
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
>
> It would be useful to add a function returning true if a string is in a list
> of arguments and false otherwise.
> Example:
> {noformat}
> if ${myEnum} = "JOHN"
> then
> ${myEnum:in("PAUL", "JOHN", "MIKE")} returns true
> and
> ${myEnum:in("RED", "GREEN", "BLUE")} returns false
> {noformat}
> This would be particularly useful in RouteAttribute processor to perform
> conformity verifications on flow files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-1826) Expression Language: add function to check enumerator
[
https://issues.apache.org/jira/browse/NIFI-1826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-1826:
-
Description:
It would be useful to add a function returning true if a string is in a list of
arguments and false otherwise.
Example:
{noformat}
if ${myEnum} = "JOHN"
then
${myEnum:in("PAUL", "JOHN", "MIKE")} returns true
and
${myEnum:in("RED", "GREEN", "BLUE")} returns false
{noformat}
This would be particularly useful in RouteAttribute processor to perform
conformity verifications on flow files.
was:
It would be useful to add a function returning true if a string is in a list of
arguments and false otherwise.
Example:
if ${myEnum} = "JOHN"
then
${myEnum:in("PAUL", "JOHN", "MIKE")} returns true
and
${myEnum:in("RED", "GREEN", "BLUE")} returns false
This would be particularly useful in RouteAttribute processor to perform
conformity verifications on flow files.
> Expression Language: add function to check enumerator
> -
>
> Key: NIFI-1826
> URL: https://issues.apache.org/jira/browse/NIFI-1826
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 0.6.1
>Reporter: Pierre Villard
>Priority: Minor
>
> It would be useful to add a function returning true if a string is in a list
> of arguments and false otherwise.
> Example:
> {noformat}
> if ${myEnum} = "JOHN"
> then
> ${myEnum:in("PAUL", "JOHN", "MIKE")} returns true
> and
> ${myEnum:in("RED", "GREEN", "BLUE")} returns false
> {noformat}
> This would be particularly useful in RouteAttribute processor to perform
> conformity verifications on flow files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-1826) Expression Language: add function to check enumerator
Pierre Villard created NIFI-1826:
Summary: Expression Language: add function to check enumerator
Key: NIFI-1826
URL: https://issues.apache.org/jira/browse/NIFI-1826
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Affects Versions: 0.6.1
Reporter: Pierre Villard
Priority: Minor
It would be useful to add a function returning true if a string is in a list of
arguments and false otherwise.
Example:
if ${myEnum} = "JOHN"
then
${myEnum:in("PAUL", "JOHN", "MIKE")} returns true
and
${myEnum:in("RED", "GREEN", "BLUE")} returns false
This would be particularly useful in RouteAttribute processor to perform
conformity verifications on flow files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (NIFI-1806) Add UTF-8 for file enconding in unit tests during maven build
[
https://issues.apache.org/jira/browse/NIFI-1806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard reassigned NIFI-1806:
Assignee: Pierre Villard
> Add UTF-8 for file enconding in unit tests during maven build
> -
>
> Key: NIFI-1806
> URL: https://issues.apache.org/jira/browse/NIFI-1806
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 0.6.1
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>Priority: Minor
> Labels: build, maven, test
>
> Following discussion here :
> https://github.com/apache/nifi/pull/366#discussion_r60829511
> For some tests, it seems to be necessary to add the following in pom file:
> {code:title=pom.xml|borderStyle=solid}
>
>
>
> org.apache.maven.plugins
> maven-surefire-plugin
>
> -Dfile.encoding=UTF-8
>
>
>
>
> {code}
> Ideally this could be added in the parent pom but it must be ensured that
> this would not affect other components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (NIFI-1814) HandleHTTPResponse processor exceptions when connection is closed
[
https://issues.apache.org/jira/browse/NIFI-1814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard reassigned NIFI-1814:
Assignee: Pierre Villard
> HandleHTTPResponse processor exceptions when connection is closed
> -
>
> Key: NIFI-1814
> URL: https://issues.apache.org/jira/browse/NIFI-1814
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 0.6.1
> Environment: Centos 6
>Reporter: Sam Hjelmfelt
>Assignee: Pierre Villard
>Priority: Blocker
>
> If a HTTP connection is closed before a response is sent, the
> HandleHTTPResponse processor will throw an exception, yield due to failure,
> then try processing the rest of the events. If multiple errors happen at
> once, this compounded delay can cause cascading issues as the other
> connections timeout and close.
> Instead, the exception should be caught and the flowfile should go to the
> error relationship. A closed connection should not fail the processor, but
> instead fail the flowfile.
> Exception:
> HandleHttpResponse[id=cca1576b-5841-3fc1-b05a-5fd0f4853cb7] failed to process
> session due to org.apache.nifi.processor.exception.FlowFileAccessException:
> Failed to export
> StandardFlowFileRecord[uuid=febaedb5-4830-46b7-b0f9-71b453a4743c,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1461688143223-3, container=contS2R1,
> section=3], offset=999718,
> length=999718],offset=0,name=1628795866820216,size=999718] to
> HttpOutput@28849fc{OPEN} due to org.eclipse.jetty.io.EofException:
> org.apache.nifi.processor.exception.FlowFileAccessException: Failed to export
> StandardFlowFileRecord[uuid=febaedb5-4830-46b7-b0f9-71b453a4743c,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1461688143223-3, container=contS2R1,
> section=3], offset=999718,
> length=999718],offset=0,name=1628795866820216,size=999718] to
> HttpOutput@28849fc{OPEN} due to org.eclipse.jetty.io.EofException
> Processor Administratively Yielded for 1 sec due to processing failure
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (NIFI-1816) HandleHTTPResponse Processor Provenance
[ https://issues.apache.org/jira/browse/NIFI-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard reassigned NIFI-1816: Assignee: Pierre Villard > HandleHTTPResponse Processor Provenance > --- > > Key: NIFI-1816 > URL: https://issues.apache.org/jira/browse/NIFI-1816 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.6.1 > Environment: Centos6 >Reporter: Sam Hjelmfelt >Assignee: Pierre Villard >Priority: Minor > > The HandleHTTPResponse processor only creates provenance events for > auto-terminated relationships (i.e. drop events). It should also create send > events. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1022) Create GetTachyon and PutTachyon Processors
[ https://issues.apache.org/jira/browse/NIFI-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-1022: - Attachment: Alluxio.xml Attached a template to try the new processors. (cf install Alluxio on local machine : http://alluxio.org/documentation/v1.0.1/en/Running-Alluxio-Locally.html) Link with PR has not been added automatically, here it is: https://github.com/apache/nifi/pull/379 > Create GetTachyon and PutTachyon Processors > --- > > Key: NIFI-1022 > URL: https://issues.apache.org/jira/browse/NIFI-1022 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Jeremy Dyer >Assignee: Pierre Villard >Priority: Minor > Attachments: Alluxio.xml > > > Provide support for Apache Tachyon by implementing a GetTachyon and > PutTachyon processor. Having the ability to both read and write to Tachyon > would assist in sharing data with external applications such as Spark. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1764) NullPointerException in PutKafka for failed segments with no delimiter and insufficient producer handling
[
https://issues.apache.org/jira/browse/NIFI-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255521#comment-15255521
]
Pierre Villard commented on NIFI-1764:
--
[~joewitt] I discussed with Oleg and the "decrementAndGet returning < 0" is
explained by a bad manipulation I did when I was in debug. I did a new check
and everything is OK on this side, the reset = false is set because of
acceptTask = true.
> NullPointerException in PutKafka for failed segments with no delimiter and
> insufficient producer handling
> -
>
> Key: NIFI-1764
> URL: https://issues.apache.org/jira/browse/NIFI-1764
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Christopher McDermott
>Assignee: Oleg Zhurakousky
> Labels: patch
> Fix For: 1.0.0, 0.7.0
>
> Attachments: 0001-NIFI-1764-numerous-edits-from-reviewing-PR-366.patch
>
>
> This NPE can happen during certain failure cases and it appears to be related
> to the lack of guarding of the failed segments attribute addition in the case
> there is no delimiter. Further, we have observed the PutKafka processor
> becoming ineffective if the established kafka client starts seeing failed
> acks/timeouts. We need to catch those cases and teardown the old client and
> create a new one instead.
> {code}
> java.lang.NullPointerException: null
> at java.lang.String.(String.java:503) ~[na:1.8.0_45]
> at
> org.apache.nifi.processors.kafka.PutKafka.buildFailedFlowFileAttributes(PutKafka.java:396)
> ~[na:na]
> at org.apache.nifi.processors.kafka.PutKafka.onTrigger(PutKafka.java:308)
> ~[na:na]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> ~[nifi-api-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1059)
> ~[nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
> [nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> [nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT]
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:123)
> [nifi-framework-core-0.7.0-SNAPSHOT.jar:0.7.0-SNAPSHOT]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_45]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_45]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_45]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1732) HandleHttpRequest should allow user to configure jetty timeout
[
https://issues.apache.org/jira/browse/NIFI-1732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255351#comment-15255351
]
Pierre Villard commented on NIFI-1732:
--
Comment from the reporter (Luke) regarding the PR:
The change looks like it should work fine. I'm unable to test the change, but
it looks correct. The only other thing I'd say is needed is the HTTP REQUEST
leakage in the StandardHttpContextMap. This happens when the timeout expires
and the HTTP ERROR comes in. Even after the REQUEST timeout is completed in
the StandardHttpContextMap, from what I saw the HTTP ERROR messages are
never cleared and thus the StandardHttpContextMap can get maxed out if too
many timeouts occur. This might be worth putting into a different JIRA
request, but I thought I'd mention it.
In other words, if the timeout threshold is not set long enough (with regard to
the flow following the request), Jetty will issue a new request to the
processor with a DispatchType ERROR. At the moment, this new request is
registered in the context map and will never be cleared. The consequence is
that the context map will be fulled of unanswered requests and will block new
incoming requests.
I tested two options:
- create an 'error' route to forward the requests of DispatchType ERROR without
registering it in the context map. Thus the initial request will be answered
even if the timeout threshold has been reached and the context map will not
become full of unanswered requests.
- when a request of DispatchType ERROR is received, directly return a HTTP
response of type TIMEOUT (error 408). Thus the timeout aspect is really applied
however the flow will be executed and an error will have to be handled at the
end of the flow when trying to answer after the timeout.
Any thoughts regarding the best option? Any other ideas?
> HandleHttpRequest should allow user to configure jetty timeout
> --
>
> Key: NIFI-1732
> URL: https://issues.apache.org/jira/browse/NIFI-1732
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Pierre Villard
> Labels: beginner, newbie
> Attachments: NIFI-1732.xml
>
>
> The following e-mail was received on the dev mailing list and points out an
> issue in the HandleHttpRequest, which is that the user cannot configure the
> jetty timeout. As a result, requests that take longer than 30 seconds timeout
> causing some problems:
> {quote}
> Hi Nifi Team,
> I've been experimenting with the HandleHttpRequest/Response processors in
> Nifi 0.5.1, and have noticed an issue that I've not been able to resolve. I'm
> hoping that I'm simply missing a configuration item, but I've been unable to
> find the solution.
> The scenario is this: HandleHttpRequest --> Long Processing (> 30 seconds)
> --> HandleHttpResponse. It appears that the jetty server backing the
> HandleHttpRequest has a built in idle time timeout of 3 ms (see
> jetty-server/src/main/java/org/eclipse/jetty/server/AbstractConnector.java
> _idleTimeout value). In my test flow, 30 seconds after a HTTP requests comes
> in, a second request comes into the flow. It has the same information, except
> the http.context.identifier and the FlowFile UUID has changed, and the
> http.dispatcher.type has changed from REQUEST to ERROR. From my online
> research
> (http://stackoverflow.com/questions/30786939/jetty-replay-request-on-timeout?),
> this re-request with a type of error comes in after jetty determines that a
> request has timed out.
> This would not normally be a big deal. I was able to RouteOnAttribute and
> capture all ERROR requests without responding. However, those requests are
> never cleared from the StandardHttpContextMap. I've tested this by setting
> the number of requests allowed by the StandardHttpContextMap to 4, and done 4
> of my long Request/Response tests. Each request is correctly responded to
> eventually in my test, but because they take over 30 seconds each also
> generates an ERROR request that is stored in the StandardHttpContextMap. If I
> then leave the system alone for much longer than the Request Timeout
> parameter in the StandardHttpContextMap and then attempt a request, I get a
> 503 response saying that the queue is full and no requests are allowed. No
> requests are allowed at all until I delete and recreate the Map.
> It seems unlikely to me that no one has attempted to use these processors in
> this fashion. However, looking through the unit test for this processor it
> seems like no where was a timeout tested over 30 seconds, so I thought it
> worth a conversation.
> So finally, is there a configuration item to extend the jetty server's idle
[jira] [Commented] (NIFI-1779) SNMP
[ https://issues.apache.org/jira/browse/NIFI-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255296#comment-15255296 ] Pierre Villard commented on NIFI-1779: -- Added duplicate link to NIFI-1537 > SNMP > - > > Key: NIFI-1779 > URL: https://issues.apache.org/jira/browse/NIFI-1779 > Project: Apache NiFi > Issue Type: Wish > Components: Core Framework >Affects Versions: 0.6.0 > Environment: n/a >Reporter: Manoj Kalikodanveetil > > Hi, > In one of NiFi 0.6.0 release notes it was mentioned that SNMP input handler > being added. But I don't see that in 0.6. Is that backed out? > Or is the plan to make use of Camel? > thanks > Manoj -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1521) Allow SSL with AMQP processors
[ https://issues.apache.org/jira/browse/NIFI-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-1521: - Fix Version/s: 0.7.0 1.0.0 > Allow SSL with AMQP processors > -- > > Key: NIFI-1521 > URL: https://issues.apache.org/jira/browse/NIFI-1521 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 0.5.0 >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Trivial > Labels: beginner > Fix For: 1.0.0, 0.7.0 > > > AMQP processor shall allow the creation of connections using SSL protocol > (not necessarily with presenting and validating certificates) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1806) Add UTF-8 for file enconding in unit tests during maven build
Pierre Villard created NIFI-1806:
Summary: Add UTF-8 for file enconding in unit tests during maven
build
Key: NIFI-1806
URL: https://issues.apache.org/jira/browse/NIFI-1806
Project: Apache NiFi
Issue Type: Bug
Components: Tools and Build
Affects Versions: 0.6.1
Reporter: Pierre Villard
Priority: Minor
Following discussion here :
https://github.com/apache/nifi/pull/366#discussion_r60829511
For some tests, it seems to be necessary to add the following in pom file:
{code:title=pom.xml|borderStyle=solid}
org.apache.maven.plugins
maven-surefire-plugin
-Dfile.encoding=UTF-8
{code}
Ideally this could be added in the parent pom but it must be ensured that this
would not affect other components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-1797) Use Compression Codec property in CreateHadoopSequenceFile processor
Pierre Villard created NIFI-1797: Summary: Use Compression Codec property in CreateHadoopSequenceFile processor Key: NIFI-1797 URL: https://issues.apache.org/jira/browse/NIFI-1797 Project: Apache NiFi Issue Type: Improvement Components: Extensions Affects Versions: 0.6.1 Reporter: Pierre Villard Priority: Minor CreateSequenceFileProcessor is extending AbstractHadoopProcessor where there is a property allowing user to define Compression Codec. However this is not used and it would be interesting to use it in SequenceFileWriterImpl. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1197) improve SSL options for getmongo and putmongo processor configuration properties
[ https://issues.apache.org/jira/browse/NIFI-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250082#comment-15250082 ] Pierre Villard commented on NIFI-1197: -- [~jskora] +1 for the patch regarding the 0.x branch > improve SSL options for getmongo and putmongo processor configuration > properties > > > Key: NIFI-1197 > URL: https://issues.apache.org/jira/browse/NIFI-1197 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration >Affects Versions: 0.7.0 >Reporter: subhash parise > Labels: patch > Fix For: 0.7.0 > > Attachments: > 0001-Fix-0.x-branch-nifi-mongodb-bundle-POM-reference-to-.patch > > > Hi Team, > Now getmongo and putmongo configuration properties are mongodb URI,database > name, collection name, etc, but if the mongodb server configured with the ssl > it won't accept ssl options in mongodb uri. > Could anyone please improve the ssl options in mongo proccessors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-1788) CreateHadoopSequenceFile shows wrong values for compression type
[ https://issues.apache.org/jira/browse/NIFI-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard reassigned NIFI-1788: Assignee: Pierre Villard > CreateHadoopSequenceFile shows wrong values for compression type > > > Key: NIFI-1788 > URL: https://issues.apache.org/jira/browse/NIFI-1788 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework, Core UI >Affects Versions: 0.6.0 >Reporter: Hugo Lemay-Proulx >Assignee: Pierre Villard >Priority: Minor > > The CreateHadoopSequenceFile processor shows that the possible values for > compression types are : > NONE > DEFAULT > BZIP > GZIP > LZ4 > SNAPPY > AUTOMATIC > but the compression type is not the compression codec to use. It is what to > compress in the sequence file: > NONE > RECORD > BLOCK > Because of that, if the selection is anything other than NONE, the processor > will throw an exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-1787) Help text has misspelling for "Route to 'matched' if any matches" routing strategy
[ https://issues.apache.org/jira/browse/NIFI-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard reassigned NIFI-1787: Assignee: Pierre Villard > Help text has misspelling for "Route to 'matched' if any matches" routing > strategy > -- > > Key: NIFI-1787 > URL: https://issues.apache.org/jira/browse/NIFI-1787 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 0.6.0 >Reporter: Andrew Lim >Assignee: Pierre Villard >Priority: Trivial > Attachments: NIFI-1787_routingStrategyHelp.png > > > On the Properties tab when configuring a RouteOnAttribute processor, the user > can select different routing strategies. If you hover over the "?" icon for > the "Route to 'matched' if any matches" property value, the help text is > displayed. The word "the" is misspelled as "hte" in the text. See attached > screenshot. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-1724) Enhance FetchFile to allow the log level for file not founds to be configurable
[ https://issues.apache.org/jira/browse/NIFI-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard reassigned NIFI-1724: Assignee: Pierre Villard (was: Oleg Zhurakousky) > Enhance FetchFile to allow the log level for file not founds to be > configurable > --- > > Key: NIFI-1724 > URL: https://issues.apache.org/jira/browse/NIFI-1724 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Christopher McDermott >Assignee: Pierre Villard >Priority: Minor > Fix For: 1.0.0, 0.7.0 > > > Sometimes a file may not exist but it is not an error. This change would > allow the user to configure a logging level for the message so that a file > not found would not alway produce a bulletin. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1765) Data Provenance: Results are not being filtered when filtering “by component name"
[ https://issues.apache.org/jira/browse/NIFI-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15244307#comment-15244307 ] Pierre Villard commented on NIFI-1765: -- [~andrewmlim] I didn't reproduce the issue neither with current master branch, nor 0.6.1 RC2. Tried with Firefox, Chrome and IE. > Data Provenance: Results are not being filtered when filtering “by component > name" > -- > > Key: NIFI-1765 > URL: https://issues.apache.org/jira/browse/NIFI-1765 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 0.6.0 >Reporter: Andrew Lim > Attachments: NIFI-1765-filterbycomponentname.png > > > In the Data Provenance window, when the user tries to filter "by component > name", the displayed results are not filtered at all. Filtering "by > component type" and "type" are working as expected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1779) SNMP
[ https://issues.apache.org/jira/browse/NIFI-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15243198#comment-15243198 ] Pierre Villard commented on NIFI-1779: -- Please have a look to NIFI-1537: there is an opened PR for SNMP processors. If you want to try it out, feedback would be very appreciated. > SNMP > - > > Key: NIFI-1779 > URL: https://issues.apache.org/jira/browse/NIFI-1779 > Project: Apache NiFi > Issue Type: Wish > Components: Core Framework >Affects Versions: 0.6.0 > Environment: n/a >Reporter: Manoj Kalikodanveetil > > Hi, > In one of NiFi 0.6.0 release notes it was mentioned that SNMP input handler > being added. But I don't see that in 0.6. Is that backed out? > Or is the plan to make use of Camel? > thanks > Manoj -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-1776) CompressContent doesn't recognize application/x-gzip as Gzip mime type
[ https://issues.apache.org/jira/browse/NIFI-1776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard reassigned NIFI-1776: Assignee: Pierre Villard > CompressContent doesn't recognize application/x-gzip as Gzip mime type > -- > > Key: NIFI-1776 > URL: https://issues.apache.org/jira/browse/NIFI-1776 > Project: Apache NiFi > Issue Type: Bug >Reporter: Joey Frazee >Assignee: Pierre Villard >Priority: Minor > > CompressContent doesn't recognize application/x-gzip as valid Gzip mime type > when setting Compression Format to use mime.type attribute. > FWIW RFC6713 [1] does recommend replacing application/x-gzip with > application/gzip but nonetheless many applications still produce and work > with this as a valid mime type (e.g., Apache Tika). So it seems like it might > be reasonable to include it. > 1. https://tools.ietf.org/html/rfc6713 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-1777) Incoming Connection Deleted While Processor Still Running
[ https://issues.apache.org/jira/browse/NIFI-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard reassigned NIFI-1777: Assignee: Pierre Villard > Incoming Connection Deleted While Processor Still Running > - > > Key: NIFI-1777 > URL: https://issues.apache.org/jira/browse/NIFI-1777 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 0.6.0, 0.6.1 >Reporter: Bryan Bende >Assignee: Pierre Villard >Priority: Minor > Fix For: 0.7.0 > > > I created a simple flow with ExecuteCommand connected to PutKafka. I had > PutKafka running and ExecuteCommand stopped, and then deleted ExecuteCommand. > At this point PutKafka showed a bulletin for being invalid because it > requires an incoming connection, but PutKafka was still running and couldn't > be stopped until adding back an incoming connection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1777) Incoming Connection Deleted While Processor Still Running
[ https://issues.apache.org/jira/browse/NIFI-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15243022#comment-15243022 ] Pierre Villard commented on NIFI-1777: -- Yes, that's what I told myself when trying to reproduce the issue. Will submit a PR shortly to prevent deleting a connection going to a running processor. > Incoming Connection Deleted While Processor Still Running > - > > Key: NIFI-1777 > URL: https://issues.apache.org/jira/browse/NIFI-1777 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 0.6.0, 0.6.1 >Reporter: Bryan Bende >Priority: Minor > Fix For: 0.7.0 > > > I created a simple flow with ExecuteCommand connected to PutKafka. I had > PutKafka running and ExecuteCommand stopped, and then deleted ExecuteCommand. > At this point PutKafka showed a bulletin for being invalid because it > requires an incoming connection, but PutKafka was still running and couldn't > be stopped until adding back an incoming connection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1777) Incoming Connection Deleted While Processor Still Running
[ https://issues.apache.org/jira/browse/NIFI-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242987#comment-15242987 ] Pierre Villard commented on NIFI-1777: -- [~bende] I just tried and did not reproduce the issue: I was able to stop PutKafka without adding an incoming connection. Did you get errors when trying to stop PutKafka? > Incoming Connection Deleted While Processor Still Running > - > > Key: NIFI-1777 > URL: https://issues.apache.org/jira/browse/NIFI-1777 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 0.6.0, 0.6.1 >Reporter: Bryan Bende >Priority: Minor > Fix For: 0.7.0 > > > I created a simple flow with ExecuteCommand connected to PutKafka. I had > PutKafka running and ExecuteCommand stopped, and then deleted ExecuteCommand. > At this point PutKafka showed a bulletin for being invalid because it > requires an incoming connection, but PutKafka was still running and couldn't > be stopped until adding back an incoming connection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-1672) Improve the Provenance Events emitted by PutKafka
[ https://issues.apache.org/jira/browse/NIFI-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard reassigned NIFI-1672: Assignee: Pierre Villard > Improve the Provenance Events emitted by PutKafka > - > > Key: NIFI-1672 > URL: https://issues.apache.org/jira/browse/NIFI-1672 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Pierre Villard >Priority: Minor > Fix For: 0.7.0 > > > The Provenance SEND events emitted by PutKafka should include the amount of > time that it took to send the data. Also, if some of the messages were sent > but not all (when using a delimiter), then we should have a SEND event that > indicates the number of events that were sent. The Transit URI should also > include the "kafka://" protocol name at the beginning and should include the > topic, such as kafka://leaderBroker:9092/topics/topicDataWasPlacedOn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-1755) Remote Process Group Status
[ https://issues.apache.org/jira/browse/NIFI-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard reassigned NIFI-1755: Assignee: Pierre Villard > Remote Process Group Status > --- > > Key: NIFI-1755 > URL: https://issues.apache.org/jira/browse/NIFI-1755 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Core UI >Reporter: Matt Gilman >Assignee: Pierre Villard >Priority: Minor > Fix For: 1.0.0 > > > If a flow has multiple Remote Process Groups (RPG) pointed at the same target > NiFi instance, each RPG will report the total Sent/Received to/from that > system. Not the amount that each individual RPG Sent/Received alone. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1757) If Bulletin Board is opened, the requests keep occurring in the background even after the Bulletin Board is closed
[ https://issues.apache.org/jira/browse/NIFI-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15238811#comment-15238811 ] Pierre Villard commented on NIFI-1757: -- [~markap14] As a side note: it seems to be Chrome specific. I reproduced the issue with Chrome but neither with Firefox nor IE. > If Bulletin Board is opened, the requests keep occurring in the background > even after the Bulletin Board is closed > -- > > Key: NIFI-1757 > URL: https://issues.apache.org/jira/browse/NIFI-1757 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Mark Payne > > I clicked to open the Bulletin Board and then closed it. Opening Chrome's > Developer Tools, I can see that the requests are still made to retrieve > bulletins every few seconds. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1758) Unable to scroll in the Comments field when configuring process group
[
https://issues.apache.org/jira/browse/NIFI-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15238779#comment-15238779
]
Pierre Villard commented on NIFI-1758:
--
It seems this is not limited to process group. I observed the same issue for
processors, input/output ports, etc.
I am not a specialist but I think it'd just require the following modification:
{code:title=nf-canvas-all.css|borderStyle=solid}
textarea {
overflow: scroll;
}
{code}
instead of
{code:title=nf-canvas-all.css|borderStyle=solid}
textarea {
overflow: hidden;
}
{code}
I let web gurus have a look into this one.
> Unable to scroll in the Comments field when configuring process group
> -
>
> Key: NIFI-1758
> URL: https://issues.apache.org/jira/browse/NIFI-1758
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Affects Versions: 0.6.0
>Reporter: Andrew Lim
>Priority: Minor
>
> When configuring a process group, the Comments field allows a large amount of
> text to be entered or pasted in. The size of the comments field displays
> about 6-7 rows (depending on what web browser is being used). However,
> scrolling is not allowed for that field, so the user is unable to easily see
> or edit the values in the field.
> A workaround is to use the keyboard arrow keys.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-1732) HandleHttpRequest should allow user to configure jetty timeout
[
https://issues.apache.org/jira/browse/NIFI-1732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-1732:
-
Attachment: NIFI-1732.xml
Template to reproduce the issue.
Found a solution, will submit a PR shortly.
> HandleHttpRequest should allow user to configure jetty timeout
> --
>
> Key: NIFI-1732
> URL: https://issues.apache.org/jira/browse/NIFI-1732
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Pierre Villard
> Labels: beginner, newbie
> Attachments: NIFI-1732.xml
>
>
> The following e-mail was received on the dev mailing list and points out an
> issue in the HandleHttpRequest, which is that the user cannot configure the
> jetty timeout. As a result, requests that take longer than 30 seconds timeout
> causing some problems:
> {quote}
> Hi Nifi Team,
> I've been experimenting with the HandleHttpRequest/Response processors in
> Nifi 0.5.1, and have noticed an issue that I've not been able to resolve. I'm
> hoping that I'm simply missing a configuration item, but I've been unable to
> find the solution.
> The scenario is this: HandleHttpRequest --> Long Processing (> 30 seconds)
> --> HandleHttpResponse. It appears that the jetty server backing the
> HandleHttpRequest has a built in idle time timeout of 3 ms (see
> jetty-server/src/main/java/org/eclipse/jetty/server/AbstractConnector.java
> _idleTimeout value). In my test flow, 30 seconds after a HTTP requests comes
> in, a second request comes into the flow. It has the same information, except
> the http.context.identifier and the FlowFile UUID has changed, and the
> http.dispatcher.type has changed from REQUEST to ERROR. From my online
> research
> (http://stackoverflow.com/questions/30786939/jetty-replay-request-on-timeout?),
> this re-request with a type of error comes in after jetty determines that a
> request has timed out.
> This would not normally be a big deal. I was able to RouteOnAttribute and
> capture all ERROR requests without responding. However, those requests are
> never cleared from the StandardHttpContextMap. I've tested this by setting
> the number of requests allowed by the StandardHttpContextMap to 4, and done 4
> of my long Request/Response tests. Each request is correctly responded to
> eventually in my test, but because they take over 30 seconds each also
> generates an ERROR request that is stored in the StandardHttpContextMap. If I
> then leave the system alone for much longer than the Request Timeout
> parameter in the StandardHttpContextMap and then attempt a request, I get a
> 503 response saying that the queue is full and no requests are allowed. No
> requests are allowed at all until I delete and recreate the Map.
> It seems unlikely to me that no one has attempted to use these processors in
> this fashion. However, looking through the unit test for this processor it
> seems like no where was a timeout tested over 30 seconds, so I thought it
> worth a conversation.
> So finally, is there a configuration item to extend the jetty server's idle
> timeout? Or is there a better way to ensure that the bogus requests don't get
> stuck permanently in the StandardHttpContextMap? I appreciate any pointers
> you can give.
> Thanks,
> Luke Coder
> BIT Systems
> CACI - NCS
> 941-907-8803 x705
> 6851 Professional Pkwy W
> Sarasota, FL 34240
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (NIFI-1732) HandleHttpRequest should allow user to configure jetty timeout
[
https://issues.apache.org/jira/browse/NIFI-1732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard reassigned NIFI-1732:
Assignee: Pierre Villard
> HandleHttpRequest should allow user to configure jetty timeout
> --
>
> Key: NIFI-1732
> URL: https://issues.apache.org/jira/browse/NIFI-1732
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Pierre Villard
> Labels: beginner, newbie
>
> The following e-mail was received on the dev mailing list and points out an
> issue in the HandleHttpRequest, which is that the user cannot configure the
> jetty timeout. As a result, requests that take longer than 30 seconds timeout
> causing some problems:
> {quote}
> Hi Nifi Team,
> I've been experimenting with the HandleHttpRequest/Response processors in
> Nifi 0.5.1, and have noticed an issue that I've not been able to resolve. I'm
> hoping that I'm simply missing a configuration item, but I've been unable to
> find the solution.
> The scenario is this: HandleHttpRequest --> Long Processing (> 30 seconds)
> --> HandleHttpResponse. It appears that the jetty server backing the
> HandleHttpRequest has a built in idle time timeout of 3 ms (see
> jetty-server/src/main/java/org/eclipse/jetty/server/AbstractConnector.java
> _idleTimeout value). In my test flow, 30 seconds after a HTTP requests comes
> in, a second request comes into the flow. It has the same information, except
> the http.context.identifier and the FlowFile UUID has changed, and the
> http.dispatcher.type has changed from REQUEST to ERROR. From my online
> research
> (http://stackoverflow.com/questions/30786939/jetty-replay-request-on-timeout?),
> this re-request with a type of error comes in after jetty determines that a
> request has timed out.
> This would not normally be a big deal. I was able to RouteOnAttribute and
> capture all ERROR requests without responding. However, those requests are
> never cleared from the StandardHttpContextMap. I've tested this by setting
> the number of requests allowed by the StandardHttpContextMap to 4, and done 4
> of my long Request/Response tests. Each request is correctly responded to
> eventually in my test, but because they take over 30 seconds each also
> generates an ERROR request that is stored in the StandardHttpContextMap. If I
> then leave the system alone for much longer than the Request Timeout
> parameter in the StandardHttpContextMap and then attempt a request, I get a
> 503 response saying that the queue is full and no requests are allowed. No
> requests are allowed at all until I delete and recreate the Map.
> It seems unlikely to me that no one has attempted to use these processors in
> this fashion. However, looking through the unit test for this processor it
> seems like no where was a timeout tested over 30 seconds, so I thought it
> worth a conversation.
> So finally, is there a configuration item to extend the jetty server's idle
> timeout? Or is there a better way to ensure that the bogus requests don't get
> stuck permanently in the StandardHttpContextMap? I appreciate any pointers
> you can give.
> Thanks,
> Luke Coder
> BIT Systems
> CACI - NCS
> 941-907-8803 x705
> 6851 Professional Pkwy W
> Sarasota, FL 34240
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1521) Allow SSL with AMQP processors
[ https://issues.apache.org/jira/browse/NIFI-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15224437#comment-15224437 ] Pierre Villard commented on NIFI-1521: -- Oleg I believe this PR was waiting for modification on StandardSSLContextService side. I created NIFI-1652 to follow on this subject. I will try to address it tomorrow. One possibility would be to update this PR to set a default ClientAuthPolicy (as it is currently done on other processors) and it would allow to close this PR. However, I am no expert in this domain, and I am not sure if there is a ClientAuthPolicy which is more suitable for AMQP. Another possibility is to add a property ClientAuthPolicy as it is the case in Cassandra processors. This would solve the issue here and would not need to change anything on StandardSSLContextService side. Thoughts? > Allow SSL with AMQP processors > -- > > Key: NIFI-1521 > URL: https://issues.apache.org/jira/browse/NIFI-1521 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 0.5.0 >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Trivial > Labels: beginner > > AMQP processor shall allow the creation of connections using SSL protocol > (not necessarily with presenting and validating certificates) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1671) TestListFile: unit test failing on Windows with specific accounts configuration
[
https://issues.apache.org/jira/browse/NIFI-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15218123#comment-15218123
]
Pierre Villard commented on NIFI-1671:
--
[~jskora] I agree, I think this is some kind of group configuration. When
running a maven build or unit tests from Eclipse, I am not using "Run as
administrator" and I get this error. I just tried to run Eclipse as
administrator and the result is the same.
I am not a Windows expert but just had a look on group policies and file
creation seems to be linked to a group "Administrateurs" rule. So it looks like
what you are suggesting and that would explain what is observed.
I fully agree this is really difficult to normalize this kind of things.
Besides, I won't be using this laptop any more in few days... Not sure there is
a need to absolutely fix something if I am the only one reporting this issue.
> TestListFile: unit test failing on Windows with specific accounts
> configuration
> ---
>
> Key: NIFI-1671
> URL: https://issues.apache.org/jira/browse/NIFI-1671
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 0.5.1
> Environment: Windows 7 Pro, Maven 3.3.9, Java 8
>Reporter: Pierre Villard
>Priority: Minor
>
> In TestListFile, with test testAttributesSet, it fails at
> assertTrue(mock1.getAttribute(ListFile.FILE_OWNER_ATTRIBUTE).contains(userName));
> If I look further:
> mock1.getAttribute(ListFile.FILE_OWNER_ATTRIBUTE) is equal to
> "BUILTIN\Administrateurs"
> when userName = System.getProperty("user.name") is equal to "pvillard"
> I believe this is something related to a very specific configuration on my
> company laptop.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-1671) TestListFile: unit test failing on Windows with specific accounts configuration
Pierre Villard created NIFI-1671:
Summary: TestListFile: unit test failing on Windows with specific
accounts configuration
Key: NIFI-1671
URL: https://issues.apache.org/jira/browse/NIFI-1671
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 0.5.1
Environment: Windows 7 Pro, Maven 3.3.9, Java 8
Reporter: Pierre Villard
Priority: Minor
In TestListFile, with test testAttributesSet, it fails at
assertTrue(mock1.getAttribute(ListFile.FILE_OWNER_ATTRIBUTE).contains(userName));
If I look further:
mock1.getAttribute(ListFile.FILE_OWNER_ATTRIBUTE) is equal to
"BUILTIN\Administrateurs"
when userName = System.getProperty("user.name") is equal to "pvillard"
I believe this is something related to a very specific configuration on my
company laptop.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1620) Allow empty Content-Type in InvokeHTTP processor
[
https://issues.apache.org/jira/browse/NIFI-1620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203758#comment-15203758
]
Pierre Villard commented on NIFI-1620:
--
OK I guess I didn't correctly understand what was the conclusion of our
previous messages. Will have a look to change the PR.
> Allow empty Content-Type in InvokeHTTP processor
>
>
> Key: NIFI-1620
> URL: https://issues.apache.org/jira/browse/NIFI-1620
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 0.5.1
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>
> External API could expect a POST request without a Content-Type property or
> at least with this property empty:
> *Error example:*
> {quote}
> You provided a non-empty HTTP "Content-Type" header ("application/json").
> This API function requires that the header be missing or empty.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-1652) Add Client Authentication policy parameter in SSL context service
Pierre Villard created NIFI-1652: Summary: Add Client Authentication policy parameter in SSL context service Key: NIFI-1652 URL: https://issues.apache.org/jira/browse/NIFI-1652 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Pierre Villard Assignee: Pierre Villard Priority: Minor Following discussion in NIFI-1521 - Add client authentication policy parameter in SSL Context service - Update every related processors to take into account this modification - Inform users regarding processors using a specific policy at this moment -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1521) Allow SSL with AMQP processors
[ https://issues.apache.org/jira/browse/NIFI-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15199326#comment-15199326 ] Pierre Villard commented on NIFI-1521: -- [~joewitt] no problem. Should I create a new JIRA for the addition of ClientAuth property in SslContextService? If yes this PR would then be merged after the new one? Besides I agree with [~mcgilman] and [~mattyb149] regarding possible confusion/compatibility issue. What is the strategy? Let the user configure the SSL Context Service to use the correct ClientAuth policy for processors where it was directly set so far? or document that for some processors this new property will have no effect? > Allow SSL with AMQP processors > -- > > Key: NIFI-1521 > URL: https://issues.apache.org/jira/browse/NIFI-1521 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 0.5.0 >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Trivial > Labels: beginner > > AMQP processor shall allow the creation of connections using SSL protocol > (not necessarily with presenting and validating certificates) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1332) GetHttp throws exception when no content returned
[ https://issues.apache.org/jira/browse/NIFI-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15199330#comment-15199330 ] Pierre Villard commented on NIFI-1332: -- [~joewitt] [~mattyb149] Could be part of 0.6.0? > GetHttp throws exception when no content returned > - > > Key: NIFI-1332 > URL: https://issues.apache.org/jira/browse/NIFI-1332 > Project: Apache NiFi > Issue Type: Bug >Reporter: Matt Burgess > > If a URL is specified in GetHttp and it returns an HTTP status code of 204 > (No content), a Null Pointer exception is thrown when trying to get the > entity/content: > java.lang.NullPointerException: null > at > org.apache.nifi.processors.standard.GetHTTP.onTrigger(GetHTTP.java:478) > ~[na:na] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1-SNAPSHOT.jar:0.4.1-SNAPSHOT] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1-SNAPSHOT.jar:0.4.1-SNAPSHOT] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1-SNAPSHOT.jar:0.4.1-SNAPSHOT] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1-SNAPSHOT.jar:0.4.1-SNAPSHOT] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_65] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > [na:1.8.0_65] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > [na:1.8.0_65] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > [na:1.8.0_65] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_65] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_65] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65] > I used http://httpstat.us/204 to reproduce this error. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1620) Allow empty Content-Type in InvokeHTTP processor
Pierre Villard created NIFI-1620:
Summary: Allow empty Content-Type in InvokeHTTP processor
Key: NIFI-1620
URL: https://issues.apache.org/jira/browse/NIFI-1620
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 0.5.1
Reporter: Pierre Villard
Assignee: Pierre Villard
Fix For: 0.6.0
External API could expect a POST request without a Content-Type property or at
least with this property empty:
*Error example:*
{quote}
You provided a non-empty HTTP "Content-Type" header ("application/json"). This
API function requires that the header be missing or empty.
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1559) Kafka processor writes some WARN message about registering an Mbean
[
https://issues.apache.org/jira/browse/NIFI-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189561#comment-15189561
]
Pierre Villard commented on NIFI-1559:
--
[~joewitt] Any specific configuration? I didn't get this log message with a
basic flow and Kafka 0.9.0.
> Kafka processor writes some WARN message about registering an Mbean
> ---
>
> Key: NIFI-1559
> URL: https://issues.apache.org/jira/browse/NIFI-1559
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 0.5.0
>Reporter: Joseph Witt
>Priority: Minor
>
> The kafka processors on startup appear to log a WARN about Mbean
> registration. This is at least alarming to a DFM.
> {quote}
> 2016-02-23 21:13:56,813 WARN [pool-29-thread-5]
> o.a.kafka.common.utils.AppInfoParser Error registering AppInfo mbean
> javax.management.InstanceAlreadyExistsException:
> kafka.producer:type=app-info,id=NiFi-2243c3f9-bd2b-4bfe-b515-09791ec25c4c
> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
> ~[na:1.8.0_65]
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
> ~[na:1.8.0_65]
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
> ~[na:1.8.0_65]
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
> ~[na:1.8.0_65]
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
> ~[na:1.8.0_65]
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
> ~[na:1.8.0_65]
> at
> org.apache.kafka.common.utils.AppInfoParser.registerAppInfo(AppInfoParser.java:57)
> ~[kafka-clients-0.9.0.0.jar:na]
> at
> org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:314)
> [kafka-clients-0.9.0.0.jar:na]
> at
> org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:194)
> [kafka-clients-0.9.0.0.jar:na]
> at
> org.apache.nifi.processors.kafka.PutKafka.createProducer(PutKafka.java:325)
> [nifi-kafka-processors-0.5.1.jar:0.5.1]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.8.0_65]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[na:1.8.0_65]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_65]
> at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_65]
> at
> org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:102)
> [nifi-framework-core-0.5.1-SNAPSHOT.jar:0.5.1]
> at
> org.apache.nifi.controller.scheduling.StandardProcessScheduler$4.run(StandardProcessScheduler.java:366)
> [nifi-framework-core-0.5.1-SNAPSHOT.jar:0.5.1-SNAPSHOT]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_65]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_65]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_65]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> [na:1.8.0_65]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_65]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_65]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1537) Add SNMP processors
[ https://issues.apache.org/jira/browse/NIFI-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188899#comment-15188899 ] Pierre Villard commented on NIFI-1537: -- [~joewitt] no problem on my side if it is not in the release next week. Besides [~michal.klempa], do you have time to have a look on this and give a feedback regarding the use case you told me? > Add SNMP processors > --- > > Key: NIFI-1537 > URL: https://issues.apache.org/jira/browse/NIFI-1537 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 0.5.0 >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Minor > Fix For: 0.6.0 > > > Add SNMP processors: > - GetSNMP to allow "get"/"walk" > - SetSNMP to allow "set" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1438) Unexpected results using MergeProcessor
[ https://issues.apache.org/jira/browse/NIFI-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-1438: - Attachment: NIFI-1438-template.xml Template to reproduce issue. > Unexpected results using MergeProcessor > > > Key: NIFI-1438 > URL: https://issues.apache.org/jira/browse/NIFI-1438 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 0.4.1 > Environment: OSX 10.10.5, Java 8u45 >Reporter: Josh Harrison > Attachments: NIFI-1438-template.xml, nifi-merge-problem.xml, > nifi-problem.tgz > > > Hello, I'm opening a ticket in reference to the stack overflow question I had > at > http://stackoverflow.com/questions/34958347/mergecontent-with-nifi-inconsistent-length > > To summarize, despite Aldrin's help, I have been unable to get the expected > merge behavior out of a template like the one attached, ingesting data like > is attached. > The goal is to ingest all of the zips in /tmp/nifidemo/source, extract the > zip files contained therein, each line being a json object. With json > routing, I extract and route for further processing ONLY items where the > "tags" item contains the tag "xyz". > These routed files should be aggregated by "mergeContent" into a bucket with, > at minimum, 1000 lines – or after being starved for 30 seconds, whatever > occurs first. > The behavior observed in my real template is replicated in this example – > merge content appears to be routing to buckets based on the original file > name, and not aggregating 1000 lines at a time as expected. Within a few > seconds of the template being run, many files are written with unexpected > line counts. > More confusingly, this isn't a consistent pattern - files may be run > repeatedly and do not generate the same number of lines in the result each > time. > The content of the input files was randomly generated so that approximately > 10% of the objects would contain the tag "xyz" (5000 lines in each input > file, there should be approximately 500 lines of – there are result files > that contain over 400 lines, but many contain 15-30 lines. There are also a > number of files with a "uuid.json" style name, all containing one line. > The attached contains a generic template that replicates the problem – it > seems to throw some errors but they don't appear to be related to the problem > I'm working on (and my real template doesn't throw the failures, but still > exhibits the same behavior). > I am running Nifi 0.4.1 on a Mac OSX 10.10.5 system and JRE 8u45. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1438) Unexpected results using MergeProcessor
[ https://issues.apache.org/jira/browse/NIFI-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15183282#comment-15183282 ] Pierre Villard commented on NIFI-1438: -- [~hijakk] I tried to have a look to your problem but it seems that the template you gave is incomplete (only containing GetFile processor). Could you update the attached template? Meanwhile, I'll try to reproduce your observation with the data you gave. Thanks! > Unexpected results using MergeProcessor > > > Key: NIFI-1438 > URL: https://issues.apache.org/jira/browse/NIFI-1438 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 0.4.1 > Environment: OSX 10.10.5, Java 8u45 >Reporter: Josh Harrison > Attachments: nifi-merge-problem.xml, nifi-problem.tgz > > > Hello, I'm opening a ticket in reference to the stack overflow question I had > at > http://stackoverflow.com/questions/34958347/mergecontent-with-nifi-inconsistent-length > > To summarize, despite Aldrin's help, I have been unable to get the expected > merge behavior out of a template like the one attached, ingesting data like > is attached. > The goal is to ingest all of the zips in /tmp/nifidemo/source, extract the > zip files contained therein, each line being a json object. With json > routing, I extract and route for further processing ONLY items where the > "tags" item contains the tag "xyz". > These routed files should be aggregated by "mergeContent" into a bucket with, > at minimum, 1000 lines – or after being starved for 30 seconds, whatever > occurs first. > The behavior observed in my real template is replicated in this example – > merge content appears to be routing to buckets based on the original file > name, and not aggregating 1000 lines at a time as expected. Within a few > seconds of the template being run, many files are written with unexpected > line counts. > More confusingly, this isn't a consistent pattern - files may be run > repeatedly and do not generate the same number of lines in the result each > time. > The content of the input files was randomly generated so that approximately > 10% of the objects would contain the tag "xyz" (5000 lines in each input > file, there should be approximately 500 lines of – there are result files > that contain over 400 lines, but many contain 15-30 lines. There are also a > number of files with a "uuid.json" style name, all containing one line. > The attached contains a generic template that replicates the problem – it > seems to throw some errors but they don't appear to be related to the problem > I'm working on (and my real template doesn't throw the failures, but still > exhibits the same behavior). > I am running Nifi 0.4.1 on a Mac OSX 10.10.5 system and JRE 8u45. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
