[jira] [Commented] (MINIFI-418) MiNiFi does not work as Windows service on 64bit machine
[
https://issues.apache.org/jira/browse/MINIFI-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289409#comment-16289409
]
Koji Kawamura commented on MINIFI-418:
--
Hi [~aldrin]
I've posted a comment on the PR, please see those. But in short, the 32 bit
minifi.exe searches 32 bit JRE and if it's not installed, then the service
fails to start as it can't start Java.
{code}
[2017-12-13 14:57:43] [error] [ 6096] Failed creating Java
[2017-12-13 14:57:43] [error] [ 6096] ServiceStart returned 1
[2017-12-13 14:57:44] [error] [ 4756] apxServiceControl(): dwState(4) !=
dwCurrentState(1); dwWin32ExitCode = 1066, dwWaitHint = 0,
dwServiceSpecificExitCode = 1
[2017-12-13 14:57:44] [error] [ 4756] apxServiceControl(): returning FALSE
[2017-12-13 14:57:44] [error] [ 4756] Failed to start service 'minifi'
[2017-12-13 14:57:44] [error] [ 4756] Commons Daemon procrun failed with exit
value: 5 (Failed to start service)
{code}
> MiNiFi does not work as Windows service on 64bit machine
>
>
> Key: MINIFI-418
> URL: https://issues.apache.org/jira/browse/MINIFI-418
> Project: Apache NiFi MiNiFi
> Issue Type: Improvement
> Components: Agent Configuration/Installation
>Affects Versions: 0.3.0
> Environment: Windows Server 2016 Datacenter, AMD64
>Reporter: Koji Kawamura
>
> MINIFI-344 added Windows service support. Running install-service.bat
> succeeded and minifi service was installed. However, the service couldn't
> start on a 64bit Windows env because the bundled 'minifi.exe' is
> 'prunsrv.exe' program from Commons Daemon for 32bit.
> Workaround is downloading Commond Daemon binary, then replace
> "minifi-home/bin/minifi.exe' with
> 'commons-daemon-1.1.0-bin-windows\amd64\prunsrv.exe'
> https://www.apache.org/dist/commons/daemon/binaries/windows/
> It would be great if it is handled automatically, but at least we should
> document it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (MINIFI-418) MiNiFi does not work as Windows service on 64bit machine
[ https://issues.apache.org/jira/browse/MINIFI-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated MINIFI-418: - Environment: Windows Server 2016 Datacenter, AMD64 > MiNiFi does not work as Windows service on 64bit machine > > > Key: MINIFI-418 > URL: https://issues.apache.org/jira/browse/MINIFI-418 > Project: Apache NiFi MiNiFi > Issue Type: Improvement > Components: Agent Configuration/Installation >Affects Versions: 0.3.0 > Environment: Windows Server 2016 Datacenter, AMD64 >Reporter: Koji Kawamura > > MINIFI-344 added Windows service support. Running install-service.bat > succeeded and minifi service was installed. However, the service couldn't > start on a 64bit Windows env because the bundled 'minifi.exe' is > 'prunsrv.exe' program from Commons Daemon for 32bit. > Workaround is downloading Commond Daemon binary, then replace > "minifi-home/bin/minifi.exe' with > 'commons-daemon-1.1.0-bin-windows\amd64\prunsrv.exe' > https://www.apache.org/dist/commons/daemon/binaries/windows/ > It would be great if it is handled automatically, but at least we should > document it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (MINIFI-419) delete-service.bat fails to delete minifi service
Koji Kawamura created MINIFI-419:
Summary: delete-service.bat fails to delete minifi service
Key: MINIFI-419
URL: https://issues.apache.org/jira/browse/MINIFI-419
Project: Apache NiFi MiNiFi
Issue Type: Bug
Components: Agent Configuration/Installation
Affects Versions: 0.3.0
Environment: Windows Server 2016 Datacenter, AMD64
Reporter: Koji Kawamura
After minifi service was installed using install-service.bat, I tried deleting
the service by delete-service.bat. The batch command failed with following
error:
{code}
C:\dev\minifi-0.3.0-vote\minifi-0.3.0\minifi-assembly\target\minifi-0.3.0-bin\minifi-0.3.0\bin>delete-service.bat
[2017-12-13 13:17:54] [error] [ 5356] Unrecognized cmd option .exe
[2017-12-13 13:17:54] [error] [ 5356] Invalid command line arguments
[2017-12-13 13:17:54] [error] [ 5356] Commons Daemon procrun failed with exit
value: 1 (Failed to parse command line arguments)
{code}
Removing the double quart around the SERVICE_NAME variable made
delete-service.bat to work.
https://github.com/apache/nifi-minifi/blob/master/minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-resources/src/main/resources/bin/delete-service.bat#L19
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (MINIFI-418) MiNiFi does not work as Windows service on 64bit machine
[ https://issues.apache.org/jira/browse/MINIFI-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated MINIFI-418: - Description: MINIFI-344 added Windows service support. Running install-service.bat succeeded and minifi service was installed. However, the service couldn't start on a 64bit Windows env because the bundled 'minifi.exe' is 'prunsrv.exe' program from Commons Daemon for 32bit. Workaround is downloading Commond Daemon binary, then replace "minifi-home/bin/minifi.exe' with 'commons-daemon-1.1.0-bin-windows\amd64\prunsrv.exe' https://www.apache.org/dist/commons/daemon/binaries/windows/ It would be great if it is handled automatically, but at least we should document it. was: MINIFI-344 added Windows service support. Running install-service.bat succeeded and minifi service was installed. However, the service couldn't start on a 64bit Windows env because the bundled 'minifi.exe' is 'prunsrv.exe' program from Commons Daemon for 32bit. Workaround is downloading Commond Daemon binary, then replace "minifi-home/bin/minifi.exe' with 'commons-daemon-1.1.0-bin-windows\amd64\prunsrv.exe' https://www.apache.org/dist/commons/daemon/binaries/windows/ Id would be great if it is handled automatically, but at least we should document it. > MiNiFi does not work as Windows service on 64bit machine > > > Key: MINIFI-418 > URL: https://issues.apache.org/jira/browse/MINIFI-418 > Project: Apache NiFi MiNiFi > Issue Type: Improvement > Components: Agent Configuration/Installation >Affects Versions: 0.3.0 >Reporter: Koji Kawamura > > MINIFI-344 added Windows service support. Running install-service.bat > succeeded and minifi service was installed. However, the service couldn't > start on a 64bit Windows env because the bundled 'minifi.exe' is > 'prunsrv.exe' program from Commons Daemon for 32bit. > Workaround is downloading Commond Daemon binary, then replace > "minifi-home/bin/minifi.exe' with > 'commons-daemon-1.1.0-bin-windows\amd64\prunsrv.exe' > https://www.apache.org/dist/commons/daemon/binaries/windows/ > It would be great if it is handled automatically, but at least we should > document it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (MINIFI-418) MiNiFi does not work as Windows service on 64bit machine
Koji Kawamura created MINIFI-418: Summary: MiNiFi does not work as Windows service on 64bit machine Key: MINIFI-418 URL: https://issues.apache.org/jira/browse/MINIFI-418 Project: Apache NiFi MiNiFi Issue Type: Improvement Components: Agent Configuration/Installation Affects Versions: 0.3.0 Reporter: Koji Kawamura MINIFI-344 added Windows service support. Running install-service.bat succeeded and minifi service was installed. However, the service couldn't start on a 64bit Windows env because the bundled 'minifi.exe' is 'prunsrv.exe' program from Commons Daemon for 32bit. Workaround is downloading Commond Daemon binary, then replace "minifi-home/bin/minifi.exe' with 'commons-daemon-1.1.0-bin-windows\amd64\prunsrv.exe' https://www.apache.org/dist/commons/daemon/binaries/windows/ Id would be great if it is handled automatically, but at least we should document it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFI-415) Bundle version number should not be compared as a simple String
[
https://issues.apache.org/jira/browse/MINIFI-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288736#comment-16288736
]
Koji Kawamura commented on MINIFI-415:
--
[~aldrin] Yeah, I think we can not make any assumption on the version string
format. Current approach would make sense in most cases. I didn't think this is
a major issue, so marked as 'Minor'. I think it's fine if user knows which
version of NAR is currently used. Logging the information would suffice. The
current code logs that, but it seems it forgot passing arguments to the logger:
{code}
logger.info("Enriching {} with bundle {}", new Object[]{});
{code}
https://github.com/apache/nifi-minifi/blob/master/minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-runtime/src/main/java/org/apache/nifi/minifi/FlowEnricher.java#L142
Fixing above logging and additionl warning message to describe a version is
automatically picked would be fine. Thanks!
> Bundle version number should not be compared as a simple String
> ---
>
> Key: MINIFI-415
> URL: https://issues.apache.org/jira/browse/MINIFI-415
> Project: Apache NiFi MiNiFi
> Issue Type: Bug
> Components: Agent Configuration/Installation
>Affects Versions: 0.3.0
>Reporter: Koji Kawamura
>Priority: Minor
>
> MINIFI-408 added support for picking the latest bundle version automatically
> if there are multiple versions for the same Nar. However, it compares version
> as simple Strings and may not be able to pick the latest one semantically.
> https://github.com/apache/nifi-minifi/pull/99/files#diff-c7d8398db8540d6e85f1a9207438ebddR138
> Following code shows the problematic inputs and possible solution. Current
> implementation picks "1.0.9", and using Version class, it can pick "1.0.12".
> {code}
> class Test {
> public static void main(String[] args) throws Exception {
> Set componentToEnrichBundleVersions = new HashSet<>();
> componentToEnrichBundleVersions.add("1.0.0");
> componentToEnrichBundleVersions.add("1.0.5");
> componentToEnrichBundleVersions.add("1.0.9");
> componentToEnrichBundleVersions.add("1.0.11-SNAPSHOT");
> componentToEnrichBundleVersions.add("1.0.12");
> // Current implementation
> final String bundleVersion =
> componentToEnrichBundleVersions.stream().sorted()
> .reduce((version, otherVersion) -> otherVersion).orElse(null);
> // Suggestion
> final Version latestVersion =
> componentToEnrichBundleVersions.stream().map(Version::fromString).sorted()
> .reduce((version, otherVersion) -> otherVersion).orElse(null);
> System.out.println(bundleVersion);
> System.out.println(latestVersion.toVersionString());
> }
> }
> class Version implements Comparable {
> private static Pattern P =
> Pattern.compile("^([\\d]+)\\.([\\d]+)\\.([\\d]+)([^\\d]*)$");
> final int major;
> final int minor;
> final int patch;
> final String opt;
> public static Version fromString(String s) {
> final Matcher matcher = P.matcher(s);
> if (matcher.matches()) {
> return new Version(
> Integer.parseInt(matcher.group(1)),
> Integer.parseInt(matcher.group(2)),
> Integer.parseInt(matcher.group(3)),
> matcher.group(4));
> }
> throw new IllegalArgumentException("Unknown version pattern " + s);
> }
> private Version(int major, int minor, int patch, String opt) {
> this.major = major;
> this.minor = minor;
> this.patch = patch;
> this.opt = opt;
> }
> @Override
> public int compareTo(@NotNull Version o) {
> final int ma = new Integer(major).compareTo(o.major);
> if (ma != 0) {
> return ma;
> }
> final int mi = new Integer(minor).compareTo(o.minor);
> if (mi != 0) {
> return mi;
> }
> final int pa = new Integer(patch).compareTo(o.patch);
> if (pa != 0) {
> return pa;
> }
> final String o1 = opt != null ? opt : "";
> final String o2 = o.opt != null ? o.opt : "";
> final int op = o1.compareTo(o2);
> return op;
> }
> public String toVersionString() {
> return String.format("%d.%d.%d%s", major, minor, patch, opt);
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (MINIFI-415) Bundle version number should not be compared as a simple String
[
https://issues.apache.org/jira/browse/MINIFI-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288615#comment-16288615
]
Koji Kawamura commented on MINIFI-415:
--
[~joewitt] I agree with that. As written in MINIFI-408, NiFi let user choose
the desired one, while MiNiFi has to pick one deterministically. I am not sure
if we can force Bundle to have a semantic versioning for custom Nars. But if it
is built by Apache NiFi project, it will be guaranteed.
Bundle class also have "buildTimestamp" property which may be used for
comparison.
> Bundle version number should not be compared as a simple String
> ---
>
> Key: MINIFI-415
> URL: https://issues.apache.org/jira/browse/MINIFI-415
> Project: Apache NiFi MiNiFi
> Issue Type: Bug
> Components: Agent Configuration/Installation
>Affects Versions: 0.3.0
>Reporter: Koji Kawamura
>Priority: Minor
>
> MINIFI-408 added support for picking the latest bundle version automatically
> if there are multiple versions for the same Nar. However, it compares version
> as simple Strings and may not be able to pick the latest one semantically.
> https://github.com/apache/nifi-minifi/pull/99/files#diff-c7d8398db8540d6e85f1a9207438ebddR138
> Following code shows the problematic inputs and possible solution. Current
> implementation picks "1.0.9", and using Version class, it can pick "1.0.12".
> {code}
> class Test {
> public static void main(String[] args) throws Exception {
> Set componentToEnrichBundleVersions = new HashSet<>();
> componentToEnrichBundleVersions.add("1.0.0");
> componentToEnrichBundleVersions.add("1.0.5");
> componentToEnrichBundleVersions.add("1.0.9");
> componentToEnrichBundleVersions.add("1.0.11-SNAPSHOT");
> componentToEnrichBundleVersions.add("1.0.12");
> // Current implementation
> final String bundleVersion =
> componentToEnrichBundleVersions.stream().sorted()
> .reduce((version, otherVersion) -> otherVersion).orElse(null);
> // Suggestion
> final Version latestVersion =
> componentToEnrichBundleVersions.stream().map(Version::fromString).sorted()
> .reduce((version, otherVersion) -> otherVersion).orElse(null);
> System.out.println(bundleVersion);
> System.out.println(latestVersion.toVersionString());
> }
> }
> class Version implements Comparable {
> private static Pattern P =
> Pattern.compile("^([\\d]+)\\.([\\d]+)\\.([\\d]+)([^\\d]*)$");
> final int major;
> final int minor;
> final int patch;
> final String opt;
> public static Version fromString(String s) {
> final Matcher matcher = P.matcher(s);
> if (matcher.matches()) {
> return new Version(
> Integer.parseInt(matcher.group(1)),
> Integer.parseInt(matcher.group(2)),
> Integer.parseInt(matcher.group(3)),
> matcher.group(4));
> }
> throw new IllegalArgumentException("Unknown version pattern " + s);
> }
> private Version(int major, int minor, int patch, String opt) {
> this.major = major;
> this.minor = minor;
> this.patch = patch;
> this.opt = opt;
> }
> @Override
> public int compareTo(@NotNull Version o) {
> final int ma = new Integer(major).compareTo(o.major);
> if (ma != 0) {
> return ma;
> }
> final int mi = new Integer(minor).compareTo(o.minor);
> if (mi != 0) {
> return mi;
> }
> final int pa = new Integer(patch).compareTo(o.patch);
> if (pa != 0) {
> return pa;
> }
> final String o1 = opt != null ? opt : "";
> final String o2 = o.opt != null ? o.opt : "";
> final int op = o1.compareTo(o2);
> return op;
> }
> public String toVersionString() {
> return String.format("%d.%d.%d%s", major, minor, patch, opt);
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (MINIFI-415) Bundle version number should not be compared as a simple String
[
https://issues.apache.org/jira/browse/MINIFI-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288606#comment-16288606
]
Koji Kawamura commented on MINIFI-415:
--
The suggested code assumes Bundle version uses semantic versioning. If it looks
reasonable, I'm happy to provide a PR using that approach. Thanks!
> Bundle version number should not be compared as a simple String
> ---
>
> Key: MINIFI-415
> URL: https://issues.apache.org/jira/browse/MINIFI-415
> Project: Apache NiFi MiNiFi
> Issue Type: Bug
> Components: Agent Configuration/Installation
>Affects Versions: 0.3.0
>Reporter: Koji Kawamura
>Priority: Minor
>
> MINIFI-408 added support for picking the latest bundle version automatically
> if there are multiple versions for the same Nar. However, it compares version
> as simple Strings and may not be able to pick the latest one semantically.
> https://github.com/apache/nifi-minifi/pull/99/files#diff-c7d8398db8540d6e85f1a9207438ebddR138
> Following code shows the problematic inputs and possible solution. Current
> implementation picks "1.0.9", and using Version class, it can pick "1.0.12".
> {code}
> class Test {
> public static void main(String[] args) throws Exception {
> Set componentToEnrichBundleVersions = new HashSet<>();
> componentToEnrichBundleVersions.add("1.0.0");
> componentToEnrichBundleVersions.add("1.0.5");
> componentToEnrichBundleVersions.add("1.0.9");
> componentToEnrichBundleVersions.add("1.0.11-SNAPSHOT");
> componentToEnrichBundleVersions.add("1.0.12");
> // Current implementation
> final String bundleVersion =
> componentToEnrichBundleVersions.stream().sorted()
> .reduce((version, otherVersion) -> otherVersion).orElse(null);
> // Suggestion
> final Version latestVersion =
> componentToEnrichBundleVersions.stream().map(Version::fromString).sorted()
> .reduce((version, otherVersion) -> otherVersion).orElse(null);
> System.out.println(bundleVersion);
> System.out.println(latestVersion.toVersionString());
> }
> }
> class Version implements Comparable {
> private static Pattern P =
> Pattern.compile("^([\\d]+)\\.([\\d]+)\\.([\\d]+)([^\\d]*)$");
> final int major;
> final int minor;
> final int patch;
> final String opt;
> public static Version fromString(String s) {
> final Matcher matcher = P.matcher(s);
> if (matcher.matches()) {
> return new Version(
> Integer.parseInt(matcher.group(1)),
> Integer.parseInt(matcher.group(2)),
> Integer.parseInt(matcher.group(3)),
> matcher.group(4));
> }
> throw new IllegalArgumentException("Unknown version pattern " + s);
> }
> private Version(int major, int minor, int patch, String opt) {
> this.major = major;
> this.minor = minor;
> this.patch = patch;
> this.opt = opt;
> }
> @Override
> public int compareTo(@NotNull Version o) {
> final int ma = new Integer(major).compareTo(o.major);
> if (ma != 0) {
> return ma;
> }
> final int mi = new Integer(minor).compareTo(o.minor);
> if (mi != 0) {
> return mi;
> }
> final int pa = new Integer(patch).compareTo(o.patch);
> if (pa != 0) {
> return pa;
> }
> final String o1 = opt != null ? opt : "";
> final String o2 = o.opt != null ? o.opt : "";
> final int op = o1.compareTo(o2);
> return op;
> }
> public String toVersionString() {
> return String.format("%d.%d.%d%s", major, minor, patch, opt);
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (MINIFI-415) Bundle version number should not be compared as a simple String
Koji Kawamura created MINIFI-415:
Summary: Bundle version number should not be compared as a simple
String
Key: MINIFI-415
URL: https://issues.apache.org/jira/browse/MINIFI-415
Project: Apache NiFi MiNiFi
Issue Type: Bug
Components: Agent Configuration/Installation
Affects Versions: 0.3.0
Reporter: Koji Kawamura
Priority: Minor
MINIFI-408 added support for picking the latest bundle version automatically if
there are multiple versions for the same Nar. However, it compares version as
simple Strings and may not be able to pick the latest one semantically.
https://github.com/apache/nifi-minifi/pull/99/files#diff-c7d8398db8540d6e85f1a9207438ebddR138
Following code shows the problematic inputs and possible solution. Current
implementation picks "1.0.9", and using Version class, it can pick "1.0.12".
{code}
class Test {
public static void main(String[] args) throws Exception {
Set componentToEnrichBundleVersions = new HashSet<>();
componentToEnrichBundleVersions.add("1.0.0");
componentToEnrichBundleVersions.add("1.0.5");
componentToEnrichBundleVersions.add("1.0.9");
componentToEnrichBundleVersions.add("1.0.11-SNAPSHOT");
componentToEnrichBundleVersions.add("1.0.12");
// Current implementation
final String bundleVersion =
componentToEnrichBundleVersions.stream().sorted()
.reduce((version, otherVersion) -> otherVersion).orElse(null);
// Suggestion
final Version latestVersion =
componentToEnrichBundleVersions.stream().map(Version::fromString).sorted()
.reduce((version, otherVersion) -> otherVersion).orElse(null);
System.out.println(bundleVersion);
System.out.println(latestVersion.toVersionString());
}
}
class Version implements Comparable {
private static Pattern P =
Pattern.compile("^([\\d]+)\\.([\\d]+)\\.([\\d]+)([^\\d]*)$");
final int major;
final int minor;
final int patch;
final String opt;
public static Version fromString(String s) {
final Matcher matcher = P.matcher(s);
if (matcher.matches()) {
return new Version(
Integer.parseInt(matcher.group(1)),
Integer.parseInt(matcher.group(2)),
Integer.parseInt(matcher.group(3)),
matcher.group(4));
}
throw new IllegalArgumentException("Unknown version pattern " + s);
}
private Version(int major, int minor, int patch, String opt) {
this.major = major;
this.minor = minor;
this.patch = patch;
this.opt = opt;
}
@Override
public int compareTo(@NotNull Version o) {
final int ma = new Integer(major).compareTo(o.major);
if (ma != 0) {
return ma;
}
final int mi = new Integer(minor).compareTo(o.minor);
if (mi != 0) {
return mi;
}
final int pa = new Integer(patch).compareTo(o.patch);
if (pa != 0) {
return pa;
}
final String o1 = opt != null ? opt : "";
final String o2 = o.opt != null ? o.opt : "";
final int op = o1.compareTo(o2);
return op;
}
public String toVersionString() {
return String.format("%d.%d.%d%s", major, minor, patch, opt);
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (MINIFI-147) MiNiFi C++ startup error due to property name mismatch in config.yml
Koji Kawamura created MINIFI-147:
Summary: MiNiFi C++ startup error due to property name mismatch in
config.yml
Key: MINIFI-147
URL: https://issues.apache.org/jira/browse/MINIFI-147
Project: Apache NiFi MiNiFi
Issue Type: Bug
Components: C++
Affects Versions: 0.1.0
Reporter: Koji Kawamura
A config.yml file created by minifi-toolkit-0.1.0 "config.sh transform" command
causes following MiNiFi C++ startup error due to several naming mismatch
between MiNiFi C++ and minifi-toolkit.
{code}
pi@raspberrypi:~/dev/nifi-minifi-cpp-0.1.0 $ terminate called after throwing an
instance of 'YAML::TypedBadConversion'
what(): bad conversion
{code}
I had to modify generated conf.yml as below to start MiNiFi C++:
* Connections:
** source name: TailFile (cpp expects)
** source id: b9340f85-0158-1000-- (toolkit generated)
** source relationship name: success (cpp)
** source relationship names: - success (toolkit generated array)
** destination name: b52ba2de-0158-1000-fda0-5861ac7dbb2f (cpp)
** destination id: b52ba2de-0158-1000-fda0-5861ac7dbb2f (toolkit)
* Remote Processing Groups (cpp)
* Remote Process Groups (toolkit)
** Port and Host properties for Input Ports are also added manually to start
MiNiFi, but this is documented properly.
MINIFI-137 has changed constant name. We need to check if such change affects
MiNiFi C++ as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (NIFI-2145) When saving flow, NiFi should allow the flow to automatically be backed up
[
https://issues.apache.org/jira/browse/NIFI-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Koji Kawamura reassigned NIFI-2145:
---
Assignee: Koji Kawamura
> When saving flow, NiFi should allow the flow to automatically be backed up
> --
>
> Key: NIFI-2145
> URL: https://issues.apache.org/jira/browse/NIFI-2145
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Configuration, Core Framework
>Reporter: Mark Payne
>Assignee: Koji Kawamura
> Fix For: 1.0.0
>
>
> Each time that a user updates the NiFi flow, the new flow is written to the
> flow.xml.gz on each node. When this occurs, we should provide the option of
> also writing the flow.xml.gz to an 'archive' folder, using a filename based
> on the current time. We should also ensure that we keep old flows for X
> (configurable) amount of time and only use a max of some configurable amount
> of disk space.
> StandardXMLFlowConfigurationDAO (in nifi-framework-core) already has a method
> for `save(FlowController controller, boolean archive)` but 'false' is always
> passed as the 'archive' field. We should instead pass in `true` if configured
> to archive. Additionally, we should change the archive filename so that it is
> based on a user-readable date/time, such as
> flow-2016-06-29-11-10-50am.xml.gz, for instance, for a flow that was saved at
> 11:10:50 AM on June 29, 2016. This allows an admin to easily revert a flow
> back to how it was at some point in history.
> Changes required to implement this include:
> 1. Update nifi.properties to include new properties (with the suggested
> defaults that may need to be adjusted):
> - nifi.flow.configuration.archive.enabled=true
> - nifi.flow.configuration.archive.max.time=30 days
> - nifi.flow.configuration.archive.max.storage=500 MB
> There already is a property named `nifi.flow.configuration.archive.dir` that
> should be used to specify where to save the archived flow. This is used
> currently only when the user clicks the 'archive' button in the UI.
> 2. Update StandardFlowService.saveFlowChanges(TimeUnit delayUnit, long delay)
> so that it passes a value of `true` for the 'archive' flag when calling the
> overridden saveFlowChanges method in the case that the
> `nifi.flow.configuration.archive.enabled` property is set to true (default).
> 3. Update StandardXMLFlowConfigurationDAO.createArchiveFile() to use a more
> user-friendly filename, as described above.
> 4. Update StandardXMLFlowConfigurationDAO.save() so that the `if (archive) {`
> branch first cleans up old/expired flows so that the `archive` directory is
> smaller than the configured max storage space and so that any archived flow
> that is older than the configured `nifi.flow.configuration.archive.max.time`
> property is removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-906) Docs do not display well on mobile/smaller res devices
[ https://issues.apache.org/jira/browse/NIFI-906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-906: --- Attachment: smartphone.png smartphone-hide-list.png pc.png pc-hide-list.png Finished making the docs to display well on mobile/smaller devices using flex-box. Please see attached images. > Docs do not display well on mobile/smaller res devices > -- > > Key: NIFI-906 > URL: https://issues.apache.org/jira/browse/NIFI-906 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation & Website > Environment: Mobile, Small res devices >Reporter: Joseph Witt >Assignee: Koji Kawamura > Attachments: pc-hide-list.png, pc.png, smartphone-hide-list.png, > smartphone.png > > > Tweet from @cyroxx > @apachenifi Unfortunately, the docs (http://nifi.apache.org/docs.html ) are > not mobile-friendly, makes it hard to read about this interesting project > https://twitter.com/cyroxx/status/637305973938483200 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-906) Docs do not display well on mobile/smaller res devices
[ https://issues.apache.org/jira/browse/NIFI-906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-906: --- Attachment: (was: small-display-hide-list.png) > Docs do not display well on mobile/smaller res devices > -- > > Key: NIFI-906 > URL: https://issues.apache.org/jira/browse/NIFI-906 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation & Website > Environment: Mobile, Small res devices >Reporter: Joseph Witt >Assignee: Koji Kawamura > > Tweet from @cyroxx > @apachenifi Unfortunately, the docs (http://nifi.apache.org/docs.html ) are > not mobile-friendly, makes it hard to read about this interesting project > https://twitter.com/cyroxx/status/637305973938483200 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-906) Docs do not display well on mobile/smaller res devices
[ https://issues.apache.org/jira/browse/NIFI-906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-906: --- Attachment: (was: large-display-hide-list.png) > Docs do not display well on mobile/smaller res devices > -- > > Key: NIFI-906 > URL: https://issues.apache.org/jira/browse/NIFI-906 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation & Website > Environment: Mobile, Small res devices >Reporter: Joseph Witt >Assignee: Koji Kawamura > > Tweet from @cyroxx > @apachenifi Unfortunately, the docs (http://nifi.apache.org/docs.html ) are > not mobile-friendly, makes it hard to read about this interesting project > https://twitter.com/cyroxx/status/637305973938483200 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-906) Docs do not display well on mobile/smaller res devices
[ https://issues.apache.org/jira/browse/NIFI-906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-906: --- Attachment: (was: large-display.png) > Docs do not display well on mobile/smaller res devices > -- > > Key: NIFI-906 > URL: https://issues.apache.org/jira/browse/NIFI-906 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation & Website > Environment: Mobile, Small res devices >Reporter: Joseph Witt >Assignee: Koji Kawamura > > Tweet from @cyroxx > @apachenifi Unfortunately, the docs (http://nifi.apache.org/docs.html ) are > not mobile-friendly, makes it hard to read about this interesting project > https://twitter.com/cyroxx/status/637305973938483200 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-906) Docs do not display well on mobile/smaller res devices
[ https://issues.apache.org/jira/browse/NIFI-906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-906: --- Attachment: (was: small-display.png) > Docs do not display well on mobile/smaller res devices > -- > > Key: NIFI-906 > URL: https://issues.apache.org/jira/browse/NIFI-906 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation & Website > Environment: Mobile, Small res devices >Reporter: Joseph Witt >Assignee: Koji Kawamura > > Tweet from @cyroxx > @apachenifi Unfortunately, the docs (http://nifi.apache.org/docs.html ) are > not mobile-friendly, makes it hard to read about this interesting project > https://twitter.com/cyroxx/status/637305973938483200 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-906) Docs do not display well on mobile/smaller res devices
[ https://issues.apache.org/jira/browse/NIFI-906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura reassigned NIFI-906: -- Assignee: Koji Kawamura > Docs do not display well on mobile/smaller res devices > -- > > Key: NIFI-906 > URL: https://issues.apache.org/jira/browse/NIFI-906 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation & Website > Environment: Mobile, Small res devices >Reporter: Joseph Witt >Assignee: Koji Kawamura > Attachments: large-display-hide-list.png, large-display.png, > small-display-hide-list.png, small-display.png > > > Tweet from @cyroxx > @apachenifi Unfortunately, the docs (http://nifi.apache.org/docs.html ) are > not mobile-friendly, makes it hard to read about this interesting project > https://twitter.com/cyroxx/status/637305973938483200 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (NIFI-906) Docs do not display well on mobile/smaller res devices
[ https://issues.apache.org/jira/browse/NIFI-906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358046#comment-15358046 ] Koji Kawamura edited comment on NIFI-906 at 6/30/16 11:38 PM: -- Tested flex-box. I still have some layout issues but it mostly works and looks as attached files. Made the left component selection pane able to be hidden, so that wider area can be used for main content. I will address remaining issues and submit a PR next week. was (Author: ijokarumawak): Tested flex-box. I still have some layout issues but it mostly works and looks as attached files. > Docs do not display well on mobile/smaller res devices > -- > > Key: NIFI-906 > URL: https://issues.apache.org/jira/browse/NIFI-906 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation & Website > Environment: Mobile, Small res devices >Reporter: Joseph Witt > Attachments: large-display-hide-list.png, large-display.png, > small-display-hide-list.png, small-display.png > > > Tweet from @cyroxx > @apachenifi Unfortunately, the docs (http://nifi.apache.org/docs.html ) are > not mobile-friendly, makes it hard to read about this interesting project > https://twitter.com/cyroxx/status/637305973938483200 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-906) Docs do not display well on mobile/smaller res devices
[ https://issues.apache.org/jira/browse/NIFI-906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-906: --- Attachment: small-display.png small-display-hide-list.png large-display.png large-display-hide-list.png Tested flex-box. I still have some layout issues but it mostly works and looks as attached files. > Docs do not display well on mobile/smaller res devices > -- > > Key: NIFI-906 > URL: https://issues.apache.org/jira/browse/NIFI-906 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation & Website > Environment: Mobile, Small res devices >Reporter: Joseph Witt > Attachments: large-display-hide-list.png, large-display.png, > small-display-hide-list.png, small-display.png > > > Tweet from @cyroxx > @apachenifi Unfortunately, the docs (http://nifi.apache.org/docs.html ) are > not mobile-friendly, makes it hard to read about this interesting project > https://twitter.com/cyroxx/status/637305973938483200 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (NIFI-906) Docs do not display well on mobile/smaller res devices
[
https://issues.apache.org/jira/browse/NIFI-906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355244#comment-15355244
]
Koji Kawamura edited comment on NIFI-906 at 6/29/16 11:21 PM:
--
I took a look at the documentation page (https://nifi.apache.org/docs.html). It
already uses foundation.js to provide responsive pages, however it doesn't work
on the documentation page, other pages looks fine.
I think there're two issues:
# NiFi Documentation is embedded as iframe. The iframe hides top-bar menu and
footer with narrower screen like mobile.
# The embedded NiFi Documentation is not responsive.
h3. Possible solution for #1
When I modified css as follows, the top-bar menu and footer was shown
correctly. I couldn't find the css file in nifi project available on Github,
though. I assume it's managed somewhere else.
{code}
/*
Iframe
*/
/* line 177, ../../../src/scss/app.scss */
div.external-guide {
position: absolute; // Change this to relative.
left: 0;
bottom: 0;
right: 0;
top: 2.8125rem; // Remove this.
z-index: 1; // Remove this.
background-color: #fff;
}
{code}
h3. Possible solution for #2
Since current web pages are using, I think we can start with foundation.js to
use in NiFi Documentation pages, or flexbox. Once we made the left component
listing pane responsive, it should be easier to see with small displays. I'm
going to test with flexbox first.
was (Author: ijokarumawak):
I took a look at the documentation page (https://nifi.apache.org/docs.html). It
already uses foundation.js to provide responsive pages, however it doesn't work
on the documentation page, other pages looks fine.
I think there're two issues:
# NiFi Documentation is embedded as iframe. The iframe hides top-bar menu and
footer with narrower screen like mobile.
# The embedded NiFi Documentation is not responsive.
h3. Possible solution for #1
When I modified css as follows, the top-bar menu and footer was shown
correctly. I couldn't find the css file in nifi project available on Github,
though. I assume it's managed somewhere else.
{code}
/*
Iframe
*/
/* line 177, ../../../src/scss/app.scss */
div.external-guide {
position: absolute; // Change this to relative.
left: 0;
bottom: 0;
right: 0;
top: 2.8125rem; // Remove this.
z-index: 1; // Remove this.
background-color: #fff;
}
{code}
h3. Possible solution for #2
Since current web pages are using, I think we can start with foundation.js to
use in NiFi Documentation pages. Once we can make the left component listing
pane responsive, it should be easier with small displays. Thoughts?
> Docs do not display well on mobile/smaller res devices
> --
>
> Key: NIFI-906
> URL: https://issues.apache.org/jira/browse/NIFI-906
> Project: Apache NiFi
> Issue Type: Bug
> Components: Documentation & Website
> Environment: Mobile, Small res devices
>Reporter: Joseph Witt
>
> Tweet from @cyroxx
> @apachenifi Unfortunately, the docs (http://nifi.apache.org/docs.html ) are
> not mobile-friendly, makes it hard to read about this interesting project
> https://twitter.com/cyroxx/status/637305973938483200
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-906) Docs do not display well on mobile/smaller res devices
[
https://issues.apache.org/jira/browse/NIFI-906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355244#comment-15355244
]
Koji Kawamura commented on NIFI-906:
I took a look at the documentation page (https://nifi.apache.org/docs.html). It
already uses foundation.js to provide responsive pages, however it doesn't work
on the documentation page, other pages looks fine.
I think there're two issues:
# NiFi Documentation is embedded as iframe. The iframe hides top-bar menu and
footer with narrower screen like mobile.
# The embedded NiFi Documentation is not responsive.
h3. Possible solution for #1
When I modified css as follows, the top-bar menu and footer was shown
correctly. I couldn't find the css file in nifi project available on Github,
though. I assume it's managed somewhere else.
{code}
/*
Iframe
*/
/* line 177, ../../../src/scss/app.scss */
div.external-guide {
position: absolute; // Change this to relative.
left: 0;
bottom: 0;
right: 0;
top: 2.8125rem; // Remove this.
z-index: 1; // Remove this.
background-color: #fff;
}
{code}
h3. Possible solution for #2
Since current web pages are using, I think we can start with foundation.js to
use in NiFi Documentation pages. Once we can make the left component listing
pane responsive, it should be easier with small displays. Thoughts?
> Docs do not display well on mobile/smaller res devices
> --
>
> Key: NIFI-906
> URL: https://issues.apache.org/jira/browse/NIFI-906
> Project: Apache NiFi
> Issue Type: Bug
> Components: Documentation & Website
> Environment: Mobile, Small res devices
>Reporter: Joseph Witt
>
> Tweet from @cyroxx
> @apachenifi Unfortunately, the docs (http://nifi.apache.org/docs.html ) are
> not mobile-friendly, makes it hard to read about this interesting project
> https://twitter.com/cyroxx/status/637305973938483200
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (NIFI-619) update MonitorActivity processor to be cluster friendly
[ https://issues.apache.org/jira/browse/NIFI-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura reassigned NIFI-619: -- Assignee: Koji Kawamura > update MonitorActivity processor to be cluster friendly > --- > > Key: NIFI-619 > URL: https://issues.apache.org/jira/browse/NIFI-619 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Brandon DeVries >Assignee: Koji Kawamura >Priority: Minor > > This processor should be able to be used to monitor activity across the > cluster. In its current state, alerting is based on activity of a single > node, not the entire cluster. > For example, in a 2 node cluster, if system A is getting data from a given > flow and system B is not, system B will alert for lack of activity even > though the flow is functioning "normally". > The ideal behavior would be fore an alert to be generated only if both > systems did not see data in the specified time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2093) Clear state link on Component State window is hidden
[ https://issues.apache.org/jira/browse/NIFI-2093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15347921#comment-15347921 ] Koji Kawamura commented on NIFI-2093: - [~mcgilman] I understand, thanks! > Clear state link on Component State window is hidden > > > Key: NIFI-2093 > URL: https://issues.apache.org/jira/browse/NIFI-2093 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Koji Kawamura > Fix For: 1.0.0 > > Attachments: ComponentState-ConsumeKafka.png > > > It seems that ComponentStateEntity should have accessPolicy so that > CanvasUtis.supportsModification() can handle whether the link is active or > not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2093) Clear state link on Component State window is hidden
[
https://issues.apache.org/jira/browse/NIFI-2093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15347451#comment-15347451
]
Koji Kawamura commented on NIFI-2093:
-
[~mcgilman] I looked at the Javascript codes and confirmed that only showState
function for Processors is using supportsModification.
It may make sense to add comparable check for Controller Services and Reporting
Tasks using supportsModification, however, if "Get" and "Clear" both require
"WRITE" access, then it should be fine as it is. If one doesn't have WRITE
access, they can't open the Component State window in the first place, because
server throws AccessDeniedException.
Although Controller Services and Reporting Tasks don't use
supportsModification, they check dataContext.accessPolicy whether to show
controlling UI inputs including show status button:
{code:title=nf-controller-services.js and nf-settings.js}
if (dataContext.accessPolicy.canRead && dataContext.accessPolicy.canWrite) {
{code}
Overall, I think current implementation is enough to block unauthorized user
clears state. Please let me know your thoughts.
> Clear state link on Component State window is hidden
>
>
> Key: NIFI-2093
> URL: https://issues.apache.org/jira/browse/NIFI-2093
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
> Fix For: 1.0.0
>
> Attachments: ComponentState-ConsumeKafka.png
>
>
> It seems that ComponentStateEntity should have accessPolicy so that
> CanvasUtis.supportsModification() can handle whether the link is active or
> not.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2093) Clear state link on Component State window is hidden
[
https://issues.apache.org/jira/browse/NIFI-2093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345944#comment-15345944
]
Koji Kawamura commented on NIFI-2093:
-
Access policy checks for Controller service and Reporting task have following
authorization check code:
{code:title=ControllerServiceResource.java}
public Response getState() {
// authorize access
serviceFacade.authorizeAccess(lookup -> {
final Authorizable controllerService =
lookup.getControllerService(id);
// Shouldn't it be READ?
controllerService.authorize(authorizer, RequestAction.WRITE);
});
}
public Response clearState() {
final boolean validationPhase = isValidationPhase(httpServletRequest);
if (validationPhase || !isTwoPhaseRequest(httpServletRequest)) {
// authorize access
serviceFacade.authorizeAccess(lookup -> {
final Authorizable controllerService =
lookup.getControllerService(id);
controllerService.authorize(authorizer, RequestAction.WRITE);
});
}
}
{code}
[~mcgilman]
Do we need something in addition to that? BTW, getState authorizes WRITE
action. Shouldn't it be READ?
> Clear state link on Component State window is hidden
>
>
> Key: NIFI-2093
> URL: https://issues.apache.org/jira/browse/NIFI-2093
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
> Fix For: 1.0.0
>
> Attachments: ComponentState-ConsumeKafka.png
>
>
> It seems that ComponentStateEntity should have accessPolicy so that
> CanvasUtis.supportsModification() can handle whether the link is active or
> not.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2093) Clear state link on Component State window is hidden
[ https://issues.apache.org/jira/browse/NIFI-2093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345696#comment-15345696 ] Koji Kawamura commented on NIFI-2093: - Thank you Matt, I've set z-index of clear-link-container to avoid dialog-buttons hides the link. The change is bundled with NIFI-2078. https://github.com/apache/nifi/pull/563 > Clear state link on Component State window is hidden > > > Key: NIFI-2093 > URL: https://issues.apache.org/jira/browse/NIFI-2093 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Koji Kawamura > Fix For: 1.0.0 > > Attachments: ComponentState-ConsumeKafka.png > > > It seems that ComponentStateEntity should have accessPolicy so that > CanvasUtis.supportsModification() can handle whether the link is active or > not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2093) Clear state link on Component State window is hidden
[ https://issues.apache.org/jira/browse/NIFI-2093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-2093: Assignee: Koji Kawamura Fix Version/s: 1.0.0 > Clear state link on Component State window is hidden > > > Key: NIFI-2093 > URL: https://issues.apache.org/jira/browse/NIFI-2093 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Koji Kawamura > Fix For: 1.0.0 > > Attachments: ComponentState-ConsumeKafka.png > > > It seems that ComponentStateEntity should have accessPolicy so that > CanvasUtis.supportsModification() can handle whether the link is active or > not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2093) Clear state link on Component State window is hidden
[ https://issues.apache.org/jira/browse/NIFI-2093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-2093: Summary: Clear state link on Component State window is hidden (was: Clear state link on Component State window is disabled) > Clear state link on Component State window is hidden > > > Key: NIFI-2093 > URL: https://issues.apache.org/jira/browse/NIFI-2093 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Koji Kawamura > Attachments: ComponentState-ConsumeKafka.png > > > It seems that ComponentStateEntity should have accessPolicy so that > CanvasUtis.supportsModification() can handle whether the link is active or > not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2094) Component State Description is not displayed correctly
[
https://issues.apache.org/jira/browse/NIFI-2094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Koji Kawamura updated NIFI-2094:
Attachment: ComponentState-GetHTTP.png
ComponentState-ConsumeKafka.png
ConsumeKafka has a long description, but using ellipsis limits the amount of
information can be displayed. There should be "show more" link or simply
display the description as it is without using ellipsis.
When a description is short like GetHTTP, the whole sentence is replaced by
"...".
> Component State Description is not displayed correctly
> --
>
> Key: NIFI-2094
> URL: https://issues.apache.org/jira/browse/NIFI-2094
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Koji Kawamura
> Fix For: 1.0.0
>
> Attachments: ComponentState-ConsumeKafka.png,
> ComponentState-GetHTTP.png
>
>
> nf-component-state.js uses ellipsis() function to trim long sentence with
> "...". But it replaces whole sentence if the description is not long enough.
> {code}
> $('#component-state-description').text(componentState.stateDescription).ellipsis();
> {code}
> GetHTTP's description is only displayed as "..." as attached screenshot.
> I think the use of ellipsis() can be removed so that the original description
> is shown as it is.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-2094) Component State Description is not displayed correctly
Koji Kawamura created NIFI-2094:
---
Summary: Component State Description is not displayed correctly
Key: NIFI-2094
URL: https://issues.apache.org/jira/browse/NIFI-2094
Project: Apache NiFi
Issue Type: Bug
Components: Core UI
Affects Versions: 1.0.0
Reporter: Koji Kawamura
Fix For: 1.0.0
nf-component-state.js uses ellipsis() function to trim long sentence with
"...". But it replaces whole sentence if the description is not long enough.
{code}
$('#component-state-description').text(componentState.stateDescription).ellipsis();
{code}
GetHTTP's description is only displayed as "..." as attached screenshot.
I think the use of ellipsis() can be removed so that the original description
is shown as it is.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-2093) Clear state link on Component State window is disabled
[ https://issues.apache.org/jira/browse/NIFI-2093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-2093: Attachment: ComponentState-ConsumeKafka.png > Clear state link on Component State window is disabled > -- > > Key: NIFI-2093 > URL: https://issues.apache.org/jira/browse/NIFI-2093 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Koji Kawamura > Attachments: ComponentState-ConsumeKafka.png > > > It seems that ComponentStateEntity should have accessPolicy so that > CanvasUtis.supportsModification() can handle whether the link is active or > not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2093) Clear state link on Component State window is disabled
Koji Kawamura created NIFI-2093: --- Summary: Clear state link on Component State window is disabled Key: NIFI-2093 URL: https://issues.apache.org/jira/browse/NIFI-2093 Project: Apache NiFi Issue Type: Bug Components: Core UI Affects Versions: 1.0.0 Reporter: Koji Kawamura It seems that ComponentStateEntity should have accessPolicy so that CanvasUtis.supportsModification() can handle whether the link is active or not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2078) State management for processors whose states are managed externally
Koji Kawamura created NIFI-2078: --- Summary: State management for processors whose states are managed externally Key: NIFI-2078 URL: https://issues.apache.org/jira/browse/NIFI-2078 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Koji Kawamura Assignee: Koji Kawamura Fix For: 1.0.0 Inherently by the nature of a given processor it may involve state managed by itself (using nifi state management), or can be managed by some external service it interacts with (kafka's offset), and theoretically some might have both going on. With the new state management, we're giving users a way to reset state managed by nifi for a given processor. But it doesnt apply to those processors who have external state. we should consider offering a way to reset state that allows a processor to call out to whatever external store it impacts -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2028) Site-to-Site Transit URI is inconsistent
[ https://issues.apache.org/jira/browse/NIFI-2028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-2028: Description: Site-to-Site client and server create provenance event at both end, and those have Transit URI in it to record how flow files are transferred. However, the URI formats are inconsistent among RAW vs HTTP. Test result as follows: - 3080: Web API port - 3081: Site-to-Site RAW Socket port *Before Fix* PUSH - RAW |Client - SEND: | nifi://localhost:3081/flow-file-uuid | |Server - RECEIVE: | nifi://localhost:3081/flow-file-uuid| PULL - RAW |Client - RECEIVE:| nifi://localhost:3081flow-file-uuid| |Server - SEND: | nifi://localhost:3081/flow-file-uuid| PUSH - HTTP |Client - SEND: | http://127.0.0.1:3080/nifi-api/flow-file-uuid| |Server - RECEIVE:| nifi://127.0.0.1:57390| PULL - HTTP |Client - RECEIVE:| http://127.0.0.1:3080/flow-file-uuid| |Server - SEND: | nifi://127.0.0.1:57673| *Issues* - PULL - RAW, Client - RECEIVE: lacks '/' in between port and flow-file uuid - RAW uses server's host and port on both end (by transit url prefix), HTTP should follow this rule - HTTP transit uri looks like REST endpoint but it is not a real endpoint. It should be an actual endpoint URI - RAW uses hostname, while HTTP uses IP address *After Fix* PUSH - RAW |Client - SEND: | nifi://localhost:3081/flow-file-uuid | |Server - RECEIVE: | nifi://localhost:3081/flow-file-uuid| PULL - RAW |Client - RECEIVE: | nifi://localhost:3081/flow-file-uuid| |Server - SEND: | nifi://localhost:3081/flow-file-uuid| PUSH - HTTP |Client - SEND: | http://localhost:3080/nifi-api/site-to-site/input-ports/$port-id/transactions/$tx-id/flow-files| |Server - RECEIVE:| http://localhost:3080/nifi-api/site-to-site/input-ports/$port-id/transactions/$tx-id/flow-files| PULL - HTTP |Client - RECEIVE: | http://localhost:3080/nifi-api/site-to-site/output-ports/$port-id/transactions/$tx-id/flow-files| |Server - SEND: | http://localhost:3080/nifi-api/site-to-site/output-ports/$port-id/transactions/$tx-id/flow-files| was: Site-to-Site client and server create provenance event at both end, and those have Transit URI in it to record how flow files are transferred. However, the URI formats are inconsistent among RAW vs HTTP. Test result as follows: - 3080: Web API port - 3081: Site-to-Site RAW Socket port *Before Fix* PUSH - RAW |Client - SEND: | nifi://localhost:3081/flow-file-uuid | |Server - RECEIVE: | nifi://localhost:3081/flow-file-uuid| PULL - RAW |Client - RECEIVE:| nifi://localhost:3081flow-file-uuid| |Server - SEND: | nifi://localhost:3081/flow-file-uuid| PUSH - HTTP |Client - SEND: | http://127.0.0.1:3080/nifi-api/flow-file-uuid| |Server - RECEIVE:| nifi://127.0.0.1:57390| PULL - HTTP |Client - RECEIVE:| http://127.0.0.1:3080/flow-file-uuid| |Server - SEND: | nifi://127.0.0.1:57673| *After Fix* PUSH - RAW |Client - SEND: | nifi://localhost:3081/flow-file-uuid | |Server - RECEIVE: | nifi://localhost:3081/flow-file-uuid| PULL - RAW |Client - RECEIVE: | nifi://localhost:3081/flow-file-uuid| |Server - SEND: | nifi://localhost:3081/flow-file-uuid| PUSH - HTTP |Client - SEND: | http://localhost:3080/nifi-api/site-to-site/input-ports/$port-id/transactions/$tx-id/flow-files| |Server - RECEIVE:| http://localhost:3080/nifi-api/site-to-site/input-ports/$port-id/transactions/$tx-id/flow-files| PULL - HTTP |Client - RECEIVE: | http://localhost:3080/nifi-api/site-to-site/output-ports/$port-id/transactions/$tx-id/flow-files| |Server - SEND: | http://localhost:3080/nifi-api/site-to-site/output-ports/$port-id/transactions/$tx-id/flow-files| > Site-to-Site Transit URI is inconsistent > > > Key: NIFI-2028 > URL: https://issues.apache.org/jira/browse/NIFI-2028 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Koji Kawamura > Fix For: 1.0.0 > > > Site-to-Site client and server create provenance event at both end, and those > have Transit URI in it to record how flow files are transferred. However, the > URI formats are inconsistent among RAW vs HTTP. > Test result as follows: > - 3080: Web API port > - 3081: Site-to-Site RAW Socket port > *Before Fix* > PUSH - RAW > |Client - SEND: | nifi://localhost:3081/flow-file-uuid | > |Server - RECEIVE: | nifi://localhost:3081/flow-file-uuid| > PULL - RAW > |Client - RECEIVE:| nifi://localhost:3081flow-file-uuid| > |Server - SEND: | nifi://localhost:3081/flow-file-uuid| > PUSH - HTTP > |Client - SEND: | http://127.0.0.1:3080/nifi-api/flow-file-uuid| > |Server - RECEIVE:| nifi://127.0.0.1:57390| > PULL - HTTP > |Client - RECEIVE:| http://127.0.0.1:3080/flow-file-uuid| > |Server - SEND: | nifi://127.0.0.1:57673| > *Issues* > - PULL - RAW, Client - REC
[jira] [Updated] (NIFI-2028) Site-to-Site Transit URI is inconsistent
[ https://issues.apache.org/jira/browse/NIFI-2028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-2028: Description: Site-to-Site client and server create provenance event at both end, and those have Transit URI in it to record how flow files are transferred. However, the URI formats are inconsistent among RAW vs HTTP. Test result as follows: - 3080: Web API port - 3081: Site-to-Site RAW Socket port *Before Fix* PUSH - RAW |Client - SEND: | nifi://localhost:3081/flow-file-uuid | |Server - RECEIVE: | nifi://localhost:3081/flow-file-uuid| PULL - RAW |Client - RECEIVE:| nifi://localhost:3081flow-file-uuid| |Server - SEND: | nifi://localhost:3081/flow-file-uuid| PUSH - HTTP |Client - SEND: | http://127.0.0.1:3080/nifi-api/flow-file-uuid| |Server - RECEIVE:| nifi://127.0.0.1:57390| PULL - HTTP |Client - RECEIVE:| http://127.0.0.1:3080/flow-file-uuid| |Server - SEND: | nifi://127.0.0.1:57673| *After Fix* PUSH - RAW |Client - SEND: | nifi://localhost:3081/flow-file-uuid | |Server - RECEIVE: | nifi://localhost:3081/flow-file-uuid| PULL - RAW |Client - RECEIVE: | nifi://localhost:3081/flow-file-uuid| |Server - SEND: | nifi://localhost:3081/flow-file-uuid| PUSH - HTTP |Client - SEND: | http://localhost:3080/nifi-api/site-to-site/input-ports/$port-id/transactions/$tx-id/flow-files| |Server - RECEIVE:| http://localhost:3080/nifi-api/site-to-site/input-ports/$port-id/transactions/$tx-id/flow-files| PULL - HTTP |Client - RECEIVE: | http://localhost:3080/nifi-api/site-to-site/output-ports/$port-id/transactions/$tx-id/flow-files| |Server - SEND: | http://localhost:3080/nifi-api/site-to-site/output-ports/$port-id/transactions/$tx-id/flow-files| was: Site-to-Site client and server create provenance event at both end, and those have Transit URI in it to record how flow files are transferred. However, the URI formats are inconsistent among RAW vs HTTP. Test result as follows: - 3080: Web API port - 3081: Site-to-Site RAW Socket port *Before Fix* PUSH - RAW |Client - SEND: | nifi://localhost:3081/flow-file-uuid | |Server - RECEIVE: | nifi://localhost:3081/flow-file-uuid| PULL - RAW |Client - RECEIVE:| nifi://localhost:3081flow-file-uuid| |Server - SEND: | nifi://localhost:3081/flow-file-uuid| PUSH - HTTP |Client - SEND: | http://127.0.0.1:3080/nifi-api/flow-file-uuid| |Server - RECEIVE:| nifi://127.0.0.1:57390| PULL - HTTP |Client - RECEIVE:| http://127.0.0.1:3080/flow-file-uuid| |Server - SEND: | nifi://127.0.0.1:57673| > Site-to-Site Transit URI is inconsistent > > > Key: NIFI-2028 > URL: https://issues.apache.org/jira/browse/NIFI-2028 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Koji Kawamura > Fix For: 1.0.0 > > > Site-to-Site client and server create provenance event at both end, and those > have Transit URI in it to record how flow files are transferred. However, the > URI formats are inconsistent among RAW vs HTTP. > Test result as follows: > - 3080: Web API port > - 3081: Site-to-Site RAW Socket port > *Before Fix* > PUSH - RAW > |Client - SEND: | nifi://localhost:3081/flow-file-uuid | > |Server - RECEIVE: | nifi://localhost:3081/flow-file-uuid| > PULL - RAW > |Client - RECEIVE:| nifi://localhost:3081flow-file-uuid| > |Server - SEND: | nifi://localhost:3081/flow-file-uuid| > PUSH - HTTP > |Client - SEND: | http://127.0.0.1:3080/nifi-api/flow-file-uuid| > |Server - RECEIVE:| nifi://127.0.0.1:57390| > PULL - HTTP > |Client - RECEIVE:| http://127.0.0.1:3080/flow-file-uuid| > |Server - SEND: | nifi://127.0.0.1:57673| > *After Fix* > PUSH - RAW > |Client - SEND: | nifi://localhost:3081/flow-file-uuid | > |Server - RECEIVE: | nifi://localhost:3081/flow-file-uuid| > PULL - RAW > |Client - RECEIVE: | nifi://localhost:3081/flow-file-uuid| > |Server - SEND: | nifi://localhost:3081/flow-file-uuid| > PUSH - HTTP > |Client - SEND: | > http://localhost:3080/nifi-api/site-to-site/input-ports/$port-id/transactions/$tx-id/flow-files| > |Server - RECEIVE:| > http://localhost:3080/nifi-api/site-to-site/input-ports/$port-id/transactions/$tx-id/flow-files| > PULL - HTTP > |Client - RECEIVE: | > http://localhost:3080/nifi-api/site-to-site/output-ports/$port-id/transactions/$tx-id/flow-files| > |Server - SEND: | > http://localhost:3080/nifi-api/site-to-site/output-ports/$port-id/transactions/$tx-id/flow-files| -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2028) Site-to-Site Transit URI is inconsistent
Koji Kawamura created NIFI-2028: --- Summary: Site-to-Site Transit URI is inconsistent Key: NIFI-2028 URL: https://issues.apache.org/jira/browse/NIFI-2028 Project: Apache NiFi Issue Type: Bug Components: Core UI Affects Versions: 1.0.0 Reporter: Koji Kawamura Assignee: Koji Kawamura Fix For: 1.0.0 Site-to-Site client and server create provenance event at both end, and those have Transit URI in it to record how flow files are transferred. However, the URI formats are inconsistent among RAW vs HTTP. Test result as follows: - 3080: Web API port - 3081: Site-to-Site RAW Socket port *Before Fix* PUSH - RAW |Client - SEND: | nifi://localhost:3081/flow-file-uuid | |Server - RECEIVE: | nifi://localhost:3081/flow-file-uuid| PULL - RAW |Client - RECEIVE:| nifi://localhost:3081flow-file-uuid| |Server - SEND: | nifi://localhost:3081/flow-file-uuid| PUSH - HTTP |Client - SEND: | http://127.0.0.1:3080/nifi-api/flow-file-uuid| |Server - RECEIVE:| nifi://127.0.0.1:57390| PULL - HTTP |Client - RECEIVE:| http://127.0.0.1:3080/flow-file-uuid| |Server - SEND: | nifi://127.0.0.1:57673| -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1982) Compressed check box in Configure Remote Port UI is not used
[ https://issues.apache.org/jira/browse/NIFI-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-1982: Attachment: configure-remote-port.png Compressed checkbox. > Compressed check box in Configure Remote Port UI is not used > > > Key: NIFI-1982 > URL: https://issues.apache.org/jira/browse/NIFI-1982 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework, Core UI >Affects Versions: 0.6.1 >Reporter: Koji Kawamura >Assignee: Koji Kawamura > Fix For: 0.7.0 > > Attachments: configure-remote-port.png > > > RIght click Remote Process Group -> Remote Ports -> (Edit icon) -> Configure > Remote Port setting window > It has "Compressed" checkbox. However if user checked the UI component, > RemoteGroupPort doesn't use it to enable compression of Site-to-Site client. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1982) Compressed check box in Configure Remote Port UI is not used
Koji Kawamura created NIFI-1982: --- Summary: Compressed check box in Configure Remote Port UI is not used Key: NIFI-1982 URL: https://issues.apache.org/jira/browse/NIFI-1982 Project: Apache NiFi Issue Type: Bug Components: Core Framework, Core UI Affects Versions: 0.6.1 Reporter: Koji Kawamura Assignee: Koji Kawamura Fix For: 0.7.0 RIght click Remote Process Group -> Remote Ports -> (Edit icon) -> Configure Remote Port setting window It has "Compressed" checkbox. However if user checked the UI component, RemoteGroupPort doesn't use it to enable compression of Site-to-Site client. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1978) Remote Process Group Yield duration is reset to default after restart
[ https://issues.apache.org/jira/browse/NIFI-1978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-1978: Description: Configure Remote Process Group has "Yield duration" input which defaults to 10 sec. However, the value is not read from saved flow.xml, then back to 10 sec after restart of NiFi. FlowFromDomFactory.java getRemoteProcessGroup should restore persisted configuration. was: Configure Remote Process Group has "Yield duration" input which defaults to 10 sec. However, the value is not read from saved flow.xml, then back to 10 sec after restart of NiFi. FlowFromDomFactory.java getRemoteProcessGroup should set > Remote Process Group Yield duration is reset to default after restart > - > > Key: NIFI-1978 > URL: https://issues.apache.org/jira/browse/NIFI-1978 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework, Core UI >Affects Versions: 0.6.1 >Reporter: Koji Kawamura >Assignee: Koji Kawamura > Fix For: 0.7.0 > > > Configure Remote Process Group has "Yield duration" input which defaults to > 10 sec. However, the value is not read from saved flow.xml, then back to 10 > sec after restart of NiFi. > FlowFromDomFactory.java getRemoteProcessGroup should restore persisted > configuration. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1978) Remote Process Group Yield duration is reset to default after restart
Koji Kawamura created NIFI-1978: --- Summary: Remote Process Group Yield duration is reset to default after restart Key: NIFI-1978 URL: https://issues.apache.org/jira/browse/NIFI-1978 Project: Apache NiFi Issue Type: Bug Components: Core Framework, Core UI Affects Versions: 0.6.1 Reporter: Koji Kawamura Assignee: Koji Kawamura Fix For: 0.7.0 Configure Remote Process Group has "Yield duration" input which defaults to 10 sec. However, the value is not read from saved flow.xml, then back to 10 sec after restart of NiFi. FlowFromDomFactory.java getRemoteProcessGroup should set -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1890) Add JSON log format support for ControllerStatusReportingTask
Koji Kawamura created NIFI-1890:
---
Summary: Add JSON log format support for
ControllerStatusReportingTask
Key: NIFI-1890
URL: https://issues.apache.org/jira/browse/NIFI-1890
Project: Apache NiFi
Issue Type: Improvement
Components: Extensions
Reporter: Koji Kawamura
ControllerStatusReportingTask can report the 5-minute status for Processors and
Connections to log files, those are informative from monitoring and alerting
system perspective. However, current log format spreads across multiple lines,
and separates columns with '|', it can be more user-friendly to integrate with
3rd party monitoring tools.
For example, if a user would like to setup monitoring & alerting mechanism on
"FlowFiles Queued" column in this log format, they has to first grep the target
log message lines, then using complex regular expression to extract target
values.
If ControllerStatusReportingTask has a option to write these statistics in JSON
format to a single log message, it would be much easier to parse log message,
extract required fields and pass those into other 3rd party monitoring tools,
such as [Logstash|https://www.elastic.co/products/logstash] and
[Kibana|https://www.elastic.co/products/kibana].
Below is the current log format for reference:
{code}
2016-05-17 09:08:23,254 INFO [Timer-Driven Process Thread-6]
o.a.n.c.C.Processors Processor Statuses:
--
| Processor Name | Processor ID |
Processor Type | Run Status | Flow
Files In | Flow Files Out |
Bytes Read | Bytes Written | Tasks |
Proc Time |
--
| GenerateFlowFile | cff43e4a-2b0c-4221-bdab-fbeb1400b94b |
GenerateFlowFile |Running | 0 / 0 bytes (+0/+0
bytes) | 100 / 100 bytes (+100/+100 bytes) | 0 bytes (+0
bytes) | 100 bytes (+100 bytes) | 100 (+100) | 00:00:00.131
(+00:00:00.131) |
| LogAttribute | bbf54a49-a948-4a17-a5fd-de95a2898bd5 |
LogAttribute |Running | 0 / 0 bytes (+0/+0
bytes) | 0 / 0 bytes (+0/+0 bytes) | 0 bytes (+0
bytes) | 0 bytes (+0 bytes) | 0 (+0) | 00:00:00.000
(+00:00:00.000) |
| UpdateAttribute| e960a719-bcf9-4bdf-9122-6438548becc6 |
UpdateAttribute |Stopped | 0 / 0 bytes (+0/+0
bytes) | 0 / 0 bytes (+0/+0 bytes) | 0 bytes (+0
bytes) | 0 bytes (+0 bytes) | 0 (+0) | 00:00:00.000
(+00:00:00.000) |
| LogAttribute | 4677d849-3042-48c7-8872-96b75eb81664 |
LogAttribute |Running | 0 / 0 bytes (+0/+0
bytes) | 0 / 0 bytes (+0/+0 bytes) | 0 bytes (+0
bytes) | 0 bytes (+0 bytes) | 0 (+0) | 00:00:00.000
(+00:00:00.000) |
--
2016-05-17 09:08:23,255 INFO [Timer-Driven Process Thread-6]
o.a.n.c.C.Connections Connection Statuses:
| Connection ID| Source |
Connection Name | Destination|
Flow Files In | Flow Files
Out |FlowFiles Queued |
--
[jira] [Assigned] (NIFI-1865) PostHTTP and ListenHTTP never shutdown LeakyBucketStreamThrottler thread
[
https://issues.apache.org/jira/browse/NIFI-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Koji Kawamura reassigned NIFI-1865:
---
Assignee: Koji Kawamura
> PostHTTP and ListenHTTP never shutdown LeakyBucketStreamThrottler thread
>
>
> Key: NIFI-1865
> URL: https://issues.apache.org/jira/browse/NIFI-1865
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 0.4.1
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>Priority: Minor
>
> LeakyBucketStreamThrottler implements close and inside it it shutdowns
> executorService.
> However, ListenHTTP and PostHTTP doesn't call close, and since those
> processors create new LeakyBucketStreamThrottler instance when it is
> scheduled, it creates new thread without closing the old ones.
> It can be reproduced by repeating stop and start those processors again and
> again. You can confirm the number of threads are increasing by following
> command:
> {code}
> $ watch "ps -M |grep {nifi pid} | wc -l"
> {code}
> or using thread dump
> {code}
> $ tail -f logs/nifi-bootstrap.log |grep LeakyBucketStreamThrottler.java
> from another terminal
> $ kill -3 {nifi pid}
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-1865) PostHTTP and ListenHTTP never shutdown LeakyBucketStreamThrottler thread
Koji Kawamura created NIFI-1865:
---
Summary: PostHTTP and ListenHTTP never shutdown
LeakyBucketStreamThrottler thread
Key: NIFI-1865
URL: https://issues.apache.org/jira/browse/NIFI-1865
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 0.4.1
Reporter: Koji Kawamura
Priority: Minor
LeakyBucketStreamThrottler implements close and inside it it shutdowns
executorService.
However, ListenHTTP and PostHTTP doesn't call close, and since those processors
create new LeakyBucketStreamThrottler instance when it is scheduled, it creates
new thread without closing the old ones.
It can be reproduced by repeating stop and start those processors again and
again. You can confirm the number of threads are increasing by following
command:
{code}
$ watch "ps -M |grep {nifi pid} | wc -l"
{code}
or using thread dump
{code}
$ tail -f logs/nifi-bootstrap.log |grep LeakyBucketStreamThrottler.java
from another terminal
$ kill -3 {nifi pid}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1860) Backward compatibility for Site-to-Site client
[
https://issues.apache.org/jira/browse/NIFI-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15276017#comment-15276017
]
Koji Kawamura commented on NIFI-1860:
-
[~mcgilman] I've sent a PR to support an old version connects to a new version
scenario. If the proposed approach is acceptable, I will update the PR with
support for a new version connects to an old version scenario. Please let me
know your thoughts.
> Backward compatibility for Site-to-Site client
> --
>
> Key: NIFI-1860
> URL: https://issues.apache.org/jira/browse/NIFI-1860
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Core Framework
>Reporter: Koji Kawamura
> Fix For: 1.0.0
>
>
> NIFI-1554 has moved /nifi-api/controller to /nifi-api/site-to-site/ .
> However, Site-to-Site client is not updated yet, and it still sends a request
> to /nifi-api/controller to obtain information for Site-to-Site.
> {code:title=EndpointConnectionPool.java}
> private ControllerDTO refreshRemoteInfo() throws IOException {
> final boolean webInterfaceSecure =
> clusterUrl.toString().startsWith("https");
> final NiFiRestApiUtil utils = new NiFiRestApiUtil(webInterfaceSecure
> ? sslContext : null);
> final ControllerDTO controller = utils.getController(apiUri +
> "/controller", commsTimeout);
> {code}
> We can simply change the URL to "/site-to-site", but if we do so, old version
> of NiFi won't be able to communicate with the new version of NiFi via
> Site-to-Site.
> Possible approaches to provide backward compatibility are:
> * A. keep the old endpoint and let it respond with 301 Moved Permanently
> * B. add a ContainerRequestFilter to modify requested url
> * C. Jetty Rewrite Handler
> http://www.eclipse.org/jetty/documentation/current/rewrite-handler.html
> Similarly when new version of NiFi tries to communicate with old version of
> NiFi, it has to send request to /controller, instead of /site-to-site. For
> this scenario, adding retry logic in new version of SIte-to-SIte client would
> be sufficient.
> # Send request to /site-to-site
> # If response code is 404 or 405, then retry sending request to /controller
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-1860) Backward compatibility for Site-to-Site client
Koji Kawamura created NIFI-1860:
---
Summary: Backward compatibility for Site-to-Site client
Key: NIFI-1860
URL: https://issues.apache.org/jira/browse/NIFI-1860
Project: Apache NiFi
Issue Type: Sub-task
Components: Core Framework
Reporter: Koji Kawamura
NIFI-1554 has moved /nifi-api/controller to /nifi-api/site-to-site/ .
However, Site-to-Site client is not updated yet, and it still sends a request
to /nifi-api/controller to obtain information for Site-to-Site.
{code:title=EndpointConnectionPool.java}
private ControllerDTO refreshRemoteInfo() throws IOException {
final boolean webInterfaceSecure =
clusterUrl.toString().startsWith("https");
final NiFiRestApiUtil utils = new NiFiRestApiUtil(webInterfaceSecure ?
sslContext : null);
final ControllerDTO controller = utils.getController(apiUri +
"/controller", commsTimeout);
{code}
We can simply change the URL to "/site-to-site", but if we do so, old version
of NiFi won't be able to communicate with the new version of NiFi via
Site-to-Site.
Possible approaches to provide backward compatibility are:
* A. keep the old endpoint and let it respond with 301 Moved Permanently
* B. add a ContainerRequestFilter to modify requested url
* C. Jetty Rewrite Handler
http://www.eclipse.org/jetty/documentation/current/rewrite-handler.html
Similarly when new version of NiFi tries to communicate with old version of
NiFi, it has to send request to /controller, instead of /site-to-site. For this
scenario, adding retry logic in new version of SIte-to-SIte client would be
sufficient.
# Send request to /site-to-site
# If response code is 404 or 405, then retry sending request to /controller
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-1857) Support HTTP(S) as a transport mechanism for Site-to-Site
Koji Kawamura created NIFI-1857: --- Summary: Support HTTP(S) as a transport mechanism for Site-to-Site Key: NIFI-1857 URL: https://issues.apache.org/jira/browse/NIFI-1857 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Koji Kawamura Assignee: Koji Kawamura We should add support for using HTTP(S) for site-to-site to be an alternative to the current socket based approach. This would support the same push based or pull based approach site-to-site offers now but it would use HTTP(S) for all interactions to include learning about ports, learning about NCM topology, and actually exchanging data. This mechanism should also support interaction via an HTTP proxy. This would also require some UI work to allow the user to specify which protocol for site-to-site to use such as 'raw' vs 'http'. We also need to document any limitations with regard to SSL support for this mode and we'd need to provide 'how-to' when using proxies like http_proxy or something else. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-1843) Default log level for logAttribute processor should be set to INFO in logback.xml
[ https://issues.apache.org/jira/browse/NIFI-1843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura reassigned NIFI-1843: --- Assignee: Koji Kawamura > Default log level for logAttribute processor should be set to INFO in > logback.xml > - > > Key: NIFI-1843 > URL: https://issues.apache.org/jira/browse/NIFI-1843 > Project: Apache NiFi > Issue Type: Bug > Components: Configuration >Affects Versions: 1.0.0 >Reporter: Matthew Clarke >Assignee: Koji Kawamura >Priority: Minor > Labels: beginner, newbie > > With the new default processor log level of "WARN" in the logback.xml, the > logAttribute processor will no longer perform as documented. The > logAttribute processor when added should still be able to write out FlowFile > Attributes to the app log by default. > This is an easy fix by simply adding by default a new logger to the > logback.xml file to set the log level for the logAttribute processor to INFO. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1759) Update to latest Commons Compress library
Koji Kawamura created NIFI-1759:
---
Summary: Update to latest Commons Compress library
Key: NIFI-1759
URL: https://issues.apache.org/jira/browse/NIFI-1759
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework, Extensions
Reporter: Koji Kawamura
Priority: Minor
The latest Commons Compress ver 1.11 was released on Apr 6, 2016.
This issue is used for tracking the changes to update our usage of Commons
Compress throughout NiFi to use this new library and to ensure tests are still
good and perhaps add some more to verify a few of following cases:
{quote}
Version 1.11 brings performance improvements to the 7ZIP code and
read-only support for ZIP archives using BZIP2 compression. In addition
it contains numerous bugfixes and enhancements, see the full list below.
The Apache Commons Compress Library defines a Java API for working with
ar, cpio, tar, zip, 7z, arj, dump, gzip, pack200, bzip2, lzma, snappy,
Z, xz and deflate files.
{quote}
quote from: [Release Announce |
http://mail-archives.apache.org/mod_mbox/www-announce/201604.mbox/%3C87bn5m6ayl.fsf%40v45346.1blu.de%3E]
[Release Note |
https://commons.apache.org/proper/commons-compress/changes-report.html#a1.11]
As of Apr 2016, following NiFi implementations uses Commons Compress 1.9
directly:
* nifi-flowfile-packager
** org.apache.nifi.util.FlowFilePackagerV1
** org.apache.nifi.util.FlowFileUnpackagerV1
* nifi-framework-cluster
** org.apache.nifi.cluster.flow.impl.DataFlowDaoImpl
* nifi-hdfs-processors
** org.apache.nifi.processors.hadoop.TarUnpackerSequenceFileWriter
* nifi-standard-processors
** org.apache.nifi.processors.standard.CompressContent
** org.apache.nifi.processors.standard.MergeContent
** org.apache.nifi.processors.standard.UnpackContent
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (NIFI-619) update MonitorActivity processor to be cluster friendly
[ https://issues.apache.org/jira/browse/NIFI-619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15234852#comment-15234852 ] Koji Kawamura edited comment on NIFI-619 at 4/11/16 10:57 AM: -- It maybe ideal to NOT alert when there is less load than the configured cluster capacity, for example in a cluster as written in the description, even if one of two nodes doesn't do any processing at all. However, in an environment that DFM expects every node should receive incoming flow file regularly, each node should report based on its own activity/inactivity. I think it'd be best to add a property named 'Monitor across cluster' (boolean) to indicate whether this processor should check inactivity cluster-wide or individual node. When 'Monitor across cluster' is true, every node update its activity using the cluster level StateProvider. Then the processor running on Primary Node checks cluster-wide inactiveness by checking each node's last activity which is stored on the StateProvider. We should be careful about updating the StateProvider too frequently. It should update StateProvider only when its activeness changes. was (Author: ijokarumawak): It maybe ideal to NOT alert when there is less load than the configured cluster capacity, for example in a cluster as written in the description, even if one of two nodes doesn't do any processing at all. However, in an environment that DFM expects every node should receive incoming flow file regularly, each node should report based on its own activity/inactivity. I think it'd be best to add a property named 'Monitor across cluster' (boolean) to indicate whether this processor should check inactivity cluster-wide or individual node. Additionally, when 'Monitor across cluster' is true, then this processor should be scheduled with 'on Primary Node'. > update MonitorActivity processor to be cluster friendly > --- > > Key: NIFI-619 > URL: https://issues.apache.org/jira/browse/NIFI-619 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Brandon DeVries >Priority: Minor > > This processor should be able to be used to monitor activity across the > cluster. In its current state, alerting is based on activity of a single > node, not the entire cluster. > For example, in a 2 node cluster, if system A is getting data from a given > flow and system B is not, system B will alert for lack of activity even > though the flow is functioning "normally". > The ideal behavior would be fore an alert to be generated only if both > systems did not see data in the specified time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-619) update MonitorActivity processor to be cluster friendly
[ https://issues.apache.org/jira/browse/NIFI-619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15234852#comment-15234852 ] Koji Kawamura commented on NIFI-619: It maybe ideal to NOT alert when there is less load than the configured cluster capacity, for example in a cluster as written in the description, even if one of two nodes doesn't do any processing at all. However, in an environment that DFM expects every node should receive incoming flow file regularly, each node should report based on its own activity/inactivity. I think it'd be best to add a property named 'Monitor across cluster' (boolean) to indicate whether this processor should check inactivity cluster-wide or individual node. Additionally, when 'Monitor across cluster' is true, then this processor should be scheduled with 'on Primary Node'. > update MonitorActivity processor to be cluster friendly > --- > > Key: NIFI-619 > URL: https://issues.apache.org/jira/browse/NIFI-619 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Brandon DeVries >Priority: Minor > > This processor should be able to be used to monitor activity across the > cluster. In its current state, alerting is based on activity of a single > node, not the entire cluster. > For example, in a 2 node cluster, if system A is getting data from a given > flow and system B is not, system B will alert for lack of activity even > though the flow is functioning "normally". > The ideal behavior would be fore an alert to be generated only if both > systems did not see data in the specified time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1736) Move org.apache.nifi.processors.kafka.StreamScanner to nifi-utils/org/apache/nifi/stream/io/util
[ https://issues.apache.org/jira/browse/NIFI-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15232045#comment-15232045 ] Koji Kawamura commented on NIFI-1736: - Thanks Joe, updated the page. > Move org.apache.nifi.processors.kafka.StreamScanner to > nifi-utils/org/apache/nifi/stream/io/util > > > Key: NIFI-1736 > URL: https://issues.apache.org/jira/browse/NIFI-1736 > Project: Apache NiFi > Issue Type: Task >Reporter: Oleg Zhurakousky >Assignee: Koji Kawamura >Priority: Trivial > Fix For: 1.0.0, 0.7.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1736) Move org.apache.nifi.processors.kafka.StreamScanner to nifi-utils/org/apache/nifi/stream/io/util
[ https://issues.apache.org/jira/browse/NIFI-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231448#comment-15231448 ] Koji Kawamura commented on NIFI-1736: - Thanks for merging the change, and sharing notes on the process! [Contributor Guide|https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide] is a guide for me or other developers to contribute, and I think there's some room for improvement. 1. It's written in lower case in the guide. 2. I'll try to do in this way next time, and again we should update the guide to illustrate that. Would you update the guide? (because I don't have privilege to do that) > Move org.apache.nifi.processors.kafka.StreamScanner to > nifi-utils/org/apache/nifi/stream/io/util > > > Key: NIFI-1736 > URL: https://issues.apache.org/jira/browse/NIFI-1736 > Project: Apache NiFi > Issue Type: Task >Reporter: Oleg Zhurakousky >Assignee: Koji Kawamura >Priority: Trivial > Fix For: 1.0.0, 0.7.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1736) Move org.apache.nifi.processors.kafka.StreamScanner to nifi-utils/org/apache/nifi/stream/io/util
[ https://issues.apache.org/jira/browse/NIFI-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15229808#comment-15229808 ] Koji Kawamura commented on NIFI-1736: - [~ozhurakousky] I've sent a PR, please check it. https://github.com/apache/nifi/pull/333 > Move org.apache.nifi.processors.kafka.StreamScanner to > nifi-utils/org/apache/nifi/stream/io/util > > > Key: NIFI-1736 > URL: https://issues.apache.org/jira/browse/NIFI-1736 > Project: Apache NiFi > Issue Type: Task >Reporter: Oleg Zhurakousky >Assignee: Koji Kawamura >Priority: Trivial > Fix For: 0.7.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-568) Create eclipse/netbeans/IntelliJ Quickstart Info
[ https://issues.apache.org/jira/browse/NIFI-568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-568: --- Attachment: nifi-checkstyle.xml nifi-checkstyle.xml (attached on this ticket) is a Checkstyle configuration file created by extracting checkstyleRules from nifi/pom.xml. Here is the steps for IntelliJ users: * Download [Checkstyle-IDEA plugin|https://plugins.jetbrains.com/plugin/1065] * Install plugin by loading the downloaded zip from: Preference -> Plugins -> Install plugin from disk... * After restarting IntelliJ, configure Checkstyle plugin to use NiFi Checkstyle configuration file from: Preference -> Other Settings -> Checkstyle * Add nifi-checkstyle.xml in Configuration File and activate it. Then you can see Checkstyle violations on editor in real-time. Could anyone update [Contributor Guide|https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide] ? I think I can't update the page. > Create eclipse/netbeans/IntelliJ Quickstart Info > > > Key: NIFI-568 > URL: https://issues.apache.org/jira/browse/NIFI-568 > Project: Apache NiFi > Issue Type: Task >Reporter: Brandon DeVries > Attachments: Apache NiFi eclipse format.xml, nifi-checkstyle.xml > > > Create Formatting definitions for common IDEs that conform with the NIFI-271 > checkstyle rules. > We also need to make a wiki page for each these explaining to people how to > get up and running in their IDE. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-1736) Move org.apache.nifi.processors.kafka.StreamScanner to nifi-utils/org/apache/nifi/stream/io/util
[ https://issues.apache.org/jira/browse/NIFI-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura reassigned NIFI-1736: --- Assignee: Koji Kawamura > Move org.apache.nifi.processors.kafka.StreamScanner to > nifi-utils/org/apache/nifi/stream/io/util > > > Key: NIFI-1736 > URL: https://issues.apache.org/jira/browse/NIFI-1736 > Project: Apache NiFi > Issue Type: Task >Reporter: Oleg Zhurakousky >Assignee: Koji Kawamura >Priority: Trivial > Fix For: 0.7.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-992) Couchbase Server Processors
[ https://issues.apache.org/jira/browse/NIFI-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14940584#comment-14940584 ] Koji Kawamura commented on NIFI-992: [~bende] Yes, that sound great. I don't know what is the preferable way to close this ticket, so I leave its status unchanged, please close this one, and I'll open new tickets for View and N1QL. Thanks again for the review process and merging my code into the project! > Couchbase Server Processors > --- > > Key: NIFI-992 > URL: https://issues.apache.org/jira/browse/NIFI-992 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Koji Kawamura > Labels: processor > Fix For: 0.4.0 > > > Processors providing data access interface with a Couchbase Server cluster. > I've started writing a set of processors for interacting with Couchbase > Server. There are several ways to integrate with Couchbase such as: > 1. Key/Value CRUD operations > 2. View (Map/Reduce) queries > 3. N1QL queries > For the first step, I'm implementing the Key/Value CRUD operations. I will > send a pull request once the code and test get clean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-992) Couchbase Server Processors
[ https://issues.apache.org/jira/browse/NIFI-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939382#comment-14939382 ] Koji Kawamura commented on NIFI-992: [~bende] I pushed the new commit to the PR for 2nd review fix. Thanks! > Couchbase Server Processors > --- > > Key: NIFI-992 > URL: https://issues.apache.org/jira/browse/NIFI-992 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Koji Kawamura > Labels: processor > > Processors providing data access interface with a Couchbase Server cluster. > I've started writing a set of processors for interacting with Couchbase > Server. There are several ways to integrate with Couchbase such as: > 1. Key/Value CRUD operations > 2. View (Map/Reduce) queries > 3. N1QL queries > For the first step, I'm implementing the Key/Value CRUD operations. I will > send a pull request once the code and test get clean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-992) Couchbase Server Processors
[ https://issues.apache.org/jira/browse/NIFI-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935403#comment-14935403 ] Koji Kawamura commented on NIFI-992: [~bende] Thanks for reviewing the code. I just pushed the new commit to the PR. Please check that again! > Couchbase Server Processors > --- > > Key: NIFI-992 > URL: https://issues.apache.org/jira/browse/NIFI-992 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Koji Kawamura > Labels: processor > > Processors providing data access interface with a Couchbase Server cluster. > I've started writing a set of processors for interacting with Couchbase > Server. There are several ways to integrate with Couchbase such as: > 1. Key/Value CRUD operations > 2. View (Map/Reduce) queries > 3. N1QL queries > For the first step, I'm implementing the Key/Value CRUD operations. I will > send a pull request once the code and test get clean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-992) Couchbase Server Processors
Koji Kawamura created NIFI-992: -- Summary: Couchbase Server Processors Key: NIFI-992 URL: https://issues.apache.org/jira/browse/NIFI-992 Project: Apache NiFi Issue Type: New Feature Components: Core Framework Reporter: Koji Kawamura Processors providing data access interface with a Couchbase Server cluster. I've started writing a set of processors for interacting with Couchbase Server. There are several ways to integrate with Couchbase such as: 1. Key/Value CRUD operations 2. View (Map/Reduce) queries 3. N1QL queries For the first step, I'm implementing the Key/Value CRUD operations. I will send a pull request once the code and test get clean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
