[jira] [Created] (MINIFICPP-458) Network management controller service for support ipv6
bqiu created MINIFICPP-458: -- Summary: Network management controller service for support ipv6 Key: MINIFICPP-458 URL: https://issues.apache.org/jira/browse/MINIFICPP-458 Project: NiFi MiNiFi C++ Issue Type: Improvement Reporter: bqiu Assignee: bqiu -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MINIFICPP-457) Network management controller service for interface binding for socket
bqiu created MINIFICPP-457: -- Summary: Network management controller service for interface binding for socket Key: MINIFICPP-457 URL: https://issues.apache.org/jira/browse/MINIFICPP-457 Project: NiFi MiNiFi C++ Issue Type: Improvement Reporter: bqiu Assignee: bqiu Network management controller service for interface binding for socket -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (MINIFICPP-404) HTTP Proxy Support for HTTP Site to Site
[ https://issues.apache.org/jira/browse/MINIFICPP-404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bqiu resolved MINIFICPP-404. Resolution: Fixed > HTTP Proxy Support for HTTP Site to Site > > > Key: MINIFICPP-404 > URL: https://issues.apache.org/jira/browse/MINIFICPP-404 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: bqiu >Assignee: bqiu >Priority: Minor > Fix For: 1.0.0 > > > HTTP Proxy Support for HTTP Site to Site > http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#configure-site-to-site-client-nifi-instance. > support for this in YAML config via > https://github.com/apache/nifi-minifi/blob/master/minifi-docs/src/main/markdown/System_Admin_Guide.md#remote-process-groups-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (MINIFICPP-405) RPG Binding to Local Network Interface
[ https://issues.apache.org/jira/browse/MINIFICPP-405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bqiu resolved MINIFICPP-405. Resolution: Fixed > RPG Binding to Local Network Interface > -- > > Key: MINIFICPP-405 > URL: https://issues.apache.org/jira/browse/MINIFICPP-405 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: bqiu >Assignee: bqiu >Priority: Minor > Fix For: 1.0.0 > > > Remote Process Group for site2site bind to a local interface name > http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#configure-site-to-site-client-nifi-instance. > YAML config via > https://github.com/apache/nifi-minifi/blob/master/minifi-docs/src/main/markdown/System_Admin_Guide.md#remote-process-groups-1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (MINIFICPP-405) RPG Binding to Local Network Interface
[ https://issues.apache.org/jira/browse/MINIFICPP-405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bqiu updated MINIFICPP-405: --- Description: Remote Process Group for site2site bind to a local interface name http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#configure-site-to-site-client-nifi-instance. YAML config via https://github.com/apache/nifi-minifi/blob/master/minifi-docs/src/main/markdown/System_Admin_Guide.md#remote-process-groups-1 was: Remote Process Group for site2site bind to a local interface name > RPG Binding to Local Network Interface > -- > > Key: MINIFICPP-405 > URL: https://issues.apache.org/jira/browse/MINIFICPP-405 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: bqiu >Assignee: bqiu >Priority: Minor > Fix For: 1.0.0 > > > Remote Process Group for site2site bind to a local interface name > http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#configure-site-to-site-client-nifi-instance. > YAML config via > https://github.com/apache/nifi-minifi/blob/master/minifi-docs/src/main/markdown/System_Admin_Guide.md#remote-process-groups-1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MINIFICPP-405) RPG Binding to Local Network Interface
bqiu created MINIFICPP-405: -- Summary: RPG Binding to Local Network Interface Key: MINIFICPP-405 URL: https://issues.apache.org/jira/browse/MINIFICPP-405 Project: NiFi MiNiFi C++ Issue Type: Improvement Affects Versions: 1.0.0 Reporter: bqiu Assignee: bqiu Fix For: 1.0.0 Remote Process Group for site2site bind to a local interface name -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MINIFICPP-404) HTTP Proxy Support for HTTP Site to Site
bqiu created MINIFICPP-404: -- Summary: HTTP Proxy Support for HTTP Site to Site Key: MINIFICPP-404 URL: https://issues.apache.org/jira/browse/MINIFICPP-404 Project: NiFi MiNiFi C++ Issue Type: Improvement Affects Versions: 1.0.0 Reporter: bqiu Assignee: bqiu Fix For: 1.0.0 HTTP Proxy Support for HTTP Site to Site http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#configure-site-to-site-client-nifi-instance. support for this in YAML config via https://github.com/apache/nifi-minifi/blob/master/minifi-docs/src/main/markdown/System_Admin_Guide.md#remote-process-groups-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MINIFICPP-403) Enable tagging of flowfiles with flow metadata information in C++
bqiu created MINIFICPP-403:
--
Summary: Enable tagging of flowfiles with flow metadata
information in C++
Key: MINIFICPP-403
URL: https://issues.apache.org/jira/browse/MINIFICPP-403
Project: NiFi MiNiFi C++
Issue Type: Improvement
Affects Versions: 1.0.0
Reporter: bqiu
Assignee: bqiu
Fix For: 1.0.0
Provide framework level support to tag flowfiles with metadata about the flow
that created them.
Design proposal
Right now, MiNiFi support core attributes like
// FlowFile Attribute
enum FlowAttribute {
// The flowfile's path indicates the relative directory to which a FlowFile
belongs and does not contain the filename
PATH = 0,
// The flowfile's absolute path indicates the absolute directory to which a
FlowFile belongs and does not contain the filename
ABSOLUTE_PATH,
// The filename of the FlowFile. The filename should not contain any directory
structure.
FILENAME,
// A unique UUID assigned to this FlowFile.
UUID,
// A numeric value indicating the FlowFile priority
priority,
// The MIME Type of this FlowFile
MIME_TYPE,
// Specifies the reason that a FlowFile is being discarded
DISCARD_REASON,
// Indicates an identifier other than the FlowFile's UUID that is known to
refer to this FlowFile.
ALTERNATE_IDENTIFIER,
MAX_FLOW_ATTRIBUTES
};
So one approach is in the flow YAML file, specific the list of core flow
attributes along with the processors that inject/import/create the flow files.
When flow was created/imported/injected by this processor, we can apply these
core attributes to the new flow.
Also user can define their own core attributes template and EL for populate
value for these core attributes, for example protocol, TTL, record route
(expected route), key, version, etc.
In current implementation, FILENAME, PATH and UUID are required attributes when
flow was created, others are optional
// Populate the default attributes
addKeyedAttribute(FILENAME,
std::to_string(getTimeNano()));
addKeyedAttribute(PATH, DEFAULT_FLOWFILE_PATH);
addKeyedAttribute(UUID,
getUUIDStr())
So if user specify the optional meta flow info section for the processor with
the key/value pairs as above, MiNiFI will add these key attributes to the flow
when flow was created by this processor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (MINIFICPP-393) add security support for MQTT
bqiu created MINIFICPP-393: -- Summary: add security support for MQTT Key: MINIFICPP-393 URL: https://issues.apache.org/jira/browse/MINIFICPP-393 Project: NiFi MiNiFi C++ Issue Type: Improvement Affects Versions: 1.0.0 Reporter: bqiu Assignee: bqiu Fix For: 1.0.0 add security support for MQTT -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-20) MiNiFi-cpp should log to $MINIFI_HOME/logs/minifi-app.log
[ https://issues.apache.org/jira/browse/MINIFICPP-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bqiu reassigned MINIFICPP-20: - Assignee: bqiu > MiNiFi-cpp should log to $MINIFI_HOME/logs/minifi-app.log > - > > Key: MINIFICPP-20 > URL: https://issues.apache.org/jira/browse/MINIFICPP-20 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Randy Gelhausen >Assignee: bqiu > > MiNiFi-java logs to $MINIFI_HOME/logs/minifi-app.log, which is very similar > to NiFi logging to $NIFI_HOME/logs/nifi-app.log > MiNiFi-cpp logs to $MINIFI_HOME/minifi-app.log.txt > Since one of the goals for MiNiFi implementations is to be as interchangeable > as possible, MiNiFi-cpp's default log configuration should be changed to > write to $MINIFI_HOME/logs/minifi-app.log -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (MINIFICPP-337) Make default log directory 'logs'
[ https://issues.apache.org/jira/browse/MINIFICPP-337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bqiu reassigned MINIFICPP-337: -- Assignee: bqiu (was: marco polo) > Make default log directory 'logs' > - > > Key: MINIFICPP-337 > URL: https://issues.apache.org/jira/browse/MINIFICPP-337 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: marco polo >Assignee: bqiu > Fix For: 0.4.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (MINIFICPP-342) MQTT framework
bqiu created MINIFICPP-342: -- Summary: MQTT framework Key: MINIFICPP-342 URL: https://issues.apache.org/jira/browse/MINIFICPP-342 Project: NiFi MiNiFi C++ Issue Type: New Feature Affects Versions: 0.3.0 Reporter: bqiu Assignee: bqiu Fix For: 0.3.0 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-336) With default GetFile settings dot files are not getting ignored on linux systems as they should
[ https://issues.apache.org/jira/browse/MINIFICPP-336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16272942#comment-16272942 ] bqiu commented on MINIFICPP-336: yes, the code section pointed by Marc has the problem. > With default GetFile settings dot files are not getting ignored on linux > systems as they should > --- > > Key: MINIFICPP-336 > URL: https://issues.apache.org/jira/browse/MINIFICPP-336 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.3.0, 0.1.0, 0.2.0 >Reporter: Joseph Witt >Assignee: marco polo > Fix For: 0.4.0 > > > With this config > Processors: > - name: GetFile > class: org.apache.nifi.processors.standard.GetFile > max concurrent tasks: 1 > scheduling strategy: TIMER_DRIVEN > scheduling period: 0 sec > penalization period: 30 sec > yield period: 1 sec > run duration nanos: 0 > auto-terminated relationships list: [] > Properties: > Batch Size: '10' > File Filter: '[^\.].*' > Ignore Hidden Files: 'true' > Input Directory: test/input > Keep Source File: 'false' > Maximum File Age: > Maximum File Size: > Minimum File Age: 0 sec > Minimum File Size: 0 B > Path Filter: > Polling Interval: 0 sec > Recurse Subdirectories: 'true' > The minifi flow picks up any files starting with '.' character right away. I > believe this is causing duplication to occur when NiFi writes to that > directory being watched, for example, because it writes the files a > hidden/dot notation then renamed it when done. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-329) Kafka Refactor
[ https://issues.apache.org/jira/browse/MINIFICPP-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16268928#comment-16268928 ] bqiu commented on MINIFICPP-329: Dup to MINIFICPP-327 > Kafka Refactor > -- > > Key: MINIFICPP-329 > URL: https://issues.apache.org/jira/browse/MINIFICPP-329 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.3.0 >Reporter: bqiu >Assignee: bqiu >Priority: Minor > Fix For: 0.3.0 > > > refactor PutKafka to PublishKafka > add kafka.key as key attribute if flow has the same in flow attribute -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (MINIFICPP-329) Kafka Refactor
bqiu created MINIFICPP-329: -- Summary: Kafka Refactor Key: MINIFICPP-329 URL: https://issues.apache.org/jira/browse/MINIFICPP-329 Project: NiFi MiNiFi C++ Issue Type: Bug Affects Versions: 0.3.0 Reporter: bqiu Assignee: bqiu Priority: Minor Fix For: 0.3.0 refactor PutKafka to PublishKafka add kafka.key as key attribute if flow has the same in flow attribute -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-274) add kafka producer
[ https://issues.apache.org/jira/browse/MINIFICPP-274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16225657#comment-16225657 ] bqiu commented on MINIFICPP-274: Will use UUID for the key for the initial development. > add kafka producer > -- > > Key: MINIFICPP-274 > URL: https://issues.apache.org/jira/browse/MINIFICPP-274 > Project: NiFi MiNiFi C++ > Issue Type: New Feature >Affects Versions: 0.3.0 >Reporter: bqiu >Assignee: bqiu >Priority: Minor > > Add kafka producer for minifi C++ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-274) add kafka producer
[ https://issues.apache.org/jira/browse/MINIFICPP-274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16225461#comment-16225461 ] bqiu commented on MINIFICPP-274: kafka producer for minifi C++ use librdkafka and its security extension key of the record is the UUID for the flow record or a JSON string contain both attributes and UUID payload of the record is the flow binary content > add kafka producer > -- > > Key: MINIFICPP-274 > URL: https://issues.apache.org/jira/browse/MINIFICPP-274 > Project: NiFi MiNiFi C++ > Issue Type: New Feature >Affects Versions: 0.3.0 >Reporter: bqiu >Assignee: bqiu >Priority: Minor > > Add kafka producer for minifi C++ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (MINIFICPP-274) add kafka producer
bqiu created MINIFICPP-274: -- Summary: add kafka producer Key: MINIFICPP-274 URL: https://issues.apache.org/jira/browse/MINIFICPP-274 Project: NiFi MiNiFi C++ Issue Type: New Feature Affects Versions: 0.3.0 Reporter: bqiu Assignee: bqiu Priority: Minor Add kafka producer for minifi C++ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (MINIFICPP-264) Implement CompressContent Processor
[ https://issues.apache.org/jira/browse/MINIFICPP-264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bqiu updated MINIFICPP-264: --- Description: Implement CompressContent Processor (was: Implement MergeContent Processor) Summary: Implement CompressContent Processor (was: Implement MergeContent Processor) > Implement CompressContent Processor > --- > > Key: MINIFICPP-264 > URL: https://issues.apache.org/jira/browse/MINIFICPP-264 > Project: NiFi MiNiFi C++ > Issue Type: New Feature >Affects Versions: 0.3.0 >Reporter: bqiu >Assignee: bqiu >Priority: Minor > Fix For: 0.3.0 > > > Implement CompressContent Processor -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (MINIFICPP-264) Implement MergeContent Processor
bqiu created MINIFICPP-264: -- Summary: Implement MergeContent Processor Key: MINIFICPP-264 URL: https://issues.apache.org/jira/browse/MINIFICPP-264 Project: NiFi MiNiFi C++ Issue Type: New Feature Affects Versions: 0.3.0 Reporter: bqiu Assignee: bqiu Priority: Minor Fix For: 0.3.0 Implement MergeContent Processor -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (MINIFICPP-256) ExecuteProcess script uses wrong path
[ https://issues.apache.org/jira/browse/MINIFICPP-256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bqiu resolved MINIFICPP-256. Resolution: Fixed merge to apache main > ExecuteProcess script uses wrong path > - > > Key: MINIFICPP-256 > URL: https://issues.apache.org/jira/browse/MINIFICPP-256 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Fredrick Stakem >Assignee: marco polo > Attachments: 1.csv, 2.csv, basic_minifi_test.xml, config.yml, > process.py > > > I am running a test using nifi to create a flow and then import this flow > into minifi c++. The flow seems to work as expected on nifi. > The flow takes a file from input directory and places it into the processing > directory. In the background another ExectureProcess processor runs a simple > python script to look at the processing directory, get any files, parse the > files, and export to an output directory. > As stated before everything works as expected nifi, but in minifi c++ the > files end up in the root folder of minifi c++ and not the output directory. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-72) Add tar and compression support for MergeContent
[ https://issues.apache.org/jira/browse/MINIFICPP-72?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16203763#comment-16203763 ] bqiu commented on MINIFICPP-72: --- Aldrin, Currently NIFI support merge format tar and zip. I already committed merge content for minifi. This jira is to add merge fomat tar and zip for minifI so that merge content processor for minifi has feature parity with nifi. I will add compress content processor for minifi in different jira > Add tar and compression support for MergeContent > > > Key: MINIFICPP-72 > URL: https://issues.apache.org/jira/browse/MINIFICPP-72 > Project: NiFi MiNiFi C++ > Issue Type: New Feature >Affects Versions: 1.0.0 >Reporter: bqiu > Fix For: 1.0.0 > > > Add tar and compression support for MergeContent > will use the https://www.libarchive.org -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-72) Add tar and compression support for MergeContent
[ https://issues.apache.org/jira/browse/MINIFICPP-72?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16172364#comment-16172364 ] bqiu commented on MINIFICPP-72: --- Hey, Aldrin the NiFi merge content processor support merge format like Binary Concatenation/TAR/ZIP. The MergeContent Processor that i did for MiNiFI only support Binary Concatenation for the merge BIN files. We need to add TAR and ZIP format. The compress content processor is different than merge content, if we get the common libarchive in, we can scope the work for compress content processor also. > Add tar and compression support for MergeContent > > > Key: MINIFICPP-72 > URL: https://issues.apache.org/jira/browse/MINIFICPP-72 > Project: NiFi MiNiFi C++ > Issue Type: New Feature >Affects Versions: 1.0.0 >Reporter: bqiu > Fix For: 1.0.0 > > > Add tar and compression support for MergeContent > will use the https://www.libarchive.org -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (MINIFICPP-72) Add tar and compression support for MergeContent
bqiu created MINIFICPP-72: - Summary: Add tar and compression support for MergeContent Key: MINIFICPP-72 URL: https://issues.apache.org/jira/browse/MINIFICPP-72 Project: NiFi MiNiFi C++ Issue Type: New Feature Affects Versions: 1.0.0 Reporter: bqiu Fix For: 1.0.0 Add tar and compression support for MergeContent will use the https://www.libarchive.org -- This message was sent by Atlassian JIRA (v6.4.14#64029)
