[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16798802#comment-16798802 ] Sergey Gubchenko commented on NIFI-4621: [~fcinqmars] working on it :) > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16798672#comment-16798672 ] Felix Cinq-Mars commented on NIFI-4621: --- Hi, Do we have any updates on this ? :) Thanks! > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710968#comment-16710968 ] ASF GitHub Bot commented on NIFI-4621: -- Github user ijokarumawak commented on a diff in the pull request: https://github.com/apache/nifi/pull/3201#discussion_r239324396 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListFTP.java --- @@ -35,17 +36,21 @@ import org.apache.nifi.components.ValidationResult; import org.apache.nifi.components.state.Scope; import org.apache.nifi.context.PropertyContext; +import org.apache.nifi.expression.ExpressionLanguageScope; import org.apache.nifi.processor.ProcessContext; import org.apache.nifi.processor.util.list.ListedEntityTracker; import org.apache.nifi.processors.standard.util.FileTransfer; import org.apache.nifi.processors.standard.util.FTPTransfer; @PrimaryNodeOnly @TriggerSerially -@InputRequirement(Requirement.INPUT_FORBIDDEN) +@InputRequirement(Requirement.INPUT_ALLOWED) --- End diff -- Let's design how we improve this processors (or create other processors if necessary) first. I remember there was some discussion on JIRA or ML before. Please check those, too. If PATh can be supplied by incoming FlowFiles, then I personally prefer passing minimum timestamps by FlowFiles, too. > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710921#comment-16710921 ] ASF GitHub Bot commented on NIFI-4621: -- Github user kislayom commented on a diff in the pull request: https://github.com/apache/nifi/pull/3201#discussion_r239314421 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListFTP.java --- @@ -35,17 +36,21 @@ import org.apache.nifi.components.ValidationResult; import org.apache.nifi.components.state.Scope; import org.apache.nifi.context.PropertyContext; +import org.apache.nifi.expression.ExpressionLanguageScope; import org.apache.nifi.processor.ProcessContext; import org.apache.nifi.processor.util.list.ListedEntityTracker; import org.apache.nifi.processors.standard.util.FileTransfer; import org.apache.nifi.processors.standard.util.FTPTransfer; @PrimaryNodeOnly @TriggerSerially -@InputRequirement(Requirement.INPUT_FORBIDDEN) +@InputRequirement(Requirement.INPUT_ALLOWED) --- End diff -- Sure, based on the input provided i will try to identify required changes. The latest timestamp management part might be tricky are we required to keep latest timestamp per path, hostname, and user wise or just the latest one ? I have so far assumed to manage latest timestamp only for the latest. If we plan to manages all timestamps i will need to rework on a number of things here, and may would like to approach the problem starting with proposed design. > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[
https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710923#comment-16710923
]
ASF GitHub Bot commented on NIFI-4621:
--
Github user kislayom commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3201#discussion_r239314794
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListFTP.java
---
@@ -35,17 +36,21 @@
import org.apache.nifi.components.ValidationResult;
import org.apache.nifi.components.state.Scope;
import org.apache.nifi.context.PropertyContext;
+import org.apache.nifi.expression.ExpressionLanguageScope;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.util.list.ListedEntityTracker;
import org.apache.nifi.processors.standard.util.FileTransfer;
import org.apache.nifi.processors.standard.util.FTPTransfer;
@PrimaryNodeOnly
@TriggerSerially
-@InputRequirement(Requirement.INPUT_FORBIDDEN)
+@InputRequirement(Requirement.INPUT_ALLOWED)
@Tags({"list", "ftp", "remote", "ingest", "source", "input", "files"})
@CapabilityDescription("Performs a listing of the files residing on an FTP
server. For each file that is found on the remote server, a new FlowFile will
be created with the filename attribute "
+ "set to the name of the file on the remote server. This can then be
used in conjunction with FetchFTP in order to fetch those files.")
+@DynamicProperty(name = "Relationship Name", value = "Attribute Expression
Language",
+expressionLanguageScope =
ExpressionLanguageScope.FLOWFILE_ATTRIBUTES, description = "Routes FlowFiles
whose attributes match the "
++ "Attribute Expression Language specified in the Dynamic Property
Value to the Relationship specified in the Dynamic Property Key")
--- End diff --
It is used to set the scope of Expression language. More details as under:
https://static.javadoc.io/org.apache.nifi/nifi-api/1.7.0/org/apache/nifi/expression/ExpressionLanguageScope.html
> Indicates the scope of expression language on a property descriptor.
Scope of the expression language is hierarchical. NONE -> VARIABLE_REGISTRY ->
FLOWFILE_ATTRIBUTES When scope is set to FlowFiles attributes, variables are
evaluated against attributes of each incoming flow file. If no matching
attribute is found, variable registry will be checked. NONE - expression
language is not supported VARIABLE_REGISTRY is hierarchically constructed as
below: | Variables defined at process group level and then, recursively, up
| to the higher process group until the root process group. |--- Variables
defined in custom properties files through the |
nifi.variable.registry.properties property in nifi.properties file. |--
Environment variables defined at JVM level and system properties.
FLOWFILE_ATTRIBUTES - will check attributes of each individual flow file
>
> Allow inputs to ListSFTP and ListFTP
>
>
> Key: NIFI-4621
> URL: https://issues.apache.org/jira/browse/NIFI-4621
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.4.0
>Reporter: Soumya Shanta Ghosh
>Assignee: Peter Wicks
>Priority: Critical
>
> ListSFTP supports listing of the supplied directory (Remote Path)
> out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password"
> / "Private Key Passphrase".
> The password can change at a regular interval (depending on organization
> policy) or the Hostname or the Remote Path can change based on some other
> requirement.
> This is a case to allow ListSFTP to leverage the use of Nifi Expression
> language so that the values of Hostname, Password and/or Remote Path can be
> set based on the attributes of an incoming flow file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[
https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710903#comment-16710903
]
Kislay Kumar commented on NIFI-4621:
[~patricker] Some code snippets from SFTPTransfer.java is under:
*Username, hostname, port:*
{quote}final String username =
ctx.getProperty(USERNAME).evaluateAttributeExpressions(flowFile).getValue();
final Session session = jsch.getSession(username,
ctx.getProperty(HOSTNAME).evaluateAttributeExpressions(flowFile).getValue(),
ctx.getProperty(PORT).evaluateAttributeExpressions(flowFile).asInteger().intValue());
{quote}
*privateKeyFile, password*
{quote}
final String privateKeyFile =
ctx.getProperty(PRIVATE_KEY_PATH).evaluateAttributeExpressions(flowFile).getValue();
if (privateKeyFile != null) {
jsch.addIdentity(privateKeyFile,
ctx.getProperty(PRIVATE_KEY_PASSPHRASE).evaluateAttributeExpressions(flowFile).getValue());
}
final String password =
ctx.getProperty(FileTransfer.PASSWORD).evaluateAttributeExpressions(flowFile).getValue();
{quote}
> Allow inputs to ListSFTP and ListFTP
>
>
> Key: NIFI-4621
> URL: https://issues.apache.org/jira/browse/NIFI-4621
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.4.0
>Reporter: Soumya Shanta Ghosh
>Assignee: Peter Wicks
>Priority: Critical
>
> ListSFTP supports listing of the supplied directory (Remote Path)
> out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password"
> / "Private Key Passphrase".
> The password can change at a regular interval (depending on organization
> policy) or the Hostname or the Remote Path can change based on some other
> requirement.
> This is a case to allow ListSFTP to leverage the use of Nifi Expression
> language so that the values of Hostname, Password and/or Remote Path can be
> set based on the attributes of an incoming flow file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710909#comment-16710909 ] ASF GitHub Bot commented on NIFI-4621: -- Github user ijokarumawak commented on a diff in the pull request: https://github.com/apache/nifi/pull/3201#discussion_r239310311 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListFTP.java --- @@ -35,17 +36,21 @@ import org.apache.nifi.components.ValidationResult; import org.apache.nifi.components.state.Scope; import org.apache.nifi.context.PropertyContext; +import org.apache.nifi.expression.ExpressionLanguageScope; import org.apache.nifi.processor.ProcessContext; import org.apache.nifi.processor.util.list.ListedEntityTracker; import org.apache.nifi.processors.standard.util.FileTransfer; import org.apache.nifi.processors.standard.util.FTPTransfer; @PrimaryNodeOnly @TriggerSerially -@InputRequirement(Requirement.INPUT_FORBIDDEN) +@InputRequirement(Requirement.INPUT_ALLOWED) --- End diff -- Changing input requirement is not enough if we want to support incoming FlowFiles. Processors need to: 1. Take input FlowFiles, LogMessage processor can be the simplest example, please look [here](https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/LogMessage.java#L125) 2. Evaluate ExpressionLanguages using the FlowFiles, again LogMessage as [an example](https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/LogMessage.java#L130) Currently [FTPTransfer.getListing()](https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/FTPTransfer.java#L187) doesn't do above things. And we also need to support existing use-cases with no input connection. Also, we need to track listing state such as latest timestamps ... etc. We need to do that efficiently with dynamically changing target paths if it's configured by incoming FlowFiles attribute. There are more things need to be done. > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[
https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710908#comment-16710908
]
ASF GitHub Bot commented on NIFI-4621:
--
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3201#discussion_r239307533
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListFTP.java
---
@@ -35,17 +36,21 @@
import org.apache.nifi.components.ValidationResult;
import org.apache.nifi.components.state.Scope;
import org.apache.nifi.context.PropertyContext;
+import org.apache.nifi.expression.ExpressionLanguageScope;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.util.list.ListedEntityTracker;
import org.apache.nifi.processors.standard.util.FileTransfer;
import org.apache.nifi.processors.standard.util.FTPTransfer;
@PrimaryNodeOnly
@TriggerSerially
-@InputRequirement(Requirement.INPUT_FORBIDDEN)
+@InputRequirement(Requirement.INPUT_ALLOWED)
@Tags({"list", "ftp", "remote", "ingest", "source", "input", "files"})
@CapabilityDescription("Performs a listing of the files residing on an FTP
server. For each file that is found on the remote server, a new FlowFile will
be created with the filename attribute "
+ "set to the name of the file on the remote server. This can then be
used in conjunction with FetchFTP in order to fetch those files.")
+@DynamicProperty(name = "Relationship Name", value = "Attribute Expression
Language",
+expressionLanguageScope =
ExpressionLanguageScope.FLOWFILE_ATTRIBUTES, description = "Routes FlowFiles
whose attributes match the "
++ "Attribute Expression Language specified in the Dynamic Property
Value to the Relationship specified in the Dynamic Property Key")
--- End diff --
What's this used for?
> Allow inputs to ListSFTP and ListFTP
>
>
> Key: NIFI-4621
> URL: https://issues.apache.org/jira/browse/NIFI-4621
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.4.0
>Reporter: Soumya Shanta Ghosh
>Assignee: Peter Wicks
>Priority: Critical
>
> ListSFTP supports listing of the supplied directory (Remote Path)
> out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password"
> / "Private Key Passphrase".
> The password can change at a regular interval (depending on organization
> policy) or the Hostname or the Remote Path can change based on some other
> requirement.
> This is a case to allow ListSFTP to leverage the use of Nifi Expression
> language so that the values of Hostname, Password and/or Remote Path can be
> set based on the attributes of an incoming flow file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710901#comment-16710901 ] Kislay Kumar commented on NIFI-4621: [~ijokarumawak] sure, I thought of putting it in review but could not find it. > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710839#comment-16710839 ] Kislay Kumar commented on NIFI-4621: Hi [~patricker] I also assumed (as the reported mentioned) initially that we might be missing EL on the processors properties. However I ended up finding that almost all properties are already EL enabled. It was only that the processor were not allowed to have input connection. If you would like i am happy to post some evidence for that. And i did test EL on the properties mentioned in the Jira. > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > Fix For: 1.9.0 > > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710366#comment-16710366 ] Peter Wicks commented on NIFI-4621: --- Hi [~kislayom], Your PR does not look correct to me. Did you check-in all of the commit's, maybe something got lost? Right now I don't see most of the changes I expected to see (EL changes you mentioned before are missing, etc...) > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Peter Wicks >Priority: Critical > Fix For: 1.9.0 > > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709875#comment-16709875 ] ASF GitHub Bot commented on NIFI-4621: -- Github user kislayom closed the pull request at: https://github.com/apache/nifi/pull/3201 > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > Fix For: 1.9.0 > > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709876#comment-16709876 ] ASF GitHub Bot commented on NIFI-4621: -- GitHub user kislayom reopened a pull request: https://github.com/apache/nifi/pull/3201 NIFI-4621 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [] Have you written or updated unit tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kislayom/nifi NIFI-4621 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3201.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3201 commit 0512754739cf63785aebfc53a89bb75d37bacbed Author: Kislay Kumar Date: 2018-12-04T12:03:56Z NIFI-4621 Enable ListSFTP and ListFTP processor to accept connection from other processors, also validated the explression language is working for properties of both the processors. commit 9da3a8700244b4fdc06c634805bbc0b25369623a Author: Kislay Kumar Date: 2018-12-04T13:05:34Z NIFI-4621 changes in processor ListSFTP and ListFTP to also have input connection and expression language for properties commit 4319d472a25fecc2bbb66501c98d91828470d15d Author: Kislay Kumar Date: 2018-12-04T12:03:56Z NIFI-4621 Enable ListSFTP and ListFTP processor to accept connection from other processors, also validated the explression language is working for properties of both the processors. NIFI-4621 changes in processor ListSFTP and ListFTP to also have input connection and expression language for properties commit d4321cf72085786f1f0b9453270fc6c7c5d32fe3 Author: kislayom Date: 2018-12-05T09:25:26Z Merge branch 'NIFI-4621' of https://github.com/kislayom/nifi into NIFI-4621 > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > Fix For: 1.9.0 > > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709840#comment-16709840 ] ASF GitHub Bot commented on NIFI-4621: -- GitHub user kislayom opened a pull request: https://github.com/apache/nifi/pull/3201 NIFI-4621 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [] Have you written or updated unit tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kislayom/nifi NIFI-4621 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3201.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3201 commit 0512754739cf63785aebfc53a89bb75d37bacbed Author: Kislay Kumar Date: 2018-12-04T12:03:56Z NIFI-4621 Enable ListSFTP and ListFTP processor to accept connection from other processors, also validated the explression language is working for properties of both the processors. commit 9da3a8700244b4fdc06c634805bbc0b25369623a Author: Kislay Kumar Date: 2018-12-04T13:05:34Z NIFI-4621 changes in processor ListSFTP and ListFTP to also have input connection and expression language for properties commit 4319d472a25fecc2bbb66501c98d91828470d15d Author: Kislay Kumar Date: 2018-12-04T12:03:56Z NIFI-4621 Enable ListSFTP and ListFTP processor to accept connection from other processors, also validated the explression language is working for properties of both the processors. NIFI-4621 changes in processor ListSFTP and ListFTP to also have input connection and expression language for properties commit d4321cf72085786f1f0b9453270fc6c7c5d32fe3 Author: kislayom Date: 2018-12-05T09:25:26Z Merge branch 'NIFI-4621' of https://github.com/kislayom/nifi into NIFI-4621 > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > Fix For: 1.9.0 > > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709591#comment-16709591 ] Koji Kawamura commented on NIFI-4621: - JFYI, you can update the same PR even after squishing commits, by adding '-f' option when you push to your remote git branch. > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > Fix For: 1.9.0 > > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709492#comment-16709492 ] Kislay Kumar commented on NIFI-4621: I have made a mistake of having multiple commits in the PR. I will squash the commits and raise PR again. ETA 9 PM AEDT. > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > Fix For: 1.9.0 > > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709493#comment-16709493 ] ASF GitHub Bot commented on NIFI-4621: -- Github user kislayom closed the pull request at: https://github.com/apache/nifi/pull/3197 > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > Fix For: 1.9.0 > > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16708677#comment-16708677 ] ASF GitHub Bot commented on NIFI-4621: -- GitHub user kislayom opened a pull request: https://github.com/apache/nifi/pull/3197 NIFI-4621 NIFI-4621 changes in processor ListSFTP and ListFTP to also have input connection and expression language for properties. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kislayom/nifi NIFI-4621 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3197.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3197 commit 0512754739cf63785aebfc53a89bb75d37bacbed Author: Kislay Kumar Date: 2018-12-04T12:03:56Z NIFI-4621 Enable ListSFTP and ListFTP processor to accept connection from other processors, also validated the explression language is working for properties of both the processors. commit 9da3a8700244b4fdc06c634805bbc0b25369623a Author: Kislay Kumar Date: 2018-12-04T13:05:34Z NIFI-4621 changes in processor ListSFTP and ListFTP to also have input connection and expression language for properties > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16708646#comment-16708646 ] ASF GitHub Bot commented on NIFI-4621: -- Github user kislayom closed the pull request at: https://github.com/apache/nifi/pull/3196 > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP and ListFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16708644#comment-16708644 ] ASF GitHub Bot commented on NIFI-4621: -- GitHub user kislayom opened a pull request: https://github.com/apache/nifi/pull/3196 NIFI-4621 Enable ListSFTP and ListFTP processor to accept connection … Modifed ListSftp and ListFtp processors so they can accept connection from other processors, also validated the expression language is working for properties of both the processors. This is for NIFI-4621. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kislayom/nifi NIFI-4621 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3196.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3196 commit 0512754739cf63785aebfc53a89bb75d37bacbed Author: Kislay Kumar Date: 2018-12-04T12:03:56Z NIFI-4621 Enable ListSFTP and ListFTP processor to accept connection from other processors, also validated the explression language is working for properties of both the processors. > Allow inputs to ListSFTP and ListFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16706078#comment-16706078 ] Kislay Kumar commented on NIFI-4621: [~patricker] : I should be able to commit in 3 days. Assessing impact on ListFTP because both of them extends ListFileTransfer. Would you suggest I implement similar changes for ListFTP as well. Personally I am inclined to modify ListFTP also. > Allow inputs to ListSFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703310#comment-16703310 ] Peter Wicks commented on NIFI-4621: --- [~kislayom] How's this going? > Allow inputs to ListSFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16678904#comment-16678904 ] Kislay Kumar commented on NIFI-4621: [~markap14], [~patricker] : Thanks for quick help. I am am hoping to come back quickly for review. > Allow inputs to ListSFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16678720#comment-16678720 ] Mark Payne commented on NIFI-4621: -- [~kislayom] I have assigned the Jira to you. You should now be able to assign Jiras to yourself, as well. [~patricker] FYI - no need for Joe to grant permissions - anyone on the PMC, I believe, should have the ability to do so. > Allow inputs to ListSFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Kislay Kumar >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16678545#comment-16678545 ] Peter Wicks commented on NIFI-4621: --- [~kislayom], I can't do this with your current permissions. [~joewitt] will need to grant you permissions to become an assignee. Hopefully he'll see this and grant you permission. Either way, you can work on it and submit a PR. > Allow inputs to ListSFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Puspendu Banerjee >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16677670#comment-16677670 ] Kislay Kumar commented on NIFI-4621: Sure, Can you please assign this task to me. It would be easy for me to keep track. > Allow inputs to ListSFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Puspendu Banerjee >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16677659#comment-16677659 ] Peter Wicks commented on NIFI-4621: --- Sounds great. Let me know when it's PR'd and I can review. > Allow inputs to ListSFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Puspendu Banerjee >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16677638#comment-16677638 ] Kislay Kumar commented on NIFI-4621: I would like to pick this task if no one is working. > Allow inputs to ListSFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Puspendu Banerjee >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538961#comment-16538961 ] Peter Wicks commented on NIFI-4621: --- [~puspendu.baner...@gmail.com] Are you working on this? > Allow inputs to ListSFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Puspendu Banerjee >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16429928#comment-16429928 ] scott wilburn commented on NIFI-4621: - I have also wanted to see this feature for ListSFTP and other List processors that do not allow an incoming relationship. In my case, I would like to use an external table to configure NiFi connections. To accomplish this, I would first read in my configuration, then create flowfiles for each connection that contains the configuration parameters as attributes. It doesn't work for SFTP unless it can read attributes from an incoming flowfile to use as configuration parameters, such as username, directory, etc. I had a conversation with devs in the dev email distro. The recommendation was to create a new processor which does not keep state. The incoming flowfile would trigger the list, which would then generate zero or more output flowfiles, one for each remote file matching. You should be able to follow the email chain discussion I had in the dev email archive for subject "ListSFTP incoming relationship". I was thinking about building a custom processor to do this also before I read your jira. > Allow inputs to ListSFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Puspendu Banerjee >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348705#comment-16348705 ] Soumya Shanta Ghosh commented on NIFI-4621: --- Hi [~puspendu.baner...@gmail.com] The description itself is my the case. This is a real scenario where we have to read list a directory and subsequently fetch the files in that directory using SFTP with the help of NiFi. Now this directory is on an edge node in the company internal network and not external network. I have written a custom processor extending the ListSFTP and deployed it as a nar file. This custom processor accepts inputs to it, transfers the attributes (which contain the username/password/keypassphrase etc.) to the out going flow file. Please let me know your views on this. Thanks and Regards, Soumya Shanta Ghosh > Allow inputs to ListSFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Assignee: Puspendu Banerjee >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4621) Allow inputs to ListSFTP
[ https://issues.apache.org/jira/browse/NIFI-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16334642#comment-16334642 ] Puspendu Banerjee commented on NIFI-4621: - Requesting for comment on pros, cons and security implication. [~soumya.ghosh] Can you provide a real life usecase for this feature? can we go for a limited number of SFTP hosts? Opening this feature for all hosts / unlimited unique sftp hosts could become a stepping stone for DDoS attack. > Allow inputs to ListSFTP > > > Key: NIFI-4621 > URL: https://issues.apache.org/jira/browse/NIFI-4621 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.4.0 >Reporter: Soumya Shanta Ghosh >Priority: Critical > > ListSFTP supports listing of the supplied directory (Remote Path) > out-of-the-box on the supplied "Hostname" using the 'Username" and 'Password" > / "Private Key Passphrase". > The password can change at a regular interval (depending on organization > policy) or the Hostname or the Remote Path can change based on some other > requirement. > This is a case to allow ListSFTP to leverage the use of Nifi Expression > language so that the values of Hostname, Password and/or Remote Path can be > set based on the attributes of an incoming flow file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
