Mike,
That is accurate that Linux will not recognize \ as a path separator. It would
have no
problem reading the attribute but would not be able to write to a
directory/file with
that name. If you wanted to use that as a path in linux you could certainly
reference it
via:
${path:replace("\\", "/")}
However, this problem already exists today with GetFile - it is using the
File.separator
so that if you have a path like "subDir1/subDir2/myFile.txt" in Windows, the
path attribute
will be "subDir1\subDir2/" with both a forward slash and a backslash. I'm
suggesting that
with ListFile instead of using "subDir1\subDir2/" as the attribute it should be
"subDir1\subDir2\"
Thanks
-Mark
> On Dec 7, 2015, at 10:30 AM, Michael Moser <[email protected]> wrote:
>
> I disagree Mark, mainly due to the flowfile attributes such as 'path' that
> processors like GetFile (and I assume ListFile) create. Windows supports
> the forward slash but Unixes do not support the back slash File.separator.
> What happens when a flowfile created on a Windows NiFi is sent over to a
> Unix NiFi? Do Expression Language expressions that expect a forward slash
> begin to fail when reading a 'path' attribute that contains backslashes?
> I'm concerned ...
>
> -- Mike
>
>
> On Mon, Dec 7, 2015 at 10:12 AM, Mark Payne <[email protected]> wrote:
>
>> Errr, actually, after thinking about it more - operators who are running
>> on Windows should probably expect \ to be used instead of /
>>
>> So I think the better solution is to get rid of the "/" anywhere in the
>> processor and tests and always use File.separator.
>>
>> Will do so and create a patch, if nobody objects to that.
>>
>> Thanks
>> -Mark
>>
>>
>>> On Dec 7, 2015, at 9:59 AM, Mark Payne <[email protected]> wrote:
>>>
>>> Tony,
>>>
>>> I think there are really two possible solutions to this:
>>>
>>> 1) Always use / in the path attributes instead of \ -- i generally
>> prefer this approach, as windows has worked with forward slashes since Win
>> 98 (I believe?).
>>> 2) Have unit test look for file.separator -- benefit here is that it is
>> consistent with the way that GetFile works, and I'd not want to change that
>> because it's quite likely that some people are routing based on the 'path'
>> attribute.
>>>
>>> Normally I would tend to make consistency a high priority. However, I
>> see ListFile / FetchFile largely as a replacement for GetFile and am
>> guessing that in the future GetFile will be deprecated and removed. So I am
>> less inclined to stay consistent between the 'old generation' and 'new
>> generation' of processors. So personally I'd prefer to go the first route.
>>>
>>> -Mark
>>>
>>>
>>>> On Dec 7, 2015, at 12:35 AM, Tony Kurc <[email protected]> wrote:
>>>>
>>>> I submitted a patch to get the test to pass (NIFI-1261). Seems a bit
>> icky,
>>>> but I'll defer to Joe Skora and Mark Payne for correct behavior.
>>>>
>>>> On Mon, Dec 7, 2015 at 12:11 AM, Tony Kurc <[email protected]> wrote:
>>>>
>>>>> Joe - I'm putting a ticket in for a fix. Looks like it was introduced
>> by
>>>>> the NIFI-1246 patch.
>>>>>
>>>>> On Sun, Dec 6, 2015 at 11:36 PM, Joe Percivall <
>>>>> [email protected]> wrote:
>>>>>
>>>>>> Yup I saw the same behavior.
>>>>>>
>>>>>> On the second try (doing mvn clean install -rf
>> :nifi-standard-processors)
>>>>>> the tailfile error went away. The listFile error still occurred
>> though.
>>>>>>
>>>>>> Joe
>>>>>> - - - - - -
>>>>>> Joseph Percivall
>>>>>> linkedin.com/in/Percivall
>>>>>> e: [email protected]
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sunday, December 6, 2015 11:32 PM, Tony Kurc <[email protected]>
>> wrote:
>>>>>> Er, just the tailfile error
>>>>>>
>>>>>> On Dec 6, 2015 11:31 PM, "Tony Kurc" <[email protected]> wrote:
>>>>>>
>>>>>>> Joe, I had this happen and it worked on a second try.
>>>>>>> On Dec 6, 2015 11:23 PM, "Joe Percivall"
>> <[email protected]
>>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Windows 8 build fails with maven 3.3.3 and Java 1.8.0_65.
>>>>>>>>
>>>>>>>> I get these error messages:
>>>>>>>>
>>>>>>>>
>>>>>>>> TestListFile.testRecurse:441 expected:<subdir1[/]subdir2/> but
>>>>>>>> was:<subdir1[\]subdir2/>
>>>>>>>>
>>>>>>
>> TestTailFile.testMultipleRolloversAfterHavingReadAllDataWhileStillRunning:381
>>>>>>>> expected:<[world]> but was:<[abc
>>>>>>>>
>>>>>>>> These were not any of the same errors I saw last time testing on
>>>>>> Windows
>>>>>>>> a couple weeks ago.
>>>>>>>>
>>>>>>>> Joe
>>>>>>>> - - - - - -
>>>>>>>> Joseph Percivall
>>>>>>>> linkedin.com/in/Percivall
>>>>>>>> e: [email protected]
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sunday, December 6, 2015 9:05 PM, Tony Kurc <[email protected]>
>>>>>> wrote:
>>>>>>>> I've gotten confirmation of CentOS 7.1.1503 x86_64, Oracle JDK 8u66
>>>>>>>> working
>>>>>>>> fine. and Fedora 23 not working with the same error that Andre
>>>>>> reported.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sun, Dec 6, 2015 at 5:50 PM, Tony Kurc <[email protected]> wrote:
>>>>>>>>
>>>>>>>>> I'll also try it on windows 10 (again x64_64)
>>>>>>>>>
>>>>>>>>> On Sun, Dec 6, 2015 at 5:36 PM, <[email protected]>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I can run it on Windows 8 tonight if no one else has.
>>>>>>>>>>
>>>>>>>>>> Joe
>>>>>>>>>>
>>>>>>>>>> Sent from my phone
>>>>>>>>>>
>>>>>>>>>>> On Dec 6, 2015, at 4:09 PM, Tony Kurc <[email protected]> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Signatures and hashes look good.
>>>>>>>>>>>
>>>>>>>>>>> Built fine on Ubuntu 14.04 x86_64. I even cursed a little bit
>>>>>> less at
>>>>>>>>>>> TestJdbcHugeStream!
>>>>>>>>>>>
>>>>>>>>>>> LICENSE, NOTICE and README look good.
>>>>>>>>>>>
>>>>>>>>>>> Docs look good.
>>>>>>>>>>>
>>>>>>>>>>> Binary ran successfully.
>>>>>>>>>>>
>>>>>>>>>>> +1
>>>>>>>>>>>
>>>>>>>>>>> Did anyone try building on windows?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> On Sat, Dec 5, 2015 at 11:46 PM, Aldrin Piri <
>>>>>> [email protected]>
>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Followed helper provided by Joe.
>>>>>>>>>>>>
>>>>>>>>>>>> Keys good.
>>>>>>>>>>>> Signatures good.
>>>>>>>>>>>> Hashes good.
>>>>>>>>>>>> Source release builds and passes contrib
>>>>>>>>>>>> Required docs present and look correct.
>>>>>>>>>>>> Checked out copy of repo for specified commit hash and diff'd
>>>>>>>> against
>>>>>>>>>>>> source bundle. Commit is as anticipated.
>>>>>>>>>>>> Ran convenience binary with varying templates all successfully.
>>>>>>>>>>>>
>>>>>>>>>>>> Release notes and upgrade/migration guides look good.
>>>>>>>>>>>>
>>>>>>>>>>>> Kudos to the community on all the efforts involved with this
>>>>>>>> release.
>>>>>>>>>>>>
>>>>>>>>>>>> +1, Release this package as Apache NiFi 0.4.0
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> On Sat, Dec 5, 2015 at 10:32 PM, Joe Witt <[email protected]>
>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hello NiFi Community,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am pleased to be calling this vote for the source release of
>>>>>>>> Apache
>>>>>>>>>>>>> NiFi 0.4.0.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The source zip, including signatures, digests, and associated
>>>>>>>>>>>>> convenience binaries can be found at:
>>>>>>>>>>>>> https://dist.apache.org/repos/dist/dev/nifi/nifi-0.4.0/
>>>>>>>>>>>>>
>>>>>>>>>>>>> The Git tag is nifi-0.4.0-RC1
>>>>>>>>>>>>> The Git commit ID is 191a56f54e3ec178f9f29e1287f23ba66dbf9e43
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=191a56f54e3ec178f9f29e1287f23ba66dbf9e43
>>>>>>>>>>>>>
>>>>>>>>>>>>> Checksums of NiFi 0.4.0 Source Release:
>>>>>>>>>>>>> MD5: b69fd7ec632d7569906e20508058556b
>>>>>>>>>>>>> SHA1: 31d88ec7a8431ba5935370eb09be7a343c46411c
>>>>>>>>>>>>>
>>>>>>>>>>>>> Release artifacts are signed with the following key:
>>>>>>>>>>>>> https://people.apache.org/keys/committer/joewitt.asc
>>>>>>>>>>>>>
>>>>>>>>>>>>> KEYS file available here:
>>>>>>>>>>>>> https://dist.apache.org/repos/dist/release/nifi/KEYS
>>>>>>>>>>>>>
>>>>>>>>>>>>> 152 issues were closed/resolved for this release:
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12333070
>>>>>>>>>>>>>
>>>>>>>>>>>>> Release note highlights:
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version0.4.0
>>>>>>>>>>>>>
>>>>>>>>>>>>> Migration/Upgrade guidance:
>>>>>>>>>>>>>
>>>>>>>> https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance
>>>>>>>>>>>>>
>>>>>> https://cwiki.apache.org/confluence/display/NIFI/Upgrading+NiFi
>>>>>>>>>>>>>
>>>>>>>>>>>>> The vote will be open for 72 hours.
>>>>>>>>>>>>> Please download the release candidate and evaluate the
>> necessary
>>>>>>>> items
>>>>>>>>>>>>> including checking hashes, signatures, build from source, and
>>>>>> test.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Then please vote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> [ ] +1 Release this package as Apache NiFi 0.4.0
>>>>>>>>>>>>> [ ] +0 no opinion
>>>>>>>>>>>>> [ ] -1 Do not release this package because...
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>
>>
>>
