Re: ExecuteStreamCommand failing to unzip incoming flowfiles

2024-02-15 Thread James McMahon
This is proving to be difficult to do in practice. Many of the filenames in the zip contain spaces and other characters, and these are failing to be passed to the tar successfully. This is the command I am testing at the command line to first extract the filenames: unzip -l

Re: ExecuteStreamCommand failing to unzip incoming flowfiles

2024-02-02 Thread Michael Moser
Yes, that's exactly what those commands do. Your linux commands like unzip and tar can probably read directly from /dev/stdin and write directly to /dev/stdout if you want to. -- Mike On Fri, Feb 2, 2024 at 9:22 AM James McMahon wrote: > Hi Michael. This is a very clever approach: convert

Re: ExecuteStreamCommand failing to unzip incoming flowfiles

2024-02-02 Thread James McMahon
Hi Michael. This is a very clever approach: convert from a zip (which UnpackContent does not preserve file metadata for extracted files) to a tar (for which UnpackContent does preserve file metadata), then employ the UnpackContent. One quick followup question. The ExecuteStreamCommand will be in

Re: ExecuteStreamCommand failing to unzip incoming flowfiles

2024-02-01 Thread Michael Moser
Hi Jim, The ExecuteStreamCommand will only output 1 flowfile, so using it to unzip in this fashion won't yield the results you need. Instead, you might try a workaround with ExecuteStreamCommand to unzip your file and then tar to repackage it. Then UnpackContent should be able to read the tar

Re: ExecuteStreamCommand failing to unzip incoming flowfiles

2024-01-31 Thread James McMahon
If anyone can show me how to get my ExecuteStreamCommand configured properly as a workaround, I am still interested in that. Jim On Wed, Jan 31, 2024 at 12:39 PM James McMahon wrote: > I tried to find a Create option for tickets here, >

Re: ExecuteStreamCommand failing to unzip incoming flowfiles

2024-01-31 Thread James McMahon
I tried to find a Create option for tickets here, https://issues.apache.org/jira/projects/NIFI/issues/NIFI-11859?filter=allopenissues . I did not find one, and suspect maybe I have no such privilege perhaps? In any case, thank you for creating that. Jim On Wed, Jan 31, 2024 at 12:37 PM Joe Witt

Re: ExecuteStreamCommand failing to unzip incoming flowfiles

2024-01-31 Thread Joe Witt
I went ahead and wrote it up here https://issues.apache.org/jira/browse/NIFI-12709 Thanks On Wed, Jan 31, 2024 at 10:30 AM James McMahon wrote: > Happy to do that Joe. How do I create and submit a JIRA for consideration? > I have not done one - at least, not for years. > If you get me started,

Re: ExecuteStreamCommand failing to unzip incoming flowfiles

2024-01-31 Thread James McMahon
Happy to do that Joe. How do I create and submit a JIRA for consideration? I have not done one - at least, not for years. If you get me started, I will do a concise and thorough description in the ticket. Sincerely, Jim On Wed, Jan 31, 2024 at 12:12 PM Joe Witt wrote: > James, > > Makes sense

Re: ExecuteStreamCommand failing to unzip incoming flowfiles

2024-01-31 Thread Joe Witt
James, Makes sense to create a JIRA to improve UnpackContent to extract these attributes in the event of a zip file that happens to present them. The concept of lastModifiedDate does appear easily accessed if available in the metadata. Owner/Creator/Creation information looks less standard in

ExecuteStreamCommand failing to unzip incoming flowfiles

2024-01-31 Thread James McMahon
I tried to use UnpackContent to extract the files within a zip file named ABC DEF (1).zip. (the filename has spaces in its name). UnpackContent seemed to work, but it did not preserve file attributes from the files in the zip. For example, the lastModifiedTime is not available so downstream I