Hello Chris, Did you find a solution for this? I looked at the underlying Apache HttpComponents API, and found InputStreamBody class. InputStreamBody doesn't copy whole byte array, but it streams data from specified input stream to the output stream per 4kb.
You may find this example useful, the example uses MultipartEntity and InputStreamBody together: https://www.programcreek.com/java-api-examples/index.php?api=org.apache.http.entity.mime.content.InputStreamBody Thanks, Koji On Mon, Sep 18, 2017 at 11:16 AM, Chris Conklin <chriseconk...@gmail.com> wrote: > Hello, > > Attempting to post to rest some multipart form data but running into memory > problems. Copying the flowfile inputstream to memory is obviously going to > be bad but can't seem to come up with another way to get the stream into the > MultipartForm. Any ideas? > > I am using 0.7.3 version of nifi for this custom processor. > > Here is a code snippet (the toBufferedInputStream works slightly better then > the new ByteArrayInputStream but eventually hits a memory issue depending on > size of heap): > > for (final FlowFile flowFile : flowFileList) { > session.read(flowFile, new InputStreamCallback() { > @Override > public void process(final InputStream rawIn) throws IOException > { > MultipartEntityBuilder builder = > MultipartEntityBuilder.create(); > builder.addTextBody("userName", "Chris"); > builder.addTextBody("password", "password"); > > //InputStream in =IOUtils.toBufferedInputStream(rawIn); > //InputStream in = new > ByteArrayInputStream(IOUtils.toByteArray(rawIn)); > > builder.addBinaryBody("file", > IOUtils.toBufferedInputStream(rawIn), ContentType.DEFAULT_BINARY , > "filename"); > HttpEntity entity2 = builder.build(); > post.setEntity(entity2); > post.setConfig(requestConfig); > } > }); > } > > > Thanks for any help anyone can offer. > > Chris >
