Unable to connect from camel-ftp to apache mina-sshd

2016-09-02 Thread Goyal, Arpit
Hi Colleagues, I am trying a very simple scenario where I am hosting apache mina-sshd as SFTP Server in my UNIT Test. Any one has idea why connection to SFTP always fails? Camel ftp - 2.16.3 Apache mina sshd - 1.2.0 Regards, Arpit. My Camel Route in test case

Re: Netty Server vs Jetty Server

2016-09-02 Thread Ranx
I think maybe a better explanation of what I'd like to accomplish is in order. The first is about CXF itself and the second is about Netty. The following is an example of a service I've set up that uses a single interface called PaymentServicesAPI which is nothing more than an interface that

Re: downloading large files in chunks

2016-09-02 Thread Quinn Stevenson
The way I know it’s streaming is running the route. You’ll see the log entries (“Download Triggered” and “Writing File”) fairly close together. Then if you watch the filesystem, you’ll see the file size on disk growing. Also, I’m using default JVM parameters, so the heap isn’t big enough

Re: downloading large files in chunks

2016-09-02 Thread Brad Johnson
https://netty.io/4.0/api/io/netty/handler/codec/http/HttpChunkedInput.html That's why I thought the Camel Netty with chunked would only read the entire stream of the specified chunk size in. On Fri, Sep 2, 2016 at 10:11 AM, S Ahmed wrote: > I'm just the consumer

Re: downloading large files in chunks

2016-09-02 Thread S Ahmed
Also, is there a way for me to test if the endpoint supports streaming? I'm on OSX so any open source tools to test this? On Fri, Sep 2, 2016 at 11:11 AM, S Ahmed wrote: > I'm just the consumer (downloading), the file can be anywhere like s3 or > centos.org! > > > > On

Netty Server vs Jetty Server

2016-09-02 Thread Ranx
When I look at this how the shared netty server works it seems very much like what I'd want for microservice bundles in an OSGi environment. http://camel.apache.org/netty-http-server-example.html Can it be used with CXF? Are there any examples of using this with CXF? If not, can one create

Re: Problem with objects not being released from memory.

2016-09-02 Thread litian
We actually finally figured out yesterday where we went wrong, after a week of trying different things. It turns out the mock statements hold the objects in memory forever. It was fine while testing since it does not fill up the memory for a few thousand messages. After removing the mock from

Re: downloading large files in chunks

2016-09-02 Thread S Ahmed
I'm just the consumer (downloading), the file can be anywhere like s3 or centos.org! On Fri, Sep 2, 2016 at 11:09 AM, Brad Johnson wrote: > By the way S. Ahmed, do you have control of both ends of this I mean > client/server or are you just on the client/consumer

Re: downloading large files in chunks

2016-09-02 Thread Brad Johnson
By the way S. Ahmed, do you have control of both ends of this I mean client/server or are you just on the client/consumer side? On Fri, Sep 2, 2016 at 10:01 AM, Brad Johnson wrote: > Absolutely. Love to set up a VM for my server. I just had a "duh" moment > when

Re: Problem with objects not being released from memory.

2016-09-02 Thread Quinn Stevenson
I’ve used HAPI quite a bit, and I’ve never noticed a memory leak with it (doesn’t mean it isn’t there though). Can you tell what object/class is holding the objects? When you said “they’re never cleaned up by the GC”, I’m assuming you forced a GC? > On Aug 30, 2016, at 1:28 PM, litian

Re: downloading large files in chunks

2016-09-02 Thread Brad Johnson
Absolutely. Love to set up a VM for my server. I just had a "duh" moment when I did it. No harm, no foul. On Fri, Sep 2, 2016 at 10:00 AM, Quinn Stevenson < qu...@pronoia-solutions.com> wrote: > Sorry - I wanted to put in and example that worked, and download something > big to make sure it

Re: downloading large files in chunks

2016-09-02 Thread Quinn Stevenson
Sorry - I wanted to put in and example that worked, and download something big to make sure it was streaming. Hopefully you needed a new CentOS image :-) > On Sep 2, 2016, at 8:58 AM, Brad Johnson wrote: > > Neat. I accidentally clicked on the link and Chrome

Re: downloading large files in chunks

2016-09-02 Thread Brad Johnson
Neat. I accidentally clicked on the link and Chrome downloaded the ISO for me. Are you propagating Trojan horses here? Heh. On Fri, Sep 2, 2016 at 9:56 AM, Quinn Stevenson wrote: > I think something like this might work for you > > > > > > >

Re: downloading large files in chunks

2016-09-02 Thread Brad Johnson
Hmmm. That could be a problem if it doesn't actually chunk. I thought it read the entire chunk into memory before letting you read it. So if the chunk size is 10mb it would download that whole 10mb and then let you read, then fetch the next 10mb and let you read. But that may not be the case. I

Re: downloading large files in chunks

2016-09-02 Thread Brad Johnson
By the way, I don't know if you said or not but do you control both sides of this or just the consumer side? On Fri, Sep 2, 2016 at 9:51 AM, Brad Johnson wrote: > Hmmm. That could be a problem if it doesn't actually chunk. I thought it > read the entire chunk into

Re: downloading large files in chunks

2016-09-02 Thread Quinn Stevenson
I think something like this might work for you > On Sep 2, 2016, at 8:51 AM, Brad Johnson wrote: > > Hmmm. That could be a problem if it doesn't actually chunk. I thought it > read the entire chunk into memory before letting you read

Re: Unmarshal fixed length Binary data

2016-09-02 Thread Brad Johnson
I second Beanio. I've used it for fixed length multi-line records and it is fabulous. On Fri, Sep 2, 2016 at 9:28 AM, Quinn Stevenson wrote: > Have you looked at the BeanIO DataFormat? (http://camel.apache.org/ > beanio.html ) >

Re: Unmarshal fixed length Binary data

2016-09-02 Thread Quinn Stevenson
Have you looked at the BeanIO DataFormat? (http://camel.apache.org/beanio.html ) > On Sep 2, 2016, at 7:35 AM, kaustubhkane wrote: > > Hi, > > We have a fixed length Binary data. > > I am looking at the Bindy Data Format and found

Re: downloading large files in chunks

2016-09-02 Thread S Ahmed
Brad, that page says this: "Notice Netty4 HTTP reads the entire stream into memory using io.netty.handler.codec.http.HttpObjectAggregator to build the entire full http message. But the resulting message is still a stream based message which is readable once." On Fri, Sep 2, 2016 at 10:26 AM, S

Re: downloading large files in chunks

2016-09-02 Thread S Ahmed
Thanks. Just to be clear, I don't run the server where I am downloading the file. I want to download files that are very large, but stream them so they are not held in memory and then written to disk. I want to stream the download straight to a file and not hold the entire file in memory. Is

Re: Adding options to default idempotent File Repo

2016-09-02 Thread Quinn Stevenson
You are correct - to customize the configuration for an Idempotent Repository, you need to create a bean for the repository and set it’s specific properties. The Idempotent Repository used by the file component is pluggable - it just needs to implement the

Unmarshal fixed length Binary data

2016-09-02 Thread kaustubhkane
Hi, We have a fixed length Binary data. I am looking at the Bindy Data Format and found that it supports unmarshalling of Fixed Length records. I looked at the implementation/code of this Bindy Fixed Lenght records in the Camel Source code (BindyFixedLengthDataFormat.java and

Mongodb Persistent tail tracking: How to chose an increasing field for tracking

2016-09-02 Thread jpeschke
Hello, Perhaps this is more a MongoDB issue, but maybe somebody has an idea: The Camel MongoDB endpoint supports the persistent tail tracking feature (which means that it stores the value of an arbitrary increasing field in a document to reset the tailable cursor to this document when