Andrew Lentvorski wrote:
S3 has good bandwidth but sucky latency. My guess is that those systems all try to write synchronously. They probably get clobbered by the round trip delays.
You keep going on about latency but I don't really understand why. You are just uploading a file. Where does latency get involved? Are you saying they take too long to send a tcp ack for each of your sent packets or something? You are probably correct about writing synchronously. They are really anal about not losing data. I am not aware of anyone trying to use S3 as their mail queue or anything like that where latency would be an issue. I only do puts and very rarely gets. Other people host their large files and website images off of S3 in which case they put each file once and then it has a lot of gets. This is primarily what it is intended for. It is not really to be a transactional object store or database which needs low latency.
You need something that will put a lot of transactions on the wire without waiting for the previous ones to complete.
We are doing backup and mass storage. Not anything transactional. I never have more than one transaction to put on the wire and it takes anywhere from minutes to hours to complete depending on the upstream bandwidth I have.
-- [email protected] http://www.kernel-panic.org/cgi-bin/mailman/listinfo/kplug-list
