Hey, I'd take a look at these options if you want to stick with Django-Storages ( http://django-storages.readthedocs.org/en/latest/index.html listed on the top of the page).
Otherwise, I wonder if you could use the hadoop fileystem or some other, similar distributed file system built for high performance computing? I agree though with your problems using NFS -- I've had a ton of trouble with it in the past. It's not pleasant to work with at all. On Tue, May 8, 2012 at 7:51 AM, Phang Mulianto <braveh...@gmail.com> wrote: > Hi, > > i also have the script to sftp using python...run with cron every 30 > mins, and suddently since last week, the eol errors come out.. > > i think this problem bcoz the sftp server.. > > are you trying rsync to transfer between host ? > > > On Tue, May 8, 2012 at 6:31 AM, Tino de Bruijn <tin...@gmail.com> wrote: > >> Hi, >> >> Unfortunatly we can't move to Amazon S3. So what other file system or >> transfer method would you recommend? We had an NFS in place, but getting it >> to play nice with permissions was quite a hassle. >> >> How do other people do this? >> >> Tino >> >> >> On Wednesday, May 2, 2012 1:25:52 AM UTC+2, Kurtis wrote: >> >>> I messed with the SFTP implementatoin a while back. I noticed it was >>> pretty tough and, if I'm not mistaken, a bit out-dated. We ended up moving >>> to Amazon S3 so it wasn't a big deal. >>> >>> Anyways, maybe you could try using another file system? SSH isn't really >>> an ideal solution for this sort of a problem -- even though it works as a >>> great starting point. >>> >>> I'm not sure if that helps much but good luck to you! >>> >>> On Tue, May 1, 2012 at 11:49 AM, Tino de Bruijn <tinodb> wrote: >>> >>>> Hi, >>>> >>>> My setup is the following: I have two webservers that serve all dynamic >>>> request. I have a seperate static file server (that also has some celery >>>> queue workers running). When images are uploaded by users, those images are >>>> uploaded to the webservers, and processed async by a queue. In the job they >>>> are transfered to the static file server and resized. For the admin >>>> however, I want the transfering and resizing done right then, so that when >>>> admins post content, it is immediatly visible. Admins post different >>>> content than users, so this is implemented by using a different field on >>>> the models. The admin images are transfered to the static file server >>>> directly by using sftpstorage from django-storages. >>>> >>>> The problem is that paramiko (used by django-storages for the sftp >>>> implementation) starts to throw EOFErrors. This doesn't always happen. When >>>> the mod_wsgi instance is fresh, everything works, but after a while these >>>> errors start occurring. Touching the wsgi.py file makes everything work >>>> again. >>>> >>>> I have build a retry loop (5 times) to try to circumvent network >>>> glitches, but it consistently fails. If it needs to retry it'll keep >>>> failing. >>>> >>>> I have two questions: >>>> >>>> 1) Does anybody have an idea why this could be happening? >>>> 2) Is this the best way to implement this strategy, or would you >>>> suggest a different technique? >>>> >>>> Thanks, >>>> >>>> >>>> Tino >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Django users" group. >>>> To view this discussion on the web visit https://groups.google.com/d/** >>>> msg/django-users/-/UXX_**dJrQJOwJ<https://groups.google.com/d/msg/django-users/-/UXX_dJrQJOwJ> >>>> . >>>> To post to this group, send email to django-users@googlegroups.com. >>>> To unsubscribe from this group, send email to django-users+unsubscribe@ >>>> **googlegroups.com <django-users%2bunsubscr...@googlegroups.com>. >>>> For more options, visit this group at http://groups.google.com/** >>>> group/django-users?hl=en<http://groups.google.com/group/django-users?hl=en> >>>> . >>>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To view this discussion on the web visit >> https://groups.google.com/d/msg/django-users/-/20YX_C3Oo3AJ. >> >> To post to this group, send email to django-users@googlegroups.com. >> To unsubscribe from this group, send email to >> django-users+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/django-users?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To post to this group, send email to django-users@googlegroups.com. > To unsubscribe from this group, send email to > django-users+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-users?hl=en. > -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
