On three URLs? Surely not...
mark
On Wed, Jul 28, 2010 at 10:24 AM, Julio Cesar Ody <julio...@gmail.com> wrote:
> I can't really tell from that stacktrace if the problem is this, but
> you may be hitting the maximum number of open files (and sockets).
>
> Are you on OSX? If so, you can expand that limit with sysctl. Try:
>
> sudo sysctl -w kern.maxfiles=36000
>
> Sets to a bit over twice as much the default. Also, check
> kern.maxfilesperproc. And ulimit.
>
>
>
>
> On Wed, Jul 28, 2010 at 10:15 AM, Robbie Shepherd
> <robbie.sheph...@gmail.com> wrote:
>>
>> Does anyone know if there's a maximum number of open connections allowed
>> when using hpricot/net::http?
>>
>> I have a bunch of xml feeds I need to read via hpricot and write to disk.
>> Whenever I hit it too hard/fast, or when I try to open several files in
>> hpricot in a loop, my app dies.
>>
>> If I do something like...
>>
>> arr = [url1, url2, url2]
>> arr.each do|d|
>> doc = Hpricot(open(d))
>> doc.search(blah...)
>> File.open("#{RAILS_ROOT}/xml/#{file_name}.xml", 'w') {|f| f.write doc }
>> end
>>
>> I get:
>>
>> OpenURI::HTTPError: 500 Internal Server Error from
>> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/open-uri.rb:277:in
>> `open_http'
>> from
>> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/open-uri.rb:616:in
>> `buffer_open'
>> from
>> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/open-uri.rb:164:in
>> `open_loop'
>> from
>> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/open-uri.rb:162:in
>> `catch'
>> from
>> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/open-uri.rb:162:in
>> `open_loop'
>> from
>> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/open-uri.rb:132:in
>> `open_uri'
>> from
>> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/open-uri.rb:518:in
>> `open'
>> from
>> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/open-uri.rb:30:in
>> `open'
>> from (irb):11
>> from (irb):11:in `each'
>> from (irb):11
>>
>> Am I missing something here, or is there a better way of doing this? I'm on
>> Ruby 1.8.7 and hpricot 0.8.2
>>
>> Thanks
>> Robbie
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Ruby or Rails Oceania" group.
>> To post to this group, send email to rails-ocea...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> rails-oceania+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/rails-oceania?hl=en.
>
>
>
> --
> http://crazyhollywood.org
>
> --
> You received this message because you are subscribed to the Google Groups
> "Ruby or Rails Oceania" group.
> To post to this group, send email to rails-ocea...@googlegroups.com.
> To unsubscribe from this group, send email to
> rails-oceania+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/rails-oceania?hl=en.
>
>
--
A UNIX signature isn't a return address, it's the ASCII equivalent of a
black velvet clown painting. It's a rectangle of carets surrounding a
quote from a literary giant of weeniedom like Heinlein or Dr. Who.
-- Chris Maeda
--
You received this message because you are subscribed to the Google Groups "Ruby
or Rails Oceania" group.
To post to this group, send email to rails-ocea...@googlegroups.com.
To unsubscribe from this group, send email to
rails-oceania+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rails-oceania?hl=en.
