Well, in terms of good Rails coding practice there's no need for most
of the code you've written so far to be in ApplicationController or a
Controller at all. It looks like functionality that should live in
app/models or in classes and modules in lib/. But that's not what's
causing your problem.
Without reading thru all of your code the obvious difference between
your local environment and Heroku is that you're running in
development mode locally. This generally means that all of your
ApplicationController code is reloaded on every request. I'm betting
that that's the origin of the issue you're seeing.
Gabriel
On Jan 3, 2:31 am, Michael Baldock <michaelb1...@googlemail.com>
wrote:
> Dear All,
>
> Fairly new to rails and Heroku, so could be doing something wacky - do
> let me know if you think my code practice is off, even if unrelated to
> this error, I'd like to learn!
>
> I'm using Rails 3.0.0, ruby 1.8.7, and 'sqlite3-ruby',
> '1.2.5', :require => 'sqlite3'.
>
> I've got an application that goes off and scrapes websites for event
> information, to do this I'm using:
>
> nokogiri - for scraping
> date - to help with parsing
> chronic - to help recognise dates
> open-uri - to open the url
>
> all of these are just gem 'gem-name', so latest unspecified versions.
>
> The code that does this scraping is in the ApplicationController class
> - a bad place for it?
> I've just got it working as a background process using delayed_job, I
> have a class called EventSearcher with the required 'perform' method.
>
> My error occurs when I upload to Heroku, if I make changes to the app,
> but NOT to ApplicationController,
> do $git add . / git commit .... / git push heroku master / heroku
> workers 1 / heroku open
> then the process runs, appears successful in the logs, but returns no
> events
>
> The error can be solved I've discovered by making small changes to
> Application controller, such as adding the line
> logger.debug"Changing AppController again"
> then doing $git add . / git commit .... / git push heroku master /
> heroku workers 1 / heroku open
> this time the process runs, appears in logs to take longer, and
> returns the scraped info as required.
>
> Any ideas why this should be??
>
> I have tried solving this myself by putting in 'logger.debug"
> commands, but these don't show on my heroku logs, any idea how to get
> them to show up?
>
> I've put the code below, for ApplicationController, EventSearcher, and
> part of SuggestedEvents it's a bit of a mess I know, but runs fine on
> local machine.
>
> -------------- Application Controller
>
> class ApplicationController < ActionController::Base
>
> require 'date'
> require 'nokogiri'
> require 'open-uri'
> require 'chronic'
>
> protect_from_forgery
> include SessionsHelper
>
> def authenticate
> deny_access unless signed_in?
> end
>
> def find_all_events
>
> @event_searches = EventSearch.all
>
> @event_searches.each do |es|
>
> do_search(es)
>
> end
> end
>
> def find_one_band_events(event_id)
>
> es = EventSearch.where(:id => event_id)[0]
> do_search(es)
> end
>
> def do_search(event_search)
>
> logger.debug"Changing AppController again"
> logger.debug"in do search"
> url = event_search.urlOne
>
> if validate(url)
>
> logger.debug"Passed Validate URL"
>
> doc = Nokogiri::HTML(open(url))
>
> #search the doc with the saved css search strings
> if !(event_search.eventDateCSS.empty?)
> startDate = doc.css(event_search.eventDateCSS) else eventDate
> = 0
> end
> if !(event_search.eventNameCSS.empty?)
> eventName = doc.css(event_search.eventNameCSS) else eventName
> = 0
> end
> if !(event_search.eventLocationCSS.empty?)
> location = doc.css(event_search.eventLocationCSS) else
> location = 0
> end
> if !(event_search.eventTimeCSS.empty?)
> time = doc.css(event_search.eventTimeCSS) else time = 0 end
> if !(event_search.priceCSS.empty?)
> price = doc.css(event_search.priceCSS) else price = 0 end
> if !(event_search.descriptionCSS.empty?)
> description = doc.css(event_search.descriptionCSS) else
> description
> = 0 end
> i=0
>
> if eventDate != 0
> while i < startDate.length # find all events on that page going
> through each case of start date
>
> SuggestedEvent.new do |@savedEvent|
>
> if eventDate != 0
> @savedEvent.eventDate =
> date_from_string(startDate[i].content)
> logger.debug(startDate[i].content)
> end
>
> test = SuggestedEvent.where(:bandName =>
> event_search.bandName,
>
> :eventDate => @savedEvent.eventDate)
>
> if !test[0]
>
> @savedEvent.bandName = event_search.bandName
> if eventName != 0
> @savedEvent.eventName =
> remove_whitespace(eventName[i].content)
> end
> if time != 0
> @savedEvent.doorsOpen =
> Chronic.parse(time[i].content) end
> if price != 0
> @savedEvent.price = price[i].content end
> if description !=0
> @savedEvent.description =
> remove_whitespace(description[i].content) end
>
> if location != 0
> savedVenueName =
> paramMatcher('Venue','venueName',
> remove_whitespace(location[i].content))
> if savedVenueName
> @savedEvent.venue = savedVenueName[0]
> @savedEvent.city_location =
> savedVenueName[1]
> @savedEvent.longitude =
> savedVenueName[2]
> @savedEvent.latitude =
> savedVenueName[3]
> else
> @newVenue = Venue.new
> @newVenue.venueName =
> remove_whitespace(location[i].content)
> @newVenue.old_venue_name =
> @newVenue.venueName
> @newVenue.save
> @savedEvent.venue =
> remove_whitespace(location[i].content)
> end #of if
> else
> @savedEvent.venue = 'No saved venue
> found'
> end #of location !=
> @savedEvent.save
> end # of if !test
> end # of .new do
> i += 1
> end # of while
>
> end #of if eventDate !=0
>
> end # of if url passes
>
> end # of def
>
> # get the date from whatever string gets thrown
> def date_from_string(date)
>
> # should remove st rd nd and th
>
> firstParse = Chronic.parse(date)
> r1 = /[a-zA-Z]/
>
> #daY Less than 12, assume chronic wrong for Britain, as long as also
> no characters such as December,
> #where it would be right
> if firstParse.day <= 12 and !r1.match(date)
>
> #swap month with day
> firstParse = firstParse.change(:day => firstParse.month, :month =>
> firstParse.day)
>
> end #of if
>
> return firstParse
>
> end #of def
>
> # remove whitespace and other characters such as newline
>
> def remove_whitespace(dirty_name)
>
> return dirty_name.split(' ').join(" ")
>
> end
>
> #find names in strings from the net
> def paramMatcher(modelString, param, cssScrapedString)
>
> case modelString
> when 'Venue'
> venues = Venue.all
> i=0
> venues.each do |v|
>
> if v.venueName.scan(cssScrapedString)[0] or
> cssScrapedString.scan(v.venueName)[0]
>
> return [v.venueName, v.city_location, v.longitude,
> v.latitude]
> end
>
> end # of .each
> end # of case
>
> return nil
> end # of def param Matcher
>
> def validate(url)
> begin
> uri = URI.parse(url)
> if uri.class != URI::HTTP
> return false
> end
> rescue URI::InvalidURIError
> return false
> end
> return true
> end
>
> end # of class
>
> ----------------- Event Searcher
>
> class EventSearcher < ApplicationController
>
> attr_accessor :all_events
> attr_accessor :one_event_id
>
> def perform
>
> logger.debug"in perform"
>
> if all_events == 1 # searches all band names
>
> logger.debug"in perform, all_events == 1"
>
> find_all_events
>
> end
>
> if one_event_id != nil #searches just one band name using id
>
> logger.debug"in perform, one_event != nil"
> logger.debug(one_event_id)
> find_one_band_events(one_event_id)
>
> end
>
> end # Of Perform
>
> end #of class
>
> -------------- Suggested Events
>
> class SuggestedEventsController < ApplicationController
>
> #.... other functions removed for brevity....
>
> def search_all_events
>
> logger.debug"Sugg E search all events"
> #the instance of the object that gets put on the queue
> @event_searcher = EventSearcher.new
> @event_searcher.one_event_id = nil
> @event_searcher.all_events = 1
> Delayed::Job.enqueue @event_searcher
>
> flash[:notice] = "Finding All events, this takes a while, refresh
> after 10 mins"
> redirect_to suggested_events_path
>
> end
>
> end
--
You received this message because you are subscribed to the Google Groups
"Heroku" group.
To post to this group, send email to her...@googlegroups.com.
To unsubscribe from this group, send email to
heroku+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/heroku?hl=en.
