Nice, maybe Americans (hey wait, you're a Canadian!) need a bit more ambient awareness of our tax-paid death toll. Could you make a version that digests a once a day report though? I'd like to see yesterday's total and a running total once per day via device updat
On Wed, Sep 10, 2008 at 9:09 AM, Anselm Hook <[EMAIL PROTECTED]> wrote: > Paige and I threw this together yesterday morning when we had a free hour - > it attempts to twitter new civilian casualties in iraq. > > Perhaps folks can suggest ways to improve it - it is kind of brute force - > mapping the deaths could also be done - but I'm assuming third party > services already do that and the location is published with the twitter > albeit using David Troy's notation. > > - me > > ---------- Forwarded message ---------- > From: paige saez <[EMAIL PROTECTED]> > Date: Tue, Sep 9, 2008 at 1:33 PM > Subject: war tweets- > To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> > > > http://twitter.com/IRAQDEATHS > > > #!/usr/local/bin/ruby > > # > # we want to fetch posts from the iraq death count and show them > # and we want to only check every day or so ( an external cron job ) > # basically then we take a look at the most recent 50 only... > # and we twitter up to 50 of them basically... > # we reverse the twitter order so that the most recent ones are first > # > > twittercap = 50 # twitter this many posts max > > # > # 1) fetch the data > # > # http://www.iraqbodycount.org/database/download/ibc-individuals > > require 'net/http' > > url = "http://www.iraqbodycount.org/database/download/ibc-individuals"; > > data = Net::HTTP.get_response(URI.parse(url)).body > > > > # > # 2) chew on the data a bit - masticate it good > # > # our first goal is to read the rather dorky csv file iraq war deaths > provides > # we have to skip past the header crap... so lets make a copy of the data > without that header > # we will just loop through all the data and pull out the good stuff and > store it into another array > # clearly not the "perfectly clean" way to do it but good enough and we > don't really care > # > # http://www.rubytips.org/2008/01/06/csv-processing-in-ruby/ > # http://fastercsv.rubyforge.org/classes/FasterCSV.html > > require 'rubygems' > require "fastercsv" > @results = FasterCSV.parse(data) # unused approach -> > read("ibc-individuals") > @deaths = [] > inside_stupid_header = true > @results.each do |death| > if death[0] == "IBC code" > inside_stupid_header = false > elsif inside_stupid_header == false > @deaths << death > end > end > > # > # 3) put the data in our storage area... keep a copy in our belly > # > # http://sqlite-ruby.rubyforge.org/sqlite3/faq.html > # > # our second goal is to store this in a structured way so we can process it > # i will bother to keep it in a database although i could just hold it > memory > # we are going to want to avoid storing duplicates - because we will call > this multiple times > # > # this gets slightly more involved > # > # 3a) let us make a table to store the data - if it already exists this > code will crash but thats ok > # we wrap the whole thing in a begin / rescue and just ignore the crash if > it happens > # > # The format seems to be here in the ibc file : "IBC code","Name or > Identifying Details","Age","Sex","Marital Status","Parental > Status","Earliest Date","Latest Date","Location" > # > # 3b) um, store everything... > > > =begin > require 'rubygems' > require 'sqlite' > # this was a manual database approach - tedious! > db = SQLite::Database.new( "endiraqwar.db" ) > begin > result = db.execute("CREATE TABLE deaths(id INTEGER, code VARCHAR(80), > name VARCHAR(255), age INTEGER, sex VARCHAR(32), marital VARCHAR(64), > parental VARCHAR(64), earliest DATE, latest DATE, location VARCHAR(255))"); > rescue > end > @deaths.each do |death| > result = db.execute("INSERT INTO ") > end > =end > > # > # 3) ok scratch the above, lets use this: > # > # http://datamapper.org/why.html > # > # lets manage the data in a way that takes advantage of object oriented > design > # we will grab datamapper which lets us pretend our data is a ruby data > object > # (instead of treating it like radioactive waste and holding it at arms > length) > # that means we have to define what our data "is" for ruby... > # also we will hop over to postgres as our back end datastore - away from > sqlite ... just because > # > > require 'rubygems' > require 'dm-core' > > DataMapper.setup(:default, { > :adapter => 'postgres', > :database => "endiraqwar", > :username => 'endiraqwar', > :password => '', > :host => 'localhost' > }) > > class Death > include DataMapper::Resource > property :id, Integer, :serial => true > property :code, String > property :name, Text > property :age, Text > property :sex, Text > property :marital, Text > property :parental, String > property :earliest, DateTime > property :latest, DateTime > property :location, Text > property :created_at, DateTime > property :posted, DateTime, :default => nil > end > > # we do not want to do this now > # because it would erase our database > # DataMapper.auto_migrate! > > # go ahead and store all the deaths > @deaths.each do |death| > if Death.first(:code => death[0] ) > # puts "We already found this death #{death[1]} #{death[0]} so not > saving" > next > end > # take a second to convert the date phrase into a machine date > death[6] = DateTime.parse(death[6]) > death[7] = DateTime.parse(death[7]) > # go ahead and make a blobby thing to hold all of this > record = Death.new( > :code => death[0], > :name => death[1], > :age => death[2], > :sex => death[3], > :marital => death[4], > :parental => death[5], > :earliest => death[6], > :latest => death[7], > :location => death[8] > ) > puts "recording the passing of #{record.name} at #{record.earliest} and > #{record.code}" > record.save > end > > # > # 4) ok cool, now we have the data - lets twitter! > # > # http://twitter.rubyforge.org/ > # http://github.com/jnunemaker/twitter/tree/master/examples/posting.rb > # > > require 'twitter' > twitter = Twitter::Base.new("endiraqwar","") > @deaths = Death.all(:order => [:earliest.desc], :limit => twittercap) > @copyofdeaths = [] > @deaths.each do |death| > @copyofdeaths << death > end > > @copyofdeaths.reverse.each do |death| > # publish deaths that are new > next if death.posted != nil > result = twitter.post("#{death.name}, #{death.age}, #{death.sex}, > #{death.marital}, #{death.parental} killed on #{death.earliest.strftime("%d > %b %Y")} at L:#{death.location}") > # remember that we already published this death > death.posted = DateTime.now > death.save > puts "posted the death of #{death.name} #{death.code}" > end > > > > -- > Paige Saez > > > 971.227.4384 > Art.Media.Love. > > paigesaez.org > makerlab.com > red76.com > > > > -- > anselm 415 215 4856 http://hook.org http://makerlab.com http://meedan.net > > _______________________________________________ > Geowanking mailing list > [email protected] > http://lists.burri.to/mailman/listinfo/geowanking > -- Dav Yaginuma http://AkuAku.org/
_______________________________________________ Geowanking mailing list [email protected] http://lists.burri.to/mailman/listinfo/geowanking
