On Apr 20, 9:18 am, "Anush J." <li...@ruby-forum.com> wrote:
> Hello guys
> After figuring out a way to load my db with info scraped of from a web
> page, the data on db is appended with "--- \n -" these characters. I'm
> not sure what the problem is.. I did change my editor to xcode.
>
> I have attached my app's
> - migration file
> - the method that is called to populate the database
> - the output on rails server
>
> You can see I'm specifically printing out the values like (eg:
> [item[0]]) before assigning it to dts.bug_id or any other db row for
> that matter. I can see the printed value is fine but after storing on
> the database they are being appended with those junk characters.
>
> Have anyone of you noticed this before. Am I doing something
> wrong..please help !!
You've got a bunch of string columns but you're assigning arrays to
them (such as [item[0]]), which forces activerecord to encode them as
yaml which is what causes the extra characters you are seeing - that
how yaml encodes arrays. If you did want to do this you should use the
serialize directive to inform activerecord that those columns should
be serialized/deserialized. If not, then just assign item[0] rather
than [item[0]]
Fred
Fred
> Thanks in advance -- Anush
>
> class CreateBugs < ActiveRecord::Migration
> def self.up
> create_table :bugs do |t|
> t.string :bug_id
> t.string :bug_pri
> t.string :bug_sev
> t.string :bug_state
> t.string :bug_date
> t.string :bug_org
> t.string :bug_asg
> t.string :bug_sum
> t.string :bug_fcomment
> t.string :bug_fdate
> t.timestamps
> end
> end
>
> def self.down
> drop_table :bugs
> end
> end
>
> def scrap_dts
> puts "Inside def scrap_dts in import controller"
> agent = Mechanize.new
> agent.basic_auth('anushj', 'porfavor')
> page =
> agent.get('http://hidden/query.cgi?action=search&name=anushj&origin=me&priority=...)
> table = agent.page.search('//*[(@id =
> "subreport_list")]//td').map(&:text).map(&:strip)
>
> table = table.split('Bug')
> table.delete_if{|x| x.size == 0}
>
> table.each do |item|
> tmp = [item[0]]
> tmp = tmp.split('-')
> if Bug.find_by_bug_id(tmp[0]).nil?
> dts = Bug.new
> dts.bug_id = tmp[0]
> puts tmp[0]
> dts.bug_pri = [item[2]]
> puts [item[2]]
> dts.bug_sev = [item[4]]
> puts [item[4]]
> dts.bug_state = [item[5]]
> puts [item[5]]
> dts.bug_date = [item[6]]
> puts [item[6]]
> dts.bug_org = [item[9]]
> puts [item[9]]
> dts.bug_asg = [item[10]]
> puts [item[10]]
> dts.save
> puts "save record!"
> #dts_num = [item[0]]
> #puts dts_num
> end
> end
>
> end
>
> DB O/P while executing scrap_dts
>
> Inside def scrap_dts in import controller
> 106083-M30S
> P1
> S1
> New
> 2011/02/16
> anushj
> mmourier
> save record!
> 115412-M30S
> P1
> S1
> Closed: Verified
> 2011/04/05
> anushj
> anushj
> save record!
> 115762-M30S
> P1
> S1
> Closed: Verified
> 2011/04/11
> anushj
> anushj
> save record!
>
> Started GET "/" for 127.0.0.1 at 2011-04-19 23:37:33 -0700
> Processing by ImportController#index as HTML
> Bug Load (0.3ms) SELECT "bugs".* FROM "bugs" WHERE "bugs"."bug_id" IN
> ('106083-M30S') LIMIT 1
> Bug Load (0.2ms) SELECT "bugs"."id" FROM "bugs" WHERE
> ("bugs"."bug_id" = '["106083-M30S"]') LIMIT 1
> AREL (0.4ms) INSERT INTO "bugs" ("bug_id", "bug_pri", "bug_sev",
> "bug_state", "bug_date", "bug_org", "bug_asg", "bug_sum",
> "bug_fcomment", "bug_fdate", "created_at", "updated_at") VALUES ('---
> - 106083-M30S
> ', '---
> - P1
> ', '---
> - S1
> ', '---
> - New
> ', '---
> - 2011/02/16
> ', '---
> - anushj
> ', '---
> - mmourier
> ', NULL, NULL, NULL, '2011-04-20 06:37:35.112929', '2011-04-20
> 06:37:35.112929')
> Bug Load (0.3ms) SELECT "bugs".* FROM "bugs" WHERE "bugs"."bug_id" IN
> ('115412-M30S') LIMIT 1
> Bug Load (0.1ms) SELECT "bugs"."id" FROM "bugs" WHERE
> ("bugs"."bug_id" = '["115412-M30S"]') LIMIT 1
> AREL (0.3ms) INSERT INTO "bugs" ("bug_id", "bug_pri", "bug_sev",
> "bug_state", "bug_date", "bug_org", "bug_asg", "bug_sum",
> "bug_fcomment", "bug_fdate", "created_at", "updated_at") VALUES ('---
> - 115412-M30S
> ', '---
> - P1
> ', '---
> - S1
> ', '---
> - "Closed: Verified"
> ', '---
> - 2011/04/05
> ', '---
> - anushj
> ', '---
> - anushj
> ', NULL, NULL, NULL, '2011-04-20 06:37:35.124746', '2011-04-20
> 06:37:35.124746')
> Bug Load (0.2ms) SELECT "bugs".* FROM "bugs" WHERE "bugs"."bug_id" IN
> ('115762-M30S') LIMIT 1
> Bug Load (0.1ms) SELECT "bugs"."id" FROM "bugs" WHERE
> ("bugs"."bug_id" = '["115762-M30S"]') LIMIT 1
> AREL (0.4ms) INSERT INTO "bugs" ("bug_id", "bug_pri", "bug_sev",
> "bug_state", "bug_date", "bug_org", "bug_asg", "bug_sum",
> "bug_fcomment", "bug_fdate", "created_at", "updated_at") VALUES ('---
> - 115762-M30S
> ', '---
> - P1
> ', '---
> - S1
> ', '---
> - "Closed: Verified"
> ', '---
> - 2011/04/11
> ', '---
> - anushj
> ', '---
> - anushj
> ', NULL, NULL, NULL, '2011-04-20 06:37:35.129591', '2011-04-20
> 06:37:35.129591')
> Bug Load (3.9ms) SELECT "bugs".* FROM "bugs"
> SQL (0.2ms) SELECT COUNT(*) FROM "bugs"
> Rendered import/index.html.erb within layouts/application (46.3ms)
> Completed 200 OK in 1769ms (Views: 51.0ms | ActiveRecord: 6.2ms)
>
> --
> Posted viahttp://www.ruby-forum.com/.
--
You received this message because you are subscribed to the Google Groups "Ruby
on Rails: Talk" group.
To post to this group, send email to rubyonrails-talk@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.
