Dug into this over the weekend and was able to solve using "escape_string":
from MySQLdb import escape_string
...
"""INSERT INTO failover ( url, status, spidername, referrer, body1,
body1hash )
VALUES ( '{0}','{1}','{2}','{3}','{4}','{5}' )"""
.format( item['url'], item['status'], spider.name, item['referrer'],*
escape_string(item['body'])*, item['bodyhash'] ))
Jim
On Friday, November 13, 2015 at 4:20:27 PM UTC-5, Jim Priest wrote:
>
> I'm trying to dump response.body into a mysql table...
>
> My items:
> class FailoverItem(scrapy.Item):
> url = scrapy.Field()
> body = scrapy.Field()
> bodyhash = scrapy.Field()
> status = scrapy.Field()
> referrer = scrapy.Field()
>
> This works:
> """INSERT INTO failover ( url, status, spidername, referrer, body1,
> body1hash )
> VALUES ( '{0}','{1}','{2}','{3}','{4}','{5}' )"""
> .format( item['url'], item['status'], spider.name,
> item['referrer'], 'fudge', item['bodyhash'] ))
>
> This fails:
> """INSERT INTO failover ( url, status, spidername, referrer, body1,
> body1hash )
> VALUES ( '{0}','{1}','{2}','{3}','{4}','{5}' )"""
> .format( item['url'], item['status'], spider.name,
> item['referrer'], item['body'], item['bodyhash'] ))
>
> Error 1064: You have an error in your SQL syntax; check the manual that
> corresponds to your MariaDB server version for the right syntax to use near
> '
> http://fonts.googleapis.com/css?family=Londrina+Outline|Roboto+Condensed:300|Rob
>
> <http://fonts.googleapis.com/css?family=Londrina+Outline%7CRoboto+Condensed:300%7CRob>'
>
> at line 2
>
> It seems like its failing with an unescaped single quote but I've googled
> and tried every which way to escape that with no luck.
>
> I think this is more of a Sql/Python issue vs. Scrapy but searching the
> list I couldn't find anything similar.
>
> Jim
>
>
>
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
