EuroPython 2016: Last day to get tickets at regular rate
We will be switching to the on-desk rates for tickets tomorrow, so today is your last chance to get tickets at the regular rate, which is about 30% less than the on-desk rate: EuroPython 2016 Registration *** https://ep2016.europython.eu/registration/ *** Day Passes -- As in the past, we will also sell day passes at the conference venue. To make things more affordable especially for students and other people who want to attend the Beginners’ Day or the sprints, we have split the day pass prices into ones valid from Monday-Friday for the main conference days and ones for the weekend days. Day passes for the first Sunday (Beginners’ Day) and the sprints weekend (valid for the day when they are purchased): * Student weekend day pass: EUR 25.00 * Personal weekend day pass: EUR 70.00 * Business weekend day pass: EUR 110.00 Day passes for the main conference (valid for the day when they are purchased): * Student conference day pass: EUR 50.00 * Personal conference day pass: EUR 140.00 * Business conference day pass: EUR 225.00 All prices include 10% Spanish VAT. Please see the registration page for full details of what is included in the ticket price. With gravitational regards, -- EuroPython 2016 Team http://ep2016.europython.eu/ http://www.europython-society.org/ PS: Please forward or retweet to help us reach all interested parties: https://twitter.com/europython/status/750955219975016448 Thanks. -- https://mail.python.org/mailman/listinfo/python-list
Re: Spot the bug: getoptquestion.py
In article , Chris Angelico wrote: >On Wed, Jul 6, 2016 at 2:04 PM, Lawrence DâOliveiro > wrote: >> On Tuesday, July 5, 2016 at 1:42:42 AM UTC+12, Chris Angelico wrote: >> >>> The getopt module is designed to match the C getopt function, which I've >>> never used; for my command-line parsing, I use argparse instead >>> (usually via some wrapper that cuts down the duplication, like clize). >> >> getopt seems so much simpler. > >Look at clize: Okay: | >>> import clize | Traceback (most recent call last): | File "", line 1, in | ImportError: No module named clize Mmm... nope. I'm not going to learn a new tool and introduce an extra dependency just to do something as basic as getopt. But then again, coming from a C background, getopt feels kind of familiar. ;-) Thanks all for the input. I think it all boils down to: "If you don't want a space in your long_option, don't put a space in there". >I just put docstrings on my functions, slap "@command" above them, and >with minimal boilerplate, I have a fully-working command line >interface. It's a wrapper around argparse. Looks neat though! -- [J|O|R] <- .signature.gz -- https://mail.python.org/mailman/listinfo/python-list
Re: Spot the bug: getoptquestion.py
On Thu, Jul 7, 2016 at 9:30 PM, Oscar wrote: > Thanks all for the input. I think it all boils down to: "If you don't > want a space in your long_option, don't put a space in there". Yeah, I guess, pretty much! > Mmm... nope. I'm not going to learn a new tool and introduce an extra > dependency just to do something as basic as getopt. But then again, > coming from a C background, getopt feels kind of familiar. ;-) >>I just put docstrings on my functions, slap "@command" above them, and >>with minimal boilerplate, I have a fully-working command line >>interface. It's a wrapper around argparse. > > Looks neat though! Yes, it's a third-party dependency. (Sorry, should have mentioned that.) You're welcome to consider that to be too much risk and/or hassle to be worth improving on getopt, but personally, I *really* like the simplicity of just writing docstrings that still read perfectly well as docstrings, and having them create my argparse configs for me. Different strokes for different horses, or something like that. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Spot the bug: getoptquestion.py
In article , Chris Angelico wrote: >Yes, it's a third-party dependency. (Sorry, should have mentioned >that.) You're welcome to consider that to be too much risk and/or >hassle to be worth improving on getopt, but personally, I *really* >like the simplicity of just writing docstrings that still read >perfectly well as docstrings, and having them create my argparse >configs for me. Different strokes for different horses, or something >like that. Well, parsing the arguments was not really the problem in my case. But after parsing I had to look at valid combinations and show helpfull messages if required information is missing or conflicting options were given. I doubt if an extra abstraction layer would have helped me. It did make my typo a bit hard to catch, though... -- [J|O|R] <- .signature.gz -- https://mail.python.org/mailman/listinfo/python-list
sqlite3 scrapy
Hi,
I have a little scrapy-script: scrapy gets the links from a webpage; this works
fine. Then I want to write these links in a sqlite3-table. There is no
error-note. But in the database are not any records. What is the problem here?
My code:
# -*- coding: utf-8 -*-
2
3 import scrapy
4 import sqlite3
5
6 class MySpider(scrapy.Spider):
7
8 name= "frisch2"
9 allowed_domains = ["frischblau.de"]
10 start_urls = ["http://www.frischblau.de/";,]
11
12 def parse(self, response):
13
14 for href in response.xpath("//ul/li/a/@href"):
15
16 url = response.urljoin(href.extract())
17 yield scrapy.Request(url, callback=self.parse_dir_contents)
18
19 def parse_dir_contents(self, response):
20
21 sql = u' '
22 conn = sqlite3.connect('frisch.db')
23 cur = conn.cursor()
24 counter = 3;
25
26 try:
27 cur.execute("SELECT SQLITE_VERSION()")
28 print "-> SQLite version: %s" % cur.fetchone()
29
30 item = response.xpath('//a/@href').extract()
31 for el in item:
32
33 #print("--> ", type(el))
34 sql = "INSERT INTO Frisch VALUES(" + unicode(counter) + ",
" + "'" + el.strip() + "');"
35
36 print (sql)
37 cur.execute(sql)
38 conn.commit
39 counter = int(counter)
40 counter += 1
41
42 except sqlite3.Error as e:
43 print "Error %s:" % e.args[0]
44 conn.close()
45
Thanks for help.
o-o
Thomas
--
https://mail.python.org/mailman/listinfo/python-list
Re: sqlite3 scrapy
On Fri, Jul 8, 2016 at 12:04 AM, wrote:
> 34 sql = "INSERT INTO Frisch VALUES(" + unicode(counter) +
> ", " + "'" + el.strip() + "');"
Don't ever do this. Instead, use parameterized queries:
cur.execute("INSERT INTO Frisch VALUES (?, ?)", (counter, el.strip()))
> 38 conn.commit
Do you know what this does?
> 39 counter = int(counter)
This is unnecessary; counter is already an integer.
Does anything actually call your code? If not, none of the above will
matter. But possibly there's an invocation that I'm not seeing.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: sqlite3 scrapy
Am Donnerstag, 7. Juli 2016 16:05:14 UTC+2 schrieb Thomas Kaufmann:
> Hi,
>
> I have a little scrapy-script: scrapy gets the links from a webpage; this
> works fine. Then I want to write these links in a sqlite3-table. There is no
> error-note. But in the database are not any records. What is the problem
> here? My code:
>
> # -*- coding: utf-8 -*-
> 2
> 3 import scrapy
> 4 import sqlite3
> 5
> 6 class MySpider(scrapy.Spider):
> 7
> 8 name= "frisch2"
> 9 allowed_domains = ["frischblau.de"]
> 10 start_urls = ["http://www.frischblau.de/";,]
> 11
> 12 def parse(self, response):
> 13
> 14 for href in response.xpath("//ul/li/a/@href"):
> 15
> 16 url = response.urljoin(href.extract())
> 17 yield scrapy.Request(url, callback=self.parse_dir_contents)
> 18
> 19 def parse_dir_contents(self, response):
> 20
> 21 sql = u' '
> 22 conn = sqlite3.connect('frisch.db')
> 23 cur = conn.cursor()
> 24 counter = 3;
> 25
> 26 try:
> 27 cur.execute("SELECT SQLITE_VERSION()")
> 28 print "-> SQLite version: %s" % cur.fetchone()
> 29
> 30 item = response.xpath('//a/@href').extract()
> 31 for el in item:
> 32
> 33 #print("--> ", type(el))
> 34 sql = "INSERT INTO Frisch VALUES(" + unicode(counter) +
> ", " + "'" + el.strip() + "');"
> 35
> 36 print (sql)
> 37 cur.execute(sql)
> 38 conn.commit
> 39 counter = int(counter)
> 40 counter += 1
> 41
> 42 except sqlite3.Error as e:
> 43 print "Error %s:" % e.args[0]
> 44 conn.close()
> 45
>
>
>
>
>
> Thanks for help.
>
> o-o
>
> Thomas
Hi Chris,
I changed my code in the way you suggested. There is no effect in the db. I
start this script on the commandline with:
tk@Hamlet:~/myscrapy/tutorial/tutorial/spiders$ scrapy crawl webbot
When I call the sql-statement in the db it works.
sqlite> INSERT INTO Frisch VALUES(4, '//:Üs.chi/hhh\n');
sqlite> SELECT * FROM Frisch;
1|Üschi/hhh
2|Üschi/hhh\n
3|//:Üs.chi/hhh\n
4|//:Üs.chi/hhh\n
--
https://mail.python.org/mailman/listinfo/python-list
Re: sqlite3 scrapy
On Fri, Jul 8, 2016 at 12:42 AM, Thomas Kaufmann wrote: > I changed my code in the way you suggested. There is no effect in the db. I > start this script on the commandline with: > > tk@Hamlet:~/myscrapy/tutorial/tutorial/spiders$ scrapy crawl webbot You have some print calls in your code. Are they happening? Do you see anything on your console? If not, you probably don't have a database issue, you have an invocation issue. If you do, please post the output along with the latest code - it might be revealing. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: sqlite3 scrapy
Thomas Kaufmann wrote:
> Am Donnerstag, 7. Juli 2016 16:05:14 UTC+2 schrieb Thomas Kaufmann:
>> Hi,
>>
>> I have a little scrapy-script: scrapy gets the links from a webpage; this
>> works fine. Then I want to write these links in a sqlite3-table. There is
>> no error-note. But in the database are not any records. What is the
>> problem here? My code:
>>
>> # -*- coding: utf-8 -*-
>> 2
>> 3 import scrapy
>> 4 import sqlite3
>> 5
>> 6 class MySpider(scrapy.Spider):
>> 7
>> 8 name= "frisch2"
>> 9 allowed_domains = ["frischblau.de"]
>> 10 start_urls = ["http://www.frischblau.de/";,]
>> 11
>> 12 def parse(self, response):
>> 13
>> 14 for href in response.xpath("//ul/li/a/@href"):
>> 15
>> 16 url = response.urljoin(href.extract())
>> 17 yield scrapy.Request(url,
>> callback=self.parse_dir_contents) 18
>> 19 def parse_dir_contents(self, response):
>> 20
>> 21 sql = u' '
>> 22 conn = sqlite3.connect('frisch.db')
>> 23 cur = conn.cursor()
>> 24 counter = 3;
>> 25
>> 26 try:
>> 27 cur.execute("SELECT SQLITE_VERSION()")
>> 28 print "-> SQLite version: %s" % cur.fetchone()
>> 29
>> 30 item = response.xpath('//a/@href').extract()
>> 31 for el in item:
>> 32
>> 33 #print("--> ", type(el))
>> 34 sql = "INSERT INTO Frisch VALUES(" + unicode(counter)
>> + ", " + "'" + el.strip() + "');" 35
>> 36 print (sql)
>> 37 cur.execute(sql)
>> 38 conn.commit
>> 39 counter = int(counter)
>> 40 counter += 1
>> 41
>> 42 except sqlite3.Error as e:
>> 43 print "Error %s:" % e.args[0]
>> 44 conn.close()
>> 45
>>
>>
>>
>>
>>
>> Thanks for help.
>>
>> o-o
>>
>> Thomas
>
> Hi Chris,
>
> I changed my code in the way you suggested.
> There is no effect in the db.
Did you replace the line
conn.commit
with
con.commit()
? If not, can you provide the updated code?
> I start this script on the commandline with:
>
> tk@Hamlet:~/myscrapy/tutorial/tutorial/spiders$ scrapy crawl webbot
>
> When I call the sql-statement in the db it works.
>
> sqlite> INSERT INTO Frisch VALUES(4, '//:Üs.chi/hhh\n');
>
> sqlite> SELECT * FROM Frisch;
> 1|Üschi/hhh
> 2|Üschi/hhh\n
> 3|//:Üs.chi/hhh\n
> 4|//:Üs.chi/hhh\n
>
>
>
>
>
>
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: sqlite3 scrapy
Peter Otten wrote: More errors than words :( >> There is no effect in the db. > > Did you replace the line > > conn.commit > > with > > con.commit() Of course the updated code should read conn.commit() > ? If not, can you provide the updated code? and that should have been "If yes, ...", i. e. you only need to provide the updated code if it does *not* work yet. -- https://mail.python.org/mailman/listinfo/python-list
Re: sqlite3 scrapy
Am Donnerstag, 7. Juli 2016 16:52:12 UTC+2 schrieb Chris Angelico: > On Fri, Jul 8, 2016 at 12:42 AM, Thomas Kaufmann wrote: > > I changed my code in the way you suggested. There is no effect in the db. I > > start this script on the commandline with: > > > > tk@Hamlet:~/myscrapy/tutorial/tutorial/spiders$ scrapy crawl webbot > > You have some print calls in your code. Are they happening? Do you see > anything on your console? If not, you probably don't have a database > issue, you have an invocation issue. If you do, please post the output > along with the latest code - it might be revealing. > > ChrisA Hi Chris, here is my table: sqlite> CREATE TABLE Frisch(id INTEGER PRIMARY KEY NOT NULL, line TEXT NOT NULL); And here is the output from my script on the console: 2016-07-07 16:59:13 [scrapy] DEBUG: Crawled (200) http://www.frischblau.de/> (referer: None) 2016-07-07 16:59:14 [scrapy] DEBUG: Crawled (200) http://www.frischblau.de/#top> (referer: http://www.frischblau.de/) -> SQLite version: 3.8.2 INSERT INTO Frisch VALUES (?, ?) (2, u'#top') INSERT INTO Frisch VALUES (?, ?) (3, u'mailto:i...@frischblau.de') INSERT INTO Frisch VALUES (?, ?) (4, u'http://www.zukunftscharta.de') INSERT INTO Frisch VALUES (?, ?) (5, u'https://www.microsoft.com/de-de/WindowsForBusiness/End-of-IE-support') INSERT INTO Frisch VALUES (?, ?) (6, u'https://contao.org/de/news/screenguide28-platz-1.html') INSERT INTO Frisch VALUES (?, ?) (7, u'http://www.ifmo.de') INSERT INTO Frisch VALUES (?, ?) (8, u'http://www.werksdesign.de') ... -- https://mail.python.org/mailman/listinfo/python-list
Re: sqlite3 scrapy
Am Donnerstag, 7. Juli 2016 16:56:48 UTC+2 schrieb Peter Otten:
> Thomas Kaufmann wrote:
>
> > Am Donnerstag, 7. Juli 2016 16:05:14 UTC+2 schrieb Thomas Kaufmann:
> >> Hi,
> >>
> >> I have a little scrapy-script: scrapy gets the links from a webpage; this
> >> works fine. Then I want to write these links in a sqlite3-table. There is
> >> no error-note. But in the database are not any records. What is the
> >> problem here? My code:
> >>
> >> # -*- coding: utf-8 -*-
> >> 2
> >> 3 import scrapy
> >> 4 import sqlite3
> >> 5
> >> 6 class MySpider(scrapy.Spider):
> >> 7
> >> 8 name= "frisch2"
> >> 9 allowed_domains = ["frischblau.de"]
> >> 10 start_urls = ["http://www.frischblau.de/";,]
> >> 11
> >> 12 def parse(self, response):
> >> 13
> >> 14 for href in response.xpath("//ul/li/a/@href"):
> >> 15
> >> 16 url = response.urljoin(href.extract())
> >> 17 yield scrapy.Request(url,
> >> callback=self.parse_dir_contents) 18
> >> 19 def parse_dir_contents(self, response):
> >> 20
> >> 21 sql = u' '
> >> 22 conn = sqlite3.connect('frisch.db')
> >> 23 cur = conn.cursor()
> >> 24 counter = 3;
> >> 25
> >> 26 try:
> >> 27 cur.execute("SELECT SQLITE_VERSION()")
> >> 28 print "-> SQLite version: %s" % cur.fetchone()
> >> 29
> >> 30 item = response.xpath('//a/@href').extract()
> >> 31 for el in item:
> >> 32
> >> 33 #print("--> ", type(el))
> >> 34 sql = "INSERT INTO Frisch VALUES(" + unicode(counter)
> >> + ", " + "'" + el.strip() + "');" 35
> >> 36 print (sql)
> >> 37 cur.execute(sql)
> >> 38 conn.commit
> >> 39 counter = int(counter)
> >> 40 counter += 1
> >> 41
> >> 42 except sqlite3.Error as e:
> >> 43 print "Error %s:" % e.args[0]
> >> 44 conn.close()
> >> 45
> >>
> >>
> >>
> >>
> >>
> >> Thanks for help.
> >>
> >> o-o
> >>
> >> Thomas
> >
> > Hi Chris,
> >
> > I changed my code in the way you suggested.
> > There is no effect in the db.
>
> Did you replace the line
>
> conn.commit
>
> with
>
> con.commit()
>
> ? If not, can you provide the updated code?
>
>
> > I start this script on the commandline with:
> >
> > tk@Hamlet:~/myscrapy/tutorial/tutorial/spiders$ scrapy crawl webbot
> >
> > When I call the sql-statement in the db it works.
> >
> > sqlite> INSERT INTO Frisch VALUES(4, '//:Üs.chi/hhh\n');
> >
> > sqlite> SELECT * FROM Frisch;
> > 1|Üschi/hhh
> > 2|Üschi/hhh\n
> > 3|//:Üs.chi/hhh\n
> > 4|//:Üs.chi/hhh\n
> >
> >
> >
> >
> >
> >
> >
Thanx a lot Peter. IT WORKS!!!
--
https://mail.python.org/mailman/listinfo/python-list
Pylint 1.6.0 released
Hi folks, I am joyful to announce the release of Pylint 1.6.0. This is the next minor release in the 1.X branch and most probably the last one, since we are preparing the taking off of Pylint 2.0. This release has a couple of small improvements, bug fixes and new checks, comparing to the last one. You can find more about was changed in this release here: https://docs.pylint.org/en/1.6.0/whatsnew/1.6.html https://docs.pylint.org/en/1.6.0/whatsnew/changelog.html#what-s-new-in-pylint-1-6-0 As usual, don't hesitate to report any new bugs you might encounter with this release. We are also looking for new contributors, so if you feel like taking a stab at a bug or a feature you would like in pylint, open up an issue and let's talk! We are also planning to do releases more often from now on and you can see what we are planning for the next releases here: https://github.com/PyCQA/pylint/milestones Thank you and enjoy, Claudiu Popa -- https://mail.python.org/mailman/listinfo/python-list
Clean Singleton Docstrings
I've got a package that contains a global ensmartened dict that allows all the various parts of my program to share state. Things like device handles, information about the application environment, etc. that are inherantly global (i.e. we're not having that debate). Implementation is: class _Registry(UserDict): """Docstring!""" ... Registry = _Registry() So Registry is now a globally accessible mutable object; no reason to complicate things with singletons or borgs or whathave you. From within the interactive console, help(foobar.Registry) gives me the _Registry documentation as expected. >From the (Linux) command line though: $ pydoc3 foobar._Registry [lots of good documentation stuff] $ pydoc3 foobar.Registry no Python documentation found for 'foobar.Registry' Is this a thing that can be fixed with a commensurate amount of effort? Thanks, Rob -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. -- https://mail.python.org/mailman/listinfo/python-list
Re: Clean Singleton Docstrings
On Fri, 8 Jul 2016 09:46 am, Rob Gaddi wrote: [...] > So Registry is now a globally accessible mutable object; no reason to > complicate things with singletons or borgs or whathave you. From > within the interactive console, help(foobar.Registry) gives me the > _Registry documentation as expected. > > From the (Linux) command line though: > $ pydoc3 foobar._Registry > [lots of good documentation stuff] > $ pydoc3 foobar.Registry > no Python documentation found for 'foobar.Registry' > > Is this a thing that can be fixed with a commensurate amount of effort? [steve@ando ~]$ python3 -m pydoc --help pydoc - the Python documentation tool pydoc ... Show text documentation on something. may be the name of a Python keyword, topic, function, module, or package, or a dotted reference to A CLASS OR FUNCTION within a module [...] (Emphasis added.) So, no, reading the docstrings from individual objects is not supported by pydoc's command line interface. You could possibly add that functionality, but I don't know how much effort it would be. -- Steven “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Re: Clean Singleton Docstrings
> On Jul 7, 2016, at 7:46 PM, Rob Gaddi > wrote: > > I've got a package that contains a global ensmartened dict that allows > all the various parts of my program to share state. The simplest solution would be to use a module as your singleton. For example, "registry.py" would work. Pydoc will show its docstring, and it will have all the features you had been using, with the added benefit of not needing to enforce its singletonness. -- https://mail.python.org/mailman/listinfo/python-list
Re: the best online course
On Thursday, July 7, 2016 at 8:38:15 AM UTC+5:30, Michael Torrie wrote: > On 07/06/2016 06:50 PM, Lawrence D’Oliveiro wrote: > >> I want to be easy and not bored so i can learn python. > > > > There is no Royal Road, nothing is going to be handed to you on a plate. > > Seconded. If he gets bored easily, he will not be very successful at > learning Python or any other programming language. The challenge of > programming by itself should be enough to give fulfillment if he's to be > successful at it. Maybe… But Ive noted that when students say “I’m bored” it often means “I dont know what the (*&*^%^$#@ is going on” -- https://mail.python.org/mailman/listinfo/python-list
