Re: [Tutor] Making http posts
2009/11/5 Kent Johnson :
> On Thu, Nov 5, 2009 at 5:06 AM, Stephen Nelson-Smith
> wrote:
>> Hello,
>>
>> I want to make an HTTP POST which will require authentication.
>
> What kind of authentication? Basic auth is easy to use from Python;
> form-based auth is a bit tricker.
>
>> This is because we are using a web tool without an API, and we want a
>> programatic way of getting content into the tool. Tech support of the
>> web tool have said we can just make a POST to the http endpoint.
>>
>> >From the below source, it looks like I just post some text to
>> /project/4254/stories, with the ID of addStoryForm.
>
> IIRC the form ID is not part of the post. The text has to be formatted
> as name=value pairs.
>
>> Is the best way to do this just a simple urllib script?
>
> urllib2. See my writeup here:
> http://personalpages.tds.net/~kent37/kk/00010.html
>
>> What's the best way to find out from the browser exactly what is being
>> posted when I use the web interface? Tcpdump? Or is the a better
>> tool?
>
> There are Firefox addons for this. It's been a while so I'm not sure
> which ones but I think either Firebug or Tamper Data will do it.
>
> Kent
> ___
> Tutor maillist - tu...@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
If you're really stuck, look into mechanize: it links together
urllib, beautiful soup and a form handler. To log in to a form, all
you need to do is:
br = mechanize.Browser()
br.open("http://path.to.form.com/form";)
#br.select_form(...) if it's not the first form on the page - see docs
br['name'] = "username"
br['password'] = "password"
response = br.submit()
then response is a beautifulsoup object of the returned page. The
browser object also updates to act on the new page as well, and has
methods to click on links and so on, exactly as you would in a
browser. Basically, if you can do it with a keyboard and your
prefered webbrowser, you can do it with mechanize, and a few things
beside. (I've bypassed a (copypasta) javascript md5 function on a
forum login before now...)
--
Rich "Roadie Rich" Lovely
There are 10 types of people in the world: those who know binary,
those who do not, and those who are off by one.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
[Tutor] Readable date arithmetic
I have the following method:
def get_log_dates(the_date_we_want_data_for):
t = time.strptime(the_date_we_want_data_for, '%Y%m%d')
t2 = datetime.datetime(*t[:-2])
extra_day = datetime.timedelta(days=1)
t3 = t2 + extra_day
next_log_date = t3.strftime('%Y%m%d')
return (the_date_we_want_data_for, next_log_date)
Quite apart from not much liking the t[123] variables, does date
arithmetic really need to be this gnarly?
How could I improve the above, especially from a readability
perspective? Or is it ok?
S.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Faster list searching?
"Tim Peters" wrote result = set(list1) - set(list2) Of course the result is a set then. Which means that if there were duplicates in list1 you only get one copy in the result. As Tim says, whether that is good, bad or irrelevant depends on the problem context. Maybe that will work fine in context, maybe not. I leave it as an exercise to figure out how to change it back into a list (hint: try the obvious way first ;-) ). But that won't replace any missing duplicates. If they are significant you'll probably need to stick with Kent's list comprehension approach. HTH, -- Alan Gauld Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Faster list searching?
[Luke Paireepinart] >> This is really just a round-about way of using sets. >> I don't really want to give a code-sample unless he's confirmed he's not >> doing this as homework, but the set version is much more simple (shorter >> code that makes more sense) and extremely quick as well. If you're >> interested in it, Bill, reply to me off-list and I'll send it to you. [also Luke Paireepinart] > Never mind about this, Kent already gave basically the same code sample I > was going to. So long as the cat's out of the list, may as well do it "the obvious" ;-) way too: result = set(list1) - set(list2) Of course the result is a set then. Maybe that will work fine in context, maybe not. I leave it as an exercise to figure out how to change it back into a list (hint: try the obvious way first ;-) ). ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Introduction - log exercise
* Antonio de la Fuente [2009-11-17 16:58:08 +]:
> Date: Tue, 17 Nov 2009 16:58:08 +
> From: Antonio de la Fuente
> To: Python Tutor mailing list
> Subject: [Tutor] Introduction - log exercise
> Organization: (muybien.org)
> User-Agent: Mutt/1.5.20 (2009-06-14)
> Message-ID: <20091117165639.ga3...@cateto>
>
> Hi everybody,
>
> This is my first post here. I have started learning python and I am new to
> programing, just some bash scripting, no much.
> Thank you for the kind support and help that you provide in this list.
>
> This is my problem: I've got a log file that is filling up very quickly, this
> log file is made of blocks separated by a blank line, inside these blocks
> there
> is a line "foo", I want to discard blocks with that line inside it, and
> create a
> new log file, without those blocks, that will reduce drastically the size of
> the
> log file.
>
> The log file is gziped, so I am going to use gzip module, and I am going to
> pass
> the log file as an argument, so sys module is required as well.
>
> I will read lines from file, with the 'for loop', and then I will check them
> for
> 'foo' matches with a 'while loop', if matches I (somehow) re-initialise the
> list, and if there is no matches for foo, I will append line to the list.
> When I
> get to a blank line (end of block), write myList to an external file. And
> start
> with another line.
>
> I am stuck with defining 'blank line', I don't manage to get throught the
> while
> loop, any hint here I will really appreciate it.
> I don't expect the solution, as I think this is a great exercise to get wet
> with python, but if anyone thinks that this is the wrong way of solving the
> problem, please let me know.
>
>
> #!/usr/bin/python
>
> import sys
> import gzip
>
> myList = []
>
> # At the moment not bother with argument part as I am testing it with a
> # testing log file
> #fileIn = gzip.open(sys.argv[1])
>
> fileIn = gzip.open('big_log_file.gz', 'r')
> fileOut = open('outputFile', 'a')
>
> for line in fileIn:
> while line != 'blank_line':
> if line == 'foo':
> Somehow re-initialise myList
> break
> else:
> myList.append(line)
> fileOut.writelines(myList)
>
>
> Somehow rename outputFile with big_log_file.gz
>
> fileIn.close()
> fileOut.close()
>
> -
>
> The log file will be fill with:
>
>
> Tue Nov 17 16:11:47 GMT 2009
> bladi bladi bla
> tarila ri la
> patatin pataton
> tatati tatata
>
> Tue Nov 17 16:12:58 GMT 2009
> bladi bladi bla
> tarila ri la
> patatin pataton
> foo
> tatati tatata
>
> Tue Nov 17 16:13:42 GMT 2009
> bladi bladi bla
> tarila ri la
> patatin pataton
> tatati tatata
>
>
> etc, etc ,etc
> ..
>
> Again, thank you.
>
This is how, with your help, finally, I wrote the script.
The way I compress the file at the end of the script, opening files
again, didn't feel right, but couldn't make it work other way, and I
was very tired at the time.
First test, seems to work, but I will do a more deep test tomorrow.
Thank you all.
#!/usr/bin/python
# Importing modules that I'm going to need
import sys
import gzip
import os
# Initialising paragraph list
paragraph = []
# Flag to signal which paragraphs to keep and which one to discard.
keep = True
# Getting file name, without extension, from parameter pass to script,
# to rename file
Re: [Tutor] Faster list searching?
On Wed, Nov 18, 2009 at 5:54 PM, Luke Paireepinart wrote: > This is really just a round-about way of using sets. > I don't really want to give a code-sample unless he's confirmed he's not > doing this as homework, but the set version is much more simple (shorter > code that makes more sense) and extremely quick as well. If you're > interested in it, Bill, reply to me off-list and I'll send it to you. > > Never mind about this, Kent already gave basically the same code sample I was going to. -Luke ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Faster list searching?
On Wed, Nov 18, 2009 at 5:03 PM, Bill Campbell wrote:
>
> When I have had to deal with large lists, I have found that using
> an intermediate dictionary can save huge amounts of time.
> Something like:
>
> dict2 = {}.fromkeys(list2)
> for x in list1:
>if x not in dist2:
>dict2[x] = True
>
> list2 = dict2.keys()
>
> This is really just a round-about way of using sets.
I don't really want to give a code-sample unless he's confirmed he's not
doing this as homework, but the set version is much more simple (shorter
code that makes more sense) and extremely quick as well. If you're
interested in it, Bill, reply to me off-list and I'll send it to you.
-Luke
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Faster list searching?
On Wed, Nov 18, 2009, GoodPotatoes wrote:
>I'm dealing with bigger lists than I have been, and noticed this is getting
>really slow. Is there a faster way to do this?
>
>for x in list1:
>if x not in list2:
>list3.append(x)
>
>My search is taking up to 5 minutes to complete.
When I have had to deal with large lists, I have found that using
an intermediate dictionary can save huge amounts of time.
Something like:
dict2 = {}.fromkeys(list2)
for x in list1:
if x not in dist2:
dict2[x] = True
list2 = dict2.keys()
Bill
--
INTERNET: b...@celestial.com Bill Campbell; Celestial Software LLC
URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way
Voice: (206) 236-1676 Mercer Island, WA 98040-0820
Fax:(206) 232-9186 Skype: jwccsllc (206) 855-5792
Intaxication: Euphoria at getting a refund from the IRS, which lasts until
you realize it was your money to start with.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Introduction - log exercise
* Christian Witts [2009-11-18 09:53:15 +0200]:
> Date: Wed, 18 Nov 2009 09:53:15 +0200
> From: Christian Witts
> To: Antonio de la Fuente
> CC: Python Tutor mailing list
> Subject: Re: [Tutor] Introduction - log exercise
> User-Agent: Thunderbird 2.0.0.23 (X11/20090817)
> Message-ID: <4b03a7e...@compuscan.co.za>
>
> How I would process it:
>
> from sys import argv, exit
> import gzip
>
> if __name__ == '__main__':
>try:
>fIn = gzip.open(argv[1])
>except IOError:
>exit('Cannot open file specified.\n')
>
>fOut = open('outputFile', 'a')
>
>datablock = []
>discard = False
>for line in fIn:
>if line.strip():
>if line.strip() == 'foo':
>discard = True
>else:
>datablock.append(line)
>else:
>if discard:
>datablock = []
>discard = False
>else:
>fOut.write(''.join(datablock))
>
>fIn.close()
>fOut.close()
I've borrowed a couple of things from your script.
It is educative for me to see different ways of solving the problem.
Thank you.
>
> --
> Kind Regards,
> Christian Witts
> Business Intelligence
>
> C o m p u s c a n | Confidence in Credit
>
> Telephone: +27 21 888 6000
> National Cell Centre: 0861 51 41 31
> Fax: +27 21 413 2424
> E-mail: cwi...@compuscan.co.za
>
> NOTE: This e-mail (including attachments )is subject to the disclaimer
> published at: http://www.compuscan.co.za/live/content.php?Item_ID=494.
> If you cannot access the disclaimer, request it from
> email.disclai...@compuscan.co.za or 0861 514131.
>
> National Credit Regulator Credit Bureau Registration No. NCRCB6
>
>
--
-
Antonio de la Fuente Martínez
E-mail: t...@muybien.org
-
... ich bin in einem dusenjet ins jahr 53 vor chr ... ich lande im
antiken Rom ... einige gladiatoren spielen scrabble ... ich rieche
PIZZA ...
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Faster list searching?
On Wed, Nov 18, 2009 at 4:51 PM, GoodPotatoes wrote: > I'm dealing with bigger lists than I have been, and noticed this is getting > really slow. Is there a faster way to do this? > > for x in list1: > if x not in list2: > list3.append(x) > > My search is taking up to 5 minutes to complete. If you can, use a set instead of list2. "x not in list2" does a linear search of list2 for x, which takes time proportional to the length of list2. Searching a set takes constant time - it doesn't depend on the size of the set. Also you can use a list comprehension to speed up the outer loop a little: set_of_list2 = set(list2) list3 = [ x for x in list1 if x not in set_of_list2 ] Note this will only work if the elements of list2 are hashable (useable as dict keys). Kent ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Faster list searching?
On Wed, Nov 18, 2009 at 1:51 PM, GoodPotatoes wrote: > I'm dealing with bigger lists than I have been, and noticed this is getting > really slow. Is there a faster way to do this? > > for x in list1: > if x not in list2: > list3.append(x) > > My search is taking up to 5 minutes to complete. greetings! hopefully this isn't a homework problem as we cannot help with those. can you give us an example of the lists that you're using (not the entire things, but just tell us what they contain)? also, use the timeit module to show us some numbers to confirm what you're seeing. my suggestion would be to use sets if possible instead of lists since those lookups are way faster (hash vs. sequence). if you're using Python 3, i think you can even build the solution set using set comprehensions. -- wesley - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - "Core Python Programming", Prentice Hall, (c)2007,2001 "Python Fundamentals", Prentice Hall, (c)2009 http://corepython.com wesley.j.chun :: wescpy-at-gmail.com python training and technical consulting cyberweb.consulting : silicon valley, ca http://cyberwebconsulting.com ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] Faster list searching?
I'm dealing with bigger lists than I have been, and noticed this is getting really slow. Is there a faster way to do this? for x in list1: if x not in list2: list3.append(x) My search is taking up to 5 minutes to complete. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python wsgi
On Wed, Nov 18, 2009 at 5:11 AM, Amit Sethi wrote: > Hi , > > How do I run a python script using wsgi? I am not using any web > framework I just wish to try out running a simple script. You need to run a WSGI-compliant server and configure it to host your application. The std lib contains a simple reference server in wsgiref, the make_server() example shows how to use it to run a demo app: http://docs.python.org/library/wsgiref.html#wsgiref.simple_server.make_server If you want something more than that, look at the list here: http://www.wsgi.org/wsgi/Servers If I were to pick from that list for simple testing I would try CherryPy. Kent ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] opening a file directly from memory
Humm. Most enlighting. For my case the solution is clearly to have an initialization file. In case the specified extension is not known then I'll prompt the user to save the file and pass on the responsibility. Txs everyone, Miguel Junte todos os seus créditos no Único da Capital Mais... ...reduza as suas despesas mensais. Saiba mais em http://www.iol.pt/correio/rodape.php?dst=0901051 ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Open Source database software
Kent Johnson wrote: On Mon, Nov 24, 2008 at 7:19 PM, Mike Meisner wrote: > 3. A good GUI front end for creating the database, creating forms for user > data input, queries, reports, etc. For this you might look at Dabo: http://dabodev.com/ I haven't worked with it myself but some people like it a lot. PostgreSQL has pgAdmin which is very nice for basic admin but not for user applications. This would also be my best advice. The framework is very solid (core as been stable for a few years know). Documentation exists, although it is lacking and scatered the mailing list is low traffic and very responsive. I would not focus on using the visual tools, although a lot can already be accomplished with them (if you dig or ask you will find info on hand coding forms). It is based on wxPython, and you can drop to that level any time. But in general you rather use the dabo interface as it will provide you with a much cleaner and productive implementation (quite marvelous), and with very good databinding. Database access is abstracted so you can develop for/with SQLite and then just change the backend to one of a few flavours of choice (such as PostgreSQL, Firebird and other commercial options as well). In a nutshelll :-) Miguel VIVA os SEUS SONHOS com o Crédito Pessoal Capital Mais. Peça aqui até 15.000 Euros: http://www.iol.pt/correio/rodape.php?dst=0901052 ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] python wsgi
Hi , How do I run a python script using wsgi? I am not using any web framework I just wish to try out running a simple script. -- A-M-I-T S|S ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Introduction - log exercise
"bob gailer" wrote "Antonio de la Fuente" wrote > if not line.isspace() and not line == 'foo': > fileOut.write(line) But then, the new log file will have all the blocks, even the ones that had 'foo' on it, even if the foo lines weren't there anymore. No? or is there anything that I don't get? I think the test should be: if not line.isspace and 'foo' not in line: fileOut.write(line) No - that misses the objective of eliminating blocks containing 'foo' Yeah, I was assuming "block" referred to a line not a paragraph. I was simply extending the original post to remove the line if it contained 'foo' rather than just being 'foo'. But it has since become obvious that the OP needs more than that! Alan G. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
