Re: Python LOC, .exe size, and refactoring
On Tue, 21 Feb 2012 19:51:07 -0800, CM wrote: > I have an application that I was hoping to reduce a bit the size of its > .exe when "packaged" with py2exe. I'm removing some Python modules such > as Tkinter, etc., but now wonder how much I could size I could reduce by > refactoring--and therefore shortening--my code. Well that will depend on how much you refactor it, but frankly, unless your code is truly awful, this will be a micro-optimization. py2exe bundles a Python runtime environment plus your files into a single exe file. Typically the runtime environment will be somewhere around 11MB for wxPython GUI apps (or 4MB with compression turned on, which will slow your application down). http://www.py2exe.org/index.cgi/SingleFileExecutable The runtime environment for Oracle's Java environment starts at 7MB and is typically 15MB, plus whatever libraries your own code produces. For dot-net applications, the framework can be up to 60MB. http://weblogs.java.net/blog/stanleyh/archive/2005/05/deployment_unde.html http://www.hanselman.com/blog/SmallestDotNetOnTheSizeOfTheNETFramework.aspx While I think 60MB for a basic calculator app is taking the piss, this is 2011 not 1987 and we don't have to support floppy disks any more. 11MB for a GUI app is nothing to be worried about. That takes, what, 3 minutes to download even on a 512 kbps link? > Is there a rule of thumb that predicts the relationship between the > number of lines of Python code and the resultant size of the application > (leaving aside the size of imported modules)? Yes. To a close approximation, for most applications: size of bundled application = ( size of Python runtime environment + size of libraries used ) Your code is most likely insignificant compared to the others. > Or is there a way to > roughly estimate how much would refactoring the code as much as I > reasonably can help? (For example, in some cases there is some cut and > paste coding...I know, it's bad). Look at it this way: take the .pyc file from your code. How big is it? Say, it's 200K. That's a BIG file -- the decimal module in the standard library is only 152K. Suppose you could cut it in half -- you would save 100K. Even if you could somehow cut it down to 1K, you've saved less than 200K. Do you care? Refactoring your code is double-plus good for maintainability. You should do it anyway. But don't do it to shrink an 11MB exe down to 10.8MB. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Should I acquire lock for logging.Handler.flush()?
On Tuesday 21 Feb 2012 12:52:14 PM Vinay Sajip wrote: > If you are using SMTPHandler, calling flush() won't do anything. > You'll probably need to subclass the handler to implement rate > limiting. Oh, no! I'm writing my own handler. https://github.com/fayazkhan/ses_handler/blob/master/ses_handler.py Now I understand from here http://docs.python.org/library/logging.html#handler-objects that emit() calls are wrapped acquire() and release() in the handle() method. Anyway, I read the source and found many interesting things that ought to be mentioned in the docs. Such as flush() should be called from close() whenever it's implemented. (FileHandler.close() is doing it) And how come close()/flush() isn't being called from inside a lock? (Handler.close() calls the module level _acquireLock() and _releaseLock()s but nothing about the instance level acquire() or release()) Or is it being locked from somewhere else? -- Fayaz Yusuf Khan Cloud developer and architect Dexetra SS, Bangalore, India fayaz.yusuf.khan_AT_gmail_DOT_com fayaz_AT_dexetra_DOT_com +91-9746-830-823 signature.asc Description: This is a digitally signed message part. -- http://mail.python.org/mailman/listinfo/python-list
Campaign to raise $200 for python code statistics tool
If $200 of contributions are made, I will create a python code statistics tool. To contribute go to this link on invested.in, a crowdfunding site (I cannot use kickstarter because I'm from Canada). http://invested.in/P2127/python-code-statistics-tool -- http://mail.python.org/mailman/listinfo/python-list
Python LOC, .exe size, and refactoring
I have an application that I was hoping to reduce a bit the size of its .exe when "packaged" with py2exe. I'm removing some Python modules such as Tkinter, etc., but now wonder how much I could size I could reduce by refactoring--and therefore shortening--my code. Is there a rule of thumb that predicts the relationship between the number of lines of Python code and the resultant size of the application (leaving aside the size of imported modules)? Or is there a way to roughly estimate how much would refactoring the code as much as I reasonably can help? (For example, in some cases there is some cut and paste coding...I know, it's bad). -- http://mail.python.org/mailman/listinfo/python-list
Re: Python as a default shell, replacement of bash, sh, cmd ?
On 2/19/12 2:16 AM, SherjilOzair wrote:
Well, if not modify python itself, I was thinking of making another shell,
which borrows a lot from python, something like merging bash and python. such
that I can do `cd ~/Desktop/dev` and `for i in open('file.txt'): print i` at
the some shell. This I think would be VERY useful.
That's an awful lot of key mashing just to replace 'cat file.txt'
for i in big small upper lower ; do for j in conf bin log ; do chmod
2755 /home/me/$i/$j ; done ; done
cat data.log | grep Error | awk -F, '{print $5, $1, $2,}' | sort
??
I believe your solution seeks a problem. I also believe a tool that
seeks to be all things generally does none of them particularly well.
Cheers.
IPyhton is very good, but after all, it is just an advanced interpreter, not a
default shell. I don't want this to run on top of bash or sh. But it should run
on its own, at shell level.
Bash and sh, according to me, have very ugly syntaxes and the general user does
not even use those. Python put on the shell would be adhering to python's
vision, i.e. bringing programming to the masses.
The general user, who earlier could not do batch operations, and had to buy
software and such for all that, could now write his open simple python script
and run it in his shell that would do as he wants.
Python on the shell could effectively remove learning grep, awk, sed, bash and
the various unix utilities.
Don't take me wrong. Those are awesome tools, and I use them. But the
awesomeness is not experienced by the general UNIX user on mac or linux. Python
could do that.
We all know how great a programming language python is. Imagine being able to
control your computer with such an elegant language. Imagine that I import some
speech recognition utility on the terminal shell, and voila, I'm speaking to
the computer and it is doing stuff on the terminal for me.
Shell would give python raw power! And Python would manage it well.
--
http://mail.python.org/mailman/listinfo/python-list
ali
http://sharecash.org/download.php?file=2652910 -- http://mail.python.org/mailman/listinfo/python-list
Re: [pyxl] xlrd 0.7.2 released!
Chris Withers wrote: > Hi All, > > I'm pleased to announce the release of xlrd 0.7.2. This release, > like the xlwt release, is long overdue and has been over 2.5 years > in the making! > [...snip...] > > I've currently only put up .tar.gz sdists, if anyone requires > anything else, please explain why and I'll be happy to add! hi, a windows installer exe would be good for the same reasons it was good for previous versions. two reasons that spring to mind immediately are: - it makes it much easier to tell what version is installed - it makes it much easier to uninstall the package i know that both of these are things that the python community does not yet seem to find useful but everyone else seems to. for instance, if i just install the new versions of these packages on windows, it will replace the python-specific files that were associated with the previous version but it will have no impact on the system's list of installed software so if i look at the add/remove software control panel, it will happily tell me that i still have the previous versions. on the other hand, if the new versions were installed with windows installers, they would be able to properly replace the old versions with the new versions and make it clear that it has done do. knowing this, i will of course uninstall the old versions before installing the new versions so at least windows won't be forced to lie to me about what versions are installed. of course, i might be wrong about all of this and setup.py may manage the windows installed software list properly and do a proper upgrade. if so, sorry for wasting time. cheers, raf -- http://mail.python.org/mailman/listinfo/python-list
Re: PyKota, Python: AttributeError: 'module' object has no attribute '_quote'
Thanks for your advice. The command itself didn't work, but it brought me on the right path and "car and engine" are running now! ;-) I fixed the problem by copying the correct pg.py-file to the python2.7 path. After that the error didn't show up again and I'm now able to create users etc. Thank you very much! Kind regards, Johannes -- View this message in context: http://python.6.n6.nabble.com/PyKota-Python-AttributeError-module-object-has-no-attribute-quote-tp4487915p4493041.html Sent from the Python - python-list mailing list archive at Nabble.com. -- http://mail.python.org/mailman/listinfo/python-list
Re: Should I acquire lock for logging.Handler.flush()?
On Feb 21, 7:23 am, Fayaz Yusuf Khan wrote: > I'm writing a custom logging Handler that sends emails through AWS Simple > Email Service using the boto library. > As there's a quota cap on how many (200) emails I can send within 24hrs, I > think I need to buffer my log messages from the emit() calls (Or is that a bad > idea?). > And I was reading the Handler documentation and was confused if I should call > the acquire() and release() methods from within a flush() call. > -- > Fayaz Yusuf Khan > Cloud developer and architect > Dexetra SS, Bangalore, India > fayaz.yusuf.khan_AT_gmail_DOT_com > fayaz_AT_dexetra_DOT_com > +91-9746-830-823 > > signature.asc > < 1KViewDownload If you are using SMTPHandler, calling flush() won't do anything. You'll probably need to subclass the handler to implement rate limiting. In the stdlib, only StreamHandler and its subclasses actually implement flush(), which flushes I/O buffers to disk. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: xlwt 0.7.3 released!
Hello chris, Thanks for the greet news! Is there any chance that xlrd read .xlsx format and arrive to decode special unicode instead of firing some unicode Exception? Cheers Karim Le 21/02/2012 10:18, Chris Withers a écrit : Hi All, I'm pleased to announce the release of xlwt 0.7.3. This release is long overdue and has been over 2.5 years in the making! The highlights: - Added user_set and best_fit attributes to Column class. - Fixed an "[Errno 0] Error" raised when Worksheet.flush_row_data() was called after Workbook.save() - Fixed an error on Windows that occurred when writing large blocks to files. - Added the ability to write rich text cells - Fixed a bug when writing MULBLANK records on big-endian platforms. - allow the active_pane on worksheets to be specified - added support for zoom (magn) factors and improved possibilities when generating split panes For a full list of the changes, please see the svn log: https://secure.simplistix.co.uk/svn/xlwt/trunk I've currently only put up .tar.gz sdists, if anyone requires anything else, please explain why and I'll be happy to add! cheers, Chris -- http://mail.python.org/mailman/listinfo/python-list
[bug] imaplib case sensitive
Hello,
Looks like imaplib is case sensitive, even though the IMAP protocol isn't:
(1) Except as noted otherwise, all alphabetic characters
are case-insensitive. The use of upper or lower case
characters to define token strings is for editorial clarity
only. Implementations MUST accept these strings in a
case-insensitive fashion.
http://tools.ietf.org/html/rfc3501#section-9
For example:
[Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49) ]
[imaplib 2.58]
import imaplib
M = imaplib.IMAP4( 'localhost', 1143 )
M.login( 'user', 'password' )
M.select()
typ, data = M.search(None, 'ALL')
for num in data[0].split():
typ, data = M.fetch(num, '(ENVELOPE)')
print 'Message %s\n%s\n' % (num, data[0][1])
M.close()
M.logout()
Traceback (most recent call last):
File "Test.py", line 3, in
M = imaplib.IMAP4( 'localhost', 1143 )
File "python2.6/imaplib.py", line 184, in __init__
self.welcome = self._get_response()
File "python2.6/imaplib.py", line 935, in _get_response
raise self.abort("unexpected response: '%s'" % resp)
imaplib.abort: unexpected response: '* ok imap4rev1'
Note the 'ok' tag in lower case.
--
http://mail.python.org/mailman/listinfo/python-list
xlrd 0.7.2 released!
Hi All, I'm pleased to announce the release of xlrd 0.7.2. This release, like the xlwt release, is long overdue and has been over 2.5 years in the making! The highlights: - All messaging and debug logging is now written to the logfile provided to open_workbook. - Tolerant handling of file with non-standard compound document header - Tolerant handling of files with extra zero bytes at end of NUMBER record - Handle mostly-BIFF8 file with BIFF5-7 WINDOWS2 record - Handle dodgy version 2.x .xls files. - Added support for HYPERLINK extraction - Added access to cell notes/comments. - Enable reading files created by pyXLWriter -- puts BIFF8 MERGEDCELLS record in a BIFF5 file. - Fixed a file-locking issue on Windows when an exception was raised in open_workbook() and on_demand was True - Book objects are now context managers - Rich text formatting information is now extracted - Page breaks information is now extracted - Some improvements in zoom factor handling - PANE records are now processed - Some memory and performance enhancements For a full list of the changes, please see the svn log: https://secure.simplistix.co.uk/svn/xlrd/trunk I've currently only put up .tar.gz sdists, if anyone requires anything else, please explain why and I'll be happy to add! cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
xlwt 0.7.3 released!
Hi All, I'm pleased to announce the release of xlwt 0.7.3. This release is long overdue and has been over 2.5 years in the making! The highlights: - Added user_set and best_fit attributes to Column class. - Fixed an "[Errno 0] Error" raised when Worksheet.flush_row_data() was called after Workbook.save() - Fixed an error on Windows that occurred when writing large blocks to files. - Added the ability to write rich text cells - Fixed a bug when writing MULBLANK records on big-endian platforms. - allow the active_pane on worksheets to be specified - added support for zoom (magn) factors and improved possibilities when generating split panes For a full list of the changes, please see the svn log: https://secure.simplistix.co.uk/svn/xlwt/trunk I've currently only put up .tar.gz sdists, if anyone requires anything else, please explain why and I'll be happy to add! cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Python as a default shell, replacement of bash, sh, cmd ?
18.02.12 20:58, SherjilOzair написав(ла): Has it been considered to add shell features to python, such that it can be used as a default shell, as a replacement for bash, etc. I'm sure everyone would agree that doing this would make the terminal very powerful. What are your views on this? Look at Hotwire (http://code.google.com/p/hotwire-shell/). -- http://mail.python.org/mailman/listinfo/python-list
Re: netCDF4 variable manipulation
On Tue, Feb 21, 2012 at 12:29 AM, Sheldon wrote:
> On Feb 21, 12:53 am, Steven D'Aprano +comp.lang.pyt...@pearwood.info> wrote:
>> On Mon, 20 Feb 2012 12:37:22 -0800, Sheldon wrote:
>> > Hi,
>>
>> > I'm trying to read a netCDF4 variable from a file (no problem) and then
>> > scale it before writing over the original variable in the file.
>>
>> > I'm using python 2.7 and the latest netCDF4 module. It's not that I keep
>> > getting an error message but I want to do this without using for loops
>> > and all the manuals seems to be skipping this task as if it is never
>> > done. I'm new to python and coming over from matlab. Does anyone know
>> > how to modify the netCDF4 variable in python and then write it back
>> > without creating a new variable, or using for loops?
> My apologies. I didn't realize this module was not popular. I'm using
> the python module "netCDF4"
> that I installed using PIP. This module gives me the ability to read
> and write netcdf4 files.
> The module is an interface to the netcdf4.1.3 and HDF51.8.8 libraries.
> What I'm trying to do is open an existing netcdf file, extract an
> variable, modify the variable, and then
> write it again to the file - effectively replacing the old variable.
>
> rgrp = netCDF4.Dataset('ODIN_NWP_2001_01_01_00.NC','r+')
> pv = rgrp.groups['Data_3D'].variables['PV']
>
> print pv
>
> float32 PV(u'level', u'lat', u'lon')
> longname: Potential Vorticity
> _FillValue: -999.0
> units: K m**2 kg**-1 s**-1
> code: 60
> scale_factor: 1.0
> add_offset: 0.0
> path = /Data_3D
> unlimited dimensions = ()
> current size = (60, 181, 360)
>
> As you can see, pv is described here as a netCDF4.Variable. I'm trying
> to modify this pv and then write it back to the file
> but the manual is vague on this part. I tried this:
>
> pv = pv*2
> ---
> TypeError Traceback (most recent call
> last)
> > 1 pv = pv*2
> TypeError: unsupported operand type(s) for *: 'netCDF4.Variable' and
> 'int'
>
> I'm unsure how this is suppose to work.
The API docs suggest that Variables don't support any arithmetic
operations, just slicing and attribute manipulation. They seem to
interoperate with numpy arrays though, so presumably you could avoid
explicit looping by loading the Variable's contents into a numpy
array, doing your calculations on that, and then assigning the
resulting numpy array back into a slice of the Variable.
Cheers,
Chris
--
http://mail.python.org/mailman/listinfo/python-list
Re: netCDF4 variable manipulation
On Feb 21, 12:53 am, Steven D'Aprano wrote:
> On Mon, 20 Feb 2012 12:37:22 -0800, Sheldon wrote:
> > Hi,
>
> > I'm trying to read a netCDF4 variable from a file (no problem) and then
> > scale it before writing over the original variable in the file.
>
> > I'm using python 2.7 and the latest netCDF4 module. It's not that I keep
> > getting an error message but I want to do this without using for loops
> > and all the manuals seems to be skipping this task as if it is never
> > done. I'm new to python and coming over from matlab. Does anyone know
> > how to modify the netCDF4 variable in python and then write it back
> > without creating a new variable, or using for loops?
>
> There is no such thing as "the netCDF4 variable" in Python -- it is not a
> built-in part of the language, and therefore there is no built-in feature
> for manipulating it.
>
> You are going to have to be more specific about what you want than just
> "how do I modify a variable with for loops?".
>
> My wild guess is that you have some sort of config file which includes a
> field called "netCDF4" and you want to update it in place. We can't tell
> you how to do this without knowing what the config file is -- is it an
> INI file, XML, JSON, YAML, Unix-style rc file, a binary pickle, or
> something else?
>
> --
> Steven
Hi,
My apologies. I didn't realize this module was not popular. I'm using
the python module "netCDF4"
that I installed using PIP. This module gives me the ability to read
and write netcdf4 files.
The module is an interface to the netcdf4.1.3 and HDF51.8.8 libraries.
What I'm trying to do is open an existing netcdf file, extract an
variable, modify the variable, and then
write it again to the file - effectively replacing the old variable.
rgrp = netCDF4.Dataset('ODIN_NWP_2001_01_01_00.NC','r+')
pv = rgrp.groups['Data_3D'].variables['PV']
print pv
float32 PV(u'level', u'lat', u'lon')
longname: Potential Vorticity
_FillValue: -999.0
units: K m**2 kg**-1 s**-1
code: 60
scale_factor: 1.0
add_offset: 0.0
path = /Data_3D
unlimited dimensions = ()
current size = (60, 181, 360)
As you can see, pv is described here as a netCDF4.Variable. I'm trying
to modify this pv and then write it back to the file
but the manual is vague on this part. I tried this:
pv = pv*2
---
TypeError Traceback (most recent call
last)
> 1 pv = pv*2
TypeError: unsupported operand type(s) for *: 'netCDF4.Variable' and
'int'
I'm unsure how this is suppose to work.
/S
--
http://mail.python.org/mailman/listinfo/python-list
