Re: What behavior would you expect?

[Paul Rubin]

Chris Angelico  writes:
>>   if len(fs) == 0:  ... # didn't get a filename
> Bikeshedding: That could be written as simply "if not fs". :)

Yeah, in that instance you could do that.  It's an unsafe practice when
None is used as the no-value marker, since the empty string is a
perfectly good string and also converts to boolean false.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What behavior would you expect?

[Chris Angelico]

On Fri, Feb 20, 2015 at 5:18 PM, Paul Rubin  wrote:
> Empty string would be bad.  Sometimes I like to simulate an option type,
> by returning the value as a 1-element list if there's a value, otherwise
> as an empty list.  So you could say
>
>   filename = get_filename(...)[0]
>
> if you want an exception in the failure case, or you could do something
> like
>
>   fs = get_filename(...)
>   if len(fs) == 0:  ... # didn't get a filename

Bikeshedding: That could be written as simply "if not fs". :)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What behavior would you expect?

[Paul Rubin]

Dan Sommers  writes:
> I'd still prefer an exception to None, and we agree on that an empty
> string is bad because it's not a non-string and it could be too easily
> mistaken for a filename.

Empty string would be bad.  Sometimes I like to simulate an option type,
by returning the value as a 1-element list if there's a value, otherwise
as an empty list.  So you could say

  filename = get_filename(...)[0]

if you want an exception in the failure case, or you could do something
like

  fs = get_filename(...)
  if len(fs) == 0:  ... # didn't get a filename
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What behavior would you expect?

[Dan Sommers]

On Thu, 19 Feb 2015 22:51:57 -0700, Jason Friedman wrote:

>>
>> I'd still advise using my_list.sort() rather than sorted(), as you
>> don't need to retain the original.
>>
>>
> Hmm.
> 
> Trying to figure out what that looks like.
> If I understand correctly, list.sort() returns None.
> What would I return to the caller?

my_list.  In the end, it'll look something like this:

def f():
my_list = some_other_function_that_returns_an_unsorted_list()
my_list.sort()
return my_list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What behavior would you expect?

[Chris Angelico]

On Fri, Feb 20, 2015 at 4:51 PM, Jason Friedman  wrote:
>> I'd still advise using my_list.sort() rather than sorted(), as you
>> don't need to retain the original.
>>
>
> Hmm.
>
> Trying to figure out what that looks like.
> If I understand correctly, list.sort() returns None.
> What would I return to the caller?

Check its docs: it sorts the list in place. So you return the list,
after you sort it. It's like appending to a list: None is returned and
the list itself is changed.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What behavior would you expect?

[Jason Friedman]

>
> I'd still advise using my_list.sort() rather than sorted(), as you
> don't need to retain the original.
>
>
Hmm.

Trying to figure out what that looks like.
If I understand correctly, list.sort() returns None.
What would I return to the caller?


> If you're going to call listdir, you probably want to use fnmatch directly.
>
>
Got it.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What behavior would you expect?

[Jason Friedman]

I think I came in a little late, and saw "2. no files match the given
> pattern," in which case I'm sticking to my story and returning an empty
> list.
>
> The original problem was "to search a directory and return the name of
> the most recent file matching a given pattern."  I'd still prefer an
> exception to None, and we agree on that an empty string is bad because
> it's not a non-string and it could be too easily mistaken for a
> filename.
>
>
Agreed.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What behavior would you expect?

[Dan Sommers]

On Fri, 20 Feb 2015 16:16:50 +1100, Chris Angelico wrote:

> On Fri, Feb 20, 2015 at 3:54 PM, Dan Sommers  wrote:
>> if there are no
>> values to return, then return an empty collection.
> 
> That one makes sense only if you were going to return a collection
> anyway, though. If you were going to return a string, returning an
> empty list on failure makes no sense. Hence the notion of returning a
> "non-string".

I think I came in a little late, and saw "2. no files match the given
pattern," in which case I'm sticking to my story and returning an empty
list.

The original problem was "to search a directory and return the name of
the most recent file matching a given pattern."  I'd still prefer an
exception to None, and we agree on that an empty string is bad because
it's not a non-string and it could be too easily mistaken for a
filename.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What behavior would you expect?

[Jason Friedman]

> def order_matching_files(a_path, a_glob="*"):
> """Search a path for files whose names match a_glob
> and return a list of the full path to such files, in descending
> order of modification time. Ignore directories."""
> previous_dir = os.getcwd()
> os.chdir(a_path)
> return_list = [os.path.join(a_path, x) for x in glob.glob(a_glob) if
> os.path.isfile(x)]
> os.chdir(previous_dir)
> return reversed(sorted(return_list, key=os.path.getmtime))
>

Oops, guess I'm not really returning a list, I'm returning an iterator.  I
should change either the comment or the return value.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What behavior would you expect?

[Chris Angelico]

On Fri, Feb 20, 2015 at 4:13 PM, Jason Friedman  wrote:
> os.chdir(a_path)
> return_list = [os.path.join(a_path, x) for x in glob.glob(a_glob) if
> os.path.isfile(x)]
> os.chdir(previous_dir)
> return reversed(sorted(return_list, key=os.path.getmtime))
>
> It's a shame that glob.glob does not take an arbitrary directory as an
> optional argument if one does not want to scan the current directory.

If you look at the glob module's docs, you'll see how it's implemented:

https://docs.python.org/2/library/glob.html

You may want to use fnmatch, or possibly just prepend a path name onto
your glob. That way, you wouldn't need to change directory, which is
almost always a bad idea for a library function.

Instead of the reversed() call, you can simply add reverse=True to the
sorted() call. And since you're not making use of the original list,
you could use its sort() method (which has key and reverse parameters
too) to sort a bit more efficiently. But I think your code is pretty
good!

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What behavior would you expect?

[Chris Angelico]

On Fri, Feb 20, 2015 at 3:54 PM, Dan Sommers  wrote:
> if there are no
> values to return, then return an empty collection.

That one makes sense only if you were going to return a collection
anyway, though. If you were going to return a string, returning an
empty list on failure makes no sense. Hence the notion of returning a
"non-string".

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What behavior would you expect?

[Jason Friedman]

I have need to search a directory and return the name of the most recent
file matching a given pattern.  Given a directory with these files and
timestamps,

>
> q.pattern1.abc Feb 13
> r.pattern1.cdf  Feb 12
> s.pattern1.efg  Feb 10
> t.pattern2.abc Feb 13
> u.pattern2.xyz  Feb 14
> v.pattern2.efg  Feb 10
>
> calling my_function("/path/to/dir", "pattern1") will return q.pattern1.abc
> and calling my_function("/path/to/dir", "pattern2") will return
> u.pattern2.xyz.
>
> My question is, what would be a reasonable behavior/result/return value if:
> 1. "/path/to/dir" does not exist or is not readable
> 2. no files match the given pattern
>
> Also, what would be a reasonable name for such a function?
>

Thank you everyone.  Taking bits of advice from each of you I tentatively
have:

import glob
import os

def order_matching_files(a_path, a_glob="*"):
"""Search a path for files whose names match a_glob
and return a list of the full path to such files, in descending
order of modification time. Ignore directories."""
previous_dir = os.getcwd()
os.chdir(a_path)
return_list = [os.path.join(a_path, x) for x in glob.glob(a_glob) if
os.path.isfile(x)]
os.chdir(previous_dir)
return reversed(sorted(return_list, key=os.path.getmtime))

It's a shame that glob.glob does not take an arbitrary directory as an
optional argument if one does not want to scan the current directory.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What behavior would you expect?

[Dan Sommers]

On Fri, 20 Feb 2015 07:11:13 +1100, Chris Angelico wrote:

> On Fri, Feb 20, 2015 at 7:03 AM, Denis McMahon  
> wrote:
>> On the one hand, the return type of a function (when it returns,
>> rather than raising an exception) should be consistent to itself,
>> even if using a language where types are not declared.
> 
> Yes, so I'd advise against having a function sometimes return a string
> and sometimes a dict. Sure. But None gets a bit of a special pass
> here; in a lot of languages, what you'd have is a "nullable" type (eg
> in C you might declare the return value as "pointer to char", and
> either return a pointer to a string, or NULL), but in Python, you
> return None for the special case. So in a sense, "str or None" is not
> so inconsistent.

I'm an old C and assembly language programmer, but the older I get, the
less I use null (in C or in Java), None (in Python), or equivalent in
other languages.  If a function can't perform its side effect (like
renaming a file), then raise an exception.  If a function called for its
value can't return that value, then raise an exception; if there are no
values to return, then return an empty collection.  How much code do we
write and read that checks for null (or None, or whatever), and how much
time do we spend writing it and reading it?

Readability counts.  Simple is better than complex.  Explicit is better
than implicit.  Errors should never pass silently.  In the face of
ambiguity, refuse the temptation to guess.  (Wow!  I knew I could turn
to the Zen for support, but I didn't realize just how much support I
would get!)

See also .

Dan
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[rurpy]

On 02/19/2015 02:23 PM, Mario Figueiredo wrote:
> On Thu, 19 Feb 2015 12:26:04 -0800 (PST), ru...@yahoo.com wrote:
>>
>> I'll point out that five people in this thread (by my
>> count) have said that Postgresql requires a significant
>> amount of work to setup and use.  Only you and Steven claim
>> the opposite.
> 
> Well, I claim the opposite too.
> 
>> Or I could have tried as you
>> and Steven suggest to "somehow" package Postgresql in my
>> app installer.  That would have been an even bigger cost
>> in my time with an uncertain outcome.
> 
> I don't see how. You said it was just a simple application this
> postgres database was serving. So I must assume you aren't talking of
> a complex setup that database synchronization or other features that
> force a manual setup.

Correct.

> For your purposes, you could just basically backup your data folder
> and package postgres with the same configuration files you have in
> your current computer.

What do you mean by "data folder"?  The directory in which 
Postgresql keeps its database data?  If so you're wrong.  That 
data is at a minimum architecture dependent.  It is also private  
to Postgresql and one would be ill advised to use that as a 
distribution format.  There is also no need to since installing
Postgresql and loading the initial data from some standard format 
is supported and will produce the same results. 

Or are you talking about some sort of data (csv files, sql files
of insert statements, etc) that you use to initially load tables
in your database.  That will be the same whether you are using
Postgresql or Sqlite so I don't see your point. 

>> None of those
>> costs would have been necessary at all had I developed
>> a self-contained Sqlite app.
> 
> The cost would have probably been much higher, depending on your
> project. SQLite would have forced you to move all your business logic
> into your code, greatly increasing code maintenance, your application
> extensibility and its ability to more easily adapt to new business
> requirements.

How so?  nobody's claimed that Sqlite is a replacement for 
Postgresql in large scale, high concurrency "heavy-duty" 
applications.  Ethan Furman posted a list of things that 
Sqlite does well from the Sqlite website.  Here is a link 
if you missed it:

  http://www.sqlite.org/whentouse.html

We are talking about using Postgresql as a backend for 
applications that fit on that list, ie applications like 
the applications already using Sqlite :-) 

>From previous posts I am guessing that what you are saying is 
that business logic should be implemented in stored procedures 
and because Sqlite does not offer stored procedures you can't 
implement business logic in Sqlite.

First, recall that we are not talking about a multi-tiered set 
of applications with a middleware layer and mutiuser backend.
We've established already that is the domain of servers like
Postgresql.

While I tend to agree with the idea that business logic 
should be in the database, I'm sure you're aware that that 
is not a universally held opinion and it is certainly not 
a universally implemented one in the domain of applications
we're talking about.  It is easy to find plenty of applications 
that implement all or part of the business logic in the app.  

Given Sqlite's architecture I'm not even sure you can say
it doesn't have stored procedures.  Since programmatic 
access is through the (python-)API you can write a module 
with all the business logic and decree that that all higher
level functions access the database through that module.
How is that effectively different than a set of stored
procedures in a client-server database?

And I can see a justification for not even going that far 
in some cases.  We are talking about small scale applications 
where the database can be considered an "implementation 
detail" of the application but it is easier to implement 
the application by taking advantage of the capabilities of 
a SQL relational database than trying to implement those 
storage details in some ad-hoc way.

> Conversely, if none of this is true concerning your particular
> project, then you just chose the wrong tool. Postgres was overkill for
> your particular needs and it was a mistake to think you need it to
> function just as a shelve on steroids.

Bingo!  You're catching on.  :-)  Again I remind you that no one
has said that Sqlite is a universal replacement for Postgresql

>> Finally keep in mind that if you develop your app using
>> Sqlite, it is likely to be far easier to migrate to
>> a heavy-duty backend like Postgresql later should you
>> need to than to go in the other direction when you find
>> out you didn't really need Postgresql after all and the
>> cost turned out to be higher than you expected.
> 
> Completely not true! For the reasons mentioned above. You are
> concentrating too much on the RDBMS aspects and completely forgetting
> about the implications in your codebase.
> 
> Whether you move from a non d

Re: 'Lite' Databases (Re: sqlite3 and dates)

[rurpy]

On 02/19/2015 01:47 PM, Chris Angelico wrote:
> On Fri, Feb 20, 2015 at 7:26 AM,   wrote:
>> I'll point out that five people in this thread (by my
>> count) have said that Postgresql requires a significant
>> amount of work to setup and use.  Only you and Steven claim
>> the opposite.  (And it sounds to me like Steven does not
>> have a lot of experience installing, configuring and
>> maintaining Postgresql -- please correct me if I am wrong
>> Steven.)
> 
> I don't know about Steven, but I do have a certain amount of
> experience installing, configuring, and maintaining;

Right.  I am aware of that as you might have guessed by the 
fact that I directed my question to Steven and not to you. :-)

> [...] the most common changes I've needed to make to Postgres
configs are:
>[...]
> 2) Increasing logging, eg turning on the logging of all queries. Not
> particularly common, but can be handy; and it's a trivial edit anyway.

It may be trivial to you but it is not going to be trivial to
many of end users the kind of apps we're talking about.  One
of the people interested in my little app was at the limits of her
ability when it came to finding a PowerPoint file she'd just been
editing so she could attach it to an email.  So if the logging 
level needs changing, the app better be prepared to do it itself.

>[...]
> So while it makes sense to talk about *experience* "installing,
> configuring, and maintaining", it's really just the installing part
> that actually demands effort. 

No, see below.

> And that can be mitigated in a few ways,
> but ultimately, there's no single simple solution for everything.

That's right but misses the obvious: some solutions are simpler 
than others.  Installing a Postgresql client-server database and 
a Python application is inherently more complex than installing 
a Python application (Sqlite being included in Python).  (Doesn't
that seem rather obvious, even to you?)

And, if using Postgresql is as easy as you claim, surely you 
don't expect to reserve its use only to your applications?  
So we could expect to see many applications, potentially all 
those currently using Sqlite, to start using Postgresql.

Do they all install and run their own independent server?  Each
with its own set of more than a half dozen processes running all
the time?  Each that has to be started up at system boot?  Each
with their own directory of Postgresql software and backend data
directory?  Where do these files all go and how do app developers 
agree on some standard to avoid chaos?  How does each app find a 
network port or socket that doesn't conflict with the others or 
with things the machine owner may run?  For a Sqlite app you are
installing files for which the rules are all pretty well established.

And if the Postresql apps share a server who arbitrates when 
app X decides it needs to set some parameter that breaks app Y?
Or change anything else on the server; things shared in common
by all the apps like users or (as you yourself mentioned) log 
file settings?

The complexities you deny exist do exist and have to be dealt 
with somewhere.  Either you, the developer have to handle them
in the installation and anticipate problems like "Install error:
/var/lib/pgsql/data: already exists" or "data directory contain 
wrong version of Postresql" or you shove the task off onto your 
users in which case they will have to have sufficient expertise 
to deal with them.

Install problems are not of course limited to server software,
they occur with simple Python apps too.  But installing a single
Python app that depends on the version of Sqlite in the minimally
supported version of Python from a self-contained installer will 
have far lower probability of problems than an automated install 
of server software in a very uncontrolled environment.  And whatever
that app install failure probability is, requiring server software 
in addition can only increase it.

> Even
> SQLite3 has issues with that - as soon as you have two programs that
> try to read the same file (eg your application, using Python's module,
> and the sqlite3 command line tool), you have version differences, and
> quite a number of my students have come to me asking why on earth they
> can't upgrade their SQLite3 properly. (Though usually both versions
> are "new enough", so it doesn't matter. It's just a point of
> confusion.)

Irrelevant.  As the application designer you are responsible for 
what you provide (the application and its backend database).  If 
I want to obtain a command line tool somewhere, it's my problem 
to worry about versions -- it has nothing to do with you.

If you mean that you as a developer have to worry about not shipping
two incompatible versions of app code, well... yeah.  I think that's
pretty much a standard part of putting together a working application.

Or is there some other point I missed?

As for no maintenance, I mentioned two common maintenance activities 
in my reply to Steven: backups and server u

Re: python implementation of a new integer encoding algorithm.

[Denis McMahon]

On Tue, 17 Feb 2015 03:22:47 -0800, janhein.vanderburg wrote:

> In http://optarbvalintenc.blogspot.nl/ I propose a new way to encode
> arbitrarily valued integers

I'm not quite sure I understand the problem that you're trying to solve 
with this.

If I want to transmit some arbitrarily huge integer value between two 
systems there are several ways to do it:

Numeric text - roughly 24 bits are used to carry 10 bits of value

Alphanumeric text - 8 bits are used to carry every 4 bits of value

A bit sequence of length n bits where 2^n > the value I wish to convey, 
providing that there is some mechanism by which the sender can tell the 
receiver "the next n bits represent a binary integer." It might be 
desirable to pad the bit stream to a byte boundary. Assuming an 8 bit 
clean transmission path and a maximum padding of 7 bits, this is 
increasingly efficient as the integer being transmitted gets larger.

The thing is, for all practical purposes, any integer that is capable of 
being processed as an integer by a computer is probably already being 
stored in a binary format in a storage space of n bytes, where n is a 
power of 2. Very little additional processing should be needed to 
packetize those n bytes and transmit them.

Additionally, you're talking about efficiency and the need to optimise, 
but you're using a scripted language. If you need a highly efficient 
protocol to transfer binary numbers between two systems with minimum 
wasted bits and maximum cpu and memory efficiency, python really isn't 
the language in which to solve your problem.

Perhaps it's time to take a step back and redefine the problem a bit more 
clearly, because at the moment I'm not sure you're solution will ever 
solve anything that needs solving.

-- 
Denis McMahon, denismfmcma...@gmail.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Python - parsing nested information and provide it in proper format from log file

[Jay T]

 have some log file which has nested data which i want to filter and provide 
specific for student with total counts

Here is my log file sample: 
Student name is ABC 
Student age is 12 
student was late 
student was late 
student was late 
Student name is DEF 
student age is 13 
student was late 
student was late

i want to parse and show data as Student name, student age , number of counts 
how many times student was late e:g 
Name Age TotalCount 
ABC 12   3 
DEF 132

Please help me with solution that will be really grateful.

thanks, Jt
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Mario Figueiredo]

On Thu, 19 Feb 2015 12:26:04 -0800 (PST), ru...@yahoo.com wrote:
>
>I'll point out that five people in this thread (by my 
>count) have said that Postgresql requires a significant 
>amount of work to setup and use.  Only you and Steven claim
>the opposite. 

Well, I claim the opposite too.

>Or I could have tried as you 
>and Steven suggest to "somehow" package Postgresql in my 
>app installer.  That would have been an even bigger cost 
>in my time with an uncertain outcome.

I don't see how. You said it was just a simple application this
postgres database was serving. So I must assume you aren't talking of
a complex setup that database synchronization or other features that
force a manual setup.

For your purposes, you could just basically backup your data folder
and package postgres with the same configuration files you have in
your current computer.

>None of those 
>costs would have been necessary at all had I developed 
>a self-contained Sqlite app.

The cost would have probably been much higher, depending on your
project. SQLite would have forced you to move all your business logic
into your code, greatly increasing code maintenance, your application
extensibility and its ability to more easily adapt to new business
requirements.

Conversely, if none of this is true concerning your particular
project, then you just chose the wrong tool. Postgres was overkill for
your particular needs and it was a mistake to think you need it to
function just as a shelve on steroids.

>
>Finally keep in mind that if you develop your app using
>Sqlite, it is likely to be far easier to migrate to
>a heavy-duty backend like Postgresql later should you 
>need to than to go in the other direction when you find 
>out you didn't really need Postgresql after all and the 
>cost turned out to be higher than you expected.

Completely not true! For the reasons mentioned above. You are
concentrating too much on the RDBMS aspects and completely forgetting
about the implications in your codebase.

Whether you move from a non distributed model to a client-server
model, or the other way around, your code will suffer major changes.
And even if you decide to keep the business logic in the client layer
(which is a mistake) when moving from SQLite to a client-server RDBMS,
you will still have to deal with a whole new set of issues regarding
the very nature of concurrent access that will essentially force you
to scrap much of your previous code.

>It is bad advise to recommend using Postgresql without 
>regard to the developer's actual needs.

Naturally. But I must say postgres isn't the bad cat you painted in
your post. It's much, much easier to distribute, deploy and manage
than you are suggesting.

For most systems where performance and database synchronization aren't
a requirement, it can be entirely automated, I know, because that's
how we had it set up on three schools where we sold our integrated
management system. I haven't had a maintenance request call in 8
months.

It's only under critical requirements that postgres necessitates a
baby sitter. And those projects don't suffer from lack of competent
administrators.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[rurpy]

On Wednesday, February 18, 2015 at 10:39:04 PM UTC-7, Ethan Furman wrote:
> On 02/18/2015 09:26 PM, memilanuk wrote:
> > On 02/18/2015 09:16 PM, Ben Finney wrote:
> >> memilanuk  writes:
> >>
> >>> In the past I've been waffling back and forth between a desktop
> >>> client/server setup, or a web-based interface with everything on one
> >>> computer. At this point I'm leaning toward the latter.
> >>
> >> So, it's been many exchanges back and forth, and you still aren't
> >> telling us what specific needs you have that SQLite can't provide. At
> >> this point I'm just going to have to wait until you can lay out the
> >> specifics.
> >>
> > 
> > Okay, let me put it like this:  if I set up a web interface using Flask for 
> > the front-end, and SQLite as the backend DB,
> > running from a PC/laptop, with anywhere from 1 to 10 people doing data 
> > entry from other devices (laptops, desktops,
> > tablets, etc.) at roughly the same time, is SQLite going to be 'concurrent' 
> > enough?
> 
> Well, having zero experience with SQLite, but having read the docs just today 
> [snip snide remark] -- I think you'll be
> fine with SQLite under those conditions.  :)

I too agree with this (with a similar caveat -- I've only 
used Sqlite for playing around and have no real experience
using it in real-world applications).  

Another thing to keep in mind is if you do need find you
need a more heavy duty backend, it will be easier to migrate
from Sqlite to Postgresql (or whatever) [*] than to go the 
other way should you later find Postgresl too heavy-weight 
and costly.


[*] Unless you are very careful to use only features in 
Postgresql that you've determined can be easily migrated 
back to Sqlite.  In practice very few people have the 
strength of character required to do this. :-)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A question about how plot from matplotlib works

[Jason Swails]

On Thu, Feb 19, 2015 at 5:47 AM, ast  wrote:

> Hello
>
>  import numpy as np
 import matplotlib.pyplot as plt
 x = np.arange(10)
 y = x**2
 x

>>> array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
>
>> y

>>> array([ 0,  1,  4,  9, 16, 25, 36, 49, 64, 81])
>
>> plt.plot(x,y)

>>> []
>
>> plt.show()

>>>
>
> The question is:
>
> plt.plot() creates an object "matplotlib.lines.Line2D" but this object is
> not referenced. So this object should disapear from memory. But
> this doesn't happens since plt.show() draws the curve on a graphic
> window. So how does it work ?


​A reference to it is put in the "active" Axes instance of the
matplotlib.pyplot namespace.  There are many things that will prevent an
object from being garbage-collected (a common source of references are
caches). [1]

​In general, matplotlib has many containers.  In particular, Line2D objects
generated by the "plot" function are added to the Axes instance from which
"plot" was called.  When you don't explicitly specify an Axes object from
which to plot, matplotlib.pyplot applies it to some "default" Axes instance
living in the matplotlib.pyplot namespace.​

This is done to give matplotlib more of a Matlab-like feel.  To demonstrate
this, let's go try and FIND that reference to the lines:

>>> import matplotlib.pyplot as plt
​>>> import numpy as np
​>>> x = np.arange(10)
​>>> y = x ** 2
​>>> x
​array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
​>>> y
​array([ 0,  1,  4,  9, 16, 25, 36, 49, 64, 81])
​>>> lines, = plt.plot(x, y)
​>>> id(lines)
​4466622800
​​
>>> lines
​
​>>> del lines
​>>> # Now let's find those lines
​... active_axes = plt.gca() # Get Current Axes
​>>> dir(active_axes)
​[..., get_lines, ...] <-- this is snipped for brevity
​>>> active_axes.get_lines()
​
​>>> active_axes.get_lines()[0]
​
​>>> id(active_axes.get_lines()[0])
​4466622800

And there we have it!  Success!  (Note, my comment indicates that the gca
in plt.gca() stands for "Get Current Axes").  I also snipped the list of
attributes in active_axes that I got from the "dir" command, since that
list is HUGE, but the method we want is, rather expectedly, "get_lines".

In *my* personal opinion, the matplotlib API is quite intuitive, such that,
coupled with Python's native introspective functions (like dir() and id())
and "help" function in the interpreter, I rarely have to consult
StackOverflow or even the API documentation online to do what I need.

For instance, you want to change the color or thickness of the error bar
hats on error bars in your plot?  Either save a reference to them when they
are generated (by plt.errorbar, for instance), or go *find* them inside the
Axes you are manipulating and set whatever properties you want.

Hope this helps,
Jason

[1] OK, so there are not many *things* -- only if there are active,
non-circular references will the object *not* be garbage-collected, loosely
speaking.  But there are many reasons and places that such references are
generated inside many APIs... caching being one of the most popular.

-- 
Jason M. Swails
BioMaPS,
Rutgers University
Postdoctoral Researcher
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Chris Angelico]

On Fri, Feb 20, 2015 at 7:26 AM,   wrote:
> I'll point out that five people in this thread (by my
> count) have said that Postgresql requires a significant
> amount of work to setup and use.  Only you and Steven claim
> the opposite.  (And it sounds to me like Steven does not
> have a lot of experience installing, configuring and
> maintaining Postgresql -- please correct me if I am wrong
> Steven.)

I don't know about Steven, but I do have a certain amount of
experience installing, configuring, and maintaining; and the biggest
thing I've noted about the "configuring" part is that you can ignore
it most of the time. The docs caution you that the defaults are
deliberately NOT set for maximum performance... but you won't normally
need maximum performance initially anyway. (In any case, "maximum
performance" requires a lot of knowledge of workloads, resource
availability, concurrency model, etc.) The defaults are usually good
enough; the most common changes I've needed to make to Postgres
configs are:

1) Weakening the default security model of listening only on
localhost, to allow other computers on the network to connect to the
database. Obviously it's correct for the default to be secure, here;
anyone who wants to allow remote access will need to be aware of what
s/he is doing anyway, so requiring that a couple of files be edited
isn't a big deal.

2) Increasing logging, eg turning on the logging of all queries. Not
particularly common, but can be handy; and it's a trivial edit anyway.

3) Permitting certain special-purpose users to bypass the normal login
mechanisms, eg for scripted backups. Similarly to #1, the defaults are
absolutely correct, and if you want to weaken your security, it's only
fitting that you have to go in and edit some configs.

Given that I frequently don't need to do _any_ of these, it's safe to
say that configuring PostgreSQL doesn't take a lot of effort at all.
Similarly, "maintaining" isn't really a thing; apart from maintaining
my own data tables (cleaning out junk tables that were for one quick
test, which I often forget to drop when I'm done with them), I don't
have to spend any effort making sure the database keeps running. (In
contrast, a busy MySQL server probably needs to have myisamchk run on
it periodically, which *does* count as maintenance.)

So while it makes sense to talk about *experience* "installing,
configuring, and maintaining", it's really just the installing part
that actually demands effort. And that can be mitigated in a few ways,
but ultimately, there's no single simple solution for everything. Even
SQLite3 has issues with that - as soon as you have two programs that
try to read the same file (eg your application, using Python's module,
and the sqlite3 command line tool), you have version differences, and
quite a number of my students have come to me asking why on earth they
can't upgrade their SQLite3 properly. (Though usually both versions
are "new enough", so it doesn't matter. It's just a point of
confusion.)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[rurpy]

On 02/19/2015 12:23 AM, Chris Angelico wrote:
> On Thu, Feb 19, 2015 at 6:07 PM, Steven D'Aprano 
>  wrote:
>> Very possibly. With modern dependency management, it isn't hard to install
>> Postgresql:
>>
>> sudo aptitude postgresql
>>
>> or equivalent should work. For primitive operating systems with no
>> dependency management available, Firefox could come with a simple script
>> which downloads, installs, configures and runs Postgresql. (Everything is
>> simple for the guy who doesn't have to do it.)
> 
> Definitely a possibility. I'm pretty sure I've seen that exact thing
> done by one application (on Windows; on a typical Linux system,
> that'll be done by simple dependency management - your package
> metadata says "depends on postgresql" and apt-get or yum or whatever
> will do the work), and it's quite common for a Windows installer
> wizard to go through a series of subcomponents (grab the .NET
> framework, grab these hotfixes that the program depends on, grab some
> adware toolbar that you forgot to untick, and *then* install the
> program you asked for).
> 
>> Possible snags:
>>
>> - Possibly postgresql is simply *too big*. Your 50MB(?) Firefox
>>turns into a 2GB install. I doubt it -- on Debian, postgresql
>>is 17MB installed. But I don't know what dependencies I'm not
>>counting.
> 
> Yeah. And frankly, I would be surprised if Firefox is only 50MB these
> days. The extra dent of PostgreSQL won't be all that significant - and
> don't forget that the SQLite3 dent can be removed, so you're talking
> about the difference between them, plus you can omit a whole bunch of
> PG's ancillaries.
> 
>> - Or it is impossible to configure without excessive amounts of
>>tech-savvy human intervention. Again, I doubt it. I seem to
>>recall needing to create a Postgresql user and password. But
>>maybe even that is too technical for the average Firefox user.
> 
> You don't even need to do that. An absolutely default Postgres on
> Debian or Ubuntu is ready to use, with peer authentication. If you can
> become root, you can then drop privs to the 'postgres' user and
> connect to the database that way.
> 
> I don't know if it's possible to do a non-root installation of
> PostgreSQL, but if it isn't today, it could easily be tomorrow, if
> someone puts in a little effort. You'd miss out on boot-time startup,
> and it'd probably have to do some password-based authentication (with
> autogenerated passwords), but it certainly could work. The Windows
> equivalent ("install for just me") is, I think, already possible.
> 
>> - Maybe there are nasty interactions between Postgresql listening
>>on some port and Windows firewall wanting to block that same port.
> 
> That's definitely an issue, given that Windows doesn't have Unix
> domain sockets. But I'm sure it's a solvable problem. How does IDLE
> cope with firewall issues?

There's an awful lot of "could"s, "possible"s etc in your 
and Steven's responses.  And most of those words apply to
issues that aren't problems at all when using Sqlite.

I'll point out that five people in this thread (by my 
count) have said that Postgresql requires a significant 
amount of work to setup and use.  Only you and Steven claim
the opposite.  (And it sounds to me like Steven does not 
have a lot of experience installing, configuring and 
maintaining Postgresql -- please correct me if I am wrong 
Steven.)

I've used Postgresql for a number of small to medium size 
projects for work and personal use.  I too think it is an 
amazing piece of work for free  software.  Consequently I 
used it for a small educational app I developed for my 
personal use.  A number of friends saw it, liked it and 
asked about using it themselves.  Unfortunately none of 
these friends were at all technical and the barrier of 
installing and configuring Postresql would have involved 
me in trying to talk them through it on the phone (risky 
as there were OSs I was unfamiliar with involved) or do 
it myself in person (one of them was 2000mi away) or do 
it remotely (again a cost in coming up with and installing 
remote desktop software on different OSs and non-technical 
doubts about its use ("I don't mind you doing this while 
I'm siting beside you but remote access is a different 
story").  Then there are the ongoing maintenance issues 
I mentioned in my last reply to Steven.

None of these would have been impossible to overcome with
more effort and expense.  Or I could have tried as you 
and Steven suggest to "somehow" package Postgresql in my 
app installer.  That would have been an even bigger cost 
in my time with an uncertain outcome.  None of those 
costs would have been necessary at all had I developed 
a self-contained Sqlite app.

Finally keep in mind that if you develop your app using
Sqlite, it is likely to be far easier to migrate to
a heavy-duty backend like Postgresql later should you 
need to than to go in the other direction when you find 
out you didn't really need Postgr

Re: 'Lite' Databases (Re: sqlite3 and dates)

[rurpy]

On 02/19/2015 12:07 AM, Steven D'Aprano wrote:
> ru...@yahoo.com wrote:
>> On 02/18/2015 07:13 PM, Steven D'Aprano wrote:
>>> Chris Angelico wrote:
> SQLite misses some important features that makes it better suited as a
> simple datastore, not much unlike shelve. And network use is not one
> of them, since you can actually implement concurrent sqlite access by
> coding an intermediate layer. Assuming of course we are talking about
> a small number of concurrent users.

 This is what I was saying: it's fine for purposes like Firefox's
 bookmarks and settings and such (which I think was what it was
 originally developed for?). Not so fine over a network.
>>>
>>> The sheer number of Firefox bugs related to its use of SQLite says
>>> different.
>>>
>>> Once upon a time, Firefox's config, bookmarks, etc. were stored in plain
>>> text files. At worst they were HTML. You could trivially read them, copy
>>> them, restore them and even (if you were careful) edit them using the
>>> text editor of your choice. Many a time I was on one machine, wanted to
>>> know a bookmark from another machine, so I would ssh across to the other
>>> machine and run grep over the bookmark file.
>>
>> I agree, I prefer plain text files whenever practical.  But since
>> the original discussion was about Sqlite vs Postgresql, not Sqlite
>> vs text files, shouldn't the question be: would Firefox be better
>> if it required you to install and configure Postgreql instead of
>> using Sqlite?
> 
> Very possibly. With modern dependency management, it isn't hard to install
> Postgresql:
> 
> sudo aptitude postgresql
> 
> or equivalent should work.

And the equivalent for Android would be?  Even with Windows, 
Mac and Linux I don't imagine they want to be in the business 
of tracking what package managers are used by what OSes and 
OS version and dealing with the inevitable problems that
will arise, particularly with a userbase that is 99.9% non-
technical.

Compare with Sqlite which require zero end-user involvement 
and puts the management in one place, in-house, and under 
control of the developers.

> For primitive operating systems with no
> dependency management available, Firefox could come with a simple script
> which downloads, installs, configures and runs Postgresql. (Everything is
> simple for the guy who doesn't have to do it.)

Right.  The key is in the last sentence above.  

> Possible snags:
> [...]
> 
> - Or it is impossible to configure without excessive amounts of
>tech-savvy human intervention. Again, I doubt it. I seem to
>recall needing to create a Postgresql user and password. But
>maybe even that is too technical for the average Firefox user.

You want hands-off for non-technical users and hands-on for
technical ones.  The last thing I want is yet another database 
server running -- if I want FF to use Postgresql, I want it 
to use a database in my existing server for which I already 
have management and backup/recovery procedures established.
So now you have to ask me if I want to use an existing server 
or not and if so what its connection details are.  And this is 
not per-install but per profile creation.

My point is not that this is an insoluble problem -- just that 
it is one of many such problems that take some non-trivial amount 
of time recognize and to address -- time which is not necessary 
when using Sqlite.  

> - Maybe there are nasty interactions between Postgresql listening
>on some port and Windows firewall wanting to block that same port.

The attack surface for any application that has a network port
is vastly greater that one that uses only filesystem apis like
sqlite.  So you've now just undertaken to be responsible to a 
much greater degree for the security of my machine.  FF does 
already have a bucketful of security issues but as a client, 
not as a server.

And what about backing up all that data in the wonderful, 
featureful database you just installed for me?  As you know, 
you can't just include the postresql data files in a file 
system backup.

And what about upgrades to the server?  Are you going to develop
your own upgrade infrastructure, or are you going to tell me to
use the standard Postgresql upgrade methodology (which involves,
in simple cases, dumping the data with pg_dumpall, doing the
upgrade, and restoring with pg_restore_all).  If the latter,
I'm sure that will be really popular with non-technical users.
Of course with sqlite, you just ship a new shared library file 
with your new version of Firefox. 

> Or... and here is a radical thought... maybe Firefox could give you the
> choice of which database? By default, it might use Sqlite, to satisfy the
> desktop users who don't want to think about it. And for those who are
> disturbed by the fragility of Sqlite on a network home directory, you just
> set a config setting in about:config to point at your existing Postgresql
> instance, and never need worry about it again.

Yes, I 

Re: How to install PIL or PILLOW on OS X Yosemite?

[Russell Owen]

On 2/15/15 8:17 PM, Ned Deily wrote:

In article ,
  KP  wrote:

just upgraded my Mac Mini to Yosemite and have never dabbled in Python on
this OS.

I see it has Python 2.7.6 installed.

When I do something like

from PIL import ImageFont, ImageDraw

it tells me that it cannot find PIL

How do I install this on Yosemite?


Suggestions: stick with Pillow which is the current, maintained fork of
the venerable PIL. Decide whether you want to use Python 3 or Python 2.
PIL/Pillow installation on OS X is more involved than on some other
platforms because it depends on a number of third-party C libraries that
are not shipped by Apple in OS X so you need to find another source for
them.  Rather than trying to build and install everything yourself or
downloading a Pillow or PIL installer, I suggest picking one of the
several fine distributors of open source packages for OS X and
installing everything you need from them (including an up-to-date Python
2 or 3) and for your future needs beyond Pillow; options include
Homebrew, MacPorts, Anaconda, Fink, and others.  Once you've installed
the base framework for the package manager you choose, installing
something like Pillow and all of its dependencies is often just a
one-line command.  It may take a little while to get used to the quirks
of the package manager you choose but, if you are going to use OS X for
development with Python or many other languages, that time spent will be
repaid many times over.


I agree that Pillow is preferable to PIL and that you may want to 
consider a 3rd party system.


If you are primarily interested in Python (and not unix-based C/C++ 
libraries and utilities then I suggest you try anaconda python.


Homebrew, MacPorts and Fink are mostly aimed at people who want to add 
missing unix libraries and tools.


If you want to stick with python.org python then a binary PIL installer 
is available here:


(I am not aware of any Pillow binaries).

-- Russell

--
https://mail.python.org/mailman/listinfo/python-list

Re: Accessible tools

[Tim Chase]

While not blind, I have an interest in accessibility and answer a
number of questions on the Blinux (Blind Linux Users) mailing list.

On 2015-02-19 08:33, Bryan Duarte wrote:
> A professor and I have been throwing around the idea of developing
> a completely text based IDE. There are a lot of reasons this could
> be beneficial to a blind developer and maybe even some sighted
> developers who are comfortable in the terminal. The idea would be
> really just to provide a way of easily navigating blocks of code
> using some kind of tabular formatting, and being able to collapse
> blocks of code and hearing from a high level information about the
> code within. All tools and features would obviously be spoken or
> output in some kind of audio manor. 

It would seem that the traditional Unix-as-IDE[1] would serve you well
here.  This is my method of choice, and it allows me to pick my
components and combine them.  I usually use tmux, though GNU screen
would do too.  Within that, I usually have the following:

- vim to edit my code.  Though swap in your favorite, whether
  emacs/emacspeak, ed/edbrowse, joe, nano, or whatever.  I know that
  at least Vim and emacs support "folding" away blocks of code (what
  you describe as "collapsing") which I usually prefix with a comment
  that would give you a description of the block

- a command-line (I use bash, some prefer zsh or tcsh or whatever)
  for things like version-control, running my code, and file
  management (move/copy/delete/rename/link/etc)

- a Python command-line REPL that allows me to do quick tests on a
  line of code as well as well as make extensive use of Python's
  built-in dir() and help() commands which are invaluable.

- when doing web-development (Django in my case), I'll often have the
  dev-server running in one pane, and a console browser like
  lynx/links/links2/elinks/w3m in another pane so that I can put my
  code through its paces

Another benefit of this is that I can run this on my development
machine, but then SSH into the machine from anywhere, reattach to the
tmux/screen session, and have the same configuration right as I left
it.

The entire tmux/screen session can be run within an accessible
terminal window (I know that some are more accessible than others),
within a terminal screen-reader session (like yasr, screader, or
emacspeak), or even remoted into via an accessible SSH program on your
platform of choice.

-tkc

[1]
http://blog.sanctum.geek.nz/series/unix-as-ide/





-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What behavior would you expect?

[Chris Angelico]

On Fri, Feb 20, 2015 at 7:03 AM, Denis McMahon  wrote:
> On the one hand, the return type of a function (when it returns, rather
> than raising an exception) should be consistent to itself, even if using
> a language where types are not declared.
>

Yes, so I'd advise against having a function sometimes return a string
and sometimes a dict. Sure. But None gets a bit of a special pass
here; in a lot of languages, what you'd have is a "nullable" type (eg
in C you might declare the return value as "pointer to char", and
either return a pointer to a string, or NULL), but in Python, you
return None for the special case. So in a sense, "str or None" is not
so inconsistent.

> I guess at the end of the day the programmer has to consider and
> determine which is most appropriate to his application, given the case.

Indeed. But knowing that your caller quite possibly won't check, what
would you do, to increase the chances of the unexpected being noticed?
Again, it comes down to expectations. If 25% of queries are going to
return "nothing found", then having that be the empty string is fine -
you can be pretty sure your users will test for it. But if that's an
extremely rare case, then it may be worth raising an exception
instead, so it's safe even if someone forgets to test for the unusual.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What behavior would you expect?

[Denis McMahon]

On Fri, 20 Feb 2015 02:08:49 +1100, Chris Angelico wrote:

> On Fri, Feb 20, 2015 at 1:16 AM, Denis McMahon
> 
> wrote:
>>> 2. no files match the given pattern
>>
>> Return either None, 0, False or an empty string.
>>
>> In both cases, it is then a matter for the calling code to catch the
>> exception or handle the return value appropriately.
> 
> I'd avoid the empty string here, because "absence of file" should be
> very different from "first file matching pattern". Imagine this naive
> code:

If your function documentation states that a failure to match any 
existing file returns an empty string (and if that's the case, then the 
documentation should do), then whoever calls the function should check 
the return value isn't an empty string.

There's two conflicting "paradigms" as it were here.

On the one hand, the return type of a function (when it returns, rather 
than raising an exception) should be consistent to itself, even if using 
a language where types are not declared.

On the other hand, a failure condition should clearly be a failure 
condition, which for a language that supports untyped functions means 
that we have the luxury of being able to return None / Nul[l] / NaN / 
False / 0 etc instead of a string / object / array / list / dictionary / 
mapping etc instead of raising an exception for the failure, and so we 
can try and handle the failure at the calling level without worrying 
about trapping exceptions.

I guess at the end of the day the programmer has to consider and 
determine which is most appropriate to his application, given the case.

As an aside, it's always a good idea to check that what you get looks 
like what you expected, whether it comes from a function call or as a 
data packet over the internet, before you start using it to do other 
things. ;)

-- 
Denis McMahon, denismfmcma...@gmail.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Marko Rauhamaa]

Terry Reedy :

> It isn't, except that Guido gets a special pass

Wusses...

Or, it's good to be king.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Terry Reedy]

On 2/19/2015 3:36 AM, Marko Rauhamaa wrote:


PS On the topic of pointlessness, why is top-posting the norm on
python-dev ... ?


It isn't, except that Guido gets a special pass and some of the posters 
travel a lot and read and reply on phones, which makes snipping and 
inline response difficult, and so they top post even though it makes 
their responsed *much* harder to understand.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[rurpy]

On 02/19/2015 09:03 AM, Tim Chase wrote:
> On 2015-02-19 15:04, Mark Lawrence wrote:
>> On 19/02/2015 14:17, Tim Chase wrote:
> Parameterized queries is just a pet peeve of mine that I wish to
> include here. SQLite misses it and I miss the fact SQLite misses
> it. The less SQL one needs to write in their code, the happier
> one should be.

 Instead, use the DB-API's parameter substitution. Put ? as a
 placeholder wherever you want to use a value, and then provide a
 tuple of values as the second argument to the cursor's execute()
 method. (Other database modules may use a different placeholder,
 such as %s or :1.) For example:..."
>>>
>>> I think Mario was referring to what other back ends call prepared
>>> statements.
>>
>> Is this
>> https://docs.python.org/3/library/sqlite3.html#sqlite3.Cursor.executemany
>> an equivalent?
> 
> Depends on whether sqlite3.Cursor.executemany() uses
> 
> https://www.sqlite.org/c3ref/stmt.html
> 
> under the hood.

So it seems that Sqlite does have prepared statements -- they 
are just accessible from the api and not from SQL.

Regarding Mark's question I would say that a more significant
difference is that executemany() requires you to know all 
the bind parameter values at the time the statement is 
executed.

A prepared statement does not require you to know any of 
the bind parameter values when you prepare the statement -- 
they are supplied to the prepared statement at any later 
time or times when you execute the prepared statement.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Tim Chase]

On 2015-02-18 20:05, ru...@yahoo.com.dmarc.invalid wrote:
> Sqlite offers concurrent access already.  
> What Sqlite doesn't offer is high performance concurrent write
> access.  That is, it locks the entire database for the duration 
> of a write operation.  Given that most such operations are pretty
> short, for a small number of concurrent writers this is not a 
> big problem.

Though the entire-database-lock, as Steven mentions in another
section of this thread, causes issues on network-shared file-systems.

-tkc




-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Chris Angelico]

On Fri, Feb 20, 2015 at 5:41 AM, Dave Angel  wrote:
> As I (and others) have said many times, making it optimal means making some
> assumptions about the distribution of likely values.

In fact, the very word "optimal" implies that. You have to have a set
of criteria on which you base your evaluation, and those criteria will
include target data.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Dave Angel]

On 02/19/2015 01:32 PM, Ian Kelly wrote:

On Thu, Feb 19, 2015 at 11:24 AM, Dave Angel  wrote:

Here's a couple of ranges of output, showing that the 7bit scheme does
better for values between 384 and 16379.

382 2 80fe --- 2 7e82
383 2 80ff --- 2 7f82
384 3 81 --- 2 0083
384  jan grew 3 81
385 3 810001 --- 2 0183
386 3 810002 --- 2 0283
387 3 810003 --- 2 0383
388 3 810004 --- 2 0483
389 3 810005 --- 2 0583

16380 3 813e7c --- 2 7cff
16380  jan grew 3 813e7c
16380 7bit grew 2 7cff
16381 3 813e7d --- 2 7dff
16382 3 813e7e --- 2 7eff
16383 3 813e7f --- 2 7fff
16384 3 813e80 --- 3 81
16384 7bit grew 3 81
16385 3 813e81 --- 3 010081
16386 3 813e82 --- 3 020081
16387 3 813e83 --- 3 030081
16388 3 813e84 --- 3 040081
16389 3 813e85 --- 3 050081

In all my experimenting, I haven't found any values where the 7bit scheme
does worse.  It seems likely that for extremely large integers, it will, but
if those are to be the intended distribution, the 7bit scheme could be
replaced by something else, like just encoding a length at the beginning,
and using raw bytes after that.


It looks like you're counting whole bytes, not bits. That would be
important since the "difficult" encoding uses fractional bytes.



Not the implementation that was shared.  I've only seen one set of 
Python code for "difficult", and it was strictly bytes.  As i said 
earlier in the message you quoted from.


Naturally, I question whether the original description makes sense for 
sub-bytes, since it was claimed that these are NOT for lists or 
sequences of arbitrary integers, but only for a single one at a time. 
Presumably mixed with other data which may or may not like bit encoding.





--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Dave Angel]

On 02/19/2015 01:34 PM, Chris Angelico wrote:

On Fri, Feb 20, 2015 at 5:24 AM, Dave Angel  wrote:

In all my experimenting, I haven't found any values where the 7bit scheme
does worse.  It seems likely that for extremely large integers, it will, but
if those are to be the intended distribution, the 7bit scheme could be
replaced by something else, like just encoding a length at the beginning,
and using raw bytes after that.


Encoding a length (as varlen) and then using eight bits to the byte
thereafter is worse for small numbers,


I only suggested this if it turns out that the distribution is primarily 
extremely large numbers, large enough that 7bit isn't good enough.


As I (and others) have said many times, making it optimal means making 
some assumptions about the distribution of likely values.



breaks even around 2**56, and
then is better. So unless your numbers are mainly going to be above
2**56, it's better to just use varlen for the entire number. On the
other hand, if you have to stream this without over-reading (imagine
streaming from a TCP/IP socket; you want to block until you have the
whole number, but not block after that), it may be more efficient to
take the length, and then do a blocking read for the main data,
instead of a large number of single-byte reads. But on the gripping
hand, you can probably just do those one-byte reads and rely on (or
implement) lower-level buffering.

Ask not the python-list for advice, because they will say both "yes"
and "no" and "maybe"... because they will say all three of "yes",
"no", "maybe", and "you don't need to do that"... erm, AMONG our
responses will be such diverse elements as...

ChrisA




--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Ian Kelly]

On Thu, Feb 19, 2015 at 11:24 AM, Dave Angel  wrote:
> Here's a couple of ranges of output, showing that the 7bit scheme does
> better for values between 384 and 16379.
>
> 382 2 80fe --- 2 7e82
> 383 2 80ff --- 2 7f82
> 384 3 81 --- 2 0083
> 384  jan grew 3 81
> 385 3 810001 --- 2 0183
> 386 3 810002 --- 2 0283
> 387 3 810003 --- 2 0383
> 388 3 810004 --- 2 0483
> 389 3 810005 --- 2 0583
>
> 16380 3 813e7c --- 2 7cff
> 16380  jan grew 3 813e7c
> 16380 7bit grew 2 7cff
> 16381 3 813e7d --- 2 7dff
> 16382 3 813e7e --- 2 7eff
> 16383 3 813e7f --- 2 7fff
> 16384 3 813e80 --- 3 81
> 16384 7bit grew 3 81
> 16385 3 813e81 --- 3 010081
> 16386 3 813e82 --- 3 020081
> 16387 3 813e83 --- 3 030081
> 16388 3 813e84 --- 3 040081
> 16389 3 813e85 --- 3 050081
>
> In all my experimenting, I haven't found any values where the 7bit scheme
> does worse.  It seems likely that for extremely large integers, it will, but
> if those are to be the intended distribution, the 7bit scheme could be
> replaced by something else, like just encoding a length at the beginning,
> and using raw bytes after that.

It looks like you're counting whole bytes, not bits. That would be
important since the "difficult" encoding uses fractional bytes.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Chris Angelico]

On Fri, Feb 20, 2015 at 5:24 AM, Dave Angel  wrote:
> In all my experimenting, I haven't found any values where the 7bit scheme
> does worse.  It seems likely that for extremely large integers, it will, but
> if those are to be the intended distribution, the 7bit scheme could be
> replaced by something else, like just encoding a length at the beginning,
> and using raw bytes after that.

Encoding a length (as varlen) and then using eight bits to the byte
thereafter is worse for small numbers, breaks even around 2**56, and
then is better. So unless your numbers are mainly going to be above
2**56, it's better to just use varlen for the entire number. On the
other hand, if you have to stream this without over-reading (imagine
streaming from a TCP/IP socket; you want to block until you have the
whole number, but not block after that), it may be more efficient to
take the length, and then do a blocking read for the main data,
instead of a large number of single-byte reads. But on the gripping
hand, you can probably just do those one-byte reads and rely on (or
implement) lower-level buffering.

Ask not the python-list for advice, because they will say both "yes"
and "no" and "maybe"... because they will say all three of "yes",
"no", "maybe", and "you don't need to do that"... erm, AMONG our
responses will be such diverse elements as...

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Dave Angel]

On 02/19/2015 10:45 AM, janhein.vanderb...@gmail.com wrote:

On Wednesday, February 18, 2015 at 11:20:12 PM UTC+1, Dave Angel wrote:

I'm not necessarily doubting it, just challenging you to provide a data
sample that actually shows it.  And of course, I'm not claiming that
7bit is in any way optimal.  You cannot define optimal without first
defining the distribution.


Weird results.


In what way weird?


For a character size 2 the growth processes are shown below.
I listed the decimal representations, the difficult representation, a stop bit 
encoding, and the number of characters they differ in length:
0:  00  00  0
1:  01  01  0
2:  10, 00  10, 00  0
3:  10, 01  10, 01  0
4:  10, 10  11, 00  0
5:  10, 11  11, 01  0
6:  11, 00.00   11, 10, 00  0
7:  11, 00.01   11, 10, 01  0
8:  11, 00.10   11, 11, 00  0
9:  11, 00.11   11, 11, 01  0
10: 11, 01.00   11, 11, 10, 00  1
11: 11, 01.01   11, 11, 10, 01  1
12: 11, 01.10   11, 11, 11, 00  1
13: 11, 01.11   11, 11, 11, 01  1
14: 11, 10.00, 00   11, 11, 11, 10, 00  1
15: 11, 10.00, 01   11, 11, 11, 10, 01  1
16: 11, 10.00, 10   11, 11, 11, 11, 00  1
17: 11, 10.00, 11   11, 11, 11, 11, 01  1
18: 11, 10.01, 00.0011, 11, 11, 11, 10, 00  1
19: 11, 10.01, 00.0111, 11, 11, 11, 10, 01  1
20: 11, 10.01, 00.1011, 11, 11, 11, 11, 00  1
21: 11, 10.01, 00.1111, 11, 11, 11, 11, 01  1
22: 11, 10.01, 01.0011, 11, 11, 11, 11, 10, 00  2
23: 11, 10.01, 01.0111, 11, 11, 11, 11, 10, 01  2
24: 11, 10.01, 01.1011, 11, 11, 11, 11, 11, 00  2
25: 11, 10.01, 01.1111, 11, 11, 11, 11, 11, 01  2
26: 11, 10.01, 10.0011, 11, 11, 11, 11, 11, 10, 00  3

I didn't take the time to prove it mathematically, but these results suggest to 
me that the complicated encoding beats the stop bit encoding.



Clearly, one wouldn't use what you call "stop bit encoding" with a 2bit 
character.  And since the Python code you posted uses an 8bit character, 
I can't validate the "difficult" column either.


I wrote the following pair of functions:

def seven_code(n):
acc = bytearray()
if n == 0:
acc.append(0)
while n > 0:
quotient, remainder = divmod(n, 128)
acc.append(remainder)
n = quotient
acc[-1] |= 0x80#add a stop bit to last byte
return acc

def seven_decode(sequ):
acc = 0
multiplier = 1
for by in sequ:
acc += (by & 0x7f) * multiplier
if by > 0x7f:
break
multiplier *= 128
return acc

Here's a couple of ranges of output, showing that the 7bit scheme does 
better for values between 384 and 16379.


382 2 80fe --- 2 7e82
383 2 80ff --- 2 7f82
384 3 81 --- 2 0083
384  jan grew 3 81
385 3 810001 --- 2 0183
386 3 810002 --- 2 0283
387 3 810003 --- 2 0383
388 3 810004 --- 2 0483
389 3 810005 --- 2 0583

16380 3 813e7c --- 2 7cff
16380  jan grew 3 813e7c
16380 7bit grew 2 7cff
16381 3 813e7d --- 2 7dff
16382 3 813e7e --- 2 7eff
16383 3 813e7f --- 2 7fff
16384 3 813e80 --- 3 81
16384 7bit grew 3 81
16385 3 813e81 --- 3 010081
16386 3 813e82 --- 3 020081
16387 3 813e83 --- 3 030081
16388 3 813e84 --- 3 040081
16389 3 813e85 --- 3 050081

In all my experimenting, I haven't found any values where the 7bit 
scheme does worse.  It seems likely that for extremely large integers, 
it will, but if those are to be the intended distribution, the 7bit 
scheme could be replaced by something else, like just encoding a length 
at the beginning, and using raw bytes after that.



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Logging with Custom Levels not working

[Didymus]

On Wednesday, February 18, 2015 at 3:16:40 PM UTC-5, Ian wrote:
> > def perror(self, message, *args, **kws):
> > """ Performance Error Message Level """
> > # Yes, logger takes its '*args' as 'args'.
> > self._log(PERROR_NUM, message, args, **kws)
> >
> > logging.Logger.perror = perror
> 
> I think you need to call self.log, not self._log. The _log method
> appears to assume that the level check has already been performed. You
> really shouldn't be calling it directly anyway, as the leading _ is an
> indication that the method is not part of the public API.

Yes, I had tried that and get:

Logged from file log5.py, line 21
Traceback (most recent call last):
  File "/usr/lib64/python2.7/logging/__init__.py", line 851, in emit
msg = self.format(record)
  File "/usr/lib64/python2.7/logging/__init__.py", line 724, in format
return fmt.format(record)
  File "/usr/lib64/python2.7/logging/__init__.py", line 464, in format
record.message = record.getMessage()
  File "/usr/lib64/python2.7/logging/__init__.py", line 328, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting


I also added:

ll = rootLogger.getEffectiveLevel()

which send back the correct level, but it's still printing out everything...

  -Tom
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Ian Kelly]

On Thu, Feb 19, 2015 at 11:04 AM, Ian Kelly  wrote:
> There's also an optimization that can be added here if we wish to
> inject a bit of cleverness. Notice that all values with more than one
> group start with 11, never 10. We can borrow a trick from IEEE
> floating point and make the leading 1 bit of the mantissa implicit for
> all values greater than 3 (we can't do it for 2 and 3 because then we
> couldn't distinguish them from 0 and 1). Applying this optimization
> removes one full group from the representation of all values greater
> than 3, which appears to make the stop-bit representation as short as
> or shorter than the "difficult" one for all the values that have been
> enumerated above.

I made a mistake. This trick only works if the second group is 10, so
it only applies to half the numbers. That makes 12 and 13 a group
longer than the difficult representation, but on the other hand 18-23
are still a group shorter.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Ian Kelly]

On Thu, Feb 19, 2015 at 8:45 AM,   wrote:
> On Wednesday, February 18, 2015 at 11:20:12 PM UTC+1, Dave Angel wrote:
>> I'm not necessarily doubting it, just challenging you to provide a data
>> sample that actually shows it.  And of course, I'm not claiming that
>> 7bit is in any way optimal.  You cannot define optimal without first
>> defining the distribution.
>
> Weird results.
> For a character size 2 the growth processes are shown below.
> I listed the decimal representations, the difficult representation, a stop 
> bit encoding, and the number of characters they differ in length:
> 0:  00  00  0
> 1:  01  01  0
> 2:  10, 00  10, 00  0
> 3:  10, 01  10, 01  0
> 4:  10, 10  11, 00  0
> 5:  10, 11  11, 01  0
> 6:  11, 00.00   11, 10, 00  0
> 7:  11, 00.01   11, 10, 01  0
> 8:  11, 00.10   11, 11, 00  0
> 9:  11, 00.11   11, 11, 01  0
> 10: 11, 01.00   11, 11, 10, 00  1
> 11: 11, 01.01   11, 11, 10, 01  1
> 12: 11, 01.10   11, 11, 11, 00  1
> 13: 11, 01.11   11, 11, 11, 01  1
> 14: 11, 10.00, 00   11, 11, 11, 10, 00  1
> 15: 11, 10.00, 01   11, 11, 11, 10, 01  1
> 16: 11, 10.00, 10   11, 11, 11, 11, 00  1
> 17: 11, 10.00, 11   11, 11, 11, 11, 01  1
> 18: 11, 10.01, 00.0011, 11, 11, 11, 10, 00  1
> 19: 11, 10.01, 00.0111, 11, 11, 11, 10, 01  1
> 20: 11, 10.01, 00.1011, 11, 11, 11, 11, 00  1
> 21: 11, 10.01, 00.1111, 11, 11, 11, 11, 01  1
> 22: 11, 10.01, 01.0011, 11, 11, 11, 11, 10, 00  2
> 23: 11, 10.01, 01.0111, 11, 11, 11, 11, 10, 01  2
> 24: 11, 10.01, 01.1011, 11, 11, 11, 11, 11, 00  2
> 25: 11, 10.01, 01.1111, 11, 11, 11, 11, 11, 01  2
> 26: 11, 10.01, 10.0011, 11, 11, 11, 11, 11, 10, 00  3
>
> I didn't take the time to prove it mathematically, but these results suggest 
> to me that the complicated encoding beats the stop bit encoding.

That stop-bit variant looks extremely inefficient (and wrong) to me.
First, 2 bits per group is probably a bad choice for a stop-bit
encoding. It saves some space for very small integers, but it won't
scale well at all. Fully half of the bits are stop bits! Secondly, I
don't understand why the leading groups are all 11s and only the later
groups introduce variability. In fact, that's practically a unary
encoding with just a small amount of binary at the end. This is what I
would expect a 2-bit stop-bit encoding to look like:

0: 00
1: 01
2: 11, 00
3: 11, 01
4: 11, 10, 00
5: 11, 10, 01
6: 11, 11, 00
7: 11, 11, 01
8: 11, 10, 10, 00
9: 11, 10, 10, 01
10: 11, 10, 11, 00
11: 11, 10, 11, 01
12: 11, 11, 10, 00
13: 11, 11, 10, 01
14: 11, 11, 11, 00
15: 11, 11, 11, 01
16: 11, 10, 10, 10, 00
17: 11, 10, 10, 10, 01
18: 11, 10, 10, 11, 00
19: 11, 10, 10, 11, 01
20: 11, 10, 11, 10, 00
21: 11, 10, 11, 10, 01
22: 11, 10, 11, 11, 00
23: 11, 10, 11, 11, 01
24: 11, 11, 10, 10, 00
25: 11, 11, 10, 10, 01
26: 11, 11, 10, 11, 00
27: 11, 11, 10, 11, 01
28: 11, 11, 11, 10, 00
29: 11, 11, 11, 10, 01
30: 11, 11, 11, 11, 00
31: 11, 11, 11, 11, 01
etc.

Notice that the size grows as O(log n), not O(n) as above. Notice also
that the only values here for which this saves space over the 7-bit
version are 0-7. Unless you expect those values to be very common, the
7-bit encoding that needs only one byte all the way up to 127 makes a
lot of sense.

There's also an optimization that can be added here if we wish to
inject a bit of cleverness. Notice that all values with more than one
group start with 11, never 10. We can borrow a trick from IEEE
floating point and make the leading 1 bit of the mantissa implicit for
all values greater than 3 (we can't do it for 2 and 3 because then we
couldn't distinguish them from 0 and 1). Applying this optimization
removes one full group from the representation of all values greater
than 3, which appears to make the stop-bit representation as short as
or shorter than the "difficult" one for all the values that have been
enumerated above.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: matplotlib.pyplot documentation

[Mark Lawrence]

On 19/02/2015 17:50, ast wrote:

hello

I got this module documentation with help(matplotlib.pyplot)

for each function there is a demo file, for example:

barbs(*args, **kw)
Plot a 2-D field of barbs.

 **Example:**
 .. plot:: mpl_examples/pylab_examples/barb_demo.py

but I dont find the barb_demo.py on my computer, I made a full search on
C:/Python34/ and it's failed

Do you know where these demo files are located ?
(windows vista, python 3.4)

thx



This http://matplotlib.org/examples/pylab_examples/barb_demo.html was 
the first hit using a search engine.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

matplotlib.pyplot documentation

[ast]

hello

I got this module documentation with help(matplotlib.pyplot)

for each function there is a demo file, for example:

   barbs(*args, **kw)
   Plot a 2-D field of barbs.

**Example:**
   
.. plot:: mpl_examples/pylab_examples/barb_demo.py


but I dont find the barb_demo.py on my computer, I made a 
full search on C:/Python34/ and it's failed


Do you know where these demo files are located ?
(windows vista, python 3.4)

thx

--
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Tim Chase]

On 2015-02-19 15:04, Mark Lawrence wrote:
> On 19/02/2015 14:17, Tim Chase wrote:
 Parameterized queries is just a pet peeve of mine that I wish to
 include here. SQLite misses it and I miss the fact SQLite misses
 it. The less SQL one needs to write in their code, the happier
 one should be.
>>>
>>> Instead, use the DB-API’s parameter substitution. Put ? as a
>>> placeholder wherever you want to use a value, and then provide a
>>> tuple of values as the second argument to the cursor’s execute()
>>> method. (Other database modules may use a different placeholder,
>>> such as %s or :1.) For example:..."
>>
>> I think Mario was referring to what other back ends call prepared
>> statements.
> 
> Is this 
> https://docs.python.org/3/library/sqlite3.html#sqlite3.Cursor.executemany
> an equivalent?

Depends on whether sqlite3.Cursor.executemany() uses

https://www.sqlite.org/c3ref/stmt.html

under the hood.

-tkc



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Accessible tools

[Jacob Kruger]

I use edSharp as my primary code/text editor, and just save code from there,
and while it doesn't offer too many things like design tools, it's pretty
much perfect in terms of code block selection, code snippets, code
navigation - to a certain extent, even in python, with regards to jumping
from block to block, etc.

Text based IDE would be nice at times, as long as you could review code
nicely enough, etc., and in terms of GUI design I generally just make use
of/work with scrollable grids via wxPython, since that cooperates well
enough in end result with both sighted, and accessibility API users, etc.,
but anyway.

It just means my interface design is done via a form of layout-by-code,
dropping controls in a flexGridSizer as well, but anyway.

My one primary thing I would like to be able to handle in terms of
development process is better means of working with debugging code
execution, etc. - at moment, primarily either print out information to
console, during dev process, or at times invoke sound effects/TTS output to
keep track, or even, occasionally use pickle to store images of objects for
later review, etc., but anyway...

Stay well

Jacob Kruger
Blind Biker
Skype: BlindZA
"Roger Wilco wants to welcome you...to the space janitor's closet..."

- Original Message - 
From: "Bryan Duarte" 

To: "Jonas Wielicki" 
Cc: 
Sent: Thursday, February 19, 2015 5:33 PM
Subject: Re: Accessible tools



Thank you jwi, and Jacob,

I took a look at that posting and it seems pretty unique. I am not much 
interested in the speech driven development, but I am very interested in 
developing an accessible IDE. A professor and I have been throwing around 
the idea of developing a completely text based IDE. There are a lot of 
reasons this could be beneficial to a blind developer and maybe even some 
sighted developers who are comfortable in the terminal. The idea would be 
really just to provide a way of easily navigating blocks of code using 
some kind of tabular formatting, and being able to collapse blocks of code 
and hearing from a high level information about the code within. All tools 
and features would obviously be spoken or output in some kind of audio 
manor.


Jacob, I know your name and I do know Jamal's name as well. I think I 
recall your names from either back in the "Mobile Speak" days, or maybe 
from the jaws mailing list. Either way thank you for the feedback and I 
will take a look at edSharp today. The Python interpreter is great for 
small tests or scripts but for lengthy programs there is no easy way to 
save your code other than capturing the entire history with extra code and 
all. How do you typically handle that issue? Thank you both.


Oh and before I forget does anyone know how to contact Eric who was 
developing that accessible speech driven IDE? Thanks

On Feb 19, 2015, at 3:08 AM, Jonas Wielicki  wrote:

Dear Bryan,

I don’t have a finished solution for you, but I would like to link you
to a previous thread on this list:


The poster seems to be researching into the direction of developing a
speech-friendly IDE. You may want to follow his work.

regards,
jwi


--
https://mail.python.org/mailman/listinfo/python-list


--
https://mail.python.org/mailman/listinfo/python-list



--
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[janhein . vanderburg]

On Thursday, February 19, 2015 at 10:52:38 AM UTC+1, Jonas Wielicki wrote:
> I read through the discussion, but until you said it directly, I did not
> realize that you wanted feedback on your *python* code.

Thanks for the tips Jonas.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[janhein . vanderburg]

On Wednesday, February 18, 2015 at 11:20:12 PM UTC+1, Dave Angel wrote:
> I'm not necessarily doubting it, just challenging you to provide a data 
> sample that actually shows it.  And of course, I'm not claiming that 
> 7bit is in any way optimal.  You cannot define optimal without first 
> defining the distribution.

Weird results.
For a character size 2 the growth processes are shown below.
I listed the decimal representations, the difficult representation, a stop bit 
encoding, and the number of characters they differ in length:
0:  00  00  0
1:  01  01  0
2:  10, 00  10, 00  0
3:  10, 01  10, 01  0
4:  10, 10  11, 00  0
5:  10, 11  11, 01  0
6:  11, 00.00   11, 10, 00  0
7:  11, 00.01   11, 10, 01  0
8:  11, 00.10   11, 11, 00  0
9:  11, 00.11   11, 11, 01  0
10: 11, 01.00   11, 11, 10, 00  1
11: 11, 01.01   11, 11, 10, 01  1
12: 11, 01.10   11, 11, 11, 00  1
13: 11, 01.11   11, 11, 11, 01  1
14: 11, 10.00, 00   11, 11, 11, 10, 00  1
15: 11, 10.00, 01   11, 11, 11, 10, 01  1
16: 11, 10.00, 10   11, 11, 11, 11, 00  1
17: 11, 10.00, 11   11, 11, 11, 11, 01  1
18: 11, 10.01, 00.0011, 11, 11, 11, 10, 00  1
19: 11, 10.01, 00.0111, 11, 11, 11, 10, 01  1
20: 11, 10.01, 00.1011, 11, 11, 11, 11, 00  1
21: 11, 10.01, 00.1111, 11, 11, 11, 11, 01  1
22: 11, 10.01, 01.0011, 11, 11, 11, 11, 10, 00  2
23: 11, 10.01, 01.0111, 11, 11, 11, 11, 10, 01  2
24: 11, 10.01, 01.1011, 11, 11, 11, 11, 11, 00  2
25: 11, 10.01, 01.1111, 11, 11, 11, 11, 11, 01  2
26: 11, 10.01, 10.0011, 11, 11, 11, 11, 11, 10, 00  3

I didn't take the time to prove it mathematically, but these results suggest to 
me that the complicated encoding beats the stop bit encoding.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Mario Figueiredo]

On Thu, Feb 19, 2015 at 11:28 AM, Mark Lawrence 
wrote:

>
> No thanks as I won't be able to find any more candidates for my dream team.
>
>
I'm glad you like me here.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Accessible tools

[Bryan Duarte]

Thank you jwi, and Jacob,

I took a look at that posting and it seems pretty unique. I am not much 
interested in the speech driven development, but I am very interested in 
developing an accessible IDE. A professor and I have been throwing around the 
idea of developing a completely text based IDE. There are a lot of reasons this 
could be beneficial to a blind developer and maybe even some sighted developers 
who are comfortable in the terminal. The idea would be really just to provide a 
way of easily navigating blocks of code using some kind of tabular formatting, 
and being able to collapse blocks of code and hearing from a high level 
information about the code within. All tools and features would obviously be 
spoken or output in some kind of audio manor. 

Jacob, I know your name and I do know Jamal's name as well. I think I recall 
your names from either back in the "Mobile Speak" days, or maybe from the jaws 
mailing list. Either way thank you for the feedback and I will take a look at 
edSharp today. The Python interpreter is great for small tests or scripts but 
for lengthy programs there is no easy way to save your code other than 
capturing the entire history with extra code and all. How do you typically 
handle that issue? Thank you both. 

Oh and before I forget does anyone know how to contact Eric who was developing 
that accessible speech driven IDE? Thanks
> On Feb 19, 2015, at 3:08 AM, Jonas Wielicki  wrote:
> 
> Dear Bryan,
> 
> I don’t have a finished solution for you, but I would like to link you
> to a previous thread on this list:
> 
> 
> The poster seems to be researching into the direction of developing a
> speech-friendly IDE. You may want to follow his work.
> 
> regards,
> jwi
> 
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Chris Angelico]

On Fri, Feb 20, 2015 at 2:04 AM, Mark Lawrence  wrote:
>> This saves the SQL processor from recompiling the SQL into internal
>> byte-code every time.  It's handy if you know a given query will run
>> multiple times with the same "shape" parameters.  It's not essential,
>> and some optimize away the need, but many back-end interfaces support
>> it.
>>
>> -tkc
>>
>
> Is this
> https://docs.python.org/3/library/sqlite3.html#sqlite3.Cursor.executemany an
> equivalent?

That's a different feature, and also useful. Personally, I've never
used executemany() for anything other than INSERT statements, though I
can imagine using it equally for UPDATE. It's useful only when you
have a bulk lot to do all at once; you can't take advantage of it to
repeat a common and complex query. Imagine you run a web server that
shows some statistical information about your session, on every page;
this may require a complex query, which you could retain from one page
request to another. But that only matters to performance.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What behavior would you expect?

[Chris Angelico]

On Fri, Feb 20, 2015 at 1:16 AM, Denis McMahon  wrote:
>> 2. no files match the given pattern
>
> Return either None, 0, False or an empty string.
>
> In both cases, it is then a matter for the calling code to catch the
> exception or handle the return value appropriately.

I'd avoid the empty string here, because "absence of file" should be
very different from "first file matching pattern". Imagine this naive
code:

fn = my_function("/path/to/dir", "pattern2")
move_to("/path/to/dir/" + fn, "/other/path")

In the failure case (nothing matches the pattern), what could happen?
Best is it raises an exception, which would come from the my_function
line. Next best, it returns None, 0, or False, and you get a TypeError
on the deletion, and can trace your way back up. Worst, it returns ""
and oops, you just moved the whole directory into the target, instead
of just one file.

Of course, non-naive code probably prefers a None/0/False return to a
raised exception, but in any case, the difference between those two
options is fairly small. But I'd avoid the potential confusion with
the empty string.

Anyway, that's just API bike-shedding :)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Mark Lawrence]

On 19/02/2015 14:17, Tim Chase wrote:

On 2015-02-19 05:32, Mark Lawrence wrote:

On 19/02/2015 00:08, Mario Figueiredo wrote:

Parameterized queries is just a pet peeve of mine that I wish to
include here. SQLite misses it and I miss the fact SQLite misses
it. The less SQL one needs to write in their code, the happier
one should be.


Instead, use the DB-API’s parameter substitution. Put ? as a
placeholder wherever you want to use a value, and then provide a
tuple of values as the second argument to the cursor’s execute()
method. (Other database modules may use a different placeholder,
such as %s or :1.) For example:..."


I think Mario was referring to what other back ends call prepared
statements. So you do something like


   sql = "..." # parameters are referenced here
   conn = sqlite3.connect(...)
   stmt = conn.prepare(sql)
   for parameters in list_of_parameters:
 stmt.execute(*parameters)

This saves the SQL processor from recompiling the SQL into internal
byte-code every time.  It's handy if you know a given query will run
multiple times with the same "shape" parameters.  It's not essential,
and some optimize away the need, but many back-end interfaces support
it.

-tkc



Is this 
https://docs.python.org/3/library/sqlite3.html#sqlite3.Cursor.executemany an 
equivalent?


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: What behavior would you expect?

[Denis McMahon]

On Wed, 18 Feb 2015 21:44:12 -0700, Jason Friedman wrote:

> My question is, what would be a reasonable behavior/result/return value
> if:

> 1. "/path/to/dir" does not exist or is not readable 

Normally I'd say raise an exception. Whether you choose to use an 
existing exception (will trying to read a non existent dir raise one 
anyway?) or define your own is up to you. This condition would probably 
indicate an error in the data received by the function - you should be 
given a readable directory.

If (and only if) being called with an invalid directory is potentially 
valid, then you could respond to (1) the same as to (2).

> 2. no files match the given pattern

Return either None, 0, False or an empty string.

In both cases, it is then a matter for the calling code to catch the 
exception or handle the return value appropriately.

-- 
Denis McMahon, denismfmcma...@gmail.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Tim Chase]

On 2015-02-19 05:32, Mark Lawrence wrote:
> On 19/02/2015 00:08, Mario Figueiredo wrote:
> > Parameterized queries is just a pet peeve of mine that I wish to
> > include here. SQLite misses it and I miss the fact SQLite misses
> > it. The less SQL one needs to write in their code, the happier
> > one should be.
> 
> Instead, use the DB-API’s parameter substitution. Put ? as a
> placeholder wherever you want to use a value, and then provide a
> tuple of values as the second argument to the cursor’s execute()
> method. (Other database modules may use a different placeholder,
> such as %s or :1.) For example:..."

I think Mario was referring to what other back ends call prepared
statements. So you do something like

  
  sql = "..." # parameters are referenced here
  conn = sqlite3.connect(...)
  stmt = conn.prepare(sql)
  for parameters in list_of_parameters:
stmt.execute(*parameters)

This saves the SQL processor from recompiling the SQL into internal
byte-code every time.  It's handy if you know a given query will run
multiple times with the same "shape" parameters.  It's not essential,
and some optimize away the need, but many back-end interfaces support
it.

-tkc



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: urgent help

[Denis McMahon]

On Thu, 19 Feb 2015 04:00:50 -0500, Dave Angel wrote:

> On 02/19/2015 03:35 AM, ismaham...@gcuf.edu.pk wrote:
>> this is the error in the following python code, can any one help me
>> error{Traceback (most recent call last):
>>File "C:\Python27\Scripts\BeOk\getBeOKExperts.py", line 6, in
>>
>>  from BeautifulSoup import BeautifulSoup
>> ImportError: No module named BeautifulSoup}
>>
>> "#encoding=utf8 from codecs import open from collections import
>> defaultdict import re
>>
>> from BeautifulSoup import BeautifulSoup

> When you can demonstrate a problem in a couple of lines of source code,
> why would you waste our bandwidth showing us dozens of unrelated  lines?
> 
> Since the error says there's no module named BeautifulSoup, perhaps
> that's because you haven't installed BeautifulSoup.  it's not in the
> standard library.
> 
> I've never used it, but a quick web search found me the page:
> 
> http://www.crummy.com/software/BeautifulSoup/bs4/doc/

  *

  *

   

>  And that seems to say the module is called bs4. 

   

  *

  *

It seems that the OP has failed to read your post, the documentation or 
the examples for the code he is using.

As a very strong hint, I have highlighted your fix for his main problem 
above with a few (ok, several) asterisks. Let's see if he can find it now.

If he can't, I don't understand why he bothered to ask for help, because 
I'm pretty sure you nailed the issue right there, and unless he's going 
to read the responses to his post to see the answers that are provided 
it's a bit stupid to post asking for help in the first place.

-- 
Denis McMahon, denismfmcma...@gmail.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: sqlite3 and dates

[Adam Funk]

On 2015-02-18, Chris Angelico wrote:

> On Thu, Feb 19, 2015 at 9:17 AM,   wrote:
>>> SQLite3 is fine for something that's basically just a more structured
>>> version of a flat file. You assume that nobody but you has the file
>>> open, and you manipulate it just the same as if it were a big fat blob
>>> of JSON, but thanks to SQLite, you don't have to rewrite the whole
>>> file every time you make a small change. That's fine.
>>
>> That's bullshit.  Sqlite offers a lot more than that including
>> a SQL interface, transactions, referential integrity, constraints
>> indexes, triggers and other general relational database features.
>>
>> That you would equate that to a JSON blob would indicate either
>> a profound ignorance about Sqlite or (more likely) a need to
>> defend your preference with complete disregard of fact.
>
> I didn't equate them. I said that SQLite3 is great if you look on it
> as an upgrade over a JSON blob. Of course it offers more features than
> that, and you don't need to swear at me to make your point.
>
> But SQLite3 is *not* great if you look on it as a database engine
> comparable with DB2, PostgreSQL, and even MySQL.

I certainly agree with that bit, but in my own code I can almost never
justify the hassle (set-up, security considerations, &c.) of using a
database server.  TBH, one reason I like SQLite3 is that I can easily
move the data file around in the filesystem or between machies.


-- 
"It is the role of librarians to keep government running in difficult
times," replied Dramoren.  "Librarians are the last line of defence
against chaos."   (McMullen 2001)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: sqlite3 and dates

[Adam Funk]

On 2015-02-18, Johannes Bauer wrote:

> On 18.02.2015 12:21, Chris Angelico wrote:
>
>> SQLite3 is fine for something that's basically just a more structured
>> version of a flat file. You assume that nobody but you has the file
>> open, and you manipulate it just the same as if it were a big fat blob
>> of JSON, but thanks to SQLite, you don't have to rewrite the whole
>> file every time you make a small change. That's fine. But it's the
>> wrong tool for any job involving multiple users over a network, and
>> quite probably the wrong tool for a lot of other jobs too.
>
> Your assessment that some tools fit certain problems and don't fit
> different problems is entirely correct. SQLite does the job that it is
> supposed to do and it fills that nieche well.
>
>> It's the
>> smallest-end piece of software that can truly be called a database. I
>> would consider it to be the wrong database for serious accounting
>> work, and that's based on the ranting of a majorly-annoyed accountant
>> who had to deal with issues in professional systems that had made
>> similar choices in back-end selection.
>
> It probably is the wrong database for serious accounting work, and it's
> probably also the wrong database for doing multivariate statistical
> analysis on sparse matrices that you store in tables.
>
> You could similarly argue that a hammer is the wrong tool to drive in a
> screw and you'd be correct in that assessment. But it's completely
> besides the point.

"If your only tool is a hammer, every problem looks like a nail."
;-)


-- 
In the 1970s, people began receiving utility bills for
-£999,999,996.32 and it became harder to sustain the 
myth of the infallible electronic brain. (Verity Stob)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A question about how plot from matplotlib works

[marco . nawijn]

On Thursday, February 19, 2015 at 11:47:53 AM UTC+1, ast wrote:
> Hello
> 
> >>> import numpy as np
> >>> import matplotlib.pyplot as plt
> >>> x = np.arange(10)
> >>> y = x**2
> >>> x
> array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
> >>> y
> array([ 0,  1,  4,  9, 16, 25, 36, 49, 64, 81])
> >>> plt.plot(x,y)
> []
> >>> plt.show()
> 
> 
> The question is:
> 
> plt.plot() creates an object "matplotlib.lines.Line2D" but this object is
> not referenced. So this object should disapear from memory. But
> this doesn't happens since plt.show() draws the curve on a graphic
> window. So how does it work ?

Hi,

I have not checked the source code, but pyplot probably implicitly
generates a few objects for you. In particular it probably creates
a default figure, so when you say "plt.plot(x,y)", behind the scenes
pyplot will request the current figure and add the Line2D items to it.

Marco
 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What the Pythons docs means by "container" ?

[Mark Lawrence]

On 19/02/2015 10:59, perfectican wrote:

in duck-typing, any objects can acts as Container. but only difference is type 
that we need to consider as a Container or not.



This comment makes no sense to me at all.  Could you please rephrase it, 
preferably supplying some context at the same time.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: can python handle CHIME .spt files (fwd)

[Laura Creighton]

So I asked Andrew about PyMol.  Here's what he said:

--- Forwarded Message

Return-Path: 
From: Andrew Dalke 
In-Reply-To: <201502190705.t1j75wx7024...@fido.openend.se>
Date: Thu, 19 Feb 2015 10:55:09 +0100
Message-Id: <47ec31ce-33ac-4426-a874-ff4107f0c...@dalkescientific.com>
On Feb 19, 2015, at 8:05 AM, Laura Creighton wrote:
> Now discussion has moved to 
> http://www.pymol.org/
> 
> Do you know if that will do the job?  Or anything else about it?

I do not know. My knowledge of that field is rather dated now. The
best I can find in a quick search is a RasMol -> PyMol converter in
2,600 lines of Python, at

  
http://sourceforge.net/projects/sbevsl/files/ConSCRIPT/ConSCRIPT-1.0/ConSCRIPT_17Jun10/
  
https://ritdml.rit.edu/bitstream/handle/1850/9019/SMottarellaAbstract2008.pdf?sequence=8

and quoting from http://blondie.dowling.edu/docman/view.php/24/692/ACA_JUL09.ppt

The latest version of ConSCRIPT has coded for almost all of the
commands possible in RasMol.  While support for some remains minimal,
included some that only return incompatibility errors, most commands,
including most of those that directly manipulate the image, are
supported.  Here are some of the commands that have recently been
added.

I have no experience with it. The review at
  https://www.mail-archive.com/jmol-users@lists.sourceforge.net/msg16583.html
says its works quite well.

I don't know how portable it is to the Chime commands.

Cheers,

Andrew
da...@dalkescientific.com



--- End of Forwarded Message
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What the Pythons docs means by "container" ?

[perfectican]

in duck-typing, any objects can acts as Container. but only difference is type 
that we need to consider as a Container or not.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: urgent help

[Mark Lawrence]

On 19/02/2015 09:48, ismaham...@gcuf.edu.pk wrote:

As Dave Angel said nearly two hours ago the module is called bs4, so the 
command you need is:-


from bs4 import BeautifulSoup

In future please don't repeat the entire email just to add a sentence or 
two, particularly when you do so three times, as some people pay for 
bandwidth.  Thanks.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

A question about how plot from matplotlib works

[ast]

Hello


import numpy as np
import matplotlib.pyplot as plt
x = np.arange(10)
y = x**2
x

array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

y

array([ 0,  1,  4,  9, 16, 25, 36, 49, 64, 81])

plt.plot(x,y)

[]

plt.show()



The question is:

plt.plot() creates an object "matplotlib.lines.Line2D" but this object is
not referenced. So this object should disapear from memory. But
this doesn't happens since plt.show() draws the curve on a graphic
window. So how does it work ?



--
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Mark Lawrence]

On 19/02/2015 09:42, Mario Figueiredo wrote:

On Thu, Feb 19, 2015 at 9:06 AM, Mark Lawrence mailto:breamore...@yahoo.co.uk>> wrote


The opinions being expressed seem to be along the lines of
"reinventing round wheels is a waste of time.  Reinventing square or
even triangular wheels is really pointless".


You obviously don't value the word 'exercise'.


If you don't like the opinions expressed here by so called
"patronizing posters" you are free to go away at any time you see fit.


If you don't like the opinions on the opinions of patronizing posters,
you are free to go away any time you see fit.



No thanks as I won't be able to find any more candidates for my dream team.

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Accessible tools

[Jonas Wielicki]

Dear Bryan,

I don’t have a finished solution for you, but I would like to link you
to a previous thread on this list:


The poster seems to be researching into the direction of developing a
speech-friendly IDE. You may want to follow his work.

regards,
jwi




signature.asc
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Accessible tools

[Jacob Kruger]

Brian, I am also a fully bind python/PHP developer, and all I really bother 
with, aside from python interpreter itself is the text editor called 
edSharp:

http://www.empowermentzone.com/edsetup.exe

That's been developed by another blind guy who's name you might have heard 
in other circles, Jamal Mazrui, and it's just a programmers text editor that 
has a bit of extra functionality built in for use along with screen readers 
like jaws and NVDA, etc.


And, there are also some other general blind programmer mailing lists - I am 
on both progra...@freelists.org as well as a couple of other topic specific 
ones, but anyway.


Stay well

Jacob Kruger
Blind Biker
Skype: BlindZA
"Roger Wilco wants to welcome you...to the space janitor's closet..."

- Original Message - 
From: "Bryan Duarte" 

To: 
Sent: Wednesday, February 18, 2015 7:43 PM
Subject: Accessible tools



Hello all,

I have been posting to another group which directed me to this group. I am 
a blind software engineering student at Arizona State University. I am 
currently doing research and have decided to use Python as my developing 
language. I am in search of an accessible IDE or other tool set which will 
allow me to use the features with a screen reader. I have tried a number 
of tools but so far have been unsuccessful in finding one that allows me 
to use the tools and features of the IDE with my screen reader.


I understand that I do not need an IDE to use Python, I am looking for an 
IDE to allow me to explore the classes and methods easily and quickly. I 
have tried the following tools.

• pyCharm
• iPy
• iPy Notebook
• Idle
• and Eclipse

So far only Eclipse was accessible but I did still incounter some 
difficulties like the IDE freezing up when tool tips would pop up. Finally 
I am using a Mac mostly but can use Windows, or Linux if need be. Thank 
you and I appreciate any and all suggestions.

--
https://mail.python.org/mailman/listinfo/python-list



--
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Mario Figueiredo]

On Thu, Feb 19, 2015 at 9:06 AM, Mark Lawrence 
wrote
>
>
> The opinions being expressed seem to be along the lines of "reinventing
> round wheels is a waste of time.  Reinventing square or even triangular
> wheels is really pointless".
>

You obviously don't value the word 'exercise'.


> If you don't like the opinions expressed here by so called "patronizing
> posters" you are free to go away at any time you see fit.
>
>
If you don't like the opinions on the opinions of patronizing posters, you
are free to go away any time you see fit.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Jonas Wielicki]

Dear Jan-Hein,

I read through the discussion, but until you said it directly, I did not
realize that you wanted feedback on your *python* code.

In that case, let me note a few things which make it unlikely that you
will get (usable) feedback:

1. The code on your website is not formatted and highlighted properly.
This makes it hard to read. Of course, people can copy-paste it into
their favourite highlighter, but that poses a hurdle some (including me)
do not want to take, because of ...

2. ... we don’t exactly know how the algorithm is supposed to work.
Unfortunately (and I’m quite sure that this is not due to the fact that
you’re not native english, because I was able to follow your mails
without issues), your description of the algorithm on your blog is full
of typos and incomplete or ambiguous grammar. It could also use some
mathematical typesetting to make it more readable.


I propose that you reformat both your description of the algorithm and
your implementation to get a better review on it. In fact, I am quite
curious about it (having implemented a MIDI-ish format I stole from
Matroska without knowing, for my pet binary storage format, where I need
small (less than 8 bit) at a number of occasions), but currently I don’t
have the time to dig through it if it doesn’t read fluently (there are
exams over here). I assume that many other people also have much other
stuff to do.


Another idea to make it more attractive for people to review your code
(if you are not after a functional review) would be to go to, e.g.,
. I am not quite sure whether it
would be "on-topic" there -- it would probably require a concise
description of the algorithm so that people can make their own
mind-model about how the algorithm is *supposed* to work and how your
code works, to compare and see potential for optimization, code wise.

best regards,
jwi

p.s.:

On your website:
> The natural character size is 2 bits, and that will eventually become
> the standard.

What is it with that statement? That bugged me when I visited the page
for the first time. Just marketing-wise, I believe you should not
confront people with a controversial statement, taking it for granted,
on the first glance. It would be better to state that this is your
implementation of Algorithm A (link to concise description included) and
you would like to get feedback on your implementation or whatever.



signature.asc
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Accessible tools

[Bryan Duarte]

Hello all,

I have been posting to another group which directed me to this group. I am a 
blind software engineering student at Arizona State University. I am currently 
doing research and have decided to use Python as my developing language. I am 
in search of an accessible IDE or other tool set which will allow me to use the 
features with a screen reader. I have tried a number of tools but so far have 
been unsuccessful in finding one that allows me to use the tools and features 
of the IDE with my screen reader. 

I understand that I do not need an IDE to use Python, I am looking for an IDE 
to allow me to explore the classes and methods easily and quickly. I have tried 
the following tools.
• pyCharm
• iPy
• iPy Notebook
• Idle
• and Eclipse

So far only Eclipse was accessible but I did still incounter some difficulties 
like the IDE freezing up when tool tips would pop up. Finally I am using a Mac 
mostly but can use Windows, or Linux if need be. Thank you and I appreciate any 
and all suggestions.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: urgent help

[ismahameed]

On Thursday, February 19, 2015 at 5:46:42 PM UTC+8, ismah...@gcuf.edu.pk wrote:
> On Thursday, February 19, 2015 at 5:31:49 PM UTC+8, ismah...@gcuf.edu.pk 
> wrote:
> > On Thursday, February 19, 2015 at 4:35:18 PM UTC+8, ismah...@gcuf.edu.pk 
> > wrote:
> > > this is the error in the following python code, can any one help me
> > > error{Traceback (most recent call last):
> > >   File "C:\Python27\Scripts\BeOk\getBeOKExperts.py", line 6, in 
> > > from BeautifulSoup import BeautifulSoup
> > > ImportError: No module named BeautifulSoup} 
> > > 
> > > 
> > > 
> > > "#encoding=utf8
> > > from codecs import open
> > > from collections import defaultdict
> > > import re
> > > 
> > > from BeautifulSoup import BeautifulSoup
> > > import mechanize
> > > import cookielib
> > > import html2text
> > > import time
> > > 
> > > 
> > > def getbr():
> > > br = mechanize.Browser()
> > > 
> > > # Cookie Jar
> > > cj = cookielib.LWPCookieJar()
> > > br.set_cookiejar(cj)
> > > 
> > > # Browser options
> > > br.set_handle_equiv(True)
> > > br.set_handle_gzip(True)
> > > br.set_handle_redirect(True)
> > > br.set_handle_referer(True)
> > > br.set_handle_robots(False)
> > > 
> > > # Follows refresh 0 but not hangs on refresh > 0
> > > br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), 
> > > max_time=1)
> > > 
> > > # User-Agent (this is cheating, ok?)
> > > br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; 
> > > en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]
> > > return br
> > > 
> > > def logthis(text):
> > > open("log.txt","a","utf8").write(text+"\n")
> > > 
> > > def getCommunity(community,url,out=""):
> > > # Browser
> > > 
> > > # The site we will navigate into, handling it's session
> > > i = 1
> > > 
> > > flag = True
> > > discussions = []
> > > baseDiscussion = []
> > > 
> > > while flag:
> > > print i
> > > currurl = url+"/"+str(i)
> > > try:
> > > br = getbr()
> > > br.open(currurl)
> > > #br.follow_link(text='link')
> > > html = br.response().read()
> > > soup = BeautifulSoup(html)
> > > if soup.find("title").string == 
> > > u'\r\n\t\u05d4\u05d5\u05d3\u05e2\u05ea \u05de\u05e2\u05e8\u05db\u05ea - 
> > > BeOK\r\n':
> > > print "done at ",i,community
> > > logthis("done at "+str(i)+" "+community)
> > > return True
> > > hrefList = soup.findAll('div',{"class":"MsgTtlChildRow"})
> > > print currurl
> > > #print hrefList
> > > for link in hrefList:
> > > #print str(link)
> > > #continue
> > > span = link.find('div',{"class":"MsgUsr"})
> > > 
> > > if "frm_mngr" in str(span):
> > > mgr = span.find("span",{"class":"frm_mngr"}).string
> > > if not "''" in mgr:
> > > continue
> > > mgr = mgr.replace("'","")
> > > date =  
> > > link.find('span',{"class":"MsgDate"}).string.split(" ")[1]
> > > #out.write(community+"\t"+mgr+"\t"+date+"\n")
> > > print community.rstrip(),date,mgr
> > > #fout = 
> > > open("corpus\\"+community+"-"+date+"-"+mgr,"w","utf8")
> > > ansDiv = 
> > > link.nextSibling.find('div',{"class":"BodyMesInner"})
> > > print "bla"
> > > ans = fixHtml2(str(ansDiv))
> > > print "bla"
> > > print ans
> > > 
> > > #fout.write(fixHtml(link.find('div',{"class":"BodyMesInner"}).string)+"\n")
> > > #fout.close()
> > > questionDiv = 
> > > link.previousSibling.find('div',{"class":"BodyMesInner"})
> > > print "bla",questionDiv
> > > quesiton = fixHtml2(str(questionDiv))
> > > print question
> > > span = None
> > > 
> > > 
> > > 
> > > soup = None
> > > br = None
> > > except:
> > > 
> > > time.sleep(60)
> > > i+=1
> > > return list(set(discussions))
> > > 
> > > def fixHtml(page):
> > > page = page.replace("","\n")
> > > page = page.replace("","\n")
> > > page = page.replace("","\n")
> > > page = page.replace("","\n")
> > > page = page.replace("","\n")
> > > page = page.replace("","\n")
> > > page = page.replace(""","'")
> > > reg = re.compile("<")
> > > reg2 = re.compile(">")
> > > page = " ".join([x[-1] for x in map(reg2.split,reg.split(page))])
> > > page = page.replace("\r\n\t\t\t","\n")
> > > return page
> > > 
> > > def fixHtml2(page):
> 

Re: urgent help

[ismahameed]

On Thursday, February 19, 2015 at 5:31:49 PM UTC+8, ismah...@gcuf.edu.pk wrote:
> On Thursday, February 19, 2015 at 4:35:18 PM UTC+8, ismah...@gcuf.edu.pk 
> wrote:
> > this is the error in the following python code, can any one help me
> > error{Traceback (most recent call last):
> >   File "C:\Python27\Scripts\BeOk\getBeOKExperts.py", line 6, in 
> > from BeautifulSoup import BeautifulSoup
> > ImportError: No module named BeautifulSoup} 
> > 
> > 
> > 
> > "#encoding=utf8
> > from codecs import open
> > from collections import defaultdict
> > import re
> > 
> > from BeautifulSoup import BeautifulSoup
> > import mechanize
> > import cookielib
> > import html2text
> > import time
> > 
> > 
> > def getbr():
> > br = mechanize.Browser()
> > 
> > # Cookie Jar
> > cj = cookielib.LWPCookieJar()
> > br.set_cookiejar(cj)
> > 
> > # Browser options
> > br.set_handle_equiv(True)
> > br.set_handle_gzip(True)
> > br.set_handle_redirect(True)
> > br.set_handle_referer(True)
> > br.set_handle_robots(False)
> > 
> > # Follows refresh 0 but not hangs on refresh > 0
> > br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), 
> > max_time=1)
> > 
> > # User-Agent (this is cheating, ok?)
> > br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; 
> > en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]
> > return br
> > 
> > def logthis(text):
> > open("log.txt","a","utf8").write(text+"\n")
> > 
> > def getCommunity(community,url,out=""):
> > # Browser
> > 
> > # The site we will navigate into, handling it's session
> > i = 1
> > 
> > flag = True
> > discussions = []
> > baseDiscussion = []
> > 
> > while flag:
> > print i
> > currurl = url+"/"+str(i)
> > try:
> > br = getbr()
> > br.open(currurl)
> > #br.follow_link(text='link')
> > html = br.response().read()
> > soup = BeautifulSoup(html)
> > if soup.find("title").string == 
> > u'\r\n\t\u05d4\u05d5\u05d3\u05e2\u05ea \u05de\u05e2\u05e8\u05db\u05ea - 
> > BeOK\r\n':
> > print "done at ",i,community
> > logthis("done at "+str(i)+" "+community)
> > return True
> > hrefList = soup.findAll('div',{"class":"MsgTtlChildRow"})
> > print currurl
> > #print hrefList
> > for link in hrefList:
> > #print str(link)
> > #continue
> > span = link.find('div',{"class":"MsgUsr"})
> > 
> > if "frm_mngr" in str(span):
> > mgr = span.find("span",{"class":"frm_mngr"}).string
> > if not "''" in mgr:
> > continue
> > mgr = mgr.replace("'","")
> > date =  
> > link.find('span',{"class":"MsgDate"}).string.split(" ")[1]
> > #out.write(community+"\t"+mgr+"\t"+date+"\n")
> > print community.rstrip(),date,mgr
> > #fout = 
> > open("corpus\\"+community+"-"+date+"-"+mgr,"w","utf8")
> > ansDiv = 
> > link.nextSibling.find('div',{"class":"BodyMesInner"})
> > print "bla"
> > ans = fixHtml2(str(ansDiv))
> > print "bla"
> > print ans
> > 
> > #fout.write(fixHtml(link.find('div',{"class":"BodyMesInner"}).string)+"\n")
> > #fout.close()
> > questionDiv = 
> > link.previousSibling.find('div',{"class":"BodyMesInner"})
> > print "bla",questionDiv
> > quesiton = fixHtml2(str(questionDiv))
> > print question
> > span = None
> > 
> > 
> > 
> > soup = None
> > br = None
> > except:
> > 
> > time.sleep(60)
> > i+=1
> > return list(set(discussions))
> > 
> > def fixHtml(page):
> > page = page.replace("","\n")
> > page = page.replace("","\n")
> > page = page.replace("","\n")
> > page = page.replace("","\n")
> > page = page.replace("","\n")
> > page = page.replace("","\n")
> > page = page.replace(""","'")
> > reg = re.compile("<")
> > reg2 = re.compile(">")
> > page = " ".join([x[-1] for x in map(reg2.split,reg.split(page))])
> > page = page.replace("\r\n\t\t\t","\n")
> > return page
> > 
> > def fixHtml2(page):
> > page = page.split('ner">')[1].split(" > print page
> > page = page.replace("","\n")
> > page = page.replace("","\n")
> > page = page.replace("","\n")
> > page = page.replace("","\n")
> > page = page.replace("","\n")
> > page = page.replace("","\n")
> > page = page.replace(""","'")
> > return page
> >

Re: urgent help

[ismahameed]

On Thursday, February 19, 2015 at 4:35:18 PM UTC+8, ismah...@gcuf.edu.pk wrote:
> this is the error in the following python code, can any one help me
> error{Traceback (most recent call last):
>   File "C:\Python27\Scripts\BeOk\getBeOKExperts.py", line 6, in 
> from BeautifulSoup import BeautifulSoup
> ImportError: No module named BeautifulSoup} 
> 
> 
> 
> "#encoding=utf8
> from codecs import open
> from collections import defaultdict
> import re
> 
> from BeautifulSoup import BeautifulSoup
> import mechanize
> import cookielib
> import html2text
> import time
> 
> 
> def getbr():
> br = mechanize.Browser()
> 
> # Cookie Jar
> cj = cookielib.LWPCookieJar()
> br.set_cookiejar(cj)
> 
> # Browser options
> br.set_handle_equiv(True)
> br.set_handle_gzip(True)
> br.set_handle_redirect(True)
> br.set_handle_referer(True)
> br.set_handle_robots(False)
> 
> # Follows refresh 0 but not hangs on refresh > 0
> br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1)
> 
> # User-Agent (this is cheating, ok?)
> br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; 
> rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]
> return br
> 
> def logthis(text):
> open("log.txt","a","utf8").write(text+"\n")
> 
> def getCommunity(community,url,out=""):
> # Browser
> 
> # The site we will navigate into, handling it's session
> i = 1
> 
> flag = True
> discussions = []
> baseDiscussion = []
> 
> while flag:
> print i
> currurl = url+"/"+str(i)
> try:
> br = getbr()
> br.open(currurl)
> #br.follow_link(text='link')
> html = br.response().read()
> soup = BeautifulSoup(html)
> if soup.find("title").string == 
> u'\r\n\t\u05d4\u05d5\u05d3\u05e2\u05ea \u05de\u05e2\u05e8\u05db\u05ea - 
> BeOK\r\n':
> print "done at ",i,community
> logthis("done at "+str(i)+" "+community)
> return True
> hrefList = soup.findAll('div',{"class":"MsgTtlChildRow"})
> print currurl
> #print hrefList
> for link in hrefList:
> #print str(link)
> #continue
> span = link.find('div',{"class":"MsgUsr"})
> 
> if "frm_mngr" in str(span):
> mgr = span.find("span",{"class":"frm_mngr"}).string
> if not "''" in mgr:
> continue
> mgr = mgr.replace("'","")
> date =  
> link.find('span',{"class":"MsgDate"}).string.split(" ")[1]
> #out.write(community+"\t"+mgr+"\t"+date+"\n")
> print community.rstrip(),date,mgr
> #fout = 
> open("corpus\\"+community+"-"+date+"-"+mgr,"w","utf8")
> ansDiv = 
> link.nextSibling.find('div',{"class":"BodyMesInner"})
> print "bla"
> ans = fixHtml2(str(ansDiv))
> print "bla"
> print ans
> 
> #fout.write(fixHtml(link.find('div',{"class":"BodyMesInner"}).string)+"\n")
> #fout.close()
> questionDiv = 
> link.previousSibling.find('div',{"class":"BodyMesInner"})
> print "bla",questionDiv
> quesiton = fixHtml2(str(questionDiv))
> print question
> span = None
> 
> 
> 
> soup = None
> br = None
> except:
> 
> time.sleep(60)
> i+=1
> return list(set(discussions))
> 
> def fixHtml(page):
> page = page.replace("","\n")
> page = page.replace("","\n")
> page = page.replace("","\n")
> page = page.replace("","\n")
> page = page.replace("","\n")
> page = page.replace("","\n")
> page = page.replace(""","'")
> reg = re.compile("<")
> reg2 = re.compile(">")
> page = " ".join([x[-1] for x in map(reg2.split,reg.split(page))])
> page = page.replace("\r\n\t\t\t","\n")
> return page
> 
> def fixHtml2(page):
> page = page.split('ner">')[1].split(" print page
> page = page.replace("","\n")
> page = page.replace("","\n")
> page = page.replace("","\n")
> page = page.replace("","\n")
> page = page.replace("","\n")
> page = page.replace("","\n")
> page = page.replace(""","'")
> return page
> 
> def getText(br,url):
> br.open(url)
> html = br.response().read()
> soup = BeautifulSoup(html)
> title = fixHtml(soup.find('h1',{'class':"articleName"}).contents[0])
> #print title
> artics = soup.findAll('div',{'class':"article"})
> text = 
> "\n"+fixHtml(str(artics[0]).split('"article">')[1].split('')[0])
> text += "\n"+ 
> fixHtml(

Re: python implementation of a new integer encoding algorithm.

[Mark Lawrence]

On 19/02/2015 08:36, Marko Rauhamaa wrote:


PS On the topic of pointlessness, why is top-posting the norm on
python-dev but shunned on python-list?



I don't know and I don't care provided top-posting remains the norm here.

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: pickle error by multiprocessing

[perfectican]

I guess, The issue is not caused by multiprocessing. make sure you are pickling 
right object that can be pickable or not.

Best way is run it in a console and test at which object the pickling error is 
occurring and then try to change the object which could be pickable.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: urgent help

[Dave Angel]

On 02/19/2015 03:35 AM, ismaham...@gcuf.edu.pk wrote:

this is the error in the following python code, can any one help me
error{Traceback (most recent call last):
   File "C:\Python27\Scripts\BeOk\getBeOKExperts.py", line 6, in 
 from BeautifulSoup import BeautifulSoup
ImportError: No module named BeautifulSoup}



"#encoding=utf8
from codecs import open
from collections import defaultdict
import re

from BeautifulSoup import BeautifulSoup



When you can demonstrate a problem in a couple of lines of source code, 
why would you waste our bandwidth showing us dozens of unrelated  lines?


Since the error says there's no module named BeautifulSoup, perhaps 
that's because you haven't installed BeautifulSoup.  it's not in the 
standard library.


I've never used it, but a quick web search found me the page:

http://www.crummy.com/software/BeautifulSoup/bs4/doc/

And that seems to say the module is called bs4.

Anyway, if you did install it, and read the directions, and are still 
stumped, you probably need to supply many other details:


1) what version of Python are you using, and do you have multiple 
versions installed

2) what OS
3) where did you download it from, and what commands did you use to 
actually install it  How did you specify which Python version it would 
install to?

4) what your import line looks like (which you did specify)
5) and of course, what the exception is (which you did include)

Other things people may need to know include what directory the bs4.pyc 
file is installed to, what your sys.path is, and so on.  But just 
answering the first questions might let you figure it out for yourself.



--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Marko Rauhamaa]

Mark Lawrence :

> The opinions being expressed seem to be along the lines of
> "reinventing round wheels is a waste of time. Reinventing square or
> even triangular wheels is really pointless".

I think it's even more pointless to mention the pointlessness of
someone's hobby.

Most pointless of all is mentioning the pointlessness of mentioning the
pointlessness of someone's hobby...


Marko

PS On the topic of pointlessness, why is top-posting the norm on
python-dev but shunned on python-list?
-- 
https://mail.python.org/mailman/listinfo/python-list

urgent help

[ismahameed]

this is the error in the following python code, can any one help me
error{Traceback (most recent call last):
  File "C:\Python27\Scripts\BeOk\getBeOKExperts.py", line 6, in 
from BeautifulSoup import BeautifulSoup
ImportError: No module named BeautifulSoup} 



"#encoding=utf8
from codecs import open
from collections import defaultdict
import re

from BeautifulSoup import BeautifulSoup
import mechanize
import cookielib
import html2text
import time


def getbr():
br = mechanize.Browser()

# Cookie Jar
cj = cookielib.LWPCookieJar()
br.set_cookiejar(cj)

# Browser options
br.set_handle_equiv(True)
br.set_handle_gzip(True)
br.set_handle_redirect(True)
br.set_handle_referer(True)
br.set_handle_robots(False)

# Follows refresh 0 but not hangs on refresh > 0
br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1)

# User-Agent (this is cheating, ok?)
br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; 
rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]
return br

def logthis(text):
open("log.txt","a","utf8").write(text+"\n")

def getCommunity(community,url,out=""):
# Browser

# The site we will navigate into, handling it's session
i = 1

flag = True
discussions = []
baseDiscussion = []

while flag:
print i
currurl = url+"/"+str(i)
try:
br = getbr()
br.open(currurl)
#br.follow_link(text='link')
html = br.response().read()
soup = BeautifulSoup(html)
if soup.find("title").string == 
u'\r\n\t\u05d4\u05d5\u05d3\u05e2\u05ea \u05de\u05e2\u05e8\u05db\u05ea - 
BeOK\r\n':
print "done at ",i,community
logthis("done at "+str(i)+" "+community)
return True
hrefList = soup.findAll('div',{"class":"MsgTtlChildRow"})
print currurl
#print hrefList
for link in hrefList:
#print str(link)
#continue
span = link.find('div',{"class":"MsgUsr"})

if "frm_mngr" in str(span):
mgr = span.find("span",{"class":"frm_mngr"}).string
if not "''" in mgr:
continue
mgr = mgr.replace("'","")
date =  
link.find('span',{"class":"MsgDate"}).string.split(" ")[1]
#out.write(community+"\t"+mgr+"\t"+date+"\n")
print community.rstrip(),date,mgr
#fout = 
open("corpus\\"+community+"-"+date+"-"+mgr,"w","utf8")
ansDiv = 
link.nextSibling.find('div',{"class":"BodyMesInner"})
print "bla"
ans = fixHtml2(str(ansDiv))
print "bla"
print ans

#fout.write(fixHtml(link.find('div',{"class":"BodyMesInner"}).string)+"\n")
#fout.close()
questionDiv = 
link.previousSibling.find('div',{"class":"BodyMesInner"})
print "bla",questionDiv
quesiton = fixHtml2(str(questionDiv))
print question
span = None



soup = None
br = None
except:

time.sleep(60)
i+=1
return list(set(discussions))

def fixHtml(page):
page = page.replace("","\n")
page = page.replace("","\n")
page = page.replace("","\n")
page = page.replace("","\n")
page = page.replace("","\n")
page = page.replace("","\n")
page = page.replace(""","'")
reg = re.compile("<")
reg2 = re.compile(">")
page = " ".join([x[-1] for x in map(reg2.split,reg.split(page))])
page = page.replace("\r\n\t\t\t","\n")
return page

def fixHtml2(page):
page = page.split('ner">')[1].split("","\n")
page = page.replace("","\n")
page = page.replace("","\n")
page = page.replace("","\n")
page = page.replace("","\n")
page = page.replace("","\n")
page = page.replace(""","'")
return page

def getText(br,url):
br.open(url)
html = br.response().read()
soup = BeautifulSoup(html)
title = fixHtml(soup.find('h1',{'class':"articleName"}).contents[0])
#print title
artics = soup.findAll('div',{'class':"article"})
text = 
"\n"+fixHtml(str(artics[0]).split('"article">')[1].split('')[0])
text += "\n"+ 
fixHtml(str(artics[1]).split('"article">')[1].split('')[0])+""
text = text.decode("utf-8")
#text = artics[0] +
#print type(title),type(text)

return title+text

def getForums(file = "links.htm"):
#out = open("beokDates","w","utf8")
soup = BeautifulSoup(open(file,"r").read())
communities = soup.findAll("a",{"class":"MainList"})
for comm in communities:
#print comm["href"]
g

Re: python implementation of a new integer encoding algorithm.

[Chris Angelico]

On Thu, Feb 19, 2015 at 7:06 PM, Mark Lawrence  wrote:
> On 19/02/2015 07:44, Mario Figueiredo wrote:
>>
>> A lot of patronizing egos running around in these groups. This is a
>> sad thread...
>>
>> What is being asked is for help, not whether this is useful or needed.
>> Jan-Hein is after some directions, not whether your bloody opinion on
>> how he should use his free time.
>
> The opinions being expressed seem to be along the lines of "reinventing
> round wheels is a waste of time.  Reinventing square or even triangular
> wheels is really pointless".

If you come to a group of architects and builders and ask them,
"What's the best way to build the steps up to my garage?", and show
them this picture...

http://cheezburger.com/6665198848

... do you expect advice to switch from concrete to wood, or to scrap
the whole idea?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Mark Lawrence]

On 19/02/2015 07:44, Mario Figueiredo wrote:

A lot of patronizing egos running around in these groups. This is a
sad thread...

What is being asked is for help, not whether this is useful or needed.
Jan-Hein is after some directions, not whether your bloody opinion on
how he should use his free time.

If the interest and usability of a project would somehow become a
problem, then boy, oh, boy; Most everyone in here, including the
patronizing posters, would probably be left without anything to code.



The opinions being expressed seem to be along the lines of "reinventing 
round wheels is a waste of time.  Reinventing square or even triangular 
wheels is really pointless".


If you don't like the opinions expressed here by so called "patronizing 
posters" you are free to go away at any time you see fit.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list