Re: ctypes & allocated memory
Hi,
> But the problem is that by specifying the type as ctypes.c_char_p,
> ctypes will hide that pointer from you and return a Python object
> instead. I'm not sure how ctypes is doing it under the hood, but I
> suspect ctypes is doing it's own strdup of the string on conversion, and
> managing that memory, but the original pointer ends up being lost and
> leaking memory.
Yes, the problem with my code I posted is that resource.getrusage report the
Python allocated memory. Once I've changed to
psutil.Process().memory_info().rss I see the memory leak. The code now looks
like:
---
...
strdup.restype = ctypes.c_void_p
free = libc.free
free.argtypes = [ctypes.c_void_p]
...
for _ in range(n):
o = strdup(data)
s = ctypes.string_at(o).decode('utf-8')
free(o)
---
This seems to fix the leak.
Thanks
--
https://mail.python.org/mailman/listinfo/python-list
Re: ctypes & allocated memory
> Does ctypes, when using restype, frees allocated memory?
>
> For example, will the memory allocated by "strdup" be freed after the "del"
> statement? If not, how can I free it?
I've tried the following program and I'm more confused now :) Can anyone
explain the output?
---
import ctypes
import gc
import resource
def mem_usage():
return resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
libc = ctypes.cdll.LoadLibrary('libc.so.6')
strdup = libc.strdup
strdup.argtypes = [ctypes.c_char_p]
strdup.restype = ctypes.c_char_p
size = 1 << 20
print(f'size: {size:,}')
data = b'x' * size # 1MB
mb = mem_usage()
print(f'memory before: {mb:,}')
n = 1000
print(f'n: {n:,}')
for _ in range(n):
strdup(data)
gc.collect()
ma = mem_usage()
diff = ma - mb
print(f'memory after: {ma:,}')
print(f'diff: {diff:,}')
print(f'diff/size: {diff/size:.2f}')
---
Which prints
---
size: 1,048,576
memory before: 21,556
n: 1,000
memory after: 1,035,180
diff: 1,013,624
diff/size: 0.97
---
--
https://mail.python.org/mailman/listinfo/python-list
ctypes & allocated memory
Hi,
Does ctypes, when using restype, frees allocated memory?
For example, will the memory allocated by "strdup" be freed after the "del"
statement? If not, how can I free it?
---
import ctypes
libc = ctypes.cdll.LoadLibrary('libc.so.6')
strdup = libc.strdup
strdup.argtypes = [ctypes.c_char_p]
strdup.restype = ctypes.c_char_p
out = strdup(b'hello').decode('utf-8')
print(out) # hello
del out
---
Thanks,
Miki
--
https://mail.python.org/mailman/listinfo/python-list
Re: Shared library missing from wheel (custom build)
Changed to self.get_ext_full_path(ext.name) and it works now. -- https://mail.python.org/mailman/listinfo/python-list
Shared library missing from wheel (custom build)
Hi,
I'm playing around with generating extension moudle to Python in Go. This is
for a blog post - not production ready.
The Go code is compiled to a shared library and the Python module is using
ctypes to call the Go code in the shared library. I know it's know a classic
extension module with PyModule_Create and friends, but that's for later...
maybe.
The setup.py is defined as
---
from subprocess import call
from setuptools import Extension, setup
from setuptools.command.build_ext import build_ext
class build_go_ext(build_ext):
def build_extension(self, ext):
ext_path = self.get_ext_filename(ext.name)
cmd = ['go', 'build', '-buildmode=c-shared', '-o', ext_path]
cmd += ext.sources
out = call(cmd)
if out != 0:
raise RuntimeError('Go build failed')
setup(
name='checksig',
version='0.1.0',
py_modules=['checksig'],
ext_modules=[
Extension('_checksig', ['checksig.go', 'export.go'])
],
cmdclass={'build_ext': build_go_ext},
)
---
When I run "python setup.py bdist_wheel", I see that that
_checksig.cpython-38-x86_64-linux-gnu.so is being built. But when I look at the
content of dist/checksig-0.1.0-cp38-cp38-linux_x86_64.whl it's not there.
What am I missing?
You can view the whole (WIP) project at
https://github.com/ardanlabs/python-go/tree/master/pyext
Thanks,
Miki
--
https://mail.python.org/mailman/listinfo/python-list
Re: Division issue with 3.8.2 on AIX 7.1
> Anyone know where can I look in the Python source code to investigate > this? Probably around https://github.com/python/cpython/blob/master/Objects/floatobject.c -- https://mail.python.org/mailman/listinfo/python-list
Re: [ANN] Python Brain Teasers Book is Out
Hi, > Would be grateful if you could post it to python-authors also: > https://mail.python.org/mailman/listinfo/python-authors Done. Though list seems very dormant. Thanks, Miki -- https://mail.python.org/mailman/listinfo/python-list
[ANN] Python Brain Teasers Book is Out
Hi All, I'm excited that my book "Python Brain Teasers: 30 brain teasers to tickle your mind and help become a better developer." is out. You can grab is from https://gum.co/iIQT (PDF & ePUB). Let me know how many teasers did you get right. If you're curious, a sample of the book, including the forward by Raymond Hettinger is at https://www.353solutions.com/python-brain-teasers Stay curious, keep hacking, Miki -- https://mail.python.org/mailman/listinfo/python-list
Re: What is the correct interpreter
Hi, > Where is the correct interpreter? > I have installed python several times > Pycharm all ways selects > > C:\ProgramFiles\JetBrains\PyCharm Community Edition 2019.3.3\bin\pycharm64.exe This is the PyCharm executable, not the Python one. See https://www.jetbrains.com/help/pycharm/configuring-local-python-interpreters.html on how to configure the interpreter. > But when I run a simple code it objects to the interpreter??? I order to help we'll need more information. What is the code you're trying to run and what is the error? Happy hacking, Miki -- https://mail.python.org/mailman/listinfo/python-list
Re: Connection refused when tryign to run bottle/flask web framweworks
If you're trying to access the machine from another machine, you need to change
the host to '0.0.0.0'. 'localhost' is the internal interface.
On Sunday, August 19, 2018 at 10:36:25 PM UTC+3, Νίκος wrote:
> Hello,
>
> i just installed bottle and flask web frameworks in my CentOS environment but
> i canno get it working even with the simpleste xample. The coonection is
> refused always.
>
> from bottle import route, run, template
>
> @route('/hello/')
> def index(name):
> return template('Hello {{name}}!', name=name)
>
> run(host='localhost', port=8080)
>
>
> Any ideas as to why i cannot access it?
> I dont have sme firewall blocking the ports 8080 or 5000.
>
> Thank you.
--
https://mail.python.org/mailman/listinfo/python-list
PyCon Israel 2018 CFP is Open
Hi, PyCon Israel 2018 call for papers is open, submit a talk today, another three tomorrow :) See more at http://il.pycon.org/2018/ All the best, -- Miki -- https://mail.python.org/mailman/listinfo/python-list
Re: I have anaconda, but Pycharm can't find it
You need to set the Python interpreter for the project to be the Anaconda one. See https://www.jetbrains.com/help/pycharm/configuring-python-interpreter.html On Monday, November 27, 2017 at 1:56:58 AM UTC+2, C W wrote: > Hello all, > > I am a first time PyCharm user. I have Python 3 and Anaconda installed. > They work together on Sublime Text, but not on Pycharm. > > Pycharm tells me it cannot find modules numpy, matplotlib, etc. > > What should I do? I tried to set the interpreter environment, and a few > other options, none seem to work. > > This is the typical solution, but it does not work > https://stackoverflow.com/questions/35623776/import-numpy-on-pycharm > > Thanks, > > -Mike -- https://mail.python.org/mailman/listinfo/python-list
Re: I have anaconda, but Pycharm can't find it
You need to set the Python interpreter for the project to be the Anaconda one. See https://www.jetbrains.com/help/pycharm/configuring-python-interpreter.html On Monday, November 27, 2017 at 1:56:58 AM UTC+2, C W wrote: > Hello all, > > I am a first time PyCharm user. I have Python 3 and Anaconda installed. > They work together on Sublime Text, but not on Pycharm. > > Pycharm tells me it cannot find modules numpy, matplotlib, etc. > > What should I do? I tried to set the interpreter environment, and a few > other options, none seem to work. > > This is the typical solution, but it does not work > https://stackoverflow.com/questions/35623776/import-numpy-on-pycharm > > Thanks, > > -Mike -- https://mail.python.org/mailman/listinfo/python-list
[ANN] Nuclio: A scalable, open source, real-time processing platform
Hi, Just wanted to share a project I'm working on. It a super fast serverless that support Python handlers as well. Check out more at https://www.iguazio.com/nuclio-new-serverless-superhero/ Code at https://github.com/nuclio/nuclio/ Happy hacking, -- Miki -- https://mail.python.org/mailman/listinfo/python-list
Re: Unable to apply stop words in Pandas dataframe
Can you show us some of the code you tried? On Monday, June 26, 2017 at 10:19:46 AM UTC+3, Bhaskar Dhariyal wrote: > Hi everyone! > > I have a dataset which I want to make model trainable. I ahve been trying to > do some thing for past 2-3 days. > > Actually I wanted to clean 'desc' and 'keywords' column from the dataset. I > am using NLTK to vectorize, than remove stopwords & alpha numeric values and > do stemming. Moreover desc column also contain NULL values. But the solution > I reached was on 2D list, to which I was not able to convert single pandas > series, so that I could change change it word vector. > > Please tell me hoe to do this using Pandas, I searched lot in stackoverflow > also but no results. > > Link to dataset: https://drive.google.com/open?id=0B1D4AyluMGU0MGVkbWdvQkM3aUk > > Thanks in advance > > Bhaskar -- https://mail.python.org/mailman/listinfo/python-list
[ANN] Second PyCon Israel June 11-14, 2017
Hi All, The second PyCon Israel will take place June 11-14, 2017. * 11 June Django girls workshop at Red Hat Israel offices in Raanana * 12-13 June PyCon Israel main event at Wohl center * 14 June PyCon Israel workshops and sprints We still have some sponsorship spots available, great recruiting and branding opportunity. Also the general session CFP is still open. Read more at http://il.pycon.org/2017/ See you there, -- The PyCon Israel Team -- https://mail.python.org/mailman/listinfo/python-list
Re: Python3, column names from array - numpy or pandas
You can do this with pandas:
import pandas as pd
from io import StringIO
io = StringIO('''\
idABCDE
10010000
10101100
10210000
10300011
''')
df = pd.read_csv(io, sep='\s+', index_col='id')
val = df.apply(lambda row: ' '.join(df.columns[row==1]), axis=1)
Google for "boolean indexing" to see what's going on :)
--
https://mail.python.org/mailman/listinfo/python-list
Re: pymssql
> for row in cus:
>print(row.budget_code)
>
>
> NameError: name 'budget_code' is not defined
You'll need to use a DictCursor to be able to access rows by name and not
position (which IMO is the preferred way).
cus = conn.cursor(pymysql.cursors.DictCursor)
cus.execute("SELECT * FROM glbud ")
for row in cus:
print(row['budget_code'])
--
https://mail.python.org/mailman/listinfo/python-list
Re: Guys, can you please share me some sites where we can practice python programs for beginners and Intermediate.
On Tuesday, June 21, 2016 at 2:03:28 PM UTC+3, Pushpanth Gundepalli wrote: > Guys, can you please share me some sites where we can practice python > programs for beginners and Intermediate. IMO you can do that at https://www.codecademy.com/learn/python -- https://mail.python.org/mailman/listinfo/python-list
Re: hourly weather forecast data
> Is there a way to get hourly weather forecast data (temperature, > chance of precipitation) from the command line in Debian Linux? If you Google for "weather API" you'll find several sites who give programmatic access to weather data. -- https://mail.python.org/mailman/listinfo/python-list
[ANN] First PyCon Israel (May 2,3)
Greetings, The first PyCon Israel is happening! It's taking place on May 2-3, 2016, will be attended by hundreds of Pythonistas, and is sponsored by major international and local companies that use Python as part of their daily work. CFP is open, please head over to http://il.pycon.org/2016/ to see more details. See you there, -- Miki -- https://mail.python.org/mailman/listinfo/python-list
Re: I can not install matplotlib, numpy, scipy, and pandas.
> When I enter into the command window "pip install matplotlib", it reads this > below (this is not the full version of it): > ... Installing scientific packages can be a pain. I recommend you take a look at https://www.continuum.io/downloads -- https://mail.python.org/mailman/listinfo/python-list
Re: Newbie XML problem
Hi,
> config = {id: 1, canvas: (3840, 1024), comment: "a comment",
> {id: 4, gate: 3, (0,0, 1280, 1024)},
> {id: 5, gate: 2, (1280, 0, 2560, 1024)},
> {id: 6, gate: 1, (2560, 0, 3840, 1024)}}
This is not valid Python. Are you trying to have a list of dicts?
> I have started to use this code - but this is beginning to feel very
> non-elegant; not the cool Python code I usually see...
ElementTree supports XPATH, using this and some list comprehension you do
things a little more easily. For example:
cfg = root.find('config[@id="B"]')
for panel in cfg.findall('panel'):
panels.append([{elem.tag: elem.text} for elem in panel])
You'll need to do some parsing for the coordinates and handle canvas and
comments separately.
HTH,
Miki
--
https://mail.python.org/mailman/listinfo/python-list
Re: compiling matplotlib in virtual env
> I been trying to compile matplotlib in a python3.4 virtual env using > pip version 1.7 on Fedora 22. I am in about 3 weeks learning python > and Django so I am not clear on the error response to: > ... > File "/usr/lib64/python3.4/distutils/version.py", line 343, in _cmp > if self.version < other.version: > TypeError: unorderable types: str() < int() Looks like a bug in distuils (which is a package used by pip). Maybe you can upgrade to a newer version of distutils? (pip3 install -U distutils) and try again? As a side note, I find using conda to install scipy related packages a much easier way. -- https://mail.python.org/mailman/listinfo/python-list
Re: numpy
On Thursday, September 10, 2015 at 1:11:59 PM UTC+3, chen...@inhand.com.cn wrote: > hi: > I have to use numpy package. My python runs on my embedded arm > device. So, how do i cross compile numpy? conda has support for ARM - http://continuum.io/blog/new-arch BTW: I suggest you write a better subject next time, almost missed this one :) http://www.catb.org/esr/faqs/smart-questions.html -- https://mail.python.org/mailman/listinfo/python-list
Re: Who uses IDLE -- please answer if you ever do, know, or teach
Greetings, > 0. Classes where Idle is used: > Where? At client site. Mostly big companies. > Level? >From beginner to advanced. > Idle users: > 1. Are you > grade school (1=12)? > undergraduate (Freshman-Senior)? > post-graduate (from whatever)? post-graduate > 2. Are you > beginner (1st class, maybe 2nd depending on intensity of first)? > post-beginner? post-beginner > 3. With respect to programming, are you > amateur (unpaid) > professional (paid for programming) professional -- https://mail.python.org/mailman/listinfo/python-list
Re: Time saving tips for Pythonists
> What are your best time saving tips when programming Python? * Use the REPL. Write small chunks of code and test them as you go * Know what's available in the standard library (sets, Counter, deque ...) * Learn how to pick good packages from PyPI (community, last commit ...) * import this -- https://mail.python.org/mailman/listinfo/python-list
Re: Problem with PyPI login
> >The forums at https://sourceforge.net/p/pypi/support-requests/ (which is > >linked from "PyPI Support") seems pretty deserted. What is the right > >location for these kind of issues? > That is one place. > https://bitbucket.org/pypa/pypi/issues Thanks! -- https://mail.python.org/mailman/listinfo/python-list
Problem with PyPI login
Greetings, After confirming email I can't login to PyPI with the password and when trying to rest I get: user "Cyberint" is unknwon to me. However when trying to re-register it says the user is already there. The forums at https://sourceforge.net/p/pypi/support-requests/ (which is linked from "PyPI Support") seems pretty deserted. What is the right location for these kind of issues? Thanks, -- Miki -- https://mail.python.org/mailman/listinfo/python-list
Re: Flask Post returning error
> If anyone may kindly suggest what is the error I am doing. It's close to impossible to know without seeing the server side code. -- https://mail.python.org/mailman/listinfo/python-list
Re: Did https://pypi.python.org/pypi/ became huge and slow?
Thanks Chris, I was hitting the wrong URL by mistake. Didn't think an extra / will make all that difference :) On Tuesday, March 10, 2015 at 2:12:13 PM UTC+2, Chris Angelico wrote: > On Tue, Mar 10, 2015 at 10:55 PM, Steven D'Aprano > wrote: > > Miki Tebeka wrote: > > > >> Greetings, > >> > >> $ time curl -I https://pypi.python.org/pypi/ > >> HTTP/1.1 200 OK > >> Date: Tue, 10 Mar 2015 10:24:30 GMT > >> ... > >> Content-Length: 9870689 > >> curl -I https://pypi.python.org/pypi/ 0.02s user 0.00s system 2% cpu > >> 12.271 total $ > >> > >> Note the long time (for comparison hitting python.org takes 0.209 total) > >> and the size (Content-Length). > >> > >> Anything gone wrong or am I missing something? > > > > I don't know. Am *I* missing something? What makes you think this is a > > problem? > > > > You're downloading via https, which has more overhead than http. There's a > > certificate that needs to be checked, the content can't be cached, and > > there's the cost of encryption. > > Those costs don't factor in here; the content-length is what the > server announces as the number of bytes of oncoming payload. > > > Do you have some reason for thinking that the content-length should not be > > 9870689 bytes? Should it be less or more? > > > > Have you tried opening the URL in your browser? > > I just tried it now, and it's comparable in a browser. 9MB is a lot > for a landing page, and while I don't have a specific record, I have a > vague recollection that it wasn't quite this big before (which would > mean that the landing page was paginated instead of having the entire > list right there). > > Oh wait. What I was remembering was https://pypi.python.org/ without > the extra pathing on it. And yes, that page _is_ short and fast. So > that's an appropriate landing page. > > To the OP: You're downloading the entire list of packages, which is > why it's taking so long. > > ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Did https://pypi.python.org/pypi/ became huge and slow?
Greetings, $ time curl -I https://pypi.python.org/pypi/ HTTP/1.1 200 OK Date: Tue, 10 Mar 2015 10:24:30 GMT ... Content-Length: 9870689 curl -I https://pypi.python.org/pypi/ 0.02s user 0.00s system 2% cpu 12.271 total $ Note the long time (for comparison hitting python.org takes 0.209 total) and the size (Content-Length). Anything gone wrong or am I missing something? Thanks, -- Miki -- https://mail.python.org/mailman/listinfo/python-list
Re: List of "python -m" tools
Thanks for all the answers! -- https://mail.python.org/mailman/listinfo/python-list
List of "python -m" tools
Greetings, I've compiled a list of "python -m" tools at pythonwise.blogspot.com/2015/01/python-m.html. Did I miss something? What are your favorite "python -m" tools? Thanks, -- Miki -- https://mail.python.org/mailman/listinfo/python-list
Re: Need Help
Greetings, > I'm a beginner of python, I just want your help. We'll gladly do it, however you need to invest some time and write better questions :) See http://www.catb.org/esr/faqs/smart-questions.html > How can I get the Failure values from the Console in to a txt or a csv file? We need more information. What is the test suite you're using? How are you running the tests? Example output ... All the best, -- Miki -- https://mail.python.org/mailman/listinfo/python-list
Re: Recommended hosting
Greetings, > I'd like to build a web site for myself, essentially a "vanity" web site to > show off whatever web development skills I have, and perhaps do some > blogging. I'm a Python developer, so I'd like to develop the site with the > following stack: > web applications written with Python and Flask, running as uwsgi > applications. These would support dynamic HTML where needed, but mostly it > would provide REST API's. IIRC you can use Flask with AppEngine (static content will be handled by AppEngine and not nginx). It's free for small-ish sites and removes a lot of operations headache. All the best, -- Miki -- https://mail.python.org/mailman/listinfo/python-list
Re: Error from pandas.io.data import DataReader
Greetings, > I am trying to run this snippet of code. > > from pandas.io.data import DataReader > ... > > I keep getting this error. > > Traceback (most recent call last): > > File "C:\Python27\download_dow.py", line 1, in > > from pandas.io.data import DataReader > > ImportError: No module named pandas.io.data * Do you have pandas installed? (Does "import pandas" work?) * If it is installed, what version do you have? ( "print(pandas.__version__") HTH, -- Miki -- https://mail.python.org/mailman/listinfo/python-list
Re: "Fuzzy" Counter?
Greetings, On Wednesday, September 24, 2014 5:57:15 PM UTC+3, Ian wrote: > Then your result depends on the order of your input, which is usually > not a good thing. As stated in previous reply - I'm OK with that. > Why would you need to determine the *number* of bins in advance? You > just need to determine where they start and stop. If for example your > epsilon is 0.5, you could determine the bins to be at [-0.5, 0.5); > [0.5, 1.5); [1.5, 2.5); ad infinitum. Then for each actual value you > encounter, you could calculate the appropriate bin, creating it first > if it doesn't already exist. I see what you mean. I thought you mean histogram like bins where you usually state the number of bins in advance. -- https://mail.python.org/mailman/listinfo/python-list
Re: "Fuzzy" Counter?
On Tuesday, September 23, 2014 7:33:06 PM UTC+3, Rob Gaddi wrote: > While you're at it, think > long and hard about that definition of fuzziness. If you can make it > closer to the concept of histogram "bins" you'll get much better > performance. The problem for me here is that I can't determine the number of bins in advance. I'd like to get frequencies. I guess every "new" (don't have any previous equal item) can be a bin. > TL;DR you need to think very hard about your problem definition and > what you want to happen before you actually try to implement this. Always a good advice :) I'm actually implementing algorithm for someone else (in the bio world where I know very little about). -- https://mail.python.org/mailman/listinfo/python-list
Re: "Fuzzy" Counter?
On Tuesday, September 23, 2014 4:37:10 PM UTC+3, Peter Otten wrote: > x eq y > y eq z > not (x eq z) > > where eq is the test given above -- should x, y, and z land in the same bin? Yeah, I know the counting depends on the order of items. But I'm OK with that. -- https://mail.python.org/mailman/listinfo/python-list
"Fuzzy" Counter?
Greetings, Before I start writing my own. Is there something like collections.Counter (fore frequencies) that does "fuzzy" matching? Meaning x is considered equal to y if abs(x - y) < epsilon. (x, y and my case will be numpy.array). Thanks, -- Miki -- https://mail.python.org/mailman/listinfo/python-list
Re: Load a CSV with different row lengths
Greetings,
> I should've mentioned that I want to import my csv as a data frame or numpy
> array or as a table.
If you know the max length of a row, then you can do something like:
def gen_rows(stream, max_length):
for row in csv.reader(stream):
yield row + ([None] * (max_length - len(line))
max_length = 10
with open('data.csv') as fo:
df = pd.DataFrame.from_records(gen_rows(fo, max_length))
HTH,
Miki
--
https://mail.python.org/mailman/listinfo/python-list
Re: NameError: name 'requests' is not defined ？
Greetings, > there is only one line in mydown.py . > > > import requests > ... > >>> import mydown > >>> requests.get(url) > > Traceback (most recent call last): > > File "", line 1, in > > NameError: name 'requests' is not defined You need to call mydown.requests, but I think you're missing the point of modules and imports. I suggest you go over https://docs.python.org/3.4/tutorial/ HTH, -- Miki -- https://mail.python.org/mailman/listinfo/python-list
Re: How to install data analysis pandas toolkit?
Greetings, > >>> import pandas as pd > > No module named numpy I find the most painless way of installing the Python scientific stack is using Anaconda http://continuum.io/downloads HTH, -- Miki -- https://mail.python.org/mailman/listinfo/python-list
Re: Installing Python 2.6.2 on Ubuntu 12.1
Greetings, > The installation went through successfully, however I > noticed that some of the _*.so files did not get built under > lib/python2.6/lib-dynload/ folder (eg _sha256.so) , which is likely the reason > why my setuptools install failed due to the error: I believe you need the developer libraries for that (including the C header files). IIRC "sudo apt-get build-dep python" should install all the required packages. HTH, -- Miki -- https://mail.python.org/mailman/listinfo/python-list
Re: Create flowcharts from Python
> Is there a library for Python that can easily create flowcharts using a > simple API? Maybe https://code.google.com/p/pydot/ ? -- https://mail.python.org/mailman/listinfo/python-list
Re: pyflakes best practices?
Greetings, > So, what's the best practice here? How do people deal with the false > positives? Is there some way to annotate the source code to tell > pyflakes to ignore something? We use flake8 (pyflakes + pep8) as pre step for the tests. We fail the tests on any output from flake8. flake8 supports ignoring certain lines by appending a comment starting with # NOQA HTH, -- Miki -- https://mail.python.org/mailman/listinfo/python-list
Re: daemon thread cleanup approach
Greetings, > Ok, so I have an issue with cleaning up threads upon a unexpected exit. What do you mean by "unexpected exit"? Uncaught exception? SIGTERM? ... > Using atexit doesn't work because it's called after the daemon threads are > killed. I don't follow. Who is killing the daemon threads? > ... > It it possible that this will cause the program to hang in any case? If due to a bug in the cleanup thread it hangs - the program will hang as well. All the best, -- Miki -- https://mail.python.org/mailman/listinfo/python-list
Re: [RELEASED] Python 2.7.7 release candidate 1
> (If you don't know what the strop > module is, go ahead and forget it now.) +1 QOTW :) -- https://mail.python.org/mailman/listinfo/python-list
Re: Running programs on mobile phones
> I have seen by chance a number of years ago a book on Python programming > for running on mobile phones (of a certain producer only). What is the > current state of the art in that? Could someone kindly give a few good > literature references? Thanks in advance. I'm not an expert, but take a look at http://kivy.org/. -- https://mail.python.org/mailman/listinfo/python-list
Re: [OT] Testing and credentials best practices?
>> How do you deal with tests (both on dev machine and Jenkins) that need >> credentials (such as AWS keys)?. > I've done several of these. Another option that may work in some > contexts is to mock the test altogether; Thanks, but mocking is last resort for me, it reduces the value of testing greatly and has the burden of catching up with the mocked service. -- https://mail.python.org/mailman/listinfo/python-list
[OT] Testing and credentials best practices?
Greetings, How do you deal with tests (both on dev machine and Jenkins) that need credentials (such as AWS keys)?. I know of the following methods: 1. Test user with known (stored in source control) limited credentials 2. ~/.secrets (or any other known location) RC file which is not in source control 3. Credentials service (such as ZooKeeper) accessed only from VPN 4. Credentials pre user encrypted (gpg) and stored in source control What method are you using? Are there any best practices in the subject? Thanks, -- Miki -- https://mail.python.org/mailman/listinfo/python-list
[ann] pypi2u - Get notified on new version of packages
Greetings, http://pypi2u.appspot.com/ is a simple service that notifies you on new versions of packages you're interested in. You can view the code, fill bugs and suggest ideas at https://bitbucket.org/tebeka/pypi2u Hope you find it useful, -- Miki -- https://mail.python.org/mailman/listinfo/python-list
Re: Python IM server
> I want to develop a instant message server, simply has user and group > entity. > Is there any better existing open-source one? > Thus I can download and have a look. You can take a look at Twisted Words (https://twistedmatrix.com/trac/wiki/TwistedWords). -- https://mail.python.org/mailman/listinfo/python-list
Re: system wide mutex
IIRC creating a directory is atomic in most environments. On Sunday, February 9, 2014 2:39:51 AM UTC-8, Asaf Las wrote: > Hi > > > > Which one is most recommended to use for mutex alike locking to > > achieve atomic access to single resource: > > > > - fcntl.lockf > > - os.open() with O_SHLOCK and O_EXLOCK > > - https://pypi.python.org/pypi/lockfile/0.9.1 > > - https://pypi.python.org/pypi/zc.lockfile/1.1.0 > > - any other ? > > > > Thanks > > > > /Asaf -- https://mail.python.org/mailman/listinfo/python-list
Re: generator slides review
> Thank you that's nicer, but ifiilterfalse is not in Python 3 (could > > use filter of course). It was renamed to filterfalse - http://docs.python.org/3.3/library/itertools.html#itertools.filterfalse -- https://mail.python.org/mailman/listinfo/python-list
Re: generator slides review
On Saturday, February 1, 2014 6:12:28 AM UTC-8, andrea crotti wrote: > I'm giving a talk tomorrow @Fosdem about generators/iterators/iterables.. > > > > The slides are here (forgive the strange Chinese characters): > > https://dl.dropboxusercontent.com/u/3183120/talks/generators/index.html#3 > > > > and the code I'm using is: > > https://github.com/AndreaCrotti/generators/blob/master/code/generators.py > > and the tests: > > https://github.com/AndreaCrotti/generators/blob/master/code/test_generators.py > > > > If anyone has any feedback or want to point out I'm saying something > > stupid I'd love to hear it before tomorrow (or also later I might give > > this talk again). > > Thanks My 2 cents: slide 4: [i*2 for i in range(10)] slide 9: while True: try: it = next(g) body(it) except StopIteration: break slide 21: from itertools import count, ifilterfalse def divided_by(p): return lambda n: n % p == 0 def primes(): nums = count(2) while True: p = next(nums) yield p nums = ifilterfalse(divided_by(p), nums) Another resource you can point to is http://www.dabeaz.com/generators/ Good luck. -- https://mail.python.org/mailman/listinfo/python-list
Re: Help me to print to screen as well as log
> I want that print "hello" should appear on screen as well as get saved in a
> log file.
> How can I accomplish this?
There are many ways to do this, here's one:
class MultiWriter(object):
def __init__(self, *writers):
self.writers = writers
self.isatty = False
def write(self, data):
for writer in self.writers:
writer.write(data)
def flush(self):
for writer in self.writers:
writer.flush()
out = open('/tmp/log.txt', 'a')
import sys
sys.stdout = MultiWriter(sys.stdout, out)
print('hello')
IMO you're better with the logging package, see
http://docs.python.org/2/howto/logging.html#configuring-logging for more
details.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Suggest an open-source issue tracker, with github integration and kanban boards?
On Wednesday, November 20, 2013 6:36:56 AM UTC-8, Alec Taylor wrote: > Anyway, here is the link: https://github.com/rauhryan/huboard I thought you wanted a Python bases solution. -- https://mail.python.org/mailman/listinfo/python-list
Re: Suggest an open-source issue tracker, with github integration and kanban boards?
> Can you recommend an open source project (or two) written in Python; > which covers multi project + sub project issue tracking linked across > github repositories? Don't know if it covers all what you need, but http://trac.edgewall.org/ is written in Python, and has many, many plugins. -- https://mail.python.org/mailman/listinfo/python-list
Re: RELEASED: Python 2.6.9 final
On Tuesday, October 29, 2013 10:48:58 AM UTC-7, Barry Warsaw wrote: > So too has my latest stint as Python Release Manager. Over the 19 years I > have been involved with Python, Thanks Barry for all the hard work. -- https://mail.python.org/mailman/listinfo/python-list
Re: Which DLL did fail to load
> Wouldn't it > better to add a feature to Python to write the name of DLL which load > has been failed? If you start Python with the -v (verbose) flag, you can see all the calls to dlopen. -- http://mail.python.org/mailman/listinfo/python-list
Re: [argparse] mutually exclusive group with 2 sets of options
> Is it possible with argparse to have this syntax for a script? > my-script (-a -b VALUE-B | -c -d VALUE-D) > > I would like to do this with the argparse module. You can probably do something similar using sub commands (http://docs.python.org/2/library/argparse.html#sub-commands). -- http://mail.python.org/mailman/listinfo/python-list
Re: Imports (in Py3), please help a novice
> Is there an import / distutils tutorial out there? I'm looking for it, but > perhaps one of you already knows where to find it. Thanks! Did you have a look at http://docs.python.org/3.3/distutils/examples.html? Another thing to do is to look at what other packages on PyPi are doing. -- http://mail.python.org/mailman/listinfo/python-list
Re: Any speech to text conversation python library for Linux and mac box
On Wednesday, June 12, 2013 8:59:44 PM UTC-7, Ranjith Kumar wrote: > I'm looking for speech to text conversation python library for linux and mac Not a Python library, but maybe you can work with http://cmusphinx.sourceforge.net/ -- http://mail.python.org/mailman/listinfo/python-list
Re: python call golang
> Now! I have written a python script . I want to call a golang script in > python script. > Who can give me some advices? See http://gopy.qur.me/extensions/examples.html and http://www.artima.com/weblogs/viewpost.jsp?thread=333589 -- http://mail.python.org/mailman/listinfo/python-list
Re: distributing a binary package
> I already have the .so files compiled. http://docs.python.org/2/distutils/setupscript.html#installing-package-data ? -- http://mail.python.org/mailman/listinfo/python-list
Re: distributing a binary package
> Basically, I'd like to know how to create a proper setup.py script http://docs.python.org/2/distutils/setupscript.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Using SciPy in application
> I want to use spline interpolation function from SciPy in an application and > at the same time, I don't want the end user to have to install SciPy > separately. You can pack you application with py2exe, pyinstaller ... and then they won't even need to install Python. Another option (which is not always possible) is to make you application a web site and then only you need to install SciPy on the server. -- http://mail.python.org/mailman/listinfo/python-list
Re: Encoding NaN in JSON
> > You understand that this will result in a chunk of text that is not JSON?
> I think he means something like this:
> >>> json.dumps([float('nan')])
> '["N/A"]'
That's exactly what I mean :)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Encoding NaN in JSON
[Roland] > yes, there is: subclass+extend the JSON-encoder, see pydoc json. Please read the original post before answering. What you suggested does not work since NaN is of float type. -- http://mail.python.org/mailman/listinfo/python-list
Re: Encoding NaN in JSON
> >>> I'm trying to find a way to have json emit float('NaN') as 'N/A'.
> Easiest way is probably to transform your object before you try to write
Yeah, that's what I ended up doing. Wondered if there's a better way ...
Thanks,
--
Miki
--
http://mail.python.org/mailman/listinfo/python-list
Re: Encoding NaN in JSON
>> I'm trying to find a way to have json emit float('NaN') as 'N/A'.
> No. There is no way to represent NaN in JSON. It's simply not part of the
> specification.
I know that. I'm trying to emit the *string* 'N/A' for every NaN.
--
http://mail.python.org/mailman/listinfo/python-list
Encoding NaN in JSON
Greetings,
I'm trying to find a way to have json emit float('NaN') as 'N/A'.
I can't seem to find a way since NaN is a float, which means overriding
"default" won't help.
Any simple way to do this?
Thanks,
--
Miki
--
http://mail.python.org/mailman/listinfo/python-list
Re: [ANNC] pynguin-0.14 python turtle graphics application
> Pynguin is a python-based turtle graphics application. I wonder why Pynguin does not get more traction in the teaching sector. Looks ideal for teaching kids. -- http://mail.python.org/mailman/listinfo/python-list
Re: CSV to matrix array
> I have a CSV file with 20 rows and 12 columns and I need to store it as a > matrix. If you can use pandas, the pandas.read_csv is what you want. -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie to python. Very newbie question
> I can't even read that mess... three nested lambda? I have to say this and other answers in this thread seem not that friendly to me. The OP said it's a newbie question, we should be more welcoming to newcomers. -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie to python. Very newbie question
> I am a newbie to python Welcome! I hope you'll do great things with Python. > and am trying to write a program that does a > sum of squares of numbers whose squares are odd. OK. > For example, for x from 1 to 100, it generates 165 as an output (sum > of 1,9,25,49,81) I don't follow, you seem to be missing a lot of numbers. For example 3^2 = 9 which is odd as well. > Here is the code I have > print reduce(lambda x, y: x+y, filter(lambda x: x%2, map(lambda x: > x*x, xrange > (10**6 = sum(x*x for x in xrange(1, 10**6, 2)) print X = Y is a syntax error. Why do you need the 2'nd part. In general, we're moving to list/generator comperhension over map/filter. Something like: print(sum(x*x for x in xrange(10**6) if (x*x)%2)) HTH, Miki -- http://mail.python.org/mailman/listinfo/python-list
Re: How to find bad row with db api executemany()?
> I can catch the exception, but don't see any way to tell which row caused the
> problem. Is this information obtainable, short of retrying each row one by
> one?
One way to debug this is to wrap the iterable passed to executemany with one
that remembers the last line. Something like:
class LastIterator(object):
def __init__(self, coll):
self.it = iter(coll)
self.last = None
def __iter__(self):
return self
def next(self):
self.last = next(self.it)
return self.last
...
li = ListIterator(items)
try:
cursor.executemany(sql, li)
except SQLError, e:
print('Error: {}, row was {}'.format(e, li.last))
--
http://mail.python.org/mailman/listinfo/python-list
Re: No errors displayed but i blank scren nstead.
> Fianlly my cgi .py script doesnt produce any more errors, i think i ahve > correct them but it present a blank screen > Any idea why? Please post the code to the script, otherwise we can't help you. -- http://mail.python.org/mailman/listinfo/python-list
Re: Binary for numpy 1.7.0 with Python 2.7.3
> I'm trying to update to both 2.7.3 and Numpy 1.7.0. Updating Python is from python.org If you're on 64bit windows, see http://www.lfd.uci.edu/~gohlke/pythonlibs/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Top 10 python features
> I would like to know what are the top 10 most important features (on your > opinion) in python. You're in luck :) Raymond Hettinger just gave "Python is Awesome" keynote at PyCon. You can view the slides at https://speakerdeck.com/pyconslides/pycon-keynote-python-is-awesome, video will follow - not sure when. -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools.filterfalse - what is it good for
> can anybody point out a situation where you really need > itertools.filterfalse() ? Sometimes you get the predicate as a parameter to another function. This way if you want to filter out things you can easily do it. Other language (such as Clojure) have a "complement" function that removes the need of filterfalse. For example (Python 3): def percent_spam(is_spam, documents): n_spam = sum(1 for _ in filter(is_spam, documents)) n_ham = sum(1 for _ in filterfalse(is_spam, documents)) return float(n_spam) / (n_ham + n_spam) -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a graphical GUI builder?
> I'm wondering if there's a utility for Python to build GUIs. IIRC the Qt builder can generate Python code. -- http://mail.python.org/mailman/listinfo/python-list
Re: The best, friendly and easy use Python Editor.
On Thursday, January 24, 2013 2:34:45 AM UTC-8, mik...@gmail.com wrote: > On Thursday, January 24, 2013 9:43:31 AM UTC, Hazard Seventyfour wrote: > > for all senior can you suggest me the best, friendly and easy use with nice > > GUI editor for me, and have many a good features such as auto complete/auto > > correct. > I personally like Eclipse as I use it for most of my projects (not only > Python) so I use Eclipse + PyDev plug-in for Python. Aptana is doing a great job bundling Eclipse + PyDev (+ other goodies). -- http://mail.python.org/mailman/listinfo/python-list
Re: How to get the selected text of the webpage in chrome through python ?
On Monday, January 7, 2013 8:20:28 PM UTC-8, iMath wrote: > How to get the selected text of the webpage in chrome through python ? You can probably use selenium to do that. -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbee question about running a script
On Sunday, January 6, 2013 8:03:43 AM UTC-8, marc.assin wrote: > I wonder how I could specify a parameter on the command line from > within the interpreter. Guido wrote some advice a while back - http://www.artima.com/weblogs/viewpost.jsp?thread=4829 Import your module and call its main. The other way is to execute in another process: from subprocess import check_call import sys check_call([sys.executable, 'myscript.py', 'arg1', 'arg2']) -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie problem with Python pandas
On Sunday, January 6, 2013 5:57:17 AM UTC-8, RueTheDay wrote: > I am getting the following error when running on Python 2.7 on Ubuntu > 12.04: > >> > > AttributeError: 'Series' object has no attribute 'str' I would *guess* that you have an older version of pandas on your Linux machine. Try "print(pd.__version__)" to see which version you have. Also, trying asking over at https://groups.google.com/forum/?fromgroups=#!forum/pydata which is more dedicated to pandas. -- http://mail.python.org/mailman/listinfo/python-list
Re: compile python 3.3 with bz2 support
On Thursday, December 20, 2012 10:27:54 PM UTC-8, Isml wrote: > I want to compile python 3.3 with bz2 support on RedHat 5.5 but fail to > do that. Here is how I do it: > 1. download bzip2 and compile it(make、make -f Makefile_libbz2_so、make > install) Why can't you use yum? (yum install libbz2-dev) > [root@localhost Python-3.3.0]# python3 -c "import bz2" > Traceback (most recent call last): > File "", line 1, in > File "/usr/local/lib/python3.3/bz2.py", line 21, in > from _bz2 import BZ2Compressor, BZ2Decompressor > ImportError: No module named '_bz2' IMO it's a linker problem. If libbz2.zo is in /usr/local/lib then try LB_LIBRARY_PATH=/usr/local/lib python3 -c 'import bz2' If this work, you can add /usr/local/lib to the linker by doing: echo /usr/local/lib >> /etc/ld.so.conf.d/local.conf ldconfig Having said that, if the yum way work - do it. > By the way, RedHat 5.5 has a built-in python 2.4.3. Would it be a problem? I don't think so. I have multiple version of Python on RedHat systems. -- http://mail.python.org/mailman/listinfo/python-list
Re: Build and runtime dependencies
On Thursday, December 20, 2012 2:11:45 PM UTC-8, Jack Silver wrote: > I have two Linux From Scratch machine. > Hence, I do not need to install all those libraries on the client machine. > Right ? It depends on what the client needs. For example if you use zlib compression in the protocol, you'll need this in the client as well to uncompress. I suggest you use the exact same Python builds on both machines, it'll save you some headache in the future. -- http://mail.python.org/mailman/listinfo/python-list
Re: [newbie] plotting pairs of data
On Wednesday, December 19, 2012 6:38:30 AM UTC-8, Thomas Bach wrote: > On Wed, Dec 19, 2012 at 05:47:30AM -0800, hugocoolens wrote: > > ['0.0364771 0.55569', '0.132688 0.808496', '0.232877 0.832833', > > '0.332702 0.849128', '0.432695 0.862158'] > xs = [ float(x) for x, _ in map(str.split, l) ] > ys = [ float(y) for _, y in map(str.split, l) ] Let's play golf :) xs, ys = zip(*(map(float, s.split()) for s in l)) -- http://mail.python.org/mailman/listinfo/python-list
Re: Help with unittest2
On Thursday, December 13, 2012 7:03:27 AM UTC-8, Daniel Laird wrote: > I do am import unittest2 as unittest > NameError: global name 'assertListEqual' is not defined According to the docs (http://docs.python.org/2/library/unittest.html#unittest.TestCase.addTypeEqualityFunc) assertListEqual and friends was added in 2.7. You can use assertEuqal, or if you don't care about order assertEqual(sorted(a), sorted(b)). -- http://mail.python.org/mailman/listinfo/python-list
Re: Running a Python app on a remote server and displaying the output files
On Saturday, December 8, 2012 12:22:35 PM UTC-8, Jason Hsu wrote: > 1. How do I run my Python script in Google App Engine and make the output > results.csv file publicly available? Probably https://developers.google.com/appengine/docs/python/blobstore/, however https://developers.google.com/appengine/docs/python/gettingstarted/staticfiles might work as well. -- http://mail.python.org/mailman/listinfo/python-list
Re: Cosine Similarity
On Thursday, December 6, 2012 2:15:53 PM UTC-8, subhaba...@gmail.com wrote: > I am looking for some example of implementing Cosine similarity in python. I > searched for hours but could not help much. NLTK seems to have a module but > did not find examples. Should be easy with numpy: import numpy as np def cos(v1, v2): return np.dot(v1, v2) / (np.sqrt(np.dot(v1, v1)) * np.sqrt(np.dot(v2, v2))) HTH, -- Miki -- http://mail.python.org/mailman/listinfo/python-list
Re: mini browser with python
> In other words I need a mini, simple browser; > something I can build that will open, read and > display a saved html file. If you want to view the "raw" HTML, use any editor. If you want to view the rendered HTML (like in a browser), you can point your favorite browser to a local file or use something like Qt+Webkit. -- http://mail.python.org/mailman/listinfo/python-list
Re: Remote server: running a Python script and making *.csv files publicly available
On Wednesday, December 5, 2012 11:14:42 AM UTC-8, Jason Hsu wrote: > make the *.csv files publicly available to read. > Can this be done on Heroku? I've gone through the tutorial, but it seems to > be geared towards people who want to create a whole web site. See one option - http://stackoverflow.com/questions/8470733/how-can-i-handle-static-files-with-python-webapp2-in-heroku > If Heroku isn't the solution for me, what are the alternatives? I tried > Google App Engine, but it requires Python 2.5 and won't work with 2.7. AppEngine does support Python 2.7, see https://developers.google.com/appengine/docs/python/runtime -- http://mail.python.org/mailman/listinfo/python-list
Re: Help "joining" two files delimited with pipe character ("|")
On Wednesday, December 5, 2012 9:57:31 AM UTC-8, Daniel Doo wrote: > I am new to Python. Is there a method to “join” two pipe delimited files > using a unique key that appears in both files? Have a look at Panda's concat (http://pandas.pydata.org/pandas-docs/dev/merging.html). It also have utilities to read delimited files into DataFrame. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Cluster
On Tuesday, December 4, 2012 11:04:15 AM UTC-8, subhaba...@gmail.com wrote: > >>> cl = HierarchicalClustering(data, lambda x,y: abs(x-y)) > but now I want to visualize it if any one suggest how may I use > visualization(like matplotlib or pyplot etc.) to see the data? One option is to use a scatter plot with different color per cluster. See the many examples in http://matplotlib.org/gallery.html. HTH, -- Miki -- http://mail.python.org/mailman/listinfo/python-list
Re: how to pass "echo t | " input to subprocess.check_output() method
On Wednesday, November 28, 2012 4:38:35 AM UTC-8, dach...@gmail.com wrote: > Thanks.. Creating two subprocesses worked for me. I did the code as below, Glad it worked. -- http://mail.python.org/mailman/listinfo/python-list
Re: "non central" package management
On Tuesday, November 27, 2012 8:45:56 PM UTC-8, Roy Smith wrote: > In the future, the plan is to build a complete fresh virtualenv for > every deployment. But we're not there yet. Maybe a repository of virtualenvs, then when deploying you can see if there's one the matches what you need and use it. Otherwise create a new one. -- http://mail.python.org/mailman/listinfo/python-list
