Re: [OT] Usage of U+00B6 PILCROW SIGN (was: generator slides review and Python doc (+/- text bug))

2014-02-04 Thread andrea crotti
2014-02-04 : > Le mardi 4 février 2014 15:39:54 UTC+1, Jerry Hill a écrit : > > Useless and really ugly. > I think this whole discussion is rather useless instead, why do you care since you're not going to use this tool anyway? -- https://mail.python.org/mailman/listinfo/python-list

Re: generator slides review

2014-02-03 Thread andrea crotti
2014-02-03 Terry Reedy : > On 2/2/2014 5:40 AM, andrea crotti wrote: >> > In general, use assert (== AssertionError) to check program logic (should > never raise). Remember that assert can be optimized away. Use other > exceptions to check user behavior. So I believe

Re: generator slides review and Python doc (+/- text bug)

2014-02-03 Thread andrea crotti
2014-02-03 : > generator slides review and Python doc > > > I do not know what tool is used to produce such > slides. > > When the mouse is over a a text like a title ( ... <\H*> ???) > the text get transformed and a colored eol is appearing. > > Example with the slide #3: > > Even numbers > becom

Re: generator slides review

2014-02-02 Thread andrea crotti
Thanks everyone for your feedback. The talk I think went well, maybe I was too fast because I only used 21 minutes. >From the audience feedback, there were some questions about my "Buggy code" example, so yes probably it's not a good example since it's too artificial. I'll have to find something

Re: generator slides review

2014-02-02 Thread andrea crotti
which explain things, to tell a clear story in a way 2014-02-02 andrea crotti : > The slides are updated now > > 2014-02-02 andrea crotti : >> 2014-02-01 Miki Tebeka : >>> >>> My 2 cents: >>> >>> slide 4: >>> [i*2 for i in range(10)]

Re: generator slides review

2014-02-02 Thread andrea crotti
The slides are updated now 2014-02-02 andrea crotti : > 2014-02-01 Miki Tebeka : >> >> My 2 cents: >> >> slide 4: >> [i*2 for i in range(10)] >> > > Well this is not correct in theory because the end should be the max > number, not the number of e

Re: generator slides review

2014-02-02 Thread andrea crotti
2014-02-01 Miki Tebeka : > > My 2 cents: > > slide 4: > [i*2 for i in range(10)] > Well this is not correct in theory because the end should be the max number, not the number of elements. So it should be [i*2 for i in range(10/2)] which might be fine but it's not really more clear imho.. > slide

Re: generator slides review

2014-02-02 Thread andrea crotti
2014-02-02 Terry Reedy : > On 2/1/2014 9:12 AM, andrea crotti wrote: > > Comments: > > The use is assert in the first slide seem bad in a couple of different > respects. > Why is it bad? It's probably not necessary but since we ask for a range it might be good to c

generator slides review

2014-02-01 Thread andrea crotti
I'm giving a talk tomorrow @Fosdem about generators/iterators/iterables.. The slides are here (forgive the strange Chinese characters): https://dl.dropboxusercontent.com/u/3183120/talks/generators/index.html#3 and the code I'm using is: https://github.com/AndreaCrotti/generators/blob/master/code/

Re: how to avoid spaghetti in Python?

2014-01-21 Thread andrea crotti
2014/1/21 CM : > I've been learning and using Python for a number of years now but never > really go particularly disciplined about all good coding practices. I've > definitely learned *some*, but I'm hoping this year to take a good step up in > terms of refactoring, maintainability, and mostly

Re: lxml tostring quoting too much

2013-08-07 Thread andrea crotti
2013/8/6 Chris Down : > On 2013-08-06 18:38, andrea crotti wrote: >> I would really like to do the following: >> >> from lxml import etree as ET >> from lxml.builder import E >> >> url = "http://something?x=10&y=20"; >> l = E

lxml tostring quoting too much

2013-08-06 Thread andrea crotti
I would really like to do the following: from lxml import etree as ET from lxml.builder import E url = "http://something?x=10&y=20"; l = E.link(url) ET.tostring(l) -> "http://something?x=10&y=20" However the lxml tostring always quotes the &, I can't find a way to tell it to avoid quoting it. Is

Re: decorator to fetch arguments from global objects

2013-06-18 Thread andrea crotti
2013/6/18 Terry Reedy > On 6/18/2013 5:47 AM, andrea crotti wrote: > >> Using a CouchDB server we have a different database object potentially >> for every request. >> >> We already set that db in the request object to make it easy to pass it >> around form

Re: decorator to fetch arguments from global objects

2013-06-18 Thread andrea crotti
2013/6/18 Wolfgang Maier > andrea crotti gmail.com> writes: > > > > > > > Using a CouchDB server we have a different database object potentially > for > every request. > > > > We already set that db in the request object to make it easy to pass it &

Re: python-django for dynamic web survey?

2013-06-18 Thread andrea crotti
Django makes your life a lot easier in many ways, but you still need some time to learn it. The task you're trying it's not trivial though, depending on your experience it might take a while with any library/framework.. -- http://mail.python.org/mailman/listinfo/python-list

decorator to fetch arguments from global objects

2013-06-18 Thread andrea crotti
Using a CouchDB server we have a different database object potentially for every request. We already set that db in the request object to make it easy to pass it around form our django app, however it would be nice if I could set it once in the API and automatically fetch it from there. Basically

Re: Getting a callable for any value?

2013-05-29 Thread andrea crotti
On 05/29/2013 06:46 PM, Croepha wrote: Is there anything like this in the standard library? class AnyFactory(object): def __init__(self, anything): self.product = anything def __call__(self): return self.product def __repr__(self): return "%s.%s(%r)" % (self.__class__.__module__, self.__class__

pip and different branches?

2013-05-20 Thread andrea crotti
We use github and we work on many different branches at the same time. The problem is that we have >5 repos now, and for each repo we might have the same branches on all of them. Now we use pip and install requirements such as: git+ssh://g...@github.com/repo.git@dev Now the problem is that the r

Re: dynamic forms generation

2013-04-19 Thread andrea crotti
Well I think since we are using django anyway (and bottle on the API side) I'm not sure why we would use flask forms for this.. Anyway the main question is probably, is it worth to try to define a DSL or not? The problem I see is that we have a lot and very complex requirements, trying to define a

dynamic forms generation

2013-04-16 Thread andrea crotti
We are re-designing a part of our codebase, which should in short be able to generate forms with custom fields. We use django for the frontend and bottle for the backend (using CouchDB as database), and at the moment we simply plug extra fields on normal django forms. This is not really scalable,

Re: Mixin way?

2013-04-03 Thread andrea crotti
2013/4/3 Steven D'Aprano > [snip] > > So, if you think of "Visitable" as a gadget that can be strapped onto > your MyObj as a component, then composition is probably a better design. > But if you think of "Visitable" as a mere collection of behaviour and > state, then a mixin is probably a better

Mixin way?

2013-04-03 Thread andrea crotti
I have some classes that have shared behaviours, for example in our scenario an object can be "visited", where something that is visitable would have some behaviour like --8<---cut here---start->8--- class Visitable(Mixin): FIELDS = { 'visits': [],

Re: groupby behaviour

2013-02-26 Thread andrea crotti
2013/2/26 Ian Kelly : > On Tue, Feb 26, 2013 at 9:27 AM, andrea crotti > wrote: >> So I was trying to use groupby (which I used in the past), but I >> noticed a very strange thing if using list on >> the result: > > As stated in the docs: > > ""&qu

Re: running multiple django/bottle instances

2013-01-07 Thread andrea crotti
che, your problem > is actually not Python related. > If you want to run your applications on different ports, take a look on e.g. > Apaches virtual host configurations. > http://httpd.apache.org/docs/2.2/vhosts/examples.html > > Am 03.01.2013 17:35, schrieb Andrea Crotti: > &

running multiple django/bottle instances

2013-01-03 Thread Andrea Crotti
I'm working on a quite complex web app that uses django and bottle (bottle for the API which is also restful). Before I came they started to use a staging server to be able to try out things properly before they get published, but now we would like to have the possibility to see multiple branches

Re: forking and avoiding zombies!

2012-12-11 Thread andrea crotti
2012/12/11 Dennis Lee Bieber : > On Tue, 11 Dec 2012 10:34:23 -0300, peter declaimed > the following in gmane.comp.python.general: > >> >> stderrfile = '%s/error.log' % os.getcwd() >> stdoutfile = '%s/out.log' % os.getcwd() >> > Ouch... > > stdoutfile = os.path.join(os.getcwd(), "o

Re: forking and avoiding zombies!

2012-12-11 Thread andrea crotti
2012/12/11 Jean-Michel Pichavant : > - Original Message - >> So I implemented a simple decorator to run a function in a forked >> process, as below. >> >> It works well but the problem is that the childs end up as zombies on >> one machine, while strangely >> I can't reproduce the same on m

Re: forking and avoiding zombies!

2012-12-11 Thread andrea crotti
2012/12/11 peter : > On 12/11/2012 10:25 AM, andrea crotti wrote: >> >> Ah sure that makes sense! >> >> But actually why do I need to move away from the current directory of >> the parent process? >> In my case it's actually useful to be in the same

Re: forking and avoiding zombies!

2012-12-11 Thread andrea crotti
Ah sure that makes sense! But actually why do I need to move away from the current directory of the parent process? In my case it's actually useful to be in the same directory, so maybe I can skip that part, or otherwise I need another chdir after.. -- http://mail.python.org/mailman/listinfo/pyth

Re: forking and avoiding zombies!

2012-12-11 Thread andrea crotti
Yes I wanted to avoid to do something too complex, anyway I'll just comment it well and add a link to the original code.. But this is now failing to me: def daemonize(stdin='/dev/null', stdout='/dev/null', stderr='/dev/null'): # Perform first fork. try: pid = os.fork() if

Re: creating size-limited tar files

2012-11-14 Thread Andrea Crotti
On 11/14/2012 04:33 PM, Dave Angel wrote: Well, as I said, I don't see how the particular timing has anything to do with the rest of the thread. If you want to do an ls within a Python program, go ahead. But if all you need can be done with ls itself, then it'll be slower to launch python just

Re: creating size-limited tar files

2012-11-14 Thread andrea crotti
2012/11/14 Dave Angel : > On 11/14/2012 10:56 AM, andrea crotti wrote: >> Ok this is all very nice, but: >> >> [andrea@andreacrotti tar_baller]$ time python2 test_pipe.py > /dev/null >> >> real 0m21.215s >> user 0m0.750s >> sys 0m1.703s >>

Re: creating size-limited tar files

2012-11-14 Thread andrea crotti
2012/11/14 Kushal Kumaran : > > Well, well, I was wrong, clearly. I wonder if this is fixable. > > -- > regards, > kushal > -- > http://mail.python.org/mailman/listinfo/python-list But would it not be possible to use the pipe in memory in theory? That would be way faster and since I have in theor

Re: creating size-limited tar files

2012-11-08 Thread andrea crotti
2012/11/8 andrea crotti : > > > > Yes yes I saw the answer, but now I was thinking that what I need is > simply this: > tar czpvf - /path/to/archive | split -d -b 100M - tardisk > > since it should run only on Linux it's probably way easier, my script > will then

Re: creating size-limited tar files

2012-11-08 Thread andrea crotti
2012/11/7 Oscar Benjamin : > > Correct. But if you read the rest of Alexander's post you'll find a > suggestion that would work in this case and that can guarantee to give > files of the desired size. > > You just need to define your own class that implements a write() > method and then distributes

Re: creating size-limited tar files

2012-11-07 Thread Andrea Crotti
On 11/07/2012 08:32 PM, Roy Smith wrote: In article <509ab0fa$0$6636$9b4e6...@newsspool2.arcor-online.net>, Alexander Blinne wrote: I don't know the best way to find the current size, I only have a general remark. This solution is not so good if you have to impose a hard limit on the resulti

Re: accepting file path or file object?

2012-11-05 Thread andrea crotti
2012/11/5 Peter Otten <__pete...@web.de>: > I sometimes do something like this: > > $ cat xopen.py > import re > import sys > from contextlib import contextmanager > > @contextmanager > def xopen(file=None, mode="r"): > if hasattr(file, "read"): > yield file > elif file == "-": >

lazy properties?

2012-11-01 Thread Andrea Crotti
Seeing the wonderful "lazy val" in Scala I thought that I should try to get the following also in Python. The problem is that I often have this pattern in my code: class Sample: def __init__(self): self._var = None @property def var(self): if self._var is None:

Re: Nice solution wanted: Hide internal interfaces

2012-10-30 Thread andrea crotti
2012/10/30 alex23 : > On Oct 30, 2:33 am, Johannes Bauer wrote: >> I'm currently looking for a good solution to the following problem: I >> have two classes A and B, which interact with each other and which >> interact with the user. Instances of B are always created by A. >> >> Now I want A to ca

Re: Nice solution wanted: Hide internal interfaces

2012-10-29 Thread andrea crotti
2012/10/29 Johannes Bauer : > Hi there, > > I'm currently looking for a good solution to the following problem: I > have two classes A and B, which interact with each other and which > interact with the user. Instances of B are always created by A. > > Now I want A to call some private methods of B

Re: Immutability and Python

2012-10-29 Thread andrea crotti
2012/10/29 Chris Angelico : > On Tue, Oct 30, 2012 at 2:55 AM, Paul Rubin wrote: >> andrea crotti writes: >>> and we want to change its state incrementing the number ... >>> the immutability purists would instead suggest to do this: >>> def increment(

Re: Immutability and Python

2012-10-29 Thread andrea crotti
2012/10/29 Jean-Michel Pichavant : > > > In an OOP language num.increment() is expected to modify the object in place. > So I think you're right when you say that functional languages technics do > not necessarily apply to Python, because they don't. > > I would add that what you're trying to sugg

Re: Immutability and Python

2012-10-29 Thread andrea crotti
2012/10/29 andrea crotti : >> > > Well sure but it doesn't modify the first object, just creates a new > one. There are in general good reasons to do that, for example I can > then compose things nicely: > > num.increment().increment() > > or I can parallelize

Re: Immutability and Python

2012-10-29 Thread andrea crotti
2012/10/29 Jean-Michel Pichavant : > > "return NumWrapper(self.number + 1) " > > still returns a(nother) mutable object. > > So what's the point of all this ? > > JM > Well sure but it doesn't modify the first object, just creates a new one. There are in general good reasons to do that, for examp

Re: resume execution after catching with an excepthook?

2012-10-25 Thread andrea crotti
2012/10/25 Steven D'Aprano : > On Wed, 24 Oct 2012 13:51:30 +0100, andrea crotti wrote: > >> So I would like to be able to ask for confirmation when I receive a C-c, >> and continue if the answer is "N/n". > > I don't think there is any way to do thi

resume execution after catching with an excepthook?

2012-10-24 Thread andrea crotti
So I would like to be able to ask for confirmation when I receive a C-c, and continue if the answer is "N/n". I'm already using an exception handler set with sys.excepthook, but I can't make it work with the confirm_exit, because it's going to quit in any case.. A possible solution would be to do

Re: locking files on Linux

2012-10-19 Thread andrea crotti
2012/10/18 Oscar Benjamin : > > The lock is cooperative. It does not prevent the file from being > opened or overwritten. It only prevents any other process from > obtaining the lock. Here you open the file with mode 'w' which > truncates the file instantly (without checking for the lock). > > > Os

Re: Testing against multiple versions of Python

2012-10-19 Thread andrea crotti
2012/10/19 Michele Simionato : > Yesterday I released a new version of the decorator module. It should run > under Python 2.4, 2.5, 2.6, 2.7, 3.0, 3.1, 3.2, 3.3. I did not have the will > to install on my machine 8 different versions of Python, so I just tested it > with Python 2.7 and 3.3. But

Re: locking files on Linux

2012-10-18 Thread andrea crotti
2012/10/18 Oscar Benjamin : > > Why not come up with a test that actually shows you if it works? Here > are two suggestions: > > 1) Use time.sleep() so that you know how long the lock is held for. > 2) Write different data into the file from each process and see what > you end up with. > Ok thank

Re: locking files on Linux

2012-10-18 Thread andrea crotti
2012/10/18 Grant Edwards : > On 2012-10-18, andrea crotti wrote: > > > File locks under Unix have historically been "advisory". That means > that programs have to _choose_ to pay attention to them. Most > programs do not. > > Linux does support mandatory lock

Re: Generating C++ code

2012-10-10 Thread andrea crotti
2012/10/10 Jean-Michel Pichavant : > Well, the C++ code will end up running on a MIPS on a SOC, unfortunately, > python is not an option here. > The xml to C++ makes a lot of sense, because only a small part of the code is > generated that way (everything related to log & fatal events). Everythin

Re: Generating C++ code

2012-10-09 Thread Andrea Crotti
On 10/09/2012 05:00 PM, Jean-Michel Pichavant wrote: Greetings, I'm trying to generate C++ code from an XML file. I'd like to use a template engine, which imo produce something readable and maintainable. My google search about this subject has been quite unsuccessful, I've been redirected to t

Re: PHP vs. Python

2012-09-25 Thread andrea crotti
2012/9/25 : > On Thursday, 23 December 2004 03:33:36 UTC+5:30, (unknown) wrote: >> Anyone know which is faster? I'm a PHP programmer but considering >> getting into Python ... did searches on Google but didn't turn much up >> on this. >> >> Thanks! >> Stephen > > > Here some helpful gudance. > >

Re: Python presentations

2012-09-24 Thread andrea crotti
For anyone interested, I already moved the slides on github (https://github.com/AndreaCrotti/pyconuk2012_slides) and for example the decorator slides will be generated from this: https://raw.github.com/AndreaCrotti/pyconuk2012_slides/master/deco_context/deco.rst Notice the literalinclude with :py

Re: 'str' object does not support item assignment

2012-09-23 Thread Andrea Crotti
On 09/23/2012 07:31 PM, jimbo1qaz wrote: spots[y][x]=mark fails with a "'str' object does not support item assignment" error,even though: a=[["a"]] a[0][0]="b" and: a=[["a"]] a[0][0]=100 both work. Spots is a nested list created as a copy of another list. But a = "a" a[0] = 'c' fails f

Re: Python presentations

2012-09-19 Thread andrea crotti
2012/9/19 Trent Nelson : > > FWIW, I gave a presentation on decorators to the New York Python > User Group back in 2008. Relevant blog post: > > http://blogs.onresolve.com/?p=48 > > There's a link to the PowerPoint presentation I used in the first > paragraph. It's in .ppt

Re: subprocess call is not waiting.

2012-09-19 Thread andrea crotti
2012/9/19 Hans Mulder : > Yes: using "top" is an observation problem. > > "Top", as the name suggests, shows only the most active processes. Sure but "ls -lR /" is a very active process if you try to run it.. Anyway as written below I don't need this anymore. > > It's quite possible that your 'ls

Re: subprocess call is not waiting.

2012-09-19 Thread andrea crotti
2012/9/18 Dennis Lee Bieber : > > Unless you have a really massive result set from that "ls", that > command probably ran so fast that it is blocked waiting for someone to > read the PIPE. I tried also with "ls -lR /" and that definitively takes a while to run, when I do this: proc = subp

Re: subprocess call is not waiting.

2012-09-18 Thread andrea crotti
I have a similar problem, something which I've never quite understood about subprocess... Suppose I do this: proc = subprocess.Popen(['ls', '-lR'], stdout=subprocess.PIPE, stderr=subprocess.PIPE) now I created a process, which has a PID, but it's not running apparently... It only seems to run whe

Re: Decorators not worth the effort

2012-09-14 Thread andrea crotti
2012/9/14 Chris Angelico : > > Trouble is, you're starting with a pretty poor algorithm. It's easy to > improve on what's poor. Memoization can still help, but I would start > with a better algorithm, such as: > > def fib(n): > if n<=1: return 1 > a,b=1,1 > for i in range(1,

Re: Decorators not worth the effort

2012-09-14 Thread andrea crotti
I think one very nice and simple example of how decorators can be used is this: def memoize(f, cache={}, *args, **kwargs): def _memoize(*args, **kwargs): key = (args, str(kwargs)) if not key in cache: cache[key] = f(*args, **kwargs) return cache[key] r

Re: Python presentations

2012-09-13 Thread Andrea Crotti
On 09/13/2012 11:58 PM, Miki Tebeka wrote: What do you think work best in general? I find typing during class (other than small REPL examples) time consuming and error prone. What works well for me is to create a slidy HTML presentation with asciidoc, then I can include code snippets that can

Re: Python presentations

2012-09-13 Thread andrea crotti
2012/9/13 William R. Wing (Bill Wing) : > > [byte] > > Speaking from experience as both a presenter and an audience member, please > be sure that anything you demo interactively you include in your slide deck > (even if only as an addendum). I assume your audience will have access to > the deck

Re: main and dependent objects

2012-09-13 Thread andrea crotti
2012/9/13 Jean-Michel Pichavant : > > Nothing shocking right here imo. It looks like a classic parent-child > implementation. > However it seems the relation between Obj and Dependent are 1-to-1. Since > Dependent need to access all Obj attributes, are you sure that Dependent and > Obj are not a

main and dependent objects

2012-09-13 Thread andrea crotti
I am in a situation where I have a class Obj which contains many attributes, and also contains logically another object of class Dependent. This dependent_object, however, also needs to access many fields of the original class, so at the moment we did something like this: class Dependent: de

Re: pyQT performance?

2012-09-10 Thread Andrea Crotti
On 09/10/2012 07:29 PM, jayden.s...@gmail.com wrote Have you ever used py2exe? After converting the python codes to executable, does it save the time of interpreting the script language? Thank a lot! Py2exe normally never speeds up anything, simply because it doesn't convert to executable, bu

Re: Why doesn't Python remember the initial directory?

2012-08-20 Thread andrea crotti
2012/8/20 Roy Smith : > In article , > Walter Hurry wrote: > >> It is difficult to think of a sensible use for os.chdir, IMHO. > > It is true that you can mostly avoid chdir() by building absolute > pathnames, but it's often more convenient to just cd somewhere and use > names relative to that.

Re: Why doesn't Python remember the initial directory?

2012-08-20 Thread andrea crotti
2012/8/20 kj : > In Roy Smith > writes> This means that no library code can ever count on, for example, > being able to reliably find the path to the file that contains the > definition of __main__. That's a weakness, IMO. One manifestation > of this weakness is that os.chdir breaks inspect.ge

Re: Sharing code between different projects?

2012-08-16 Thread andrea crotti
2012/8/16 andrea crotti : > > > Unfortunately I think you guess wrong > http://forums.perforce.com/index.php?/topic/553-perforce-svnexternals-equivalent/ > Anyway with views and similar things is not that hard to implement the > same thing.. I'm very happy to say that I fi

Re: how to call perl script from html using python

2012-08-16 Thread andrea crotti
2012/8/16 Pervez Mulla : > > Hey Steven , > > Thank you for your response, > > I will in detail now about my project, > > Actually the project entire backend in PERL language , Am using Django > framework for my front end . > I have written code for signup page in python , which is working perfec

Re: Sharing code between different projects?

2012-08-16 Thread andrea crotti
2012/8/16 Jean-Michel Pichavant : > > SVN allows to define external dependencies, where one repository will > actually checkout another one at a specific version. If SVN does it, I guess > any decent SCM also provide such feature. > > Assuming our project is named 'common', and you have 2 projects

Re: Sharing code between different projects?

2012-08-15 Thread andrea crotti
Also looking at logilab-common I thought that it would be great if we could actually make this "common" library even open source, and use it as one of the other many external libraries. Since Python code is definitively not the the core business of this company I might even convince them, but the

Re: Sharing code between different projects?

2012-08-15 Thread andrea crotti
2012/8/14 Cameron Simpson : > > Having just skimmed this thread, one thing I haven't quite seen suggested is > this: > > Really do make a third "utilities" project, and treat "the project" and > "deploy" as separate notions. So to actually run/deploy project A's code > you'd have a short script tha

Re: Sharing code between different projects?

2012-08-14 Thread andrea crotti
2012/8/14 Jean-Michel Pichavant : > > I can think of logilab-common (http://www.logilab.org/848/) > > Having a company-wide python module properly distributed is one to achieve > your goal. Without distributing your module to the public, there's a way to > have a pypi-like server runnning on your p

Re: Sharing code between different projects?

2012-08-14 Thread andrea crotti
2012/8/13 Rob Day : > I'd just create a module - called shared_utils.py or similar - and import > that in both projects. It might be a bit messy if there's no 'unifying > theme' to the module - but surely it'd be a lot less messy than your > TempDirectory class, and anyone else who knows Python wil

Sharing code between different projects?

2012-08-13 Thread andrea crotti
I am in the situation where I am working on different projects that might potentially share a lot of code. I started to work on project A, then switched completely to project B and in the transiction I copied over a lot of code with the corresponding tests, and I started to modify it. Now it's ti

Re: CRC-checksum failed in gzip

2012-08-02 Thread andrea crotti
2012/8/2 andrea crotti : > > Ok sure that makes sense, but then this explanation is maybe not right > anymore, because I'm quite sure that the file object is *not* shared > between threads, everything happens inside a thread.. > > I managed to get some errors doing this

Re: CRC-checksum failed in gzip

2012-08-02 Thread andrea crotti
2012/8/2 Laszlo Nagy : > > Your example did not share the file object between threads. Here an example > that does that: > > class OpenAndRead(threading.Thread): > def run(self): > global fz > fz.read(100) > > if __name__ == '__main__': > >fz = gzip.open('out2.txt.gz') >

Re: CRC-checksum failed in gzip

2012-08-02 Thread andrea crotti
2012/8/1 Steven D'Aprano : > > When you start using threads, you have to expect these sorts of > intermittent bugs unless you are very careful. > > My guess is that you have a bug where two threads read from the same file > at the same time. Since each read shares state (the position of the file >

Re: CRC-checksum failed in gzip

2012-08-01 Thread andrea crotti
2012/8/1 Laszlo Nagy : > >> Thanks a lot, that makes a lot of sense.. I haven't given this detail >> before because I didn't write this code, and I forgot that there were >> threads involved completely, I'm just trying to help to fix this bug. >> >> Your explanation makes a lot of sense, but it's

Re: CRC-checksum failed in gzip

2012-08-01 Thread andrea crotti
2012/8/1 Steven D'Aprano : > On Wed, 01 Aug 2012 14:01:45 +0100, andrea crotti wrote: > >> Full traceback: >> >> Exception in thread Thread-8: > > "DANGER DANGER DANGER WILL ROBINSON!!!" > > Why didn't you say that there were threads involve

Re: Pass data to a subprocess

2012-08-01 Thread andrea crotti
2012/8/1 Laszlo Nagy : > > So detaching the child process will not make IPC stop working. But exiting > from the original parent process will. (And why else would you detach the > child?) > > -- > http://mail.python.org/mailman/listinfo/python-list Well it makes perfect sense if it stops working

Re: CRC-checksum failed in gzip

2012-08-01 Thread andrea crotti
2012/8/1 Laszlo Nagy : >>there seems to be no clear pattern and just randmoly fails. The file >> is also just open for read from this program, >>so in theory no way that it can be corrupted. > > Yes, there is. Gzip stores CRC for compressed *blocks*. So if the file is > not flushed to the d

Re: Pass data to a subprocess

2012-08-01 Thread andrea crotti
2012/8/1 Roy Smith : > In article , > Laszlo Nagy wrote: > >> Yes, I think that is correct. Instead of detaching a child process, you >> can create independent processes and use other frameworks for IPC. For >> example, Pyro. It is not as effective as multiprocessing.Queue, but in >> return, you

Re: CRC-checksum failed in gzip

2012-08-01 Thread andrea crotti
Full traceback: Exception in thread Thread-8: Traceback (most recent call last): File "/user/sim/python/lib/python2.7/threading.py", line 530, in __bootstrap_inner self.run() File "/user/sim/tests/llif/AutoTester/src/AutoTester2.py", line 67, in run self.processJobData(jobData, logger)

Re: CRC-checksum failed in gzip

2012-08-01 Thread andrea crotti
2012/8/1 Laszlo Nagy : > On 2012-08-01 12:39, andrea crotti wrote: >> >> We're having some really obscure problems with gzip. >> There is a program running with python2.7 on a 2.6.18-128.el5xen (red >> hat I think) kernel. >> >> Now this program doe

CRC-checksum failed in gzip

2012-08-01 Thread andrea crotti
We're having some really obscure problems with gzip. There is a program running with python2.7 on a 2.6.18-128.el5xen (red hat I think) kernel. Now this program does the following: if filename == 'out2.txt': out2 = open('out2.txt') elif filename == 'out2.txt.gz' out2 = open('out2.txt.gz'

Re: Pass data to a subprocess

2012-08-01 Thread andrea crotti
2012/8/1 Laszlo Nagy : > On thing is sure: os.fork() doesn't work under Microsoft Windows. Under > Unix, I'm not sure if os.fork() can be mixed with > multiprocessing.Process.start(). I could not find official documentation on > that. This must be tested on your actual platform. And don't forget t

Re: Pass data to a subprocess

2012-08-01 Thread andrea crotti
2012/8/1 Laszlo Nagy : >> I was just surprised that it worked better than I expected even >> without Pipes and Queues, but now I understand why.. >> >> Anyway now I would like to be able to detach subprocesses to avoid the >> nasty code reloading that I was talking about in another thread, but >> t

Re: Pass data to a subprocess

2012-07-31 Thread andrea crotti
2012/7/31 Laszlo Nagy : >> I think I got it now, if I already just mix the start before another add, >> inside the Process.run it won't see the new data that has been added after >> the start. So this way is perfectly safe only until the process is launched, >> if it's running I need to use some mu

Re: Pass data to a subprocess

2012-07-31 Thread andrea crotti
> > > def procs(): > mp = MyProcess() > # with the join we are actually waiting for the end of the running time > mp.add([1,2,3]) > mp.start() > mp.add([2,3,4]) > mp.join() > print(mp) > I think I got it now, if I already just mix the start before another add, inside th

Re: py2c - an open source Python to C/C++ is looking for developers

2012-07-30 Thread andrea crotti
2012/7/30 : > I created py2c ( http://code.google.com/p/py2c )- an open source Python to > C/C++ translator! > py2c is looking for developers! > To join create a posting in the py2c-discuss Google Group or email me! > Thanks > PS:I hope this is the appropiate group for this message. > -- > http:/

Re: reloading code and multiprocessing

2012-07-27 Thread andrea crotti
2012/7/25 andrea crotti : > > I would also like to avoid this in general, but we have many > subprocesses to launch and some of them might take weeks, so we need > to have a process which is always running, because there is never a > point in time where we can just say let's

regexps to objects

2012-07-27 Thread andrea crotti
I have some complex input to parse (with regexps), and I would like to create nice objects directy from them. The re module doesn't of course try to conver to any type, so I was playing around to see if it's worth do something as below, where I assign a constructor to every regexp and build an obje

Re: Dumping all the sql statements as backup

2012-07-25 Thread andrea crotti
2012/7/25 Jack Since you know the content of what the sql code is, why not just build > the sql file(s) needed and store them so that in case of a burp you can > just execute the code file. If you don't know the exact sql code, dump > it to a file as the statements are constructed... The only prob

Dumping all the sql statements as backup

2012-07-25 Thread andrea crotti
I have some long running processes that do very long simulations which at the end need to write things on a database. At the moment sometimes there are network problems and we end up with half the data on the database. The half-data problem is probably solved easily with sessions and sqlalchemy (

Re: reloading code and multiprocessing

2012-07-25 Thread andrea crotti
2012/7/23 Chris Angelico : > > That would probably be correct. However, I still think you may be > fighting against the language instead of playing to its strengths. > > I've never fiddled with sys.modules like that, but I know some have, > without problem. > > ChrisA > -- > http://mail.python.org/

Re: reloading code and multiprocessing

2012-07-23 Thread andrea crotti
2012/7/20 Chris Angelico : > On Thu, Jul 19, 2012 at 8:15 PM, andrea crotti > wrote: >> We need to be able to reload code on a live system. This live system >> has a daemon process always running but it runs many subprocesses with >> multiprocessing, and the subprocesses

reloading code and multiprocessing

2012-07-19 Thread andrea crotti
We need to be able to reload code on a live system. This live system has a daemon process always running but it runs many subprocesses with multiprocessing, and the subprocesses might have a short life... Now I found a way to reload the code successfully, as you can see from this testcase: def

Re: assertraises behaviour

2012-07-17 Thread Andrea Crotti
To clarify my "problem", I just thought that assertRaises if used as context manager should behave as following: - keep going if the exception declared is raised - re-raise an error even if catched after the declared exception was catched I was also confused by the doc a bit: "Test that an excep

  1   2   3   4   >