io.TextIOWrapper and io.RawIOBase
Documentation for `io.TextIOWrapper` [[1]] suggests that the buffer supplied to the constructor should be of `io.BufferedIOBase` type: > A buffered text stream over a `BufferedIOBase` binary stream. Looking at the implementation of `io.TextIOWrapper` in both `Modules/_io/textio.c` [[2]] and `Lib/_pyio.py` [[3]], it seems like `io.TextIOWrapper` will only use the following attributes and functions on the supplied `buffer` object: ```python buffer.closed buffer.close() buffer._dealloc_warn() # ^ only from Modules/_io/textio.c, looks like failures will be ignored buffer.fileno() buffer.flush() buffer.isatty() buffer.name # ^ only when TextIOWrapper.buffer is accessed buffer.raw() # ^ only if buffer is instance of BufferedReader, BufferedWriter # or BufferedRandom, also only from Modules/_io/textio.c buffer.read() buffer.read1() # only if buffer has a read1 method buffer.readable() buffer.seek() buffer.seekable() buffer.tell() buffer.truncate() buffer.writable() buffer.write() ``` More specifically, `io.TextIOWrapper` looks like it will work fine with `buffer` objects that does not have any of the following attributes and methods that only exists in `io.BufferedIOBase`, but not in `io.RawIOBase`: ```python buffer.raw # ^ this is only called if buffer is an instance of # explicit subclasses of `io.BufferedIOBase` buffer.detatch() # this is never called buffer.read1() # this is only used if it exists buffer.readinto() # this is never called buffer.readinto1() # this is never called ``` Or stated differently, it looks like `io.TextIOWrapper` will work equally well with buffer objects that are either `io.RawIOBase` or `io.BufferedIOBase` types. So the question is, if my assessment is correct, should the documentation not be updated to clearly state that the `buffer` can be either a `io.RawIOBase` or `io.BufferedIOBase` object? (text written for python 3.7.12) [1]: https://docs.python.org/3.7/library/io.html#io.TextIOWrapper [2]: https://github.com/python/cpython/blob/v3.7.12/Modules/_io/textio.c [3]: https://github.com/python/cpython/blob/v3.7.12/Lib/_pyio.py -- https://mail.python.org/mailman/listinfo/python-list
Issue combining gzip and subprocess
Hi there, We tried to gzip the output of a shell command, but this results in a strange error: the resulting file seems to be the concatenation of the plaintext file with the zipped content. For example: f = gzip.open(filename, 'w') subprocess.check_call(['ls','-la'], stdout=f) f.close() Using a normal file works as expected, but a GzipFile results in a file containing what looks like the unzipped data, followed by the zipped data. I suspect this may have something to do with limitations of GzipFile such as it not implementing truncate(). Does anyone have an explanation / can you suggest a nice solution for doing what we are trying to do? Regards - Iwan -- http://mail.python.org/mailman/listinfo/python-list
Re: How to debug this import problem?
Mmm, we solved half of the cause of this one. Test runs are kicked off via setuptools's test command. But this happens programmatically, and successively in one process. But setuptools's test command clears all modules imported during a test run from sys.modules - hence it is intended that modules would be re- imported. Why certain of our code managed to still get to the classes contained in prior imports is still a mystery though. -Iwan -- http://mail.python.org/mailman/listinfo/python-list
Re: How to debug this import problem?
Hi Scott, Diez, On May 8, 8:21 pm, Scott David Daniels wrote: > Diez B. Roggisch wrote: > > Try putting an "import pdb; pdb.set_trace()" on top of the decimal module. [snip] > You can also run Python with the "-v" or "-vv" flags to get output > about exactly what files are getting imported from the file > system and when. "-v" iswhat it got, "-vv" is everywhere it tries. Thanks, yes we did do the pdb.set_trace(), but everything looked fine (ie, the paths used to import each module from was the same each time) in there except for the fact that id(sys) and id(decimal) were different each time decimal got imported. Python -vv also reveals that the paths used to import the modules from are always the same paths and correct. However, on the stack trace, there is another strangeness: We have a little script that loads the tests that have to be run. This loading happens 3 times, and the module is imported 3 times - each time as a result of loading tests. This happens with the following line: module = __import__(moduleName, globals(), locals(), [className]) I was wondering whether this could have something to do with it? Regards - Iwan -- http://mail.python.org/mailman/listinfo/python-list
How to debug this import problem?
Hi there, We have a rather complicated program which does a bit of os.chdir and sys.path manipulations. In between all of this, it imports the decimal module several times. However, it imports a new instance of decimal sometimes. (Which is a problem, since a decimal.Decimal (imported at point A) now is not the same as that imported somewhere else. In trying to figure out what's happening, we've changed the code in decimal to print out id(sys) when decimal gets imported. This also gives back different values at different times. My suspicion is that when importing python tries to check sys.modules - but each time sys is a different sys already. Any ideas of how we can gather more data to find out exactly what causes this? The path manipulations is a suspect, but unfortunately we cannot remove it. If we can trace the problem from the import end though, we may be able to isolate the exact one which is the problem. This code incidentally also runs in a virtualenv environment AND uses setuptools. None of these complications can be removed... Regards - Iwan -- http://mail.python.org/mailman/listinfo/python-list
awrkawrprpkar
http://www.martin-stosch-superstar.wg.am/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Structuring larger applications - ideas
I know my foreign (to python) one class per module idea is what makes life more difficult for me here. And there is an argument to be made out that it is a stupid remnant I stick to from having used it in other programming languages (do I have to admit C++ in my background?) Two small examples of where it it useful for me: my development environment is run by make to a large extent. Many standard UNIX tools have the intelligence to deal with filenames, and if you know a file corresponds to a class, you have a lot more info available in your makefile. Also, if I use my version control software (currently gnu arch) to see what's changed, for example, I get a list of changes files which again gives me more info because I know a file corresponds to a class. So, I can do a number of such small things using that convention that works well with standard old UNIX tools. And I find that valuable. As for the dependencies- I'm trying to keep the specification of them simple. From a design point of view, it makes sense to me to only specify ONCE that largeish-collection-of-classes-A depends on largeish-collection-of-classes-B. As another simple small example: say I change the name of large-collection-of-classes-B - this way I only need to do it once, else I need to grep and sed all over in order to do it. It just feels cleaner and easier. I know essentially, if you stick to having many classes in a python module, you have what I referred to in the previous paragraph, because you can stick to importing other modules at the beginning of a module and so have them and their contens available in the entire file. Still, I'm investigating whether I can bend the ever-so-flexible python to work well with my admittedly somewhat strange requirement. And, off course, I'd like to hear what others think. -i -- http://mail.python.org/mailman/listinfo/python-list
Re: Structuring larger applications - ideas
I know my foreign (to python) one class per module idea is what makes life more difficult for me here. And there is an argument to be made out that it is a stupid remnant I stick to from having used it in other programming languages (do I have to admit C++ in my background?) Two small examples of where it it useful for me: my development environment is run by make to a large extent. Many standard UNIX tools have the intelligence to deal with filenames, and if you know a file corresponds to a class, you have a lot more info available in your makefile. Also, if I use my version control software (currently gnu arch) to see what's changed, for example, I get a list of changes files which again gives me more info because I know a file corresponds to a class. So, I can do a number of such small things using that convention that works well with standard old UNIX tools. And I find that valuable. As for the dependencies- I'm trying to keep the specification of them simple. From a design point of view, it makes sense to me to only specify ONCE that largeish-collection-of-classes-A depends on largeish-collection-of-classes-B. As another simple small example: say I change the name of large-collection-of-classes-B - this way I only need to do it once, else I need to grep and sed all over in order to do it. It just feels cleaner and easier. I know essentially, if you stick to having many classes in a python module, you have what I referred to in the previous paragraph, because you can stick to importing other modules at the beginning of a module and so have them and their contens available in the entire file. Still, I'm investigating whether I can bend the ever-so-flexible python to work well with my admittedly somewhat strange requirement. And, off course, I'd like to hear what others think. -i -- http://mail.python.org/mailman/listinfo/python-list
Re: Ruby on Rails or Perl's Maypole..is there a Python equivalent
Gary Nutbeam wrote: needing to learn Ruby. But why wouldn't you just use Rails and learn Ruby in the process? The "effort" required to learn Ruby pales in comparisson to the advantages using Ruby on Rails might give you, imho. Ruby is an excellent language, not much different from Python with its own set of advantages and problems (I really mis python's white-space indentation for example, but that is fully compensated by Ruby's nice OOP features). With a book like "Programming Ruby" you would be up to speed in a few days. Rails gives you much more than a comparable set of Python libraries which are gobled together with sticky tape. It provides you not just with an superbly integrated and consistent set of components. Rails gives you: - (real) automation (take a look at scaffolding for quick prototyping) - terrific documentation (the videos are *not* a gimmick, for example) - an enthousiastic, supportive user community (that alone is an incredible help and time saver) have fun, Iwan -- http://mail.python.org/mailman/listinfo/python-list
Re: Python evolution: Unease
Also, you keep talking about "the core python team" on the basis, it would appear, of reading one document by Guido. Have you bothered doing a MINIMUM of homework, such as, looking at http://www.amk.ca/diary/archives/cat_python.html Well, you being a member of that core team (meaning nog an organisational unit, but the group of people doing the really hard job, getting Python to work. An excellent job at that :-) I can repect you if not branding me a lamer at least admonishing me for not coming up with a thorough factual statement. But like I stated: "ramblings", remember. I'm not completely unknown with the workings of our species myself, though. Especially when discourse and policy is dictated by a select group of people (meaning: the one who actually create python, no criticism there) with final abritary powers for one indidual (again, no criticism), mindset *is* just as important as stated fact. Mindset will dictate future discourse and action. And I do sense (reading planet python/this newsgroup) a mindset or at least a tendency by the people who really matter in these discussion to keep on adding features to the syntax; to add "structure" to Python. My personal preference would be to leave the language alone for a while and to improve its infrastructure. Regards, Iwan -- http://mail.python.org/mailman/listinfo/python-list
Python evolution: Unease
Please ignore if you are allergic to ramblings :-) Despite a puritan streak I've always tried to refrain from language wars or syntax bickering; call it enforced pragmatism. That's the main reason why I've liked Python: it's elegant and simple and still dynamic and flexible. You could do worse for a clean and pragmatic language. I do know my Smaltalk from my Common Lisp and my Ruby from my C#, so I think I'm quite capable of escaping the "Blub paradox" http://c2.com/cgi/wiki?BlubParadox. I do miss some slick features in Python. But the nice thing about Python is that in those cases I can use its dynamism to implement it myself (and usually somebody else has done it for me, of course). In the end I'm not a language guru nor a framework wizard, but a mere mortal who designs and writes programs for end-users. For that task I need: a better standard ide, an integrated db interface with a proper set of db drivers (!!), a better debugger, a standard widget/windows toolkit, something akin to a standard for web programming, better documentation, a standard lib which is better organized, a formalized set of protocols and patterns for program construction. And an interpreter which is fast enough to avoid using C or Pyrex in most obvious cases. Many will say that Van Rossum's brainstorms/proposals as depicted in http://www.artima.com/weblogs/viewpost.jsp?thread=86641 will help in the above mentioned. And I'm certainly not against Optional ype checking. But I see little to no efforts from the core python team to address my needs as listed above. They seem mainly to focus on the core attributes and syntax of the language. Very little or no efforts are taken to improve the infrastructure around the language. And then I read the following sentence by Van Rossum: "In order to make type inferencing a little more useful, I'd like to restrict certain forms of extreme dynamic behavior in Python" In the end, it's mindset which counts. And I think that mindset is going to be determine the way foreward for Python: more features, increased complexity, less dynamism. Lots of syntax crud, without addressing the need to improve the infrastructure around the language. In short: I symphatize Patrick Logan's feeling: http://patricklogan.blogspot.com/2005/01/road-to-ruin.html Regards, Iwan van der Kleyn -- http://mail.python.org/mailman/listinfo/python-list
