Re: [Tutor] converting tab-delimited text files to csv
Luke Paireepinart wrote: > csv is comma-separated values, right? > you should be able to just do a string replace of tabs -> commas on each > line in the new file... > or is the csv format more complicated than that? Yes, it is more complicated than that because the data itself may contain commas. Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] converting tab-delimited text files to csv
Switanek, Nick wrote:
> I have a long tab-delimited text file that I’d like to convert into csv
> format so I can read it into a statistics package.
Are you sure the statistics package can't read tab-delimited data
directly? For example in R you can use read.delim().
> Here’s what I’ve tried to do:
>
>
>
> import csv
>
> inputFileList = file(‘input.txt’).readlines()
>
> writer = csv.writer(file(‘output.csv', 'wb'))
>
> writer.writerows(inputFileList)
>
> del writer
>
> When I try to open the resulting file in Excel (I’ve used a sample file
> of 100 rows), however, it appears that I’m delimiting every character,
> as each character has its own column. The rows in the original text file
> are combinations of integers and character strings (some of which
> include double quotes). Do I need to alter the ‘dialect’ or what?
The argument to writerows() should be a list of sequences. You are
passing a list of strings, so the strings are interpreted as sequences
of characters.
You need to divide the input rows at the tabs. You will also want to
strip the trailing newlines off the input lines. Try this:
inputFileList = [ line.rstrip('\n').split('\t') for line in
open('input.txt') ]
Kent
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] converting tab-delimited text files to csv
Switanek, Nick wrote: > > I have a long tab-delimited text file that I’d like to convert into > csv format so I can read it into a statistics package. > > I’ve been using Excel to do the format conversion up till now, but now > I have more rows than Excel can handle, and would like to avoid going > through Excel if possible. > > I’ve found the csv module but am confused by its documentation, as I > am with the few threads on the subject generally. > > Here’s what I’ve tried to do: > > import csv > > inputFileList = file(‘input.txt’).readlines() > > writer = csv.writer(file(‘output.csv', 'wb')) > > writer.writerows(inputFileList) > > del writer > > When I try to open the resulting file in Excel (I’ve used a sample > file of 100 rows), however, it appears that I’m delimiting every > character, as each character has its own column. The rows in the > original text file are combinations of integers and character strings > (some of which include double quotes). Do I need to alter the > ‘dialect’ or what? > > Many thanks in advance for your help! > csv is comma-separated values, right? you should be able to just do a string replace of tabs -> commas on each line in the new file... or is the csv format more complicated than that? -Luke ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] converting tab-delimited text files to csv
I have a long tab-delimited text file that I'd like to convert into csv
format so I can read it into a statistics package.
I've been using Excel to do the format conversion up till now, but now I
have more rows than Excel can handle, and would like to avoid going
through Excel if possible.
I've found the csv module but am confused by its documentation, as I am
with the few threads on the subject generally.
Here's what I've tried to do:
import csv
inputFileList = file('input.txt').readlines()
writer = csv.writer(file('output.csv', 'wb'))
writer.writerows(inputFileList)
del writer
When I try to open the resulting file in Excel (I've used a sample file
of 100 rows), however, it appears that I'm delimiting every character,
as each character has its own column. The rows in the original text file
are combinations of integers and character strings (some of which
include double quotes). Do I need to alter the 'dialect' or what?
Many thanks in advance for your help!
Nick
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Cause-Effect, Isikawa, fishbone diagram
Hi János, Reportlab is a very complete PDF library, but it is not very simple, and does not have shortcuts for what you need to do. You'd have to come up with the code to get a fishbone diagram from the textfile. But it does help you with the PDF. There are some graphing packages, but they're most for scientific graphs. Like http://matplotlib.sourceforge.net/ Hope that helps, Hugo János Juhász wrote: > Dear All, > > does someone know any python based solution to draw a > cause-effect diagram into PDF from a simple textfile ? > > It is also called a Fishbone Diagram, because of its shape, > or an Ishikawa Chart, after its originator, Kaoru Ishikawa > I feel, it can be converted from a structure like this. > > Title > Effect > Cause1 > Secundary > Tertiary > Tertiary > Cause2 > Secundary > Tertiary > Cause3 > Secundary > Tertiary > > It is probably a simple function. > > > > Yours sincerely, > __ > Janos Juhasz > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python code to connect using PPPoE
Hi Johan, PPPoE is both in the Linux Kernel (for the low level work) and as some executables and scripts. It fou want to establish a connection from a Python script (E.G. ehn the computer has no internet access and you need it) The best way would be to call the appropriate utilities, and parse their output. This means calling adsl-connect, ficonfig, and the like, maybe parsing the output from ifconfig. Take a look at the module named subprocess to do that. Reimplementing these utilities in Python is some work, but doable. Implementing PPPoE in Python, reading the device files and all that... well... it's almost nonsense. For this, you have to use the kernel. Johan Geldenhuys wrote: > Kent, > I want to establish the connection with Python. I think in Linux you can use > a PPPoE package to make life easier, but I don't want to use the Kernel to > do that. > > Johan > ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] How to use diff.py?
I've found diff.py in Python25\Tools\Scripts. But I don't see how to use it to find the differences between 2 files, say file1.txt and file2.txt. I was hoping it would work like unix's diff, but could someone explain? Thanks, Dick Moores UliPad <>: http://wiki.woodpecker.org.cn/moin/UliPad ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] question about *args and functions
Great, gents, thanks. tried it out and is working fine, this will clean up a lot of stuff for me. thanks for your help ! shawn On 1/26/07, Wesley Brooks <[EMAIL PROTECTED]> wrote: Greetings, You could default it to None and check in your script to see if it has changed. def some_function(req_var, req_var2, un_req_var=None): if un_req_var != None: dosomething else: dosomethingelse Wesley Brooks. On 26/01/07, shawn bright <[EMAIL PROTECTED]> wrote: > lo there all, > > if i have a function that sometimes needs a value passed to it and sometimes > not, is this where i use *args ? > > like this > > def some_function(req_var, req_var2, un_req_var): > do some stuff > return value > > how would i use this if sometimes i need to process un_req_var and sometimes > not ? > > thanks > > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > > > ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] question about *args and functions
Greetings, You could default it to None and check in your script to see if it has changed. def some_function(req_var, req_var2, un_req_var=None): if un_req_var != None: dosomething else: dosomethingelse Wesley Brooks. On 26/01/07, shawn bright <[EMAIL PROTECTED]> wrote: > lo there all, > > if i have a function that sometimes needs a value passed to it and sometimes > not, is this where i use *args ? > > like this > > def some_function(req_var, req_var2, un_req_var): > do some stuff > return value > > how would i use this if sometimes i need to process un_req_var and sometimes > not ? > > thanks > > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > > > ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] question about *args and functions
shawn bright wrote: > lo there all, > > if i have a function that sometimes needs a value passed to it and > sometimes not, is this where i use *args ? No, use an optional argument. > > like this > > def some_function(req_var, req_var2, un_req_var): > do some stuff > return value > > how would i use this if sometimes i need to process un_req_var and > sometimes not ? def some_function(req_var, req_var2, un_req_var=None): do some stuff return value Now the caller can write some_function(1, 2) or some_function(1, 2, 3). You can distinguish the two by checking for 'un_req_var is None'. If None is a legitimate value for un_req_var then you need to pick some other sentinal value. If there is no built-in value that works, create your own: missing = object() def some_function(req_var, req_var2, un_req_var=missing): Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] question about *args and functions
2007/1/26, shawn bright <[EMAIL PROTECTED]>: lo there all, if i have a function that sometimes needs a value passed to it and sometimes not, is this where i use *args ? No. *args is used if there are arguments that could occur more than once. like this def some_function(req_var, req_var2, un_req_var): do some stuff return value how would i use this if sometimes i need to process un_req_var and sometimes not ? Use: def some_function(req_var, req_var2, un_req_var = None): do some stuff return value Now, the function can be called both with and without un_req_var, and if it is called without un_req_var, within the function body un_req_var is considered to be None. -- Andre Engels, [EMAIL PROTECTED] ICQ: 6260644 -- Skype: a_engels ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] question about *args and functions
lo there all, if i have a function that sometimes needs a value passed to it and sometimes not, is this where i use *args ? like this def some_function(req_var, req_var2, un_req_var): do some stuff return value how would i use this if sometimes i need to process un_req_var and sometimes not ? thanks ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Optimal solution in dealing with huge databases inpython
On Friday 26 January 2007 00:40, Alan Gauld wrote:
> "Shadab Sayani" <[EMAIL PROTECTED]> wrote
>
> > I got your point.But before inserting data I need to store it
> > into a file in a format supported by postgresql.Wont this
> > operation incur a performance hit as it includes writing
> > to a file which is on disk?
>
> Unless your data is already in a format the database
> understands you will have to reformat it before loading it.
> There are basically two options:
> 1) read the unformatted data piece by piece, reformat
> it and load it to the database item by item.
> 2) read the unformatted data and write it to an
> intermediate file in a format supported by the
> database, then load the formatted data in bulk.
>
> The second approach is nearly always faster than
> the first for large data sets. This is due to many things
> including transactional overheads in the first approach,
> caching issues, availability of bulk optimisations in
> the database itself, etc.
>
> Writing to a flat file is much faster than writing to a
> database. Reformatting data is a complex business.
> Python is good at complex processing and writing
> to flat files. SQL is good at writing to databases but poor
> at complex processing. So use Python for its
> strengths and SQL for its strengths and you get
> optimal results.
>
> HTH,
>
> Alan G
This subject has come up several times in the psycopg mailing list. The
author of psycopg (Federico Di Gregorio) has suggested the best to handle
this is as follows:
"When I'd have some more time I'll write a
class to do that without the need to create a temporary file. Now that I
think about it probably the best way would be to use two threads
writing/reading from a pipe, so that the postgresql one doesn't have to
wait for the other and data is always ready."
In response too:
This is a copout and uses temporary files, but does the trick:
Usage is pretty simple:
orac = ora.cursor()
orac.execute('select * from gene.geneinfo')
orac.arraysize=10
oraf = OracleFile2(orac)
pgc = pg.cursor()
pgc.copy_from(oraf,'loader.gene_info')
class CursorFile(object):
def __init__(self,cursor):
self._cursor = cursor
self._file = tempfile.TemporaryFile()
self.write_file()
def write_file(self):
print 'fetching'
dat = self._cursor.fetchmany()
while len(dat)>0:
print 'writing'
for row in dat:
self._file.writelines("\t".join(map(str,row)) + "\n")
print "fetching"
dat = self._cursor.fetchmany()
print 'done writing'
self._file.flush()
self._file.seek(0)
def read(self,size=-1):
dat = self._file.read(size)
return dat
def readline(self,size=-1):
return self._file.readline(size)
I hope this helps
--
John Fabiani
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Inheriting Classes and __init__
Thanks again, that is a great help. Wesley Brooks On 26/01/07, Kent Johnson <[EMAIL PROTECTED]> wrote: > Wesley Brooks wrote: > > Dear Users, > > > > I would like a class to inherit the methods from wxVTKRenderWindow, > > but to add a few lines of code to __init__ . I would prefer not to > > copy the whole init method from the inherited class into the > > inheriting class. What is the best approach for this? I guess that if > > I have an init in the inheriting class it would overide the init in > > the inherited class? > > Yes, your __init__() will override the base class __init__(). To include > the base class functionality, just call the base class __init__() from > your __init__(). The syntax for this is a little different from the > usual method call; in your __init__() include this call: >wxVTKRenderWindow.__init__(self, args) > > where args is whatever argument list you want to pass to the base class > __init__(). > > > ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Inheriting Classes and __init__
Wesley Brooks wrote: > Dear Users, > > I would like a class to inherit the methods from wxVTKRenderWindow, > but to add a few lines of code to __init__ . I would prefer not to > copy the whole init method from the inherited class into the > inheriting class. What is the best approach for this? I guess that if > I have an init in the inheriting class it would overide the init in > the inherited class? Yes, your __init__() will override the base class __init__(). To include the base class functionality, just call the base class __init__() from your __init__(). The syntax for this is a little different from the usual method call; in your __init__() include this call: wxVTKRenderWindow.__init__(self, args) where args is whatever argument list you want to pass to the base class __init__(). ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Inheriting Classes and __init__
Dear Users, I would like a class to inherit the methods from wxVTKRenderWindow, but to add a few lines of code to __init__ . I would prefer not to copy the whole init method from the inherited class into the inheriting class. What is the best approach for this? I guess that if I have an init in the inheriting class it would overide the init in the inherited class? Yours faithfully, Wesley Brooks ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] issuing system commands
Dan Klose wrote:
> Hi All,
>
> I am trying to break my nasty habit of doing EVERYTHING in perl and for
> this task I figure python is one of the better options.
>
> I would like to do several things:
>
> 1. take user input - this will be a password
> 2. use some sort of function that converts the unser input to * or the
> typical '' no output of linux.
See getpass.getpass()
> 3. Take this input and use it in the os.system to issue an su command.
>
> All I have so far is:
>
> import os, sys
> password = raw_input('Enter USER password: ')
> print "you entered ", password
> os.system("su perlmunky password")
>
> I guess point three is more of a system dependent problem - As I work on
> a mac and linux is there a better way of issuing these commands? All I
> really want to do is mount/unmount directories and shift / delete
> files. I know how to do this in perl, but as I said I really want to
> kick the habit.
Not sure what you mean by mounting a directory but you can manipulate
files and directories directly in Python. See the os and shutil modules.
Kent
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
[Tutor] issuing system commands
Hi All,
I am trying to break my nasty habit of doing EVERYTHING in perl and for this
task I figure python is one of the better options.
I would like to do several things:
1. take user input - this will be a password
2. use some sort of function that converts the unser input to * or the
typical '' no output of linux.
3. Take this input and use it in the os.system to issue an su command.
All I have so far is:
import os, sys
password = raw_input('Enter USER password: ')
print "you entered ", password
os.system("su perlmunky password")
I guess point three is more of a system dependent problem - As I work on a
mac and linux is there a better way of issuing these commands? All I really
want to do is mount/unmount directories and shift / delete files. I know
how to do this in perl, but as I said I really want to kick the habit.
Any help - pointers - advice would be most welcome.
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] how to pause or stop a running python script?
Wong Vincent wrote:
> Hi,
> If I use run a script which consist the following line:
> -->os.popen("python c:\\test.py")
>
> Is it possible for me to interrupt the pause/stop the running of
> "test.py" halfway?
Maybe this thread on c.l.python is helpful...
http://tinyurl.com/2bpvjo
Kent
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Best IDE for Python
On 1/26/07, OkaMthembo <[EMAIL PROTECTED]> wrote: > how useable is vim on Windows? Very! I've used Vim on Windows since the 5.3 version, it only gets better. Like Alan said there is a learning curve, but once you've "got it" there is no going back. > i wish i could learn Unix. which distro do you think is good to learn? If you just want to have the power of some Unix tools then take a look at http://gnuwin32.sourceforge.net/ , they all run fine in the standard Windows shell. If you want to try a Unix variant then find one of the many "Live" CD's out there which you can use before you slice up your disk. I'm a big fan of NetBSD but at the moment I'm running OpenBSD 4.0 on my PIII laptop and it's very nice. -- () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] how to pause or stop a running python script?
"Wong Vincent" <[EMAIL PROTECTED]> wrote
> If I use run a script which consist the following line:
> -->os.popen("python c:\\test.py")
>
> Is it possible for me to interrupt the pause/stop the running of
> "test.py" halfway?
There is a recipe for this somewhere.
I don't think you can do it with vanilla popen because you
need the process ID. You can do some trickery using multiple
threads, or you can use lower level tools to launch the process
(spawn etc).
However you should look at the subprocess module which
supercedes popen. I strongly suspect that the Popen class
will have some options that help here, although I confess I
haven't actually tried it!
http://docs.python.org/lib/node533.html
Alan G.
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Optimal solution in dealing with huge databases inpython
"Shadab Sayani" <[EMAIL PROTECTED]> wrote > > I got your point.But before inserting data I need to store it > into a file in a format supported by postgresql.Wont this > operation incur a performance hit as it includes writing > to a file which is on disk? Unless your data is already in a format the database understands you will have to reformat it before loading it. There are basically two options: 1) read the unformatted data piece by piece, reformat it and load it to the database item by item. 2) read the unformatted data and write it to an intermediate file in a format supported by the database, then load the formatted data in bulk. The second approach is nearly always faster than the first for large data sets. This is due to many things including transactional overheads in the first approach, caching issues, availability of bulk optimisations in the database itself, etc. Writing to a flat file is much faster than writing to a database. Reformatting data is a complex business. Python is good at complex processing and writing to flat files. SQL is good at writing to databases but poor at complex processing. So use Python for its strengths and SQL for its strengths and you get optimal results. HTH, Alan G ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Best IDE for Python
"OkaMthembo" <[EMAIL PROTECTED]> wrote > how useable is vim on Windows? Very usable. I'm an old Unix user who used to use emacs primarily. But when I moved to Windoze I found I didn't like emacs as much so I moved to vim and I don't regret it. > I saw on its site that it was developed > primarily as a Unix tool. Its an "Improved" version of the vi(sual) editor which is the standard editor on Unix. But there is nothing that is Unix specific about it. But if you have never used vi before there is a steep learning curve because vi was designed to be "easy to use" for experts not novices. ie it concentrates on delivering advanced features in minimal keystrokes rather than on making the basic features easy to remember. It also works best if you are a touch typist since the commands are arranged such that the most commonly used are the keys under your fingers in the home position - efficiency for experts again... > i wish i could learn Unix. Try using cygwin as an intermediate step. Even if yuou never make the move to unix you will have some power tools for Windows. If you do move to Unix you will feel at home after using cygwin. > which distro do you think is good to learn? There is too much made of distros IMHO. Once you have them installed they are all more or less the same. The same tools, the same GUIs. Its only things like the package installers and some admin tools that are different. Alan G. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Best IDE for Python
hi guys, how useable is vim on Windows? i saw on its site that it was developed primarily as a Unix tool. i wish i could learn Unix. which distro do you think is good to learn? ive been eyeing FreeBSD but ive got untouched Ubuntus. thanks_up_ front() "Shortash" On 1/25/07, Mike Hansen <[EMAIL PROTECTED]> wrote: > > > Activestate just released Komodo Edit and Komodo IDE. I > believe both do > > auto-completion that shows a list of possibilities. Komodo > Edit is free > > and Komodo IDE cost some $$. > > IMHO: If one programs in more than one programming language, the > ultimate would be an editor/IDE that can accommodate all with similar > functionality and behavior for all. > > MTCW > tim I agree. Depending on what I'm doing, I toggle between VIM and Komodo. I like that both can handle multiple languages. Mike - NOTICE: This e-mail transmission and any documents or files attached to it contain information for the sole use of the above-identified individual or entity. Its contents may be privileged, confidential, and exempt from disclosure under the law. Any dissemination, distribution, or copying of this communication is strictly prohibited. Please notify the sender immediately if you are not the intended recipient. FGNS ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor -- "The Stupidry Foundry" ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Best IDE for Python
On Thursday 25 January 2007 03:12, Shadab Sayani wrote: > Hi, > I am using vim editor to code my project in python.Is there a good IDE > where in I type the name of the class object and then dot then all the > attributes of the object are displayed so on.I tried to install IDLE but I > have no idea how to install tkinter? Any help that enables me to use good > IDE as soon as possible is appreciated Thanks and Regards, > Shadab > I use ... http://www.die-offenbachs.de/detlev/eric.html ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
