[Tutor] how to read a non english text file

2005-11-25 Thread enas khalil 
hello everyone   still my question about if i could read and process a non english language text file using python   
		 Yahoo! Music Unlimited - Access over 1 million songs. Try it free.___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] how to read a non english text file

2005-11-25 Thread Kent Johnson 
enas khalil wrote:
 hello everyone
 still my question about if i could read and process  a non english 
 language text file using python

To read the file use
import codecs
data = codecs.open('myfile.txt', 'r', 'cp1256').read()

This will give you a Unicode string in data. I don't know how to use NLTK to 
tag this, I hope this will get you started.

Kent

-- 
http://www.kentsjohnson.com

___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] how to read a non english text file

2005-11-25 Thread Basem Narmok 
enas khalil wrote:

hello everyone 
  still my question about if i could read and process  a non english language 
 text file using python 
  

Hi Enas,
I think you mean Arabic language, yes of course you can, and I think you 
should see the Duali project:
http://www.arabeyes.org/project.php?proj=Duali

Duali is a simple Arabic spell checker written in Python, Duali will be 
a good start if you are interested in Arabic language, It will be hard 
if you are a newbie but try and see the code you will learn a lot.
you can see a screenshot for Duali running at : 
http://art.arabeyes.org/duali/duali_0_1_1
What you need to learn is Unicode with Python, Duali is a very good example.

Good Luck

Basem Narmok
___
Tutor maillist  -  Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor