I’m confused. I don’t know if I can help. > On Jul 22, 2020, at 11:11 AM, Liu Zheng <[email protected]> wrote: > > Hi, glad you solved the problem. Yes, both the request.FILES[‘file’] and the > chardet file handler are binary handlers. Binary handler presents the raw > data. chardet takes a sequence or raw data and then detect the encoding > format. With its prediction, if you want to open that puece of data in text > mode, you can use the .decode(<encoding format>) method of bytes object to > get a python string. > > On Wed, 22 Jul 2020 at 11:04 PM, Kovy Jacob <[email protected] > <mailto:[email protected]>> wrote: > That’s probably not the proper answer, but that’s the best I can do. Sorry :-( > > >> On Jul 22, 2020, at 10:46 AM, Ronaldo Mata <[email protected] >> <mailto:[email protected]>> wrote: >> >> Yes, the problem here is that the files will be loaded by the user, so I >> don't know what delimiter I will receive. This is not a base command that I >> am using, it is the logic that I want to incorporate in a view >> >> El mié., 22 jul. 2020 a las 10:43, Kovy Jacob (<[email protected] >> <mailto:[email protected]>>) escribió: >> Ah, so is the problem that you don’t always know what the delimiter is when >> you read it? If yes, what is the use case for this? You might not need a >> universal solution, maybe just put all the info into a csv yourself, >> manually. >> >>> On Jul 22, 2020, at 10:39 AM, Ronaldo Mata <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Hi Kovy, I'm using csv module, but I need to handle the delimiters of the >>> files, sometimes you come separated by "," others by ";" and rarely by "|" >>> >>> El mié., 22 jul. 2020 a las 10:28, Kovy Jacob (<[email protected] >>> <mailto:[email protected]>>) escribió: >>> Could you just use the standard python csv module? >>> >>>> On Jul 22, 2020, at 10:25 AM, Ronaldo Mata <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> Hi Liu thank for your answer. >>>> >>>> This has been a headache, I am trying to read the file using >>>> csv.DictReader initially i had an error trying to get the dict keys when >>>> iterating by rows, and i thought it could be encoding (for this reason i >>>> wanted to prepare the view to use the correct encoding). for that reason I >>>> asked my question. >>>> >>>> 1) your first approach doesn't work, if i send utf-8 file, chardet returns >>>> ascii as encoding. it seems request.FILES ['file']. read () returns a >>>> binary with that encoding. >>>> >>>> 2) In the end I realized that the problem was the delimiter of the csv but >>>> predicting it is another problem. >>>> >>>> Anyway, it was a task that I had to do and that was my limitation. I think >>>> there must be a library that does all this, uploading a csv file is common >>>> practice in many web apps. >>>> >>>> El mar., 21 jul. 2020 a las 13:47, Liu Zheng (<[email protected] >>>> <mailto:[email protected]>>) escribió: >>>> Hi. First of all, I think it's impossible to perfectly detect encoding >>>> without further information. See the answer in this SO post: >>>> https://stackoverflow.com/questions/436220/how-to-determine-the-encoding-of-text >>>> >>>> <https://stackoverflow.com/questions/436220/how-to-determine-the-encoding-of-text> >>>> There are many packages and tools to help detect encoding format, but >>>> keep in mind that they are only giving educated guesses. (Most of the >>>> time, the guess is correct, but do check the dev page to see whether there >>>> are known issues related to your problem.) >>>> >>>> Now let's say you have decided to use chardet. Check its doc page for the >>>> usage: https://chardet.readthedocs.io/en/latest/usage.html#usage >>>> <https://chardet.readthedocs.io/en/latest/usage.html#usage> You'll have >>>> more than one solutions. Here are some examples: >>>> >>>> 1. If the files uploaded to your server are all expected to be small csv >>>> files (less than a few MB and not many users do it concurrently), you can >>>> do the following: >>>> >>>> #in the view to handle the uploaded file: (assume file input name is just >>>> "file") >>>> file_content = request.FILES['file'].read() >>>> chardet.detect(file_content) >>>> >>>> 2. Also, chardet seems to support incremental (line-by-line) detection >>>> https://chardet.readthedocs.io/en/latest/usage.html#example-detecting-encoding-incrementally >>>> >>>> <https://chardet.readthedocs.io/en/latest/usage.html#example-detecting-encoding-incrementally> >>>> >>>> Given this, we can also read from requests.FILES line by line and pass >>>> each line to chardet >>>> >>>> from chardet.universaldetector import UniversalDetector >>>> >>>> #somewhere in a view function >>>> detector = UniversalDetector() >>>> file_handle = request.FILES['file'] >>>> for line in file_handle: >>>> detector.feed(line) >>>> if detector.done: break >>>> detector.close() >>>> # result available as a dict at detector.result >>>> >>>> >>>> >>>> >>>> >>>> On Tuesday, July 21, 2020 at 7:09:35 AM UTC+8, Ronaldo Mata wrote: >>>> How to deal with encoding when you try to read a csv file on view. >>>> >>>> I have a view to upload csv file, in this view I read file and save each >>>> row as new record. >>>> >>>> My bug is when I try to upload a csv file with a differente encoding (not >>>> UTF-8) >>>> >>>> how to handle this on django (using request.FILES) I was researching and I >>>> found chardet but I don't know how to pass it a request.FILES. I need help >>>> please. >>>> >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "Django users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an >>>> email to [email protected] >>>> <mailto:[email protected]>. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/django-users/64307441-0e65-45a2-b917-ece15a4ea729o%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/django-users/64307441-0e65-45a2-b917-ece15a4ea729o%40googlegroups.com?utm_medium=email&utm_source=footer>. >>>> >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "Django users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an >>>> email to [email protected] >>>> <mailto:[email protected]>. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/django-users/CAP%3DoziQuZyb74Wsk%2BnjngUpSccOKCYRM_C%3D7KgGX%2BgV5wRzHwQ%40mail.gmail.com >>>> >>>> <https://groups.google.com/d/msgid/django-users/CAP%3DoziQuZyb74Wsk%2BnjngUpSccOKCYRM_C%3D7KgGX%2BgV5wRzHwQ%40mail.gmail.com?utm_medium=email&utm_source=footer>. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Django users" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected] >>> <mailto:[email protected]>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/django-users/91E9FE01-4701-478C-B575-2BD5BA5DCE86%40gmail.com >>> >>> <https://groups.google.com/d/msgid/django-users/91E9FE01-4701-478C-B575-2BD5BA5DCE86%40gmail.com?utm_medium=email&utm_source=footer>. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Django users" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected] >>> <mailto:[email protected]>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/django-users/CAP%3DoziSjnUSkWgHqb1RzsSHsUURLM9%3DPP0ZNX_zORkp3v-L1%2BQ%40mail.gmail.com >>> >>> <https://groups.google.com/d/msgid/django-users/CAP%3DoziSjnUSkWgHqb1RzsSHsUURLM9%3DPP0ZNX_zORkp3v-L1%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] >> <mailto:[email protected]>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-users/1471A9A8-8BFD-41B0-9AC4-2EA424F1F989%40gmail.com >> >> <https://groups.google.com/d/msgid/django-users/1471A9A8-8BFD-41B0-9AC4-2EA424F1F989%40gmail.com?utm_medium=email&utm_source=footer>. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] >> <mailto:[email protected]>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-users/CAP%3DoziR%3DrkT%3DCHquc%3DOCB1WbmLFdGuJy0CWadM7bMs8-cGGPNw%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/django-users/CAP%3DoziR%3DrkT%3DCHquc%3DOCB1WbmLFdGuJy0CWadM7bMs8-cGGPNw%40mail.gmail.com?utm_medium=email&utm_source=footer>. > > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-users/1DD30686-3E37-4217-AC5A-F865A522F059%40gmail.com > > <https://groups.google.com/d/msgid/django-users/1DD30686-3E37-4217-AC5A-F865A522F059%40gmail.com?utm_medium=email&utm_source=footer>. > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-users/CAGQ3pf-hZFLu6JpfTg7qj0jJ92v5br38z9Dx2m%3DkKwouiZZhFw%40mail.gmail.com > > <https://groups.google.com/d/msgid/django-users/CAGQ3pf-hZFLu6JpfTg7qj0jJ92v5br38z9Dx2m%3DkKwouiZZhFw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
-- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/73558DAD-CAE6-4275-A8F0-F3A7C47E1514%40gmail.com.
