That’s probably not the proper answer, but that’s the best I can do. Sorry :-(
> On Jul 22, 2020, at 10:46 AM, Ronaldo Mata <[email protected]> wrote: > > Yes, the problem here is that the files will be loaded by the user, so I > don't know what delimiter I will receive. This is not a base command that I > am using, it is the logic that I want to incorporate in a view > > El mié., 22 jul. 2020 a las 10:43, Kovy Jacob (<[email protected] > <mailto:[email protected]>>) escribió: > Ah, so is the problem that you don’t always know what the delimiter is when > you read it? If yes, what is the use case for this? You might not need a > universal solution, maybe just put all the info into a csv yourself, manually. > >> On Jul 22, 2020, at 10:39 AM, Ronaldo Mata <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hi Kovy, I'm using csv module, but I need to handle the delimiters of the >> files, sometimes you come separated by "," others by ";" and rarely by "|" >> >> El mié., 22 jul. 2020 a las 10:28, Kovy Jacob (<[email protected] >> <mailto:[email protected]>>) escribió: >> Could you just use the standard python csv module? >> >>> On Jul 22, 2020, at 10:25 AM, Ronaldo Mata <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Hi Liu thank for your answer. >>> >>> This has been a headache, I am trying to read the file using csv.DictReader >>> initially i had an error trying to get the dict keys when iterating by >>> rows, and i thought it could be encoding (for this reason i wanted to >>> prepare the view to use the correct encoding). for that reason I asked my >>> question. >>> >>> 1) your first approach doesn't work, if i send utf-8 file, chardet returns >>> ascii as encoding. it seems request.FILES ['file']. read () returns a >>> binary with that encoding. >>> >>> 2) In the end I realized that the problem was the delimiter of the csv but >>> predicting it is another problem. >>> >>> Anyway, it was a task that I had to do and that was my limitation. I think >>> there must be a library that does all this, uploading a csv file is common >>> practice in many web apps. >>> >>> El mar., 21 jul. 2020 a las 13:47, Liu Zheng (<[email protected] >>> <mailto:[email protected]>>) escribió: >>> Hi. First of all, I think it's impossible to perfectly detect encoding >>> without further information. See the answer in this SO post: >>> https://stackoverflow.com/questions/436220/how-to-determine-the-encoding-of-text >>> >>> <https://stackoverflow.com/questions/436220/how-to-determine-the-encoding-of-text> >>> There are many packages and tools to help detect encoding format, but keep >>> in mind that they are only giving educated guesses. (Most of the time, the >>> guess is correct, but do check the dev page to see whether there are known >>> issues related to your problem.) >>> >>> Now let's say you have decided to use chardet. Check its doc page for the >>> usage: https://chardet.readthedocs.io/en/latest/usage.html#usage >>> <https://chardet.readthedocs.io/en/latest/usage.html#usage> You'll have >>> more than one solutions. Here are some examples: >>> >>> 1. If the files uploaded to your server are all expected to be small csv >>> files (less than a few MB and not many users do it concurrently), you can >>> do the following: >>> >>> #in the view to handle the uploaded file: (assume file input name is just >>> "file") >>> file_content = request.FILES['file'].read() >>> chardet.detect(file_content) >>> >>> 2. Also, chardet seems to support incremental (line-by-line) detection >>> https://chardet.readthedocs.io/en/latest/usage.html#example-detecting-encoding-incrementally >>> >>> <https://chardet.readthedocs.io/en/latest/usage.html#example-detecting-encoding-incrementally> >>> >>> Given this, we can also read from requests.FILES line by line and pass each >>> line to chardet >>> >>> from chardet.universaldetector import UniversalDetector >>> >>> #somewhere in a view function >>> detector = UniversalDetector() >>> file_handle = request.FILES['file'] >>> for line in file_handle: >>> detector.feed(line) >>> if detector.done: break >>> detector.close() >>> # result available as a dict at detector.result >>> >>> >>> >>> >>> >>> On Tuesday, July 21, 2020 at 7:09:35 AM UTC+8, Ronaldo Mata wrote: >>> How to deal with encoding when you try to read a csv file on view. >>> >>> I have a view to upload csv file, in this view I read file and save each >>> row as new record. >>> >>> My bug is when I try to upload a csv file with a differente encoding (not >>> UTF-8) >>> >>> how to handle this on django (using request.FILES) I was researching and I >>> found chardet but I don't know how to pass it a request.FILES. I need help >>> please. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Django users" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected] >>> <mailto:[email protected]>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/django-users/64307441-0e65-45a2-b917-ece15a4ea729o%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/django-users/64307441-0e65-45a2-b917-ece15a4ea729o%40googlegroups.com?utm_medium=email&utm_source=footer>. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Django users" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected] >>> <mailto:[email protected]>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/django-users/CAP%3DoziQuZyb74Wsk%2BnjngUpSccOKCYRM_C%3D7KgGX%2BgV5wRzHwQ%40mail.gmail.com >>> >>> <https://groups.google.com/d/msgid/django-users/CAP%3DoziQuZyb74Wsk%2BnjngUpSccOKCYRM_C%3D7KgGX%2BgV5wRzHwQ%40mail.gmail.com?utm_medium=email&utm_source=footer>. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] >> <mailto:[email protected]>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-users/91E9FE01-4701-478C-B575-2BD5BA5DCE86%40gmail.com >> >> <https://groups.google.com/d/msgid/django-users/91E9FE01-4701-478C-B575-2BD5BA5DCE86%40gmail.com?utm_medium=email&utm_source=footer>. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] >> <mailto:[email protected]>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-users/CAP%3DoziSjnUSkWgHqb1RzsSHsUURLM9%3DPP0ZNX_zORkp3v-L1%2BQ%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/django-users/CAP%3DoziSjnUSkWgHqb1RzsSHsUURLM9%3DPP0ZNX_zORkp3v-L1%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>. > > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-users/1471A9A8-8BFD-41B0-9AC4-2EA424F1F989%40gmail.com > > <https://groups.google.com/d/msgid/django-users/1471A9A8-8BFD-41B0-9AC4-2EA424F1F989%40gmail.com?utm_medium=email&utm_source=footer>. > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-users/CAP%3DoziR%3DrkT%3DCHquc%3DOCB1WbmLFdGuJy0CWadM7bMs8-cGGPNw%40mail.gmail.com > > <https://groups.google.com/d/msgid/django-users/CAP%3DoziR%3DrkT%3DCHquc%3DOCB1WbmLFdGuJy0CWadM7bMs8-cGGPNw%40mail.gmail.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/1DD30686-3E37-4217-AC5A-F865A522F059%40gmail.com.
