Ah, so is the problem that you don’t always know what the delimiter is when you read it? If yes, what is the use case for this? You might not need a universal solution, maybe just put all the info into a csv yourself, manually.
> On Jul 22, 2020, at 10:39 AM, Ronaldo Mata <[email protected]> wrote: > > Hi Kovy, I'm using csv module, but I need to handle the delimiters of the > files, sometimes you come separated by "," others by ";" and rarely by "|" > > El mié., 22 jul. 2020 a las 10:28, Kovy Jacob (<[email protected] > <mailto:[email protected]>>) escribió: > Could you just use the standard python csv module? > >> On Jul 22, 2020, at 10:25 AM, Ronaldo Mata <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hi Liu thank for your answer. >> >> This has been a headache, I am trying to read the file using csv.DictReader >> initially i had an error trying to get the dict keys when iterating by rows, >> and i thought it could be encoding (for this reason i wanted to prepare the >> view to use the correct encoding). for that reason I asked my question. >> >> 1) your first approach doesn't work, if i send utf-8 file, chardet returns >> ascii as encoding. it seems request.FILES ['file']. read () returns a binary >> with that encoding. >> >> 2) In the end I realized that the problem was the delimiter of the csv but >> predicting it is another problem. >> >> Anyway, it was a task that I had to do and that was my limitation. I think >> there must be a library that does all this, uploading a csv file is common >> practice in many web apps. >> >> El mar., 21 jul. 2020 a las 13:47, Liu Zheng (<[email protected] >> <mailto:[email protected]>>) escribió: >> Hi. First of all, I think it's impossible to perfectly detect encoding >> without further information. See the answer in this SO post: >> https://stackoverflow.com/questions/436220/how-to-determine-the-encoding-of-text >> >> <https://stackoverflow.com/questions/436220/how-to-determine-the-encoding-of-text> >> There are many packages and tools to help detect encoding format, but keep >> in mind that they are only giving educated guesses. (Most of the time, the >> guess is correct, but do check the dev page to see whether there are known >> issues related to your problem.) >> >> Now let's say you have decided to use chardet. Check its doc page for the >> usage: https://chardet.readthedocs.io/en/latest/usage.html#usage >> <https://chardet.readthedocs.io/en/latest/usage.html#usage> You'll have more >> than one solutions. Here are some examples: >> >> 1. If the files uploaded to your server are all expected to be small csv >> files (less than a few MB and not many users do it concurrently), you can do >> the following: >> >> #in the view to handle the uploaded file: (assume file input name is just >> "file") >> file_content = request.FILES['file'].read() >> chardet.detect(file_content) >> >> 2. Also, chardet seems to support incremental (line-by-line) detection >> https://chardet.readthedocs.io/en/latest/usage.html#example-detecting-encoding-incrementally >> >> <https://chardet.readthedocs.io/en/latest/usage.html#example-detecting-encoding-incrementally> >> >> Given this, we can also read from requests.FILES line by line and pass each >> line to chardet >> >> from chardet.universaldetector import UniversalDetector >> >> #somewhere in a view function >> detector = UniversalDetector() >> file_handle = request.FILES['file'] >> for line in file_handle: >> detector.feed(line) >> if detector.done: break >> detector.close() >> # result available as a dict at detector.result >> >> >> >> >> >> On Tuesday, July 21, 2020 at 7:09:35 AM UTC+8, Ronaldo Mata wrote: >> How to deal with encoding when you try to read a csv file on view. >> >> I have a view to upload csv file, in this view I read file and save each row >> as new record. >> >> My bug is when I try to upload a csv file with a differente encoding (not >> UTF-8) >> >> how to handle this on django (using request.FILES) I was researching and I >> found chardet but I don't know how to pass it a request.FILES. I need help >> please. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] >> <mailto:[email protected]>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-users/64307441-0e65-45a2-b917-ece15a4ea729o%40googlegroups.com >> >> <https://groups.google.com/d/msgid/django-users/64307441-0e65-45a2-b917-ece15a4ea729o%40googlegroups.com?utm_medium=email&utm_source=footer>. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] >> <mailto:[email protected]>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-users/CAP%3DoziQuZyb74Wsk%2BnjngUpSccOKCYRM_C%3D7KgGX%2BgV5wRzHwQ%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/django-users/CAP%3DoziQuZyb74Wsk%2BnjngUpSccOKCYRM_C%3D7KgGX%2BgV5wRzHwQ%40mail.gmail.com?utm_medium=email&utm_source=footer>. > > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-users/91E9FE01-4701-478C-B575-2BD5BA5DCE86%40gmail.com > > <https://groups.google.com/d/msgid/django-users/91E9FE01-4701-478C-B575-2BD5BA5DCE86%40gmail.com?utm_medium=email&utm_source=footer>. > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-users/CAP%3DoziSjnUSkWgHqb1RzsSHsUURLM9%3DPP0ZNX_zORkp3v-L1%2BQ%40mail.gmail.com > > <https://groups.google.com/d/msgid/django-users/CAP%3DoziSjnUSkWgHqb1RzsSHsUURLM9%3DPP0ZNX_zORkp3v-L1%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/1471A9A8-8BFD-41B0-9AC4-2EA424F1F989%40gmail.com.
