Hi Kovy, this is not solved. Liu Zheng but using
chardet(request.FILES['file'].read()) return encoding "ascii" is not
correct, I've uploaded a file using utf-7 as encoding for example and the
result is wrog. and then I tried
request.FILES['file'].read().decode('ascii') and not work return bad data.
Example for @ string return "+AEA-" string.El mié., 22 jul. 2020 a las 11:16, Kovy Jacob (<[email protected]>) escribió: > I’m confused. I don’t know if I can help. > > On Jul 22, 2020, at 11:11 AM, Liu Zheng <[email protected]> wrote: > > Hi, glad you solved the problem. Yes, both the request.FILES[‘file’] and > the chardet file handler are binary handlers. Binary handler presents the > raw data. chardet takes a sequence or raw data and then detect the encoding > format. With its prediction, if you want to open that puece of data in text > mode, you can use the .decode(<encoding format>) method of bytes object to > get a python string. > > On Wed, 22 Jul 2020 at 11:04 PM, Kovy Jacob <[email protected]> wrote: > >> That’s probably not the proper answer, but that’s the best I can do. >> Sorry :-( >> >> >> On Jul 22, 2020, at 10:46 AM, Ronaldo Mata <[email protected]> >> wrote: >> >> Yes, the problem here is that the files will be loaded by the user, so I >> don't know what delimiter I will receive. This is not a base command that I >> am using, it is the logic that I want to incorporate in a view >> >> El mié., 22 jul. 2020 a las 10:43, Kovy Jacob (<[email protected]>) >> escribió: >> >>> Ah, so is the problem that you don’t always know what the delimiter is >>> when you read it? If yes, what is the use case for this? You might not need >>> a universal solution, maybe just put all the info into a csv yourself, >>> manually. >>> >>> On Jul 22, 2020, at 10:39 AM, Ronaldo Mata <[email protected]> >>> wrote: >>> >>> Hi Kovy, I'm using csv module, but I need to handle the delimiters of >>> the files, sometimes you come separated by "," others by ";" and rarely by >>> "|" >>> >>> El mié., 22 jul. 2020 a las 10:28, Kovy Jacob (<[email protected]>) >>> escribió: >>> >>>> Could you just use the standard python csv module? >>>> >>>> On Jul 22, 2020, at 10:25 AM, Ronaldo Mata <[email protected]> >>>> wrote: >>>> >>>> Hi Liu thank for your answer. >>>> >>>> This has been a headache, I am trying to read the file using >>>> csv.DictReader initially i had an error trying to get the dict keys when >>>> iterating by rows, and i thought it could be encoding (for this reason i >>>> wanted to prepare the view to use the correct encoding). for that reason I >>>> asked my question. >>>> >>>> 1) your first approach doesn't work, if i send utf-8 file, chardet >>>> returns ascii as encoding. it seems request.FILES ['file']. read () returns >>>> a binary with that encoding. >>>> >>>> 2) In the end I realized that the problem was the delimiter of the csv >>>> but predicting it is another problem. >>>> >>>> Anyway, it was a task that I had to do and that was my limitation. I >>>> think there must be a library that does all this, uploading a csv file is >>>> common practice in many web apps. >>>> >>>> El mar., 21 jul. 2020 a las 13:47, Liu Zheng (<[email protected]>) >>>> escribió: >>>> >>>>> Hi. First of all, I think it's impossible to perfectly detect encoding >>>>> without further information. See the answer in this SO post: >>>>> https://stackoverflow.com/questions/436220/how-to-determine-the-encoding-of-text >>>>> There >>>>> are many packages and tools to help detect encoding format, but keep in >>>>> mind that they are only giving educated guesses. (Most of the time, the >>>>> guess is correct, but do check the dev page to see whether there are known >>>>> issues related to your problem.) >>>>> >>>>> Now let's say you have decided to use chardet. Check its doc page for >>>>> the usage: https://chardet.readthedocs.io/en/latest/usage.html#usage >>>>> You'll >>>>> have more than one solutions. Here are some examples: >>>>> >>>>> 1. If the files uploaded to your server are all expected to be small >>>>> csv files (less than a few MB and not many users do it concurrently), you >>>>> can do the following: >>>>> >>>>> #in the view to handle the uploaded file: (assume file input name is >>>>> just "file") >>>>> file_content = request.FILES['file'].read() >>>>> chardet.detect(file_content) >>>>> >>>>> 2. Also, chardet seems to support incremental (line-by-line) detection >>>>> https://chardet.readthedocs.io/en/latest/usage.html#example-detecting-encoding-incrementally >>>>> >>>>> Given this, we can also read from requests.FILES line by line and pass >>>>> each line to chardet >>>>> >>>>> from chardet.universaldetector import UniversalDetector >>>>> >>>>> #somewhere in a view function >>>>> detector = UniversalDetector() >>>>> file_handle = request.FILES['file'] >>>>> for line in file_handle: >>>>> detector.feed(line) >>>>> if detector.done: break >>>>> detector.close() >>>>> # result available as a dict at detector.result >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Tuesday, July 21, 2020 at 7:09:35 AM UTC+8, Ronaldo Mata wrote: >>>>>> >>>>>> How to deal with encoding when you try to read a csv file on view. >>>>>> >>>>>> I have a view to upload csv file, in this view I read file and save >>>>>> each row as new record. >>>>>> >>>>>> My bug is when I try to upload a csv file with a differente encoding >>>>>> (not UTF-8) >>>>>> >>>>>> how to handle this on django (using request.FILES) I was researching >>>>>> and I found chardet but I don't know how to pass it a request.FILES. I >>>>>> need >>>>>> help please. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Django users" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/django-users/64307441-0e65-45a2-b917-ece15a4ea729o%40googlegroups.com >>>>> <https://groups.google.com/d/msgid/django-users/64307441-0e65-45a2-b917-ece15a4ea729o%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Django users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/django-users/CAP%3DoziQuZyb74Wsk%2BnjngUpSccOKCYRM_C%3D7KgGX%2BgV5wRzHwQ%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/django-users/CAP%3DoziQuZyb74Wsk%2BnjngUpSccOKCYRM_C%3D7KgGX%2BgV5wRzHwQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Django users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/django-users/91E9FE01-4701-478C-B575-2BD5BA5DCE86%40gmail.com >>>> <https://groups.google.com/d/msgid/django-users/91E9FE01-4701-478C-B575-2BD5BA5DCE86%40gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Django users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/django-users/CAP%3DoziSjnUSkWgHqb1RzsSHsUURLM9%3DPP0ZNX_zORkp3v-L1%2BQ%40mail.gmail.com >>> <https://groups.google.com/d/msgid/django-users/CAP%3DoziSjnUSkWgHqb1RzsSHsUURLM9%3DPP0ZNX_zORkp3v-L1%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Django users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/django-users/1471A9A8-8BFD-41B0-9AC4-2EA424F1F989%40gmail.com >>> <https://groups.google.com/d/msgid/django-users/1471A9A8-8BFD-41B0-9AC4-2EA424F1F989%40gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-users/CAP%3DoziR%3DrkT%3DCHquc%3DOCB1WbmLFdGuJy0CWadM7bMs8-cGGPNw%40mail.gmail.com >> <https://groups.google.com/d/msgid/django-users/CAP%3DoziR%3DrkT%3DCHquc%3DOCB1WbmLFdGuJy0CWadM7bMs8-cGGPNw%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-users/1DD30686-3E37-4217-AC5A-F865A522F059%40gmail.com >> <https://groups.google.com/d/msgid/django-users/1DD30686-3E37-4217-AC5A-F865A522F059%40gmail.com?utm_medium=email&utm_source=footer> >> . >> > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-users/CAGQ3pf-hZFLu6JpfTg7qj0jJ92v5br38z9Dx2m%3DkKwouiZZhFw%40mail.gmail.com > <https://groups.google.com/d/msgid/django-users/CAGQ3pf-hZFLu6JpfTg7qj0jJ92v5br38z9Dx2m%3DkKwouiZZhFw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-users/73558DAD-CAE6-4275-A8F0-F3A7C47E1514%40gmail.com > <https://groups.google.com/d/msgid/django-users/73558DAD-CAE6-4275-A8F0-F3A7C47E1514%40gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAP%3DoziSHnZFKiXON8b5Jn7hu7LVX-jHCOQ%2BHUSeiBO%3DF3Q_yxw%40mail.gmail.com.
