Hi Adam, Still not sure how to get past this. I see it is an n-dash (I should have looked this up, rather than assumed it was related to Chinese).
The question for me is why is this code assuming all ascii since it allowed an ndash into the database? Is there a way to fix this code, rather update the content? Darcy On Saturday, September 16, 2017 at 12:59:44 AM UTC+10, Adam Cox wrote: > > Darcy, it looks like this is not a chinese character, but a long dash. > > This issue seems to be well-summed up in this stack exchange answer: > https://stackoverflow.com/a/5387966/3873885 > > Essentially, in this line > > csvwriter.writerow({k: str(v).encode('utf8') for k, v in > csv_record.items()}) > > the str() operation is encoding v (which in this case is a unicode object) > to ascii, the default encoding for a str object in python 2.7. Then, that > ascii-encoded string is further encoded into utf-8. I assume the initial > str() operation is meant to handle integers and other non-text obects, but > you've found an example where because u'\u2013' (unicode character 2013 > <http://www.fileformat.info/info/unicode/char/2013/index.htm>) cannot be > encoded in ascii, it hits an error even before it has a chance to encode to > utf-8. > So, I think that line in the code could be improved. > > It looks like that line comes from the related resource export > <https://github.com/archesproject/arches/blob/stable/3.x/arches/management/commands/packages.py#L352> > > process. Maybe you have a long dash in one of the notes that you have about > a resource to resource relationship? Otherwise, I'm really not sure where > that problem character would come from... > > Hope that's at least a little helpful. > > Adam > > On Thu, Sep 14, 2017 at 8:01 PM, Darcy Christ <da...@1000camels.com > <javascript:>> wrote: > >> I am having trouble exporting data from v3 >> >> I have add this to my config: >> >> EXPORT_CONFIG = os.path.normpath(os.path.join(PACKAGE_ROOT, >> 'source_data', 'business_data', 'resource_export_mappings.json')) >> >> >> And then I get an error while trying to export. Given that it is related >> to encoding, could it be any chinese characters I might have in the data? >> >> >> (hkarches) [hkarches@heritage hongkong]$ python manage.py packages -o >> export_resources -d '../hongkong_data' >> operation: export_resources >> package: hongkong >> Writing 3 ACTIVITY.E7 resources >> Writing 370 INFORMATION_RESOURCE.E73 resources >> Writing 1205 HERITAGE_RESOURCE.E18 resources >> Writing 545 ACTOR.E39 resources >> Writing 6 HISTORICAL_EVENT.E5 resources >> Writing 0 HERITAGE_RESOURCE_GROUP.E27 resources >> Traceback (most recent call last): >> File "manage.py", line 28, in <module> >> execute_from_command_line(sys.argv) >> File >> "/home/hkarches/lib/python2.7/site-packages/django/core/management/__init__.py", >> >> line 399, in execute_from_command_line >> utility.execute() >> File >> "/home/hkarches/lib/python2.7/site-packages/django/core/management/__init__.py", >> >> line 392, in execute >> self.fetch_command(subcommand).run_from_argv(self.argv) >> File >> "/home/hkarches/lib/python2.7/site-packages/django/core/management/base.py", >> line 242, in run_from_argv >> self.execute(*args, **options.__dict__) >> File >> "/home/hkarches/lib/python2.7/site-packages/django/core/management/base.py", >> line 285, in execute >> output = self.handle(*args, **options) >> File >> "/home/hkarches/lib/python2.7/site-packages/arches/management/commands/packages.py", >> >> line 106, in handle >> self.export_resources(package_name, options['dest_dir']) >> File >> "/home/hkarches/lib/python2.7/site-packages/arches/management/commands/packages.py", >> >> line 351, in export_resources >> csvwriter.writerow({k: str(v).encode('utf8') for k, v in >> csv_record.items()}) >> File >> "/home/hkarches/lib/python2.7/site-packages/arches/management/commands/packages.py", >> >> line 351, in <dictcomp> >> csvwriter.writerow({k: str(v).encode('utf8') for k, v in >> csv_record.items()}) >> UnicodeEncodeError: 'ascii' codec can't encode character u'\u2013' in >> position 220: ordinal not in range(128) >> >> -- >> -- To post, send email to arches...@googlegroups.com <javascript:>. To >> unsubscribe, send email to archesprojec...@googlegroups.com <javascript:>. >> For more information, visit >> https://groups.google.com/d/forum/archesproject?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Arches Project" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to archesprojec...@googlegroups.com <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > > -- -- To post, send email to archesproject@googlegroups.com. To unsubscribe, send email to archesproject+unsubscr...@googlegroups.com. For more information, visit https://groups.google.com/d/forum/archesproject?hl=en --- You received this message because you are subscribed to the Google Groups "Arches Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to archesproject+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
