OK that was it. Authority Files need to be encoded in ANSI/ASCII for Arches to import them. Wouldn't it be better to use UTF-8?
Cheers, Tobias 2014-04-01 17:48 GMT+02:00 Tobias Kohr <tobiask...@gmail.com>: > Adding this line > > COMPONENT CERTAINTY TYPE.E55,COMPONENT CERTAINTY TYPE AUTHORITY > DOCUMENT.csv,EH SOURCE DATA > > to ENTITY_TYPE_X_ADOC.csv doesn't solve the problem. What > does authoritydocconceptschemename refer to? The DB scheme? > > The Authority Files should be encoded in UTF-8 (on Linux), right? Does > with/without BOM make a difference? > Notepad++ on Windows tells me that the Authority Files of the default cds > package are encoded in ANSI, however, which surprises me. > I will try to see if the error is an encoding problem. > > Best, > Tobias > > > 2014-04-01 17:16 GMT+02:00 Adam Lodge <alo...@fargeo.com>: > > I'm not sure, but it's the first thing I'd try. >> >> Adam >> On Apr 1, 2014 8:14 AM, "Tobias Kohr" <tobiask...@gmail.com> wrote: >> >>> Adam, >>> >>> alright, I was not aware of the existence of this file (using Arches >>> v2). I guess we need include our newly defined entity type here. Is this >>> the reason for the error? >>> >>> Thanks! >>> -Tobias >>> >>> >>> 2014-04-01 17:02 GMT+02:00 Adam Lodge <alo...@fargeo.com>: >>> >>>> Tobias, >>>> >>>> No such thing as a stupid question. >>>> >>>> Assuming that you are running v2 of Arches, the file should exist in >>>> this folder: source_data\concepts\authority_files , and its name is >>>> actually ENTITY_TYPE_X_ADOC.csv >>>> >>>> If you're running an earlier version, just send me COMPONENT CERTAINTY >>>> TYPE AUTHORITY DOCUMENT.CSV and we'll start there. >>>> >>>> -- >>>> Adam Lodge >>>> Geospatial Systems Consultant >>>> Farallon Geographics >>>> 415.317.6625 >>>> >>>> On Tuesday, April 1, 2014 at 7:57 AM, Tobias Kohr wrote: >>>> >>>> Hi, >>>> >>>> I feel a little stupid asking this, but which file do you mean >>>> with ENTITY_TYPE_X_AUTHDOC.csv? (Perhaps already the solution to my >>>> problem?) >>>> >>>> Cheers, >>>> Tobias >>>> >>>> On Tuesday, April 1, 2014 4:46:28 PM UTC+2, Adam Lodge wrote: >>>> >>>> Tobias, >>>> >>>> Could you send me a the COMPONENT CERTAINTY TYPE AUTHORITY DOCUMENT.CSV >>>> file and the ENTITY_TYPE_X_AUTHDOC.csv file? With those, I can probably >>>> tell you what the issue is. >>>> >>>> Adam >>>> >>>> -- >>>> Adam Lodge >>>> Geospatial Systems Consultant >>>> Farallon Geographics >>>> 415.317.6625 >>>> >>>> On Tuesday, April 1, 2014 at 7:43 AM, Tobias Kohr wrote: >>>> >>>> Hi Dennis, Koen, et al. >>>> >>>> we will have a closer look at the CIDOC extension and try to keep in >>>> mind that people have different interpretations for uncerainty. >>>> >>>> Regarding the technical implementation we're encountering problems in >>>> step 3, running install_packages.sh which throws the following error: >>>> >>>> root@srv-i3-fundstellendb:/arches-web/archesproject/build# source >>>> install_packages.sh >>>> Install packages defined in settings.py >>>> operation: install >>>> ...||ABSOLUTE DATING METHOD AUTHORITY DOCUMENT.CSV >>>> ADMINISTRATIVE SUBDIVISION AUTHORITY DOCUMENT.CSV >>>> ADMINISTRATIVE SUBDIVISION AUTHORITY DOCUMENT.VALUES.CSV >>>> ADMINISTRATIVE SUBDIVISION TYPE AUTHORITY DOCUMENT.CSV >>>> ARCHAEOLOGICAL COMPONENT TYPE AUTHORITY DOCUMENT.CSV >>>> ARCHAEOLOGICAL HERITAGE (ARTIFACT) TYPE AUTHORITY DOCUMENT.CSV >>>> ARCHAEOLOGICAL HERITAGE (SITE) TYPE AUTHORITY DOCUMENT.CSV >>>> ARCHAEOLOGICAL TECHNIQUE AUTHORITY DOCUMENT.CSV >>>> ARCHES RESOURCE CROSS-REFERENCE RELATIONSHIP TYPE AUTHORITY DOCUMENT.CSV >>>> ARCHES RESOURCE CROSS-REFERENCE RELATIONSHIP TYPE AUTHORITY >>>> DOCUMENT.VALUES.CSV >>>> ARCHITECTURAL COMPONENT TYPE AUTHORITY DOCUMENT.CSV >>>> ARCHITECTURAL HERITAGE TYPE AUTHORITY DOCUMENT.CSV >>>> ARCHITECTURAL TECHNIQUE AUTHORITY DOCUMENT.CSV >>>> COMPONENT CERTAINTY TYPE AUTHORITY DOCUMENT.CSV >>>> Traceback (most recent call last): >>>> File "../manage.py", line 28, in <module> >>>> execute_from_command_line(sys.argv) >>>> File "/arches-web/archesproject/virtualenv/ENV/local/lib/ >>>> python2.7/site-packages/django/core/management/__init__.py", line 399, >>>> in execute_from_command_line >>>> utility.execute() >>>> File "/arches-web/archesproject/virtualenv/ENV/local/lib/ >>>> python2.7/site-packages/django/core/management/__init__.py", line 392, >>>> in execute >>>> self.fetch_command(subcommand).run_from_argv(self.argv) >>>> File "/arches-web/archesproject/virtualenv/ENV/local/lib/ >>>> python2.7/site-packages/django/core/management/base.py", line 242, in >>>> run_from_argv >>>> self.execute(*args, **options.__dict__) >>>> File "/arches-web/archesproject/virtualenv/ENV/local/lib/ >>>> python2.7/site-packages/django/core/management/base.py", line 285, in >>>> execute >>>> output = self.handle(*args, **options) >>>> File "/arches-web/archesproject/build/management/commands/packages.py", >>>> line 47, in handle >>>> self.load_package(package) >>>> File "/arches-web/archesproject/build/management/commands/packages.py", >>>> line 52, in load_package >>>> install(settings.ROOT_DIR) >>>> File "/arches-web/archesproject/packages/cdscert/setup.py", line 60, >>>> in install >>>> authority_files.load_authority_files(package_settings.ROOT_DIR) >>>> File >>>> "/arches-web/archesproject/packages/cdscert/install/authority_files.py", >>>> line 22, in load_authority_files >>>> load_authority_file(cursor, mapping_files_directory, file_name) >>>> File >>>> "/arches-web/archesproject/packages/cdscert/install/authority_files.py", >>>> line 63, in load_authority_file >>>> concepts.insert_concept(settings.DATA_CONCEPT_SCHEME, >>>> adoc_dict['PREFLABEL'], '', 'en-us', adoc_dict['CONCEPTID']) >>>> KeyError: 'CONCEPTID' >>>> >>>> Our provisional Authority Files look like this: >>>> >>>> - COMPONENT CERTAINTY TYPE AUTHORITY DOCUMENT.csv >>>> conceptid,PrefLabel,AltLabels,ParentConceptid,ConceptType,Provider >>>> COMPONENT_CERTAINTY_1,certain,,COMPONENT CERTAINTY TYPE AUTHORITY >>>> DOCUMENT.csv,Index,i3mainz >>>> COMPONENT_CERTAINTY_2,uncertain,,COMPONENT CERTAINTY TYPE AUTHORITY >>>> DOCUMENT.csv,Index,i3mainz >>>> COMPONENT_CERTAINTY_3,unknown,,COMPONENT CERTAINTY TYPE AUTHORITY >>>> DOCUMENT.csv,Index,i3mainz >>>> >>>> - COMPONENT CERTAINTY TYPE AUTHORITY DOCUMENT.values.csv (do we need >>>> this one?) >>>> conceptid,Value,ValueType,Provider >>>> COMPONENT_CERTAINTY_1,1,sortorder,i3mainz >>>> COMPONENT_CERTAINTY_2,2,sortorder,i3mainz >>>> COMPONENT_CERTAINTY_3,3,sortorder,i3mainz >>>> >>>> Can anybody tell us what's wrong with our conceptid? Does the >>>> authority_files.py search for the ID in any additional place, where we >>>> should reference it? >>>> >>>> Cheers, >>>> Tobias >>>> >>>> On Monday, March 31, 2014 11:22:27 PM UTC+2, Koen Van Daele wrote: >>>> >>>> Hi all, >>>> >>>> I just wanted to get back at what Dennis said at the beginning of this >>>> thread. Im quite curious how you will get people to agree on (un)certainty. >>>> If feels like a very natural idea to talk and think about, but I haven't >>>> really seen it function properly in practice. >>>> >>>> We once did an experiment where we had 10 people who were used to >>>> entering data in our archaeological inventory system enter the same site. >>>> We paired the archaeologists: one more more experienced data entry person >>>> (a few years experience) and one newbie (a few months), so they would be >>>> forced to really think things through and discuss. In our database we have >>>> a field for certain the data entry person is about the location of the >>>> site, ie. about the polygon they might have drawn on a map. This field only >>>> allowed 5 choices, ranging from 1 (I'm sure it's exactly where it needs to >>>> be) to 5 (I have no idea whatsoever where the site is). We had a very >>>> detailed manual with examples of all these cases, what to use when, ... >>>> Final result of our experiment: every group had entered the location >>>> with a different level of certainty. So, based on the exact same >>>> information they had all drawn totally different conclusions. And this was >>>> about something as simple as the location of the site. >>>> >>>> So, I'm very curious about how you manage to prevent stuff like this >>>> from happening. >>>> >>>> The other thing I wonder about: how does certainty affect searching? >>>> Should a search for 'churches' only return sites that have a certain >>>> "certainty" attached to the interpretation? Are you working with sliding >>>> scale of certainty (ie. we are 75% percent certain about this statement) or >>>> a binary one (we're certain or uncertain)? >>>> >>>> Cheers, >>>> Koen >>>> ________________________________________ >>>> Van: arches...@googlegroups.com [arches...@googlegroups.com] namens >>>> dwut...@fargeo.com [dwut...@fargeo.com] >>>> Verzonden: donderdag 27 maart 2014 22:36 >>>> Aan: thomas....@gmail.com >>>> CC: arches...@googlegroups.com >>>> Onderwerp: Re: [Arches] "uncertain information" in Arches >>>> >>>> Thomas, >>>> >>>> Good question! You are quite correct that we haven't tried to include >>>> uncertainty in Arches. >>>> >>>> One reason is pretty basic: certainty is quite subjective from person >>>> to person. For example: most people agree that the earth is spherical. >>>> But a "flat-earther" may be very certain that the earth is not a sphere, >>>> but is instead a plane. His certainty does not make him correct, it merely >>>> states the degree to which he believes in his interpretation. Clearly, you >>>> can be very certain and very wrong at the same time. I guess my point is >>>> that in many cases "certainty" says more about the person making the >>>> assertion than it does about the thing being described. >>>> >>>> OK, all philosophy aside, one could easily extend any Arches graph to >>>> include a "certainty node". Such a node could point to a thesaurus (as >>>> many of the nodes in Arches already do), allowing a user to select from a >>>> list of "uncertainty levels". Really, any Arches graph could include a >>>> "certainty node" under any entity that you might want to qualify (for >>>> example, one certainty node for period and another certainty node for >>>> heritage type). >>>> >>>> Really, the hard part is not in getting Arches to allow you to add an >>>> "uncertainty level" to your cultural heritage data. Rather, the difficult >>>> thing is to decide how you'll get different people to agree on what >>>> constitutes certain vs. uncertain interpretations of heritage. >>>> >>>> Sorry that I can't be any more helpful... However, I'm very interested to >>>> hear how you will model uncertainty and how you will get people to >>>> implement it consistently. Please keep me posted! >>>> >>>> Cheers, >>>> >>>> Dennis >>>> >>>> >>>> On Mar 27, 2014, at 2:41, thomas....@gmail.com<mailto:th >>>> omas....@gmail.com> wrote: >>>> >>>> I have a question about conceptual modeling in CIDOC CRM, maybe there >>>> is someone one the list who is able to provide some guidance. >>>> >>>> As posted before, we are trying to integrate research data of neolithic >>>> sites into Arches. Now, naturally a significant part of this data has a >>>> level of "certainty" to which the information is correct. e.g. a site can >>>> consist of some features for certain (in this case modeled in the >>>> Archaeological Heritage (Site).E27 - Component.E18 relationship) but if >>>> others exist is uncertain. We believe this valuable information should not >>>> get lost (quite often theory construction is based on such information). >>>> >>>> For example it could be uncertain if an archaeological feature is to be >>>> named "pit" or "ditch" - or if it exists at all. Another example could be >>>> the questionable relationship of a findspot to a certain archaeological >>>> period. To make it even more difficult, different authors could have >>>> different thoughts on that. >>>> >>>> As far as we can see, the expression of such "uncertainty" is not >>>> covered by Arches yet. Is there a concept for the integration of such data >>>> in the future? We are currently thinking into potential solutions but are >>>> struggeling to find adequate expressions for uncertain information in >>>> CIDOC. >>>> >>>> thanks, Thomas >>>> >>>> -- >>>> -- To post, send email to arches...@googlegroups.com<mailto: >>>> arches...@googlegroups.com>. To unsubscribe, send email to >>>> archesprojec...@googlegroups.com<mailto:archesproject+ >>>> unsubscr...@googlegroups.com>. For more information, visit >>>> https://groups.google.com/d/forum/archesproject?hl=en >>>> --- You received this message because you are subscribed to the Google >>>> Groups "Arches Project" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to archesprojec...@googlegroups.com<mailto:archesproject+ >>>> unsubscr...@googlegroups.com>. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> -- >>>> -- To post, send email to arches...@googlegroups.com. To unsubscribe, >>>> send email to archesprojec...@googlegroups.com. For more information, >>>> visit https://groups.google.com/d/forum/archesproject?hl=en >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Arches Project" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to archesprojec...@googlegroups.com<mailto:archesproject+ >>>> unsubscr...@googlegroups.com>. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> -- >>>> -- To post, send email to arches...@googlegroups.com. To unsubscribe, >>>> send email to archesprojec...@googlegroups.com. For more information, >>>> visit https://groups.google.com/d/forum/archesproject?hl=en >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Arches Project" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to archesprojec...@googlegroups.com. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> -- >>>> -- To post, send email to archesproject@googlegroups.com. To >>>> unsubscribe, send email to archesproject+unsubscr...@googlegroups.com. >>>> For more information, visit >>>> https://groups.google.com/d/forum/archesproject?hl=en >>>> >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Arches Project" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to archesproject+unsubscr...@googlegroups.com. >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> >>> > -- -- To post, send email to archesproject@googlegroups.com. To unsubscribe, send email to archesproject+unsubscr...@googlegroups.com. For more information, visit https://groups.google.com/d/forum/archesproject?hl=en --- You received this message because you are subscribed to the Google Groups "Arches Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to archesproject+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
