It would be better, and thanks for pointing it out. I'm surprised it's not set up that way already. I'll that as a ticket on bitbucket.
Dennis should be getting back to you on setting up a time to chat via Skype. Best, Adam -- Adam Lodge Geospatial Systems Consultant Farallon Geographics 415.317.6625 On Tuesday, April 1, 2014 at 9:16 AM, Tobias Kohr wrote: > OK that was it. Authority Files need to be encoded in ANSI/ASCII for Arches > to import them. Wouldn't it be better to use UTF-8? > > Cheers, > Tobias > > > > 2014-04-01 17:48 GMT+02:00 Tobias Kohr <tobiask...@gmail.com > (mailto:tobiask...@gmail.com)>: > > Adding this line > > > > COMPONENT CERTAINTY TYPE.E55,COMPONENT CERTAINTY TYPE AUTHORITY > > DOCUMENT.csv,EH SOURCE DATA > > > > to ENTITY_TYPE_X_ADOC.csv doesn't solve the problem. What does > > authoritydocconceptschemename refer to? The DB scheme? > > > > The Authority Files should be encoded in UTF-8 (on Linux), right? Does > > with/without BOM make a difference? > > Notepad++ on Windows tells me that the Authority Files of the default cds > > package are encoded in ANSI, however, which surprises me. > > I will try to see if the error is an encoding problem. > > > > Best, > > Tobias > > > > > > > > 2014-04-01 17:16 GMT+02:00 Adam Lodge <alo...@fargeo.com > > (mailto:alo...@fargeo.com)>: > > > > > I'm not sure, but it's the first thing I'd try. > > > Adam > > > On Apr 1, 2014 8:14 AM, "Tobias Kohr" <tobiask...@gmail.com > > > (mailto:tobiask...@gmail.com)> wrote: > > > > Adam, > > > > > > > > alright, I was not aware of the existence of this file (using Arches > > > > v2). I guess we need include our newly defined entity type here. Is > > > > this the reason for the error? > > > > > > > > Thanks! > > > > -Tobias > > > > > > > > > > > > > > > > 2014-04-01 17:02 GMT+02:00 Adam Lodge <alo...@fargeo.com > > > > (mailto:alo...@fargeo.com)>: > > > > > Tobias, > > > > > > > > > > No such thing as a stupid question. > > > > > > > > > > Assuming that you are running v2 of Arches, the file should exist in > > > > > this folder: source_data\concepts\authority_files , and its name is > > > > > actually ENTITY_TYPE_X_ADOC.csv > > > > > > > > > > If you're running an earlier version, just send me COMPONENT > > > > > CERTAINTY TYPE AUTHORITY DOCUMENT.CSV and we'll start there. > > > > > > > > > > -- > > > > > Adam Lodge > > > > > Geospatial Systems Consultant > > > > > Farallon Geographics > > > > > 415.317.6625 (tel:415.317.6625) > > > > > > > > > > > > > > > On Tuesday, April 1, 2014 at 7:57 AM, Tobias Kohr wrote: > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > I feel a little stupid asking this, but which file do you mean with > > > > > > ENTITY_TYPE_X_AUTHDOC.csv? (Perhaps already the solution to my > > > > > > problem?) > > > > > > > > > > > > Cheers, > > > > > > Tobias > > > > > > > > > > > > On Tuesday, April 1, 2014 4:46:28 PM UTC+2, Adam Lodge wrote: > > > > > > > Tobias, > > > > > > > > > > > > > > Could you send me a the COMPONENT CERTAINTY TYPE AUTHORITY > > > > > > > DOCUMENT.CSV file and the ENTITY_TYPE_X_AUTHDOC.csv file? With > > > > > > > those, I can probably tell you what the issue is. > > > > > > > > > > > > > > Adam > > > > > > > > > > > > > > -- > > > > > > > Adam Lodge > > > > > > > Geospatial Systems Consultant > > > > > > > Farallon Geographics > > > > > > > 415.317.6625 (tel:415.317.6625) > > > > > > > > > > > > > > > > > > > > > On Tuesday, April 1, 2014 at 7:43 AM, Tobias Kohr wrote: > > > > > > > > > > > > > > > Hi Dennis, Koen, et al. > > > > > > > > > > > > > > > > we will have a closer look at the CIDOC extension and try to > > > > > > > > keep in mind that people have different interpretations for > > > > > > > > uncerainty. > > > > > > > > > > > > > > > > Regarding the technical implementation we're encountering > > > > > > > > problems in step 3, running install_packages.sh > > > > > > > > (http://install_packages.sh) which throws the following error: > > > > > > > > > > > > > > > > root@srv-i3-fundstellendb:/arches-web/archesproject/build# > > > > > > > > source install_packages.sh (http://install_packages.sh) > > > > > > > > Install packages defined in settings.py > > > > > > > > operation: install > > > > > > > > ...||ABSOLUTE DATING METHOD AUTHORITY DOCUMENT.CSV > > > > > > > > ADMINISTRATIVE SUBDIVISION AUTHORITY DOCUMENT.CSV > > > > > > > > ADMINISTRATIVE SUBDIVISION AUTHORITY DOCUMENT.VALUES.CSV > > > > > > > > ADMINISTRATIVE SUBDIVISION TYPE AUTHORITY DOCUMENT.CSV > > > > > > > > ARCHAEOLOGICAL COMPONENT TYPE AUTHORITY DOCUMENT.CSV > > > > > > > > ARCHAEOLOGICAL HERITAGE (ARTIFACT) TYPE AUTHORITY DOCUMENT.CSV > > > > > > > > ARCHAEOLOGICAL HERITAGE (SITE) TYPE AUTHORITY DOCUMENT.CSV > > > > > > > > ARCHAEOLOGICAL TECHNIQUE AUTHORITY DOCUMENT.CSV > > > > > > > > ARCHES RESOURCE CROSS-REFERENCE RELATIONSHIP TYPE AUTHORITY > > > > > > > > DOCUMENT.CSV > > > > > > > > ARCHES RESOURCE CROSS-REFERENCE RELATIONSHIP TYPE AUTHORITY > > > > > > > > DOCUMENT.VALUES.CSV > > > > > > > > ARCHITECTURAL COMPONENT TYPE AUTHORITY DOCUMENT.CSV > > > > > > > > ARCHITECTURAL HERITAGE TYPE AUTHORITY DOCUMENT.CSV > > > > > > > > ARCHITECTURAL TECHNIQUE AUTHORITY DOCUMENT.CSV > > > > > > > > COMPONENT CERTAINTY TYPE AUTHORITY DOCUMENT.CSV > > > > > > > > Traceback (most recent call last): > > > > > > > > File "../manage.py", line 28, in <module> > > > > > > > > execute_from_command_line(sys.argv) > > > > > > > > File > > > > > > > > "/arches-web/archesproject/virtualenv/ENV/local/lib/python2.7/site-packages/django/core/management/__init__.py", > > > > > > > > line 399, in execute_from_command_line > > > > > > > > utility.execute() > > > > > > > > File > > > > > > > > "/arches-web/archesproject/virtualenv/ENV/local/lib/python2.7/site-packages/django/core/management/__init__.py", > > > > > > > > line 392, in execute > > > > > > > > self.fetch_command(subcommand).run_from_argv(self.argv) > > > > > > > > File > > > > > > > > "/arches-web/archesproject/virtualenv/ENV/local/lib/python2.7/site-packages/django/core/management/base.py", > > > > > > > > line 242, in run_from_argv > > > > > > > > self.execute(*args, **options.__dict__) > > > > > > > > File > > > > > > > > "/arches-web/archesproject/virtualenv/ENV/local/lib/python2.7/site-packages/django/core/management/base.py", > > > > > > > > line 285, in execute > > > > > > > > output = self.handle(*args, **options) > > > > > > > > File > > > > > > > > "/arches-web/archesproject/build/management/commands/packages.py", > > > > > > > > line 47, in handle > > > > > > > > self.load_package(package) > > > > > > > > File > > > > > > > > "/arches-web/archesproject/build/management/commands/packages.py", > > > > > > > > line 52, in load_package > > > > > > > > install(settings.ROOT_DIR) > > > > > > > > File "/arches-web/archesproject/packages/cdscert/setup.py", > > > > > > > > line 60, in install > > > > > > > > > > > > > > > > authority_files.load_authority_files(package_settings.ROOT_DIR) > > > > > > > > File > > > > > > > > "/arches-web/archesproject/packages/cdscert/install/authority_files.py", > > > > > > > > line 22, in load_authority_files > > > > > > > > load_authority_file(cursor, mapping_files_directory, > > > > > > > > file_name) > > > > > > > > File > > > > > > > > "/arches-web/archesproject/packages/cdscert/install/authority_files.py", > > > > > > > > line 63, in load_authority_file > > > > > > > > concepts.insert_concept(settings.DATA_CONCEPT_SCHEME, > > > > > > > > adoc_dict['PREFLABEL'], '', 'en-us', adoc_dict['CONCEPTID']) > > > > > > > > KeyError: 'CONCEPTID' > > > > > > > > > > > > > > > > Our provisional Authority Files look like this: > > > > > > > > > > > > > > > > - COMPONENT CERTAINTY TYPE AUTHORITY DOCUMENT.csv > > > > > > > > conceptid,PrefLabel,AltLabels,ParentConceptid,ConceptType,Provider > > > > > > > > > > > > > > > > COMPONENT_CERTAINTY_1,certain,,COMPONENT CERTAINTY TYPE > > > > > > > > AUTHORITY DOCUMENT.csv,Index,i3mainz > > > > > > > > COMPONENT_CERTAINTY_2,uncertain,,COMPONENT CERTAINTY TYPE > > > > > > > > AUTHORITY DOCUMENT.csv,Index,i3mainz > > > > > > > > COMPONENT_CERTAINTY_3,unknown,,COMPONENT CERTAINTY TYPE > > > > > > > > AUTHORITY DOCUMENT.csv,Index,i3mainz > > > > > > > > > > > > > > > > - COMPONENT CERTAINTY TYPE AUTHORITY DOCUMENT.values.csv (do we > > > > > > > > need this one?) > > > > > > > > conceptid,Value,ValueType,Provider > > > > > > > > COMPONENT_CERTAINTY_1,1,sortorder,i3mainz > > > > > > > > COMPONENT_CERTAINTY_2,2,sortorder,i3mainz > > > > > > > > COMPONENT_CERTAINTY_3,3,sortorder,i3mainz > > > > > > > > > > > > > > > > Can anybody tell us what's wrong with our conceptid? Does the > > > > > > > > authority_files.py search for the ID in any additional place, > > > > > > > > where we should reference it? > > > > > > > > > > > > > > > > Cheers, > > > > > > > > Tobias > > > > > > > > > > > > > > > > On Monday, March 31, 2014 11:22:27 PM UTC+2, Koen Van Daele > > > > > > > > wrote: > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > I just wanted to get back at what Dennis said at the > > > > > > > > > beginning of this thread. Im quite curious how you will get > > > > > > > > > people to agree on (un)certainty. If feels like a very > > > > > > > > > natural idea to talk and think about, but I haven't really > > > > > > > > > seen it function properly in practice. > > > > > > > > > > > > > > > > > > We once did an experiment where we had 10 people who were > > > > > > > > > used to entering data in our archaeological inventory system > > > > > > > > > enter the same site. We paired the archaeologists: one more > > > > > > > > > more experienced data entry person (a few years experience) > > > > > > > > > and one newbie (a few months), so they would be forced to > > > > > > > > > really think things through and discuss. In our database we > > > > > > > > > have a field for certain the data entry person is about the > > > > > > > > > location of the site, ie. about the polygon they might have > > > > > > > > > drawn on a map. This field only allowed 5 choices, ranging > > > > > > > > > from 1 (I'm sure it's exactly where it needs to be) to 5 (I > > > > > > > > > have no idea whatsoever where the site is). We had a very > > > > > > > > > detailed manual with examples of all these cases, what to use > > > > > > > > > when, ... > > > > > > > > > Final result of our experiment: every group had entered the > > > > > > > > > location with a different level of certainty. So, based on > > > > > > > > > the exact same information they had all drawn totally > > > > > > > > > different conclusions. And this was about something as simple > > > > > > > > > as the location of the site. > > > > > > > > > > > > > > > > > > So, I'm very curious about how you manage to prevent stuff > > > > > > > > > like this from happening. > > > > > > > > > > > > > > > > > > The other thing I wonder about: how does certainty affect > > > > > > > > > searching? Should a search for 'churches' only return sites > > > > > > > > > that have a certain "certainty" attached to the > > > > > > > > > interpretation? Are you working with sliding scale of > > > > > > > > > certainty (ie. we are 75% percent certain about this > > > > > > > > > statement) or a binary one (we're certain or uncertain)? > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > Koen > > > > > > > > > ________________________________________ > > > > > > > > > Van: arches...@googlegroups.com [arches...@googlegroups.com] > > > > > > > > > namens dwut...@fargeo.com [dwut...@fargeo.com] > > > > > > > > > Verzonden: donderdag 27 maart 2014 22:36 > > > > > > > > > Aan: thomas....@gmail.com > > > > > > > > > CC: arches...@googlegroups.com > > > > > > > > > Onderwerp: Re: [Arches] "uncertain information" in Arches > > > > > > > > > > > > > > > > > > Thomas, > > > > > > > > > > > > > > > > > > Good question! You are quite correct that we haven’t tried > > > > > > > > > to include uncertainty in Arches. > > > > > > > > > > > > > > > > > > One reason is pretty basic: certainty is quite subjective > > > > > > > > > from person to person. For example: most people agree that > > > > > > > > > the earth is spherical. But a “flat-earther” may be very > > > > > > > > > certain that the earth is not a sphere, but is instead a > > > > > > > > > plane. His certainty does not make him correct, it merely > > > > > > > > > states the degree to which he believes in his interpretation. > > > > > > > > > Clearly, you can be very certain and very wrong at the same > > > > > > > > > time. I guess my point is that in many cases “certainty” > > > > > > > > > says more about the person making the assertion than it does > > > > > > > > > about the thing being described. > > > > > > > > > > > > > > > > > > OK, all philosophy aside, one could easily extend any Arches > > > > > > > > > graph to include a “certainty node”. Such a node could point > > > > > > > > > to a thesaurus (as many of the nodes in Arches already do), > > > > > > > > > allowing a user to select from a list of “uncertainty > > > > > > > > > levels”. Really, any Arches graph could include a “certainty > > > > > > > > > node” under any entity that you might want to qualify (for > > > > > > > > > example, one certainty node for period and another certainty > > > > > > > > > node for heritage type). > > > > > > > > > > > > > > > > > > Really, the hard part is not in getting Arches to allow you > > > > > > > > > to add an “uncertainty level” to your cultural heritage data. > > > > > > > > > Rather, the difficult thing is to decide how you’ll get > > > > > > > > > different people to agree on what constitutes certain vs. > > > > > > > > > uncertain interpretations of heritage. > > > > > > > > > > > > > > > > > > Sorry that I can’t be any more helpful… However, I’m very > > > > > > > > > interested to hear how you will model uncertainty and how you > > > > > > > > > will get people to implement it consistently. Please keep me > > > > > > > > > posted! > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > > > > > > > > Dennis > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mar 27, 2014, at 2:41, > > > > > > > > > thomas....@gmail.com<mailto:thomas....@gmail.com> wrote: > > > > > > > > > > > > > > > > > > I have a question about conceptual modeling in CIDOC CRM, > > > > > > > > > maybe there is someone one the list who is able to provide > > > > > > > > > some guidance. > > > > > > > > > > > > > > > > > > As posted before, we are trying to integrate research data of > > > > > > > > > neolithic sites into Arches. Now, naturally a significant > > > > > > > > > part of this data has a level of "certainty" to which the > > > > > > > > > information is correct. e.g. a site can consist of some > > > > > > > > > features for certain (in this case modeled in the > > > > > > > > > Archaeological Heritage (Site).E27 - Component.E18 > > > > > > > > > relationship) but if others exist is uncertain. We believe > > > > > > > > > this valuable information should not get lost (quite often > > > > > > > > > theory construction is based on such information). > > > > > > > > > > > > > > > > > > For example it could be uncertain if an archaeological > > > > > > > > > feature is to be named "pit" or "ditch" - or if it exists at > > > > > > > > > all. Another example could be the questionable relationship > > > > > > > > > of a findspot to a certain archaeological period. To make it > > > > > > > > > even more difficult, different authors could have different > > > > > > > > > thoughts on that. > > > > > > > > > > > > > > > > > > As far as we can see, the expression of such "uncertainty" is > > > > > > > > > not covered by Arches yet. Is there a concept for the > > > > > > > > > integration of such data in the future? We are currently > > > > > > > > > thinking into potential solutions but are struggeling to find > > > > > > > > > adequate expressions for uncertain information in CIDOC. > > > > > > > > > > > > > > > > > > thanks, Thomas > > > > > > > > > > > > > > > > > > -- > > > > > > > > > -- To post, send email to > > > > > > > > > arches...@googlegroups.com<mailto:arches...@googlegroups.com>. > > > > > > > > > To unsubscribe, send email to > > > > > > > > > archesprojec...@googlegroups.com<mailto:archesproject+unsubscr...@googlegroups.com>. > > > > > > > > > For more information, visit > > > > > > > > > https://groups.google.com/d/forum/archesproject?hl=en > > > > > > > > > --- You received this message because you are subscribed to > > > > > > > > > the Google Groups "Arches Project" group. > > > > > > > > > To unsubscribe from this group and stop receiving emails from > > > > > > > > > it, send an email to > > > > > > > > > archesprojec...@googlegroups.com<mailto:archesproject+unsubscr...@googlegroups.com>. > > > > > > > > > > > > > > > > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > -- To post, send email to arches...@googlegroups.com. To > > > > > > > > > unsubscribe, send email to archesprojec...@googlegroups.com. > > > > > > > > > For more information, visit > > > > > > > > > https://groups.google.com/d/forum/archesproject?hl=en > > > > > > > > > --- > > > > > > > > > You received this message because you are subscribed to the > > > > > > > > > Google Groups "Arches Project" group. > > > > > > > > > To unsubscribe from this group and stop receiving emails from > > > > > > > > > it, send an email to > > > > > > > > > archesprojec...@googlegroups.com<mailto:archesproject+unsubscr...@googlegroups.com>. > > > > > > > > > > > > > > > > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > > > > > > > -- > > > > > > > > -- To post, send email to arches...@googlegroups.com. To > > > > > > > > unsubscribe, send email to archesprojec...@googlegroups.com. > > > > > > > > For more information, visit > > > > > > > > https://groups.google.com/d/forum/archesproject?hl=en > > > > > > > > --- > > > > > > > > You received this message because you are subscribed to the > > > > > > > > Google Groups "Arches Project" group. > > > > > > > > To unsubscribe from this group and stop receiving emails from > > > > > > > > it, send an email to archesprojec...@googlegroups.com. > > > > > > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > > > > -- > > > > > > -- To post, send email to archesproject@googlegroups.com > > > > > > (mailto:archesproject@googlegroups.com). To unsubscribe, send email > > > > > > to archesproject+unsubscr...@googlegroups.com > > > > > > (mailto:archesproject+unsubscr...@googlegroups.com). For more > > > > > > information, visit > > > > > > https://groups.google.com/d/forum/archesproject?hl=en > > > > > > > > > > > > --- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "Arches Project" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > send an email to archesproject+unsubscr...@googlegroups.com > > > > > > (mailto:archesproject+unsubscr...@googlegroups.com). > > > > > > > > > > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > > > -- -- To post, send email to archesproject@googlegroups.com. To unsubscribe, send email to archesproject+unsubscr...@googlegroups.com. For more information, visit https://groups.google.com/d/forum/archesproject?hl=en --- You received this message because you are subscribed to the Google Groups "Arches Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to archesproject+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
