Re: [Archivesspace_Users_Group] Import Data Errors

[Mayo, Dave]

What Kate said, and also: for anything that we figured out during our 
migration, you can check individual EAD files with somewhat better error 
reporting by putting them through https://eadchecker.lib.harvard.edu/

--
Dave Mayo (he/him)
Senior Digital Library Software Engineer
Harvard University > HUIT > LTS

From:  on behalf of 
"Bowers, Kate A." 
Reply-To: Archivesspace Users Group 

Date: Tuesday, January 29, 2019 at 2:21 PM
To: Archivesspace Users Group 
Subject: Re: [Archivesspace_Users_Group] Import Data Errors

EAD has fewer contraints than ArchivesSpace.  Those error messages are not 
terribly helpful. At the very end of this article (before the end notes) is a 
list of the constraints in AS that will throw an error if your EAD does not 
comply:
https://journal.code4lib.org/articles/12239

I think the messages are coming from what we have as number 26 on the list 
“Absence of both unittitle and unitdate at a subordinate level causes import to 
fail”. Our solution was to grab data via a script from the parent . You 
may have few enough to do this by hand.  So, for example, we had s that 
contained only  or only  or only . These were all valid EAD 
but not acceptable to ArchivesSpace.

Below is a bit more digestible (but still Harvard-centric and definitely a 
working document) list of these issues and the practice that is needed going 
forward to ensure AS will ingest EAD.  Your error messages are coming from the 
3rd thing listed under .

Element category

Practice going forward

Reason

Reason category

Schema

Use the canonical EAD schema, not the Harvard modified schema

AS expects data that conforms to the canonical schema.

Schema



Do not use frontmatter

This will be added by EAD export from AS

Former practice no longer needed



 Always include

 Required for saving in AS, will ingest, but annoying to users who edit 
resources.





Do not use descgrp

This will not load to AS

Practice not supported by AS EAD ingest



Do not embed  within 

Roundtripped EAD from AS will be invalid

Practice not supported by AS EAD export



Always include an @level attribute value on all s. If using 
@level="otherlevel" always include an @otherlevel value.

 without a @level will ingest as "otherlevel" but lack @otherlevel value

New attribute data required



All components should have ; in cases where formerly archivsts might 
have used only , the parent 's unittitle is often a good choice

The component-centered display in ArchivesSpace makes any component lacking a 
the context provided by  text vague and cryptic, hampering 
recognition and interpretation of the component.

New content strongly recommended



All components must have either a  or a 

EAD lacking this data will not load to AS

New content required



 should stand alone, not be embedded within 

Will load twice into AS

Practice not supported by AS EAD ingest



 must include type and label attributes, cannot describe multiple 
containers in one container element, and should not include type of container 
as part of content.

AS accommodates container numbers and types, but does not accommodate note-like 
container information. In addition, AS creates "top container" records based on 
EAD ingest. These are linked records. Placing a range of boxes, for example, in 
a single container element creates incorrect data about containers.

Data model in AS is different from EAD



Do not encode  in 

Finding aid will load, but data will be lost. Use  or other 
appropriate element

Practice not supported by AS EAD ingest



@role ???

 Disappears on ingest





 statement should include ingest information

Include ingest to AS in your creation statement, e.g. "Created in Oxygen on 
2016-11-18; ingested to AS on 2016-12-12"

New content required



One Digital Object per Archival Object

Automated linking to objects in the DRS is based on the ref_id of the Archival 
Object, which is used as an owner supplied name in DRS.

New limit



Supply a title for digital archival objects; use  of parent 

xlink@title attribute is required by AS ingest

New attribute data required



???

 Disappears on ingest?





To achieve thumbnails,  must be coded thus: 

 ?





Collection-level  is required

EAD lacking this data will not load to AS

New content required



Do not encode mixed content within 

Finding aid will ingest, but content will be lost. Specifically, if a 
 has some child elements, any text that is not inside a child element 
will be left behind during ingest. An entirely plain-text  is OK.

New limit



 must begin with a number

EAD with non-numerical extent will not load to AS

New limit



 should be used more sparingly, consider using only if @href values 
link Harvard-managed links

Link rot (has nothing to do with AS), except that during migration, links 
became noticeable and rot was there

New recommendation



Do not encode nested indexes

Import may succeed, but data will be lacking from AS

Practice not supported by AS 

Re: [Archivesspace_Users_Group] Import Data Errors

[Bowers, Kate A.]

EAD has fewer contraints than ArchivesSpace.  Those error messages are not 
terribly helpful. At the very end of this article (before the end notes) is a 
list of the constraints in AS that will throw an error if your EAD does not 
comply:
https://journal.code4lib.org/articles/12239

I think the messages are coming from what we have as number 26 on the list 
“Absence of both unittitle and unitdate at a subordinate level causes import to 
fail”. Our solution was to grab data via a script from the parent . You 
may have few enough to do this by hand.  So, for example, we had s that 
contained only  or only  or only . These were all valid EAD 
but not acceptable to ArchivesSpace.

Below is a bit more digestible (but still Harvard-centric and definitely a 
working document) list of these issues and the practice that is needed going 
forward to ensure AS will ingest EAD.  Your error messages are coming from the 
3rd thing listed under .

Element category

Practice going forward

Reason

Reason category

Schema

Use the canonical EAD schema, not the Harvard modified schema

AS expects data that conforms to the canonical schema.

Schema



Do not use frontmatter

This will be added by EAD export from AS

Former practice no longer needed



 Always include

 Required for saving in AS, will ingest, but annoying to users who edit 
resources.





Do not use descgrp

This will not load to AS

Practice not supported by AS EAD ingest



Do not embed  within 

Roundtripped EAD from AS will be invalid

Practice not supported by AS EAD export



Always include an @level attribute value on all s. If using 
@level="otherlevel" always include an @otherlevel value.

 without a @level will ingest as "otherlevel" but lack @otherlevel value

New attribute data required



All components should have ; in cases where formerly archivsts might 
have used only , the parent 's unittitle is often a good choice

The component-centered display in ArchivesSpace makes any component lacking a 
the context provided by  text vague and cryptic, hampering 
recognition and interpretation of the component.

New content strongly recommended



All components must have either a  or a 

EAD lacking this data will not load to AS

New content required



 should stand alone, not be embedded within 

Will load twice into AS

Practice not supported by AS EAD ingest



 must include type and label attributes, cannot describe multiple 
containers in one container element, and should not include type of container 
as part of content.

AS accommodates container numbers and types, but does not accommodate note-like 
container information. In addition, AS creates "top container" records based on 
EAD ingest. These are linked records. Placing a range of boxes, for example, in 
a single container element creates incorrect data about containers.

Data model in AS is different from EAD



Do not encode  in 

Finding aid will load, but data will be lost. Use  or other 
appropriate element

Practice not supported by AS EAD ingest



@role ???

 Disappears on ingest





 statement should include ingest information

Include ingest to AS in your creation statement, e.g. "Created in Oxygen on 
2016-11-18; ingested to AS on 2016-12-12"

New content required



One Digital Object per Archival Object

Automated linking to objects in the DRS is based on the ref_id of the Archival 
Object, which is used as an owner supplied name in DRS.

New limit



Supply a title for digital archival objects; use  of parent 

xlink@title attribute is required by AS ingest

New attribute data required



???

 Disappears on ingest?





To achieve thumbnails,  must be coded thus: 

 ?





Collection-level  is required

EAD lacking this data will not load to AS

New content required



Do not encode mixed content within 

Finding aid will ingest, but content will be lost. Specifically, if a 
 has some child elements, any text that is not inside a child element 
will be left behind during ingest. An entirely plain-text  is OK.

New limit



 must begin with a number

EAD with non-numerical extent will not load to AS

New limit



 should be used more sparingly, consider using only if @href values 
link Harvard-managed links

Link rot (has nothing to do with AS), except that during migration, links 
became noticeable and rot was there

New recommendation



Do not encode nested indexes

Import may succeed, but data will be lacking from AS

Practice not supported by AS EAD ingest



Instead of creating es, add controlaccess terms to components

This allows search and retrieval of all components across the whole corpus 
rather than in one finding aid.

Better data model for discovery and retrieval



Do not encode nested indexentries

Import may succeed, but data will be lacking from AS

Practice not supported by AS EAD ingest



Do not encode nested lists

Import may succeed, but data will be lacking from AS

Practice not supported by AS EAD ingest



Do not use  or 

Import may 

Re: [Archivesspace_Users_Group] Resource records not showing up

[Chelsee Boehm]

Hi everyone,

I don't have access to the server to do a full re-index, but I was able to 
access those troubled resource records through the Background Jobs page. Making 
edits does make them appear in the search.

Thank you to everyone who helped me on this! I appreciate it very much!

Chelsee

From: archivesspace_users_group-boun...@lyralists.lyrasis.org 
[mailto:archivesspace_users_group-boun...@lyralists.lyrasis.org] On Behalf Of 
Blake Carver
Sent: Sunday, January 27, 2019 2:35 PM
To: Archivesspace Users Group
Subject: Re: [Archivesspace_Users_Group] Resource records not showing up


>>  but you might also be able to get some of those newly-ingested finding aids 
>> to be indexed by opening them in the staff interface and re-saving them one 
>> at a time.



Oh yeah, that too! Which is good if there's only a few. One other thing that 
reminded me of, if you update all the mtimes on your records that kinda does 
the same thing. I've had to do it a few times that way for various reasons I 
made a copy and paste gist here that I use:



https://gist.github.com/Blake-/538c8d7cc7ade39efc372a3e3e190873



This is one of those "nothing else seems to work" things though. Or maybe "I 
only have access to MySQL" things.


From: archivesspace_users_group-boun...@lyralists.lyrasis.org 
 on behalf of Custer, 
Mark 
Sent: Sunday, January 27, 2019 2:20:43 PM
To: Archivesspace Users Group
Subject: Re: [Archivesspace_Users_Group] Resource records not showing up


Chelsee,



In case you don't have immediate access to the server to force a full re-index, 
check out the Background Job pages (or Import Job pages, depending on which 
version of ASpace you're using).  Even though the job can have a status of 
"Completed," it's possible that no records were ingested.  If records were 
ingested, though, then you should be able to click on any of the links directly 
on the background job page, under the New and Modified Records heading (e.g. 
http://test.archivesspace.org/staff/jobs/1992, log in with user=admin and 
password=admin, and scroll to the bottom of that page).  After doing that, try 
editing the top-level Resource record and re-saving it (just a meaningless edit 
will do, like adding and then removing a space to the Resource's title).  I 
don't know why, but we have at least one finding aid that generally requires us 
to edit it like this before it gets re-indexed.



I think you'll need to follow Blake's advice to re-index everything, but you 
might also be able to get some of those newly-ingested finding aids to be 
indexed by opening them in the staff interface and re-saving them one at a time.



I hope that helps,



Mark





From: archivesspace_users_group-boun...@lyralists.lyrasis.org 
[mailto:archivesspace_users_group-boun...@lyralists.lyrasis.org] On Behalf Of 
Blake Carver
Sent: Sunday, 27 January, 2019 8:17 AM
To: archivesspace_users_group@lyralists.lyrasis.org
Subject: Re: [Archivesspace_Users_Group] Resource records not showing up



Most likely a problem with your index.

I would force a full reindex and see if that fixes things.

Stop ArchivesSpace

Delete everything in the /data/ directory. (yes, everything. no, you don't need 
anything inside there. yes, you need the actual directory.)

Start ArchivesSpace

Wait until it finishes up indexing and see if the records are back. That takes 
somewhere between 3 minutes and 3 days depending on how many records you have 
and how powerful of a server you're running on there.



If they're not back, look at the file in /logs/ archivesspace.out and see if 
you can find any lines with ERROR or also FATAL, those lines should provide 
clues. This location assumes you're running on Linux, logs seem to end up 
different places on Windows sometimes.



From: 
archivesspace_users_group-boun...@lyralists.lyrasis.org
 
mailto:archivesspace_users_group-boun...@lyralists.lyrasis.org>>
 on behalf of Chelsee Boehm 
mailto:chelsee.bo...@ishs.idaho.gov>>
Sent: Saturday, January 26, 2019 5:27:50 PM
To: 
archivesspace_users_group@lyralists.lyrasis.org
Subject: [Archivesspace_Users_Group] Resource records not showing up



Hi all,



I searched the List's Archives to see if I could find an answer to this 
question, and I didn't see anything, so I hope I am not asking pretty obvious 
or repeat question.



I am currently working on linking related Accession and Resource records before 
we launch our public site.



I was searching for a few specific Resource records, and could not find any of 
them. I checked back on my import jobs, and I can see where I successfully 
imported those records, but when I search for them they've done some sort of 
magic disappearing act!



I managed to find one of the Resource records from the troubled batch, as it 
had earlier been linked to it to an Accession 

Re: [Archivesspace_Users_Group] Import Data Errors

[Margaret Kidd]

It is probably an empty date tag in the components list. Check to see if it
something like this:
 My stuff   

This is technically an empty tag since there isn't a date after
"inclusive." AS does not like empty tags. So you will either need to add
title, date, etc. or delete the empty tags. If you have a lot of finding
aids you can do batch validation to see which ones need to be fixed (see
the blog post Validation Scenarios
by
Yale for help). I did this with my finding aids and then manually fixed the
problems I found. If I'd known how to write a script to do it for me I
would have, but it wasn't so many that I couldn't do it manually. Others
might have some suggestions on automating the process.

Hope this helps!

--

Margaret Turman Kidd

Access and Electronic Records Archivist, Special Collections & Archives

VCU Libraries | Tompkins-McCaw Library for the Health Sciences

Virginia Commonwealth University

509 N. 12th Street / Box 980582, Richmond, VA 23298-0582

(804) 828-3152

ki...@vcu.edu

Pronouns: she/her



On Tue, Jan 29, 2019 at 12:16 PM Ryan Flahive  wrote:

> Morning Folks,
>
>
>
> I’m new to this group as my ArchiveSpace server was just set up a couple
> days ago.
>
>
>
> Due to circumstances too lengthy to describe here, I am manually building
> this database from exported EAD files from my Archivist’s Toolkit system.
> My first few imports were successful, but then a few failed due to these
> errors:
>
>
>
> Date: one or more required (or enter a Title)
>
> Title: must not be an empty string (or enter a Date)
>
>
>
> These records have titles and dates. Can anyone shed some light on how I
> resolve this issue? Feel free to email me with suggestions!
>
>
>
> Thanks!
>
>
>
> *Ryan S. Flahive*
>
> Archivist
>
> *INSTITUTE OF AMERICAN INDIAN ARTS*
>
> 83 Avan Nu Po Road, Santa Fe, NM 87508
>
> P. 505-424-2392
>
> E. *rflah...@iaia.edu *
>
> *www.iaia.edu *
>
>
>
> IAIA's Mission: To empower creativity and leadership in Native arts and
> cultures through higher education, lifelong learning, and outreach.
>
>
> ___
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group@lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>
___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

Re: [Archivesspace_Users_Group] Import Data Errors

[Majewski, Steven Dennis (sdm7g)]

I believe all of the dsc components must have either a unittitle or unitdate. 


> On Jan 29, 2019, at 12:16 PM, Ryan Flahive  wrote:
> 
> Morning Folks,
>  
> I’m new to this group as my ArchiveSpace server was just set up a couple days 
> ago.
>  
> Due to circumstances too lengthy to describe here, I am manually building 
> this database from exported EAD files from my Archivist’s Toolkit system. My 
> first few imports were successful, but then a few failed due to these errors:
>  
> Date: one or more required (or enter a Title)
> Title: must not be an empty string (or enter a Date)
>  
> These records have titles and dates. Can anyone shed some light on how I 
> resolve this issue? Feel free to email me with suggestions!
>  
> Thanks!
>  
> Ryan S. Flahive
> Archivist
> INSTITUTE OF AMERICAN INDIAN ARTS
> 83 Avan Nu Po Road, Santa Fe, NM 87508
> P. 505-424-2392
> E. rflah...@iaia.edu 
> www.iaia.edu  
>  
> IAIA's Mission: To empower creativity and leadership in Native arts and 
> cultures through higher education, lifelong learning, and outreach.
>  
> ___
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group@lyralists.lyrasis.org 
> 
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group 
> 


smime.p7s
Description: S/MIME cryptographic signature
___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

[Archivesspace_Users_Group] Import Data Errors

[Ryan Flahive]

Morning Folks,

I’m new to this group as my ArchiveSpace server was just set up a couple days 
ago.

Due to circumstances too lengthy to describe here, I am manually building this 
database from exported EAD files from my Archivist’s Toolkit system. My first 
few imports were successful, but then a few failed due to these errors:

Date: one or more required (or enter a Title)
Title: must not be an empty string (or enter a Date)

These records have titles and dates. Can anyone shed some light on how I 
resolve this issue? Feel free to email me with suggestions!

Thanks!

Ryan S. Flahive
Archivist
INSTITUTE OF AMERICAN INDIAN ARTS
83 Avan Nu Po Road, Santa Fe, NM 87508
P. 505-424-2392
E. rflah...@iaia.edu
www.iaia.edu

IAIA's Mission: To empower creativity and leadership in Native arts and 
cultures through higher education, lifelong learning, and outreach.

___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group