[dspace-tech] Re: [Dspace-tech] Bulk METS ingest
Hi Pedro,
I will attach two files - the whole program and the pom.xml. The program
runs standalone on the server, for every item reads a directory, zips it
and creates the item. The program is as I needed it in my case. I think you
need a Java developer to make it work for you.
Best regards
Evgeni
On Tuesday, May 31, 2016 at 2:58:05 PM UTC+3, Pedro Amorim wrote:
>
> Hello Evgeni,
>
> Yes that is definetly an option for me, even being an alternative solution.
> I was already planning on developing a script in shell to package folders
> in zips and run dspace/bin packager for each of them.
>
> Would you please elaborate on:
>
> // get parent object
> parentObj = HandleManager
> .resolveToObject(context, itemParentHandle);
>
> // create a single item
> itemObj = metsIngester.ingest(
> context,
> parentObj,
> zippedItemPath.toFile(),
> pkgParams,
> null
> );
>
> I'm not the most experienced Java developer and wouldn't know how to
> implement custom code in DSpace.
>
> Thank you so much for replying.
>
> Cheers,
>
> Pedro Amorim
>
>
--
You received this message because you are subscribed to the Google Groups
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.
// DSPItemCreator.java
//
package bg.nalis.dsptools;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.PrintWriter;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.TreeMap;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
import org.dspace.content.DSpaceObject;
import org.dspace.content.packager.PackageIngester;
import org.dspace.content.packager.PackageParameters;
import org.dspace.core.ConfigurationManager;
import org.dspace.core.Context;
import org.dspace.core.PluginManager;
import org.dspace.eperson.EPerson;
import org.dspace.handle.HandleManager;
import org.dspace.servicemanager.DSpaceKernelInit;
/**
* The class creates items.
*/
public class DSPItemCreator {
// logWriter
private BufferedWriter logWriter;
// - Final Fields
private static final String ADMIN_PERSON_EMAIL = "ad...@admin.xx";
// directory
private static final String ITEMS_DIR = "/stagingarea/dtlexports/mets/";
// fileNames
private static final String ITEMS_MAP = "items_map.txt";
private static final String COLLECTIONS_MAP = "collections_map.txt";
private static final String ITEM_ZIPPED = "item.zip";
private static final String ERRLOG = "errlog.txt";
private static final Charset FILEENCODING = StandardCharsets.UTF_8;
// - Fields
private Map itemsMap;
private Map collectionsMap;
private List itemsList;
// - Methods
/**
*
*/
public static void main(String[] args) {
new DSPItemCreator().go();
}
/**
*
*/
private void go() {
// try to create logWriter and exit on failure
try {
Path logPath = Paths.get(ERRLOG);
logWriter = Files.newBufferedWriter(logPath, FILEENCODING);
}
catch (IOException e) {
return;
}
try {
createItem();
}
catch (Throwable t) {
t.printStackTrace();
logTrace(t);
}
try {
logWriter.close();
}
catch (Exception ex) {}
}
/**
*
*/
private void createItem() {
// context
Context context = null;
// parameters
PackageParameters pkgParams = new PackageParameters();
pkgParams.setWorkflowEnabled(false);
// ingester
PackageIngester metsIngester = null;
try {
// read files
itemsMap = readFileToMap(ITEMS_MAP, FILEENCODING);
collectionsMap = readFileToMap(COLLECTIONS_MAP, FILEENCODING);
// read directory
Path dirPath = Paths.get(ITEMS_DIR);
File dir = dirPath.toFile();
String[] subDirs = dir.list();
itemsList = Arrays.asList(subDirs);
Collections.sort(itemsList);
//--- start services ---
( DSpaceKernelInit
.getKernel(null) )
.start(ConfigurationManager.getProperty("dspace.dir"));
// get ingester
metsIngester = (PackageIngester) PluginManager
.getN
[dspace-tech] Re: [Dspace-tech] Bulk METS ingest
Hello Evgeni, Yes that is definetly an option for me, even being an alternative solution. I was already planning on developing a script in shell to package folders in zips and run dspace/bin packager for each of them. Would you please elaborate on: // get parent object parentObj = HandleManager .resolveToObject(context, itemParentHandle); // create a single item itemObj = metsIngester.ingest( context, parentObj, zippedItemPath.toFile(), pkgParams, null ); I'm not the most experienced Java developer and wouldn't know how to implement custom code in DSpace. Thank you so much for replying. Cheers, Pedro Amorim -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
[dspace-tech] Re: [Dspace-tech] Bulk METS ingest
Hi Pedro, Recently I migrated thousands of objects to DSpace, but I made a separate zip file for every one. The zip file contains the files (bitstreams) and the mets file. The zip files are ingested programmatically: // get parent object parentObj = HandleManager .resolveToObject(context, itemParentHandle); // create a single item itemObj = metsIngester.ingest( context, parentObj, zippedItemPath.toFile(), pkgParams, null ); I don't know is this an option for you? About your first email: When I look at the mets.xml file of an item with Edit this Item / Item Bitstreams / View - I see that it is exactly as ingested. Perhaps in your case something got distorted during the export. In my case the struct map in the mets.xml describes the structure of the book. Somethig like . . . Best regards Evgeni On Monday, May 30, 2016 at 6:09:57 PM UTC+3, Pedro Amorim wrote: > > Hello Evgeni, >> > > I am also having the same problem. > I'll share the post I made in dspace-community for introduction: > > https://groups.google.com/forum/#!searchin/dspace-community/Questions$20regarding$20METS$2FAIP$20ingestion$20in$20DSpace/dspace-community/1OrWeDfnD20/ECZWJv9oBgAJ > > 1) > If I import the collection using -t METS, the following error message > happens: > java.lang.ClassCastException: org.dspace.content.Community cannot be cast > to org.dspace.content.Collection > > 2) > If I import the collection using -t AIP (after changing the PROFILE, etc), > the following happens: > "METS document is missing the required structMap[@LABEL='Parent'] element." > > NOTE: I did not export from an existing DSpace. I'm importing METS items > for the first time in DSpace. > What I *think* is happening is, for the -t METS, DSpace won't recognize > the mets package as a collection, instead it will interpret it as an Item, > as such, when interpreting the parent in the structMap of the collection > being imported (a community), it'll interpret it as a collection, instead > of a community. Hence the message. > After that I tried with -t AIP and have the message in 2). What I *think* > happens in this case is it's complaining because the structMap in the ITEMS > (not the COLLECTION package being imported) is non-existent. But why would > it exist? I'm importing the COLLECTION package that already defines each > ITEM package contained in it. Additionally, even if I wanted to inject some > structMap for each ITEM (just to bypass this) how would I? I don't know the > handle of the parent collection because I'm submitting it for the first > time. > > I hope this wasn't too confusing. > Any help would be greatly appreciated. > > Thank you, > > Pedro Amorim > > > > > > > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
[dspace-tech] Re: [Dspace-tech] Bulk METS ingest
segunda-feira, 30 de Maio de 2016 às 15:09:57 UTC, Pedro Amorim escreveu: > > Hello Evgeni, >> > > I am also having the same problem. > I'll share the post I made in dspace-community for introduction: > > https://groups.google.com/forum/#!searchin/dspace-community/Questions$20regarding$20METS$2FAIP$20ingestion$20in$20DSpace/dspace-community/1OrWeDfnD20/ECZWJv9oBgAJ > > 1) > If I import the collection using -t METS, the following error message > happens: > java.lang.ClassCastException: org.dspace.content.Community cannot be cast > to org.dspace.content.Collection > > 2) > If I import the collection using -t AIP (after changing the PROFILE, etc), > the following happens: > "METS document is missing the required structMap[@LABEL='Parent'] element." > > NOTE: I did not export from an existing DSpace. I'm importing METS items > for the first time in DSpace. > What I *think* is happening is, for the -t METS, DSpace won't recognize > the mets package as a collection, instead it will interpret it as an Item, > as such, when interpreting the parent in the structMap of the collection > being imported (a community), it'll interpret it as a collection, instead > of a community. Hence the message. > After that I tried with -t AIP and have the message in 2). What I *think* > happens in this case is it's complaining because the structMap in the ITEMS > (not the COLLECTION package being imported) is non-existent. But why would > it exist? I'm importing the COLLECTION package that already defines each > ITEM package contained in it. Additionally, even if I wanted to inject some > structMap for each ITEM (just to bypass this) how would I? I don't know the > handle of the parent collection because I'm submitting it for the first > time. > > I hope this wasn't too confusing. > Any help would be greatly appreciated. > > Thank you, > > Pedro Amorim > > > > > > > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
[dspace-tech] Re: [Dspace-tech] Bulk METS ingest
> > Hello Evgeni, > I am also having the same problem. I'll share the post I made in dspace-community for introduction: https://groups.google.com/forum/#!searchin/dspace-community/Questions$20regarding$20METS$2FAIP$20ingestion$20in$20DSpace/dspace-community/1OrWeDfnD20/ECZWJv9oBgAJ 1) If I import the collection using -t METS, the following error message happens: java.lang.ClassCastException: org.dspace.content.Community cannot be cast to org.dspace.content.Collection 2) If I import the collection using -t AIP (after changing the PROFILE, etc), the following happens: "METS document is missing the required structMap[@LABEL='Parent'] element." NOTE: I did not export from an existing DSpace. I'm importing METS items for the first time in DSpace. What I *think* is happening is, for the -t METS, DSpace won't recognize the mets package as a collection, instead it will interpret it as an Item, as such, when interpreting the parent in the structMap of the collection being imported (a community), it'll interpret it as a collection, instead of a community. Hence the message. After that I tried with -t AIP and have the message in 2). What I *think* happens in this case is it's complaining because the structMap in the ITEMS (not the COLLECTION package being imported) is non-existent. But why would it exist? I'm importing the COLLECTION package that already defines each ITEM package contained in it. Additionally, even if I wanted to inject some structMap for each ITEM (just to bypass this) how would I? I don't know the handle of the parent collection because I'm submitting it for the first time. I hope this wasn't too confusing. Any help would be greatly appreciated. Thank you, Pedro Amorim -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech+unsubscr...@googlegroups.com. To post to this group, send email to dspace-tech@googlegroups.com. Visit this group at https://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
