Re: [Dspace-tech] Requesting Troubleshooting Suggestions - AIP Export Errors

[Tim Donohue]

Patrick,

Sounds good.  Let us know what you find out.  If you notice any other 
oddity/issue, we'll definitely want to get it resolved for 1.8.0.

Also, if you find DS-930 is the cause, you could also just "patch" your 
local code with the patch that is attached to DS-930.  It's literally a 
one-line fix to the RoleCrosswalk.java class.  So, it's up to you if 
you'd rather "turn off" the RoleCrosswalk (as described in the previous 
email) or just apply the patch to fix it so that it properly closes its 
database connections.

Good luck,

- Tim

On Monday, August 22, 2011 2:37:13 PM, Patrick Etienne wrote:
> Tim,
> 
> Thanks so much! This is wonderful information. I really appreciate your 
> taking the time to write all this up. I'm going to be looking into the 
> things you mentioned here and do a little exploring to see if I can 
> eliminate the problem or at least use this information to narrow down 
> even further what is really going on. I likely won't email back about i
t 
> for perhaps two weeks (other things going on), but I'll certainly get 
> back to you with whatever information that I'm able that might help 
> toward the stability of the AIP tool.
> 
> Thanks again!
> 
>   - Patrick
> 
> On Mon, Aug 22, 2011 at 2:09 PM, Tim Donohue  > wrote:
> 
> Hi Patrick,
> 
> 
> On 8/19/2011 2:33 PM, Patrick Etienne wrote:
> 
> To start, I should toss out a little system information:
> DSpace v. 1.7.2
> Java version "1.6.0_26"
> PostgresSQL 8.4.7
> All on the same RedHat box.
> 
> 
> Thanks!  This all looks reasonable/normal.
> 
> 
> The next important detail is that I should have removed the
> "manifestOnly" option from the command as I've experienced the error
> with manifestOnly set to true as well as left out (defaulted to
> false).
> As a side note, this use of manifestOnly was more of an aftertho
ught
> (I'd noted it's experimental nature), and I'm not certain that it
> actually does quite what I was looking for. My purpose is setting up
> instances with data so that I can fully test themes I'm building for
> various institutions, but I'm not really needing "content files"
> (bitstreams), just the communities, sub-communities, collections,
> item-pages, and /references/ to content files (having files or not
> doesn't really matter as much, but it'd be preferable to leave
> out the
> asset store). From the description in your email, it does sound
> indeed
> as though setting the manifestOnly option to "true" would be
> good for my
> use-case /as long as/ the feature was stable (which, as it has been
> said, it's not as of yet). But again, the manifestOnly option is
> not a
> priority (for me) at this point.
> 
> 
> OK, good to know. Yes, t
he manifestOnly option will not give you
> bitstreams (as you expected). It just includes metadata & structure
> (communities/collections etc). However, as mentioned, it is still
> very experimental. I've admittedly never tried to migrate content
> using the manifestOnly option -- so, I'm not sure how stable the
> import will be.
> 
> But, as the import isn't the problem right now...we can leave that
> for later.
> 
> 
> It sounds as though the next piece of the puzzle might be the
> database
> connection. The postgres service that I'm using for the
> instance(s) is
> on the same machine as the dspace instance(s). I think that
> eliminates
> network troubles as a potential cause (not that there couldn't be
> something else going on with the postgres service).
> 
> Next up would be a little more detail concerning the behavior of the
> attempted command. The error does no
t happen immediately, it
> does run
> for a while before erroring out.
> 
> 
> 
> After this I ended up shutting down all the tomcat instances save
> for the large one and also increased the db.maxconnections to
> 400 (just
> to give it a whirl). This seemed to allow the AIP export to go even
> longer before erroring out.
> 
> 
> Hmm..I wonder.
> 
>  >From the reaction after increasing 'db.maxconnections', it sounds
> as though it could be that your DSpace is just running out of
> database connections (as though connections are not being closed
> properly).
> 
> It's possible you are encountering this bug (which will be fixed in
> 1.8.0):
> https://jira.duraspace.org/__browse/DS-930
> 
> 
> The AIP Export uses the RoleCrosswalk listed in DS-930 (to export
> user permissions/groups into the DSPACE-ROLES schema). We realized
>  
   there was a small bug in the RoleCrosswalk, where it wasn't closing
>

Re: [Dspace-tech] Requesting Troubleshooting Suggestions - AIP Export Errors

[Patrick Etienne]

Tim,

Thanks so much! This is wonderful information. I really appreciate your
taking the time to write all this up. I'm going to be looking into the
things you mentioned here and do a little exploring to see if I can
eliminate the problem or at least use this information to narrow down even
further what is really going on. I likely won't email back about it for
perhaps two weeks (other things going on), but I'll certainly get back to
you with whatever information that I'm able that might help toward the
stability of the AIP tool.

Thanks again!

 - Patrick

On Mon, Aug 22, 2011 at 2:09 PM, Tim Donohue  wrote:

> Hi Patrick,
>
>
> On 8/19/2011 2:33 PM, Patrick Etienne wrote:
>
>  To start, I should toss out a little system information:
>> DSpace v. 1.7.2
>> Java version "1.6.0_26"
>> PostgresSQL 8.4.7
>> All on the same RedHat box.
>>
>
> Thanks!  This all looks reasonable/normal.
>
>
>  The next important detail is that I should have removed the
>> "manifestOnly" option from the command as I've experienced the error
>> with manifestOnly set to true as well as left out (defaulted to false).
>> As a side note, this use of manifestOnly was more of an afterthought
>> (I'd noted it's experimental nature), and I'm not certain that it
>> actually does quite what I was looking for. My purpose is setting up
>> instances with data so that I can fully test themes I'm building for
>> various institutions, but I'm not really needing "content files"
>> (bitstreams), just the communities, sub-communities, collections,
>> item-pages, and /references/ to content files (having files or not
>> doesn't really matter as much, but it'd be preferable to leave out the
>> asset store). From the description in your email, it does sound indeed
>> as though setting the manifestOnly option to "true" would be good for my
>> use-case /as long as/ the feature was stable (which, as it has been
>> said, it's not as of yet). But again, the manifestOnly option is not a
>> priority (for me) at this point.
>>
>
> OK, good to know. Yes, the manifestOnly option will not give you bitstreams
> (as you expected). It just includes metadata & structure
> (communities/collections etc). However, as mentioned, it is still very
> experimental. I've admittedly never tried to migrate content using the
> manifestOnly option -- so, I'm not sure how stable the import will be.
>
> But, as the import isn't the problem right now...we can leave that for
> later.
>
>
>  It sounds as though the next piece of the puzzle might be the database
>> connection. The postgres service that I'm using for the instance(s) is
>> on the same machine as the dspace instance(s). I think that eliminates
>> network troubles as a potential cause (not that there couldn't be
>> something else going on with the postgres service).
>>
>> Next up would be a little more detail concerning the behavior of the
>> attempted command. The error does not happen immediately, it does run
>> for a while before erroring out.
>>
> 
>
>  After this I ended up shutting down all the tomcat instances save
>> for the large one and also increased the db.maxconnections to 400 (just
>> to give it a whirl). This seemed to allow the AIP export to go even
>> longer before erroring out.
>>
>
> Hmm..I wonder.
>
> From the reaction after increasing 'db.maxconnections', it sounds as though
> it could be that your DSpace is just running out of database connections (as
> though connections are not being closed properly).
>
> It's possible you are encountering this bug (which will be fixed in 1.8.0):
> https://jira.duraspace.org/**browse/DS-930
>
> The AIP Export uses the RoleCrosswalk listed in DS-930 (to export user
> permissions/groups into the DSPACE-ROLES schema). We realized there was a
> small bug in the RoleCrosswalk, where it wasn't closing its DB Connections
> properly.
>
> So, if you don't care whether you are moving user permissions/groups, you
> can turn "off" usage of the RoleCrosswalk by changing the following in your
> dspace.cfg.
>
> First, you'd want to turn off the "DSPACE-ROLES" schema export, by removing
> it from this line (also remove the comma)
>
> aip.disseminate.techMD = PREMIS, DSPACE-ROLES
>
> Second, you'd want to turn off the "METSRIGHTS" schema export, by removing
> it from this line (also remove the last comma)
>
> aip.disseminate.rightsMD = DSpaceDepositLicense:DSPACE_**DEPLICENSE, \
>CreativeCommonsRDF:DSPACE_**CCRDF, CreativeCommonsText:DSPACE_**CCTEXT,
> METSRIGHTS
>
> What this does is disables the export of users/groups (DSPACE-ROLES) and
> permissions (METSRIGHTS) in AIPs. WARNING: in most backup scenarios, you
> would NOT want to do this, as you cannot restore users/groups/permissions if
> they are not in the AIPs. However, for your purposes of moving around test
> content, you may not care whether users/groups/permissions are moved
> successfully.
>
> More info about DSPACE-ROLES and METSRIGHTS schemas can be found at:
> https://wiki.d

Re: [Dspace-tech] Requesting Troubleshooting Suggestions - AIP Export Errors

[Tim Donohue]

Hi Patrick,

On 8/19/2011 2:33 PM, Patrick Etienne wrote:

> To start, I should toss out a little system information:
> DSpace v. 1.7.2
> Java version "1.6.0_26"
> PostgresSQL 8.4.7
> All on the same RedHat box.

Thanks!  This all looks reasonable/normal.

> The next important detail is that I should have removed the
> "manifestOnly" option from the command as I've experienced the error
> with manifestOnly set to true as well as left out (defaulted to false).
> As a side note, this use of manifestOnly was more of an afterthought
> (I'd noted it's experimental nature), and I'm not certain that it
> actually does quite what I was looking for. My purpose is setting up
> instances with data so that I can fully test themes I'm building for
> various institutions, but I'm not really needing "content files"
> (bitstreams), just the communities, sub-communities, collections,
> item-pages, and /references/ to content files (having files or not
> doesn't really matter as much, but it'd be preferable to leave out the
> asset store). From the description in your email, it does sound indeed
> as though setting the manifestOnly option to "true" would be good for my
> use-case /as long as/ the feature was stable (which, as it has been
> said, it's not as of yet). But again, the manifestOnly option is not a
> priority (for me) at this point.

OK, good to know. Yes, the manifestOnly option will not give you 
bitstreams (as you expected). It just includes metadata & structure 
(communities/collections etc). However, as mentioned, it is still very 
experimental. I've admittedly never tried to migrate content using the 
manifestOnly option -- so, I'm not sure how stable the import will be.

But, as the import isn't the problem right now...we can leave that for 
later.

> It sounds as though the next piece of the puzzle might be the database
> connection. The postgres service that I'm using for the instance(s) is
> on the same machine as the dspace instance(s). I think that eliminates
> network troubles as a potential cause (not that there couldn't be
> something else going on with the postgres service).
>
> Next up would be a little more detail concerning the behavior of the
> attempted command. The error does not happen immediately, it does run
> for a while before erroring out.

> After this I ended up shutting down all the tomcat instances save
> for the large one and also increased the db.maxconnections to 400 (just
> to give it a whirl). This seemed to allow the AIP export to go even
> longer before erroring out.

Hmm..I wonder.

 From the reaction after increasing 'db.maxconnections', it sounds as 
though it could be that your DSpace is just running out of database 
connections (as though connections are not being closed properly).

It's possible you are encountering this bug (which will be fixed in 1.8.0):
https://jira.duraspace.org/browse/DS-930

The AIP Export uses the RoleCrosswalk listed in DS-930 (to export user 
permissions/groups into the DSPACE-ROLES schema). We realized there was 
a small bug in the RoleCrosswalk, where it wasn't closing its DB 
Connections properly.

So, if you don't care whether you are moving user permissions/groups, 
you can turn "off" usage of the RoleCrosswalk by changing the following 
in your dspace.cfg.

First, you'd want to turn off the "DSPACE-ROLES" schema export, by 
removing it from this line (also remove the comma)

aip.disseminate.techMD = PREMIS, DSPACE-ROLES

Second, you'd want to turn off the "METSRIGHTS" schema export, by 
removing it from this line (also remove the last comma)

aip.disseminate.rightsMD = DSpaceDepositLicense:DSPACE_DEPLICENSE, \
 CreativeCommonsRDF:DSPACE_CCRDF, CreativeCommonsText:DSPACE_CCTEXT, 
METSRIGHTS

What this does is disables the export of users/groups (DSPACE-ROLES) and 
permissions (METSRIGHTS) in AIPs. WARNING: in most backup scenarios, you 
would NOT want to do this, as you cannot restore 
users/groups/permissions if they are not in the AIPs. However, for your 
purposes of moving around test content, you may not care whether 
users/groups/permissions are moved successfully.

More info about DSPACE-ROLES and METSRIGHTS schemas can be found at:
https://wiki.duraspace.org/display/DSDOC/DSpace+AIP+Format

If you are encountering the DS-930 bug, disabling both "DSPACE-ROLES" 
and "METSRIGHTS" may actually allow the export to succeed.  But, as 
stated, you won't have any user/groups/permissions info in your AIPs. 
So, when you re-import those AIPs, everything will just end up with 
generic default permissions (i.e. publicly accessible, but a System 
Administrator has rights to edit everything, etc)


> I discovered while tweaking with the database settings was that running
> the AIP export while tailing the log file showed that the
> org.postgresql.util.PSQLException popped up at least a few times for
> each AIP export I'd attempted to do.

Is it that same PSQLException each time, with the same error 
message/description?  Does it pop up for ev

Re: [Dspace-tech] Requesting Troubleshooting Suggestions - AIP Export Errors

[Patrick Etienne]

Tim -

Many thanks for your help! I'm going to try to provide some additional
information to see about teasing out the cause of this particular problem.

To start, I should toss out a little system information:
DSpace v. 1.7.2
Java version "1.6.0_26"
PostgresSQL 8.4.7
All on the same RedHat box.

The next important detail is that I should have removed the "manifestOnly"
option from the command as I've experienced the error with manifestOnly set
to true as well as left out (defaulted to false). As a side note, this use
of manifestOnly was more of an afterthought (I'd noted it's experimental
nature), and I'm not certain that it actually does quite what I was looking
for. My purpose is setting up instances with data so that I can fully test
themes I'm building for various institutions, but I'm not really needing
"content files" (bitstreams), just the communities, sub-communities,
collections, item-pages, and *references* to content files (having files or
not doesn't really matter as much, but it'd be preferable to leave out the
asset store). From the description in your email, it does sound indeed as
though setting the manifestOnly option to "true" would be good for my
use-case *as long as* the feature was stable (which, as it has been said,
it's not as of yet). But again, the manifestOnly option is not a priority
(for me) at this point.

It sounds as though the next piece of the puzzle might be the database
connection. The postgres service that I'm using for the instance(s) is on
the same machine as the dspace instance(s). I think that eliminates network
troubles as a potential cause (not that there couldn't be something else
going on with the postgres service).

Next up would be a little more detail concerning the behavior of the
attempted command. The error does not happen immediately, it does run for a
while before erroring out. I'm not sure about how to get exact statistics
for how much of the process is being completed, but I can give some info.
The instance I'm attempting to pull from has 28 communities and 82
sub-communities (all but a few have items). I'm not sure how many
collections but it should have between 6,000 to 8,000 items. I've run the
command numerous times and seem to semi-randomly get between 55mb and
125mb's worth of data. That ranges between 2, 6, or more communities
depending on the run (with collection and item numbers varying widely). In
attempting to do some more detailed troubleshooting I delved into a DEBUG
level of dspace logs. Unfortunately the DEBUG level only gave me one
additional line from which to investigate the issue:

2011-08-19 14:05:51,512 DEBUG net.sf.ehcache.CacheManager @ CacheManager
already shutdown

The rest of the log file didn't show anything relevant. From there I was
going to try to take a look at the postgres logs but 1) I don't have read
access to them, 2) They don't have "query level" information (not sure how
necessary that would be), and 3) My sysadmin seemed to think that it
wouldn't worth checking at this point. He did, however, suggest some other
things to check on which I have. We have several (around 10) tomcat
instances serving dspace installations on this server. Half have no data and
are never used. The other half have descent stores of info (communities,
collections, items, bitstreams) but are very infrequently accessed. One
instance is large and has more regular (but hard for me to define in volume)
usage. I ended up changing all the instances to have a db.maxidle value of 5
(rather than the default -1 [unlimited]). This seemed to enable the AIP
export process to run longer before it errored out. After this I ended up
shutting down all the tomcat instances save for the large one and also
increased the db.maxconnections to 400 (just to give it a whirl). This
seemed to allow the AIP export to go even longer before erroring out.
Another interesting (to me at least) thing is that I also tried changing the
db.maxidle down to a value of 1. The trial here lasted a long time, got a
lot of data, but ended due to exceeding the db.maxwait interval (I'd set it
to 1 milliseconds) for the idle thread. I'm not sure what all exactly
this would tell us, but at least some of it will be relevant/helpful info.
One other thing that I discovered while tweaking with the database settings
was that running the AIP export while tailing the log file showed that
the org.postgresql.util.PSQLException
popped up at least a few times for each AIP export I'd attempted to do.

"Is it a specific object in DSpace that causes the error?"
This is a great question. I'm not sure how to answer it though (would
probably need a "query level" of postgres logs?). I can say that for certain
this was the case for an earlier error (not one that related to postgres
issues) that we'd encountered with the AIP tool. The cause of the earlier
error was that we'd somehow managed to have a couple items in the instance
that did not have titles. After making sure that the items were removed or
gi

Re: [Dspace-tech] Requesting Troubleshooting Suggestions - AIP Export Errors

[Tim Donohue]

One last note Patrick...I just realized, your commands are slightly 
"wrong" (though I'm not sure if this is the cause or not).

With the "manifestOnly=true" option, DSpace will only export mets.xml 
files.  You look to be trying to create a ZIP package 
(harvest_test-aip.zip).  But, in reality the AIP Export is creating 
*XML* files only.   If you want to generate a Zip package, you then you 
need to remove the "manifestOnly=true" option.

I'm not sure that this is the cause of your issues though. But, I 
wanted to be clear that "manfiestOnly=true" only creates XML files 
(with just metadata, and no content files).

- Tim

On Thursday, August 18, 2011 10:59:11 AM, Tim Donohue wrote:
> Hi Patrick,
> 
> A few questions for you:
> * What version of DSpace are you using? 1.7.0, 1.7.1 or 1.7.2?
> * Just a generic question -- is there a reason why you want to run this 
> in "manifestOnly=true" mode? In the docs, it's worth noting that that 
> setting is marked as "experimental" an
d that it should "never be set to 
> true if you want to be able to restore content file" (In other words, 
> with this option set to "true", you are only exporting Metadata, and not 
> content files -- admittedly, this option is still very "experimental" in 
> nature) See: 
> https://wiki.duraspace.org/display/DSDOC/AIP+Backup+and+Restore#AIPBackupandRestore-AdditionalPackagerOptions
>  
> 
> 
> That all being said, a "socket closed" exception usually means that the 
> JDBC client somehow lost its connection to your Database for some 
> reason. It could happen for a variety of reasons. If the database is on 
> a separate server, it could be a network problem (though if it's a 
> consistent error, this may be less likely). T
> here could also be a bug in the code somewhere that you are hitting 
> (where somehow the connection was accidentally closed or dropped). 
> Others may know of other scenarios (these are just the ones I'm aware of 
> off the top of my head).
> 
> Does this
 error always happen immediately? or does the command run for a 
> while and then error out (i.e. are some/many AIPs generated successfully 
> before the error)? Is it a specific object in DSpace that causes the error?
> 
> I can also try to do more testing on my end to see if I can replicate it 
> by running a sitewide export with the "manifest=true" option set 
> (though, we have our DSpace 1.8.0 Feature Freeze tomorrow, so I may not 
> get back to this until Monday, to be honest).
> 
> - Tim
> 
> On Wednesday, August 17, 2011 3:45:23 PM, Patrick Etienne wrote:
>> DSpace Techies,
>>
>> I've been attempting to leverage the new AIP tool in order to export
>> data from one dspace instance into another for the purpose of testing
>> development instances (wit
> h which I'm building themes) and have come
>> across a java error that I can't seem to figure out how to resolve.
>> I'm not the sys admin of servers responsible for hosting the instances
>> but do have access to enough t
hat I should be able to do some
>> troubleshooting. The error is as follows:
>>
>> -
>>
>> [dspace_gkr@gkr-dev exports]$ ../gkrtest/bin/dspace packager -d -a -t
>> AIP -o manifestOnly=true -e patrick.etie...@library.gatech.edu -i
>> GKRTEST/0 harvest_test-aip.zip
>>
>> Disseminating DSpace SITE [ hdl=GKRTEST/0 ] to harvest_test-aip.zip
>>
>> Also disseminating all child objects (recursive mode)..
>> This may take a while, please check your logs for ongoing status while
>> we process each package.
>> Exception: An I/O error occured while sending to the backend.
>> org.postgresql.util.PSQLException: An I/O error occured while sending
>> to the backend.
>> at 
>> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:214) 
>>
>> at org.postgres
> ql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:452)
>> at 
>> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:354)
>>  
>>
>> at 
>> org.
postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:258)
 
>>
>> at 
>> org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
>>  
>>
>> at 
>> org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
>>  
>>
>> at 
>> org.dspace.storage.rdbms.DatabaseManager.queryTable(DatabaseManager.java:225)
>>  
>>
>> at 
>> org.dspace.authorize.AuthorizeManager.getPolicies(AuthorizeManager.java:511) 
>>
>> at 
>> org.dspace.content.crosswalk.METSRightsCrosswalk.disseminateElement(METSRightsCrosswalk.java:156)
>>  
>>
>> at 
>> org.dspace.content.packager.AbstractMETSDisseminator.crosswalkToMetsElement(AbstractMETSDisseminator.java:1342)
>>  
>>
>> at 
>> org.dspace.content.packager.AbstractMETSDisseminator.makeMdSec(AbstractMETSDisseminator.java:597)
>>  
>>
>> at org.ds
> pace.content.packager.AbstractMETSDisseminator.addToAmdSec(AbstractMETSDisseminator.java:710)
>  
> 
>> at 
>> org.dspace.c
ontent.pa

Re: [Dspace-tech] Requesting Troubleshooting Suggestions - AIP Export Errors

[Tim Donohue]

Hi Patrick,

A few questions for you:
* What version of DSpace are you using? 1.7.0, 1.7.1 or 1.7.2?
* Just a generic question -- is there a reason why you want to run this 
in "manifestOnly=true" mode?  In the docs, it's worth noting that that 
setting is marked as "experimental" and that it should "never be set to 
true if you want to be able to restore content file" (In other words, 
with this option set to "true", you are only exporting Metadata, and 
not content files -- admittedly, this option is still very 
"experimental" in nature) See: 
https://wiki.duraspace.org/display/DSDOC/AIP+Backup+and+Restore#AIPBackupandRestore-AdditionalPackagerOptions

That all being said, a "socket closed" exception usually means that the 
JDBC client somehow lost its connection to your Database for some 
reason.  It could happen for a variety of reasons. If the database is 
on a separate server, it could be a network problem (though if it's a 
consistent error, this may be less likely). T
here could also be a bug 
in the code somewhere that you are hitting (where somehow the 
connection was accidentally closed or dropped).  Others may know of 
other scenarios (these are just the ones I'm aware of off the top of my 
head).

Does this error always happen immediately? or does the command run for 
a while and then error out (i.e. are some/many AIPs generated 
successfully before the error)?  Is it a specific object in DSpace that 
causes the error?

I can also try to do more testing on my end to see if I can replicate 
it by running a sitewide export with the "manifest=true" option set 
(though, we have our DSpace 1.8.0 Feature Freeze tomorrow, so I may not 
get back to this until Monday, to be honest).

- Tim

On Wednesday, August 17, 2011 3:45:23 PM, Patrick Etienne wrote:
> DSpace Techies,
> 
> I've been attempting to leverage the new AIP tool in order to export
> data from one dspace instance into another for the purpose of testing
> development instances (wit
h which I'm building themes) and have come
> across a java error that I can't seem to figure out how to resolve.
> I'm not the sys admin of servers responsible for hosting the instances
> but do have access to enough that I should be able to do some
> troubleshooting. The error is as follows:
> 
> -
> 
> [dspace_gkr@gkr-dev exports]$ ../gkrtest/bin/dspace packager -d -a -t
> AIP -o manifestOnly=true -e patrick.etie...@library.gatech.edu -i
> GKRTEST/0 harvest_test-aip.zip
> 
> Disseminating DSpace SITE [ hdl=GKRTEST/0 ] to harvest_test-aip.zip
> 
> Also disseminating all child objects (recursive mode)..
> This may take a while, please check your logs for ongoing status while
> we process each package.
> Exception: An I/O error occured while sending to the backend.
> org.postgresql.util.PSQLException: An I/O error occured while sending
> to the backend.
>   at 
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:214)
>   at org.postgres
ql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:452)
>   at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:354)
>   at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:258)
>   at 
> org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
>   at 
> org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
>   at 
> org.dspace.storage.rdbms.DatabaseManager.queryTable(DatabaseManager.java:225)
>   at 
> org.dspace.authorize.AuthorizeManager.getPolicies(AuthorizeManager.java:511)
>   at 
> org.dspace.content.crosswalk.METSRightsCrosswalk.disseminateElement(METSRightsCrosswalk.java:156)
>   at 
> org.dspace.content.packager.AbstractMETSDisseminator.crosswalkToMetsElement(AbstractMETSDisseminator.java:1342)
>   at 
> org.dspace.content.packager.AbstractMETSDisseminator.makeMdSec(AbstractMETSDisseminator.java:597)
>   at org.ds
pace.content.packager.AbstractMETSDisseminator.addToAmdSec(AbstractMETSDisseminator.java:710)
>   at 
> org.dspace.content.packager.AbstractMETSDisseminator.addAmdSec(AbstractMETSDisseminator.java:737)
>   at 
> org.dspace.content.packager.AbstractMETSDisseminator.makeManifest(AbstractMETSDisseminator.java:889)
>   at 
> org.dspace.content.packager.AbstractMETSDisseminator.disseminate(AbstractMETSDisseminator.java:230)
>   at 
> org.dspace.content.packager.DSpaceAIPDisseminator.disseminate(DSpaceAIPDisseminator.java:157)
>   at 
> org.dspace.content.packager.AbstractPackageDisseminator.disseminateAll(AbstractPackageDisseminator.java:86)
>   at 
> org.dspace.content.packager.AbstractPackageDisseminator.disseminateAll(AbstractPackageDisseminator.java:120)
>   at 
> org.dspace.content.packager.AbstractPackageDisseminator.disseminateAll(AbstractPackageDisseminator.java:141)
>   at 
> org.dspace.content.p

[Dspace-tech] Requesting Troubleshooting Suggestions - AIP Export Errors

[Patrick Etienne]

DSpace Techies,

I've been attempting to leverage the new AIP tool in order to export
data from one dspace instance into another for the purpose of testing
development instances (with which I'm building themes) and have come
across a java error that I can't seem to figure out how to resolve.
I'm not the sys admin of servers responsible for hosting the instances
but do have access to enough that I should be able to do some
troubleshooting. The error is as follows:

-

[dspace_gkr@gkr-dev exports]$ ../gkrtest/bin/dspace packager -d -a -t
AIP -o manifestOnly=true -e patrick.etie...@library.gatech.edu -i
GKRTEST/0 harvest_test-aip.zip

Disseminating DSpace SITE [ hdl=GKRTEST/0 ] to harvest_test-aip.zip

Also disseminating all child objects (recursive mode)..
This may take a while, please check your logs for ongoing status while
we process each package.
Exception: An I/O error occured while sending to the backend.
org.postgresql.util.PSQLException: An I/O error occured while sending
to the backend.
at 
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:214)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:452)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:354)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:258)
at 
org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
at 
org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
at 
org.dspace.storage.rdbms.DatabaseManager.queryTable(DatabaseManager.java:225)
at 
org.dspace.authorize.AuthorizeManager.getPolicies(AuthorizeManager.java:511)
at 
org.dspace.content.crosswalk.METSRightsCrosswalk.disseminateElement(METSRightsCrosswalk.java:156)
at 
org.dspace.content.packager.AbstractMETSDisseminator.crosswalkToMetsElement(AbstractMETSDisseminator.java:1342)
at 
org.dspace.content.packager.AbstractMETSDisseminator.makeMdSec(AbstractMETSDisseminator.java:597)
at 
org.dspace.content.packager.AbstractMETSDisseminator.addToAmdSec(AbstractMETSDisseminator.java:710)
at 
org.dspace.content.packager.AbstractMETSDisseminator.addAmdSec(AbstractMETSDisseminator.java:737)
at 
org.dspace.content.packager.AbstractMETSDisseminator.makeManifest(AbstractMETSDisseminator.java:889)
at 
org.dspace.content.packager.AbstractMETSDisseminator.disseminate(AbstractMETSDisseminator.java:230)
at 
org.dspace.content.packager.DSpaceAIPDisseminator.disseminate(DSpaceAIPDisseminator.java:157)
at 
org.dspace.content.packager.AbstractPackageDisseminator.disseminateAll(AbstractPackageDisseminator.java:86)
at 
org.dspace.content.packager.AbstractPackageDisseminator.disseminateAll(AbstractPackageDisseminator.java:120)
at 
org.dspace.content.packager.AbstractPackageDisseminator.disseminateAll(AbstractPackageDisseminator.java:141)
at 
org.dspace.content.packager.AbstractPackageDisseminator.disseminateAll(AbstractPackageDisseminator.java:132)
at 
org.dspace.content.packager.AbstractPackageDisseminator.disseminateAll(AbstractPackageDisseminator.java:152)
at org.dspace.app.packager.Packager.disseminate(Packager.java:635)
at org.dspace.app.packager.Packager.main(Packager.java:460)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:183)
Caused by: java.net.SocketException: Socket closed
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:99)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at 
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at org.postgresql.core.PGStream.flush(PGStream.java:532)
at 
org.postgresql.core.v3.QueryExecutorImpl.sendSync(QueryExecutorImpl.java:672)
at 
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:187)
... 27 more

-

I know that our sys admin has been able to do successful exports of
single items, but so far we've been unable to do a full site export.
I'll need a lot of data in order to adequately test the themes I'm
working on. If anyone could point me in the right direction (or even
just make some suggestions) on how to resolve this issue, it would be
greatly appreciated!

 - Patrick E.

-- 
Patrick K. Etienne
Systems Analyst
Georgia Institute of Technology
Library & Infor