Hi everyone,
Thanks for the responses. I always appreciate the fact that post a
question/problem on the forum and the next morning there are stack of
replies.
+1 to everyone!
As it turns out the problem was entirely mine :-)
However, I've discovered how the file manager works out the percentage
file transferred and why it thinks the file Percentage Complete was 0%.
The magic all happens with the following piece of code:
private long getBytesTransferred(Reference r) {
File destFile = null;
try {
destFile = new File(new URI(r.getDataStoreReference()));
* long bytesTransferred = destFile.length();*
return bytesTransferred;
} catch (URISyntaxException e) {
LOG.log(Level.WARNING,
"URISyntaxException when checking size of destFile: ["
+ r.getDataStoreReference() + "]: Message: "
+ e.getMessage());
return -1L;
}
}
So the file transfer size is worked out on the fly - its not stored in the
catalog.
The origin of the weirdness relates to my Versioner.
My versioner stores files to
file:/var/kat/archive/data/comm/[StartDate]/1234567890.h5
My original code (pre-OODT) stored the data in the same way. However, for
OODT I store my dates in UTC, and thus there is a 2 hour difference between
my preOODT and OODT implementations, so for my product 1287353769.h5 the
references are:
origReference "file:/var/kat/archive/data/comm/2010/10/*18*/1287353769.h5"
(id=115)
dataStoreReference "file:/var/kat/archive/data/comm/2010/10/*17*/1287353769.h5"
(id=112)
Since I used the InPlaceDataTransfer the data reference was updated but the
file was not moved.
So files captured between midnight and 2am will now have a new reference to
the time shifted directory (e.g. moved from the 18'th to the 17'th)
When the %complete is calculated it uses the dataStoreReference.
Is it possible for the file manager to know that the InPlaceDataTransfer
has been used and that the reference should not change? I guess this would
not be possible.
Cheers,
Tom
On 17 July 2012 05:29, Sheryl John <[email protected]> wrote:
> > On Jul 16, 2012 8:26 AM, "Thomas Bennett" <[email protected]> wrote:
>> > Hi,
>> >
>> > I've recently created a Lucene file manager catalog. I cataloged about
>> 2000 files "in place", but have encountered a problem.
>> >
>> > From the fmbrowser some of the files are listed as being:
>> >
>> > Product Transfer Status Percent Complete Received
>> Time References Metadata
>> > 1287587805.h5 RECEIVED 100% UNKNOWN
>> View References View Metadata
>> > 1287353769.h5 RECEIVED 0% UNKNOWN
>> View References View Metadata
>> > 1287355996.h5 RECEIVED 0% UNKNOWN
>> View References View Metadata
>> > 1287425397.h5 RECEIVED 100% UNKNOWN
>> View References View Metadata
>> >
>> > The problem lies with the two products that show % complete being 0%.
>> >
>> > When I use fmprod to download the product:
>> >
>> > wget http://
>> ..../fmprod/data?productID=26b9cb0b-c5dd-11e1-bb9f-036596780c6e
>> >
>> > I get:
>> > HTTP request sent, awaiting response... 500 Internal Server Error
>> > 2012-07-14 20:20:38 ERROR 500: Internal Server Error.
>> >
>> > So I'd like to "edit" the lucene catalog to update this piece of
>> metadata to reflect that the % complete is 100. Are there any tools
>> available to do this?
>> >
>> > Also at some point I must figure out why its happening :)
>> >
>> > Cheers,
>> > Tom
>>
>
