-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6409/
-----------------------------------------------------------
(Updated Aug. 7, 2012, 12:38 a.m.)
Review request for oodt, Chris Mattmann and Paul Ramirez.
Description (updated)
-------
A NumberFormatException occurs when using the cas-product RSS service to view
file transfers in progress, as detailed by OODT-483. I traced this exception
to two methods in the XmlRpcStructFactory class, getXmlRpcFileTransferStatus
and getFileTransferStatusFromXmlRpc:
public static Hashtable<String, Object>
getXmlRpcFileTransferStatus(FileTransferStatus status)
{
Hashtable<String, Object> statusHash = new Hashtable<String, Object>();
statusHash.put("bytesTransferred",
Long.toBinaryString(status.getBytesTransferred()));
statusHash.put("parentProduct", getXmlRpcProduct(status.getParentProduct()));
statusHash.put("fileRef", getXmlRpcReference(status.getFileRef()));
return statusHash;
}
public static FileTransferStatus
getFileTransferStatusFromXmlRpc(Hashtable<String, Object> statusHash)
{
FileTransferStatus status = new FileTransferStatus();
status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
status.setParentProduct(getProductFromXmlRpc((Hashtable<String, Object>)
statusHash.get("parentProduct")));
status.setFileRef(getReferenceFromXmlRpc((Hashtable<String, Object>)
statusHash.get("fileRef")));
return status;
}
The following line in getXmlRpcFileTransferStatus converts the value returned
by status.getBytesTransferred() to a binary representation in a string and
stores the result in the statusHash Hashtable:
statusHash.put("bytesTransferred",
Long.toBinaryString(status.getBytesTransferred()));
The following line in getFileTransferStatusFromXmlRpc retrieves the value for
key "bytesTransferred" from the supplied statusHash Hashtable, but it does not
assume that the number is in binary format and attempts to convert it directly
to a long to pass to the setBytesTransferred() method for the status (type
FileTransferStatus) object:
status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString()));
This causes a NumberFormatException for larger files because the parseLong
can't fit the binary representation inside a long.
Here are two alternative simple fixes:
1) in getXmlRpcFileTransferStatus use 'toString' instead of 'toBinaryString' to
store a decimal representation of the number in the string (in statusHash):
statusHash.put("bytesTransferred",
Long.toString(status.getBytesTransferred()));
2) in getFileTransferStatusFromXmlRpc use a radix argument for parseLong to
specify that the "bytesTransferred" string (from statusHash) represents a
binary number:
status.setBytesTransferred(Long.parseLong(statusHash.get("bytesTransferred").toString(),
2));
I have uploaded two patches, one for each of the above suggestions.
How important is the binary representation? Is it used to prevent loss of
precision? Looking back at the commit history for XmlRpcStructFactory, it
looks like the 'toBinaryString' conversion was introduced as a result of
OODT-54. This was part of a bug fix to prevent loss of precision that happened
when casting from longs to integers and then back again.
The changes can be seen here (see changes for lines 75 and 86):
http://svn.apache.org/viewvc/oodt/trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/XmlRpcStructFactory.java?r1=1096655&r2=1070942&diff_format=h
This addresses bugs OODT-483 and OODT-54.
https://issues.apache.org/jira/browse/OODT-483
https://issues.apache.org/jira/browse/OODT-54
Diffs
-----
trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/util/XmlRpcStructFactory.java
1365193
Diff: https://reviews.apache.org/r/6409/diff/
Testing
-------
I have tested both of the patches by using File Manager to ingest a large file
(over 100MB) and then running the following from the command line to access the
RSS output from the RSSProductTransferServlet in the 'cas-product' (fmprod) web
application:
curl "http://localhost:8080/cas-product-0.5-SNAPSHOT/viewTransfers";
An example output is shown below:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<rss xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas"; version="2.0">
<channel>
<title>File Manager Transfers</title>
<link>http://localhost:8080/cas-product-0.5-SNAPSHOT/viewTransfers</link>
<description>Current Files Being Transferred to the File Manager</description>
<language>en-us</language>
<copyright>Copyright 2010: Apache Software Foundation</copyright>
<pubDate>Mon, 06 Aug 2012 12:07:41 BST</pubDate>
<category>data transfer</category>
<generator>CAS File Manager</generator>
<lastBuildDate>Mon, 06 Aug 2012 12:07:41 BST</lastBuildDate>
<item>
<title>t1.m4v</title>
<description>GenericFile</description>
<link>http://localhost:8080/cas-product-0.5-SNAPSHOT/viewTransfer?ref=file:///usr/local/oodt/cas-filemgr-0.5/test/t1.m4v&size=124699632</link>
<pubDate>Mon, 06 Aug 2012 13:07:32 BST</pubDate>
<source>file manager transfers</source>
<cas:source>file manager transfers</cas:source>
<cas:bytesTransferred>101998592</cas:bytesTransferred>
<cas:totalBytes>124699632</cas:totalBytes>
<cas:percentComplete>0.8179542342193921</cas:percentComplete>
</item>
</channel>
</rss>
Thanks,
Ross Laidlaw
