Re: Review Request 10096: OODT-611: Implement a JAX-RS service for delivery of file manager products in raw data formats (file, zip)

[Chris Mattmann]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10096/#review23131
---

Ship it!


Ship It!

- Chris Mattmann


On July 13, 2013, 11:31 a.m., Ross Laidlaw wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10096/
> ---
> 
> (Updated July 13, 2013, 11:31 a.m.)
> 
> 
> Review request for oodt and Rishi Verma.
> 
> 
> Bugs: OODT-611
> https://issues.apache.org/jira/browse/OODT-611
> 
> 
> Repository: oodt
> 
> 
> Description
> ---
> 
> Summary
> ===
> 
> This patch implements a new JAX-RS service for the CAS-Product web 
> application.  It ports over functionality from the DataDeliveryServlet and 
> DatasetDeliveryServlet classes (and associated utility classes) from the 
> oodt.apache.org.cas.product.data package.
> 
> 
> URLs
> 
> 
> Example URLs are:
> 
> /reference?productID=123abc&refIndex=0&format=file
> /product?productID=123abc&format=zip
> /dataset?typeID=urn:oodt:GenericFile&format=zip
> 
> 
> For backwards compatibility, I've also included the 'data' URL as an 
> equivalent to the 'product' URL, for example:
> 
> /data?productID=123abc&format=zip
> 
> 
> While this project is in progress, I've mapped the JAX-RS servlet to the 
> '/service/*' URL so the new URLs don't clash with URLs from the original HTTP 
> servlets, which are still active.  For example, if the web application is 
> deployed to http://localhost:8080/fmprod the full URLs would be as follows:
> 
> http://localhost:8080/fmprod/service/reference?productID=123abc&refIndex=0&format=file
> http://localhost:8080/fmprod/service/product?productID=123abc&format=zip
> http://localhost:8080/fmprod/service/dataset?typeID=urn:oodt:GenericFile&format=zip
> 
> 
> Design
> ==
> 
> I've added a new package: oodt.apache.org.cas.product.service.
> 
> Within this package, I've created four sub-packages:
> 
> * configurations: (currently empty) will contain classes for reading and 
> processing XML configuration files (e.g. for RDF, RSS)
> * exceptions: custom exception classes that extend 
> javax.ws.rs.WebApplicationException to return specific HTTP response status 
> codes plus messages
> * responders: classes that provide HTTP responses in different formats
> * resources: JAX-RS resources representing file manager entities
> 
> The above design will hopefully make it fairly straightforward to add new 
> output formats by adding new responders and configurations.  For example, the 
> aim is to add RdfResponder and RssResponder classes to the responders 
> package, and RdfConfiguration and RssConfiguration classes to the 
> configurations package for OODT-612 and OODT-613.
> 
> 
> Notes
> =
> 
> (Please ignore the first diff (version 1) attached to this review request - 
> it's from an old issue OODT-470.)
> 
> I've moved the original test classes from src/test to src/test/java, and the 
> test resources (test.logging.properties) from src/testdata to 
> src/test/resources.  This matches the recommended Maven project structure.  
> But these details have been omitted from the patch, as Review Board did not 
> cope well with the details of the moved files - instead it shows the files as 
> having been deleted.
> 
> I added a test 'base' class 'ResourceTestBase' but found that the 
> maven-surefire-plugin tried to run this as a JUnit test class and complained 
> that there were no tests to run.  As a workaround, I changed the filename 
> recognition pattern to '/*Test.class' so that only classes ending in 'Test' 
> (rather than having 'Test' anywhere in the class name) would be recognised as 
> JUnit test classes by the surefire plugin.
> 
> For zipping products, I've assumed that if the first reference is a 
> directory, then it's a hierarchical product and all contents of that 
> directory are included in the zip.  We might want to do something more 
> sophisticated, such as iterating over the list of references, finding the 
> root directory and building a temp structure to zip (or process in another 
> format).
> 
> Currently there's quite a lot of repetition in the ZipResponder class and 
> some of the test classes.  I'd like to refactor these after I've implemented 
> the other main tasks for the project: OODT-612 and OODT-613.
> 
> The attached patch also contains a few minor updates to the CAS-Product POM 
> and original servlets that I spotted while carrying out this work.
> 
> 
> Diffs
> -
> 
>   /trunk/webapp/fmprod/pom.xml 1502765 
>   
> /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/data/DataUtils.java
>  1502765 
>   
> /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/exceptions/BadRequestException.java
>  PRE-CREATION 
>   
> /trunk/webapp/fmprod/src/m

Re: Review Request 10096: OODT-611: Implement a JAX-RS service for delivery of file manager products in raw data formats (file, zip)

[Ross Laidlaw]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10096/
---

(Updated July 13, 2013, 9:31 a.m.)


Review request for oodt and Rishi Verma.


Changes
---

Hi Rishi & Chris!

Many thanks for your guidance and advice.  I've uploaded an updated diff.  In 
the ZipResponder I've added some logging at Level.FINE when deleting files, as 
per Chris' suggestion.  In the FileResponder, I've replaced my clumsy attempt 
at MIME type detection with the proper Tika.detect(String).  I've used version 
0.8 of Tika for now, but I'll take a look at supplying the patch to upgrade to 
v1.4 across OODT.  I've also added a comment to the POM to clarify that Zip4j 
is ALv2 licensed (with reference to the link that Chris provided).

To test the log levels, I used the settings shown below in the 
'test.logging.properties' file.  This file doesn't appear in the Review Board 
diff because I moved it from src/testdata to src/test/resources.

###
handlers = java.util.logging.ConsoleHandler
.level = FINE
java.util.logging.ConsoleHandler.level = FINE
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
org.apache.oodt.cas.product.service = FINE
org.apache.oodt.cas.filemgr.level = SEVERE
org.apache.commons.httpclient.level = WARNING
httpclient.wire.level = WARNING
org.apache.cxf.level = WARNING
###


Ross


Bugs: OODT-611
https://issues.apache.org/jira/browse/OODT-611


Repository: oodt


Description
---

Summary
===

This patch implements a new JAX-RS service for the CAS-Product web application. 
 It ports over functionality from the DataDeliveryServlet and 
DatasetDeliveryServlet classes (and associated utility classes) from the 
oodt.apache.org.cas.product.data package.


URLs


Example URLs are:

/reference?productID=123abc&refIndex=0&format=file
/product?productID=123abc&format=zip
/dataset?typeID=urn:oodt:GenericFile&format=zip


For backwards compatibility, I've also included the 'data' URL as an equivalent 
to the 'product' URL, for example:

/data?productID=123abc&format=zip


While this project is in progress, I've mapped the JAX-RS servlet to the 
'/service/*' URL so the new URLs don't clash with URLs from the original HTTP 
servlets, which are still active.  For example, if the web application is 
deployed to http://localhost:8080/fmprod the full URLs would be as follows:

http://localhost:8080/fmprod/service/reference?productID=123abc&refIndex=0&format=file
http://localhost:8080/fmprod/service/product?productID=123abc&format=zip
http://localhost:8080/fmprod/service/dataset?typeID=urn:oodt:GenericFile&format=zip


Design
==

I've added a new package: oodt.apache.org.cas.product.service.

Within this package, I've created four sub-packages:

* configurations: (currently empty) will contain classes for reading and 
processing XML configuration files (e.g. for RDF, RSS)
* exceptions: custom exception classes that extend 
javax.ws.rs.WebApplicationException to return specific HTTP response status 
codes plus messages
* responders: classes that provide HTTP responses in different formats
* resources: JAX-RS resources representing file manager entities

The above design will hopefully make it fairly straightforward to add new 
output formats by adding new responders and configurations.  For example, the 
aim is to add RdfResponder and RssResponder classes to the responders package, 
and RdfConfiguration and RssConfiguration classes to the configurations package 
for OODT-612 and OODT-613.


Notes
=

(Please ignore the first diff (version 1) attached to this review request - 
it's from an old issue OODT-470.)

I've moved the original test classes from src/test to src/test/java, and the 
test resources (test.logging.properties) from src/testdata to 
src/test/resources.  This matches the recommended Maven project structure.  But 
these details have been omitted from the patch, as Review Board did not cope 
well with the details of the moved files - instead it shows the files as having 
been deleted.

I added a test 'base' class 'ResourceTestBase' but found that the 
maven-surefire-plugin tried to run this as a JUnit test class and complained 
that there were no tests to run.  As a workaround, I changed the filename 
recognition pattern to '/*Test.class' so that only classes ending in 'Test' 
(rather than having 'Test' anywhere in the class name) would be recognised as 
JUnit test classes by the surefire plugin.

For zipping products, I've assumed that if the first reference is a directory, 
then it's a hierarchical product and all contents of that directory are 
included in the zip.  We might want to do something more sophisticated, such as 
iterating over the list of references, finding the root directory and building 
a temp structure to zip (or process in another format).

Currently there's quite a lot of repetition in the Zi

Re: Review Request 10096: OODT-611: Implement a JAX-RS service for delivery of file manager products in raw data formats (file, zip)

[Ross Laidlaw]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10096/
---

(Updated July 13, 2013, 11:31 a.m.)


Review request for oodt and Rishi Verma.


Changes
---

I just spotted and corrected some mistakes in the Javadocs that I made in the 
original diff.  (Therefore - please view changes from diff r2 to r4 show 
updated code since original diff).


Bugs: OODT-611
https://issues.apache.org/jira/browse/OODT-611


Repository: oodt


Description
---

Summary
===

This patch implements a new JAX-RS service for the CAS-Product web application. 
 It ports over functionality from the DataDeliveryServlet and 
DatasetDeliveryServlet classes (and associated utility classes) from the 
oodt.apache.org.cas.product.data package.


URLs


Example URLs are:

/reference?productID=123abc&refIndex=0&format=file
/product?productID=123abc&format=zip
/dataset?typeID=urn:oodt:GenericFile&format=zip


For backwards compatibility, I've also included the 'data' URL as an equivalent 
to the 'product' URL, for example:

/data?productID=123abc&format=zip


While this project is in progress, I've mapped the JAX-RS servlet to the 
'/service/*' URL so the new URLs don't clash with URLs from the original HTTP 
servlets, which are still active.  For example, if the web application is 
deployed to http://localhost:8080/fmprod the full URLs would be as follows:

http://localhost:8080/fmprod/service/reference?productID=123abc&refIndex=0&format=file
http://localhost:8080/fmprod/service/product?productID=123abc&format=zip
http://localhost:8080/fmprod/service/dataset?typeID=urn:oodt:GenericFile&format=zip


Design
==

I've added a new package: oodt.apache.org.cas.product.service.

Within this package, I've created four sub-packages:

* configurations: (currently empty) will contain classes for reading and 
processing XML configuration files (e.g. for RDF, RSS)
* exceptions: custom exception classes that extend 
javax.ws.rs.WebApplicationException to return specific HTTP response status 
codes plus messages
* responders: classes that provide HTTP responses in different formats
* resources: JAX-RS resources representing file manager entities

The above design will hopefully make it fairly straightforward to add new 
output formats by adding new responders and configurations.  For example, the 
aim is to add RdfResponder and RssResponder classes to the responders package, 
and RdfConfiguration and RssConfiguration classes to the configurations package 
for OODT-612 and OODT-613.


Notes
=

(Please ignore the first diff (version 1) attached to this review request - 
it's from an old issue OODT-470.)

I've moved the original test classes from src/test to src/test/java, and the 
test resources (test.logging.properties) from src/testdata to 
src/test/resources.  This matches the recommended Maven project structure.  But 
these details have been omitted from the patch, as Review Board did not cope 
well with the details of the moved files - instead it shows the files as having 
been deleted.

I added a test 'base' class 'ResourceTestBase' but found that the 
maven-surefire-plugin tried to run this as a JUnit test class and complained 
that there were no tests to run.  As a workaround, I changed the filename 
recognition pattern to '/*Test.class' so that only classes ending in 'Test' 
(rather than having 'Test' anywhere in the class name) would be recognised as 
JUnit test classes by the surefire plugin.

For zipping products, I've assumed that if the first reference is a directory, 
then it's a hierarchical product and all contents of that directory are 
included in the zip.  We might want to do something more sophisticated, such as 
iterating over the list of references, finding the root directory and building 
a temp structure to zip (or process in another format).

Currently there's quite a lot of repetition in the ZipResponder class and some 
of the test classes.  I'd like to refactor these after I've implemented the 
other main tasks for the project: OODT-612 and OODT-613.

The attached patch also contains a few minor updates to the CAS-Product POM and 
original servlets that I spotted while carrying out this work.


Diffs (updated)
-

  /trunk/webapp/fmprod/pom.xml 1502765 
  
/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/data/DataUtils.java
 1502765 
  
/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/exceptions/BadRequestException.java
 PRE-CREATION 
  
/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/exceptions/InternalServerErrorException.java
 PRE-CREATION 
  
/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/exceptions/NotFoundException.java
 PRE-CREATION 
  
/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/resources/DatasetResource.java
 PRE-CREATION 
  
/trun

Re: Review Request 10096: OODT-611: Implement a JAX-RS service for delivery of file manager products in raw data formats (file, zip)

[Chris Mattmann]

> On July 8, 2013, 1:55 p.m., Chris Mattmann wrote:
> > /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/FileResponder.java,
> >  line 49
> > 
> >
> > Not sure this is needed, can be replaced with call to Tika.detect per 
> > http://tika.apache.org/1.4/detection.html
> 
> Rishi Verma wrote:
> Just an FYI, tika.detect will likely give you a Tika version conflict 
> with OODT's Metadata or FileManager Tika bundled JAR (0.8 vs. 1.4). This JAR 
> contains a different method signature for detecting mime types, and is not 
> compatible with Tika.detect.
> 
> I've documented this issue at: 
> https://issues.apache.org/jira/browse/OODT-630
> 
> You might want to use the deprecated version of that method until 
> OODT-630 is resolved (which will be soon :)
> 
> http://tika.apache.org/0.8/api/org/apache/tika/mime/MimeTypes.html#getMimeType(java.lang.String)

Why not just upgrade Tika in File Manager to 1.4? 


- Chris


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10096/#review22812
---


On July 8, 2013, 5:41 a.m., Ross Laidlaw wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10096/
> ---
> 
> (Updated July 8, 2013, 5:41 a.m.)
> 
> 
> Review request for oodt and Rishi Verma.
> 
> 
> Bugs: OODT-611
> https://issues.apache.org/jira/browse/OODT-611
> 
> 
> Repository: oodt
> 
> 
> Description
> ---
> 
> Summary
> ===
> 
> This patch implements a new JAX-RS service for the CAS-Product web 
> application.  It ports over functionality from the DataDeliveryServlet and 
> DatasetDeliveryServlet classes (and associated utility classes) from the 
> oodt.apache.org.cas.product.data package.
> 
> 
> URLs
> 
> 
> Example URLs are:
> 
> /reference?productID=123abc&refIndex=0&format=file
> /product?productID=123abc&format=zip
> /dataset?typeID=urn:oodt:GenericFile&format=zip
> 
> 
> For backwards compatibility, I've also included the 'data' URL as an 
> equivalent to the 'product' URL, for example:
> 
> /data?productID=123abc&format=zip
> 
> 
> While this project is in progress, I've mapped the JAX-RS servlet to the 
> '/service/*' URL so the new URLs don't clash with URLs from the original HTTP 
> servlets, which are still active.  For example, if the web application is 
> deployed to http://localhost:8080/fmprod the full URLs would be as follows:
> 
> http://localhost:8080/fmprod/service/reference?productID=123abc&refIndex=0&format=file
> http://localhost:8080/fmprod/service/product?productID=123abc&format=zip
> http://localhost:8080/fmprod/service/dataset?typeID=urn:oodt:GenericFile&format=zip
> 
> 
> Design
> ==
> 
> I've added a new package: oodt.apache.org.cas.product.service.
> 
> Within this package, I've created four sub-packages:
> 
> * configurations: (currently empty) will contain classes for reading and 
> processing XML configuration files (e.g. for RDF, RSS)
> * exceptions: custom exception classes that extend 
> javax.ws.rs.WebApplicationException to return specific HTTP response status 
> codes plus messages
> * responders: classes that provide HTTP responses in different formats
> * resources: JAX-RS resources representing file manager entities
> 
> The above design will hopefully make it fairly straightforward to add new 
> output formats by adding new responders and configurations.  For example, the 
> aim is to add RdfResponder and RssResponder classes to the responders 
> package, and RdfConfiguration and RssConfiguration classes to the 
> configurations package for OODT-612 and OODT-613.
> 
> 
> Notes
> =
> 
> (Please ignore the first diff (version 1) attached to this review request - 
> it's from an old issue OODT-470.)
> 
> I've moved the original test classes from src/test to src/test/java, and the 
> test resources (test.logging.properties) from src/testdata to 
> src/test/resources.  This matches the recommended Maven project structure.  
> But these details have been omitted from the patch, as Review Board did not 
> cope well with the details of the moved files - instead it shows the files as 
> having been deleted.
> 
> I added a test 'base' class 'ResourceTestBase' but found that the 
> maven-surefire-plugin tried to run this as a JUnit test class and complained 
> that there were no tests to run.  As a workaround, I changed the filename 
> recognition pattern to '/*Test.class' so that only classes ending in 'Test' 
> (rather than having 'Test' anywhere in the class name) would be recognised as 
> JUnit test classes by the surefire plugin.
> 
> For zipping products, I've assumed that if the first reference is a 
> d

Re: Review Request 10096: OODT-611: Implement a JAX-RS service for delivery of file manager products in raw data formats (file, zip)

[Rishi Verma]

> On July 8, 2013, 1:55 p.m., Chris Mattmann wrote:
> > /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/FileResponder.java,
> >  line 49
> > 
> >
> > Not sure this is needed, can be replaced with call to Tika.detect per 
> > http://tika.apache.org/1.4/detection.html

Just an FYI, tika.detect will likely give you a Tika version conflict with 
OODT's Metadata or FileManager Tika bundled JAR (0.8 vs. 1.4). This JAR 
contains a different method signature for detecting mime types, and is not 
compatible with Tika.detect.

I've documented this issue at: https://issues.apache.org/jira/browse/OODT-630

You might want to use the deprecated version of that method until OODT-630 is 
resolved (which will be soon :)
http://tika.apache.org/0.8/api/org/apache/tika/mime/MimeTypes.html#getMimeType(java.lang.String)


> On July 8, 2013, 1:55 p.m., Chris Mattmann wrote:
> > /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/FileResponder.java,
> >  line 127
> > 
> >
> > Why would this ever be triggered? Or if it is triggered, shouldn't we 
> > just call e.g., Tika.detect here per: 
> > http://tika.apache.org/1.4/detection.html

See tika comment above.


- Rishi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10096/#review22812
---


On July 8, 2013, 5:41 a.m., Ross Laidlaw wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10096/
> ---
> 
> (Updated July 8, 2013, 5:41 a.m.)
> 
> 
> Review request for oodt and Rishi Verma.
> 
> 
> Bugs: OODT-611
> https://issues.apache.org/jira/browse/OODT-611
> 
> 
> Repository: oodt
> 
> 
> Description
> ---
> 
> Summary
> ===
> 
> This patch implements a new JAX-RS service for the CAS-Product web 
> application.  It ports over functionality from the DataDeliveryServlet and 
> DatasetDeliveryServlet classes (and associated utility classes) from the 
> oodt.apache.org.cas.product.data package.
> 
> 
> URLs
> 
> 
> Example URLs are:
> 
> /reference?productID=123abc&refIndex=0&format=file
> /product?productID=123abc&format=zip
> /dataset?typeID=urn:oodt:GenericFile&format=zip
> 
> 
> For backwards compatibility, I've also included the 'data' URL as an 
> equivalent to the 'product' URL, for example:
> 
> /data?productID=123abc&format=zip
> 
> 
> While this project is in progress, I've mapped the JAX-RS servlet to the 
> '/service/*' URL so the new URLs don't clash with URLs from the original HTTP 
> servlets, which are still active.  For example, if the web application is 
> deployed to http://localhost:8080/fmprod the full URLs would be as follows:
> 
> http://localhost:8080/fmprod/service/reference?productID=123abc&refIndex=0&format=file
> http://localhost:8080/fmprod/service/product?productID=123abc&format=zip
> http://localhost:8080/fmprod/service/dataset?typeID=urn:oodt:GenericFile&format=zip
> 
> 
> Design
> ==
> 
> I've added a new package: oodt.apache.org.cas.product.service.
> 
> Within this package, I've created four sub-packages:
> 
> * configurations: (currently empty) will contain classes for reading and 
> processing XML configuration files (e.g. for RDF, RSS)
> * exceptions: custom exception classes that extend 
> javax.ws.rs.WebApplicationException to return specific HTTP response status 
> codes plus messages
> * responders: classes that provide HTTP responses in different formats
> * resources: JAX-RS resources representing file manager entities
> 
> The above design will hopefully make it fairly straightforward to add new 
> output formats by adding new responders and configurations.  For example, the 
> aim is to add RdfResponder and RssResponder classes to the responders 
> package, and RdfConfiguration and RssConfiguration classes to the 
> configurations package for OODT-612 and OODT-613.
> 
> 
> Notes
> =
> 
> (Please ignore the first diff (version 1) attached to this review request - 
> it's from an old issue OODT-470.)
> 
> I've moved the original test classes from src/test to src/test/java, and the 
> test resources (test.logging.properties) from src/testdata to 
> src/test/resources.  This matches the recommended Maven project structure.  
> But these details have been omitted from the patch, as Review Board did not 
> cope well with the details of the moved files - instead it shows the files as 
> having been deleted.
> 
> I added a test 'base' class 'ResourceTestBase' but found that the 
> maven-surefire-plugin tried to run this as a JUnit test class and complained 
> that there were no tests to run.

Re: Review Request 10096: OODT-611: Implement a JAX-RS service for delivery of file manager products in raw data formats (file, zip)

[Chris Mattmann]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10096/#review22812
---



/trunk/webapp/fmprod/pom.xml


reformatting (seems orthogonal)



/trunk/webapp/fmprod/pom.xml


Worth noting this is ALv2 licensed, per: 
http://www.lingala.net/zip4j/about.php



/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/FileResponder.java


Not sure this is needed, can be replaced with call to Tika.detect per 
http://tika.apache.org/1.4/detection.html



/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/FileResponder.java


Why would this ever be triggered? Or if it is triggered, shouldn't we just 
call e.g., Tika.detect here per: http://tika.apache.org/1.4/detection.html



/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/ZipResponder.java


probably worth logging here to Level.FINE if removal was unsuccessful.


- Chris Mattmann


On July 8, 2013, 5:41 a.m., Ross Laidlaw wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10096/
> ---
> 
> (Updated July 8, 2013, 5:41 a.m.)
> 
> 
> Review request for oodt and Rishi Verma.
> 
> 
> Bugs: OODT-611
> https://issues.apache.org/jira/browse/OODT-611
> 
> 
> Repository: oodt
> 
> 
> Description
> ---
> 
> Summary
> ===
> 
> This patch implements a new JAX-RS service for the CAS-Product web 
> application.  It ports over functionality from the DataDeliveryServlet and 
> DatasetDeliveryServlet classes (and associated utility classes) from the 
> oodt.apache.org.cas.product.data package.
> 
> 
> URLs
> 
> 
> Example URLs are:
> 
> /reference?productID=123abc&refIndex=0&format=file
> /product?productID=123abc&format=zip
> /dataset?typeID=urn:oodt:GenericFile&format=zip
> 
> 
> For backwards compatibility, I've also included the 'data' URL as an 
> equivalent to the 'product' URL, for example:
> 
> /data?productID=123abc&format=zip
> 
> 
> While this project is in progress, I've mapped the JAX-RS servlet to the 
> '/service/*' URL so the new URLs don't clash with URLs from the original HTTP 
> servlets, which are still active.  For example, if the web application is 
> deployed to http://localhost:8080/fmprod the full URLs would be as follows:
> 
> http://localhost:8080/fmprod/service/reference?productID=123abc&refIndex=0&format=file
> http://localhost:8080/fmprod/service/product?productID=123abc&format=zip
> http://localhost:8080/fmprod/service/dataset?typeID=urn:oodt:GenericFile&format=zip
> 
> 
> Design
> ==
> 
> I've added a new package: oodt.apache.org.cas.product.service.
> 
> Within this package, I've created four sub-packages:
> 
> * configurations: (currently empty) will contain classes for reading and 
> processing XML configuration files (e.g. for RDF, RSS)
> * exceptions: custom exception classes that extend 
> javax.ws.rs.WebApplicationException to return specific HTTP response status 
> codes plus messages
> * responders: classes that provide HTTP responses in different formats
> * resources: JAX-RS resources representing file manager entities
> 
> The above design will hopefully make it fairly straightforward to add new 
> output formats by adding new responders and configurations.  For example, the 
> aim is to add RdfResponder and RssResponder classes to the responders 
> package, and RdfConfiguration and RssConfiguration classes to the 
> configurations package for OODT-612 and OODT-613.
> 
> 
> Notes
> =
> 
> (Please ignore the first diff (version 1) attached to this review request - 
> it's from an old issue OODT-470.)
> 
> I've moved the original test classes from src/test to src/test/java, and the 
> test resources (test.logging.properties) from src/testdata to 
> src/test/resources.  This matches the recommended Maven project structure.  
> But these details have been omitted from the patch, as Review Board did not 
> cope well with the details of the moved files - instead it shows the files as 
> having been deleted.
> 
> I added a test 'base' class 'ResourceTestBase' but found that the 
> maven-surefire-plugin tried to run this as a JUnit test class and complained 
> that there were no tests to run.  As a workaround, I changed the filename 
> recognition pattern to '/*Test.class' so that only classes ending in 'Test' 
> (rather than having 'Test' anywhere in the class name) would be recognised as 
> JUnit test classes by the surefire plugin.
> 
> For zippi

Re: Review Request 10096: OODT-611: Implement a JAX-RS service for delivery of file manager products in raw data formats (file, zip)

[Chris Mattmann]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10096/#review22813
---

Ship it!


Ship It!

- Chris Mattmann


On July 8, 2013, 5:41 a.m., Ross Laidlaw wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10096/
> ---
> 
> (Updated July 8, 2013, 5:41 a.m.)
> 
> 
> Review request for oodt and Rishi Verma.
> 
> 
> Bugs: OODT-611
> https://issues.apache.org/jira/browse/OODT-611
> 
> 
> Repository: oodt
> 
> 
> Description
> ---
> 
> Summary
> ===
> 
> This patch implements a new JAX-RS service for the CAS-Product web 
> application.  It ports over functionality from the DataDeliveryServlet and 
> DatasetDeliveryServlet classes (and associated utility classes) from the 
> oodt.apache.org.cas.product.data package.
> 
> 
> URLs
> 
> 
> Example URLs are:
> 
> /reference?productID=123abc&refIndex=0&format=file
> /product?productID=123abc&format=zip
> /dataset?typeID=urn:oodt:GenericFile&format=zip
> 
> 
> For backwards compatibility, I've also included the 'data' URL as an 
> equivalent to the 'product' URL, for example:
> 
> /data?productID=123abc&format=zip
> 
> 
> While this project is in progress, I've mapped the JAX-RS servlet to the 
> '/service/*' URL so the new URLs don't clash with URLs from the original HTTP 
> servlets, which are still active.  For example, if the web application is 
> deployed to http://localhost:8080/fmprod the full URLs would be as follows:
> 
> http://localhost:8080/fmprod/service/reference?productID=123abc&refIndex=0&format=file
> http://localhost:8080/fmprod/service/product?productID=123abc&format=zip
> http://localhost:8080/fmprod/service/dataset?typeID=urn:oodt:GenericFile&format=zip
> 
> 
> Design
> ==
> 
> I've added a new package: oodt.apache.org.cas.product.service.
> 
> Within this package, I've created four sub-packages:
> 
> * configurations: (currently empty) will contain classes for reading and 
> processing XML configuration files (e.g. for RDF, RSS)
> * exceptions: custom exception classes that extend 
> javax.ws.rs.WebApplicationException to return specific HTTP response status 
> codes plus messages
> * responders: classes that provide HTTP responses in different formats
> * resources: JAX-RS resources representing file manager entities
> 
> The above design will hopefully make it fairly straightforward to add new 
> output formats by adding new responders and configurations.  For example, the 
> aim is to add RdfResponder and RssResponder classes to the responders 
> package, and RdfConfiguration and RssConfiguration classes to the 
> configurations package for OODT-612 and OODT-613.
> 
> 
> Notes
> =
> 
> (Please ignore the first diff (version 1) attached to this review request - 
> it's from an old issue OODT-470.)
> 
> I've moved the original test classes from src/test to src/test/java, and the 
> test resources (test.logging.properties) from src/testdata to 
> src/test/resources.  This matches the recommended Maven project structure.  
> But these details have been omitted from the patch, as Review Board did not 
> cope well with the details of the moved files - instead it shows the files as 
> having been deleted.
> 
> I added a test 'base' class 'ResourceTestBase' but found that the 
> maven-surefire-plugin tried to run this as a JUnit test class and complained 
> that there were no tests to run.  As a workaround, I changed the filename 
> recognition pattern to '/*Test.class' so that only classes ending in 'Test' 
> (rather than having 'Test' anywhere in the class name) would be recognised as 
> JUnit test classes by the surefire plugin.
> 
> For zipping products, I've assumed that if the first reference is a 
> directory, then it's a hierarchical product and all contents of that 
> directory are included in the zip.  We might want to do something more 
> sophisticated, such as iterating over the list of references, finding the 
> root directory and building a temp structure to zip (or process in another 
> format).
> 
> Currently there's quite a lot of repetition in the ZipResponder class and 
> some of the test classes.  I'd like to refactor these after I've implemented 
> the other main tasks for the project: OODT-612 and OODT-613.
> 
> The attached patch also contains a few minor updates to the CAS-Product POM 
> and original servlets that I spotted while carrying out this work.
> 
> 
> Diffs
> -
> 
>   /trunk/webapp/fmprod/pom.xml 1498211 
>   
> /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/data/DataUtils.java
>  1498211 
>   
> /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/exceptions/BadRequestException.java
>  PRE-CREATION 
>   
> /trunk/webapp/fmprod/src/main/

Review Request 10096: OODT-611: Implement a JAX-RS service for delivery of file manager products in raw data formats (file, zip)

[Ross Laidlaw]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10096/
---

Review request for oodt and Rishi Verma.


Summary (updated)
-

OODT-611: Implement a JAX-RS service for delivery of file manager products in 
raw data formats (file, zip)


Bugs: OODT-611
https://issues.apache.org/jira/browse/OODT-611


Repository: oodt


Description (updated)
---

Summary
===

This patch implements a new JAX-RS service for the CAS-Product web application. 
 It ports over functionality from the DataDeliveryServlet and 
DatasetDeliveryServlet classes (and associated utility classes) from the 
oodt.apache.org.cas.product.data package.


URLs


Example URLs are:

/reference?productID=123abc&refIndex=0&format=file
/product?productID=123abc&format=zip
/dataset?typeID=urn:oodt:GenericFile&format=zip


For backwards compatibility, I've also included the 'data' URL as an equivalent 
to the 'product' URL, for example:

/data?productID=123abc&format=zip


While this project is in progress, I've mapped the JAX-RS servlet to the 
'/service/*' URL so the new URLs don't clash with URLs from the original HTTP 
servlets, which are still active.  For example, if the web application is 
deployed to http://localhost:8080/fmprod the full URLs would be as follows:

http://localhost:8080/fmprod/service/reference?productID=123abc&refIndex=0&format=file
http://localhost:8080/fmprod/service/product?productID=123abc&format=zip
http://localhost:8080/fmprod/service/dataset?typeID=urn:oodt:GenericFile&format=zip


Design
==

I've added a new package: oodt.apache.org.cas.product.service.

Within this package, I've created four sub-packages:

* configurations: (currently empty) will contain classes for reading and 
processing XML configuration files (e.g. for RDF, RSS)
* exceptions: custom exception classes that extend 
javax.ws.rs.WebApplicationException to return specific HTTP response status 
codes plus messages
* responders: classes that provide HTTP responses in different formats
* resources: JAX-RS resources representing file manager entities

The above design will hopefully make it fairly straightforward to add new 
output formats by adding new responders and configurations.  For example, the 
aim is to add RdfResponder and RssResponder classes to the responders package, 
and RdfConfiguration and RssConfiguration classes to the configurations package 
for OODT-612 and OODT-613.


Notes
=

(Please ignore the first diff (version 1) attached to this review request - 
it's from an old issue OODT-470.)

I've moved the original test classes from src/test to src/test/java, and the 
test resources (test.logging.properties) from src/testdata to 
src/test/resources.  This matches the recommended Maven project structure.  But 
these details have been omitted from the patch, as Review Board did not cope 
well with the details of the moved files - instead it shows the files as having 
been deleted.

I added a test 'base' class 'ResourceTestBase' but found that the 
maven-surefire-plugin tried to run this as a JUnit test class and complained 
that there were no tests to run.  As a workaround, I changed the filename 
recognition pattern to '/*Test.class' so that only classes ending in 'Test' 
(rather than having 'Test' anywhere in the class name) would be recognised as 
JUnit test classes by the surefire plugin.

For zipping products, I've assumed that if the first reference is a directory, 
then it's a hierarchical product and all contents of that directory are 
included in the zip.  We might want to do something more sophisticated, such as 
iterating over the list of references, finding the root directory and building 
a temp structure to zip (or process in another format).

Currently there's quite a lot of repetition in the ZipResponder class and some 
of the test classes.  I'd like to refactor these after I've implemented the 
other main tasks for the project: OODT-612 and OODT-613.

The attached patch also contains a few minor updates to the CAS-Product POM and 
original servlets that I spotted while carrying out this work.


Diffs (updated)
-

  /trunk/webapp/fmprod/pom.xml 1498211 
  
/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/data/DataUtils.java
 1498211 
  
/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/exceptions/BadRequestException.java
 PRE-CREATION 
  
/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/exceptions/InternalServerErrorException.java
 PRE-CREATION 
  
/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/exceptions/NotFoundException.java
 PRE-CREATION 
  
/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/resources/DatasetResource.java
 PRE-CREATION 
  
/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/resources/ProductResou