Thanks Bruce -- attachment didn't come through (I think the mailing list strips them) so if you want to send me the attachment, chris.a.mattm...@nasa.gov works for that.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -----Original Message----- From: Bruce Barkstrom <brbarkst...@gmail.com> Reply-To: "dev@oodt.apache.org" <dev@oodt.apache.org> Date: Friday, June 28, 2013 12:49 PM To: "dev@oodt.apache.org" <dev@oodt.apache.org> Subject: Re: [jira] [Resolved] (OODT-639) Add a versioner based on Product Type Metadata > > > >It's a reasonable start. I had done something similar >for CERES. However, I think there are some kinds >of collections (like Rock Cores and the NOAA Emergency >Response Imagery collection) where the number of layers >is different. > >Also, since I don't think you'd seen my examples of the >FCA web site approach, I've attached my presentation >for the WMSCI meeting the week after next. Unzip >the file (it's got the paper in pdf form). The presentation >is the .ppt file. There are two example web sites included. >You should be able to link from the appropriate pages to >the web sites. The first example has the navigation from >no attributes to enough to select an object. The second >is a reasonable facsimile to the way the NOAA storm damage >site works in the top level or two. Note particularly the lack >of having to input a keyword to navigate through the site. > >Thanks for the response. > >Bruce B. > >On Fri, Jun 28, 2013 at 2:01 PM, Mattmann, Chris A (398J) ><chris.a.mattm...@jpl.nasa.gov> wrote: > >Hey Bruce, > >Totally agree. In Apache OODT, we have the layered, hierarchical attribute >approach, already, wherein which the first layer is the "Product Type" >concept, >capturing aggregate information bout a set of products (name, id, >classification, >aggregate free form metadata, sets of metadata extractors, and >"versioning" or >file placement on disk policy, etc.). The next level is the Product level, >where >we capture product (or frequently changing) level metadata; apply the >versioning >scheme for file placement, and extract metadata using the per product type >specified >metadata extractors. Attributes at the product level are hierarchical in >the sense >that they are defined by per product type policy (though we can accept >others) and >are specified in the OODT File Manager server through the use of a >particular >ValidationLayer, and RepositoryManager selected for the running system. > >HTH! > >Cheers, >Chris > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Chris Mattmann, Ph.D. >Senior Computer Scientist >NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >Office: 171-266B, Mailstop: 171-246 >Email: chris.a.mattm...@nasa.gov >WWW: http://sunset.usc.edu/~mattmann/ >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Adjunct Assistant Professor, Computer Science Department >University of Southern California, Los Angeles, CA 90089 USA >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > > > >-----Original Message----- >From: Bruce Barkstrom <brbarkst...@gmail.com> >Reply-To: "dev@oodt.apache.org" <dev@oodt.apache.org> >Date: Wednesday, June 26, 2013 2:50 PM >To: "dev@oodt.apache.org" <dev@oodt.apache.org> >Subject: Re: [jira] [Resolved] (OODT-639) Add a versioner based on Product >Type Metadata > > >>I'll at least mention an issue we've discovered when examining existing >>Earth science data collections: the collections have a layered or >>hierarchical >>structure, with each layer having different kinds of metadata (or, if I >>did >>it >>a bit more formally, different types of metadata). In Formal Concept >>Analysis, it shows up as hierarchical attributes. >> >>The easiest example is perhaps the NOAA Emergency Response Imagery >>collection, where the top layer divides about 250,000 digital photos into >>about twenty collections - one for each disastrous storm that needed an >>emergency response. Examples include Hurricane Sandy, Hurricane >>Katrina, and the Tuscaloosa Tornado. The top level metadata fields >>are (Storm Name, Storm Date, and Storm Location). One level below >>that, the storm collections bifurcate into image collections organized >>by airplane flight path or organized by coarse geographic boxes. >>Each storm collection has this bifurcation - and so once a user >>has distinguished which storm collection he or she is interested in, >>Storm Name (or Date or Location) is not helpful in distinguishing >>a flight path collection from a geographic collection. >> >>The level below that for the flight path requires distinguishing which >>flight path is the one you want. Once you decide that, you can get >>a zipped file with several hundred jpg images. On the other hand, >>if you go the geographic boxes, you can get individual images. >>The metadata types for the boxes are quire different from the metadata >>for the flight paths. >> >>The same kind of structure crops up for rock core archives (wells, boxes >>in a well, core fragments in a box), as well as for many of the satellite >>data collections. >> >>The usual library science approach (e.g. Functional Requirements >>for Bibliographic Records, and its relatives) assumes all the inventoried >>objects in an archive should have the same types of metadata. With >>the Emergency Response Imagery collection, the "Responsible Party" >>field is sort of "Dept. of Commerce, NOAA, National Ocean Survey, >>National Geodetic Survey, Emergency Response Imagery Project" >>and its the same for every image or zipped image file. At least in >>this case, an "Author" metadata field (assuming that Dept. of Commerce >>is a Responsible Party and that a Responsible Party is equivalent >>to an Author) is of NO help at distinguishing which of the quarter >>million files you might want to get. In FCA, the algorithms would >>ignore the field if applied to the whole collection. >> >>Some care would appear to be appropriate. >> >>Bruce b. >> >>On Sat, Jun 22, 2013 at 1:31 PM, Chris A. Mattmann (JIRA) >><j...@apache.org>wrote: >> >>> >>> [ >>> >>>https://issues.apache.org/jira/browse/OODT-639?page=com.atlassian.jira.p >>>l >>>ugin.system.issuetabpanels:all-tabpanel] >>> >>> Chris A. Mattmann resolved OODT-639. >>> ------------------------------------ >>> >>> Resolution: Fixed >>> >>> - fixed in r1495761. Added unit test and doc updates to all product >>>types >>> suggesting how to use the new versioner. >>> >>> > Add a versioner based on Product Type Metadata >>> > ---------------------------------------------- >>> > >>> > Key: OODT-639 >>> > URL: >https://issues.apache.org/jira/browse/OODT-639 ><https://issues.apache.org/jira/browse/OODT-639> >>> > Project: OODT >>> > Issue Type: New Feature >>> > Components: file manager >>> > Reporter: Chris A. Mattmann >>> > Assignee: Chris A. Mattmann >>> > Fix For: 0.6 >>> > >>> > >>> > Add a versioner that allows users to input the filePathSpec for the >>> MetadataBasedVersioner using ProductTypeMetadata, e.g., >>> > {code:xml} >>> > <type id="foo" name="Foo"> >>> > <typeMetadata> >>> > <keyval> >>> > <key>filePathSpec</key> >>> > <val>/[AcquisitionDate]/[Filename]</val> >>> > </keyval> >>> > </typeMetadata> >>> > .. >>> > </type> >>> > {code} >>> >>> -- >>> This message is automatically generated by JIRA. >>> If you think it was sent incorrectly, please contact your JIRA >>> administrators >>> For more information on JIRA, see: >>>http://www.atlassian.com/software/jira >>> > > > > > > >
