Oh absolutely. That's one of the reasons I'm writing all this stuff down :)
Valerie A. Mallder New Horizons Deputy Mission System Engineer Johns Hopkins University/Applied Physics Laboratory > -----Original Message----- > From: Tyler Palsulich [mailto:tpalsul...@gmail.com] > Sent: Wednesday, February 04, 2015 5:03 PM > To: dev > Subject: RE: FW: Tyler - I may need your help > > You're welcome! I'm happy to help. If you get a chance, can you create a wiki > page > with how you set up your deployment? > > Thanks, > Tyler > On Feb 4, 2015 4:44 PM, "Mallder, Valerie" <valerie.mall...@jhuapl.edu> > wrote: > > > Ok, thanks! > > > > > > Valerie A. Mallder > > New Horizons Deputy Mission System Engineer Johns Hopkins > > University/Applied Physics Laboratory > > > > > > > -----Original Message----- > > > From: Tyler Palsulich [mailto:tpalsul...@gmail.com] > > > Sent: Wednesday, February 04, 2015 4:40 PM > > > To: dev > > > Subject: RE: FW: Tyler - I may need your help > > > > > > Hi, > > > > > > Yes, removing the / was intended. Having it broke some of the http tests. > > > > > > Have a good day! > > > Tyler > > > On Feb 4, 2015 3:58 PM, "Mallder, Valerie" > > > <valerie.mall...@jhuapl.edu> > > > wrote: > > > > > > > Hi Tyler, > > > > > > > > I am running back through all of the local changes that I had to > > > > make to OODT 0.8 just to make sure I had written them all down. > > > > And I noticed that the change to MimeTypeUtils.java is different > > > > now than it was on Jan 24 when you sent me the link below. In the > > > > version of the change that I looked at on Jan 24, line 64 had been > > > > changed by adding a '/" at the front of the filename on that line. > > > > But, now that "/" is > > gone. (i.e. > > > > "/tika-mimetypes.xml" is now "tika-mimetypes.xml." Can you verify > > > > that removing the "/" was intended? > > > > > > > > Thanks, > > > > Val > > > > > > > > > > > > > > > > Valerie A. Mallder > > > > New Horizons Deputy Mission System Engineer Johns Hopkins > > > > University/Applied Physics Laboratory > > > > > > > > > -----Original Message----- > > > > > From: Tyler Palsulich [mailto:tpalsul...@gmail.com] > > > > > Sent: Saturday, January 24, 2015 11:02 AM > > > > > To: dev > > > > > Subject: Re: FW: Tyler - I may need your help > > > > > > > > > > Hi Val, > > > > > > > > > > Please see the update for OODT-805 here < > > > > https://github.com/apache/oodt/commit/a44c0e871f4db4af645f00533b00 > > > > de1e > > > > 91d > > > > > 5bcc3> > > > > > . > > > > > > > > > > Re: Radix, is MimeTypeUtils used in Radix? I don't see any > > > > > references to > > > > it. > > > > > > > > > > Tyler > > > > > > > > > > On Fri, Jan 23, 2015 at 4:02 PM, Mallder, Valerie < > > > > valerie.mall...@jhuapl.edu> > > > > > wrote: > > > > > > > > > > > Hi Tyler, > > > > > > > > > > > > Yes, this fix did take care of my problem. Thanks so much! > > > > > > > > > > > > Chris, if you want to make a new OODT 0.8.1, be sure to also > > > > > > include the fix for OODT-805 below in the radix installation. > > > > > > My system is back up and running now. > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Val > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Valerie A. Mallder > > > > > > New Horizons Deputy Mission System Engineer Johns Hopkins > > > > > > University/Applied Physics Laboratory > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: Tyler Palsulich [mailto:tpalsul...@gmail.com] > > > > > > > Sent: Thursday, January 22, 2015 10:35 PM > > > > > > > To: dev > > > > > > > Subject: Re: FW: Tyler - I may need your help > > > > > > > > > > > > > > Hi Val, > > > > > > > > > > > > > > Please see OODT-805 and > > > > > > > > > > > > > https://github.com/apache/oodt/commit/cf1220d4ac66ccefc8e510c6 > > > > > > 2fb6 > > > > > > b38c > > > > > > f529f > > > > > > > fb2 > > > > > > > for what I believe is the fix. > > > > > > > > > > > > > > Can you make the MimeTypeUtils changes locally or try out trunk? > > > > > > > > > > > > > > Let me know! > > > > > > > Tyler > > > > > > > > > > > > > > On Thu, Jan 22, 2015 at 5:40 PM, Tyler Palsulich > > > > > > > <tpalsul...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > Hi Val, > > > > > > > > > > > > > > > > Yes, I think you've hit the nail on the head -- if Tika > > > > > > > > isn't passed your updated mimetypes configuration file > > > > > > > > (with your custom types), then those files will not be > > > > > > > > properly identified. I'll look into this issue more > > > > > > > > tonight and hopefully find a fix. :) > > > > > > > > > > > > > > > > > by default tika only knows about xml files, text files, > > > > > > > > application/octet-stream files. > > > > > > > > I'm not sure what you mean by this? Tika knows about much > > > > > > > > more than that, but is there an OODT config that overrides that? > > > > > > > > > > > > > > > > > I'm a newbie with Java and I can't guarantee I would be > > > > > > > > > able to build a > > > > > > > > JUnit test program very easily. But I will continue to > > > > > > > > investigate and see what I can do. > > > > > > > > No worries! :) If you have time and want to try your hand > > > > > > > > at it, the best way to learn is by looking at the existing > > > > > > > > tests, like in > > > > > > > > https://github.com/apache/oodt/blob/trunk/metadata/src/tes > > > > > > > > t/or g/ap ache > > > > > > > > /oodt/cas/metadata/util/TestMimeTypeUtils.java > > > > > > > > . > > > > > > > > > > > > > > > > Have a good night, > > > > > > > > Tyler > > > > > > > > > > > > > > > > On Thu, Jan 22, 2015 at 2:22 PM, Mallder, Valerie < > > > > > > > > valerie.mall...@jhuapl.edu> wrote: > > > > > > > > > > > > > > > >> Hi Tyler, > > > > > > > >> > > > > > > > >> Can you tell me more about the tika-mimetypes.xml file? > > > > > > > >> Is this a new 'required' file? I'm not 100% sure about > > > > > > > >> this yet, but it seems to me that, since > > > > > > > >> MimeTypeUtils.java instantiates Tika with the default > > > > > > > >> constructor, and never explicitly tells Tika which > > > > > > > >> mime-types file to use (even though the correct > > > > > > > >> mime-types.xml file is passed to the MimeTypeUtils > > > > > > > >> constructor from > > > > > > > >> MimeExtractorRepo) there is no place where the contents > > > > > > > >> of my mime-types.xml file is being read and stored in the > > > > > > > >> Tika's MimeTypeRegistry, and by default tika only knows > > > > > > > >> about xml files, > > > > text > > > > > files, application/octet-stream files. > > > > > > > >> > > > > > > > >> I will keep looking at this tomorrow and verify which the > > > > > > > >> file that is passed to the Tika's MimeTypesFactory class, > > > > > > > >> but I have to head > > > > > > home now. > > > > > > > >> > > > > > > > >> Val > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> Valerie A. Mallder > > > > > > > >> New Horizons Deputy Mission System Engineer Johns Hopkins > > > > > > > >> University/Applied Physics Laboratory > > > > > > > >> > > > > > > > >> > > > > > > > >> -----Original Message----- > > > > > > > >> From: Mallder, Valerie > > > > > > > >> Sent: Thursday, January 22, 2015 11:42 AM > > > > > > > >> To: dev > > > > > > > >> Subject: RE: Tyler - I may need your help > > > > > > > >> > > > > > > > >> Hi Tyler, > > > > > > > >> > > > > > > > >> I have defined a few custom mime types in my > > > > > > > >> filemgr/etc/mime-types.xml file. The contents of my file > > > > > > > >> looks exactly like the contents of > > > > > > > >> http://svn.apache.org/viewvc/oodt/tags/0.8/filemgr/src/ma > > > > > > > >> in/r esou rces /mime-types.xml with the addition of > > > > > > > >> project-specific mime-types . > > > > > > > >> The tika-mimetypes.xml file you pointed me to has ~2000 > > > > > > > >> additional lines in it as compared to the > > > > > > > >> http://svn.apache.org/viewvc/oodt/tags/0.8/filemgr/src/ma > > > > > > > >> in/r > > > > > > > >> esou > > > > > > > >> rces > > > > > > > >> /mime-types.xml > > > > > > > >> file and the > > > > > > > >> http://svn.apache.org/viewvc/oodt/tags/0.8/mvn/archetypes > > > > > > > >> /rad > > > > > > > >> ix/s > > > > > > > >> rc/m > > > > > > > >> ain/resources/archetype-resources/filemgr/src/main/resour > > > > > > > >> ces/ etc/ mime -types.xml file. So, it is definitely > > > > > > > >> different than the one I've been using. But, I copied it > > > > > > > >> over and added my mime types to it, and it didn't help. > > > > > > > >> The mime types it is returning are 'reasonable' > > > > > > > >> mime-types to return, they are just not the mime-types > > > > > > > >> that I defined them as. For instance, I have *.sfdu > > > > > > > >> files and *.out files that contain binary data, and tika > > > > > > > >> says they are "application/octet-stream" files. I also > > > > > > > >> have *.ecsv files that contain text, and tika says they are > "text/plain" files. > > > > > > > >> > > > > > > > >> But here are the mime-types I defined for these files for > > > > > > > >> my project, and these are the mime-types that have > > > > > > > >> defined extractors for. None of these filename > > > > > > > >> extensions "*.out, *.ecsv, and *.sfdu" are defined > > > > > > > >> elsewhere in the mime-types.xml > > > > file. > > > > > > > >> > > > > > > > >> <mime-type type="product/fei-out"> > > > > > > > >> <glob pattern="*.out"/> </mime-type> > > > > > > > >> > > > > > > > >> <mime-type type="product/fei-ecsv"> > > > > > > > >> <glob pattern="*.ecsv"/> </mime-type> > > > > > > > >> > > > > > > > >> <mime-type type="product/fei-sfdu"> > > > > > > > >> <glob pattern="*.sfdu"/> </mime-type> > > > > > > > >> > > > > > > > >> I'm a newbie with Java and I can't guarantee I would be > > > > > > > >> able to build a JUnit test program very easily. But I > > > > > > > >> will continue to investigate and see what I can do. > > > > > > > >> > > > > > > > >> Thanks! > > > > > > > >> > > > > > > > >> Val > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> Valerie A. Mallder > > > > > > > >> New Horizons Deputy Mission System Engineer Johns Hopkins > > > > > > > >> University/Applied Physics Laboratory > > > > > > > >> > > > > > > > >> > > > > > > > >> > -----Original Message----- > > > > > > > >> > From: Tyler Palsulich [mailto:tpalsul...@gmail.com] > > > > > > > >> > Sent: Wednesday, January 21, 2015 5:13 PM > > > > > > > >> > To: dev > > > > > > > >> > Subject: Re: Tyler - I may need your help > > > > > > > >> > > > > > > > > >> > Hi Val, > > > > > > > >> > > > > > > > > >> > Hmm... Is there a particular (wrong) mime-type that > > > > > > > >> > keeps getting detected (like text/plain, or something)? > > > > > > > >> > I'm curious if the type is just returning a default. > > > > > > > >> > Or, is it a seemingly random file type? What > > > > > > > >> are the contents of your mime-types.xml file? > > > > > > > >> > If it's different than > > > > > > > >> > https://raw.githubusercontent.com/apache/tika/trunk/tik > > > > > > > >> > a- > > > > > > > >> > core/src/main/resources/org/apache/tika/mime/tika-mimet > > > > > > > >> > ypes > > > > > > > >> > .xml > > > > > > > >> > , > > > > > > > >> > can you try copying it over? > > > > > > > >> > > > > > > > > >> > I'm not sure I'll be able to replicate your error on my > > > > > > > >> > computer without a bit of difficulty. Do you think > > > > > > > >> > there is any way you could create a JUnit test case with the > problem? > > > > > > > >> > > > > > > > > >> > Tyler > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > On Wed, Jan 21, 2015 at 1:26 PM, Mallder, Valerie < > > > > > > > >> > valerie.mall...@jhuapl.edu> > > > > > > > >> > wrote: > > > > > > > >> > > > > > > > > >> > > Hi Tyler, > > > > > > > >> > > > > > > > > > >> > > I'm have been looking into an issue that cropped up > > > > > > > >> > > in my OODT system when I upgraded to OODT 0.8. The > > > > > > > >> > > issue is, my AutoDetectProductCrawler, which is > > > > > > > >> > > launched from a PGETaskInstance is unable to > > > > > > > >> > > determine the mime-type for my product files. I am > > > > > > > >> > > using the same filemgr/etc/mime-types.xml file that I > > > > > > > >> > > was using with OODT 0.7, and I am using the same > > > > > > > >> > > oodt/extensions/policy/mime-extractor-map.xml file > > > > > > > >> > > that I was using with OODT 0.7, but now, in > > > > > > > >> > > MimeTypeRepo::getExtractorSpecsForFile, > > > > > > > >> > > the call to > > > > > > > >> > > this.mimeRepo.getMimeType(file) is returning the > > > > > > > >> > > wrong mime-types for all of my files, and so the > > > > > > > >> > > AutoDetectProductCrawler is telling me I have no > > > > > > > >> > > extractor > > > > specs for > > > > > my files. > > > > > > > >> > > > > > > > > > >> > > I noticed that you did some work on MimeTypeUtils for > > > > > > > >> > > OODT-630 in OODT 0.8. At first glance, it doesn't' > > > > > > > >> > > look like any of this work would be directly > > > > > > > >> > > responsible. Can you think of anything that might be > > > > > > > >> > > causing this to happen? I don't know anything about > > > > > > > >> > > tika. Do I need to make any changes to my policy > > > > > > > >> > > files to remain > > > > > > > compatible. > > > > > > > >> > > Just looking for clues on how to resolve this. I > > > > > > > >> > > have verified by adding log messages throughout the > > > > > > > >> > > code that, prior to launching the > > > > > > > >> > > AutoDetectProductCrawler, all of the policy files > > > > > > are read > > > > > > > correctly. > > > > > > > >> > > The MimeExtractorConfigReader is reading the correct > > > > > > > >> > > mim-extractor-map.xml file, and it is calling > > > > > > > >> > > setMimeRepoFile with the correct mime-types.xml file, > > > > > > > >> > > and it is setting the correct extractor config file, etc. > > > > > > > >> > > But, once AutoDetectProductCrawler starts crawling it > > > > > > > >> > > try to getExtractorSpecsForFile but determines the > > > > > > > >> > > wrong mime type and then > > > > > > > >> > can't find the extractor spec. > > > > > > > >> > > > > > > > > > >> > > Thanks, > > > > > > > >> > > Val > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > Valerie A. Mallder > > > > > > > >> > > > > > > > > > >> > > New Horizons Deputy Mission System Engineer The Johns > > > > > > > >> > > Hopkins University/Applied Physics Laboratory > > > > > > > >> > > 11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723 > > > > > > > >> > > 240-228-7846 (Office) 410-504-2233 (Blackberry) > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > >
