Re: Deploying SOLR (was:Re: Fail to ingest a test file to file manager on SuSe Linux)
Wei, if you want to test the 0.12-SNAPSHOT radix again you should get time stamps now. Tom On Wed, Feb 10, 2016 at 6:25 PM, Tom Barberwrote: > Hey Wei: > > https://dl.dropboxusercontent.com/u/8503756/opsui-patch.png > > Will that do you? ;) > > I have a patch, I'll dump it into trunk and if Chris doesn't like it I may > have to tweak it, but I think there was a minor disconnect between the Solr > deserializer and the non solr stuff. > > Tom > > On Wed, Feb 10, 2016 at 6:55 AM, Tom Barber > wrote: > >> Okay cool. I'm not familiar with that part of the code base, but I'll >> prod around and see what I can come up with. >> >> Tom >> >> On Wed, Feb 10, 2016 at 12:11 AM, Shi, Wei wrote: >> >>> Hi Tom, >>> The installation went very well. I tested ingestion and deletion, both >>> worked as expected. >>> >>> The only thing I noticed is ³Received Time² displayed as ³UNKNOWN² after >>> the file is ingested. >>> >>> Thanks, >>> >>> Wei >>> >>> On 2/9/16, 6:08 PM, "Tom Barber" wrote: >>> >>> >Okay Wei, Jordan >>> > >>> >Can you guys test this for me whilst I pass out: >>> > >>> >mvn -DarchetypeGroupId=org.apache.oodt >>> >-DarchetypeArtifactId=radix-archetype -DarchetypeVersion=0.12-SNAPSHOT >>> >-Doodt=0.12-SNAPSHOT -DgroupId=com.mycompany -DartifactId=oodt >>> >-Dversion=0.1 -DarchetypeRepository= >>> >https://repository.apache.org/content/groups/snapshots/ >>> archetype:generate >>> >-Pfm-solr-catalog >>> > >>> >mv oodt oodt-src; cd oodt-src; mvn clean package -Pfm-solr-catalog >>> > >>> >mkdir ../oodt; tar -xvf >>> >distribution/target/oodt-distribution-0.1-bin.tar.gz -C ../oodt >>> > >>> >cd ../oodt; ./bin/oodt start >>> > >>> > >>> >Its 0.12 RADIX with the start script patched so SOLR starts. >>> > >>> >Notice the -P profile flag for solr, it should deploy your solr config >>> >with >>> >no extra work. >>> > >>> >Let me know how it goes, I've not tested ingestion but I have verified >>> >that >>> >solr and the filemgr are running. >>> > >>> >Tom >>> > >>> >On Tue, Feb 9, 2016 at 9:17 PM, Tom Barber >>> >wrote: >>> > >>> >> You don't need vagrant if you don't want it and you can use maven 3 >>> BTW >>> >> On 9 Feb 2016 22:13, "Tom Barber" wrote: >>> >> >>> >>> Blimey! >>> >>> >>> >>> Okay you guys are mostly working. I'm gonna fix the solr maven >>> profile >>> >>> before I cut 0.12 because it mostly works and it's a lot easier than >>> >>>that >>> >>> solr tutorial! :) >>> >>> On 9 Feb 2016 22:06, "Shi, Wei" wrote: >>> >>> >>> Here is a list I compiled: >>> >>> 1. Download and Install VirtualBox 5.0 >>> 2. Install Vagrant on Windows 7. It will be installed in >>> VirtualBox >>> >>> >>> >>> http://www.seascapewebdesign.com/blog/part-1-getting-started-vagrant-wi >>> ndows-7-and-8 >>> 3. Install Java >>> >>> >>> >>> https://www.digitalocean.com/community/tutorials/how-to-install-java-on >>> -ubuntu-with-apt-get >>> 4. Install Maven 2.2.1 (download from maven site cause the blog >>> provided link is not working) >>> >>> >>> >>> http://archive.apache.org/dist/maven/binaries/apache-maven-2.2.1-bin.ta >>> r.gz >>> >>> >>> >>> https://saeedkhattak.wordpress.com/2013/03/05/install-maven-in-ubuntu-1 >>> 2-04/ >>> 5. Install OODT >>> >>> https://cwiki.apache.org/confluence/display/OODT/RADiX+Powered+By+OODT >>> 6. You may need to open port 8080 to allow guest to access >>> >>> >>> >>> http://serverfault.com/questions/248568/how-to-open-a-8080-port-for-an- >>> application-with-iptables >>> 7. Recompile filemgr in 0.12 and copy over, Note: logs folder >>> is >>> missing. Create logs folder under filemgr >>> 8. File Manger User Guide: >>> >>> >>> https://cwiki.apache.org/confluence/display/OODT/OODT+Filemgr+User+Guid >>> e >>> 9. Solr >>> >>> >>> >>> https://cwiki.apache.org/confluence/display/OODT/Solr+File+Manager+Quic >>> k+Start+Guide >>> 10. Curator Setup >>> >>> >>> >>> http://svn.apache.org/repos/asf/oodt/site/components/maven/curator/user >>> /basic.html >>> >>> >>> >>> -Original Message- >>> From: Tom Barber [mailto:tom.bar...@meteorite.bi] >>> Sent: Tuesday, February 09, 2016 3:28 PM >>> To: dev@oodt.apache.org >>> Subject: Deploying SOLR (was:Re: Fail to ingest a test file to file >>> manager on SuSe Linux) >>> >>> Out of curiosity, Wei and Jordan if he checks this thread, what >>> instructions/hacks are you using to deploy OODT & SOLR? >>> >>> I know how I do it, I wondered how others do it, ie, what docs do >>> you >>> follow? >>> >>> Cheers >>> >>> Tom >>>
Re: Deploying SOLR (was:Re: Fail to ingest a test file to file manager on SuSe Linux)
Hey Wei: https://dl.dropboxusercontent.com/u/8503756/opsui-patch.png Will that do you? ;) I have a patch, I'll dump it into trunk and if Chris doesn't like it I may have to tweak it, but I think there was a minor disconnect between the Solr deserializer and the non solr stuff. Tom On Wed, Feb 10, 2016 at 6:55 AM, Tom Barberwrote: > Okay cool. I'm not familiar with that part of the code base, but I'll prod > around and see what I can come up with. > > Tom > > On Wed, Feb 10, 2016 at 12:11 AM, Shi, Wei wrote: > >> Hi Tom, >> The installation went very well. I tested ingestion and deletion, both >> worked as expected. >> >> The only thing I noticed is ³Received Time² displayed as ³UNKNOWN² after >> the file is ingested. >> >> Thanks, >> >> Wei >> >> On 2/9/16, 6:08 PM, "Tom Barber" wrote: >> >> >Okay Wei, Jordan >> > >> >Can you guys test this for me whilst I pass out: >> > >> >mvn -DarchetypeGroupId=org.apache.oodt >> >-DarchetypeArtifactId=radix-archetype -DarchetypeVersion=0.12-SNAPSHOT >> >-Doodt=0.12-SNAPSHOT -DgroupId=com.mycompany -DartifactId=oodt >> >-Dversion=0.1 -DarchetypeRepository= >> >https://repository.apache.org/content/groups/snapshots/ >> archetype:generate >> >-Pfm-solr-catalog >> > >> >mv oodt oodt-src; cd oodt-src; mvn clean package -Pfm-solr-catalog >> > >> >mkdir ../oodt; tar -xvf >> >distribution/target/oodt-distribution-0.1-bin.tar.gz -C ../oodt >> > >> >cd ../oodt; ./bin/oodt start >> > >> > >> >Its 0.12 RADIX with the start script patched so SOLR starts. >> > >> >Notice the -P profile flag for solr, it should deploy your solr config >> >with >> >no extra work. >> > >> >Let me know how it goes, I've not tested ingestion but I have verified >> >that >> >solr and the filemgr are running. >> > >> >Tom >> > >> >On Tue, Feb 9, 2016 at 9:17 PM, Tom Barber >> >wrote: >> > >> >> You don't need vagrant if you don't want it and you can use maven 3 BTW >> >> On 9 Feb 2016 22:13, "Tom Barber" wrote: >> >> >> >>> Blimey! >> >>> >> >>> Okay you guys are mostly working. I'm gonna fix the solr maven >> profile >> >>> before I cut 0.12 because it mostly works and it's a lot easier than >> >>>that >> >>> solr tutorial! :) >> >>> On 9 Feb 2016 22:06, "Shi, Wei" wrote: >> >>> >> Here is a list I compiled: >> >> 1. Download and Install VirtualBox 5.0 >> 2. Install Vagrant on Windows 7. It will be installed in >> VirtualBox >> >> >> >> http://www.seascapewebdesign.com/blog/part-1-getting-started-vagrant-wi >> ndows-7-and-8 >> 3. Install Java >> >> >> >> https://www.digitalocean.com/community/tutorials/how-to-install-java-on >> -ubuntu-with-apt-get >> 4. Install Maven 2.2.1 (download from maven site cause the blog >> provided link is not working) >> >> >> >> http://archive.apache.org/dist/maven/binaries/apache-maven-2.2.1-bin.ta >> r.gz >> >> >> >> https://saeedkhattak.wordpress.com/2013/03/05/install-maven-in-ubuntu-1 >> 2-04/ >> 5. Install OODT >> >> https://cwiki.apache.org/confluence/display/OODT/RADiX+Powered+By+OODT >> 6. You may need to open port 8080 to allow guest to access >> >> >> >> http://serverfault.com/questions/248568/how-to-open-a-8080-port-for-an- >> application-with-iptables >> 7. Recompile filemgr in 0.12 and copy over, Note: logs folder is >> missing. Create logs folder under filemgr >> 8. File Manger User Guide: >> >> >> https://cwiki.apache.org/confluence/display/OODT/OODT+Filemgr+User+Guid >> e >> 9. Solr >> >> >> >> https://cwiki.apache.org/confluence/display/OODT/Solr+File+Manager+Quic >> k+Start+Guide >> 10. Curator Setup >> >> >> >> http://svn.apache.org/repos/asf/oodt/site/components/maven/curator/user >> /basic.html >> >> >> >> -Original Message- >> From: Tom Barber [mailto:tom.bar...@meteorite.bi] >> Sent: Tuesday, February 09, 2016 3:28 PM >> To: dev@oodt.apache.org >> Subject: Deploying SOLR (was:Re: Fail to ingest a test file to file >> manager on SuSe Linux) >> >> Out of curiosity, Wei and Jordan if he checks this thread, what >> instructions/hacks are you using to deploy OODT & SOLR? >> >> I know how I do it, I wondered how others do it, ie, what docs do you >> follow? >> >> Cheers >> >> Tom >> ** >> Electronic Mail is not secure, may not be read every day, and should >> not >> be used for urgent or sensitive issues >> >> >>> >> >> ** >> Electronic Mail is not secure, may not be read every day, and should
Re: Deploying SOLR (was:Re: Fail to ingest a test file to file manager on SuSe Linux)
If this is testing well on Jenkins I am +1 to rolling the release candidate. Good work Tom. On Wed, Feb 10, 2016 at 12:03 PM, Shi, Weiwrote: > Thank you Tom! > > Wei > > -Original Message- > From: Tom Barber [mailto:tom.bar...@meteorite.bi] > Sent: Wednesday, February 10, 2016 2:01 PM > To: dev@oodt.apache.org > Subject: Re: Deploying SOLR (was:Re: Fail to ingest a test file to file > manager on SuSe Linux) > > Wei, if you want to test the 0.12-SNAPSHOT radix again you should get time > stamps now. > > Tom > > On Wed, Feb 10, 2016 at 6:25 PM, Tom Barber > wrote: > > > Hey Wei: > > > > https://dl.dropboxusercontent.com/u/8503756/opsui-patch.png > > > > Will that do you? ;) > > > > I have a patch, I'll dump it into trunk and if Chris doesn't like it I > > may have to tweak it, but I think there was a minor disconnect between > > the Solr deserializer and the non solr stuff. > > > > Tom > > > > On Wed, Feb 10, 2016 at 6:55 AM, Tom Barber > > wrote: > > > >> Okay cool. I'm not familiar with that part of the code base, but I'll > >> prod around and see what I can come up with. > >> > >> Tom > >> > >> On Wed, Feb 10, 2016 at 12:11 AM, Shi, Wei > wrote: > >> > >>> Hi Tom, > >>> The installation went very well. I tested ingestion and deletion, > >>> both worked as expected. > >>> > >>> The only thing I noticed is ³Received Time² displayed as ³UNKNOWN² > >>> after the file is ingested. > >>> > >>> Thanks, > >>> > >>> Wei > >>> > >>> On 2/9/16, 6:08 PM, "Tom Barber" wrote: > >>> > >>> >Okay Wei, Jordan > >>> > > >>> >Can you guys test this for me whilst I pass out: > >>> > > >>> >mvn -DarchetypeGroupId=org.apache.oodt > >>> >-DarchetypeArtifactId=radix-archetype -DarchetypeVersion=0.12-SNAPSHOT > >>> >-Doodt=0.12-SNAPSHOT -DgroupId=com.mycompany -DartifactId=oodt > >>> >-Dversion=0.1 -DarchetypeRepository= > >>> >https://repository.apache.org/content/groups/snapshots/ > >>> archetype:generate > >>> >-Pfm-solr-catalog > >>> > > >>> >mv oodt oodt-src; cd oodt-src; mvn clean package -Pfm-solr-catalog > >>> > > >>> >mkdir ../oodt; tar -xvf > >>> >distribution/target/oodt-distribution-0.1-bin.tar.gz -C ../oodt > >>> > > >>> >cd ../oodt; ./bin/oodt start > >>> > > >>> > > >>> >Its 0.12 RADIX with the start script patched so SOLR starts. > >>> > > >>> >Notice the -P profile flag for solr, it should deploy your solr > >>> >config with no extra work. > >>> > > >>> >Let me know how it goes, I've not tested ingestion but I have > >>> >verified that solr and the filemgr are running. > >>> > > >>> >Tom > >>> > > >>> >On Tue, Feb 9, 2016 at 9:17 PM, Tom Barber > >>> > > >>> >wrote: > >>> > > >>> >> You don't need vagrant if you don't want it and you can use maven > >>> >> 3 > >>> BTW > >>> >> On 9 Feb 2016 22:13, "Tom Barber" wrote: > >>> >> > >>> >>> Blimey! > >>> >>> > >>> >>> Okay you guys are mostly working. I'm gonna fix the solr maven > >>> profile > >>> >>> before I cut 0.12 because it mostly works and it's a lot easier > >>> >>>than that solr tutorial! :) On 9 Feb 2016 22:06, "Shi, Wei" > >>> >>> wrote: > >>> >>> > >>> Here is a list I compiled: > >>> > >>> 1. Download and Install VirtualBox 5.0 > >>> 2. Install Vagrant on Windows 7. It will be installed in > >>> VirtualBox > >>> > >>> > >>> > >>> http://www.seascapewebdesign.com/blog/part-1-getting-started-vagrant > >>> -wi > >>> ndows-7-and-8 > >>> 3. Install Java > >>> > >>> > >>> > >>> https://www.digitalocean.com/community/tutorials/how-to-install-java > >>> -on > >>> -ubuntu-with-apt-get > >>> 4. Install Maven 2.2.1 (download from maven site cause the > blog > >>> provided link is not working) > >>> > >>> > >>> > >>> http://archive.apache.org/dist/maven/binaries/apache-maven-2.2.1-bin > >>> .ta > >>> r.gz > >>> > >>> > >>> > >>> https://saeedkhattak.wordpress.com/2013/03/05/install-maven-in-ubunt > >>> u-1 > >>> 2-04/ > >>> 5. Install OODT > >>> > >>> https://cwiki.apache.org/confluence/display/OODT/RADiX+Powered+By+OO > >>> DT > >>> 6. You may need to open port 8080 to allow guest to access > >>> > >>> > >>> > >>> http://serverfault.com/questions/248568/how-to-open-a-8080-port-for- > >>> an- > >>> application-with-iptables > >>> 7. Recompile filemgr in 0.12 and copy over, Note: logs folder > >>> is > >>> missing. Create logs folder under filemgr > >>> 8. File Manger User Guide: > >>> > >>> > >>> https://cwiki.apache.org/confluence/display/OODT/OODT+Filemgr+User+G > >>> uid > >>> e > >>> 9. Solr > >>> > >>> > >>> > >>> https://cwiki.apache.org/confluence/display/OODT/Solr+File+Manager+Q > >>> uic > >>>
Re: Deploying SOLR (was:Re: Fail to ingest a test file to file manager on SuSe Linux)
The Jenkins tests threw up but thats some weird transient smtp issue. I'm just checking OODT-992. re the build, I'll run it on minotaur and check and if it works cut RC1 On Wed, Feb 10, 2016 at 8:59 PM, Lewis John Mcgibbney < lewis.mcgibb...@gmail.com> wrote: > If this is testing well on Jenkins I am +1 to rolling the release > candidate. > Good work Tom. > > On Wed, Feb 10, 2016 at 12:03 PM, Shi, Weiwrote: > > > Thank you Tom! > > > > Wei > > > > -Original Message- > > From: Tom Barber [mailto:tom.bar...@meteorite.bi] > > Sent: Wednesday, February 10, 2016 2:01 PM > > To: dev@oodt.apache.org > > Subject: Re: Deploying SOLR (was:Re: Fail to ingest a test file to file > > manager on SuSe Linux) > > > > Wei, if you want to test the 0.12-SNAPSHOT radix again you should get > time > > stamps now. > > > > Tom > > > > On Wed, Feb 10, 2016 at 6:25 PM, Tom Barber > > wrote: > > > > > Hey Wei: > > > > > > https://dl.dropboxusercontent.com/u/8503756/opsui-patch.png > > > > > > Will that do you? ;) > > > > > > I have a patch, I'll dump it into trunk and if Chris doesn't like it I > > > may have to tweak it, but I think there was a minor disconnect between > > > the Solr deserializer and the non solr stuff. > > > > > > Tom > > > > > > On Wed, Feb 10, 2016 at 6:55 AM, Tom Barber > > > wrote: > > > > > >> Okay cool. I'm not familiar with that part of the code base, but I'll > > >> prod around and see what I can come up with. > > >> > > >> Tom > > >> > > >> On Wed, Feb 10, 2016 at 12:11 AM, Shi, Wei > > wrote: > > >> > > >>> Hi Tom, > > >>> The installation went very well. I tested ingestion and deletion, > > >>> both worked as expected. > > >>> > > >>> The only thing I noticed is ³Received Time² displayed as ³UNKNOWN² > > >>> after the file is ingested. > > >>> > > >>> Thanks, > > >>> > > >>> Wei > > >>> > > >>> On 2/9/16, 6:08 PM, "Tom Barber" wrote: > > >>> > > >>> >Okay Wei, Jordan > > >>> > > > >>> >Can you guys test this for me whilst I pass out: > > >>> > > > >>> >mvn -DarchetypeGroupId=org.apache.oodt > > >>> >-DarchetypeArtifactId=radix-archetype > -DarchetypeVersion=0.12-SNAPSHOT > > >>> >-Doodt=0.12-SNAPSHOT -DgroupId=com.mycompany -DartifactId=oodt > > >>> >-Dversion=0.1 -DarchetypeRepository= > > >>> >https://repository.apache.org/content/groups/snapshots/ > > >>> archetype:generate > > >>> >-Pfm-solr-catalog > > >>> > > > >>> >mv oodt oodt-src; cd oodt-src; mvn clean package -Pfm-solr-catalog > > >>> > > > >>> >mkdir ../oodt; tar -xvf > > >>> >distribution/target/oodt-distribution-0.1-bin.tar.gz -C ../oodt > > >>> > > > >>> >cd ../oodt; ./bin/oodt start > > >>> > > > >>> > > > >>> >Its 0.12 RADIX with the start script patched so SOLR starts. > > >>> > > > >>> >Notice the -P profile flag for solr, it should deploy your solr > > >>> >config with no extra work. > > >>> > > > >>> >Let me know how it goes, I've not tested ingestion but I have > > >>> >verified that solr and the filemgr are running. > > >>> > > > >>> >Tom > > >>> > > > >>> >On Tue, Feb 9, 2016 at 9:17 PM, Tom Barber > > >>> > > > >>> >wrote: > > >>> > > > >>> >> You don't need vagrant if you don't want it and you can use maven > > >>> >> 3 > > >>> BTW > > >>> >> On 9 Feb 2016 22:13, "Tom Barber" > wrote: > > >>> >> > > >>> >>> Blimey! > > >>> >>> > > >>> >>> Okay you guys are mostly working. I'm gonna fix the solr maven > > >>> profile > > >>> >>> before I cut 0.12 because it mostly works and it's a lot easier > > >>> >>>than that solr tutorial! :) On 9 Feb 2016 22:06, "Shi, Wei" > > >>> >>> wrote: > > >>> >>> > > >>> Here is a list I compiled: > > >>> > > >>> 1. Download and Install VirtualBox 5.0 > > >>> 2. Install Vagrant on Windows 7. It will be installed in > > >>> VirtualBox > > >>> > > >>> > > >>> > > >>> http://www.seascapewebdesign.com/blog/part-1-getting-started-vagrant > > >>> -wi > > >>> ndows-7-and-8 > > >>> 3. Install Java > > >>> > > >>> > > >>> > > >>> https://www.digitalocean.com/community/tutorials/how-to-install-java > > >>> -on > > >>> -ubuntu-with-apt-get > > >>> 4. Install Maven 2.2.1 (download from maven site cause the > > blog > > >>> provided link is not working) > > >>> > > >>> > > >>> > > >>> http://archive.apache.org/dist/maven/binaries/apache-maven-2.2.1-bin > > >>> .ta > > >>> r.gz > > >>> > > >>> > > >>> > > >>> https://saeedkhattak.wordpress.com/2013/03/05/install-maven-in-ubunt > > >>> u-1 > > >>> 2-04/ > > >>> 5. Install OODT > > >>> > > >>> https://cwiki.apache.org/confluence/display/OODT/RADiX+Powered+By+OO > > >>> DT > > >>> 6. You may need to open port 8080 to allow guest to access > > >>> > > >>> > > >>> > > >>>
Re: Deploying SOLR (was:Re: Fail to ingest a test file to file manager on SuSe Linux)
Okay the count is wrong, as I have 1 MP3 file and 2 Generic files and both flag 3. I'll figuer out OODT-992 before shipping. On Wed, Feb 10, 2016 at 9:04 PM, Tom Barberwrote: > The Jenkins tests threw up but thats some weird transient smtp issue. I'm > just checking OODT-992. re the build, I'll run it on minotaur and check and > if it works cut RC1 > > On Wed, Feb 10, 2016 at 8:59 PM, Lewis John Mcgibbney < > lewis.mcgibb...@gmail.com> wrote: > >> If this is testing well on Jenkins I am +1 to rolling the release >> candidate. >> Good work Tom. >> >> On Wed, Feb 10, 2016 at 12:03 PM, Shi, Wei wrote: >> >> > Thank you Tom! >> > >> > Wei >> > >> > -Original Message- >> > From: Tom Barber [mailto:tom.bar...@meteorite.bi] >> > Sent: Wednesday, February 10, 2016 2:01 PM >> > To: dev@oodt.apache.org >> > Subject: Re: Deploying SOLR (was:Re: Fail to ingest a test file to file >> > manager on SuSe Linux) >> > >> > Wei, if you want to test the 0.12-SNAPSHOT radix again you should get >> time >> > stamps now. >> > >> > Tom >> > >> > On Wed, Feb 10, 2016 at 6:25 PM, Tom Barber >> > wrote: >> > >> > > Hey Wei: >> > > >> > > https://dl.dropboxusercontent.com/u/8503756/opsui-patch.png >> > > >> > > Will that do you? ;) >> > > >> > > I have a patch, I'll dump it into trunk and if Chris doesn't like it I >> > > may have to tweak it, but I think there was a minor disconnect between >> > > the Solr deserializer and the non solr stuff. >> > > >> > > Tom >> > > >> > > On Wed, Feb 10, 2016 at 6:55 AM, Tom Barber >> > > wrote: >> > > >> > >> Okay cool. I'm not familiar with that part of the code base, but I'll >> > >> prod around and see what I can come up with. >> > >> >> > >> Tom >> > >> >> > >> On Wed, Feb 10, 2016 at 12:11 AM, Shi, Wei >> > wrote: >> > >> >> > >>> Hi Tom, >> > >>> The installation went very well. I tested ingestion and deletion, >> > >>> both worked as expected. >> > >>> >> > >>> The only thing I noticed is ³Received Time² displayed as ³UNKNOWN² >> > >>> after the file is ingested. >> > >>> >> > >>> Thanks, >> > >>> >> > >>> Wei >> > >>> >> > >>> On 2/9/16, 6:08 PM, "Tom Barber" wrote: >> > >>> >> > >>> >Okay Wei, Jordan >> > >>> > >> > >>> >Can you guys test this for me whilst I pass out: >> > >>> > >> > >>> >mvn -DarchetypeGroupId=org.apache.oodt >> > >>> >-DarchetypeArtifactId=radix-archetype >> -DarchetypeVersion=0.12-SNAPSHOT >> > >>> >-Doodt=0.12-SNAPSHOT -DgroupId=com.mycompany -DartifactId=oodt >> > >>> >-Dversion=0.1 -DarchetypeRepository= >> > >>> >https://repository.apache.org/content/groups/snapshots/ >> > >>> archetype:generate >> > >>> >-Pfm-solr-catalog >> > >>> > >> > >>> >mv oodt oodt-src; cd oodt-src; mvn clean package -Pfm-solr-catalog >> > >>> > >> > >>> >mkdir ../oodt; tar -xvf >> > >>> >distribution/target/oodt-distribution-0.1-bin.tar.gz -C ../oodt >> > >>> > >> > >>> >cd ../oodt; ./bin/oodt start >> > >>> > >> > >>> > >> > >>> >Its 0.12 RADIX with the start script patched so SOLR starts. >> > >>> > >> > >>> >Notice the -P profile flag for solr, it should deploy your solr >> > >>> >config with no extra work. >> > >>> > >> > >>> >Let me know how it goes, I've not tested ingestion but I have >> > >>> >verified that solr and the filemgr are running. >> > >>> > >> > >>> >Tom >> > >>> > >> > >>> >On Tue, Feb 9, 2016 at 9:17 PM, Tom Barber >> > >>> > >> > >>> >wrote: >> > >>> > >> > >>> >> You don't need vagrant if you don't want it and you can use maven >> > >>> >> 3 >> > >>> BTW >> > >>> >> On 9 Feb 2016 22:13, "Tom Barber" >> wrote: >> > >>> >> >> > >>> >>> Blimey! >> > >>> >>> >> > >>> >>> Okay you guys are mostly working. I'm gonna fix the solr maven >> > >>> profile >> > >>> >>> before I cut 0.12 because it mostly works and it's a lot easier >> > >>> >>>than that solr tutorial! :) On 9 Feb 2016 22:06, "Shi, Wei" >> > >>> >>> wrote: >> > >>> >>> >> > >>> Here is a list I compiled: >> > >>> >> > >>> 1. Download and Install VirtualBox 5.0 >> > >>> 2. Install Vagrant on Windows 7. It will be installed in >> > >>> VirtualBox >> > >>> >> > >>> >> > >>> >> > >>> >> http://www.seascapewebdesign.com/blog/part-1-getting-started-vagrant >> > >>> -wi >> > >>> ndows-7-and-8 >> > >>> 3. Install Java >> > >>> >> > >>> >> > >>> >> > >>> >> https://www.digitalocean.com/community/tutorials/how-to-install-java >> > >>> -on >> > >>> -ubuntu-with-apt-get >> > >>> 4. Install Maven 2.2.1 (download from maven site cause the >> > blog >> > >>> provided link is not working) >> > >>> >> > >>> >> > >>> >> > >>> >> http://archive.apache.org/dist/maven/binaries/apache-maven-2.2.1-bin >> > >>> .ta >> > >>> r.gz >> > >>> >> > >>> >> > >>> >> > >>> >>
Re: Deploying SOLR (was:Re: Fail to ingest a test file to file manager on SuSe Linux)
Boom. Wei, once the latest snapshot has built, you'll get a correct product count in solr as well. Cut o'clock. On Wed, Feb 10, 2016 at 9:05 PM, Tom Barberwrote: > Okay the count is wrong, as I have 1 MP3 file and 2 Generic files and both > flag 3. > > I'll figuer out OODT-992 before shipping. > > On Wed, Feb 10, 2016 at 9:04 PM, Tom Barber > wrote: > >> The Jenkins tests threw up but thats some weird transient smtp issue. I'm >> just checking OODT-992. re the build, I'll run it on minotaur and check and >> if it works cut RC1 >> >> On Wed, Feb 10, 2016 at 8:59 PM, Lewis John Mcgibbney < >> lewis.mcgibb...@gmail.com> wrote: >> >>> If this is testing well on Jenkins I am +1 to rolling the release >>> candidate. >>> Good work Tom. >>> >>> On Wed, Feb 10, 2016 at 12:03 PM, Shi, Wei wrote: >>> >>> > Thank you Tom! >>> > >>> > Wei >>> > >>> > -Original Message- >>> > From: Tom Barber [mailto:tom.bar...@meteorite.bi] >>> > Sent: Wednesday, February 10, 2016 2:01 PM >>> > To: dev@oodt.apache.org >>> > Subject: Re: Deploying SOLR (was:Re: Fail to ingest a test file to file >>> > manager on SuSe Linux) >>> > >>> > Wei, if you want to test the 0.12-SNAPSHOT radix again you should get >>> time >>> > stamps now. >>> > >>> > Tom >>> > >>> > On Wed, Feb 10, 2016 at 6:25 PM, Tom Barber >>> > wrote: >>> > >>> > > Hey Wei: >>> > > >>> > > https://dl.dropboxusercontent.com/u/8503756/opsui-patch.png >>> > > >>> > > Will that do you? ;) >>> > > >>> > > I have a patch, I'll dump it into trunk and if Chris doesn't like it >>> I >>> > > may have to tweak it, but I think there was a minor disconnect >>> between >>> > > the Solr deserializer and the non solr stuff. >>> > > >>> > > Tom >>> > > >>> > > On Wed, Feb 10, 2016 at 6:55 AM, Tom Barber >> > >>> > > wrote: >>> > > >>> > >> Okay cool. I'm not familiar with that part of the code base, but >>> I'll >>> > >> prod around and see what I can come up with. >>> > >> >>> > >> Tom >>> > >> >>> > >> On Wed, Feb 10, 2016 at 12:11 AM, Shi, Wei >>> > wrote: >>> > >> >>> > >>> Hi Tom, >>> > >>> The installation went very well. I tested ingestion and deletion, >>> > >>> both worked as expected. >>> > >>> >>> > >>> The only thing I noticed is ³Received Time² displayed as ³UNKNOWN² >>> > >>> after the file is ingested. >>> > >>> >>> > >>> Thanks, >>> > >>> >>> > >>> Wei >>> > >>> >>> > >>> On 2/9/16, 6:08 PM, "Tom Barber" wrote: >>> > >>> >>> > >>> >Okay Wei, Jordan >>> > >>> > >>> > >>> >Can you guys test this for me whilst I pass out: >>> > >>> > >>> > >>> >mvn -DarchetypeGroupId=org.apache.oodt >>> > >>> >-DarchetypeArtifactId=radix-archetype >>> -DarchetypeVersion=0.12-SNAPSHOT >>> > >>> >-Doodt=0.12-SNAPSHOT -DgroupId=com.mycompany -DartifactId=oodt >>> > >>> >-Dversion=0.1 -DarchetypeRepository= >>> > >>> >https://repository.apache.org/content/groups/snapshots/ >>> > >>> archetype:generate >>> > >>> >-Pfm-solr-catalog >>> > >>> > >>> > >>> >mv oodt oodt-src; cd oodt-src; mvn clean package -Pfm-solr-catalog >>> > >>> > >>> > >>> >mkdir ../oodt; tar -xvf >>> > >>> >distribution/target/oodt-distribution-0.1-bin.tar.gz -C ../oodt >>> > >>> > >>> > >>> >cd ../oodt; ./bin/oodt start >>> > >>> > >>> > >>> > >>> > >>> >Its 0.12 RADIX with the start script patched so SOLR starts. >>> > >>> > >>> > >>> >Notice the -P profile flag for solr, it should deploy your solr >>> > >>> >config with no extra work. >>> > >>> > >>> > >>> >Let me know how it goes, I've not tested ingestion but I have >>> > >>> >verified that solr and the filemgr are running. >>> > >>> > >>> > >>> >Tom >>> > >>> > >>> > >>> >On Tue, Feb 9, 2016 at 9:17 PM, Tom Barber >>> > >>> > >>> > >>> >wrote: >>> > >>> > >>> > >>> >> You don't need vagrant if you don't want it and you can use >>> maven >>> > >>> >> 3 >>> > >>> BTW >>> > >>> >> On 9 Feb 2016 22:13, "Tom Barber" >>> wrote: >>> > >>> >> >>> > >>> >>> Blimey! >>> > >>> >>> >>> > >>> >>> Okay you guys are mostly working. I'm gonna fix the solr maven >>> > >>> profile >>> > >>> >>> before I cut 0.12 because it mostly works and it's a lot easier >>> > >>> >>>than that solr tutorial! :) On 9 Feb 2016 22:06, "Shi, Wei" >>> > >>> >>> wrote: >>> > >>> >>> >>> > >>> Here is a list I compiled: >>> > >>> >>> > >>> 1. Download and Install VirtualBox 5.0 >>> > >>> 2. Install Vagrant on Windows 7. It will be installed in >>> > >>> VirtualBox >>> > >>> >>> > >>> >>> > >>> >>> > >>> >>> http://www.seascapewebdesign.com/blog/part-1-getting-started-vagrant >>> > >>> -wi >>> > >>> ndows-7-and-8 >>> > >>> 3. Install Java >>> > >>> >>> > >>> >>> > >>> >>> > >>> >>> https://www.digitalocean.com/community/tutorials/how-to-install-java >>> > >>> -on >>> >
Re: Fail to ingest a test file to file manager on SuSe Linux
No problems :) On Tue, Feb 9, 2016 at 12:17 PM, Shi, Weiwrote: > Sorry, it was my mistake. I specified repository manager directory wrong. > > Wei > > -Original Message- > From: Lewis John Mcgibbney [mailto:lewis.mcgibb...@gmail.com] > Sent: Tuesday, February 09, 2016 1:11 PM > To: dev@oodt.apache.org > Subject: Re: Fail to ingest a test file to file manager on SuSe Linux > > File manager log should provide more insight into the the exception is as > right now it seems to be too vague for me to debug. > Lewis > > On Tue, Feb 9, 2016 at 9:20 AM, Shi, Wei wrote: > > > Yes, the Solr is on 8080. The version is 4.2.0 > > > > -Original Message- > > From: Lewis John Mcgibbney [mailto:lewis.mcgibb...@gmail.com] > > Sent: Tuesday, February 09, 2016 11:36 AM > > To: dev@oodt.apache.org > > Subject: Re: Fail to ingest a test file to file manager on SuSe Linux > > > > Is your Solr server running on port 8080 or another port? > > Which version of Solr are you running? > > > > On Tue, Feb 9, 2016 at 8:19 AM, Shi, Wei wrote: > > > > > Hi Every, > > > I tried to setup the OODT on SuSe Linux server. The installation > > > went well. I also installed Solr. > > > Now the ingest test failed because of Bad Request. > > > I am using jdk 8. > > > > > > Any ideas? I attached error message below. > > > > > > ./filemgr-client --url http://localhost:9000 --operation > > > --ingestProduct --productName test.txt --productStructure Flat > > > --productTypeName GenericFile --metadataFile > > > file:///tmp/test.txt.met --refs file:///tmp/test.txt > > > > > > log4j:WARN No appenders could be found for logger > > > (org.springframework.context.support.FileSystemXmlApplicationContext). > > > log4j:WARN Please initialize the log4j system properly. > > > Feb 09, 2016 11:05:55 AM > > > org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient > > > INFO: Loading File Manager Configuration Properties from: > > > [../etc/filemgr.properties] > > > Feb 09, 2016 11:05:56 AM > > > org.apache.oodt.cas.filemgr.catalog.solr.SolrCatalog addProduct > > > INFO: Adding product:test.txt > > > Feb 09, 2016 11:05:56 AM > > > org.apache.oodt.cas.filemgr.catalog.solr.SolrClient index > > > INFO: Posting message: > > name="id">b69ee1e0-f9f4-4cf0-80e8-232f74e31d12 > > name="CAS.ProductId">b69ee1e0-f9f4-4cf0-80e8-232f74e31d12 > > el d name="CAS.ProductName">test.txt > > name="CAS.ProductTypeName">GenericFile > > name="CAS.ProductReceivedTime">2016-02-09T11:05:56Z > > name="CAS.ProductTypeId">urn:oodt:GenericFile > > name="CAS.ProductStructure">Flat > > name="CAS.ProductTransferStatus">TRANSFERING to > URL: > > > http://localhost:8080/solr/update > > > Feb 09, 2016 11:05:56 AM > > > org.apache.oodt.cas.filemgr.catalog.solr.SolrClient index > > > SEVERE: HTTP method failed: HTTP/1.1 400 Bad Request Feb 09, 2016 > > > 11:05:56 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager > > > catalogProduct > > > SEVERE: ingestProduct: CatalogException when adding Product: > > > test.txt to > > > Catalog: Message: HTTP method failed: HTTP/1.1 400 Bad Request Feb > > > 09, > > > 2016 11:05:56 AM > > > org.apache.oodt.cas.filemgr.system.XmlRpcFileManager > > > ingestProductCore > > > SEVERE: HTTP method failed: HTTP/1.1 400 Bad Request Feb 09, 2016 > > > 11:05:56 AM > > > org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient > > > ingestProduct > > > SEVERE: Failed to ingest product [ name:test.txt] :java.lang.Exception: > > > org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: > > > Error ingesting product > > > [org.apache.oodt.cas.filemgr.structs.Product@469d4b24] > > > : HTTP method failed: HTTP/1.1 400 Bad Request -- rolling back > > > ingest Feb 09, 2016 11:05:56 AM > > > org.apache.oodt.cas.filemgr.catalog.solr.SolrClient delete > > > INFO: Posting message:id:null to URL: > > > http://localhost:8080/solr/update?commit=true > > > Feb 09, 2016 11:05:56 AM > > > org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction > > > execute > > > SEVERE: java.lang.Exception: > > > org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: > > > Error ingesting product > > > [org.apache.oodt.cas.filemgr.structs.Product@469d4b24] > > > : HTTP method failed: HTTP/1.1 400 Bad Request > > > ERROR: Failed to ingest product 'test.txt' : java.lang.Exception: > > > org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: > > > Error ingesting product > > > [org.apache.oodt.cas.filemgr.structs.Product@469d4b24] > > > : HTTP method failed: HTTP/1.1 400 Bad Request > > > > > > > > > Thanks, > > > > > > Wei Shi, M.B.A., M.S. > > > Architect > > > UM Comprehensive Cancer Center > > > NCRC 100/134 > > > O: 734-763-8440 C: 248-835-8453 > > > shi...@med.umich.edu > > > > > > > > > > > > ** > > > Electronic Mail is not secure, may not be read every day, and should > > > not be used for urgent or sensitive issues >
RE: Fail to ingest a test file to file manager on SuSe Linux
Sorry, it was my mistake. I specified repository manager directory wrong. Wei -Original Message- From: Lewis John Mcgibbney [mailto:lewis.mcgibb...@gmail.com] Sent: Tuesday, February 09, 2016 1:11 PM To: dev@oodt.apache.org Subject: Re: Fail to ingest a test file to file manager on SuSe Linux File manager log should provide more insight into the the exception is as right now it seems to be too vague for me to debug. Lewis On Tue, Feb 9, 2016 at 9:20 AM, Shi, Weiwrote: > Yes, the Solr is on 8080. The version is 4.2.0 > > -Original Message- > From: Lewis John Mcgibbney [mailto:lewis.mcgibb...@gmail.com] > Sent: Tuesday, February 09, 2016 11:36 AM > To: dev@oodt.apache.org > Subject: Re: Fail to ingest a test file to file manager on SuSe Linux > > Is your Solr server running on port 8080 or another port? > Which version of Solr are you running? > > On Tue, Feb 9, 2016 at 8:19 AM, Shi, Wei wrote: > > > Hi Every, > > I tried to setup the OODT on SuSe Linux server. The installation > > went well. I also installed Solr. > > Now the ingest test failed because of Bad Request. > > I am using jdk 8. > > > > Any ideas? I attached error message below. > > > > ./filemgr-client --url http://localhost:9000 --operation > > --ingestProduct --productName test.txt --productStructure Flat > > --productTypeName GenericFile --metadataFile > > file:///tmp/test.txt.met --refs file:///tmp/test.txt > > > > log4j:WARN No appenders could be found for logger > > (org.springframework.context.support.FileSystemXmlApplicationContext). > > log4j:WARN Please initialize the log4j system properly. > > Feb 09, 2016 11:05:55 AM > > org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient > > INFO: Loading File Manager Configuration Properties from: > > [../etc/filemgr.properties] > > Feb 09, 2016 11:05:56 AM > > org.apache.oodt.cas.filemgr.catalog.solr.SolrCatalog addProduct > > INFO: Adding product:test.txt > > Feb 09, 2016 11:05:56 AM > > org.apache.oodt.cas.filemgr.catalog.solr.SolrClient index > > INFO: Posting message: > name="id">b69ee1e0-f9f4-4cf0-80e8-232f74e31d12 > name="CAS.ProductId">b69ee1e0-f9f4-4cf0-80e8-232f74e31d12 > el d name="CAS.ProductName">test.txt > name="CAS.ProductTypeName">GenericFile > name="CAS.ProductReceivedTime">2016-02-09T11:05:56Z > name="CAS.ProductTypeId">urn:oodt:GenericFile > name="CAS.ProductStructure">Flat > name="CAS.ProductTransferStatus">TRANSFERING to URL: > > http://localhost:8080/solr/update > > Feb 09, 2016 11:05:56 AM > > org.apache.oodt.cas.filemgr.catalog.solr.SolrClient index > > SEVERE: HTTP method failed: HTTP/1.1 400 Bad Request Feb 09, 2016 > > 11:05:56 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager > > catalogProduct > > SEVERE: ingestProduct: CatalogException when adding Product: > > test.txt to > > Catalog: Message: HTTP method failed: HTTP/1.1 400 Bad Request Feb > > 09, > > 2016 11:05:56 AM > > org.apache.oodt.cas.filemgr.system.XmlRpcFileManager > > ingestProductCore > > SEVERE: HTTP method failed: HTTP/1.1 400 Bad Request Feb 09, 2016 > > 11:05:56 AM > > org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient > > ingestProduct > > SEVERE: Failed to ingest product [ name:test.txt] :java.lang.Exception: > > org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: > > Error ingesting product > > [org.apache.oodt.cas.filemgr.structs.Product@469d4b24] > > : HTTP method failed: HTTP/1.1 400 Bad Request -- rolling back > > ingest Feb 09, 2016 11:05:56 AM > > org.apache.oodt.cas.filemgr.catalog.solr.SolrClient delete > > INFO: Posting message:id:null to URL: > > http://localhost:8080/solr/update?commit=true > > Feb 09, 2016 11:05:56 AM > > org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction > > execute > > SEVERE: java.lang.Exception: > > org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: > > Error ingesting product > > [org.apache.oodt.cas.filemgr.structs.Product@469d4b24] > > : HTTP method failed: HTTP/1.1 400 Bad Request > > ERROR: Failed to ingest product 'test.txt' : java.lang.Exception: > > org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: > > Error ingesting product > > [org.apache.oodt.cas.filemgr.structs.Product@469d4b24] > > : HTTP method failed: HTTP/1.1 400 Bad Request > > > > > > Thanks, > > > > Wei Shi, M.B.A., M.S. > > Architect > > UM Comprehensive Cancer Center > > NCRC 100/134 > > O: 734-763-8440 C: 248-835-8453 > > shi...@med.umich.edu > > > > > > > > ** > > Electronic Mail is not secure, may not be read every day, and should > > not be used for urgent or sensitive issues > > > > > > -- > *Lewis* > ** > Electronic Mail is not secure, may not be read every day, and should > not be used for urgent or sensitive issues > -- *Lewis* ** Electronic Mail is not
Fail to ingest a test file to file manager on SuSe Linux
Hi Every, I tried to setup the OODT on SuSe Linux server. The installation went well. I also installed Solr. Now the ingest test failed because of Bad Request. I am using jdk 8. Any ideas? I attached error message below. ./filemgr-client --url http://localhost:9000 --operation --ingestProduct --productName test.txt --productStructure Flat --productTypeName GenericFile --metadataFile file:///tmp/test.txt.met --refs file:///tmp/test.txt log4j:WARN No appenders could be found for logger (org.springframework.context.support.FileSystemXmlApplicationContext). log4j:WARN Please initialize the log4j system properly. Feb 09, 2016 11:05:55 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient INFO: Loading File Manager Configuration Properties from: [../etc/filemgr.properties] Feb 09, 2016 11:05:56 AM org.apache.oodt.cas.filemgr.catalog.solr.SolrCatalog addProduct INFO: Adding product:test.txt Feb 09, 2016 11:05:56 AM org.apache.oodt.cas.filemgr.catalog.solr.SolrClient index INFO: Posting message:b69ee1e0-f9f4-4cf0-80e8-232f74e31d12b69ee1e0-f9f4-4cf0-80e8-232f74e31d12test.txtGenericFile2016-02-09T11:05:56Zurn:oodt:GenericFileFlatTRANSFERING to URL:http://localhost:8080/solr/update Feb 09, 2016 11:05:56 AM org.apache.oodt.cas.filemgr.catalog.solr.SolrClient index SEVERE: HTTP method failed: HTTP/1.1 400 Bad Request Feb 09, 2016 11:05:56 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager catalogProduct SEVERE: ingestProduct: CatalogException when adding Product: test.txt to Catalog: Message: HTTP method failed: HTTP/1.1 400 Bad Request Feb 09, 2016 11:05:56 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager ingestProductCore SEVERE: HTTP method failed: HTTP/1.1 400 Bad Request Feb 09, 2016 11:05:56 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct SEVERE: Failed to ingest product [ name:test.txt] :java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@469d4b24] : HTTP method failed: HTTP/1.1 400 Bad Request -- rolling back ingest Feb 09, 2016 11:05:56 AM org.apache.oodt.cas.filemgr.catalog.solr.SolrClient delete INFO: Posting message:id:null to URL:http://localhost:8080/solr/update?commit=true Feb 09, 2016 11:05:56 AM org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction execute SEVERE: java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@469d4b24] : HTTP method failed: HTTP/1.1 400 Bad Request ERROR: Failed to ingest product 'test.txt' : java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@469d4b24] : HTTP method failed: HTTP/1.1 400 Bad Request Thanks, Wei Shi, M.B.A., M.S. Architect UM Comprehensive Cancer Center NCRC 100/134 O: 734-763-8440 C: 248-835-8453 shi...@med.umich.edu ** Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues
Re: Fail to ingest a test file to file manager on SuSe Linux
Is your Solr server running on port 8080 or another port? Which version of Solr are you running? On Tue, Feb 9, 2016 at 8:19 AM, Shi, Weiwrote: > Hi Every, > I tried to setup the OODT on SuSe Linux server. The installation went > well. I also installed Solr. > Now the ingest test failed because of Bad Request. > I am using jdk 8. > > Any ideas? I attached error message below. > > ./filemgr-client --url http://localhost:9000 --operation --ingestProduct > --productName test.txt --productStructure Flat --productTypeName > GenericFile --metadataFile file:///tmp/test.txt.met --refs > file:///tmp/test.txt > > log4j:WARN No appenders could be found for logger > (org.springframework.context.support.FileSystemXmlApplicationContext). > log4j:WARN Please initialize the log4j system properly. > Feb 09, 2016 11:05:55 AM > org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient > INFO: Loading File Manager Configuration Properties from: > [../etc/filemgr.properties] > Feb 09, 2016 11:05:56 AM > org.apache.oodt.cas.filemgr.catalog.solr.SolrCatalog addProduct > INFO: Adding product:test.txt > Feb 09, 2016 11:05:56 AM > org.apache.oodt.cas.filemgr.catalog.solr.SolrClient index > INFO: Posting message: name="id">b69ee1e0-f9f4-4cf0-80e8-232f74e31d12 name="CAS.ProductId">b69ee1e0-f9f4-4cf0-80e8-232f74e31d12 name="CAS.ProductName">test.txt name="CAS.ProductTypeName">GenericFile name="CAS.ProductReceivedTime">2016-02-09T11:05:56Z name="CAS.ProductTypeId">urn:oodt:GenericFile name="CAS.ProductStructure">Flat name="CAS.ProductTransferStatus">TRANSFERING to URL: > http://localhost:8080/solr/update > Feb 09, 2016 11:05:56 AM > org.apache.oodt.cas.filemgr.catalog.solr.SolrClient index > SEVERE: HTTP method failed: HTTP/1.1 400 Bad Request > Feb 09, 2016 11:05:56 AM > org.apache.oodt.cas.filemgr.system.XmlRpcFileManager catalogProduct > SEVERE: ingestProduct: CatalogException when adding Product: test.txt to > Catalog: Message: HTTP method failed: HTTP/1.1 400 Bad Request > Feb 09, 2016 11:05:56 AM > org.apache.oodt.cas.filemgr.system.XmlRpcFileManager ingestProductCore > SEVERE: HTTP method failed: HTTP/1.1 400 Bad Request > Feb 09, 2016 11:05:56 AM > org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct > SEVERE: Failed to ingest product [ name:test.txt] :java.lang.Exception: > org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error > ingesting product [org.apache.oodt.cas.filemgr.structs.Product@469d4b24] > : HTTP method failed: HTTP/1.1 400 Bad Request -- rolling back ingest > Feb 09, 2016 11:05:56 AM > org.apache.oodt.cas.filemgr.catalog.solr.SolrClient delete > INFO: Posting message:id:null to URL: > http://localhost:8080/solr/update?commit=true > Feb 09, 2016 11:05:56 AM > org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction execute > SEVERE: java.lang.Exception: > org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error > ingesting product [org.apache.oodt.cas.filemgr.structs.Product@469d4b24] > : HTTP method failed: HTTP/1.1 400 Bad Request > ERROR: Failed to ingest product 'test.txt' : java.lang.Exception: > org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error > ingesting product [org.apache.oodt.cas.filemgr.structs.Product@469d4b24] > : HTTP method failed: HTTP/1.1 400 Bad Request > > > Thanks, > > Wei Shi, M.B.A., M.S. > Architect > UM Comprehensive Cancer Center > NCRC 100/134 > O: 734-763-8440 C: 248-835-8453 > shi...@med.umich.edu > > > > ** > Electronic Mail is not secure, may not be read every day, and should not > be used for urgent or sensitive issues > -- *Lewis*
RE: Fail to ingest a test file to file manager on SuSe Linux
Yes, the Solr is on 8080. The version is 4.2.0 -Original Message- From: Lewis John Mcgibbney [mailto:lewis.mcgibb...@gmail.com] Sent: Tuesday, February 09, 2016 11:36 AM To: dev@oodt.apache.org Subject: Re: Fail to ingest a test file to file manager on SuSe Linux Is your Solr server running on port 8080 or another port? Which version of Solr are you running? On Tue, Feb 9, 2016 at 8:19 AM, Shi, Weiwrote: > Hi Every, > I tried to setup the OODT on SuSe Linux server. The installation went > well. I also installed Solr. > Now the ingest test failed because of Bad Request. > I am using jdk 8. > > Any ideas? I attached error message below. > > ./filemgr-client --url http://localhost:9000 --operation > --ingestProduct --productName test.txt --productStructure Flat > --productTypeName GenericFile --metadataFile file:///tmp/test.txt.met > --refs file:///tmp/test.txt > > log4j:WARN No appenders could be found for logger > (org.springframework.context.support.FileSystemXmlApplicationContext). > log4j:WARN Please initialize the log4j system properly. > Feb 09, 2016 11:05:55 AM > org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient > INFO: Loading File Manager Configuration Properties from: > [../etc/filemgr.properties] > Feb 09, 2016 11:05:56 AM > org.apache.oodt.cas.filemgr.catalog.solr.SolrCatalog addProduct > INFO: Adding product:test.txt > Feb 09, 2016 11:05:56 AM > org.apache.oodt.cas.filemgr.catalog.solr.SolrClient index > INFO: Posting message: name="id">b69ee1e0-f9f4-4cf0-80e8-232f74e31d12 name="CAS.ProductId">b69ee1e0-f9f4-4cf0-80e8-232f74e31d12 d name="CAS.ProductName">test.txt name="CAS.ProductTypeName">GenericFile name="CAS.ProductReceivedTime">2016-02-09T11:05:56Z name="CAS.ProductTypeId">urn:oodt:GenericFile name="CAS.ProductStructure">Flat name="CAS.ProductTransferStatus">TRANSFERING to URL: > http://localhost:8080/solr/update > Feb 09, 2016 11:05:56 AM > org.apache.oodt.cas.filemgr.catalog.solr.SolrClient index > SEVERE: HTTP method failed: HTTP/1.1 400 Bad Request Feb 09, 2016 > 11:05:56 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager > catalogProduct > SEVERE: ingestProduct: CatalogException when adding Product: test.txt > to > Catalog: Message: HTTP method failed: HTTP/1.1 400 Bad Request Feb 09, > 2016 11:05:56 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager > ingestProductCore > SEVERE: HTTP method failed: HTTP/1.1 400 Bad Request Feb 09, 2016 > 11:05:56 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient > ingestProduct > SEVERE: Failed to ingest product [ name:test.txt] :java.lang.Exception: > org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error > ingesting product > [org.apache.oodt.cas.filemgr.structs.Product@469d4b24] > : HTTP method failed: HTTP/1.1 400 Bad Request -- rolling back ingest > Feb 09, 2016 11:05:56 AM > org.apache.oodt.cas.filemgr.catalog.solr.SolrClient delete > INFO: Posting message:id:null to URL: > http://localhost:8080/solr/update?commit=true > Feb 09, 2016 11:05:56 AM > org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction execute > SEVERE: java.lang.Exception: > org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error > ingesting product > [org.apache.oodt.cas.filemgr.structs.Product@469d4b24] > : HTTP method failed: HTTP/1.1 400 Bad Request > ERROR: Failed to ingest product 'test.txt' : java.lang.Exception: > org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error > ingesting product > [org.apache.oodt.cas.filemgr.structs.Product@469d4b24] > : HTTP method failed: HTTP/1.1 400 Bad Request > > > Thanks, > > Wei Shi, M.B.A., M.S. > Architect > UM Comprehensive Cancer Center > NCRC 100/134 > O: 734-763-8440 C: 248-835-8453 > shi...@med.umich.edu > > > > ** > Electronic Mail is not secure, may not be read every day, and should > not be used for urgent or sensitive issues > -- *Lewis* ** Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues
RE: Deploying SOLR (was:Re: Fail to ingest a test file to file manager on SuSe Linux)
Here is a list I compiled: 1. Download and Install VirtualBox 5.0 2. Install Vagrant on Windows 7. It will be installed in VirtualBox http://www.seascapewebdesign.com/blog/part-1-getting-started-vagrant-windows-7-and-8 3. Install Java https://www.digitalocean.com/community/tutorials/how-to-install-java-on-ubuntu-with-apt-get 4. Install Maven 2.2.1 (download from maven site cause the blog provided link is not working) http://archive.apache.org/dist/maven/binaries/apache-maven-2.2.1-bin.tar.gz https://saeedkhattak.wordpress.com/2013/03/05/install-maven-in-ubuntu-12-04/ 5. Install OODT https://cwiki.apache.org/confluence/display/OODT/RADiX+Powered+By+OODT 6. You may need to open port 8080 to allow guest to access http://serverfault.com/questions/248568/how-to-open-a-8080-port-for-an-application-with-iptables 7. Recompile filemgr in 0.12 and copy over, Note: logs folder is missing. Create logs folder under filemgr 8. File Manger User Guide: https://cwiki.apache.org/confluence/display/OODT/OODT+Filemgr+User+Guide 9. Solr https://cwiki.apache.org/confluence/display/OODT/Solr+File+Manager+Quick+Start+Guide 10. Curator Setup http://svn.apache.org/repos/asf/oodt/site/components/maven/curator/user/basic.html -Original Message- From: Tom Barber [mailto:tom.bar...@meteorite.bi] Sent: Tuesday, February 09, 2016 3:28 PM To: dev@oodt.apache.org Subject: Deploying SOLR (was:Re: Fail to ingest a test file to file manager on SuSe Linux) Out of curiosity, Wei and Jordan if he checks this thread, what instructions/hacks are you using to deploy OODT & SOLR? I know how I do it, I wondered how others do it, ie, what docs do you follow? Cheers Tom ** Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues
Deploying SOLR (was:Re: Fail to ingest a test file to file manager on SuSe Linux)
Out of curiosity, Wei and Jordan if he checks this thread, what instructions/hacks are you using to deploy OODT & SOLR? I know how I do it, I wondered how others do it, ie, what docs do you follow? Cheers Tom
Re: Fail to ingest a test file to file manager on SuSe Linux
File manager log should provide more insight into the the exception is as right now it seems to be too vague for me to debug. Lewis On Tue, Feb 9, 2016 at 9:20 AM, Shi, Weiwrote: > Yes, the Solr is on 8080. The version is 4.2.0 > > -Original Message- > From: Lewis John Mcgibbney [mailto:lewis.mcgibb...@gmail.com] > Sent: Tuesday, February 09, 2016 11:36 AM > To: dev@oodt.apache.org > Subject: Re: Fail to ingest a test file to file manager on SuSe Linux > > Is your Solr server running on port 8080 or another port? > Which version of Solr are you running? > > On Tue, Feb 9, 2016 at 8:19 AM, Shi, Wei wrote: > > > Hi Every, > > I tried to setup the OODT on SuSe Linux server. The installation went > > well. I also installed Solr. > > Now the ingest test failed because of Bad Request. > > I am using jdk 8. > > > > Any ideas? I attached error message below. > > > > ./filemgr-client --url http://localhost:9000 --operation > > --ingestProduct --productName test.txt --productStructure Flat > > --productTypeName GenericFile --metadataFile file:///tmp/test.txt.met > > --refs file:///tmp/test.txt > > > > log4j:WARN No appenders could be found for logger > > (org.springframework.context.support.FileSystemXmlApplicationContext). > > log4j:WARN Please initialize the log4j system properly. > > Feb 09, 2016 11:05:55 AM > > org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient > > INFO: Loading File Manager Configuration Properties from: > > [../etc/filemgr.properties] > > Feb 09, 2016 11:05:56 AM > > org.apache.oodt.cas.filemgr.catalog.solr.SolrCatalog addProduct > > INFO: Adding product:test.txt > > Feb 09, 2016 11:05:56 AM > > org.apache.oodt.cas.filemgr.catalog.solr.SolrClient index > > INFO: Posting message: > name="id">b69ee1e0-f9f4-4cf0-80e8-232f74e31d12 > name="CAS.ProductId">b69ee1e0-f9f4-4cf0-80e8-232f74e31d12 > d name="CAS.ProductName">test.txt > name="CAS.ProductTypeName">GenericFile > name="CAS.ProductReceivedTime">2016-02-09T11:05:56Z > name="CAS.ProductTypeId">urn:oodt:GenericFile > name="CAS.ProductStructure">Flat > name="CAS.ProductTransferStatus">TRANSFERING to URL: > > http://localhost:8080/solr/update > > Feb 09, 2016 11:05:56 AM > > org.apache.oodt.cas.filemgr.catalog.solr.SolrClient index > > SEVERE: HTTP method failed: HTTP/1.1 400 Bad Request Feb 09, 2016 > > 11:05:56 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager > > catalogProduct > > SEVERE: ingestProduct: CatalogException when adding Product: test.txt > > to > > Catalog: Message: HTTP method failed: HTTP/1.1 400 Bad Request Feb 09, > > 2016 11:05:56 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager > > ingestProductCore > > SEVERE: HTTP method failed: HTTP/1.1 400 Bad Request Feb 09, 2016 > > 11:05:56 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient > > ingestProduct > > SEVERE: Failed to ingest product [ name:test.txt] :java.lang.Exception: > > org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error > > ingesting product > > [org.apache.oodt.cas.filemgr.structs.Product@469d4b24] > > : HTTP method failed: HTTP/1.1 400 Bad Request -- rolling back ingest > > Feb 09, 2016 11:05:56 AM > > org.apache.oodt.cas.filemgr.catalog.solr.SolrClient delete > > INFO: Posting message:id:null to URL: > > http://localhost:8080/solr/update?commit=true > > Feb 09, 2016 11:05:56 AM > > org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction execute > > SEVERE: java.lang.Exception: > > org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error > > ingesting product > > [org.apache.oodt.cas.filemgr.structs.Product@469d4b24] > > : HTTP method failed: HTTP/1.1 400 Bad Request > > ERROR: Failed to ingest product 'test.txt' : java.lang.Exception: > > org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error > > ingesting product > > [org.apache.oodt.cas.filemgr.structs.Product@469d4b24] > > : HTTP method failed: HTTP/1.1 400 Bad Request > > > > > > Thanks, > > > > Wei Shi, M.B.A., M.S. > > Architect > > UM Comprehensive Cancer Center > > NCRC 100/134 > > O: 734-763-8440 C: 248-835-8453 > > shi...@med.umich.edu > > > > > > > > ** > > Electronic Mail is not secure, may not be read every day, and should > > not be used for urgent or sensitive issues > > > > > > -- > *Lewis* > ** > Electronic Mail is not secure, may not be read every day, and should not > be used for urgent or sensitive issues > -- *Lewis*
Re: Deploying SOLR (was:Re: Fail to ingest a test file to file manager on SuSe Linux)
Okay Wei, Jordan Can you guys test this for me whilst I pass out: mvn -DarchetypeGroupId=org.apache.oodt -DarchetypeArtifactId=radix-archetype -DarchetypeVersion=0.12-SNAPSHOT -Doodt=0.12-SNAPSHOT -DgroupId=com.mycompany -DartifactId=oodt -Dversion=0.1 -DarchetypeRepository= https://repository.apache.org/content/groups/snapshots/ archetype:generate -Pfm-solr-catalog mv oodt oodt-src; cd oodt-src; mvn clean package -Pfm-solr-catalog mkdir ../oodt; tar -xvf distribution/target/oodt-distribution-0.1-bin.tar.gz -C ../oodt cd ../oodt; ./bin/oodt start Its 0.12 RADIX with the start script patched so SOLR starts. Notice the -P profile flag for solr, it should deploy your solr config with no extra work. Let me know how it goes, I've not tested ingestion but I have verified that solr and the filemgr are running. Tom On Tue, Feb 9, 2016 at 9:17 PM, Tom Barberwrote: > You don't need vagrant if you don't want it and you can use maven 3 BTW > On 9 Feb 2016 22:13, "Tom Barber" wrote: > >> Blimey! >> >> Okay you guys are mostly working. I'm gonna fix the solr maven profile >> before I cut 0.12 because it mostly works and it's a lot easier than that >> solr tutorial! :) >> On 9 Feb 2016 22:06, "Shi, Wei" wrote: >> >>> Here is a list I compiled: >>> >>> 1. Download and Install VirtualBox 5.0 >>> 2. Install Vagrant on Windows 7. It will be installed in VirtualBox >>> >>> http://www.seascapewebdesign.com/blog/part-1-getting-started-vagrant-windows-7-and-8 >>> 3. Install Java >>> >>> https://www.digitalocean.com/community/tutorials/how-to-install-java-on-ubuntu-with-apt-get >>> 4. Install Maven 2.2.1 (download from maven site cause the blog >>> provided link is not working) >>> >>> http://archive.apache.org/dist/maven/binaries/apache-maven-2.2.1-bin.tar.gz >>> >>> https://saeedkhattak.wordpress.com/2013/03/05/install-maven-in-ubuntu-12-04/ >>> 5. Install OODT >>> https://cwiki.apache.org/confluence/display/OODT/RADiX+Powered+By+OODT >>> 6. You may need to open port 8080 to allow guest to access >>> >>> http://serverfault.com/questions/248568/how-to-open-a-8080-port-for-an-application-with-iptables >>> 7. Recompile filemgr in 0.12 and copy over, Note: logs folder is >>> missing. Create logs folder under filemgr >>> 8. File Manger User Guide: >>> https://cwiki.apache.org/confluence/display/OODT/OODT+Filemgr+User+Guide >>> 9. Solr >>> >>> https://cwiki.apache.org/confluence/display/OODT/Solr+File+Manager+Quick+Start+Guide >>> 10. Curator Setup >>> >>> http://svn.apache.org/repos/asf/oodt/site/components/maven/curator/user/basic.html >>> >>> >>> >>> -Original Message- >>> From: Tom Barber [mailto:tom.bar...@meteorite.bi] >>> Sent: Tuesday, February 09, 2016 3:28 PM >>> To: dev@oodt.apache.org >>> Subject: Deploying SOLR (was:Re: Fail to ingest a test file to file >>> manager on SuSe Linux) >>> >>> Out of curiosity, Wei and Jordan if he checks this thread, what >>> instructions/hacks are you using to deploy OODT & SOLR? >>> >>> I know how I do it, I wondered how others do it, ie, what docs do you >>> follow? >>> >>> Cheers >>> >>> Tom >>> ** >>> Electronic Mail is not secure, may not be read every day, and should not >>> be used for urgent or sensitive issues >>> >>
Re: Deploying SOLR (was:Re: Fail to ingest a test file to file manager on SuSe Linux)
Okay cool. I'm not familiar with that part of the code base, but I'll prod around and see what I can come up with. Tom On Wed, Feb 10, 2016 at 12:11 AM, Shi, Weiwrote: > Hi Tom, > The installation went very well. I tested ingestion and deletion, both > worked as expected. > > The only thing I noticed is ³Received Time² displayed as ³UNKNOWN² after > the file is ingested. > > Thanks, > > Wei > > On 2/9/16, 6:08 PM, "Tom Barber" wrote: > > >Okay Wei, Jordan > > > >Can you guys test this for me whilst I pass out: > > > >mvn -DarchetypeGroupId=org.apache.oodt > >-DarchetypeArtifactId=radix-archetype -DarchetypeVersion=0.12-SNAPSHOT > >-Doodt=0.12-SNAPSHOT -DgroupId=com.mycompany -DartifactId=oodt > >-Dversion=0.1 -DarchetypeRepository= > >https://repository.apache.org/content/groups/snapshots/ > archetype:generate > >-Pfm-solr-catalog > > > >mv oodt oodt-src; cd oodt-src; mvn clean package -Pfm-solr-catalog > > > >mkdir ../oodt; tar -xvf > >distribution/target/oodt-distribution-0.1-bin.tar.gz -C ../oodt > > > >cd ../oodt; ./bin/oodt start > > > > > >Its 0.12 RADIX with the start script patched so SOLR starts. > > > >Notice the -P profile flag for solr, it should deploy your solr config > >with > >no extra work. > > > >Let me know how it goes, I've not tested ingestion but I have verified > >that > >solr and the filemgr are running. > > > >Tom > > > >On Tue, Feb 9, 2016 at 9:17 PM, Tom Barber > >wrote: > > > >> You don't need vagrant if you don't want it and you can use maven 3 BTW > >> On 9 Feb 2016 22:13, "Tom Barber" wrote: > >> > >>> Blimey! > >>> > >>> Okay you guys are mostly working. I'm gonna fix the solr maven profile > >>> before I cut 0.12 because it mostly works and it's a lot easier than > >>>that > >>> solr tutorial! :) > >>> On 9 Feb 2016 22:06, "Shi, Wei" wrote: > >>> > Here is a list I compiled: > > 1. Download and Install VirtualBox 5.0 > 2. Install Vagrant on Windows 7. It will be installed in > VirtualBox > > > > http://www.seascapewebdesign.com/blog/part-1-getting-started-vagrant-wi > ndows-7-and-8 > 3. Install Java > > > > https://www.digitalocean.com/community/tutorials/how-to-install-java-on > -ubuntu-with-apt-get > 4. Install Maven 2.2.1 (download from maven site cause the blog > provided link is not working) > > > > http://archive.apache.org/dist/maven/binaries/apache-maven-2.2.1-bin.ta > r.gz > > > > https://saeedkhattak.wordpress.com/2013/03/05/install-maven-in-ubuntu-1 > 2-04/ > 5. Install OODT > > https://cwiki.apache.org/confluence/display/OODT/RADiX+Powered+By+OODT > 6. You may need to open port 8080 to allow guest to access > > > > http://serverfault.com/questions/248568/how-to-open-a-8080-port-for-an- > application-with-iptables > 7. Recompile filemgr in 0.12 and copy over, Note: logs folder is > missing. Create logs folder under filemgr > 8. File Manger User Guide: > > > https://cwiki.apache.org/confluence/display/OODT/OODT+Filemgr+User+Guid > e > 9. Solr > > > > https://cwiki.apache.org/confluence/display/OODT/Solr+File+Manager+Quic > k+Start+Guide > 10. Curator Setup > > > > http://svn.apache.org/repos/asf/oodt/site/components/maven/curator/user > /basic.html > > > > -Original Message- > From: Tom Barber [mailto:tom.bar...@meteorite.bi] > Sent: Tuesday, February 09, 2016 3:28 PM > To: dev@oodt.apache.org > Subject: Deploying SOLR (was:Re: Fail to ingest a test file to file > manager on SuSe Linux) > > Out of curiosity, Wei and Jordan if he checks this thread, what > instructions/hacks are you using to deploy OODT & SOLR? > > I know how I do it, I wondered how others do it, ie, what docs do you > follow? > > Cheers > > Tom > ** > Electronic Mail is not secure, may not be read every day, and should > not > be used for urgent or sensitive issues > > >>> > > ** > Electronic Mail is not secure, may not be read every day, and should not > be used for urgent or sensitive issues > >