Hi Clay,
thanks for your advice. Indeed I digged quiet deep into the source and got it
finally running. The trick was to include tika-core. The default executor of
Oak is quiet hacky, it swallows exceptions thrown by its threads without
logging. So I didn’t get the class not found exception, caused by missing tika
dependencies, at least in my tomcat deployment.
Just for the records I post my working config. I also added a custom
OakExecutorProvider to handle the memory leaks by orphan exectors at oak
shutdown.
I hope it may help someone facing similar problems.
Best Leonhard Wachutka
In pom.xml:
<dependency>
<groupId>org.apache.tika</groupId>
<artifactId>tika-core</artifactId>
<version>1.9</version>
</dependency>
SolrServerProviderFactroy:
public class SolrServerProviderFactroy {
@Bean
public RemoteSolrServerProvider remoteSolrServerProvider()
{
RemoteSolrServerConfiguration remoteSolrConfig = new
RemoteSolrServerConfiguration(null, null, 1, 1, null,
"http://192.168.2.250:8983/solr/oak";);
return new RemoteSolrServerProvider(remoteSolrConfig);
}
@Bean
public OakSolrConfigurationProvider oakSolrConfigProvider()
{
OakSolrConfiguration oakSolrconfiguration = new
DefaultSolrConfiguration() {
@Override
public OakSolrConfiguration.CommitPolicy getCommitPolicy() {
return OakSolrConfiguration.CommitPolicy.HARD;
}
@Override
public boolean useForPropertyRestrictions() {
return false;
}
@Override
public boolean useForPrimaryTypes() {
return false;
}
@Override
public boolean useForPathRestrictions() {
return false;
}
@Override
public String getFieldNameFor(Type<?> propertyType) {
return null;
}
};
return new DefaultSolrConfigurationProvider(oakSolrconfiguration);
}
}
protected Repository createRepository() throws Exception {
// return JackRabbit repository.
log.info("Start of Apache Jackrabbit OAK Repository.\n{host:" + host
+"\ndbName:"+ dbName+"}");
DB db = new MongoClient(host, 27017).getDB(dbName);
documentNodeStore = new DocumentMK.Builder().
setMongoDB(db).getNodeStore();
oak = new Oak(documentNodeStore)
.with(new SolrIndexInitializer(true))
.with(AggregateIndexProvider.wrap(new
SolrQueryIndexProvider(solrServerProvider, oakSolrConfigurationProvider)))
.with(new SolrIndexEditorProvider(solrServerProvider,
oakSolrConfigurationProvider))
.with( new NodeStateSolrServersObserver())
.withAsyncIndexing()
.with(executorProvider.defaultExecutorService())
.with(executorProvider.defaultScheduledExecutor());
Repository repo = new Jcr(oak).createRepository();
return repo;
}
public class OakExecutorProvider {
final static Logger logger =
LoggerFactory.getLogger(OakExecutorProvider.class);
private ScheduledThreadPoolExecutor scheduledThreadPoolExecutor;
private ThreadPoolExecutor threadPoolExecutor;
public ScheduledExecutorService defaultScheduledExecutor() {
scheduledThreadPoolExecutor = new ScheduledThreadPoolExecutor(32, new
ThreadFactory() {
private final AtomicInteger counter = new AtomicInteger();
@Override
public Thread newThread(@Nonnull Runnable r) {
Thread thread = new Thread(r, createName());
thread.setDaemon(true);
return thread;
}
private String createName() {
return "oak-scheduled-executor-" + counter.getAndIncrement();
}
});
scheduledThreadPoolExecutor.setKeepAliveTime(1, TimeUnit.MINUTES);
scheduledThreadPoolExecutor.allowCoreThreadTimeOut(true);
return scheduledThreadPoolExecutor;
}
/**
* Default {@code ExecutorService} used for scheduling concurrent tasks.
* This default spawns as many threads as required with a priority of
* {@code Thread.MIN_PRIORITY}. Idle threads are pruned after one minute.
* @return fresh ExecutorService
*/
public ExecutorService defaultExecutorService() {
threadPoolExecutor = new ThreadPoolExecutor(0, Integer.MAX_VALUE, 60L,
TimeUnit.SECONDS,
new SynchronousQueue<Runnable>(), new ThreadFactory() {
private final AtomicInteger counter = new AtomicInteger();
@Override
public Thread newThread(@Nonnull Runnable r) {
Thread thread = new Thread(r, createName());
thread.setDaemon(true);
thread.setPriority(Thread.MIN_PRIORITY);
return thread;
}
private String createName() {
return "oak-executor-" + counter.getAndIncrement();
}
});
threadPoolExecutor.setKeepAliveTime(1, TimeUnit.MINUTES);
threadPoolExecutor.allowCoreThreadTimeOut(true);
return threadPoolExecutor;
}
public void shutdown()
{
logger.debug("Initializing shutdown of Oak worker Threads");
try {
threadPoolExecutor.shutdown();
scheduledThreadPoolExecutor.shutdown();
threadPoolExecutor.awaitTermination(30, TimeUnit.SECONDS);
scheduledThreadPoolExecutor.awaitTermination(30, TimeUnit.SECONDS);
} catch (InterruptedException ex) {
logger.error("Executor shutdown error");
throw new RuntimeException(ex);
}
logger.debug("Finished shutdown of Oak worker Threads");
}
}
-----Ursprüngliche Nachricht-----
Von: Clay Ferguson [mailto:[email protected]]
Gesendet: Mittwoch, 5. August 2015 16:37
An: [email protected]
Betreff: Re: Solr Configuration
Leonhard,
Check out the example FullTextSolrSearchTest.java, and in general search the
entire codebase for "SolrIndex". I found it impossible to learn Jackrabbit by
reading docs. The only good way is just download the entire source (1.2.3 being
latest), and then search for test case examples to see how it's done. There are
so many different versions of stuff documented online you can really only get
"hints" until you go directly into the source.
Best regards,
Clay Ferguson
meta64.com
On Wed, Aug 5, 2015 at 7:55 AM, lw <[email protected]> wrote:
> Hi all,
>
>
>
> I am trying to get a remote solr instance working with oak. Because of
> lack of good documentation, I tried find out the proper configuration
> myself, however, if I commit new nodes they are not send to the Solr
> server for indexing. Also no queries seem to be forwarded to the Solar
> instance. I use the attached code to initialize the repository and use
> jackrabbit ocm to persist nodes.
>
> Debugging shows that SolrIndexEditorProvider gets invoced after a
> session.save(), but it seems to look for
> "SolrQueryIndex.TYPE.equals(type)". I don't get what this type is used
> for and how set up it in the right way.
>
> For the first try I just want all properties of my node indexed by Solr.
> Has someone additional resources, code or hints what I have to do?
>
> Thanks in advance for your help
>
>
> Best Leonhard Wachutka
>
>
>
>
>
> protected Repository createRepository() throws Exception {
>
> RemoteSolrServerConfiguration remoteSolrConfig = new
> RemoteSolrServerConfiguration(null, null, 1, 1, null, "
> http://192.168.2.250:8983/solr/oak";);
> SolrServerProvider solrServerProvider = new
> RemoteSolrServerProvider(remoteSolrConfig);
>
> OakSolrConfiguration oakSolrconfiguration = new
> DefaultSolrConfiguration() {
> @Override
> public CommitPolicy getCommitPolicy() {
> return CommitPolicy.HARD;
> }
> @Override
> public boolean useForPropertyRestrictions() {
> return true;
> }
> @Override
> public boolean useForPrimaryTypes() {
> return true;
> }
> @Override
> public boolean useForPathRestrictions() {
> return true;
> }
> };
>
> OakSolrConfigurationProvider oakSolrConfigurationProvider =
> new DefaultSolrConfigurationProvider(oakSolrconfiguration);
>
> // return JackRabbit repository.
> log.info("Start of Apache Jackrabbit OAK Repository.\n{host:"
> + host +"\ndbName:"+ dbName+"}");
>
> DB db = new MongoClient(host, 27017).getDB(dbName);
> documentNodeStore = new DocumentMK.Builder().
> setMongoDB(db).getNodeStore();
>
> oak = new Oak(documentNodeStore)
> .with(new SolrIndexInitializer(true))
> .with(AggregateIndexProvider.wrap(new
> SolrQueryIndexProvider(solrServerProvider, oakSolrConfigurationProvider)))
> .with(new SolrIndexEditorProvider(solrServerProvider,
> oakSolrConfigurationProvider));
>
> Repository repo = new Jcr(oak).createRepository();
> return repo;
> }
>
