Are you saying to do a local mini-collection and then mirror final result
to the real one?
What about deletions? Per-entry cleanup statements and so on? DIH does full
updates, not just additions.
Or did I miss the focus?
Regards,
Alex
On 15 Dec 2015 11:46 pm, "Erik Hatcher" <erik.hatc...@gmail.com> wrote:
> With time shaken loose, IMO ideally what we do (under
> https://issues.apache.org/jira/browse/SOLR-7188 <
> https://issues.apache.org/jira/browse/SOLR-7188> probably) is create an
> update processor that *forwards* to a _real_ Solr collection update
> handler, and fire up EmbeddedSolrServer in a client-side command-line tool
> that can run /update/extract, DIH stuff, etc - does what it does now to
> extract, parse, and build documents and then forwards them via javabin to a
> live Solr collection. I’m not sure that SOLR-7188 currently spells it out
> like that, but it is a nice, clean, straightforward path from DIH and Tika
> embedded inside a real Solr cluster to leveraging and scaling it on its
> own. We’d lose the DIH admin UI, but that’s ok by me.
>
> —
> Erik Hatcher, Senior Solutions Architect
> http://www.lucidworks.com <http://www.lucidworks.com/>
>
>
>
> > On Dec 15, 2015, at 9:23 AM, Davis, Daniel (NIH/NLM) [C] <
> daniel.da...@nih.gov> wrote:
> >
> > I am aware of the problems with the implementation of DIH, but is there
> any problem with the XML driven data import capability?
> > Could it be rewritten (using modern XPath) to run as a part of SolrJ?
> >
> > I've been interested in that, but I just haven't been able to shake
> loose the time.
> >
> > -----Original Message-----
> > From: Upayavira [mailto:u...@odoko.co.uk]
> > Sent: Tuesday, December 15, 2015 5:04 AM
> > To: solr-user@lucene.apache.org
> > Subject: Re: Is DIH going to be removed from Solr future versions?
> >
> > I doubt DIH will be "removed". It more likely will be relegated - still
> there, but emphasised less.
> >
> > Another possibility that has been mooted is to extract it, so that it
> can run outside of Solr. This strikes me as the best option. Having it run
> inside Solr strikes me as architecturally wrong, and also problematic in a
> SolrCloud world. Taking the DIH codebase and running it
> > *outside* Solr you get the best of DIH without the same set of issues.
> >
> > Upayavira
> >
> > On Tue, Dec 15, 2015, at 05:47 AM, Anil Cherian wrote:
> >> Dear Team,
> >>
> >> I use DIH extensively and even wrote my own custom transformers in
> >> some situations.
> >> Recently during an architecture discussion one of my team members told
> >> that Solr is going to take away DIH from its future versions.
> >>
> >> Is that true?
> >>
> >> Also is using DIH for say 2 or 3 million docs a good option for
> >> indexing an XML content data set. I am planning to use it either by
> >> calling separate entities parallely or multiple /dataimport in
> >> solrconfig.xml.
> >>
> >> Cld you please reply at your earliest convenience as it is an
> >> important decision for us to continue on DIH or not!
> >>
> >> Thanks and Rgds,
> >> Anil.
>
>
