Upayavira, manual commit isn't a good advice, especially with small bulks
or single document, is it? I see recommendations on using
autoCommit+autoSoftCommit instead of manual commit mostly.
вт, 4 авг. 2015 г. в 1:00, Upayavira <u...@odoko.co.uk>:
> SolrJ is just a "SolrClient". In pseudocode, you say:
>
> SolrClient client = new
> SolrClient("http://localhost:8983/solr/whatever";);
>
> List<SolrInputDocument> docs = new ArrayList<>();
> SolrInputDocument doc = new SolrInputDocument();
> doc.addField("id", "abc123");
> doc.addField("some-text-field", "I like it when the sun shines");
> docs.add(doc);
> client.add(docs);
> client.commit();
>
> (warning, the above is typed from memory)
>
> So, the question is simply how many documents do you add to docs before
> you do client.add(docs);
>
> And how often (if at all) do you call client.commit().
>
> So when you are told "Use SolrJ", really, you are being told to write
> some Java code that happens to use the SolrJ client library for Solr.
>
> Upayavira
>
>
> On Mon, Aug 3, 2015, at 10:01 PM, Alexandre Rafalovitch wrote:
> > Well,
> >
> > If it is just file names, I'd probably use SolrJ client, maybe with
> > Java 8. Read file names, split the name into parts with regular
> > expressions, stuff parts into different field names and send to Solr.
> > Java 8 has FileSystem walkers, etc to make it easier.
> >
> > You could do it with DIH, but it would be with nested entities and the
> > inner entity would probably try to parse the file. So, a lot of wasted
> > effort if you just care about the file names.
> >
> > Or, I would just do a directory listing in the operating system and
> > use regular expressions to split it into CSV file, which I would then
> > import into Solr directly.
> >
> > In all of these cases, the question would be which field is the ID of
> > the record to ensure no duplicates.
> >
> > Regards,
> > Alex.
> >
> > ----
> > Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
> > http://www.solr-start.com/
> >
> >
> > On 3 August 2015 at 15:34, Mugeesh Husain <muge...@gmail.com> wrote:
> > > @Alexandre No i dont need a content of a file. i am repeating my
> requirement
> > >
> > > I have a 40 millions of files which is stored in a file systems,
> > > the filename saved as ARIA_SSN10_0007_LOCATION_0000129.pdf
> > >
> > > I just split all Value from a filename only,these values i have to
> index.
> > >
> > > I am interested to index value to solr not file contains.
> > >
> > > I have tested the DIH from a file system its work fine but i dont know
> how
> > > can i implement my code in DIH
> > > if my code get some value than how i can i index it using DIH.
> > >
> > > If i will use DIH then How i will make split operation and get value
> from
> > > it.
> > >
> > >
> > >
> > >
> > >
> > > --
> > > View this message in context:
> http://lucene.472066.n3.nabble.com/Can-Apache-Solr-Handle-TeraByte-Large-Data-tp3656484p4220552.html
> > > Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Best regards,
Konstantin Gribov
