[
https://issues.apache.org/jira/browse/LUCENENET-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049906#comment-13049906
]
Digy commented on LUCENENET-417:
--------------------------------
OK, File System Indexer is a good example.
Have you tested your code, since I see some codes like
*AbstractField.cs*
{code}
if (fieldsData is System.IO.Stream)
{
var streamReader = new
System.IO.BinaryReader((System.IO.Stream)fieldsData);
byte[] data = new byte[streamReader.BaseStream.Length]; // <----
streamReader.Read(data, 0, data.Length);
return data;
}
{code}
or
*Field.cs*
{code}
if (fieldsData is Lucene.Net.Util.InternalStreamReader)
{
using (var reader =
((Lucene.Net.Util.InternalStreamReader)fieldsData))
{
reader.DiscardBufferedData();
reader.BaseStream.Seek(0, SeekOrigin.Begin);
fieldsData = reader.ReadToEnd(); // <----
}
}
{code}
Wouldn't they be costly in terms of mem usage?
DIGY
PS: I still think there should be other solutions(outside of Lucene.Net) to
handle these type of problems.
> implement streams as field values
> ---------------------------------
>
> Key: LUCENENET-417
> URL: https://issues.apache.org/jira/browse/LUCENENET-417
> Project: Lucene.Net
> Issue Type: New Feature
> Components: Lucene.Net Core
> Reporter: Christopher Currens
> Attachments: StreamValues.patch
>
>
> Adding binary values to a field is an expensive operation, as the whole
> binary data must be loaded into memory and then written to the index. Adding
> the ability to use a stream instead of a byte array could not only speed up
> the indexing process, but reducing the memory footprint as well.
> -Java lucene has the ability to use a TextReader the both analyze and store
> text in the index.- Lucene.NET lacks the ability to store string data in the
> index via streams. This should be a feature added into Lucene .NET as well.
> My thoughts are to add another Field constructor, that is Field(string name,
> System.IO.Stream stream, System.Text.Encoding encoding), that will allow the
> text to be analyzed and stored into the index.
> Comments about this approach are greatly appreciated.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
