Is there a maximum size to objects in the blob store? How are objects
stored? As a stored field?
I've got some machine learning models that are 2-4Gb in size, and whilst
machine learning models is one of the intended uses of the blob store,
putting GB of data in it scares me a little. Is it
No, the maximum size is limited to 2MB for now. The use-case behind
the blob store is to store small jars (custom plugins) and stopwords,
synonyms etc (even though those aren't usable right now) so maybe we
can relax the limits a little bit. However, it is definitely not meant
for GBs of data.
On
It's unfortunate that the Blob Store API wasn't named "Small File Store
API" to convey to users its intended purpose.
That said, maybe you could use the same technique as is recommended for
large synonym files: Break them into a sequence of smaller files and then
take advantage of the fact that
I checked the code and the limit is actually 5MB and configurable via
the blob.max.size.mb config property. I posted a comment on the Solr doc
for this.
In any case, thanks for sharing info that you gleaned from the conference,
for all of us who couldn't make it.
-- Jack Krupansky
On Tue, Oct
Yes, sorry I checked as well and the limit is 5MB. And it is
configurable using the property mentioned by Jack. Thanks for
correcting me.
On Tue, Oct 20, 2015 at 7:48 PM, Jack Krupansky
wrote:
> I checked the code and the limit is actually 5MB and configurable via
> the
What is this limit limiting? Is this effectively a stored field, and the
bigger it gets, the more issues we'll have with segment merges/etc?
Upayavira
On Tue, Oct 20, 2015, at 09:25 AM, Shalin Shekhar Mangar wrote:
> Yes, sorry I checked as well and the limit is 5MB. And it is
> configurable
Okay, thx. I heard it mentioned at Lucene Revolution as a location for
storing machine learning models. Do people really have models coming in
at under 2Mb?
It'd be good to get this limitation into the BlobStore docs.
Upayavira
On Tue, Oct 20, 2015, at 07:19 AM, Shalin Shekhar Mangar wrote:
>