[ https://issues.apache.org/jira/browse/SOLR-10981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650765#comment-16650765 ]
David Smiley commented on SOLR-10981: ------------------------------------- This is looking very good Andrew! Thanks for your patience. I applied the patch locally and ran tests and poked around. I only made slight edits and did some reformatting. I like that the test was revamped to use try-with-resources. I'm inclined to alter the ref-guide change. I don't think it's worthy of a header. I edited the ref guide notes as follows: {code:java} The source of the data can be compressed using gzip, and Solr will generally detect this. The detection is based on either the presence of a `Content-Encoding: gzip` HTTP header or the file ending with .gz or .gzip. Gzip doesn't apply to `stream.body`.{code} WDYT? CHANGES.txt will credit you first, then Jan and I. > Allow update to load gzip files > -------------------------------- > > Key: SOLR-10981 > URL: https://issues.apache.org/jira/browse/SOLR-10981 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrJ > Affects Versions: 6.6 > Reporter: Andrew Lundgren > Assignee: David Smiley > Priority: Major > Labels: patch > Fix For: 7.6, master (8.0) > > Attachments: SOLR-10981.patch, SOLR-10981.patch, SOLR-10981.patch, > SOLR-10981.patch, SOLR-10981.patch, SOLR-10981.patch > > > We currently import large CSV files. We store them in gzip files as they > compress at around 80%. > To import them we must gunzip them and then import them. After that we no > longer need the decompressed files. > This patch allows directly opening either URL, or local files that are > gzipped. > For URLs, to determine if the file is gzipped, it will check the content > encoding=="gzip" or if the file ends in ".gz" > For files, if the file ends in ".gz" then it will assume the file is gzipped. > I have tested the patch with 4.10.4, 6.6.0, 7.0.1 and master from git. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org