[jira] [Commented] (SOLR-11198) downconfig downloads empty file as folder
[ https://issues.apache.org/jira/browse/SOLR-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133608#comment-16133608 ] Isabelle Giguere commented on SOLR-11198: - Compiled and tested on Windows to confirm the patch works (obviously, if you look at it ;) ) > downconfig downloads empty file as folder > - > > Key: SOLR-11198 > URL: https://issues.apache.org/jira/browse/SOLR-11198 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.6 > Environment: Windows 7 >Reporter: Isabelle Giguere >Assignee: Erick Erickson >Priority: Minor > Fix For: 6.7, master (8.0), 7.1 > > Attachments: SOLR-11198.patch, SOLR-11198.patch > > > With Solr 6.6.0, when downloading a config from Zookeeper (3.4.10), if a file > is empty, it is downloaded as a folder (on Windows, at least). > A Zookeeper browser (Eclipse: Zookeeper Explorer) shows the file as a file, > however, in ZK. > Noticed because we keep an empty synonyms.txt file in the Solr config > provided with our product, in case a client would want to use it. > The workaround is simple, since the file allows comments: just add a comment, > so it is not empty. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11198) downconfig downloads empty file as folder
[ https://issues.apache.org/jira/browse/SOLR-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117000#comment-16117000 ] ASF subversion and git services commented on SOLR-11198: Commit 83e3276225691c2c710e5fc89df1a1605a2b4112 in lucene-solr's branch refs/heads/branch_6x from [~erickerickson] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=83e3276 ] SOLR-11198: fix test failures (cherry picked from commit 53db72c) > downconfig downloads empty file as folder > - > > Key: SOLR-11198 > URL: https://issues.apache.org/jira/browse/SOLR-11198 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.6 > Environment: Windows 7 >Reporter: Isabelle Giguere >Assignee: Erick Erickson >Priority: Minor > Fix For: 6.7, master (8.0), 7.1 > > Attachments: SOLR-11198.patch, SOLR-11198.patch > > > With Solr 6.6.0, when downloading a config from Zookeeper (3.4.10), if a file > is empty, it is downloaded as a folder (on Windows, at least). > A Zookeeper browser (Eclipse: Zookeeper Explorer) shows the file as a file, > however, in ZK. > Noticed because we keep an empty synonyms.txt file in the Solr config > provided with our product, in case a client would want to use it. > The workaround is simple, since the file allows comments: just add a comment, > so it is not empty. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11198) downconfig downloads empty file as folder
[ https://issues.apache.org/jira/browse/SOLR-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116999#comment-16116999 ] ASF subversion and git services commented on SOLR-11198: Commit 2c281457dce8b4a09f5b3c101c92b03d28e3d994 in lucene-solr's branch refs/heads/branch_6x from [~erickerickson] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2c28145 ] SOLR-11198: downconfig downloads empty file as folder, test failures possible fix and logging (cherry picked from commit e053e22) (cherry picked from commit a3c360e) > downconfig downloads empty file as folder > - > > Key: SOLR-11198 > URL: https://issues.apache.org/jira/browse/SOLR-11198 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.6 > Environment: Windows 7 >Reporter: Isabelle Giguere >Assignee: Erick Erickson >Priority: Minor > Fix For: 6.7, master (8.0), 7.1 > > Attachments: SOLR-11198.patch, SOLR-11198.patch > > > With Solr 6.6.0, when downloading a config from Zookeeper (3.4.10), if a file > is empty, it is downloaded as a folder (on Windows, at least). > A Zookeeper browser (Eclipse: Zookeeper Explorer) shows the file as a file, > however, in ZK. > Noticed because we keep an empty synonyms.txt file in the Solr config > provided with our product, in case a client would want to use it. > The workaround is simple, since the file allows comments: just add a comment, > so it is not empty. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11198) downconfig downloads empty file as folder
[ https://issues.apache.org/jira/browse/SOLR-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116888#comment-16116888 ] ASF subversion and git services commented on SOLR-11198: Commit 20a963cd7185d22a13a3801b8c06a9498cf39b1c in lucene-solr's branch refs/heads/branch_7x from [~erickerickson] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=20a963c ] SOLR-11198: fix test failures (cherry picked from commit 53db72c) > downconfig downloads empty file as folder > - > > Key: SOLR-11198 > URL: https://issues.apache.org/jira/browse/SOLR-11198 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.6 > Environment: Windows 7 >Reporter: Isabelle Giguere >Assignee: Erick Erickson >Priority: Minor > Fix For: 6.7, master (8.0), 7.1 > > Attachments: SOLR-11198.patch, SOLR-11198.patch > > > With Solr 6.6.0, when downloading a config from Zookeeper (3.4.10), if a file > is empty, it is downloaded as a folder (on Windows, at least). > A Zookeeper browser (Eclipse: Zookeeper Explorer) shows the file as a file, > however, in ZK. > Noticed because we keep an empty synonyms.txt file in the Solr config > provided with our product, in case a client would want to use it. > The workaround is simple, since the file allows comments: just add a comment, > so it is not empty. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11198) downconfig downloads empty file as folder
[ https://issues.apache.org/jira/browse/SOLR-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116856#comment-16116856 ] ASF subversion and git services commented on SOLR-11198: Commit 53db72c5985fd6d0027b6888683973ae764c2f85 in lucene-solr's branch refs/heads/master from [~erickerickson] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=53db72c ] SOLR-11198: fix test failures > downconfig downloads empty file as folder > - > > Key: SOLR-11198 > URL: https://issues.apache.org/jira/browse/SOLR-11198 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.6 > Environment: Windows 7 >Reporter: Isabelle Giguere >Assignee: Erick Erickson >Priority: Minor > Fix For: 6.7, master (8.0), 7.1 > > Attachments: SOLR-11198.patch, SOLR-11198.patch > > > With Solr 6.6.0, when downloading a config from Zookeeper (3.4.10), if a file > is empty, it is downloaded as a folder (on Windows, at least). > A Zookeeper browser (Eclipse: Zookeeper Explorer) shows the file as a file, > however, in ZK. > Noticed because we keep an empty synonyms.txt file in the Solr config > provided with our product, in case a client would want to use it. > The workaround is simple, since the file allows comments: just add a comment, > so it is not empty. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11198) downconfig downloads empty file as folder
[ https://issues.apache.org/jira/browse/SOLR-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115994#comment-16115994 ] ASF subversion and git services commented on SOLR-11198: Commit a3c360eddfee65d9eae70106ac941bcf6b946852 in lucene-solr's branch refs/heads/branch_7x from [~erickerickson] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a3c360e ] SOLR-11198: downconfig downloads empty file as folder, test failures possible fix and logging (cherry picked from commit e053e22) > downconfig downloads empty file as folder > - > > Key: SOLR-11198 > URL: https://issues.apache.org/jira/browse/SOLR-11198 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.6 > Environment: Windows 7 >Reporter: Isabelle Giguere >Assignee: Erick Erickson >Priority: Minor > Fix For: 6.7, master (8.0), 7.1 > > Attachments: SOLR-11198.patch, SOLR-11198.patch > > > With Solr 6.6.0, when downloading a config from Zookeeper (3.4.10), if a file > is empty, it is downloaded as a folder (on Windows, at least). > A Zookeeper browser (Eclipse: Zookeeper Explorer) shows the file as a file, > however, in ZK. > Noticed because we keep an empty synonyms.txt file in the Solr config > provided with our product, in case a client would want to use it. > The workaround is simple, since the file allows comments: just add a comment, > so it is not empty. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11198) downconfig downloads empty file as folder
[ https://issues.apache.org/jira/browse/SOLR-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115992#comment-16115992 ] ASF subversion and git services commented on SOLR-11198: Commit e053e2298fb2a6c67e3ccc244413b98648c249cb in lucene-solr's branch refs/heads/master from [~erickerickson] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e053e22 ] SOLR-11198: downconfig downloads empty file as folder, test failures possible fix and logging > downconfig downloads empty file as folder > - > > Key: SOLR-11198 > URL: https://issues.apache.org/jira/browse/SOLR-11198 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.6 > Environment: Windows 7 >Reporter: Isabelle Giguere >Assignee: Erick Erickson >Priority: Minor > Fix For: 6.7, master (8.0), 7.1 > > Attachments: SOLR-11198.patch, SOLR-11198.patch > > > With Solr 6.6.0, when downloading a config from Zookeeper (3.4.10), if a file > is empty, it is downloaded as a folder (on Windows, at least). > A Zookeeper browser (Eclipse: Zookeeper Explorer) shows the file as a file, > however, in ZK. > Noticed because we keep an empty synonyms.txt file in the Solr config > provided with our product, in case a client would want to use it. > The workaround is simple, since the file allows comments: just add a comment, > so it is not empty. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11198) downconfig downloads empty file as folder
[ https://issues.apache.org/jira/browse/SOLR-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115875#comment-16115875 ] ASF subversion and git services commented on SOLR-11198: Commit c0ee11f0a90b60a0db82132fa9c495a859438386 in lucene-solr's branch refs/heads/branch_6x from [~erickerickson] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c0ee11f ] SOLR-11198: downconfig downloads empty file as folder > downconfig downloads empty file as folder > - > > Key: SOLR-11198 > URL: https://issues.apache.org/jira/browse/SOLR-11198 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.6 > Environment: Windows 7 >Reporter: Isabelle Giguere >Assignee: Erick Erickson >Priority: Minor > Attachments: SOLR-11198.patch > > > With Solr 6.6.0, when downloading a config from Zookeeper (3.4.10), if a file > is empty, it is downloaded as a folder (on Windows, at least). > A Zookeeper browser (Eclipse: Zookeeper Explorer) shows the file as a file, > however, in ZK. > Noticed because we keep an empty synonyms.txt file in the Solr config > provided with our product, in case a client would want to use it. > The workaround is simple, since the file allows comments: just add a comment, > so it is not empty. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11198) downconfig downloads empty file as folder
[ https://issues.apache.org/jira/browse/SOLR-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115852#comment-16115852 ] ASF subversion and git services commented on SOLR-11198: Commit 002b008c83f79e01ddd22d53f99c33976f966366 in lucene-solr's branch refs/heads/branch_7x from [~erickerickson] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=002b008 ] SOLR-11198: downconfig downloads empty file as folder (cherry picked from commit 522367c) > downconfig downloads empty file as folder > - > > Key: SOLR-11198 > URL: https://issues.apache.org/jira/browse/SOLR-11198 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.6 > Environment: Windows 7 >Reporter: Isabelle Giguere >Assignee: Erick Erickson >Priority: Minor > Attachments: SOLR-11198.patch > > > With Solr 6.6.0, when downloading a config from Zookeeper (3.4.10), if a file > is empty, it is downloaded as a folder (on Windows, at least). > A Zookeeper browser (Eclipse: Zookeeper Explorer) shows the file as a file, > however, in ZK. > Noticed because we keep an empty synonyms.txt file in the Solr config > provided with our product, in case a client would want to use it. > The workaround is simple, since the file allows comments: just add a comment, > so it is not empty. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11198) downconfig downloads empty file as folder
[ https://issues.apache.org/jira/browse/SOLR-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115850#comment-16115850 ] ASF subversion and git services commented on SOLR-11198: Commit 522367cc4db3b23ddd728b407f2d10326bb9c858 in lucene-solr's branch refs/heads/master from [~erickerickson] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=522367c ] SOLR-11198: downconfig downloads empty file as folder > downconfig downloads empty file as folder > - > > Key: SOLR-11198 > URL: https://issues.apache.org/jira/browse/SOLR-11198 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.6 > Environment: Windows 7 >Reporter: Isabelle Giguere >Assignee: Erick Erickson >Priority: Minor > Attachments: SOLR-11198.patch > > > With Solr 6.6.0, when downloading a config from Zookeeper (3.4.10), if a file > is empty, it is downloaded as a folder (on Windows, at least). > A Zookeeper browser (Eclipse: Zookeeper Explorer) shows the file as a file, > however, in ZK. > Noticed because we keep an empty synonyms.txt file in the Solr config > provided with our product, in case a client would want to use it. > The workaround is simple, since the file allows comments: just add a comment, > so it is not empty. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11198) downconfig downloads empty file as folder
[ https://issues.apache.org/jira/browse/SOLR-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115470#comment-16115470 ] Erick Erickson commented on SOLR-11198: --- I think I see the problem and that this particular issue should be fixed, i.e. empty znodes with no children should be files locally rather than directories. I don't think I want to try anything fancier. If a user copies down from ZK and the ZK state changes (i.e. a node with data gets children etc.) then erroring out and recommending that they use a clean local directory seems reasonable. The rest of this is details you can read through if you want to or wonder what the heck the second point means. I think the problem is in ZkMaintenanceUtils.downloadFromZk. This code {code} if (children.size() == 0) { // If we didn't copy data down, then we also didn't create the file. But we still need a marker on the local // disk so create a dir. if (copyDataDown(zkClient, zkPath, file.toFile()) == 0) { Files.createDirectories(file); } {code} copyDataDown returns, well, zero if there's no data in the znode. I changed this code a while back because recursive copy wasn't working correctly (SOLR-10108) and I suspect it was introduced then. And it went in Solr 6.6 so it fits Isabelle's testing, including the bit about putting a comment in the stopwords.txt makes it a file rather than a directory.. zNodes can have both data and children. The logic says, in effect, "if the znode has no data and no children it'll be mapped into an empty directory". There's logic in there that if a znode has both data and children, it's made into a directory with a special file containing its data (zknode.data). The simple fix would be just to decide the other way, i.e. a znode with no data and no children would become an empty file locally rather than a directory. That actually seems OK since I don't think ZK cares. The more I think about this the more I think the correct behavior is the easy fix above. ZK doesn't care; a znode can have data added and children added at will so if we make the local node an empty text file it'd be copied back up as a znode just like any other, albeit one without data or children, but that's OK as far as ZK is concerned. That doesn't preclude someone adding children to the ZK node after pushing it back up via another mechanism. There'll still be an edge case where - someone copies an empty znode from ZK and it becomes a file - the ZK node gets children - the person tries the copy again from ZK to local Since the copy down now already has a text file for that znode and then tries to make a directory there it'll probably error out. At least it better. I think we can live with that, it would be a good thing to add a test though. I'm torn about whether to try to "do the right thing" in the case above. First, it appears to much of an edge case. One could write something like: if (the local file is zero length) { remove the file create a directory in it's place } Which seems relatively safe. Except that doesn't handle the case where - a znode exists with data - downconfig copies it locally as a text file - the znode gets children through some other mechanism - downconfig is run again In that case "the right thing" would be to move the data to the zknode.data and make the local node into a directory and continue. Probably could do this "en passant" by just deleting the text file and continuing, the directory would be add it in its place and children would be added. Deleting data on the local disk makes me nervous though. I'll assign it to myself, if anyone else wants to take it feel free. > downconfig downloads empty file as folder > - > > Key: SOLR-11198 > URL: https://issues.apache.org/jira/browse/SOLR-11198 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.6 > Environment: Windows 7 >Reporter: Isabelle Giguere >Priority: Minor > > With Solr 6.6.0, when downloading a config from Zookeeper (3.4.10), if a file > is empty, it is downloaded as a folder (on Windows, at least). > A Zookeeper browser (Eclipse: Zookeeper Explorer) shows the file as a file, > however, in ZK. > Noticed because we keep an empty synonyms.txt file in the Solr config > provided with our product, in case a client would want to use it. > The workaround is simple, since the file allows comments: just add a comment, > so it is not empty. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11198) downconfig downloads empty file as folder
[ https://issues.apache.org/jira/browse/SOLR-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115023#comment-16115023 ] Isabelle Giguere commented on SOLR-11198: - My command (on Windows) > java -cp "C:\Path\to\lib\*" org.apache.solr.cloud.ZkCLI -cmd downconfig > -zkhost localhost:2181/ot/solr -confname otif_en -confdir > "C:\Path\to\output\otif_en" Note that I cannot reproduce the issue using Zookeeper 3.4.6 and Solr 5.4.1, with exactly the same command. We just upgraded to Zookeeper 3.4.10 and Solr 6.6.0 I'll try Zookeeper 3.4.6 with Solr 6.6.0, and Zookeeper 3.4.10 with Solr 5.4.1, if it can help narrow things down. > downconfig downloads empty file as folder > - > > Key: SOLR-11198 > URL: https://issues.apache.org/jira/browse/SOLR-11198 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.6 > Environment: Windows 7 >Reporter: Isabelle Giguere >Priority: Minor > > With Solr 6.6.0, when downloading a config from Zookeeper (3.4.10), if a file > is empty, it is downloaded as a folder (on Windows, at least). > A Zookeeper browser (Eclipse: Zookeeper Explorer) shows the file as a file, > however, in ZK. > Noticed because we keep an empty synonyms.txt file in the Solr config > provided with our product, in case a client would want to use it. > The workaround is simple, since the file allows comments: just add a comment, > so it is not empty. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11198) downconfig downloads empty file as folder
[ https://issues.apache.org/jira/browse/SOLR-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114961#comment-16114961 ] Cassandra Targett commented on SOLR-11198: -- We need a bit more info on how you are downloading the config - the command you are using - to be sure there isn't an error somewhere in there. I tried it on my machine (Mac), and it worked fine, so we want to be clear how you are using it so we can reproduce. Here's the command I used: {{./bin/solr zk cp zk:/configs/gettingstarted/stopwords.txt file:/Desktop -z localhost:9983}} (When I did this, I modified the default stopwords.txt so it is empty). > downconfig downloads empty file as folder > - > > Key: SOLR-11198 > URL: https://issues.apache.org/jira/browse/SOLR-11198 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.6 > Environment: Windows 7 >Reporter: Isabelle Giguere >Priority: Minor > > With Solr 6.6.0, when downloading a config from Zookeeper (3.4.10), if a file > is empty, it is downloaded as a folder (on Windows, at least). > A Zookeeper browser (Eclipse: Zookeeper Explorer) shows the file as a file, > however, in ZK. > Noticed because we keep an empty synonyms.txt file in the Solr config > provided with our product, in case a client would want to use it. > The workaround is simple, since the file allows comments: just add a comment, > so it is not empty. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org