[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16875008#comment-16875008 ] Mikhail Khludnev commented on SOLR-9242: I'm wondering why BackupRepository isn't ever closed. I suppose it causes leaks hdfs stuff in TestHdfsCloudBackupRestore. Isn't it worth to close repo every time? Or it's better to keep repo instances in BRFactory? I'm asking in scope of SOLR-9961 where we need to spawn threadpool once or per repo. > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker >Priority: Major > Fix For: 6.2, 7.0 > > Attachments: 7726.log.gz, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch, > SOLR-9242_followup2.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15487169#comment-15487169 ] Varun Thacker commented on SOLR-9242: - Great! Then we don't need any more work on this > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2, master (7.0) > > Attachments: 7726.log.gz, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch, > SOLR-9242_followup2.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15487139#comment-15487139 ] Steve Rowe commented on SOLR-9242: -- bq. Steve Rowe You had mentioned that this reproduced for you. Does it still reproduce ? I just tried it on trunk and it no longer reproduces for me. > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2, master (7.0) > > Attachments: 7726.log.gz, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch, > SOLR-9242_followup2.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[
https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15486457#comment-15486457
]
Varun Thacker commented on SOLR-9242:
-
We should resolve this issue since this feature has been released.
bq. Secondly I see the operation is being retried on failure. That doesn't look
good.
This was fixed in SOLR-9445
If {{ant test -Dtestcase=TestHdfsCloudBackupRestore -Dtests.method=test
-Dtests.seed=9470C0688BFAF297 -Dtests.slow=true
-Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt
-Dtests.locale=es-MX -Dtests.timezone=HST -Dtests.asserts=true
-Dtests.file.encoding=UTF-8}} still reproduces then lets track it on a separate
Jira.
I ran it a few times and it didn't fail for me on my Mac, I didn't see any
jenkins failure on this recently as well. Maybe it was fixed as part of
SOLR-9444 ?
[~steve_rowe] You had mentioned that this reproduced for you. Does it still
reproduce ? If yes then I'll create a separate Jira and try to fix it
> Collection level backup/restore should provide a param for specifying the
> repository implementation it should use
> -
>
> Key: SOLR-9242
> URL: https://issues.apache.org/jira/browse/SOLR-9242
> Project: Solr
> Issue Type: Improvement
>Reporter: Hrishikesh Gadre
>Assignee: Varun Thacker
> Fix For: 6.2, master (7.0)
>
> Attachments: 7726.log.gz, SOLR-9242.patch, SOLR-9242.patch,
> SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch,
> SOLR-9242_followup2.patch
>
>
> SOLR-7374 provides BackupRepository interface to enable storing Solr index
> data to a configured file-system (e.g. HDFS, local file-system etc.). This
> JIRA is to track the work required to extend this functionality at the
> collection level.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15484186#comment-15484186 ] Hrishikesh Gadre commented on SOLR-9242: [~varunthacker] Now that SOLR-9444 is resolved, should we close this JIRA? Or are there any recent test failures due to this functionality? > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2, master (7.0) > > Attachments: 7726.log.gz, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch, > SOLR-9242_followup2.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439198#comment-15439198 ] Uwe Schindler commented on SOLR-9242: - Thanks! I have seen it and already respondend. I have an proposal, but I did not look into the code fully. Sorry for the noise here, but URI.getPath() and URL.getPath() are 2 of the methods that are used wrongly all the time. The Maven People already have a nice blog post about that, because theis is also an issue when building classpaths and so on. If you don't read the documentation of those methods correctly, its almost always wrong. The problem is Linux, where you mostly don't see the corner cases, but people on Windows with whitespace in path names and drive letters always hit those issues. > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2, master (7.0) > > Attachments: 7726.log.gz, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch, > SOLR-9242_followup2.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439146#comment-15439146 ] Varun Thacker commented on SOLR-9242: - Hi Uwe, I created SOLR-9444 to track this. I'll start looking into this ASAP . Would appreciate your feedback on the Jira as well > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2, master (7.0) > > Attachments: 7726.log.gz, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch, > SOLR-9242_followup2.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439145#comment-15439145 ] Varun Thacker commented on SOLR-9242: - Hi Uwe, I created SOLR-9444 to track this. I'll start looking into this ASAP . Would appreciate your feedback on the Jira as well > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2, master (7.0) > > Attachments: 7726.log.gz, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch, > SOLR-9242_followup2.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438930#comment-15438930 ] Uwe Schindler commented on SOLR-9242: - I just checked the code: This is a major desaster. The method URI#getPath() and URL.getPath() was already proposed to be put on the forbidden-apis list. I think it's time to do this. This is not acceptable, we should fix asap. To me that would have been also a reason to -1 the release of Solr 5.2! > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2, master (7.0) > > Attachments: 7726.log.gz, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch, > SOLR-9242_followup2.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438925#comment-15438925 ] Uwe Schindler commented on SOLR-9242: - Hi, can we fix this please in a correct way? The Stackoverflow issue is IMHO showing incompetence of the typical "Stackoverflow" user. URI.getPath() returns the URI path and never ever a filesystem path. The fix is also incorrect. It also breaks if you have whitespace in your path!!! This explains now why this test always fails on my system! To convert an URI to a Path use Paths.get(URI), and not URI.getPath(). > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2, master (7.0) > > Attachments: 7726.log.gz, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch, > SOLR-9242_followup2.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15424567#comment-15424567 ] Varun Thacker commented on SOLR-9242: - hdfsbackuprestore_shard1_0_replica0 - doesn't sound right. We should create another issue to make sure its atleast replica1 Secondly I see the operation is being retried on failure. That doesn't look good. I'll try looking at the main issue as well > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2, master (7.0) > > Attachments: 7726.log.gz, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch, > SOLR-9242_followup2.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421104#comment-15421104 ] Hrishikesh Gadre commented on SOLR-9242: [~steve_rowe] Thanks for the update. Let me take a look and get back to you. > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2, master (7.0) > > Attachments: 7726.log.gz, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch, > SOLR-9242_followup2.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[
https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421046#comment-15421046
]
Steve Rowe commented on SOLR-9242:
--
My Jenkins found a {{TestHdfsCloudBackupRestore}} failure that reproduces for
me on master:
{noformat}
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestHdfsCloudBackupRestore -Dtests.method=test
-Dtests.seed=9470C0688BFAF297 -Dtests.slow=true
-Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt
-Dtests.locale=es-MX -Dtests.timezone=HST -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
[junit4] ERROR 38.6s J5 | TestHdfsCloudBackupRestore.test <<<
[junit4]> Throwable #1:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
from server at http://127.0.0.1:37868/solr: Collection
'hdfsbackuprestore_restored' exists, no action taken.
[junit4]>at
__randomizedtesting.SeedInfo.seed([9470C0688BFAF297:1C24FFB225069F6F]:0)
[junit4]>at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:608)
[junit4]>at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
[junit4]>at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
[junit4]>at
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:413)
[junit4]>at
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:366)
[junit4]>at
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1291)
[junit4]>at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
[junit4]>at
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
[junit4]>at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
[junit4]>at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
[junit4]>at
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:232)
[junit4]>at
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:126)
[junit4]>at java.lang.Thread.run(Thread.java:745)
{noformat}
> Collection level backup/restore should provide a param for specifying the
> repository implementation it should use
> -
>
> Key: SOLR-9242
> URL: https://issues.apache.org/jira/browse/SOLR-9242
> Project: Solr
> Issue Type: Improvement
>Reporter: Hrishikesh Gadre
>Assignee: Varun Thacker
> Fix For: 6.2, master (7.0)
>
> Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch,
> SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch,
> SOLR-9242_followup2.patch
>
>
> SOLR-7374 provides BackupRepository interface to enable storing Solr index
> data to a configured file-system (e.g. HDFS, local file-system etc.). This
> JIRA is to track the work required to extend this functionality at the
> collection level.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15411235#comment-15411235 ] ASF subversion and git services commented on SOLR-9242: --- Commit 9c37aaabe4e1b3f66cd0034e2e3fe0399e82fffd in lucene-solr's branch refs/heads/branch_6x from [~varunthacker] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9c37aaa ] SOLR-9242: fix windows path issue + load live cluster properties. This closes #62 > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2, master (7.0) > > Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch, > SOLR-9242_followup2.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408803#comment-15408803 ] ASF GitHub Bot commented on SOLR-9242: -- Github user asfgit closed the pull request at: https://github.com/apache/lucene-solr/pull/62 > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2 > > Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch, > SOLR-9242_followup2.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408802#comment-15408802 ] ASF subversion and git services commented on SOLR-9242: --- Commit d5a7ca79f3ac88d4de54c013eb6b29f72e52c907 in lucene-solr's branch refs/heads/master from [~varunthacker] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d5a7ca7 ] SOLR-9242: fix windows path issue + load live cluster properties. This closes #62 > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2 > > Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch, > SOLR-9242_followup2.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15407912#comment-15407912 ] Hrishikesh Gadre commented on SOLR-9242: [~varunthacker] Thanks for the review and updated patch. Looks good to me! > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2 > > Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch, > SOLR-9242_followup2.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15406641#comment-15406641 ] ASF GitHub Bot commented on SOLR-9242: -- GitHub user hgadre opened a pull request: https://github.com/apache/lucene-solr/pull/62 [SOLR-9242] Fix unit test failure on Windows platform. The root cause of the failure is the usage of URI::getPath() method in the backup/restore functionality (e.g. in BackupManager::downloadFromZK OR in the OverseerCollectionMessageHandler::processBackupAction methods). On the Windows platform, usage of URI.getPath() returns an invalid path string (e.g. URI file:///C:/lucene-solr/solr returns /C:/lucene-solr/solr as the result of getPath() method). Refer to following discussion for more details, http://stackoverflow.com/questions/9834776/java-nio-file-path-issue Since the caller may have used this method to generate the string representation for the pathComponents, we implement a work-around specifically for Windows platform to remove the leading '/' character. You can merge this pull request into a Git repository by running: $ git pull https://github.com/hgadre/lucene-solr SOLR-9242_fix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/lucene-solr/pull/62.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #62 > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2 > > Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15406644#comment-15406644 ] ASF GitHub Bot commented on SOLR-9242: -- Github user hgadre commented on the issue: https://github.com/apache/lucene-solr/pull/62 @vthacker could you please take a look? > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2 > > Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15394308#comment-15394308 ] ASF subversion and git services commented on SOLR-9242: --- Commit 12449282624a2342dde6a51a90872b104a23560a in lucene-solr's branch refs/heads/branch_6x from [~varunthacker] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1244928 ] SOLR-9242: Disabling TestLocalFSCloudBackupRestore on Windows till we fix it > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2 > > Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15394305#comment-15394305 ] ASF subversion and git services commented on SOLR-9242: --- Commit 219406d912e027195145de2e77f35f41a6116c6d in lucene-solr's branch refs/heads/master from [~varunthacker] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=219406d ] SOLR-9242: Disabling TestLocalFSCloudBackupRestore on Windows till we fix it > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2 > > Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387322#comment-15387322 ] Varun Thacker commented on SOLR-9242: - Thanks for Investigating! I've been swamped with work otherwise I would have got to this sooner. > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2 > > Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386814#comment-15386814 ] Hrishikesh Gadre commented on SOLR-9242: Found one more unit test failure on Windows. Investigating... http://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/5996/ > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2 > > Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376105#comment-15376105 ] Hrishikesh Gadre commented on SOLR-9242: [~varunthacker] Sorry for late reply. It seems like the changes committed regarding the force loading the cluster properties are in direct contradiction with SOLR-9106. Since this is unrelated to backup/restore, I think we should have handled it separately. Regarding windows failure - I don't have the windows setup. But let me figure something out and provide a fix... > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2 > > Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15374696#comment-15374696 ] ASF subversion and git services commented on SOLR-9242: --- Commit 1dfc637f2ca10242afeb12ba1ad35357ed611029 in lucene-solr's branch refs/heads/branch_6x from [~varunthacker] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1dfc637 ] SOLR-9242: Move license headers to the top + force refresh cluster propery before reading the 'location' param > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2 > > Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15374693#comment-15374693 ] ASF subversion and git services commented on SOLR-9242: --- Commit eefdc62c997f7936b6db203111d8149dc934b243 in lucene-solr's branch refs/heads/master from [~varunthacker] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=eefdc62 ] SOLR-9242: Move license headers to the top + force refresh cluster propery before reading the 'location' param > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2 > > Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch, SOLR-9242_followup.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[
https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370234#comment-15370234
]
Varun Thacker commented on SOLR-9242:
-
There's another type of test failure :
https://builds.apache.org/job/Lucene-Solr-Tests-master/1270/console
In this case the ZK watch for updating the clusterprop occurs after the backup
command has been invoked . Hence it doesn't find the value set in the
cluster-prop api
{code}
[junit4] 2> 819240 INFO (qtp102693249-7831) [n:127.0.0.1:52710_solr]
o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterprop with params
val=/location/does/not/exist&name=location&action=CLUSTERPROP&wt=javabin&version=2
and sendToOCPQueue=true
[junit4] 2> 819241 INFO (qtp102693249-7831) [n:127.0.0.1:52710_solr]
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections
params={val=/location/does/not/exist&name=location&action=CLUSTERPROP&wt=javabin&version=2}
status=0 QTime=0
[junit4] 2> 819242 INFO (qtp102693249-7825) [n:127.0.0.1:52710_solr]
o.a.s.h.a.CollectionsHandler Invoked Collection Action :backup with params
name=invalidbackuprequest&action=BACKUP&collection=backuprestore&wt=javabin&version=2
and sendToOCPQueue=true
[junit4] 2> 819242 INFO (zkCallback-1166-thread-1) []
o.a.s.c.c.ZkStateReader Loaded cluster properties:
{location=/location/does/not/exist}
[junit4] 2> 819242 INFO
(zkCallback-1160-thread-2-processing-n:127.0.0.1:41141_solr)
[n:127.0.0.1:41141_solr] o.a.s.c.c.ZkStateReader Loaded cluster properties:
{location=/location/does/not/exist}
[junit4] 2> 819242 ERROR (qtp102693249-7825) [n:127.0.0.1:52710_solr]
o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: 'location' is
not specified as a query parameter or as a default repository property or as a
cluster property.
[junit4] 2>at
org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation$28.call(CollectionsHandler.java:822)
[junit4] 2>at
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:207)
[junit4] 2>at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)
[junit4] 2>at
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:659)
[junit4] 2>at
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)
[junit4] 2>at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
[junit4] 2>at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
[junit4] 2>at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] 2>at
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:111)
[junit4] 2>at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] 2>at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
[junit4] 2>at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2>at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
[junit4] 2>at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
[junit4] 2>at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2>at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
[junit4] 2>at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2>at
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
[junit4] 2>at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2>at
org.eclipse.jetty.server.Server.handle(Server.java:518)
[junit4] 2>at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
[junit4] 2>at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
[junit4] 2>at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2>at
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2>at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2>at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
[junit4] 2>at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
[junit4] 2>at
org.eclipse.jetty.util.thread.QueuedThreadPoo
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369165#comment-15369165 ] David Smiley commented on SOLR-9242: BTW we should be putting our ASF source headers at the very top. I notice these files put them in other places, and in at least one place moved existing ones to another ordering. I wish precommit with complain about this. > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Fix For: 6.2 > > Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[
https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15367524#comment-15367524
]
Varun Thacker commented on SOLR-9242:
-
We have a test failure for windows :
Log excerpt :
{code}
[junit4] 2> Caused by: java.nio.file.InvalidPathException: Illegal char
<:> at index 2:
/C:/Users/jenkins/workspace/Lucene-Solr-6.x-Windows/solr/build/solr-core/test/J1/temp/solr.cloud.TestLocalFSCloudBackupRestore_168D4B6DEE507089-001/tempDir-002/mytestbackup
[junit4] 2>at
sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
[junit4] 2>at
sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
[junit4] 2>at
sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
[junit4] 2>at sun.nio.fs.WindowsPath.parse(WindowsPath.java:94)
[junit4] 2>at
sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:255)
[junit4] 2>at java.nio.file.Paths.get(Paths.java:84)
[junit4] 2>at
org.apache.solr.core.backup.repository.LocalFileSystemRepository.createURI(LocalFileSystemRepository.java:62)
[junit4] 2>at
org.apache.solr.handler.SnapShooter.initialize(SnapShooter.java:85)
[junit4] 2>at
org.apache.solr.handler.SnapShooter.(SnapShooter.java:79)
[junit4] 2>at
org.apache.solr.handler.admin.CoreAdminOperation$19.call(CoreAdminOperation.java:873)
[junit4] 2>... 30 more
{code}
Jenkins failure link :
http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/305/
> Collection level backup/restore should provide a param for specifying the
> repository implementation it should use
> -
>
> Key: SOLR-9242
> URL: https://issues.apache.org/jira/browse/SOLR-9242
> Project: Solr
> Issue Type: Improvement
>Reporter: Hrishikesh Gadre
>Assignee: Varun Thacker
> Fix For: 6.2
>
> Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch,
> SOLR-9242.patch, SOLR-9242.patch
>
>
> SOLR-7374 provides BackupRepository interface to enable storing Solr index
> data to a configured file-system (e.g. HDFS, local file-system etc.). This
> JIRA is to track the work required to extend this functionality at the
> collection level.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15363935#comment-15363935 ] ASF subversion and git services commented on SOLR-9242: --- Commit 413ea476700429bd39c659cdd0bc6682263b0545 in lucene-solr's branch refs/heads/branch_6x from [~varunthacker] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=413ea47 ] SOLR-9242: Collection backup/restore to provide a param for specifying underlying storage repository to use > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15363930#comment-15363930 ] ASF subversion and git services commented on SOLR-9242: --- Commit bfe5c5ae499499d4198caa71442eb3e4eba45237 in lucene-solr's branch refs/heads/master from [~varunthacker] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bfe5c5a ] SOLR-9242: Collection backup/restore to provide a param for specifying underlying storage repository to use > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch, SOLR-9242.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15361830#comment-15361830 ] Hrishikesh Gadre commented on SOLR-9242: [~varunthacker] Thanks for the review comments. The updated patch looks good mostly. Only couple of minor comments, bq. This prop solr.hdfs.confdir in the solr.xml file never seemed to be getting used? I removed it and the tests pass. Do we need this? The HdfsBackupRepository uses HdfsDirectory internally. This optional property is being used by the HdfsDirectoryFactory. My guess is that in the test environment Hadoop conf directory is not created/initialized. I don't think there is any harm in keeping this configuration option. bq. Changed setRepository(Optional repository) to setRepository(String repository . Seems cleaner from an API perspective given it's a setter. I am OK with changing the setRepository parameter from Optional to String. But it looks like you have changed the attribute type as well. Since we are using Java 8 now, we should use Optional type to clearly specify optional attributes. This helps to improve the readability of the code. Here is a good article about the Java 8 Optional type. http://blog.codefx.org/techniques/intention-revealing-code-java-8-optional/ How about following? - The parameter type of setter should be String. The setter method should initialize the attribute based on its nullability. - The type of attribute as well as the getter should be Optional. This clearly indicates that the attribute is optional (without having to read the code comment in the CollectionAdminRequest class). - In the getParams method - replace the null check with isPresent() method call. > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch, > SOLR-9242.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359007#comment-15359007 ] Varun Thacker commented on SOLR-9242: - Hi Hrishikesh, Thanks for the patch! I'll have a look at this over the weekend. > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Attachments: SOLR-9242.patch, SOLR-9242.patch, SOLR-9242.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357790#comment-15357790 ] Hrishikesh Gadre commented on SOLR-9242: [~varunthacker] Filed SOLR-9268 to track API support for solr.xml configuration. > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Attachments: SOLR-9242.patch, SOLR-9242.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356701#comment-15356701 ] Varun Thacker commented on SOLR-9242: - Hi Hrishikesh, bq. we can restrict the users to configure only a single repository at-a-time. This will avoid the problem mentioned above and they can use the current property. Personally I don't like the idea of limiting our users to one repo in all of the 6.x line. Let's say we follow this order: 1. If "location" param was provided as a query param use that 2. Else if the "repository" in the solr.xml has a "location" param use that. 3. If the "repository" specified doesn't specify a "location" param then see if it's specified via the cluster property API . The code will throw an error if the location was bogus or was with not for this repo. It has to fail as the repository will fail to read / write to that location. I thought about the "repoName:/path" syntax idea that you proposed . Seems to me that we want to do all of this because solr.xml doesn't have an API to update it . We have to hand edit the file and upload to ZK at the very least. I think let's not complicate it any further? Keep the location cluster prop for now the way it is and support it. We can work towards adding API support to solr.xml and then get rid the "location" cluster prop or the entire cluster prop API in the future. > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Attachments: SOLR-9242.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353377#comment-15353377 ] Hrishikesh Gadre commented on SOLR-9242: [~varunthacker] [~markrmil...@gmail.com] Any thoughts on this? > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Attachments: SOLR-9242.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348809#comment-15348809 ] Hrishikesh Gadre commented on SOLR-9242: [~varunthacker] Thanks for the review. bq. But now that its out and documented I don't think we can drop it easily. Seems like a good way to override location anyways? We'll need to add support for it now Since we allow users to configure multiple repositories in solr.xml, we can not use the current cluster property as is. This is because user may want to specify different location for different file-systems (or repositories). Hence at minimum we need one cluster property per repository configuration (e.g. name could be -location). For 6.x releases, we can restrict the users to configure only a *single* repository at-a-time. This will avoid the problem mentioned above and they can use the current property. For 7.x releases, we have multiple options, - Support configuring multiple cluster properties one per repository configuration. This will require some changes in CLUSTERPROP API since currently it requires fixed (or well-known) property names. In this case we will remove the current cluster property as well as ability to configure *default* location via solr.xml - Continue using the current mechanism of configuring *default* location via solr.xml. Just remove the current cluster property What do you think? > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Attachments: SOLR-9242.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9242) Collection level backup/restore should provide a param for specifying the repository implementation it should use
[ https://issues.apache.org/jira/browse/SOLR-9242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348213#comment-15348213 ] Varun Thacker commented on SOLR-9242: - Hi, I had a quick peek into the patches and looks good overall! Some nice cleanups as well. bq. Removed the cluster property to define a default backup location. The default will now be associated with the backup repository configuration. I wanted to get this hashed out before Solr 6.1 came out so that we know whether we want to support this or not. But now that its out and documented I don't think we can drop it easily. Seems like a good way to override location anyways? We'll need to add support for it now > Collection level backup/restore should provide a param for specifying the > repository implementation it should use > - > > Key: SOLR-9242 > URL: https://issues.apache.org/jira/browse/SOLR-9242 > Project: Solr > Issue Type: Improvement >Reporter: Hrishikesh Gadre >Assignee: Varun Thacker > Attachments: SOLR-9242.patch > > > SOLR-7374 provides BackupRepository interface to enable storing Solr index > data to a configured file-system (e.g. HDFS, local file-system etc.). This > JIRA is to track the work required to extend this functionality at the > collection level. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
