[
https://issues.apache.org/jira/browse/SOLR-7187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365865#comment-14365865
]
Mike Drob commented on SOLR-7187:
---------------------------------
Took another crack at this, I haven't been able to find an elegant solution
yet. The {{OverseerCollectionProcessor}} has no reference to the file system to
do the directory clean-up. I was thinking that maybe we can add to the
{{CoreAdminHandler}} as part of the unload command, if the parent directory is
empty (i.e. we've deleted all the data in HDFS) then delete the core directory
too. Following this logic, the handler delegates the deletes to the
{{HdfsDirectoryFactory}}, but it's all actually implemented in
{{CachingDirectoryFactory}} and only happens on {{close()}}. But by the time
that we get to closing the directory factory, we've lost all information about
whether we are deleting instance and data directories, we just have a list of
things to remove and it's tought to extrapolate from there. So this approach
might be a dead end.
I think some of this will be made easier with zk=truth and it makes sense to
revisit again at that point.
> SolrCloud does not fully clean collection after delete
> ------------------------------------------------------
>
> Key: SOLR-7187
> URL: https://issues.apache.org/jira/browse/SOLR-7187
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
> Affects Versions: 4.10.3
> Reporter: Mike Drob
> Attachments: log.out.gz
>
>
> When I attempt to delete a collection using
> {{/admin/collections?action=DELETE&name=collection1}} if I go into HDFS I
> will still see remnants from the collection. No files, but empty directories
> stick around.
> {noformat}
> [root@solr1 ~]# sudo -u hdfs hdfs dfs -ls -R /solr/collection1
> drwxr-xr-x - solr solr 0 2015-03-03 15:42
> /solr/collection1/core_node1
> drwxr-xr-x - solr solr 0 2015-03-03 15:42
> /solr/collection1/core_node2
> drwxr-xr-x - solr solr 0 2015-03-03 15:42
> /solr/collection1/core_node3
> drwxr-xr-x - solr solr 0 2015-03-03 15:42
> /solr/collection1/core_node4
> drwxr-xr-x - solr solr 0 2015-03-03 15:42
> /solr/collection1/core_node5
> drwxr-xr-x - solr solr 0 2015-03-03 15:42
> /solr/collection1/core_node6
> {noformat}
> (Edit: I had the wrong log portion here originally)
> In the logs, after deleting all the data, I see:
> {noformat}
> 2015-03-03 16:15:14,762 INFO org.apache.solr.servlet.SolrDispatchFilter:
> [admin] webapp=null path=/admin/cores
> params={deleteInstanceDir=true&action=UNLOAD&core=collection1_shard5_replica1&wt=javabin&qt=/admin/cores&deleteDataDir=true&version=2}
> status=0 QTime=362
> 2015-03-03 16:15:14,787 INFO org.apache.solr.common.cloud.ZkStateReader: A
> cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> 2015-03-03 16:15:14,854 INFO org.apache.solr.cloud.DistributedQueue:
> LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
> NodeChildrenChanged
> 2015-03-03 16:15:14,879 INFO org.apache.solr.cloud.DistributedQueue:
> LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
> NodeChildrenChanged
> 2015-03-03 16:15:14,896 INFO org.apache.solr.cloud.DistributedQueue:
> LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
> NodeChildrenChanged
> 2015-03-03 16:15:14,920 INFO org.apache.solr.cloud.DistributedQueue:
> LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
> NodeChildrenChanged
> 2015-03-03 16:15:15,151 INFO org.apache.solr.cloud.DistributedQueue:
> LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
> NodeChildrenChanged
> 2015-03-03 16:15:15,170 INFO org.apache.solr.cloud.DistributedQueue:
> LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
> NodeChildrenChanged
> 2015-03-03 16:15:15,279 INFO org.apache.solr.common.cloud.ZkStateReader: A
> cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> 2015-03-03 16:15:15,546 INFO org.apache.solr.cloud.DistributedQueue:
> LatchChildWatcher fired on path:
> /overseer/collection-queue-work/qnr-0000000016 state: SyncConnected type
> NodeDataChanged
> 2015-03-03 16:15:15,562 INFO org.apache.solr.cloud.DistributedQueue:
> LatchChildWatcher fired on path: /overseer/collection-queue-work state:
> SyncConnected type NodeChildrenChanged
> 2015-03-03 16:15:15,562 INFO
> org.apache.solr.cloud.OverseerCollectionProcessor: Overseer Collection
> Processor: Message id:/overseer/collection-queue-work/qn-0000000016 complete,
> response:{success={solr1.example.com:8983_solr={responseHeader={status=0,QTime=207}},solr1.example.com:8983_solr={responseHeader={status=0,QTime=243}},solr1.example.com:8983_solr={responseHeader={status=0,QTime=243}},solr1.example.com:8983_solr={responseHeader={status=0,QTime=342}},solr1.example.com:8983_solr={responseHeader={status=0,QTime=346}},solr1.example.com:8983_solr={responseHeader={status=0,QTime=362}}}}
> {noformat}
> This might be related to SOLR-5023, but I'm not sure.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
