[
https://issues.apache.org/jira/browse/SOLR-17460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884426#comment-17884426
]
Arda commented on SOLR-17460:
-----------------------------
To successfully migrate the collection with all data intact, I followed these
steps:
*1. Prepare the Collection Configuration:*
First, I retrieved the Solr collection's configuration files from the Solr 7.0
cluster and modified them to be compatible with Solr 8.4.
I then created the new collection on the Solr 8.4 cluster without any data,
using the updated configuration.
*2. Copy Shard Core Nodes Locally:*
I copied each shard’s core nodes from the HDFS path of the Solr 7.0 cluster to
a local directory. Using the Solr Web UI, I identified which nodes corresponded
to each shard and replica for the collection. Note that the core node paths may
differ for each collection.
Here’s an example of the commands used:
_# Shard 1, 2, and 3_
hdfs dfs --copyToLocal <solrCollectionPath>/core_node1 <localCorePath> _#
Shard 1_
hdfs dfs --copyToLocal <solrCollectionPath>/core_node2 <localCorePath> _#
Shard 2_
hdfs dfs --copyToLocal <solrCollectionPath>/core_node3 <localCorePath> _#
Shard 3_
_# Replica nodes_
hdfs dfs --copyToLocal <solrCollectionPath>/core_node6 <localCorePath> _#
Replica 1_
hdfs dfs --copyToLocal <solrCollectionPath>/core_node8 <localCorePath> _#
Replica 2_
hdfs dfs --copyToLocal <solrCollectionPath>/core_node10 <localCorePath> _#
Replica 3_
After copying the core node files locally, I transferred them to the Solr 8.4
cluster.
*3. Copy Shard Core Nodes to HDFS on the New Cluster:*
I copied the shard core node files into the appropriate HDFS directory on the
new Solr 8.4 cluster.
*Important:* The shard core nodes must be copied into the exact corresponding
location. For example, if shard 1 was stored in *_core_node1_* in the old
cluster and is now assigned to *_core_node5_* in the new cluster, you must copy
the data from *_core_node1_* to *_core_node5._*
Example commands:
_# Copy shard 1, 2, and 3 core nodes_
hdfs dfs -put <localCorePath>/core_node1 <newSolrCollectionPath>/core_node5 _#
Shard 1_
hdfs dfs -put <localCorePath>/core_node2 <newSolrCollectionPath>/core_node6 _#
Shard 2_
hdfs dfs -put <localCorePath>/core_node3 <newSolrCollectionPath>/core_node7 _#
Shard 3_
_# Copy replica core nodes_
hdfs dfs -put <localCorePath>/core_node6 <newSolrCollectionPath>/core_node11 _#
Replica 1_
hdfs dfs -put <localCorePath>/core_node8 <newSolrCollectionPath>/core_node12 _#
Replica 2_
hdfs dfs -put <localCorePath>/core_node10 <newSolrCollectionPath>/core_node9 _#
Replica 3_
*4. Adjust Ownership in HDFS:*
I changed the ownership of the collection’s HDFS path on the new cluster to
ensure Solr had the necessary permissions to access the data.
hdfs dfs -chown -R solr:solr <newSolrCollectionPath>
To verify that the files were correctly copied, I compared the file sizes on
both clusters using the following command:
hdfs dfs -du -s -h -v -x <collectionPath>
*5. Reload the Collection:*
Finally, I reloaded the collection via the Solr Web UI on the new Solr 8.4
cluster.
To confirm the successful migration of all data, I queried the collection and
verified that the document count matched the original.
With these steps, I was able to migrate the entire collection along with all
data. You can check the document count by running a simple query to verify that
the migration was successful.
> Error During Collection Migration from Solr 7.0 to Solr 8.4: Missing Files
> and Shard Restoration Failures
> ---------------------------------------------------------------------------------------------------------
>
> Key: SOLR-17460
> URL: https://issues.apache.org/jira/browse/SOLR-17460
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: hdfs, SolrCloud
> Affects Versions: 7.0, 8.4
> Reporter: Arda
> Priority: Minor
> Labels: backup, restore
>
> I was attempting to migrate a collection with 3 shards from a Solr 7.0
> cluster to a Solr 8.4 cluster. The data is stored in HDFS. I followed the
> backup-restore process but encountered issues with two of the shards during
> the restoration.
> h1. *Migration Process:*
> *1-* *Backup Command:* To avoid timeouts, I initiated the backup with an
> async parameter:
> curl -k --negotiate -u :
> 'https://<solrNode>:<solrPort>/solr/admin/collections?action=BACKUP&name=<backupName>&collection=<solrCollectionName>x&location=<hdfsPath>&
> async=12346'
> *2- Copy Backup to Local:* After the backup, I copied the data from HDFS to
> the local filesystem:
> hdfs dfs --copyToLocal <backupPath> <localPath>
> *3- Transfer Backup to New Cluster:* I then copied the backup files from the
> older Solr node to the newer one:
> scp -pr <localPath> <username>@<ip>:<localPathDestination>
> *4- Prepare New HDFS Path:* On the new Solr cluster, I created a new
> directory in HDFS and adjusted ownership:
> hdfs dfs -mkdir <pathName2>
> hdfs dfs -chown solr:solr <pathName2>
> *5- Copy Backup to New HDFS Location:* I transferred the backup data from
> local to the new HDFS path. Before that, I deleted
> "<str>queryDocAuthorization</str>" parts from solrconfig.xml file to become
> compatible with the newer version.
> hdfs dfs --copyFromLocal <localPathDestination> <pathName2>
> *6- Restore Collection:* Finally, I ran the restore command:
> curl -k --negotiate -u :
> 'https://<solrNode>:<solrPort>/solr/admin/collections?action=RESTORE&name=<backupName>&collection=<solrCollectionName>x&location=<hdfs_path>&
> async=12345'
> h1.
> *Issue:*
> After the restore process completed, I found that two of the shards could not
> be restored. The logs displayed the following errors:
> *Error During Shard Restoration:*
> ERROR [c: <solrCollectionName> s: shard2 r:core_node5 x: :
> <solrCollectionName>_shard2_replica_n4] o.a.s.h.RequestHandlerBase
> org.apache.solr.common. SolrException: Error CREATEing SolrCore
> '<solrCollectionName>_shard2_replica_n4': Unable to create core
> [:<solrCollectionName>_shard2_replica_n4] Caused by:
> org.apache.solr.handler.component.QueryDocAuthorizationComponent.....
> *FileNotFoundException and Index Corruption:*
> WARN
> (parallelCoreAdminExecutor-6-thread-7-processing-n:<solrNode>:<solrPort>_solrx:<solrCollectionName>_shard2
> _replica_n1 <numbers> RESTORECORE) [x:<solrCollectionName>_shard2_replica_n1]
> o.a.s.h. RestoreCore Could not switch to restored index. Rolling back to the
> current index => org.apache.lucene.index.CorruptindexException: Unexpected
> file read error while reading index.
> (resource=BufferedChecksumIndexInput(segments_1g9dk))
> Caused by: java.io. FileNotFoundException: File does not exist:
> hdfs://<hdfsPath>/core_node2/data/restore/<fileName>
> It appears that Solr is looking for a file in HDFS that doesn't exist,
> despite no manual deletions being made. I cannot determine why these specific
> shards failed to restore, or why the system is unable to locate the required
> files.
> Expected Behavior:
> The backup and restore process should complete without errors, and all shards
> should be restored successfully to the new cluster.
> Actual Behavior:
> Two shards failed to restore, with errors related to missing files and index
> corruption.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
