Re: How to migrate model/cube metadat across cluster
Hi Lionel, Sorry for my misunderstanding, you are right, I think in your situation, use metastore.sh back is a better way. Best wishes, Xiaoxiang Yu 在 2019/9/19 10:42，“lionel@oocl.com” 写入: Hi Note that the different Kylin environments should share the same Hadoop cluster, including HDFS, HBase and HIVE. The doc says that should share the same Hadoop cluster, including HDFS, HBase and HIVE, right? Thanks & Regards Lionel -Original Message- From: Xiaoxiang Yu Sent: Thursday, September 19, 2019 10:32 AM To: LIONEL TAO (OPS-IRIS-ISD-OOCLL/ZHA) ; dev@kylin.apache.org Subject: Re: How to migrate model/cube metadat across cluster Hi, As the doc says, it could migrate cube between different clusters. If you have a kylin instance in QA cluster (qa_node:7070), and you want to migrate to a PROD cluster (prod_node:7070), you can use qa_node:7070 as srcKylinConfigUri and prod_node:7070 as dstKylinConfigUri, thus to migrate across cluster. Best wishes, Xiaoxiang Yu 在 2019/9/19 10:21，“lionel@oocl.com” 写入: Hi Xiaoxiang, CubeMigrationCLI.java can migrate a cube from a Kylin environment to another, for example, promote a well tested cube from the testing env to production env. Note that the different Kylin environments should share the same Hadoop cluster, including HDFS, HBase and HIVE. Per the document, cannot migrate across cluster? We are finding a solution to do this and QA & PROD are in different cluster. Can you help give more advice? Thanks & Regards Lionel -Original Message- From: Xiaoxiang Yu Sent: Thursday, September 19, 2019 10:07 AM To: dev@kylin.apache.org; LIONEL TAO (OPS-IRIS-ISD-OOCLL/ZHA) Subject: Re: How to migrate model/cube metadat across cluster Hi Lionel, It is a good practice to do a full test at test env and migrate it to another env, I think you can use CubeMigrationCLI to meet your request. Please check https://apc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fkylin.apache.org%2Fdocs%2Fhowto%2Fhowto_use_cli.htmldata=02%7C01%7Clionel.tao%40oocl.com%7C2610d4d2698b44a0bf6908d73ca98c1d%7C7851b4cc2c5c459f96d916731d6b4ca4%7C0%7C0%7C637044571243399491sdata=%2FLMuIwEDnbaulQoZONRm5fNMseHb%2BdH3dkAOfc5N%2Ft8%3Dreserved=0 for more detail. Best wishes, Xiaoxiang Yu 在 2019/9/19 01:12，“lionel@oocl.com” 写入: Hi Kylin dev, We are now trying Kylin for our OLAP system, and now have some questions need clarify with you : 1. See that cannot use CLI to migrate data from QA environment to PROD if they are in different cluster, what's the limitation when migrate across clusters and then how can do the migration? 2. We have tried with metastore.sh backup and then restore in another cluster, Will this be a solution for migration? And any impact if using this solution? Thanks & Regards Lionel Disclaimer : This email and all contents are subject to the following disclaimer: https://apc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Femaildisclaimer.oocl.com%2Fdefault.htmldata=02%7C01%7Clionel.tao%40oocl.com%7C2610d4d2698b44a0bf6908d73ca98c1d%7C7851b4cc2c5c459f96d916731d6b4ca4%7C0%7C0%7C637044571243399491sdata=%2BHDGzQz0VlgJhx3vqt6b5xadXgkwhf4GvepE0djVQC0%3Dreserved=0 Disclaimer : This email and all contents are subject to the following disclaimer: https://apc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Femaildisclaimer.oocl.com%2Fdefault.htmldata=02%7C01%7Clionel.tao%40oocl.com%7C2610d4d2698b44a0bf6908d73ca98c1d%7C7851b4cc2c5c459f96d916731d6b4ca4%7C0%7C0%7C637044571243399491sdata=%2BHDGzQz0VlgJhx3vqt6b5xadXgkwhf4GvepE0djVQC0%3Dreserved=0 Disclaimer : This email and all contents are subject to the following disclaimer: http://emaildisclaimer.oocl.com/default.html
RE: How to migrate model/cube metadat across cluster
Hi Note that the different Kylin environments should share the same Hadoop cluster, including HDFS, HBase and HIVE. The doc says that should share the same Hadoop cluster, including HDFS, HBase and HIVE, right? Thanks & Regards Lionel -Original Message- From: Xiaoxiang Yu Sent: Thursday, September 19, 2019 10:32 AM To: LIONEL TAO (OPS-IRIS-ISD-OOCLL/ZHA) ; dev@kylin.apache.org Subject: Re: How to migrate model/cube metadat across cluster Hi, As the doc says, it could migrate cube between different clusters. If you have a kylin instance in QA cluster (qa_node:7070), and you want to migrate to a PROD cluster (prod_node:7070), you can use qa_node:7070 as srcKylinConfigUri and prod_node:7070 as dstKylinConfigUri, thus to migrate across cluster. Best wishes, Xiaoxiang Yu 在 2019/9/19 10:21，“lionel@oocl.com” 写入: Hi Xiaoxiang, CubeMigrationCLI.java can migrate a cube from a Kylin environment to another, for example, promote a well tested cube from the testing env to production env. Note that the different Kylin environments should share the same Hadoop cluster, including HDFS, HBase and HIVE. Per the document, cannot migrate across cluster? We are finding a solution to do this and QA & PROD are in different cluster. Can you help give more advice? Thanks & Regards Lionel -Original Message- From: Xiaoxiang Yu Sent: Thursday, September 19, 2019 10:07 AM To: dev@kylin.apache.org; LIONEL TAO (OPS-IRIS-ISD-OOCLL/ZHA) Subject: Re: How to migrate model/cube metadat across cluster Hi Lionel, It is a good practice to do a full test at test env and migrate it to another env, I think you can use CubeMigrationCLI to meet your request. Please check https://apc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fkylin.apache.org%2Fdocs%2Fhowto%2Fhowto_use_cli.htmldata=02%7C01%7Clionel.tao%40oocl.com%7C2610d4d2698b44a0bf6908d73ca98c1d%7C7851b4cc2c5c459f96d916731d6b4ca4%7C0%7C0%7C637044571243399491sdata=%2FLMuIwEDnbaulQoZONRm5fNMseHb%2BdH3dkAOfc5N%2Ft8%3Dreserved=0 for more detail. Best wishes, Xiaoxiang Yu 在 2019/9/19 01:12，“lionel@oocl.com” 写入: Hi Kylin dev, We are now trying Kylin for our OLAP system, and now have some questions need clarify with you : 1. See that cannot use CLI to migrate data from QA environment to PROD if they are in different cluster, what's the limitation when migrate across clusters and then how can do the migration? 2. We have tried with metastore.sh backup and then restore in another cluster, Will this be a solution for migration? And any impact if using this solution? Thanks & Regards Lionel Disclaimer : This email and all contents are subject to the following disclaimer: https://apc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Femaildisclaimer.oocl.com%2Fdefault.htmldata=02%7C01%7Clionel.tao%40oocl.com%7C2610d4d2698b44a0bf6908d73ca98c1d%7C7851b4cc2c5c459f96d916731d6b4ca4%7C0%7C0%7C637044571243399491sdata=%2BHDGzQz0VlgJhx3vqt6b5xadXgkwhf4GvepE0djVQC0%3Dreserved=0 Disclaimer : This email and all contents are subject to the following disclaimer: https://apc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Femaildisclaimer.oocl.com%2Fdefault.htmldata=02%7C01%7Clionel.tao%40oocl.com%7C2610d4d2698b44a0bf6908d73ca98c1d%7C7851b4cc2c5c459f96d916731d6b4ca4%7C0%7C0%7C637044571243399491sdata=%2BHDGzQz0VlgJhx3vqt6b5xadXgkwhf4GvepE0djVQC0%3Dreserved=0 Disclaimer : This email and all contents are subject to the following disclaimer: http://emaildisclaimer.oocl.com/default.html
RE: How to migrate model/cube metadat across cluster
Hi Xiaoxiang, CubeMigrationCLI.java can migrate a cube from a Kylin environment to another, for example, promote a well tested cube from the testing env to production env. Note that the different Kylin environments should share the same Hadoop cluster, including HDFS, HBase and HIVE. Per the document, cannot migrate across cluster? We are finding a solution to do this and QA & PROD are in different cluster. Can you help give more advice? Thanks & Regards Lionel -Original Message- From: Xiaoxiang Yu Sent: Thursday, September 19, 2019 10:07 AM To: dev@kylin.apache.org; LIONEL TAO (OPS-IRIS-ISD-OOCLL/ZHA) Subject: Re: How to migrate model/cube metadat across cluster Hi Lionel, It is a good practice to do a full test at test env and migrate it to another env, I think you can use CubeMigrationCLI to meet your request. Please check https://apc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fkylin.apache.org%2Fdocs%2Fhowto%2Fhowto_use_cli.htmldata=02%7C01%7Clionel.tao%40oocl.com%7C872328d1b6be478d200208d73ca61c5f%7C7851b4cc2c5c459f96d916731d6b4ca4%7C0%7C0%7C637044556465806576sdata=4qGoP1v25b3QRDH1fJVjOuvr%2BnCPHmRsNgXad2Qwp%2FI%3Dreserved=0 for more detail. Best wishes, Xiaoxiang Yu 在 2019/9/19 01:12，“lionel@oocl.com” 写入: Hi Kylin dev, We are now trying Kylin for our OLAP system, and now have some questions need clarify with you : 1. See that cannot use CLI to migrate data from QA environment to PROD if they are in different cluster, what's the limitation when migrate across clusters and then how can do the migration? 2. We have tried with metastore.sh backup and then restore in another cluster, Will this be a solution for migration? And any impact if using this solution? Thanks & Regards Lionel Disclaimer : This email and all contents are subject to the following disclaimer: https://apc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Femaildisclaimer.oocl.com%2Fdefault.htmldata=02%7C01%7Clionel.tao%40oocl.com%7C872328d1b6be478d200208d73ca61c5f%7C7851b4cc2c5c459f96d916731d6b4ca4%7C0%7C0%7C637044556465806576sdata=Af82tzdedXb%2BjZ%2BkbEggDnHCmLvyTKjM%2FnAQCKMXiY8%3Dreserved=0 Disclaimer : This email and all contents are subject to the following disclaimer: http://emaildisclaimer.oocl.com/default.html
Re: How to migrate model/cube metadat across cluster
Hi Lionel, You might want to look into CubeMigrationCLI.java for migrating from lower to higher environments. http://kylin.apache.org/docs/howto/howto_use_cli.html I hope this helps! Best Regards, Nikhil Jain On 2019/9/19, 1:12 AM, "lionel@oocl.com" wrote: Hi Kylin dev, We are now trying Kylin for our OLAP system, and now have some questions need clarify with you : 1. See that cannot use CLI to migrate data from QA environment to PROD if they are in different cluster, what's the limitation when migrate across clusters and then how can do the migration? 2. We have tried with metastore.sh backup and then restore in another cluster, Will this be a solution for migration? And any impact if using this solution? Thanks & Regards Lionel Disclaimer : This email and all contents are subject to the following disclaimer: http://emaildisclaimer.oocl.com/default.html
Re: How to migrate model/cube metadat across cluster
Hi Lionel, It is a good practice to do a full test at test env and migrate it to another env, I think you can use CubeMigrationCLI to meet your request. Please check http://kylin.apache.org/docs/howto/howto_use_cli.html for more detail. Best wishes, Xiaoxiang Yu 在 2019/9/19 01:12，“lionel@oocl.com” 写入: Hi Kylin dev, We are now trying Kylin for our OLAP system, and now have some questions need clarify with you : 1. See that cannot use CLI to migrate data from QA environment to PROD if they are in different cluster, what's the limitation when migrate across clusters and then how can do the migration? 2. We have tried with metastore.sh backup and then restore in another cluster, Will this be a solution for migration? And any impact if using this solution? Thanks & Regards Lionel Disclaimer : This email and all contents are subject to the following disclaimer: http://emaildisclaimer.oocl.com/default.html
How to migrate model/cube metadat across cluster
Hi Kylin dev, We are now trying Kylin for our OLAP system, and now have some questions need clarify with you : 1. See that cannot use CLI to migrate data from QA environment to PROD if they are in different cluster, what's the limitation when migrate across clusters and then how can do the migration? 2. We have tried with metastore.sh backup and then restore in another cluster, Will this be a solution for migration? And any impact if using this solution? Thanks & Regards Lionel Disclaimer : This email and all contents are subject to the following disclaimer: http://emaildisclaimer.oocl.com/default.html
