[GitHub] [carbondata] shenjiayu17 commented on pull request #4293: Bump jackson-databind from 2.10.0 to 2.12.7.1 in /integration/presto
shenjiayu17 commented on PR #4293: URL: https://github.com/apache/carbondata/pull/4293#issuecomment-1316314203 Build Failed with Spark 2.4.5, Please check CI http://159.138.8.58:12602/job/ApacheCarbon_PR_Builder_2.4.5/4665/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4293: Bump jackson-databind from 2.10.0 to 2.12.7.1 in /integration/presto
shenjiayu17 commented on PR #4293: URL: https://github.com/apache/carbondata/pull/4293#issuecomment-1316224942 Build Failed with Spark 2.3.4, Please check CI http://159.138.8.58:12602/job/ApacheCarbonPRBuilder2.3/6374/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4293: Bump jackson-databind from 2.10.0 to 2.12.7.1 in /integration/presto
shenjiayu17 commented on PR #4293: URL: https://github.com/apache/carbondata/pull/4293#issuecomment-1316135307 Build Failed with Spark 3.1, Please check CI http://159.138.8.58:12602/job/ApacheCarbon_PR_Builder_3.1/757/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] dependabot[bot] opened a new pull request, #4293: Bump jackson-databind from 2.10.0 to 2.12.7.1 in /integration/presto
dependabot[bot] opened a new pull request, #4293: URL: https://github.com/apache/carbondata/pull/4293 Bumps [jackson-databind](https://github.com/FasterXML/jackson) from 2.10.0 to 2.12.7.1. Commits See full diff in https://github.com/FasterXML/jackson/commits";>compare view [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/carbondata/network/alerts). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] dependabot[bot] commented on pull request #4291: Bump jackson-databind from 2.10.0 to 2.13.4.1 in /integration/presto
dependabot[bot] commented on PR #4291: URL: https://github.com/apache/carbondata/pull/4291#issuecomment-1316017304 Superseded by #4293. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] dependabot[bot] closed pull request #4291: Bump jackson-databind from 2.10.0 to 2.13.4.1 in /integration/presto
dependabot[bot] closed pull request #4291: Bump jackson-databind from 2.10.0 to 2.13.4.1 in /integration/presto URL: https://github.com/apache/carbondata/pull/4291 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] jbonofre commented on pull request #4290: [CARBONDATA-TEST] jenkins test
jbonofre commented on PR #4290: URL: https://github.com/apache/carbondata/pull/4290#issuecomment-1313965613 I'm working on fixing Jenkins. For now, I have an issue to build locally on my machine. I will create few PRs about that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4292: Bump solr-core from 6.3.0 to 8.8.2 in /index/lucene
shenjiayu17 commented on PR #4292: URL: https://github.com/apache/carbondata/pull/4292#issuecomment-1307758128 Build Failed with Spark 2.3.4, Please check CI http://159.138.8.58:12602/job/ApacheCarbonPRBuilder2.3/6373/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4292: Bump solr-core from 6.3.0 to 8.8.2 in /index/lucene
shenjiayu17 commented on PR #4292: URL: https://github.com/apache/carbondata/pull/4292#issuecomment-1307652430 Build Failed with Spark 3.1, Please check CI http://159.138.8.58:12602/job/ApacheCarbon_PR_Builder_3.1/756/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4292: Bump solr-core from 6.3.0 to 8.8.2 in /index/lucene
shenjiayu17 commented on PR #4292: URL: https://github.com/apache/carbondata/pull/4292#issuecomment-1307482632 Build Failed with Spark 2.4.5, Please check CI http://159.138.8.58:12602/job/ApacheCarbon_PR_Builder_2.4.5/4664/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] dependabot[bot] opened a new pull request, #4292: Bump solr-core from 6.3.0 to 8.8.2 in /index/lucene
dependabot[bot] opened a new pull request, #4292: URL: https://github.com/apache/carbondata/pull/4292 Bumps solr-core from 6.3.0 to 8.8.2. [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/carbondata/network/alerts). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4193: [WIP] Fix Show Schema Issue
shenjiayu17 commented on PR #4193: URL: https://github.com/apache/carbondata/pull/4193#issuecomment-1283410162 Build Failed with Spark 2.3.4, Please check CI http://159.138.8.58:12602/job/ApacheCarbonPRBuilder2.3/6372/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4291: Bump jackson-databind from 2.10.0 to 2.13.4.1 in /integration/presto
shenjiayu17 commented on PR #4291: URL: https://github.com/apache/carbondata/pull/4291#issuecomment-1283368136 Build Failed with Spark 3.1, Please check CI http://159.138.8.58:12602/job/ApacheCarbon_PR_Builder_3.1/755/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4291: Bump jackson-databind from 2.10.0 to 2.13.4.1 in /integration/presto
shenjiayu17 commented on PR #4291: URL: https://github.com/apache/carbondata/pull/4291#issuecomment-1283251703 Build Failed with Spark 2.4.5, Please check CI http://159.138.8.58:12602/job/ApacheCarbon_PR_Builder_2.4.5/4663/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4291: Bump jackson-databind from 2.10.0 to 2.13.4.1 in /integration/presto
shenjiayu17 commented on PR #4291: URL: https://github.com/apache/carbondata/pull/4291#issuecomment-1283151994 Build Failed with Spark 2.3.4, Please check CI http://159.138.8.58:12602/job/ApacheCarbonPRBuilder2.3/6371/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] dependabot[bot] commented on pull request #4270: Bump jackson-databind from 2.10.0 to 2.12.6.1 in /integration/presto
dependabot[bot] commented on PR #4270: URL: https://github.com/apache/carbondata/pull/4270#issuecomment-1283089668 Superseded by #4291. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] dependabot[bot] closed pull request #4270: Bump jackson-databind from 2.10.0 to 2.12.6.1 in /integration/presto
dependabot[bot] closed pull request #4270: Bump jackson-databind from 2.10.0 to 2.12.6.1 in /integration/presto URL: https://github.com/apache/carbondata/pull/4270 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] dependabot[bot] opened a new pull request, #4291: Bump jackson-databind from 2.10.0 to 2.13.4.1 in /integration/presto
dependabot[bot] opened a new pull request, #4291: URL: https://github.com/apache/carbondata/pull/4291 Bumps [jackson-databind](https://github.com/FasterXML/jackson) from 2.10.0 to 2.13.4.1. Commits See full diff in https://github.com/FasterXML/jackson/commits";>compare view [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/carbondata/network/alerts). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4283: [WIP] Upgrade to Spark 3.3.0
shenjiayu17 commented on PR #4283: URL: https://github.com/apache/carbondata/pull/4283#issuecomment-1272585346 Build Failed with Spark 2.4.5, Please check CI http://159.138.8.58:12602/job/ApacheCarbon_PR_Builder_2.4.5/4662/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4284: Bump libthrift from 0.9.3 to 0.14.0 in /format
shenjiayu17 commented on PR #4284: URL: https://github.com/apache/carbondata/pull/4284#issuecomment-1272583565 Build Failed with Spark 2.4.5, Please check CI http://159.138.8.58:12602/job/ApacheCarbon_PR_Builder_2.4.5/4661/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4286: Bump zookeeper from 3.4.7 to 3.4.14 in /core
shenjiayu17 commented on PR #4286: URL: https://github.com/apache/carbondata/pull/4286#issuecomment-1272582775 Build Failed with Spark 2.4.5, Please check CI http://159.138.8.58:12602/job/ApacheCarbon_PR_Builder_2.4.5/4660/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
shenjiayu17 commented on PR #4287: URL: https://github.com/apache/carbondata/pull/4287#issuecomment-1272564013 Build Failed with Spark 2.4.5, Please check CI http://159.138.8.58:12602/job/ApacheCarbon_PR_Builder_2.4.5/4659/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4290: [CARBONDATA-TEST] jenkins test
shenjiayu17 commented on PR #4290: URL: https://github.com/apache/carbondata/pull/4290#issuecomment-1272545391 Build Failed with Spark 2.4.5, Please check CI http://159.138.8.58:12602/job/ApacheCarbon_PR_Builder_2.4.5/4658/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4290: [CARBONDATA-TEST] jenkins test
shenjiayu17 commented on PR #4290: URL: https://github.com/apache/carbondata/pull/4290#issuecomment-1272527964 Build Failed with Spark 2.3.4, Please check CI http://159.138.8.58:12602/job/ApacheCarbonPRBuilder2.3/6370/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4290: [CARBONDATA-TEST] jenkins test
shenjiayu17 commented on PR #4290: URL: https://github.com/apache/carbondata/pull/4290#issuecomment-1272511318 Build Failed with Spark 3.1, Please check CI http://159.138.8.58:12602/job/ApacheCarbon_PR_Builder_3.1/754/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4290: [CARBONDATA-TEST] jenkins test
shenjiayu17 commented on PR #4290: URL: https://github.com/apache/carbondata/pull/4290#issuecomment-1272474933 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4290: [CARBONDATA-TEST] jenkins test
shenjiayu17 commented on PR #4290: URL: https://github.com/apache/carbondata/pull/4290#issuecomment-1272453534 Build Failed with Spark 2.3.4, Please check CI http://159.138.8.58:12602/job/ApacheCarbonPRBuilder2.3/6369/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 opened a new pull request, #4290: [CARBONDATA-TEST] jenkins test
shenjiayu17 opened a new pull request, #4290: URL: https://github.com/apache/carbondata/pull/4290 ### Why is this PR needed? Todo ### What changes were proposed in this PR? Todo ### Does this PR introduce any user interface change? - No ### Is any new testcase added? - No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4284: Bump libthrift from 0.9.3 to 0.14.0 in /format
shenjiayu17 commented on PR #4284: URL: https://github.com/apache/carbondata/pull/4284#issuecomment-1272443022 Build Failed with Spark 2.3.4, Please check CI http://159.138.8.58:12602/job/ApacheCarbonPRBuilder2.3/6368/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4289: [CARBONDATA-CI-TEST] test new jenkins
shenjiayu17 commented on PR #4289: URL: https://github.com/apache/carbondata/pull/4289#issuecomment-1272442947 Build Failed with Spark 2.3.4, Please check CI http://159.138.8.58:12602/job/ApacheCarbonPRBuilder2.3/6367/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4289: [CARBONDATA-CI-TEST] test new jenkins
shenjiayu17 commented on PR #4289: URL: https://github.com/apache/carbondata/pull/4289#issuecomment-1210213607 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4289: [CARBONDATA-CI-TEST] test new jenkins
CarbonDataQA2 commented on PR #4289: URL: https://github.com/apache/carbondata/pull/4289#issuecomment-1200088644 Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/793/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
CarbonDataQA2 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1200088642 Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/794/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4289: [CARBONDATA-CI-TEST] test new jenkins
CarbonDataQA2 commented on PR #4289: URL: https://github.com/apache/carbondata/pull/4289#issuecomment-1200082616 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4664/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4289: [CARBONDATA-CI-TEST] test new jenkins
CarbonDataQA2 commented on PR #4289: URL: https://github.com/apache/carbondata/pull/4289#issuecomment-1200082430 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6407/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
CarbonDataQA2 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1200080698 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6408/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
CarbonDataQA2 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1200080551 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4665/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
shenjiayu17 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1200060281 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4289: [CARBONDATA-CI-TEST] test new jenkins
shenjiayu17 commented on PR #4289: URL: https://github.com/apache/carbondata/pull/4289#issuecomment-1200059300 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4289: [CARBONDATA-CI-TEST] test new jenkins
CarbonDataQA2 commented on PR #4289: URL: https://github.com/apache/carbondata/pull/4289#issuecomment-1191266444 Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/792/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4289: [CARBONDATA-CI-TEST] test new jenkins
CarbonDataQA2 commented on PR #4289: URL: https://github.com/apache/carbondata/pull/4289#issuecomment-1191237956 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6406/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4289: [CARBONDATA-CI-TEST] test new jenkins
CarbonDataQA2 commented on PR #4289: URL: https://github.com/apache/carbondata/pull/4289#issuecomment-1191140542 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4663/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 opened a new pull request, #4289: [CARBONDATA-CI-TEST] test new jenkins
shenjiayu17 opened a new pull request, #4289: URL: https://github.com/apache/carbondata/pull/4289 ### Why is this PR needed? Test ### What changes were proposed in this PR? Test ### Does this PR introduce any user interface change? - No ### Is any new testcase added? - No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
CarbonDataQA2 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1188925677 Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/791/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
CarbonDataQA2 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1188912379 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6405/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
CarbonDataQA2 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1188912378 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4662/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
shenjiayu17 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1188734870 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
asfgit closed pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash URL: https://github.com/apache/carbondata/pull/4276 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
Indhumathi27 commented on PR #4276: URL: https://github.com/apache/carbondata/pull/4276#issuecomment-1188593741 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
CarbonDataQA2 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1188575323 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4660/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
CarbonDataQA2 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1188575301 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6403/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
CarbonDataQA2 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1188574627 Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/789/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
shenjiayu17 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1188566729 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
CarbonDataQA2 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1188552188 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4659/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
CarbonDataQA2 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1188552101 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6402/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
CarbonDataQA2 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1188551437 Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/788/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
shenjiayu17 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1188545214 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
CarbonDataQA2 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1186778649 Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/787/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
CarbonDataQA2 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1186778203 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6401/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
CarbonDataQA2 commented on PR #4288: URL: https://github.com/apache/carbondata/pull/4288#issuecomment-1186778097 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4658/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] shenjiayu17 opened a new pull request, #4288: [CARBONDATA-4347] Improve performance when delete empty partition directory
shenjiayu17 opened a new pull request, #4288: URL: https://github.com/apache/carbondata/pull/4288 Why is this PR needed? When delete multiple segments, carbondata tries to judge if partition directory is empty and delete it on each segments, but partiton path may be the same between two segments. So there is repetitive action. What changes were proposed in this PR? Collect a set of partition directories that are from segments to be deleted, judge empty and delete after all segments deleted. Does this PR introduce any user interface change? No Is any new testcase added? No ### Why is this PR needed? ### What changes were proposed in this PR? ### Does this PR introduce any user interface change? - No - Yes. (please explain the change and update document) ### Is any new testcase added? - No - Yes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
CarbonDataQA2 commented on PR #4276: URL: https://github.com/apache/carbondata/pull/4276#issuecomment-1184540573 Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/786/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
CarbonDataQA2 commented on PR #4276: URL: https://github.com/apache/carbondata/pull/4276#issuecomment-1184535449 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4657/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
CarbonDataQA2 commented on PR #4276: URL: https://github.com/apache/carbondata/pull/4276#issuecomment-1184499722 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6400/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on a diff in pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
Indhumathi27 commented on code in PR #4276: URL: https://github.com/apache/carbondata/pull/4276#discussion_r921037157 ## docs/faq.md: ## @@ -481,4 +482,8 @@ TableStatusRecovery.main(args) --> args is of length two: 1. Database Name 2. Ta TableStatus Recovery tool cannot recover table status version files for the below two scenarios 1. After compaction, if table status file is lost, cannot recover compacted commit transaction, as the lost version file only has merged load details. 2. After Delete segment by Id/Date, if table status file is lost, cannot recover deleted segment commit transaction, as the lost version file only has the segment status as deleted. -3. Table status recovery on materialized view table is not supported. \ No newline at end of file +3. Table status recovery on materialized view table is not supported. + +## Why deleted partition data still showing in file system +Please enable the feature **moving deleted partition data to trash** by making the `carbon.enable.partitiondata.trash` property true. Review Comment: Can mention like below. By default, the dropped partition data will not be physically removed from the table store until the table is dropped. Enable [carbon.enable.partitiondata.trash](./configuration-parameters.md#system-configuration) property to move all the dropped partitions data to trash during alter table DROP PARTITION operation itself. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on a diff in pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
Indhumathi27 commented on code in PR #4276: URL: https://github.com/apache/carbondata/pull/4276#discussion_r921033456 ## docs/ddl-of-carbondata.md: ## @@ -1107,6 +1107,12 @@ Users can specify which columns to include and exclude for local dictionary gene ALTER TABLE locationTable DROP PARTITION (country = 'US'); ``` + The below flag is set to enable the feature of moving the deleted partition data to trash when dropped partition command is given. Review Comment: better to add it as a note ```suggestion **NOTE:** Enable [carbon.enable.partitiondata.trash](./configuration-parameters.md#system-configuration) to move dropped partition data to trash during alter table DROP PARTITION. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on a diff in pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
Indhumathi27 commented on code in PR #4276:
URL: https://github.com/apache/carbondata/pull/4276#discussion_r921026787
##
core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java:
##
@@ -2886,5 +2886,15 @@ private CarbonCommonConstants() {
public static final String CARBON_ENABLE_MULTI_VERSION_TABLE_STATUS_DEFAULT
= "false";
+ /**
+ * This flag is set to enable the feature of moving the partition data to
trash
Review Comment:
```suggestion
* Enable this property to move the dropped partition data to trash on
ALTER DROP PARTITION operation
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on a diff in pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
Indhumathi27 commented on code in PR #4276: URL: https://github.com/apache/carbondata/pull/4276#discussion_r921025292 ## docs/configuration-parameters.md: ## @@ -58,6 +58,7 @@ This section provides the details of all the configurations required for the Car | carbon.lock.class | (none) | This specifies the implementation of ICarbonLock interface to be used for acquiring the locks in case of concurrent operations | | carbon.data.file.version | V3 | This specifies carbondata file format version. Carbondata file format has evolved with time from V1 to V3 in terms of metadata storage and IO level pruning capabilities. You can find more details [here](https://carbondata.apache.org/file-structure-of-carbondata.html#carbondata-file-format). | | spark.carbon.hive.schema.store | false | Carbondata currently supports 2 different types of metastores for storing schemas. This property specifies if Hive metastore is to be used for storing and retrieving table schemas | +| carbon.enable.partitiondata.trash | false | This property is set to enable the feature of moving the deleted partition data to trash when dropped partition command is given Review Comment: ```suggestion | carbon.enable.partitiondata.trash | false | This property, when enabled, will move the dropped partition data to trash on ALTER DROP PARTITION operation. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
CarbonDataQA2 commented on PR #4287: URL: https://github.com/apache/carbondata/pull/4287#issuecomment-1184133097 Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/783/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
CarbonDataQA2 commented on PR #4276: URL: https://github.com/apache/carbondata/pull/4276#issuecomment-1184122482 Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/785/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
CarbonDataQA2 commented on PR #4276: URL: https://github.com/apache/carbondata/pull/4276#issuecomment-1184120916 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4656/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
CarbonDataQA2 commented on PR #4287: URL: https://github.com/apache/carbondata/pull/4287#issuecomment-1184113053 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4654/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
CarbonDataQA2 commented on PR #4276: URL: https://github.com/apache/carbondata/pull/4276#issuecomment-118410 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6399/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
CarbonDataQA2 commented on PR #4287: URL: https://github.com/apache/carbondata/pull/4287#issuecomment-1184107772 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6397/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
akashrn5 commented on PR #4276: URL: https://github.com/apache/carbondata/pull/4276#issuecomment-1184067700 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] maheshrajus commented on a diff in pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
maheshrajus commented on code in PR #4276: URL: https://github.com/apache/carbondata/pull/4276#discussion_r920769079 ## docs/ddl-of-carbondata.md: ## @@ -1107,6 +1107,12 @@ Users can specify which columns to include and exclude for local dictionary gene ALTER TABLE locationTable DROP PARTITION (country = 'US'); ``` + The below flag is set to enable the feature of moving the deleted partition data to trash when dropped partition command is given. + ``` + carbon.enable.partitiondata.trash + ``` Review Comment: done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on a diff in pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
akashrn5 commented on code in PR #4276: URL: https://github.com/apache/carbondata/pull/4276#discussion_r920757032 ## docs/ddl-of-carbondata.md: ## @@ -1107,6 +1107,12 @@ Users can specify which columns to include and exclude for local dictionary gene ALTER TABLE locationTable DROP PARTITION (country = 'US'); ``` + The below flag is set to enable the feature of moving the deleted partition data to trash when dropped partition command is given. + ``` + carbon.enable.partitiondata.trash + ``` Review Comment: please add in configuration md file also -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ShreelekhyaG commented on pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
ShreelekhyaG commented on PR #4287: URL: https://github.com/apache/carbondata/pull/4287#issuecomment-1183991323 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] maheshrajus commented on a diff in pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
maheshrajus commented on code in PR #4276:
URL: https://github.com/apache/carbondata/pull/4276#discussion_r920739192
##
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/partition/CarbonAlterTableDropHivePartitionCommand.scala:
##
@@ -106,6 +108,32 @@ case class CarbonAlterTableDropHivePartitionCommand(
ifExists,
purge,
retainData).run(sparkSession)
+ val isPartitionDataTrashEnabled = CarbonProperties.getInstance()
+
.getProperty(CarbonCommonConstants.CARBON_ENABLE_PARTITION_DATA_TRASH,
+
CarbonCommonConstants.CARBON_ENABLE_PARTITION_DATA_TRASH_DEFAULT).toBoolean
+ if (isPartitionDataTrashEnabled) {
+// move the partition files to trash folder which are dropped
+val droppedPartitionNames = partitions.map { partition =>
+ partition.spec.map { specs => specs._1 + "=" + specs._2 }
+}
+val timeStamp = CarbonUpdateUtil.readCurrentTime();
Review Comment:
ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on a diff in pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
akashrn5 commented on code in PR #4276:
URL: https://github.com/apache/carbondata/pull/4276#discussion_r920728055
##
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/partition/CarbonAlterTableDropHivePartitionCommand.scala:
##
@@ -106,6 +108,32 @@ case class CarbonAlterTableDropHivePartitionCommand(
ifExists,
purge,
retainData).run(sparkSession)
+ val isPartitionDataTrashEnabled = CarbonProperties.getInstance()
+
.getProperty(CarbonCommonConstants.CARBON_ENABLE_PARTITION_DATA_TRASH,
+
CarbonCommonConstants.CARBON_ENABLE_PARTITION_DATA_TRASH_DEFAULT).toBoolean
+ if (isPartitionDataTrashEnabled) {
+// move the partition files to trash folder which are dropped
+val droppedPartitionNames = partitions.map { partition =>
+ partition.spec.map { specs => specs._1 + "=" + specs._2 }
+}
+val timeStamp = CarbonUpdateUtil.readCurrentTime();
Review Comment:
for time and in this case it will be milliSec only and many places used
directly, so use directly only
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
akashrn5 commented on PR #4287: URL: https://github.com/apache/carbondata/pull/4287#issuecomment-1183956145 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
CarbonDataQA2 commented on PR #4287: URL: https://github.com/apache/carbondata/pull/4287#issuecomment-1183172937 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6396/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
CarbonDataQA2 commented on PR #4287: URL: https://github.com/apache/carbondata/pull/4287#issuecomment-1183012054 Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/782/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
CarbonDataQA2 commented on PR #4287: URL: https://github.com/apache/carbondata/pull/4287#issuecomment-1183011121 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4653/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ShreelekhyaG commented on pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
ShreelekhyaG commented on PR #4287: URL: https://github.com/apache/carbondata/pull/4287#issuecomment-1182968335 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
CarbonDataQA2 commented on PR #4287: URL: https://github.com/apache/carbondata/pull/4287#issuecomment-1182936703 Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/781/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
CarbonDataQA2 commented on PR #4287: URL: https://github.com/apache/carbondata/pull/4287#issuecomment-1182888005 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4652/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
CarbonDataQA2 commented on PR #4287: URL: https://github.com/apache/carbondata/pull/4287#issuecomment-1182881418 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6395/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ShreelekhyaG commented on a diff in pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
ShreelekhyaG commented on code in PR #4287:
URL: https://github.com/apache/carbondata/pull/4287#discussion_r918947827
##
core/src/main/java/org/apache/carbondata/core/util/BlockletIndexUtil.java:
##
@@ -193,7 +154,11 @@ private static BlockMetaInfo createBlockMetaInfo(
CarbonFile carbonFile = FileFactory.getCarbonFile(carbonDataFile);
return new BlockMetaInfo(new String[] { "localhost" },
carbonFile.getSize());
default:
-return
fileNameToMetaInfoMapping.get(FileFactory.getFormattedPath(carbonDataFile));
+if (!FileFactory.isFileExist(carbonDataFile)) {
Review Comment:
I have added a generic comment in line:111. As the list files code is
removed, I feel adding an example would create confusion now. I have added an
example and percentage improvement in the pr description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] ShreelekhyaG commented on pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
ShreelekhyaG commented on PR #4287: URL: https://github.com/apache/carbondata/pull/4287#issuecomment-1182767186 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
CarbonDataQA2 commented on PR #4287: URL: https://github.com/apache/carbondata/pull/4287#issuecomment-1181993356 Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/779/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
CarbonDataQA2 commented on PR #4287: URL: https://github.com/apache/carbondata/pull/4287#issuecomment-1181947855 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4650/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
CarbonDataQA2 commented on PR #4287: URL: https://github.com/apache/carbondata/pull/4287#issuecomment-1181946225 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6393/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ShreelekhyaG commented on a diff in pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
ShreelekhyaG commented on code in PR #4287:
URL: https://github.com/apache/carbondata/pull/4287#discussion_r918947827
##
core/src/main/java/org/apache/carbondata/core/util/BlockletIndexUtil.java:
##
@@ -193,7 +154,11 @@ private static BlockMetaInfo createBlockMetaInfo(
CarbonFile carbonFile = FileFactory.getCarbonFile(carbonDataFile);
return new BlockMetaInfo(new String[] { "localhost" },
carbonFile.getSize());
default:
-return
fileNameToMetaInfoMapping.get(FileFactory.getFormattedPath(carbonDataFile));
+if (!FileFactory.isFileExist(carbonDataFile)) {
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
CarbonDataQA2 commented on PR #4276: URL: https://github.com/apache/carbondata/pull/4276#issuecomment-1181498684 Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/778/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
CarbonDataQA2 commented on PR #4276: URL: https://github.com/apache/carbondata/pull/4276#issuecomment-1181479275 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6392/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
CarbonDataQA2 commented on PR #4276: URL: https://github.com/apache/carbondata/pull/4276#issuecomment-1181479270 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4647/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on a diff in pull request #4287: [CARBONDATA-4346] Remove list files while query and invalid cache
akashrn5 commented on code in PR #4287:
URL: https://github.com/apache/carbondata/pull/4287#discussion_r918566559
##
core/src/main/java/org/apache/carbondata/core/util/BlockletIndexUtil.java:
##
@@ -193,7 +154,11 @@ private static BlockMetaInfo createBlockMetaInfo(
CarbonFile carbonFile = FileFactory.getCarbonFile(carbonDataFile);
return new BlockMetaInfo(new String[] { "localhost" },
carbonFile.getSize());
default:
-return
fileNameToMetaInfoMapping.get(FileFactory.getFormattedPath(carbonDataFile));
+if (!FileFactory.isFileExist(carbonDataFile)) {
Review Comment:
please add comments with example how this will help, also add the
percentage improvement of count(*) for partition and non partition table in the
description
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] maheshrajus commented on a diff in pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
maheshrajus commented on code in PR #4276:
URL: https://github.com/apache/carbondata/pull/4276#discussion_r918548519
##
core/src/main/java/org/apache/carbondata/core/util/CleanFilesUtil.java:
##
@@ -206,10 +206,12 @@ private static void getStaleSegmentFiles(CarbonTable
carbonTable, List s
/**
* This method will delete all the empty partition folders starting from the
table path
*/
- private static void deleteEmptyPartitionFoldersRecursively(CarbonFile
tablePath) {
+ public static void deleteEmptyPartitionFoldersRecursively(CarbonFile
tablePath) {
CarbonFile[] listOfFiles = tablePath.listFiles();
if (listOfFiles.length == 0) {
tablePath.delete();
+ // if parent file folder also empty then delete that too.
+ deleteEmptyPartitionFoldersRecursively(tablePath.getParentFile());
Review Comment:
this i added as part of bug fix. If partition folder is empty after moving
to trash, it will delete empty folder. After moving if database folder also
empty then it will delete that too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [carbondata] maheshrajus commented on a diff in pull request #4276: [CARBONDATA-4338] moving dropped partition data to trash
maheshrajus commented on code in PR #4276:
URL: https://github.com/apache/carbondata/pull/4276#discussion_r918539981
##
core/src/main/java/org/apache/carbondata/core/util/TrashUtil.java:
##
@@ -268,4 +268,45 @@ public static String getCompleteTrashFolderPath(String
tablePath, long timeStamp
timeStampSubFolder + CarbonCommonConstants.FILE_SEPARATOR +
CarbonTablePath
.SEGMENT_PREFIX + segmentNumber;
}
+
+ /**
+ * This will give the complete path of the trash folder with the timestamp
and the partition name
+ *
+ * @param tablePath absolute table path
+ * @param timeStampSubFolder the timestamp for the clean files operation
+ * @param partitionName partition name for which files are moved to the
trash folder
+ */
+ public static String getCompleteTrashFolderPathForPartition(String tablePath,
+ long timeStampSubFolder, String partitionName) {
+return CarbonTablePath.getTrashFolderPath(tablePath) +
CarbonCommonConstants.FILE_SEPARATOR
++ timeStampSubFolder + CarbonCommonConstants.FILE_SEPARATOR
++ partitionName;
+ }
+
+ /**
+ * The below method copies dropped partition files to the trash folder.
+ *
+ * @param filesToCopy absolute path of the files to copy to the
trash folder
+ * @param trashFolderWithTimestamp trashfolderpath with complete timestamp
and segment number
+ */
+ public static void copyFilesToTrash(String filesToCopy, String
trashFolderWithTimestamp) {
+try {
+ if (!FileFactory.isFileExist(trashFolderWithTimestamp)) {
+FileFactory.mkdirs(trashFolderWithTimestamp);
+ }
+ // check if file exists before copying
+ if (FileFactory.isFileExist(filesToCopy)) {
+CarbonFile folder = FileFactory.getCarbonFile(filesToCopy);
+CarbonFile[] dataFiles = folder.listFiles();
+for (CarbonFile carbonFile : dataFiles) {
+ copyFileToTrashFolder(carbonFile.getAbsolutePath(),
trashFolderWithTimestamp);
+}
+ } else {
+LOGGER.warn("Folder not copied to trash as partition folder does not
exist");
+ }
+} catch (IOException e) {
+ // If file is already moved or not found then continue with other files
+ LOGGER.warn("Unable to copy file to trash folder as file already moved
or not found.", e);
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
