[GitHub] [kylin] asfgit commented on issue #709: KYLIN-3982 Add measures without purging segments
asfgit commented on issue #709: KYLIN-3982 Add measures without purging segments URL: https://github.com/apache/kylin/pull/709#issuecomment-504725709 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] Qsimple opened a new pull request #709: KYLIN-3982 Add measures without purging segments
Qsimple opened a new pull request #709: KYLIN-3982 Add measures without purging segments URL: https://github.com/apache/kylin/pull/709 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] codecov-io commented on issue #707: KYLIN-4013 Only show the cubes under one model
codecov-io commented on issue #707: KYLIN-4013 Only show the cubes under one model URL: https://github.com/apache/kylin/pull/707#issuecomment-504718946 # [Codecov](https://codecov.io/gh/apache/kylin/pull/707?src=pr=h1) Report > Merging [#707](https://codecov.io/gh/apache/kylin/pull/707?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/970899011211b285b78cfc3a63ac5025436ddb59?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/kylin/pull/707?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #707 +/- ## - Coverage 25.74% 25.73% -0.01% + Complexity 6014 6012 -2 Files 1385 1385 Lines 8242482424 Branches 1155711557 - Hits 2121621210 -6 - Misses5916759170 +3 - Partials 2041 2044 +3 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/707?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/707/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `72.97% <0%> (-5.41%)` | `6% <0%> (-1%)` | | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/707/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `69.3% <0%> (-0.92%)` | `7% <0%> (ø)` | | | [...a/org/apache/kylin/dict/Number2BytesConverter.java](https://codecov.io/gh/apache/kylin/pull/707/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L051bWJlcjJCeXRlc0NvbnZlcnRlci5qYXZh) | `81.74% <0%> (-0.8%)` | `17% <0%> (-1%)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/707?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/707?src=pr=footer). Last update [9708990...bb228a7](https://codecov.io/gh/apache/kylin/pull/707?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] Qsimple opened a new pull request #707: KYLIN-4013 Only show the cubes under one model
Qsimple opened a new pull request #707: KYLIN-4013 Only show the cubes under one model URL: https://github.com/apache/kylin/pull/707 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #707: KYLIN-4013 Only show the cubes under one model
asfgit commented on issue #707: KYLIN-4013 Only show the cubes under one model URL: https://github.com/apache/kylin/pull/707#issuecomment-504717706 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls commented on issue #707: KYLIN-4013 Only show the cubes under one model
coveralls commented on issue #707: KYLIN-4013 Only show the cubes under one model URL: https://github.com/apache/kylin/pull/707#issuecomment-504718829 ## Pull Request Test Coverage Report for [Build 4647](https://coveralls.io/builds/24152042) * **0** of **0** changed or added relevant lines in **0** files are covered. * **4** unchanged lines in **1** file lost coverage. * Overall coverage increased (+**0.01%**) to **28.216%** --- | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/24152042/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L555) | 4 | 77.81% | | Totals | [](https://coveralls.io/builds/24152042) | | :-- | --: | | Change from base [Build 4633](https://coveralls.io/builds/24099387): | 0.01% | | Covered Lines: | 23257 | | Relevant Lines: | 82424 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #711: KYLIN-4054 Logger of HCreateTableJob record error message
asfgit commented on issue #711: KYLIN-4054 Logger of HCreateTableJob record error message URL: https://github.com/apache/kylin/pull/711#issuecomment-504844932 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] hit-lacus opened a new pull request #711: KYLIN-4054 Logger of HCreateTableJob record error message
hit-lacus opened a new pull request #711: KYLIN-4054 Logger of HCreateTableJob record error message URL: https://github.com/apache/kylin/pull/711 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] codecov-io commented on issue #710: KYLIN-4046 Refine JDBC Source(source.default=8)
codecov-io commented on issue #710: KYLIN-4046 Refine JDBC Source(source.default=8) URL: https://github.com/apache/kylin/pull/710#issuecomment-504846974 # [Codecov](https://codecov.io/gh/apache/kylin/pull/710?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@760aefd`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `25.3%`. [](https://codecov.io/gh/apache/kylin/pull/710?src=pr=tree) ```diff @@Coverage Diff@@ ## master #710 +/- ## = Coverage ? 25.71% Complexity? 6018 = Files ? 1387 Lines ?82732 Branches ?11605 = Hits ?21278 Misses?59403 Partials ? 2051 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/710?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...ylin/source/jdbc/metadata/DefaultJdbcMetadata.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL21ldGFkYXRhL0RlZmF1bHRKZGJjTWV0YWRhdGEuamF2YQ==) | `95.83% <0%> (ø)` | `8 <0> (?)` | | | [...ain/java/org/apache/kylin/job/JoinedFlatTable.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9Kb2luZWRGbGF0VGFibGUuamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...in/source/jdbc/metadata/Oracle11gJdbcMetadata.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL21ldGFkYXRhL09yYWNsZTExZ0pkYmNNZXRhZGF0YS5qYXZh) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...ain/java/org/apache/kylin/source/jdbc/SqlUtil.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL1NxbFV0aWwuamF2YQ==) | `56.75% <0%> (ø)` | `22 <0> (?)` | | | [...in/java/org/apache/kylin/common/SourceDialect.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9Tb3VyY2VEaWFsZWN0LmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...ylin/source/jdbc/metadata/JdbcMetadataFactory.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL21ldGFkYXRhL0pkYmNNZXRhZGF0YUZhY3RvcnkuamF2YQ==) | `20% <0%> (ø)` | `1 <0> (?)` | | | [...in/source/jdbc/metadata/SQLServerJdbcMetadata.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL21ldGFkYXRhL1NRTFNlcnZlckpkYmNNZXRhZGF0YS5qYXZh) | `85.71% <0%> (ø)` | `4 <0> (?)` | | | [.../kylin/source/jdbc/metadata/MySQLJdbcMetadata.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL21ldGFkYXRhL015U1FMSmRiY01ldGFkYXRhLmphdmE=) | `93.75% <0%> (ø)` | `6 <0> (?)` | | | [.../org/apache/kylin/source/jdbc/JdbcTableReader.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL0pkYmNUYWJsZVJlYWRlci5qYXZh) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...ylin/source/jdbc/extensible/JdbcHiveInputBase.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL2V4dGVuc2libGUvSmRiY0hpdmVJbnB1dEJhc2UuamF2YQ==) | `84% <100%> (ø)` | `2 <0> (?)` | | | ... and [4 more](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/710?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/710?src=pr=footer). Last update [760aefd...a8b0f60](https://codecov.io/gh/apache/kylin/pull/710?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to
[GitHub] [kylin] nichunen merged pull request #711: KYLIN-4054 Logger of HCreateTableJob record error message
nichunen merged pull request #711: KYLIN-4054 Logger of HCreateTableJob record error message URL: https://github.com/apache/kylin/pull/711 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen merged pull request #705: Add docs for KYLIN-4033
nichunen merged pull request #705: Add docs for KYLIN-4033 URL: https://github.com/apache/kylin/pull/705 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] hit-lacus closed pull request #703: Revert "#KYLIN-4044, upgrade curator dependency version"
hit-lacus closed pull request #703: Revert "#KYLIN-4044, upgrade curator dependency version" URL: https://github.com/apache/kylin/pull/703 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen merged pull request #680: KYLIN-4035 Calculate column cardinality by using spark engine
nichunen merged pull request #680: KYLIN-4035 Calculate column cardinality by using spark engine URL: https://github.com/apache/kylin/pull/680 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls commented on issue #711: KYLIN-4054 Logger of HCreateTableJob record error message
coveralls commented on issue #711: KYLIN-4054 Logger of HCreateTableJob record error message URL: https://github.com/apache/kylin/pull/711#issuecomment-504851650 ## Pull Request Test Coverage Report for [Build 4656](https://coveralls.io/builds/24161630) * **0** of **33** **(0.0%)** changed or added relevant lines in **4** files are covered. * **8** unchanged lines in **5** files lost coverage. * Overall coverage increased (+**0.0003%**) to **28.185%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-common/src/main/java/org/apache/kylin/common/persistence/JDBCConnectionManager.java](https://coveralls.io/builds/24161630/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fpersistence%2FJDBCConnectionManager.java#L65) | 0 | 2 | 0.0% | [tool/src/main/java/org/apache/kylin/tool/CubeMetaIngester.java](https://coveralls.io/builds/24161630/source?filename=tool%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Ftool%2FCubeMetaIngester.java#L173) | 0 | 4 | 0.0% | [storage-hbase/src/main/java/org/apache/kylin/storage/hbase/steps/CreateHTableJob.java](https://coveralls.io/builds/24161630/source?filename=storage-hbase%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstorage%2Fhbase%2Fsteps%2FCreateHTableJob.java#L110) | 0 | 13 | 0.0% | [core-storage/src/main/java/org/apache/kylin/storage/gtrecord/GTCubeStorageQueryBase.java](https://coveralls.io/builds/24161630/source?filename=core-storage%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstorage%2Fgtrecord%2FGTCubeStorageQueryBase.java#L308) | 0 | 14 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/24161630/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62) | 1 | 81.08% | | [server-base/src/main/java/org/apache/kylin/rest/util/QueryRequestLimits.java](https://coveralls.io/builds/24161630/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Futil%2FQueryRequestLimits.java#L72) | 1 | 47.62% | | [core-storage/src/main/java/org/apache/kylin/storage/gtrecord/GTCubeStorageQueryBase.java](https://coveralls.io/builds/24161630/source?filename=core-storage%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstorage%2Fgtrecord%2FGTCubeStorageQueryBase.java#L582) | 1 | 0.0% | | [core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/24161630/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124) | 2 | 68.46% | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/24161630/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L449) | 3 | 78.42% | | Totals | [](https://coveralls.io/builds/24161630) | | :-- | --: | | Change from base [Build 4655](https://coveralls.io/builds/24161515): | 0.0003% | | Covered Lines: | 23256 | | Relevant Lines: | 82513 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls commented on issue #708: KYLIN-4031 RestClient will throw exception with message contains clea…
coveralls commented on issue #708: KYLIN-4031 RestClient will throw exception with message contains clea… URL: https://github.com/apache/kylin/pull/708#issuecomment-504723245 ## Pull Request Test Coverage Report for [Build 4648](https://coveralls.io/builds/24152434) * **0** of **1** **(0.0%)** changed or added relevant line in **1** file are covered. * **1** unchanged line in **1** file lost coverage. * Overall coverage increased (+**0.01%**) to **28.215%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-common/src/main/java/org/apache/kylin/common/restclient/RestClient.java](https://coveralls.io/builds/24152434/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Frestclient%2FRestClient.java#L102) | 0 | 1 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [server-base/src/main/java/org/apache/kylin/rest/util/QueryRequestLimits.java](https://coveralls.io/builds/24152434/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Futil%2FQueryRequestLimits.java#L72) | 1 | 47.62% | | Totals | [](https://coveralls.io/builds/24152434) | | :-- | --: | | Change from base [Build 4633](https://coveralls.io/builds/24099387): | 0.01% | | Covered Lines: | 23256 | | Relevant Lines: | 82424 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] luguosheng1314 merged pull request #693: KYLIN-3933 Invalid replica set cache after status updated
luguosheng1314 merged pull request #693: KYLIN-3933 Invalid replica set cache after status updated URL: https://github.com/apache/kylin/pull/693 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] hit-lacus commented on issue #710: KYLIN-4046 Refine JDBC Source(source.default=8)
hit-lacus commented on issue #710: KYLIN-4046 Refine JDBC Source(source.default=8) URL: https://github.com/apache/kylin/pull/710#issuecomment-504847464 Fix rat-plugin error. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls commented on issue #710: KYLIN-4046 Refine JDBC Source(source.default=8)
coveralls commented on issue #710: KYLIN-4046 Refine JDBC Source(source.default=8) URL: https://github.com/apache/kylin/pull/710#issuecomment-504769865 ## Pull Request Test Coverage Report for [Build 4650](https://coveralls.io/builds/24157131) * **68** of **249** **(27.31%)** changed or added relevant lines in **14** files are covered. * **64** unchanged lines in **6** files lost coverage. * Overall coverage decreased (**-0.04%**) to **28.162%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java](https://coveralls.io/builds/24157131/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2FJoinedFlatTable.java#L244) | 0 | 1 | 0.0% | [core-metadata/src/main/java/org/apache/kylin/metadata/model/TblColRef.java](https://coveralls.io/builds/24157131/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetadata%2Fmodel%2FTblColRef.java#L256) | 4 | 5 | 80.0% | [source-jdbc/src/main/java/org/apache/kylin/source/jdbc/metadata/DefaultJdbcMetadata.java](https://coveralls.io/builds/24157131/source?filename=source-jdbc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fjdbc%2Fmetadata%2FDefaultJdbcMetadata.java#L79) | 0 | 1 | 0.0% | [source-jdbc/src/main/java/org/apache/kylin/source/jdbc/metadata/JdbcMetadataFactory.java](https://coveralls.io/builds/24157131/source?filename=source-jdbc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fjdbc%2Fmetadata%2FJdbcMetadataFactory.java#L35) | 0 | 1 | 0.0% | [source-jdbc/src/main/java/org/apache/kylin/source/jdbc/metadata/MySQLJdbcMetadata.java](https://coveralls.io/builds/24157131/source?filename=source-jdbc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fjdbc%2Fmetadata%2FMySQLJdbcMetadata.java#L71) | 0 | 1 | 0.0% | [source-jdbc/src/main/java/org/apache/kylin/source/jdbc/metadata/SQLServerJdbcMetadata.java](https://coveralls.io/builds/24157131/source?filename=source-jdbc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fjdbc%2Fmetadata%2FSQLServerJdbcMetadata.java#L66) | 0 | 1 | 0.0% | [source-jdbc/src/main/java/org/apache/kylin/source/jdbc/SqlUtil.java](https://coveralls.io/builds/24157131/source?filename=source-jdbc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fjdbc%2FSqlUtil.java#L73) | 0 | 2 | 0.0% | [source-jdbc/src/main/java/org/apache/kylin/source/jdbc/metadata/Oracle11gJdbcMetadata.java](https://coveralls.io/builds/24157131/source?filename=source-jdbc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fjdbc%2Fmetadata%2FOracle11gJdbcMetadata.java#L9) | 0 | 3 | 0.0% | [core-metadata/src/main/java/org/apache/kylin/metadata/model/PartitionDesc.java](https://coveralls.io/builds/24157131/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetadata%2Fmodel%2FPartitionDesc.java#L235) | 7 | 11 | 63.64% | [source-jdbc/src/main/java/org/apache/kylin/source/jdbc/JdbcExplorer.java](https://coveralls.io/builds/24157131/source?filename=source-jdbc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fjdbc%2FJdbcExplorer.java#L65) | 0 | 7 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/24157131/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L553) | 1 | 78.12% | | [server-base/src/main/java/org/apache/kylin/rest/util/QueryRequestLimits.java](https://coveralls.io/builds/24157131/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Futil%2FQueryRequestLimits.java#L72) | 1 | 47.62% | | [core-metadata/src/main/java/org/apache/kylin/metadata/model/PartitionDesc.java](https://coveralls.io/builds/24157131/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetadata%2Fmodel%2FPartitionDesc.java#L327) | 1 | 70.73% | | [source-jdbc/src/main/java/org/apache/kylin/source/jdbc/metadata/JdbcMetadataFactory.java](https://coveralls.io/builds/24157131/source?filename=source-jdbc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fjdbc%2Fmetadata%2FJdbcMetadataFactory.java#L31) | 2 | 40.0% | | [source-jdbc/src/main/java/org/apache/kylin/source/jdbc/JdbcHiveInputBase.java](https://coveralls.io/builds/24157131/source?filename=source-jdbc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fjdbc%2FJdbcHiveInputBase.java#L229) | 5 | 23.96% | | [source-jdbc/src/main/java/org/apache/kylin/source/jdbc/JdbcExplorer.java](https://coveralls.io/builds/24157131/source?filename=source-jdbc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fjdbc%2FJdbcExplorer.java#L51) | 54 | 0.0% | | Totals | [![Coverage
[GitHub] [kylin] hit-lacus opened a new pull request #710: KYLIN-4046 Refine JDBC Source(source.default=8)
hit-lacus opened a new pull request #710: KYLIN-4046 Refine JDBC Source(source.default=8) URL: https://github.com/apache/kylin/pull/710 Currently, the function of ingest data from RDBMS(kylin.source.default=8) to Kylin has some problems , in this patch, I want to fix: - fix case-sensitive - fix weak dialect compatibility - fix mis-use quote character - start support oracle 11g This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #710: KYLIN-4046 Refine JDBC Source(source.default=8)
asfgit commented on issue #710: KYLIN-4046 Refine JDBC Source(source.default=8) URL: https://github.com/apache/kylin/pull/710#issuecomment-504767850 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] codecov-io edited a comment on issue #710: KYLIN-4046 Refine JDBC Source(source.default=8)
codecov-io edited a comment on issue #710: KYLIN-4046 Refine JDBC Source(source.default=8) URL: https://github.com/apache/kylin/pull/710#issuecomment-504846974 # [Codecov](https://codecov.io/gh/apache/kylin/pull/710?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@9a72b9e`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `25.63%`. [](https://codecov.io/gh/apache/kylin/pull/710?src=pr=tree) ```diff @@Coverage Diff@@ ## master #710 +/- ## = Coverage ? 25.72% Complexity? 6018 = Files ? 1386 Lines ?82720 Branches ?11603 = Hits ?21276 Misses?59394 Partials ? 2050 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/710?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...ylin/source/jdbc/metadata/JdbcMetadataFactory.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL21ldGFkYXRhL0pkYmNNZXRhZGF0YUZhY3RvcnkuamF2YQ==) | `25% <ø> (ø)` | `1 <0> (?)` | | | [...ylin/source/jdbc/metadata/DefaultJdbcMetadata.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL21ldGFkYXRhL0RlZmF1bHRKZGJjTWV0YWRhdGEuamF2YQ==) | `95.83% <0%> (ø)` | `8 <0> (?)` | | | [...ain/java/org/apache/kylin/job/JoinedFlatTable.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9Kb2luZWRGbGF0VGFibGUuamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | | | [.../org/apache/kylin/source/jdbc/JdbcTableReader.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL0pkYmNUYWJsZVJlYWRlci5qYXZh) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...ain/java/org/apache/kylin/source/jdbc/SqlUtil.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL1NxbFV0aWwuamF2YQ==) | `56.75% <0%> (ø)` | `22 <0> (?)` | | | [...in/java/org/apache/kylin/common/SourceDialect.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9Tb3VyY2VEaWFsZWN0LmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...in/source/jdbc/metadata/SQLServerJdbcMetadata.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL21ldGFkYXRhL1NRTFNlcnZlckpkYmNNZXRhZGF0YS5qYXZh) | `85.71% <0%> (ø)` | `4 <0> (?)` | | | [.../kylin/source/jdbc/metadata/MySQLJdbcMetadata.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL21ldGFkYXRhL015U1FMSmRiY01ldGFkYXRhLmphdmE=) | `93.75% <0%> (ø)` | `6 <0> (?)` | | | [...ylin/source/jdbc/extensible/JdbcHiveInputBase.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL2V4dGVuc2libGUvSmRiY0hpdmVJbnB1dEJhc2UuamF2YQ==) | `84% <100%> (ø)` | `2 <0> (?)` | | | [...ava/org/apache/kylin/source/jdbc/JdbcExplorer.java](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree#diff-c291cmNlLWpkYmMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9qZGJjL0pkYmNFeHBsb3Jlci5qYXZh) | `35.86% <14.28%> (ø)` | `10 <0> (?)` | | | ... and [3 more](https://codecov.io/gh/apache/kylin/pull/710/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/710?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/710?src=pr=footer). Last update [9a72b9e...579078f](https://codecov.io/gh/apache/kylin/pull/710?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
[GitHub] [kylin] nichunen merged pull request #699: fix add jacoco.version in pom.xml
nichunen merged pull request #699: fix add jacoco.version in pom.xml URL: https://github.com/apache/kylin/pull/699 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] Qsimple closed pull request #709: KYLIN-3982 Add measures without purging segments
Qsimple closed pull request #709: KYLIN-3982 Add measures without purging segments URL: https://github.com/apache/kylin/pull/709 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] Qsimple opened a new pull request #712: KYLIN-3982 Add measures without purging segments
Qsimple opened a new pull request #712: KYLIN-3982 Add measures without purging segments URL: https://github.com/apache/kylin/pull/712 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #712: KYLIN-3982 Add measures without purging segments
asfgit commented on issue #712: KYLIN-3982 Add measures without purging segments URL: https://github.com/apache/kylin/pull/712#issuecomment-505079110 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] shaofengshi merged pull request #661: KYLIN-4015 "Build UHC Dictionary" step filter ".dci" files
shaofengshi merged pull request #661: KYLIN-4015 "Build UHC Dictionary" step filter ".dci" files URL: https://github.com/apache/kylin/pull/661 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen commented on a change in pull request #664: KYLIN-4017 Build engine get zk(zookeeper) lock failed when building job, it causes the whole build engine doesn't work
nichunen commented on a change in pull request #664: KYLIN-4017 Build engine
get zk(zookeeper) lock failed when building job, it causes the whole build
engine doesn't work
URL: https://github.com/apache/kylin/pull/664#discussion_r293188955
##
File path: core-common/src/main/java/org/apache/kylin/common/util/ZKUtil.java
##
@@ -84,7 +84,7 @@ public void onRemoval(RemovalNotification notification
logger.error("Error at closing " + curator, ex);
}
}
-}).expireAfterWrite(1, TimeUnit.DAYS).build();
+}).expireAfterWrite(1, TimeUnit.DAYS).build();//never expired
Review comment:
OK
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] nichunen merged pull request #664: KYLIN-4017 Build engine get zk(zookeeper) lock failed when building job, it causes the whole build engine doesn't work
nichunen merged pull request #664: KYLIN-4017 Build engine get zk(zookeeper) lock failed when building job, it causes the whole build engine doesn't work URL: https://github.com/apache/kylin/pull/664 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] hn5092 commented on a change in pull request #676: KYLIN-4033 Can not access Kerberized Cluster with DebugTomcat
hn5092 commented on a change in pull request #676: KYLIN-4033 Can not access
Kerberized Cluster with DebugTomcat
URL: https://github.com/apache/kylin/pull/676#discussion_r292724726
##
File path: server/src/main/java/org/apache/kylin/rest/DebugTomcat.java
##
@@ -99,6 +107,25 @@ private static File findFile(String dir, String ptn) {
}
return null;
}
+public static boolean isKerberosEnabled() {
+Configuration conf = new Configuration();
+boolean authEnabled = conf.getBoolean("hadoop.security.authorization",
false);
+String authMethod = conf.get("hadoop.security.authentication");
+return authEnabled &&
authMethod.toLowerCase(Locale.ROOT).equals("kerberos");
Review comment:
UserGroupInformation.isSecurityEnabled()
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] hn5092 commented on a change in pull request #676: KYLIN-4033 Can not access Kerberized Cluster with DebugTomcat
hn5092 commented on a change in pull request #676: KYLIN-4033 Can not access
Kerberized Cluster with DebugTomcat
URL: https://github.com/apache/kylin/pull/676#discussion_r292726096
##
File path: server/src/main/java/org/apache/kylin/rest/DebugTomcat.java
##
@@ -99,6 +107,25 @@ private static File findFile(String dir, String ptn) {
}
return null;
}
+public static boolean isKerberosEnabled() {
+Configuration conf = new Configuration();
+boolean authEnabled = conf.getBoolean("hadoop.security.authorization",
false);
+String authMethod = conf.get("hadoop.security.authentication");
+return authEnabled &&
authMethod.toLowerCase(Locale.ROOT).equals("kerberos");
+}
+
+public static void authKrb5() {
+System.setProperty("java.security.krb5.conf", "/etc/krb5.conf");
+Configuration conf = new Configuration();
+UserGroupInformation.setConfiguration(conf);
Review comment:
UGI needn't set
hadoopconf.org.apache.hadoop.security.UserGroupInformation#isSecurityEnabled
will init it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] hn5092 commented on a change in pull request #676: KYLIN-4033 Can not access Kerberized Cluster with DebugTomcat
hn5092 commented on a change in pull request #676: KYLIN-4033 Can not access
Kerberized Cluster with DebugTomcat
URL: https://github.com/apache/kylin/pull/676#discussion_r292726863
##
File path: server/src/main/java/org/apache/kylin/rest/DebugTomcat.java
##
@@ -99,6 +107,25 @@ private static File findFile(String dir, String ptn) {
}
return null;
}
+public static boolean isKerberosEnabled() {
+Configuration conf = new Configuration();
+boolean authEnabled = conf.getBoolean("hadoop.security.authorization",
false);
+String authMethod = conf.get("hadoop.security.authentication");
+return authEnabled &&
authMethod.toLowerCase(Locale.ROOT).equals("kerberos");
+}
+
+public static void authKrb5() {
+System.setProperty("java.security.krb5.conf", "/etc/krb5.conf");
Review comment:
System prop should be noted in dev doc.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] hn5092 commented on a change in pull request #676: KYLIN-4033 Can not access Kerberized Cluster with DebugTomcat
hn5092 commented on a change in pull request #676: KYLIN-4033 Can not access
Kerberized Cluster with DebugTomcat
URL: https://github.com/apache/kylin/pull/676#discussion_r292726635
##
File path: server/src/main/java/org/apache/kylin/rest/DebugTomcat.java
##
@@ -99,6 +107,25 @@ private static File findFile(String dir, String ptn) {
}
return null;
}
+public static boolean isKerberosEnabled() {
+Configuration conf = new Configuration();
+boolean authEnabled = conf.getBoolean("hadoop.security.authorization",
false);
+String authMethod = conf.get("hadoop.security.authentication");
+return authEnabled &&
authMethod.toLowerCase(Locale.ROOT).equals("kerberos");
+}
+
+public static void authKrb5() {
+System.setProperty("java.security.krb5.conf", "/etc/krb5.conf");
+Configuration conf = new Configuration();
+UserGroupInformation.setConfiguration(conf);
+try {
+UserGroupInformation.loginUserFromKeytab(
Review comment:
Use another way? Systemprop or env and note to dev doc.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] hn5092 commented on a change in pull request #676: KYLIN-4033 Can not access Kerberized Cluster with DebugTomcat
hn5092 commented on a change in pull request #676: KYLIN-4033 Can not access
Kerberized Cluster with DebugTomcat
URL: https://github.com/apache/kylin/pull/676#discussion_r292726863
##
File path: server/src/main/java/org/apache/kylin/rest/DebugTomcat.java
##
@@ -99,6 +107,25 @@ private static File findFile(String dir, String ptn) {
}
return null;
}
+public static boolean isKerberosEnabled() {
+Configuration conf = new Configuration();
+boolean authEnabled = conf.getBoolean("hadoop.security.authorization",
false);
+String authMethod = conf.get("hadoop.security.authentication");
+return authEnabled &&
authMethod.toLowerCase(Locale.ROOT).equals("kerberos");
+}
+
+public static void authKrb5() {
+System.setProperty("java.security.krb5.conf", "/etc/krb5.conf");
Review comment:
System prop should note to dev doc.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] hn5092 commented on a change in pull request #676: KYLIN-4033 Can not access Kerberized Cluster with DebugTomcat
hn5092 commented on a change in pull request #676: KYLIN-4033 Can not access
Kerberized Cluster with DebugTomcat
URL: https://github.com/apache/kylin/pull/676#discussion_r292726635
##
File path: server/src/main/java/org/apache/kylin/rest/DebugTomcat.java
##
@@ -99,6 +107,25 @@ private static File findFile(String dir, String ptn) {
}
return null;
}
+public static boolean isKerberosEnabled() {
+Configuration conf = new Configuration();
+boolean authEnabled = conf.getBoolean("hadoop.security.authorization",
false);
+String authMethod = conf.get("hadoop.security.authentication");
+return authEnabled &&
authMethod.toLowerCase(Locale.ROOT).equals("kerberos");
+}
+
+public static void authKrb5() {
+System.setProperty("java.security.krb5.conf", "/etc/krb5.conf");
+Configuration conf = new Configuration();
+UserGroupInformation.setConfiguration(conf);
+try {
+UserGroupInformation.loginUserFromKeytab(
Review comment:
Use another way? Systemprop or env and noted to dev doc.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] hit-lacus commented on a change in pull request #680: KYLIN-4035 Calculate column cardinality by using spark engine
hit-lacus commented on a change in pull request #680: KYLIN-4035 Calculate
column cardinality by using spark engine
URL: https://github.com/apache/kylin/pull/680#discussion_r294071741
##
File path:
engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkColumnCardinality.java
##
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.engine.spark;
+
+import org.apache.commons.cli.Option;
+import org.apache.commons.cli.OptionBuilder;
+import org.apache.commons.cli.Options;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.io.IntWritable;
+import org.apache.hadoop.io.LongWritable;
+import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat;
+import org.apache.kylin.common.util.AbstractApplication;
+import org.apache.kylin.common.util.Bytes;
+import org.apache.kylin.common.util.HadoopUtil;
+import org.apache.kylin.common.util.OptionsHelper;
+import org.apache.kylin.engine.mr.common.BatchConstants;
+import org.apache.kylin.measure.hllc.HLLCounter;
+import org.apache.spark.SparkConf;
+import org.apache.spark.api.java.JavaPairRDD;
+import org.apache.spark.api.java.JavaRDD;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.apache.spark.api.java.function.PairFlatMapFunction;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import scala.Tuple2;
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+public class SparkColumnCardinality extends AbstractApplication implements
Serializable {
+protected static final Logger logger =
LoggerFactory.getLogger(SparkColumnCardinality.class);
+
+public static final Option OPTION_TABLE_NAME =
OptionBuilder.withArgName(BatchConstants.ARG_TABLE_NAME).hasArg()
+.isRequired(true).withDescription("Table
Name").create(BatchConstants.ARG_TABLE_NAME);
+public static final Option OPTION_OUTPUT =
OptionBuilder.withArgName(BatchConstants.ARG_OUTPUT).hasArg()
+
.isRequired(true).withDescription("Output").create(BatchConstants.ARG_OUTPUT);
+public static final Option OPTION_PRJ =
OptionBuilder.withArgName(BatchConstants.ARG_PROJECT).hasArg()
+.isRequired(true).withDescription("Project
name").create(BatchConstants.ARG_PROJECT);
+public static final Option OPTION_COLUMN_COUNT =
OptionBuilder.withArgName(BatchConstants.CFG_OUTPUT_COLUMN).hasArg()
+.isRequired(true).withDescription("column
count").create(BatchConstants.CFG_OUTPUT_COLUMN);
+
+private Options options;
+
+public SparkColumnCardinality() {
+options = new Options();
+options.addOption(OPTION_TABLE_NAME);
+options.addOption(OPTION_OUTPUT);
+options.addOption(OPTION_PRJ);
+options.addOption(OPTION_COLUMN_COUNT);
+}
+
+@Override
+protected Options getOptions() {
+return options;
+}
+
+@Override
+protected void execute(OptionsHelper optionsHelper) throws Exception {
+String tableName = optionsHelper.getOptionValue(OPTION_TABLE_NAME);
+String output = optionsHelper.getOptionValue(OPTION_OUTPUT);
+int columnCnt =
Integer.valueOf(optionsHelper.getOptionValue(OPTION_COLUMN_COUNT));
+
+Class[] kryoClassArray = new
Class[]{Class.forName("scala.reflect.ClassTag$$anon$1"),
+
Class.forName("org.apache.kylin.engine.mr.steps.SelfDefineSortableKey")};
+
+SparkConf conf = new SparkConf().setAppName("Calculate table:" +
tableName);
+//set spark.sql.catalogImplementation=hive, If it is not set,
SparkSession can't read hive metadata, and throw
"org.apache.spark.sql.AnalysisException"
+conf.set("spark.sql.catalogImplementation", "hive");
+//serialization conf
+conf.set("spark.serializer",
"org.apache.spark.serializer.KryoSerializer");
+conf.set("spark.kryo.registrator",
"org.apache.kylin.engine.spark.KylinKryoRegistrator");
+conf.set("spark.kryo.registrationRequired",
"true").registerKryoClasses(kryoClassArray);
+
+KylinSparkJobListener jobListener = new KylinSparkJobListener();
+try
[GitHub] [kylin] hit-lacus commented on a change in pull request #680: KYLIN-4035 Calculate column cardinality by using spark engine
hit-lacus commented on a change in pull request #680: KYLIN-4035 Calculate
column cardinality by using spark engine
URL: https://github.com/apache/kylin/pull/680#discussion_r294071741
##
File path:
engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkColumnCardinality.java
##
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.engine.spark;
+
+import org.apache.commons.cli.Option;
+import org.apache.commons.cli.OptionBuilder;
+import org.apache.commons.cli.Options;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.io.IntWritable;
+import org.apache.hadoop.io.LongWritable;
+import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat;
+import org.apache.kylin.common.util.AbstractApplication;
+import org.apache.kylin.common.util.Bytes;
+import org.apache.kylin.common.util.HadoopUtil;
+import org.apache.kylin.common.util.OptionsHelper;
+import org.apache.kylin.engine.mr.common.BatchConstants;
+import org.apache.kylin.measure.hllc.HLLCounter;
+import org.apache.spark.SparkConf;
+import org.apache.spark.api.java.JavaPairRDD;
+import org.apache.spark.api.java.JavaRDD;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.apache.spark.api.java.function.PairFlatMapFunction;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import scala.Tuple2;
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+public class SparkColumnCardinality extends AbstractApplication implements
Serializable {
+protected static final Logger logger =
LoggerFactory.getLogger(SparkColumnCardinality.class);
+
+public static final Option OPTION_TABLE_NAME =
OptionBuilder.withArgName(BatchConstants.ARG_TABLE_NAME).hasArg()
+.isRequired(true).withDescription("Table
Name").create(BatchConstants.ARG_TABLE_NAME);
+public static final Option OPTION_OUTPUT =
OptionBuilder.withArgName(BatchConstants.ARG_OUTPUT).hasArg()
+
.isRequired(true).withDescription("Output").create(BatchConstants.ARG_OUTPUT);
+public static final Option OPTION_PRJ =
OptionBuilder.withArgName(BatchConstants.ARG_PROJECT).hasArg()
+.isRequired(true).withDescription("Project
name").create(BatchConstants.ARG_PROJECT);
+public static final Option OPTION_COLUMN_COUNT =
OptionBuilder.withArgName(BatchConstants.CFG_OUTPUT_COLUMN).hasArg()
+.isRequired(true).withDescription("column
count").create(BatchConstants.CFG_OUTPUT_COLUMN);
+
+private Options options;
+
+public SparkColumnCardinality() {
+options = new Options();
+options.addOption(OPTION_TABLE_NAME);
+options.addOption(OPTION_OUTPUT);
+options.addOption(OPTION_PRJ);
+options.addOption(OPTION_COLUMN_COUNT);
+}
+
+@Override
+protected Options getOptions() {
+return options;
+}
+
+@Override
+protected void execute(OptionsHelper optionsHelper) throws Exception {
+String tableName = optionsHelper.getOptionValue(OPTION_TABLE_NAME);
+String output = optionsHelper.getOptionValue(OPTION_OUTPUT);
+int columnCnt =
Integer.valueOf(optionsHelper.getOptionValue(OPTION_COLUMN_COUNT));
+
+Class[] kryoClassArray = new
Class[]{Class.forName("scala.reflect.ClassTag$$anon$1"),
+
Class.forName("org.apache.kylin.engine.mr.steps.SelfDefineSortableKey")};
+
+SparkConf conf = new SparkConf().setAppName("Calculate table:" +
tableName);
+//set spark.sql.catalogImplementation=hive, If it is not set,
SparkSession can't read hive metadata, and throw
"org.apache.spark.sql.AnalysisException"
+conf.set("spark.sql.catalogImplementation", "hive");
+//serialization conf
+conf.set("spark.serializer",
"org.apache.spark.serializer.KryoSerializer");
+conf.set("spark.kryo.registrator",
"org.apache.kylin.engine.spark.KylinKryoRegistrator");
+conf.set("spark.kryo.registrationRequired",
"true").registerKryoClasses(kryoClassArray);
+
+KylinSparkJobListener jobListener = new KylinSparkJobListener();
+try
[GitHub] [kylin] codecov-io commented on issue #694: KYLIN-4049 Make storage clean configurable for refreshing job
codecov-io commented on issue #694: KYLIN-4049 Make storage clean configurable for refreshing job URL: https://github.com/apache/kylin/pull/694#issuecomment-503039537 # [Codecov](https://codecov.io/gh/apache/kylin/pull/694?src=pr=h1) Report > Merging [#694](https://codecov.io/gh/apache/kylin/pull/694?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/2cfead0b3c7772366e233c6bc61942ca38c68642?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `0%`. [](https://codecov.io/gh/apache/kylin/pull/694?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #694 +/- ## - Coverage 25.73% 25.72% -0.01% + Complexity 6011 6010 -1 Files 1385 1385 Lines 8240182401 Branches 1155311553 - Hits 2120421196 -8 - Misses5915459159 +5 - Partials 2043 2046 +3 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/694?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...pache/kylin/storage/hbase/steps/HBaseJobSteps.java](https://codecov.io/gh/apache/kylin/pull/694/diff?src=pr=tree#diff-c3RvcmFnZS1oYmFzZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RvcmFnZS9oYmFzZS9zdGVwcy9IQmFzZUpvYlN0ZXBzLmphdmE=) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/694/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `72.97% <0%> (-5.41%)` | `6% <0%> (-1%)` | | | [...he/kylin/job/impl/threadpool/DefaultScheduler.java](https://codecov.io/gh/apache/kylin/pull/694/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL3RocmVhZHBvb2wvRGVmYXVsdFNjaGVkdWxlci5qYXZh) | `74.41% <0%> (-2.33%)` | `12% <0%> (ø)` | | | [.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/694/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=) | `63.84% <0%> (-2.31%)` | `0% <0%> (ø)` | | | [...a/org/apache/kylin/dict/Number2BytesConverter.java](https://codecov.io/gh/apache/kylin/pull/694/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L051bWJlcjJCeXRlc0NvbnZlcnRlci5qYXZh) | `81.74% <0%> (-0.8%)` | `17% <0%> (-1%)` | | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/694/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `69.6% <0%> (-0.61%)` | `7% <0%> (ø)` | | | [...core/storage/columnar/GeneralColumnDataReader.java](https://codecov.io/gh/apache/kylin/pull/694/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3N0b3JhZ2UvY29sdW1uYXIvR2VuZXJhbENvbHVtbkRhdGFSZWFkZXIuamF2YQ==) | `94.73% <0%> (+5.26%)` | `7% <0%> (+1%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/694?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/694?src=pr=footer). Last update [2cfead0...d58951d](https://codecov.io/gh/apache/kylin/pull/694?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen merged pull request #694: KYLIN-4049 Make storage clean configurable for refreshing job
nichunen merged pull request #694: KYLIN-4049 Make storage clean configurable for refreshing job URL: https://github.com/apache/kylin/pull/694 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls commented on issue #694: KYLIN-4049 Make storage clean configurable for refreshing job
coveralls commented on issue #694: KYLIN-4049 Make storage clean configurable for refreshing job URL: https://github.com/apache/kylin/pull/694#issuecomment-503040338 ## Pull Request Test Coverage Report for [Build 4623](https://coveralls.io/builds/24057638) * **0** of **1** **(0.0%)** changed or added relevant line in **1** file are covered. * **6** unchanged lines in **4** files lost coverage. * Overall coverage decreased (**-0.006%**) to **28.21%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [storage-hbase/src/main/java/org/apache/kylin/storage/hbase/steps/HBaseJobSteps.java](https://coveralls.io/builds/24057638/source?filename=storage-hbase%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstorage%2Fhbase%2Fsteps%2FHBaseJobSteps.java#L245) | 0 | 1 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/24057638/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L553) | 1 | 78.12% | | [core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/24057638/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62) | 1 | 81.08% | | [core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/24057638/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124) | 2 | 68.46% | | [core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultScheduler.java](https://coveralls.io/builds/24057638/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultScheduler.java#L194) | 2 | 80.23% | | Totals | [](https://coveralls.io/builds/24057638) | | :-- | --: | | Change from base [Build 4621](https://coveralls.io/builds/24053032): | -0.006% | | Covered Lines: | 23245 | | Relevant Lines: | 82401 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls commented on issue #689: KYLIN-4047 Use push-down query when division dynamic column cube query is not supported
coveralls commented on issue #689: KYLIN-4047 Use push-down query when division dynamic column cube query is not supported URL: https://github.com/apache/kylin/pull/689#issuecomment-502610604 ## Pull Request Test Coverage Report for [Build 4613](https://coveralls.io/builds/24033066) * **4** of **10** **(40.0%)** changed or added relevant lines in **2** files are covered. * **6** unchanged lines in **2** files lost coverage. * Overall coverage increased (+**0.0009%**) to **28.214%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-metadata/src/main/java/org/apache/kylin/exception/QueryOnCubeException.java](https://coveralls.io/builds/24033066/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fexception%2FQueryOnCubeException.java#L27) | 2 | 8 | 25.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/24033066/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124) | 2 | 68.46% | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/24033066/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L449) | 4 | 78.12% | | Totals | [](https://coveralls.io/builds/24033066) | | :-- | --: | | Change from base [Build 4606](https://coveralls.io/builds/23990229): | 0.0009% | | Covered Lines: | 23249 | | Relevant Lines: | 82403 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #695: KYLIN-4050 Fix a CLI option parse error in FlinkMergingDictionary and some little code refactor
asfgit commented on issue #695: KYLIN-4050 Fix a CLI option parse error in FlinkMergingDictionary and some little code refactor URL: https://github.com/apache/kylin/pull/695#issuecomment-503079241 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #695: KYLIN-4050 Fix a CLI option parse error in FlinkMergingDictionary and some little code refactor
asfgit commented on issue #695: KYLIN-4050 Fix a CLI option parse error in FlinkMergingDictionary and some little code refactor URL: https://github.com/apache/kylin/pull/695#issuecomment-503079244 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #695: KYLIN-4050 Fix a CLI option parse error in FlinkMergingDictionary and some little code refactor
asfgit commented on issue #695: KYLIN-4050 Fix a CLI option parse error in FlinkMergingDictionary and some little code refactor URL: https://github.com/apache/kylin/pull/695#issuecomment-503079237 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #695: KYLIN-4050 Fix a CLI option parse error in FlinkMergingDictionary and some little code refactor
asfgit commented on issue #695: KYLIN-4050 Fix a CLI option parse error in FlinkMergingDictionary and some little code refactor URL: https://github.com/apache/kylin/pull/695#issuecomment-503079242 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] yanghua opened a new pull request #695: KYLIN-4050 Fix a CLI option parse error in FlinkMergingDictionary and some little code refactor
yanghua opened a new pull request #695: KYLIN-4050 Fix a CLI option parse error in FlinkMergingDictionary and some little code refactor URL: https://github.com/apache/kylin/pull/695 JIRA : https://issues.apache.org/jira/browse/KYLIN-4050 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] liuzx32 commented on issue #654: Bug Fix: remove HBase tableName toUpperCase()
liuzx32 commented on issue #654: Bug Fix: remove HBase tableName toUpperCase() URL: https://github.com/apache/kylin/pull/654#issuecomment-502554022 @nichunen Thank you, I will try it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] liuzx32 edited a comment on issue #657: adjust HBase tableName toUpperCase for HFile
liuzx32 edited a comment on issue #657: adjust HBase tableName toUpperCase for HFile URL: https://github.com/apache/kylin/pull/657#issuecomment-502587332 @nichunen Because HBase tableName is case sensitive. HBase tableName.toUpperCase() will different from the original table, leading to table can not be found. # And it's only for hbase, not for hive, in the branch master-hadoop3.1. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #688: remove tableName.toUppderCase() for hbase table Name,
asfgit commented on issue #688: remove tableName.toUppderCase() for hbase table Name, URL: https://github.com/apache/kylin/pull/688#issuecomment-502592818 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #688: remove tableName.toUppderCase() for hbase table Name,
asfgit commented on issue #688: remove tableName.toUppderCase() for hbase table Name, URL: https://github.com/apache/kylin/pull/688#issuecomment-502592834 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] liuzx32 opened a new pull request #688: remove tableName.toUppderCase() for hbase table Name,
liuzx32 opened a new pull request #688: remove tableName.toUppderCase() for hbase table Name, URL: https://github.com/apache/kylin/pull/688 **Description** HBase is case sensitive，tableName.toUpperCase() will lead to the created table could not be found. If my tableName of HBase is lowercase, kylin can't find the table created in the HFile generated step. **Improvement** just remove toUpperCase() for HBase tableName on master-hadoop3.1. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #688: remove tableName.toUppderCase() for hbase table Name,
asfgit commented on issue #688: remove tableName.toUppderCase() for hbase table Name, URL: https://github.com/apache/kylin/pull/688#issuecomment-502592827 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #688: remove tableName.toUppderCase() for hbase table Name,
asfgit commented on issue #688: remove tableName.toUppderCase() for hbase table Name, URL: https://github.com/apache/kylin/pull/688#issuecomment-502592820 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] wangxiaojing123 opened a new pull request #689: KYLIN-4047 Use push-down query when division dynamic column cube query is not supported
wangxiaojing123 opened a new pull request #689: KYLIN-4047 Use push-down query when division dynamic column cube query is not supported URL: https://github.com/apache/kylin/pull/689 Scene： SQL:select round (car_dismatch_cnt*100/answer_order_cnt, 4) from db.table1. - 1、Executes the SQL on a project where its cube has db.table1,it will throws the right side of the BinaryTuple Expressions columns is not supported for / exception and is not queried through pushdown; - 2、When change a project where not create the table cube , the same query can be executed to push down and get the right result. - 3、 Expect that no matter whether a table cube is created or not, when the cube query is not success, the result can still be obtained by pushdown query, which will give users a consistent experience. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #689: KYLIN-4047 Use push-down query when division dynamic column cube query is not supported
asfgit commented on issue #689: KYLIN-4047 Use push-down query when division dynamic column cube query is not supported URL: https://github.com/apache/kylin/pull/689#issuecomment-502593815 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #689: KYLIN-4047 Use push-down query when division dynamic column cube query is not supported
asfgit commented on issue #689: KYLIN-4047 Use push-down query when division dynamic column cube query is not supported URL: https://github.com/apache/kylin/pull/689#issuecomment-502593816 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #689: KYLIN-4047 Use push-down query when division dynamic column cube query is not supported
asfgit commented on issue #689: KYLIN-4047 Use push-down query when division dynamic column cube query is not supported URL: https://github.com/apache/kylin/pull/689#issuecomment-502593817 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #689: KYLIN-4047 Use push-down query when division dynamic column cube query is not supported
asfgit commented on issue #689: KYLIN-4047 Use push-down query when division dynamic column cube query is not supported URL: https://github.com/apache/kylin/pull/689#issuecomment-502593818 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] majic31 commented on a change in pull request #680: KYLIN-4035 Calculate column cardinality by using spark engine
majic31 commented on a change in pull request #680: KYLIN-4035 Calculate column
cardinality by using spark engine
URL: https://github.com/apache/kylin/pull/680#discussion_r294143074
##
File path:
engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkColumnCardinality.java
##
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.engine.spark;
+
+import org.apache.commons.cli.Option;
+import org.apache.commons.cli.OptionBuilder;
+import org.apache.commons.cli.Options;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.io.IntWritable;
+import org.apache.hadoop.io.LongWritable;
+import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat;
+import org.apache.kylin.common.util.AbstractApplication;
+import org.apache.kylin.common.util.Bytes;
+import org.apache.kylin.common.util.HadoopUtil;
+import org.apache.kylin.common.util.OptionsHelper;
+import org.apache.kylin.engine.mr.common.BatchConstants;
+import org.apache.kylin.measure.hllc.HLLCounter;
+import org.apache.spark.SparkConf;
+import org.apache.spark.api.java.JavaPairRDD;
+import org.apache.spark.api.java.JavaRDD;
+import org.apache.spark.api.java.JavaSparkContext;
+import org.apache.spark.api.java.function.PairFlatMapFunction;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import scala.Tuple2;
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+public class SparkColumnCardinality extends AbstractApplication implements
Serializable {
+protected static final Logger logger =
LoggerFactory.getLogger(SparkColumnCardinality.class);
+
+public static final Option OPTION_TABLE_NAME =
OptionBuilder.withArgName(BatchConstants.ARG_TABLE_NAME).hasArg()
+.isRequired(true).withDescription("Table
Name").create(BatchConstants.ARG_TABLE_NAME);
+public static final Option OPTION_OUTPUT =
OptionBuilder.withArgName(BatchConstants.ARG_OUTPUT).hasArg()
+
.isRequired(true).withDescription("Output").create(BatchConstants.ARG_OUTPUT);
+public static final Option OPTION_PRJ =
OptionBuilder.withArgName(BatchConstants.ARG_PROJECT).hasArg()
+.isRequired(true).withDescription("Project
name").create(BatchConstants.ARG_PROJECT);
+public static final Option OPTION_COLUMN_COUNT =
OptionBuilder.withArgName(BatchConstants.CFG_OUTPUT_COLUMN).hasArg()
+.isRequired(true).withDescription("column
count").create(BatchConstants.CFG_OUTPUT_COLUMN);
+
+private Options options;
+
+public SparkColumnCardinality() {
+options = new Options();
+options.addOption(OPTION_TABLE_NAME);
+options.addOption(OPTION_OUTPUT);
+options.addOption(OPTION_PRJ);
+options.addOption(OPTION_COLUMN_COUNT);
+}
+
+@Override
+protected Options getOptions() {
+return options;
+}
+
+@Override
+protected void execute(OptionsHelper optionsHelper) throws Exception {
+String tableName = optionsHelper.getOptionValue(OPTION_TABLE_NAME);
+String output = optionsHelper.getOptionValue(OPTION_OUTPUT);
+int columnCnt =
Integer.valueOf(optionsHelper.getOptionValue(OPTION_COLUMN_COUNT));
+
+Class[] kryoClassArray = new
Class[]{Class.forName("scala.reflect.ClassTag$$anon$1"),
+
Class.forName("org.apache.kylin.engine.mr.steps.SelfDefineSortableKey")};
+
+SparkConf conf = new SparkConf().setAppName("Calculate table:" +
tableName);
+//set spark.sql.catalogImplementation=hive, If it is not set,
SparkSession can't read hive metadata, and throw
"org.apache.spark.sql.AnalysisException"
+conf.set("spark.sql.catalogImplementation", "hive");
+//serialization conf
+conf.set("spark.serializer",
"org.apache.spark.serializer.KryoSerializer");
+conf.set("spark.kryo.registrator",
"org.apache.kylin.engine.spark.KylinKryoRegistrator");
+conf.set("spark.kryo.registrationRequired",
"true").registerKryoClasses(kryoClassArray);
+
+KylinSparkJobListener jobListener = new KylinSparkJobListener();
+try
[GitHub] [kylin] liuzx32 edited a comment on issue #657: adjust HBase tableName toUpperCase for HFile
liuzx32 edited a comment on issue #657: adjust HBase tableName toUpperCase for HFile URL: https://github.com/apache/kylin/pull/657#issuecomment-502587332 @nichunen Because HBase tableName is case sensitive. HBase tableName.toUpperCase() will different from the original table, leading to table can not be found. # And it's only for hbase, not for hive. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] liuzx32 commented on issue #657: adjust HBase tableName toUpperCase for HFile
liuzx32 commented on issue #657: adjust HBase tableName toUpperCase for HFile URL: https://github.com/apache/kylin/pull/657#issuecomment-502587332 @nichunen Because HBase tableName is case sensitive. HBase tableName.toUpperCase() will different from the original table, leading to table can not be found. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
nichunen commented on a change in pull request #629: KYLIN-3983 Add extra
metadata for measure. Add test case and example/…
URL: https://github.com/apache/kylin/pull/629#discussion_r282519903
##
File path: core-cube/src/main/java/org/apache/kylin/measure/MeasureManager.java
##
@@ -0,0 +1,698 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.measure;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import org.apache.kylin.common.KylinConfig;
+import org.apache.kylin.common.persistence.JsonSerializer;
+import org.apache.kylin.common.persistence.ResourceStore;
+import org.apache.kylin.common.persistence.Serializer;
+import org.apache.kylin.common.util.AutoReadWriteLock;
+import org.apache.kylin.cube.CubeInstance;
+import org.apache.kylin.cube.CubeManager;
+import org.apache.kylin.cube.CubeUpdate;
+import org.apache.kylin.cube.model.CubeDesc;
+import org.apache.kylin.metadata.MetadataConstants;
+import org.apache.kylin.metadata.cachesync.Broadcaster;
+import org.apache.kylin.metadata.cachesync.CachedCrudAssist;
+import org.apache.kylin.metadata.cachesync.CaseInsensitiveStringCache;
+import org.apache.kylin.metadata.model.ISegment;
+import org.apache.kylin.metadata.model.MeasureDesc;
+import org.apache.kylin.metadata.model.SegmentStatusEnum;
+import org.apache.kylin.metadata.project.ProjectInstance;
+import org.apache.kylin.metadata.project.ProjectManager;
+import org.apache.kylin.metadata.project.RealizationEntry;
+import org.apache.kylin.metadata.realization.RealizationType;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Locale;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import static org.apache.kylin.common.util.AutoReadWriteLock.AutoLock;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.CREATE;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.DROP;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.UPDATE;
+
+public class MeasureManager {
+
+public static final Serializer MEASURE_SERIALIZER = new
JsonSerializer<>(MeasureInstance.class);
+
+private static final Logger LOG =
LoggerFactory.getLogger(MeasureManager.class);
+public static final String MEASURE = "measure";
+public static final String CUBE = "cube";
+public static final String CUBE_DESC = "cube_desc";
+public static final String CUBE_MEASURE = "cube_measure";
+
+public static MeasureManager getInstance(KylinConfig config) {
+return config.getManager(MeasureManager.class);
+}
+
+// called by reflection
+static MeasureManager newInstance(KylinConfig config) throws IOException {
+return new MeasureManager(config);
+}
+
+// ===
+
+private KylinConfig config;
+
+// measure instance store path ==> MeasureInstance
+private CaseInsensitiveStringCache measureMap;
+private CubeL2Cache cubeL2Cache;
+
+private CachedCrudAssist crud;
+
+private AutoReadWriteLock measureMapLock = new AutoReadWriteLock();
+
+private MeasureManager(KylinConfig config) throws IOException {
+LOG.info("Initializing MeasureManager with config " + config);
+this.config = config;
+this.measureMap = new CaseInsensitiveStringCache<>(config, "measure");
+this.cubeL2Cache = new CubeL2Cache(getCubeManager());
+
+this.crud = new CachedCrudAssist(getStore(),
ResourceStore.MEASURE_RESOURCE_ROOT, MeasureInstance.class, measureMap) {
+@Override
+protected MeasureInstance initEntityAfterReload(MeasureInstance
entity, String resourceName) {
+CubeInstance cube =
getCubeManager().getCube(entity.getCubeName());
+entity.init(cube);
+
+return entity;
+}
+};
+
+// touch lower level metadata before registering my listener
+
[GitHub] [kylin] nichunen commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
nichunen commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/… URL: https://github.com/apache/kylin/pull/629#discussion_r282483148 ## File path: core-cube/src/main/java/org/apache/kylin/cube/CubeManager.java ## @@ -238,7 +239,7 @@ public CubeInstance createCube(String cubeName, String projectName, CubeDesc des ProjectManager.getInstance(config).moveRealizationToProject(RealizationType.CUBE, cubeName, projectName, owner); - +getMeasureManager().createMeasuresOnCube(cube); Review comment: why not put the logic of update measure in updateSegmentsOnCube() This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
nichunen commented on a change in pull request #629: KYLIN-3983 Add extra
metadata for measure. Add test case and example/…
URL: https://github.com/apache/kylin/pull/629#discussion_r282517673
##
File path: core-cube/src/main/java/org/apache/kylin/measure/MeasureManager.java
##
@@ -0,0 +1,698 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.measure;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import org.apache.kylin.common.KylinConfig;
+import org.apache.kylin.common.persistence.JsonSerializer;
+import org.apache.kylin.common.persistence.ResourceStore;
+import org.apache.kylin.common.persistence.Serializer;
+import org.apache.kylin.common.util.AutoReadWriteLock;
+import org.apache.kylin.cube.CubeInstance;
+import org.apache.kylin.cube.CubeManager;
+import org.apache.kylin.cube.CubeUpdate;
+import org.apache.kylin.cube.model.CubeDesc;
+import org.apache.kylin.metadata.MetadataConstants;
+import org.apache.kylin.metadata.cachesync.Broadcaster;
+import org.apache.kylin.metadata.cachesync.CachedCrudAssist;
+import org.apache.kylin.metadata.cachesync.CaseInsensitiveStringCache;
+import org.apache.kylin.metadata.model.ISegment;
+import org.apache.kylin.metadata.model.MeasureDesc;
+import org.apache.kylin.metadata.model.SegmentStatusEnum;
+import org.apache.kylin.metadata.project.ProjectInstance;
+import org.apache.kylin.metadata.project.ProjectManager;
+import org.apache.kylin.metadata.project.RealizationEntry;
+import org.apache.kylin.metadata.realization.RealizationType;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Locale;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import static org.apache.kylin.common.util.AutoReadWriteLock.AutoLock;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.CREATE;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.DROP;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.UPDATE;
+
+public class MeasureManager {
+
+public static final Serializer MEASURE_SERIALIZER = new
JsonSerializer<>(MeasureInstance.class);
+
+private static final Logger LOG =
LoggerFactory.getLogger(MeasureManager.class);
+public static final String MEASURE = "measure";
+public static final String CUBE = "cube";
+public static final String CUBE_DESC = "cube_desc";
+public static final String CUBE_MEASURE = "cube_measure";
Review comment:
what is "cube_measure" for?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] nichunen commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
nichunen commented on a change in pull request #629: KYLIN-3983 Add extra
metadata for measure. Add test case and example/…
URL: https://github.com/apache/kylin/pull/629#discussion_r282458112
##
File path:
core-common/src/main/java/org/apache/kylin/common/KylinConfigBase.java
##
@@ -2184,4 +2184,32 @@ public boolean isStreamingStandAloneMode() {
public String getLocalStorageImpl() {
return getOptional("kylin.stream.settled.storage", null);
}
+
+public String getHBaseColumnQualifierPrefix() {
Review comment:
Configs should be grouped, configs start with "kylin.metadata.*" should be
in one group
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] nichunen commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
nichunen commented on a change in pull request #629: KYLIN-3983 Add extra
metadata for measure. Add test case and example/…
URL: https://github.com/apache/kylin/pull/629#discussion_r282502592
##
File path: core-cube/src/main/java/org/apache/kylin/measure/CubeL2Cache.java
##
@@ -0,0 +1,188 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.measure;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+import org.apache.kylin.cube.CubeInstance;
+import org.apache.kylin.cube.CubeManager;
+import org.apache.kylin.metadata.model.ISegment;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentSkipListMap;
+import java.util.stream.Collectors;
+
+public class CubeL2Cache {
+
+private static final Logger LOG =
LoggerFactory.getLogger(CubeL2Cache.class);
+
+private CubeManager cubeMgr;
+private Map cubeCaches = new
ConcurrentSkipListMap<>(String.CASE_INSENSITIVE_ORDER);
+
+
+public CubeL2Cache(CubeManager cubeMgr) {
+this.cubeMgr = cubeMgr;
+}
+
+// cat
+public List getMeasuresOnCubeAllowMiss(String cubeName) {
+CubeCache cache = cubeCaches.get(cubeName);
+cache = cache == null ? reloadCacheAllowMiss(cubeName) : cache;
Review comment:
redundant code
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] nichunen merged pull request #635: KYLIN-3934 add config for sqoop config null-string and null-non-string
nichunen merged pull request #635: KYLIN-3934 add config for sqoop config null-string and null-non-string URL: https://github.com/apache/kylin/pull/635 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] Qsimple commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
Qsimple commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/… URL: https://github.com/apache/kylin/pull/629#discussion_r282714032 ## File path: core-cube/src/main/java/org/apache/kylin/cube/CubeManager.java ## @@ -238,7 +239,7 @@ public CubeInstance createCube(String cubeName, String projectName, CubeDesc des ProjectManager.getInstance(config).moveRealizationToProject(RealizationType.CUBE, cubeName, projectName, owner); - +getMeasureManager().createMeasuresOnCube(cube); Review comment: I don't understand this. why need to put "Create A" into "Update B". This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] codecov-io commented on issue #637: KYLIN-3998 Make "bpus-min-benefit-ratio" configurable in cube planner phase 1
codecov-io commented on issue #637: KYLIN-3998 Make "bpus-min-benefit-ratio" configurable in cube planner phase 1 URL: https://github.com/apache/kylin/pull/637#issuecomment-491114316 # [Codecov](https://codecov.io/gh/apache/kylin/pull/637?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@3f0ca84`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `0%`. [](https://codecov.io/gh/apache/kylin/pull/637?src=pr=tree) ```diff @@Coverage Diff@@ ## master #637 +/- ## = Coverage ? 25.37% Complexity? 5859 = Files ? 1379 Lines ?81871 Branches ?11479 = Hits ?20775 Misses?59073 Partials ? 2023 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/637?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../kylin/engine/mr/common/CuboidRecommenderUtil.java](https://codecov.io/gh/apache/kylin/pull/637/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvY29tbW9uL0N1Ym9pZFJlY29tbWVuZGVyVXRpbC5qYXZh) | `0% <0%> (ø)` | `0 <0> (?)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/637?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/637?src=pr=footer). Last update [3f0ca84...2f4841e](https://codecov.io/gh/apache/kylin/pull/637?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] Wayne1c opened a new pull request #637: KYLIN-3998 Make "bpus-min-benefit-ratio" configurable in cube planner phase 1
Wayne1c opened a new pull request #637: KYLIN-3998 Make "bpus-min-benefit-ratio" configurable in cube planner phase 1 URL: https://github.com/apache/kylin/pull/637 …hase 1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #637: KYLIN-3998 Make "bpus-min-benefit-ratio" configurable in cube planner phase 1
asfgit commented on issue #637: KYLIN-3998 Make "bpus-min-benefit-ratio" configurable in cube planner phase 1 URL: https://github.com/apache/kylin/pull/637#issuecomment-491110024 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] Qsimple commented on issue #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
Qsimple commented on issue #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/… URL: https://github.com/apache/kylin/pull/629#issuecomment-490970062 Yes, I agree with you. Another PR will be prepared to replace this one. Tks for you, chunen This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls edited a comment on issue #631: Add additional unit tests
coveralls edited a comment on issue #631: Add additional unit tests URL: https://github.com/apache/kylin/pull/631#issuecomment-490178792 ## Pull Request Test Coverage Report for [Build 4480](https://coveralls.io/builds/23288694) * **0** of **0** changed or added relevant lines in **0** files are covered. * **380** unchanged lines in **11** files lost coverage. * Overall coverage increased (+**0.09%**) to **27.955%** --- | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/steps/MergeStatisticsWithOldStep.java](https://coveralls.io/builds/23288694/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fsteps%2FMergeStatisticsWithOldStep.java#L123) | 1 | 0.0% | | [core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultScheduler.java](https://coveralls.io/builds/23288694/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultScheduler.java#L194) | 2 | 80.23% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/common/CubeStatsWriter.java](https://coveralls.io/builds/23288694/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fcommon%2FCubeStatsWriter.java#L48) | 10 | 62.07% | | [source-jdbc/src/main/java/org/apache/kylin/source/jdbc/JdbcHiveInputBase.java](https://coveralls.io/builds/23288694/source?filename=source-jdbc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fjdbc%2FJdbcHiveInputBase.java#L175) | 11 | 12.26% | | [core-metadata/src/main/java/org/apache/kylin/metadata/expression/CaseTupleExpression.java](https://coveralls.io/builds/23288694/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetadata%2Fexpression%2FCaseTupleExpression.java#L53) | 16 | 51.85% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/steps/MergeStatisticsStep.java](https://coveralls.io/builds/23288694/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fsteps%2FMergeStatisticsStep.java#L83) | 23 | 0.0% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/steps/MergeDictionaryMapper.java](https://coveralls.io/builds/23288694/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fsteps%2FMergeDictionaryMapper.java#L136) | 30 | 0.0% | | [engine-mr/src/main/java/org/apache/kylin/engine/mr/common/CubeStatsReader.java](https://coveralls.io/builds/23288694/source?filename=engine-mr%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fmr%2Fcommon%2FCubeStatsReader.java#L169) | 52 | 0.0% | | [engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkMergingDictionary.java](https://coveralls.io/builds/23288694/source?filename=engine-spark%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fengine%2Fspark%2FSparkMergingDictionary.java#L92) | 59 | 0.0% | | [core-metadata/src/main/java/org/apache/kylin/metadata/filter/CompareTupleFilter.java](https://coveralls.io/builds/23288694/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetadata%2Ffilter%2FCompareTupleFilter.java#L64) | 62 | 33.13% | | Totals | [](https://coveralls.io/builds/23288694) | | :-- | --: | | Change from base [Build 4453](https://coveralls.io/builds/23203528): | 0.09% | | Covered Lines: | 22887 | | Relevant Lines: | 81871 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen merged pull request #637: KYLIN-3998 Make "bpus-min-benefit-ratio" configurable in cube planner phase 1
nichunen merged pull request #637: KYLIN-3998 Make "bpus-min-benefit-ratio" configurable in cube planner phase 1 URL: https://github.com/apache/kylin/pull/637 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #636: Modify the display format of 'Get job list' api
asfgit commented on issue #636: Modify the display format of 'Get job list' api URL: https://github.com/apache/kylin/pull/636#issuecomment-490797168 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] GinaZhai opened a new pull request #636: Modify the display format of 'Get job list' api
GinaZhai opened a new pull request #636: Modify the display format of 'Get job list' api URL: https://github.com/apache/kylin/pull/636 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] codecov-io edited a comment on issue #624: KYLIN-4001 Allow user-specified time format using real-time
codecov-io edited a comment on issue #624: KYLIN-4001 Allow user-specified time format using real-time URL: https://github.com/apache/kylin/pull/624#issuecomment-487864284 # [Codecov](https://codecov.io/gh/apache/kylin/pull/624?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@d16bdd3`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `34.69%`. [](https://codecov.io/gh/apache/kylin/pull/624?src=pr=tree) ```diff @@Coverage Diff@@ ## master #624 +/- ## = Coverage ? 25.38% Complexity? 5864 = Files ? 1382 Lines ?81918 Branches ?11481 = Hits ?20795 Misses?59099 Partials ? 2024 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/624?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...ache/kylin/stream/source/kafka/DateTimeParser.java](https://codecov.io/gh/apache/kylin/pull/624/diff?src=pr=tree#diff-c3RyZWFtLXNvdXJjZS1rYWZrYS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RyZWFtL3NvdXJjZS9rYWZrYS9EYXRlVGltZVBhcnNlci5qYXZh) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...he/kylin/stream/core/source/MessageParserInfo.java](https://codecov.io/gh/apache/kylin/pull/624/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3NvdXJjZS9NZXNzYWdlUGFyc2VySW5mby5qYXZh) | `0% <0%> (ø)` | `0 <0> (?)` | | | [.../kylin/stream/source/kafka/AbstractTimeParser.java](https://codecov.io/gh/apache/kylin/pull/624/diff?src=pr=tree#diff-c3RyZWFtLXNvdXJjZS1rYWZrYS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RyZWFtL3NvdXJjZS9rYWZrYS9BYnN0cmFjdFRpbWVQYXJzZXIuamF2YQ==) | `100% <100%> (ø)` | `1 <1> (?)` | | | [...lin/stream/source/kafka/TimedJsonStreamParser.java](https://codecov.io/gh/apache/kylin/pull/624/diff?src=pr=tree#diff-c3RyZWFtLXNvdXJjZS1rYWZrYS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RyZWFtL3NvdXJjZS9rYWZrYS9UaW1lZEpzb25TdHJlYW1QYXJzZXIuamF2YQ==) | `71.76% <50%> (ø)` | `12 <0> (?)` | | | [...ache/kylin/stream/source/kafka/LongTimeParser.java](https://codecov.io/gh/apache/kylin/pull/624/diff?src=pr=tree#diff-c3RyZWFtLXNvdXJjZS1rYWZrYS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RyZWFtL3NvdXJjZS9rYWZrYS9Mb25nVGltZVBhcnNlci5qYXZh) | `57.14% <57.14%> (ø)` | `3 <3> (?)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/624?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/624?src=pr=footer). Last update [d16bdd3...b186aeb](https://codecov.io/gh/apache/kylin/pull/624?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] codecov-io edited a comment on issue #624: KYLIN-4001 Allow user-specified time format using real-time
codecov-io edited a comment on issue #624: KYLIN-4001 Allow user-specified time format using real-time URL: https://github.com/apache/kylin/pull/624#issuecomment-487864284 # [Codecov](https://codecov.io/gh/apache/kylin/pull/624?src=pr=h1) Report > Merging [#624](https://codecov.io/gh/apache/kylin/pull/624?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/d16bdd3a546ce7c37602471e1fdac1a3ad4e8be9?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `34.69%`. [](https://codecov.io/gh/apache/kylin/pull/624?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #624 +/- ## + Coverage 25.37% 25.37% +<.01% - Complexity 5859 5863 +4 Files 1379 1382 +3 Lines 8187181918 +47 Branches 1147911481 +2 + Hits 2077220788 +16 - Misses5907559104 +29 - Partials 2024 2026 +2 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/624?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...ache/kylin/stream/source/kafka/DateTimeParser.java](https://codecov.io/gh/apache/kylin/pull/624/diff?src=pr=tree#diff-c3RyZWFtLXNvdXJjZS1rYWZrYS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RyZWFtL3NvdXJjZS9rYWZrYS9EYXRlVGltZVBhcnNlci5qYXZh) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...he/kylin/stream/core/source/MessageParserInfo.java](https://codecov.io/gh/apache/kylin/pull/624/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3NvdXJjZS9NZXNzYWdlUGFyc2VySW5mby5qYXZh) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [.../kylin/stream/source/kafka/AbstractTimeParser.java](https://codecov.io/gh/apache/kylin/pull/624/diff?src=pr=tree#diff-c3RyZWFtLXNvdXJjZS1rYWZrYS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RyZWFtL3NvdXJjZS9rYWZrYS9BYnN0cmFjdFRpbWVQYXJzZXIuamF2YQ==) | `100% <100%> (ø)` | `1 <1> (?)` | | | [...lin/stream/source/kafka/TimedJsonStreamParser.java](https://codecov.io/gh/apache/kylin/pull/624/diff?src=pr=tree#diff-c3RyZWFtLXNvdXJjZS1rYWZrYS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RyZWFtL3NvdXJjZS9rYWZrYS9UaW1lZEpzb25TdHJlYW1QYXJzZXIuamF2YQ==) | `71.76% <50%> (-4.95%)` | `12 <0> (ø)` | | | [...ache/kylin/stream/source/kafka/LongTimeParser.java](https://codecov.io/gh/apache/kylin/pull/624/diff?src=pr=tree#diff-c3RyZWFtLXNvdXJjZS1rYWZrYS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc3RyZWFtL3NvdXJjZS9rYWZrYS9Mb25nVGltZVBhcnNlci5qYXZh) | `57.14% <57.14%> (ø)` | `3 <3> (?)` | | | [...he/kylin/job/impl/threadpool/DefaultScheduler.java](https://codecov.io/gh/apache/kylin/pull/624/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL3RocmVhZHBvb2wvRGVmYXVsdFNjaGVkdWxlci5qYXZh) | `74.41% <0%> (-2.33%)` | `12% <0%> (ø)` | | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/624/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `70.21% <0%> (+0.91%)` | `7% <0%> (ø)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/624?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/624?src=pr=footer). Last update [d16bdd3...1612287](https://codecov.io/gh/apache/kylin/pull/624?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] hit-lacus commented on a change in pull request #624: KYLIN-4001 Allow user-specified time format using real-time
hit-lacus commented on a change in pull request #624: KYLIN-4001 Allow
user-specified time format using real-time
URL: https://github.com/apache/kylin/pull/624#discussion_r283117876
##
File path:
stream-source-kafka/src/main/java/org/apache/kylin/stream/source/kafka/TimedJsonStreamParser.java
##
@@ -88,6 +91,21 @@ public TimedJsonStreamParser(List cols,
MessageParserInfo parserInfo)
}
logger.info("Using parser field mapping by {}",
parserInfo.getColumnToSourceFieldMapping());
}
+this.tsParser = parserInfo.getTsParser();
+
+if (!StringUtils.isEmpty(tsParser)) {
+try {
+Class clazz = Class.forName(tsParser);
+Constructor constructor =
clazz.getConstructor(MessageParserInfo.class);
+streamTimeParser = (AbstractTimeParser)
constructor.newInstance(parserInfo);
+} catch (Exception e) {
+throw new IllegalStateException("Invalid StreamingConfig,
tsParser " + tsParser + ", tsPattern " + parserInfo.getTsPattern() + ".", e);
+}
+}else{
Review comment:
code not formated
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] coveralls edited a comment on issue #624: KYLIN-4001 Allow user-specified time format using real-time
coveralls edited a comment on issue #624: KYLIN-4001 Allow user-specified time format using real-time URL: https://github.com/apache/kylin/pull/624#issuecomment-487863142 ## Pull Request Test Coverage Report for [Build 4486](https://coveralls.io/builds/23331565) * **20** of **49** **(40.82%)** changed or added relevant lines in **5** files are covered. * **5** unchanged lines in **2** files lost coverage. * Overall coverage increased (+**0.006%**) to **27.855%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [stream-source-kafka/src/main/java/org/apache/kylin/stream/source/kafka/LongTimeParser.java](https://coveralls.io/builds/23331565/source?filename=stream-source-kafka%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fsource%2Fkafka%2FLongTimeParser.java#L50) | 10 | 14 | 71.43% | [stream-core/src/main/java/org/apache/kylin/stream/core/source/MessageParserInfo.java](https://coveralls.io/builds/23331565/source?filename=stream-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fcore%2Fsource%2FMessageParserInfo.java#L52) | 0 | 6 | 0.0% | [stream-source-kafka/src/main/java/org/apache/kylin/stream/source/kafka/TimedJsonStreamParser.java](https://coveralls.io/builds/23331565/source?filename=stream-source-kafka%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fsource%2Fkafka%2FTimedJsonStreamParser.java#L98) | 8 | 14 | 57.14% | [stream-source-kafka/src/main/java/org/apache/kylin/stream/source/kafka/DateTimeParser.java](https://coveralls.io/builds/23331565/source?filename=stream-source-kafka%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fstream%2Fsource%2Fkafka%2FDateTimeParser.java#L32) | 0 | 13 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-job/src/main/java/org/apache/kylin/job/impl/threadpool/DefaultScheduler.java](https://coveralls.io/builds/23331565/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fthreadpool%2FDefaultScheduler.java#L194) | 2 | 80.23% | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/23331565/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L449) | 3 | 78.42% | | Totals | [](https://coveralls.io/builds/23331565) | | :-- | --: | | Change from base [Build 4485](https://coveralls.io/builds/23331346): | 0.006% | | Covered Lines: | 22818 | | Relevant Lines: | 81918 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
nichunen commented on a change in pull request #629: KYLIN-3983 Add extra
metadata for measure. Add test case and example/…
URL: https://github.com/apache/kylin/pull/629#discussion_r283126065
##
File path: core-cube/src/main/java/org/apache/kylin/measure/MeasureManager.java
##
@@ -0,0 +1,698 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.measure;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import org.apache.kylin.common.KylinConfig;
+import org.apache.kylin.common.persistence.JsonSerializer;
+import org.apache.kylin.common.persistence.ResourceStore;
+import org.apache.kylin.common.persistence.Serializer;
+import org.apache.kylin.common.util.AutoReadWriteLock;
+import org.apache.kylin.cube.CubeInstance;
+import org.apache.kylin.cube.CubeManager;
+import org.apache.kylin.cube.CubeUpdate;
+import org.apache.kylin.cube.model.CubeDesc;
+import org.apache.kylin.metadata.MetadataConstants;
+import org.apache.kylin.metadata.cachesync.Broadcaster;
+import org.apache.kylin.metadata.cachesync.CachedCrudAssist;
+import org.apache.kylin.metadata.cachesync.CaseInsensitiveStringCache;
+import org.apache.kylin.metadata.model.ISegment;
+import org.apache.kylin.metadata.model.MeasureDesc;
+import org.apache.kylin.metadata.model.SegmentStatusEnum;
+import org.apache.kylin.metadata.project.ProjectInstance;
+import org.apache.kylin.metadata.project.ProjectManager;
+import org.apache.kylin.metadata.project.RealizationEntry;
+import org.apache.kylin.metadata.realization.RealizationType;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Locale;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import static org.apache.kylin.common.util.AutoReadWriteLock.AutoLock;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.CREATE;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.DROP;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.UPDATE;
+
+public class MeasureManager {
+
+public static final Serializer MEASURE_SERIALIZER = new
JsonSerializer<>(MeasureInstance.class);
+
+private static final Logger LOG =
LoggerFactory.getLogger(MeasureManager.class);
+public static final String MEASURE = "measure";
+public static final String CUBE = "cube";
+public static final String CUBE_DESC = "cube_desc";
+public static final String CUBE_MEASURE = "cube_measure";
Review comment:
> When Listener receive "cube_measure", MeasureManager will batch-process
the MeasureInstances on the cube
I found you only add "/measure" as metadata in
org.apache.kylin.common.persistence.ResourceStore, but no "cube_measure"
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] nichunen merged pull request #631: Add additional unit tests
nichunen merged pull request #631: Add additional unit tests URL: https://github.com/apache/kylin/pull/631 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] Qsimple commented on a change in pull request #629: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
Qsimple commented on a change in pull request #629: KYLIN-3983 Add extra
metadata for measure. Add test case and example/…
URL: https://github.com/apache/kylin/pull/629#discussion_r283127561
##
File path: core-cube/src/main/java/org/apache/kylin/measure/MeasureManager.java
##
@@ -0,0 +1,698 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.measure;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import org.apache.kylin.common.KylinConfig;
+import org.apache.kylin.common.persistence.JsonSerializer;
+import org.apache.kylin.common.persistence.ResourceStore;
+import org.apache.kylin.common.persistence.Serializer;
+import org.apache.kylin.common.util.AutoReadWriteLock;
+import org.apache.kylin.cube.CubeInstance;
+import org.apache.kylin.cube.CubeManager;
+import org.apache.kylin.cube.CubeUpdate;
+import org.apache.kylin.cube.model.CubeDesc;
+import org.apache.kylin.metadata.MetadataConstants;
+import org.apache.kylin.metadata.cachesync.Broadcaster;
+import org.apache.kylin.metadata.cachesync.CachedCrudAssist;
+import org.apache.kylin.metadata.cachesync.CaseInsensitiveStringCache;
+import org.apache.kylin.metadata.model.ISegment;
+import org.apache.kylin.metadata.model.MeasureDesc;
+import org.apache.kylin.metadata.model.SegmentStatusEnum;
+import org.apache.kylin.metadata.project.ProjectInstance;
+import org.apache.kylin.metadata.project.ProjectManager;
+import org.apache.kylin.metadata.project.RealizationEntry;
+import org.apache.kylin.metadata.realization.RealizationType;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Locale;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import static org.apache.kylin.common.util.AutoReadWriteLock.AutoLock;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.CREATE;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.DROP;
+import static org.apache.kylin.metadata.cachesync.Broadcaster.Event.UPDATE;
+
+public class MeasureManager {
+
+public static final Serializer MEASURE_SERIALIZER = new
JsonSerializer<>(MeasureInstance.class);
+
+private static final Logger LOG =
LoggerFactory.getLogger(MeasureManager.class);
+public static final String MEASURE = "measure";
+public static final String CUBE = "cube";
+public static final String CUBE_DESC = "cube_desc";
+public static final String CUBE_MEASURE = "cube_measure";
Review comment:
Yes，cube_measure means the Measure Instances on same cube have been updated
and needed to be reloaded. The most methods in MeasureManager will batch
process on Measure instances on same cube and will publish an cube_measure
entitie's events with cube's name. MeasureManager can also modify single
Measure instance and trigger a 'measure' event, which is seldomly called. I ll
remove cube and cube_dese which doesn't matter any more
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] coveralls commented on issue #639: Add unit tests three
coveralls commented on issue #639: Add unit tests three URL: https://github.com/apache/kylin/pull/639#issuecomment-491921501 ## Pull Request Test Coverage Report for [Build 4490](https://coveralls.io/builds/23356161) * **0** of **0** changed or added relevant lines in **0** files are covered. * **6** unchanged lines in **2** files lost coverage. * Overall coverage increased (+**0.03%**) to **27.984%** --- | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/23356161/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62) | 1 | 81.08% | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/23356161/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L439) | 5 | 77.81% | | Totals | [](https://coveralls.io/builds/23356161) | | :-- | --: | | Change from base [Build 4487](https://coveralls.io/builds/23340471): | 0.03% | | Covered Lines: | 22911 | | Relevant Lines: | 81871 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] codecov-io commented on issue #639: Add unit tests three
codecov-io commented on issue #639: Add unit tests three URL: https://github.com/apache/kylin/pull/639#issuecomment-491922284 # [Codecov](https://codecov.io/gh/apache/kylin/pull/639?src=pr=h1) Report > Merging [#639](https://codecov.io/gh/apache/kylin/pull/639?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/c39b238fb2f4999f5484a7d2fa0b55245db5534c?src=pr=desc) will **increase** coverage by `0.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/kylin/pull/639?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#639 +/- ## === + Coverage 25.48% 25.5% +0.01% - Complexity 58925900 +8 === Files 13791379 Lines 81871 81871 Branches 11479 11479 === + Hits 20863 20878 +15 + Misses58986 58964 -22 - Partials 20222029 +7 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/639?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/639/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `72.97% <0%> (-5.41%)` | `6% <0%> (-1%)` | | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/639/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `69.3% <0%> (-0.92%)` | `7% <0%> (ø)` | | | [...apache/kylin/engine/mr/common/HadoopCmdOutput.java](https://codecov.io/gh/apache/kylin/pull/639/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvY29tbW9uL0hhZG9vcENtZE91dHB1dC5qYXZh) | `14.7% <0%> (+14.7%)` | `4% <0%> (+4%)` | :arrow_up: | | [...ylin/engine/mr/common/DefaultX509TrustManager.java](https://codecov.io/gh/apache/kylin/pull/639/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvY29tbW9uL0RlZmF1bHRYNTA5VHJ1c3RNYW5hZ2VyLmphdmE=) | `35.71% <0%> (+35.71%)` | `5% <0%> (+5%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/639?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/639?src=pr=footer). Last update [c39b238...4c5f7f1](https://codecov.io/gh/apache/kylin/pull/639?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #639: Add unit tests three
asfgit commented on issue #639: Add unit tests three URL: https://github.com/apache/kylin/pull/639#issuecomment-491899719 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] TheRealHaui opened a new pull request #639: Add unit tests three
TheRealHaui opened a new pull request #639: Add unit tests three URL: https://github.com/apache/kylin/pull/639 Added additional Unit Tests to increase code coverage. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen merged pull request #638: KYLIN-3983 Add extra metadata for measure. Add test case and example/…
nichunen merged pull request #638: KYLIN-3983 Add extra metadata for measure. Add test case and example/… URL: https://github.com/apache/kylin/pull/638 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen merged pull request #639: Add unit tests three
nichunen merged pull request #639: Add unit tests three URL: https://github.com/apache/kylin/pull/639 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] codecov-io commented on issue #640: Add unit tests four
codecov-io commented on issue #640: Add unit tests four URL: https://github.com/apache/kylin/pull/640#issuecomment-492119779 # [Codecov](https://codecov.io/gh/apache/kylin/pull/640?src=pr=h1) Report > Merging [#640](https://codecov.io/gh/apache/kylin/pull/640?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/20fa8b1530e0d09f4393242db282dff43b930b00?src=pr=desc) will **increase** coverage by `0.05%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/kylin/pull/640?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #640 +/- ## + Coverage 25.48% 25.53% +0.05% - Complexity 5892 5919 +27 Files 1379 1379 Lines 8187181871 Branches 1147911479 + Hits 2086620908 +42 + Misses5898458932 -52 - Partials 2021 2031 +10 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/640?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/640/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `72.97% <0%> (-5.41%)` | `6% <0%> (-1%)` | | | [...ream/core/storage/columnar/ColumnarStoreCache.java](https://codecov.io/gh/apache/kylin/pull/640/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3N0b3JhZ2UvY29sdW1uYXIvQ29sdW1uYXJTdG9yZUNhY2hlLmphdmE=) | `48.83% <0%> (-3.49%)` | `7% <0%> (ø)` | | | [.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/640/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=) | `63.84% <0%> (-2.31%)` | `0% <0%> (ø)` | | | [...he/kylin/job/impl/threadpool/DefaultScheduler.java](https://codecov.io/gh/apache/kylin/pull/640/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL3RocmVhZHBvb2wvRGVmYXVsdFNjaGVkdWxlci5qYXZh) | `76.74% <0%> (+2.32%)` | `12% <0%> (ø)` | :arrow_down: | | [...pache/kylin/engine/mr/common/JobInfoConverter.java](https://codecov.io/gh/apache/kylin/pull/640/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvY29tbW9uL0pvYkluZm9Db252ZXJ0ZXIuamF2YQ==) | `29.13% <0%> (+13.38%)` | `21% <0%> (+14%)` | :arrow_up: | | [...apache/kylin/engine/mr/common/HadoopCmdOutput.java](https://codecov.io/gh/apache/kylin/pull/640/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvY29tbW9uL0hhZG9vcENtZE91dHB1dC5qYXZh) | `14.7% <0%> (+14.7%)` | `4% <0%> (+4%)` | :arrow_up: | | [...n/engine/mr/steps/lookup/LookupExecutableUtil.java](https://codecov.io/gh/apache/kylin/pull/640/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvc3RlcHMvbG9va3VwL0xvb2t1cEV4ZWN1dGFibGVVdGlsLmphdmE=) | `24.44% <0%> (+24.44%)` | `5% <0%> (+5%)` | :arrow_up: | | [...ylin/engine/mr/common/DefaultX509TrustManager.java](https://codecov.io/gh/apache/kylin/pull/640/diff?src=pr=tree#diff-ZW5naW5lLW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9lbmdpbmUvbXIvY29tbW9uL0RlZmF1bHRYNTA5VHJ1c3RNYW5hZ2VyLmphdmE=) | `35.71% <0%> (+35.71%)` | `5% <0%> (+5%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/640?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/640?src=pr=footer). Last update [20fa8b1...2fb6db4](https://codecov.io/gh/apache/kylin/pull/640?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] TheRealHaui opened a new pull request #640: Add unit tests four
TheRealHaui opened a new pull request #640: Add unit tests four URL: https://github.com/apache/kylin/pull/640 Added Unit Tests to improve code coverage. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] TempleZhou opened a new pull request #641: KYLIN-3978 InternalErrorException: null with precise count distinct
TempleZhou opened a new pull request #641: KYLIN-3978 InternalErrorException: null with precise count distinct URL: https://github.com/apache/kylin/pull/641 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #640: Add unit tests four
asfgit commented on issue #640: Add unit tests four URL: https://github.com/apache/kylin/pull/640#issuecomment-492110321 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #641: KYLIN-3978 InternalErrorException: null with precise count distinct
asfgit commented on issue #641: KYLIN-3978 InternalErrorException: null with precise count distinct URL: https://github.com/apache/kylin/pull/641#issuecomment-492110333 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen commented on issue #612: KYLIN-3961 Optimize TopNCounter's merge function to reduce TopNCounter's error size.
nichunen commented on issue #612: KYLIN-3961 Optimize TopNCounter's merge function to reduce TopNCounter's error size. URL: https://github.com/apache/kylin/pull/612#issuecomment-492491490 Hi @zhaojintaozhao I agree that your pr will increase the accuracy of the counter number in some cases, but I'm afraid it will also break the results' oder. Look at the ut I've added in your pr. The counter with item "DD" is supposed to be the top 1 after two times of merging, but with the code change, it will be lost This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] nichunen merged pull request #641: KYLIN-3978 InternalErrorException: null with precise count distinct
nichunen merged pull request #641: KYLIN-3978 InternalErrorException: null with precise count distinct URL: https://github.com/apache/kylin/pull/641 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] codecov-io edited a comment on issue #645: Add unit tests five
codecov-io edited a comment on issue #645: Add unit tests five URL: https://github.com/apache/kylin/pull/645#issuecomment-493550575 # [Codecov](https://codecov.io/gh/apache/kylin/pull/645?src=pr=h1) Report > Merging [#645](https://codecov.io/gh/apache/kylin/pull/645?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/876d420812a09270455bfbb78204010c81cab612?src=pr=desc) will **increase** coverage by `0.09%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/kylin/pull/645?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #645 +/- ## + Coverage 25.67% 25.77% +0.09% - Complexity 5959 5990 +31 Files 1382 1382 Lines 8214482144 Branches 1150911509 + Hits 2109021169 +79 + Misses5901158925 -86 - Partials 2043 2050 +7 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/645?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/645/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=) | `63.84% <0%> (-2.31%)` | `0% <0%> (ø)` | | | [...pache/kylin/job/impl/curator/CuratorScheduler.java](https://codecov.io/gh/apache/kylin/pull/645/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL2N1cmF0b3IvQ3VyYXRvclNjaGVkdWxlci5qYXZh) | `64.4% <0%> (-0.85%)` | `7% <0%> (ø)` | | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/645/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `69.6% <0%> (-0.61%)` | `7% <0%> (ø)` | | | [...rg/apache/kylin/storage/hybrid/HybridInstance.java](https://codecov.io/gh/apache/kylin/pull/645/diff?src=pr=tree#diff-Y29yZS1zdG9yYWdlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdG9yYWdlL2h5YnJpZC9IeWJyaWRJbnN0YW5jZS5qYXZh) | `44.27% <0%> (+6.1%)` | `17% <0%> (+4%)` | :arrow_up: | | [.../java/org/apache/kylin/storage/StorageContext.java](https://codecov.io/gh/apache/kylin/pull/645/diff?src=pr=tree#diff-Y29yZS1zdG9yYWdlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdG9yYWdlL1N0b3JhZ2VDb250ZXh0LmphdmE=) | `30.1% <0%> (+30.1%)` | `10% <0%> (+10%)` | :arrow_up: | | [...kylin/storage/translate/FuzzyValueCombination.java](https://codecov.io/gh/apache/kylin/pull/645/diff?src=pr=tree#diff-Y29yZS1zdG9yYWdlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdG9yYWdlL3RyYW5zbGF0ZS9GdXp6eVZhbHVlQ29tYmluYXRpb24uamF2YQ==) | `94.23% <0%> (+94.23%)` | `17% <0%> (+17%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/645?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/645?src=pr=footer). Last update [876d420...2a42d10](https://codecov.io/gh/apache/kylin/pull/645?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls commented on issue #645: Add unit tests five
coveralls commented on issue #645: Add unit tests five URL: https://github.com/apache/kylin/pull/645#issuecomment-493675333 ## Pull Request Test Coverage Report for [Build 4510](https://coveralls.io/builds/23468162) * **0** of **0** changed or added relevant lines in **0** files are covered. * **4** unchanged lines in **3** files lost coverage. * Overall coverage increased (+**0.1%**) to **28.271%** --- | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/23468162/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L553) | 1 | 78.12% | | [core-job/src/main/java/org/apache/kylin/job/impl/curator/CuratorScheduler.java](https://coveralls.io/builds/23468162/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fcurator%2FCuratorScheduler.java#L145) | 1 | 68.64% | | [core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/23468162/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124) | 2 | 68.46% | | Totals | [](https://coveralls.io/builds/23468162) | | :-- | --: | | Change from base [Build 4509](https://coveralls.io/builds/23465482): | 0.1% | | Covered Lines: | 23223 | | Relevant Lines: | 82144 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] coveralls edited a comment on issue #645: Add unit tests five
coveralls edited a comment on issue #645: Add unit tests five URL: https://github.com/apache/kylin/pull/645#issuecomment-493675333 ## Pull Request Test Coverage Report for [Build 4511](https://coveralls.io/builds/23468323) * **0** of **0** changed or added relevant lines in **0** files are covered. * **8** unchanged lines in **3** files lost coverage. * Overall coverage increased (+**0.1%**) to **28.271%** --- | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-job/src/main/java/org/apache/kylin/job/impl/curator/CuratorScheduler.java](https://coveralls.io/builds/23468323/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fcurator%2FCuratorScheduler.java#L145) | 1 | 68.64% | | [core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/23468323/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124) | 2 | 68.46% | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/23468323/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L439) | 5 | 77.81% | | Totals | [](https://coveralls.io/builds/23468323) | | :-- | --: | | Change from base [Build 4509](https://coveralls.io/builds/23465482): | 0.1% | | Covered Lines: | 23223 | | Relevant Lines: | 82144 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] asfgit commented on issue #653: minor, switch to openjdk8 for travis
asfgit commented on issue #653: minor, switch to openjdk8 for travis URL: https://github.com/apache/kylin/pull/653#issuecomment-494814406 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [kylin] shaofengshi opened a new pull request #653: minor, switch to openjdk8 for travis
shaofengshi opened a new pull request #653: minor, switch to openjdk8 for travis URL: https://github.com/apache/kylin/pull/653 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
