[jira] [Commented] (KYLIN-4125) Kylin upgraded from springmvc architecture to spring boot architecture
[ https://issues.apache.org/jira/browse/KYLIN-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900678#comment-16900678 ] zhao jintao commented on KYLIN-4125: Hi Team: Next, I will do these things; # Modify the project's pom.xml to import the spring boot dependency. # Upgrade kylin-server from spring mvc to springboot 1.5.5; modify kylin-server code to support spring boot. # Modify the build&package script; package kylin into a war package and run it as external tomcat mode. It's the same as current deployment. I have implemented this part of the functionality and conducted a simple test, expecting to submit it to the community this week. > Kylin upgraded from springmvc architecture to spring boot architecture > -- > > Key: KYLIN-4125 > URL: https://issues.apache.org/jira/browse/KYLIN-4125 > Project: Kylin > Issue Type: Improvement > Components: REST Service >Reporter: zhao jintao >Assignee: zhao jintao >Priority: Minor > > Hi Team: > Kylin is based on the spring mvc architecture, but the spring mvc > configuration is more complicated. It is cumbersome when integrateing new > components. > Now， The mainstream of the industry has been based on the spring boot > architecture. Spring boot can be automatically configured to reduce the > complexity of project integration; promote the expansion and implementation > of microservice architecture. More and more project architectures have been > upgraded from springmvc to spring boot. > Kylin can also be upgraded from the springmvc architecture to the spring boot > architecture. > Do you have any suggestions? -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (KYLIN-4125) Kylin upgraded from springmvc architecture to spring boot architecture
[ https://issues.apache.org/jira/browse/KYLIN-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900666#comment-16900666 ] zhao jintao commented on KYLIN-4125: Hi Shaofeng: I will submit my code about kylin using the springboot architecture to the "KYLIN-4125" branch; and will discuss the issue of using springboot architecture at [d...@kylin.apache.org.|mailto:d...@kylin.apache.org.] Thank you. > Kylin upgraded from springmvc architecture to spring boot architecture > -- > > Key: KYLIN-4125 > URL: https://issues.apache.org/jira/browse/KYLIN-4125 > Project: Kylin > Issue Type: Improvement > Components: REST Service >Reporter: zhao jintao >Assignee: zhao jintao >Priority: Minor > > Hi Team: > Kylin is based on the spring mvc architecture, but the spring mvc > configuration is more complicated. It is cumbersome when integrateing new > components. > Now， The mainstream of the industry has been based on the spring boot > architecture. Spring boot can be automatically configured to reduce the > complexity of project integration; promote the expansion and implementation > of microservice architecture. More and more project architectures have been > upgraded from springmvc to spring boot. > Kylin can also be upgraded from the springmvc architecture to the spring boot > architecture. > Do you have any suggestions? -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (KYLIN-4125) Kylin upgraded from springmvc architecture to spring boot architecture
[ https://issues.apache.org/jira/browse/KYLIN-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900630#comment-16900630 ] Shaofeng SHI commented on KYLIN-4125: - Please also raise a discussion in the d...@kylin.apache.org mailing list. I created a branch to stage the code changes: "KYLIN-4125" > Kylin upgraded from springmvc architecture to spring boot architecture > -- > > Key: KYLIN-4125 > URL: https://issues.apache.org/jira/browse/KYLIN-4125 > Project: Kylin > Issue Type: Improvement > Components: REST Service >Reporter: zhao jintao >Assignee: zhao jintao >Priority: Minor > > Hi Team: > Kylin is based on the spring mvc architecture, but the spring mvc > configuration is more complicated. It is cumbersome when integrateing new > components. > Now， The mainstream of the industry has been based on the spring boot > architecture. Spring boot can be automatically configured to reduce the > complexity of project integration; promote the expansion and implementation > of microservice architecture. More and more project architectures have been > upgraded from springmvc to spring boot. > Kylin can also be upgraded from the springmvc architecture to the spring boot > architecture. > Do you have any suggestions? -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (KYLIN-4125) Kylin upgraded from springmvc architecture to spring boot architecture
[ https://issues.apache.org/jira/browse/KYLIN-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900618#comment-16900618 ] Shaofeng SHI commented on KYLIN-4125: - Jintao, we can create a separate branch (fork from master), and then you can raise PR to that branch, just like the "flink_engine" we have today. Does that work for you? Thanks! > Kylin upgraded from springmvc architecture to spring boot architecture > -- > > Key: KYLIN-4125 > URL: https://issues.apache.org/jira/browse/KYLIN-4125 > Project: Kylin > Issue Type: Improvement > Components: REST Service >Reporter: zhao jintao >Assignee: zhao jintao >Priority: Minor > > Hi Team: > Kylin is based on the spring mvc architecture, but the spring mvc > configuration is more complicated. It is cumbersome when integrateing new > components. > Now， The mainstream of the industry has been based on the spring boot > architecture. Spring boot can be automatically configured to reduce the > complexity of project integration; promote the expansion and implementation > of microservice architecture. More and more project architectures have been > upgraded from springmvc to spring boot. > Kylin can also be upgraded from the springmvc architecture to the spring boot > architecture. > Do you have any suggestions? -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (KYLIN-4125) Kylin upgraded from springmvc architecture to spring boot architecture
[ https://issues.apache.org/jira/browse/KYLIN-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900593#comment-16900593 ] zhao jintao commented on KYLIN-4125: I implemented kylin upgrade from spring mvc4 to springboot1.x.kylin can build and query normally;but the upgrade framework requires more people to participate and verify in the community. Need I create a new branch to submit the code? or submitted directly to the master branch. > Kylin upgraded from springmvc architecture to spring boot architecture > -- > > Key: KYLIN-4125 > URL: https://issues.apache.org/jira/browse/KYLIN-4125 > Project: Kylin > Issue Type: Improvement > Components: REST Service >Reporter: zhao jintao >Assignee: zhao jintao >Priority: Minor > > Hi Team: > Kylin is based on the spring mvc architecture, but the spring mvc > configuration is more complicated. It is cumbersome when integrateing new > components. > Now， The mainstream of the industry has been based on the spring boot > architecture. Spring boot can be automatically configured to reduce the > complexity of project integration; promote the expansion and implementation > of microservice architecture. More and more project architectures have been > upgraded from springmvc to spring boot. > Kylin can also be upgraded from the springmvc architecture to the spring boot > architecture. > Do you have any suggestions? -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (KYLIN-4125) Kylin upgraded from springmvc architecture to spring boot architecture
zhao jintao created KYLIN-4125: -- Summary: Kylin upgraded from springmvc architecture to spring boot architecture Key: KYLIN-4125 URL: https://issues.apache.org/jira/browse/KYLIN-4125 Project: Kylin Issue Type: Improvement Components: REST Service Reporter: zhao jintao Assignee: zhao jintao Hi Team: Kylin is based on the spring mvc architecture, but the spring mvc configuration is more complicated. It is cumbersome when integrateing new components. Now， The mainstream of the industry has been based on the spring boot architecture. Spring boot can be automatically configured to reduce the complexity of project integration; promote the expansion and implementation of microservice architecture. More and more project architectures have been upgraded from springmvc to spring boot. Kylin can also be upgraded from the springmvc architecture to the spring boot architecture. Do you have any suggestions? -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [kylin] codecov-io commented on issue #787: KYLIN-4108 Show slow query hit cube info
codecov-io commented on issue #787: KYLIN-4108 Show slow query hit cube info URL: https://github.com/apache/kylin/pull/787#issuecomment-518244924 # [Codecov](https://codecov.io/gh/apache/kylin/pull/787?src=pr&el=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@d2c3cd3`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `7.69%`. [](https://codecov.io/gh/apache/kylin/pull/787?src=pr&el=tree) ```diff @@Coverage Diff@@ ## master #787 +/- ## = Coverage ? 25.75% Complexity? 6048 = Files ? 1394 Lines ?83016 Branches ?11610 = Hits ?21381 Misses?59581 Partials ? 2054 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/787?src=pr&el=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...va/org/apache/kylin/rest/service/QueryService.java](https://codecov.io/gh/apache/kylin/pull/787/diff?src=pr&el=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3Qvc2VydmljZS9RdWVyeVNlcnZpY2UuamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...kylin/query/relnode/OLAPToEnumerableConverter.java](https://codecov.io/gh/apache/kylin/pull/787/diff?src=pr&el=tree#diff-cXVlcnkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3F1ZXJ5L3JlbG5vZGUvT0xBUFRvRW51bWVyYWJsZUNvbnZlcnRlci5qYXZh) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...rg/apache/kylin/query/util/QueryInfoCollector.java](https://codecov.io/gh/apache/kylin/pull/787/diff?src=pr&el=tree#diff-cXVlcnkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3F1ZXJ5L3V0aWwvUXVlcnlJbmZvQ29sbGVjdG9yLmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | | | [...rg/apache/kylin/rest/service/BadQueryDetector.java](https://codecov.io/gh/apache/kylin/pull/787/diff?src=pr&el=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3Qvc2VydmljZS9CYWRRdWVyeURldGVjdG9yLmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | | | [.../apache/kylin/metadata/badquery/BadQueryEntry.java](https://codecov.io/gh/apache/kylin/pull/787/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWV0YWRhdGEvYmFkcXVlcnkvQmFkUXVlcnlFbnRyeS5qYXZh) | `45.61% <40%> (ø)` | `12 <1> (?)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/787?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/787?src=pr&el=footer). Last update [d2c3cd3...0425e56](https://codecov.io/gh/apache/kylin/pull/787?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4108) Show slow query hit cube in slow query page
[ https://issues.apache.org/jira/browse/KYLIN-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900070#comment-16900070 ] ASF GitHub Bot commented on KYLIN-4108: --- Wayne1c commented on pull request #787: KYLIN-4108 Show slow query hit cube info URL: https://github.com/apache/kylin/pull/787 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Show slow query hit cube in slow query page > --- > > Key: KYLIN-4108 > URL: https://issues.apache.org/jira/browse/KYLIN-4108 > Project: Kylin > Issue Type: Improvement > Components: Metrics >Reporter: Chao Long >Assignee: Chao Long >Priority: Major > Fix For: v3.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [kylin] Wayne1c opened a new pull request #787: KYLIN-4108 Show slow query hit cube info
Wayne1c opened a new pull request #787: KYLIN-4108 Show slow query hit cube info URL: https://github.com/apache/kylin/pull/787 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (KYLIN-4124) Fix bug in map partition function en cuboid children is empty or null
vinoyang created KYLIN-4124:
---
Summary: Fix bug in map partition function en cuboid children is
empty or null
Key: KYLIN-4124
URL: https://issues.apache.org/jira/browse/KYLIN-4124
Project: Kylin
Issue Type: Bug
Components: Flink Engine
Reporter: vinoyang
Assignee: vinoyang
There is a bug in the {{mapPartition}} method:
{code:java}
// if still empty or null
if (myChildren == null || myChildren.size() == 0) {
return;//it should be continue;
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [kylin] nichunen merged pull request #776: KYLIN-4107 Fix bug of StorageCleanupJob fails to delete Hive tables with "Argument list too long" error
nichunen merged pull request #776: KYLIN-4107 Fix bug of StorageCleanupJob fails to delete Hive tables with "Argument list too long" error URL: https://github.com/apache/kylin/pull/776 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (KYLIN-4107) StorageCleanupJob fails to delete Hive tables with "Argument list too long" error
[
https://issues.apache.org/jira/browse/KYLIN-4107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900043#comment-16900043
]
ASF subversion and git services commented on KYLIN-4107:
Commit d2c3cd342777320185604969d5977bbb75d3058c in kylin's branch
refs/heads/master from weibinzhu
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=d2c3cd3 ]
KYLIN-4107 Fix bug of StorageCleanupJob fails to delete Hive tables with
"Argument list too long" error
> StorageCleanupJob fails to delete Hive tables with "Argument list too long"
> error
> -
>
> Key: KYLIN-4107
> URL: https://issues.apache.org/jira/browse/KYLIN-4107
> Project: Kylin
> Issue Type: Bug
> Components: Storage - HBase
>Affects Versions: v2.6.2
> Environment: CentOS 7.6, HDP 2.6.5, Kylin 2.6.3
>Reporter: Vsevolod Ostapenko
>Assignee: weibin0516
>Priority: Major
> Fix For: v3.0.0-beta
>
>
> On a system with multiple Kylin developers that experiment with cube design
> and (re)build/drop cube segments often intermediate Hive tables and HBase
> left over tables accumulate very quickly.
> After a certain point storage cleanup cannot be executed using suggested
> method:
> {{${KYLIN_HOME}/bin/kylin.sh org.apache.kylin.tool.StorageCleanupJob --delete
> true}}
> Apparently, storage cleanup job creates a single shell command to drop all
> Hive tables, which fails to execute because command line is just too long.
> For example:
> {quote}
> 2019-07-23 17:47:31,611 ERROR [main] job.StorageCleanupJob:377 : Error during
> deleting Hive tables
> java.io.IOException: Cannot run program "/bin/bash": error=7, Argument list
> too long
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
> at
> org.apache.kylin.common.util.CliCommandExecutor.runNativeCommand(CliCommandExecutor.java:133)
> at
> org.apache.kylin.common.util.CliCommandExecutor.execute(CliCommandExecutor.java:89)
> at
> org.apache.kylin.common.util.CliCommandExecutor.execute(CliCommandExecutor.java:83)
> at
> org.apache.kylin.rest.job.StorageCleanupJob.deleteHiveTables(StorageCleanupJob.java:409)
> at
> org.apache.kylin.rest.job.StorageCleanupJob.cleanUnusedIntermediateHiveTableInternal(StorageCleanupJob.java:375)
> at
> org.apache.kylin.rest.job.StorageCleanupJob.cleanUnusedIntermediateHiveTable(StorageCleanupJob.java:278)
> at
> org.apache.kylin.rest.job.StorageCleanupJob.cleanup(StorageCleanupJob.java:151)
> at
> org.apache.kylin.rest.job.StorageCleanupJob.execute(StorageCleanupJob.java:145)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> at org.apache.kylin.tool.StorageCleanupJob.main(StorageCleanupJob.java:27)
> Caused by: java.io.IOException: error=7, Argument list too long
> at java.lang.UNIXProcess.forkAndExec(Native Method)
> at java.lang.UNIXProcess.(UNIXProcess.java:247)
> at java.lang.ProcessImpl.start(ProcessImpl.java:134)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
> ... 10 more
> {quote}
> Instead of composing one long command, storage cleanup need to generate a
> script and feed that into beeline or hive CLI.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (KYLIN-4107) StorageCleanupJob fails to delete Hive tables with "Argument list too long" error
[
https://issues.apache.org/jira/browse/KYLIN-4107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900042#comment-16900042
]
ASF GitHub Bot commented on KYLIN-4107:
---
nichunen commented on pull request #776: KYLIN-4107 Fix bug of
StorageCleanupJob fails to delete Hive tables with "Argument list too long"
error
URL: https://github.com/apache/kylin/pull/776
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> StorageCleanupJob fails to delete Hive tables with "Argument list too long"
> error
> -
>
> Key: KYLIN-4107
> URL: https://issues.apache.org/jira/browse/KYLIN-4107
> Project: Kylin
> Issue Type: Bug
> Components: Storage - HBase
>Affects Versions: v2.6.2
> Environment: CentOS 7.6, HDP 2.6.5, Kylin 2.6.3
>Reporter: Vsevolod Ostapenko
>Assignee: weibin0516
>Priority: Major
> Fix For: v3.0.0-beta
>
>
> On a system with multiple Kylin developers that experiment with cube design
> and (re)build/drop cube segments often intermediate Hive tables and HBase
> left over tables accumulate very quickly.
> After a certain point storage cleanup cannot be executed using suggested
> method:
> {{${KYLIN_HOME}/bin/kylin.sh org.apache.kylin.tool.StorageCleanupJob --delete
> true}}
> Apparently, storage cleanup job creates a single shell command to drop all
> Hive tables, which fails to execute because command line is just too long.
> For example:
> {quote}
> 2019-07-23 17:47:31,611 ERROR [main] job.StorageCleanupJob:377 : Error during
> deleting Hive tables
> java.io.IOException: Cannot run program "/bin/bash": error=7, Argument list
> too long
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
> at
> org.apache.kylin.common.util.CliCommandExecutor.runNativeCommand(CliCommandExecutor.java:133)
> at
> org.apache.kylin.common.util.CliCommandExecutor.execute(CliCommandExecutor.java:89)
> at
> org.apache.kylin.common.util.CliCommandExecutor.execute(CliCommandExecutor.java:83)
> at
> org.apache.kylin.rest.job.StorageCleanupJob.deleteHiveTables(StorageCleanupJob.java:409)
> at
> org.apache.kylin.rest.job.StorageCleanupJob.cleanUnusedIntermediateHiveTableInternal(StorageCleanupJob.java:375)
> at
> org.apache.kylin.rest.job.StorageCleanupJob.cleanUnusedIntermediateHiveTable(StorageCleanupJob.java:278)
> at
> org.apache.kylin.rest.job.StorageCleanupJob.cleanup(StorageCleanupJob.java:151)
> at
> org.apache.kylin.rest.job.StorageCleanupJob.execute(StorageCleanupJob.java:145)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> at org.apache.kylin.tool.StorageCleanupJob.main(StorageCleanupJob.java:27)
> Caused by: java.io.IOException: error=7, Argument list too long
> at java.lang.UNIXProcess.forkAndExec(Native Method)
> at java.lang.UNIXProcess.(UNIXProcess.java:247)
> at java.lang.ProcessImpl.start(ProcessImpl.java:134)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
> ... 10 more
> {quote}
> Instead of composing one long command, storage cleanup need to generate a
> script and feed that into beeline or hive CLI.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (KYLIN-4123) ClassCastException when performing Integration Tests
[
https://issues.apache.org/jira/browse/KYLIN-4123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1685#comment-1685
]
Xiaoxiang Yu commented on KYLIN-4123:
-
Hi, this looks duplicated with
https://issues.apache.org/jira/browse/KYLIN-4040, you may try to use HDP
2.4.0.0-169 to run Kylin's integration test.
> ClassCastException when performing Integration Tests
>
>
> Key: KYLIN-4123
> URL: https://issues.apache.org/jira/browse/KYLIN-4123
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v2.6.3
> Environment: Hortonworks HDP 3.0.1.0-187 VirtualBox image
>Reporter: Mikhail Zhukov
>Priority: Blocker
> Labels: test
>
> Hello!
> When I'm trying to perform integration tests I get following error:
> *Command*:
> mvn verify -fae -Dhdp.version=3.0.1.0-187 -P sandbox
>
> *Error message:*
> 17:06:08.511 [Scheduler 727225988 Job
> aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-392] ERROR
> org.apache.kylin.engine.mr.common.HadoopShellExecutable - error execute
> HadoopShellExecutable\{id=aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-03, name=Build
> Dimension Dictionary, state=RUNNING}
> *java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be
> cast to org.apache.hadoop.hive.serde2.io.ShortWritable*
> at
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableShortObjectInspector.getPrimitiveJavaObject(WritableShortObjectInspector.java:46)
> ~[hive-serde-3.1.0.jar:3.1.0]
> at
> org.apache.hive.hcatalog.data.HCatRecordSerDe.serializePrimitiveField(HCatRecordSerDe.java:278)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at
> org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeField(HCatRecordSerDe.java:199)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:53)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:97)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at
> org.apache.hive.hcatalog.mapreduce.HCatRecordReader.nextKeyValue(HCatRecordReader.java:204)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at
> org.apache.hive.hcatalog.data.transfer.impl.HCatInputFormatReader$HCatRecordItr.hasNext(HCatInputFormatReader.java:107)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at
> org.apache.kylin.source.hive.HiveTableReader.next(HiveTableReader.java:90)
> ~[kylin-source-hive-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.dict.lookup.SnapshotTable.takeSnapshot(SnapshotTable.java:98)
> ~[kylin-core-dictionary-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.dict.lookup.SnapshotManager.buildSnapshot(SnapshotManager.java:137)
> ~[kylin-core-dictionary-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.cube.CubeManager$DictionaryAssist.buildSnapshotTable(CubeManager.java:1202)
> ~[kylin-core-cube-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.cube.CubeManager.buildSnapshotTable(CubeManager.java:1117)
> ~[kylin-core-cube-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:123)
> ~[kylin-core-cube-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:69)
> ~[kylin-core-cube-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:73)
> ~[kylin-engine-mr-2.6.3.jar:2.6.3]
> at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:93)
> ~[kylin-engine-mr-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
> [kylin-engine-mr-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
> [kylin-core-job-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> [kylin-core-job-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
> [kylin-core-job-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> [kylin-core-job-2.6.3.jar:2.6.3]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_172]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_172]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
> 17:06:08.539 [Scheduler 727225988 Job
> aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-392] ERROR
> org.apache.kylin.job.execution.AbstractExecutable - error running Executable:
> CubingJob\{id=aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1, name=BUILD CUBE -
> ci_inner_join_cube - 1970010100_2012060100 - GMT+08:00 2019-08-02
[jira] [Updated] (KYLIN-4123) ClassCastException when performing Integration Tests
[
https://issues.apache.org/jira/browse/KYLIN-4123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Zhukov updated KYLIN-4123:
--
Fix Version/s: (was: v2.6.3)
> ClassCastException when performing Integration Tests
>
>
> Key: KYLIN-4123
> URL: https://issues.apache.org/jira/browse/KYLIN-4123
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
> Environment: Hortonworks HDP 3.0.1.0-187 VirtualBox image
>Reporter: Mikhail Zhukov
>Priority: Blocker
> Labels: test
>
> Hello!
> When I'm trying to perform integration tests I get following error:
> *Command*:
> mvn verify -fae -Dhdp.version=3.0.1.0-187 -P sandbox
>
> *Error message:*
> 17:06:08.511 [Scheduler 727225988 Job
> aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-392] ERROR
> org.apache.kylin.engine.mr.common.HadoopShellExecutable - error execute
> HadoopShellExecutable\{id=aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-03, name=Build
> Dimension Dictionary, state=RUNNING}
> *java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be
> cast to org.apache.hadoop.hive.serde2.io.ShortWritable*
> at
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableShortObjectInspector.getPrimitiveJavaObject(WritableShortObjectInspector.java:46)
> ~[hive-serde-3.1.0.jar:3.1.0]
> at
> org.apache.hive.hcatalog.data.HCatRecordSerDe.serializePrimitiveField(HCatRecordSerDe.java:278)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at
> org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeField(HCatRecordSerDe.java:199)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:53)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:97)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at
> org.apache.hive.hcatalog.mapreduce.HCatRecordReader.nextKeyValue(HCatRecordReader.java:204)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at
> org.apache.hive.hcatalog.data.transfer.impl.HCatInputFormatReader$HCatRecordItr.hasNext(HCatInputFormatReader.java:107)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at
> org.apache.kylin.source.hive.HiveTableReader.next(HiveTableReader.java:90)
> ~[kylin-source-hive-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.dict.lookup.SnapshotTable.takeSnapshot(SnapshotTable.java:98)
> ~[kylin-core-dictionary-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.dict.lookup.SnapshotManager.buildSnapshot(SnapshotManager.java:137)
> ~[kylin-core-dictionary-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.cube.CubeManager$DictionaryAssist.buildSnapshotTable(CubeManager.java:1202)
> ~[kylin-core-cube-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.cube.CubeManager.buildSnapshotTable(CubeManager.java:1117)
> ~[kylin-core-cube-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:123)
> ~[kylin-core-cube-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:69)
> ~[kylin-core-cube-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:73)
> ~[kylin-engine-mr-2.6.3.jar:2.6.3]
> at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:93)
> ~[kylin-engine-mr-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
> [kylin-engine-mr-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
> [kylin-core-job-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> [kylin-core-job-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
> [kylin-core-job-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> [kylin-core-job-2.6.3.jar:2.6.3]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_172]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_172]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
> 17:06:08.539 [Scheduler 727225988 Job
> aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-392] ERROR
> org.apache.kylin.job.execution.AbstractExecutable - error running Executable:
> CubingJob\{id=aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1, name=BUILD CUBE -
> ci_inner_join_cube - 1970010100_2012060100 - GMT+08:00 2019-08-02
> 22:01:57, state=RUNNING}
> 17:06:08.572 [pool-12-thread-4] ERROR
> org.apache.kylin.job.impl.threadpool.DefaultScheduler - ExecuteException
> job:aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1
> org.apach
[jira] [Updated] (KYLIN-4123) ClassCastException when performing Integration Tests
[
https://issues.apache.org/jira/browse/KYLIN-4123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Zhukov updated KYLIN-4123:
--
Affects Version/s: v2.6.3
> ClassCastException when performing Integration Tests
>
>
> Key: KYLIN-4123
> URL: https://issues.apache.org/jira/browse/KYLIN-4123
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v2.6.3
> Environment: Hortonworks HDP 3.0.1.0-187 VirtualBox image
>Reporter: Mikhail Zhukov
>Priority: Blocker
> Labels: test
>
> Hello!
> When I'm trying to perform integration tests I get following error:
> *Command*:
> mvn verify -fae -Dhdp.version=3.0.1.0-187 -P sandbox
>
> *Error message:*
> 17:06:08.511 [Scheduler 727225988 Job
> aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-392] ERROR
> org.apache.kylin.engine.mr.common.HadoopShellExecutable - error execute
> HadoopShellExecutable\{id=aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-03, name=Build
> Dimension Dictionary, state=RUNNING}
> *java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be
> cast to org.apache.hadoop.hive.serde2.io.ShortWritable*
> at
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableShortObjectInspector.getPrimitiveJavaObject(WritableShortObjectInspector.java:46)
> ~[hive-serde-3.1.0.jar:3.1.0]
> at
> org.apache.hive.hcatalog.data.HCatRecordSerDe.serializePrimitiveField(HCatRecordSerDe.java:278)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at
> org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeField(HCatRecordSerDe.java:199)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:53)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:97)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at
> org.apache.hive.hcatalog.mapreduce.HCatRecordReader.nextKeyValue(HCatRecordReader.java:204)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at
> org.apache.hive.hcatalog.data.transfer.impl.HCatInputFormatReader$HCatRecordItr.hasNext(HCatInputFormatReader.java:107)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at
> org.apache.kylin.source.hive.HiveTableReader.next(HiveTableReader.java:90)
> ~[kylin-source-hive-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.dict.lookup.SnapshotTable.takeSnapshot(SnapshotTable.java:98)
> ~[kylin-core-dictionary-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.dict.lookup.SnapshotManager.buildSnapshot(SnapshotManager.java:137)
> ~[kylin-core-dictionary-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.cube.CubeManager$DictionaryAssist.buildSnapshotTable(CubeManager.java:1202)
> ~[kylin-core-cube-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.cube.CubeManager.buildSnapshotTable(CubeManager.java:1117)
> ~[kylin-core-cube-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:123)
> ~[kylin-core-cube-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:69)
> ~[kylin-core-cube-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:73)
> ~[kylin-engine-mr-2.6.3.jar:2.6.3]
> at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:93)
> ~[kylin-engine-mr-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
> [kylin-engine-mr-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
> [kylin-core-job-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> [kylin-core-job-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
> [kylin-core-job-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> [kylin-core-job-2.6.3.jar:2.6.3]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_172]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_172]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
> 17:06:08.539 [Scheduler 727225988 Job
> aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-392] ERROR
> org.apache.kylin.job.execution.AbstractExecutable - error running Executable:
> CubingJob\{id=aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1, name=BUILD CUBE -
> ci_inner_join_cube - 1970010100_2012060100 - GMT+08:00 2019-08-02
> 22:01:57, state=RUNNING}
> 17:06:08.572 [pool-12-thread-4] ERROR
> org.apache.kylin.job.impl.threadpool.DefaultScheduler - ExecuteException
> job:aaebe71d-67ba-43e8-8d9e-e
[jira] [Updated] (KYLIN-4123) ClassCastException when performing Integration Tests
[
https://issues.apache.org/jira/browse/KYLIN-4123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Zhukov updated KYLIN-4123:
--
Description:
Hello!
When I'm trying to perform integration tests I get following error:
*Command*:
mvn verify -fae -Dhdp.version=3.0.1.0-187 -P sandbox
*Error message:*
17:06:08.511 [Scheduler 727225988 Job aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-392]
ERROR org.apache.kylin.engine.mr.common.HadoopShellExecutable - error execute
HadoopShellExecutable\{id=aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-03, name=Build
Dimension Dictionary, state=RUNNING}
*java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast
to org.apache.hadoop.hive.serde2.io.ShortWritable*
at
org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableShortObjectInspector.getPrimitiveJavaObject(WritableShortObjectInspector.java:46)
~[hive-serde-3.1.0.jar:3.1.0]
at
org.apache.hive.hcatalog.data.HCatRecordSerDe.serializePrimitiveField(HCatRecordSerDe.java:278)
~[hive-hcatalog-core-3.1.0.jar:3.1.0]
at
org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeField(HCatRecordSerDe.java:199)
~[hive-hcatalog-core-3.1.0.jar:3.1.0]
at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:53)
~[hive-hcatalog-core-3.1.0.jar:3.1.0]
at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:97)
~[hive-hcatalog-core-3.1.0.jar:3.1.0]
at
org.apache.hive.hcatalog.mapreduce.HCatRecordReader.nextKeyValue(HCatRecordReader.java:204)
~[hive-hcatalog-core-3.1.0.jar:3.1.0]
at
org.apache.hive.hcatalog.data.transfer.impl.HCatInputFormatReader$HCatRecordItr.hasNext(HCatInputFormatReader.java:107)
~[hive-hcatalog-core-3.1.0.jar:3.1.0]
at org.apache.kylin.source.hive.HiveTableReader.next(HiveTableReader.java:90)
~[kylin-source-hive-2.6.3.jar:2.6.3]
at
org.apache.kylin.dict.lookup.SnapshotTable.takeSnapshot(SnapshotTable.java:98)
~[kylin-core-dictionary-2.6.3.jar:2.6.3]
at
org.apache.kylin.dict.lookup.SnapshotManager.buildSnapshot(SnapshotManager.java:137)
~[kylin-core-dictionary-2.6.3.jar:2.6.3]
at
org.apache.kylin.cube.CubeManager$DictionaryAssist.buildSnapshotTable(CubeManager.java:1202)
~[kylin-core-cube-2.6.3.jar:2.6.3]
at org.apache.kylin.cube.CubeManager.buildSnapshotTable(CubeManager.java:1117)
~[kylin-core-cube-2.6.3.jar:2.6.3]
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:123)
~[kylin-core-cube-2.6.3.jar:2.6.3]
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:69)
~[kylin-core-cube-2.6.3.jar:2.6.3]
at
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:73)
~[kylin-engine-mr-2.6.3.jar:2.6.3]
at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:93)
~[kylin-engine-mr-2.6.3.jar:2.6.3]
at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
[kylin-engine-mr-2.6.3.jar:2.6.3]
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
[kylin-core-job-2.6.3.jar:2.6.3]
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
[kylin-core-job-2.6.3.jar:2.6.3]
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
[kylin-core-job-2.6.3.jar:2.6.3]
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
[kylin-core-job-2.6.3.jar:2.6.3]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_172]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_172]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
17:06:08.539 [Scheduler 727225988 Job
aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-392] ERROR
org.apache.kylin.job.execution.AbstractExecutable - error running Executable:
CubingJob\{id=aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1, name=BUILD CUBE -
ci_inner_join_cube - 1970010100_2012060100 - GMT+08:00 2019-08-02
22:01:57, state=RUNNING}
17:06:08.572 [pool-12-thread-4] ERROR
org.apache.kylin.job.impl.threadpool.DefaultScheduler - ExecuteException
job:aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1
org.apache.kylin.job.exception.ExecuteException:
org.apache.kylin.job.exception.ExecuteException:
org.apache.kylin.engine.mr.exception.HadoopShellException:
java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast
to org.apache.hadoop.hive.serde2.io.ShortWritable
at
org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableShortObjectInspector.getPrimitiveJavaObject(WritableShortObjectInspector.java:46)
at
org.apache.hive.hcatalog.data.HCatRecordSerDe.serializePrimitiveField(HCatRecordSerDe.java:278)
at
org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeField(HCatRecordSerDe.java:199)
at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHC
[jira] [Updated] (KYLIN-4123) ClassCastException when performing Integration Tests
[
https://issues.apache.org/jira/browse/KYLIN-4123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Zhukov updated KYLIN-4123:
--
Description:
Hello!
When I'm trying to perform integration tests I get following error:
*Command*:
mvn verify -fae -Dhdp.version=3.0.1.0-187 -P sandbox
*Error message:*
17:06:08.511 [Scheduler 727225988 Job aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-392]
ERROR org.apache.kylin.engine.mr.common.HadoopShellExecutable - error execute
HadoopShellExecutable\{id=aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-03, name=Build
Dimension Dictionary, state=RUNNING}
*java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast
to org.apache.hadoop.hive.serde2.io.ShortWritable*
at
org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableShortObjectInspector.getPrimitiveJavaObject(WritableShortObjectInspector.java:46)
~[hive-serde-3.1.0.jar:3.1.0]
at
org.apache.hive.hcatalog.data.HCatRecordSerDe.serializePrimitiveField(HCatRecordSerDe.java:278)
~[hive-hcatalog-core-3.1.0.jar:3.1.0]
at
org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeField(HCatRecordSerDe.java:199)
~[hive-hcatalog-core-3.1.0.jar:3.1.0]
at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:53)
~[hive-hcatalog-core-3.1.0.jar:3.1.0]
at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:97)
~[hive-hcatalog-core-3.1.0.jar:3.1.0]
at
org.apache.hive.hcatalog.mapreduce.HCatRecordReader.nextKeyValue(HCatRecordReader.java:204)
~[hive-hcatalog-core-3.1.0.jar:3.1.0]
at
org.apache.hive.hcatalog.data.transfer.impl.HCatInputFormatReader$HCatRecordItr.hasNext(HCatInputFormatReader.java:107)
~[hive-hcatalog-core-3.1.0.jar:3.1.0]
at org.apache.kylin.source.hive.HiveTableReader.next(HiveTableReader.java:90)
~[kylin-source-hive-2.6.3.jar:2.6.3]
at
org.apache.kylin.dict.lookup.SnapshotTable.takeSnapshot(SnapshotTable.java:98)
~[kylin-core-dictionary-2.6.3.jar:2.6.3]
at
org.apache.kylin.dict.lookup.SnapshotManager.buildSnapshot(SnapshotManager.java:137)
~[kylin-core-dictionary-2.6.3.jar:2.6.3]
at
org.apache.kylin.cube.CubeManager$DictionaryAssist.buildSnapshotTable(CubeManager.java:1202)
~[kylin-core-cube-2.6.3.jar:2.6.3]
at org.apache.kylin.cube.CubeManager.buildSnapshotTable(CubeManager.java:1117)
~[kylin-core-cube-2.6.3.jar:2.6.3]
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:123)
~[kylin-core-cube-2.6.3.jar:2.6.3]
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:69)
~[kylin-core-cube-2.6.3.jar:2.6.3]
at
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:73)
~[kylin-engine-mr-2.6.3.jar:2.6.3]
at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:93)
~[kylin-engine-mr-2.6.3.jar:2.6.3]
at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
[kylin-engine-mr-2.6.3.jar:2.6.3]
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
[kylin-core-job-2.6.3.jar:2.6.3]
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
[kylin-core-job-2.6.3.jar:2.6.3]
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
[kylin-core-job-2.6.3.jar:2.6.3]
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
[kylin-core-job-2.6.3.jar:2.6.3]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_172]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_172]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
17:06:08.539 [Scheduler 727225988 Job
aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-392] ERROR
org.apache.kylin.job.execution.AbstractExecutable - error running Executable:
CubingJob\{id=aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1, name=BUILD CUBE -
ci_inner_join_cube - 1970010100_2012060100 - GMT+08:00 2019-08-02
22:01:57, state=RUNNING}
Could you please share some suggestions on this?
Thanks!
was:
Hello!
When I'm trying to perform integration tests I get following error:
*Command*:
mvn verify -fae -Dhdp.version=3.0.1.0-187 -P sandbox
*Error message:*
17:06:08.511 [Scheduler 727225988 Job aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-392]
ERROR org.apache.kylin.engine.mr.common.HadoopShellExecutable - error execute
HadoopShellExecutable\{id=aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-03, name=Build
Dimension Dictionary, state=RUNNING}
java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast
to org.apache.hadoop.hive.serde2.io.ShortWritable
at
org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableShortObjectInspector.getPrimitiveJavaObject(WritableShortObjectInspector.java:46)
~[hive-serde-3.1.0.jar:3.1.0]
at
org.apache.hive.hcatalo
[jira] [Created] (KYLIN-4123) ClassCastException when performing Integration Tests
Mikhail Zhukov created KYLIN-4123:
-
Summary: ClassCastException when performing Integration Tests
Key: KYLIN-4123
URL: https://issues.apache.org/jira/browse/KYLIN-4123
Project: Kylin
Issue Type: Bug
Components: Tools, Build and Test
Environment: Hortonworks HDP 3.0.1.0-187 Virtualbox image
Reporter: Mikhail Zhukov
Fix For: v2.6.3
Hello!
When I'm trying to perform integration tests I get following error:
*Command*:
mvn verify -fae -Dhdp.version=3.0.1.0-187 -P sandbox
*Error message:*
17:06:08.511 [Scheduler 727225988 Job aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-392]
ERROR org.apache.kylin.engine.mr.common.HadoopShellExecutable - error execute
HadoopShellExecutable\{id=aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-03, name=Build
Dimension Dictionary, state=RUNNING}
java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast
to org.apache.hadoop.hive.serde2.io.ShortWritable
at
org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableShortObjectInspector.getPrimitiveJavaObject(WritableShortObjectInspector.java:46)
~[hive-serde-3.1.0.jar:3.1.0]
at
org.apache.hive.hcatalog.data.HCatRecordSerDe.serializePrimitiveField(HCatRecordSerDe.java:278)
~[hive-hcatalog-core-3.1.0.jar:3.1.0]
at
org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeField(HCatRecordSerDe.java:199)
~[hive-hcatalog-core-3.1.0.jar:3.1.0]
at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:53)
~[hive-hcatalog-core-3.1.0.jar:3.1.0]
at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:97)
~[hive-hcatalog-core-3.1.0.jar:3.1.0]
at
org.apache.hive.hcatalog.mapreduce.HCatRecordReader.nextKeyValue(HCatRecordReader.java:204)
~[hive-hcatalog-core-3.1.0.jar:3.1.0]
at
org.apache.hive.hcatalog.data.transfer.impl.HCatInputFormatReader$HCatRecordItr.hasNext(HCatInputFormatReader.java:107)
~[hive-hcatalog-core-3.1.0.jar:3.1.0]
at org.apache.kylin.source.hive.HiveTableReader.next(HiveTableReader.java:90)
~[kylin-source-hive-2.6.3.jar:2.6.3]
at
org.apache.kylin.dict.lookup.SnapshotTable.takeSnapshot(SnapshotTable.java:98)
~[kylin-core-dictionary-2.6.3.jar:2.6.3]
at
org.apache.kylin.dict.lookup.SnapshotManager.buildSnapshot(SnapshotManager.java:137)
~[kylin-core-dictionary-2.6.3.jar:2.6.3]
at
org.apache.kylin.cube.CubeManager$DictionaryAssist.buildSnapshotTable(CubeManager.java:1202)
~[kylin-core-cube-2.6.3.jar:2.6.3]
at org.apache.kylin.cube.CubeManager.buildSnapshotTable(CubeManager.java:1117)
~[kylin-core-cube-2.6.3.jar:2.6.3]
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:123)
~[kylin-core-cube-2.6.3.jar:2.6.3]
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:69)
~[kylin-core-cube-2.6.3.jar:2.6.3]
at
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:73)
~[kylin-engine-mr-2.6.3.jar:2.6.3]
at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:93)
~[kylin-engine-mr-2.6.3.jar:2.6.3]
at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
[kylin-engine-mr-2.6.3.jar:2.6.3]
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
[kylin-core-job-2.6.3.jar:2.6.3]
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
[kylin-core-job-2.6.3.jar:2.6.3]
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
[kylin-core-job-2.6.3.jar:2.6.3]
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
[kylin-core-job-2.6.3.jar:2.6.3]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_172]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_172]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
17:06:08.539 [Scheduler 727225988 Job aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-392]
ERROR org.apache.kylin.job.execution.AbstractExecutable - error running
Executable: CubingJob\{id=aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1, name=BUILD CUBE
- ci_inner_join_cube - 1970010100_2012060100 - GMT+08:00 2019-08-02
22:01:57, state=RUNNING}
Could you please share some suggestions on this?
Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (KYLIN-4123) ClassCastException when performing Integration Tests
[
https://issues.apache.org/jira/browse/KYLIN-4123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Zhukov updated KYLIN-4123:
--
Environment: Hortonworks HDP 3.0.1.0-187 VirtualBox image (was:
Hortonworks HDP 3.0.1.0-187 Virtualbox image)
> ClassCastException when performing Integration Tests
>
>
> Key: KYLIN-4123
> URL: https://issues.apache.org/jira/browse/KYLIN-4123
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
> Environment: Hortonworks HDP 3.0.1.0-187 VirtualBox image
>Reporter: Mikhail Zhukov
>Priority: Blocker
> Labels: test
> Fix For: v2.6.3
>
>
> Hello!
> When I'm trying to perform integration tests I get following error:
> *Command*:
> mvn verify -fae -Dhdp.version=3.0.1.0-187 -P sandbox
>
> *Error message:*
> 17:06:08.511 [Scheduler 727225988 Job
> aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-392] ERROR
> org.apache.kylin.engine.mr.common.HadoopShellExecutable - error execute
> HadoopShellExecutable\{id=aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-03, name=Build
> Dimension Dictionary, state=RUNNING}
> java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast
> to org.apache.hadoop.hive.serde2.io.ShortWritable
> at
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableShortObjectInspector.getPrimitiveJavaObject(WritableShortObjectInspector.java:46)
> ~[hive-serde-3.1.0.jar:3.1.0]
> at
> org.apache.hive.hcatalog.data.HCatRecordSerDe.serializePrimitiveField(HCatRecordSerDe.java:278)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at
> org.apache.hive.hcatalog.data.HCatRecordSerDe.serializeField(HCatRecordSerDe.java:199)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:53)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at org.apache.hive.hcatalog.data.LazyHCatRecord.get(LazyHCatRecord.java:97)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at
> org.apache.hive.hcatalog.mapreduce.HCatRecordReader.nextKeyValue(HCatRecordReader.java:204)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at
> org.apache.hive.hcatalog.data.transfer.impl.HCatInputFormatReader$HCatRecordItr.hasNext(HCatInputFormatReader.java:107)
> ~[hive-hcatalog-core-3.1.0.jar:3.1.0]
> at
> org.apache.kylin.source.hive.HiveTableReader.next(HiveTableReader.java:90)
> ~[kylin-source-hive-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.dict.lookup.SnapshotTable.takeSnapshot(SnapshotTable.java:98)
> ~[kylin-core-dictionary-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.dict.lookup.SnapshotManager.buildSnapshot(SnapshotManager.java:137)
> ~[kylin-core-dictionary-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.cube.CubeManager$DictionaryAssist.buildSnapshotTable(CubeManager.java:1202)
> ~[kylin-core-cube-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.cube.CubeManager.buildSnapshotTable(CubeManager.java:1117)
> ~[kylin-core-cube-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:123)
> ~[kylin-core-cube-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:69)
> ~[kylin-core-cube-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:73)
> ~[kylin-engine-mr-2.6.3.jar:2.6.3]
> at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:93)
> ~[kylin-engine-mr-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
> [kylin-engine-mr-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
> [kylin-core-job-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> [kylin-core-job-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167)
> [kylin-core-job-2.6.3.jar:2.6.3]
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> [kylin-core-job-2.6.3.jar:2.6.3]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_172]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_172]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
> 17:06:08.539 [Scheduler 727225988 Job
> aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1-392] ERROR
> org.apache.kylin.job.execution.AbstractExecutable - error running Executable:
> CubingJob\{id=aaebe71d-67ba-43e8-8d9e-eb7aa98b49e1, name=BUILD CUBE -
> ci_inner_join_cube - 1970010100_2012060100 - GMT+08:00 2019-08-02
> 22:01:57, state=RUNNING}
>
> Could you please share some suggestions on this?
> Thanks!
