[jira] [Commented] (KYLIN-3370) Enhance segment pruning
[ https://issues.apache.org/jira/browse/KYLIN-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535627#comment-16535627 ] ASF subversion and git services commented on KYLIN-3370: Commit ab7bdaea290123b1823eb8e1f65d4206d198f6d5 in kylin's branch refs/heads/sync from Li Yang [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=ab7bdae ] KYLIN-3370 bug fix in UpdateCubeInfoAfterMergeStep > Enhance segment pruning > --- > > Key: KYLIN-3370 > URL: https://issues.apache.org/jira/browse/KYLIN-3370 > Project: Kylin > Issue Type: Improvement > Components: Metadata, Query Engine >Reporter: Shaoxiong Zhan >Assignee: Shaoxiong Zhan >Priority: Major > Fix For: v2.5.0 > > > 1.Compute and store all dimension's range in cubeSegment,; > 2.If the query condition don't satisfy dimension range, then no need to scan > the segment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3370) Enhance segment pruning
[ https://issues.apache.org/jira/browse/KYLIN-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535625#comment-16535625 ] ASF subversion and git services commented on KYLIN-3370: Commit a08fed5b5273c16abfaf6a7a09ba44545069e88a in kylin's branch refs/heads/sync from Li Yang [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=a08fed5 ] KYLIN-3370 bug fix in UpdateCubeInfoAfterMergeStep > Enhance segment pruning > --- > > Key: KYLIN-3370 > URL: https://issues.apache.org/jira/browse/KYLIN-3370 > Project: Kylin > Issue Type: Improvement > Components: Metadata, Query Engine >Reporter: Shaoxiong Zhan >Assignee: Shaoxiong Zhan >Priority: Major > Fix For: v2.5.0 > > > 1.Compute and store all dimension's range in cubeSegment,; > 2.If the query condition don't satisfy dimension range, then no need to scan > the segment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3428) java.lang.OutOfMemoryError: Requested array size exceeds VM limit
[ https://issues.apache.org/jira/browse/KYLIN-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534945#comment-16534945 ] ASF subversion and git services commented on KYLIN-3428: Commit 84c5449dfe90028d23ae76847951d415711f54f9 in kylin's branch refs/heads/2.4.x from [~yangcao] [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=84c5449 ] KYLIN-3428 too large dict file cause OOM Signed-off-by: shaofengshi > java.lang.OutOfMemoryError: Requested array size exceeds VM limit > - > > Key: KYLIN-3428 > URL: https://issues.apache.org/jira/browse/KYLIN-3428 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.1.0, v2.2.0, v2.3.0, v2.3.1, v2.4.0 > Environment: kylin v2.2.0 jdk7 >Reporter: yangcao >Assignee: yangcao >Priority: Critical > Labels: Build_Base_Cuboid, MAP, OOM > Fix For: v2.4.1, v2.5.0 > > Attachments: patch-v3.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > LOG: > 2018-06-26 15:50:24,032 INFO [main] org.apache.kylin.dict.DictionaryManager: > DictionaryManager(1499050426) loading DictionaryInfo(loadDictObj:true) at > /dict/xxx.xxx/C7/036b7ca0-8733-4c0c-99f5-5122919fd3dd.dict 2018-06-26 > 15:50:25,586 ERROR [main] org.apache.kylin.engine.mr.KylinMapper: > com.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError: > Requested array size exceeds VM limit at > com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2232) at > com.google.common.cache.LocalCache.get(LocalCache.java:3965) at > com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at > com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829) > at > org.apache.kylin.dict.DictionaryManager.getDictionaryInfo(DictionaryManager.java:118) > at org.apache.kylin.cube.CubeManager.getDictionary(CubeManager.java:271) at > org.apache.kylin.cube.CubeSegment.getDictionary(CubeSegment.java:320) at > org.apache.kylin.cube.kv.CubeDimEncMap.getDictionary(CubeDimEncMap.java:86) > at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:65) at > org.apache.kylin.cube.kv.RowKeyColumnIO.getColumnLength(RowKeyColumnIO.java:43) > at org.apache.kylin.cube.kv.RowKeyEncoder.(RowKeyEncoder.java:59) at > org.apache.kylin.cube.kv.AbstractRowKeyEncoder.createInstance(AbstractRowKeyEncoder.java:48) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.(BaseCuboidBuilder.java:84) > at > org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.doSetup(BaseCuboidMapperBase.java:70) > at > org.apache.kylin.engine.mr.steps.HiveToBaseCuboidMapper.doSetup(HiveToBaseCuboidMapper.java:36) > at org.apache.kylin.engine.mr.KylinMapper.setup(KylinMapper.java:48) at > org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at > org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at > java.security.AccessController.doPrivileged(Native Method) at > javax.security.auth.Subject.doAs(Subject.java:415) at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1707) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: > java.lang.OutOfMemoryError: Requested array size exceeds VM limit at > java.util.Arrays.copyOf(Arrays.java:2271) at > java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113) at > java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93) > at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140) at > org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1793) at > org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769) at > org.apache.commons.io.IOUtils.copy(IOUtils.java:1744) at > org.apache.kylin.common.persistence.FileResourceStore.getResourceImpl(FileResourceStore.java:123) > at > org.apache.kylin.common.persistence.ResourceStore.getResource(ResourceStore.java:154) > at org.apache.kylin.dict.DictionaryManager.load(DictionaryManager.java:418) > at org.apache.kylin.dict.DictionaryManager$1.load(DictionaryManager.java:101) > at org.apache.kylin.dict.DictionaryManager$1.load(DictionaryManager.java:98) > at > com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568) > at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350) > at > com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313) > at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228) at > com.google.common.cache.LocalCache.get(LocalCache.java:3965) at > com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at >
[jira] [Commented] (KYLIN-3427) Convert to HFile in Spark
[ https://issues.apache.org/jira/browse/KYLIN-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534826#comment-16534826 ] ASF subversion and git services commented on KYLIN-3427: Commit 0f209517898b9599e62fde68a70ca62851f7e175 in kylin's branch refs/heads/2.4.x from shaofengshi [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=0f20951 ] KYLIN-3427 fix UT in AdminServiceTest > Convert to HFile in Spark > - > > Key: KYLIN-3427 > URL: https://issues.apache.org/jira/browse/KYLIN-3427 > Project: Kylin > Issue Type: Improvement > Components: Spark Engine >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI >Priority: Major > Fix For: v2.5.0 > > > Some references: > https://www.opencore.com/blog/2016/10/efficient-bulk-load-of-hbase-using-spark/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3428) java.lang.OutOfMemoryError: Requested array size exceeds VM limit
[ https://issues.apache.org/jira/browse/KYLIN-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534823#comment-16534823 ] ASF subversion and git services commented on KYLIN-3428: Commit 59354acced7fdb35e0325fb7a8ea2e8c862f658c in kylin's branch refs/heads/master from [~yangcao] [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=59354ac ] KYLIN-3428 too large dict file cause OOM Signed-off-by: shaofengshi > java.lang.OutOfMemoryError: Requested array size exceeds VM limit > - > > Key: KYLIN-3428 > URL: https://issues.apache.org/jira/browse/KYLIN-3428 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.1.0, v2.2.0, v2.3.0, v2.3.1, v2.4.0 > Environment: kylin v2.2.0 jdk7 >Reporter: yangcao >Assignee: yangcao >Priority: Critical > Labels: Build_Base_Cuboid, MAP, OOM > Fix For: v2.4.1, v2.5.0 > > Attachments: patch-v3.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > LOG: > 2018-06-26 15:50:24,032 INFO [main] org.apache.kylin.dict.DictionaryManager: > DictionaryManager(1499050426) loading DictionaryInfo(loadDictObj:true) at > /dict/xxx.xxx/C7/036b7ca0-8733-4c0c-99f5-5122919fd3dd.dict 2018-06-26 > 15:50:25,586 ERROR [main] org.apache.kylin.engine.mr.KylinMapper: > com.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError: > Requested array size exceeds VM limit at > com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2232) at > com.google.common.cache.LocalCache.get(LocalCache.java:3965) at > com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at > com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829) > at > org.apache.kylin.dict.DictionaryManager.getDictionaryInfo(DictionaryManager.java:118) > at org.apache.kylin.cube.CubeManager.getDictionary(CubeManager.java:271) at > org.apache.kylin.cube.CubeSegment.getDictionary(CubeSegment.java:320) at > org.apache.kylin.cube.kv.CubeDimEncMap.getDictionary(CubeDimEncMap.java:86) > at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:65) at > org.apache.kylin.cube.kv.RowKeyColumnIO.getColumnLength(RowKeyColumnIO.java:43) > at org.apache.kylin.cube.kv.RowKeyEncoder.(RowKeyEncoder.java:59) at > org.apache.kylin.cube.kv.AbstractRowKeyEncoder.createInstance(AbstractRowKeyEncoder.java:48) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.(BaseCuboidBuilder.java:84) > at > org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.doSetup(BaseCuboidMapperBase.java:70) > at > org.apache.kylin.engine.mr.steps.HiveToBaseCuboidMapper.doSetup(HiveToBaseCuboidMapper.java:36) > at org.apache.kylin.engine.mr.KylinMapper.setup(KylinMapper.java:48) at > org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at > org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at > java.security.AccessController.doPrivileged(Native Method) at > javax.security.auth.Subject.doAs(Subject.java:415) at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1707) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: > java.lang.OutOfMemoryError: Requested array size exceeds VM limit at > java.util.Arrays.copyOf(Arrays.java:2271) at > java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113) at > java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93) > at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140) at > org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1793) at > org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769) at > org.apache.commons.io.IOUtils.copy(IOUtils.java:1744) at > org.apache.kylin.common.persistence.FileResourceStore.getResourceImpl(FileResourceStore.java:123) > at > org.apache.kylin.common.persistence.ResourceStore.getResource(ResourceStore.java:154) > at org.apache.kylin.dict.DictionaryManager.load(DictionaryManager.java:418) > at org.apache.kylin.dict.DictionaryManager$1.load(DictionaryManager.java:101) > at org.apache.kylin.dict.DictionaryManager$1.load(DictionaryManager.java:98) > at > com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568) > at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350) > at > com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313) > at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228) at > com.google.common.cache.LocalCache.get(LocalCache.java:3965) at > com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at >
[jira] [Created] (KYLIN-3443) Optimize cube job in Spark
Shaofeng SHI created KYLIN-3443: --- Summary: Optimize cube job in Spark Key: KYLIN-3443 URL: https://issues.apache.org/jira/browse/KYLIN-3443 Project: Kylin Issue Type: Improvement Components: Spark Engine Reporter: Shaofeng SHI -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3442) Fact distinct columns in Spark
Shaofeng SHI created KYLIN-3442: --- Summary: Fact distinct columns in Spark Key: KYLIN-3442 URL: https://issues.apache.org/jira/browse/KYLIN-3442 Project: Kylin Issue Type: Improvement Components: Spark Engine Reporter: Shaofeng SHI -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3441) Merge cube segments in Spark
Shaofeng SHI created KYLIN-3441: --- Summary: Merge cube segments in Spark Key: KYLIN-3441 URL: https://issues.apache.org/jira/browse/KYLIN-3441 Project: Kylin Issue Type: Improvement Components: Spark Engine Reporter: Shaofeng SHI -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3430) Global Dictionary Cleanup
[
https://issues.apache.org/jira/browse/KYLIN-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534657#comment-16534657
]
kangkaisen commented on KYLIN-3430:
---
Hi, [~temple.zhou].
There is also a config "kylin.dictionary.append-version-ttl", the default value
is three days. which means we will not delete the dir Within three days.
The Global Dictionary is not cube level, multiple cubes could use the same
Global Dictionary. So we shouldn't delete Global Dictionary when we drop one
cube.
But you remind me, if the Global Dictionary related cube are all dropped,
currently, we will never delete the Global Dictionary in HDFS. This is a issue
indeed, thanks you very much! I thought wrong.
As for how to check a Global Dictionary whether has reference or not, I think
we could get the active Global Dictionary from the cube metadata, if all cube
don't use a Global Dictionary, we could think the Global Dictionary is useless
and delete it.
Thanks you !
> Global Dictionary Cleanup
> -
>
> Key: KYLIN-3430
> URL: https://issues.apache.org/jira/browse/KYLIN-3430
> Project: Kylin
> Issue Type: Improvement
> Components: Tools, Build and Test
>Affects Versions: v2.1.0, v2.2.0, v2.3.0, v2.3.1, v2.4.0
>Reporter: Temple Zhou
>Assignee: Temple Zhou
>Priority: Major
> Attachments: KYLIN-3430.master.001.patch
>
>
> I had run "{{./bin/metastore.sh clean --delete true" to cleanup my Kylin
> metadata, but, after that, the Global Dictionary still exists in my HDFS and
> the size of directory "/kylin_metadata/resources/GlobalDict/dict" hasn't
> shrunk.}}
>
> {{BTW: I'm very sure that there are redundant Global Dictionaries.}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3428) java.lang.OutOfMemoryError: Requested array size exceeds VM limit
[ https://issues.apache.org/jira/browse/KYLIN-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yangcao updated KYLIN-3428: --- Attachment: (was: patch-v2.patch) > java.lang.OutOfMemoryError: Requested array size exceeds VM limit > - > > Key: KYLIN-3428 > URL: https://issues.apache.org/jira/browse/KYLIN-3428 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.1.0, v2.2.0, v2.3.0, v2.3.1, v2.4.0 > Environment: kylin v2.2.0 jdk7 >Reporter: yangcao >Assignee: yangcao >Priority: Critical > Labels: Build_Base_Cuboid, MAP, OOM > Fix For: v2.4.1, v2.5.0 > > Attachments: patch-v3.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > LOG: > 2018-06-26 15:50:24,032 INFO [main] org.apache.kylin.dict.DictionaryManager: > DictionaryManager(1499050426) loading DictionaryInfo(loadDictObj:true) at > /dict/xxx.xxx/C7/036b7ca0-8733-4c0c-99f5-5122919fd3dd.dict 2018-06-26 > 15:50:25,586 ERROR [main] org.apache.kylin.engine.mr.KylinMapper: > com.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError: > Requested array size exceeds VM limit at > com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2232) at > com.google.common.cache.LocalCache.get(LocalCache.java:3965) at > com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at > com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829) > at > org.apache.kylin.dict.DictionaryManager.getDictionaryInfo(DictionaryManager.java:118) > at org.apache.kylin.cube.CubeManager.getDictionary(CubeManager.java:271) at > org.apache.kylin.cube.CubeSegment.getDictionary(CubeSegment.java:320) at > org.apache.kylin.cube.kv.CubeDimEncMap.getDictionary(CubeDimEncMap.java:86) > at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:65) at > org.apache.kylin.cube.kv.RowKeyColumnIO.getColumnLength(RowKeyColumnIO.java:43) > at org.apache.kylin.cube.kv.RowKeyEncoder.(RowKeyEncoder.java:59) at > org.apache.kylin.cube.kv.AbstractRowKeyEncoder.createInstance(AbstractRowKeyEncoder.java:48) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.(BaseCuboidBuilder.java:84) > at > org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.doSetup(BaseCuboidMapperBase.java:70) > at > org.apache.kylin.engine.mr.steps.HiveToBaseCuboidMapper.doSetup(HiveToBaseCuboidMapper.java:36) > at org.apache.kylin.engine.mr.KylinMapper.setup(KylinMapper.java:48) at > org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at > org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at > java.security.AccessController.doPrivileged(Native Method) at > javax.security.auth.Subject.doAs(Subject.java:415) at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1707) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: > java.lang.OutOfMemoryError: Requested array size exceeds VM limit at > java.util.Arrays.copyOf(Arrays.java:2271) at > java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113) at > java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93) > at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140) at > org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1793) at > org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769) at > org.apache.commons.io.IOUtils.copy(IOUtils.java:1744) at > org.apache.kylin.common.persistence.FileResourceStore.getResourceImpl(FileResourceStore.java:123) > at > org.apache.kylin.common.persistence.ResourceStore.getResource(ResourceStore.java:154) > at org.apache.kylin.dict.DictionaryManager.load(DictionaryManager.java:418) > at org.apache.kylin.dict.DictionaryManager$1.load(DictionaryManager.java:101) > at org.apache.kylin.dict.DictionaryManager$1.load(DictionaryManager.java:98) > at > com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568) > at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350) > at > com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313) > at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228) at > com.google.common.cache.LocalCache.get(LocalCache.java:3965) at > com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at > com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829) > at > org.apache.kylin.dict.DictionaryManager.getDictionaryInfo(DictionaryManager.java:118) > at org.apache.kylin.cube.CubeManager.getDictionary(CubeManager.java:271) at >
[jira] [Closed] (KYLIN-3428) java.lang.OutOfMemoryError: Requested array size exceeds VM limit
[ https://issues.apache.org/jira/browse/KYLIN-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yangcao closed KYLIN-3428. -- Resolution: Fixed > java.lang.OutOfMemoryError: Requested array size exceeds VM limit > - > > Key: KYLIN-3428 > URL: https://issues.apache.org/jira/browse/KYLIN-3428 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.1.0, v2.2.0, v2.3.0, v2.3.1, v2.4.0 > Environment: kylin v2.2.0 jdk7 >Reporter: yangcao >Assignee: yangcao >Priority: Critical > Labels: Build_Base_Cuboid, MAP, OOM > Fix For: v2.4.1, v2.5.0 > > Attachments: patch-v3.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > LOG: > 2018-06-26 15:50:24,032 INFO [main] org.apache.kylin.dict.DictionaryManager: > DictionaryManager(1499050426) loading DictionaryInfo(loadDictObj:true) at > /dict/xxx.xxx/C7/036b7ca0-8733-4c0c-99f5-5122919fd3dd.dict 2018-06-26 > 15:50:25,586 ERROR [main] org.apache.kylin.engine.mr.KylinMapper: > com.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError: > Requested array size exceeds VM limit at > com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2232) at > com.google.common.cache.LocalCache.get(LocalCache.java:3965) at > com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at > com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829) > at > org.apache.kylin.dict.DictionaryManager.getDictionaryInfo(DictionaryManager.java:118) > at org.apache.kylin.cube.CubeManager.getDictionary(CubeManager.java:271) at > org.apache.kylin.cube.CubeSegment.getDictionary(CubeSegment.java:320) at > org.apache.kylin.cube.kv.CubeDimEncMap.getDictionary(CubeDimEncMap.java:86) > at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:65) at > org.apache.kylin.cube.kv.RowKeyColumnIO.getColumnLength(RowKeyColumnIO.java:43) > at org.apache.kylin.cube.kv.RowKeyEncoder.(RowKeyEncoder.java:59) at > org.apache.kylin.cube.kv.AbstractRowKeyEncoder.createInstance(AbstractRowKeyEncoder.java:48) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.(BaseCuboidBuilder.java:84) > at > org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.doSetup(BaseCuboidMapperBase.java:70) > at > org.apache.kylin.engine.mr.steps.HiveToBaseCuboidMapper.doSetup(HiveToBaseCuboidMapper.java:36) > at org.apache.kylin.engine.mr.KylinMapper.setup(KylinMapper.java:48) at > org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at > org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at > java.security.AccessController.doPrivileged(Native Method) at > javax.security.auth.Subject.doAs(Subject.java:415) at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1707) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: > java.lang.OutOfMemoryError: Requested array size exceeds VM limit at > java.util.Arrays.copyOf(Arrays.java:2271) at > java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113) at > java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93) > at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140) at > org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1793) at > org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769) at > org.apache.commons.io.IOUtils.copy(IOUtils.java:1744) at > org.apache.kylin.common.persistence.FileResourceStore.getResourceImpl(FileResourceStore.java:123) > at > org.apache.kylin.common.persistence.ResourceStore.getResource(ResourceStore.java:154) > at org.apache.kylin.dict.DictionaryManager.load(DictionaryManager.java:418) > at org.apache.kylin.dict.DictionaryManager$1.load(DictionaryManager.java:101) > at org.apache.kylin.dict.DictionaryManager$1.load(DictionaryManager.java:98) > at > com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568) > at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350) > at > com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313) > at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228) at > com.google.common.cache.LocalCache.get(LocalCache.java:3965) at > com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at > com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829) > at > org.apache.kylin.dict.DictionaryManager.getDictionaryInfo(DictionaryManager.java:118) > at org.apache.kylin.cube.CubeManager.getDictionary(CubeManager.java:271) at >
[jira] [Updated] (KYLIN-3428) java.lang.OutOfMemoryError: Requested array size exceeds VM limit
[ https://issues.apache.org/jira/browse/KYLIN-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yangcao updated KYLIN-3428: --- Attachment: patch-v3.patch > java.lang.OutOfMemoryError: Requested array size exceeds VM limit > - > > Key: KYLIN-3428 > URL: https://issues.apache.org/jira/browse/KYLIN-3428 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.1.0, v2.2.0, v2.3.0, v2.3.1, v2.4.0 > Environment: kylin v2.2.0 jdk7 >Reporter: yangcao >Assignee: yangcao >Priority: Critical > Labels: Build_Base_Cuboid, MAP, OOM > Fix For: v2.4.1, v2.5.0 > > Attachments: patch-v2.patch, patch-v3.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > LOG: > 2018-06-26 15:50:24,032 INFO [main] org.apache.kylin.dict.DictionaryManager: > DictionaryManager(1499050426) loading DictionaryInfo(loadDictObj:true) at > /dict/xxx.xxx/C7/036b7ca0-8733-4c0c-99f5-5122919fd3dd.dict 2018-06-26 > 15:50:25,586 ERROR [main] org.apache.kylin.engine.mr.KylinMapper: > com.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError: > Requested array size exceeds VM limit at > com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2232) at > com.google.common.cache.LocalCache.get(LocalCache.java:3965) at > com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at > com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829) > at > org.apache.kylin.dict.DictionaryManager.getDictionaryInfo(DictionaryManager.java:118) > at org.apache.kylin.cube.CubeManager.getDictionary(CubeManager.java:271) at > org.apache.kylin.cube.CubeSegment.getDictionary(CubeSegment.java:320) at > org.apache.kylin.cube.kv.CubeDimEncMap.getDictionary(CubeDimEncMap.java:86) > at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:65) at > org.apache.kylin.cube.kv.RowKeyColumnIO.getColumnLength(RowKeyColumnIO.java:43) > at org.apache.kylin.cube.kv.RowKeyEncoder.(RowKeyEncoder.java:59) at > org.apache.kylin.cube.kv.AbstractRowKeyEncoder.createInstance(AbstractRowKeyEncoder.java:48) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.(BaseCuboidBuilder.java:84) > at > org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.doSetup(BaseCuboidMapperBase.java:70) > at > org.apache.kylin.engine.mr.steps.HiveToBaseCuboidMapper.doSetup(HiveToBaseCuboidMapper.java:36) > at org.apache.kylin.engine.mr.KylinMapper.setup(KylinMapper.java:48) at > org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at > org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at > java.security.AccessController.doPrivileged(Native Method) at > javax.security.auth.Subject.doAs(Subject.java:415) at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1707) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: > java.lang.OutOfMemoryError: Requested array size exceeds VM limit at > java.util.Arrays.copyOf(Arrays.java:2271) at > java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113) at > java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93) > at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140) at > org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1793) at > org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769) at > org.apache.commons.io.IOUtils.copy(IOUtils.java:1744) at > org.apache.kylin.common.persistence.FileResourceStore.getResourceImpl(FileResourceStore.java:123) > at > org.apache.kylin.common.persistence.ResourceStore.getResource(ResourceStore.java:154) > at org.apache.kylin.dict.DictionaryManager.load(DictionaryManager.java:418) > at org.apache.kylin.dict.DictionaryManager$1.load(DictionaryManager.java:101) > at org.apache.kylin.dict.DictionaryManager$1.load(DictionaryManager.java:98) > at > com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568) > at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350) > at > com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313) > at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228) at > com.google.common.cache.LocalCache.get(LocalCache.java:3965) at > com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at > com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829) > at > org.apache.kylin.dict.DictionaryManager.getDictionaryInfo(DictionaryManager.java:118) > at org.apache.kylin.cube.CubeManager.getDictionary(CubeManager.java:271) at >
[jira] [Reopened] (KYLIN-3428) java.lang.OutOfMemoryError: Requested array size exceeds VM limit
[ https://issues.apache.org/jira/browse/KYLIN-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yangcao reopened KYLIN-3428: > java.lang.OutOfMemoryError: Requested array size exceeds VM limit > - > > Key: KYLIN-3428 > URL: https://issues.apache.org/jira/browse/KYLIN-3428 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.1.0, v2.2.0, v2.3.0, v2.3.1, v2.4.0 > Environment: kylin v2.2.0 jdk7 >Reporter: yangcao >Assignee: yangcao >Priority: Critical > Labels: Build_Base_Cuboid, MAP, OOM > Fix For: v2.4.1, v2.5.0 > > Attachments: patch-v2.patch, patch-v3.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > LOG: > 2018-06-26 15:50:24,032 INFO [main] org.apache.kylin.dict.DictionaryManager: > DictionaryManager(1499050426) loading DictionaryInfo(loadDictObj:true) at > /dict/xxx.xxx/C7/036b7ca0-8733-4c0c-99f5-5122919fd3dd.dict 2018-06-26 > 15:50:25,586 ERROR [main] org.apache.kylin.engine.mr.KylinMapper: > com.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError: > Requested array size exceeds VM limit at > com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2232) at > com.google.common.cache.LocalCache.get(LocalCache.java:3965) at > com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at > com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829) > at > org.apache.kylin.dict.DictionaryManager.getDictionaryInfo(DictionaryManager.java:118) > at org.apache.kylin.cube.CubeManager.getDictionary(CubeManager.java:271) at > org.apache.kylin.cube.CubeSegment.getDictionary(CubeSegment.java:320) at > org.apache.kylin.cube.kv.CubeDimEncMap.getDictionary(CubeDimEncMap.java:86) > at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:65) at > org.apache.kylin.cube.kv.RowKeyColumnIO.getColumnLength(RowKeyColumnIO.java:43) > at org.apache.kylin.cube.kv.RowKeyEncoder.(RowKeyEncoder.java:59) at > org.apache.kylin.cube.kv.AbstractRowKeyEncoder.createInstance(AbstractRowKeyEncoder.java:48) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.(BaseCuboidBuilder.java:84) > at > org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.doSetup(BaseCuboidMapperBase.java:70) > at > org.apache.kylin.engine.mr.steps.HiveToBaseCuboidMapper.doSetup(HiveToBaseCuboidMapper.java:36) > at org.apache.kylin.engine.mr.KylinMapper.setup(KylinMapper.java:48) at > org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at > org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at > java.security.AccessController.doPrivileged(Native Method) at > javax.security.auth.Subject.doAs(Subject.java:415) at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1707) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: > java.lang.OutOfMemoryError: Requested array size exceeds VM limit at > java.util.Arrays.copyOf(Arrays.java:2271) at > java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113) at > java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93) > at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140) at > org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1793) at > org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769) at > org.apache.commons.io.IOUtils.copy(IOUtils.java:1744) at > org.apache.kylin.common.persistence.FileResourceStore.getResourceImpl(FileResourceStore.java:123) > at > org.apache.kylin.common.persistence.ResourceStore.getResource(ResourceStore.java:154) > at org.apache.kylin.dict.DictionaryManager.load(DictionaryManager.java:418) > at org.apache.kylin.dict.DictionaryManager$1.load(DictionaryManager.java:101) > at org.apache.kylin.dict.DictionaryManager$1.load(DictionaryManager.java:98) > at > com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568) > at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350) > at > com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313) > at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228) at > com.google.common.cache.LocalCache.get(LocalCache.java:3965) at > com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at > com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829) > at > org.apache.kylin.dict.DictionaryManager.getDictionaryInfo(DictionaryManager.java:118) > at org.apache.kylin.cube.CubeManager.getDictionary(CubeManager.java:271) at >
[jira] [Commented] (KYLIN-3427) Convert to HFile in Spark
[ https://issues.apache.org/jira/browse/KYLIN-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534521#comment-16534521 ] ASF subversion and git services commented on KYLIN-3427: Commit bec6e5965a7a547d8dd346023e93a6e287c8a9db in kylin's branch refs/heads/master from shaofengshi [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=bec6e59 ] KYLIN-3427 fix UT in AdminServiceTest > Convert to HFile in Spark > - > > Key: KYLIN-3427 > URL: https://issues.apache.org/jira/browse/KYLIN-3427 > Project: Kylin > Issue Type: Improvement > Components: Spark Engine >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI >Priority: Major > Fix For: v2.5.0 > > > Some references: > https://www.opencore.com/blog/2016/10/efficient-bulk-load-of-hbase-using-spark/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (KYLIN-3428) java.lang.OutOfMemoryError: Requested array size exceeds VM limit
[ https://issues.apache.org/jira/browse/KYLIN-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yangcao closed KYLIN-3428. -- Resolution: Fixed > java.lang.OutOfMemoryError: Requested array size exceeds VM limit > - > > Key: KYLIN-3428 > URL: https://issues.apache.org/jira/browse/KYLIN-3428 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v2.1.0, v2.2.0, v2.3.0, v2.3.1, v2.4.0 > Environment: kylin v2.2.0 jdk7 >Reporter: yangcao >Assignee: yangcao >Priority: Critical > Labels: Build_Base_Cuboid, MAP, OOM > Fix For: v2.4.1, v2.5.0 > > Attachments: patch-v2.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > LOG: > 2018-06-26 15:50:24,032 INFO [main] org.apache.kylin.dict.DictionaryManager: > DictionaryManager(1499050426) loading DictionaryInfo(loadDictObj:true) at > /dict/xxx.xxx/C7/036b7ca0-8733-4c0c-99f5-5122919fd3dd.dict 2018-06-26 > 15:50:25,586 ERROR [main] org.apache.kylin.engine.mr.KylinMapper: > com.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError: > Requested array size exceeds VM limit at > com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2232) at > com.google.common.cache.LocalCache.get(LocalCache.java:3965) at > com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at > com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829) > at > org.apache.kylin.dict.DictionaryManager.getDictionaryInfo(DictionaryManager.java:118) > at org.apache.kylin.cube.CubeManager.getDictionary(CubeManager.java:271) at > org.apache.kylin.cube.CubeSegment.getDictionary(CubeSegment.java:320) at > org.apache.kylin.cube.kv.CubeDimEncMap.getDictionary(CubeDimEncMap.java:86) > at org.apache.kylin.cube.kv.CubeDimEncMap.get(CubeDimEncMap.java:65) at > org.apache.kylin.cube.kv.RowKeyColumnIO.getColumnLength(RowKeyColumnIO.java:43) > at org.apache.kylin.cube.kv.RowKeyEncoder.(RowKeyEncoder.java:59) at > org.apache.kylin.cube.kv.AbstractRowKeyEncoder.createInstance(AbstractRowKeyEncoder.java:48) > at > org.apache.kylin.engine.mr.common.BaseCuboidBuilder.(BaseCuboidBuilder.java:84) > at > org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.doSetup(BaseCuboidMapperBase.java:70) > at > org.apache.kylin.engine.mr.steps.HiveToBaseCuboidMapper.doSetup(HiveToBaseCuboidMapper.java:36) > at org.apache.kylin.engine.mr.KylinMapper.setup(KylinMapper.java:48) at > org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at > org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at > org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at > java.security.AccessController.doPrivileged(Native Method) at > javax.security.auth.Subject.doAs(Subject.java:415) at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1707) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: > java.lang.OutOfMemoryError: Requested array size exceeds VM limit at > java.util.Arrays.copyOf(Arrays.java:2271) at > java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113) at > java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93) > at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140) at > org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1793) at > org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769) at > org.apache.commons.io.IOUtils.copy(IOUtils.java:1744) at > org.apache.kylin.common.persistence.FileResourceStore.getResourceImpl(FileResourceStore.java:123) > at > org.apache.kylin.common.persistence.ResourceStore.getResource(ResourceStore.java:154) > at org.apache.kylin.dict.DictionaryManager.load(DictionaryManager.java:418) > at org.apache.kylin.dict.DictionaryManager$1.load(DictionaryManager.java:101) > at org.apache.kylin.dict.DictionaryManager$1.load(DictionaryManager.java:98) > at > com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568) > at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350) > at > com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313) > at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228) at > com.google.common.cache.LocalCache.get(LocalCache.java:3965) at > com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at > com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829) > at > org.apache.kylin.dict.DictionaryManager.getDictionaryInfo(DictionaryManager.java:118) > at org.apache.kylin.cube.CubeManager.getDictionary(CubeManager.java:271) at >
[jira] [Commented] (KYLIN-2683) Support reloading kerberos token of BeelineHiveClient
[ https://issues.apache.org/jira/browse/KYLIN-2683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534476#comment-16534476 ] Kanta Kuramoto commented on KYLIN-2683: --- I'm sorry that the problem doesn't appear in our current environment. I'd like to close this ticket. I will reopen it if I find this problem again. > Support reloading kerberos token of BeelineHiveClient > - > > Key: KYLIN-2683 > URL: https://issues.apache.org/jira/browse/KYLIN-2683 > Project: Kylin > Issue Type: Bug > Components: Security >Reporter: Kanta Kuramoto >Assignee: Kanta Kuramoto >Priority: Minor > Labels: scope, security > Attachments: KYLIN-2683.patch, kerberos_auth.png > > > When the datasource is kerberized, the behavior of reloading kerberos token > is different between "Cube Build" and "Load Hive Table". > I summarized the detail of this behavior in the attached image. > > I think BeelineHiveClient#init shuold be implementad like following. > http://appcrawler.com/wordpress/2015/06/18/examples-of-connecting-to-kerberos-hive-in-jdbc/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (KYLIN-2683) Support reloading kerberos token of BeelineHiveClient
[ https://issues.apache.org/jira/browse/KYLIN-2683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kanta Kuramoto closed KYLIN-2683. - Resolution: Not A Problem > Support reloading kerberos token of BeelineHiveClient > - > > Key: KYLIN-2683 > URL: https://issues.apache.org/jira/browse/KYLIN-2683 > Project: Kylin > Issue Type: Bug > Components: Security >Reporter: Kanta Kuramoto >Assignee: Kanta Kuramoto >Priority: Minor > Labels: scope, security > Attachments: KYLIN-2683.patch, kerberos_auth.png > > > When the datasource is kerberized, the behavior of reloading kerberos token > is different between "Cube Build" and "Load Hive Table". > I summarized the detail of this behavior in the attached image. > > I think BeelineHiveClient#init shuold be implementad like following. > http://appcrawler.com/wordpress/2015/06/18/examples-of-connecting-to-kerberos-hive-in-jdbc/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
