Hi ShaoFeng,
We are using Hive as data source for pulling in fact data for cube building,
Also the commit id as I see is: 0169dc0950fc278239413cfb71313ff9ae8edf7d
Will also investigate myself, and try to resolve the same and share a patch.
Thanks,
Ketan
> On 22-Dec-2017, at 7:41 PM, ShaoFeng Shi <shaofeng...@apache.org> wrote:
>
> Hi Ketan,
>
> Is your Cube data from Hive or Kafka?
>
> The line number seems couldn't match with the Kylin 2.2 source code; Can
> you double confirm the Kylin version? In Kylin installation folder, there
> is a commit id sha file, please provide that file content to us so we can
> exactly match the line numbers. Thanks.
>
> 2017-12-22 18:12 GMT+08:00 ShaoFeng Shi <shaofeng...@apache.org>:
>
>> It should be a bug; I took a quick look but didn't find the root cause
>> (very busy in the year-end). Could you please investigate it and then
>> contribute a patch? Thanks!
>>
>> 2017-12-22 16:41 GMT+08:00 ketan dikshit <kdcool6...@yahoo.com.invalid>:
>>
>>> Hi Guys,
>>> Any update on this, as I see this is a bug or ?
>>>
>>> We are passing in null as TSRange and the check
>>> Preconditions.checkArgument(tsRange != null); is always failing.
>>>
>>> Help would be appreciated.
>>>
>>> Thanks,
>>> Ketan
>>>
>>>> On 21-Dec-2017, at 3:06 PM, ketan dikshit <kdcool6...@yahoo.com.INVALID>
>>> wrote:
>>>>
>>>> Hi Guys,
>>>> I am using Kylin 2.2 and getting some issues with Kylin auto merge.
>>>> The error log in kylin.log is here:
>>>>
>>>> 2017-12-21 08:31:18,811 INFO [Thread-50937] service.CubeService:552 :
>>> checking keepCubeRetention
>>>> 2017-12-21 08:31:18,811 DEBUG [Thread-50937] model.Segments:201 : Cube
>>> publishers_v4 has 1 building segments
>>>> 2017-12-21 08:31:18,811 ERROR [Thread-50937] service.CacheService:87 :
>>> Error in updateOnNewSegmentReady()
>>>> java.lang.IllegalArgumentException
>>>> at com.google.common.base.Preconditions.checkArgument(Precondit
>>> ions.java:76)
>>>> at org.apache.kylin.cube.CubeManager.mergeSegments(CubeManager.
>>> java:544)
>>>> at org.apache.kylin.rest.service.CubeService.mergeCubeSegment(C
>>> ubeService.java:601)
>>>> at org.apache.kylin.rest.service.CubeService.updateOnNewSegment
>>> Ready(CubeService.java:545)
>>>> at org.apache.kylin.rest.service.CubeService$$FastClassBySpring
>>> CGLIB$$17a07c0e.invoke(<generated>)
>>>> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodPro
>>> xy.java:204)
>>>> at org.springframework.aop.framework.CglibAopProxy$DynamicAdvis
>>> edInterceptor.intercept(CglibAopProxy.java:669)
>>>> at org.apache.kylin.rest.service.CubeService$$EnhancerBySpringC
>>> GLIB$$e4f6b188.updateOnNewSegmentReady(<generated>)
>>>> at org.apache.kylin.rest.service.CacheService$1$1.run(CacheServ
>>> ice.java:85)
>>>>
>>>>
>>>> Auto Merge Thresholds
>>>> 8 (Hours)
>>>> 1 (Days)
>>>> 7 (Days)
>>>> 15 (Days)
>>>>
>>>>
>>>> As I see inside the code, I see,
>>>> private void mergeCubeSegment(String cubeName) {
>>>> CubeInstance cube = getCubeManager().getCube(cubeName);
>>>> if (!cube.needAutoMerge()) {
>>>> return;
>>>> }
>>>>
>>>> synchronized (CubeService.class) {
>>>> try {
>>>> cube = getCubeManager().getCube(cubeName);
>>>> SegmentRange offsets = cube.autoMergeCubeSegments();
>>>> if (offsets != null) {
>>>> CubeSegment newSeg = getCubeManager().mergeSegments(cube,
>>> null, offsets, true);
>>>> logger.debug("Will submit merge job on " + newSeg);
>>>> DefaultChainedExecutable job =
>>> EngineFactory.createBatchMergeJob(newSeg, "SYSTEM");
>>>> getExecutableManager().addJob(job);
>>>> } else {
>>>> logger.debug("Not ready for merge on cube " + cubeName);
>>>> }
>>>> } catch (IOException e) {
>>>> logger.error("Failed to auto merge cube " + cubeName, e);
>>>> }
>>>> }
>>>> }
>>>>
>>>> ie; getCubeManager().mergeSegments(cube, null, offsets, true); is
>>> passing the TSRange as ‘null’;
>>>> to CubeManager:public CubeSegment mergeSegments(CubeInstance cube,
>>> TSRange tsRange, SegmentRange segRange, boolean force) method;
>>>> which is defined as:
>>>> if (cube.getSegments().isEmpty())
>>>> throw new IllegalArgumentException("Cube " + cube + " has no
>>> segments");
>>>>
>>>> checkInputRanges(tsRange, segRange);
>>>> checkBuildingSegment(cube);
>>>> checkCubeIsPartitioned(cube);
>>>>
>>>> if (cube.getSegments().getFirstSegment().isOffsetCube()) {
>>>> // offset cube, merge by date range?
>>>> if (segRange == null && tsRange != null) {
>>>> Pair<CubeSegment, CubeSegment> pair =
>>> cube.getSegments(SegmentStatusEnum.READY)
>>>> .findMergeOffsetsByDateRange(tsRange, Long.MAX_VALUE);
>>>> if (pair == null)
>>>> throw new IllegalArgumentException("Find no segments to
>>> merge by " + tsRange + " for cube " + cube);
>>>> segRange = new SegmentRange(pair.getFirst().getSegRange().start,
>>> pair.getSecond().getSegRange().end);
>>>> }
>>>> tsRange = null;
>>>> Preconditions.checkArgument(segRange != null);
>>>> } else {
>>>> segRange = null;
>>>> Preconditions.checkArgument(tsRange != null);
>>>> }
>>>>
>>>> In my case, this is getting invoked,
>>>> else {
>>>> segRange = null;
>>>> Preconditions.checkArgument(tsRange != null);
>>>> }
>>>>
>>>> and as we have passed in tsRange as NULL , so we get this exception:
>>>> java.lang.IllegalArgumentException
>>>> at com.google.common.base.Preconditions.checkArgument(Precondit
>>> ions.java:76)
>>>> at org.apache.kylin.cube.CubeManager.mergeSegments(CubeManager.
>>> java:544)
>>>> at org.apache.kylin.rest.service.CubeService.mergeCubeSegment(C
>>> ubeService.java:601)
>>>> at org.apache.kylin.rest.service.CubeService.updateOnNewSegment
>>> Ready(CubeService.java:545)
>>>> at org.apache.kylin.rest.service.CubeService$$FastClassBySpring
>>> CGLIB$$17a07c0e.invoke(<generated>)
>>>> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodPro
>>> xy.java:204)
>>>> at org.springframework.aop.framework.CglibAopProxy$DynamicAdvis
>>> edInterceptor.intercept(CglibAopProxy.java:669)
>>>> at org.apache.kylin.rest.service.CubeService$$EnhancerBySpringC
>>> GLIB$$e4f6b188.updateOnNewSegmentReady(<generated>)
>>>> at org.apache.kylin.rest.service.CacheService$1$1.run(CacheServ
>>> ice.java:85)
>>>>
>>>> And the auto merge fails.
>>>>
>>>> Let me know in case I am missing on something here, or a fix maybe.
>>>>
>>>>
>>>> Thanks,
>>>> Ketan
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>> --
>> Best regards,
>>
>> Shaofeng Shi 史少锋
>>
>>
>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
