[jira] [Created] (KYLIN-4155) Cube status can not change immediately when executed disable or enable button in web

2019-09-02 Thread luguosheng (Jira) 
luguosheng created KYLIN-4155:
-

 Summary: Cube status can not change immediately when executed 
disable or enable button in web
 Key: KYLIN-4155
 URL: https://issues.apache.org/jira/browse/KYLIN-4155
 Project: Kylin
  Issue Type: Bug
Reporter: luguosheng
Assignee: luguosheng


When user execute disable or enable button in cube list dropdown menu , and 
then api response was back successfully,  cube status tag can not change. 



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[GitHub] [kylin] coveralls commented on issue #821: Update prepare-libs.sh

2019-09-02 Thread GitBox 
coveralls commented on issue #821: Update prepare-libs.sh
URL: https://github.com/apache/kylin/pull/821#issuecomment-527307321
 
 
   ## Pull Request Test Coverage Report for [Build 
4969](https://coveralls.io/builds/25491665)
   
   * **0** of **0**   changed or added relevant lines in **0** files are 
covered.
   * **3** unchanged lines in **3** files lost coverage.
   * Overall coverage remained the same at **28.085%**
   
   ---
   
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-|--|--: |
   | 
[core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/25491665/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L553)
 | 1 | 78.12% |
   | 
[core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/25491665/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62)
 | 1 | 81.08% |
   | 
[core-job/src/main/java/org/apache/kylin/job/impl/curator/CuratorScheduler.java](https://coveralls.io/builds/25491665/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fcurator%2FCuratorScheduler.java#L145)
 | 1 | 68.64% |
   
   
   |  Totals | [![Coverage 
Status](https://coveralls.io/builds/25491665/badge)](https://coveralls.io/builds/25491665)
 |
   | :-- | --: |
   | Change from base [Build 4960](https://coveralls.io/builds/25458324): |  
0.0% |
   | Covered Lines: | 23452 |
   | Relevant Lines: | 83505 |
   
   ---
   #   - [Coveralls](https://coveralls.io)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [kylin] codecov-io commented on issue #821: Update prepare-libs.sh

2019-09-02 Thread GitBox 
codecov-io commented on issue #821: Update prepare-libs.sh
URL: https://github.com/apache/kylin/pull/821#issuecomment-527306637
 
 
   # [Codecov](https://codecov.io/gh/apache/kylin/pull/821?src=pr=h1) Report
   > Merging [#821](https://codecov.io/gh/apache/kylin/pull/821?src=pr=desc) 
into 
[master](https://codecov.io/gh/apache/kylin/commit/de23e721859e45e1db50353d2af53c8267402862?src=pr=desc)
 will **decrease** coverage by `<.01%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/kylin/pull/821/graphs/tree.svg?width=650=JawVgbgsVo=150=pr)](https://codecov.io/gh/apache/kylin/pull/821?src=pr=tree)
   
   ```diff
   @@ Coverage Diff  @@
   ## master #821  +/-   ##
   
   - Coverage 25.63%   25.62%   -0.01% 
   + Complexity 6051 6050   -1 
   
 Files  1398 1398  
 Lines 8350583505  
 Branches  1169711697  
   
   - Hits  2140421396   -8 
   - Misses6005060056   +6 
   - Partials   2051 2053   +2
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/kylin/pull/821?src=pr=tree) | Coverage Δ 
| Complexity Δ | |
   |---|---|---|---|
   | 
[...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/821/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==)
 | `72.97% <0%> (-5.41%)` | `6% <0%> (-1%)` | |
   | 
[...org/apache/kylin/rest/util/QueryRequestLimits.java](https://codecov.io/gh/apache/kylin/pull/821/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvdXRpbC9RdWVyeVJlcXVlc3RMaW1pdHMuamF2YQ==)
 | `35.71% <0%> (-4.77%)` | `5% <0%> (-1%)` | |
   | 
[...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/821/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh)
 | `69.6% <0%> (-1.52%)` | `7% <0%> (ø)` | |
   | 
[...pache/kylin/job/impl/curator/CuratorScheduler.java](https://codecov.io/gh/apache/kylin/pull/821/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9pbXBsL2N1cmF0b3IvQ3VyYXRvclNjaGVkdWxlci5qYXZh)
 | `64.4% <0%> (-0.85%)` | `7% <0%> (ø)` | |
   | 
[...core/storage/columnar/GeneralColumnDataReader.java](https://codecov.io/gh/apache/kylin/pull/821/diff?src=pr=tree#diff-c3RyZWFtLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3N0cmVhbS9jb3JlL3N0b3JhZ2UvY29sdW1uYXIvR2VuZXJhbENvbHVtbkRhdGFSZWFkZXIuamF2YQ==)
 | `94.73% <0%> (+5.26%)` | `7% <0%> (+1%)` | :arrow_up: |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/kylin/pull/821?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/kylin/pull/821?src=pr=footer). Last 
update 
[de23e72...dd9d153](https://codecov.io/gh/apache/kylin/pull/821?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [kylin] asf-ci commented on issue #821: Update prepare-libs.sh

2019-09-02 Thread GitBox 
asf-ci commented on issue #821: Update prepare-libs.sh
URL: https://github.com/apache/kylin/pull/821#issuecomment-527302043
 
 
   Can one of the admins verify this patch?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [kylin] hyunseokJang commented on issue #821: Update prepare-libs.sh

2019-09-02 Thread GitBox 
hyunseokJang commented on issue #821: Update prepare-libs.sh
URL: https://github.com/apache/kylin/pull/821#issuecomment-527302067
 
 
   add.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [kylin] hyunseokJang opened a new pull request #821: Update prepare-libs.sh

2019-09-02 Thread GitBox 
hyunseokJang opened a new pull request #821: Update prepare-libs.sh
URL: https://github.com/apache/kylin/pull/821
 
 
   add lib ( storage stream )


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Resolved] (KYLIN-3625) Query engine for Parquet

2019-09-02 Thread Chao Long (Jira) 


 [ 
https://issues.apache.org/jira/browse/KYLIN-3625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chao Long resolved KYLIN-3625.
--
Resolution: Resolved

> Query engine for Parquet
> 
>
> Key: KYLIN-3625
> URL: https://issues.apache.org/jira/browse/KYLIN-3625
> Project: Kylin
>  Issue Type: Sub-task
>  Components: Query Engine, Storage - Parquet
>Reporter: Shaofeng SHI
>Assignee: Chao Long
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Resolved] (KYLIN-3626) Allow customization for Parquet path

2019-09-02 Thread Chao Long (Jira) 


 [ 
https://issues.apache.org/jira/browse/KYLIN-3626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chao Long resolved KYLIN-3626.
--
Resolution: Resolved

> Allow customization for Parquet path
> 
>
> Key: KYLIN-3626
> URL: https://issues.apache.org/jira/browse/KYLIN-3626
> Project: Kylin
>  Issue Type: Sub-task
>  Components: Job Engine, Metadata, Storage - Parquet
>Reporter: Shaofeng SHI
>Assignee: Chao Long
>Priority: Major
>
> Due to the difference of HDFS and object store like S3/blob store, Kylin 
> needs to allow different path patterns for the files when putting to the 
> underlying storage. 
>  
> For example, to HDFS, this pattern is good:
> /kylin/metadata_name/cube_name/segment_name/file1.parquet
>  
> But, the above pattern is not good for S3, since S3 uses the first couple 
> bytes of the path to do the hashing, so we'd better add salt ahead of the 
> original pathname:
> /salt/kylin/metadata_name/cube_name/segment_name/file1.parquet
> or do a reverse order:
> /segment_name/cube_name/metadata_name/kylin/file1.parquet
>  
> To get the flexibility, we should make this extensible.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Resolved] (KYLIN-3777) Remove dict from query

2019-09-02 Thread Chao Long (Jira) 


 [ 
https://issues.apache.org/jira/browse/KYLIN-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chao Long resolved KYLIN-3777.
--
Resolution: Resolved

> Remove dict from query
> --
>
> Key: KYLIN-3777
> URL: https://issues.apache.org/jira/browse/KYLIN-3777
> Project: Kylin
>  Issue Type: Sub-task
>Reporter: Chao Long
>Assignee: Chao Long
>Priority: Major
>
> To improve query performance, we should remove dict from query side.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Resolved] (KYLIN-4085) Segment parallel building may cause segment not found

2019-09-02 Thread Chao Long (Jira) 


 [ 
https://issues.apache.org/jira/browse/KYLIN-4085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chao Long resolved KYLIN-4085.
--
Resolution: Fixed

> Segment parallel building may cause segment not found
> -
>
> Key: KYLIN-4085
> URL: https://issues.apache.org/jira/browse/KYLIN-4085
> Project: Kylin
>  Issue Type: Bug
>  Components: Metadata
>Reporter: Chao Long
>Assignee: Chao Long
>Priority: Major
> Fix For: v3.0.0-beta
>
>
> In the case of multi-node and parallel building one same cube, the 
> JDBCResourceStore split the update metadata step in two sql, which can't 
> guarantee the atomicity.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Resolved] (KYLIN-4089) Integration test failed with JDBCMetastore

2019-09-02 Thread Chao Long (Jira) 


 [ 
https://issues.apache.org/jira/browse/KYLIN-4089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chao Long resolved KYLIN-4089.
--
Resolution: Fixed

> Integration test failed with JDBCMetastore
> --
>
> Key: KYLIN-4089
> URL: https://issues.apache.org/jira/browse/KYLIN-4089
> Project: Kylin
>  Issue Type: Bug
>  Components: Integration, Tools, Build and Test
>Reporter: Chao Long
>Assignee: Chao Long
>Priority: Major
> Fix For: v3.0.0-beta
>
>
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.822 
> s <<< FAILURE! - in 
> org.apache.kylin.storage.hbase.ITAclTableMigrationToolTest[ERROR] 
> testBasic(org.apache.kylin.storage.hbase.ITAclTableMigrationToolTest)  Time 
> elapsed: 0.812 s  <<< ERROR!java.lang.NullPointerException
>   at 
> org.apache.kylin.storage.hbase.ITAclTableMigrationToolTest.testBasic(ITAclTableMigrationToolTest.java:95)



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[GitHub] [kylin] coveralls commented on issue #820: KYLIN-4148 Execute 'bin/kylin-port-replace-util.sh' to change port wi…

2019-09-02 Thread GitBox 
coveralls commented on issue #820: KYLIN-4148 Execute 
'bin/kylin-port-replace-util.sh' to change port wi…
URL: https://github.com/apache/kylin/pull/820#issuecomment-527277205
 
 
   ## Pull Request Test Coverage Report for [Build 
4967](https://coveralls.io/builds/25490357)
   
   * **0** of **0**   changed or added relevant lines in **0** files are 
covered.
   * **9** unchanged lines in **4** files lost coverage.
   * Overall coverage decreased (**-0.007%**) to **28.077%**
   
   ---
   
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-|--|--: |
   | 
[core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/25490357/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62)
 | 1 | 81.08% |
   | 
[core-job/src/main/java/org/apache/kylin/job/impl/curator/CuratorScheduler.java](https://coveralls.io/builds/25490357/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fcurator%2FCuratorScheduler.java#L145)
 | 1 | 68.64% |
   | 
[core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/25490357/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124)
 | 2 | 68.46% |
   | 
[core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/25490357/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L439)
 | 5 | 77.81% |
   
   
   |  Totals | [![Coverage 
Status](https://coveralls.io/builds/25490357/badge)](https://coveralls.io/builds/25490357)
 |
   | :-- | --: |
   | Change from base [Build 4960](https://coveralls.io/builds/25458324): |  
-0.007% |
   | Covered Lines: | 23446 |
   | Relevant Lines: | 83505 |
   
   ---
   #   - [Coveralls](https://coveralls.io)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (KYLIN-4148) Execute 'bin/kylin-port-replace-util.sh' to change port will cause the configuration of 'kylin.metadata.url' lost

2019-09-02 Thread ASF GitHub Bot (Jira) 


[ 
https://issues.apache.org/jira/browse/KYLIN-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921104#comment-16921104
 ] 

ASF GitHub Bot commented on KYLIN-4148:
---

luguosheng1314 commented on pull request #820: KYLIN-4148 Execute 
'bin/kylin-port-replace-util.sh' to change port wi…
URL: https://github.com/apache/kylin/pull/820
 
 
   …ll cause the configuration of  'kylin.metadata.url' lost
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Execute 'bin/kylin-port-replace-util.sh' to change port will cause the 
> configuration of  'kylin.metadata.url' lost
> --
>
> Key: KYLIN-4148
> URL: https://issues.apache.org/jira/browse/KYLIN-4148
> Project: Kylin
>  Issue Type: Bug
>Reporter: luguosheng
>Assignee: luguosheng
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[GitHub] [kylin] asf-ci commented on issue #820: KYLIN-4148 Execute 'bin/kylin-port-replace-util.sh' to change port wi…

2019-09-02 Thread GitBox 
asf-ci commented on issue #820: KYLIN-4148 Execute 
'bin/kylin-port-replace-util.sh' to change port wi…
URL: https://github.com/apache/kylin/pull/820#issuecomment-527272200
 
 
   Can one of the admins verify this patch?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [kylin] luguosheng1314 opened a new pull request #820: KYLIN-4148 Execute 'bin/kylin-port-replace-util.sh' to change port wi…

2019-09-02 Thread GitBox 
luguosheng1314 opened a new pull request #820: KYLIN-4148 Execute 
'bin/kylin-port-replace-util.sh' to change port wi…
URL: https://github.com/apache/kylin/pull/820
 
 
   …ll cause the configuration of  'kylin.metadata.url' lost


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (KYLIN-4143) truncate spark executable job output

2019-09-02 Thread ZhouKang (Jira) 


 [ 
https://issues.apache.org/jira/browse/KYLIN-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhouKang updated KYLIN-4143:

Attachment: KYLIN-4143.master.001.patch

> truncate spark executable job output 
> -
>
> Key: KYLIN-4143
> URL: https://issues.apache.org/jira/browse/KYLIN-4143
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v2.5.2
>Reporter: ZhouKang
>Priority: Major
> Attachments: KYLIN-4143.master.001.patch
>
>
>  
> truncate spark job's output when the job exec ret is not equal 0, which made 
> the execute output content too large.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Updated] (KYLIN-4143) truncate spark executable job output

2019-09-02 Thread ZhouKang (Jira) 


 [ 
https://issues.apache.org/jira/browse/KYLIN-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhouKang updated KYLIN-4143:

Description: 
 

truncate spark job's output when the job exec ret is not equal 0, which made 
the execute output content too large.

  was:
 

truncate spark job's output when the job exec ret is not equal 0, which made 
the execute output is too large.


> truncate spark executable job output 
> -
>
> Key: KYLIN-4143
> URL: https://issues.apache.org/jira/browse/KYLIN-4143
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v2.5.2
>Reporter: ZhouKang
>Priority: Major
>
>  
> truncate spark job's output when the job exec ret is not equal 0, which made 
> the execute output content too large.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Updated] (KYLIN-4143) truncate spark executable job output

2019-09-02 Thread ZhouKang (Jira) 


 [ 
https://issues.apache.org/jira/browse/KYLIN-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhouKang updated KYLIN-4143:

Description: 
 

truncate spark job's output when the job exec ret is not equal 0, which made 
the execute output is too large.

  was:
Kylin: 2.5.2

job engine: spark

 

After long time running, job executor will record plenty of content in job 
output file, which make ExecutableManager get output fail.  And the job will be 
marked as "error".
{code:java}
// code placeholder
2019-08-15 11:58:34,442 ERROR [Scheduler 1278051386 Job 
db9b01d0-fd8c-d168-8cfa-2e11d807972c-128] spark.SparkExecutable:359 : error run 
spark job:
 java.lang.RuntimeException: 
org.apache.kylin.job.exception.PersistentException: 
com.fasterxml.jackson.databind.JsonMappingException: Unexpected end-of-input in 
VALUE_STRING
 at [Source: (DataInputStream); line: 5, column: 10728969]
 at [Source: (DataInputStream); line: 5, column: 15] (through reference chain: 
org.apache.kylin.job.dao.ExecutableOutputPO["content"])
 at 
org.apache.kylin.job.execution.ExecutableManager.getOutput(ExecutableManager.java:164)
 at 
org.apache.kylin.job.execution.AbstractExecutable.getOutput(AbstractExecutable.java:414)
 at 
org.apache.kylin.job.execution.AbstractExecutable.isDiscarded(AbstractExecutable.java:525)
 at 
org.apache.kylin.engine.spark.SparkExecutable.doWork(SparkExecutable.java:304)
 at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
 at 
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
 at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
 at 
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)
 Caused by: org.apache.kylin.job.exception.PersistentException: 
com.fasterxml.jackson.databind.JsonMappingException: Unexpected end-of-input in 
VALUE_STRING
 at [Source: (DataInputStream); line: 5, column: 10728969]
 at [Source: (DataInputStream); line: 5, column: 15] (through reference chain: 
org.apache.kylin.job.dao.ExecutableOutputPO["content"])
 at org.apache.kylin.job.dao.ExecutableDao.getJobOutput(ExecutableDao.java:356)
 at 
org.apache.kylin.job.execution.ExecutableManager.getOutput(ExecutableManager.java:159)

 

... 10 more
 Caused by: com.fasterxml.jackson.databind.JsonMappingException: Unexpected 
end-of-input in VALUE_STRING
 at [Source: (DataInputStream); line: 5, column: 10728969]
 at [Source: (DataInputStream); line: 5, column: 15] (through reference chain: 
org.apache.kylin.job.dao.ExecutableOutputPO["content"])
 at 
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:391)
 at 
com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:351)
 at 
com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapAndThrow(BeanDeserializerBase.java:1704)
 at 
com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:290)
 at 
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
 at 
com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
 at 
com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3058)
 at org.apache.kylin.common.util.JsonUtil.readValue(JsonUtil.java:69)
 at 
org.apache.kylin.common.persistence.JsonSerializer.deserialize(JsonSerializer.java:46)
 at 
org.apache.kylin.common.persistence.ResourceStore.getResource(ResourceStore.java:182)
{code}
 


> truncate spark executable job output 
> -
>
> Key: KYLIN-4143
> URL: https://issues.apache.org/jira/browse/KYLIN-4143
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v2.5.2
>Reporter: ZhouKang
>Priority: Major
>
>  
> truncate spark job's output when the job exec ret is not equal 0, which made 
> the execute output is too large.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Updated] (KYLIN-4143) truncate spark executable job output

2019-09-02 Thread ZhouKang (Jira) 


 [ 
https://issues.apache.org/jira/browse/KYLIN-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhouKang updated KYLIN-4143:

Summary: truncate spark executable job output   (was: Get job status fail 
when too much content in job output)

> truncate spark executable job output 
> -
>
> Key: KYLIN-4143
> URL: https://issues.apache.org/jira/browse/KYLIN-4143
> Project: Kylin
>  Issue Type: Bug
>Affects Versions: v2.5.2
>Reporter: ZhouKang
>Priority: Major
>
> Kylin: 2.5.2
> job engine: spark
>  
> After long time running, job executor will record plenty of content in job 
> output file, which make ExecutableManager get output fail.  And the job will 
> be marked as "error".
> {code:java}
> // code placeholder
> 2019-08-15 11:58:34,442 ERROR [Scheduler 1278051386 Job 
> db9b01d0-fd8c-d168-8cfa-2e11d807972c-128] spark.SparkExecutable:359 : error 
> run spark job:
>  java.lang.RuntimeException: 
> org.apache.kylin.job.exception.PersistentException: 
> com.fasterxml.jackson.databind.JsonMappingException: Unexpected end-of-input 
> in VALUE_STRING
>  at [Source: (DataInputStream); line: 5, column: 10728969]
>  at [Source: (DataInputStream); line: 5, column: 15] (through reference 
> chain: org.apache.kylin.job.dao.ExecutableOutputPO["content"])
>  at 
> org.apache.kylin.job.execution.ExecutableManager.getOutput(ExecutableManager.java:164)
>  at 
> org.apache.kylin.job.execution.AbstractExecutable.getOutput(AbstractExecutable.java:414)
>  at 
> org.apache.kylin.job.execution.AbstractExecutable.isDiscarded(AbstractExecutable.java:525)
>  at 
> org.apache.kylin.engine.spark.SparkExecutable.doWork(SparkExecutable.java:304)
>  at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
>  at 
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
>  at 
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
>  at 
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at java.lang.Thread.run(Thread.java:748)
>  Caused by: org.apache.kylin.job.exception.PersistentException: 
> com.fasterxml.jackson.databind.JsonMappingException: Unexpected end-of-input 
> in VALUE_STRING
>  at [Source: (DataInputStream); line: 5, column: 10728969]
>  at [Source: (DataInputStream); line: 5, column: 15] (through reference 
> chain: org.apache.kylin.job.dao.ExecutableOutputPO["content"])
>  at 
> org.apache.kylin.job.dao.ExecutableDao.getJobOutput(ExecutableDao.java:356)
>  at 
> org.apache.kylin.job.execution.ExecutableManager.getOutput(ExecutableManager.java:159)
>  
> ... 10 more
>  Caused by: com.fasterxml.jackson.databind.JsonMappingException: Unexpected 
> end-of-input in VALUE_STRING
>  at [Source: (DataInputStream); line: 5, column: 10728969]
>  at [Source: (DataInputStream); line: 5, column: 15] (through reference 
> chain: org.apache.kylin.job.dao.ExecutableOutputPO["content"])
>  at 
> com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:391)
>  at 
> com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:351)
>  at 
> com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapAndThrow(BeanDeserializerBase.java:1704)
>  at 
> com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:290)
>  at 
> com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
>  at 
> com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
>  at 
> com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3058)
>  at org.apache.kylin.common.util.JsonUtil.readValue(JsonUtil.java:69)
>  at 
> org.apache.kylin.common.persistence.JsonSerializer.deserialize(JsonSerializer.java:46)
>  at 
> org.apache.kylin.common.persistence.ResourceStore.getResource(ResourceStore.java:182)
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Created] (KYLIN-4154) Metadata inconsistency between multi Kylin server caused by Broadcaster closing

2019-09-02 Thread Chao Long (Jira) 
Chao Long created KYLIN-4154:


 Summary: Metadata inconsistency between multi Kylin server caused 
by Broadcaster closing 
 Key: KYLIN-4154
 URL: https://issues.apache.org/jira/browse/KYLIN-4154
 Project: Kylin
  Issue Type: Bug
  Components: Metadata
Reporter: Chao Long
Assignee: Chao Long
 Fix For: v3.0.0


To avoid Broadcaster memory leak, KYLIN-4131 close the metadata sync thread 
after receiving "Sync All" event, but there may be some events in the waiting 
queue, which haven't been synced yet. So we should sync all events before 
closing sync thread.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[GitHub] [kylin] coveralls commented on issue #819: Kylin 3392

2019-09-02 Thread GitBox 
coveralls commented on issue #819: Kylin 3392
URL: https://github.com/apache/kylin/pull/819#issuecomment-527038341
 
 
   ## Pull Request Test Coverage Report for [Build 
4966](https://coveralls.io/builds/25476126)
   
   * **78** of **129**   **(60.47%)**  changed or added relevant lines in 
**18** files are covered.
   * **11** unchanged lines in **7** files lost coverage.
   * Overall coverage increased (+**0.09%**) to **28.177%**
   
   ---
   
   |  Changes Missing Coverage | Covered Lines | Changed/Added Lines | % |
   | :-|--||---: |
   | 
[core-metadata/src/main/java/org/apache/kylin/measure/basic/DoubleMaxAggregator.java](https://coveralls.io/builds/25476126/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmeasure%2Fbasic%2FDoubleMaxAggregator.java#L50)
 | 7 | 8 | 87.5%
   | 
[core-metadata/src/main/java/org/apache/kylin/measure/basic/DoubleMinAggregator.java](https://coveralls.io/builds/25476126/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmeasure%2Fbasic%2FDoubleMinAggregator.java#L50)
 | 7 | 8 | 87.5%
   | 
[core-metadata/src/main/java/org/apache/kylin/measure/basic/DoubleSumAggregator.java](https://coveralls.io/builds/25476126/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmeasure%2Fbasic%2FDoubleSumAggregator.java#L50)
 | 9 | 10 | 90.0%
   | 
[core-metadata/src/main/java/org/apache/kylin/measure/basic/LongIngester.java](https://coveralls.io/builds/25476126/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmeasure%2Fbasic%2FLongIngester.java#L36)
 | 0 | 1 | 0.0%
   | 
[core-metadata/src/main/java/org/apache/kylin/measure/basic/LongMaxAggregator.java](https://coveralls.io/builds/25476126/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmeasure%2Fbasic%2FLongMaxAggregator.java#L50)
 | 7 | 8 | 87.5%
   | 
[core-metadata/src/main/java/org/apache/kylin/measure/basic/LongMinAggregator.java](https://coveralls.io/builds/25476126/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmeasure%2Fbasic%2FLongMinAggregator.java#L50)
 | 7 | 8 | 87.5%
   | 
[core-metadata/src/main/java/org/apache/kylin/measure/basic/LongSumAggregator.java](https://coveralls.io/builds/25476126/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmeasure%2Fbasic%2FLongSumAggregator.java#L50)
 | 9 | 10 | 90.0%
   | 
[core-metadata/src/main/java/org/apache/kylin/metadata/datatype/BigDecimalSerializer.java](https://coveralls.io/builds/25476126/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmetadata%2Fdatatype%2FBigDecimalSerializer.java#L77)
 | 7 | 8 | 87.5%
   | 
[core-metadata/src/main/java/org/apache/kylin/measure/basic/BigDecimalMaxAggregator.java](https://coveralls.io/builds/25476126/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmeasure%2Fbasic%2FBigDecimalMaxAggregator.java#L45)
 | 4 | 6 | 66.67%
   | 
[core-metadata/src/main/java/org/apache/kylin/measure/basic/BigDecimalMinAggregator.java](https://coveralls.io/builds/25476126/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmeasure%2Fbasic%2FBigDecimalMinAggregator.java#L45)
 | 4 | 6 | 66.67%
   
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-|--|--: |
   | 
[core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://coveralls.io/builds/25476126/source?filename=core-dictionary%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fdict%2Flookup%2Fcache%2FRocksDBLookupTable.java#L62)
 | 1 | 81.08% |
   | 
[core-job/src/main/java/org/apache/kylin/job/impl/curator/CuratorScheduler.java](https://coveralls.io/builds/25476126/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2Fimpl%2Fcurator%2FCuratorScheduler.java#L145)
 | 1 | 68.64% |
   | 
[core-metadata/src/main/java/org/apache/kylin/measure/basic/DoubleIngester.java](https://coveralls.io/builds/25476126/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmeasure%2Fbasic%2FDoubleIngester.java#L38)
 | 1 | 0.0% |
   | 
[core-common/src/main/java/org/apache/kylin/common/util/BytesUtil.java](https://coveralls.io/builds/25476126/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Futil%2FBytesUtil.java#L232)
 | 2 | 26.04% |
   | 
[core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/25476126/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124)
 | 2 | 68.46% |
   | 
[core-metadata/src/main/java/org/apache/kylin/measure/basic/BigDecimalIngester.java](https://coveralls.io/builds/25476126/source?filename=core-metadata%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fmeasure%2Fbasic%2FBigDecimalIngester.java#L31)
 | 2 | 0.0% |
   |

[GitHub] [kylin] codecov-io commented on issue #819: Kylin 3392

2019-09-02 Thread GitBox 
codecov-io commented on issue #819: Kylin 3392
URL: https://github.com/apache/kylin/pull/819#issuecomment-527037206
 
 
   # [Codecov](https://codecov.io/gh/apache/kylin/pull/819?src=pr=h1) Report
   > Merging [#819](https://codecov.io/gh/apache/kylin/pull/819?src=pr=desc) 
into 
[master](https://codecov.io/gh/apache/kylin/commit/de23e721859e45e1db50353d2af53c8267402862?src=pr=desc)
 will **increase** coverage by `0.07%`.
   > The diff coverage is `48.06%`.
   
   [![Impacted file tree 
graph](https://codecov.io/gh/apache/kylin/pull/819/graphs/tree.svg?width=650=JawVgbgsVo=150=pr)](https://codecov.io/gh/apache/kylin/pull/819?src=pr=tree)
   
   ```diff
   @@ Coverage Diff @@
   ## master#819  +/-   ##
   ===
   + Coverage 25.63%   25.7%   +0.07% 
   - Complexity 60516104  +53 
   ===
 Files  13981399   +1 
 Lines 83505   83581  +76 
 Branches  11697   11716  +19 
   ===
   + Hits  21404   21483  +79 
   + Misses60050   60033  -17 
   - Partials   20512065  +14
   ```
   
   
   | [Impacted 
Files](https://codecov.io/gh/apache/kylin/pull/819?src=pr=tree) | Coverage Δ 
| Complexity Δ | |
   |---|---|---|---|
   | 
[...org/apache/kylin/measure/basic/DoubleIngester.java](https://codecov.io/gh/apache/kylin/pull/819/diff?src=pr=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWVhc3VyZS9iYXNpYy9Eb3VibGVJbmdlc3Rlci5qYXZh)
 | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: |
   | 
[...rg/apache/kylin/measure/basic/CountAggregator.java](https://codecov.io/gh/apache/kylin/pull/819/diff?src=pr=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWVhc3VyZS9iYXNpYy9Db3VudEFnZ3JlZ2F0b3IuamF2YQ==)
 | `0% <0%> (ø)` | `0 <0> (?)` | |
   | 
[...apache/kylin/measure/basic/BigDecimalIngester.java](https://codecov.io/gh/apache/kylin/pull/819/diff?src=pr=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWVhc3VyZS9iYXNpYy9CaWdEZWNpbWFsSW5nZXN0ZXIuamF2YQ==)
 | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: |
   | 
[...g/apache/kylin/measure/basic/BasicMeasureType.java](https://codecov.io/gh/apache/kylin/pull/819/diff?src=pr=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWVhc3VyZS9iYXNpYy9CYXNpY01lYXN1cmVUeXBlLmphdmE=)
 | `10.16% <0%> (-0.36%)` | `1 <0> (ø)` | |
   | 
[...a/org/apache/kylin/measure/basic/LongIngester.java](https://codecov.io/gh/apache/kylin/pull/819/diff?src=pr=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWVhc3VyZS9iYXNpYy9Mb25nSW5nZXN0ZXIuamF2YQ==)
 | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: |
   | 
[...apache/kylin/metadata/datatype/LongSerializer.java](https://codecov.io/gh/apache/kylin/pull/819/diff?src=pr=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWV0YWRhdGEvZGF0YXR5cGUvTG9uZ1NlcmlhbGl6ZXIuamF2YQ==)
 | `38.46% <100%> (+38.46%)` | `3 <2> (+3)` | :arrow_up: |
   | 
[...ache/kylin/metadata/datatype/DoubleSerializer.java](https://codecov.io/gh/apache/kylin/pull/819/diff?src=pr=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWV0YWRhdGEvZGF0YXR5cGUvRG91YmxlU2VyaWFsaXplci5qYXZh)
 | `73.33% <100%> (+73.33%)` | `5 <4> (+5)` | :arrow_up: |
   | 
[...n/java/org/apache/kylin/common/util/BytesUtil.java](https://codecov.io/gh/apache/kylin/pull/819/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi91dGlsL0J5dGVzVXRpbC5qYXZh)
 | `23.26% <15.38%> (-1.74%)` | `27 <1> (ø)` | |
   | 
[...e/kylin/measure/basic/BigDecimalMinAggregator.java](https://codecov.io/gh/apache/kylin/pull/819/diff?src=pr=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWVhc3VyZS9iYXNpYy9CaWdEZWNpbWFsTWluQWdncmVnYXRvci5qYXZh)
 | `75% <50%> (+47.22%)` | `9 <2> (+6)` | :arrow_up: |
   | 
[...e/kylin/measure/basic/BigDecimalMaxAggregator.java](https://codecov.io/gh/apache/kylin/pull/819/diff?src=pr=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vbWVhc3VyZS9iYXNpYy9CaWdEZWNpbWFsTWF4QWdncmVnYXRvci5qYXZh)
 | `75% <50%> (+47.22%)` | `9 <2> (+6)` | :arrow_up: |
   | ... and [27 
more](https://codecov.io/gh/apache/kylin/pull/819/diff?src=pr=tree-more) | |
   
   --
   
   [Continue to review full report at 
Codecov](https://codecov.io/gh/apache/kylin/pull/819?src=pr=continue).
   > **Legend** - [Click here to learn 
more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute  (impact)`, `ø = not affected`, `? = missing data`
   > Powered by 
[Codecov](https://codecov.io/gh/apache/kylin/pull/819?src=pr=footer). Last 
update 
[de23e72...3e84b43](https://codecov.io/gh/apache/kylin/pull/819?src=pr=lastupdated).
 Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

[jira] [Commented] (KYLIN-3392) support NULL value in Sum, Max, Min Aggregation

2019-09-02 Thread Shaofeng SHI (Jira) 


[ 
https://issues.apache.org/jira/browse/KYLIN-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920660#comment-16920660
 ] 

Shaofeng SHI commented on KYLIN-3392:
-

Related PR: [https://github.com/apache/kylin/pull/819]

> support NULL value in Sum, Max, Min Aggregation
> ---
>
> Key: KYLIN-3392
> URL: https://issues.apache.org/jira/browse/KYLIN-3392
> Project: Kylin
>  Issue Type: Bug
>Reporter: Yifei Wu
>Assignee: Yifei Wu
>Priority: Major
>
> It is treated as 0 when confronted with NULL value in KYLIN's basic aggregate 
> measure (like sum, max, min). However, to distinguish the NULL value with 0 
> is very necessary.
> It should be like this
> *sum(null, null) = null*
> *sum(null, 1) = 1*
> *max(null, null) = null*
> *max(null, -1) = -1*
> *min(null,  -1）= -1*
>  in accordance with Hive and SparkSQL



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[GitHub] [kylin] asf-ci commented on issue #819: Kylin 3392

2019-09-02 Thread GitBox 
asf-ci commented on issue #819: Kylin 3392
URL: https://github.com/apache/kylin/pull/819#issuecomment-527029728
 
 
   Can one of the admins verify this patch?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [kylin] vaful opened a new pull request #819: Kylin 3392

2019-09-02 Thread GitBox 
vaful opened a new pull request #819: Kylin 3392
URL: https://github.com/apache/kylin/pull/819
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (KYLIN-3976) Failed to find metadata store by url: kylin_metadata@hbase

2019-09-02 Thread Yaqian Zhang (Jira) 


[ 
https://issues.apache.org/jira/browse/KYLIN-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920649#comment-16920649
 ] 

Yaqian Zhang commented on KYLIN-3976:
-

Hello Xihao:

       Maybe this article can help you: 
[https://zhuanlan.zhihu.com/p/62187552|https://zhuanlan.zhihu.com/p/62187552.]

       BTW, For such questions, you can send an email to the kylin mail group 
for help.       
       Join Kylin user mail group: 
[user-subscr...@kylin.apache.org|mailto:user-subscr...@kylin.apache.org]
       Join Kylin dev mail group: [dev-subscr...@kylin.apache.org 
|mailto:dev-subscr...@kylin.apache.org]
       

> Failed to find metadata store by url: kylin_metadata@hbase
> --
>
> Key: KYLIN-3976
> URL: https://issues.apache.org/jira/browse/KYLIN-3976
> Project: Kylin
>  Issue Type: Bug
>  Components: Metadata
> Environment: 
> kylin2.3.2+hadoop3.1.2+hive2.3.4+hbase1.2.0+zookeeper3.4.5
>Reporter: ZhangXihao
>Priority: Blocker
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> 2019-04-24 17:30:01,894 DEBUG [main] hbase.HBaseConnection:337 : Creating 
> HTable 'kylin_metadata'
> 2019-04-24 17:30:01,936 INFO [main] Configuration.deprecation:1174 : 
> hadoop.native.lib is deprecated. Instead, use io.native.lib.available
> 2019-04-24 17:30:02,089 DEBUG [main] util.ZookeeperDistributedLock:223 : 
> 7116@ambari02 trying to unlock 
> /kylin/kylin_metadata/create_htable/kylin_metadata/lock
> 2019-04-24 17:30:02,099 INFO [main] util.ZookeeperDistributedLock:234 : 
> 7116@ambari02 released lock at 
> /kylin/kylin_metadata/create_htable/kylin_metadata/lock
> Exception in thread "main" java.lang.IllegalArgumentException: Failed to find 
> metadata store by url: kylin_metadata@hbase
>  at 
> org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:96)
>  at 
> org.apache.kylin.common.persistence.ResourceStore.getStore(ResourceStore.java:108)
>  at 
> org.apache.kylin.rest.service.AclTableMigrationTool.checkIfNeedMigrate(AclTableMigrationTool.java:94)
>  at 
> org.apache.kylin.tool.AclTableMigrationCLI.main(AclTableMigrationCLI.java:41)
> Caused by: java.lang.reflect.InvocationTargetException
>  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>  at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>  at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>  at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>  at 
> org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:90)
>  ... 3 more
> Caused by: java.lang.NoSuchMethodError: 
> org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)Lorg/apache/hadoop/hbase/HTableDescriptor;
>  at 
> org.apache.kylin.storage.hbase.HBaseConnection.createHTableIfNeeded(HBaseConnection.java:344)
>  at 
> org.apache.kylin.storage.hbase.HBaseResourceStore.createHTableIfNeeded(HBaseResourceStore.java:111)
>  at 
> org.apache.kylin.storage.hbase.HBaseResourceStore.(HBaseResourceStore.java:92)
>  ... 8 more
> 2019-04-24 17:30:02,105 INFO [close-hbase-conn] hbase.HBaseConnection:137 : 
> Closing HBase connections...
> 2019-04-24 17:30:02,107 INFO [close-hbase-conn] 
> client.ConnectionManager$HConnectionImplementation:1830 : Closing zookeeper 
> sessionid=0x16a4c1bd0c80008
> 2019-04-24 17:30:02,111 INFO [close-hbase-conn] zookeeper.ZooKeeper:684 : 
> Session: 0x16a4c1bd0c80008 closed
> 2019-04-24 17:30:02,111 INFO [main-EventThread] zookeeper.ClientCnxn:512 : 
> EventThread shut down
> 2019-04-24 17:30:02,114 INFO [Thread-7] zookeeper.ZooKeeper:684 : Session: 
> 0x16a4c1bd0c80009 closed
> 2019-04-24 17:30:02,114 INFO [main-EventThread] zookeeper.ClientCnxn:512 : 
> EventThread shut down



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Assigned] (KYLIN-4080) Project schema update event casues error reload NEW DataModelDesc

2019-09-02 Thread Yaqian Zhang (Jira) 


 [ 
https://issues.apache.org/jira/browse/KYLIN-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yaqian Zhang reassigned KYLIN-4080:
---

Assignee: Yaqian Zhang

> Project schema update event casues error reload NEW DataModelDesc
> -
>
> Key: KYLIN-4080
> URL: https://issues.apache.org/jira/browse/KYLIN-4080
> Project: Kylin
>  Issue Type: Bug
>  Components: Metadata
>Affects Versions: v2.5.2
>Reporter: Yuzhang QIU
>Assignee: Yaqian Zhang
>Priority: Blocker
>
> Hi, dear Kylin dev team:
>When create new DataModelDesc, DataModelManager.createDataModelDese:246 
> will temporarily add the new model name into selected project(project1) 
> cache, but won't persist it. The TEMPORARY ADD operation will make the model 
> reloading successful, rather than throw "No project found for model ..." 
> exception(at ProjectManager:391).
>However, If there have another threads are processing  "Broadcasting 
> update project_schema, project1", it will clean up cache of project1 and 
> reload it, which will reset the "TEMPORARY ADD" operation. Meanwhile, the 
> model saving thread has persisted the DataModelDesc and start to reload it, 
> but will find there have "No project for this model".
>   The new model can't be created again because the conflict timestamp and 
> can't be reloaded into cache because the abrove problem. 
>How do you think about this??
>   
>Best regards
>   
>yuzhang



--
This message was sent by Atlassian Jira
(v8.3.2#803003)