[GitHub] [incubator-doris] kangkaisen commented on pull request #3703: [Load] Add more metric to trace the time cost in stream load and make brpc_num_threads configurable

2020-05-29 Thread GitBox
kangkaisen commented on pull request #3703: URL: https://github.com/apache/incubator-doris/pull/3703#issuecomment-636276709 @caiconghui Hi, If you only change the webserver_num_workers from 5 to 48, how many times stream load performance will improve?

[GitHub] [incubator-doris] kangkaisen closed issue #3530: Group by query will core in Debug mode BE

2020-05-29 Thread GitBox
kangkaisen closed issue #3530: URL: https://github.com/apache/incubator-doris/issues/3530 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [incubator-doris] kangkaisen commented on a change in pull request #3722: [Bug] Fix bug that runningprofile show time problem in FE web page and add the runingprofile doc

2020-05-29 Thread GitBox
kangkaisen commented on a change in pull request #3722: URL: https://github.com/apache/incubator-doris/pull/3722#discussion_r432806386 ## File path: docs/zh-CN/administrator-guide/running-profile.md ## @@ -0,0 +1,151 @@ +--- +{ +"title": "查询执行的统计", +"language": "zh-CN"

[GitHub] [incubator-doris] morningman commented on a change in pull request #3724: Fix large string val allocation failure

2020-05-29 Thread GitBox
morningman commented on a change in pull request #3724: URL: https://github.com/apache/incubator-doris/pull/3724#discussion_r432806370 ## File path: be/src/exprs/bitmap_function.cpp ## @@ -488,7 +488,8 @@ StringVal BitmapFunctions::bitmap_from_string(FunctionContext* ctx,

[GitHub] [incubator-doris] morningman commented on pull request #3723: Fix UT ThreadPoolManagerTest failure

2020-05-29 Thread GitBox
morningman commented on pull request #3723: URL: https://github.com/apache/incubator-doris/pull/3723#issuecomment-636268081 > 700 is larger the task sleep time 500. I see, thant u~ This is an automated message from

[GitHub] [incubator-doris] morningman commented on pull request #3731: [Bug][FsBroker] NPE throw when username is empty

2020-05-29 Thread GitBox
morningman commented on pull request #3731: URL: https://github.com/apache/incubator-doris/pull/3731#issuecomment-636267883 > empty user_name is valid? Not valid, but is OK. This is just a self self-protection.

[GitHub] [incubator-doris] kangkaisen commented on pull request #3731: [Bug][FsBroker] NPE throw when username is empty

2020-05-29 Thread GitBox
kangkaisen commented on pull request #3731: URL: https://github.com/apache/incubator-doris/pull/3731#issuecomment-636266561 empty user_name is valid? This is an automated message from the Apache Git Service. To respond to

[GitHub] [incubator-doris] kangkaisen commented on pull request #3723: Fix UT ThreadPoolManagerTest failure

2020-05-29 Thread GitBox
kangkaisen commented on pull request #3723: URL: https://github.com/apache/incubator-doris/pull/3723#issuecomment-636265936 > > ``` > > java.lang.AssertionError: expected:<0> but was:<1> > > at org.apache.doris.common.ThreadPoolManagerTest.testNormal(ThreadPoolManagerTest.java:64)

[GitHub] [incubator-doris] chaoyli merged pull request #3690: [compaction] Update cumulative point calculate algorithm

2020-05-29 Thread GitBox
chaoyli merged pull request #3690: URL: https://github.com/apache/incubator-doris/pull/3690 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[incubator-doris] branch master updated (7524c5e -> 43d25af)

2020-05-29 Thread lichaoyong
This is an automated email from the ASF dual-hosted git repository. lichaoyong pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 7524c5e [Memory Engine] Add MemSubTablet, MemTablet, WriteTx, PartialRowBatch (#3637) add

[incubator-doris] branch master updated: Fix UT ThreadPoolManagerTest failure (#3723)

2020-05-29 Thread lichaoyong
This is an automated email from the ASF dual-hosted git repository. lichaoyong pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 5cb4063 Fix UT

[GitHub] [incubator-doris] chaoyli merged pull request #3723: Fix UT ThreadPoolManagerTest failure

2020-05-29 Thread GitBox
chaoyli merged pull request #3723: URL: https://github.com/apache/incubator-doris/pull/3723 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-doris] chaoyli merged pull request #3637: [Memory Engine] Add MemSubTablet, MemTablet, WriteTx, PartialRowBatch

2020-05-29 Thread GitBox
chaoyli merged pull request #3637: URL: https://github.com/apache/incubator-doris/pull/3637 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[incubator-doris] branch master updated: [Memory Engine] Add MemSubTablet, MemTablet, WriteTx, PartialRowBatch (#3637)

2020-05-29 Thread lichaoyong
This is an automated email from the ASF dual-hosted git repository. lichaoyong pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 7524c5e [Memory Engine] Add

[GitHub] [incubator-doris] morningman commented on a change in pull request #3716: [Spark load] Fe submit spark etl job

2020-05-29 Thread GitBox
morningman commented on a change in pull request #3716: URL: https://github.com/apache/incubator-doris/pull/3716#discussion_r432800096 ## File path: fe/src/main/java/org/apache/doris/catalog/OlapTable.java ## @@ -530,6 +530,30 @@ public KeysType getKeysType() { return

[GitHub] [incubator-doris] morningman commented on a change in pull request #3715: [Spark load] Fe create job

2020-05-29 Thread GitBox
morningman commented on a change in pull request #3715: URL: https://github.com/apache/incubator-doris/pull/3715#discussion_r432798579 ## File path: fe/src/main/java/org/apache/doris/common/Config.java ## @@ -491,6 +491,12 @@ @ConfField(mutable = true, masterOnly = true)

[GitHub] [incubator-doris] wyb commented on a change in pull request #3715: [Spark load] Fe create job

2020-05-29 Thread GitBox
wyb commented on a change in pull request #3715: URL: https://github.com/apache/incubator-doris/pull/3715#discussion_r432797917 ## File path: fe/src/main/java/org/apache/doris/analysis/ResourceDesc.java ## @@ -0,0 +1,121 @@ +// Licensed to the Apache Software Foundation (ASF)

[GitHub] [incubator-doris] wyb commented on a change in pull request #3715: [Spark load] Fe create job

2020-05-29 Thread GitBox
wyb commented on a change in pull request #3715: URL: https://github.com/apache/incubator-doris/pull/3715#discussion_r432797382 ## File path: fe/src/main/java/org/apache/doris/analysis/BrokerDesc.java ## @@ -17,61 +17,36 @@ package org.apache.doris.analysis; -import

[GitHub] [incubator-doris] wyb commented on a change in pull request #3715: [Spark load] Fe create job

2020-05-29 Thread GitBox
wyb commented on a change in pull request #3715: URL: https://github.com/apache/incubator-doris/pull/3715#discussion_r432797335 ## File path: fe/src/main/java/org/apache/doris/load/loadv2/JobState.java ## @@ -21,6 +21,7 @@ public enum JobState { UNKNOWN, // this is only

[GitHub] [incubator-doris] wyb edited a comment on issue #3433: [Spark load] Doris support Spark load

2020-05-29 Thread GitBox
wyb edited a comment on issue #3433: URL: https://github.com/apache/incubator-doris/issues/3433#issuecomment-621142283 **Design doc** #2855 [Proposal] support spark load #2887 [Proposal] Support Spark Convert Doris Segment #3010 Spark load interface

[GitHub] [incubator-doris] wyb edited a comment on issue #3433: [Spark load] Doris support Spark load

2020-05-29 Thread GitBox
wyb edited a comment on issue #3433: URL: https://github.com/apache/incubator-doris/issues/3433#issuecomment-635180109 **Resource manager** #3418 [Spark load] Add resource manager (Merged) **Fe schedule job execution** #3712 [Spark load] Add spark etl job config #3718

[GitHub] [incubator-doris] wyb edited a comment on issue #3433: [Spark load] Doris support Spark load

2020-05-29 Thread GitBox
wyb edited a comment on issue #3433: URL: https://github.com/apache/incubator-doris/issues/3433#issuecomment-621142283 design doc #2855 [Proposal] support spark load #2887 [Proposal] Support Spark Convert Doris Segment #3010 Spark load interface

[GitHub] [incubator-doris] wyb edited a comment on issue #3433: [Spark load] Doris support Spark load

2020-05-29 Thread GitBox
wyb edited a comment on issue #3433: URL: https://github.com/apache/incubator-doris/issues/3433#issuecomment-635180109 #3418 [Spark load] Add resource manager (Merged) #3712 [Spark load] Add spark etl job config #3718 [Spark load] Update push task thrift interface #3715

[GitHub] [incubator-doris] wyb removed a comment on issue #3433: [Spark load] Doris support Spark load

2020-05-29 Thread GitBox
wyb removed a comment on issue #3433: URL: https://github.com/apache/incubator-doris/issues/3433#issuecomment-635310967 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-doris] wyb edited a comment on issue #3433: [Spark load] Doris support Spark load

2020-05-29 Thread GitBox
wyb edited a comment on issue #3433: URL: https://github.com/apache/incubator-doris/issues/3433#issuecomment-635180109 #3418 [Spark load] Add resource manager (finished) #3712 [Spark load] Add spark etl job config #3718 [Spark load] Update push task thrift interface #3715

[GitHub] [incubator-doris] wyb removed a comment on issue #3433: [Spark load] Doris support Spark load

2020-05-29 Thread GitBox
wyb removed a comment on issue #3433: URL: https://github.com/apache/incubator-doris/issues/3433#issuecomment-635181748 Add resource manager #3418 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [incubator-doris] morningman commented on pull request #3604: (#3464) fix Query failed when fact table has no data in join case

2020-05-29 Thread GitBox
morningman commented on pull request #3604: URL: https://github.com/apache/incubator-doris/pull/3604#issuecomment-635962189 > I think we can set `partition join` as default when we're unable to estimate the cost accurately and broadcast join cost equals partition join cost and without

[GitHub] [incubator-doris] morningman commented on pull request #3723: Fix UT ThreadPoolManagerTest failure

2020-05-29 Thread GitBox
morningman commented on pull request #3723: URL: https://github.com/apache/incubator-doris/pull/3723#issuecomment-635959969 > ``` > java.lang.AssertionError: expected:<0> but was:<1> >at org.apache.doris.common.ThreadPoolManagerTest.testNormal(ThreadPoolManagerTest.java:64) >

[GitHub] [incubator-doris] morningman opened a new pull request #3731: [Bug][FsBroker] NPE throw when username is empty

2020-05-29 Thread GitBox
morningman opened a new pull request #3731: URL: https://github.com/apache/incubator-doris/pull/3731 When using Broker with an empty username, a NPE is thrown, which is not expected. Fix: #3730 This is an automated

[GitHub] [incubator-doris] morningman opened a new issue #3730: [Bug][Broker] Null pointer exception when using broker

2020-05-29 Thread GitBox
morningman opened a new issue #3730: URL: https://github.com/apache/incubator-doris/issues/3730 **Describe the bug** When using broker with an empty username, NPE is thrown `IllegalArgumentException: Null user` **To Reproduce** create a broker load with broker property:

[GitHub] [incubator-doris] wangbo edited a comment on issue #3433: [Spark load] Doris support Spark load

2020-05-29 Thread GitBox
wangbo edited a comment on issue #3433: URL: https://github.com/apache/incubator-doris/issues/3433#issuecomment-626104019 **Count Distinct Module** #3319 Support Java Verision HyperLogLog(REVIEWING) #3061 Doris Support Using Hive Table to Build Global Dict(TESTING) #3088 Support

[GitHub] [incubator-doris] wangbo opened a new pull request #3729: (#3728) Using SparkDpp to complete some calculation in Spark Load

2020-05-29 Thread GitBox
wangbo opened a new pull request #3729: URL: https://github.com/apache/incubator-doris/pull/3729 see #3728 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [incubator-doris] wangbo opened a new issue #3728: [Spark Load] Using SparkDpp to complete ETL in Spark Load

2020-05-29 Thread GitBox
wangbo opened a new issue #3728: URL: https://github.com/apache/incubator-doris/issues/3728 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-doris] chaoyli closed issue #3442: [Memory Engine] Add MemTablet type to TabletMeta

2020-05-29 Thread GitBox
chaoyli closed issue #3442: URL: https://github.com/apache/incubator-doris/issues/3442 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[incubator-doris] branch master updated (93aae6b -> c967eaf)

2020-05-29 Thread lichaoyong
This is an automated email from the ASF dual-hosted git repository. lichaoyong pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 93aae6b [Bug] fix mixed used of counter (#3720) add c967eaf [Memory Engine] Add TabletType to

[GitHub] [incubator-doris] chaoyli merged pull request #3668: [Memory Engine] Add TabletType to PartitionInfo and TabletMeta

2020-05-29 Thread GitBox
chaoyli merged pull request #3668: URL: https://github.com/apache/incubator-doris/pull/3668 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-doris] wangbo opened a new pull request #3727: (#3726) [Spark Load] Rollup Tree Builder

2020-05-29 Thread GitBox
wangbo opened a new pull request #3727: URL: https://github.com/apache/incubator-doris/pull/3727 see #3726 1 A tree data structure to describe doris table's rollup 2 A builder to build the data structure This is an

[GitHub] [incubator-doris] wangbo opened a new issue #3726: [Spark Load] Rollup Tree Builder

2020-05-29 Thread GitBox
wangbo opened a new issue #3726: URL: https://github.com/apache/incubator-doris/issues/3726 In Spark Load,we want to aggregate doris table's rollup table. First,we need a data structure to describe the ```derivative relationship``` (such as ```rollup A``` can be calculated from

[GitHub] [incubator-doris] vagetablechicken opened a new issue #3725: Convert shouldn't core in HttpHandlers

2020-05-29 Thread GitBox
vagetablechicken opened a new issue #3725: URL: https://github.com/apache/incubator-doris/issues/3725 **Describe the bug** We use std conversion funcs, but don't catch the exception.

[GitHub] [incubator-doris] wangbo opened a new pull request #3063: (#3061) [Spark Load] Doris Support Using Hive Table to Build Global Dict

2020-05-29 Thread GitBox
wangbo opened a new pull request #3063: URL: https://github.com/apache/incubator-doris/pull/3063 #3061 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [incubator-doris] wangbo closed pull request #3063: (#3061) [Spark Load] Doris Support Using Hive Table to Build Global Dict

2020-05-29 Thread GitBox
wangbo closed pull request #3063: URL: https://github.com/apache/incubator-doris/pull/3063 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-doris] EmmyMiao87 commented on pull request #3601: fix large string val allocation failure, #3600

2020-05-29 Thread GitBox
EmmyMiao87 commented on pull request #3601: URL: https://github.com/apache/incubator-doris/pull/3601#issuecomment-635895100 This pr has some error. The correct pr is here #3724 This is an automated message from the Apache

[GitHub] [incubator-doris] EmmyMiao87 closed pull request #3601: fix large string val allocation failure, #3600

2020-05-29 Thread GitBox
EmmyMiao87 closed pull request #3601: URL: https://github.com/apache/incubator-doris/pull/3601 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [incubator-doris] EmmyMiao87 opened a new pull request #3724: Fix large string val allocation failure

2020-05-29 Thread GitBox
EmmyMiao87 opened a new pull request #3724: URL: https://github.com/apache/incubator-doris/pull/3724 Large bitmap will need use StringVal to allocate large memory, which is large than MAX_INT. The overflow will cause serialization failure of bitmap. Fixed #3600 Change-Id:

[GitHub] [incubator-doris] decster opened a new pull request #3723: Fix UT ThreadPoolManagerTest failure

2020-05-29 Thread GitBox
decster opened a new pull request #3723: URL: https://github.com/apache/incubator-doris/pull/3723 ``` java.lang.AssertionError: expected:<0> but was:<1> at org.apache.doris.common.ThreadPoolManagerTest.testNormal(ThreadPoolManagerTest.java:64) - ```

[GitHub] [incubator-doris] chaoyli closed issue #3721: mix used of counter

2020-05-29 Thread GitBox
chaoyli closed issue #3721: URL: https://github.com/apache/incubator-doris/issues/3721 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [incubator-doris] chaoyli merged pull request #3720: [Bug] fix mixed used of counter

2020-05-29 Thread GitBox
chaoyli merged pull request #3720: URL: https://github.com/apache/incubator-doris/pull/3720 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[incubator-doris] branch master updated: [Bug] fix mixed used of counter (#3720)

2020-05-29 Thread lichaoyong
This is an automated email from the ASF dual-hosted git repository. lichaoyong pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 93aae6b [Bug] fix mixed used of

[GitHub] [incubator-doris] EmmyMiao87 opened a new issue #3344: [Proposal] Materialized View 2.0

2020-05-29 Thread GitBox
EmmyMiao87 opened a new issue #3344: URL: https://github.com/apache/incubator-doris/issues/3344 # The status of materialized view 1.0 In the present, we have supported the materialized views in Doris 0.12 version. The materialized view selector supports to select the most efficient