Hi Xiaoxiang My case is not in date range and I need to do daily.
1. How can I automate the build index daily for newest data? 2. Can I apply the above automate process for near realtime or realtime data (load realtime data from Hive into new index/segment) Thank you very much for your help On Thu, 2 Nov 2023 at 12:58 Xiaoxiang Yu <x...@apache.org> wrote: > If the new data 's date range is covered by a segment in your model, you > should refresh your existing segment, refer to : > > > https://kylin.apache.org/5.0/docs/modeling/load_data/segment_operation_settings/intro#segment-operation > . > > If not, create a new segment and build index, refer to : > https://kylin.apache.org/5.0/docs/modeling/load_data/by_date > > ------------------------ > With warm regard > Xiaoxiang Yu > > > > On Thu, Nov 2, 2023 at 11:57 AM Nam Đỗ Duy <na...@vnpay.vn.invalid> wrote: > > > Thank you XiangXiao, I still have 1 question as follows: > > > > When the Hive Datasource to be added with new data, how to reflect those > in > > Cube (index) and query result? > > > > > > On Thu, Nov 2, 2023 at 10:00 AM Xiaoxiang Yu <x...@apache.org> wrote: > > > > > Congratulations, hope you will make good use of the ability of Kylin 5 > > for > > > your use cases. > > > > > > > > > ------------------------ > > > With warm regard > > > Xiaoxiang Yu > > > > > > > > > > > > On Thu, Nov 2, 2023 at 10:50 AM Nam Đỗ Duy <na...@vnpay.vn.invalid> > > wrote: > > > > > >> The query is too fast, less than a second, can you make it a little > bit > > >> slower so that I can see it clearly 😀😀 > > >> [image: image.png] > > >> > > >> On Thu, Nov 2, 2023 at 9:32 AM Nam Đỗ Duy <na...@vnpay.vn> wrote: > > >> > > >>> Thank you Xiaoxiang for the guideline. Will definitely read > > >>> it carefully. Kindly help the following questions: > > >>> > > >>> 1. Computed column > > >>> > > >>> I created a “computed column” and add it to dimensions (among other > > >>> dimensions) > > >>> > > >>> When I use query to select the computed column it returned error > > >>> > > >>> 2. Datatype optimization: will you think that the int be better than > > >>> string for key join columns? > > >>> > > >>> Please advise > > >>> > > >>> > > >>> On Wed, 1 Nov 2023 at 17:32 Xiaoxiang Yu <x...@apache.org> wrote: > > >>> > > >>>> Yes, that is almost correct. > > >>>> > > >>>> If you have a lot of complex queries, and you want to using Kylin 5 > to > > >>>> accelerate them, the recommended steps of mine are as follows: > > >>>> > > >>>> 1. You analyse all queries and collect all join relation/pattern. > > >>>> 2. You create Models for each specific join relation/pattern, with > the > > >>>> join > > >>>> relation you find in above step. > > >>>> 3. You analyse and collect dimensions and measures from all queries, > > and > > >>>> add them to the corresponding Model. > > >>>> 4. You build segments of all Models with proper data range. > > >>>> 5. You turned off the pushdown switch, and sent all queries to > Kylin. > > If > > >>>> there are some queries which failed, fix them. > > >>>> Here are some common situations. > > >>>> 5.1 Join relation/pattern is not matched > > >>>> 5.2 If the join relation is matched, the Model might not contain > > >>>> every > > >>>> column that your query needs, please check kylin.query.log with > > keyword > > >>>> ' > > >>>> unmatched'. > > >>>> 6. (Optional) If you find some of your queries do not exactly match > > with > > >>>> your Index(your query on [colA, colB], but your index contains more > > >>>> columns > > >>>> than colA and colB), you can add some aggregate groups(or smaller > > Table > > >>>> Index) to optimize the query performance. > > >>>> > > >>>> > > >>>> > > >>>> ------------------------ > > >>>> With warm regard > > >>>> Xiaoxiang Yu > > >>>> > > >>>> > > >>>> > > >>>> On Wed, Nov 1, 2023 at 5:57 PM Nam Đỗ Duy <na...@vnpay.vn.invalid> > > >>>> wrote: > > >>>> > > >>>> > Thank you Xiaoxiang, I nearly got to the point. > > >>>> > > > >>>> > So can I interpret that: 1 model equal (~) to a set of Joins of > > >>>> (Dim/Fact) > > >>>> > table, that is to say we need to create several models according > to > > >>>> > multiple kinds of joins queries? > > >>>> > > > >>>> > Best regards > > >>>> > > > >>>> > On Wed, Nov 1, 2023 at 4:50 PM Xiaoxiang Yu <x...@apache.org> > > wrote: > > >>>> > > > >>>> >> Have you ever tried to analyse the reason why your query can not > > hit > > >>>> >> Model 'sample_ssb'? > > >>>> >> It is because the join relation of your query is not suitable for > > the > > >>>> >> join relation/pattern of Model 'sample_ssb'. > > >>>> >> > > >>>> >> Your query used a join relation/pattern like: A inner join B. > > >>>> >> But the Model 'sample_ssb' used a join relation/pattern like : A > > >>>> inner > > >>>> >> join B inner join C. > > >>>> >> > > >>>> >> If you are familiar with the definition of Inner join, you may > know > > >>>> that > > >>>> >> the > > >>>> >> relation/pattern 'A inner join B inner join C' will have a chance > > >>>> >> to lose some rows when compared to pattern 'A inner join B'. > > >>>> >> So the Model 'sample_ssb' will be excluded to serve your query. > > >>>> >> > > >>>> >> That is to say, you need to create a new model that is similar to > > >>>> Model > > >>>> >> 'sample_ssb', > > >>>> >> but with additional tables removed. > > >>>> >> > > >>>> >> > > >>>> >> > > >>>> >> ------------------------ > > >>>> >> With warm regard > > >>>> >> Xiaoxiang Yu > > >>>> >> > > >>>> >> > > >>>> >> > > >>>> >> On Wed, Nov 1, 2023 at 5:21 PM Nam Đỗ Duy <na...@vnpay.vn.invalid > > > > >>>> wrote: > > >>>> >> > > >>>> >>> Hi Xiaoxiang, > > >>>> >>> > > >>>> >>> Thank you very much > > >>>> >>> > > >>>> >>> I have clearer picture of Kylin already thanks to your > > explanation. > > >>>> >>> > > >>>> >>> Now back to the sample project of SSB in attached photo, when I > > run > > >>>> this > > >>>> >>> query with push_down option OFF, why the OLAP error appears, and > > in > > >>>> such > > >>>> >>> case, how to create a new cube for this query? > > >>>> >>> > > >>>> >>> [image: image.png] > > >>>> >>> > > >>>> >>> On Wed, Nov 1, 2023 at 3:49 PM Xiaoxiang Yu <x...@apache.org> > > >>>> wrote: > > >>>> >>> > > >>>> >>>> Here is some of my explanation and it may not be perfect. > > >>>> >>>> Segment in Kylin is part of model/cube pre-computed data, in > most > > >>>> >>>> cases, divided by date column. > > >>>> >>>> > > >>>> >>>> Here is some difference between Segment and Snapshot. > > >>>> >>>> Segment, whose source data comes from one fact table joins some > > >>>> dimension > > >>>> >>>> tables with 'specific date range', is 'precomputed', and will > > >>>> accelerate > > >>>> >>>> complex query. > > >>>> >>>> Snapshot, whose source data comes from one specific dimension > > >>>> table without > > >>>> >>>> specific date range, is "not precomputed", and can join with > > >>>> segments > > >>>> >>>> at runtime . > > >>>> >>>> > > >>>> >>>> - > https://kylin.apache.org/5.0/docs/snapshot/snapshot_management > > >>>> >>>> - > > >>>> >>>> > > >>>> > > > https://kylin.apache.org/5.0/docs/modeling/load_data/segment_operation_settings/intro > > >>>> >>>> > > >>>> >>>> ------------------------ > > >>>> >>>> With warm regard > > >>>> >>>> Xiaoxiang Yu > > >>>> >>>> > > >>>> >>>> > > >>>> >>>> > > >>>> >>>> On Wed, Nov 1, 2023 at 3:53 PM Nam Đỗ Duy <na...@vnpay.vn> > > wrote: > > >>>> >>>> > > >>>> >>>>> Thank you again, very smart of you to automatically select > cube > > >>>> for a > > >>>> >>>>> certain query. Sorry If I ask too much: Is the concept of > > Segment > > >>>> in Kylin > > >>>> >>>>> model similar to Slice-and-Dice concept of Cube, what is the > > >>>> different > > >>>> >>>>> between Kylin Segment and Kylin Snapshot? > > >>>> >>>>> > > >>>> >>>>> PS. I sent you the log files for your help in investigating > why > > my > > >>>> >>>>> cube has not been used. > > >>>> >>>>> > > >>>> >>>>> On Wed, Nov 1, 2023 at 2:36 PM Xiaoxiang Yu <x...@apache.org> > > >>>> wrote: > > >>>> >>>>> > > >>>> >>>>>> I guess there is a misunderstanding from your sentences. > > >>>> >>>>>> > > >>>> >>>>>> -- 'I need to select Cube from a combo box below the query > > >>>> window' > > >>>> >>>>>> It is not right to use 'need', that combo box is for some > > >>>> specific > > >>>> >>>>>> cases(for example, Kylin did not choose a cube which is the > > most > > >>>> >>>>>> efficient), not the most cases. > > >>>> >>>>>> In most cases(both for Kylin 4 and Kylin 5), you don't need > to > > >>>> select > > >>>> >>>>>> a Cube in the combo box, Kylin will do the choice for you. > > >>>> >>>>>> > > >>>> >>>>>> ------------------------ > > >>>> >>>>>> With warm regard > > >>>> >>>>>> Xiaoxiang Yu > > >>>> >>>>>> > > >>>> >>>>>> > > >>>> >>>>>> > > >>>> >>>>>> On Wed, Nov 1, 2023 at 3:24 PM Nam Đỗ Duy > > <na...@vnpay.vn.invalid > > >>>> > > > >>>> >>>>>> wrote: > > >>>> >>>>>> > > >>>> >>>>>>> Hi Xiaoxiang, sorry if I made you confused (Anyway, it is > > just a > > >>>> >>>>>>> question of a beginner) > > >>>> >>>>>>> > > >>>> >>>>>>> "obviously" means "clearly" > > >>>> >>>>>>> > > >>>> >>>>>>> because I need to select Cube from a combo box below the > query > > >>>> window > > >>>> >>>>>>> > > >>>> >>>>>>> Thank you very much > > >>>> >>>>>>> > > >>>> >>>>>>> On Wed, Nov 1, 2023 at 2:20 PM Xiaoxiang Yu < > x...@apache.org> > > >>>> wrote: > > >>>> >>>>>>> > > >>>> >>>>>>>> From my side, I cannot understand why you say Kylin 4 is > > 'very > > >>>> >>>>>>>> obviously'. Can you give an example? > > >>>> >>>>>>>> From the source code, the basic logic of choosing the right > > >>>> >>>>>>>> cube/model are similar. > > >>>> >>>>>>>> ------------------------ > > >>>> >>>>>>>> With warm regard > > >>>> >>>>>>>> Xiaoxiang Yu > > >>>> >>>>>>>> > > >>>> >>>>>>>> > > >>>> >>>>>>>> > > >>>> >>>>>>>> On Wed, Nov 1, 2023 at 3:01 PM Nam Đỗ Duy <na...@vnpay.vn> > > >>>> wrote: > > >>>> >>>>>>>> > > >>>> >>>>>>>>> Thank you for your kind reply, please answer 1 more > question > > >>>> about > > >>>> >>>>>>>>> version 5: > > >>>> >>>>>>>>> > > >>>> >>>>>>>>> In version 4.x we run query against a Cube very obviously, > > >>>> but in > > >>>> >>>>>>>>> version 5, the cube usage is a implication socan you > advise: > > >>>> for a given > > >>>> >>>>>>>>> query, which model will be used, which index (cube) will > be > > >>>> used for this > > >>>> >>>>>>>>> query? > > >>>> >>>>>>>>> > > >>>> >>>>>>>>> Thank you > > >>>> >>>>>>>>> > > >>>> >>>>>>>>> On Wed, Nov 1, 2023 at 1:42 PM Xiaoxiang Yu < > > x...@apache.org> > > >>>> >>>>>>>>> wrote: > > >>>> >>>>>>>>> > > >>>> >>>>>>>>>> 1. How do I measure the size of the index (cube) in > version > > >>>> 5? > > >>>> >>>>>>>>>> You can check storage of specific Indexes from the > Index > > >>>> page. > > >>>> >>>>>>>>>> > > >>>> >>>>>>>>>> > > >>>> > > > https://kylin.apache.org/5.0/docs/modeling/model_design/aggregation_group#view-aggregate-index > > >>>> >>>>>>>>>> or > > >>>> >>>>>>>>>> > > >>>> > > > https://kylin.apache.org/5.0/assets/images/index_1-6ad3f55183d4ed61962359d9408ba192.png > > >>>> >>>>>>>>>> > > >>>> >>>>>>>>>> > > >>>> >>>>>>>>>> 2. How to create the cardinality for each column? > > >>>> >>>>>>>>>> You should check this link : > > >>>> >>>>>>>>>> > > https://kylin.apache.org/5.0/docs/datasource/data_sampling/ > > >>>> . > > >>>> >>>>>>>>>> > > >>>> >>>>>>>>>> 3. In your default project sample named SSB project, you > > have > > >>>> >>>>>>>>>> only 4 simple aggregate group index and no table index as > > in > > >>>> attached file > > >>>> >>>>>>>>>> so what is the best strategy to select index for our > OLAP? > > >>>> >>>>>>>>>> 1. There does exist a 'Base Table Index' by default > > >>>> >>>>>>>>>> actually, its id is 20000000001. > > >>>> >>>>>>>>>> 2. I think it is a good question and Kylin 5 lacks > > such a > > >>>> >>>>>>>>>> guide for better modeling. You are free to ask your > > question > > >>>> to > > >>>> >>>>>>>>>> mailing list and I will try to reply. > > >>>> >>>>>>>>>> > > >>>> >>>>>>>>>> ------------------------ > > >>>> >>>>>>>>>> With warm regard > > >>>> >>>>>>>>>> Xiaoxiang Yu > > >>>> >>>>>>>>>> > > >>>> >>>>>>>>>> > > >>>> >>>>>>>>>> > > >>>> >>>>>>>>>> On Wed, Nov 1, 2023 at 2:12 PM Xiaoxiang Yu < > > x...@apache.org > > >>>> > > > >>>> >>>>>>>>>> wrote: > > >>>> >>>>>>>>>> > > >>>> >>>>>>>>>>> OK, I didn't read all the mail history so I > misunderstand > > >>>> the > > >>>> >>>>>>>>>>> situation. Looks like you need to analyse > > >>>> >>>>>>>>>>> the cause why the query didn't hit the cube correctly. > > >>>> >>>>>>>>>>> > > >>>> >>>>>>>>>>> Please generate query diagnosis package and send it to > me > > >>>> >>>>>>>>>>> privately. I will analyse the query log. > > >>>> >>>>>>>>>>> You can refer to the following steps in screenshots. > > >>>> >>>>>>>>>>> > > >>>> >>>>>>>>>>> [image: image.png] > > >>>> >>>>>>>>>>> > > >>>> >>>>>>>>>>> If the screenshots are not displaying correctly, please > > read > > >>>> >>>>>>>>>>> this guide : > > >>>> >>>>>>>>>>> > > >>>> >>>>>>>>>>> > > >>>> > > > https://kylin.apache.org/5.0/docs/operations/system-operation/diagnosis/#generate-query-diagnosis-package-in-web-ui > > >>>> >>>>>>>>>>> > > >>>> >>>>>>>>>>> By the way, you need to analyse the cause by reading > > >>>> >>>>>>>>>>> kylin.query.log, not the kylin.log, > > >>>> >>>>>>>>>>> refer to > > >>>> >>>>>>>>>>> > > >>>> https://kylin.apache.org/5.0/docs/operations/logs/system_log > > >>>> >>>>>>>>>>> > > >>>> >>>>>>>>>>> ------------------------ > > >>>> >>>>>>>>>>> With warm regard > > >>>> >>>>>>>>>>> Xiaoxiang Yu > > >>>> >>>>>>>>>>> > > >>>> >>>>>>>>>>> > > >>>> >>>>>>>>>>> > > >>>> >>>>>>>>>>> On Wed, Nov 1, 2023 at 12:18 PM Nam Đỗ Duy < > > na...@vnpay.vn> > > >>>> >>>>>>>>>>> wrote: > > >>>> >>>>>>>>>>> > > >>>> >>>>>>>>>>>> Thank you Xiaoxiang for your advice. As my title email > > >>>> shown, I > > >>>> >>>>>>>>>>>> guessed that the OLAP functionalities has not been > > >>>> correctly set up in my > > >>>> >>>>>>>>>>>> computer. > > >>>> >>>>>>>>>>>> > > >>>> >>>>>>>>>>>> The evidence about it is that: when I disable the > > Pushdown > > >>>> >>>>>>>>>>>> option box to use solely the precomputation cube only, > it > > >>>> showed following > > >>>> >>>>>>>>>>>> error: Please kindly advise how to properly build the > > OLAP > > >>>> >>>>>>>>>>>> > > >>>> >>>>>>>>>>>> LIMIT 500": No realization found for OLAPContext, > > >>>> MODEL_UNMATCHED_JOIN, > > >>>> > rel#2240:KapTableScan.OLAP.[](table=[VNEVENT_HIVE_DWH_400MILLION_ROWS, > > >>>> FACTUSEREVENT],ctx=0@null,fields=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, > > >>>> 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]) > > >>>> >>>>>>>>>>>> > > >>>> >>>>>>>>>>>> > > >>>> >>>>>>>>>>>> > > >>>> >>>>>>>>>>>> On Wed, Nov 1, 2023 at 10:40 AM Xiaoxiang Yu < > > >>>> x...@apache.org> > > >>>> >>>>>>>>>>>> wrote: > > >>>> >>>>>>>>>>>> > > >>>> >>>>>>>>>>>>> Hi, > > >>>> >>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>> Yesterday, I tried to see if query pushdown > > functions > > >>>> work > > >>>> >>>>>>>>>>>>> well in the Kylin5 docker, and all of my queries > return > > >>>> proper responses . > > >>>> >>>>>>>>>>>>> After checking your logs from Shaofeng, I found > > these > > >>>> >>>>>>>>>>>>> error messages repeated many times: > > >>>> >>>>>>>>>>>>> 1. 'java.io.IOException: All datanodes > > >>>> >>>>>>>>>>>>> DatanodeInfoWithStorage[127.0.0.1:9866 > > >>>> ,DS-5093899b-06c7-4386-95d5-6fc271d92b52,DISK] > > >>>> >>>>>>>>>>>>> are bad. Aborting...' > > >>>> >>>>>>>>>>>>> 2. 'curator.ConnectionState : Connection timed out > > for > > >>>> >>>>>>>>>>>>> connection string (localhost:2181) and timeout > (15000) / > > >>>> elapsed (41794) > > >>>> >>>>>>>>>>>>> org.apache.curator.CuratorConnectionLossException: > > >>>> >>>>>>>>>>>>> KeeperErrorCode = ConnectionLoss' > > >>>> >>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>> I guess the root cause is that the container > didn't > > >>>> not > > >>>> >>>>>>>>>>>>> have enough resources. I found you query on a table > > called > > >>>> >>>>>>>>>>>>> 'XXX_hive_dwh_400million_rows', looks like you gave a > > >>>> complex query on a > > >>>> >>>>>>>>>>>>> table which contains 400 million rows? > > >>>> >>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>> Since I am the uploader of kylin5 's docker > image, I > > >>>> want > > >>>> >>>>>>>>>>>>> to give some explainment. Kylin5 docker is not a place > > >>>> for performance > > >>>> >>>>>>>>>>>>> benchmarks, it is only for demonstration. It is only > > >>>> allocated with very > > >>>> >>>>>>>>>>>>> little resources(8G memory) if you are using the > default > > >>>> command from > > >>>> >>>>>>>>>>>>> docker hub page. Before I uploaded my image, I only > > >>>> tested my image using > > >>>> >>>>>>>>>>>>> the ssb dataset, which the biggest table only contains > > >>>> about 60k rows. If > > >>>> >>>>>>>>>>>>> you are using a larger dataset and complexer queries, > > you > > >>>> have to scale the > > >>>> >>>>>>>>>>>>> resource properly. Try querying tables which contain > not > > >>>> more than 100k > > >>>> >>>>>>>>>>>>> rows by default. > > >>>> >>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>> Here are some tips which may help you to check if > > the > > >>>> >>>>>>>>>>>>> daemon service is in health status and > > >>>> resources(particularly disk space) > > >>>> >>>>>>>>>>>>> is configured properly. > > >>>> >>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>> 1. Checking HDFS 's web ui( > > >>>> >>>>>>>>>>>>> http://localhost:9870/dfshealth.html#tab-datanode ) > to > > >>>> >>>>>>>>>>>>> confirm whether HDFS service is in 'In service' > status. > > >>>> >>>>>>>>>>>>> 2. Checking Datanode 's log in > > >>>> >>>>>>>>>>>>> > > >>>> `/opt/hadoop-3.2.1/logs/hadoop-root-datanode-Kylin5-Machine.log`, > > check if > > >>>> >>>>>>>>>>>>> there is any error message. Like: cat > > >>>> >>>>>>>>>>>>> > > >>>> /opt/hadoop-3.2.1/logs/hadoop-root-datanode-Kylin5-Machine.log | > grep > > ERROR > > >>>> >>>>>>>>>>>>> | wc -l > > >>>> >>>>>>>>>>>>> 3. Checking if your docker engine is configured > with > > >>>> >>>>>>>>>>>>> enough disk space, if you are using Docker Desktop > like > > >>>> me,please go to > > >>>> >>>>>>>>>>>>> "Settings" - "Resources" - "Advanced", make sure you > > have > > >>>> allocated 40GB+ > > >>>> >>>>>>>>>>>>> disk space to the docker container. > > >>>> >>>>>>>>>>>>> 4. Checking the available disk space of your > > >>>> container by > > >>>> >>>>>>>>>>>>> `df -h`, make sure the 'Use%' of 'overlay' is less > than > > >>>> 60% . > > >>>> >>>>>>>>>>>>> 5. Checking the load average/ cpu usage/ jvm gc. > > Make > > >>>> sure > > >>>> >>>>>>>>>>>>> these metrics are not really high when you send a > query. > > >>>> >>>>>>>>>>>>> ------------------------ > > >>>> >>>>>>>>>>>>> With warm regard > > >>>> >>>>>>>>>>>>> Xiaoxiang Yu > > >>>> >>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>> On Tue, Oct 31, 2023 at 5:13 PM Nam Đỗ Duy > > >>>> >>>>>>>>>>>>> <na...@vnpay.vn.invalid> wrote: > > >>>> >>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>> Hi ShaoFeng > > >>>> >>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>> Thank you very much for your valuable feedback > > >>>> >>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>> I saw the application to be there (if I see it right) > > as > > >>>> in > > >>>> >>>>>>>>>>>>>> the attachment photo. Kindly advise so that I can run > > >>>> this query on OLAP. > > >>>> >>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>> PS. I sent you the log file in private. > > >>>> >>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>> [image: image.png] > > >>>> >>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>> On Tue, Oct 31, 2023 at 3:11 PM ShaoFeng Shi < > > >>>> >>>>>>>>>>>>>> shaofeng...@apache.org> wrote: > > >>>> >>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>>> Can you provide the messages in logs/kylin.log when > > >>>> >>>>>>>>>>>>>>> executing the SQL? and you can also check the Spark > UI > > >>>> from yarn resource > > >>>> >>>>>>>>>>>>>>> manager (there should be one running application > > called > > >>>> Spardar, which is > > >>>> >>>>>>>>>>>>>>> Kylin's backend spark application). If the > application > > >>>> is not there, it may > > >>>> >>>>>>>>>>>>>>> indicates the yarn doesn't have resource to startup > > it. > > >>>> >>>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>>> Best regards, > > >>>> >>>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>>> Shaofeng Shi 史少锋 > > >>>> >>>>>>>>>>>>>>> Apache Kylin PMC, > > >>>> >>>>>>>>>>>>>>> Apache Incubator PMC, > > >>>> >>>>>>>>>>>>>>> Email: shaofeng...@apache.org > > >>>> >>>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>>> Apache Kylin FAQ: > > >>>> >>>>>>>>>>>>>>> > https://kylin.apache.org/docs/gettingstarted/faq.html > > >>>> >>>>>>>>>>>>>>> Join Kylin user mail group: > > >>>> user-subscr...@kylin.apache.org > > >>>> >>>>>>>>>>>>>>> Join Kylin dev mail group: > > >>>> dev-subscr...@kylin.apache.org > > >>>> >>>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>>> Nam Đỗ Duy <na...@vnpay.vn> 于2023年10月31日周二 10:35写道： > > >>>> >>>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>>>> Dear Sir/Madam, > > >>>> >>>>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>>>> I have a fact with 500million rows then I build > > model, > > >>>> >>>>>>>>>>>>>>>> index according to the website help. > > >>>> >>>>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>>>> I chose full incremental because this is the first > > >>>> times I > > >>>> >>>>>>>>>>>>>>>> load data > > >>>> >>>>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>>>> I create both index types Aggregate group index, > > table > > >>>> >>>>>>>>>>>>>>>> index as photo attached. > > >>>> >>>>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>>>> But the query always failed after timeout of 300 > > >>>> seconds (I > > >>>> >>>>>>>>>>>>>>>> run in docker), I dont want to increase the value > of > > >>>> 300 seconds because I > > >>>> >>>>>>>>>>>>>>>> wish the OLAP can run within 1 minutes (is that > > >>>> possible?) > > >>>> >>>>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>>>> It seems that the OLAP function in indexing not > > >>>> working to > > >>>> >>>>>>>>>>>>>>>> speedup the query by precomputed cube. > > >>>> >>>>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>>>> Can you advise to check whether the index did > really > > >>>> work? > > >>>> >>>>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>>>> It is quite urgent task for me so prompt response > is > > >>>> highly > > >>>> >>>>>>>>>>>>>>>> appreciated. > > >>>> >>>>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>>>> Thank you very much > > >>>> >>>>>>>>>>>>>>>> > > >>>> >>>>>>>>>>>>>>> > > >>>> > > >>> > > >
