[
https://issues.apache.org/jira/browse/KYLIN-5603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17739518#comment-17739518
]
huangsheng commented on KYLIN-5603:
-----------------------------------
h1. Fix Design
# The V3 dictionary is changed from project level to project level, which is
consistent with the V2 dictionary
#
org.apache.kylin.engine.spark.builder.v3dict.DictionaryBuilder#buildGlobalDict
method adds parameter dbName, dictionary storage path adds DB information
h2. Test Evidence
# create a model：
{code:java}
select count(distinct v_revenue) as revenue
from p_lineorder
left join dates on lo_orderdate = d_datekey
where d_year = 1993
and lo_discount between 1 and 3
and lo_quantity < 25; {code}
# Build the V2 dictionary !image-2023-07-03-17-23-36-745.png!
# Upgrade the V2 dictionary to V3 with the following conf:
{code:java}
kylin.build.is-v3dict-enable:true
kylin.build.is-v2dict-enable:false
kylin.build.is-convert-v3dict-enable:true {code}
V3 dictionary datas: !image-2023-07-03-17-28-10-272.png!
# Global level V3 upgrade project level V3
## First use the old version of KE to build the V3 dictionary. Generate
dictionary file: /k_team/mming0929/dict/global_dict_v3/P_LINEORDER/V_REVENUE
## Build the model with the new version. Generate dictionary file:
/k_team/mming0929/DictV3Upgrade/dict/global_dict_v3/SSB10/P_LINEORDER/V_REVENUE
!image-2023-07-03-17-30-18-755.png!
## Key Log Information: v3dict.DictionaryBuilder : V3 Dict build mode is
V3UPGRADE
> After the V2 dictionary is automatically upgraded to the V3 dictionary, the
> dictionary data is abnormal, resulting in incorrect query results
> ---------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: KYLIN-5603
> URL: https://issues.apache.org/jira/browse/KYLIN-5603
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
> Affects Versions: 5.0-alpha
> Reporter: huangsheng
> Assignee: huangsheng
> Priority: Major
> Fix For: 5.0-alpha
>
> Attachments: image-2023-07-03-17-23-36-745.png,
> image-2023-07-03-17-28-10-272.png, image-2023-07-03-17-30-18-755.png
>
>
> After v2 upgrades the v3 dictionary, if v2 and v3 are mixed, the query result
> is incorrect
> Root Cause
> The project opens the v3 dictionary construction, but when the v3 dictionary
> construction reads the v2 dictionary, the db name is not included in the
> path, so the v2 dictionary will not be read, causing the v2 upgrade and the
> v3 dictionary conversion step to be skipped directly, and v3 is equivalent to
> recoding. Therefore, the v2 upgrade v3 dictionary must not be available, and
> the encoding is disordered.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
