This is an automated email from the ASF dual-hosted git repository.
eldenmoon pushed a change to branch branch-2.0-var
in repository https://gitbox.apache.org/repos/asf/doris.git
discard f91ba2f0e4b [Bug](variant) make schema update in tablet atomic
discard 089560206a6 set scanner thread name by thread pool, query type and
query id suffix (#25164)
omit 39b83792c78 [feature](autobucekt) Add support autobucket min buckets
config Add support autobucket (#24920)
omit c92ae4c3786 [pick-from](struct) fix struct lost child info
omit f0f0aa6326c [pick-from](deciaml) fix decimal type lost scale and
precision when using get_type_as_type_descriptor
omit 2079172d044 [pick-from](cast) fix jsonb cast crash when function
context is nullptr
omit dfb4a33f6df fix rebase
omit fa083c65d03 [fix](index compaction)skip index compaction when no
output segment
omit 5dbe71cbe8f [improve](scan) support lower the thread priority of scan
thread
omit f31ee10c90f [chore] (compile) fix compile
omit c1d85fd0faf [fix](tablet clone) fix clone backend chose wrong disk
(#23729)
omit 410cb97254d [improvement](create tablet) backend create tablet round
robin among disks (#23218)
omit af10d3bacff [to-pick](fix) fix variant sparse column read with wrong
check, for example inverted index read situation it may result in may
next_batch for one column, so for ExtractReader the variant dst size is
increasing
omit b92d2e523d2 [enhancement](load) log info for rowset
omit 95da8adc97a add config to modify tablet schema recycle interval
omit 1d3bb19bc44 add more gh case
omit f494511f7b4 [to-pick] avoid read schema cache for variant
omit 12d8af92293 [to-pick] finalize src object before doing alignment in
align_variant_by_name_and_type
omit b7e12359a60 [chore](var) fix rebase compile
omit 48237d7bb22 [to-pick](refactor) remove _rows_read from reader make it
more stateless
omit 1faa6e6b140 fix jsonb string cast to string return quoted string like
"a"
omit cb9ac5fa2d2 return error instead of CHECK when row count not matched
after cast from variant inner column
omit 9c4a46642d7 [performace](var) improve insert_from performance when
both src and dst is scalar variant with same type
omit b491a636f51 avoid unnessarary info log
omit 5843105bf15 remove shared reader to avoid complicated shared state
omit 704957a0d3f [to-pick](desc) avoid merge empty schemas when
fetch_remote_tablet_schema
omit 4a09013896a [to-pick] disable schema cache for variant
omit 429904ed299 [Refactor](reader) refactor reader to decouple from
segment iterator
omit 2f8ac913643 add command admin set replica version
omit 6f91dfc376b temp disable twophase opt
omit 86720978863 [to-pick] fix column object with nothing root type lost
num_rows info
omit 815c4a34bcd [to-pick] fix MemoryScratchSink with boolean type and
complex type
omit 9ffd766987a [to-pick] fix hirachinal data read queries
omit db52b302ade fix rebase branch-2.0
omit 211c327fc00 [to-pick] fix variant aligment
omit 6045fc62543 [to-pick](mow) fix variant on merge-on-write model with
partial update
omit 3e668a9534c [refactor](read) seperate hirachinal reader from segment
reader
omit 5aa5fbc8716 chore
omit c1686d0b0df remove flatten variant in olap scanner
omit 76ab0fe1890 [to-pick] fix segment iterator return emtpy row may cause
inpit block's types missmatched, since _current_return_columns may converted to
segment column type which is inconsistent with type from input block
omit 987df374cdf [to-pick]support expr in MemoryScratchSink and make
arrow::Schema recalculate with block info
omit 8bf24594bb1 [to-pick]support convert jsonb to complex type by parsing
omit 05110d6c2d9 [improvement](desc table) add session variable to control
extend variant column (#23147)
omit ad378d73c9e fix merge object caused assertion error
omit 0c613387bde [to-pick]Fix crash when root sparse column encode string
column to jsonb column
omit 5d5fdf8f1ff [to-pick]fix fetch_remote_tablet_schema coordinator flying
rpc request not all joined may cause heap use after free
omit b0f963c9b95 [to-pick]fix topn desc permute with empty block caused
ColumnObject::permute crash
omit 95fec9619cf [to-pick]finalize when ColumnObject::filter
omit c5fc99624ae avoid too may `failed to parse` log when parse variant
failed
omit 8c3eaf4097e [improve](variant)Desc variant (#23111)
omit ce192e81e00 variant cast remove dependancy of schema util
omit 10930c08981 fix clang not used error
omit 3afc5e12e4e modify memtable
omit fb03992ee39 [to-pick]support filter invalid variant rows
omit 36111449485 [to-pick]fix unique key compaction
omit 4506173a0f6 [to-pick]fix no partition serialize object cause be core
dump
omit 79d68703d3c [to-pick]support unique table and modify some log
omit 4883a229e76 [Feature-Variant](Variant Type) support variant type
add 089fa1eb087 [Enhancement](inverted index) fix compound predicates
error (#24300)
add 60be43e0e8b [fix](planner)update explode slot's nullable info in
analyze phase #24879 (#24916)
add 9097525afcb [Fix](inverted index) fix tokenize function coredump
(#24896)
add 65a745fb968 [feature](function) add ip function ipv4numtostring (alias
inet_ntoa) (#20936)
add 20fb8691b09 [bug](scanner) Fix memory out of bound in scanner
scheduler (#24840)
add d865cc8282b [feature](javaudf)support no input java udf (#24457)
add 64466b8f2fb [fix](merge-on-write) MergeIndexDeleteBitmapCalculator
stack overflow (#24913)
add 78ce6b67220 [feature](autobucekt) Add support autobucket min buckets
config Add support autobucket (#24920)
add c55ff1615a0 [improvement](compaction) output tablet_id when be core
dumped. (#24952)
add f792f275c9e [Fix](Planner) disable bitmap type in compare expression
(#24792)
add 7b715fe93a6 [fix](Nereids) could not prune datev1 partition column
(#24959)
add 73331c1fc86 fix fe compile
add da27dce699a [fix](nereids) push down filter through partition topn
(#24944)
add c57dbe33da7 [Bug](pipeline) Fix block reusing for union source
operator (#24977)
add fde77e787ff [fix](Nereids) some expression not cast in InPredicate
(#24987)
add 5140ae86c44 [fix](schema) Table column order is changed if add a
column and do truncate (#24981)
add 8d9a150badd [bug](json)Fix some problems of json function on Nereids
(#24898)
add f8d85d85d2a [Fix](inverted index) make parser mode coarse grained by
default (#24949)
add 7a85f040925 [fix](single-replica-load) fix duplicated done run in
request_slave_tablet_pull_rowset (#25013)
add 0806ead35a7 [Bug](migration) Enhance invalid behavior handling in
`copy_files_to` (#25017)
add 71c69d97d4f fix be ut
add e1dfa47cf50 [fix](planner) createColumnAndViewDefs method use wrong
analyzer (#25005)
add ae923f7094e change branch-2.0 version to 2.0.2-rc05 (#25031)
new 3bb6cc476f4 [Feature-Variant](Variant Type) support variant type
new 15a8e55bc19 [to-pick]support unique table and modify some log
new 47f429eb757 [to-pick]fix no partition serialize object cause be core
dump
new e954c40db16 [to-pick]fix unique key compaction
new f050f89dc3d [to-pick]support filter invalid variant rows
new fccb4751433 modify memtable
new 1d4a4bfb093 fix clang not used error
new 4db7b706511 variant cast remove dependancy of schema util
new a3cf9acfa4e [improve](variant)Desc variant (#23111)
new e366adf8325 avoid too may `failed to parse` log when parse variant
failed
new 7d207723d54 [to-pick]finalize when ColumnObject::filter
new 1c94f7f9521 [to-pick]fix topn desc permute with empty block caused
ColumnObject::permute crash
new 020422b686c [to-pick]fix fetch_remote_tablet_schema coordinator flying
rpc request not all joined may cause heap use after free
new a18eb9df4d9 [to-pick]Fix crash when root sparse column encode string
column to jsonb column
new 6417fd1fece fix merge object caused assertion error
new bbd297dc2b2 [improvement](desc table) add session variable to control
extend variant column (#23147)
new 901518f2770 [to-pick]support convert jsonb to complex type by parsing
new 396ba3eee10 [to-pick]support expr in MemoryScratchSink and make
arrow::Schema recalculate with block info
new 5f6b009e3da [to-pick] fix segment iterator return emtpy row may cause
inpit block's types missmatched, since _current_return_columns may converted to
segment column type which is inconsistent with type from input block
new 0f08ec7eda4 remove flatten variant in olap scanner
new cfdc2c88589 chore
new 13ae48c2186 [refactor](read) seperate hirachinal reader from segment
reader
new 0840802da13 [to-pick](mow) fix variant on merge-on-write model with
partial update
new 727f14015a4 [to-pick] fix variant aligment
new baf88ef4a52 fix rebase branch-2.0
new 3568b1b0585 [to-pick] fix hirachinal data read queries
new 4764485704e [to-pick] fix MemoryScratchSink with boolean type and
complex type
new 864996e2ee2 [to-pick] fix column object with nothing root type lost
num_rows info
new 0d95571efe0 temp disable twophase opt
new 2b4d144bd39 add command admin set replica version
new b1e37f9a379 [Refactor](reader) refactor reader to decouple from
segment iterator
new c3244691f1d [to-pick] disable schema cache for variant
new 5b70dbd92f2 [to-pick](desc) avoid merge empty schemas when
fetch_remote_tablet_schema
new c20ba460c7b remove shared reader to avoid complicated shared state
new 349634508c0 avoid unnessarary info log
new d3135cd2171 [performace](var) improve insert_from performance when
both src and dst is scalar variant with same type
new cbfb4d74097 return error instead of CHECK when row count not matched
after cast from variant inner column
new 7fbd469c6e6 fix jsonb string cast to string return quoted string like
"a"
new 69243eb301f [to-pick](refactor) remove _rows_read from reader make it
more stateless
new a18633aa59f [chore](var) fix rebase compile
new 32cb4967360 [to-pick] finalize src object before doing alignment in
align_variant_by_name_and_type
new fda9cb77e6e [to-pick] avoid read schema cache for variant
new 5a8a49bb9f5 add more gh case
new d1239d30b0d add config to modify tablet schema recycle interval
new b3424767b89 [enhancement](load) log info for rowset
new 0381ad61057 [to-pick](fix) fix variant sparse column read with wrong
check, for example inverted index read situation it may result in may
next_batch for one column, so for ExtractReader the variant dst size is
increasing
new 6ad97b61098 [improvement](create tablet) backend create tablet round
robin among disks (#23218)
new 22a5e262619 [fix](tablet clone) fix clone backend chose wrong disk
(#23729)
new 450594bad58 [chore] (compile) fix compile
new 5a9aad9d775 [improve](scan) support lower the thread priority of scan
thread
new 9988010672b [fix](index compaction)skip index compaction when no
output segment
new 78cc9ad99c8 fix rebase
new 573068e2564 [pick-from](cast) fix jsonb cast crash when function
context is nullptr
new 2cf2d87bc0f [pick-from](deciaml) fix decimal type lost scale and
precision when using get_type_as_type_descriptor
new e5026c5f5c6 [pick-from](struct) fix struct lost child info
new 92f08ef8673 set scanner thread name by thread pool, query type and
query id suffix (#25164)
new 9023cff8dbd [Bug](variant) make schema update in tablet atomic
new 1ca690f3f24 [fix](planner) Fix select table tablet not effective
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (f91ba2f0e4b)
\
N -- N -- N refs/heads/branch-2.0-var (1ca690f3f24)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 58 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/src/common/signal_handler.h | 4 +
be/src/io/fs/local_file_system.cpp | 6 +-
be/src/io/fs/local_file_system.h | 6 +-
be/src/olap/delete_bitmap_calculator.cpp | 4 +-
be/src/olap/rowset/beta_rowset.cpp | 10 +-
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 15 ++-
be/src/olap/schema_change.cpp | 3 +
be/src/olap/tablet.cpp | 2 +
be/src/pipeline/exec/union_source_operator.cpp | 2 +-
be/src/service/internal_service.cpp | 2 +-
be/src/util/jsonb_document.h | 22 ++--
be/src/vec/common/format_ip.cpp | 78 ++++++++++++++
be/src/vec/exec/scan/scanner_scheduler.cpp | 4 +-
be/src/vec/exec/scan/scanner_scheduler.h | 3 +-
.../src/vec/functions/function_ip.cpp | 35 ++----
be/src/vec/functions/function_ip.h | 120 +++++++++++++++++++++
be/src/vec/functions/function_tokenize.cpp | 43 +++-----
be/src/vec/functions/simple_function_factory.h | 2 +
be/test/io/fs/local_file_system_test.cpp | 2 +-
.../sql-functions/ip-functions/INET_NTOA.md | 70 ++++++++++++
.../ip-functions/IPV4-NUM-TO-STRING.md | 70 ++++++++++++
docs/sidebars.json | 8 ++
.../sql-functions/ip-functions/INET_NTOA.md | 71 ++++++++++++
.../ip-functions/IPV4-NUM-TO-STRING.md | 71 ++++++++++++
.../main/java/org/apache/doris/common/Config.java | 30 ++++++
fe/fe-core/src/main/cup/sql_parser.cup | 5 +-
.../org/apache/doris/analysis/BinaryPredicate.java | 2 +-
.../java/org/apache/doris/analysis/CaseExpr.java | 6 ++
.../org/apache/doris/analysis/CreateViewStmt.java | 2 +-
.../main/java/org/apache/doris/analysis/Expr.java | 11 ++
.../org/apache/doris/analysis/InPredicate.java | 1 +
.../apache/doris/analysis/InvertedIndexUtil.java | 2 +-
.../org/apache/doris/analysis/LateralViewRef.java | 8 +-
.../doris/catalog/BuiltinScalarFunctions.java | 2 +-
.../apache/doris/datasource/InternalCatalog.java | 27 +++--
.../glue/translator/ExpressionTranslator.java | 2 +-
.../org/apache/doris/nereids/rules/RuleSet.java | 4 +-
.../org/apache/doris/nereids/rules/RuleType.java | 2 +
.../rules/HiveDefaultPartitionEvaluator.java | 4 +-
.../rules/OneRangePartitionEvaluator.java | 18 ++--
.../rules/expression/rules/PartitionPruner.java | 48 ++++++++-
.../expression/rules/SimplifyInPredicate.java | 27 ++++-
.../rewrite/CreatePartitionTopNFromWindow.java | 7 +-
.../PushdownFilterThroughPartitionTopN.java | 97 +++++++++++++++++
.../expressions/functions/scalar/JsonContains.java | 2 +-
.../expressions/functions/scalar/JsonLength.java | 6 +-
.../trees/expressions/literal/DateTimeLiteral.java | 4 +
.../expressions/literal/StringLikeLiteral.java | 19 ++++
.../org/apache/doris/nereids/types/CharType.java | 6 --
.../org/apache/doris/nereids/types/StringType.java | 6 --
.../apache/doris/nereids/types/VarcharType.java | 6 --
.../doris/nereids/util/TypeCoercionUtils.java | 13 +--
.../org/apache/doris/planner/OlapScanNode.java | 21 +++-
.../org/apache/doris/analysis/SelectStmtTest.java | 29 ++++-
.../nereids/datasets/ssb/SSBJoinReorderTest.java | 4 +-
.../rules/expression/SimplifyInPredicateTest.java | 7 +-
.../doris/nereids/util/TypeCoercionUtilsTest.java | 15 ++-
.../org/apache/doris/planner/QueryPlanTest.java | 2 +-
.../doris/planner/TableFunctionPlanTest.java | 14 +--
gensrc/script/doris_builtins_functions.py | 8 ++
gensrc/script/gen_build_version.sh | 2 +-
.../test_inlineview_with_lateralview.out | 8 ++
.../data/datatype_p0/bitmap/test_bitmap_int.out | Bin 315 -> 350 bytes
.../data/inverted_index_p0/test_tokenize.out | 3 +
.../data/javaudf_p0/test_javaudf_no_input.out | 23 ++++
.../data/nereids_p0/union/test_union.out | 6 ++
.../push_filter_through_ptopn.out | 52 +++++++++
.../ip_functions/test_ip_functions.out | 11 ++
.../java/org/apache/doris/udf/NoInputTest.java | 32 +-----
.../test_inlineview_with_lateralview.groovy | 58 ++++++++++
.../datatype_p0/bitmap/test_bitmap_int.groovy | 33 ++++++
.../suites/inverted_index_p0/test_tokenize.groovy | 2 +
.../suites/javaudf_p0/test_javaudf_no_input.groovy | 75 +++++++++++++
.../jsonb_p0/test_jsonb_load_and_function.groovy | 3 +-
.../test_jsonb_load_unique_key_and_function.groovy | 3 +-
.../nereids_p0/join/bucket_shuffle_join.groovy | 2 +-
.../suites/nereids_p0/union/test_union.groovy | 44 ++++++++
.../push_filter_through_ptopn.groovy | 53 +++++++++
.../ip_functions/test_ip_functions.groovy | 33 ++----
.../json_functions/test_json_function.groovy | 6 +-
regression-test/suites/view_p0/view_p0.groovy | 6 ++
81 files changed, 1329 insertions(+), 246 deletions(-)
create mode 100644 be/src/vec/common/format_ip.cpp
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/literal/StringLikeLiteral.java
=> be/src/vec/functions/function_ip.cpp (52%)
create mode 100644 be/src/vec/functions/function_ip.h
create mode 100644
docs/en/docs/sql-manual/sql-functions/ip-functions/INET_NTOA.md
create mode 100644
docs/en/docs/sql-manual/sql-functions/ip-functions/IPV4-NUM-TO-STRING.md
create mode 100644
docs/zh-CN/docs/sql-manual/sql-functions/ip-functions/INET_NTOA.md
create mode 100644
docs/zh-CN/docs/sql-manual/sql-functions/ip-functions/IPV4-NUM-TO-STRING.md
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PushdownFilterThroughPartitionTopN.java
create mode 100644
regression-test/data/correctness_p0/test_inlineview_with_lateralview.out
create mode 100644 regression-test/data/javaudf_p0/test_javaudf_no_input.out
create mode 100644
regression-test/data/nereids_syntax_p0/push_filter_through_ptopn.out
create mode 100644
regression-test/data/query_p0/sql_functions/ip_functions/test_ip_functions.out
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/literal/StringLikeLiteral.java
=>
regression-test/java-udf-src/src/main/java/org/apache/doris/udf/NoInputTest.java
(53%)
create mode 100644
regression-test/suites/correctness_p0/test_inlineview_with_lateralview.groovy
create mode 100644
regression-test/suites/javaudf_p0/test_javaudf_no_input.groovy
create mode 100644
regression-test/suites/nereids_syntax_p0/push_filter_through_ptopn.groovy
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/literal/StringLikeLiteral.java
=>
regression-test/suites/query_p0/sql_functions/ip_functions/test_ip_functions.groovy
(52%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
