xiaokang commented on code in PR #41482:
URL: https://github.com/apache/doris/pull/41482#discussion_r1821831025
##########
be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:
##########
@@ -665,8 +665,10 @@ Status
VerticalSegmentWriter::_append_block_with_variant_subcolumns(RowsInBlock&
_opts.rowset_ctx->merged_tablet_schema =
_opts.rowset_ctx->tablet_schema;
}
TabletSchemaSPtr update_schema;
+ bool check_schema_size = true;
Review Comment:
unrelated change?
##########
cloud/src/common/config.h:
##########
@@ -135,8 +135,10 @@ CONF_mBool(snapshot_get_tablet_stats, "true");
// Value codec version
CONF_mInt16(meta_schema_value_version, "1");
-// Limit kv size of Schema SchemaDictKeyList, default 10MB
-CONF_mInt32(schema_dict_kv_size_limit, "10485760");
+// Limit kv size of Schema SchemaDictKeyList, default 5MB
+CONF_mInt32(schema_dict_kv_size_limit, "5242880");
Review Comment:
Why change limit smaller?
##########
cloud/src/meta-service/meta_service_schema.cpp:
##########
@@ -331,6 +340,12 @@ void read_schema_from_dict(MetaServiceCode& code,
std::string& msg, const std::s
LOG(INFO) << "Get schema_dict, column size=" << dict.column_dict_size()
<< ", index size=" << dict.index_dict_size();
+ // Return dict, let backend to fill schema with dict info
+ if (schema_op == GetRowsetRequest::RETURN_DICT) {
+ response->mutable_schema_dict()->Swap(&dict);
Review Comment:
If the dict is very large, the response is memory consuming.
Will it still be better than before, if only small ratio of dict is usefull
for the response rowsets?
##########
be/src/cloud/pb_convert.cpp:
##########
@@ -185,6 +221,9 @@ void cloud_rowset_meta_to_doris(RowsetMetaPB* out, const
RowsetMetaCloudPB& in)
if (in.has_tablet_schema()) {
cloud_tablet_schema_to_doris(out->mutable_tablet_schema(),
in.tablet_schema());
}
+ if (dict != nullptr) {
Review Comment:
If dict is nullptr due to some exception and schema is not filled by meta
service, the state is not valid. So you need to check the state if dict is
nullptr.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
