[kudu] branch branch-1.16.x updated: [docs] Add release notes for 1.16.0
This is an automated email from the ASF dual-hosted git repository. abukor pushed a commit to branch branch-1.16.x in repository https://gitbox.apache.org/repos/asf/kudu.git The following commit(s) were added to refs/heads/branch-1.16.x by this push: new bc7d073 [docs] Add release notes for 1.16.0 bc7d073 is described below commit bc7d073e3984004ad41229204ad1bfd46fb20011 Author: Attila Bukor AuthorDate: Fri Feb 25 15:04:00 2022 +0100 [docs] Add release notes for 1.16.0 Change-Id: I0beb2e182af261ce785cfe21c7de34ca953e6a32 Reviewed-on: http://gerrit.cloudera.org:8080/18275 Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin Reviewed-by: Andrew Wong --- docs/release_notes.adoc | 156 +--- 1 file changed, 147 insertions(+), 9 deletions(-) diff --git a/docs/release_notes.adoc b/docs/release_notes.adoc index 39f8387..21e6cd3 100644 --- a/docs/release_notes.adoc +++ b/docs/release_notes.adoc @@ -31,27 +31,156 @@ [[rn_1.16.0_upgrade_notes]] == Upgrade Notes - -[[rn_1.16.0_obsoletions]] -== Obsoletions - - [[rn_1.16.0_deprecations]] == Deprecations - -Support for Python 2.x and Python 3.4 and earlier is deprecated and may be removed in the next minor release. +* Support for Python 2.x and Python 3.4 and earlier is deprecated and may be + removed in the next minor release. [[rn_1.16.0_new_features]] == New features +* Clients can now require authentication and encryption instead of depending on + server-side settings + (link:https://issues.apache.org/jira/browse/KUDU-1921[KUDU-1921]). + +* Kudu Masters now automatically attempt to add themselves to an existing + cluster if there is a healthy Raft quorum among Kudu Masters. + +* A new tool `kudu master unsafe_rebuild` is added to reconstruct the master + catalog from tablet metadata collected from tablet servers. This can be used + in emergencies to restore access to tables when all masters are unavailable. + +* A new tool `kudu table set_replication_factor` is added to alter the + replication factor of a table. The tool immediately updates table metadata in + the master, and the master will asynchronously effect the new replication + factor. Progress can be monitored by running `ksck`. + +* It’s now possible to require a minimum replication factor for a Kudu table. + This can be achieved by customizing the setting for the newly introduced + `--min_num_replicas` kudu-master’s flag. For example, setting + `--min_num_replicas=3` enforces every newly created table to have at least 3 + replicas for each of its tablets, so there cannot be a data loss when just a + single tablet server in the cluster fails irrecoverably. For the sake of + backward compatibility, `--min_num_replicas` is set to 1 by default. + +* It’s now possible to track startup progress on the `/startup` page on the web + UI. There are also metrics added to track the overall server startup progress + as well as the processing of the log block containers and starting of the + tablets (link:https://issues.apache.org/jira/browse/KUDU-1959[KUDU-1959]). + +* A new tool `kudu table add_column` is added to add columns to existing tables + using the CLI + (link:https://issues.apache.org/jira/browse/KUDU-3339[KUDU-3339]). + +* A new tool `kudu tserver unregister` is added to remove a dead tablet server + from the cluster without restarting the masters + (link:https://issues.apache.org/jira/browse/KUDU-2915[KUDU-2915]). + [[rn_1.16.0_improvements]] == Optimizations and improvements +* Kudu will now more aggressively fsync consensus-related metadata when metadata + is configured to be on an XFS mount. This may lead to increased contention on + the device that backs metadata, but will prevent corruption in the event of an + outage (link:https://issues.apache.org/jira/browse/KUDU-2195[KUDU-2195]). + +* A clearer message is logged when the Ranger subprocess crashes, to specify a + problem with the Ranger client. + +* Two new flags have been introduced for the `kudu table scan` and `kudu perf + table_scan` CLI tools: `--row_count_only` and `--report_scanner_stats`. With + these new flags, the above mentioned CLI tools allow to issue scan requests + equivalent to running “SELECT COUNT(1) FROM ” from impala-shell. + These new provisions are useful in detecting and troubleshooting scan + performance issues. + +* Added replica selection configuration knob for the `kudu table scan` and `kudu + perf table_scan` CLI tools: it’s controlled by the `--replica_selection` flag. + +* To improve security, the following flags are now marked as sensitive and will + be redacted in the logs and WebUI when the redaction is enabled: + ** `--webserver_private_key_file` + ** `--webserver_private_key_password_cmd` + ** `--webserver_password_file` + +* The logic to select the effective time source when running with + `--time_source=auto` has been updated. The `builtin` time source would be + auto-
[kudu] branch branch-1.16.x updated: Bump version to 1.16.0
This is an automated email from the ASF dual-hosted git repository. abukor pushed a commit to branch branch-1.16.x in repository https://gitbox.apache.org/repos/asf/kudu.git The following commit(s) were added to refs/heads/branch-1.16.x by this push: new 3bc7f4e Bump version to 1.16.0 3bc7f4e is described below commit 3bc7f4e834597bc0610a0b30caef003cd68ac120 Author: Attila Bukor AuthorDate: Thu Mar 3 18:46:49 2022 +0100 Bump version to 1.16.0 Change-Id: I5999b9c9d19ddb4549478dfe7531b17f9162360c Reviewed-on: http://gerrit.cloudera.org:8080/18287 Reviewed-by: Alexey Serbin Tested-by: Attila Bukor --- version.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/version.txt b/version.txt index 026c697..15b989e 100644 --- a/version.txt +++ b/version.txt @@ -1 +1 @@ -1.16.0-SNAPSHOT +1.16.0
[kudu] 02/02: [thirdparty] build curl without brotli
This is an automated email from the ASF dual-hosted git repository.
alexey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit 1750aedbbe20134b7dec52cd52ac6c2e330d77e0
Author: Alexey Serbin
AuthorDate: Thu Mar 3 14:39:34 2022 -0800
[thirdparty] build curl without brotli
The default build configuration for curl has the brotli compression
opportunistically enabled. If brotli libraries and headers are found
at the build machine, the configure script picks them up and then the
generated curl library depends on corresponding brotli libraries. That
on itself requires extra brotli libraries to be linked in when building
statically linked kudu binaries (that's so for RELEASE builds).
This patch disables broli for the libcurl library build as a part of
Kudu's thirdparty.
Change-Id: I14ef696302cc1653f3efd6dba5714ce3927c1d20
Reviewed-on: http://gerrit.cloudera.org:8080/18288
Tested-by: Alexey Serbin
Reviewed-by: Andrew Wong
---
thirdparty/build-definitions.sh | 1 +
1 file changed, 1 insertion(+)
diff --git a/thirdparty/build-definitions.sh b/thirdparty/build-definitions.sh
index 7a1ce24..d851e32 100644
--- a/thirdparty/build-definitions.sh
+++ b/thirdparty/build-definitions.sh
@@ -742,6 +742,7 @@ build_curl() {
--disable-smtp \
--disable-telnet \
--disable-tftp \
+--without-brotli \
--without-libidn2 \
--without-libpsl \
--without-librtmp \
[kudu] branch master updated (43905d8 -> 1750aed)
This is an automated email from the ASF dual-hosted git repository. alexey pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/kudu.git. from 43905d8 [security] avoid allocating string in SimpleAcl::UserAllowed() new f4b6d89 KUDU-3197 [tserver] optimal Schema's memory used, using std::shared_ptr new 1750aed [thirdparty] build curl without brotli The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: src/kudu/client/client-test.cc | 10 ++--- src/kudu/common/schema.h | 1 + src/kudu/integration-tests/linked_list-test-util.h | 2 +- src/kudu/master/sys_catalog.cc | 5 ++- src/kudu/tablet/all_types-scan-correctness-test.cc | 2 +- src/kudu/tablet/cfile_set.cc | 4 +- src/kudu/tablet/cfile_set.h| 2 +- src/kudu/tablet/diff_scan-test.cc | 6 +-- src/kudu/tablet/diskrowset.cc | 8 ++-- src/kudu/tablet/mt-tablet-test.cc | 2 +- src/kudu/tablet/ops/alter_schema_op.cc | 10 ++--- src/kudu/tablet/ops/alter_schema_op.h | 8 ++-- src/kudu/tablet/ops/op.h | 4 +- src/kudu/tablet/ops/write_op.h | 7 ++- src/kudu/tablet/rowset_metadata.h | 2 +- src/kudu/tablet/tablet-schema-test.cc | 10 ++--- src/kudu/tablet/tablet-test-util.h | 3 +- src/kudu/tablet/tablet.cc | 47 +++- src/kudu/tablet/tablet.h | 6 +-- src/kudu/tablet/tablet_bootstrap.cc| 8 ++-- src/kudu/tablet/tablet_metadata.cc | 51 +++--- src/kudu/tablet/tablet_metadata.h | 29 +--- src/kudu/tablet/tablet_replica-test.cc | 10 ++--- src/kudu/tools/kudu-tool-test.cc | 8 ++-- src/kudu/tools/tool_action_fs.cc | 8 ++-- src/kudu/tools/tool_action_local_replica.cc| 15 --- src/kudu/tserver/scanners.cc | 5 ++- src/kudu/tserver/tablet_server-test.cc | 2 +- src/kudu/tserver/tablet_service.cc | 23 ++ src/kudu/tserver/tserver_path_handlers.cc | 12 ++--- thirdparty/build-definitions.sh| 1 + 31 files changed, 162 insertions(+), 149 deletions(-)
[kudu] 01/02: KUDU-3197 [tserver] optimal Schema's memory used, using std::shared_ptr
This is an automated email from the ASF dual-hosted git repository.
alexey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit f4b6d8917b79b9de53957174ade1a7ffc76e0090
Author: shenxingwuying
AuthorDate: Tue Jan 4 20:07:29 2022 +0800
KUDU-3197 [tserver] optimal Schema's memory used, using std::shared_ptr
Change TabletMeta's variable Schema* to std::shared_ptr
to reduce memory used when alter schema.
Because TabletMeta save old_schemas to reserve the elder schemas
when alter schema, maybe they have been used by scanners or
compaction jobs. As jira KUDU-3197 said, frequently alter schema will
lead to tserver's memory becomes very large, just like memory leak,
especially column's number is very large.
The jira issued by wangningito, and I continue his work, and
now use std::shared_ptr instead of scoped_refptr, because
scoped_refptr causes too many changes, just as:
https://gerrit.cloudera.org/c/18098/
Change-Id: Ic284dde108c49130419d876c6698b40c195e9b35
Reviewed-on: http://gerrit.cloudera.org:8080/18255
Tested-by: Kudu Jenkins
Reviewed-by: Andrew Wong
---
src/kudu/client/client-test.cc | 10 ++---
src/kudu/common/schema.h | 1 +
src/kudu/integration-tests/linked_list-test-util.h | 2 +-
src/kudu/master/sys_catalog.cc | 5 ++-
src/kudu/tablet/all_types-scan-correctness-test.cc | 2 +-
src/kudu/tablet/cfile_set.cc | 4 +-
src/kudu/tablet/cfile_set.h| 2 +-
src/kudu/tablet/diff_scan-test.cc | 6 +--
src/kudu/tablet/diskrowset.cc | 8 ++--
src/kudu/tablet/mt-tablet-test.cc | 2 +-
src/kudu/tablet/ops/alter_schema_op.cc | 10 ++---
src/kudu/tablet/ops/alter_schema_op.h | 8 ++--
src/kudu/tablet/ops/op.h | 4 +-
src/kudu/tablet/ops/write_op.h | 7 ++-
src/kudu/tablet/rowset_metadata.h | 2 +-
src/kudu/tablet/tablet-schema-test.cc | 10 ++---
src/kudu/tablet/tablet-test-util.h | 3 +-
src/kudu/tablet/tablet.cc | 47 +++-
src/kudu/tablet/tablet.h | 6 +--
src/kudu/tablet/tablet_bootstrap.cc| 8 ++--
src/kudu/tablet/tablet_metadata.cc | 51 +++---
src/kudu/tablet/tablet_metadata.h | 29 +---
src/kudu/tablet/tablet_replica-test.cc | 10 ++---
src/kudu/tools/kudu-tool-test.cc | 8 ++--
src/kudu/tools/tool_action_fs.cc | 8 ++--
src/kudu/tools/tool_action_local_replica.cc| 15 ---
src/kudu/tserver/scanners.cc | 5 ++-
src/kudu/tserver/tablet_server-test.cc | 2 +-
src/kudu/tserver/tablet_service.cc | 23 ++
src/kudu/tserver/tserver_path_handlers.cc | 12 ++---
30 files changed, 161 insertions(+), 149 deletions(-)
diff --git a/src/kudu/client/client-test.cc b/src/kudu/client/client-test.cc
index 94ec37d..75b2bce 100644
--- a/src/kudu/client/client-test.cc
+++ b/src/kudu/client/client-test.cc
@@ -4709,7 +4709,7 @@ TEST_F(ClientTest, TestBasicAlterOperations) {
->Default(KuduValue::CopyString("hello!"));
ASSERT_OK(table_alterer->Alter());
ASSERT_EQ(4, tablet_replica->tablet()->metadata()->schema_version());
-Schema schema = tablet_replica->tablet()->metadata()->schema();
+Schema schema = *tablet_replica->tablet()->metadata()->schema();
ColumnSchema col_schema = schema.column(schema.find_column("string_val"));
ASSERT_FALSE(col_schema.has_read_default());
ASSERT_TRUE(col_schema.has_write_default());
@@ -4723,7 +4723,7 @@ TEST_F(ClientTest, TestBasicAlterOperations) {
->Default(KuduValue::FromInt(54321));
ASSERT_OK(table_alterer->Alter());
ASSERT_EQ(5, tablet_replica->tablet()->metadata()->schema_version());
-Schema schema = tablet_replica->tablet()->metadata()->schema();
+Schema schema = *tablet_replica->tablet()->metadata()->schema();
ColumnSchema col_schema =
schema.column(schema.find_column("non_null_with_default"));
ASSERT_TRUE(col_schema.has_read_default()); // Started with a default
ASSERT_TRUE(col_schema.has_write_default());
@@ -4737,7 +4737,7 @@ TEST_F(ClientTest, TestBasicAlterOperations) {
->RemoveDefault();
ASSERT_OK(table_alterer->Alter());
ASSERT_EQ(6, tablet_replica->tablet()->metadata()->schema_version());
-Schema schema = tablet_replica->tablet()->metadata()->schema();
+Schema schema = *tablet_replica->tablet()->metadata()->schema();
ColumnSchema col_schema = schema.column(schema.find_column("string_val"));
ASSERT_FALSE(col_schema.has_read_default());
ASSERT_FALSE(col_schema.has_write_defau
