(kudu) branch master updated: KUDU-3534 [compaction] Log timestamp of two matching DELETE REDO mutations.

2023-12-22 Thread alexey
This is an automated email from the ASF dual-hosted git repository.

alexey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git


The following commit(s) were added to refs/heads/master by this push:
 new 1f6f7e5ba KUDU-3534 [compaction] Log timestamp of two matching DELETE 
REDO mutations.
1f6f7e5ba is described below

commit 1f6f7e5bafb6a9f497ea92b62c1f268a2fe6c3c3
Author: Ashwani Raina 
AuthorDate: Thu Dec 14 18:18:32 2023 +0530

KUDU-3534 [compaction] Log timestamp of two matching DELETE REDO mutations.

This patch just adds an info message to log timestamp value in case
two REDO mutations of type DELETE are found to be stamped with same
time. This is an undesired condition that could possibly happen
due to corruption of mutation entries. The value will give us an
idea whether it is 0, garbled or close to some valid timestamp.

Change-Id: I508254a83046818b81db4577bf07265b46a13c9a
Reviewed-on: http://gerrit.cloudera.org:8080/20792
Reviewed-by: Abhishek Chennaka 
Tested-by: Abhishek Chennaka 
Reviewed-by: Wang Xixu <1450306...@qq.com>
Reviewed-by: Alexey Serbin 
---
 src/kudu/tablet/compaction.cc | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/src/kudu/tablet/compaction.cc b/src/kudu/tablet/compaction.cc
index 8f2bba384..2291ea0eb 100644
--- a/src/kudu/tablet/compaction.cc
+++ b/src/kudu/tablet/compaction.cc
@@ -476,7 +476,12 @@ int CompareDuplicatedRows(const CompactionInputRow& left,
 // the row again. If that's the case, the delete with the higher timestamp
 // defines the newer input, or
 int ret = left_last->timestamp().CompareTo(right_last->timestamp());
-CHECK_NE(0, ret);
+// Log timestamp and mutation history, invalid ts value could mean 
potential corruption.
+CHECK_NE(0, ret)
+<< Substitute("different DELETE REDO mutations must not have same 
timestamp, "
+  "Left Redo Mutations: $0, Right Redo Mutations: $1",
+  Mutation::StringifyMutationList(*left.row.schema(), 
left.redo_head),
+  Mutation::StringifyMutationList(*right.row.schema(), 
right.redo_head));
 return ret;
   }
 



(kudu) branch master updated: [webserver] add security-related HTTP headers

2023-12-22 Thread alexey
This is an automated email from the ASF dual-hosted git repository.

alexey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git


The following commit(s) were added to refs/heads/master by this push:
 new 26ad5ad96 [webserver] add security-related HTTP headers
26ad5ad96 is described below

commit 26ad5ad962fe05af4b69730ffded9414fd13a5e5
Author: Alexey Serbin 
AuthorDate: Thu Feb 17 17:54:11 2022 -0800

[webserver] add security-related HTTP headers

To please various security scanners, this patch adds the following
HTTP headers into Kudu embedded webserver's responses:
  * Cache-Control (set to 'no-store' by default, see [1])
  * X-Content-Type-Options (set to 'nosniff' by default, see [2])
  * Strict-Transport-Security (see [3] and below for details)

The embedded webserver adds the HTTP strict transport security
(HSTS) header 'Strict-Transport-Security' for responses sent from HTTPS
(i.e. TLS-protected) endpoints if --webserver_hsts_max_age_seconds is
set to a non-negative value.  The header contains the 'max-age'
attribute as specified by the flag, and adds the optional
'includeSubDomains' attribute as per set setting of the
--webserver_hsts_include_sub_domains flag.  The HSTS header isn't added
to the responses sent from plain HTTP endpoints (BTW, it seems most
browsers simply ignore the HSTS header anyway if it's received from
an HTTP, not an HTTPS endpoint).

Essentially, the HSTS header for Kudu is a no-op since the embedded
webserver doesn't serve both HTTP and HTTPS endpoints at the same time:
one can enable one or the other, but never both.  However, many security
scanners almost cry "Security breach" if they don't see the header :)

Adding the HSTS header isn't enabled by default since it could make
other plain HTTP endpoints at the same node/hostname inaccessible.
To enable adding the HSTS header for HTTPS responses, set the
--webserver_hsts_max_age_seconds flag to a non-negative integer.
Enable it with care and only if you know what you are doing!

One extra test added and a few existing ones updated correspondingly
to cover the newly introduced functionality.

[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control
[2] 
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Content-Type-Options
[3] 
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Strict-Transport-Security

Change-Id: Id844b9588196b3d608765d0f16f5caec1c414d41
Reviewed-on: http://gerrit.cloudera.org:8080/18253
Tested-by: Alexey Serbin 
Reviewed-by: Attila Bukor 
---
 src/kudu/server/webserver-test.cc | 76 ++-
 src/kudu/server/webserver.cc  | 74 ++
 2 files changed, 142 insertions(+), 8 deletions(-)

diff --git a/src/kudu/server/webserver-test.cc 
b/src/kudu/server/webserver-test.cc
index 472a64e62..924e11128 100644
--- a/src/kudu/server/webserver-test.cc
+++ b/src/kudu/server/webserver-test.cc
@@ -24,6 +24,7 @@
 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -61,8 +62,13 @@ using std::vector;
 using std::unique_ptr;
 using strings::Substitute;
 
+DECLARE_bool(webserver_hsts_include_sub_domains);
+DECLARE_int32(webserver_hsts_max_age_seconds);
 DECLARE_int32(webserver_max_post_length_bytes);
 DECLARE_string(trusted_certificate_file);
+DECLARE_string(webserver_cache_control_options);
+DECLARE_string(webserver_x_content_type_options);
+DECLARE_string(webserver_x_frame_options);
 
 DEFINE_bool(test_sensitive_flag, false, "a sensitive flag");
 TAG_FLAG(test_sensitive_flag, sensitive);
@@ -360,14 +366,42 @@ TEST_F(SpnegoWebserverTest, 
TestAuthNotRequiredForOptions) {
 TEST_F(WebserverTest, TestIndexPage) {
   curl_.set_return_headers(true);
   ASSERT_OK(curl_.FetchURL(url_, &buf_));
-  // Check expected header.
+
+  // Check for the expected headers.
+  ASSERT_STR_CONTAINS(buf_.ToString(), "Cache-Control: no-store");
   ASSERT_STR_CONTAINS(buf_.ToString(), "X-Frame-Options: DENY");
+  ASSERT_STR_CONTAINS(buf_.ToString(), "X-Content-Type-Options: nosniff");
+
+  FLAGS_webserver_hsts_max_age_seconds = 1000;
+  // The HTTP strict transport security policy (HSTS) header should be absent
+  // in the response sent from the plain HTTP (i.e. non-TLS) endpoint.
+  ASSERT_STR_NOT_CONTAINS(buf_.ToString(), "Strict-Transport-Security");
 
   // Should have expected title.
   ASSERT_STR_CONTAINS(buf_.ToString(), "Kudu");
 
   // Should have link to default path handlers (e.g memz)
   ASSERT_STR_CONTAINS(buf_.ToString(), "memz");
+
+  // Check that particular headers are generated as expected when customizing
+  // the Cache-Control and X-Content-Type-Options headers.
+  FLAGS_webserver_cache_control_options = "no-cache";
+  FLAGS_webserver_x_frame_options = "SAMEORIGIN";
+  FLAGS_webserver_x_content_type_optio