Re: [PR] Support etcd client authentication [skywalking-banyandb]
hanahmily merged PR #331: URL: https://github.com/apache/skywalking-banyandb/pull/331 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Support etcd client authentication [skywalking-banyandb]
hailin0 commented on PR #331: URL: https://github.com/apache/skywalking-banyandb/pull/331#issuecomment-1771972416 @hanahmily PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Support etcd client authentication [skywalking-banyandb]
hailin0 commented on code in PR #331:
URL:
https://github.com/apache/skywalking-banyandb/pull/331#discussion_r1365725043
##
docs/installation/cluster.md:
##
@@ -28,3 +28,45 @@ The host is registered to the etcd cluster by the
`banyand-server` automatically
- `node-host-provider=hostname` : Default. The OS's hostname is registered as
the host part in the address.
- `node-host-provider=ip` : The OS's the first non-loopback active IP
address(IPv4) is registered as the host part in the address.
- `node-host-provider=flag` : `node-host` is registered as the host part in
the address.
+
+## Etcd Client Authentication
+
+etcd supports through tls certificates and RBAC based authentication for both
clients to server communication. This section is intended to help users set up
authentication in etcd client.
Review Comment:
fixed
##
test/integration/etcd/client_test.go:
##
@@ -0,0 +1,304 @@
+// Licensed to Apache Software Foundation (ASF) under one or more contributor
+// license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright
+// ownership. Apache Software Foundation (ASF) licenses this file to you under
+// the Apache License, Version 2.0 (the "License"); you may
+// not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package integration_etcd_test
+
+import (
+ "context"
+ "crypto/tls"
+ "fmt"
+ "log"
+ "net/url"
+ "path/filepath"
+ "time"
+
+ . "github.com/onsi/ginkgo/v2"
+ . "github.com/onsi/gomega"
+ "github.com/onsi/gomega/gleak"
+ "go.etcd.io/etcd/client/pkg/v3/transport"
+ clientv3 "go.etcd.io/etcd/client/v3"
+ "go.etcd.io/etcd/server/v3/embed"
+
+ databasev1
"github.com/apache/skywalking-banyandb/api/proto/banyandb/database/v1"
+ "github.com/apache/skywalking-banyandb/banyand/metadata/schema"
+ "github.com/apache/skywalking-banyandb/pkg/test"
+ "github.com/apache/skywalking-banyandb/pkg/test/flags"
+ "github.com/apache/skywalking-banyandb/pkg/test/helpers"
+ "github.com/apache/skywalking-banyandb/pkg/test/setup"
+)
+
+const host = "127.0.0.1"
+
+const namespace = "liaison-test"
+
+const nodeHost = "liaison-1"
+
+var _ = Describe("Client Test", func() {
+ var (
+ dirstring
+ dirSpaceDeffunc()
+ caFilePath string
+ serverKeyFilePath string
+ serverCertFilePath string
+ clientKeyFilePath string
+ clientCertFilePath string
+ goods []gleak.Goroutine
+ )
+ BeforeEach(func() {
+ var err error
+ dir, dirSpaceDef, err = test.NewSpace()
+ Expect(err).ShouldNot(HaveOccurred())
+ caFilePath, err = filepath.Abs("./cafile/ca.crt")
+ Expect(err).ShouldNot(HaveOccurred())
+ serverKeyFilePath, err =
filepath.Abs("./cafile/server-serverusage.key.insecure")
+ Expect(err).ShouldNot(HaveOccurred())
+ serverCertFilePath, err =
filepath.Abs("./cafile/server-serverusage.crt")
+ Expect(err).ShouldNot(HaveOccurred())
+ clientKeyFilePath, err =
filepath.Abs("./cafile/client-clientusage.key.insecure")
+ Expect(err).ShouldNot(HaveOccurred())
+ clientCertFilePath, err =
filepath.Abs("./cafile/client-clientusage.crt")
+ Expect(err).ShouldNot(HaveOccurred())
+
+ goods = gleak.Goroutines()
+ })
+ AfterEach(func() {
+ dirSpaceDef()
+ Eventually(gleak.Goroutines,
flags.EventuallyTimeout).ShouldNot(gleak.HaveLeaked(goods))
+ })
+
+ It("should be using user/password connect etcd server successfully",
func() {
+ serverTLSInfo := transport.TLSInfo{}
+ clientTLSInfo := transport.TLSInfo{}
+ clientConfig, err := clientTLSInfo.ClientConfig()
+ Expect(err).ShouldNot(HaveOccurred())
+ username := "banyandb"
+ password := "banyandb"
+
+ etcdServer, err := createEtcdServer(dir, serverTLSInfo)
+ Expect(err).ShouldNot(HaveOccurred())
+ // wait for e.Server to join the cluster
+ <-etcdServer.Server.ReadyNotify()
+ etcdEndpoint := etcdServer.Config().ListenClientUrls[0].String()
+ defer etcdServer.Close()
+
+
Re: [PR] Support etcd client authentication [skywalking-banyandb]
hailin0 commented on code in PR #331: URL: https://github.com/apache/skywalking-banyandb/pull/331#discussion_r1365724065 ## docs/installation/cluster.md: ## @@ -28,3 +28,45 @@ The host is registered to the etcd cluster by the `banyand-server` automatically - `node-host-provider=hostname` : Default. The OS's hostname is registered as the host part in the address. - `node-host-provider=ip` : The OS's the first non-loopback active IP address(IPv4) is registered as the host part in the address. - `node-host-provider=flag` : `node-host` is registered as the host part in the address. + +## Etcd Client Authentication + +etcd supports through tls certificates and RBAC based authentication for both clients to server communication. This section is intended to help users set up authentication in etcd client. + +### Client-to-server authentication with username/password + +The etcd user can be setup by the [etcd authentication guide](https://etcd.io/docs/v3.5/op-guide/authentication/) + +The username/password is configured in the following command: Review Comment: added `note` ## docs/installation/cluster.md: ## @@ -28,3 +28,45 @@ The host is registered to the etcd cluster by the `banyand-server` automatically - `node-host-provider=hostname` : Default. The OS's hostname is registered as the host part in the address. - `node-host-provider=ip` : The OS's the first non-loopback active IP address(IPv4) is registered as the host part in the address. - `node-host-provider=flag` : `node-host` is registered as the host part in the address. + +## Etcd Client Authentication + +etcd supports through tls certificates and RBAC based authentication for both clients to server communication. This section is intended to help users set up authentication in etcd client. + +### Client-to-server authentication with username/password + +The etcd user can be setup by the [etcd authentication guide](https://etcd.io/docs/v3.5/op-guide/authentication/) + +The username/password is configured in the following command: + +- `etcd-username`: The username for etcd client authentication. +- `etcd-password`: The password for etcd client authentication. + +```shell +$ ./banyand-server storage --etcd-endpoints=your-endpoints --etcd-username=your-username --etcd-password=your-password +$ ./banyand-server liaison --etcd-endpoints=your-endpoints --etcd-username=your-username --etcd-password=your-password +``` + +### Client-to-server transport security with HTTPS + +The etcd trusted certificate file can be setup by the [etcd transport security model](https://etcd.io/docs/v3.5/op-guide/security/#example-1-client-to-server-transport-security-with-https) + +- `etcd-tls-ca-file`: The path of the trusted certificate file. + +```shell +$ ./banyand-server storage --etcd-endpoints=your-https-endpoints --etcd-tls-ca-file=youf-file-path +$ ./banyand-server liaison --etcd-endpoints=your-https-endpoints --etcd-tls-ca-file=youf-file-path +``` + +### Client-to-server authentication with HTTPS client certificates Review Comment: fixed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Support etcd client authentication [skywalking-banyandb]
hailin0 commented on code in PR #331:
URL:
https://github.com/apache/skywalking-banyandb/pull/331#discussion_r1365725432
##
test/integration/etcd/client_test.go:
##
@@ -0,0 +1,304 @@
+// Licensed to Apache Software Foundation (ASF) under one or more contributor
+// license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright
+// ownership. Apache Software Foundation (ASF) licenses this file to you under
+// the Apache License, Version 2.0 (the "License"); you may
+// not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package integration_etcd_test
+
+import (
+ "context"
+ "crypto/tls"
+ "fmt"
+ "log"
+ "net/url"
+ "path/filepath"
+ "time"
+
+ . "github.com/onsi/ginkgo/v2"
+ . "github.com/onsi/gomega"
+ "github.com/onsi/gomega/gleak"
+ "go.etcd.io/etcd/client/pkg/v3/transport"
+ clientv3 "go.etcd.io/etcd/client/v3"
+ "go.etcd.io/etcd/server/v3/embed"
+
+ databasev1
"github.com/apache/skywalking-banyandb/api/proto/banyandb/database/v1"
+ "github.com/apache/skywalking-banyandb/banyand/metadata/schema"
+ "github.com/apache/skywalking-banyandb/pkg/test"
+ "github.com/apache/skywalking-banyandb/pkg/test/flags"
+ "github.com/apache/skywalking-banyandb/pkg/test/helpers"
+ "github.com/apache/skywalking-banyandb/pkg/test/setup"
+)
+
+const host = "127.0.0.1"
+
+const namespace = "liaison-test"
+
+const nodeHost = "liaison-1"
+
+var _ = Describe("Client Test", func() {
+ var (
+ dirstring
+ dirSpaceDeffunc()
+ caFilePath string
+ serverKeyFilePath string
+ serverCertFilePath string
+ clientKeyFilePath string
+ clientCertFilePath string
+ goods []gleak.Goroutine
+ )
+ BeforeEach(func() {
+ var err error
+ dir, dirSpaceDef, err = test.NewSpace()
+ Expect(err).ShouldNot(HaveOccurred())
+ caFilePath, err = filepath.Abs("./cafile/ca.crt")
Review Comment:
fixed
##
.licenserc.yaml:
##
@@ -85,6 +85,7 @@ header: # `header` section is configurations for source codes
license header.
- 'ui'
- '.github/PULL_REQUEST_TEMPLATE'
- "**/*.prof"
+- "**/test/integration/etcd/cafile/**"
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] Support etcd client authentication [skywalking-banyandb]
hailin0 commented on code in PR #331: URL: https://github.com/apache/skywalking-banyandb/pull/331#discussion_r1365724556 ## docs/installation/cluster.md: ## @@ -28,3 +28,45 @@ The host is registered to the etcd cluster by the `banyand-server` automatically - `node-host-provider=hostname` : Default. The OS's hostname is registered as the host part in the address. - `node-host-provider=ip` : The OS's the first non-loopback active IP address(IPv4) is registered as the host part in the address. - `node-host-provider=flag` : `node-host` is registered as the host part in the address. + +## Etcd Client Authentication + +etcd supports through tls certificates and RBAC based authentication for both clients to server communication. This section is intended to help users set up authentication in etcd client. + +### Client-to-server authentication with username/password + +The etcd user can be setup by the [etcd authentication guide](https://etcd.io/docs/v3.5/op-guide/authentication/) + +The username/password is configured in the following command: + +- `etcd-username`: The username for etcd client authentication. +- `etcd-password`: The password for etcd client authentication. + +```shell +$ ./banyand-server storage --etcd-endpoints=your-endpoints --etcd-username=your-username --etcd-password=your-password +$ ./banyand-server liaison --etcd-endpoints=your-endpoints --etcd-username=your-username --etcd-password=your-password +``` + +### Client-to-server transport security with HTTPS Review Comment: fixed ## docs/installation/cluster.md: ## @@ -28,3 +28,45 @@ The host is registered to the etcd cluster by the `banyand-server` automatically - `node-host-provider=hostname` : Default. The OS's hostname is registered as the host part in the address. - `node-host-provider=ip` : The OS's the first non-loopback active IP address(IPv4) is registered as the host part in the address. - `node-host-provider=flag` : `node-host` is registered as the host part in the address. + +## Etcd Client Authentication + +etcd supports through tls certificates and RBAC based authentication for both clients to server communication. This section is intended to help users set up authentication in etcd client. + +### Client-to-server authentication with username/password Review Comment: fixed ## docs/installation/cluster.md: ## @@ -28,3 +28,45 @@ The host is registered to the etcd cluster by the `banyand-server` automatically - `node-host-provider=hostname` : Default. The OS's hostname is registered as the host part in the address. - `node-host-provider=ip` : The OS's the first non-loopback active IP address(IPv4) is registered as the host part in the address. - `node-host-provider=flag` : `node-host` is registered as the host part in the address. + +## Etcd Client Authentication Review Comment: fixed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Support etcd client authentication [skywalking-banyandb]
hanahmily commented on code in PR #331:
URL:
https://github.com/apache/skywalking-banyandb/pull/331#discussion_r1365102335
##
.licenserc.yaml:
##
@@ -85,6 +85,7 @@ header: # `header` section is configurations for source codes
license header.
- 'ui'
- '.github/PULL_REQUEST_TEMPLATE'
- "**/*.prof"
+- "**/test/integration/etcd/cafile/**"
Review Comment:
Would you please put them into `testdata`, which is ignored by the checker
automatically? You can put this folder in any place.
##
test/integration/etcd/client_test.go:
##
@@ -0,0 +1,304 @@
+// Licensed to Apache Software Foundation (ASF) under one or more contributor
+// license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright
+// ownership. Apache Software Foundation (ASF) licenses this file to you under
+// the Apache License, Version 2.0 (the "License"); you may
+// not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package integration_etcd_test
+
+import (
+ "context"
+ "crypto/tls"
+ "fmt"
+ "log"
+ "net/url"
+ "path/filepath"
+ "time"
+
+ . "github.com/onsi/ginkgo/v2"
+ . "github.com/onsi/gomega"
+ "github.com/onsi/gomega/gleak"
+ "go.etcd.io/etcd/client/pkg/v3/transport"
+ clientv3 "go.etcd.io/etcd/client/v3"
+ "go.etcd.io/etcd/server/v3/embed"
+
+ databasev1
"github.com/apache/skywalking-banyandb/api/proto/banyandb/database/v1"
+ "github.com/apache/skywalking-banyandb/banyand/metadata/schema"
+ "github.com/apache/skywalking-banyandb/pkg/test"
+ "github.com/apache/skywalking-banyandb/pkg/test/flags"
+ "github.com/apache/skywalking-banyandb/pkg/test/helpers"
+ "github.com/apache/skywalking-banyandb/pkg/test/setup"
+)
+
+const host = "127.0.0.1"
+
+const namespace = "liaison-test"
+
+const nodeHost = "liaison-1"
+
+var _ = Describe("Client Test", func() {
+ var (
+ dirstring
+ dirSpaceDeffunc()
+ caFilePath string
+ serverKeyFilePath string
+ serverCertFilePath string
+ clientKeyFilePath string
+ clientCertFilePath string
+ goods []gleak.Goroutine
+ )
+ BeforeEach(func() {
+ var err error
+ dir, dirSpaceDef, err = test.NewSpace()
+ Expect(err).ShouldNot(HaveOccurred())
+ caFilePath, err = filepath.Abs("./cafile/ca.crt")
Review Comment:
Please use
https://github.com/apache/skywalking-banyandb/blob/main/test/integration/standalone/other/tls_test.go#L47
to load files
##
docs/installation/cluster.md:
##
@@ -28,3 +28,45 @@ The host is registered to the etcd cluster by the
`banyand-server` automatically
- `node-host-provider=hostname` : Default. The OS's hostname is registered as
the host part in the address.
- `node-host-provider=ip` : The OS's the first non-loopback active IP
address(IPv4) is registered as the host part in the address.
- `node-host-provider=flag` : `node-host` is registered as the host part in
the address.
+
+## Etcd Client Authentication
Review Comment:
```suggestion
## Etcd Authentication
```
##
test/integration/etcd/client_test.go:
##
@@ -0,0 +1,304 @@
+// Licensed to Apache Software Foundation (ASF) under one or more contributor
+// license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright
+// ownership. Apache Software Foundation (ASF) licenses this file to you under
+// the Apache License, Version 2.0 (the "License"); you may
+// not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package integration_etcd_test
+
+import (
+ "context"
+ "crypto/tls"
+ "fmt"
+ "log"
+ "net/url"
+ "path/filepath"
+ "time"
+
+ . "github.com/onsi/ginkgo/v2"
+ . "github.com/onsi/gomega"
+ "github.com/onsi/gomega/gleak"
+
Re: [PR] Support etcd client authentication [skywalking-banyandb]
hailin0 commented on PR #331: URL: https://github.com/apache/skywalking-banyandb/pull/331#issuecomment-1768623495 > You should create test cases to ensure the added items are verified. You can utilize "test/integration/distributed/setup/node_test.go" as a reference for writing the test cases. @hanahmily please help to review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@skywalking.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
