[
https://issues.apache.org/jira/browse/HDDS-17721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng reassigned HDDS-17721:
---------------------------------
Assignee: Siyao Meng
Description:
This is a regression from HDDS-17235.
Since HDDS-17235, the inter-SCM gRPC server ({{InterSCMGrpcProtocolService}})
accepts only SCM certificates when {{hdds.grpc.tls.enabled}} is true. Recon
downloads the SCM DB through this server
({{StorageContainerServiceProviderImpl#getSCMDBSnapshot}}) with its own leaf
certificate, so the TLS handshake fails. Recon logs "Unable to obtain SCM DB
Snapshot" and {{ReconContext}} reports {{GET_SCM_DB_SNAPSHOT_FAILED}}. This
happens whenever Recon needs a full SCM DB sync: initial sync, container count
drift above {{ozone.recon.scm.container.threshold}}, or a manual DB sync.
Letting the gRPC server also accept Recon is not safe: a Recon certificate has
no field that a Datanode or OM cannot also get into its own certificate
(subject and extensions come from the CSR, and the role checked at issuance is
not recorded in the certificate or the SCM certificate store).
Proposed fix:
* Recon downloads the SCM DB from the leader SCM's {{/dbCheckpoint}} HTTP(S)
endpoint, with SPNEGO when {{hdds.scm.http.auth.type}} is {{kerberos}}. It
verifies the checkpoint completion marker, like the OM DB download does. If
{{hdds.grpc.tls.enabled}} is true but {{ozone.http.policy}} does not enable
HTTPS, it warns that the transfer is not encrypted, as the OM DB download over
HTTP already is in that configuration.
* {{SCMDBCheckpointServlet}} allows the Recon principal
({{ozone.recon.kerberos.principal}}), the same way {{OMDBCheckpointServlet}}
does. Like the gRPC download, it flushes the SCM transaction buffer and uses
the current DB store before taking the checkpoint.
* The inter-SCM gRPC server stays SCM only.
Recon then needs to reach the leader SCM's HTTP(S) port instead of
{{ozone.scm.grpc.port}}. SCM should be upgraded before Recon, since an older
SCM rejects Recon on {{/dbCheckpoint}} when SPNEGO and admin authorization are
enabled.
was:This is a regression from HDDS-17235.
> Recon cannot download SCM DB when gRPC TLS is enabled
> -----------------------------------------------------
>
> Key: HDDS-17721
> URL: https://issues.apache.org/jira/browse/HDDS-17721
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Recon, SCM
> Affects Versions: 2.2.2
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Blocker
>
> This is a regression from HDDS-17235.
> Since HDDS-17235, the inter-SCM gRPC server ({{InterSCMGrpcProtocolService}})
> accepts only SCM certificates when {{hdds.grpc.tls.enabled}} is true. Recon
> downloads the SCM DB through this server
> ({{StorageContainerServiceProviderImpl#getSCMDBSnapshot}}) with its own leaf
> certificate, so the TLS handshake fails. Recon logs "Unable to obtain SCM DB
> Snapshot" and {{ReconContext}} reports {{GET_SCM_DB_SNAPSHOT_FAILED}}. This
> happens whenever Recon needs a full SCM DB sync: initial sync, container
> count drift above {{ozone.recon.scm.container.threshold}}, or a manual DB
> sync.
> Letting the gRPC server also accept Recon is not safe: a Recon certificate
> has no field that a Datanode or OM cannot also get into its own certificate
> (subject and extensions come from the CSR, and the role checked at issuance
> is not recorded in the certificate or the SCM certificate store).
> Proposed fix:
> * Recon downloads the SCM DB from the leader SCM's {{/dbCheckpoint}} HTTP(S)
> endpoint, with SPNEGO when {{hdds.scm.http.auth.type}} is {{kerberos}}. It
> verifies the checkpoint completion marker, like the OM DB download does. If
> {{hdds.grpc.tls.enabled}} is true but {{ozone.http.policy}} does not enable
> HTTPS, it warns that the transfer is not encrypted, as the OM DB download
> over HTTP already is in that configuration.
> * {{SCMDBCheckpointServlet}} allows the Recon principal
> ({{ozone.recon.kerberos.principal}}), the same way {{OMDBCheckpointServlet}}
> does. Like the gRPC download, it flushes the SCM transaction buffer and uses
> the current DB store before taking the checkpoint.
> * The inter-SCM gRPC server stays SCM only.
> Recon then needs to reach the leader SCM's HTTP(S) port instead of
> {{ozone.scm.grpc.port}}. SCM should be upgraded before Recon, since an older
> SCM rejects Recon on {{/dbCheckpoint}} when SPNEGO and admin authorization
> are enabled.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
