woblerr commented on issue #80: URL: https://github.com/apache/cloudberry-backup/issues/80#issuecomment-4170215979
Hi ### 1. Monitoring: Prometheus Metrics Exporter There is currently no built-in way to monitor backup status via metrics. I have a ready-to-use Prometheus exporter — [gpbackup_exporter](https://github.com/woblerr/gpbackup_exporter) — that collects metrics from the `gpbackup_history.db` file. It already uses some calls from the original [gpbackman](https://github.com/woblerr/gpbackman) library (which is now integrated into cloudberry-backup). The exporter provides the following metrics: | Metric | Description | |--------|-------------| | `gpbackup_backup_status` | Success/failure per backup | | `gpbackup_backup_deletion_status` | Deletion lifecycle tracking | | `gpbackup_backup_info` | Backup metadata (version, compression, plugin, etc.) | | `gpbackup_backup_duration_seconds` | Backup duration | | `gpbackup_backup_since_last_completion_seconds` | Time since last completed backup | | `gpbackup_exporter_status` | Exporter health / data fetch status | There is also a ready-made [Grafana dashboard](https://grafana.com/grafana/dashboards/22543-gpbackup-exporter-dashboard/) for visualization. If this is relevant to the project's roadmap, I can port the exporter into cloudberry-backup. ### 2. Backup Encryption Currently backup data is stored **unencrypted**. All modern backup utilities (including [WAL-G](https://github.com/wal-g/wal-g) for physical backups) support data encryption. I would suggest splitting this feature request into two parts: - **S3 plugin encryption (higher priority):** Encrypt backup data before uploading to S3. This is critical because object storage is often shared or accessible over the network, making unencrypted data a significant security risk. - **Local/mounted storage encryption:** Encrypt backups when data is stored on local disks or mounted file storage (i.e., when `--backup-dir` is used with gpbackup). While the risk is somewhat lower than with S3 (since access control is handled at the OS/filesystem level), encryption at rest is still a best practice. Of course, encryption should be an option, and users can choose whether to use it or not. In addition, it would be valuable to provide **documentation on how to implement encryption when using custom plugins**, so that third-party plugin authors can follow a consistent approach. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
