Deployment Examples#

This directory contains complete deployment examples for the HSM Secrets Operator in production environments.

Files#

complete-setup.yaml - Full production deployment with all components

Additional Configurations: See the config directory for CRDs, RBAC, and operator deployments, or the helm directory for Helm chart deployments.

Complete Setup#

The complete-setup.yaml file demonstrates a full production deployment including:

Core Components#

HSM Secrets Operator deployment
HSMDevice configuration with mirroring
Production HSMSecret resources
RBAC configuration

Sample Application#

Web application deployment using HSM secrets
Database credentials from HSM
TLS certificates from HSM
High availability configuration

Production Features#

Pod anti-affinity for distribution across nodes
Horizontal Pod Autoscaler for scaling
Pod Disruption Budget for availability
Network policies for security
Resource limits and health checks

Monitoring#

ServiceMonitor for Prometheus integration
Metrics collection from operator
HSM device health monitoring

Deployment Steps#

1. Prerequisites#

Ensure you have:

Kubernetes cluster (v1.20+)
HSM devices connected to nodes
OpenSC libraries installed
Prometheus Operator (for monitoring)

2. Label HSM Nodes#

Label nodes that have HSM devices:

kubectl label node <node-name> hsm.j5t.io/enabled=true

3. Deploy the Operator#

# Deploy CRDs first
kubectl apply -f config/crd/bases/

# Deploy the operator
kubectl apply -f config/default/

# Or use the complete setup
kubectl apply -f examples/deployment/complete-setup.yaml

4. Verify Deployment#

# Check operator pods
kubectl get pods -n hsm-secrets-operator-system

# Check HSM devices
kubectl get hsmdevice

# Check secrets (multiple ways)
kubectl hsm list                # via kubectl-hsm plugin
kubectl get hsmsecret          # via CRDs
kubectl get secret             # via K8s secrets

5. Test Secret Operations#

Option A: Using kubectl-hsm plugin (recommended):

# Check health
kubectl hsm health

# Create a test secret
kubectl hsm create test-secret --from-literal=key=value

# List secrets
kubectl hsm list

# Get secret details
kubectl hsm get test-secret

Option B: Using REST API (advanced):

# Port forward to access API locally
kubectl port-forward -n hsm-secrets-operator-system service/hsm-secrets-operator-api 8090:8090

# Test health endpoint
curl http://localhost:8090/api/v1/health

Configuration Options#

HSM Device Configuration#

spec:
  deviceType: PicoHSM  # or SmartCardHSM, Generic
  usb:
    vendorId: "20a0"
    productId: "4230"
  nodeSelector:
    hsm.j5t.io/enabled: "true"
  mirroring:
    policy: "ReadOnly"
    syncInterval: 300
    autoFailover: true

Secret Configuration#

spec:
  hsmPath: "secrets/production/my-secret"
  secretName: "my-secret"
  autoSync: true
  syncInterval: 600
  secretType: Opaque  # or kubernetes.io/tls, etc.

Security Considerations#

1. RBAC#

Limit access to HSMSecret resources
Use service accounts with minimal permissions
Separate dev/staging/prod access

2. Network Security#

Use NetworkPolicies to restrict traffic
Enable TLS for API communication
Use private networks where possible

3. HSM Security#

Properly configure HSM authentication
Regular security updates for OpenSC libraries
Monitor HSM access and operations

4. Secret Management#

Use strong passwords and keys
Implement secret rotation policies
Monitor secret access and changes

Monitoring and Alerting#

Key Metrics to Monitor#

HSM device connectivity
Secret sync status and lag
API response times and errors
Operator pod health

Sample Alerts#

# HSM Device Down
- alert: HSMDeviceDown
  expr: hsm_device_connected == 0
  for: 5m
  labels:
    severity: critical
  annotations:
    summary: "HSM device is disconnected"

# Secret Sync Failed
- alert: SecretSyncFailed
  expr: hsm_secret_sync_failed > 0
  for: 2m
  labels:
    severity: warning
  annotations:
    summary: "Secret synchronization failed"

Troubleshooting#

Common Issues#

HSM Device Not Found

# Check USB devices
lsusb

# Check OpenSC
pkcs11-tool --list-slots

# Check node labels
kubectl describe node <node-name>

Secrets Not Syncing

# Check HSMSecret status
kubectl describe hsmsecret <secret-name>

# Check operator logs
kubectl logs -n hsm-secrets-operator-system deployment/hsm-secrets-operator-controller-manager

API Not Responding

# Check API service
kubectl get service -n hsm-secrets-operator-system

# Check API logs
kubectl logs -n hsm-secrets-operator-system -l control-plane=controller-manager

Backup and Recovery#

Backup HSM Configuration#

# Export HSMDevice configurations
kubectl get hsmdevice -o yaml > hsm-devices-backup.yaml

# Export HSMSecret configurations
kubectl get hsmsecret --all-namespaces -o yaml > hsm-secrets-backup.yaml

Recovery Process#

Restore HSM devices and configure authentication
Deploy operator and CRDs
Apply HSMDevice configurations
Apply HSMSecret configurations
Verify secret synchronization

Configure Feed