Module 35: Centralized Configuration with Spring Cloud Config

In a distributed system, managing configuration across 50+ microservice instances is a logistical nightmare. If you need to change a database password or a feature flag, you don't want to rebuild and redeploy 50 JAR files. You need a Centralized Configuration Server.

Spring Cloud Config provides server and client-side support for externalized configuration in a distributed system. With the Config Server, you have a central place to manage external properties for applications across all environments.

1. The Source of Truth: Config Server

The Config Server is a standalone microservice that centralizes all application properties. It typically uses a Git Repository as the backend, providing:

• Versioning: Every configuration change is a Git commit. You can see historical values and audit changes.
• Auditing: Git's author and timestamp identify exactly who changed which property and when.
• Rollback: Instantly revert to a previous configuration state by resetting the Git branch or pointing to a previous commit ID.
• Environment Separation: Use different branches (e.g., dev, prod) or logical labels to separate environment-specific configs.

2. Hardware-Mirror: The "Config Thundering Herd"

When a large cluster (e.g., 500 service instances) restarts simultaneously—whether due to a power failure, a major data center migration, or a logic-triggered crash—a phenomenon known as the Thundering Herd occurs.

Physical Hardware Impact:

1. NIC Saturation: Every instance calls the Config Server at the same millisecond to fetch its bootstrap.yml or application.yml. This creates a massive spike in Network Ingress on the Config Server's NIC. If you have 500 instances each requesting 100KB of configuration, that is a sudden 50MB burst.
2. CPU Spikes (The Decryption Tax): If you use encryption (Symmetric or Asymmetric), the Config Server must decrypt secrets for every request. RSA decryption is CPU-intensive. Hundreds of concurrent decryption requests can peg the Config Server's CPU at 100%, causing request timeouts for the remaining fleet.
3. Git I/O Latency: If the server is configured to clone-on-start (the default), the Config Server must talk to a remote Git provider (GitHub/GitLab). This introduces External Latency and eats up Disk I/O and Memory on the Config Server host.

Hardware-Mirror Rule: For high-density clusters, always enable local filesystem caching on the Config Server (spring.cloud.config.server.git.basedir). Furthermore, scale the Config Server horizontally behind a Load Balancer and ensure the host has a high TCP Backlog setting in the Linux kernel (net.core.somaxconn) to prevent dropped connection packets during the burst.

3. Server Implementation: Building the Brain

To build the server, you need the spring-cloud-config-server dependency.

Maven Dependencies

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-config-server</artifactId>
</dependency>

Main Class

@EnableConfigServer // CRITICAL: This activates the server logic
@SpringBootApplication
public class ConfigServerApplication {
    public static void main(String[] args) {
        SpringApplication.run(ConfigServerApplication.class, args);
    }
}

Server Configuration (application.yml)

server:
  port: 8888
spring:
  cloud:
    config:
      server:
        git:
          uri: https://github.com/topictrick/config-repo
          search-paths: 'services/{application}' # Search by app name
          clone-on-start: true

4. Client Implementation: Import vs. Bootstrap

In Spring Boot 3.x, the way clients fetch config has changed. The legacy bootstrap.yml method is now optional, replaced by the spring.config.import property.

Client Dependencies

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-config</artifactId>
</dependency>

Client Configuration (application.yml)

spring:
  application:
    name: inventory-service
  config:
    import: "optional:configserver:http://localhost:8888"

The optional: prefix is vital during local development; it prevents the application from failing if the Config Server isn't reachable.

5. Security: Asymmetric Encryption (RSA)

Storing passwords in plain text in Git is a critical security vulnerability. While Symmetric encryption (AES) is easy, Asymmetric Encryption (RSA) is the enterprise gold standard.

The Hardware Key

1. Generate a Keystore: Use the JDK keytool to create a .jks file.
   bash

   keytool -genkeypair -alias configserver -keyalg RSA \
     -dname "CN=Config Server, OU=Hardware" -keypass secret -keystore configserver.jks -storepass secret

2. Configure the Server:
   yaml

   encrypt:
     key-store:
       location: classpath:/configserver.jks
       password: secret
       alias: configserver
       secret: secret

3. Encrypted Value: Precede your secrets with {cipher} in the Git YAML.
   yaml

   db:
     password: "{cipher}MIIBvQYJKoZIhvcNAQEB..."

Hardware-Mirror Insight: Asymmetric decryption requires significantly more CPU Cycles than symmetric. If your cluster is very large, consider decrypting secrets on the Client side to distribute the CPU load, though this requires distributing the private key to every node (increasing security risk).

6. Dynamic Refresh with Spring Cloud Bus

When you change a property in Git, the services don't automatically update. You have three choices:

1. Restart: (Slow, Hardware intensive).
2. Manual /refresh: POST to every node (Impossible at scale).
3. Spring Cloud Bus: (Recommended).

By integrating Module 33: Spring Cloud Bus, you can broadcast a refresh event to the entire fleet via RabbitMQ or Kafka.

# Call this once on the Config Server or any instance
curl -X POST http://config-server:8888/actuator/busrefresh

This single call triggers a cluster-wide event, causing every node to re-fetch its configuration and re-instantiate @RefreshScope beans instantly.

7. Multi-Backend Strategy: Git + Vault

For high-security environments, you can combine Git (for non-sensitive app settings) with HashiCorp Vault (for secrets).

spring:
  profiles:
    active: git, vault
  cloud:
    config:
      server:
        git:
          uri: https://...
        vault:
          host: vault.internal.hardware.com
          port: 8200

This hybrid approach allows developers to manage logic-driven config in Git while the Security Team manages tokens and keys in the dedicated Vault hardware.

8. High Availability: Fail-Fast and Retries

If the Config Server is down, your application shouldn't just crash.

Implementation: Retry Logic

<dependency>
    <groupId>org.springframework.retry</groupId>
    <artifactId>spring-retry</artifactId>
</dependency>

spring:
  cloud:
    config:
      fail-fast: true
      retry:
        max-attempts: 6
        initial-interval: 1000

This ensures that if the Config Server is struggling with a Thundering Herd, the client will wait and retry rather than immediately dying.

9. Real-World Case Study: The "Phantom" Bootstrap Lag

The Scenario:

A team reported that their microservices were taking 25 seconds to start on their production hardware, compared to 5 seconds in staging.

The Diagnosis:

By analyzing the Network I/O and Disk Watcher on the Config Server, we found that the Config Server was set to clone-on-start: true and had no local cache. Every time a new pod started, the Config Server would perform a deep git clone from a remote GitLab instance across a high-latency VPN.

The Fix:

1. Enabled spring.cloud.config.server.git.basedir to use the local NVMe SSD as a cache.
2. Set clone-on-start: false to allow the server to serve existing cached data while fetching updates asynchronously.
3. Result: Startup time dropped from 25s to 3s.

10. Summary

Centralized configuration transforms your microservices from rigid, static binaries into a dynamic, "Living" cluster. By understanding the Hardware-Mirror implications of encryption CPU tax and NIC saturation, you can design a Config architecture that handles thousands of nodes and provides sub-second update propagation.

In the next module, Module 36: Service Discovery with Consul, we'll explore how services find the "Brain" and each other in a fluid hardware environment.

Next Steps:

1. Set up a Git repository with an application.yml.
2. Build a Config Server and point it to your repo.
3. Connect a client and verify it receives properties via curl localhost:8080/actuator/env.
4. Implement RSA Encryption and verify the Config Server decrypts values on-the-fly.

Frequently Asked Questions

Q: What is Spring Cloud Config Server and why use it over environment variables?

Spring Cloud Config Server is a centralised configuration service that stores and serves configuration for all your microservices from a Git repository (or Vault, filesystem, etc.). Unlike environment variables, it supports versioned config history, per-environment profiles, dynamic refresh without restart, and secret encryption at rest. It is the standard solution for managing configuration in a fleet of microservices that would otherwise require updating environment variables on every instance individually.

Q: How do I secure secrets in Spring Cloud Config?

Spring Cloud Config supports symmetric and asymmetric encryption of property values using the /encrypt and /decrypt endpoints. Store sensitive values encrypted in Git (prefixed with {cipher}) and the Config Server decrypts them on the fly before serving to clients. For production, integrate with HashiCorp Vault as a backend — Config Server has a native Vault backend that fetches secrets directly from Vault's secret store, keeping them out of Git entirely.

Q: What happens if the Config Server is unavailable when a service starts up?

By default, the client will fail fast and throw an exception, preventing startup. This is the spring.cloud.config.fail-fast=true behaviour, which is recommended for production. Alternatively, set fail-fast=false to fall back to local defaults. For resilience, run the Config Server as a highly available cluster behind a load balancer, or use spring.cloud.config.retry properties to retry with exponential backoff before giving up.