Typical N-tier architecture under load:
  10,000 concurrent users
  → 10,000 requests to application servers (scale horizontally ✓)
  → 10,000 database queries (cannot scale horizontally ✗)
  
Database connection limits:
  PostgreSQL: ~200 connections per instance
  10,000 requests → 9,800 requests queued → latency explodes
  
Adding read replicas helps reads, but not writes.
Sharding helps writes, but adds complexity and can't do cross-shard joins.

┌─────────────────────────────────────────────────────┐
│                Processing Unit (PU)                  │
│  ┌──────────────┐  ┌────────────────────────────┐   │
│  │  Application │  │  In-Memory Data Grid (IMDG) │   │
│  │  Logic       │←→│  (local partition of data)  │   │
│  └──────────────┘  └────────────────────────────┘   │
│         ↕ messaging                                  │
└─────────────────────────────────────────────────────┘
         ↑                     ↑
         PU 1                  PU 2 (identical, different data partition)

Shared infrastructure:
┌─────────────────────────────────────────────────────┐
│  Messaging Grid (async event bus between PUs)        │
│  Virtualization/Middleware (routing, load balancing) │
│  Data Grid (distributed IMDG coordinator)            │
└─────────────────────────────────────────────────────┘

Persistence layer (async, not on hot path):
┌─────────────────────────────────────────────────────┐
│  Database writers                                    │
│  Database (PostgreSQL, Cassandra)                    │
└─────────────────────────────────────────────────────┘

// Processing unit startup
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.map.IMap;
import com.hazelcast.config.Config;

public class OrderProcessingUnit {
    private final HazelcastInstance hz;
    private final IMap<String, Order> orders;
    private final IMap<String, UserSession> sessions;

    public OrderProcessingUnit() {
        Config config = new Config();
        config.setClusterName("production");
        
        // Configure data partitioning and replication
        config.getMapConfig("orders")
            .setBackupCount(1)              // 1 synchronous backup
            .setAsyncBackupCount(1)         // 1 async backup
            .setEvictionConfig(            // evict LRU when memory > 80%
                new EvictionConfig()
                    .setEvictionPolicy(EvictionPolicy.LRU)
                    .setMaxSizePolicy(MaxSizePolicy.USED_HEAP_PERCENTAGE)
                    .setSize(80)
            )
            .setTimeToLiveSeconds(86400);   // 24h TTL for order data

        // Near cache: per-node local cache for hot reads
        config.getMapConfig("sessions")
            .getNearCacheConfig()
            .setTimeToLiveSeconds(300)
            .setMaxIdleSeconds(60);

        this.hz = Hazelcast.newHazelcastInstance(config);
        this.orders = hz.getMap("orders");
        this.sessions = hz.getMap("sessions");
    }

    // Write to in-memory grid (no database call)
    public Order createOrder(CreateOrderRequest request) {
        Order order = Order.create(request.getUserId(), request.getItems());
        
        // Atomic put-if-absent (distributed)
        if (orders.putIfAbsent(order.getId(), order) != null) {
            throw new DuplicateOrderException(order.getId());
        }
        
        // Publish event for async DB write and downstream consumers
        publishEvent(new OrderCreatedEvent(order));
        
        return order;
    }

    // Read from memory — sub-millisecond
    public Optional<Order> getOrder(String orderId) {
        return Optional.ofNullable(orders.get(orderId));
    }

    // Distributed lock for critical sections
    public void confirmOrder(String orderId) {
        // Lock prevents two PUs from confirming the same order simultaneously
        var lock = hz.getCPSubsystem().getLock("order:" + orderId);
        lock.lock();
        try {
            Order order = orders.get(orderId);
            if (order == null) throw new OrderNotFoundException(orderId);
            order.confirm();
            orders.put(orderId, order);
        } finally {
            lock.unlock();
        }
    }
}

// Event publishing via Hazelcast topic
public class EventBus {
    private final HazelcastInstance hz;

    public void publish(DomainEvent event) {
        hz.getTopic(event.getTopicName()).publish(event);
    }

    public <T extends DomainEvent> void subscribe(
        String topic,
        MessageListener<T> listener
    ) {
        hz.getTopic(topic).addMessageListener(listener);
    }
}

// Async database writer — listens for events and persists asynchronously
@Component
public class OrderPersistenceWorker {
    
    @PostConstruct
    public void start() {
        eventBus.subscribe("order.created", message -> {
            OrderCreatedEvent event = message.getMessageObject();
            // Write to database asynchronously (not on hot path)
            orderRepository.save(event.getOrder());
        });
        
        eventBus.subscribe("order.confirmed", message -> {
            OrderConfirmedEvent event = message.getMessageObject();
            orderRepository.updateStatus(event.getOrderId(), OrderStatus.CONFIRMED);
        });
    }
}

Hazelcast partitioning with 271 partitions (default):
  4 PU nodes, each owns ~67 partitions
  Each partition has 1 primary copy + 1 backup

  Partition assignment by consistent hashing of key:
  key "order:abc123" → hash → partition 142 → PU node 2 (primary), PU node 3 (backup)

  Node joins/leaves: only ~1/N of partitions migrate

// Custom partition strategy to co-locate related data
// All of a user's orders go to the same partition
public class UserAwarePartitionStrategy implements PartitioningStrategy {
    @Override
    public Object getPartitionKey(Object key) {
        if (key instanceof String orderId && orderId.startsWith("order:")) {
            // Extract userId from "order:{userId}:{uuid}"
            return orderId.split(":")[1];
        }
        return key;
    }
}

// Apply to map config
config.getMapConfig("orders")
    .setPartitioningStrategyConfig(
        new PartitioningStrategyConfig(UserAwarePartitionStrategy.class.getName())
    );

Failure scenario:
  4 PU nodes, replication factor = 1 (1 primary + 1 backup)
  
  PU Node 2 fails:
  1. Hazelcast detects failure via heartbeat (default: 5 seconds)
  2. Partitions from Node 2 promoted from backup to primary on Node 3
  3. New backups created on remaining nodes
  4. Traffic continues — brief pause during promotion (~200ms)
  5. When Node 2 restarts, it rejoins and receives its partitions back

// Listen for partition events
hz.getPartitionService().addMigrationListener(new MigrationListener() {
    @Override
    public void migrationStarted(MigrationEvent event) {
        log.info("Partition {} migrating from {} to {}",
            event.getPartitionId(), event.getOldMember(), event.getNewMember());
    }
    
    @Override
    public void migrationCompleted(MigrationEvent event) {
        metrics.increment("partition.migration.completed");
    }
    
    @Override
    public void migrationFailed(MigrationEvent event) {
        alerts.fire("Partition migration failed: " + event.getPartitionId());
    }
});

// Elastic scaling: new PU joins and receives its share of partitions
// Kubernetes horizontal pod autoscaler works naturally with Hazelcast

// HPA configuration in Kubernetes
/*
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: order-processing-unit
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: order-pu
  minReplicas: 3
  maxReplicas: 20
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: External
    external:
      metric:
        name: hazelcast_owned_partition_count
      target:
        type: AverageValue
        averageValue: "30"  # scale up when each node owns > 30 partitions
*/

Microservices: independent services, each with own DB
  Problem: still hits DB bottleneck per service at high load

CQRS: separate read/write models
  Problem: eventual consistency, but DB is still the write bottleneck

SBA: in-memory state, async DB writes
  Advantage: removes DB from hot path entirely
  Cost: operational complexity, memory costs, eventual persistence

Best combined pattern:
  SBA for high-concurrency write path (order creation, session management)
  CQRS read model (Elasticsearch, PostgreSQL) for complex queries
  Event sourcing for audit trail and replay