System Design with Java — Java

System design is about making architecture decisions that allow your application to handle scale, failure, and complexity. Java's ecosystem — Spring Cloud, Kafka, Redis, Docker — gives you production-grade tools for every challenge. This guide covers the most important patterns you will face in real-world Java backend systems.

Step 1 — Monolith vs Microservices

Monolith: When to Keep It

Single deployable unit: all code runs in one JVM process.
Simple to develop, test, and deploy initially.
Strong consistency: all data in one DB, no distributed transactions.
Best for: startups, small teams, MVP, apps under ~50 developers.
Problem: as teams grow, deployment coupling, long build times, scaling individual components is impossible.

Microservices: When to Migrate

Each service owns a specific business domain (User Service, Order Service, Payment Service).
Each service has its OWN database (Database per Service pattern).
Services communicate via REST APIs or messaging (Kafka, RabbitMQ).
Independent deployment: deploy Order Service without touching User Service.
Independent scaling: scale Payment Service x10 without scaling User Service.
Cost: network latency, eventual consistency, distributed tracing complexity.

Step 2 — Spring Cloud: Microservices Toolkit

Spring Cloud Components

Spring Cloud Gateway: API Gateway — single entry point, routing, rate limiting, auth.
Spring Cloud Netflix Eureka: Service Discovery — services register themselves, others look them up by name.
Spring Cloud Config: Centralized configuration server — manage config for all services in one place.
Spring Cloud LoadBalancer: Client-side load balancing (replacement for Ribbon).
Resilience4j: Circuit Breaker, Rate Limiter, Retry, Bulkhead.

server:
  port: 8080

spring:
  cloud:
    gateway:
      routes:
        - id: user-service
          uri: lb://USER-SERVICE   # 'lb://' = load-balanced via Eureka
          predicates:
            - Path=/api/users/**
          filters:
            - StripPrefix=0
            - name: RequestRateLimiter
              args:
                redis-rate-limiter.replenishRate: 100   # 100 req/sec
                redis-rate-limiter.burstCapacity: 200

        - id: order-service
          uri: lb://ORDER-SERVICE
          predicates:
            - Path=/api/orders/**
          filters:
            - AddRequestHeader=X-Gateway-Source, spring-cloud-gateway

        - id: auth-route
          uri: lb://USER-SERVICE
          predicates:
            - Path=/api/auth/**
          filters:
            - name: CircuitBreaker
              args:
                name: authCircuitBreaker
                fallbackUri: forward:/fallback/auth

// user-service/src/main/java/.../UserServiceApplication.java
@SpringBootApplication
@EnableEurekaClient // register this service with Eureka
public class UserServiceApplication {
    public static void main(String[] args) {
        SpringApplication.run(UserServiceApplication.class, args);
    }
}

// user-service/application.yml
// spring:
//   application:
//     name: USER-SERVICE   <- how other services discover this service
// eureka:
//   client:
//     serviceUrl:
//       defaultZone: http://eureka-server:8761/eureka/

// order-service: call user-service by name, not hardcoded URL
@Configuration
class RestConfig {
    @Bean
    @LoadBalanced // makes RestTemplate use service discovery
    public RestTemplate restTemplate() {
        return new RestTemplate();
    }
}

@Service
class OrderService {
    @Autowired
    private RestTemplate restTemplate;

    public UserDto getUserForOrder(Long userId) {
        // 'USER-SERVICE' resolved via Eureka — load balanced!
        return restTemplate.getForObject(
            "http://USER-SERVICE/api/users/" + userId,
            UserDto.class
        );
    }
}

Step 3 — Circuit Breaker Pattern with Resilience4j

When a downstream service is slow or failing, a naive caller will block threads waiting for timeout, potentially cascading the failure across your entire system. The Circuit Breaker pattern detects failures and 'opens' the circuit to fail fast, allowing the system to recover.

Circuit Breaker States

CLOSED (normal): requests pass through. Failure rate is monitored.
OPEN (failing): requests immediately fail with fallback. No calls to downstream service. Waits for wait duration.
HALF-OPEN (recovering): allows a limited number of test requests. If they succeed: CLOSED. If they fail: OPEN again.

import io.github.resilience4j.circuitbreaker.annotation.CircuitBreaker;
import io.github.resilience4j.retry.annotation.Retry;
import io.github.resilience4j.timelimiter.annotation.TimeLimiter;
import org.springframework.stereotype.Service;
import java.util.concurrent.CompletableFuture;

@Service
public class PaymentService {

    @CircuitBreaker(name = "paymentService", fallbackMethod = "paymentFallback")
    @Retry(name = "paymentService")           // retry before circuit opens
    @TimeLimiter(name = "paymentService")     // fail if takes > 2s
    public CompletableFuture<String> processPayment(PaymentRequest request) {
        return CompletableFuture.supplyAsync(() -> {
            // Call external payment gateway
            return externalPaymentGateway.charge(request);
        });
    }

    // Fallback: called when circuit is OPEN or an exception occurs
    private CompletableFuture<String> paymentFallback(
            PaymentRequest request, Throwable ex) {
        System.err.println("Payment service unavailable: " + ex.getMessage());
        // Could: queue payment for retry, return pending status, notify ops team
        return CompletableFuture.completedFuture("PAYMENT_QUEUED_FOR_RETRY");
    }
}

// application.yml configuration:
// resilience4j:
//   circuitbreaker:
//     instances:
//       paymentService:
//         registerHealthIndicator: true
//         slidingWindowSize: 10          # evaluate last 10 calls
//         minimumNumberOfCalls: 5
//         failureRateThreshold: 50       # open if 50% of calls fail
//         waitDurationInOpenState: 10s   # stay open 10s before half-open
//         permittedCallsInHalfOpenState: 3
//   retry:
//     instances:
//       paymentService:
//         maxAttempts: 3
//         waitDuration: 500ms
//         retryExceptions:
//           - java.net.ConnectException
//           - java.util.concurrent.TimeoutException
//   timelimiter:
//     instances:
//       paymentService:
//         timeoutDuration: 2s

Step 4 — Event-Driven Architecture with Apache Kafka

In microservices, synchronous REST calls create tight coupling and cascading failures. Kafka enables asynchronous, decoupled communication: services publish events to topics, other services subscribe and react independently. This gives you resilience, decoupling, and the ability to replay events.

<dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka</artifactId>
</dependency>

spring:
  kafka:
    bootstrap-servers: localhost:9092
    producer:
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      value-serializer: org.springframework.kafka.support.serializer.JsonSerializer
      acks: all             # wait for all replicas to acknowledge (strongest durability)
      retries: 3
    consumer:
      group-id: order-service-group
      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      value-deserializer: org.springframework.kafka.support.serializer.JsonDeserializer
      auto-offset-reset: earliest  # read from beginning if no committed offset
      enable-auto-commit: false     # manual commit for exactly-once processing
      properties:
        spring.json.trusted.packages: "com.yourapp.events"

import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.support.SendResult;
import org.springframework.stereotype.Component;
import java.util.concurrent.CompletableFuture;

// Event record (DTO published to Kafka)
record OrderPlacedEvent(
    String orderId,
    Long customerId,
    double totalAmount,
    String status,
    java.time.Instant timestamp
) {}

@Component
public class OrderEventPublisher {

    private static final String TOPIC = "order-events";
    private final KafkaTemplate<String, OrderPlacedEvent> kafkaTemplate;

    public OrderEventPublisher(KafkaTemplate<String, OrderPlacedEvent> template) {
        this.kafkaTemplate = template;
    }

    public void publishOrderPlaced(OrderPlacedEvent event) {
        // Key = orderId: ensures all events for the same order go to the same partition
        // This guarantees ordering of events for a given order
        CompletableFuture<SendResult<String, OrderPlacedEvent>> future =
            kafkaTemplate.send(TOPIC, event.orderId(), event);

        future.whenComplete((result, ex) -> {
            if (ex != null) {
                System.err.println("Failed to publish event: " + ex.getMessage());
                // In production: write to outbox table for retry (Transactional Outbox pattern)
            } else {
                System.out.printf("Published to %s partition %d offset %d%n",
                    result.getRecordMetadata().topic(),
                    result.getRecordMetadata().partition(),
                    result.getRecordMetadata().offset());
            }
        });
    }
}

import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.support.Acknowledgment;
import org.springframework.stereotype.Component;

@Component
public class NotificationConsumer {

    @KafkaListener(
        topics = "order-events",
        groupId = "notification-service-group",
        containerFactory = "kafkaListenerContainerFactory"
    )
    public void handleOrderPlaced(
            ConsumerRecord<String, OrderPlacedEvent> record,
            Acknowledgment ack) { // manual acknowledgment
        try {
            OrderPlacedEvent event = record.value();
            System.out.printf("Processing order %s for customer %d%n",
                event.orderId(), event.customerId());

            // Process the event
            sendConfirmationEmail(event);
            updateAnalytics(event);

            // Commit offset ONLY after successful processing
            // If we crash here, Kafka will redeliver from last committed offset
            ack.acknowledge();

        } catch (Exception e) {
            System.err.println("Failed to process event: " + e.getMessage());
            // Don't acknowledge — Kafka will redeliver
            // After max retries, event goes to Dead Letter Topic (DLT)
        }
    }

    // Handle dead letter topic — events that couldn't be processed after all retries
    @KafkaListener(topics = "order-events.DLT", groupId = "dlt-handler")
    public void handleDeadLetter(ConsumerRecord<String, OrderPlacedEvent> record) {
        System.err.println("Dead letter: " + record.value());
        // Alert ops team, store to DB for manual inspection
    }

    private void sendConfirmationEmail(OrderPlacedEvent event) { /* ... */ }
    private void updateAnalytics(OrderPlacedEvent event) { /* ... */ }
}

Step 5 — Saga Pattern: Distributed Transactions

In microservices, you cannot use a single database transaction spanning multiple services. The Saga pattern coordinates distributed transactions through a sequence of local transactions, with compensating transactions for rollback on failure.

Choreography vs Orchestration Saga

Choreography: services react to events from each other. No central coordinator. Simple, but hard to track the overall flow when there are many services.
Orchestration: a central Saga Orchestrator tells each service what to do and handles failures. Easier to visualize and manage. Use Axon Framework or Temporal for this.
Example — Place Order Saga: 1) Order Service creates Order (PENDING). 2) Payment Service charges card. 3) Inventory Service reserves items. 4) Shipping Service schedules delivery. 5) Order Service marks CONFIRMED.
Compensation: If step 3 fails: Inventory → rollback, Payment Service → refund charge, Order Service → mark CANCELLED.

import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

// This is a simplified orchestration saga — production use Axon or Temporal
@Service
public class PlaceOrderSaga {

    private final OrderRepository orderRepository;
    private final PaymentServiceClient paymentClient;
    private final InventoryServiceClient inventoryClient;
    private final KafkaTemplate<String, ?> kafka;

    public PlaceOrderSaga(OrderRepository orderRepo,
                           PaymentServiceClient paymentClient,
                           InventoryServiceClient inventoryClient,
                           KafkaTemplate<String, ?> kafka) {
        this.orderRepository  = orderRepo;
        this.paymentClient    = paymentClient;
        this.inventoryClient  = inventoryClient;
        this.kafka            = kafka;
    }

    @Transactional
    public Order execute(PlaceOrderCommand command) {
        // Step 1: Create order in PENDING state
        Order order = new Order(command.customerId(), command.items());
        order.setStatus(OrderStatus.PENDING);
        orderRepository.save(order);

        String paymentId = null;
        try {
            // Step 2: Charge payment
            paymentId = paymentClient.charge(
                command.customerId(), order.getTotalAmount());

            // Step 3: Reserve inventory
            inventoryClient.reserve(order.getId(), command.items());

            // Step 4: Confirm order
            order.setStatus(OrderStatus.CONFIRMED);
            orderRepository.save(order);

            // Step 5: Publish event for downstream services
            kafka.send("order-events",
                new OrderConfirmedEvent(order.getId(), order.getCustomerId()));

            return order;

        } catch (PaymentException e) {
            // Payment failed — only cancel order, nothing else was done
            order.setStatus(OrderStatus.CANCELLED);
            orderRepository.save(order);
            throw new OrderException("Payment failed: " + e.getMessage());

        } catch (InventoryException e) {
            // Inventory failed — COMPENSATE: refund payment
            if (paymentId != null) {
                paymentClient.refund(paymentId); // compensating transaction
            }
            order.setStatus(OrderStatus.CANCELLED);
            orderRepository.save(order);
            throw new OrderException("Inventory unavailable: " + e.getMessage());
        }
    }
}

Step 6 — Caching with Redis

import org.springframework.cache.annotation.*;
import org.springframework.stereotype.Service;

// Add to application.yml:
// spring:
//   cache:
//     type: redis
//   data:
//     redis:
//       host: localhost
//       port: 6379
//       timeout: 2000ms

// Add to main class: @EnableCaching

@Service
@CacheConfig(cacheNames = "users") // default cache name for this service
public class CachedUserService {

    @Cacheable(key = "#id") // cache result; skip method if cache hit
    public UserResponse getUserById(Long id) {
        // Only called on cache MISS. Result is stored in Redis with key "users::1"
        return userRepository.findById(id)
            .map(this::toResponse)
            .orElseThrow();
    }

    @Cacheable(key = "'all'") // literal key
    public List<UserResponse> getAllUsers() {
        return userRepository.findAll().stream().map(this::toResponse).toList();
    }

    @CachePut(key = "#result.id") // update cache after method executes
    public UserResponse updateUser(Long id, UserUpdateRequest req) {
        // Always executes (not skipped). Puts fresh result into cache.
        return doUpdate(id, req);
    }

    @CacheEvict(key = "#id") // remove entry from cache
    public void deleteUser(Long id) {
        userRepository.deleteById(id);
    }

    @CacheEvict(allEntries = true) // clear entire cache
    public void clearCache() {}
}

// For more control: use RedisTemplate directly
import org.springframework.data.redis.core.RedisTemplate;

@Service
public class RateLimiterService {
    private final RedisTemplate<String, Long> redis;

    public boolean isAllowed(String userId, int maxRequests, long windowSeconds) {
        String key = "rate_limit:" + userId;
        Long count = redis.opsForValue().increment(key);
        if (count == 1) {
            redis.expire(key, windowSeconds, java.util.concurrent.TimeUnit.SECONDS);
        }
        return count <= maxRequests;
    }
}

Step 7 — Distributed Tracing with Micrometer + Zipkin

When a request spans 5 microservices, finding where latency or errors occur is nearly impossible without distributed tracing. Every request gets a unique Trace ID that flows across all services.

<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-tracing-bridge-otel</artifactId>
</dependency>
<dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-exporter-zipkin</artifactId>
</dependency>

management:
  tracing:
    sampling:
      probability: 1.0  # trace 100% of requests (use 0.1 in high-traffic prod)
  zipkin:
    tracing:
      endpoint: http://zipkin:9411/api/v2/spans

logging:
  pattern:
    level: "%5p [%X{traceId:-},%X{spanId:-}]" # include trace/span IDs in every log line

Step 8 — Key Architecture Decisions and Patterns

Database per Service Pattern

Each microservice has its OWN database. No service touches another service's DB directly.
User Service → PostgreSQL. Order Service → PostgreSQL. Catalog Service → MongoDB. Session → Redis.
Services share data through APIs or events — never through shared DB tables.
This enables independent scaling, different DB technologies, and prevents schema coupling.

Outbox Pattern: Reliable Event Publishing

Problem: you save to DB and publish to Kafka in the same operation — what if Kafka publish fails after DB commit? Events are lost.
Solution: write the event to an 'outbox' table in the SAME DB transaction as your business data. A separate poller reads the outbox and publishes to Kafka, then marks published.
This gives you exactly-once semantics: if the poller crashes, it restarts and re-publishes from the outbox. Idempotent consumers handle duplicates.
Tools: Debezium (CDC-based), custom Spring Scheduled poller.

CQRS: Command Query Responsibility Segregation

Separate the read model from the write model.
Write side: validates commands, updates the main DB, publishes events.
Read side: subscribes to events, builds denormalized read-optimized projections in a separate store (Elasticsearch, Redis, a read replica).
Benefit: read queries hit a fast, pre-aggregated store. Writes go to a normalized DB. Both can scale independently.
Java stack: Axon Framework, Spring + Kafka + Elasticsearch.