Routing Reads to Aurora Reader Endpoints with Connection Pools

This guide is part of AWS Aurora Connection Scaling. Aurora gives every DB cluster a reader endpoint, a DNS name that resolves to one of the available Aurora Replicas each time it is looked up. That is load balancing per connection, not per query. A connection pool opens its connections once and keeps them for as long as maxLifetime allows, so the distribution of reads is decided at the moment the pool was filled and stays that way. The typical symptom appears after Aurora Auto Scaling adds replicas during a traffic peak:

CloudWatch, 14:20 UTC, DB cluster orders-aurora
  orders-aurora-instance-2   CPUUtilization 91%   DatabaseConnections 118
  orders-aurora-instance-3   CPUUtilization 88%   DatabaseConnections 114
  orders-aurora-instance-4   CPUUtilization  4%   DatabaseConnections   2   (added 14:02 by auto scaling)
  orders-aurora-instance-5   CPUUtilization  3%   DatabaseConnections   1   (added 14:05 by auto scaling)

The new replicas are healthy and idle while the original ones saturate. The objective is to make pools pick up new replicas within minutes, spread connections evenly, and keep reads correct when replica lag or a failover changes which instance is behind the endpoint.

Rapid incident diagnosis

Compare per-instance connections and CPU. The imbalance is visible in CloudWatch per instance, or directly from each replica:

for id in orders-aurora-instance-2 orders-aurora-instance-3 orders-aurora-instance-4 orders-aurora-instance-5; do
  aws cloudwatch get-metric-statistics --namespace AWS/RDS --metric-name DatabaseConnections \
    --dimensions Name=DBInstanceIdentifier,Value=$id --statistics Average \
    --start-time $(date -u -d '-10 min' +%FT%TZ) --end-time $(date -u +%FT%TZ) --period 600 \
    --query "Datapoints[0].Average" --output text | xargs echo "$id"
done

Check how old the pool’s connections are. On a PostgreSQL-compatible replica, backend start times show whether connections predate the replicas that were added:

SELECT application_name, count(*) AS conns,
       min(backend_start) AS oldest, max(backend_start) AS newest
FROM pg_stat_activity
WHERE backend_type = 'client backend'
GROUP BY application_name;

Connections that are hours old on the busy replicas confirm the pool is not recycling.

Check DNS caching in the client. The reader endpoint’s DNS record has a very short TTL, but a JVM or a sidecar may cache resolutions much longer. In a JVM, the effective setting is networkaddress.cache.ttl:

jcmd $(pgrep -f orders-api) VM.system_properties | grep -i networkaddress
jcmd $(pgrep -f orders-api) VM.command_line

If a cached address is reused, every new connection within the cache period goes to the same replica, even when the pool does recycle.

Separate imbalance from lag. If reads return stale data rather than being slow, check AuroraReplicaLag per instance. Lag is usually tens of milliseconds; values in seconds indicate a replica struggling with write-heavy replay, and routing reads away from it is a different fix from rebalancing.

Connection distribution across Aurora replicas behind the reader endpoint An application pool of 240 connections resolved the reader endpoint while only instances 2 and 3 existed, so all connections sit on them. Instances 4 and 5, added later by auto scaling, receive almost none. App pools 240 connections opened at 13:40 Reader endpoint DNS per connection instance-2 · 118 connections · CPU 91% instance-3 · 114 connections · CPU 88% instance-4 · 2 connections · CPU 4% instance-5 · 1 connection · CPU 3% added by auto scaling at 14:02 and 14:05
The reader endpoint balances at connection time. Pools filled before auto scaling added replicas keep every connection on the original two instances until those connections are recycled.

Mathematical sizing or parameter formula

Time to rebalance. A pool with maximum connection lifetime L replaces its connections continuously; HikariCP applies a small random variance so they do not all expire together. After a replica is added, the share of connections opened since then — each of which has a fair chance of landing on the new replica — grows roughly linearly:

rebalanced_fraction(t) ≈ min( 1, t ÷ L )
time_to_balance        ≈ L + dns_cache_ttl

With maxLifetime of 30 minutes, a replica added at 14:02 carries its share only by about 14:32 — long after the peak it was added for. With 5 minutes, it is balanced by roughly 14:07.

Cost of shorter lifetimes. Every expiry is a new connection: TCP, TLS and authentication, typically 20–60 ms on Aurora, plus IAM token generation if IAM authentication is used. The rate of new connections is:

new_connections_per_second = pool_connections ÷ L

240 connections with a 5-minute lifetime means 240 ÷ 300 = 0.8 new connections per second across the fleet — negligible for the database. At 30 seconds it would be 8 per second, which starts to matter for CPU on small instances and for IAM rate limits.

Per-replica target. For even load, each replica should hold about:

connections_per_replica = total_reader_pool_connections ÷ replica_count

240 connections over 4 replicas is 60 each. Check this against each instance’s max_connections, which for Aurora is derived from instance memory — and note that replicas added by auto scaling may be the same class, but an instance manually added to a mixed-class DB cluster may have a much lower ceiling.

Rebalancing after a replica is added for two maxLifetime values The share of connections on a newly added replica rises to its fair share of 25 percent within 5 minutes with a 5 minute maxLifetime, but takes 30 minutes with a 30 minute maxLifetime. maxLifetime 5 min maxLifetime 30 min fair share 25% of 4 replicas replica added +5 min +30 min still underused while the peak lasts
Shorter connection lifetimes move load onto a new replica sooner at the cost of more reconnections. Five minutes balances quickly while adding under one new connection per second.

Exact remediation & configuration

Shorten connection lifetime for reader pools only. Writer pools do not benefit, and keeping their lifetime longer avoids unnecessary reconnections. With HikariCP in Spring Boot:

app:
  datasource:
    reader:
      jdbc-url: jdbc:postgresql://orders-aurora.cluster-ro-abc123.eu-west-1.rds.amazonaws.com:5432/orders
      maximum-pool-size: 20
      minimum-idle: 5
      max-lifetime: 300000        # 5 minutes: new replicas pick up load quickly
      keepalive-time: 60000
      idle-timeout: 120000
    writer:
      jdbc-url: jdbc:postgresql://orders-aurora.cluster-abc123.eu-west-1.rds.amazonaws.com:5432/orders
      maximum-pool-size: 20
      max-lifetime: 1800000       # 30 minutes

Stop the JVM from caching the reader address. Set the DNS cache TTL low at startup — before any lookup happens — so each new connection resolves the endpoint again:

JAVA_TOOL_OPTIONS="-Dsun.net.inetaddr.ttl=1 -Dsun.net.inetaddr.negative.ttl=1"

The equivalent security property is networkaddress.cache.ttl=1 in java.security. Node.js does not cache DNS by default, but many HTTP and database client libraries install a caching lookup; check yours.

Or select replicas in the client. DNS round-robin with a short TTL still distributes randomly, and with a handful of pods the result can be uneven. The AWS Advanced JDBC Wrapper discovers the DB cluster’s topology and chooses reader instances itself, with a selectable strategy:

app:
  datasource:
    reader:
      jdbc-url: jdbc:aws-wrapper:postgresql://orders-aurora.cluster-ro-abc123.eu-west-1.rds.amazonaws.com:5432/orders
      data-source-properties:
        wrapperPlugins: initialConnection,auroraConnectionTracker,failover2,efm2
        readerInitialConnectionHostSelectorStrategy: roundRobin
        wrapperDialect: aurora-pg

Route stale-sensitive reads to the writer. A read that must see a write the same request just made — read-your-writes — should use the writer pool. Aurora replica lag is small but not zero, and reader rebalancing does not change that. The application-level routing pattern is in Configuring Spring Boot Read Replica DataSource Routing.

Keep headroom for failover. When the writer fails, Aurora promotes a replica; for a short period there is one fewer reader, and the reader endpoint’s connections on the promoted instance are dropped. Size the reader pool so that total_reader_connections ÷ (replica_count − 1) still fits each instance’s max_connections. Failover handling in the pool is covered in Handling Aurora Failover in Connection Pools.

DNS-based and driver-based reader selection With DNS, each new connection resolves the reader endpoint, which returns a random replica if the client does not cache the address. With a topology-aware driver, the pool queries DB cluster topology and opens each connection to the next replica in turn. New connection maxLifetime expired Resolve reader endpoint DNS cache TTL 1 s Random replica even on average New connection maxLifetime expired Topology query instance list Next replica in rotation even with few pods Both need connections to expire before they can rebalance
Two mechanisms spread reads. DNS with a short cache distributes at random per connection; a topology-aware driver rotates through the known readers explicitly.

Validation & verification

Rebalance after scaling. In staging, run a steady read load, then add a replica and record connections per instance every minute:

aws rds create-db-instance --db-instance-identifier orders-aurora-instance-6 \
  --db-cluster-identifier orders-aurora --engine aurora-postgresql --db-instance-class db.r7g.large
# then re-run the per-instance DatabaseConnections loop from the diagnosis section once a minute

Within about maxLifetime plus a minute of the new instance becoming available, its connection count should approach total ÷ replicas, and its CPUUtilization should approach the others’.

DNS resolution per connection. From inside a pod, resolve the endpoint repeatedly; the addresses should vary:

for i in $(seq 1 20); do getent hosts orders-aurora.cluster-ro-abc123.eu-west-1.rds.amazonaws.com | awk '{print $1}'; sleep 1; done | sort | uniq -c

Then confirm the application’s own connections vary too, by checking the replica each new pooled connection landed on:

SELECT aurora_db_instance_identifier();   -- run through the reader pool several times

Reconnection cost. Check that the new-connection rate matches the formula and that reconnections do not show up in request latency. On each replica, pg_stat_database.sessions (PostgreSQL 14 and later) should grow by roughly pool_connections ÷ maxLifetime_seconds per second across the fleet.

Failover. Trigger a failover in staging with aws rds failover-db-cluster --db-cluster-identifier orders-aurora and confirm reader traffic continues with errors limited to the connections on the promoted instance, and that all replicas carry load again within one maxLifetime.

Reader auto scaling and pool sizing together

Aurora Auto Scaling adds and removes replicas based on a target metric, usually average reader CPU or average connections. Both interact with connection pools in ways that can defeat the policy.

CPU-based scaling with sticky pools. When pools do not rebalance, a new replica stays idle and average reader CPU barely falls, so the policy keeps adding replicas that receive no traffic. With maxLifetime shortened as described above, each new replica absorbs load within minutes and the average falls, which lets the policy settle. Set the policy’s scale-out cooldown longer than maxLifetime, so it waits to see the effect of the last replica before adding another:

aws application-autoscaling put-scaling-policy \
  --service-namespace rds --resource-id cluster:orders-aurora \
  --scalable-dimension rds:cluster:ReadReplicaCount \
  --policy-name reader-cpu-60 --policy-type TargetTrackingScaling \
  --target-tracking-scaling-policy-configuration '{
    "TargetValue": 60.0,
    "PredefinedMetricSpecification": { "PredefinedMetricType": "RDSReaderAverageCPUUtilization" },
    "ScaleOutCooldown": 600,
    "ScaleInCooldown": 900
  }'

Connection-based scaling. Scaling on RDSReaderAverageDatabaseConnections measures pool sizes, not load. Application pools with a fixed minimum hold the same number of connections whether busy or idle, so the metric responds to deployments and pod counts rather than traffic. Prefer CPU for most workloads.

Scale-in drops connections. When a replica is removed, its connections fail on next use. Pools with validation or keepaliveTime replace them quickly, but requests using those connections at the moment of removal see errors. A long scale-in cooldown and a minimum replica count that covers baseline traffic limit how often it happens.

Setting Recommended starting value Reason
Reader pool maxLifetime 5 minutes New replicas absorb load quickly
Scale-out cooldown 10 minutes (longer than maxLifetime) Observe the last replica’s effect before adding another
Scale-in cooldown 15 minutes Avoid oscillation and repeated connection drops
Minimum replicas Enough for baseline reads plus one Failover and scale-in headroom
Target metric Reader average CPU Tracks load, not pool configuration

Frequently Asked Questions

Would a very short maxLifetime such as 30 seconds balance even faster?
Yes, but it trades rebalancing speed for constant reconnection work, IAM token requests and occasional latency spikes when a request waits for a fresh connection. Minutes, not seconds, is the practical range.
Does RDS Proxy solve reader balancing?
An RDS Proxy reader endpoint distributes its backend connections across replicas and can hide replica churn from clients, but the application’s connections to the proxy still persist, and the proxy’s own balancing applies per borrowed backend connection. It helps most for many short-lived clients such as functions.
What about custom endpoints?
Custom endpoints group specific instances — for example, larger instances for analytical reads. They behave like the reader endpoint for balancing, so the same lifetime and DNS guidance applies to each custom endpoint’s pool.
Should the writer pool also have a short lifetime?
It helps after a failover, when the writer endpoint moves to a new instance: connections that survive on the old writer — now a reader — would fail on writes. Failover-aware drivers handle this directly; without one, a moderate writer lifetime of 10–30 minutes bounds how long stale connections survive.
Do reader pools need a different acquisition timeout from writer pools?
Often a shorter one. A read that cannot get a connection quickly can sometimes fall back to the writer or to a cache, whereas a write has nowhere else to go. Shorter reader timeouts combined with a fallback keep read latency bounded during replica churn.