Capacity, scaling and availability#
What one workspace can be expected to do, which parts of the services keep state in process memory (and therefore limit you to one instance), where the bottlenecks are, and what has to change to run more than one replica. No load test has been run (by design of the security plan, and not yet as a performance exercise): the figures here are derived from the code and the reference installation, not measured throughput.
1. Reference installation#
The release was installed and exercised on a single arm64 VM (Oracle Always Free Ampere A1, up to 4 OCPU / 24 GB), one
workspace, nginx on the host, Garage as the object store. The public sandbox runs the whole stack inside these container
limits (docker-compose.sandbox.yml), which is the smallest configuration that has been operated:
| Container | Memory | CPU |
|---|---|---|
| core-api | 768 MB | 1.0 |
| asset-processor | 1 GB (image decoding with sharp) | 1.0 |
| gateway | 384 MB | 0.5 |
| admin-dashboard, editor (nginx) | 256 MB each | 0.5 each |
| postgres | 512 MB | 1.0 |
| garage | 512 MB | 1.0 |
That is about 3.7 GB of memory for a small, low-traffic workspace with capped assets. A production workspace with real traffic, large images or the page generators needs more; plan from the table below and measure.
2. Where the load goes#
| Work | Cost driver | Notes |
|---|---|---|
| Every authenticated request | 3 database queries (session + user, last_seen update, effective permissions) |
no session cache; the sessions table is written on every request |
| Public read API | published content filtered in application code, per-address limits (3000 / min) | pages are small JSON trees; search runs on search_documents; published static sites do not call the API at all, SSR/ISR sites call it from one address |
| Image upload and transforms | sharp decoding, up to 100 MP per image, 50 MB per file | memory spikes in asset-processor; the main reason for its 1 GB |
| Form upload bodies | held in memory up to the website's limit (never above 25 MB) per concurrent upload | worst case about 25 MB × concurrent uploads; limited by 30 bodies per address per minute |
| Website export / import | the whole bundle is built or parsed in core-api memory (the proxy allows 100 MB bodies) | run large imports off-peak |
Site generation (publish-site.sh) |
CPU and disk of the machine that runs it (Next/Nuxt build) | not part of the services |
| AI calls | upstream latency; 20 / minute per user | the provider is the limit |
| Gateway proxy | upstream latency, 30 s timeout, per-key rps limits | |
| Database connections | each backend opens a pg pool with the library default (10 connections) |
3 backends ≈ 30 connections against PostgreSQL's default max_connections of 100; replicas multiply this |
Starting point for a production workspace (to be confirmed by your own test): 2 vCPU / 4 GB for the three backends and PostgreSQL together, plus disk for the assets; scale memory first (asset-processor), then CPU.
3. State that lives in process memory#
These make a service per-process: with two replicas each keeps its own copy, which changes behaviour (not data):
| In memory | Service | Effect with more than one replica or after a restart |
|---|---|---|
| Login failure counters (5 per address+e-mail, 25 per e-mail, 15-minute lock) | core-api | the lockout threshold multiplies by the number of replicas; a restart clears locks |
Pending MFA challenges (mfaToken) |
core-api | a login that reaches replica A for the password step and replica B for the code step fails; a restart cancels logins in progress |
| MFA code failure locks, used TOTP codes (replay guard), forgot-password counters | core-api | replay protection and limits are per replica |
| Rate limiters: public, search, query, form submit, upload approval, PWA subscribe, AI | core-api | per-replica counts |
| Upload body limiter | asset-processor | per replica |
| Gateway per-key rate limit (sliding window) | gateway | per replica; GATEWAY_RATE_LIMIT_BACKEND=redis is accepted but not implemented (falls back to memory) |
| Authored-notify and authored-upload-field caches (30 s), site-root cache | core-api | a page published on A is seen by B within 30 s |
| Form-notify recipient throttle (3 mails / hour) | core-api | per replica |
| Mail-provider and preset files watched in memory | core-api, gateway | consistent if the files are the same |
What is not in memory: sessions (database), users and permissions (database), all content, upload tickets (signed, stateless), upload rows (database). So a second replica does not need sticky sessions for normal use; only the MFA challenge needs the same replica for both steps.
Other single-instance behaviours:
- Migrations run at start in core-api with no lock. Start one core-api first; the compose file makes the others wait.
- Timers: the clean-up of unused form uploads runs in every asset-processor replica (idempotent).
- Startup tasks (admin seed, role seed, backfills) run in every core-api start and are idempotent.
4. What it takes to run more than one replica#
Not supported today; the work, in order of importance:
- Move the pending MFA challenges, login/MFA/forgot-password counters and the public/upload/form limiters to a shared store (the sessions table or Redis), or put them behind a sticky proxy rule.
- Implement the Redis backend of the gateway limiter that the configuration already names.
- Take a PostgreSQL advisory lock around the migration step (or run migrations as a separate job).
- Size PostgreSQL connections (
pool max) for the replicas, or add a pooler (PgBouncer). - Shared
LOG_DIRis not needed (logs go to stdout and the DAM); the object store is already shared. - Test the whole thing: there is no multi-replica test.
5. Availability#
- No high availability. One container per service per workspace; Compose
restart: unless-stoppedbrings a crashed process back in seconds; the proxy returns 502 meanwhile. Planned downtime: an update recreates the containers. - A database outage makes every API route fail (health stays green, see observability).
- Published static sites keep serving from their own hosting while the CMS is down; sites rendered on a server or incrementally regenerated show stale content or errors depending on their cache settings.
- For a managed-database setup point
DB_HOSTat it withDB_SSL=verify-full; the application tolerates the database restarting (the pool reconnects) but in-flight requests fail.
6. Limits that are enforced#
| Limit | Value | Where |
|---|---|---|
| Request body | 100 MB (WS_MAX_BODY) |
nginx |
| JSON body | 50 MB (1 MB in the sandbox) | core-api, asset-processor |
| Asset file | 50 MB (ASSET_MAX_FILE_BYTES) |
asset-processor |
| Image pixels | 100 MP (ASSET_MAX_INPUT_PIXELS) |
asset-processor |
| Assets in the library | unlimited unless ASSET_MAX_COUNT |
asset-processor |
| Form upload | per website, at most 25 MB, 10 files, 1 GB a day | core-api, asset-processor |
| Per-address request limits | see configuration reference section 5 | core-api, asset-processor, gateway |
7. Questions to ask before sizing a customer#
Expected visitors per day and whether the sites are static, server-rendered or API-driven; number of editors working at the same time; total asset volume and the largest images; whether form uploads and AI are on; the required RPO/RTO (backup); whether a managed PostgreSQL is available. A load test against the real configuration (public API, login, image upload, form upload) is the recommended gate before a launch.
Last updated