PROJECT / SAMPLE
The safest rewrite was not rewriting it.
Instrumentation and behavioral tests made a fragile legacy service understandable enough to repair incrementally.
- Date
- 2026-04
- Type
- software / reliability
- Status
- fictional fixture
- Published
- prototype only
Problem
A small essential service had accumulated years of patches, undocumented dependencies, and manual restart rituals. Nobody understood it well enough to change it confidently.
Constraints
- Existing behavior was the only reliable specification.
- The service could not tolerate a long replacement project.
- Several external dependencies were no longer maintained.
Observed
- Most incidents originated in two recoverable dependency failures.
- Obsolete paths carried disproportionate operational risk.
- A rewrite would have recreated years of undocumented edge cases.
Work
- Instrumented the service before rewriting any path.
- Reconstructed its runtime and dependency map.
- Added tests around observed production behavior.
- Removed obsolete paths and isolated failure modes.
- Migrated the highest-risk components incrementally.
Decisions
- Treat production behavior as evidence, not an inconvenience.
- Remove risk in descending order rather than replacing the system at once.
- Document recovery before changing architecture.
Result / SAMPLE
The service became boring enough to operate and safe enough for ordinary maintenance without a risky replacement program.
- 37 incidents/quarter → 2
- MTTR: ~95 min → 11 min
- full rewrite avoided
What we did not do
We did not rewrite a system we did not yet understand. Instrumentation bought more certainty than new code would have.
Technical index
observability / behavioral testing / legacy systems / incremental migration