Issue Affecting Payment Validation Services
Postmortem
Overview
On 3 July 2026, we identified an issue affecting payment validation services in the production environment. Some clients experienced errors when validating payments via the platform and API. The issue was mitigated through cache reloads and service recovery actions, with a permanent fix subsequently implemented and deployed. Service was restored and monitored until stability was confirmed.
Client Impact
During the incident, some clients may have experienced:
- Errors when validating payments
- Intermittent failures during payment creation and validation workflows
- Issues processing certain bulk payment uploads
The impact was limited and intermittent. No data integrity issues were identified, and no client action was required.
Root Cause
The issue was caused by a missing payment validation cache entry required by the Payments Engine. Under certain conditions, the cache was not correctly repopulated following service events, causing validation requests to fail. A related service change contributed to the cache inconsistency. Mitigations were applied to restore the cache, and permanent fixes were implemented to improve cache management and prevent recurrence.