Mohammed Mudassir Ahmed

The store every money movement posts through stalled at roughly 25 transactions per second on a single account. The team read the resulting retry storm as insufficient capacity and wanted to scale the database out.

I reconstructed a failing request query-by-query from the logs and found a 28.5ms UPDATE sitting in lock wait. The design held one row per account and updated it on every posting, so concurrent writers serialised behind a single row lock. The system was contended, not underprovisioned — and adding capacity would have made it worse, because more writers means more contention on the same row.

I re-architected it onto hash-partitioned buckets, so a hot account's balance spreads across buckets that grow and shrink with measured write contention. A control loop sizes the partition set, bounded by a growth cap per cycle and a cross-pod cooldown so concurrent pods cannot oscillate a hot account. Accounts partition independently, so throughput now scales horizontally.

The decision I would defend hardest: I built the reconciliation and self-repair harness before migrating. It caught a snapshot-ordering defect and a floating-point precision drift while both were still cheap. On a ledger, a correctness bug found in production is not a bug — it is someone's money.

Built from scratch: the system that meters usage and bills every client on the platform — tiered and pooled pricing, invoice generation and customer balances.

A single AspectJ annotation marks a billable action. The event leaves the request thread as protobuf over gRPC, publishes to Kafka, and batched consumers with a dedicated retry topic price it and persist it. Metering therefore adds zero latency to the request it measures. Statements generate off a read replica, never the write path, and every invoice reconciles line-by-line against raw usage.

Much of this was built with heavy AI assistance under my architectural direction, and it bills real customers in production today — which is the only test of AI-assisted development that counts.

Building the platform, not using the tools. Hugo Minions automates backend work scoped to a verifiable done-condition — a failing test, a stack trace, a defined refactor, PR triage — with OpenHands as the execution substrate.

The engineering is in the guardrails. Every agent runs least-privilege against an explicit tool allow-list, side effects are capped, anything irreversible waits at a human approval gate, and model output is schema-validated so a malformed or invented response fails closed rather than propagating. The reviewer is accountable, not the generator.

Alongside it: company-wide adoption of Claude Code as the standard workflow, per-service context files across 11+ services so agents reason from real architecture rather than whatever files they happened to open, a GitHub Actions agent that diagnoses and repairs failing tests on pull requests, and an autonomous debugging workflow that reproduces a reported fault and reports the narrowed cause.

I own the technical direction for the platform's crypto domain and authored the high-level design, driving it to a validated go. A three-module split along licensing lines — broker, on-chain execution, custody — so regulatory scope maps onto deployable boundaries rather than cutting across them.

An immutable double-entry posting log credits only on finality: twelve confirmations on Ethereum, L1-anchored for Base. An omnibus wallet with per-customer deposit addresses and sweeps. A block-anchored solvency invariant that freezes the system on breach rather than continuing to serve. Threat-modelled address poisoning and forged transfer events.

The signature store is keyed on (wallet, chain, nonce) after I established that MPC signing is non-deterministic — an assumption that would have caused silent duplicate-signature bugs had it gone unchecked. I also ran the custody vendor evaluation end to end and wrote the recommendation leadership acted on.

Run and step state persisted in MySQL, so a flow survives a process restart and resumes at the step it stopped on. Per-step retry policies and timeouts, parallel step groups, sub-workflows, asynchronous third-party callbacks, pause and resume, and explicit escalation to human review where a decision genuinely needs a person. Pluggable dispatchers — same-thread, thread-offload, queue-backed — chosen per step, with state transitions guarded by Redisson distributed locks.

Back-office and money flows live or die on that last part. They are mostly waiting: on a document, on a third party, on somebody's approval. A system that cannot survive a deploy mid-flow, or that quietly double-charges when a callback arrives twice, is worse than the manual process it replaced. Currently running a Temporal proof-of-concept to evaluate replacing it.

The business needed new financial products faster than engineering could build them. I built a configuration-driven engine where a product's fees, taxes, interest and yield accrual, transaction limits, overdrafts and credit limits are expressed as data rather than code, evaluated in under 30ms from precomputed configurations and in-memory execution paths, with per-customer override APIs.

Launch time fell by more than 60%, and a dozen product variations now run on it with no new services and no new code. That was the ship-something-that-works call — and knowing when you are in that situation rather than the re-architect-it one is most of the job.

Built the compliance service from scratch: KYC and identity verification through third-party providers, customer profile lifecycle, account closure, resubmission and remediation flows. At its centre is a rule engine that screens every transaction before it executes, within roughly 100ms.

Rules are held as data, not code, so new eligibility, fraud and AML detections ship without a deploy. The same evaluation drives automated dispute workflows and the transaction-monitoring foundation.

Designed the platform-wide layer that encrypts all sensitive customer data — envelope encryption, AES-GCM with a KEK/DEK hierarchy, master keys in AWS KMS, adding only a few milliseconds. Exposed as a library other engineers call rather than a service they integrate with, which is precisely why adoption reached every service handling PII.

Alongside it, the platform-wide audit system recording every request that crosses the platform — REST, gRPC, database writes, queue events — as an immutable append-only log, because a regulator will eventually ask you to reconstruct a decision from a year ago.

The first production platform was Python on AWS Lambda — accounts, payments, scheduling, background jobs, every third-party integration — which I built and operated. When cold starts and concurrency became the constraint, I led the rebuild into Java and Spring Boot microservices, more than doubling throughput and eliminating cold starts.

I then migrated the system of record from Amazon QLDB to MySQL, moving 100% of live customers through an idempotent, resumable per-customer pipeline with zero downtime and zero data loss. Separately, two platform-wide messaging migrations — SQS to Kafka to NATS JetStream — behind a pluggable provider, including consumer replay and backfill without duplicating downstream effects.

Moved 11 services to gRPC and Java virtual threads, replacing thread-per-request pools that collapsed under IO-bound fan-out — cutting inter-service latency by more than 30% and raising concurrent capacity roughly fivefold.

Then a throughput collapse that showed idle CPU. Two days to find it: a virtual thread pins its carrier thread when it blocks inside a synchronized block, starving the scheduler. Fixed with reentrant locks — and then wired the JDK's own thread-pinning events into our dashboards, so that entire fault class is now a graph rather than an investigation.

Built the cash and payments service from an empty repository — accounts, wallets, multi-currency balances, deposits, payouts, transfers, beneficiaries, overdrafts and limits — then the full card lifecycle on top: authorization, clearing, settlement, reversals, refunds and disputes.

That meant integrating third-party issuer processors (NymCard, Clowd9), licensed partner banks including Griffin in its first year under a full UK banking licence, domestic and cross-border rails — FAST in Singapore, PayPak and PaySys in Pakistan, SWIFT — and FX and remittance providers (Currencycloud, Dandelion Payments, SentBe).

Every connector absorbs its own auth, rate limits, retry-and-backoff and reconciliation behind one internal contract, so the domain code never learns which processor it is talking to. Every movement applies exactly once under failure and retry — idempotency keys, MySQL trigger-level guards and Redisson distributed locks after a cache-based mutex proved unsafe across pods — and reconciles against the processor's and the rail's own records rather than against our own optimism.

Every banking flow — activation, refunds, transactions, card ordering, round-ups, investments, onboarding — had grown its own ad-hoc orchestration. I designed a centralized state machine framework that standardised lifecycle management across all of them, and built the intent-processing engine on top of it as the spine: 20+ step-based async processors under mutex-based concurrency control, with full CRUD, pagination and typed intent references.

It became the platform's standard async-work primitive — new flows compose existing steps rather than inventing orchestration, which is where the delivery gain came from. Reliability and development velocity improved roughly 30–40%.

Implemented trace-ID propagation through async execution, schedulers and virtual threads, so a request stays followable once it leaves the thread that started it — the point where most tracing quietly gives up.

Built the Grafana and Prometheus dashboards and alerting the team operates on, Sentry triage across 65 services, and Elasticsearch/Kibana for log search. Then wrote an internal observability CLI that queries logs, metrics and alerts across roughly twenty environments through a self-healing credential flow — tokens acquired, cached and transparently re-acquired on auth failure, so engineers never handle a credential by hand during an incident.

I carry rotational on-call for what I ship, and diagnose lock contention, hot partitions and slow consumers in live systems.