Idempotency

Idempotency is the property of a system that ensures an action is applied at most once. This may sound strange if you haven't heard of it before but but the idea behind idempotency is this: in an idempotent system if you accidentally send the same instruction 1,000 times, it will only be executed once. This has two very powerful implications in financial systems:

• It protects from duplicate charges and payouts. Duplicate charges make for unhappy customers and duplicate payouts make for unhappy operating accounts. Both will cost you money, time, and reputation when they happen.
• It makes retry policies extremely straight forward. In an idempotency protect system it is always ok to blindly retry an action you aren't sure finished. Whether it finished or not, you can be confident that it will never be executed a second time.

Idempotency keys

This behavior is accomplished through the use of idempotency keys. There are unique identifiers that are used a single time for an unit of financial work. Idempotency keys should be unique based on the business action you are taking. For example, if you are paying your employees for payroll you might use an idempotency key of payroll_07_15_2023:employeeId. This means even if someone runs payroll a second, third, or fourth time for the same period, the system will pay each person a single time for that period.

Idempotency key structure

String Theory uses idempotency keys made up of two fields: type and token. The type is the category of key. It should be specific enough to be unique to the action type, but not unique to the action. The token should be a unique identifier for this specific key. In this section we'll show them in the format of type:token. Some good and bad examples of idempotency keys are provided below.

• order:1234
• payout:a3nb90018c
• payroll_07_15_2023:employee_819234
• manual_deposit:429ff7e16837b203
• manual:cs
• employee_819234:pay

The core to a good idempotency key is knowing the business context for the financial change and building the key based on the business rules. Some example:

• Do you charge a guest only once per order? order:orderId might be a good idempotency candidate
• Do you payout out to sellers at most once per day? payout_02_15_2023:sellerId might be a good idempotency candidate
• Do you only allow coupons to be redeemed once? coupon:couponId might be a good idempotency key

These are just examples, but good idempotency keys are closely tied to how your business functions. You need to think critically about when actions should be repeatable, and when they should be unique.

-- Do not randomly generate idempotency keys --

Don't generate an idempotency key just to meet the requirement of having one. It is important to think about your use case and if the operation should be allowed to be repeated. Passing random keys for every operation will create the potential for duplicate financial actions in your system that are difficult and costly to fix. In particular these issues tend to be very impactful to reputation as overcharging customers or over paying employees degrades trust rapidly.

Idempotency contract

String Theory requires idempotency keys on every interaction with the API. The side effects (actual impacts) of each API call are atomic, and idempotent. If the idempotency key has ever been used before the system will return the result of the old operation, instead of applying the effect again. The contract with String Theory for idempotency keys is as follows:

String Theory will only execute operations once per idempotency key.

Common use cases

Idempotency keys are important in most operations, but we'll walk through some common scenarios where you'll almost certainly care or not care about an idempotency key.

Deposits (Important)

When depositing money into the system from either a UI or through an API call it is important to have a strong idempotency key. If you don't have one then multiple fast clicks of the button (UI) or automated retries (API) will make extra deposits into the system. These are correctable mistakes, but a good idempotency key will prevent them from being possible in the first place.

Withdraws from groups/threads (Important)

Groups and threads act similar to accounts for withdraws. As long as they have enough money to satisfy the amount of the withdraw being asked for, they'll allow it. This means if we have a $4,000 in an account and accidentally request $2,000 twice, both withdraws will go through.

You can avoid this in two ways:

1. Use a unique idempotency key with the withdraw so it'll only happen once.
2. Withdraw by targeting knots, instead of groups or threads, which cannot be duplicated.

Transfers from groups/threads (Important)

Similar to withdrawals, transferring from a thread or a group has the potential to duplicate if there is still enough money left. Be careful to specify a strong idempotency key when working with these operations: particularly if the targeted group has a large amount of money in it.

Specifying knots

If you are specifying knots for an operation, you do not need to worry about having a strong idempotency key. Knots are immutable, so once they've been changed once they can never be changed again. This makes them operations targeting specific knots effectively idempotent.

Exchanges

Exchanging between world currencies, you don't need to be hyper vigilant about idempotency. If String Theory sees that the source and destination currency for an exchange are the same, it will not repeat the exchange.

However, if you are doing custom exchanges you need to be careful. For instance, if you are exchanging to accrue interest you need to make sure you provide a strong idempotency key for the interest computation so it doesn't increase the amount twice.

Questions?

Idempotency can be a difficult concept. It requires both engineering understanding as well a business context to get right. If you need help determining good idempotency keys for your system the String Theory team is always available to help.