Blockchain reorgs for Managers and Auditors

ABA's photo
ABA
·

12 min read

Table of contents

Executive overview

  • There exists a blockchain event referred to as blockchain reorganization or reorgs. When this event happens, transactions can be discarded or reordered within a block and this can span across multiple blocks

  • Depending on the Web3 product type being built, reorgs can be a natural issue that needs to be taken into consideration. See Products susceptible to blockchain reorgs for more information

  • Depending on the blockchain, these events are more or less frequent. In some extreme cases a reorg has affected as much as 5 minutes of history. See Metrics for protocols and auditors for more information

  • A coordinated malicious attack can also cause a blockchain reorg. Although improbable it has happened in the past leading to a double spending attack

  • For auditors, there are examples provided of issues for each product type

Detailed overview

In Web3 we often hear the idiom the blockchain is immutable but there are cases when it's not, or to be more precise it becomes practically immutable (reach finality) only after some time has passed since the block, which contains users' transactions, was minted. This is due to events called blockchain reorganizations, or reorgs for short.

Paradigm has one of the best articles on reorgs, how and why it happens. I highly recommend you give it a read as we will only briefly touch on why it fundamentally happens. From the mentioned article, we have the following good block reorg definition:

A reorg is an event where a block that was part of the canonical chain becomes no longer part of the canonical chain because a competing block beat it out.

Reorgs happen because of the way blockchain validation client softer works. The algorithm that determines which is the canonical chain (fork choice rule), under certain circumstances, can conclude that the current chain history is not optimal and choose a different chain from those provided by competing miners (fork chains), to become the canonical chain.

When a block reorg happens, as an external observer of the chain you see that:

  • several blocks have been discarded and replaced with others

  • this does not imply that all transactions in the original blocks were discarded, a large number will probably still be included in the new blocks, possibly even all but not in the same order

Reorgs appear more often than people realize. Using on-chain explorers such as polygonscan.com/blocks_forked, for Polygon reorgs, you can see they appear often:

polygonscan blocks forked

The first reorg in the image indicates a ReorgDepth of 11, meaning 11 blocks were discarded (approx. 22 seconds of history was changed).

Products susceptible to blockchain reorgs

Protocols or systems that are naturally vulnerable to blockchains reorgs are:

Products vulnerable to reorgsExamples and details
On-chain random based systemsExamples:
- betting protocols
- on-chain casinos
- random reward systems
Off-ramp crypto systemsExamples:
- centralized exchanges (CEX)
- crypto-to-fiat providers
Protocols that require users to create something and then interact with it in a two step manner- yes, this is vague
- the majority of problematic systems here are those that have user deploy a contract then interact with it

Example behavior to look for:
- deploying a vault in one transaction and depositing funds to it in another
- user creates a proposal then votes on it in an on-chain voting system
Cross-chain mechanisms- yes, this is also vague

Examples:
- cross-chain communication protocols
- token bridges
- cross-chain gas acquisition mechanisms
Systems where an on-chain action has a gradual effect depending on its proximity to a specific block or timestampExamples:
- on-chain Dutch auctions
Blockchain validation softwareExample:
- node validation clients

Metrics for protocols and auditors

When designing or auditing a protocol that may be vulnerable to reorg issues there are some data points that must be taken into account which are particular to each chain. The following table serves as a reference point.

BlockchainReorgs/dayMax depth/dayLargest known reorgAverage block time (approx. in seconds)
Ethereum10-251112.08
Cronos10+5-775.6
Polygon PoS2-105-151572.2
BNB Chain0-2153
Avalanche C-Chainreorgs not possiblereorgs not possiblereorgs not possible2
Fantomreorgs not possiblereorgs not possiblereorgs not possible1.41
Arbitrum One0000.25
Optimism0002
Base0002
zkSync Era0002

Notes and observations

  • the above data has been collected either from 3rd party aggregators or from sites displaying the raw data directly. They are also taken after any major changes in the network, e.g. for Ethereum after PoS and for BNB Chain after their latest major scalability changes that drastically reduced reorg rates

  • When deciding how much blocks to wait before confirming a transaction off-chain in sensitive systems (such as crypto to fiat off-ramps) it is best to wait a number of blocks more then highest know reorg depth on that chain that happened historically

    • For example, deposits on a CEX from Polygon can wait even 200 blocks before being regarded as final

  • L2s theoretically can experience reorgs but it has never happened in practice (at least not know to or found out by the author of this article).

  • In the References section you can find links from where the above numbers were collected

Product detailed reorg issues explained

For each of the aforementioned type of products or system susceptible to blockchain reorg issues that were mentioned, a detailed description follows.

On-chain random based systems

On-chain protocols that have a random component are naturally affected by blockchain reorgs. These type of protocols can be betting, casino types, games with a random reward system, practically anything that calculates randomness using an oracle.

Example of situations that can appear:

  • a player wins a good reward in random draw but due to a blockchain reorg he loses it

  • a gambler loses an on-chain dice game but due to a blockchain reorg he now actually wins

In practice, Chainlink's VRF (Verifiable Random Function) is often used to calculate random number. This API accepts a block confirmation time, which, to quote the documentation represents:

how many blocks the VRF service waits before writing a fulfillment to the chain to make potential rewrite attacks unprofitable in the context of your application and its value-at-risk.

Examples of this type of vulnerability seen in audits:

To mitigate this issue, it is enough to choose an appropriate block confirmation time. See Metrics for protocols and auditors for recommended numbers.

Off-ramp systems

To off-ramp crypto, refers to the process of exchanging cryptocurrencies for fiat currency. Systems doing crypto off-ramping include:

  • centralized exchanges (CEX)

  • crypto-to-fiat providers

Crypto deposits into these systems are confirmed after a specific number of blocks have passed since the transaction in order to mitigate any issues possibly caused by blockchain reorgs.

This article gives some specific block confirmations requirements found on know CEXs. Example Coinbase requires 35 block confirmations for ETH deposits while Binance requires only 12.

As an observation, CEXs tend to severely over-wait for confirmations. Example, a blockchain reorg on Ethereum of more then 2 blocks has not happened in years.

Off-ramp systems, particularly CEXs, are an attractive target for 51% attacks. In the past, there were several cases where attackers gained 51% of a network validation resources (hashing rate) and proceed to initiate withdrawals on several exchanges then reorg the chain in order to re-initiate the withdrawals on other exchanges.

Most notable 51% attacks are mentioned in this article and involve the Verge, Ethereum Classic, Bitcoin Gold, Feathercoin and Vertcoin tokens.

Mitigating this form of attack is not straight forward. One component of the solution is to overly increase the required confirmation count. Although in those cases this is not enough, as the network takeover can last an undefined amount of time. An on-chain monitoring solution coupled with withdrawals/deposits pausing need to implemented.

Protocols that require users to create something and then interact with it in a two step manner

(yes, the title is too big)

This type of reorg issues are the most frequently seen in this vulnerability category. They appear when a protocol requires users to first create or initiate a component and then, in a different transaction to interact with it.

Another important and key precondition is that the deployed contracts use the Solidity new keyword for contract creation (or the CREATE opcode). Both of these rely on the calling contract nonce to determine the address of the newly-deployed contract and do not take into consideration any custom input.

Example, a protocol uses a factory-like pattern to deploy contracts, whos functions are permissionless. Say the factory logic deploys a contract via one call while giving special permissions to the deployer. After the deploy, the user send a deposit to the newly deployed contract.

An example attack scenario in the above case:

  • alice creates a vault via the factory contract (transaction A)

  • alice then deposits into the vault tokens (transaction B)

  • a block reorg happens and transaction A is discarded (transaction B still exists)

  • normally transaction B now would revert if executed

  • bob deploys himself the vault that initially alice did (via frontrunning) and because of the underlying issue, it is created on the same address as the initial vault

  • alice's deposit reaches now bob's vault, and he can withdraw it

Another interesting example seen ITAW (In The Audit Wilderness) is:

  • alice creates a proposal; proposal ID is determined by a counter increment only (transaction A)

  • alice votes on her proposal (transaction B)

  • a block reorg happens and transaction A is discarded (transaction B still exists)

  • normally transaction B now would revert if executed

  • bob creates himself a different proposal (via frontrunning) which receives the same ID as alice's due to faulty implementation

  • alice's vote now reaches a malicious proposal

Examples of this type of vulnerability seen in audits:

To mitigate the issue use the create2 opcode and have the salt contain elements distinctive of the caller (e.g. hash on the inputs plus the msg.sender).

Cross-chain mechanisms

Similar to how CEXs or crypto off-ramp solutions need to consider block reorganizations so does any cross-chain mechanism that needs an off-chain component to transfer data from one chain to another.

Issues can appear if one of the destination or source chain reorgs which may create situations where a bridging was removed on the source chain but on the destination chain it was created. Leading to double spending.

Examples:

  • cross-chain communication protocols

  • token bridges

  • cross-chain gas acquisition mechanisms (e.g. gasbot)

Examples of this type of vulnerability seen in audits:

Mitigation is again straightforward and involves waiting a specific amount of blocks before considering a transaction as done (finalized). See Metrics for protocols and auditors for exact numbers

Systems where an on-chain action has a gradual effect depending on its proximity to a specific block or timestamp

(yes, this is another title that is too big)

There are protocols for which the result of an on-chain action depend gradually on their proximity to a specific block number or timestamp. There are, by their nature vulnerable to reorgs since transactions can be shuffled within a reorg leading to different outcome then expected.

The predominant (only) example see in this case is that of a Dutch auction:

  • alice initiates a payment at auctionStartBlock + 3 needing to pay price/3

  • a block reorg happens and alice's transaction is now put at auctionStartBlock + 2

  • the transaction goes through and alice ends up paying price/2

Examples of this type of vulnerability seen in audits:

The issue is mitigated per project value logic. In the Dutch auction example, a maximum payable amount should also be passed as a parameter when initiating the buy.

Blockchain validation software

There are over 1000 different blockchains in existence, each with its own validation logic and client software. While there are some common validation software being used, in the rare cases a blockchain development team decides to implement their custom validation logic, it is imperative to take blockchain reorgs into consideration.

Consider an example where a validation node always validates the oldest block. But because of reorgs, that block is often discarded and depending on the validation logic, the node may be slashed and lose funds.

Examples of this type of vulnerability seen in audits:

Conclusion

Blockchain reorganization is an event that has always existed and will probably exist for years to come. Reorgs impact specific application types and as such need to be taken into consideration otherwise severe financial loss may happen, as it has in the past.

As a manager, if the project your team is building is the type that can suffer when reorgs happen, share this link with your team and ask them to check. Resolving reorg issues should not have its own KPI, it is enough to include it in the (hopefully already existing) security related ones.

As an auditor you can either focus on the type of protocols that are susceptible to reorg issues or focus on the event itself. It is enough to ask the question:

how would a block reorg affect this protocol?

and to follow up from there to identify if such issues exist. The current article shows issues regarding specific products but a pattern can be deduced and applied to similar protocols.

References

reorgBlockchainWeb3smart contract auditorsmanagement