Replace-by-Fee (RBF) is a concept by which a wallet can broadcast a transaction that aims to overwrite a previously broadcasted but unconfirmed transaction sitting in the mempool, most often in order to boost the mining fee and expediate the mining of the transaction.
This RBF ability is useful because it allows a user to broadcast a transaction with very little mining fee (say this user is not in a hurry to see the payment go through), and then later on change their mind and “bump” up their fee in order to hasten the mining of the transaction.
The feature has potential implications for the continuing development of a well liquid fee market, as participants can in theory lowball their fees and later on bump them up as needed, leading to better price discovery for fees. It is a fairly used feature—about 10% of bitcoin transactions signal as Opt-in-RBF enabled (the variant this article will focuses on)—and it is likely to continue being used in the future as block space becomes more and more compteted for.
We should note that there are in fact 4 RBF variants, and that the choice of which variant(s) to accept and retransmit is decided at the node level (this is referred to as a node policy). This article focuses on the most popular of the 4 variants—the one implemented by most wallets—called Opt-in RBF. It is described in BIP125 which was authored by Peter Todd and David Harding.
Here is how it happens at the network level: when a node receives a transaction that spends a given set of UTXOs and verifies it as valid, it broadcasts it to its peers and includes it in its mempool. If a new transaction that spends some or all of the same UTXOs is recieved, a prudent way to approach the situation would be to not accept and retransmit the new transaction. The reason for this is that it could lead to DoS attacks—imagine your node being flooded with transactions that all spend the same utxo, meaning only one of those can ultimately be valid, but having to broadcast all of those to your peers. Such an attack would barely cost anything to perform but could potentially jam a lot of nodes. Instead, most nodes will reject new “overridding” transactions broadcast to them, except under very specific circumstances; the transaction has to fall under one of the 4 variants of RBF as described in the RBF Wiki.
Nowadays the most prevalent type of RBF is the Opt-in RBF variant, where a transaction can signal that it is ok with being replaced by a new transaction. It does not increase transaction size.