One of the essential attributes of blockchain technology is the dispersion of data among distributed and transparent ledgers instead of centralized, permissioned databases characteristic of Web2 architectures. By disseminating transactional records globally, blockchains have changed how people think about data ownership, access, and storage. But this design is not without limitations. When data is duplicated across nodes, it creates a storage headache, which worsens as networks grow. This, in turn, leads to problems with scalability, performance, and availability.
The issue of storage is one of the most commonly discussed challenges facing blockchains today. All blockchain transactions are recorded and preserved on the network’s ledger. As more transactions are executed on the network, more data is created, necessitating an increase in storage capacity. Moreover, blockchains are immutable, meaning that storage requirements constantly grow because nothing is ever deleted from the ledger.
In this article, we’ll examine blockchain’s storage constraints and some potential solutions to the problem.
Where is blockchain data stored?
Blockchain data is hosted on globally distributed machines referred to as nodes. Nodes essentially run software to validate and store information about the network’s state. There are various types of nodes serving different functions. Some may retain a full copy of the ledger, while others store only the most recent blocks. Although this architecture may vary from one network to another, a full node typically stores the entire network state, which is a complete history of transactions executed on the blockchain. Running a network node requires meeting some minimum hardware requirements. In the case of Bitcoin, among other requirements, a device must have at least 500 GB of free storage space with a minimum read/write speed of 100 MB/s to run a node.
Why is there a blockchain storage problem?
As Ethereum co-founder Vitalik Buterin argues, storage limitation imposes a severe constraint on blockchain scalability. In an ideal scenario, considerably more users on blockchain networks would run their own nodes, but this requires significant hardware and bandwidth resources (a minimum of 1TB of SSD storage is needed to run Eth 2.0 full nodes) that are prohibitively high for the average user. A quick peek at Etherscan shows an average of fewer than 10,000 nodes running on the Ethereum network over the past 30 days. This has raised questions about computational limits for blockchains and just how decentralized networks might be in the future.
With growing hardware requirements comes the need for specialized projects running blockchain nodes as a service. Infura and Alchemy are two leading projects maintaining nodes for Web3 protocols and developers. But these services have raised concerns as they centralize blockchain data in the hands of specialized service providers, creating a single point of failure (SPOF) and privacy risks.
Are there any viable solutions to the growing blockchain storage problem?
Several solutions have been developed to tackle the blockchain storage problem, mainly:
- Sharding: Sharding is an optimization technique that entails partitioning the blockchain workload into various shards, with dedicated nodes focusing on unique data types. This frees up other nodes to take on more computational tasks. This reduces the amount of storage space each node must allocate for the distributed ledger. The critical benefit of sharding is that it increases on-chain storage capacity without relying on 3rd parties like Infura. This means that storage capacity does not come at the expense of decentralization, and at the same time, the network’s attack surface is not increased. A potential downside is that it remains limited in terms of the extent to which it can remedy the storage problem.
- Pruning: Another approach to improving on-chain storage is by locally removing older or less relevant information from a specific node category. This is known as pruning. By eliminating older transactional data, storage can be freed up, enabling more people to run nodes without meeting stringent hardware requirements. However, pruning carries certain risks. For instance, if an attacker targeted an older block that had been pruned, the entire network may be compromised.
Blockchains are designed to be fault-tolerant systems. This means that they remain highly available even in the absence of some network participants. However, serious limitations on on-chain storage could significantly impact network performance. As transaction data grows, so too does necessary storage needs. Achieving decentralization amidst this ever-growing demand requires a highly distributed infrastructure that is not beyond the affordability of users. By lowering hardware requirements, blockchains achieve greater security and decentralization.