There's an article doing the rounds at the moment titled 'Fundamental challenges with public blockchains' that gives a really good look into the state of crypto at the moment - check it out if you haven't seen it already. I found this article super interesting as it summarises really well the state of confusion that exists around crypto at the moment in Silicon Valley.
The most telling thing about these hurdles can be seen from the term 'public blockchains' in the title rather than Bitcoin. This strange aversion to the word Bitcoin (and cryptocurrency) is one of the main things driving architects into corners where they find themselves grappling with seemingly impossible problems. Unfortunately, it means that people have started assuming problems with certain blockchains are problems in general, and that they apply to Bitcoin as well. For most of the 'challenges' mentioned, Bitcoin and its economic security model is both the source of the problem and of the solution, in that competition creates problems that in turn create their own solutions - this is what evolutionary capitalism teaches us.
With that, I thought it would actually be a good idea to step through the various hurdles listed, since it's important to understand that these aren't actually "technical barriers". Although they seem technical in nature, they're really social/political barriers that result from the wrong mental models. Interestingly, more than anything, they're the result of a security model that is working, within an ecosystem where people don't really know why. There's still a lot we have to learn, but for the systems to succeed, and for us to succeed with it, it's going to take alignment of interests in the ecosystem, and a reconciling of differences. This is the biggest game in the world. There's a lot to gain from competing with Bitcoin, but if you can't beat it at its own game, then you're still better to join 'em.
As developers, I believe it’s critical that we shift some of our focus away from shiny new ICOs to the real technological challenges standing in our way.
It's pretty normal for developers to see themselves as experts, I prefer to see them as servants to their users. It's pretty normal for VC companies to separate users and customers, with database business models. But with crypto, we can't separate them. As developers, we should make sure we have a good understanding of how to deliver value to our customers. In crypto these are end users, who require a reliable, low cost medium of exchange, investors, who want a reliable deflationary store of value, and miners, who work to maintain the network infrastructure. As developers, our incentives should be aligned with one of the three groups above. It's important to understand the role of developers in these ecosystems is very different from the typical, technocratic governance the Valley is used to. Humility goes a long way in crypto, and it pays off especially well here. After all, this is all about trust. Check out Daniel Krawisz' video for more on this topic.
Currently, all public blockchain consensus protocols have a challenging limitation: every fully participating node in the network must process every transaction.
Why? Well, recall that blockchains are fundamentally "decentralized" — which means that no central party is responsible for securing and maintaining the system. Instead, every single node on the network is responsible for securing the system by processing every transaction and maintaining a copy of the entire state.
This is the tried and true argument against the adoption of Bitcoin, but it's built upon false assumptions.
There is such a thing as a 'fully participating node'
What is a "fully participating node"? Remember, Bitcoin is a competitive system. It is not egalitarian. Someone always makes a profit, whether that be in transaction processing, the block reward, or the transaction in the first place. This means it is not possible to define 'full participation' in the security model, because someone is always doing 'more' than the other in order to win business.
Often the term full node is used as a reference for a node that downloads, verified, stores and distributes the full blockchain. However, this node is not fully participating, since it isn't mining at all, let alone winning. A node is either a wallet, holding and spending bitcoin, or a merchant, earning bitcoin by selling goods or providing services. There aren't any other types of nodes, as every participant in the ecosystem is either looking to make money or to spend it. Miners count as service providers, and developers can sit on either side.
A 'fully participating node' could more accurately be defined as a miner who both earns and spends bitcoin for all its economic activity. At a protocol level, individual roles aren't specifiable. That's the reason for the Proof of Work game. It creates a definition of participation as solving a problem that meets the specified level of difficulty for the hash rate, meaning that all incentives of any particular node have some way of mapping back down to hash rate, whether that's improving the ability to command and control the hash power directly, or convincing others with hash power to mine in your favour. I like to use the term 'fully sacrificial' node. The only real way to define a node as being 'full' is that it has given everything within its power to protect the network. If you're a wallet that means selling all your Bitcoin for hashpower. This is an extremely irrational act, and really one of martyrdom. Defining it in such a way makes it easier to understand why it's the last thing you'd ever want to do, and why participation is measured by hashrate.
Nothing forces a node to store a copy of the entire blockchain
The only thing that results in nodes holding a copy of the blockchain are the degree to which they require the information in order to protect their position. There is nothing anyone can do to force a miner to host the whole blockchain, if he doesn't want to. It is a risk he takes for himself. Likewise, there is nothing that forces every user to host the entire blockchain. Remember, the permissionless nature of the system, and the use of cryptography to build functioning economies, is the idea that there can be rule of law without force. People are addicted to force. They find the idea of being self-sufficient frightening. It takes a while to understand that words like 'need', 'must', 'everyone' etc. are mostly obsolete. Enlightened self interest through co-dependence is the name of the game. In fact, Satoshi's whitepaper includes an entire section on 'Reclaiming Disk Space':
Once the latest transaction in a coin is buried under enough blocks, the spent transactions before it can be discarded to save disk space.
Not every single node on the network is responsible
The whole point of the Proof of Work game is that nodes cannot be trusted. The only thing we can trust is the difficulty of solving the problem, and the economic interests of everyone involved. This means that, quite crucially, 'everyone is responsible' is different from 'everyone must do everything'. We collectively need to protect the network. But the whole point of rewarding nodes that contribute hashpower is to free everyone else of the burden of having to worry about attacks on monetary policy or denial of service. Szabo practically screamed it at us.
A security model dependent on 'full-nodes' who don't mine is in contradiction to bitcoin having value. Because for it to have value, it needs a network effect of people who believe it has value, and will do two things: validate transactions they receive and report them to the network if they're invalid, or validate miners blocks and produce a new candidate block if they're invalid. As typically defined, a 'full node' is not a wallet or a miner. It's a node that 'validates transactions' / 'secures the network' yet its costs of operation are not compensated in-protocol. This is a contradiction. To believe that full nodes are required in order to maintain protocol security, we'd also have to believe that miners won't check each others' work and users won't validate their incoming transactions. If we believed those things, then bitcoin would have no value, as its entire value is derived from the belief in its resilience and utility by its users and investors. If bitcoin has no value, nothing will hold it together. Without supply of security (hashrate) and demand for security (transactions), and an ability for the provider of the security to profit, then it will cease to exist.
The 'solutions' mentioned are not scalability solutions
This is because they don't increase the economic activity exposed to the Proof of Work Game - they reduce it, by moving it elsewhere. Usually, the incentive is for someone other than the miners to profit from the network effect. There is no easy and fast way for money to flow between all the different types of applications if they are all on separate blockchains, since they all have different security models, but more importantly, slow things down considerably thanks to incompatible consistency models.
Instead of scaling the economic activity exposed to bitcoin protocol, the solutions mentioned all move economic activity away from it. Of course, their goal is to free up space for 'more valid' economic activity, but unfortunately this is a fallacy. As security is purely economic, moving any amount of economic activity off-chain by definition reduces the security, not increase it. A chain without room for 'regular' transactions will not be secure for high-value ones, as the number of people who have direct interests in enforcing the protocol rules will be considerably lower. Instead, those users will likely have financial motivation to invest in systems with competing network effects, like we see with Bitcoin Cash and Ethereum attempting 'flippenings'.
Unfortunately there is no way to get security from other main chain without paying for it. This is the tragedy of the commons. If we want to be able to rely on the mining network for information security, we're going to have to pay for it. If we move economic activity away from the validation/optimisation game and instead rely on a third party to secure our money, we're just going to get stuck with the same old problems Bitcoin set out to solve. All off-chain systems make a tradeoff for security. The funniest is the Lightning Network, which is based around the idea of 'hubs' that forward money by signing transactions. Each of these transactions have dependencies to funds locked to a private key, exposing them to QC risk, and requiring their key to be present on every single signing node. Can you imagine the Lightning Network equivalent of Amazon CloudFront storing their master password to their bank account?
In practice, scalability is merely the result of two things: the holders of transaction outputs collectively storing the blockchain, and increasing the throughput of nodes performing the mining / transaction processing operations. We have already seen the absolutely incredible result of the bitcoin ecosystem to optimise the hell out of things far beyond what the traditional economy would ever think possible in the emergence of the sha256 ASIC industry. There is no reason that Bitcoin wouldn't be capable of a similar abundance of transaction processing throughput if the miners competed over it.
The fear associated with the loss of the chain, and the failure of consensus, is the exact thing that gears people into action. Fear of loss is one of the most motivating factors in human evolution. Without the fear of losing access to the chain, there would be nothing to actually get people to democratise transaction processing through hyperoptimisation.
The solution to scalability is easy. Simply process progressively larger blocks. The real challenge is in convincing people to do so. We're currently seeing an active experiment for the feasibility of doing so, that once it's played out, will signal to Bitcoin's miners whether they should adopt the same strategy.
Scaling throughput is easy. The cost of infrastructure is subsidised by the rise in price associated with rising demand for infrastructure, as already happened with sha256. But if investors think in the short-term, they will not realise the potential gains associated with their holding. In order to maximise the return on their investment, they actually need to invest their returns in things that will grow the network. That means spending some proportion of coins on a bigger computer, joining the mining network, or spending time optimising the software in between (with the free time that's been earned from appreciated assets).
Privacy remains a fundamental hindrance for individuals, organizations, and industries that care about privacy and individual sovereignty. Many of us who are obsessed with blockchain and cryptocurrency have a concerted interested in enabling a trustless and censorship-resistant system that brings financial empowerment to the individual. Paradoxically, we’re using a public, easily traceable ledger to do so.
Because everyone is attempting to use this 'public' resource as if it is public. It's not! It's a private money, purely held together by the sheer profit-seeking motives of every participant in the ecosystem. If someone can make money off our information, and we give it to them, they will find a way to profit from it. That's not bad per se, there's value in sharing information if it results in a mutually beneficial relationship, where both parties want to profit off each other. Facebook is a good example of a bad relationship - they want to sell our information to advertisers, but they don't make anything from us. Information privacy has a lot more to do with what we share, who we share it with, and why than it does how we shared it. If we shared a secret over an HTTP connection, it would have been shared not only between us but also with every intermediary. Likewise, if we send a ZKP Zcash transaction to someone who has to comply with KYC then 'protocol privacy' might as well have been the TSA -- security theatre. What's more, if I have visibility into the network, I can identify relationships between nodes without having to worry at all about the exact contents of the transaction. Just like a phone call, it's the metadata that counts.
People are addicted to authority; it's familiar to them. They want the protocol to protect them, just like the government. The premise of a private protocol is really representative of a desire not to have to secure one's own data. Developers actually cause more harm than good here, because they typically treat Users like they are dumb or need to be protected. It's much more valuable for developers to educate their users on security and privacy than for them to come up with complex cryptographic primitives that people find impossible to understand. When we stop treating users like children, we'll have made some progress towards a more secure economy. Supporting attitudes that encourage people not to take personal responsibility for their actions is causing more harm than good. When it comes to personal information, that's not something anyone should outsource to someone else. To live a completely open life is to live at the whims of other people. If we want control over our lives and money, we need to know how to protect our own wealth. This is an aspect of self-sovereignty. What's important about these technologies aren't the 'privacy' or 'anonymity' that they provide us - we will always be limited by our own personal security. So unless we move our own personal wealth into the control of a third party, we're never going to be safe.
Above everything, the nodes who have the best ability to make private transactions are the miners. A miner gets its coins out of the abyss, and can write transactions directly to the chain that haven't touched the internet. Combined with best practices in threshold sigs and payment channels (existing tech), there's no technical challenge to solve here. It's just education. Chains like Monero and Zcash just encourage people to use the internet to gain privacy.
The best way to keep things private is to have control over how our packets are routed. We should avoid intermediaries at all costs.
I'm sounding like a broken record here, but verification is not someone else's job, and it already has a role in the game theory. Verifying code is not a "HUGE unsolved problem" any more than programming is a huge unsolved problem - it's completely open ended, and often just pushes a bug elsewhere. What's more important is who wrote what code and why. If the developer of the code is the one verifying it, we probably missed the whole trust-but-verify idea in the first place.
Unfortunately blockchains with poor security models like Ethereum, where any arbitrary code can be added to the protocol at any point in time, don't align incentives correctly around contract verification. That's why people have lost so much money from Ethereum. We could argue that Ethereum contracts are different from the underlying blockchain protocol, but in practice and in social terms, they're indistinguishable from one another. The EIP/ERC process is as official as the development of the protocol standard. As the economic activity in a contract matters just as much as the overall economy, when contract issues impact the base protocol, maintainers will be interested in protecting it, as we saw with the DAO.
Verification is code that contributes to a convincing argument. If we want to use code that was written by someone else, then we should verify it ourselves! If we want others to use it, they should verify it too. Appealing to authority - asking the developer to formally verify his code - does nothing to protect us from an honest mistake. It's a natural instinct, point the finger, but being able to point at someone when something goes wrong won't do anything to change the past. The more effort a single trusted developer puts into something, the less that code can be trusted. The more the developer does, the less likely it is that users will compile it, run the tests, read and understand the code or try to break it. The solution here is the solution everywhere else in the Proof of Work Game. If we aren't paying someone else to perform validation for us (as the miners do), then we're a product. Don't trust a contract, but don't want to pay someone to verify it? Then don't use it, or take the responsibility for the risk. Appealing to authority in a system without trusted authorities and only people with competing interests is a great way to guarantee we'll lose out.
Storage is effectively the same problem as scalability, but the suggestion is that its possible to separate the two. Unfortunately, this is not really true, due to what's known as the 'data availability problem'. The solution to the data availability problem is the blockchain and economically bound consensus.
In more practical terms, it's important to realise that data storage today is not a significant constraint for these types of systems. Currently, the cost of a single Bitcoin transaction is more than the price of a hard disk to store the entire chain. There is an abundance of storage capacity available for storing these transactions. If we believe that storage is constrained, then we're just subsidising the business of miners.
The idea that resilient, long-term storage of information in the chain is a problem is actually a little crazy - since it's the entire value proposition of a decentralised statement of record. We can't solve the problem of the cost of data storage by not storing data; that's like solving third world hunger by telling people not to eat. When the entire value proposition is immutable storage, we can't solve the problem by not offering the solution people are looking for.
The solution is staring everyone in the face, but they don't want to accept it: The transaction fees are there to pay for the additional cost of each transaction beyond those directly involved.
Data storage imposes a huge cost on a decentralized network where every full node has to store more and more data into infinity.
Again, it's profit that matters, not costs. Mining works at massive scale because it's profitable, in spite of its expense. If the revenue received by a miner for processing a transaction is greater than the cost to store it, it's not a 'huge cost', it's profit! If someone is running a node that's not mining, it's not processing any transactions or adding any security to the network, it's just making less than it's spending to stay online. Nodes that can't sustain themselves are not useful to depend on for security. What we really see then is the benefits of the economics of scale - because some people can't compete with the big boys, we see the same as we do with hyperscalers - storing lots of data is cheaper if you're in the business of storing lots of data.
Want to reduce the cost of storage? We should demand that the miners store it, since we're already paying them for it. None of the mentioned storage 'solutions' will result in a lower cost than competition over that cost by Bitcoin miners. This is Filecoin's entire premise, and it's already built into Bitcoin.
As public blockchains like Bitcoin that use proof-of-work consensus continue to scale, increasingly, more energy will be wasted. If the goal is for the public blockchain to scale to millions of users and transactions, the unsustainable wasted energy and computation costs of proof-of-work are not conducive to this outcome.
I disagree with this premise entirely. It's proof of work, and efficient wasted computation (that is, energy that could not be put to a more profitable use) that enables the blockchain to scale to millions of users, by building a more logical and convincing argument for its security as money. Proof-of-Work has scaled enormously, to several ExaHashes today, 9 years after the first block was mined, and the energy wasted for each of these hashes is orders of magnitude lower today than was needed in 2009.
This argument is not new with Bitcoin. "If we don't stop burning fossil fuels ..." "If we don't stop building houses ..." "If we don't stop trying to live beyond our means ..." then we will never make any progress! It's this constant desire to do more with less that results in growth and progress. Check this section from George Land's book Grow or Die, written half a century ago. The argument applies as much today as it did to anything in the 1970s. Then re-read Szabo's blog, understanding that Bitcoin is the best machine for energy optimisation ever built. Port some software to the bitcoin mining network and be amazed at the energy efficiency gains that the market can extract from it if we allow them the freedom of profit.
ASICs aren't a weakness, they're a strength.
ASICs ensure that energy pointed towards the chain is in line with the security model to defend the currency against third party attacks. Requiring a significant capital investment of specialised hardware aligns the incentives of the network with the role to be played by miners. It protects against vulnerabilities. Competitive consensus is not about fairness, it's specifically about unfairness. Artificial restrictions against optimisation create morally unfair systems because they deny the market the ability to make a profit, denying harder-working providers the opportunity to enter the race and encouraging low performance. David Vorick from Sia wrote a great post on this topic.
Mining Pools counterbalance Industrial Miners.
They're the Yin and Yang of Bitcoin. Giving smaller miners who aren't capable of competing at industrial scale the ability to join together to compete with larger miners ensures enough liquid mining to hold industry to account. The distributed nature of mining pools grants them a defensibility that is completely impractical with industrial mining. What's more, the efficiency costs of smaller miners are easily outweighed by their influence. They have greater access to liquid capital and more attention overall for investment strategy. An industrial miner may have lower operational costs, but they have must more expensive human capital, which is the single biggest contributor to TCO. Since Bitcoin is a Political Economy, an industrial miner makes a greater profit, but is restricted much more in their strategy.
Useful PoW warps incentives.
It gives a party who gets value from the computational outcome the ability to mine at a loss. This is not necessarily a bad thing. Exposure to the chain state enables any miner to provide a large number of highly valuable services. But chain state is universally monetisable, whereas 'useful' Proof of Work is likely specialised. These should be run on Bitcoin, not as Layer 0.
This brings us to really the crux of the whole topic, the idea of 'governance'. The word is a red flag. It means the same as control, or centralisation, or in a more basic form 'decision making'. So there's this idea that we need a mechanism to make decisions - whether that be over the rules, rolling back state or banning certain users altogether. The whole point of Bitcoin is that there is no authority that can make those kinds of decisions - they're made by the market. It's a response to this whole idea of centralised decision making and economic planning in the first place. It shows that we can achieve a stable system with a Nash Equilibrium - between Order and Chaos - without a central authority. If we rely on the competitive pressures of the market to keep the blockchain secure, we're going to be protected much better long-term.
By demanding that all transactions and consensus decisions are resolved through Proof of Work, the only 'governance' standards that matter are the ones that are beaten to death through competition and stand the test of time. Any other 'governance' protocol which relies on voting, identity, latency or any other system is always going to fail when subjected to economic pressure. Using anything but political economics for governing a political economy will simply result in disastrous risks of 51% attack.
'Standards', like the EVM, the 1MB block size, the Bitcoin Core codebase and SegWit all restrict optimisation and competition by forcing people to behave in one way or another. Often they're designed to set artificial high costs for standard resources, such that the proprietors of the standards can implement their own centralised alternatives. In the case of ERC20, ethereum actually won out economic activity over individual altcoins, by making a standard that could support less complex tokens. So standards, like protocols, should compete with one another. If someone's being forced to conform to a particular standard, it's likely that someone else is benefiting more than they are.
To receive the benefits of self-sovereignty, we need to take on the responsibility of protecting our interests. Unfortunately, it seems like not many people want to take it.
This one's easy. Pay people to make tools, or make some and sell them.
If the topics above weren't enough, yes, Quantum Computing is at the bottom of the list. Also known as the boogieman! What's actually interesting about the quantum computing 'threat' is that blockchains are a much more powerful threat today, with distributed cracking and DDoScoins being completely practical to build today, with immediate security implications. We've seen that modern computers are woefully insecure. With data markets coming online, the network which is currently mostly void of malicious actors will suddenly become full of them. But the tech can also be used for good here, which is where we come back to quantum computation. Proper use of the blockchain (no address re-use, shared keys, actively flowing coins) make malicious modification of the chain completely impractical for attack by quantum computers. A relationship with a miner is like a protective shield. So like privacy, the solution is education, not more complex crypto algos.
It's wrong to interpret these challenges as 'technical problems'. They may seem technical in nature, but the reason they exist is because of political challenges. It's typical for developers to claim authority in arguments by claiming they are Experts, but being unwilling to listen will always lead to less growth than if the best ideas won out. This technology is not compatible with Expertise, since it's irrational not to adopt the most profitable strategy, no matter where we heard it from.
With stakes so high, it's highly unlikely that developers incentives are entirely aligned with everyone's. Contributions to the Linux Kernel are dominated by Intel, Red Hat, Linaro, Samsung, SUSE and IBM, who represent commercial Linux distributions and the dominant CPU architectures of today - x86, ARM and POWER. Until we see similar competition over contributions in Bitcoin, we won't see a significant shift in the discussion. Now at least, there are competing implementations. As with any open source project, with a fork comes the potential to merge back together, as Node.js and IO.js did in 2015. We'll see what happens with Bitcoin Cash over the next couple of years.
Currently, development of Bitcoin, like many other cryptocurrencies, is dominated by firms without clear relationships to mining firms, making it difficult to predict the motivations behind different contributions. Balaji Srinivasan wrote that the lack of competition in protocol development is Bitcoin's greatest weakness.
Early competition for developer rewards which enabled breakthrough productivity in the early group behind ethereum has now vanished, with no one productively improving the production system. It is now a stagnating ecosystem, not an ecosystem of growth.
The blockchain represents the hostile environment that the internet, and the market, has always been. It isn't someone else's job to fix our problems, unless it is in their interest to do so. Most of these problems are failures of the individual (appeals to authority) or political issues where the narrative is that the issue is 'technical' and if we can't prove our tech chops, then we can't participate in the discussion. Developer dictators should get off their high horses if they want to be rewarded for contributing value to the protocol.
If we can't precisely understand how the promoter of a particular solution will profit from its adoption, then its likely that their incentives do not align with ours. Right now, less than developers formally verifying their code, we need developers verifying each others code, and not trusting the incentives of the person who wrote it.
By and large, Bitcoin shows us that performance improvement and trustworthiness are direct benefits of predictable motives. If both parties are behaving in a highly trustworthy manner, they are likely to realise performance benefits (profit), since one man can understand the other. If people are behaving in a confusing, accusatory, or authoritative manner where they're demanding something of others, then they're backing themselves into a corner. To optimise anything, we need some freedom to experiment and freedom to experience small failures that can be continuously corrected over time.
Systems like PoS which punish people for behaving incorrectly aren't able to create this environment since the safest strategy is always to do exactly what we're being told. Bitcoin shows us that austerity is far more wasteful and immoral than redundant execution will ever be.
If we care about these issues, and we want to have a voice, we're going to have to pick a team. At the moment, there aren't that many. But with the growth in Bitcoin's price, anyone holding coins has no excuse not to invest some of their time or money in protecting their interests - by getting involved in the mining game.
I'll leave you with One Very Important Thought.