2022-01-02
Whiteboard: DApps and Development
- A New Year technical whiteboard on Cardano dapp development, arguing why extended UTXO starts functional and dials expressiveness forward instead of dialing an accounts model back.
- Walks through on-chain scaling via bigger blocks, pipelining, and input endorsers, then the three off-chain mechanisms: Hydra, off-chain computing, and side chains.
- Frames 2022 as an engineering and expressiveness problem worked out with DeFi vendors through CIPs and workshops, with an October hard-fork deadline.
- Closes on the philosophy of starting with low expressiveness and full decentralization to preserve censorship resistance, versus chasing performance and network effect first.
64 entries
Opens January 2nd 2022 with a New Year whiteboard on dapps, DeFi, and development, promising more pedagogy in 2022 from him and Lars.
Shares the meditation books he ordered from Jon Kabat-Zinn's reading list, including Matthieu Ricard's Why Meditate and several Zen books.
Recommends starting with the introduction at docs.cardano.org, which covers Plutus, Rosetta, native tokens, the extended UTXO model, and the ledger model.
For a deeper introduction cites his paper with James Chapman and Lars Brunjes, UTXO versus Account-Based Smart Contract Blockchain Paradigms, which contrasts Solidity on Ethereum with Plutus on Cardano.
Reads the paper's conclusion introducing an idealized extended UTXO abstraction with novel results on alpha conversion and observational equivalence that Ethereum lacks, since accounts lend to imperative programming over global state while UTXO lends to a functional style where a contract maps a UTXO list to a UTXO list.
He is not dogmatic and believes in a multi-paradigm world blending functional and imperative styles, since expressiveness and model matter from the very beginning.
Places Bitcoin Script as the foundational functional-like model, a stack-based assembly based on Forth with a big limitation on expressiveness.
The other extreme is full program environments like the JVM and CLR with maximal expressiveness running full apps like Minecraft, while the EVM restricts some things to avoid security issues.
The extended UTXO model was written to make UTXO more extensible, targeting a sweet spot between Bitcoin and the EVM.
Asks whether the EVM's extra features and larger attack surface are really required for DeFi, oracles, stablecoins, or NFTs, since if not you pay for them without using them.
The imperative global-state model is hard to scale and unused expressiveness can still be exploited by an attacker, which is why the Bitcoin crowd stays conservative with over a trillion dollars of value.
2022 is about asking what expressiveness DeFi actually requires, and where more is needed you write CIPs, with three already written in collaboration with DeFi vendors.
Extended UTXO is a brand-new model that Ergo brought to market first, giving great advice and clarity as both grow toward the sweet spot.
Preserving functional style and local state gives security since it is a mathematical object easy to model with preconditions, invariants, and property-based testing.
Specifications enable design by contract, verification, and certifying software against standards for high assurance, which is possible but very expensive in the imperative style.
The choice was to start with a functional UTXO model and dial expressiveness forward based on need rather than dialing an accounts model back, preserving deterministic cost and local state for predictable contract behavior.
Thirteen years of Bitcoin history since 2009 are there to draw from for making the model performant, starting with block timing as the blockchain's heartbeat where transactions aggregate.
The most obvious scaling is bigger blocks, restarting the block-size agenda with node 1.33 after a 12.5 percent increase in 2021, part of an optimization program led by Galois, a military contractor.
Bigger blocks fit more transactions until reaching a threshold set by propagation, using the empirical measure of 95 percent of peers receiving a block within five seconds.
Data structures and coding theory like fountain codes optimize propagation, citing Arizona State research by the Dash community and Stanford's David Tse lab on the co-development of proof of stake and the network stack.
Block time has dead space, and pipelining does work during that dead time to increase throughput alongside faster libraries, reduced node syncing, and better network propagation.
Asks why not do blocks in parallel, a DAG design he mentioned in his 2017 whiteboard, breaking into key blocks and input blocks with micro-blocks constructed in between.
The Parallel Chains paper was written in 2018 and input endorsers were introduced in the 2016 Ouroboros paper, a concept the industry chases via IOTA's Tangle and metastable consensus.
Avalanche and sharding protocols like Polyshard from Viveck Cadambe's work and Ethereum 2 pursue this holy grail, constrained by network throughput.
You can ratchet TPS as high as a million if you can propagate it, raising a data-representation question where block processors have a different view than the network yet keep inclusive accountability via Mithril.
The other issue is optimism about network conditions, since all DAG and input-block protocols assume people are honest.
Literature shows many attacks where dishonest actors create conflicting transactions that degrade performance at serialization, sometimes worse than a single-shard protocol, which is why protocols add slashing, fraud proofs, and bonds and why they thought so hard about Parallel Chains and input endorsers.
Ouroboros needs no punitive bonding or slashing now though it could be added later, and realistically pipelining plus this gets 500 to 1000 TPS mixing scripts and value transactions.
Evolving the network stack widens that window and punitive measures enable deeper sharding, but pushing further decreases safety and increases centralization to where you must reset the network, which he does not consider a real cryptocurrency.
The science is done, so after pipelining, input endorsers plus aggressive optimization are the top priority, with an October deadline across three 2022 hard forks and cost no object.
Dialing up expressiveness brings more use and utility to take Cardano from millions to billions of users, and the network must handle that capacity.
Cardano is number one in GitHub commits with almost 15 companies working in parallel and no academic wait, since 130 papers are published and it is now an engineering coding problem.
Extended UTXO's locality and determinism make off-chain work easier via three mechanisms, the first being Hydra, the payment and state-channel idea with a node already on testnet.
SPOs are the natural operators for Hydra, and that Lightning paved the way but is not expressive enough, so Cardano's extended UTXO does it faster and better.
Wants a 1.0 Hydra on mainnet before year end, moving transactions to a nearly-feeless layer-two network across thousands of stake pools for micro-transactions.
The second mechanism is off-chain computing, already happening with SundaeSwap and EigenLayer, citing the ACE asynchronous contract execution paper from ETH Zurich that designates off-chain people to compute and prove correctness under an m-of-n trust assumption.
Off-chain computing maps well to the delegated SPO model and the UTXO model, since local determinism makes proofs easier to build and verify without global state synchronization.
The third mechanism is side chains, where Ouroboros builds a strong root of trust from 3000 SPOs, using cryptographic sortition to pick a subset each epoch to bootstrap a fast side chain.
Side chains use high-performance BFT protocols like Harmony One's RapidChain or Algorand for fast finality, something EOS chased without the theoretical underpinning of establishing a strong root of trust first.
Side chains like the Yella, Catalyst, or EVM chains run optimized protocols with thousands of TPS via permissioned BFT whose permissioning comes from the decentralized main chain.
A fixed known quorum lets you optimize the network stack for high throughput, the settlement-layer versus computation-layer distinction from his 2016 Cardano whitepaper, a direction Ethereum 2 chases with its beacon chain and shards.
The proof-of-stake side chains paper is Peter Gazi's from 2018 to 2019, real peer-reviewed theory they implemented, with OBFT shipped in the Byron reboot.
The Mamba project re-implemented Ouroboros BFT for the Scala EVM code inherited from ETC, in scope for 2022 along with EigenLayer explorations through partners.
Micro-workshops and summits like the one in Texas, where every DEX is building similar off-chain infrastructure, bring them together with the Cardano DeFi Alliance to learn shared patterns.
Workshops will request more expressiveness via new CIPs, with three slated for June adding read-only UTXO and new cryptographic primitives to improve off-chain computing and decentralization.
Points developers to the Cardano Stack Exchange and a developer Discord with over 11,000 members, with heavy investment from IO, the Foundation, Emurgo, and dcSpark.
Says community members should demand builders contribute to alliances, documentation, workshops, and open source, and notes block contents get optimized via script compression that Galois improved after inefficient Haskell libraries.
Side chains are an endless river constrained only by SPO hardware, and more side chains mean more profit and better hardware, a very scalable model.
Functional programming is a big ask, which is why they support a multi-model world with side chains like dcSpark's EVM side chain you can deploy on today with Ethereum tooling.
Extended UTXO is new and hard but made easier together via the CIP process, and that the when-PAB question is like asking when-JVM since it has many components, with PAB diversity coming by rewriting parts in TypeScript, compiling with GHCJS, and building a light wallet for the browser.
The competitive disadvantage of a year spent improving the model is offset by side-chain and EVM support, so for immediate commercial needs you use dcSpark and the imminent Mamba side chains with Solidity.
You either accept Ethereum's baggage or become a Plutus Pioneer of a new model with off-chain advantages via Hydra and verification that avoids the 10.5 billion dollars of DeFi hacks while giving the cheapest operating cost.
Says developing for the iPhone or Android when new was hard too, and tells Bitcoin maximalists to view Cardano as the beta test for Bitcoin, since supporting smart contracts means either outsourcing off Bitcoin's trust model or extending Bitcoin with extended UTXO.
Predicts a polyglot future where apps use functional and imperative, on-chain and off-chain, with two on-chains meaning Cardano plus its side chains plus another network like Ethereum or Bitcoin.
Asks why maximalists fight when an SPO could route all these things and profit, noting wrapped Bitcoin on Ethereum already exists.
His company has 600 people with over 100 engineers devoted to this stack plus more than a dozen companies, and they will cut a check to accelerate anything needed since the research is done.
It is easy to start with high expressiveness and centralization but harder to start low and ratchet up, which preserves the most important thing, censorship resistance and decentralization, or else why not just use Amazon Web Services.
Verifying massive Solidity code is enormously expensive, so doing it right the first time with good principles makes the whole ecosystem better, a game of averages the industry is currently losing.
The industry gets away with it because some people get rich and control the media, pushing hopes and dreams while the majority lose everything, which is why decentralization must be the number one concern.
Third-generation protocols market their performance but never control, safety, consumer protections, or legacy interoperability, because they just want your money and network effect.
Once truly decentralized you cannot fix it later since you need the consent of users who do not care about decentralization, like Ethereum needing a difficulty bomb to fire proof-of-work miners in the merge.
Is why you start with low expressiveness and full decentralization and systematically ratchet up performance while adding primitives like Mithril, pipelining, and input endorsers, keeping the soul of the project and the pioneers who share the founders' philosophy.
Everyone works toward a collective human vision of a fairer world with principles baked in, batteries not included, since VCs and governments will not help because they profit from the old system, unlike Bitcoin's can't-be-evil-by-design dream of inclusive accountability people signed up for unmotivated by money.
Six years of voluminous science moved the global commons toward an egalitarian world held equally under math, which Cardano reflects with scalability, interoperability, identity, and governance, and that being number one in GitHub commits is real patent-free open-source effort solving unsexy hard problems with the honesty to document mistakes.