From Byron to Shelley: part two, the journey to the mainnet
Continuing on to Shelley with the decentralization of block production
11 May 2020 Kevin Hammond 5 mins read
Today, we’re kicking off the ‘Friends & Family’ testnet, which will allow us to establish a robust network to test and iterate, before we roll it out to the wider community. We’ve gathered a small number of around 20 ‘pioneers’ to help us with this important initial work. By the time you probably read this, they’ll be briefed and we’ll have things underway.
In my last blog, I outlined how the Shelley experience will roll out within clearly defined phases. These first three phases will involve exploring and testing the new Shelley capabilities via a series of testnets. I thought it might be useful to offer a glimpse ahead to provide some additional context.
The rollout of the testnets will happen very much in parallel with our continued progress towards mainnet. So alongside the work on the Haskell Shelley testnet, the mainnet will be systematically upgraded to support the Shelley era protocol, that will enable staking, delegation and metadata.
Similarly, IOHK’s block-producing and public-facing relay nodes on the mainnet will be upgraded so that they are ready for Shelley, and the Blockchain Explorer, Daedalus Wallet, Wallet CLI and other user-facing software will be polished so that it can be used for mainnet.
Users will soon be able to go to the official Cardano websites or other providers such as Yoroi, to download a new wallet – currently in development – that will work with both Byron and Shelley era blocks. The Shelley-era Daedalus wallet will contain all the logic for staking and delegation that has been tested on the Incentivized Testnet, plus new features that are specific to the full Shelley protocol. Stakepool operators, exchanges and others will also be able to download Shelley-compatible nodes and to adapt their own software to support the new Shelley client API. However, during this period, the mainnet will still be running in Byron reboot mode with federated consensus governed by the OBFT (Ouroboros Byzantine Fault Tolerance) algorithm. Think of it as a time when forward compatibility is being integrated but not yet ‘switched on’.
The move to Shelley will be accomplished using the new hard fork combinator that has been developed by IOHK.The hard fork combinator allows a node to transition from one blockchain protocol to another. The Cardano node software that is running on the mainnet will gradually evolve so that it is able to deal with both Byron era and Shelley era blocks, and will be modified to include the hard fork combinator. When the time comes to move the mainnet from the Byron era to the Shelley era, IOHK will trigger a hard fork.
‘Switching on’ Shelley
This will activate the hard fork combinator within the nodes, and the nodes will then change from only producing Byron era blocks to only producing Shelley era blocks. After the hard fork, no new Byron era blocks will be recorded on the blockchain, and the nodes will be able to support distributed block production, staking and delegation. They will have seamlessly switched from the OBFT to the Ouroboros Praos consensus mechanism. We will have entered the Shelley era on the mainnet.
Distributed stake pools and Decentralization of Block Production
Central to Shelley is the idea of decentralization. IOHK believes that companies, systems, and platforms run by a single individual or central authority are more vulnerable and less fair. This is why it is crucial that we move block production to our supporters rather than keeping the power contained within our organizations.
The Cardano blockchain currently operates on a federated basis. Effectively, nodes ‘controlled’ by IOHK and EMURGO are responsible for block production, while Daedalus wallet users act as the nodes of the network. Shelley will mark the ‘beginning of the end’ of that era, as we move from Byron’s static federated system to an active, decentralized system.
At the moment, core nodes and relays are owned and operated by IOHK. The network propagates through relays into each individual wallet. Once the system decentralizes, nodes will be run by stake pool operators and networked with individual wallets. Once control of the system is transferred over, the community will be fully running the Cardano ecosystem.
Federated Block Production (Byron)
Following the hard fork, IOHK’s existing core nodes will initially produce all of the Shelley blocks, as in the Byron era. However, this will change over time, under the control of the built-in d (decentralization) parameter. This parameter can be considered as a control valve that allows increasing amounts of decentralization.
In the decentralization phase, the federated system will still produce a (steadily decreasing) portion of the blocks. As this happens, stake pools will begin registering, operating, and producing blocks and will start to earn rewards in proportion to the stake that is delegated to them. As time passes, more blocks will be made by stake pools and fewer will be made by the core consensus nodes. The balance between the two will be controlled by the d parameter.
Distributed Block Production (Shelley and Beyond)
We will use metrics like the amount of ada that has been staked to determine how quickly to change the d parameter and so to decentralize the network. Once the network has fully decentralized, the stake pools will completely take over block production. At that point, we will then be able to shut down the core consensus nodes and disable the d parameter. This is the first step towards the full decentralization of Cardano. We will return to that in future blog posts, when we discuss some of the exciting developments that the Shelley mainnet will enable.
The road towards Shelley has been long but creating a global financial and social operating system takes time, scientific rigor, and the support of an informed, passionate community. As always, we thank you for your support and encourage you to follow our official channels closely as the Haskell Shelley testnet and subsequent phases roll out.
Following the successful Byron reboot of Cardano, we are beginning our phased transition to the Shelley mainnet. This means moving from a static, federated system to a dynamic, decentralized Cardano blockchain.
The process begins with a series of Haskell Shelley testnets, culminating in the Shelley upgrade hybrid phase.
The Haskell Shelley testnets will be a different experience from the previous Incentivized Testnet (ITN) for both stake pool operators and general users/ada holders. This is because the ITN and the Haskell testnets have been created with different goals in mind.
The ITN was designed to give stake pool operators experience in building their critical infrastructure, while allowing IOHK’s engineers to test the new incentive mechanisms with real ada, delegated by actual ada holders. The Haskell Shelley testnet is about ensuring that the Shelley mainnet is calibrated to be a best-in-class experience from day one. Unlike the ITN, the Haskell Shelley testnet will not involve ‘regular’ ada holders: the testnet will not be incentivized. Each phase is intended to run for a much shorter period – weeks rather than months. We will, of course, be testing out the operation of the wallet, explorer, and so on, but using a faucet distributing test ada that doesn’t offer rewards. Ada holders will be able to try Daedalus and the explorer on the public testnet and provide feedback, but without using real ada.
The Shelley experience will roll out within clearly defined phases. The first three phases will involve exploring and testing the new Shelley capabilities and moving to a situation where we are ready for full Shelley mainnet deployment.
Phase 1: Pioneers and the ‘Friends & Family’ phase
The rollout will begin with an invitation-only ‘friends and family’ testnet. During this phase, IOHK will first spin up and run a Shelley-only test network internally. We will then invite about 20 trusted stake pool operators – we’re calling them ‘pioneers’ – to join this (initially closed) network. These operators will comprise a small group who have demonstrated a high level of technical skill and community contribution during the ITN.
These pioneers will blaze the trail for others to follow as we head to full Shelley deployment on the Cardano mainnet. In this important first phase, we’ll be asking them to perform specific functionality tests to capture their valuable feedback while exploring the capabilities of the Haskell Shelley platform. We expect to invite more pioneers to join us – a few at a time – as we add features and prove the reliability of the testnet.
In this ‘closed alpha’ testing phase, IOHK will focus on tuning system parameters such as the saturation threshold, network resilience, and decentralization. Furthermore, IOHK’s engineers will see the Ouroboros Praos consensus mechanism working outside of simulation. The pioneer phase will give IOHK’s engineers the opportunity to address any issues in a controlled environment, with feedback and support from stake pool operators, before moving to the next phase. The findings will be communicated to the Cardano community and opportunities will be taken to learn about and improve the Shelley system.
This phase will also be all about producing high-quality technical documentation and support. The pioneers (supported by the community as a whole) will help us produce documentation that will make it easy to set up and run stakepools, and give our technical support team an understanding of the issues that our users will face.
Phase 2: Opening up the testnet – the public phase
The community response to the ITN was incredible, and we are blessed with having a wealth of skilled stake pool operators in the community. We’ll keep everyone informed through every step of the process and – as ever – our repos will be fully open. But for purely practical reasons, we’ll be working 1-2-1 with just a small group of around 20 operators at first. But our goal is to open things up as soon as we can, with full public access in the next phase.
This will allow all the stake pool operators who participated in the ITN to redeploy their previously constructed infrastructure, and to tune their stake pool to the new Haskell settings. This testnet will run as closely as possible to mainnet conditions, including mixing Byron and Shelley era blocks.
During each evolution in the transition to Shelley, IOHK is placing an emphasis on community training and collaboration. Decentralization of knowledge is just as important as decentralization of the platform. Pioneer participants in the alpha testnet will provide crucial support in advising the remaining stake pool operators on configuration and use of the Shelley system. And as operators acclimatize, we’ll also be asking them to support and bring new operators on board.
Phase 3: The balance check
The third and final phase prior to mainnet deployment is the balance check. This will bring together the Byron and ITN transaction histories, and prepare the mainnet for the Shelley era. At this point, the ITN rewards and mainnet balances will be consolidated. After this point, it will no longer be possible to earn rewards on the ITN. However, users will be able to check their rewards and confirm them in mainnet wallets. We’ll share full details of what ada holders need to do to reclaim their ITN rewards a little nearer the time. The balance check phase will last for only a couple of weeks before we start moving towards decentralized stake pools and the Shelley era.
How we will select the pioneers
We are selecting the pioneer group based on a number of criteria, devised in collaboration with the team at the Cardano Foundation. Pool operators must have a deep knowledge of running stake pools on the ITN, as well as competency working with Linux, and come from a range of backgrounds and geographical locations. Some will be working with cloud solutions providers to run their pool, others with their own hardware – we’ll have a mix. By selecting pioneers from different geographical regions, we will be able to ensure a global reach, and test out our new network implementation.
Pioneers will be expected to commit a set number of hours per week to supporting the rollout program, give direct feedback and provide advice to the community and mentor others at subsequent phases. Bringing others on board and supporting them along the way will be a crucial part of the role. To be clear, as ever, all our repos will be open so we encourage everyone to get involved. As always, IOHK’s developers value input from every member of the Cardano community. Anyone who wishes to is encouraged to spin up their own nodes. If they are skilled developers they can also recommend enhancements to the Shelley Haskell code base because all the information will be published through GitHub.
We’ll be looking to expand the network rapidly with more pools as soon as this earliest testing phase delivers the results we want.
Ensuring an easier start for everyone
The Haskell Shelley code base has been developed with formal methods and the high assurance Haskell programming language. So while we anticipate that some minor elements will need addressing, we believe that the initial experience should be free of any major issues. This is the approach that we have used for the Byron reboot, with great success, and we will be building on the code base that we have developed there. The approach will deliver even greater benefits for Shelley and beyond, by allowing us to deploy software much more quickly than in the past, with new features subject to rigorous and careful checks even before coding has been completed.
Our goal is to provide a plug-and-play solution to get stake pool operators up and running. This means they should be able to pick up a pre-prepared docker image or AWS instance, for example, and their stake pool will be launched. We will, of course, also provide standalone binaries and source code for those with more experience, or who have specific configuration requirements.
We’re now in the final stages of preparation and things are heating up (you may have recently seen a tweet that the new node has produced its first block). With that successful first step completed, we’ll be sharing dates and more details very soon. We’ll also be publishing further blogs outlining the other key steps and milestones in the process. Keep an eye out for those and meanwhile stay tuned to IOHK’s social channels. We’ll be sure to let you know as we start rolling things out.
Explaining the Shelley Incentivized Testnet incentive model
Learn about Cardano’s incentive model, rewards, and our plan for the Incentivized Testnet
5 December 2019 Kevin Hammond 7 mins read
We’ve talked a lot about incentives lately. That’s because right now we’re running the Incentivized Testnet: a Shelley testnet that provides an opportunity for stakeholders to delegate their stake or operate a stake pool to earn real ada rewards. Later this month, anybody that had ada in either a Daedalus or Yoroi mainnet wallet during the balance snapshot (taken on November 29) will be able to participate in the Incentivized Testnet, as either a delegator or stake pool operator, or both.
One of our key goals for the Incentivized Testnet is to test – in a real-world setting – the assumptions made in the Ouroboros incentives whitepaper, which uses game theory to calculate the incentives required to ensure consistent, active, and strong participation within a blockchain network.
The foundation of Cardano is mathematics; its central pillar, however, is a philosophy aimed at creating a fairer, more transparent, and more equitable system, decentralized and globally distributed.
Why incentives matter
Successful systems depend upon the adequate supply of incentives. Think of a company. A company must sufficiently incentivize its employees to work. This doesn’t just mean turning up to work – existing within the system – but performing a specific function to the desired standard. The same is true (and, arguably, is more crucial) for decentralized systems. Cardano is a decentralized network of global participants, each of whom must be adequately incentivized to take part and perform their roles, with the understanding that the network’s interests align with their own.
A brief overview of the incentives mechanism
Cardano’s incentives model begins with an assumption of rationality: that each player will act to maximize their own returns. These returns are the system’s incentives, and can take the form of tangible rewards – such as money – or intangible rewards, such as esteem, reputation, status, identity, or fulfilment.
Selfless acts are rare. As individuals, we pursue strategies that reward us, directly or indirectly. A network of participants each acting out of self-interest, however, can lead to chaos. That’s why successful systems codify – in protocols, rules, or laws – when and how much each participant will be rewarded. One of the core principles of game theory is that an ideal system is one where a selfish participant, acting in their own best interests, is also, by design, acting in the best interests of the system.
This is the function of Ouroboros’ incentives mechanism: a set of instructions that specify how and when rewards are paid out, and in what proportions to reward different levels of stake contribution. It allows a distributed network of participants to coordinate and collaborate in a decentralized system and receive rewards in accordance with their self-interest, while still contributing to the long-term health of the network.
Aims of Cardano’s incentive model
Equality and fairness are key to the sustainability of any future system, but can only be assured by the system itself, independent of individual goals or self-interest. Individuals must be free to exercise their ingenuity and maximize their outcomes, as long as doing so does not impede the operation of the network or restrict the possibilities of another (for example, by gaining a disproportionate amount of control). If one participant is the winner every time, other participants are disincentivized and, eventually, disenfranchised. The final implementation of Cardano’s incentives mechanism, as outlined in the incentives whitepaper, incorporates these factors, ensuring that that the biggest doesn’t always win, and that not only the richest get richer.
This is one of the aims of the game theory underpinning the incentives model – to test the thresholds and parameters for exploitation and the alignment of individual and collective interest – and is similarly one of the aims of the Incentivized Testnet. Over time, we will introduce new factors to the rewards calculation and monitor the impact on participant behavior.
Testing the incentive model
The incentives model we’re introducing to the Incentivized Testnet is not the final model. We plan to use this phase to test the incentives model incrementally, verifying our assumptions and exploring whether the network and participants respond in the way we anticipate.
We will not only be testing our game theory, however. We’ll also be testing the technology, ensuring that additional factors for reward calculations are only included once a baseline model is proven to be secure and stable.
In the beginning, various factors will not be included in the rewards calculation. These include factors to increase the number of stake pools and to better rank stake pools according to their desirability. Other factors will be included but in a limited capacity, and their function and calculation will evolve over time. This includes stake pool ranking. At first, the ranking will be based on a stake pool’s performance but, as we progress through the Incentivized Testnet, will transition to be based on desirability (a combination of cost, margin, pledged stake, and performance).
We’ll then gradually introduce additional factors into the rewards calculation, beginning with factors to encourage growth in the number of stake pools and to ensure the system promotes the most desirable stake pools. Each of these is important, and introducing them in a staged approach will allow us to ensure they function as intended and that each has the intended effect on the network.
Incentivized Testnet rewards
The rewards for delegating stake or operating a stake pool on the Incentivized Testnet depend upon the percentage of network participation. An approximate 3.8 million ada will be awarded per epoch. If 50 percent of the network participates, then we estimate the annual return for delegation will be approximately 7 to 8 percent but could, if network participation is lower, be as high as 13-15 percent. These figures are subject to treasury taxes and stake pool fees. A rewards calculator is now available on the Incentivized Testnet website which, in addition to other variables, allows you to calculate approximate rewards relative to different levels of network participation. Here’s a sneak peek:
Approximate delegation rewards calculation at 30% participation
Approximate delegation rewards calculation at 50% participation
Meanwhile, for stake pool operators, the rewards for stake pool operation, assuming a pledged amount of 10,000,000 ada, a 10 dollar daily stake pool operating fee, 50 percent network participation in the Incentivized Testnet, and the operator margin set to 10 percent, the total return rate for stake pool delegation will be approximately 12 to 13 percent. We will be updating the calculator over time to include more sophisticated rewards calculation modelling.
Approximate stake pool operation rewards calculation at 50% participation
More coming soon
This is a testnet, and as such involves an iterative process to reach our desired end: a complete and fully functioning incentives mechanism – as described in the Ouroboros whitepaper – that rewards network participants accurately and fairly in proportion to their contribution, while preventing any single actor from gaining a disproportionate amount of control over the network. We’ll be actively monitoring participant behavior throughout the testnet, to determine when and what additional factors may be included in the rewards calculation.
To learn more about the Incentivized Testnet, visit our website. If you’re interested in running a stake pool, register your interest and explore our testnet website for step-by-step instructions. And, as always, follow us on Twitter or sign up to our email list for the latest progress updates.