Combinator makes easy work of Shelley hard fork

A behind-the-scenes look with Duncan Coutts and Edsko de Vries

7 May 2020 Anthony Quinn 11 mins read

Combinator makes easy work of Shelley hard fork

Since the launch of the Incentivized Testnet marked the coming of the Shelley era last year, the Cardano platform has entered a fast-moving period of development. The Ouroboros Classic consensus protocol has supported Cardano Byron and the ada cryptocurrency for the past 30 months, and we’ll soon be switching to Ouroboros Praos. This is the version of our proof-of-stake (PoS) protocol that will initially power Shelley as Cardano decentralizes. It builds in the staking process with monetary rewards for ada holders and stake pool owners.

We upgraded Cardano on February 20 with a hard fork that switched the mainnet from the original consensus protocol, Ouroboros Classic, to an updated version, Ouroboros BFT. This BFT hard fork began a transition period under Ouroboros BFT, a slimmed-down version of the protocol designed to help us make the switch to Praos, while still preventing any malicious behaviour. Many probably didn’t even notice. For Daedalus wallet users, it meant a standard software update. Exchanges had to upgrade manually, but they had several weeks to do this and we were on hand to help.

The next event was the ‘Byron reboot’ on March 30. This released totally new code for many of the Cardano components, including a new node to support delegation and decentralization, and future Shelley features. A big advantage of the new code base is that it has been redesigned to be modular, so many components can be changed without affecting the others.

In turn, the BFT will act as the jumping off point for the Shelley hard fork, which will happen once we’re happy with the Haskell testnet. This second hard fork will be a similar process to the February one for exchanges, ada holders and wallet users, and, hopefully, just as much of a non-event.

However, while everything looks smooth on the surface, there is a lot of hidden activity going on. Like a duck serenely swimming across a pond – while its feet are furiously paddling below the calm waters – our blockchain engineers are hard at it.

So, we sat down two of the leading engineers on the Cardano project, Duncan Coutts and Edsko de Vries, to find out how they’ve done it. Duncan has been Cardano’s architectural lead for the past three years, and between them, Duncan and Edsko have spent 35 years using Haskell, the programming language being used to develop Cardano.

Duncan, how did you do it?

As described in the Cardano roadmap, IOHK’s blockchain engineers believe in smooth code updates. Instead of trying to do the jump from Ouroboros Classic to Praos in a single update – which would be an incredibly complex task – it’s been a two-stage approach using Ouroboros BFT as an intermediary (Figure 1). The BFT code is compatible with both the Byron-era federated nodes and the Shelley-style nodes released in the Byron reboot. It's like a relay race: one runner (in our case, running one protocol) enters the handover box where the other runner is waiting; they synchronise their speeds (so they're perfectly compatible with each other) and then hand over the baton (operating the mainnet), and then the new runner with the baton continues from the handover box for the next lap.

Mainnet Byron to Shelley roadmap
Figure 1. Mainnet Byron to Shelley roadmap

The Daedalus Flight process has helped us quickly develop and test a new wallet and, once everyone is running that on the mainnet, and once we finish swapping over the core nodes, the old code is redundant. We are in that transition phase right now, with a new mainnet Daedalus wallet released on April 24.

Our aim is to have a ‘graceful entry into Shelley’, as IOHK chief Charles Hoskinson describes in his whiteboard video about the hard fork. A vital tool in making this move has been creating a hard fork combinator.

That sounds like farm machinery. What is it?

A combinator is just a technical term for something that combines other things. For example, addition is a combinator on numbers. A hard fork combinator combines two protocols into a single protocol. We call this a sequential combination of the two protocols because it runs the first protocol for a while and at some point it switches over to the second. In our case, this is two versions of Ouroboros as we move from BTF to Praos.

The clever part of all this has been the use of discrete modules that do their job, while knowing as little as possible about each other and the blockchain. Simplicity is the key here and this process of taking out the details we call ‘abstraction’. Most of the consensus modules don’t even have to know that they’re dealing with a cryptocurrency and could be putting pretty much anything on a blockchain. For example, we’ve done seminars using the example of a Pokémon ledger on a Ouroboros blockchain. The only thing that’s different is the ledger rules; the consensus is all the same. You just set it up – ‘instantiate’ it in the programming jargon – with the rules for playing Pokémon rather than for UTXO-style accounting.[For readers with a technical interest, watch out for Edsko delving further into the ‘abstraction’ process and combinators in a future blog post.]

You make it sound simple

In fact, it’s tricky because Cardano is running the ada cryptocurrency, and a pile of other things, at the same time. Think of it as changing all the wheels on a car while you’re driving along and towing a caravan. So we have to be sure we can do this in a totally reliable way.

We could have tackled this as a one-off task, but it made sense to do it in a generic way using a protocol combinator. We chose this route because we get a better result and the testing that is vital to ensuring the code works is made far easier. On top of that, there will be more hard forks to come, which made the choice even clearer. For example, as we near the culmination of Cardano’s development and move through the Goguen, Basho, and Voltaire eras, there will be at least one hard fork at each stage.

So how did you cope with the tricky bits?

Well, first off, we had to do it without research to turn to. The researchers describe a single protocol as a free-standing, perfect thing. But that’s not where we are. We are trying to run Praos after having started with a chain that was using something else. What Edsko’s working on, going from one protocol to another in a generic way, is just not covered in the research. And it’s hard, it’s complicated. All the details need a lot of thinking, a lot of scratching your head. But switching between Cardano code bases is not the sort of thing the academics can expect to get published. It doesn’t have a novel aspect and is seen as just an implementation issue.

Edsko, can you give us an example?

As Duncan says, for the researchers, these implementation issues are trivial but dealing with them is 99% of what we do. Take the problem of time for a blockchain. It’s what I’ve been banging my head against for a couple of weeks. Time is divided into slots where the chain can contain at most one block per slot. We often need to convert between slot numbers and time in the real world, for example when a node needs to know ‘Is it my turn?’ to generate the next block. This is fundamental to Cardano, but the length of a slot changes after the hard fork. For Byron, a slot is 20 seconds; for Shelley, it will be two seconds, or perhaps one. To really complicate things, the exact point of time when the hard fork is made is decided on the chain itself. Yet, I need to know when the changeover point is. It’s a quandary: to do slot conversions I need to know the state of the blockchain, but to know the state I need to know the slot conversions!

This is real chicken-and-egg territory with many complex things to disentangle. We have to be very precise with how we do things. It might be trivial in theory, but it’s very difficult to disentangle things and make sure it’s not a circular problem.

We can’t afford for it to be wrong, so how do you know you’re right?

Duncan: That’s an excellent question. My reply is that you come to the answer on two levels. The first is intellectual: you analyse the problem, you do the maths, you talk to colleagues and wrestle with it until you can see how it all fits together. Second, we do all our QuickCheck testing to give us the confidence that this does what we think it does. We do extensive testing that really takes us into the unusual cases that you might never think of, including this changeover. We can do 100,000 tests every time we change a line of code. [Lars Brünjes has written about how John Hughes, one of the creators of Haskell, has helped IOHK develop its testing strategies.]

Edsko: Yes, I agree with those two points. In terms of the combinator, I resolve these things by thinking about the guarantees that the code I write needs to provide, and which guarantees that it, in turn, needs from the ledger. I sketch a mathematical proof that this ‘if-then’ reasoning is indeed justified, and then turn to the formal method teams. The formal methods team are the people who set up the mathematical rules that describe the blockchain, and they can then tweak the rules in such a way that they provide the required guarantees.

In terms of Duncan’s second point, I know the time issue I mentioned above is correct by thinking hard mathematically, and by testing. Timing decisions are easy when we have the full blockchain, but are hard when we have to make predictions about the future. Fortunately, the way we set things up means I can easily create testing blockchains. So, I can create a full blockchain, then slice this chain in half. I take the first half and consider that to be in the present; and set the other half in the future. Then I can use the ‘present’ (first half) to make predictions about the ‘future’ (the second half) and verify them against the whole thing (on which the calculations are easy). If they match, then I know everything is OK.

When did you start on this?

Right after Duncan came up with the genius idea of the OBFT. So I’ve been thinking about the combinator on and off for about 18 months. It was a design goal from the very beginning of our modular rewrite of Ouroboros starting in October 2018, with my first commit to the GitHub repository. We had a prototype demonstration with OBFT and Praos soon after, in December 2018.

And how many people have been involved?

Duncan: Many people have been working on the consensus code, but whenever we have anything really hard, like this combinator, we give it to Edsko: he’s our software engineer extraordinaire! This is Haskell programming as free climbing, rather than climbing with the ropes of formal methods for support.

Any final thoughts?

Duncan: The code that was running is right now being phased out and pretty soon the code that was running Cardano a month ago will no longer exist. Once everyone is running the new code on the mainnet and once we finish swapping over the core nodes, the old code is redundant. We are in that transition phase right now and no one is shouting that the sky is falling. No one has noticed it.

Edsko: That is quite an achievement in its own right. The idea of OBFT was crucial in making the transition, but it’s not relevant any more once we make that transition to Shelley. This has been a way for us genuinely to ditch legacy code, which is often very difficult to do, as the banks know to their cost.

Duncan: And if it all works fine, you won’t notice anything.

Duncan and Edsko, thank you for your time. I think we’d better let you both get back to it.

From Byron to Shelley: Part one, the testnets

The evolution to decentralization continues with a series of three Haskell Shelley testnets

29 April 2020 Kevin Hammond 7 mins read

From Byron to Shelley: Part one, the testnets

Following the successful Byron reboot of Cardano, we are beginning our phased transition to the Shelley mainnet. This means moving from a static, federated system to a dynamic, decentralized Cardano blockchain.

The process begins with a series of Haskell Shelley testnets, culminating in the Shelley upgrade hybrid phase.

The Haskell Shelley testnets will be a different experience from the previous Incentivized Testnet (ITN) for both stake pool operators and general users/ada holders. This is because the ITN and the Haskell testnets have been created with different goals in mind.

The ITN was designed to give stake pool operators experience in building their critical infrastructure, while allowing IOHK’s engineers to test the new incentive mechanisms with real ada, delegated by actual ada holders. The Haskell Shelley testnet is about ensuring that the Shelley mainnet is calibrated to be a best-in-class experience from day one. Unlike the ITN, the Haskell Shelley testnet will not involve ‘regular’ ada holders: the testnet will not be incentivized. Each phase is intended to run for a much shorter period – weeks rather than months. We will, of course, be testing out the operation of the wallet, explorer, and so on, but using a faucet distributing test ada that doesn’t offer rewards. Ada holders will be able to try Daedalus and the explorer on the public testnet and provide feedback, but without using real ada.

The Shelley experience will roll out within clearly defined phases. The first three phases will involve exploring and testing the new Shelley capabilities and moving to a situation where we are ready for full Shelley mainnet deployment.

Phase 1: Pioneers and the ‘Friends & Family’ phase

The rollout will begin with an invitation-only ‘friends and family’ testnet. During this phase, IOHK will first spin up and run a Shelley-only test network internally. We will then invite about 20 trusted stake pool operators – we’re calling them ‘pioneers’ – to join this (initially closed) network. These operators will comprise a small group who have demonstrated a high level of technical skill and community contribution during the ITN.

These pioneers will blaze the trail for others to follow as we head to full Shelley deployment on the Cardano mainnet. In this important first phase, we’ll be asking them to perform specific functionality tests to capture their valuable feedback while exploring the capabilities of the Haskell Shelley platform. We expect to invite more pioneers to join us – a few at a time – as we add features and prove the reliability of the testnet.

In this ‘closed alpha’ testing phase, IOHK will focus on tuning system parameters such as the saturation threshold, network resilience, and decentralization. Furthermore, IOHK’s engineers will see the Ouroboros Praos consensus mechanism working outside of simulation. The pioneer phase will give IOHK’s engineers the opportunity to address any issues in a controlled environment, with feedback and support from stake pool operators, before moving to the next phase. The findings will be communicated to the Cardano community and opportunities will be taken to learn about and improve the Shelley system.

This phase will also be all about producing high-quality technical documentation and support. The pioneers (supported by the community as a whole) will help us produce documentation that will make it easy to set up and run stakepools, and give our technical support team an understanding of the issues that our users will face.

Phase 2: Opening up the testnet – the public phase

The community response to the ITN was incredible, and we are blessed with having a wealth of skilled stake pool operators in the community. We’ll keep everyone informed through every step of the process and – as ever – our repos will be fully open. But for purely practical reasons, we’ll be working 1-2-1 with just a small group of around 20 operators at first. But our goal is to open things up as soon as we can, with full public access in the next phase.

This will allow all the stake pool operators who participated in the ITN to redeploy their previously constructed infrastructure, and to tune their stake pool to the new Haskell settings. This testnet will run as closely as possible to mainnet conditions, including mixing Byron and Shelley era blocks.

During each evolution in the transition to Shelley, IOHK is placing an emphasis on community training and collaboration. Decentralization of knowledge is just as important as decentralization of the platform. Pioneer participants in the alpha testnet will provide crucial support in advising the remaining stake pool operators on configuration and use of the Shelley system. And as operators acclimatize, we’ll also be asking them to support and bring new operators on board.

Phase 3: The balance check

The third and final phase prior to mainnet deployment is the balance check. This will bring together the Byron and ITN transaction histories, and prepare the mainnet for the Shelley era. At this point, the ITN rewards and mainnet balances will be consolidated. After this point, it will no longer be possible to earn rewards on the ITN. However, users will be able to check their rewards and confirm them in mainnet wallets. We’ll share full details of what ada holders need to do to reclaim their ITN rewards a little nearer the time. The balance check phase will last for only a couple of weeks before we start moving towards decentralized stake pools and the Shelley era.

How we will select the pioneers

We are selecting the pioneer group based on a number of criteria, devised in collaboration with the team at the Cardano Foundation. Pool operators must have a deep knowledge of running stake pools on the ITN, as well as competency working with Linux, and come from a range of backgrounds and geographical locations. Some will be working with cloud solutions providers to run their pool, others with their own hardware – we’ll have a mix. By selecting pioneers from different geographical regions, we will be able to ensure a global reach, and test out our new network implementation.

Pioneers will be expected to commit a set number of hours per week to supporting the rollout program, give direct feedback and provide advice to the community and mentor others at subsequent phases. Bringing others on board and supporting them along the way will be a crucial part of the role. To be clear, as ever, all our repos will be open so we encourage everyone to get involved. As always, IOHK’s developers value input from every member of the Cardano community. Anyone who wishes to is encouraged to spin up their own nodes. If they are skilled developers they can also recommend enhancements to the Shelley Haskell code base because all the information will be published through GitHub.

We’ll be looking to expand the network rapidly with more pools as soon as this earliest testing phase delivers the results we want.

Ensuring an easier start for everyone

The Haskell Shelley code base has been developed with formal methods and the high assurance Haskell programming language. So while we anticipate that some minor elements will need addressing, we believe that the initial experience should be free of any major issues. This is the approach that we have used for the Byron reboot, with great success, and we will be building on the code base that we have developed there. The approach will deliver even greater benefits for Shelley and beyond, by allowing us to deploy software much more quickly than in the past, with new features subject to rigorous and careful checks even before coding has been completed.

Our goal is to provide a plug-and-play solution to get stake pool operators up and running. This means they should be able to pick up a pre-prepared docker image or AWS instance, for example, and their stake pool will be launched. We will, of course, also provide standalone binaries and source code for those with more experience, or who have specific configuration requirements.

We’re now in the final stages of preparation and things are heating up (you may have recently seen a tweet that the new node has produced its first block). With that successful first step completed, we’ll be sharing dates and more details very soon. We’ll also be publishing further blogs outlining the other key steps and milestones in the process. Keep an eye out for those and meanwhile stay tuned to IOHK’s social channels. We’ll be sure to let you know as we start rolling things out.

Merging formal methods and agile development to build Cardano

IOHK formal methods director Philipp Kant lays out our methodology for building software with flexibility and precision

9 April 2020 Philipp Kant 7 mins read

Merging formal methods and agile development to build Cardano

Form and function

IOHK is building Cardano into a global financial and social operating system. This enormous task requires both quick iteration and absolute precision. It is why IOHK has chosen to combine the speed of agile development with high assurance code and formal methods. Fusing flexibility and formality led our engineers to pioneer this modern development philosophy.

IOHK believes firmly in research, formal methods, functional programming, and building in a rigorous manner. As a competitor in the blockchain development industry, we also have to consistently demonstrate progress and create value for our global community of stakeholders. This means we can’t compromise on robustness or on development speed and flexibility. In an ever-changing marketplace this is a challenge, so our developers have to strike a balance.

Agility versus formality

The start-up standard for developing technology has been to build a minimum viable product quickly and then continually iterate until it is ready for the mass market. This is known as an agile process. It is a great way of showing that a project is advancing while eventually building a fully functional product. However, an agile methodology assumes there will be bugs and weaknesses in each step of development that can be ironed out later. This is fine if there is no value at risk – but, with virtual currencies, there is an enormous amount of money and stakeholder trust on the line.

Building a digital asset on a blockchain provides several challenges to overcome in terms of organizing a development process. As a proof-of-stake cryptocurrency, Cardano is a distributed system in an adversarial environment where consistent performance is critical. The protocol has to maintain security in the face of malicious actors attempting sabotage. This means that no one can afford to build quickly and deal with problems later.

Trust is essential for a currency to be accepted and correctness proofs are an important way to increase the veracity of a system. This is why the code should not only be correct, but there should be evidence of its correctness, such as extensive meaningful tests and mathematical proofs. In a young industry like cryptocurrencies, IOHK engineers have to anticipate the addition of new features while maintaining the correctness guarantees established in the initial version. The platform can only scale globally if it is able to grow while maintaining security and utility for everyone. This is why Cardano developers streamlined their methodology, combining a variety of tools ranging from property-based testing all the way to machine verifiable proofs, to create high assurance software even in the presence of changing requirements.

Research to code

The methodology begins with scientific research. To date, IOHK has released more than 60 research papers that have contributed to creating the platform. Each paper examines a critical aspect of blockchain technology from first principles. How do we gain consensus in a decentralized way? How is a smart contract designed? What is the right reward structure to incentivize good behavior? IOHK researchers examine these questions, and submit their answers to scientific journals and conferences. These papers contain proofs that must pass rigorous peer review. Then, to ensure that the quality of our software does justice to the science, it is developed using formal methods.

In essence, this means that IOHK engineers specify what the code should do mathematically. That way, they can ensure that when the code is run, it contains the desired properties designed into it. The code is written in Haskell, a high-assurance functional programming language with a strong type system. While Haskell is a great tool for implementing reliable software, it is not foolproof, so the code still needs to be tested. A great way to write tests is using QuickCheck, which allows developers to state properties that should always hold in a program. QuickCheck then generates test cases, and searches for minimal counterexamples that violate those properties.

In code that interacts with the external world, in particular network applications, it can be hard to find minimal counterexamples. This is because the order of execution is not deterministic: it can change every time the software is run. The same code can be run hundreds of times, and only fail once. We can get around this by using simulations with deterministic execution order. Running tests in simulation allows us to reliably find and fix a class of bugs in testing, which would otherwise only occur randomly in production.

Bridging the gap

To get a picture of the development methodology employed for Cardano, let’s consider the metaphor of bridge building. When a civil engineer builds a bridge, a large portion of their time is spent behind a desk. The civil engineer plans a design, calculates the statics, and orders geographic surveys. During that time, nothing happens at the building site. An observer would be unable to see any progress being made. For building bridges, this is the correct approach. If the planning is not accurate, it is difficult and expensive to correct problems at a later stage. Ultimately, the result would be a delayed bridge at a higher cost, or one which fails completely. Lack of visible progress is a good price to pay for a functional and safe bridge.

When building software, making changes in later stages is much easier than in construction. That is what enables the common agile development approach. If an agile developer was building a bridge, they would construct a pillar in one rapid sprint and then the next in a second sprint. The gap between the pillars would be spanned in a final sprint and, if things didn’t hold up, the developers would add on one more sprint to fix any issues. While progress would be demonstrable at the building site, the final product would likely have a great deal of problems built into it. This creates clean up work at the end of the project which could have been avoided by better planning at an earlier stage. Furthermore, the minimum viable product would likely be given to a small group of people for a test drive with the expectation that it would fail in order to alert developers of bugs. Needless to say, it is best that bridges aren’t designed in this way.

When hearing the words 'formal methods’, a lot of people in software development think about the civil engineering approach, which is dubbed ‘waterfall’ and generally shunned. This is a common but unfortunate misunderstanding. Indeed, using appropriate formal techniques allows us to have our cake, and eat it too: to have an overall design (a deliberate design, not an accidental one from fitting together pieces developed in sprints), to show progress continuously, and to retain the ability to react to changing requirements.

A key technique employed by IOHK developers is executable specifications. These are high level designs, which abstract over low level details, written in a language that the computer can understand and execute. Executable specifications can be used as prototypes to show progress, get feedback from users, and test assumptions. On top of that, lower level details can be added via successive refinements. Our developers build the bridge to solve the biggest problems first then add pillars to reinforce it at a later time. In a software system, the pillars would be features like saving data to disk, or using performant algorithms, which are needed for a final product, but which are not essential to demonstrate the overall functionality.

Using executable specifications, we get the benefits of proper planning without sacrificing flexibility. IOHK developers can fix what the system should look like on a large scale, and then implement suitable components as needed. Continuous testing guarantees that each component fits the overall design. This helps prevent problems that are common in a late integration approach. With this methodology, we get the best of both worlds: we can use a top-down design (avoiding late integration troubles, having a good handle on the overall design at all times), and have working code early (demonstrating progress, and allowing for tests and feedback through the whole process).

Future proof

Ultimately, the method of construction should be determined by what is being built. IOHK is building a global social and financial operating system which requires rigor and speed. Formal versus agile is a false dichotomy. Instead, we’re continuing to develop our methodology which fuses the best of both approaches: formal techniques within an agile delivery framework, with robust, higher assurance code upon which we can build for all our futures.

Architecting Shelley: an interview with Duncan Coutts

A fireside chat with Duncan Coutts, Cardano's chief technical architect, about Haskell and delivering Shelley

7 April 2020 Eric Czuleger 6 mins read

Architecting Shelley: an interview with Duncan Coutts

Duncan Coutts has been an important guide on the road to the Cardano Shelley mainnet. Long time supporters of IOHK are likely familiar with his signature long hair, beard, and penchant for drinking tea while discussing decentralization in front of a white board. He recently sat down for an interview to discuss the upcoming Byron reboot, the Haskell Shelley testnet, and the conclusion of the pre-Shelley development cycle. Coutts, who has been working with IOHK since 2016, brings a wealth of knowledge from working with the Haskell programming language for nearly 20 years and helping found the Well-Typed consultancy.

What’s your role at IOHK?

I’m the chief technical architect for the Cardano project and I’m primarily responsible for the design and implementation of the node. This means that I collaborate with the teams that work on consensus, ledger, networking, and other things. Ultimately, I work to bring everyone together around the same design after a discussion with the team leaders. The design of Cardano is the product of joint work by many individuals working together.

What does the Haskell programming language bring to Cardano?

Haskell is an enabler. It makes it easier for us to follow the approach that we believe is right, which is driven by computer science. We know how to do things properly; computer science tells us how. We just need to pick the appropriate techniques to do that. Haskell makes that easier.

It’s a good fit for Cardano because it suits the high-assurance, specification-driven software that is vital for a blockchain. Haskell helps us find systematic ways of avoiding mistakes. In essence, it’s a better mousetrap.

You’ve been working with Haskell for a long time. How have you seen the landscape of functional programming change?

People take it seriously now. When I started as an undergraduate in 1999, I thought that Haskell was amazing. Other students thought, ‘Wow that’s totally impractical. How will you ever get a job?’

At the time, functional programming was an academic curiosity. There wasn’t any prebuilt code and it wasn’t machine readable, which meant that Haskell wasn’t usable for a wide range of people. There wasn’t the tooling, range of libraries, or experience. That has changed over the years: the tooling got better, the libraries got better. IOHK has helped develop the infrastructure for building and distributing open-source Haskell code and the number of libraries exploded. That, combined with more teaching and a gradual change of attitude in the industry, means that people take it more seriously now. Haskell hasn’t changed as much as the industry around us has.

What’s the biggest change from an industry point of view?

There are two things. The first is that attitudes are changing, albeit slowly. People are changing their opinions about what they consider a sensible language choice. Previously, everything had to be in C or Java or maybe Python, but eventually good ideas make progress, even if it takes a long time. You can make a lot of progress by just recognizing that a good idea is a good idea. The mainstream does pick up on important developments, even if it does take 10 or 15 years. The industry has not embraced functional programming wholesale yet, but individual programmers have taken up various ideas. That makes Haskell look less radical.

If you look at a language like Rust, it has some of the clever type systems of Haskell, although it doesn’t have any functional programming ideas. Even Java and C++ have some functional programming ideas in them these days, so Haskell is not quite so far from the mainstream as it used to be.

The second major change has been performance, which is getting much better. We’ve recently become competitive with Java in terms of performance. It makes people say, ‘Wow, Haskell is so fast,’ but that’s because they’re comparing it to Python and PHP rather than C. So that’s another way of saying that Haskell has improved slightly, but the industry environment around it has changed as well.

You have been heavily involved in the Byron reboot which was kicked off last week. Why was this work important?

The Byron reboot is the culmination of over 18 months of hard work across multiple IOHK development teams, and constitutes a complete overhaul of the node infrastructure with 100% fresh code. The reboot introduces an extensible, modular design within the node itself, separating out the ledger, consensus, and networking components, as well as improvements and new functionality in the wallet backend and the Cardano explorer.

For Daedalus users, the Byron reboot will see us moving to a regular update cadence [see our recent piece on Daedalus Flight for more on that], after which they should find that Daedalus is faster, more reliable, and uses less memory. A lot of the issues users have experienced with Daedalus in the past were due to the underlying node, rather than Daedalus itself. The Byron reboot will go a long way to improving things, and users should see Daedalus syncing and restoring wallets within minutes, even when downloading the entire Cardano blockchain.

As the chief architect, your job is to lay the foundation for Cardano’s future. What have you focused on to achieve this?

The most important aspect in terms of flexibility for the future is keeping different functions separate. One of the big improvements of the Byron reboot is that the ledger rules will be totally independent of the consensus implementation; this modularity means that the ledger rules are perfectly clean mathematical functions, which is a core aspect of functional programming.

As a result, everything is easier to test, tweak, and change, both now and in the future. The consensus algorithm isn’t entangled with the details of the ledger rules, so we can alter the ledger rules without changing the consensus implementation. This makes integrating Plutus and smart contracts functionality much easier and will also help in the future when we are adding treasury and governance features.

The consensus implementation itself has also been parameterized so that we can transition from the Ouroboros Classic consensus protocol to BFT and then Praos, which also provides flexibility for future versions of the protocol that haven't been developed yet.

Shelley is a big step towards the future of Cardano, but what is the significance of compiling Haskell into JavaScript and WebAssembly?

We’re interested in compiling to JavaScript or WebAssembly because of Plutus. We want to have Plutus contracts or Plutus applications that can be distributed to users, which would include custom interfaces and custom logic with the user rather than in a server. Compiling to JavaScript allows us to do that; you can compile the Plutus code once and distribute it to users on different platforms.


Thanks to Duncan Coutts for his time. As chief technical architect, he’s a cornerstone of the Cardano project and has been fundamental to the ongoing success of the platform. For more interviews with the team, stay tuned to our social channels and the IOHK blog.

We need you for a Daedalus testing program!

A call for the Cardano community to help shape IOHK’s ada wallet for the Shelley era

1 April 2020 Anthony Quinn 7 mins read

We need you for a Daedalus testing program!

The Daedalus team is opening up IOHK’s wallet testing program to ada owners. The aim is to seek the help of a broad range of people who can test – on a rolling basis – the latest interface features being readied for the next release version of Daedalus. This will be a fully-functioning version of the wallet, called Daedalus Flight, so you will be able to spend and receive ada as usual – and give the team valuable feedback to improve the experience with each release.

We put some questions to Daedalus Product Manager, Darko Mijić, to tease out the details for people who want to take part.

Darko, can you expand on the thinking behind the program?

I’ve been working at IOHK for almost four years. It’s been amazing to see the way the Incentivized Testnet has galvanized the Cardano community in the past few months. The community helped us develop the stake pools for Shelley, and, of course, to test and develop a version of the Daedalus wallet for the ITN. So, building on that strategy, we came up with the idea of Daedalus Flight. This is a pre-release version of the Daedalus wallet that users can test with real ada transactions on the mainnet. People can join the program by downloading and using the Flight wallet alongside their usual Daedalus – or Yoroi – wallet to find and report issues so we can fix them before the actual release. We’ve seen that people want to help – and they will gain early access to the new features of the next production Daedalus release for the mainnet.

It’s similar to beta testing, or the ‘canary’ approach Google uses for its Chrome browser. Adding this tactic to our release process creates another valuable source of testing data. We will be able to improve the quality assurance of our new software versions and move more quickly into full production by first trialing these changes in Daedalus Flight with a subset of participating users.

But is it for everyone?

This program is especially important because it will give us a bigger sample of users who are using Daedalus in various ways on a wide variety of software and hardware configurations. We want to bring confident ada owners on board to help us develop the wallet by testing features and suggesting their own ideas. People who volunteer for this must already be familiar with using the Daedalus wallet. People need to be skilled in using computers – copying and backing up files, moving things around between folders – and prepared to deal with any issues that arise. We’re not expecting IT experts, but you do need to be confident about computers and making ada transactions.

How will the process work?

We complete a Daedalus development sprint every two weeks and quality assurance is a continuous effort during the development of every feature. At the end of each sprint, we will now create a new Daedalus Flight release, and, starting with the first flight candidate, we do our tests internally. But testing in a lab for a mass-market desktop product is not a real test. Under the new flight process, we will release a series of flight candidates and work on testing, finding issues and fixing issues with members of the Cardano community who have joined the program to help us. Flight candidate releases will be delivered through the Daedalus newsfeed, after an initial download from a new Flight option on the official Daedalus website, which is the only source of the Flight wallet.

Once we reach our final candidate, and we are confident that the release ‘can fly’, we will be releasing it to production for all Daedalus users.

This process will repeat every two weeks. There will be instances when we don’t release the production version after completing a flight release. This will happen when we are building big, multi-sprint features that cannot be delivered partially.

When you download Daedalus Flight, you can compare balances and transaction histories and see that all your wallet data has moved across. If there’s a problem, you just report it and go back to using your usual wallet while we fix things and release another candidate.

Why is this on the mainnet? Won’t it put our ada at risk?

Your ada will not be at risk. The Flight wallet is a real wallet and can do everything your usual wallet can. We’re adding features to make things easier for users. We could have done this on a testnet but the real test is where the real wallets live, on the mainnet. ‘Real’ user testing only happens after a wallet update is released to the mainnet. This flight process is secure because it is a completely separate wallet installation. We import your wallets from your production version of Daedalus. The production version of Daedalus stays untouched and fully functional.

So, we keep our present wallets?

Yes, you keep all your present wallets. You can use them alongside each Daedalus Flight candidate.

How will we give feedback?

There is an option to open a support request directly from the Daedalus interface. These requests will be handled separately by the support desk, and you can attach a wallet log so we can investigate your problem. If you just want to suggest an idea, you can do that too by clicking ‘Support request’ in the help menu.

Can you give us a sneak preview of the features?

Well, first of all, there will be Yoroi support in the new Flight wallet. Yoroi users will be able to use the same wallet both in Flight and Yoroi. Transactions will be in sync. Then, there will be a transaction filtering function in your transaction history so you can filter transactions by type, date, and ada amount. There will also be a warning in the transaction confirmation window making sure users understand that Daedalus Flight transactions are real ada transactions on the mainnet. Other neat touches will include parallel wallet restoration and a resync wallet function. Later on, we will be adding cool features like hardware wallet support.

How long will the Daedalus Flight program last?

The program started with the Byron reboot on March 31, with the first flight release and its candidate #1 release. If we discover and fix issues, we will issue a second candidate. When we get to a fully stable candidate, we will release it to all Daedalus users. April 6 is the planned date for the first production-ready Daedalus, but this depends on the results and user data from Daedalus Flight.

The next release and its series of candidates can be expected for April 14, and then every two weeks. Because this is a new product, we will try this process out for a month or two. If successful, we plan to make this permanent practice in the long term. It is important to note that the special Daedalus version for the Incentivized Testnet will still exist – that is a completely separate product.

Any final message?

People should be clear that the Daedalus Flight wallet is used with your mainnet ada. It’s a real wallet with all the core functions of Daedalus. It’s just the new features, which are not to do with transactions, that we’re testing. Going back to the Chrome comparison, think of it as buying something from Amazon using a beta test release of a web browser; you are doing real things with test software.

We hope you’ll be as excited as we are by this innovation – and can help our Daedalus development really fly! And remember, for the latest information and news, visit the official Daedalus website.