Hydrozoa: Lightweight and scalable Hydra Heads

[GENERAL] Name and Surname of Main Applicant

George Flerovsky

[GENERAL] Email address of Main Applicant

george@flerovsky.ca

Additional Applicants

None.

[GENERAL] Please specify how many months you expect your project to last (from 2-12 months)

9

[GENERAL] Please indicate if your proposal has been auto-translated into English from another language.

No

[GENERAL] Does your project have any dependencies on other organizations, technical or otherwise?

Yes

[GENERAL] If YES, please describe what the dependency is and why you believe it is essential for your project’s delivery. If NO, please write “No dependencies.” in this field.

Yes, this project should ideally be aligned with the Hydra core dev team's development roadmap to minimize incompatible forking and duplication of effort. A healthy working relationship with that team has already been established in previous projects, and we will pursue this project in close collaboration with them.

[GENERAL] Will your project outputs be fully Open Source?

Yes

[GENERAL] If NO, please describe which outputs are not going to be open source. If YES, please write “Project will be fully open source.” in this field.

The project will be fully open source.

[METADATA] Category of Proposal

Dev Tools, API or Library

[IMPACT] Please describe your proposed solution.

This project will implement the Hydrozoa protocol — an adaptation of the Coordinated Hydra Head protocol with more lightweight and scalable Hydra Heads.

Hydrozoa introduces cheap and deterministic state update transactions as the default way to commit utxos to and decommit utxos from the Hydra Head. A similarly cheap finalization procedure can be used to stop operating the head if consensus can be maintained throughout its life. The halt and dissolve procedures are an alternative workflow available if L2 consensus breaks down — optimized analogs to the close and fanout operations in the Coordinated Hydra Heads protocol.

Introduction

The Hydra Head protocol allows a group of participants to temporarily transfer control over a subset of utxos from the Cardano mainnet (L1) ledger to a private unanimous consensus protocol (L2) run by the participants instead of the Ouroboros consensus protocol running on the whole Cardano mainnet. While the Hydra Head controls these utxos, transactions involving them are quickly broadcast among the participants and then confirmed with immediate finality as soon as all participants sign a corresponding snapshot. The utxos resulting from the L2 transactions can be brought back under L1 control by finalizing an L2 snapshot (via Close/Contest transactions) and then fanning them out on L1 from the state utxo.

The protocol was developed before Plutus was added to mainnet, effective design patterns emerged for building dApps in the eUTXO model, and the Vasil enhancements enabled a completely different approach to dApp design on Cardano. It can be improved by applying the lessons learned over the past year.

Design

The main design ideas of Hydrozoa are:

1. Eliminate the initialization phase — open the Hydra Head immediately with an empty L2 active utxo set.
2. Commit utxos on L1 to the head simply by sending them to a native script address controlled by the participants' keys.
3. Transact on L2 like before, using Cardano ledger rules to apply transactions to the L2 active utxo set.
4. Decommit utxos via a new L2 request to remove certain utxos from the L2 active utxo set, validated like an input-only transaction via modified Cardano ledger rules.
5. While the head is open, periodically collect commits and release decommits on L1, updating the major version in the L1 head state. Semantic versioning is used because these updates are backward incompatible with previous snapshots — updates change the balance of funds at the head state utxo.
6. If participants can maintain L2 consensus, finalize the head, and release its L2 active utxo set on L1 just as cheaply and efficiently as if all utxos decommitted.
7. If L2 consensus breaks down, halt the head and dissolve its remaining funds to produce outputs corresponding to the L2 active utxo set of the snapshot with which it was halted. Dissolution is specified and authorized via multi-signed certificates instead of multi-signed transactions.
8. Spin off the contestation mechanism so that it can only affect the head state when the contestation deadline elapses, or all participants have exercised their right to contest.
9. Updates should be used during normal operation to commit and decommit utxos, finalization should be used if the participants want to stop operating the head, and the contestation mechanism should only be used to halt the head if consensus breaks down.

The L1 component of the Hydra Head protocol becomes simpler, cheaper, and more scalable. In exchange, the L2 component is somewhat more complicated. However, managing complexity in the L2 and off-chain components is usually easier and cheaper than in the L1 components, which incur transaction fees and often use smart contracts that must be carefully developed and audited.

The full design of the Hydrozoa protocol has already been meticulously thought-out and described in the following whitepaper:

• Hydrozoa: Lightweight and scalable Hydra Heads

The paper has been shared with the Hydra core dev team and was well-received by them. They have forwarded it to the IOG researchers who developed the original Hydra Head protocol for further comment. In principle and at a high level, the participants in the discussion thread agree that the features described in the Hydrozoa paper make technical sense and could be feasible:

• Discussion thread with core devs on the Hydra repository's Ideas forum

Project scope

The whitepaper has described the mechanisms of the protocol in great detail so that a technical reader can form a concrete understanding of its operation. To this, a full specification would add concrete definitions for the following parts of the design:

• The validators and minting policies -- datums, redeemers, local utxo logic, etc.
• The auxiliary deterministic algorithms:

1. "Algo_settle" to select the largest prefix of commits to collect per major snapshot and derive settlement transaction trees.
2. "Algo_dissolve-cert" to derive dissolution certificate trees.
3. "Algo_dissolve-tx" to derive dissolution transaction trees from dissolution certificate trees.

• The asynchronous messaging protocol used by participants in the Hydrozoa head.

Once the above is sufficiently specified, a smart contract dev team can proceed with implementation. The on-chain logic is expected to be straightforward (its simplicity was an explicit goal of the design). However, the off-chain logic is significantly more complex than in the Hydra Head protocol currently implemented in the input-output-hk/hydra repository. This is where the bulk of the Hydrozoa project's effort will be directed.

The Hydrozoa project will be developed with the Hydra repository in mind, and the deliverables of our project should strive to be applicable to that repository. To the extent possible, we will contribute PRs and ideas wherever appropriate.

[IMPACT] How does your proposed solution address the challenge and what benefits will this bring to the Cardano ecosystem?

As we discovered while applying the Hydra Head technology in the development of a non-trivial smart contract application, the Hydra Head still has a few major limitations that are an obstacle to its widespread adoption in commercially viable apps on Cardano mainnet.

As we mentioned in the hydra auction whitepaper (Jan 2023), the main obstacles to commercial viability and scalability of a typical hydra-based application are:

• The set of participants in a Hydra Head is fixed before the head opens -- no new participants can enter the head while it is open, and no participant can leave the head without causing the head to close.
• The set of utxos and funds in a Hydra Head is fixed before the head opens -- no new funds can be brought into the head or withdrawn out of the head until it is closed.
• There is significant overhead in cost and time associated with opening and closing hydra heads.

Hydrozoa addresses the second limitation directly by embracing incremental commits/decommits to/from open heads at the core of the protocol. Furthermore, the first limitations can potentially be lifted via the "major snapshot" mechanism that Hydrozoa introduces -- through the implementation of dynamic membership within a head will be left for a future project.

Lifting these restrictions will allow a diverse range of L2-enabled applications to flourish on Cardano because it will become cheap and easy to open and operate hydra heads for short periods of time or as persistent services. Furthermore, these lightweight hydra heads will make launching a network of heads more feasible, similar to Bitcoin's Lightning network.

[IMPACT] How do you intend to measure the success of your project?

The success of this project will be determined by whether the mechanisms contemplated in the Hydrozoa protocol design bear out in practice, providing the following improvements over the current Hydra Head implementation:

• An easier and cheaper process to open, operate, and close hydra heads.
• Easy and cost-effective commit and decommit transactions to move funds in and out of open hydra heads.

These improvements should be compared against the additional complexity and computational resources required to run the L2/off-chain part of the Hydrozoa protocol relative to the Hydra Head protocol.

We will keep these benefit/cost metrics in mind during the project timeline and develop benchmarks to monitor them as early as feasible in the project roadmap.

[IMPACT] Please describe your plans to share the outputs and results of your project?

The entire project will be open source and richly documented. It will be pursued with regular communication with relevant stakeholders like the Hydra core dev team, the Cardano developer community, and entrepreneurs in the ecosystem.

It is anticipated that many spinoff applications of the Hydrozoa application may form from the usage examples that we will develop during the course of the project -- we already have informal concepts/designs for auctions, voting, stablecoins, and other applications that use the Hydrozoa features. We are well-connected with the Cardano entrepreneurs and developers who may be interested in pursuing these new designs.

[CAPABILITY/ FEASIBILITY] What is your capability to deliver your project with high levels of trust and accountability?

George Flerovsky is the author of the Hydrozoa whitepaper. He has significant experience with the Hydra Head protocol and implementation via his involvement in the Hydra auction project from Sep 2022 to Jul 2023. He has carefully studied the associated research papers and specs and he has developed an intimate understanding of the Hydra implementation in input-output-hk/hydra repository. He has contributed to discussions about important recent features in the Hydra roadmap (released in version 0.11.0):

• Commits from external wallets
• Allow commits with multiple UTxO
• Support timed transactions on L2

George has been prolific in the smart contract application development ecosystem on Cardano. He has helped develop the design, define the project roadmap and budget, and oversee the successful development of the following applications:

• Hydra auction -- an L1/L2 hybrid application that can host the bidding process inside of a Hydra Head, providing a livelier and cheaper bidding experience.
• Grabbit -- an L1-based auction platform that allows bidders to participate without interference from each other or the seller (i.e. no utxo contention) and without any privileged third parties (i.e. batchers).
• Cardano Open Oracle Protocol (COOP) -- a protocol and SDK for publishing and consuming on-chain data with proof of provenance from the publisher and flexibility in publication cost-sharing between the publisher and consumer of data.
• Equine -- an NFT-based horse racing, breeding, and trading game.
• Metera -- a decentralized protocol to create tokenized portfolios of assets balanced via a smart-contract-based pricing function that incorporates performance metrics and ESG ratings provided via oracles.
• Optim -- liquidity bonds and other innovative yield products built on Cardano.
• LambdaBuffers -- a schema language and code generation toolkit that allows developers to define algebraic data types in a language-agnostic format and then derive corresponding code in a plethora of supported languages.
• Dynamic NFT metadata systems for multiple games
• Governance/DAOs

George is also the current chair of the Cardano DeFi Alliance and of the Cardano Developer Experience Working Group. He intends to keep those groups connected to the progress of the Hydrozoa project so that the project remains relevant to the entrepreneurs, users, and developers in those groups.

The rest of the development team will be provided by MLabs, under a sub-contract. MLabs is a leading consultancy in the Cardano ecosystem with a proven track record and significant experience. Its team consists of seasoned engineers, each holding expertise in their respective fields. Moreover, MLabs has consistently demonstrated its ability to deliver complicated projects with a high degree of trust and accountability. It has an extensive portfolio of satisfied client projects as well as several popular Catalyst projects.

[CAPABILITY/ FEASIBILITY] What are the main goals for the project and how will you validate if your approach is feasible?

The main goal of the project is to develop a prototype implementation of the Hydrozoa protocol that can demonstrably:

• Open and close a head with low overhead costs and latency.
• Allow utxos to be committed and decommited to/from the head while it is open, simply and with low costs.

If the implementation meets these feature targets, then it will validate the Hydrozoa approach and make it very appealing for entrepreneurs and developers to adopt L2 scaling in their applications.

[CAPABILITY/ FEASIBILITY] Please provide a detailed breakdown of your project’s milestones and each of the main tasks or activities to reach the milestone plus the expected timeline for the delivery.

M1 – Specification and architecture (Nov 2023)

• Specify the on-chain validators and minting policies.
• Specify the auxiliary deterministic algorithms used by the off-chain protocol on L2.
• Specify the asynchronous messaging protocol between participants on L2.
• Explore options and define the component architecture of on-chain scripts, off-chain code, data systems, and their corresponding APIs.
• Set up the project repository, module structure, project board, and team processes.

M2 – Basic Head state model and on-chain transitions (Dec 2023)

• Implement the state model for L1 state transitions.
• Implement the on-chain validators and minting policies for L1 state transitions.
• Implement basic off-chain tx building code for L1 transitions.

M3 – Off-chain data systems and ledgers (Jan 2024)

• Implement the data models and systems to track pending commits, active utxo set, decommitted utxo set, etc.
• Implement the L2 consensus state model and data systems.

M4 – Incremental commits/decommits (Feb 2024)

• Implement basic L1 and L2 transactions for committing and decommitting utxos.
• Implement corresponding effects on L2 data systems and ledgers.

M5 – Snapshots creation/verification and participant messaging protocol (Mar 2024)

• Implement the snapshot creation process.
• Implement the snapshot verification process.

M6 – Halting, contestation, and dissolution (Apr 2024)

• Implement the halting and contestation mechanisms for closing the head in case of L2 consensus failure.
• Implement the dissolution process based on the prevailing snapshot.

M7 – Deterministic algorithms for certificate/tx trees (May 2024)

• Implement the "Algo_settle" algorithm to derive settlement transaction trees.
• Implement the "Algo_dissolve-cert" algorithm to derive dissolution certificate trees.
• Implement the "Algo_dissolve-tx" algorithm to derive dissolution transaction trees from dissolution certificate trees.
• Expand the L2 protocol to support more decommits between snapshots and larger utxo sets than those that fit in a single L1 transaction, as they can now be processed by transaction trees.

M8 – Full protocol and testnet deployment (Jun 2024)

• Integrate the various parts of the Hydrozoa protocol.
• Optimize the flow and efficiency of transactions and operations.
• Prepare and deploy the full protocol on testnet.

M9 – Wrap-up (Jul 2024)

• Complete the test suite with adequate coverage of the various architecture components.
• Finalize documentation — usage, deployment, examples, tutorials, etc.

[CAPABILITY/ FEASIBILITY] Please describe the deliverables, outputs and intended outcomes of each milestone.

M1 – Specification and architecture (Nov 2023)

Deliverables:

• Specification document for on-chain validators and minting policies.
• Specification document for the auxiliary deterministic algorithms.
• Specification document for the async messaging protocol.
• Architecture document.
• Open-source project repository and project board.

Acceptance criteria:

• Specifications are clearly written and complete, sufficient for implementation.
• The open-source project repository and project board are well-organized and provide a good structure for team members to contribute.

M2 – Basic Head state model and on-chain transitions (Dec 2023)

Deliverables:

• On-chain and off-chain code for the L1 state of the Hydrozoa protocol.
• Implementation of the L1 state model.

Acceptance criteria:

• On-chain and off-chain code covers the L1 state and transitions described in the specifications.

M3 – Off-chain data systems and ledgers (Jan 2024)

Deliverables:

• Data models and systems that track pending commits, the L2 ledger's active utxo set, utxos decommitted by L2 participants, and other relevant data for the L2 protocol.
• Implementation of the L2 consensus state model.

Acceptance criteria:

• Data models/systems and the L2 consensus state model conform to the specifications.

M4 – Incremental commits/decommits (Feb 2024)

Deliverables:

• The codebase includes commit and decommit operations for open Hydrozoa heads.

Acceptance criteria:

• A CLI-based demo is recorded to showcase the commit/decommit feature.

M5 – Snapshots creation/verification and participant messaging protocol (Mar 2024)

Deliverables:

• The codebase includes snapshot operations and participant messaging protocol.

Acceptance criteria:

• Code is well-documented and conforms to the specifications.

M6 – Halting, contestation, and dissolution (Apr 2024)

Deliverables:

• The codebase includes halting/contestation/dissolution operations.

Acceptance criteria:

• Code is well-documented and conforms to the specifications.

M7 – Deterministic algorithms for certificate/tx trees (May 2024)

Deliverables:

• The codebase includes the auxiliary deterministic algorithms that are necessary to allow the L2 utxo set grow in size and to handle more commits between snapshots.

Acceptance criteria:

• Code is well-documented and conforms to the specifications.

M8 – Full protocol and testnet deployment (Jun 2024)

Deliverables:

• The full workflow of the Hydrozoa protocol is available in the codebase.

Acceptance criteria:

• A CLI-based demo is recorded to showcase the full Hydrozoa protocol in operation.
• Code is well-documented and conforms to the specifications.

M9 – Wrap-up (Jul 2024)

Deliverables:

• Finalized documentation, examples, and tutorials for the project.
• Complete test suite for the major components of the Hydrozoa architecture.

Acceptance criteria:

• Documentation, examples, and tutorials are well-written and approachable by a technical and non-technical audience, as appropriate.
• The test suite has high coverage of the codebase.

[RESOURCES & VALUE FOR MONEY] Please provide a detailed budget breakdown of the proposed work and resources.

The budget breaks down into the following activities:

• 300 h = Specification and research
• 220 h = L1 on-chain code implementation
• 250 h = L2 data engineering
• 400 h = L2 messaging and consensus protocol
• 200 h = Testing
• 150 h = Documentation
• 120 h = Deployment
• 350 h = Change budget reserved for technical/architecture changes that may come up during the project, based on stakeholder feedback and to provide a better final outcome in the project.

The total hours of effort involved with the above activities is 2090 h. At the standard $95/h rate, the total USD cost of the project is $198,550.

We apply a 0.21 USD/ADA exchange rate, which is the average rate over the past few months, conservatively adjusted downward to ensure that the project will remain viable despite moderate price shocks that may occur before or during the project timeline.

The corresponding total ADA cost of the project is 945,476 ADA.

We propose a linear budget allocation schedule for the milestones, with a double allocation for the last milestone:

• M1 = 10%
• M2 = 10%
• M3 = 10%
• M4 = 10%
• M5 = 10%
• M6 = 10%
• M7 = 10%
• M8 = 10%
• M9 = 20%

[RESOURCES & VALUE FOR MONEY] How does the cost of the project represent value for money for the Cardano ecosystem?

The costs described in the budget breakdown of this proposal have been developed based on a detailed plan of the activities required to achieve the milestones. We have high confidence in the comprehensiveness and accurate sizing of these activities, as we have engaged in very similar projects to this in the past at complexity level and subject domain.

The USD hourly rate is a typical rate that we and our fellow developers in the smart contracts and dApp sector have used in past projects to provide high-quality services, with a proven track record of delivering projects despite the complexity of viable applications in this space. This requires experts on staff for a diverse range of niche skillsets and deep knowledge of the Cardano and Hydra protocols, the EUTXO model, and the modern practices of dApp design.

The USD/ADA exchange rate reflects the average rate over the past few months, conservatively discounted down to 0.21 USD/ADA to allow the project to remain viable despite potential price shocks that may occur after proposal submission and during the project timeline.

[IMPORTANT NOTE] The Applicant agreed to Fund10 rules and also that data in the Submission Form and other data provided by the project team during the course of the project will be publicly available.

I Accept