Elixir implementation of Ouroboros networking

[GENERAL] Name and surname of main applicant

Carlos Souza

[GENERAL] Are you delivering this project as an individual or as an entity (whether formally incorporated or not)

Individual

[GENERAL] Please specify how many months you expect your project to last (from 2-12 months)

6

[GENERAL] Please indicate if your proposal has been auto-translated into English from another language

No

[GENERAL] Summarize your solution to the problem (200-character limit including spaces)

Develop an Elixir client library for Ouroboros networking protocols, enabling Cardano development in Elixir and on the Erlang VM (aka BEAM)

[GENERAL] Does your project have any dependencies on other organizations, technical or otherwise?

Yes

[GENERAL] If YES, please describe what the dependency is and why you believe it is essential for your project’s delivery. If NO, please write “No dependencies.”

This project relies on Pallas by TxPipe (in particular, pallas-network) and gOuroboros by Blink-Labs, whose source code will be used as a reference - in particular for the CBOR parsing alongside official documentation.

[GENERAL] Will your project’s output/s be fully open source?

Yes

[GENERAL] Please provide here more information on the open source status of your project outputs

The project will be released under the Apache License 2.0 and publicly available from the start.

[METADATA] Horizons

Developer Tools

[SOLUTION] Please describe your proposed solution

To develop a library in the Elixir programming language designed specifically to implement Cardano’s Ouroboros node-to-client (N2C) protocol. This library will facilitate direct integration and interaction with a Cardano node and serve as a robust foundation for developing future Elixir based projects and services.

By leveraging the capabilities of Elixir and the BEAM (Erlang’s VM), this library aims to expand the developer ecosystem, enabling a wide range of technologies that compile to the BEAM to build innovative and efficient applications on the Cardano platform.

Given the lack of Elixir libraries for low-level Cardano protocols, the effort involved in adding support for each of the mini-protocol will also include developing the necessary Elixir code for CBOR parsing. This effort will also be able to be re-used by future Elixir projects that need to work with Cardano CBOR.

The development of this project will be done using a phased Cardano era approach (starting era yet TBD), starting with the preview testnet and then evolving into mainnet as features are validated.

[IMPACT] Please define the positive impact your project will have on the wider Cardano community

This project aims to deliver significant value to the Cardano community by harnessing the strengths of Elixir, Erlang, and the BEAM—technologies renowned for building highly concurrent, fault-tolerant, and resilient distributed systems. These characteristics are in line with what is expected from a blockchain network, thus making Elixir an ideal language to build systems on Cardano.

Elixir is a functional, dynamic programming language that runs on the Erlang Virtual Machine. The underlying Erlang VM (known as the BEAM) was developed by Ericsson and has been battle tested for over 30 years. A few examples of services and tools built using this technology include WhatsApp, RabbitMQ, Discord and AdRoll just to name a few. These are all services known for their high availability and throughput.

There exists a substantial, yet untapped, ecosystem of Elixir developers who currently do not have the tools to build on Cardano. By providing these developers with familiar and powerful tools integrated directly with Cardano's infrastructure, this project will not only broaden the developer base but also foster innovation and fresh perspectives within the community. This expansion encourages a cross-pollination of ideas and practices from different tech backgrounds, enhancing the innovative capacity of the Cardano ecosystem.

Additionally, this project will significantly diversify the tools and technologies available to Cardano developers. By expanding the technological landscape of the Cardano ecosystem, this project will enable a broader range of developers to contribute to and benefit from the ongoing growth and success of Cardano.

And if you'd like one more reason to bet on Elixir, then allow me to quote Charles H. himself:

It's one of the sexiest languages ever written

[CAPABILITY & FEASIBILITY] What is your capability to deliver your project with high levels of trust and accountability? How do you intend to validate if your approach is feasible?

The main applicant of this proposal has worked in the software development industry for over 15 years, with the last 7 years dedicated to consulting in Elixir. He's published over 70 video courses published on Pluralsight on various technology topics including the Elixir programming language - https://app.pluralsight.com/profile/author/carlos-souza

The feasibility of this project has been validated through a Proof of Concept which successfully connects to a Cardano node through a UNIX domain socket and performs a simple query using the node-to-client (N2C) local state query protocol. The source code for this POC is available at https://github.com/caike/rex

The accountability of the applicants can be verified through the contributions to the Cardano ecosystem. A few examples include:

1- Created Xogmios, an open-source Elixir client for Ogmios - https://github.com/wowica/xogmios

2 - Created an open-source Elixir-based block watcher Blocks https://blocks.workwithcardano.com/ using the aforementioned Xogmios library.

1- Operate the JUNGLE [JNGL] Stake Pool for 2 years (> 500 blocks minted to date) - https://cexplorer.io/pool/pool1p8ap6v2k60kkav3hqcchwltxhuwpfw2qtj8txcw4scp76hdvgmc

2- Contributed with a number of open-source projects including Lucid, Aiken, Helios, Oura, Fracada dApp, Roundtable, Lucid-Evolution, Blaze and others.

3- Continuously create developer focused Cardano educational content on the following youtube channel https://youtube.com/@workwithcardano

Through our combined experience developing numerous Elixir applications, including Xogmios, the Elixir client for Ogmios, and contributions to other Cardano projects we have gained an understanding of the different Ouroboros n2c mini-protocols. This gives us confidence in the feasibility of this project and in the fact that this team is well suited to deliver its promise.

[PROJECT MILESTONES] What are the key milestones you need to achieve in order to complete your project successfully?

Note: The milestones of this project contain the different node-to-client (N2C) mini-protocols that will be implemented. Each milestone will add support for a mini-protocol based on the currently documented specification found on https://ouroboros-network.cardano.intersectmbo.org/pdfs/network-spec/network-spec.pdf

This first milestone (partially implemented through the PoC) includes:

1- Establishing a connection with an existing Cardano node using UNIX socket domain. Potentially supporting a Demeter.run TLS endpoint as well, for increased developer experience. The primary focus will be the most recent 32784 protocol version. Support for older versions is possible, but not guaranteed in the scope of this proposal.

2- Support for some Local State Queries. The queries are yet TBD and not all of them will be possible to cover in this proposal.

3- A custom Dockerfile that allows running this project without having to provision a local Elixir development environment.

This first milestone will involve a re-write of the initial Proof of Concept to better leverage Elixir's and OTP's concurrency features as well as proper documentation and a Dockerfile to allow easy on-boarding of new Elixir developers looking to use this library.

This milestone adds support for Local Transaction Submission mini-protocol. It includes submitting a finalized transaction (signed) and interpreting whether the transaction was accepted or rejected. The actual building of the transaction is outside of the scope of this proposal.

This milestone adds support for the Chain Sync mini-protocol. The goal is to be able to sync with a particular point on the chain and start requesting block information such that it can be reported to upstream clients of this library. There are many scenarios needed for full support of this mini-protocol and not all of them will be included in this proposal. It is expected that most of the work in this milestone will be dedicated to parsing the CBOR for the different types of transactions. In order to reduce the scope of this proposal and keep it within budget, the initial focus will be on syncing blocks from one particular era and then gradually adding support for additional eras as time and budget allows. The era in question is yet TBD.

This final milestone will include a video presenting the library, demonstrating each one of the mini-protocols implemented in the previous milestones and talking about next steps.

[RESOURCES] Who is in the project team and what are their roles?

Carlos Souza - Elixir Developer and Tech lead

https://github.com/caike

https://www.linkedin.com/in/caike/

https://twitter.com/caike

Dave Miner - Elixir Developer

https://github.com/daveminer

https://www.linkedin.com/in/dave-w-miner/

[BUDGET & COSTS] Please provide a cost breakdown of the proposed work and resources

Milestone 1: 28% (115 hours - 25,200 ADA)

Research, development and documentation for implementing the Handshake and Local State Query mini-protocols. Dockerfile for running the project.

Milestone 2: 20% (80 hours - 18,000 ADA)

Research, development and documentation for implementing Local Tx Submission mini-protocol.

Milestone 3: 50% (200 hours - 45,000 ADA)

Research, development and documentation for implementing Chain Sync mini-protocol.

Milestone 4: 2% (5 hours - 1,800 ADA)

Produce a close out video presenting the result of this project and demonstrating each one of the mini-protocols implemented in the previous milestones.

[VALUE FOR MONEY] How does the cost of the project represent value for money for the Cardano ecosystem?

This project has potential to onboard a new wave of developers to Cardano, bringing knowledge from an ecosystem with over 30 years of success in the telecommunications industry. The efficiency and robustness of Elixir and BEAM-based applications cannot be overstated, and adding this technology to the currently available libraries for building in Cardano represents tremendous value for the money.