MLabs - LambdaBuffers (was Cardano dApp schemas) code generation backends for Rust, Javascript and Aiken

[GENERAL] Name and surname of main applicant

[GENERAL] Email address of main applicant

Additional applicants

Drazen Popovic

[GENERAL] Please specify how many months you expect your project to last (from 2-12 months)

[GENERAL] Please indicate if your proposal has been auto-translated into English from another language.

[GENERAL] Does your project have any dependencies on other organizations, technical or otherwise?

[GENERAL] If YES, please describe what the dependency is and why you believe it is essential for your project’s delivery. If NO, please write “No dependencies.” .

No dependencies.

[GENERAL] Will your project’s output/s be fully open source?

[GENERAL] If NO, please describe which outputs are not going to be open source. If YES, please write “Project will be fully open source.”

Project will be fully open source.

[METADATA] Category of proposal

[IMPACT] Please describe your proposed solution.

Problem

There are no language-agnostic mechanisms for specifying the types of information and values used in Cardano DApps, a detrimental fact to productivity and cross-language environment interoperability. A significant percentage of DApp development effort is spent on dealing with serialization/deserialization issues between on-chain code types, off-chain/frontend, and analytics/databases. This major source of delay in project timelines could be eliminated if types would be language agnostically defined once per DApp; with serialization/deserialization code for multiple targets generated automatically from the agnostic definition.

Currently, type definitions for a typical Cardano DApp must be specified in the Haskell programming language. Together with the PlutusTx or Plutarch libraries, Haskell is the de-facto main tool for writing Plutus programs. However, we want to facilitate expansion to other languages getting traction in this space, specifically Aiken for on-chain Plutus scripts.

The lack of language-agnostic tools for specifying Cardano DApps data types (e.g. datums, redeemers, etc.) hurts productivity. Furthermore, it constitutes a fundamental hurdle to cross-language environment interoperability and the proliferation of generic tooling to enable rich interaction with DApps (e.g. database explorers).

The mentioned issue of language-tied typing has become more severe with the proliferation of different PAB platforms, each written in different languages. PAB platforms enable developers to build software that gives users the ability to interact with Cardano DApps from a variety of platforms (e.g., server, desktop, browser, mobile etc.). Examples of such platforms include Atlas (Haskell), Cardano Transaction Lib (Purescript), Lucid (Javascript) and Cardano Serialization Lib (Rust).

To make Cardano DApps available across all the aforementioned platforms, structured information needs to be effectively shared in a language-agnostic manner and made available to different language environments. Furthermore, blockchain analytics applications, DApp aggregators, and DApp interoperability will require light DApp-type schemas that are decoupled from the DApp’s implementation code - it is not very practical to have to import a DApp’s codebase in order to interpret its datums, redeemers, and other on-chain data.

We want to consolidate the solution to this issue into a single project that takes type definitions out of a language-specific environment and into a configuration language that can be then fed to a reliable code generation tool that targets a number of different language environments.

The intent behind this project is to align the Cardano tech environment with modern best practices that facilitate building robust, technology stack-agnostic Cardano DApps.

Solution

Under Catalyst Fund9 we’re nearing completion of the LambdaBuffers project (was Cardano dApp schemas) and are looking to extend our support from Haskell and Purescript to Rust, Javascript and Aiken language environments.

The project implements a configuration language for specifying DApp types/schemas. The language makes use of code-generation tooling, which using the schemas produces ‘type libraries’ (aka. typelibs) for different language environments. Each typelib is generated alongside supporting libraries for handling encoding, serialization and other essential operations. All the mentioned processes are automated, maintaining correctness and compatibility between different language environments.

LambdaBuffers serve as a language-neutral, extensible mechanism for specifying type definitions and code-generating different language environments library implementations.

Market

Developers building Cardano DApps and extended tooling and infrastructure.

Features

• Configuration language for specifying data types aligned with the Plutus data model.
• Support for sum, record/product and polymorphic types.
• Automated production of type libraries from configuration-based specifications.
• Integration with build systems associated with the target language environments (e.g., Cabal, Spago, Node, Cargo, Nix etc.).
• Standardized data type associated operations, including equality, JSON encodings and Plutus data encodings.

Related work

For the reader’s reference, the following standards and technologies have solved a similar problem in traditional software sectors:

1. ASN.1 - adopted by widely used cryptographic standards (see X.509),
2. Google Protobuf - used by major tech companies as the foundation for RPC and microservice-based architectures,
3. Apache Thrift - used by major tech companies as the foundation for RPC and microservice-based architectures,
4. Apache Avro - configuration-based schema definition mechanism used across different Apache projects,
5. CDDL - used by Cardano itself at the network/transport level, fairly complicated standard with little supporting tooling available,
6. XDR - used for RPC in Linux-based enterprise networks,
7. Microsoft IDL - used by Microsoft DCE/MS-RPC protocols and a foundational technology for MS Windows-based enterprise networks,
8. ADL https://github.com/timbod7/adl - data specification technology inspired by and built for functional programming language environments.
9. CIP-0057 https://developers.cardano.org/docs/governance/cardano-improvement-proposals/cip-0057/ - JSON-based approach to specifying Plutus encodings via JSON schema.

[IMPACT] How does your proposed solution address the challenge and what benefits will this bring to the Cardano ecosystem?

Intended Challenge – Fund10: Developer Ecosystem - The Evolution

Challenge Statement – “How do we equip and support developers with tools and working infrastructure? How do we make Cardano their first choice when it comes to building dApps, innovating and collaborating?”

What does this proposal entail?

The current iteration of this project entails building support for 3 new languages, namely Rust, Javascript and Aiken.

How does increased throughput on Cardano help developers?

Typical Cardano DApp projects involve a very diverse team. For instance, Plutus scripts are written in Plutarch, PlutusTx and Aiken.

The frontend and backend teams use various PABs to build transactions, which include Atlas (Haskell), Cardano Transaction Lib (Purescript), Cardano Serialization Lib (Rust) and Lucid (Javascript).

We hope this proposal will make such diverse teams work together more effectively by consolidating all data type definitions in a single place and reusing them reliably across their respective technological environments.

The successful adoption of this project would radically increase developer productivity while decreasing the amount of unnecessary, tedious, error-prone and boilerplate code that needs to be manually written and curated. Type definitions would serve as documentation for the project, as well, and could very well be foundational to the proliferation of generic tools and infrastructure that can interact with schema-enabled Cardano DApp projects.

[IMPACT] How do you intend to measure the success of your project?

We will measure:

• the number of features implemented,
• the number of issues discovered and resolved,
• the usability of Cardano DApp schemas to projects that have incorporated it,
• the number of contributors adding to the GitHub repo,
• the number of Cardano projects relying on LambdaBuffers toolkit,
• the general perception of the project in the Cardano developer community.

We expect growth/positive results in these areas and are committed to meeting the milestones we have established throughout this proposal.

[IMPACT] Please describe your plans to share the outputs and results of your project?

In the spirit of complete transparency with the Cardano community, the outputs of our proposal will be widely available. Foremost, as an open-source project, the GitHub repo at https://github.com/mlabs-haskell/lambda-buffers will be the obvious resource for updates, documentation, and educational resources. This repo will be updated as the project progresses - a primary deliverable of this proposal.

MLabs has a noteworthy social media presence. We'll make active use of these channels to share regular updates with less technical community members. Finally, we'll actively share progress reports through the typical, publically-available Catalyst channels. MLabs has a strong compliance record, and this will not change throughout the progress of this proposal.

[CAPABILITY/ FEASIBILITY] What is your capability to deliver your project with high levels of trust and accountability?

LambdaBuffers project is nearing its completion in terms of its Fund9 deliverables, and with that, the overall research, design and implementation has already been done. Adding new language backends comes with little uncertainty as the framework was designed with such additions in mind.

[CAPABILITY/ FEASIBILITY] What are the main goals for the project and how will you validate if your approach is feasible?

The main goal in this iteration of the LambdaBuffers project is to add support for 3 new additional languages, namely Rust, Javascript and Aiken. The initial research phase will be necessary to understand if there are any technical limitations to the targeted backends and may prove to require some additional work in relevant upstream code bases (Aiken in particular).

[CAPABILITY/ FEASIBILITY] Please provide a detailed breakdown of your project’s milestones and each of the main tasks or activities to reach the milestone plus the expected timeline for the delivery.

1st month - Rust support

• Research the available libraries used in the Cardano ecosystem, namely Aiken Rust packages and Cardano Serialization Library and decide on MVP integration paths with LambdaBuffers,
• Implement the Rust code generation module (type definition and implementation printing),
• Implement LambdaBuffers Prelude runtime library for Rust which includes the standardized LambdaBuffers JSON encodings for both opaque and transparent LambdaBuffers types,
• Implement the LambdaBuffers Prelude test suite that provides guarantees that the new backend works as intended,
• Implement LambdaBuffers Plutus runtime library for Rust which includes the standardized LambdaBuffers PlutusData encodings for both opaque and transparent LambdaBuffers types,
• Implement the LambdaBuffers Plutus test suite that provides guarantees that the new backend works as intended,
• Implement Nix wrappers for the Rust backend,
• Document LambdaBuffers usage patterns for Rust.

2nd month - Javascript support

• Research the available libraries used in the Cardano ecosystem, namely Lucid and decide on MVP integration paths with LambdaBuffers,
• Implement the Javascript code generation module (type definition and implementation printing),
• Implement LambdaBuffers Prelude runtime library for Javascript which includes the standardized LambdaBuffers JSON encodings for both opaque and transparent LambdaBuffers types,
• Implement the LambdaBuffers Prelude test suite that provides guarantees that the new backend works as intended,
• Implement LambdaBuffers Plutus runtime library for Javascript which includes the standardized LambdaBuffers PlutusData encodings for both opaque and transparent LambdaBuffers types,
• Implement the LambdaBuffers Plutus test suite that provides guarantees that the new backend works as intended,
• Implement Nix wrappers for the Javascript backend,
• Document LambdaBuffers usage patterns for Javascript.

3rd month - Aiken support

• Research the Aiken internals (Blueprints and modular encoding providers) and decide on MVP integration paths with LambdaBuffers,
• Implement the Aiken code generation module (type definition and implementation printing),
• Implement the LambdaBuffers Plutus test suite that provides guarantees that the new backend works as intended,
• Implement Nix wrappers for the Aiken backend,
• Document LambdaBuffers usage patterns for Aiken.

[CAPABILITY/ FEASIBILITY] Please describe the deliverables, outputs and intended outcomes of each milestone.

1) Rust support

• LambdaBuffers Prelude package support which includes the code-generation module, runtime library, test suite and documentation,
• LambdaBuffers Plutus package support which includes the code-generation module, runtime library, test suite and documentation,
• All provided at https://github.com/mlabs-haskell/lambda-buffers

2) Javascript support

• LambdaBuffers Prelude package support which includes the code-generation module, runtime library, test suite and documentation,
• LambdaBuffers Plutus package support which includes the code-generation module, runtime library, test suite and documentation,
• All provided at https://github.com/mlabs-haskell/lambda-buffers

3) Aiken support

• LambdaBuffers Plutus package support which includes the code-generation module, runtime library, test suite and documentation,
• All are provided at https://github.com/mlabs-haskell/lambda-buffers.

[RESOURCES & VALUE FOR MONEY] Please provide a detailed budget breakdown of the proposed work and resources.

Subtotal: 576 hours @95/hour = $54,720 USD

Total (@ rate $0.21 USD / ADA): 260571

**In the interest of full transparency, please note we have applied a conservative USD/ADA exchange rate in pricing this proposal. This is to ensure our operations remain stable regardless of market conditions. Although we firmly believe the future of Cardano is bright, we recognize the price of ADA and all cryptocurrencies is inherently volatile. Our financial obligations are denominated in fiat. Most importantly, this includes the salary of our engineers whose hard work makes projects like this possible. 

In the unlikely scenario of severe negative price movement beyond our forecasted rate, it is possible that MLabs may need to temporarily suspend work on this proposal until the market recovers. Rest assured, this decision would be made solely to protect our business's long-term viability and never taken lightly. 

We appreciate your understanding and support, and we are excited to see what we can achieve together.

[RESOURCES & VALUE FOR MONEY] How does the cost of the project represent value for money for the Cardano ecosystem?

In a survey ran by the LambdaBuffers team and made available at https://github.com/mlabs-haskell/lambda-buffers/blob/main/docs/feedback/questionnaire-results.pdf it was determined that 10-20% of a DApp project time is spent on managing PlutusData encodings of DApp types. The goal of this project is to eliminate this cost in its entirety.

[IMPORTANT NOTE] The Applicant agreed to Fund10 rules and also that data in the Submission Form and other data provided by the project team during the course of the project will be publicly available.