XSY - High-performance on-chain standard library for Plutarch and Aiken

[GENERAL] Name and surname of main applicant

[GENERAL] Are you delivering this project as an individual or as an entity (whether formally incorporated or not)

[GENERAL] Please specify how many months you expect your project to last (from 2-12 months)

[GENERAL] Please indicate if your proposal has been auto-translated into English from another language

[GENERAL] Summarize your solution to the problem (200-character limit including spaces)

We propose a production grade performant and secure on-chain standard library for Plutarch, with potential bindings to Aiken. These will enable developers in building efficient smart contracts.

[GENERAL] Does your project have any dependencies on other organizations, technical or otherwise?

[GENERAL] If YES, please describe what the dependency is and why you believe it is essential for your project’s delivery. If NO, please write “No dependencies.”

No dependencies.

[GENERAL] Will your project’s output/s be fully open source?

[GENERAL] Please provide here more information on the open source status of your project outputs

All outputs will be open sourced under the Apache 2 license.

The code will be published to the following repository:

xsy-booster (github.com)

[SOLUTION] Please describe your proposed solution

We propose the development of a high-performance, secure on-chain standard library for Plutarch, aiming to empower developers to create more efficient and robust smart contracts. This initiative seeks to provide access to curated Plutarch libraries and Aiken bindings with improved execution performance and appropriate abstractions, thus significantly enhancing the development experience within the Cardano ecosystem. Our solution focuses on security, performance, accessibility, and comprehensive documentation, setting a foundational layer for production projects.

Core Principles:

• Security and Assurance: Prioritizing safety, our library will undergo extensive testing and security assessments. It's designed to support critical on-chain projects in production environments, ensuring reliability and trustworthiness.

• Performance: Leveraging the expertise of seasoned Cardano developers, we will optimize every aspect of the library to deliver exceptional performance, enabling the creation of fast and cost-effective smart contracts.

• Accessibility: With a commitment to the development community, the library will feature intuitive abstractions and utilities, enhancing ease of use without sacrificing control or efficiency.

• Comprehensive Documentation: To facilitate widespread adoption and effective implementation, we will provide detailed documentation, including guides, API references, and practical examples, illustrating the application of best practices and design patterns.

[IMPACT] Please define the positive impact your project will have on the wider Cardano community

The Cardano ecosystem has seen remarkable growth in tooling and resources, lowering entry barriers for DApp and smart contract development. Despite these advancements, due to computational constraints around on-chain smart contracts, the need for low-level control to optimize performance remains a critical challenge. Plutarch stands out by offering granular control over contract logic. However, lower-level control introduces a trade-off: on one hand, experienced developers are able to craft highly optimized contracts, reducing resource usage and operational costs. On the other hand, the complexity of working with these tools can deter less experienced developers, thus creating barriers to adoption. As a result, many projects develop their own performant libraries, adding to the ecosystem's fragmentation. Our proposed on-chain standard library aims to bridge this gap by offering a robust performance-oriented foundation on top of Plutarch and providing accessible abstractions/utilities to a wider audience. Note that the targeted audience is not limited to Plutarch developers as we will also provide Aiken bindings. However, compiled UPLC code can in theory be imported and used within any on-chain framework. By centralizing and optimizing the library, we reduce redundancy across projects and streamline development processes within the ecosystem.

The subsequent sections outline common performance challenges and new features to be included in the library:

High Assurance

Given the current state of the ecosystem, there is great burden upon DApp developers to ensure the assurance of their smart contracts. However, underlying standard library functions in many cases, are not extensively tested or formally verified. The proposed standard library will be designed with great attention to security and ease of use. The library will be comprehensively tested using property-based testing and formal verification techniques to assert robustness of all library functionalities.

Unrolling recursive calls

Due to the absence of native support for recursion in UPLC, all recursive functions (e.g. pall, pelemAt, etc) rely on the Y-combinator to facilitate recursion. While this method is theoretically robust, it is inefficient in practice as recursive functions are passed repeatedly along the recursion chain. Most of the time, the required number of recursive calls are predictable according to the application context. We therefore propose to unroll recursive calls prior to the UPLC generation. This approach enables a strategic balance between script size (e.g. unrolling of tail calls when traversing a list) and execution unit consumption (i.e., recursion expansion at runtime), potentially yielding substantial performance improvements.

Converting between Scott and Builtin Data

A significant opportunity for performance enhancement lies in the trade-off between using Scott-encoded data versus builtin data structures in smart contract logic. Scott encoding generally offers efficiency advantages when handling complex data structures. However, this efficiency comes with a cost: serializing and de-serializing data into Scott encoding can incur substantial performance penalties. PlutusTx adopts this approach by default, i.e., encoding all builtin structures into Scott encoding. In contrast, tools like Aiken exclusively operate over builtin structures, which might appear to optimize performance by avoiding data structure conversions. However, solely using builtin structures carries drawbacks especially when they are fully traversed or manipulated multiple times. Plutarch provides developers with granular control over these two operating levels, but this can be challenging for those with limited experience in profiling on-chain scripts. To aid developers in making informed choices regarding the appropriate level of abstraction, the standard library will include both utilities and comprehensive documentation.

Early failure

Haskell developers commonly use sum-type data structures such as Maybe and Either to represent function return types that might fail or result in unexpected states. While these patterns are considered idiomatic in Haskell development, they are often viewed as anti-patterns in the development of on-chain smart contracts for two primary reasons:

1. Additional Overheads: These structures add overhead to the existing domain data structures. In a resource-constrained environment, such overheads can significantly increase both script size and memory usage.
2. Failure Propagation: Failures are typically propagated through the execution logic of the contract until they are handled at a higher level. This propagation can occur during critical operations such as searches, look-ups, and contract validation logic. It is thus considered a best practice in on-chain development to terminate execution as early as possible when errors are detected. This approach not only reduces execution unit costs, especially in complex contracts, but also minimizes potential security vulnerabilities that could arise from improper handling of failures.

As an example, consider the `pfind` function below:

This function searches for an element in a list that satisfies a specific predicate f. In practice, pfind might be used to locate an input within a transaction that contains a certain NFT, triggering additional logic. Therefore, it would be more efficient to immediately terminate the execution if the condition is not met. In doing so, this will save on execution units and reduce potential security vulnerabilities that may follow from not adequately addressing the PNothing case.

Also note that the recursive function accepts an argument f, which remains constant throughout the entire recursive execution. In such cases, it would be more efficient to move f to an outer lambda such that it only gets applied once instead of being applied at each recursive call. This is all to illustrate that the current standard library is not sufficient for production use-cases.

[CAPABILITY & FEASIBILITY] What is your capability to deliver your project with high levels of trust and accountability? How do you intend to validate if your approach is feasible?

Our team consists of highly skilled developers with experience developing open-source tooling (Convex) in the ecosystem. Our developers are extremely experienced in all stages of DApp development on Cardano from design and architecture all the way to mainnet releases. They are intimately familiar with the requirements of DApp protocols, and the nuances of smart contract development on Cardano. We are uniquely positioned to deliver this critical infrastructure to the Cardano ecosystem.

Historically a number of promising well intentioned tools and libraries in the ecosystem did not see much traction upon release. Often this can be the result of the tooling being developed in a vacuum without feedback from production use-cases.

In order to make sure that the proposed framework is well-equipped for production use, we will develop and revise it with feedback from a live mainnet protocol.

[PROJECT MILESTONES] What are the key milestones you need to achieve in order to complete your project successfully?

Analysis & Library architecture

• Research existing libraries, patterns and projects
• Analyze shortcomings, common security vulnerabilities and identify bottlenecks
• Set standards for desired benchmarking and performance targets
• Convergence on the design of the library as well as the interface

Library Implementation

• Implementation of the library
• Utilized in parallel as the foundation for an existing DApp on Cardano for regular feedback on performance and developer experience.

Iterative Improvement

1. Security and assurance analysis via various testing and verification techniques
2. Performance analysis and comparisons to show clear improvements
3. Refining interface and design via community feedback reviews.
4. Provide comprehensive documentation and usage examples catered for different developers with varying experience.
5. Thorough documentation of trade-offs between different patterns and contexts in which they are applicable.
6. Comparative analysis and performance profiling of the library.

Suite of compiled UPLC of the library:

• Provide a release of compiled UPLC binaries that can be imported into various frameworks.
• Instructions on importing and using released binaries into various frameworks

Close-out Video and Report

The video will walk through the entire repository. It will demonstrate how developers can import the library and use it in their own projects. The report will detail the development process of the library along with the future roadmap.

[RESOURCES] Who is in the project team and what are their roles?

Jean-Frédéric Etienne has more than 15 years of experience in safety and threat analysis and is an expert in several formal verification techniques. He is currently the architecture and technical lead for the Djed implementation on Cardano and has put in place a property-based testing methodology to extensively assess the correctness and robustness of Plutus smart contracts against all potential attacks. He has also specified and proved the adaptation of the Djed protocol on the EUTxO model and has developed a set of Plutus libraries to produce optimized on-chain code.

Jean-Frédéric will be working on security analysis and testing.

Jann Müller is a Haskell programmer with years of experience in writing scalable, mission-critical systems. He has been working with Plutus since its inception and is the maintainer of the sc-tools library for Cardano apps. He is the lead developer of Djed and will be working on the off-chain parts of the framework. 

Philip DiSarro is an expert in the field of Compiler Development & Programming Language Theory. He has made significant open-source contributions to the Cardano developer ecosystem. As a co-chair of the IOHK developer experience working group he worked to identify and resolve pain points that DApp developers experience in Cardano. He has a vast wealth of professional experience in smart contract security and auditing on Cardano; and was responsible for the identification and resolution of a large number of critical exploits in production open-source smart contracts. Recently, Philip has concentrated his efforts on designing and deploying effective zero-knowledge proof applications within the Cardano ecosystem. Philip is a senior Haskell developer on the XSY team, a consultant and lecturer for Emurgo, and the CEO and co-founder of Anastasia Labs.

Philip will be contributing to the implementation of the efficient library functions for Plutarch.

Amir H. Meyssami Rad is a Haskell developer and a member of the XSY development team with over two years of experience in developing on Cardano. His expertise spans wallet integration tools, off-chain transaction library development, DApp and on-chain development using Plutus across various projects. His focus will be on implementation, testing and performance benchmarking of the standard library.

Konstantinos Lambrou-Latreille is a Haskell programmer with more than 5 years of relevant experience. He worked for 3 years in IOG on Plutus off-chain tooling such as a node emulator for testing Plutus applications, a transaction building library, and a chain-indexer. He will also be working on the off-chain part of the framework.

[BUDGET & COSTS] Please provide a cost breakdown of the proposed work and resources

2 Engineers total:

On-chain Engineer

• Implementation
• 24 weeks
• 100k Ada

Engineer

• Research, documentation, testing and benchmarking
• 24 weeks
• 100k Ada

[VALUE FOR MONEY] How does the cost of the project represent value for money for the Cardano ecosystem?

The absence of a standard library that is sufficiently optimized for production use has resulted in a situation where each product independently develops and maintains their own libraries. This is a huge redundant waste of development time and money. This library aims to resolve that by completely removing the need for developers to reinvent the wheel for every project.