Helios is missing an in-language unit-testing feature. The current JS approach requires too much boilerplate and leads to too much context switching.
This is the total amount allocated to Helios unit testing framework with code-coverage statistics.
I propose an OOP-like approach, by creating a new built-in called TestingContext`, which has conventional test-runner methods. The UPLC CEK machine will be extended to calculate code-coverage stats.
No dependencies
These features will be part of the following repository: https://github.com/HyperionBT/helios
This repository is already open-source. The license is BSD-3-Clause
Implement the ability to test Helios functions from within Helios scripts themselves. The CLI can then automatically detect any testing scripts in the workspace/repository, run them, and generate a test report.
An OOP unit-testing framework, which fits nicely into the current Helios language paradigm, will allow us to avoid introducing new syntax that is only intended for unit testing. The test-runner methods will support automatic fuzzy arguments of the functions being tested.
Finally, code-coverage stats will be collected from test runs by using the code mapping that is included in UPLC scripts generated by the Helios compiler.
The lack of in-language unit testing is deterring some teams from using Helios. The current approach of wrapping Helios test scripts in JS test-runners is too cumbersome.
This proposal will alleviate these issues and allow developers to write Helios unit tests in a way that is hopefully on par with mainstream languages.
The Helios language already has many built-in types and macros. A TestingContext object would introduce a few additional macros, for which the core architecture is already there.
The current Helios JS fuzzy tester can be used in the background to generate fuzzy arguments for test runs.
Collecting code-coverage statistics requires collecting code-map sites from UPLC run. This code mapping has already been implemented.
Investigate how unit-testing frameworks work in a variety of mainstream languages. Make a synthesis of this investigation in order to determine the best possible design of TestingContext . Consult members of the Helios discord for their opinions/recommendations/experiences.
Acceptance criteria:
Share a report containing the details and conclusion of this investigation
Collect branch/function scope info upon compiling a Helios program. Collect evaluated code-map sites from a UPLC evaluation. Combines these two pieces of information to generate code-coverage reports.
Acceptance criteria:
Add a tutorial to the Helios book demonstrating how to generate code-coverage reports during a test run or emulator run.
Implement the TestingContext type along with fuzzy testing and other macros. (Macros are Helios-specific UPLC functions that connect to a service/library outside of the UPLC environment).
Make sure the CLI can autodetect all testing scripts in a repository/workspace, run them, and then generate a report.
Acceptance criteria:
Add a tutorial to the Helios book on how to write unit tests and run them.
Christian Schmitz: creator of Helios
https://www.linkedin.com/in/christian-schmitz-ba0a23b9/
I will try to dedicate 0.2 FTE to this. If I can't find the time I will reach out to other Cardano developers via the Helios discord.
Required skills for code-coverage stats implementation and CLI changes: significant experience with JS/TS, experience with the Helios codebase (bonus).
Required skills for the investigative phase: significant experience with unit-testing frameworks.
Milestone 1: 5 FTE days
Milestone 2: 5 FTE days
Milestone 3: 10 FTE days
Total: 20 FTE days => 160 hours @ 8hours/day => 12000 USD @ 75 USD/hour => 32000 ADA @ 0.38USD/ADA
Timeline: though milestones 1 and 2 can run in parallel, there aren't enough FTEs available to do that. 20 FTE days => 100 days @ 0.2 FTE => 5 months
An easy-to-use unit-testing framework and code-coverage stats could bring more teams to use Helios.
Let's assume that Helios will be able to attract 10% more teams annually coming from outside the Cardano ecosystem. At the current rate that is about 0.5 teams per year (not including very small teams and hobbyists).
Assuming that only 1 in 10 teams can deliver a successful dApp on Cardano, and that single successful dApp brings a TVL of 1M USD.
According to this estimate, this feature will roughly be able to add 50k USD TVL to Cardano per year.