Mafoc Chain Indexer with integrated Ouroboros networking ("integrated cardano-node")

[GENERAL] Name and surname of main applicant

Markus Läll

[GENERAL] Are you delivering this project as an individual or as an entity (whether formally incorporated or not)

Entity (Incorporated)

[GENERAL] Co proposers and Additional applicants

-

[GENERAL] Please specify how many months you expect your project to last (from 2-12 months)

7

[GENERAL] Please indicate if your proposal has been auto-translated into English from another language.

No

[GENERAL] Summarize your solution to the problem (200-character limit including spaces)

Mafoc will integrate Ouroboros networking into the chain indexer and doesn't require a cardano-node to be run on the side.

[GENERAL] Does your project have any dependencies on other organizations, technical or otherwise?

Yes

[GENERAL] If YES, please describe what the dependency is and why you believe it is essential for your project’s delivery. If NO, please write “No dependencies.”

There are only technical dependencies, libraries produced by IOG:

• cardano-ledger
• cardano-api
• ouroboros-consensus
• cardano-node

[GENERAL] Will your project’s output/s be fully open source?

Yes

[GENERAL] Please provide here more information on the open source status of your project outputs

Will be available under Apache-2.0 license as a source repository on github.

[SOLUTION] Please describe your proposed solution.

Integrate cardano-node into Mafoc, using the know-how already present in the current cardano-node, hooking into the correct locations to receive the blocks, and drop unnecessary functionality (tx submission etc).

Other indexers (e.g Cardano DB Sync) require a cardano-node running on the side to consume blocks from. This is an accidental requirement because any indexer actually needs access only to the blocks get whatever data it needs. But the node does a lot more, including keeping a ledger state (full utxo set, and more) which takes up a lot of memory (~13GB as of now). Cardano DB Sync also keeps a ledger state, doubling the memory requirement. This issue is alleviated by integrating cardano-node directly into the indexer.

My project will engage dApp developers who need access to on-chain data and are frustrated in running the heavy-weight Cardano DB Sync as their only option.

Demonstrating the functionality is straightforward: indexing any data off the chain without cardano-node becomes possible.

[IMPACT] Please define the positive impact your project will have on the wider Cardano community.

Ability to run an indexer without cardano-node and Cardano DB Sync together, removing the huge RAM requirement to get any on-chain data.

Impact of the project can be measured by how many of the dApp developers switch from Cardano DB Sync to Mafoc.

The produced library and executable will remain open source for anyone to reuse.

[CAPABILITY & FEASIBILITY] What is your capability to deliver your project with high levels of trust and accountability? How do you intend to validate if your approach is feasible?

Mafoc, a chain indexer is already implemented as a PoC with ~14 prebuilt indexer (https://captainhaskell.ee/mafoc/docs/category/indexers), but currently requires a cardano-node to connect to. Integrating the node is the logical next step and requires a big engineering push to get done.

Mafoc is also constantly improving as can be seen from the github repository: https://github.com/eyeinsky/mafoc/.

[Project Milestones] What are the key milestones you need to achieve in order to complete your project successfully?

Get a dirty integration of cardano-node into Mafoc, such that it "somehow does something without the cardano-node" :)

Refactor out unrelated services, refactor the node itself such that it becomes easy to integrate, both for Mafoc and other clients.

Improve performance by indexing the chain in parallel from multiple other block sources over the network.

Integrate the refactored cardano-node back into IOHK's code base for upstream use.

(what is said in Milestone 3)

[RESOURCES] Who is in the project team and what are their roles?

It's only me :)

[BUDGET & COSTS] Please provide a cost breakdown of the proposed work and resources.

No external services will be bought, all of the funds will go into development time.

[VALUE FOR MONEY] How does the cost of the project represent value for money for the Cardano ecosystem?

IOHK is developing the Marconi chain indexer, which nominally has a similar goal. That team has ~5 people, if they make around 100k$ per year then they have a run rate of 0.5mln$ per year. Marconi has been in development for over two years. Mafoc has already surpassed them in terms of speed and also has extensive tests against Cardano DB Sync.

The cost of integating cardano-node into Mafoc is relatively well costed.

[IMPORTANT NOTE] The Applicant agrees to Fund Rules and also that data in the submission form and other data provided by the project team during the course of the project will be publicly available.

I Accept