Nodes used exclusively as data source for client tools have different requirements than block producers / relays nodes. Many performance / cost optimizations are not currently possible.
We'll develop a new node, fine-tuned to solve a very narrow scope: keeping an updated copy of the ledger and replying to queries from trusted clients, while requiring a small fraction of the resources
This is the total amount allocated to Dolos: Cardano “Data Node”
Cardano nodes can assume one of two roles:
Each of these roles has concrete responsibilities and runtime requirements. Criteria such as network topology, resource allocation, backup procedures, etc vary by role.
We argue that there’s a 3rd role that should be treated independently with the goal of optimizing its workload: nodes that are used with the sole purpose of resolving local state queries or serving as data source for downstream tools that require ledger data.
There are many potential optimizations for nodes performing this type of workload that are not currently possible with the Cardano node:
The goal of this project is to provide a very limited and focused version of the Cardano node that can be used by DevOps as a cost-effective, performant option to deploy data nodes side-by-side with the producer / relay nodes.
This new role would be useful in the following scenarios:
Data nodes will share some of the features with the mainstream Cardano node:
This new type of node will also provide features not currently available in the mainstream Cardano node:
Dolos will be developed as an open-source project using Rust as its main development language. The code will do heavy usage of the Pallas library developed by this team and already available as version 0.11.
Having an efficient way to access ledger data without incurring high infrastructure costs will accelerate development for both solo developers and small to medium sized teams. A lower entry barrier will drive more developers to the ecosystem.
New wire protocols such as HTTP/JSON and gRPC will also widen the spectrum of options for developers to integrate ledger data without the requirement of integrating low-level mini-protocols directly into their projects.
Our experience developing Oura and Pallas allowed us to gain knowledge and implement libraries that will serve as foundational components for this project. Regardless of this advantage, there are some known-unknowns that could present a risk:
Is important to highlight that we consider this project feasible because the complexity of a data node is orders of magnitude lower than the complexity of a full-node. Please note that we are NOT proposing an alternative to the mainstream Cardano node written in Rust, that would be imposible to achieve with our current development bandwidth.
Milestone #1: PoC
Milestone #2: Features
Milestone #3: Hardening / Documentation
Santiago Carmuega will lead the software development effort. He is a senior developer with over 20 years of experience in software development and very active in Cardano Open Source ecosystem leading TxPipe.
Alejandro Drabenche will be the SRE in charge of validating, deploying and testing the project at each milestone. He is a senior System Administrator with over 15 years of experience. He has been working on blockchain for over 5 years.
Florencia Luna will be in charge of the technical writing. She is a junior developer with experience in technical writing.
Federico S. Weill will be the project manager. He is a senior project manager, he has a PhD. in science and he has lead more than 10 research projects during the last 20 years managing resources and people.
We're planning on hiring a new software developer with experience in Rust to contribute to the codebase starting from milestone #2.
If the project achieves a good level of adoption, we intend to return to Catalyst for a v2 of the tool after we gathered enough feedback from real-world usage.
Progress of the project will be measured by released versions matching the scope of the predetermined milestones.
Projects within the Cardano ecosystem use the new Data Node as a way to optimize their infrastructure costs and improve the performance of their data-intensive workloads.
This is an entirely new proposal.
We're TxPipe, a small team of seasoned developers passionate about Open Source and Cardano. It is the team behind existing projects such as Oura & Pallas. We specialize in backend development, data pipelines and cloud architecture. Rust is our language of choice.