Amaru: P2P networking component by TxPipe

[GENERAL] Name and surname of main applicant

Santiago Carmuega

[GENERAL] Are you delivering this project as an individual or as an entity (whether formally incorporated or not)

Entity (Incorporated)

[GENERAL] Please specify how many months you expect your project to last (from 2-12 months)

4

[GENERAL] Please indicate if your proposal has been auto-translated into English from another language

No

[GENERAL] Summarize your solution to the problem (200-character limit including spaces)

We'll contribute a new Rust library that implements a peer-to-peer networking compatible with Cardano to be used by Amaru and any other Rust-based application that requires it.

[GENERAL] Does your project have any dependencies on other organizations, technical or otherwise?

No

[GENERAL] If YES, please describe what the dependency is and why you believe it is essential for your project’s delivery. If NO, please write “No dependencies.”

No dependencies.

[GENERAL] Will your project’s output/s be fully open source?

Yes

[GENERAL] Please provide here more information on the open source status of your project outputs

The source code will be released under the Apache 2.0 license and it will welcome external contributions.

[SOLUTION] Please describe your proposed solution

To implement a P2P networking layer compatible with Cardano, we'll focus on the following individual components:

Peer Discovery

A mechanism to establish which peers are participating in the network without relying on a static topology. In our scope, peer discovery includes:

• Ledger Relays: our component will be able to discover peers by inspecting the ledger state and accessing relay information available in SPO registration certificates.
• Peer-sharing: our component will be able to request known peers from already connected nodes using the peer-sharing mini-protocol.
• Bootstrap Topology: our component will accept a bootstrapping configuration that specifies an initial set of known peers to use as seed for crawling new peers using peer-sharing.

Outbound Routing

A component that manages the outbound traffic to peers using different heuristics with the goal of optimizing overall network efficiency. In our scope, the routing heuristics will include:

• Health Checks: our component will monitor outbound connections at the TCP level and at the application level using the Ouroboros keep-alive mini-protocol.
• Random Churn: to avoid getting stuck on a local maxima of efficiency, our component will introduce a configurable level of chaos by eagerly switching to random new peers from the list of all available.
• Latency Score: our component will keep a ranking of peers in terms of network performance by computing a rolling average of the chain-sync latency of each. This dynamic ranking will be used as score to prioritize connections.

Inbound Monitor

A component that is in charge of monitoring inbound connections with the goal of optimizing efficiency and to mitigate abuses / attacks.

• Max connections: our component will keep track of the total amount of connections and restrict any new connections after a configurable max threshold has been reached.
• Rate-limiting / Throttling: our component will keep track of data throughput metrics for each peer and throttle the communication using a known algorithm such as "leaky bucket".
• Inactivity Churn: our component will measure inactivity span for each peer and disconnect peers that exceed a configurable max threshold.

The approach we're planning to use for the implementation of these components consists on taking the Haskell node as inspiration for the overall mechanics but without doing a strict reverse engineering of its current architecture. We believe that copying the exact implementation might not provide the benefits we're looking for in terms of node diversity.

It's important to note that many P2P interactions in Cardano are not part of a specification and, thus, not mandatory.

[IMPACT] Please define the positive impact your project will have on the wider Cardano community

As a general concern and an indirect, long-term effect of this proposal, node diversity provides several benefits to the ecosystem:

• Resilience against vulnerability and bugs. In a single-implementation ecosystem, any zero-day vulnerability or severe bug could be “catastrophic” to the whole network.
• Resilience against organizational issues. In an ecosystem where the development of the node is handled by a single team, any organizational issues (developer rotation, shifts in priority, budget limitation, etc) can have drastic consequences on the evolution of the protocol.
• Foster different and novel approaches to existing problems. Independent teams or different tech-stacks will yield different ways of solving the same problems, potentially encountering better ways to solve already implemented concerns.

On a more direct relationship, this proposal has short-term benefits too:

• A P2P module improves the resilience of Amaru (and any other integrator of the library) against network partitions and adversarial conditions.
• A P2P module will potentially improve the overall performance of the network by optimizing block and transaction propagation.

[CAPABILITY & FEASIBILITY] What is your capability to deliver your project with high levels of trust and accountability? How do you intend to validate if your approach is feasible?

TxPipe is an active member of the Cardano ecosystem

TxPipe has been developing open-source tools for the Cardano ecosystem for over 2 years and we're not going anywhere. Evidence of our commitment can be found by evaluating the continuous activity of our public code repositories.

Experience developing in the Cardano ecosystem

TxPipe has helped developed several dApps for the Cardano ecosystem. This experience allows us to evaluate the feasibility of the project and its potential benefit from a developer's perspective.

Successful Catalyst proposals

We have successfully completed several Catalyst proposals. This may serve as evidence that our team has the required capabilities to fulfill these type of projects.

Development process will be public and open-source

Both the output and the development process will be public and open-source. This approach provides an easy way for the Catalyst team and the Cardano community to evaluate the progress at each step of the process.

[PROJECT MILESTONES] What are the key milestones you need to achieve in order to complete your project successfully?

Peer Discovery

Milestone outputs

• publicly available source-code of a peer-discovery module in Rust
• a published library that can be used as a dependency in Rust applications
• publicly available documentation of the module's interface
• report showing benchmarks and integration testing results

Acceptance criteria

• the newly created module provides a way to discover network peers from on-chain relay registration.
• the newly created module provides a way to discover network peers from bootstrapping peers using the peer-sharing mini-protocol.

Evidence of milestone completion

• Source-code available in the Github repository
• Rust crate published at crates.io
• Documentation published at docs.rs
• Testing reports available in the Github repository

Outbound Routing

Milestone outputs

• publicly available source-code of an outbound routing module in Rust
• a published library that can be used as a dependency in Rust applications
• publicly available documentation of the module's interface
• report showing benchmarks and integration testing results

Acceptance criteria

• the newly created component manages outbound traffic to peers using different heuristics to optimize overall traffic.

Evidence of milestone completion

• Source-code available in the Github repository
• Rust crate published at crates.io
• Documentation published at docs.rs
• Testing reports available in the Github repository

Inbound Monitor

Milestone outputs

• publicly available source-code of an inbound monitor module in Rust
• a published library that can be used as a dependency in Rust applications
• publicly available documentation of the module's interface
• report showing benchmarks and integration testing results

Acceptance criteria

• the newly created component can monitor inbound connections and enforce rate-limiting / throttling.

Evidence of milestone completion

• Source-code available in the Github repository
• Rust crate published at crates.io
• Documentation published at docs.rs
• Testing reports available in the Github repository

Final Report

Milestone outputs

• publicly available source-code for a complete p2p module in Rust
• a published library that can be used as a dependency in Rust applications
• publicly available documentation of the module's interface
• a video showcasing the final output of the project
• a final report describing the project process and outcome

Acceptance criteria

• the final module provides a working P2P networking layer compatible with Cardano
• the final video complies with Catalyst requirements
• the final report complies with Catalyst requirements

Evidence of milestone completion

• Source-code available in the Github repository
• Rust crate published at crates.io
• Documentation published at docs.rs
• Testing reports available in the Github repository
• Final video uploaded to Google drive
• Final report uploaded to Google drive

[RESOURCES] Who is in the project team and what are their roles?

• Santiago Carmuega - TxPipe : Rust Developer /Tech Lead - Github
• Federico Weill - TxPipe : Project Manager - Linkedin
• Alejandro Avagnina - TxPipe : Site Reliability Engineer - Github
• Paulo Bressan - TxPipe: Rust Developer- Github
• Felipe Gonzalez - TxPipe: Rust Developer - Github
• Alexander Falcucci - TxPipe : Rust Developer - Github

[BUDGET & COSTS] Please provide a cost breakdown of the proposed work and resources

FTE = Full-time equivalent

Total budget = ₳ 148,125

Software development

• Rust developer: 2 FTE x 3 months = ₳ 135,000
• Technical Writer: 1/2 FTE x 1 months = ₳ 3,750

Project Management

• Project Manager: 1/4 FTE x 5 months = ₳ 9,375

[VALUE FOR MONEY] How does the cost of the project represent value for money for the Cardano ecosystem?

The bulk of the budget falls under the software development category. TxPipe has extensive experience in the field, allowing it to provide good value for money. The hourly rates are defined using fair market prices. The estimation for the level of effort takes into account all of the optimizations that our team is capable of providing after years of experience developing software solutions in the Cardano ecosystem.