Sundae Labs Hydra Transaction Stream Plugin

[GENERAL] Name and Surname of Main Applicant

[GENERAL] Email address of Main Applicant

Additional Applicants

Pi Lanningham @ Sundae Labs

Artem Wright @ Sundae Labs

[GENERAL] Please specify how many months you expect your project to last (from 2-12 months)

[GENERAL] Please indicate if your proposal has been auto-translated into English from another language.

[GENERAL] Does your project have any dependencies on other organizations, technical or otherwise?

[GENERAL] If YES, please describe what the dependency is and why you believe it is essential for your project’s delivery. If NO, please write “No dependencies.” in this field.

The Hydra team will need to approve and merge the pull request for the upgrade, and they have already voiced an interest in working with us on this project.

[GENERAL] Will your project outputs be fully Open Source?

[GENERAL] If NO, please describe which outputs are not going to be open source. If YES, please write “Project will be fully open source.” in this field.

Project will be fully open source.

[METADATA] Category of Proposal

[IMPACT] Please describe your proposed solution.

This solution addresses the need for a customizable and flexible mechanism to log, analyze, and react to the event and transaction flow in a Hydra node. Without such a mechanism, developers face difficulties in integrating with a hydra node; Some events aren’t exposed at all, while others require expensive network hops, new serialization formats, and more.

We propose working with the Hydra team to externalize these events, writing them to disk or an arbitrary socket. A small plugin at the other end of that socket could then integrate this stream of events into any number of existing formats or tools.

For example, Kupo, Carp, or Oura could easily read from this socket and implement the same indexing functions they provide for the Cardano layer 1. Sundae Labs plans to use this to implement a “transaction archive” file format, used in a layer 2 scalability solution called “Gummiworm” that we’re building on top of Hydra.

[IMPACT] How does your proposed solution address the challenge and what benefits will this bring to the Cardano ecosystem?

The described solution offers a flexible and efficient model that allows for event introspection within the Hydra node. This provides a customizable approach to logging and analyzing transaction flows, which addresses the existing challenge of system integration and data management.

Within the context of the Cardano ecosystem, this brings several benefits:

1. Increased Interoperability: By providing a pluggable event stream architecture for each hydra head, the solution can integrate more smoothly with various existing data pipeline tool
2. Customizability: Developers can build their own modules tailored to their specific needs, promoting flexibility, efficiency, and versatility. 
3. Improved Usability: The plugin model can make the Hydra node more user-friendly, as it allows developers to work in languages and formats they’re more familiar with, and observe a wider spectra of events from the Hydra node. 
4. Efficient Data Management: Integrating directly with a binary stream of events can alllow the developer to choose their own tradeoffs between efficiency and accessibility or readability; for example, some tools don’t want to pay the overhead for a websocket / JSON based AP

[IMPACT] How do you intend to measure the success of your project?

Adoption Rate: This is a straightforward metric that measures the number of developers or entities that integrate with the hydra node using this capability. A high adoption rate would suggest the solution is addressing a real need within the ecosystem.

Integration Efficiency: Measuring the time and resources it takes to integrate the Hydra node transaction streams compared to the existing APIs. A decrease in time or resources needed indicates success.

Developer Feedback: Surveys or interviews with developers can provide qualitative data about how the new feature is perceived and its impact on their workflow. Positive feedback indicates the solution is beneficial.

System Performance: Monitor the overall performance of the system compared to streaming transactions over an HTTP API. Improvements in system efficiency, reduced error rates, or increased transaction throughput can indicate success.

[IMPACT] Please describe your plans to share the outputs and results of your project?

1. A detailed Architectural Decision Record (ADR) in accordance with the Hydra project standards.
2. Project Blog Posts and Updates: Regularly post updates on the project's progress on popular platforms like Medium, the project website’s own blog, and YouTube channel. These posts can detail recent accomplishments, hurdles overcome, and next steps.
3. Release Notes: With each significant release or update, publish detailed release notes outlining the changes, improvements, or bug fixes. These can be included directly in the project repository or on the project's website.
4. Webinars and Online Tutorials: Conduct webinars or create online tutorials to demonstrate the functionality of the ledger-mode fork of the Hydra Node. This will help users understand its features and how to use them.
5. Community Forum Discussions: Use platforms like the Cardano community forum or Reddit to engage with the community. Answer questions, ask for feedback, and participate in discussions about the project.
6. Presentations at Conferences or Meetups: Present the project's results at blockchain or Cardano specific conferences and meetups. This not only reaches a larger audience but also provides an opportunity to receive direct feedback from peers and users.
7. Social Media Announcements: Leverage Twitter, LinkedIn or other popular social media platforms to share project milestones and engage with the community.
8. Documentation: Finally, create robust documentation that thoroughly explains how to use the new ledger-mode fork, its benefits, and potential use-cases. This documentation should be easily accessible from the project's website and repository.

[CAPABILITY/ FEASIBILITY] What is your capability to deliver your project with high levels of trust and accountability?

Sundae Labs is a proven and trusted contributor to the Cardano Ecosystem. We built the first AMM DEX on Cardano, and currently supply turnkey Rewards and Governance products/services to other high-profile projects in the Cardano ecosystem. Additionally, we actively support various ad-hoc activities like Cardano security and scalability R&D. We also contribute actively to ongoing IOG initiatives like Voltaire and other core tech topics of discussion. Learn more at https://sundae.fi.

Specific to this proposal, Sundae Labs was the first team to demonstrate an end-to-end demo of Hydra in a dApp, and were also the first to ever run smart contracts on Hydra (before the Hydra team themselves!). We’ve worked closely with the Hydra team in the past.

[CAPABILITY/ FEASIBILITY] What are the main goals for the project and how will you validate if your approach is feasible?

The main goals of the project would be:

1. Development of a Plugin Model: The main objective of this project is to develop a pluggable event-stream architecture within the Hydra node. 
2. Integration with Existing Data Pipelines: A critical goal of the project is to ensure the plugin model is compatible with existing data pipeline tools. This will facilitate smoother integration of transaction streams from the Hydra node into these pipelines, boosting system interoperability and usability.
3. Community Adoption and Usage: One of the project's objectives is to increase adoption and usage of the new pluggable "archive" model among the developer community within the Cardano ecosystem. Encouraging utilization of this new model will foster a more robust, versatile, and efficient ecosystem.

Validation of the approach's feasibility could be conducted through:

1. Technical Feasibility Assessment: A technical review will be performed, in cooperation with the Hydra team, to evaluate whether the proposed solution is possible within the existing technical constraints of the Hydra node. This will involve prototyping and testing the event stream model within a controlled environment to assess its functionality and compatibility with the Hydra node, and result in an Architectural Decision Record (ADR).
2. User Acceptance Testing: User acceptance testing with a select group of developers can provide insights into whether the proposed solution meets the needs of the intended users. This can include feedback on the model's usability, the ease of integration with existing systems, and its effectiveness in improving data management and analysis.
3. Monitoring Early Adoption: Following the launch of the solution, monitoring the rate of early adoption among the developer community can provide a good indication of whether the solution is meeting its intended purpose and adding value to the ecosystem. Higher rates of adoption could suggest the solution is effectively addressing the identified problem.
4. Ongoing Feedback and Iteration: Continual gathering of user feedback post-launch, and making necessary adjustments based on this feedback, can provide further validation of the approach's feasibility and effectiveness. This feedback can also help identify any unforeseen issues or opportunities for further enhancements.

[CAPABILITY/ FEASIBILITY] Please provide a detailed breakdown of your project’s milestones and each of the main tasks or activities to reach the milestone plus the expected timeline for the delivery.

Milestone 1: Project Planning and Design (2 weeks)

• Define the project's scope and objectives.
• Coordinate with core Hydra team on a project plan and ADR.
• Design the architecture of the event stream.

Milestone 2: Development of Plugin Model (4 weeks)

• Develop the core functionalities of the event stream model.
• As a real world use case, we’ll use this model to produce regular on-disk “archives” of transactions for offline analysis as part of the Gummiworm protocol.

Milestone 3: Integration with an existing tool (2 weeks)

• Work with at least one developer of an existing indexer tool, such as Oura from TxPipe, to develop a plugin that can integrate Hydra into the indexer tool. If you are the maintainer of one of these tools and would like to work with us, please reach out! 

Milestone 4: User Acceptance Testing (UAT) and Feedback Collection (2 weeks)

• Select a group of developers for user acceptance testing.
• Conduct UAT and gather feedback on usability, integration ease, and effectiveness.
• Make necessary adjustments based on UAT results and feedback.

Milestone 5: Launch and Early Adoption Monitoring (2 weeks)

• Launch the event stream model to the broader community.
• Monitor early adoption rates among developers and the hydra team.
• Collect and analyze feedback from early adopters.

[CAPABILITY/ FEASIBILITY] Please describe the deliverables, outputs and intended outcomes of each milestone.

Milestone 1: Project Planning and Design

• Deliverables: Project Plan, Design Document
• Outputs: Clear project timeline, roles & responsibilities, and architecture for the event stream model.
• Intended Outcome: The team has a clear understanding of the project's direction, goals, and design considerations.

Milestone 2: Development of Event Stream Model

• Deliverables: Developed Event Stream Model, Test Reports
• Outputs: Functioning event stream model ready for testing.
• Intended Outcome: A stable version of the event stream model that meets the requirements and expectations defined in the project plan.

Milestone 3: Integration with Existing Data Pipelines

• Deliverables: Integration Test Reports, 
• Outputs: An integrated system that ensures a smooth data flow between the Hydra node and at least one real world indexing tool.
• Intended Outcome: A robust and reliable plugin model that fits seamlessly within existing data pipeline infrastructures.

Milestone 4: User Acceptance Testing (UAT) and Feedback Collection

• Deliverables: UAT Reports, Collected Feedback
• Outputs: Comprehensive user feedback and identified issues or improvements.
• Intended Outcome: A validated plugin model that meets users' needs and expectations.

Milestone 5: Launch and Early Adoption Monitoring

• Deliverables: Launch Event, Early Adoption Report
• Outputs: Successful launch, initial user engagement, and gathered usage data.
• Intended Outcome: Effective dissemination of the model in the broader community and positive initial user engagement.

[RESOURCES & VALUE FOR MONEY] Please provide a detailed budget breakdown of the proposed work and resources.

Total budget: $172.5k USD / 588,000 ADA / 3 months

Engineers (x2): $25k/mo

Project Management: $5k/mo

Administrative: $2.5k/mo

[RESOURCES & VALUE FOR MONEY] How does the cost of the project represent value for money for the Cardano ecosystem?

This project, with its emphasis on creating an external event stream model for the Hydra node, represents significant value for money for the Cardano ecosystem in several ways:

1. Improved Ecosystem Functionality: By providing a flexible and customizable mechanism for reacting to events within the Hydra node, this project enables robust off-chain infrastructure for Hydra projects. This leads to a better developer experience for projects building with Hydra.
2. Increased Interoperability: The integration of transaction streams from the Hydra node into existing data pipeline ecosystems significantly improves system interoperability, and reduces developer friction when adding or adopting Hydra. This improvement makes it easier for developers to work within the Hydra ecosystem.
3. Enhanced Developer Experience: With an external event stream, developers can more easily understand, analyze, and utilize transaction data, and do so from the technology stacks that they’re familiar with. This improves the developer experience and potentially attracts more developers to the Hydra ecosystem, fostering innovation and growth, and broader scalability for Cardano.
4. Potential for New Use Cases: By making all hydra events more accessible and manageable, this project may enable new use cases within the Hydra ecosystem, such as advanced analytics, compliance tracking, or other data-driven applications. This could bring more users, developers, and businesses to the Cardano network.
5. Long-term Sustainability: The project is not a one-off solution. By integrating directly into the hydra node event stream, future changes and improvements to the Hydra node will feed into this pipeline, amplifying its benefit over time.
6. Open source innovation: This project will serve as a high-quality example for how to contribute to the Hydra protocol, building out a legacy of true open source and community innovation.

In summary, the cost of implementing this solution is justified by the multi-dimensional value it adds to the Cardano ecosystem, enhancing its functionality, usability, and future growth potential.

[IMPORTANT NOTE] The Applicant agreed to Fund10 rules and also that data in the Submission Form and other data provided by the project team during the course of the project will be publicly available.