Cloud Native Blockchain Archive Node by Blink Labs

[GENERAL] Name and surname of main applicant

Blink Labs

[GENERAL] Are you delivering this project as an individual or as an entity (whether formally incorporated or not)

Entity (Incorporated)

[GENERAL] Please specify how many months you expect your project to last (from 2-12 months)

5

[GENERAL] Please indicate if your proposal has been auto-translated into English from another language

No

[GENERAL] Summarize your solution to the problem (200-character limit including spaces)

Create a Cardano blockchain node which communicates with the Cardano network to archive and index block data for future retrieval using cloud native object storage.

[GENERAL] Does your project have any dependencies on other organizations, technical or otherwise?

No

[GENERAL] If YES, please describe what the dependency is and why you believe it is essential for your project’s delivery. If NO, please write “No dependencies.”

No dependencies.

[GENERAL] Will your project’s output/s be fully open source?

Yes

[GENERAL] Please provide here more information on the open source status of your project outputs

The project is and will continue to be open sourced under the Apache 2.0 license.

[METADATA] Horizons

Infrastructure

[SOLUTION] Please describe your proposed solution

With the advent of cloud computing, a new class of applications has emerged. These applications are optimized for the elastic nature of the cloud and utilize web standards to produce rich applications as cloud native technologies. Cloud native technologies continue to gain traction, with more than half of organizations running most of their workloads in the cloud. As the historical size of the Cardano blockchain grows, organizations will be seeking ways to integrate Cardano data into their existing infrastructure, data pipelines, and business intelligence workflows, by leveraging cloud native object storage, such as Amazon’s S3 or Google’s Cloud Storage. They will also be looking for ways to utilize this data source to support elastic infrastructure.

Archive nodes are common in other blockchains. An archive node provides a complete historical record of every block and transaction state change for a blockchain and is optimized for these types of requests. As the size of the Cardano blockchain grows, local storage of the entire chain will become a growing issue of cost and capacity. Creating an archive node allows creation of other nodes which can prune immutable block data from their own storage when paired with an archive node.

Blink Labs will build capabilities into its Go Cardano node to support an archive mode of operation which will allow it to both archive data and utilize archive data from a cloud native object store for servicing requests. Archive node usage will be built into the Go Cardano node as a supported out-of-box feature, allowing the Go Cardano node to be used in a mode which prunes the archive data when used in conjunction with an archive node.

[IMPACT] Please define the positive impact your project will have on the wider Cardano community

Building cloud native technologies with Cardano support opens up the user base to organizations around the globe. With the growth of cloud computing across all verticals of business, building the capability for organizations to be early adopters of Cardano blockchain technology could give them a competitive edge in their industry. Catering to the business needs of enterprise applications by providing data services integrated into their existing infrastructure and business practices will expand the potential customer base of Cardano by unlocking new use cases, such as cloud native applications, machine learning, or artificial intelligence.

Storing Cardano blockchain data in cloud object storage makes it available to countless software applications designed to leverage this information storage class as input.

[CAPABILITY & FEASIBILITY] What is your capability to deliver your project with high levels of trust and accountability? How do you intend to validate if your approach is feasible?

Blink Labs has decades of experience delivering open source software. Our team also has experience building global scale data applications and infrastructure for companies like Yahoo and Box. We have a proven track record of delivery and have successfully completed two previous Catalyst Fund 9 proposals. We have two open funded proposals from Fund 11, have met our milestones and are wrapping up closeout videos now; we are just starting milestone 1 objectives for our Fund 12 proposal which is slated to complete before Fund 13 onboarding completes. This proposal builds on those previous proposals and our Go node proposal from Fund 12. Our work is open source and public for all to view. Our team is highly cohesive and have worked together for years, allowing us to provide accurate time and cost estimates to our work. Funding allows us to dedicate engineering time to an open source project, rapidly accelerating its development.

Blink Labs will create GitHub issues for the individual items of work and will track those items using a project Kanban board on GitHub.

[PROJECT MILESTONES] What are the key milestones you need to achieve in order to complete your project successfully?

Milestone 1 will decouple the storage implementation in the Go node from the underlying storage systems, creating an interface which can be used by multiple backends.

• Create relational database interface
• Create key-value storage interface
• Migrate sqlite implementation to new relational interface
• Migrate badger implementation to new key-value interface

Milestone 2 will implement storage for Google Cloud Storage for data storage and retrieval in the archive node mode.

• Implement relational interface over Google Cloud Storage
• Implement key-value interface over Google Cloud Storage
• Implement optional KMS (Key Management Service) based encryption

Milestone 3 will implement storage for Amazon AWS S3 for data storage and retrieval in the archive node mode.

• Implement relational interface over AWS S3
• Implement key-value interface over AWS S3
• Implement optional KMS (Key Management Service) based encryption

Milestone 4 will implement data pruning and transparent data proxying from a Go node in pruning mode to an archive node.

• Develop transparent data proxying for immutable block data
• Implement data pruning for immutable data

Our 5th and final milestone will wrap up the project, including the Project closeout report and video. The project closeout report will include a comprehensive summary of project objectives and evidence of goals achieved. The project closeout video will provide a visual overview of the project's journey, start to finish, highlight key achievements, and showcase the net product: Cloud Native Blockchain Archive Node.

[RESOURCES] Who is in the project team and what are their roles?

The project development will be completed by the Blink Labs engineering team. Our team includes Principal / Staff Engineers, with more than 50 years combined experience. This project will be primarily assigned to Chris Gianelloni (https://github.com/wolf31o2) for development completion, with additional Blink Labs developers providing code review, documentation, and support.

Additional Blink Labs staff may be used for administrative functions, such as project management and community reporting.

[BUDGET & COSTS] Please provide a cost breakdown of the proposed work and resources

Funding will primarily pay for engineering time. There is also a small piece set aside for paying administrative staff for project management.

The project budget is 28165 ADA per month to pay for Engineering (26666.4) and Project Management (1500) for the first four milestones. The final milestone will be an additional 6000 ADA for the project manager. Each of the first four milestones corresponds to a month, and time is allocated to take approximately 50 percent of a full time software engineer or 80 hours and approximately 10 hours of project management. The final milestone includes closeout reporting and video production and will take approximately 40 hours of project management time.

No additional funding is necessary for third party products or services.

[VALUE FOR MONEY] How does the cost of the project represent value for money for the Cardano ecosystem?

Costs are estimated on a 333.33 ADA per hour rate for a senior golang blockchain software engineer working remotely within the United States (~ $192k USD annually) and 150 ADA per hour rate for a project manager working remotely (~ $86k USD annually)