Db-Sync Enterprise: high-availability setup of Cardano’s Db-Sync to run queries and access on-chain data as a service

[GENERAL] Name and surname of main applicant

Thomas Kaliakos

[GENERAL] Are you delivering this project as an individual or as an entity (whether formally incorporated or not)

Entity (Not Incorporated)

[GENERAL] Requested funds in ada

60000

[GENERAL] Please specify how many months you expect your project to last (from 2-12 months)

6

[GENERAL] Please indicate if your proposal has been auto-translated into English from another language.

No

[GENERAL] Summarize your solution to the problem (200-character limit including spaces)

We are developing and maintaining a high-availability infrastructure running Db-Sync to minimise downtime. Cardano projects can be hooked up to our Db-Sync Enterprise solution in short time.

[GENERAL] Does your project have any dependencies on other organizations, technical or otherwise?

No

[GENERAL] If YES, please describe what the dependency is and why you believe it is essential for your project’s delivery. If NO, please write “No dependencies.”

No dependencies.

[GENERAL] Will your project’s output/s be fully open source?

Yes

[GENERAL] Please provide here more information on the open source status of your project outputs

Project will be fully open source under Apache license v2.0

[SOLUTION] Please describe your proposed solution.

We will offer on demand all the on chain data in a relational database that can be used by any client. In this way the pain of querying on-chain data, combining realtime and historical data will be mitigated.

[IMPACT] Please define the positive impact your project will have on the wider Cardano community.

We will offer to Cardano’s ecosystem a low barrier entry for querying realtime and historical on-chain data. It will be “Db-Sync as a Service”, pay as you go, and scalable to the user’s project needs.

[CAPABILITY & FEASIBILITY] What is your capability to deliver your project with high levels of trust and accountability? How do you intend to validate if your approach is feasible?

We have more than two years experience with our own developed Db-Sync Enterprise protocol which minimises downtime of a Db-Sync pipeline.

The project team covers all aspects required to achieve the stated goals.

[Project Milestones] What are the key milestones you need to achieve in order to complete your project successfully?

M1: setup redundant and high-availability infrastructure for the project (2 months)

This process entails setting up the servers to run the Cardano node, db-sync, and a Postgres database. The setup includes configuring two fully functional db-sync pipelines, including the mentioned components. The criteria for acceptance will focus on ensuring these pipelines run smoothly, testing failover capabilities.

M2: create monitoring and dashboard view (2 months)

This milestone's outputs will be the monitoring of the whole system. It will involve monitoring and alerting at different levels: underlying servers, running systems, database usage.

Server monitoring:

We will monitor CPU, I/O, disk and memory for every server, setting up relevant alerts for unexpected usage levels,

Running systems:

We will monitor process health and uptime

Database usage:

We will monitor typical database metrics (replica lag, queries) as well as db-sync related metrics (eg. block height lag).

The acceptance criteria will be having 3 dashboards where all the server/process instances will be monitored and test the successful delivery of alerts for all the relevant metrics

M3: automatise user signup (2 months)

This milestone involves setting up a web App, where each user will be able to signup and select what type/how many database instances would like to use. After the successful signup, the relevant resources will be created automatically and the connection details will be shared with the user.

The user will have the option to tear down the spawned instances and pay only for the time they were used.

Acceptance criteria: Perform a successful user signup and use the connection details to connect to the read replica and perform a query successfully. Perform a successful tear down. Validate pay-as-you-go charging.

[RESOURCES] Who is in the project team and what are their roles?

Alexander Diemand (cardanobigquery - https://www.linkedin.com/in/axeld/): architecture & design, project management, communication, documentation

Thomas Kaliakos (thomaska - https://www.linkedin.com/in/tkaliakos/): data engineering, data quality responsibility, documentation

Bitseat Tadesse (bitseatt - https://www.linkedin.com/in/bitseat/): data science, social networks, documentation

[BUDGET & COSTS] Please provide a cost breakdown of the proposed work and resources.

B1: 20 PD architecture & design, project management, communication

B2: 20 PD system engineering (Devops) for pipeline setup

B3: 20 PD software engineering

B4: 20 PD data science: dashboard development

B5: hardware costs: est. $1000 per month (x6 months)

(PD = person day; 8 hrs/day; 1 hr = $90)

Total Budget: $63600

[VALUE FOR MONEY] How does the cost of the project represent value for money for the Cardano ecosystem?

By sharing our redundant high-availability Db-Sync pipeline we can provide users with a sophisticated and reliable Db-Sync connection for a fair price.

We believe this offering enables a number of Cardano projects as it lowers entry barriers.

[IMPORTANT NOTE] The Applicant agrees to Fund Rules and also that data in the submission form and other data provided by the project team during the course of the project will be publicly available.

I Accept