CHARLI3: Live API with all Cardano DEX aggregated data

[GENERAL] Name and surname of main applicant

Robert Hever

[GENERAL] Are you delivering this project as an individual or as an entity (whether formally incorporated or not)

Entity (Incorporated)

[GENERAL] Please specify how many months you expect your project to last (from 2-12 months)

3

[GENERAL] Please indicate if your proposal has been auto-translated into English from another language

No

[GENERAL] Summarize your solution to the problem (200-character limit including spaces)

1. Use Charli3 Dendrite to extract on-chain pool depth, pricing, and trading data from all Cardano DEXs, every block.
2. Create a database with aggregated processed data
3. Build an API endpoint integration

[GENERAL] Does your project have any dependencies on other organizations, technical or otherwise?

Yes

[GENERAL] If YES, please describe what the dependency is and why you believe it is essential for your project’s delivery. If NO, please write “No dependencies.”

Charli3 is partnering with the founder of Steelswap to deliver this project.

We previously delivered project 1100093 together.

Both teams are using in-house resources, unless we determine that gaps exist (e.g. front-end developer) during development.

[GENERAL] Will your project’s output/s be fully open source?

No

[GENERAL] Please provide here more information on the open source status of your project outputs

The only part of the solution that is open-sourced is Charli3 Dendrite. We will build a data extraction solution using Dendrite as the foundation.

[METADATA] Horizons

Oracles

[SOLUTION] Please describe your proposed solution

Context of the problem

The problem is that APIs are available from independent DEXs, but that data does not account for data from most or all DEXs on Cardano. The majority of the liquidation events on Cardano are due to using unreliable and poor quality data sources. As we enter the bull market, these poor quality sources will get exposed. If you are going to use an API endpoint, you might as well get the highest quality one available. Charli3 Oracles leverages its alert and monitoring systems, robust infrastructure, and innovative open-source tooling to provide the most comprehensive Cardano on-chain data in an easily consumable format. Moreover, the broader crypto community receives data through poor quality sources that do not provide accurate, reliable, or secure data. A major DEX, Coinbase for example, displays WMTx as of Oct 16 2024 using Coingecko and Coinmarketcap. These pull from only a few sources and consume outdated unsupported APIs. We believe this project addresses the need for accurate reliable secure on-chain CNT data in an easibly consumable and cost effective standard.

TL;DR

An API endpoint to consume

• Aggregated live (last block) CNT pricing data
• Aggregated live (last block) TVL in ADA or $USD
• Trading volumes over different periods
• Daily
• Hourly
• Custom
• Aggregated historical data over different periods
• Custom time period

Summary

We will build a tool to extract all on-chain Cardano DEX activity (pool data, trading data, and pricing data). This tool will leverage our open-sourced solution, Charli3 Dendrite. The raw data from the chain will be processed in a database and turned into consumable format/categories including, but not limited to, the following:

• Aggregated live (last block) CNT pricing data
• Aggregated live (last block) TVL in ADA or $USD
• Trading volumes over different periods
• Daily
• Hourly
• Custom
• Aggregated historical data over different periods
• Custom time period

We will build an API endpoint to consume the data in the database along with documentation (YAML) to use it. After years of monitoring the Charli3 Oracle service, we have 4 alert and monitoring systems to catch issues before they impact reported data. We will leverage one of those monitoring systems to ensure the API service enjoys the same 99.999% uptimes as our oracle service.

Lastly, we will build a front-end to acquire an API key and promote the service to major solutions outside of Cardano.

Steps to deliver the solution:

1. Data Extraction from the chain
2. Database MVP and API endpoint
3. Database with full processed data suite
4. Documentation and Front-end
5. Monitoring and Live service

Milestone 1: Data Extraction from the chain (completed 10%)

• Architecting solution
• Configuring Charli3 Dendrite, testing and debugging

Milestone 2: Build MVP Database and API (completed 25%)

• Build Database and API endpoint
• Develop aggregation methodology and implement database processing

Milestone 3: Building a comprehensive database of processed data (Completed 75%)

• Database development to create categories for API endpoint
• Hire front-end developer

Milestone 4: Create Landing Page and Documentation (Completed 95%)

• Build landing page
• Create documentation
• Testing and debugging

Final Milestone: Publish live landing page and deploy live service + Close out video and report completed (Completed 100%)

• Final deployment and testing

[IMPACT] Please define the positive impact your project will have on the wider Cardano community

MEASURING SUCCESS

We can easily track usage of the API and promote usage by the Cardano community as well as those outside the community. High usage is a sign of a positive impact for Cardano.

IMPACTS of project on Cardano Community

Lack of accurate up-to-date information on Cardano chain activity

Cardano has been battling a reputation problem with the wider crypto world for many years. This problem includes things such as being called a "ghost chain" and hordes of inaccurate posts about the on-chain volume and amount of projects and tokens available.

This Data suite solution aims to add to the battle against this misinformation and FUD by showing all of the data on Cardano for CNTs and showing off the speed and accuracy of the product.

IMPACT: We hope this leads to less misinformation about Cardano.

Major platforms outside Cardano using poor quality sources (e.g. Coinbase uses coingecko for WMT)

A major issue for Cardano's visibility is the lack of data about the cardano ecosystem in the wider crypto world. We aim to bring a full suite of constantly up-to-date data using similar methods to our oracle operations to ensure proper pricing data with security against hurtful and outlying values. This data would then be partnered with leading trading and price comparison platforms for accurate and enforceable data on Cardano.

The largest platforms of all don't even show Cardano data at all! They would benefit from this data to give ease of access to Cardano data for their mass of users, increasing the visibility of the Cardano ecosystem and economy.

• Trading view
• Coinbase
• Dexscreener
• Binance
• Etoro
• Metatrader
• and many many more.

These are typically provided by oracles in the ecosystem, notably Chain.link and Pyth. We will leverage our reputation as a trusted provider of data (note enterprise levels of uptimes) to approach these platforms with our API.

IMPACT: Major platforms reporting accurate Cardano CNT information when they do get listed

Replacing poor quality API data sources for Cardano Dapps

Non secure price reporting of Cardano assets on platforms such as

• Coingecko
• Coinmarketcap
• Coinbase

have triggered mass liquidations in the Cardano ecosystem with projects using these sources and a cheap and insecure way to gain data for their platform.

To combat this, we are creating this free to low cost data suite with aggregation methods used in our oracle operations that improve security and accurate price and data visibility for the users of these outside platforms.

The benefits of Live aggregation of all DEXs...

• CNT prices
• Liquidity pools
• Trading volumes
• Historical pool data
• Trading activity

... is the removal of outlier pricing data protects against:

• Liquidity removal
• Flash crashes
• DEX reporting errors
• Data source operational errors

IMPACT: Cardano Dapps are safer for the end-user

Wallets using different quality sources

It has been observed that many on-chain applications have been impacted by inaccurate and unreliable data sourcing. Specifically, Cardano wallets have received numerous complaints from the community about the prices of assets varying between wallets and not being consistent. This has led to users making misinformed trading decisions with their assets.

IMPACT: Wallets have a high quality source of truth through a low cost, fast, and monitored enterprise solution.

Demand for such a solution

Our team has already received expressions of interest for this product from a few wallets and other applications in both private and public conversations.

IMPACT: Immediate adoption and usage in Cardano

[CAPABILITY & FEASIBILITY] What is your capability to deliver your project with high levels of trust and accountability? How do you intend to validate if your approach is feasible?

Best Fit

Charli3 has a singular mission of delivering accurate, reliable, and secure data on-chain. We've built a robust network to supply on-chain data, an aggregator to consume on-chain data, and 3 years of experience monitoring data systems to ensure uptimes of 99.99%+. Moreover, this project is a natural extension of Charli3 Dendrite, an open-sourced on-chain DEX aggregator solution.

Charli3 Team and History of Delivering Projects

The Charli3 team and partners have 10 funded proposals. 3 have successfully completed. 5 are in-progress and scheduled to complete by November 2024.

The key foundations for this project are already complete from Fund11: Project 1100093.

We are beginning 2 other projects this quarter.

We have an in-house development team that started in 2021. Our Oracle solution with push-based architecture has been live since 2022, the first Decentralized Oracle Solution on Cardano. You can view the feeds at portal.charli3.io.

We are currently working Anastasia Labs, Metalamp, Txpipe. We've worked with other labs such as Plank and successfully passed a Certik level 2 audit (the first on Cardano to pass one).

Our team has stayed committed through a difficult bear market subsidizing the majority of our data feeds to enable projects to build using them.

Our current offerings includes the following active services: push-based oracle data feeds, pull-based oracle data feeds, Proof of Reserves (USDM).

[PROJECT MILESTONES] What are the key milestones you need to achieve in order to complete your project successfully?

Milestone 1: Data Extraction from the chain

>> Order book (trading activity) and liquidity pool data updated per block

A: Milestone outputs

• Configure data extraction from all supported DEXs using Charli3 Dendrite

B: Acceptance criteria

• All supported Cardano DEX's Order book (trading activity) and Liquidity pool data updated once per block using Charli3 Dendrite

C: Evidence of milestone completion

• Demo video of extraction of raw data from all DEXs

Milestone 2: Build MVP Database and API 

>> Build and populate a database with raw data from the Dendrite solution, process aggregate current CNT price data (last block) that is available through an API endpoint

A: Milestone outputs

• Create database and integrate with Charli3 Dendrite configuration from milestone 1
• Process data in the database to provide aggregate current (last block) CNT price data
• Create API endpoint to call and receive data

B: Acceptance criteria

• Able to call database using API endpoint and receive desired aggregate current CNT price data

C: Evidence of milestone completion

• Demo video of working API endpoint showing AC

Milestone 3: Building a comprehensive database of processed data

A: Milestone outputs

• Enhance the database by processing data into the following categories for consumption via the API endpoint:
• Trading volumes
• Daily 
• Hourly
• Other
• Historical pool data
• Periods TBD
• TVL of pools in ADA or $USD

B: Acceptance criteria

• Able to call database using API endpoint and receive trading volume data, historical pool data, and aggregate current (last block) TVL data

C: Evidence of milestone completion

• Demo video of working API endpoint showing AC

Milestone 4: Create Landing Page and Documentation

A: Milestone outputs

• Creating YAML / Documentation
• Creating basic landing page to find documentation and get API key

B: Acceptance criteria

• Documentation completed
• Landing page completed
• End-user can obtain API key

C: Evidence of milestone completion

• Share link to landing page

n/a

Final Milestone: Publish live landing page and deploy live service + Close out video and report completed

A: Milestone outputs

• Deploy landing page on Charli3 accessible to public
• Deploy service on main-net ready for usage
• Complete close out video
• Complete close out report

B: Acceptance criteria

• End-user is able to goto the landing page, get an API key, and start using the full functionality of the service
• Close out report completed
• Close out video completed

C: Evidence of milestone completion

• Link to live page
• Close out report accepted
• Close out video accepted

[RESOURCES] Who is in the project team and what are their roles?

Project leads - Business side:

Damon Zwarich

Robert Hever

Technical lead:

Founder of Steelswap.io (https://x.com/ElderM)

Development team:

Talha Hussain

Deep Bhatt

Jose Seraquive

[BUDGET & COSTS] Please provide a cost breakdown of the proposed work and resources

Milestone 1: Data Extraction from the chain (completed 10%)

• Architecting solution (Technical Lead - 20 hours x USD$150hr) $3000
• Configuring Charli3 Dendrite, testing and debugging (1 developers - 3 weeks x USD$100hr) $12000

Estimate: USD$15,000

Milestone 2: Build MVP Database and API (completed 25%)

• Build Database and API endpoint (1 back-end developers - 3 weeks x USD$100hr) $12000
• Develop aggregation methodology and implement database processing (1 developers 3 weeks x USD$100hr) $12000
• Hosting costs during build (USD$750)

Estimate: USD$24,750

Milestone 3: Building a comprehensive database of processed data (Completed 75%)

• Database development to create categories for API endpoint (1 back end developer - 6 weeks x USD$100hr) $24000
• Hosting costs during build (USD$750)
• Hire front-end developer (USD$500)

Estimate: USD$25,250

Milestone 4: Create Landing Page and Documentation (Completed 95%)

• Build landing page (1 front-end developer - 2 weeks x set rate USD$2500) $2500
• Create documentation (1 developer - 10 hours at USD$75hr) $750
• Testing and debugging (1 developers - 2 weeks x USD$80hr) $6400
• Hosting costs during build (USD$750) $750

Estimate: USD$10,400

Final Milestone: Publish live landing page and deploy live service + Close out video and report completed (Completed 100%)

• Hosting costs during build (USD$750) $750
• Final deployment and testing (1 developer - 40 hours x USD$100hr) $4000

Estimated: USD$4,750

Estimated Total Cost: USD$80,150 | 235,000 ADA Oct 9th 2024 conversion - Adding 5% for conservative estimate to proposal request

[VALUE FOR MONEY] How does the cost of the project represent value for money for the Cardano ecosystem?

• The cost of platforms on Cardano using insecure data for their operations has already been in the $100s of thousands of dollars. This is a free to low cost alternative to increase security.

• This product also will help users make easier and accurately informed decisions when trading through wallets and other platforms that adopt it.

• Reducing the spread of FUD and misinformation about Cardano is immeasurable for the ecosystem.

• Providing a simple API endpoint with a powerful suite of on-chain Cardano data, updated live to major web2 financial platforms (e.g. trading view) will hopefully bring more users to Cardano, bring more liquidity on-chain, and promote the chain to the wider crypto audience

There are alternative APIs available already arguably of significantly lower quality. The Charli3 team is obsessed with accuracy, reliability, and security -- this is our business. Most of the solutions out there have an API that has partial information (e.g. from a single DEX) or aggregate from other APIs (e.g. coingecko), this API will be comprehensive and use as many DEXs as possible. Typically, these APIs are provided by the decentralized oracles in the ecosystem for this obsession with accuracy, reliability, and security -- but also because Oracles are in the business of delivering data (monitoring data streams 24/7 and living and dying by uptime and accuracy).

In addition, the Steelswap team are also experts on aggregating of on-chain data. The partnership between us and the value we can drive together is already evident by Charli3 Dendrite: the open sourced DEX aggregator already in use.

For those reasons, the project budget is a small cost to provide an impactful solution to Cardano and the wider crypto market. Thank you for your support of Charli3 and Steelswap.