Cardano Native Token Dex Price Indexer in Kotlin

[GENERAL] Name and surname of main applicant

[GENERAL] Are you delivering this project as an individual or as an entity (whether formally incorporated or not)

[GENERAL] Requested funds in ada

[GENERAL] Please specify how many months you expect your project to last (from 2-12 months)

[GENERAL] Please indicate if your proposal has been auto-translated into English from another language

[GENERAL] Summarize your solution to the problem (200-character limit including spaces)

A Kotlin based open-source historical and latest price indexer for Cardano Native Tokens. This will be the open sourcing of the indexer used by https://realfi.info 

[GENERAL] Does your project have any dependencies on other organizations, technical or otherwise?

[GENERAL] If YES, please describe what the dependency is and why you believe it is essential for your project’s delivery. If NO, please write “No dependencies.”

The following dependencies are expected to be persistent for this system. These do not include any dependencies of components that are modular and replaceable within the system:

Cardano Foundation / Bloxbean - Technical dependency on the open-source Java Yaci Cardano mini-protocols and Cardano-client libraries.

Intersect Members Based Organisation (MBO) - Technical dependency on the open-source Cardano Node software.

[GENERAL] Will your project’s output/s be fully open source?

[GENERAL] Please provide here more information on the open source status of your project outputs

License. This project will be released under the permissive MIT License. This license is chosen for its permissive nature allowing others to use, modify and distribute the code without restriction. We are committed to full transparency and community collaboration thoughout the projects lifecycle. 

Project Repository. The entire project including the core indexer and the API webserver component will remain available at the following GitHub repository: https://github.com/Edgxtech/prise 

[METADATA] SDG rating

9 - Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation

[SOLUTION] Please describe your proposed solution

Overview. This proposal is to develop a Decentralised Exchange (DEX) price indexer using the productive Kotlin language that is accessible and useful for a broad range of developers. The indexer will utilise the Ourobourous mini-protocols to sync the blockchain, compute trades and corresponding prices. This solves the problem of not having a Kotlin price indexer for bootstrapping new applications, analytics or reporting of Cardano related statistics. This also solves a problem of not having enough accessible examples of how to extract insights from the Cardano blockchain. The software, or an extension, will also be running with Realfi.info, providing a data API with a free tier and that is accessible by data scientists and traders without requiring them to operate the software themselves. The tool has been running since Jul 2022.

Key Characteristics. The indexer will have modular components allowing some options for external datasources (other than the blockchain data itself) such as token metadata providers and extensions for new or upgraded DEX smart contracts. The choice of Kotlin means modifications and extensions will be relatively simple and accessible by a wide developer base. The indexer will be operable and modifiable by moderately skilled IT operators or software developers.

Key differentiators. There are various general purpose Cardano indexers including DbSync and Carp. Various ecosystem tools or Decentralised Applications (dApps) have also developed bespoke solutions. There are also some examples of database indexer frameworks, however currently there are no examples of an open-source Kotlin based historical price indexer that can perform the functions as described.

Functional Description. The high-level functional requirements of the system are as follows;

• Generate and persist historical candles (open, high, low, close, volume)
• continually generate for multiple resolutions; weekly, daily, hourly, 15mins 
• continually generate for all Cardano Native Tokens traded on Cardano
• continually generate from on-chain transactions data of a configured set of Cardano DEX’s
• Generate and persist latest prices 
• continually generate for a configured set of Cardano DEX's
• Monitoring and maintenance
• expose key metrics compatible with prometheus
• log key events to local filesystem 
• Provide query API
• enable query of historical and latest prices

Constraints. A summary of key constraints (must-do's) of the system are as follows;

• Minimise latency of data generation so far as possible within the software design
• Availability; the software shall have continual uptime
• Use a mainstream and supported database for easy deployment and maintenance 

Roadmap. This effort is part of a broader open-sourcing agenda leading to more comprehensive portfolio reporting and Decentralised Application (dApp) user funds transparency & accountability, in the following order;

CNT Price Indexer - This Proposal

-> Realfi.info (Backend) - Tentative next Project

-> Realfi.info (Frontend) - Tentative final project

[IMPACT] Please define the positive impact your project will have on the wider Cardano community

Contribution of know how.

• This project will add open-source know how for generating historical and latest price data for relatively complicated extended Unspent Transaction Output (eUTXO) blockchains 
• This project will add open-source know how for using the various Java Virtual Machine (JVM) libraries with Kotlin for the Cardano blockchain, including Java Cardano Client Lib and Yaci 

Reporting and Statistics.

• This project will be a minor contribution towards the effort for third party analysis firms to better represent Cardano in reporting and comparisons with other blockchains 

Enhanced public price data.

• Open sourcing the product will allow external subject matter experts to contribute enhancements or extensions, leading to better capabilities including to the hosted free data api at realfi.info 

Reduce complexity, effort & costs for new projects.

• Applications requiring price data may have a direct head start by avoiding bespoke development, or an indirect head start by accessing the know how 

Inclusive Accountability.

• The longer-term objective of open-sourcing the entire realfi.info tool, with the price indexer as the first stage, will provide, in addition to Decentralised Applications (dApp) themselves directly, the knowledge (in code) for how to account for users funds

[CAPABILITY & FEASIBILITY] What is your capability to deliver your project with high levels of trust and accountability? How do you intend to validate if your approach is feasible?

Capability.

• The GitHub repository with a starting (fully functional) indexer is currently live demonstrating the software architecture and design patterns to be used for the final product, this proves the system can be delivered by our team
• We are committed to the Cardano ecosystem demonstrated through Stakepool operations since the beginning of the Shelley Era and through the development and operation of the Realfi.info tool for ~2 years as a self-funded community contribution
• Realfi.info has had multi-wallet native tokens integrated since Jan 2022, its first Liquidity pools since Jan 2022, staking since Mar 2022, lending since Jul 2022. These integrations are 99% achieved via blockchain data rather than from dApp provided apis and partnerships. These contributions demonstrate a deep understanding of the Cardano technical infrastructure

Validating Feasibility 

• Acceptance Testing: Internal testing is factored into the milestones and make up of the team to ensure the system meets requirements
• Operational Testing: Internal testing is factored into the milestones and make up of the team to ensure the system meets user needs in a representative operating environment
• Community Engagement: Regular interaction will be via GitHub and other channels to ensure the system aligns with user needs

[PROJECT MILESTONES] What are the key milestones you need to achieve in order to complete your project successfully?

Milestone 1: Proof of Concept progress release 

A: Expected Duration: 2 months, Max Duration: 4 months 

B: Activities

• Design, development and testing in development and representative environments to confirm ability to function as intended

C: Milestone Outputs 

• Release of a system containing the foundational software architecture and providing the core functional requirement of generating and persisting price data 
• At least 1x (one) generic ChainDatabaseService implementation module
• At least 1x (one) TokenMetadataService implementation module
• At least 3x (three) DEX classifier(s)

D: Acceptance Criteria 

• Demonstration of capability to meet functional requirements, remote or as agreed at time of acceptance

E: Evidence of Milestone Completion 

• GitHub release representing proof of concept, containing software code and documentation as applicable

Milestone 2: Initial release 

A: Expected Duration: 4 months, Max Duration: 6 months 

B: Activities

• Design, development and testing in development and representative environments to confirm ability to function as intended

C: Milestone Outputs 

• Release of a system capable of performing the defined functional requirements
• At least 2x (one) generic ChainDatabaseService implementation module(s)
• At least 1x (one) TokenMetadataService implementation module
• At least 4x (four) DEX classifier(s)

D: Acceptance Criteria 

• Demonstration of capability to meet functional requirements, remote or as agreed at time of acceptance

E: Evidence of Milestone Completion 

• GitHub release representing the initial release containing software code and documentation as applicable

Milestone 3: Final Release, project enters continual improvement cycle 

A: Expected Duration: 2 months, Max Duration: 4 months 

B: Activities

• Testing in a representative environment to characterise performance and confirm ability to meet user needs

C: Milestone Outputs 

• Software capable of suitable performance for the intended operating environment and capable of being operated by a moderately skilled IT operator
• Refinement and improvements as necessary

D: Acceptance Criteria 

• Demonstration of capability to meet user needs in a representative operating environment, remote or as agree at time of acceptance

E: Evidence of Milestone Completion 

• GitHub release representing the final release containing software code and documentation as applicable
• Test & Evaluation reporting as applicable

[RESOURCES] Who is in the project team and what are their roles?

We are a small team is comprised of: 

Tim Edge | Engineer 

Twitter | LinkedIn | Development Portfolio

Experienced Systems and Software Engineer primarily within the Information and Communications Technology industry with 15+ years' experience. Tim is responsible for development and maintenance. 

Edgar ‘Jobe’ Gasper | Test & Evaluation

Twitter | LinkedIn

Experienced Information and Communications Technology practitioner with 17+ years experience. Jobe provides test & evaluation support and design guidance. 

We are active members of the Cardano community, regularly engaging in discussions to progress the Cardano ecosystem, including as co-hosts of the Malaysian Cardano Summit.

[BUDGET & COSTS] Please provide a cost breakdown of the proposed work and resources

Reduced rates are proposed to represent a very good value for money proposition. A flat rate is used: $500/day for principal team members. 

Milestones: 

• 1: Proof of concept release:

Software Engineer: 20 days * $500 = $10,000  

Test & Evaluation: 5 days * $500 = $2,500 

• 2: Initial release: 

Software Engineer: 25 days * $500 = $12,500 

Test & Evaluation: 5 days * $500 = $2,500 

• 3: Final release:

Software Engineer: 10 days * $500 = $5,000 

Test & Evaluation: 5 days * $500 = $2,500 

The total comes to $35,000 (~= 75,000 ADA at time of writing) and will cover costs for one lead developer and one testing/operations support. Tasks may be delegated to other developers to meet the budget and schedule.

[VALUE FOR MONEY] How does the cost of the project represent value for money for the Cardano ecosystem?

Development & Testing Costs. The cost estimate is based on reduced software and Information Communications Technology industry standard employment rates. This proposal reflects the skills and expertise of the development team, and is justified by the clarity of system requirements and progress on the product to date.

Value for Money.  

• Enhancing Developer Accessibility: Lowering the barrier to entry for new developers and streamlining the development process for experienced software developers 
• Long-Term Community Support: Offering ongoing support as custodians of the code repository
• Supplementation. We are supplementing operating costs through running a Stakepool