Add datalog support and power to cardano developers

[Proposal setup] Proposal title

Please provide your proposal title

Add datalog support and power to cardano developers

[Proposal Summary] Budget Information

Enter the amount of funding you are requesting in ADA

100000

[Proposal Summary] Time

Please specify how many months you expect your project to last

10

[Proposal Summary] Translation Information

Please indicate if your proposal has been auto-translated

No

Original Language

en

[Proposal Summary] Problem Statement

What is the problem you want to solve?

There is no Cardano-specific tool providing for datalog/logic-like queries for the blockchain data that is easy to use in the apps that is production ready and interoperable with existing components.

[Proposal Summary] Project Dependencies

Does your project have any dependencies on other organizations, technical or otherwise?

No

Describe any dependencies or write 'No dependencies'

No dependencies

[Proposal Summary] Project Open Source

Will your project's outputs be fully open source?

Yes

License and Additional Information

All source-code developed for this proposal will be released under Apache 2.0 license.

[Theme Selection] Theme

Please choose the most relevant theme and tag related to the outcomes of your proposal

Developer Tools

[Campaign Category] Category Questions

Mention your open source license and describe your open source license rationale.

Apache 2.0 license is very permissive as it puts minimal restrictions on how the software can be used, modified, and distributed, also in commercial setting. Thanks to that it is contributing to the widespread use of the software. It is also very stable and sustainable solution that is encouraging future collaboration.

How do you make sure your source code is accessible to the public from project start, and people are informed?

Project will be open source and public from day 1, in github, under cardano-logica name. As we aim to make it interoperable with several key components (like oura, dolos, etc) we will reach to the respective teams to make them informed about our progress and usefulness. We plan to use discord and also communication via github extensively.

How will you provide high quality documentation?

Proper tutorials and dcumentation is key for project traction and we put it as a top priority. We plan to make roadmap, all development communication, tutorials, also in video form, available as soon as possible. Documentation will be easy to use, actionable, hands on and we understand that superb quality is crucial for the project success.

[Your Project and Solution] Solution

Please describe your proposed solution and how it addresses the problem

We need to lower the entry-barrier for projects that want to use rule-based querying of data in Cardano. And also build app/services logic using logic-programming paradigm. By providing a ready to be plug-in backend, we can bring down the development cost for apps that want to experiment and utilize this approach. The cost reduction will be significant as the main effort for developers will be the logic they are after, the rest pieces, ie., backend will be available. Moreover, we plan to add logic snippets being the part of the library that could be easily used and compose complex logic block by block. By providing a ready-to-be plugged-in backend for rule based querying of Cardano data, we estimate this solution could reduce development costs by circa 80-90% utilizing this approach.

This reduction would come from the following:

1. Eliminating the need to build custom data retrieval and transformational layers and simplifying integration with existing data systems through a standardized interface (saving 2-3 dev months)

2. Reducing the learning curve for understanding and working with blockchain data structures (saving 1 month onboarding time)

The cost reduction is more particularly significant for small - medium sized dev teams where specialised blockchain expertise may be limited.

In particular we are going to deliver the following:

• add logic-query component that is going to be able to attach to available data sinks, first postgre sql, then the data node (dolos) and finally as oura component, and execute datalog queries
• add compelling examples and authoritative 10 tutorials involving utxo accounting, smart contracts, key hierarchies, etc. to lower substantially the entry into the logic paradigm of retrieving and processing information. We aim to create a logic library containing those concepts to help developers build real-world products rather than reinventing the wheel for the n-th time.
• prepare the logic-query component to work in various deployments, as a standalone binary, docker component. We will also provide easy to use development environments to have a try with this new component in various setups through proper nix flakes

The project will be implemented in Rust.

[Your Project and Solution] Impact

Please define the positive impact your project will have on the wider Cardano community

The project will open a new way of handling blockchain data, ie. using logic programming paradigm. As this approach has its merits for some use cases we believe, based on our research and long time observation of the cardano ecosystem, it is going to lower the barrier for app/service developers and businesses to embrace that avenue. And hence enhance adoption. As a consequence, we believe at the end of the first year of the project development there will be dapps/services powered with cardano-logica that produce txs that fill-in blocks.

We plan not only to provide easily pluggable component to the existing ecosystem tools providing blockchain data, but also prepare a number of working examples how this approach could be used (like show how nontrivial cohort analysis can be realized using this approach or validation of user introduced invariants could increase the confidence in the dedicated functionalities the user architects). Meaning educational goal.

Datalog powered querying allows handling use cases that deal with many-to-many relations between data in a more expressive way. As rules can be composed with each other one can express nontrivial relation between data that is very difficult to embrace using the relational approach that is prevalent now. Moreover, we plan to extract useful building block rules ready to be used and to be composed that we will identify in this domain as generic. We hope not only to provide an immediately-to-be-used solution that fits in different tech setups currently used, but also show by example how to use it practically. We are confident that this is going to be a very useful addition amid developer tools that will be immediately available and developer friendly.

We believe that any new component added to the existing ecosystem should work seamlessly with other components and should be zero-cost (or at least with not significant effort) integrable for the developer already having its own setup.

[Your Project and Solution] Capabilities & Feasibility

What is your capability to deliver your project with high levels of trust and accountability? How do you intend to validate if your approach is feasible?

Pawel Jakubas, PhD, is an active member of the Cardano ecosystem, contributing since July 2018 in wallet backend, cardano-addresses to name a few. Before that he worked in academia (solid state simulations using HPC), Samsung R&D and data analysis/ML startups. Pawel works currently for Cardano Foundation in HAL team. Cardano repos that Pawel contributed:

https://github.com/IntersectMBO/cardano-addresses

https://github.com/cardano-foundation/cardano-wallet

https://github.com/txpipe/vrf

https://github.com/txpipe/kes

https://github.com/cardano-foundation/antithesis

Oksana Volnianska, PhD, is a senior researcher specializing in condensed matter physics theory with extensive data analysis/data querying experience. She worked in a big data analysis startup as a data scientist. Drawing on this extensive experience she is eager to pioneer the application of Prolog's logic programming paradigm to blockchain systems.

Robert Zakrzewski is a mathematician that switched into software engineering. He has 20+ years of experience in many diverse projects (from banking to automotive) as a software engineer and architect. He is proficient in many technology stacks, Rust, Go, C++, to name a few. Recently he started to contribute to various blockchain projects and one of his applications won one of the blockchain hackathons (https://github.com/zedar/aleph-notifications)..)

Experience developing in the Cardano ecosystem

Pawel, with nearly 7 years of direct Cardano ecosystem development experience, has been a cornerstone contributor to critical Cardano Infrastructure. As a core developer of cardano-wallet and key contributor to cardano-addresses, he has deep expertise in the fundamental components that power Cardano's user-facing applications. His extensive work spans both IOHK and the Cardano Foundation where he has been specialising in wallet backend systems, REST APIs, and sophisticated cryptographic implementations including key derivation schemes, multisignature wallets, VRF, KES and zero-knowledge proof technologies. This experience allows us to evaluate the feasibility of the project, our ability to deliver it and its potential benefit from a developer's perspective.

Successful numerous projects

We have successfully completed many projects working separately in the highest calibre places (Samsung R&D, Pekao BP bank, IOG, …). We also worked together on a big data analytics project in 2016. We are fully committed to delivering high quality, enterprise grade software products, with minimum maintenance costs, with the user put at the front. This may serve as evidence that our team has the required capabilities to fulfill these types of projects. Pawel is cardano veteran working full time in the ecosystem since mid 2018, Oksana and Robert are both very experience technologist that will give a fresh look, rigor and make sure that the final version of cardano-logica will be easy to pick up and use for newcomers to the cardano ecosystem.

Development process will be public and open-source

Both the output and the development process will be public and open-source. This approach provides an easy way for the Catalyst team and the Cardano community to evaluate the progress at each step of the process. We also promise to be highly responsive in github space (through discussions/issues) and discord.

[Milestones] Project Milestones

Milestone Title

Provide prototype datalog/logic programming layer on top of postgre sql database.

Milestone Outputs

1. At this stage we plan to have a separate component that is able to talk with the database that has blocks stored.
2. First easy queries can be realized for the blocks
3. CLI for the component is provided and the user can (i) connect to a given store (ii) inject datalog file to invoke the query on ra blockchain's blocks

Acceptance Criteria

1. The component in the form of binary is able to connect to the database that has blocks stored, we have this tested, documented and anyone can checks that
2. A number of datalog queries can be instantiated for the blocks, they are tested and documented.
3. CLI is added for the component and the user can realize the functions specified in the milestone

Evidence of Completion

One can easily, invoke datalog powered component and get query results from blocks. The documented way fits with the reality and the user can call datalog queries on raw blockchain's blocks.

Delivery Month

3

Cost

30000

Progress

30 %

Milestone Title

The component is made industry ready in deployment sense for postrgre sql.

Milestone Outputs

1. Light binary for the component available

2. Docker image for the component is ready

3. nix flake for development is available

4. Tests for CLI and datalog queries added

5. more sophisticated family of queries are demonstrated

6. proper CI is added in github

Acceptance Criteria

1. Light binary works, is documented and usable
2. Docker image works, is documented and usable
3. nix flake for development works
4. Tests suite contains tests for CLI and datalog queries
5. First datalog query for Cardano tutorial is created
6. CI is triggered every PR push and when branch is merged into main

Evidence of Completion

One can easily invoke datalog powered component and get data from postgre sql blockchain database. The documented way fits with the reality and the user can call connected postgre database using datalog schema.

Delivery Month

4

Cost

40000

Progress

70 %

Milestone Title

Integration with Cardano data node

Milestone Outputs

1. cardano-logica is able to connect to the dolos data node and together realize datalog queries
2. cardano-logica is consumer in oura and works for any data node being a producer in oura
3. Tutorials and easy to use and understand examples for the point 1 and 2 are provided
4. The most reusable examples gathered until now will be added to the library for the easy reuse
5. Provide roadmap for the next development phase (datalog in smart contracts, integration with other components)

Acceptance Criteria

1. cardano-logica is seamlessly interoperable with dolos using docker, binary and nix
2. cardano-logica is seamlessly interoperable with oura using docker, binary and nix
3. Useful and easily digestable content is provided
4. There is library with primitives that people can use as import and use simple rules and compose into complex ones and it is easy to realize sophisticated validation, transaction logic, etc.
5. Roadmap is published in github

Evidence of Completion

One can create pipeline using dolos and oura and cardano-logica within a while and realize datalog queries. There is library with valuable primtives and simple rules and a clear instruction how to import it and use it to build complex business application using the pipeline that is easily available using binaries, docker and nix flake.

Delivery Month

3

Cost

30000

Progress

100 %

[Final Pitch] Budget & Costs

Please provide a cost breakdown of the proposed work and resources

The team will work continuously for 10 months on the project. Each week everyone is expected to spend 15 hours per week on the project, meaning 60 hours per month. It is 180 hours worked per month, for which we attribute the cost of 10k Ada. 10 months give 100k ada.

The cost breakdown per milestone is presented in Milestones section above.

[Final Pitch] Value for Money

How does the cost of the project represent value for the Cardano ecosystem?

There is no Cardano-specific tool providing for datalog/logic-like queries for the blockchain data that is easy to use in the apps that is production ready and interoperable with existing components. Having that is bound to open new avenues for developers building complex logic, struggling with sophisticated validation. We are going to make sure the component is industry grade, well documented, and easily to use within existent ecosystem, both in the development and deployment sense.

[Required Acknowledgements] Consent & Confirmation

Terms and Conditions:

Yes