Catalyst & Cardano DReps Evaluation & Reputation Protocol [by TrustLevel]

[GENERAL] Name and surname of main applicant

Dominik Tilman

[GENERAL] Are you delivering this project as an individual or as an entity (whether formally incorporated or not)

Entity (Incorporated)

[GENERAL] Requested funds in ada

245000

[GENERAL] Please specify how many months you expect your project to last (from 2-12 months)

9

[GENERAL] Please indicate if your proposal has been auto-translated into English from another language

No

[GENERAL] Summarize your solution to the problem (200-character limit including spaces)

Develop a transparent evaluation system for DReps that integrates both quantitative metrics and community feedback to enhance accountability in Cardano & Catalyst.

[GENERAL] Does your project have any dependencies on other organizations, technical or otherwise?

No

[GENERAL] If YES, please describe what the dependency is and why you believe it is essential for your project’s delivery. If NO, please write “No dependencies.”

No dependencies.

[GENERAL] Will your project’s output/s be fully open source?

Yes

[GENERAL] Please provide here more information on the open source status of your project outputs

All project outputs, including developed software and documentation, will be fully open source under the Apache 2.0 or MIT License.

[SOLUTION] Please describe your proposed solution

Background:

-> DReps in Catalyst

In Project Catalyst, decentralized representatives (DReps) will be responsible for representing the interests of those who delegate their voting power to them:

• Voting on Proposals: DReps analyze and vote on various project proposals within Catalyst, essentially acting on behalf of community members who may not have the time or expertise to evaluate each proposal.
• Scaling Governance: As Catalyst grows, DReps help manage the volume of proposals and ensure that all submissions receive the attention they deserve, supporting the sustainability of the innovation fund.

-> DReps in Cardano (Including SPOs and the CC)

Within the broader Cardano ecosystem, the governance model proposed under CIP-1694 introduces a structured approach where DReps, Stake Pool Operators (SPOs), and the Constitutional Committee (CC) all have distinct but complementary roles:

• DReps: Similar to their role in Catalyst, DReps in Cardano are tasked with representing the ada holders who delegate their voting power to them. They participate in the governance by voting on broader network decisions, such as Cardano Improvement Proposals (CIPs) and other crucial governance actions.
• Stake Pool Operators (SPOs): SPOs maintain the Cardano network infrastructure by running nodes. In the governance model, they also have the right to submit governance actions and vote on them.
• Constitutional Committee (CC): The CC is a formal body to oversee the constitutionality of governance actions. They review proposals and governance actions to ensure they adhere to the foundational principles laid out in Cardano's "constitution," acting as a safeguard against potential abuses of power.

The need for transparent evaluations in DAO governance:

While the introduction of DReps in Catalyst promises several benefits, it also raises some concerns:

1. Conflict of Interest: DReps hold substantial voting power and could potentially vote on their own projects for personal gain, risking the integrity of the decision-making process.
2. Risk of Corruption: The influential position of dReps could make them targets for bribery and undue influence from project teams.
3. External Influence: The involvement of dReps could attract large investors ("whales") and investment funds, potentially leading to collusion and favoritism in project selections.

These issues highlight the need for reliable governance and transparency measures to prevent abuses and ensure that dReps contribute positively to the Catalyst ecosystem. Our proposal is intended to prevent DReps from exploiting their position unnoticed, and at the same time we want to give those DReps who do continuously valuable work for the ecosystem the opportunity to be independently assessed and rewarded with a high reputation. 

Proposal Objectives:

With the introduction of the proposed evaluation framework for DReps, we want to achieve the following aspects for the community:

• Trust Building: Transparency is foundational to building and maintaining trust within any community. Especially in DAOs, where centralized oversight does not exist, it is crucial that participants trust the processes and the individuals (DREPs) making decisions on their behalf.
• Accountability: When actions and decisions are visible and traceable, DReps are more accountable for their actions.
• Informed Decision Making: Transparency enables the community to make more informed decisions about whom to trust and delegate their voting power to.
• Reducing Corruption and Bias: Transparent evaluations helps to minimize the risk of corrupt practices and biases as every action and its justification are open to scrutiny.
• Community Engagement: When members feel confident that the system is transparent and fair, they are more likely to participate actively in Governance.

Components of our proposed DREP evaluation system:

• Quantitative Metrics: Our framework will incorporates a set of quantitative metrics that objectively measure the performance of DREPs based on their voting behavior and the outcomes of their decisions. These metrics are designed to be transparent and easily understandable, ensuring that all community members can see how each metric is calculated and what it represents:
• Voting Consistency: Measures how consistently DReps vote in line with the community preferences or successful project outcomes.
• Impact Score: Assesses the influence of DReps' votes on the success rate of projects they support, calculated through outcomes like project completion rates and post-implementation reviews.
• Participation Rate: Tracks the frequency and regularity of DReps involvement in voting and other governance activities, highlighting their commitment to their role.
• Qualitative Assessments: To complement the quantitative data, the framework integrates qualitative assessments that provide deeper insights into the DReps' contributions. These assessments are gathered through community feedback mechanisms and peer reviews, adding a layer of subjective evaluation that captures aspects of performance not easily quantified.
• Community Feedback: Allows community members to submit reviews or ratings based on their interactions and experiences with DReps. This feedback can include aspects such as communication quality, responsiveness, and advocacy for community interests.
• Expert Reviews: Engages subject matter experts to assess the strategic impact and foresight of DReps' decisions, providing insights into their understanding and handling of complex governance issues.
• Reputation Scores: Both quantitative metrics and qualitative assessments feed into a dynamic reputation scoring system. This system calculates a composite score for each DRep, reflecting their overall effectiveness and reliability as a representative. The reputation scores are updated regularly to reflect the most current data, ensuring they remain relevant and timely.
• On-chain Integration: To ensure integrity and prevent manipulation, all aspects of the evaluation process, especially the collection of votes and feedback, are recorded on the blockchain.
• Community Involvement and Oversight: Our intension is to design this evaluation framework with strong community involvement. We want to facilitate open forums for feedback on the system, and give the possibility for community-driven modifications to ensure that the evaluation framework remains aligned with the needs and values of the Cardano and Catalyst community.

Technical Approach:

1. Data Collection and Analysis Tool

• Tool to extract and analyze voting data from Cardano's blockchain.
• Data Correlation: We will use algorithms to correlate voting behaviors with project outcomes in Catalyst.

2. Community Interface

• Simple User Dashboard for viewing DReps' metrics and reputation scores.
• Feedback Portal for submitting community feedback.
• Visualization Tools to display data trends and insights.

3. Quantitative and Qualitative Data Processing

• Metrics Engine to calculate quantitative metrics like Voting Consistency and Impact Score.
• Assessment Module to analyze community feedback and expert reviews.

4. Reputation Scoring System

• Scoring Algorithm to compute composite reputation scores from quantitative and qualitative data.
• Regular Updates: Automate data pipelines to keep reputation scores current.

5. On-Chain Integration

• Smart Contracts for secure, transparent recording of votes and feedback.

6. Integration with Cardano and Catalyst:

• Integration with Catalyst Voices and other community portals / applications (like DRepWatch from NMKR).

POC work:

-> This projects will build on top of the successful proposal of Fund 11: Reputation Scores for Catalyst Reviewers

(https://projectcatalyst.io/funds/11/cardano-use-cases-concept/reputation-scores-for-catalyst-proposers-and-reviewers-by-lidonation-and-trustlevel)

[IMPACT] Please define the positive impact your project will have on the wider Cardano community

This DREP evaluation system could help to improve transparency and accountability within both Cardano and Catalyst and to build trust in DREPs and their governance and voting decisions.

To measure the success and impact of the DRep evaluation system, the following three impact metrics can be tracked:

1. Improvement in Funding Outcomes in Catalyst:

• Metric: Success rate of projects or initiatives endorsed by DReps.
• Method: Track and analyze the outcomes of projects or governance actions that DREPs have voted for or supported. Success can be measured through project completion rates, fulfillment of project goals, and impact assessments.
• Goal: Increase the success rate of supported projects, demonstrating that DREPs are effectively identifying and backing initiatives that contribute positively to the ecosystem.

2. Community Satisfaction and Trust in Cardano & Catalyst DREPs:

• Metric: Community satisfaction with DREP performance and trust in their governance decisions and voting decisions.
• Method: Regularly conduct satisfaction surveys to measure trust levels and satisfaction with DREP decisions.
• Goal: Improve community satisfaction and trust levels over time, to show that the evaluation system is effective in promoting better governance.

3. Dynamics of Delegated Voting Power:

• Metric: Changes in the number of delegations, total delegated voting power, fluctuations in delegated voting power to individual DREPs.
• Method: Track and analyze the volume of voting power delegated to DREPs over time. This involves monitoring the number of delegations (how many individual delegators each DREP has), the total amount of voting power delegated (sum of all delegated ADA to DREPs), and the changes in these figures over set periods.
• Goal: Maintain or increase the total delegated voting power and the number of delegations to DREPs over time, while minimizing large swings in delegated power away from any particular DREP. Positive changes in these metrics would suggest that DREPs are perceived as effective and trustworthy.

[CAPABILITY & FEASIBILITY] What is your capability to deliver your project with high levels of trust and accountability? How do you intend to validate if your approach is feasible?

Know-how and Partnerships:

TrustLevel has a proven track record of successful participation in and contribution to the voting and reviewing processes across multiple platforms, including Project Catalyst, Arbitrum, and SingularityNet, over the last few years. Our experience has equipped us with a deep understanding of the nuanced challenges and specific requirements of effective voting and reviewing systems.

We have active collaborations with the following teams in Cardano:

- Lidonation: Reputation-Scores for Catalyst Reviewers (https://projectcatalyst.io/funds/11/cardano-use-cases-concept/reputation-scores-for-catalyst-proposers-and-reviewers-by-lidonation-and-trustlevel)

- Photrek: Development of a community tool for voting calculations and community engagement in SingularityNet (https://proposals.deepfunding.ai/graduated/accepted/ed600af3-885c-45bc-a874-56d2dde371ce)

- SidanLab and MeshJS: Smart Contract Development (https://projectcatalyst.io/funds/11/cardano-open-developers/aiken-open-source-smart-contract-library-by-meshjs-and-trustlevel)

Validation of Feasibility:

• Pilot Testing: We will conduct a series of pilot tests to refine the system, using real-world data and scenarios to ensure that the system performs as intended.
• Feedback Loops: We will implement feedback mechanisms to gather insights from the community and the Catalyst team, which will be used continuously to improve the system.

Community Collaboration: 

There are already a number of projects around DReps being developed by Lidonation, NMKR, Photrek and others. We have active collaborations or are in contact with most of these initiatives and are very interested and open to deeper collaboration to join forces with the community. 

Contributions and Publications:

We have published several articles and blogs about our findings and insights into the voting and reviewing processes:

• Reputation POC work: https://github.com/TrustLevel
• Impact Assessments: https://docs.google.com/document/d/1JEti-4I9l-3LsAvf_JaTRXVsIRgPooffhcGO1zQjzr0/edit?usp=sharing
• Proposal Evaluations in Grant programs: https://medium.com/@dominik_36234/from-chaos-to-clarity-proposal-evaluations-in-web3-grant-programs-22d13222c69e
• Analysis of Decentralized Innovation Funds: https://medium.com/@dominik_36234/searching-for-signals-of-trust-in-an-ocean-of-noise-lessons-from-the-fellowship-part-2-acad30e9ae45
• Reputation Protocols: https://docs.google.com/document/d/1ghLOLB9sUMuyxlQ8GXkSsH10_k0gPN3R90FyD3ZBwm0/edit?usp=sharing

[PROJECT MILESTONES] What are the key milestones you need to achieve in order to complete your project successfully?

We propose a 9-month timeline for implementation of all milestones, assuming a start date of August 1, 2024.

M1: Development of the framework and design of the DRep evaluation system.

• Timeline: Month 1-2
• Tasks:
• Define KPIs for quantitative metrics.
• Outline the qualitative assessment methodologies, including community feedback channels and expert review protocols.
• Design the data collection and analysis infrastructure to ensure all evaluation data is transparent and easily accessible.
• Output and Acceptance Criteria:
• Document outlining the evaluation framework.
• Technical specifications for the data collection and analysis tools.
• Evidence of milestone completion:
• Published documents, specification and code on github repository

M2: Backend Development of the MVP of the evaluation system.

• Timeline: Month 3-4
• Tasks:
• Build the backend infrastructure for data collection and processing.
• Integrate prototype systems with a blockchain testnet to ensure compatibility and security.
• Output and Acceptance Criteria:
• A functional MVP of the backend.
• Evidence of milestone completion:
• Published documents, specification and code on Github repository

M3: UI Development of the MVP of the evaluation system.

• Timeline: Month 5-6
• Tasks:
• Develop the community interface for submitting feedback and viewing DREP scores.
• Output and Acceptance Criteria:
• A functional MVP of the user interface.
• Evidence of milestone completion:
• Published documents, specification and code on Github repository

M4: Testing and integration of the system with the other community platforms.*

• Timeline: Month 7-8
• Tasks:
• Integrate feedback and refine the system based on test results.
• Finalize integration features for tracking and storing evaluation data securely.
• Deliverables and Acceptance Criteria:
• A fully tested and functional MVP of the evaluation system.
• Integration documentation and system deployment on other community platforms.
• Evidence of milestone completion:
• Published documents, specification and code on Github repository

*Other DREP systems are under development at the moment by Lidonation, NMKR and others. We intend to integrate our solution with existing infrastructure of the ecosystem. 

M5: Engage with the community to promote the system and officially close out the project.

• Timeline: Month 9
• Tasks:
• Launch of community workshops and training sessions to educate about the new system.
• Gather and incorporate final feedback from the community to ensure the system is well-received and functional.
• Create documentation and support materials.
• Deliverables:
• Workshops and feedback sessions.
• Final project report.
• Acceptance Criteria:
• Completion of all documentation and support materials.
• Formal project closeout report and video.
• Evidence of milestone completion:
• Published documents, specification and code on Github repository.
• Submitted report and video.

[RESOURCES] Who is in the project team and what are their roles?

TrustLevel was founded by Dominik Tilman with the vision to develop methods and protocols to make the reliability of data and information measurable. Since then, we have received various grants (Cardano, SingularityNet, Arbitrum) and projects that have continuously improved our knowledge and developed tools and enabled us to provide better reviewing and voting processes and systems in decentralised communities. All our outputs are open-source. 

TrustLevel Core Team: 

• Dominik Tilman: Project Lead
• Dominik has been actively engaged since Fund 3 in Project Catalyst and involved in multiple funded projects. All proposals are either successfully delivered or on track. He is founder of TrustLevel.io & Conu21.com and has 15+ years experience in innovation management and company building.
• Links: https://www.linkedin.com/in/dominikstumpp/; www.trustlevel.io; www.conu21.com 
• Roman Preuss: Full-Stack Developer
• https://de.linkedin.com/in/roman-preuss-0b145558
• Josch Rossa: Full-stack developer and LLMs
• https://github.com/josros
• Alex Ramalho: Full Stack AI Developer
• https://alexramalho.dev
• Sergey K.: ML + Blockchain Developer
• https://www.linkedin.com/i

Community contributors:

We want to build the system together with the community, i.e. we will work with bounties and hope to distribute a considerable part of the funds among valuable contributors. We will manage bounties via Dework.

[BUDGET & COSTS] Please provide a cost breakdown of the proposed work and resources

Total Requested Funding: 245,000 ADA

• Milestone 1: Design (Months 1-2)
• Amount: 50,000 ADA
• Estimated Hours: ~300 hours at 150 ADA/hour = 45,000 ADA

+ 5,000 ADA for Licences and Tools.

• Justification: This phase involves conceptual and planning work, including the development of quantitative and qualitative metrics and the overall system design.

• Milestone 2: MVP Backend (Months 3-4)
• Estimated Hours: ~300 hours at 150 ADA/hour = 45,000 ADA
• Justification: This milestone focuses on developing the backend.

• Milestone 3: MVP UI (Months 5-6)
• Estimated Hours: ~300 hours at 150 ADA/hour = 45,000 ADA
• Justification: This milestone focuses on developing the UI.

• Milestone 4: Test and Integration (Months 7-8)
• Estimated Hours: ~400 hours at 150 ADA/hour = 60,000 ADA
• Justification: During this phase, we will test the tool with real world data and will integrate it with other platform in Cardano and Catalyst (e.g Voices for Catalyst Data Ingestion or other community build tools).

• Milestone 5: Community Engagement & Closeout (Months 9)
• Amount: 45,000 ADA
• Justification: The final phase focuses on deploying the system, conducting community engagement, and project closeout. We think community engagement (workshops and also marketing) are a as important as developing the tool itself, because we want to ensure, that the tool is being used at the end.

Note: Budget is calculated with an hourly rate at 75 USD.

[VALUE FOR MONEY] How does the cost of the project represent value for money for the Cardano ecosystem?

Strategic Investment: The investment in this project represents substantial value for money by providing crucial improvements to Catalyst and Cardano DRep system, which is foundational to the funding and development of the Cardano ecosystem. Improved review processes will enhance project selection quality and governance action directly influencing the efficiency and effectiveness of resource allocation within the Cardano community.

Expertise and Complexity: Costs reflect fair compensation for specialized skills in data analysis and software development. The budget allocations align with prevailing rates in the industry, determined by the experience and skill set of professionals.

Risk mitigation: As a team, we willingly accept the currency risk of being paid in ADA, demonstrating our commitment and adaptability in a dynamic cryptocurrency environment. A decrease in the ADA price is a risk we bear, while any increase allows us to expand the scope.