Total to date

.

Catalyst Assessment and Review Phases Overview

In the current Catalyst process, after the proposals are finalized, there are three phases before community voting:

1. Community Advisors (CAs) assess the proposals and give them a rating between 1 and 5 in three categories: Impact, Feasibility and Auditability, along with a rationale explaining each rating given;
2. Proposers review the CAs assessments, when they are able to flag low quality/unfair assessments to be checked by the Veteran Community Advisors (vCAs).
3. vCAs review all assessments of all proposals, starting by those flagged by the Proposers in the previous phase.

.

Current Problems/Limitations

Currently, during the assessment phase, CAs have to navigate through IdeaScale website and search for proposals to access. In order to better distribute the assessments between proposals, CAs have to manually search for the least assessed proposals, which is inefficient, tiring and time consuming. During Fund4 a document created by IOG was distributed indicating the proposals that were missing a minimum number of CA reviews.

The assessments review done by both Proposers and vCAs is currently performed using an Excel sheet, which is functional to perform the job, but in the case of vCAs reviews, there is no coordination during this phase. This leads to an imbalance in the reviews of assessments: some assessments receive lots of reviews, and others receive none.

As seen in Fund4, CAs and vCAs got overloaded with the amount of proposals they needed to assess and assessments to review. Considering the expected exponential increase in Cardano Project Catalyst's size, it is crucial to improve the efficiency of CAs and vCAs work.

.

Proposed Solution

To solve these problems, we are proposing three tools:

1. A CA Tool, to automatically suggest a proposal to be assessed by the CA based and his challenge preference and the least assessed proposals;
2. A Proposer Tool, to make it easier to go through all the assessments, set them as 'checked' and/or flag them;
3. A vCA Tool, to automatically suggest an assessment to be reviews by the vCA based on his filter choices and the least reviews assessments;

These tools aim to make the process of assessment and review easier, and also to improve/create coordination between CAs and vCAs during these phases.

The team is already engaged with a plan for each tool, and a prototype each tool is already being designed and built.

Work is currently being done on these tools with the intention to have their prototype operational to be used in Fund5 as an option to the current process. Then, based on users' feedback and team's insights, the prototype will evolve into an improved solution.

Since the CAs and vCAs work is paid by Project Catalyst as well, we believe that these tools will increase the Return on Investment related to these activities.

.

Roadmap

• Design user flow and interface for all three tools - Complete (See: https://miro.com/app/board/o9J_lN0yFEM=/))
• Create working prototype of all three tools - CA Tool: Complete. Proposer and vCA: To be complete in time for Fund5 Review-of-Assessments Phase (See: https://cardanocataly.st/proposer-tool , https://cardanocataly.st/ca-tool/ , https://cardanocataly.st/vca-tool/))
• Improve prototype based on community feedback after Fund5 (To be done in time for Fund6)

.

KPIs

• Number of CAs that used the CA tool in Fund5. Goal: > 20 CAs or 20% of CAs
• Number of vCAs that used the vCA tool in Fund5. Goal: > 10 vCAs or 20% of vCAs
• Number of Proposers that used the Proposer tool in Fund5. Goal: > 30 Proposers or 20% of Proposers

.

Technical Specs

CA Tool

The CA Tool is a simple SPA (Single Page Application) based on the JavaScript framework Vue.js.

All the proposals in the "Assess" phase will be preloaded in the tool (using an export provided by the Project Catalyst Team) and the end user will have the opportunity to filter by categories and get a suggestion on the next proposal to be reviewed.

In parallel, a script will be developed that will keep the number of assessments received by each proposal updated: in this way the suggestion on the next proposal to be evaluated is based on the number of reviews received, with the idea of ​​distributing the work of the CA.

For each proposal, the CA that will use the tool will have the possibility to mark it as "reviewed". This information will be stored in the browser (using the local storage) and it would be useful for the CA to to get an overview about the work already done (and also to double check in a simple way if every review was submitted correctly).

.

vCA Tool

The VCA Tool is a simple SPA (Single Page Application) based on the JavaScript framework Vue.js.

The main idea behind the design of this tool is to maintain everything decentralized and compatible with the process followed for Fund4 using Google Sheets.

Every vCA will receive an export file from the Project Catalyst Team that could be imported into the tool.

Using the local storage technology through Vuex (a state management library for Vue.js applications), the tool will store locally in the browser all the reviews made by the vCA. For each assessment review completed the client will also increase the counter on the backend. In this way, the updated count of reviews received by each assessment will be always available to the tool used by others vCAs, in order to coordinate better the workload (no specific info about the review will be shared with the server, only the "reviewed / non reviewed" state).

The backend is an API made with node.js connected to redis and deployed to Heroku.

In addition the vCA will have advanced filters for the assessments (eg. by assessor, proposal, question, rating, length of text and sorted by number of reviews received, random, etc.) and a page with the statistics (of his own work and global).

When the vCA has finished he can simply export the file with his reviews and share it with the Project Catalyst Team.

The files (import and export) used by the vCA Tool are fully compatible with the process followed for Fund4, so it is not mandatory to use it and a vCA can still use Google Sheet.

.

Proposer Tool

The Proposer Tool is really similar to the vCA Tool, but some features are excluded from it.

The Proposer Tool is a simple SPA (Single Page Application) based on the JavaScript framework Vue.js.

Every proposer will receive an export file containing the assessments received for their own proposal. Using the local storage technology through Vuex (a state management library for Vue.js applications), the tool will store locally in the browser all the flags reported by the proposer.

When the Proposer has finished he can simply export the file with the assessments reported and share it with the Project Catalyst Team.

The files (import and export) used by the vCA Tool are fully compatible with the process followed for Fund4, so it is not mandatory to use it and a vCA can still use Google Sheet.

.

Costs Breakdown

• Design user flow and interface for all three tools: $1000
• CA Tool Frontend Development: $600
• vCA Tool Frontend Development: $800
• vCA Tool Backend Development: $200
• Proposer Tool Frontend Development: $600
• Project admin and management: $1400

.