Improve the Quality of CA Reviews

Expanded Problem Statement

Despite the accolades for being the largest decentralised funding project with collaboration at the heart of its success, focusing on these positives and not recognising the current weaknesses, could introduce problems as Catalyst grows.

While collaboration is an important attribute of Catalyst, it is naive to ignore the fact that Catalyst is also competitive. There have been more losing proposals than winners. So in addition to pitching the value they can bring, some projects also resort to criticising proposals that are competing for the limited funds available in their challenge.

The absence of identity and credential verification for Community Advisers (CAs) and Veteran Community Advisers (VCAs) means that project members can create alternate accounts to submit negative reviews for competing projects. The VCA review process can also be influenced by negative actors.

Impact

Due to the large volume of proposals it is unrealistic to expect voters to fully absorb every proposal. Some voters are therefore resorting to the CA reviews for guidance on which proposals to vote for.

This was highlighted by a Japanese community member, who surveyed the Japanese community's participation in Catalyst. Of those that voted, a large number indicated that they didn't understand the proposals well enough to make an informed decision, so followed the CA ratings when voting.

Competition can cause unreasonable reviews which leads to uninformed voters and a less optimised ROI.

Review Quality

While verifying every Community Advisor is not scalable, the use of VCAs to assess the quality of CA reviews is on the surface a good idea. However due to the technical nature of some proposals, it is important that VCAs are able to understand the proposal and its reviews to properly assess their quality. As VCAs are being paid to quality check reviews, this capability is important for the Cardano community to get value for money.

Potential Solution

To ensure that bad actors do not infiltrate the VCA ranks, the purpose of this proposal is to indicate the community's request for the Catalyst team to perform identity verification checks of VCAs to ensure they are not assessing CA reviews where there is a conflict of interest (e.g. the proposal is in competition with another project they are aligned with).

Further, VCA experience should be collected and documented to ensure they have the ability to assess technical proposals and their reviews. If someone challenges the ability for a VCA to properly QA a proposal or review, then this documentation can be referred to. If VCAs know that their ability to properly assess proposal reviews is open to dispute, they will be more inclined to only assess those they are able to.

While this may initially fall to the Catalyst team to oversee, as Catalyst becomes more decentralised, a VCA identity verification and technical review process can be outsourced to community elected organisations.

**************

By voting for this proposal you are indicating your support for the introduction of this process.