Distributed Auditability
Problem
No common audit standards in funded proposals.
No consistent evaluation of funded proposals.
Funded reporting is manual.
Solution
Identify common community standards.
Build consistent evaluation tools.
Automate funded proposal reporting.
Completed outcome
Total to date
This is the total amount allocated to Distributed Auditability.
About this idea
---------------------------------------------------------------------------------------------------------
Build an automated distributed auditability system.
At its heart Project Catalyst is an experiment to develop distributed governance for the Cardano blockchain. So, meeting the challenge to improve and grow auditability requires a distributed approach. An effective and equitable distribution of auditability will require connections between the funded proposer and the community to be established, the communal development of transparent audit methods and constant iteration.
---------------------------------------------------------------------------------------------------------
Revisiting the Problem Statement
Project Catalyst is rapidly expanding, and its decentralized funding platform needs to develop common audit standards to apply to its funded proposals. At large scale if there is no way to compare the performance of proposals then there is no means by which return on intent can be assessed.
No consistent evaluation will lead to a lack of understanding of why proposals fail or succeed. This will obscure what can be learnt from the Catalyst experiment.
Manual reporting is time-consuming, prone to error and creates inconsistencies between proposals.
---------------------------------------------------------------------------------------------------------
Solution Overview
Drawing on lessons learnt to date tracking and documenting Catalyst Swarm and Quality Assurance DAO this proposal will research reporting workflows, then develop and deploy automated reporting & tracking using Continuous Integration tools such as GitHub Actions.
A library of Continuous Integration tools will assist with consistent evaluation of funded proposals and make reporting processes more transparent.
Automated reporting & tracking is less time-consuming than manual reporting. Each event is run through a batch process that exposes any errors and is reproducible; thereby promoting consistent reporting.
---------------------------------------------------------------------------------------------------------
Relevant experience (identifying information)
Stephen Whitenstall has 30 years' experience in organizing academic, community and business projects. Relevant experience for this proposal includes Program Test Management (IBM) and Quality Assurance Management (BNP-Paribas).
He currently develops, maintains documentation and provides technical project management support for Catalyst Swarm.
In Fund 5 of Project Catalyst Stephen's QA-DAO (https://stephen-rowan.gitbook.io/quality-assurance-dao/) submitted a proposal "Quality Assurance DAO" in the Developer Ecosystem Challenge that was funded in August 2021.
Stephen provides support to Catalyst Swarm by documenting and tracking it's activities. This has been done continuously since 7th July 2021 by raising GitHub issues that track Core Swarm Activities such as meetings, After Town Halls, events (Idea Fest) and Saturday Swarm Sessions. The evolution of the governance of Catalyst Swarm is documented and updated here - https://catalyst-swarm.gitbook.io/catalyst-swarm-genesis/governance/overview .
You can support the activities of QA-DAO by becoming a GitHub sponsor https://github.com/sponsors/Quality-Assurance-DAO .
https://www.linkedin.com/in/stephen-whitenstall-166727210
---------------------------------------------------------------------------------------------------------
Roadmap
---------------------------------------------------------------------------------------------------------
The present – Fund 5 QA-DAO proposal and community collaborations.
In August 2021 the F5: Developer ecosystem proposal Quality-Assurance-DAO ( https://cardano.ideascale.com/a/dtd/Quality-Assurance-DAO/351678-48088 ) was successful in receiving votes and was funded. This confirmed that voters wish to see Quality Assurance Implemented in Project Catalyst and provides support for this proposal.
---------------------------------------------------------------------------------------------------------
Phase 0 : Prior to Funding - Research and prototyping – present to November 2021
---------------------------------------------------------------------------------------------------------
Research
How different teams and individuals report the progress of their funded proposals will be researched.
This Fund 6 Auditability proposal will draw upon the successful Fund 5 Developer ecosystem proposal for research. This will be done in collaboration with the funded cohort, IOG and Catalyst ecosystem toolmakers such as Catalyst Swarm.
Lessons Learnt to date
The ongoing experience of working with a widely distributed team such as Catalyst Swarm, working on many different projects, has led to automation solutions such as Issue templates and forms.
Qualitative lessons have also been learnt within Catalyst Swarm in respect of the amount of reporting and tracking required. In terms of what is manageable and what is meaningful to team members. The best incentive is a team understanding of who contributes what to a project - so that work is fairly attributed and costed.
---------------------------------------------------------------------------------------------------------
Phase 1 - Continuous Integration Development - November 2021 to December 2021
---------------------------------------------------------------------------------------------------------
Once funded Distributed Auditability will build upon lessons learnt in research and prototyping to automate reporting & tracking using Continuous Integration tools such as GitHub Actions.
Building on prototypes from Phase 0 specific auditing workflows will be automated using GitHub Actions.
Auditability Workflows
Data from funded cohort reporting, Swarm tracking and qualitative research (into how reporting varies) will inform the design of auditability workflows. An auditability workflow will model the working practices of participants, the metrics they report on, whether this fit into their own documentation and how they might benefit from automation.
Auditability Workshops
In addition a related proposal Open-Source-Training ( F6: DLT Entrepreneurship Toolbox proposal ( https://cardano.ideascale.com/a/dtd/Open-Source-Training/368678-48088 ) offered in collaboration with the Catalyst School will will offer workshops to assist early stage entrepreneurs in discovering their workflows and examining how these may be documented, reported on and automated. If the Open-Source-Training is funded data from workshop workflows will also feedback into Continuous Integration Development.
Continuous Integration Development
Auditability workflows identified from tracking and research will be translated to GitHub Actions for development and testing.
This will comprise of :
- Batch jobs that will raise & close tracking issues, meeting minutes, contribution surveys and biweekly reports automatically.
- Interface with Catalyst Fund data to prefill issue templates & reports.
- Interface with and update static site website with team workflow status and reporting.
- Setup trigger events on pull requests.
- Sending status bots to Discord servers.
---------------------------------------------------------------------------------------------------------
Phase 2 - Continuous Integration Deployment - January 2022 to February 2022
---------------------------------------------------------------------------------------------------------
From January 10th 2022 QA-DAO will begin to deploy Continuous Integration tools at pilot GitHub Organisations such as Catalyst Swarm, SecretDecks and Quality Assurance DAO.
- Auditability workflows identified in Development (above) will be deployed and tested in the pilot GitHub Organisations.
- Deployment and test reports will be produced to document how effective the tools are.
---------------------------------------------------------------------------------------------------------
Definition of success after 3, 6 and 12 months
---------------------------------------------------------------------------------------------------------
3 months (November 2021 to January 2022)
Continuous Integration Development
GitHub Actions are being developed and tested in the following stages :
- Continuous Integration Development 1 - Monday, November 15th to Tuesday November 16th 2021
- Continuous Integration Development 2 - Monday, November 22nd to Tuesday November 23rd 2021
- Continuous Integration Development 3 - Monday, November 29th to Tuesday November 30th 2021
- Continuous Integration Development 4 - Monday, December 6th to Tuesday December 7th 2021
- Continuous Integration Development 5 - Monday, December 13th to Tuesday December 14th 2021
- Break
Continuous Integration Deployment
GitHub Actions are being deployed and tested in pilot GitHub Organisations in the following stages :
- Continuous Integration Deployment 6 - Monday, January 10th to Tuesday January 11th 2022
- Continuous Integration Deployment 7 - Monday, January 17th to Tuesday January 18th 2022
- Continuous Integration Deployment 8 - Monday, January 24th to Tuesday January 25th 2022
Marketing to funded cohort and proposers
Promotion, engagement and communication on Social Media (Twitter, Telegram, Youtube, Discord).
Reporting
Deployment and test reports are produced and communicated on Social Media.
---------------------------------------------------------------------------------------------------------
6 months (February 2022 to April 2022)
Pilot GitHub Organisations and some funded proposals are using the automated auditability workflow tools.
The Cardano and Catalyst community are using the Final Report on Development & Deployment for reference.
---------------------------------------------------------------------------------------------------------
12 months (May 2022 to October 2022)
Pilot GitHub Organisations and some funded proposals are using the automated auditability workflow tools.
The Cardano and Catalyst community are using the Final Report on Development & Deployment for reference.
---------------------------------------------------------------------------------------------------------
Expected launch date : February 2022
---------------------------------------------------------------------------------------------------------
Budget
Continuous Integration Development
5 Stages of 16 hours Development
= 80 hours
Continuous Integration Deployment
3 Stages of 16 hours Deployment
= 48 hours
Documentation and presentation
8 stages with 8 hours preparation each
= 64 hours
Final Report on Development & Deployment
= 16 hours
Marketing
Promotion, engagement and communication on Social Media (Twitter, Telegram, Youtube, Discord).
10 hours for 8 stages
---------------------------------------------------------------------------------------------------------
Total Hours = 218
$ 40 per hour*
Total = $ 8720
*Hourly rate does not match commercial rates for this work - but is set at a lower rate given the experimental & community context.
---------------------------------------------------------------------------------------------------------
Key Metrics to measure
- Number of funded proposals using the automated auditability workflow tools.
- Number of commits in funded proposal repositories (Github)
- Quality of performance, achievements & financials reports using the tools compared to those not using them.
- Number of onboarded developers using Continuous Integration
Monthly report
NB: Monthly reporting was deprecated from January 2024 and replaced fully by the Milestones Program framework. Learn more here
Progress report
Progress report
Progress report
Team
Stephen has 30 years' experience in organizing academic, community and business projects (See Detailed plan).
