Last updated 11 months ago
Ouroboros Stress Test and Early Warning System
Problem
Ouroboros has exceptional security but as the value of Cardano increases subtle vulnerabilities become sources to leak value. The evolution of the protocol requires rigorous tests and detectors.
Total to date
This is the total amount allocated to Ouroboros Stress Test and Early Warning System.
About this idea
[GENERAL] Name and surname of main applicant
Kenric Nelson[GENERAL] Are you delivering this project as an individual or as an entity (whether formally incorporated or not)
Entity (Incorporated)[GENERAL] Please specify how many months you expect your project to last (from 2-12 months)
4[GENERAL] Please indicate if your proposal has been auto-translated into English from another language
No[GENERAL] Summarize your solution to the problem (200-character limit including spaces)
Photrek has developed unique robustness metrics for information systems. Dr. Aman will use these probabilistic methods to design early warning detectors that we’ll evaluate on protocol testnets.
[GENERAL] Does your project have any dependencies on other organizations, technical or otherwise?
No[GENERAL] If YES, please describe what the dependency is and why you believe it is essential for your project’s delivery. If NO, please write “No dependencies.”
No dependencies.
[GENERAL] Will your project’s output/s be fully open source?
Yes[GENERAL] Please provide here more information on the open source status of your project outputs
YES
Photrek is developing Communities of Innovation centered around development and utilization of open-source capabilities. The Risk Intelligence Community of Innovation will focus on development of machine intelligence algorithms for managing risk. This effort grew out of an open-source collaboration called Machine Intelligence for Complex Systems which produced the following open source code bases:
Nonlinear Statistical Coupling Library - Apache v2.0 License
Coupled Variational Autoencoder - GNU v3.0 License
[SOLUTION] Please describe your proposed solution
Cardano seeks to achieve the most decentralized, secure blockchain network as the foundation for an antifragile financial operating system. To achieve this goal, it is essential that the evolution of the Ouroboros protocol be designed and developed alongside a suite of rigorous stress tests and early warning detection systems. Ouroboros Praos established Cardano as a leader in secure decentralized proof-of-stake. The protocol continues to evolve with the anticipated release of Ouroboros Leios which will support Input Endorsers and enhanced optimization of transaction efficiency.
Photrek’s first stress test and detection system will focus on the ability of rogue actors to drain value from the network through manipulation of block production timing. If a multipool or a consortium of SPOs were to share slot leader selection information, that knowledge could be used to strategically withhold block production for the purpose of creating blockchain segments controlled by these Byzantine actors. Figure 1 shows the global block production predictions and experimental measurements. Collusion is difficult to detect because it can “hide” in the inherent uncertainty of Nakamoto proof-of-stake and result in smaller pools failing to mint their block assignments to the broader network. Although this type of event is expected to be a rare event, systematic exploitation could drain a substantial amount of value out of the network overtime. Photrek will quantify the real-world vulnerability, compare that with design analysis, and develop detectors for early warning.
Figure 1: Block Production
The Cardano community needs tools and educational materials to locally monitor the accuracy of the slot leader probabilities. Local alerts are needed when the accuracy of the probabilities drops below historic trends. Furthermore, analytical tools are necessary to identify which cluster of nodes are attempting a Byzantine attack. Furthermore an understanding of how upgrades to the Ouroboros protocol impact these vulnerabilities is needed.
Photrek and its partners will develop a Stress Test and Early Warning System that provides Cardano with
- The ability to simulate vulnerabilities in the timing of block production on testnets;
- A histogram of the probability forecasts of the selected slot leader;
- A visualization comparing the delegation ratio forecasts with the histogram of the actual percentages of slot leader selection;
- Accuracy, Robustness and Decisiveness metrics regarding slot selection;
- Educational resources to enhance knowledge within the SPO community regarding Byzantine attacks and how the SPO Health Monitor can mitigate the threat.
Figure 2 shows an example of a Risk Monitor plot, which will be the first tool developed for the Early Warning Detection System. This example was of an image generator algorithm. The histogram can show the distributional character of the forecasts. The Accuracy shows the central-tendency on a log scale; the Robustness shows the outlier performance; and the Decisiveness shows the ability to forecast the correct class (slot leader).
Cardano protocol developers and SPOs will be able to utilize the Risk Monitor to actively track the accuracy and robustness of the slot leader forecasts. This information can provide an early warning if slot leader selection and thus block production is being manipulated via block withholding or similar network distribution based attacks. The analysis will be completed across the network nodes and for individual nodes. By developing historical records of the Robustness-Accuracy-Decisiveness (R-A-D) metrics, alerts based on significant deviations from historical trends can be created. These alerts can for instance be used to determine whether multipool operators are withholding release of blocks in effort to increase their probability of slot leader selection.
Figure 2: Risk Monitor
How we perceive the problem we are solving:
The problem at hand involves ensuring accountability among SPOs based on the honest behavior outlined by the theoretical predictions in the Ouroboros Praos paper. This involves enabling real-time analysis of each party's data dissemination across the network. The introduction of a numerical risk model would empower developers and SPOs to conduct an empirical analysis of network health, thereby swiftly identifying any Byzantine behavior.
- Another concern that needs addressing is the possibility of multiple pool operators leveraging their size to impact the network negatively. An example of this issue can be found in an article on https://adapulse.io/multiple-stakepool-operators-are-harming-cardano/
- Furthermore, an enhancement we are considering is the provision of an "active" local calculation mechanism for block creation probability. This feature would greatly aid in monitoring the performance of the network.
Our reasons for approaching it in the way that we have:
- This approach takes into account real world network behavior instead of theoretical estimates. Nakamoto proof-of-stake allows for a larger validator set (due to being more bandwidth efficient) at the cost of short-term statistical uncertainty. This probabilistic approach coupled with a dynamically available distributed network provides a path for dishonest SPOs to accrue greater than their fair share of rewards.
- Such an effect would be small and would be more easily found by having multiple SPOs analyzing the network concurrently, in real-time.
Who our project will engage:
- The Cardano developer and SPO community will be the primary target for engagement within this project. The team plans to connect with the community through dedicated chat groups on Telegram and Discord, as well as by hosting breakout rooms. The team will introduce the proposed solution and reach out to developers and SPOs who express interest, with the aim of collaborating on testing and validation of new protocol designs and ongoing operation of the network.
How we will demonstrate or prove our impact:
- The impact of this solution will be evidenced through several steps:
- A complete testnet experiment to gauge the impact on probability accuracy. This will provide an understanding of how accurate the solution works in real-world conditions.
- A demonstration of the sensitivity of the model to the accumulation of stake. This will provide insights on the model’s ability to handle changes in stake distribution and their impact on the network.
Explanation of our unique solution, who will benefit, and why this should be important to Cardano:
- Our solution is unique because of its localized approach to pool performance evaluation. Ouroboros is unique among proof of stake blockchains by virtue of its probabilistic nature. Local calculation of the relevant statistics for a pool are vital to understand pool performance.
- Complex model due to a combination of network topology, game theory of information propagation, and leader election process is well suited to a (multi-vectored) risk analysis.
- This comprehensive approach will benefit all participating in the network, particularly SPOs by providing a more robust and transparent understanding of network health and performance. Underscoring why this should be of significant importance to the Cardano community.
[IMPACT] Please define the positive impact your project will have on the wider Cardano community
This project will have a significant impact on and value for the Cardano ecosystem by
- providing a tool that enables developers to be rigorous in the evaluation of protocol upgrades and for SPO operators to be vigilant in monitoring the health of the network. This results in enhancements to the overall decentralization and trustworthiness thus adding value to the entire Cardano ecosystem.
The project will provide significant impact through various ways:
- By enabling empirical network analysis, it can optimize the design and testing for how information propagates throughout the network, leading to enhanced efficiency and reliability.
- Helping ensure fair and honest block production which is essential for the ecosystem’s growth and strength and will attract both existing and new users to engage more deeply with the Cardano ecosystem.
- Providing real-time analysis of the likelihood that a string of events has occurred. This can be instrumental in identifying irregularities or potential issues within the system.
- Ex. what are the odds that pool A has created 10 out of the last 20 blocks?
We will aim to simulate anomalous events on the Cardano testnest and to engage with 3 SPOs to compare local measurements of network health using the Risk Monitor. These measures will demonstrate to the Cardano community the feasibility of having a decentralized measure of network health.
This tool will promote honest SPO block creation behaviors through the community accountability afforded by introducing locally computed probability metrics. In the short term, this may lead to improved trust within the community, as new Ouroboros designs are rigorously evaluated and SPOs have increased visibility into the relationship between actual leader distribution and the design goals. In the long term, this will enhance the level of decentralization within the ecosystem, thus strengthening the reputation and integrity of the Cardano network, attracting more users and developers.
[CAPABILITY & FEASIBILITY] What is your capability to deliver your project with high levels of trust and accountability? How do you intend to validate if your approach is feasible?
Capability to Deliver
- Photrek is a trusted partner with the Cardano community, currently supporting Input Output Global on research regarding Alternative Voting Methods, and developing a Sociocratic Circle to manage a community DRep.
- Photrek’s expertise in analysis of probabilistic forecasts is evidenced through originating concepts in research publications (e.g., On average uncertainty, Assessing Probabilistic Inference and applying those concepts to real-world problems (including analyzing election forecasts and behavior analysis in financial markets).
- Photrek’s team includes applicable expertise including (1) experienced Cardano stake pool operators and (2) researchers in Proof-of-Stake protocols’ eligibility thresholds.
- Photrek’s team includes significant software development expertise in creating a variety of tools with contextually-appropriate User Experience/User Interface.These span (1) real-time updatable graphical analytics to (2) terminal-based minimalistic utilities such as Text User Interfaces. This will be leveraged to develop tools supporting meaningful use-cases for stake pool operators.
- Photrek’s successful track record in prior Catalyst funds was facilitated by business processes including web-based time reporting and weekly virtual meetings to monitor progress.
Validation of Feasibility
- Develop Ouroboros Risk Monitor tool suite: Develop software performing static event data analysis for developers and SPOs to evaluate and monitor block eligibility. This includes development of appropriate visualization and user-interface. Suite will be iteratively refined in response to validation and usability objectives’ progress.
- Validate tool operability: Initially examine historical per-epoch block eligibility for presence of byzantine behavior. Subsequently, extend to per-slot real-time analysis. Compare empirical block production frequency against Cardano protocol’s eligibility forecasts. Consider the prospect of evaluation of byzantine behavior detection within an appropriate SPO emulation environment (e.g., a standalone or a test net).
- Validate tool usability: Engage at least 3 SPOs for feedback on tool utility and incorporate the feedback into our incremental development.
- Present capability to Cardano community: Upload open-source Risk Monitor suite to publicly-accessible software repository along with educational materials, use-case examples, and documentation artifacts.
Trustworthy Management Processes
- Photrek plans to submit monthly progress reports, ensuring that expenditures are tied to developments which incrementally achieve program objectives.
- Photrek will host Catalyst After Town Hall meetings to discuss project results.
- Photrek’s successful track record in prior Catalyst funds was facilitated by business processes including web-based time reporting and weekly virtual meetings to monitor progress.
- Photrek has instituted a Sociocratic Operating Agreement that forms peer-to-peer circles interlinked by reciprocal representatives. Each member of a circle has the authority to raise reasoned objections that are resolved as a team toward consent decisions. This provides a high level of accountability throughout our endeavors.
[PROJECT MILESTONES] What are the key milestones you need to achieve in order to complete your project successfully?
Stand-up initial Risk Monitor (1 month)
Milestone outcome:
- Establish a baseline repository for maintained implementation of Risk Monitor tool suite
- Implement privately-accessible version-controlled baseline software
Acceptance criteria:
- Gather/analyze block eligibility data from Cardano protocol.
- Incorporate eligibility data into probabilistic risk metric computation.
- Present computation results in an understandable/meaningful format.
Proof of milestone completion:
- Report detailing initial Risk Monitor capability detailing what was accomplished
Validate Risk Monitor on Testnest (1 month)
Milestone outcome:
Update repository including operability scripts, data, and test tools.
Acceptance criteria:
- Evaluate tool operation on historical block eligibility (static event data).
- Evaluate tool operation with real-time block data using pre-production, preview and/or Guild testnet.
- Confirmation that tools operate in a manner satisfying project objectives.
Proof of milestone completion:
- Report detailing the outputs from the tool and initial findings from historical and emulated byzantine detection.
Validate usability of Risk Monitor (1 month)
Milestone outcome:
- Developer and SPO feedback on capability updates
Acceptance criteria:
- Gather/incorporate developer and SPO feedback.
- Extend tool capabilities to priority use-cases identified by developers and SPOs.
- Risk Monitor operation confirmed as meaningful/useful to developers and SPOs.
Proof of milestone completion:
- Document detailing the findings and capability updates from developer and SPO feedback.
Present Risk Monitor tools and documentation (1 month)
Milestone outcome:
- Prepare educational documentation on risk metric usage/interpretation.
- Upload developed software to publicly-accessible repository.
Acceptance criteria:
- Operational Risk Monitor capability and documentation in a format suitable for usage within the wider Cardano community.
- Proof of milestone completion:Risk Monitor software, installation instructions, educational material, and testing scripts.
- Video Close-out report
[RESOURCES] Who is in the project team and what are their roles?
Kenric Nelson, Ph.D.: Principal Investigator, President of Photrek, Improve design of Risk Metric and oversee execution of the project
Jose De Gamboa Program Manager, Photrek Cardano Lead, Customer Engagement and Project Execution
James Aman, Ph.D.: Teaching Professor at Rice University, Protocol Security Engineer, design byzantine experiment attacks
Ray Mata: SPO Specialist, Contribute to experiment design and usability tests
Biographies of the Project Team
Dr. Kenric Nelson is Founder and President of Photrek, LLC which is developing novel approaches to Complex Decision Systems, including dynamics of cryptocurrency protocols, sensor systems for machine intelligence, robust machine learning methods, and novel estimation methods. He served on the Cardano Catalyst Circle governance council and is leading a revitalization of Sociocracy for All’s
Dr. James Aman is a web3 practitioner, researcher, and educator. Since his introduction to blockchain in 2015, he has developed several decentralized applications across various blockchain networks and architected protocols such as Ouroboros Taktikos, a regularized Nakamoto proof-of-stake protocol. James currently serves as a faculty member at Rice University where he teaches an introductory blockchain course for undergraduate and graduate students that combines the technical and business impacts of decentralized ledger technologies. James also co-founded Topl, a novel layer-one protocol for building sustainable and inclusive communities. While at Topl, he led the teams responsible for the engineering and research efforts in implementing the Topl protocol including novel implementations of the key-evolving signature scheme and verifiable random functions required by the Ouroboros protocol.
Raymond Mata's professional career spans three decades, encompasses a diverse range of experiences and accomplishments. After graduating with a Pre-Med background in Psychology and Biology in Boston, he advanced to a freelance IT consultant and Server Engineer for a corporate datacenter serving 10,000 employees. Raymond later relocated to Hawaii and discovered his passion for education while teaching High School Computer Science and in parallel established a successful IT Managed Services Provider business with a ZERO-data loss track record providing technology solutions and critical support to local businesses across the islands for nearly two decades. Raymond then ventured into the blockchain crypto industry, inspired by its capacity to do good and address wealth distribution inequality. He opened business as a dedicated Cardano stake pool operator for GROW, staked with ~49M ADA and largely recognized for his on-going charitable child-saving initiatives. His present work at the intersection of blockchain and AI technology provides services to projects in the space ranging from community manager, game development & testing to blockchain infrastructure provider and systems administrator that all underscore his passion and commitment to the future of humanity and self-sovereignty.
[BUDGET & COSTS] Please provide a cost breakdown of the proposed work and resources
Milestone 1: ₳ 38,000
Milestone 2: ₳ 36,000
Milestone 3: ₳ 36,000
Final Milestone: ₳ 40,000
Project costs are broken into the following categories as referenced in the table below:
Engineering
- Salaries to developers and consultants
Customer Engagement
- Compensation for community outreach work
Computing resources
- Hosting services
- Other computing costs
Program management
- Project reporting
- Meeting time
- Project coordination
Contingency
[VALUE FOR MONEY] How does the cost of the project represent value for money for the Cardano ecosystem?
Photrek is led by a five person Leadership Circle that uses consent-based decision making to make sure that our policies, objectives, and execution are consistent with building a collaborative environment for our team, our partners, and our customers. We work to develop Enlightened Pathways that enhance the intelligence and sustainability of the planet. Our approach to corporate governance is specified in the Photrek Sociocratic Operating Agreement.
Towards that end, we utilize competitive pricing methods that provide the highest value to our customers and support our team members with fulfilling lives. Our rates are based on self-employment in the US & Canada. The rates take into account the employment overheads of the resources contracted. The amounts are calculated for each milestone based on the hours to complete. For example the chart for engineering and scientific salaries in the Commonwealth of Massachusetts is provided here: https://www.mass.gov/guides/salary-and-compensation.
