Last updated 2 years ago
SPO Health Monitor
Problem
Cardano's block-creation can be gamed by SPOs colluding to withhold blocks, boosting future win chances. SPOs lack decentralized tools to stop this.
Total to date
This is the total amount allocated to SPO Health Monitor.
About this idea
[GENERAL] Name and surname of main applicant
Juana Attieh[GENERAL] Are you delivering this project as an individual or as an entity (whether formally incorporated or not)
Entity (Incorporated)[GENERAL] Co proposers and Additional applicants
Dr. James Aman
[GENERAL] Requested funds in ada
199400[GENERAL] Please specify how many months you expect your project to last (from 2-12 months)
4[GENERAL] Please indicate if your proposal has been auto-translated into English from another language.
No[GENERAL] Summarize your solution to the problem (200-character limit including spaces)
A SPO Health Monitor suite of tools is needed to strengthen the decentralized integrity of the SPO network. Photrek will develop a utility comparing the forecasted & actual probabilities of selection.
[GENERAL] Does your project have any dependencies on other organizations, technical or otherwise?
No[GENERAL] If YES, please describe what the dependency is and why you believe it is essential for your project’s delivery. If NO, please write “No dependencies.”
No dependencies.
[GENERAL] Will your project’s output/s be fully open source?
Yes[GENERAL] Please provide here more information on the open source status of your project outputs
Photrek is developing Communities of Innovation centered around development and utilization of open-source capabilities. The Risk Intelligence Community of Innovation will focus on development of machine intelligence algorithms for managing risk. This effort grew out of an open-source collaboration called Machine Intelligence for Complex Systems which produced the following open source code bases:
Nonlinear Statistical Coupling Library - Apache v2.0 License
Coupled Variational Autoencoder - GNU v3.0 License
[SOLUTION] Please describe your proposed solution.
Cardano seeks to achieve the most decentralized, secure blockchain network as the foundation for an antifragile financial operating system. To achieve this goal, it is essential that the SPOs are acting independently. If a multipool or a consortium of SPOs were to share slot leader selection information, that knowledge could be used to strategically withhold block production for the purpose of creating blockchain segments controlled by these Byzantine actors. Figure 1 shows the global block production predictions and experimental measurements. Collusion is difficult to detect because it can “hide” in the inherent uncertainty of Nakamoto proof-of-stake and result in smaller pools failing to mint their block assignments to the broader network.
Figure 1: Block Production
The SPO community needs tools and educational materials to locally monitor the accuracy of the slot leader probabilities. Local alerts are needed when the accuracy of the probabilities drops below historic trends. Furthermore, analytical tools are necessary to identify which cluster of nodes are attempting a Byzantine attack.
Photrek and its partners will develop a SPO Health Monitor that provides Cardano SPOs with
- A histogram of the probability forecasts of the selected slot leader;
- A visualization comparing the delegation ratio forecasts with the histogram of the actual percentages of slot leader selection;
- An Accuracy metric of the forecasts based on the geometric mean of the forecasted probabilities;
- Robustness and Decisiveness metrics that indicate the degree of variability in the forecasts
- Educational resources to enhance knowledge within the SPO community regarding Byzantine attacks and how the SPO Health Monitor can mitigate the threat.
Figure 2 shows an example of the Risk Monitor plot. This example was of an image generator algorithm. The histogram can show the distributional character of the forecasts. The Accuracy shows the central-tendency on a log scale; the Robustness shows the outlier performance; and the Decisiveness shows the ability to forecast the correct class (slot leader).
Cardano SPOs will be able to utilize the Risk Monitor to actively track the accuracy and robustness of the slot leader forecasts. This information can provide an early warning if slot leader selection and thus block production is being manipulated via block withholding or similar network distribution based attacks. The analysis will be completed across the network nodes and for individual nodes. By developing historical records of the Robustness-Accuracy-Decisiveness (R-A-D) metrics, alerts based on significant deviations from historical trends can be created. These alerts can for instance be used to determine whether multipool operators are withholding release of blocks in effort to increase their probability of slot leader selection.
Figure 2: Risk Monitor
How we perceive the problem we are solving:
The problem at hand involves ensuring accountability among SPOs based on the honest behavior outlined by the theoretical predictions in the Ouroboros Praos paper. This involves enabling real-time analysis of each party's data dissemination across the network. The introduction of a numerical risk model would empower each SPO to conduct an empirical analysis of network health, thereby swiftly identifying any Byzantine behavior.
- Another concern that needs addressing is the possibility of multiple pool operators leveraging their size to impact the network negatively. An example of this issue can be found in an article on https://adapulse.io/multiple-stakepool-operators-are-harming-cardano/
- Furthermore, an enhancement we are considering is the provision of an "active" local calculation mechanism for block creation probability. This feature would greatly aid in monitoring the performance of the network.
Our reasons for approaching it in the way that we have:
- This approach takes into account real world network behavior instead of theoretical estimates. Nakamoto proof-of-stake allows for a larger validator set (due to being more bandwidth efficient) at the cost of short-term statistical uncertainty. This probabilistic approach coupled with a dynamically available distributed network provides a path for dishonest SPOs to accrue greater than their fair share of rewards.
- Such an effect would be small and would be more easily found by having multiple SPOs analyzing the network concurrently, in real-time.
Who our project will engage:
- The SPO community will be the primary target for engagement within this project. The team plans to connect with the community through dedicated SPO chat groups on Telegram and Discord, as well as by hosting breakout rooms. The team will introduce the proposed solution and reach out to SPOs who express interest, with the aim of collaborating on testing and validation of the solution.
How we will demonstrate or prove our impact:
- The impact of this solution will be evidenced through several steps:
- A complete testnet experiment to gauge the impact on probability accuracy. This will provide an understanding of how accurate the solution works in real-world conditions.
- A demonstration of the sensitivity of the model to the accumulation of stake. This will provide insights on the model’s ability to handle changes in stake distribution and their impact on the network.
Explanation of our unique solution, who will benefit, and why this should be important to Cardano:
- Our solution is unique because of its localized approach to pool performance evaluation. Ouroboros is unique among proof of stake blockchains by virtue of its probabilistic nature. Local calculation of the relevant statistics for a pool are vital to understand pool performance.
- Complex model due to a combination of network topology, game theory of information propagation, and leader election process is well suited to a (multi-vectored) risk analysis.
- This comprehensive approach will benefit all participating in the network, particularly SPOs by providing a more robust and transparent understanding of network health and performance. Underscoring why this should be of significant importance to the Cardano community.
[IMPACT] Please define the positive impact your project will have on the wider Cardano community.
This project will have a significant impact on and value for the Cardano ecosystem by
- providing a tool that promotes and ensures honest behavior of SPO operators. This results in enhancements to the overall decentralization, trustworthiness and health of the network, thus adding value to the entire Cardano ecosystem.
The project will provide significant impact through various ways:
- By enabling empirical network analysis, it can optimize the propagation of information throughout the network, leading to enhanced efficiency and reliability.
- Helping ensure fair and honest block production which is essential for the ecosystem’s growth and strength and will attract both existing and new users to engage more deeply with the Cardano ecosystem.
- Providing real-time analysis of the likelihood that a string of events has occurred. This can be instrumental in identifying irregularities or potential issues within the system.
- Ex. what are the odds that pool A has created 10 out of the last 20 blocks?
We will aim to have 3 SPOs compare local measurements of network health using the Risk Monitor. These measures will demonstrate to the Cardano community the feasibility of having a decentralized measure of network health.
This tool will promote honest SPO block creation behaviors through the community accountability afforded by introducing locally computed probability metrics. In the short term, this may lead to improved trust within the community, as SPOs can be better held accountable. This could result in higher engagement and active participation within the network. In the long term, this will enhance the level of decentralization within the ecosystem, thus strengthening the reputation and integrity of the Cardano network, attracting more users and developers.
[CAPABILITY & FEASIBILITY] What is your capability to deliver your project with high levels of trust and accountability? How do you intend to validate if your approach is feasible?
Capability to Deliver
- Photrek’s expertise in analysis of probabilistic forecasts is evidenced through originating concepts in research publications (e.g., On average uncertainty, Assessing Probabilistic Inference and applying those concepts to real-world problems (including analyzing election forecasts and behavior analysis in financial markets).
- Photrek’s team includes applicable expertise including (1) experienced Cardano stake pool operators and (2) researchers in Proof-of-Stake protocols’ eligibility thresholds.
- Photrek’s team includes significant software development expertise in creating a variety of tools with contextually-appropriate User Experience/User Interface.These span (1) real-time updatable graphical analytics to (2) terminal-based minimalistic utilities such as Text User Interfaces. This will be leveraged to develop tools supporting meaningful use-cases for stake pool operators.
- Photrek’s successful track record in prior Catalyst funds was facilitated by business processes including web-based time reporting and weekly virtual meetings to monitor progress.
Validation of Feasibility
- Develop SPO Risk Monitor tool suite: Develop software performing static event data analysis for SPO to evaluate and monitor block eligibility. This includes development of appropriate visualization and user-interface. Suite will be iteratively refined in response to validation and usability objectives’ progress.
- Validate tool operability: Initially examine historical per-epoch block eligibility for presence of byzantine behavior. Subsequently, extend to per-slot real-time analysis. Compare empirical block production frequency against Cardano protocol’s eligibility forecasts. Consider the prospect of evaluation of byzantine behavior detection within an appropriate SPO emulation environment (e.g., a standalone or a test net).
- Validate tool usability: Engage at least 3 SPOs for feedback on tool utility and incorporate the feedback into our incremental development.
- Present capability to Cardano community: Upload open-source Risk Monitor suite to publicly-accessible software repository along with educational materials, use-case examples, and documentation artifacts.
Trustworthy Management Processes
- Photrek plans to submit monthly progress reports, ensuring that expenditures are tied to developments which incrementally achieve program objectives.
- Photrek will host Catalyst After Town Hall meetings to discuss project results.
- Photrek’s successful track record in prior Catalyst funds was facilitated by business processes including web-based time reporting and weekly virtual meetings to monitor progress.
- Photrek has instituted a Sociocratic Operating Agreement that forms peer-to-peer circles interlinked by reciprocal representatives. Each member of a circle has the authority to raise reasoned objections that are resolved as a team toward consent decisions. This provides a high level of accountability throughout our endeavors.
[Project Milestones] What are the key milestones you need to achieve in order to complete your project successfully?
Stand-up initial Risk Monitor (end of month 1)
- Task 1.1: Gather/analyze block eligibility data from Cardano protocol.
- Task 1.2: Incorporate eligibility data into probabilistic risk metric computation.
- Task 1.3: Present computation results in an understandable/meaningful format.
- Deliverable: baseline repository-maintained implementation of Risk Monitor tool suite.
- Outputs: privately-accessible version-controlled baseline software implementation
- Outcome: stand-up of initial Risk Monitor capability
Validate operability of Risk Monitor (end of month 2)
- Task 2.1: Evaluate tool operation on historical block eligibility (static event data).
- Task 2.2: Evaluate tool operation with real-time block production data.
- Deliverable: updated repository including operability scripts, data, and test tools.
- Outputs: initial evaluation findings from historical and emulated byzantine detection.
- Outcome: confirmation that tools operate in manner satisfying project objectives.
Validate usability of Risk Monitor (end of month 3)
- Task 3.1: Gather/incorporate SPO feedback.
- Task 3.2: Extend tool capabilities to priority use-cases identified by SPOs.
- Deliverable: documented findings and capability updates from SPO feedback.
- Outputs: findings incorporated into a section of regular progress status report.
- Outcome: Risk Monitor operation confirmed as meaningful/useful to SPOs.
N/A
N/A
Present Risk Monitor tools and documentation (end of month 4)
- Task 4.1: Prepare educational documentation on risk metric usage/interpretation.
- Task 4.2: Upload developed software to publicly-accessible repository.
- Deliverable: updated implementation within publicly-accessible open-source software repository.
- Outputs: Risk Monitor software, installation instructions, educational material, and testing scripts.
- Outcome: operational Risk Monitor capability and documentation in a format suitable for usage within the wider Cardano community.
[RESOURCES] Who is in the project team and what are their roles?
Juana Attieh: Principal Investigator, SPO owner who will complete SPO Community feedback
Kenric Nelson, Ph.D.: President of Photrek, Improve design of Risk Metric and oversee execution of the project
James Aman, Ph.D.: Teaching Professor at Rice University, Protocol Security Engineer, design byzantine experiment attacks
Blake Anderton, Ph.D.: Software & Algorithm Developer, configure Risk Monitor for usability with Cardano SPO community
Amir Azem: SPO Specialist, Contribute to experiment design and usability tests
Megan Hess, Communications Lead, Reporting and Administration of Project
Biographies of the Project Team
Juana Attieh, a graduate of Management Engineering from the University of Waterloo, is the co-founder of FLUUS, a solution unlocking Instant Settlements for Emerging Markets. Alongside her role at FLUUS, Juana established LALKUL, a Cardano Stake Pool with a mission to integrate blockchain technology in the MENA region. Furthermore, as a co-founder of the Cardano MENA community, Juana is committed to fostering decentralized governance and contributing towards optimal solutions for self-organizing systems. With her work, Juana seeks to reimagine societies, unlock untapped potential, and provide inclusive opportunities to those who need them most.
Dr. Kenric Nelson is Founder and President of Photrek, LLC which is developing novel approaches to Complex Decision Systems, including dynamics of cryptocurrency protocols, sensor systems for machine intelligence, robust machine learning methods, and novel estimation methods. He served on the Cardano Catalyst Circle governance council and is contributing to designs of decentralized governance. Prior to launching Photrek, Nelson was a Research Professor with Boston University Electrical & Computer Engineering (2014-2019) and Sr. Principal Systems Engineer with Raytheon Company (2007-2019). He has pioneered novel approaches to measuring and fusing information. His nonlinear statistical coupling methods have been used to improve the accuracy and robustness of radar signal processing, sensor fusion, and machine learning algorithms. His education in electrical engineering includes a B.S. degree Summa Cum Laude from Tulane University, a M.S. degree from Rensselaer Polytechnic Institute, and a Ph.D. degree from Boston University. His management education includes an Executive Certificate from MIT Sloan and participation in NSF’s I-Corp program.
Dr. James Aman is a web3 practitioner, researcher, and educator. Since his introduction to blockchain in 2015, he has developed several decentralized applications across various blockchain networks and architected protocols such as Ouroboros Taktikos, a regularized Nakamoto proof-of-stake protocol. James currently serves as a faculty member at Rice University where he teaches an introductory blockchain course for undergraduate and graduate students that combines the technical and business impacts of decentralized ledger technologies. James also co-founded Topl, a novel layer-one protocol for building sustainable and inclusive communities. While at Topl, he led the teams responsible for the engineering and research efforts in implementing the Topl protocol including novel implementations of the key-evolving signature scheme and verifiable random functions required by the Ouroboros protocol.
Dr. Blake Anderton is Founder and President of Acuition Solutions, LLC which develops machine intelligence solutions tailored for customers in operations research, precision agriculture, and environmental science. Anderton specializes in applied remote sensing, autonomous platforms/UAVs, decision support systems, quantitative finance, and human-centric AI. Anderton has 21 years experience in software development across environments spanning embedded systems to cloud computing. He holds a Ph. D. in Optical Science (U. of Arizona), M.S. in Electrical Engineering (U. of Alabama-Huntsville), and B.S. in both Physics and Engineering Mechanics (Lipscomb U.).
Amir Azem holds a Masters Degree in Chemical Engineering with a minor in International Business. With 12 years of experience working in Canada, he later relocated to the Middle East to contribute to private companies in Saudi Arabia and the UAE. While his expertise primarily lies in establishing entities and facilitating business trade, his passion for AI and technology sparked an interest in blockchain. Amir has self-taught himself on Cardano, which has significantly piqued his curiosity. He took his enthusiasm further by launching a Stake Pool called HAWAK, which has been operational on the testnet for over 18 months and on the Mainnet for nearly a year. Beyond his personal ventures, Amir is actively committed to fostering a thriving blockchain community in the Middle East. He aims to raise awareness about various blockchain technologies, support web3 initiatives, and collaborate with other passionate individuals in the region.
Megan Hess Communications Lead at Photrek, serves as the facilitator in Photrek’s Cardano Circle, coordinating initiatives supporting governance and risk intelligence. Megan collaborated with Photrek on their first Catalyst project, Diversifying Voting Influence. She was selected by her Photrek colleagues to represent the contracting team on Photrek’s Advisory Circle. She has a Bachelor’s degree in Physics from the University of Denver and has taught Middle/High School Math and Physics.
[BUDGET & COSTS] Please provide a cost breakdown of the proposed work and resources.
Milestone 1: Stand-up initial Risk Monitor.
- Budget: ₳ 49,700
Milestone 2: Validate operability of Risk Monitor
- Budget: ₳ 46,000
Milestone 3: Validate usability of Risk Monitor
- Budget: ₳ 47,800
Milestone 4: Present Risk Monitor tools and documentation
- Budget: ₳ 55,900
Total: ₳ 199,400
[VALUE FOR MONEY] How does the cost of the project represent value for money for the Cardano ecosystem?
Photrek is led by a five person Leadership Circle that uses consent-based decision making to make sure that our policies, objectives, and execution are consistent with building a collaborative environment for our team, our partners, and our customers. We work to develop Enlightened Pathways that enhance the intelligence and sustainability of the planet. Our approach to corporate governance is specified in the Photrek Sociocratic Operating Agreement.
Towards that end, we utilize competitive pricing methods that provide the highest value to our customers and support our team members with fulfilling lives. Our rates are based on self-employment in the US & Canada. The rates take into account the employment overheads of the resources contracted. The amounts are calculated for each milestone based on the hours to complete. For example the chart for engineering and scientific salaries in the Commonwealth of Massachusetts is provided here: https://www.mass.gov/guides/salary-and-compensation.
