NuNet: Running OSS ChatGPT alternatives on decentralized network // Enabling cheap and reliable Open Source chatbots alternatives on decentralized hardware supporting independence from big tech solutions.

[GENERAL] Name and surname of main applicant

Ilija Radeljic

[GENERAL] Email address of main applicant

ilija.radeljic@nunet.io

Additional applicants

Avimanyu Bandyopadhyay, PhD candidate

Lead Researcher and systems scientist

Kabir Veitas, PhD

CEO and lead architect

[GENERAL] Please specify how many months you expect your project to last (from 2-12 months)

10

[GENERAL] Please indicate if your proposal has been auto-translated into English from another language.

No

[GENERAL] Does your project have any dependencies on other organizations, technical or otherwise?

Yes

[GENERAL] If YES, please describe what the dependency is and why you believe it is essential for your project’s delivery. If NO, please write “No dependencies.” .

For the development we shall use only open source frameworks as follows: 

• OpenSource ChatGPT alternatives (e.g. Google Flan-T5 Large, Open Llama 7B, etc.)

These technical dependencies do not require permissions to use outside Open Source licensing and are taken into account in our feasibility study. Therefore, integrating these dependencies will not cause any delays.

https://gitlab.com/nunet/ml-on-gpu/ml-on-gpu-service/-/blob/develop/examples/pytorch/flan-t5-large_chatbot_checkpointed.py

https://gitlab.com/nunet/ml-on-gpu/ml-on-gpu-service/-/blob/develop/examples/pytorch/open_llama_7b_chatbot_checkpointed.py

[GENERAL] Will your project’s output/s be fully open source?

Yes

[GENERAL] If NO, please describe which outputs are not going to be open source. If YES, please write “Project will be fully open source.”

Project will be fully open source in line with our licensing policy.

[METADATA] Category of Proposal

dApp

[METADATA] SDG rating

SDG Goals

• 9 - Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation
• 10 - Reduce inequality within and among countries

SDG Subgoals

• 9.1 - Develop quality, reliable, sustainable and resilient infrastructure, including regional and transborder infrastructure, to support economic development and human well-being, with a focus on affordable and equitable access for all
• 9.5 - Enhance scientific research, upgrade the technological capabilities of industrial sectors in all countries, in particular developing countries, including, by 2030, encouraging innovation and substantially increasing the number of research and development workers per 1 million people and public and private research and development spending

[IMPACT] Please describe your proposed solution.

Problem:

The current state of global computing is dominated by a few large infrastructure and AI model providers who control the market by locking users into their computing and data silos. However, most of their models (e.g. OpenAI large language model) is being trained on the public data of Internet users. Consequently , cutting-edge AI and ML technologies are not affordable for most except the largest corporations, which further widens the gap between big players and everybody else, while both raw data as well as training models are publicly available and belong to all humanity. 

Unique solution:

A decentralized computing infrastructure of NuNet, powered by Cardano transaction and settlement layer, can provide a unique solution to these problems. NuNet is building a globally decentralized computing framework that combines the latent computing power and storage of independently owned compute devices across the globe into a dynamic ecosystem of compute resources. The application of this platform will democratize access to AI and ML technologies and trained models and unlock the potential of AI development to all humanity rather than selected bid tech players.

Detailed approach:

1. Decentralized Computing Infrastructure: NuNet's decentralized computing framework with integration with Cardano blockchain creates decentralized infrastructure for deploying AI models and running inference on them. This infrastructure would leverage the latent computing power of devices across the globe, creating a dynamic ecosystem of compute resources.
2. Tokenomic Ecosystem: The infrastructure would be supported by a tokenomic ecosystem based on the NuNet Utility Token (NTX). This would provide individual rewards for the provision of computing resources, data and incentivizing participation in the network.
3. Open-Source Platform and Protocols: The development of open-source platforms, protocols, and APIs is tailored to run complex geographically distributed computing workflows that would facilitate the deployment of AI models. Blockchain integrations would ensure the security and transparency of the system.
4. Conversation Saving: Conversation saving can be enabled for AI models deployed on this infrastructure, enhancing their performance and utility. This feature can be particularly useful for language models like Google Flan-T5 Large and Open Llama 7B, which have been successfully tested on NuNet.

Proposed use of funds: 

The funds would be used for the development and maintenance of the decentralized computing infrastructure, including the creation of open-source platforms and protocols, blockchain integrations, and the implementation of the tokenomic ecosystem. Funds would also be allocated for testing and optimizing training and storage of open source AI models on the infrastructure, as well as for community engagement and expansion efforts.

Proposed structure:

The proposed structure would involve deploying open source AI models on a decentralized network of independently owned compute devices, integrated with the Cardano blockchain and managed through NuNet's computing framework. The tokenomic ecosystem would be based on the NuNet Utility Token (NTX), and the system would be governed by open-source platforms, protocols, and APIs.

Benefits for the Cardano ecosystem:

The integration of a decentralized computing infrastructure on Cardano would bring several benefits to the Cardano ecosystem. It would democratize access to AI and ML technologies, allowing individuals, researchers, and SMEs to affordably develop and apply new technologies. It would also unlock the potential of dormant computing resources, creating a dynamic ecosystem for compute resources. Furthermore, it would enhance the utility and performance of AI models deployed on the Cardano blockchain, contributing to the advancement of AI research and development in the Cardano ecosystem.

[IMPACT] How does your proposed solution address the challenge and what benefits will this bring to the Cardano ecosystem?

The proposal addresses the following directions of the challenge:

• Business solutions - Software products, data management, process management, data management solutions (CRM, ERP etc), privacy products.
• Artificial intelligence

Introduction:

The NuNet platform, as outlined in the proposal, currently offers an infrastructure for machine learning computations. However, the potential of this platform can be significantly expanded by integrating an additional option for training an inference Open Source machine learning and AI models and enabling their free access. This would allow the platform to cater to a wider range of use-cases, and radically increase broad access to differently trained AI as well as Large language models (LLM).

Impact on the Cardano Ecosystem:

The integration of tools and technologies for training, inference, hosting and monetizing Open Source language models and chatbots into NuNet platform can have a transformative impact on the Cardano ecosystem. The potential influx of users would not only increase the number of transactions but also strengthen the Cardano community by bringing in new perspectives and ideas.

The proposed solution addresses a key challenge in the Artificial Intelligence and Cardano ecosystems: the need for a more versatile computational platform that can democratize access to cutting edge AI solutions. By offering a solution that can handle both machine learning and non-machine learning computations, the platform can become a one-stop solution for researchers, thereby solving this issue.

Benefits:

The proposed solution addresses the AI challenge by democratizing access to AI and Machine Learning technologies and unlocking the potential of dormant computing resources. By integrating a decentralized computing infrastructure on Cardano, powered by NuNet, we can create a dynamic ecosystem for compute resources that is accessible to all, not just large corporations. This approach can help to bridge the gap between big players and everybody else in the field of AI and ML.

The impact on the Cardano ecosystem would be significant.

Firstly, the integration of a decentralized computing infrastructure on Cardano would bring a new level of functionality to the blockchain, enabling it to support complex AI models and workflows. This would enhance the utility of the Cardano blockchain and could attract more developers and users to the ecosystem.

Secondly, the proposed solution would unlock the potential of dormant computing resources, creating a dynamic ecosystem for compute resources. This would incentivize participation in the Cardano ecosystem, as individuals could be rewarded for providing computing resources through the tokenomic ecosystem based on the NuNet Utility Token (NTX).

Thirdly, the proposed solution would democratize access to AI and ML technologies, allowing individuals, researchers, and SMEs to affordably develop and apply new technologies. This could lead to an increase in innovative projects and applications being developed on the Cardano blockchain, further strengthening the ecosystem.

In terms of quantifiable impact, it's challenging to provide exact figures without more detailed analysis. However, given the rapid growth in AI and ML technologies, with the global AI market projected to reach USD 360.36 billion by 2028, and the increasing demand for decentralized computing resources, as evidenced by the edge computing market expected to grow to USD 111.3 billion by 2028, we believe that the proposed solution could attract a significant number of new users and developers to the Cardano ecosystem. This could potentially lead to an increase in the number of transactions on the Cardano blockchain, contributing to its growth and development. The integration of these high-growth sectors with the Cardano ecosystem could also enhance its visibility and reputation in the broader technology and business communities, attracting further investment and innovation.

[IMPACT] How do you intend to measure the success of your project?

Quantitative Metrics:

1. User Adoption: A key sign of success would be the influx of new users and developers drawn to the open-source chatbot alternatives as a result of the initiative. This can be quantified by monitoring the growth in user base, the number of unique visitors to the platforms, and the number of developers utilizing the infrastructure for deploying AI models.

2. Transaction Volume: Track the number of transactions on the platform. An increase in transaction volume would indicate a higher usage of the platform.

3. Successful Deployments: A surge in the number of interactions on the open-source chatbot platforms would signify a higher degree of activity and engagement within the ecosystem, suggesting that the initiative has been successful in attracting users and developers.

Qualitative Metrics:

1. User Feedback: User feedback and reviews can offer valuable subjective insights into the success of the initiative. This could include surveys, interviews, and user reviews, which can help to assess user satisfaction and pinpoint areas for enhancement.

2. Community Engagement: The degree of community involvement can be quantified by observing activity on AI forums, social media, and other community platforms. This could include the number of posts, comments, likes, shares, and followers, as well as the community's sentiment towards the initiative.

3. Innovation and Development: The number of innovative projects and applications being developed as a result of the initiative can also serve as a measure of success. This could be tracked by monitoring the number of new projects, the amount of funding secured, and the impact of these projects on the chatbot landscape and beyond.

By monitoring these metrics over time, we can gain a comprehensive understanding of the initiative's impact on the open-source chatbot landscape, both in the short and long term. These measures are realistic as they are based on observable and measurable indicators, and they consider both the numerical and subjective aspects of the initiative's success.

Some of the direct benefits to the Cardano ecosystem are:

• Number of projects using cheaper GPU resources for Open Source ChatGPT alternatives
• Computing resources used in the processes are to be compensated in NTX, which is a Cardano Native Token
• Each exchange of value will be done as a Smart Contract on Cardano
• Currently over 2000+ people are in NuNet Discord testing the various builds of the NuNet platform

Some of the indirect benefits to the Cardano ecosystem are:

• Cardano becomes the settlement layer for decentralized Open Source computing frameworks used in training Open Source ChatGPT alternatives
• Other solutions can be built on top of the framework, greatly expanding the potential business models
• With the right onramp/offramp solutions Web2 users can utilize compute power without even realizing the Web3 layer underneath. NuNet is interested in joint work with the experts in this field.

[IMPACT] Please describe your plans to share the outputs and results of your project?

Spreading Outputs Over a Timescale

Our project plan includes clear milestones and deliverables, which will be shared publicly as they are completed. This incremental release of outputs will ensure a continuous stream of updates for the community.

This approach lets us provide updates on a regular basis, and offers users the chance to provide feedback that we can use to guide subsequent development.

Sharing Outputs, Impacts, and Opportunities

We intend to leverage various communication channels to share our project's outputs, impacts, and opportunities:

1. GitLab: The primary hub for our technical work, hosting our codebase, documentation, and issue tracking. This will be the main point of reference for the details of our project.
2. Social Platforms: We plan to regularly post updates on our progress on platforms like Twitter, LinkedIn, and Reddit. This will include major milestones, bug fixes, and insights from our development work.
3. Technical Discussions: We will continue to hold weekly technical discussion where we discuss the technical aspects of our work. This provides a forum for live Q&A and discussion with our community.
4. Blogs: A regular blogs to summarize the progress we have made, highlighting key achievements and outlining the next steps in our project.

Testing and further research

As an open-source project, our outputs will be freely accessible for further research and development. We encourage the community's involvement in testing our solutions to enhance their real-world performance.

Community Testing: We'll invite our users to participate in alpha and beta testing phases, where they can help identify bugs and suggest improvements. We'll use GitLab's issue tracking for managing feedback and provide guidelines for issue reporting and feature suggestions.

Internally, we'll use project insights and community feedback to guide our future work, optimize performance, and prioritize new features. Our aim is to foster a collaborative development ecosystem that is robust, relevant, and of high quality.

[CAPABILITY/ FEASIBILITY] What is your capability to deliver your project with high levels of trust and accountability?

NuNet has a proven track record in developing and managing complex technological projects. As part of the SingularityNET ecosystem, NuNet supports the decentralized operation of their global AI ecosystem. NuNet has also successfully finished Fund8 that facilitates a decentralized GPU ML Cloud for the open web. This experience and expertise make NuNet well-equipped to handle the complexities of this project.

Our team is populated with seasoned software engineers with excellent skills to leverage and deploy open source AI models in a decentralized manner. NuNet past work includes the implementation of projects similar to the one proposed here, showcasing our readiness to tackle the unique challenges this project poses.

NuNet is committed to Open Source Software development from the inception. Therefore, all our development and progress is available for public scrutiny at all times as well as open collaboration with the community. We actively invite and work with the community in regards to contribution, usage, work and testing of the platform codebase.

Link: https://gitlab.com/nunet

NuNet licencing policy:

https://docs.nunet.io/nunet-licensing-policy/

Openness and Responsibility:

We have established a robust framework to ensure openness and responsibility in the execution of the project and the management of finances:

1. Elaborate Budgeting: We present an exhaustive budget layout at the start of the project that details the fund allocation across various tasks. This leaves no room for ambiguity regarding the utilization of funds.

2. Periodic Reporting: Regular updates regarding the project and financial statements will be shared, offering complete transparency into the progression of the project and the use of funds.

3. External Auditing: We are open to audits conducted by independent third parties at regular intervals. This ensures responsibility and openness in our financial management.

4. Escrow Mechanisms: To further reassure proper use of funds, we can utilize an escrow service. This arrangement ensures that the project funds are held by a third party and released according to pre-set milestones. This provides an extra layer of assurance for the funds.

5. Payment Based on Milestones: Our payment structure is built around specific, agreed-upon milestones. This ensures that funds are released as we achieve these milestones. The completion of each milestone can be verified, ensuring you pay only for verifiable progress.

These measures reflect our commitment to openness, responsibility, and proper management of funds. We believe that these factors, along with our technical capabilities, make us an ideal choice to successfully execute this project.

We understand that not all steps we have implemented are valid for the Catalyst proposal but it demonstrates the internal working procedures we have in place.

Catalyst Experience

NuNet also has received the funding for proposals in Fund7 and Fund8. One proposal is successfully closed and the other is close to completion with one technical obstacle left to be solved. Overall, the funds were spent as intended on the development which can be monitored on Gitlab with daily commits since the award.

https://gitlab.com/groups/nunet/-/milestones/19#tab-issues

https://gitlab.com/groups/nunet/-/milestones/20#tab-issues

Financial Stability

As a 28+ strong team, we have independent funding to develop the core platform with a cash runway for at least 1-1.5 years. Cardano Catalyst proposals are used to extend the functionality and add features to the platform in order to enrich the possible use cases.

The financial report is publicly available and can be reviewed here:

https://medium.com/nunet/nunet-financial-report-2022-and-outlook-for-2023-405d38397629

[CAPABILITY/ FEASIBILITY] What are the main goals for the project and how will you validate if your approach is feasible?

Main goals:

1. Democratize Access to AI and ML Technologies: One of the primary goals of the project is to democratize access to AI and ML technologies. By creating a decentralized computing infrastructure on Cardano, we aim to make these technologies accessible to all, not just large corporations.

2. Unlock Dormant Computing Resources: Another key goal is to unlock the potential of dormant computing resources. By incentivizing individuals to contribute their computing resources to the network, we aim to create a dynamic ecosystem of compute resources.

3. Enhance the Cardano Ecosystem: The project also aims to enhance the Cardano ecosystem by integrating a decentralized computing infrastructure. This would bring a new level of functionality to the blockchain and could attract more developers and users to the ecosystem.

4. Promote Innovation and Development: Finally, the project aims to promote innovation and development within the Cardano ecosystem. By democratizing access to AI and ML technologies, we hope to encourage the development of new projects and applications on the Cardano blockchain.

To validate if our approach is feasible, we will use a combination of quantitative and qualitative measures. These could include:

1. User Adoption and Transaction Volume: We will track the number of new users and developers attracted to the Cardano ecosystem, as well as the number of transactions on the Cardano blockchain. An increase in these metrics would indicate that our approach is successful.

2. Community Engagement: We will measure the level of community engagement by tracking activity on forums, social media, and other community platforms. This will give us an indication of the community's interest and involvement in the project.

3. Feedback and Reviews: We will seek feedback and reviews from users and developers to gauge their satisfaction with the project and identify areas for improvement. This will provide valuable qualitative insights into the success of the project.

4. Innovation and Development: We will monitor the number of innovative projects and applications being developed on the Cardano blockchain as a result of the project. This will give us an indication of the project's impact on innovation and development within the Cardano ecosystem.

By tracking these metrics over time, we will be able to validate if our approach is feasible and if we are achieving our project goals.

[CAPABILITY/ FEASIBILITY] Please provide a detailed breakdown of your project’s milestones and each of the main tasks or activities to reach the milestone plus the expected timeline for the delivery.

Milestone 1: Project Commencement and detailed architecture design

• Main Tasks / Activities: 
• Forming the squad, building communication platforms, conducting weekly brainstorming sessions for comprehensive understanding of the Cardano universe and its latest computational offerings, identifying the prerequisites for a Open Source ChatGPT model training, inference and hosting, drafting a comprehensive project blueprint and architecture design, and conducting a practicability study based on the specified prerequisites and available resources.
• Expected Delivery Time: 5 Weeks

Milestone 2: Implementation of detailed architecture

• Main Tasks / Activities: 
• Implement functionality to train, host and deploy Open Source AI models (e.g. language models like Google Flan-T5 Large and Open Llama 7B) via NuNet platform;
• Implement mechanisms of monetizing different models and sue them in independent chatbot implementations on the network;
• Implement ability for community members to perform iterative learning based on existing models and propose offer customized and adjusted models for border community;
• Augment all components of the platform as needed and implement additional features as defined in architecture design; 
• Expected Delivery Time: 20 Weeks

Milestone 3: Internal, community testing and resolving additional issues

• Main Tasks / Activities: 
• Internal testing on closed testing network;
• Large scale community testing on testnet/preprod;
• Resolving bug reports and implement identified additional features;
• Expected Delivery Time: 4 Weeks

Milestone 4: Production release

• Main Tasks / Activities: 
• Release of the tested solution and features on the Cardano mainnet;
• Expected Delivery Time: 3 Weeks

Milestone 5: Technical documentation and result dissemination

• Main Tasks / Activities: 
• Writing documentation 
• Dissemination of results in the community
• Expected Delivery Time: 2.5 Weeks

Milestone 6: Community Engagement and Expansion for diverse AI models

• Main Tasks / Activities: 
• Engage with the Cardano community, solicit feedback and input, conduct AI webinars and workshops, initiate expansion efforts to attract more users and developers.
• Expected Delivery Time: 5 Weeks

[CAPABILITY/ FEASIBILITY] Please describe the deliverables, outputs and intended outcomes of each milestone.

Milestone 1: Project Commencement and detailed architecture design

• Deliverables: 
• Project blueprint, general architecture and applied research, practicability analysis, project roadmap, defining risk management strategy and resource allocation plan.
• Outcomes: 
• Detailed documents of all strategies, including a timeline, resource distribution, and dependencies.

Milestone 2: Implementation of detailed architecture

• Deliverables: 
• Updated platform components and their functionalities as needed for training, inference and hosting Open source ChatGPT alternatives;
• Outcomes: 
• A functioning decentralized computing infrastructure that is integrated with the Cardano blockchain and supported by a tokenomic ecosystem. This infrastructure will be ready for testing and optimization. 

Milestone 3: Internal, community testing and resolving additional issues

• Deliverables: 
• Comprehensive testing reports, optimized infrastructure, tested AI models; Bug fixes and new features based on the community testing and reports;
• Outcomes: 
• A fully tested and optimized infrastructure that is ready for use. AI models that have been successfully tested on the infrastructure and are ready for deployment.

Milestone 4: Production release

• Deliverables: 
• Deployment of the finalized and tested system on the Cardano mainnet.
• Outcomes: 
• A live, functioning system on the Cardano mainnet enabling community developers to train and host their versions of OSS ChatGPT alternatives, application developers to integrate them into their business models and customers to request inference based on them.

Milestone 5: Technical documentation and result dissemination

• Deliverables:  
• Prepared comprehensive technical and usage documentation for users and community developers;
• Outcomes: 
• Users are trained and informed on how to effectively use the system and train, deploy and use Open Source ChatGPT alternatives and models, improved ability to develop full applications involving data access for models as well as host the models themselves on decentralized hardware, increased number of NTX settlement transactions on Cardano blockchain and in general.

Milestone 6: Community Engagement and Expansion for diverse AI models

• Deliverables: 
• Community engagement activities (e.g., webinars, workshops), feedback reports, expansion plans.
• Outcomes: 
• Increased engagement and participation from the Cardano community, valuable feedback and input that can be used to improve the project, plans for attracting more users and developers to the ecosystem.

Each milestone’s progress will be tracked through the completion of the stated expected results and the achievement of the anticipated impact. Regular project update meetings and reports will provide visibility into the project's progress, and any issues or delays will be addressed through the project's risk management process. The overall project management methodology will be agile, with regular sprint planning, daily stand-up meetings, and retrospective meetings. Key performance indicators will be defined to track the progress and success of the project. The team will regularly communicate with stakeholders and the Cardano community to keep them updated on the progress and gather feedback.

[RESOURCES & VALUE FOR MONEY] Please provide a detailed budget breakdown of the proposed work and resources.

Each project is examined in great detail which can be seen in the proposed budgeting sheet. This results in pre-feasibility and feasibility studies which minimize the risk of budget overruns.

Project management in NuNet is on a high level with employed techniques such as Agile, Scrum, CCPM and others resulting in a good daily overview of the project progress.

The project is complex and involves research and development uncertainties however, NuNet is a well funded deep tech startup and in case of budget overruns will continue to develop until delivered due to this being a critical part of the overall NuNet development plan. This is evidenced by the funding received in Cardano Catalyst Fund 7 and 8 where NuNet has continued the work despite the substantial unexpected technical roadblocks and time impact.

[RESOURCES & VALUE FOR MONEY] How does the cost of the project represent value for money for the Cardano ecosystem?

The costs of the project are based on the average salary levels of engineers currently employed by NuNet. Since the team is fully distributed and remote, it is challenging to have a suitable median cost that covers the range of countries (India, Pakistan, Ethiopia, Brasil, Egypt, UAE, UK, Italy and others).

We believe that the costs are reasonable and reflect the seniority and knowledge of various positions involved in the delivering of the proposal.

In line of full openness, in the budget table can be seen the very granular distribution of costs, all the way to the hours of each position for each milestone.

In addition, fully remote workers can compete for jobs in Western countries driving the individual compensation levels much higher than in their native countries.

[IMPORTANT NOTE] The Applicant agreed to Fund10 rules and also that data in the Submission Form and other data provided by the project team during the course of the project will be publicly available.

I Accept