Fix Web3 rabbit holes

[IMPACT] Please describe your proposed solution.

The title of the proposal comes from this paragraph, taken from The Architecture of a Web 3.0 application (preethikasireddy.com) :

"Moreover, the cost of storing the full ... blockchain goes up as your DApp scales, and you need to add more nodes to expand your infrastructure. That’s why, as your infrastructure becomes more complex, you’ll need full-time DevOps engineers. They’ll help you maintain the infrastructure to ensure reliable uptime and fast response times.

All that to say, avoiding these headaches is why many DApps choose to use services like Infura or Alchemy to manage their node infrastructure for them. Of course, there’s a trade-off since this creates a centralized chokepoint, but let’s leave that rabbit hole for another day. ;)"

The proposed solution is a Haskell framework for writing DApps as well as writing blockchain infrastructure which:

1 Could start programs on request and stop them when inactive, If it is inactive, the program will stop and serialize his state to IPFS registers. Upon request, the portion of the program involved in the response would be activated again in this server or any other computer resource, including the computer which initiated the request or another running instance. A execution policy depending on the user/wallet contracts with computer resource providers would decide where it would be executed.

A execution has two elements: a program and an execution path in the program. The first may be stored in the form of source code in a globally accessible repo such is Git or IPFS, It could be a binary executable or a image, which includes the program and all other programs upon which it depends. In the other side the execution path is a set of execution results that can be replayed in the executable to recover the execution state in the same of in other machine. A certain state may be identified as the hash of a point in the program. A certain execution point may not include all the execution path but the execution path since the previous point. if such point is not running the algorithm could be applied recursively until there is a point running or the point 0 is reached in which case the program must be started from scratch and the sum of all the paths applied. This may be called "continuation-path" architecture.

In the other side, a request would contain essentially, the identifier of an execution point, and a REST route which will be the execution path after this point. if the point if not running, will be restored following the algorithm described above before applying the execution path of the request.

2 Could migrate at runtime from node to node. Migration is an special case of the former. Since migration will be a primitive, the programmers would be allowed to create smart contract for buying/auctioning computer resources, so that programs automatically would make use of these smart contracts to locate computer resources to run. This migration also solves the problem of scalability.

3 Run in server as well as Web browsers, The framework will run in both environments, DApps source code could be compiled to run in web browsers. There are Web environments for running haskell code like GHCJS or Asterius. The first has been used in commercial applications and the author has experience building programs that compiles in both server and browser and interoperate seamlessly as if browser programs were first class in the network. Recovery of execution state work fine also in Web nodes. There is also a web framework that includes that as well as widget/forms creation and management (Axiom). The client programs could also be programmed in javascript or any other language and interact with the Haskell code trough HTTP endpoints.

4 Addressed by IPNS unique identifiers which are location and IP independent. To solve the problem of addressing a service even if his location changes, IPNS is the way to identify changing registers with fixed identifiers in the IPFS network. A register can be accessed globally no matter where it was stored. Once the program is relocated or stopped, the program will update the IPNS register, which track the list of locations where the program is running. If there is none, the scheduler (a master program) will try to find suitable resources to run the program, including the requester computer if there is no other resource available for that user. The IPNS register format will be designed so that programmers will define access rights and smart contracts to allow a market of computer resources for running blockchain infrastructure without human intervention. A DNS-like service based on smart contracts will allow to give meaningful identifiers for services (Phase 2). This DNS will be modelized similarly to Forever Domains using Cardano blockchain tecnology.

The program could service network requests trough different protocols: sockets, websockets and HTTP.

A scheduler which would run in each node will be necessary for receiving requests in any computer participating in the blockchain network. This program will receive request for execution but also, in the requester machine, will locate suitable nodes which would answer the request of the user by looking at the IPNS register of the service. if no suitable resource are available and the user included no way to negotiate it with possible resource providers, the local machine could be used to satisfy the request, using the serialized state, stored in IPFS registers, generated by previous executions of the application in other nodes. Note that not all the state of the program is necessary, but only the state necessary for recreating the execution points necessary for satisfying the request.

Note for programming experts: DApps involving smart contracts are inherently stateful; They proceed in steps that imply changes of session states. They also involve different users. This is evident since Plutus has a "state machine" mode to deal with contracts with medium to high complexity. Programming and maintaining such contracts is very difficult. In the heart, this proposal uses the results of F3's "Write Dapps as continuos workflows" (more info) where instead of using state machines, the multiuser workflow is written like a continuous program without "artifacts" and breaks of flow to deal with endpoints, routing, no logic needed to check state transitions, no invalid transitions etc. Distributing the execution of such programs is the goal of this proposal

[IMPACT] Please describe how your proposed solution will address the Challenge that you have submitted it in.

The danger of centralization at dominant IT service providers will be overcome, as the intelligence for migration, failover, scalability, backup, etc., is built into the application. It doesn't need specialized human intervention, except to fix programming errors, since the program now controls the entire life cycle of the application, not just its execution on a particular machine. The dependence on a certain infrastructure or configuration of a certain service provider will be overcome. Infrastructure maintainers will focus on keeping generic hardware and software running, rather than user applications.

Also, the framework uses Haskell, since this language has the tools necessary for this development: such are continuations, pure state and others. This is the language of the Cardano infrastructure. That will support and enhance the Cardano technical approach.

In a further project, some smart contracts and a new coin could be developed for trading computer resources without human intervention as the program goes. We are not talking about storage but also CPU utilization, network usage etc. This would incentívate the adoption of Cardano and Cardano technologies in the computer industry.

The goal is to create and deploy a full stack of robust truly decentralized blockchain software and applications and making it easier than in any other languages. The ideal goal is to package functionalities and facilities that would allow creating and managing DApps in haskell easier than in JavaScript.

Adoption: It is not necessary to adapt the current infrastructure of blockchain software and installations; Even for a single DApp program, the platform will be useful ,since it will discharge load from the DApp servers to the client computers.

[IMPACT] What are the main risks that could prevent you from delivering the project successfully and please explain how you will mitigate each risk?

The most challenging parts of the development is already done: serialization and recovery of thread state from storage is already done to solve the problem of the potential long duration of smart contracts; Smart contracts could execute for long periods of time, so that they must overcome potential stop and restart of the program. This is a result of the F3 project "Write DApps as continuous workflows".

It is however necessary to test this functionality in real scenarios and to create additional infrastructure for IPFS storage, IPNS register definition and job scheduling.

IPFS (used for storage) and IPNS (for identifiers of services) could cause problems of performance. This will be alleviated by caching in the local IPFS node. If remote access to IPFS content becomes critical, there have been improvements in IPFS/IPNS to reduce the response times.

The scheduler is not developed. There is a service for activating and calling applications -containerized or not in docker containers- which work with IP addresses. The goal is to make it work with IPNS addresses.

There is a lack of definition of how best schedule requests: Initially a simple policy of letting each client define the location of some machine (or his own machine) which could serve the request, if there is no IPNS register for that service in the internet. Requests from other users to the same service will offer his own nodes as potential hosting of the application, which will be registered in the IPNS registere. This will create a P2P network without a market. The program can load balance among the different copies of the program thanks to the easy way to communicate between nodes using this continuation-path architecture -explained above- of the platform.

In a phase 2, an specific coin and some smart contracts for auctioning computer resources could be developed to create a market of computer resources.

[FEASIBILITY] Please provide a detailed plan, including timeline and key milestones for delivering your proposal.

Unlike in the previous project in which I put an unrealistic low budget, I will now give a realistic estimation of costs maintaining it as tightly as possible.

I want for one side get paid for the job and in the other side, make sure that the community actually want to have this done and make use of the result of this project, so that I would have the incentives and the community pressure to do it successfully.

The breakdown is as follows:

Phase 1

-------

Optimize thread state serialization 

•  Such functionality has been developed but 
•                     it needs testing and optimization

improve API interface         

• such functionality has been developed but    
• it need improvements:
• Example: support file uploads

IPFS/IPNS interface          

• Using the Web interface of IPFS 

research IPNS register format     

• A table of IPs which serve each service but
• also other metadata to optimize access
• and future contracts for computer resource
• negotiations

Scheduler               

• Development/testing of the IPNS-based 
• scheduler from the current IP-based scheduler

P2P load balancing           

• Development of load balancing among 1 month
• nodes servicing requests for a given user/Wallet

Timeline: A first preview will be presented during the first 8 months to allow programmers to allow evaluate the platform. This will produce valuable feedback to improve the requirements and the robustness of the product.

Phase 2 (Not Included in this funding)

--------  

negotiation of contracts for computer resources

BlockChain DNS

[FEASIBILITY] Please provide a detailed budget breakdown.

Unlike in the previous project in which I put an unrealistic low budget, I will now give a realistic estimation of costs.

I want for one side get paid for the job and in the other side, make sure that the community actually likes to have this done and make use of the result of this project, so that I would have the incentives and the community interest to do it successfully

The breakdown is as follows:

Phase 1

-------

Optimize thread state serialization:         

• 1 month

improve API interface                

• 1 month

IPFS/IPNS interface          

• 2 months

research IPNS register format          

• 1 month

Scheduler                   

• 2 months

P2P load balancing             

• 1 month

Working part-time 4 hours a day one person. This is a 16 months work

( 32 $/h * 4 h/day * 30 day/month * 16 months= 61,440 $ )

[FEASIBILITY] Please provide details of the people who will work on the project.

To start up the project and minimise costs (maximizing the chances of being founded) the "team" will be constituted of only one person.

In next phases not included in this budget, the team could be expanded to one more part time or full time.

Besides the above links, this is my twitter profle: (8) Alberto G. Corona (@AGoCorona) / Twitter

[FEASIBILITY] If you are funded, will you return to Catalyst in a later round for further funding? Please explain why / why not.

For the tasks labeled as Phase two, not included in this work, I could request further funding

[FEASIBILITY] Are you or any member of your team working on any other proposals in this Fund9?

[FEASIBILITY] Are you or your team working on any other proposals from previous Funds?

[AUDITABILITY] Please describe what you will measure to track your project's progress, and how will you measure these?

The project will be opensource, therefore verifiable by anyone

Updates will be published on my github profile and my Twitter profile.

A first prototype during the first half of the project will be liberated to get feedback from the developer community

As a measure of interest It would count the likes of the github project, number of issues, number of forks. Since it is a software development hard mesures are difficult to stablish

[AUDITABILITY] What does success for this project look like?

The goal would be to see 1 or 2 major projects taking advantage of the platform

[AUDITABILITY] Please provide information on whether this proposal is a continuation of a previously funded project in Catalyst or an entirely new one.

This proposals draw results from F3's "Write DApps as continuous workflows" which is intended to finish in September.

As I detailed above, serialization and recovery of thread state from storage is necessary and it is is already done to solve the problem of the potential long duration of smart contracts; Smart contracts could execute for long periods of time, so that they must overcome potential stop and restart of the program. This is a result of the F3 project "Write DApps as continuous workflows".

This functionality could solver a wider range of problems such is migration, scalability, backup and other usually hand-made task of the program lifecycle, and this is the reason of this new proposal.