Transaction monitoring as a service

[IMPACT]

Need for this solution:

Every dApp on Cardano has use cases to make transactions with ADA or native assets. 

Once a transaction is initiated it gets added to the mempool and from then there is a delay by which the transaction is on chain and gets added into a block accepted by the network. In such situations the dApp is waiting for an update on the transaction. 

Common approaches used by dApps to get an update on transaction status are

• Periodically query the blockchain with the transaction id to get the status.

• Ask the end user to manually check the status of the transaction and perform some action on the dApp to initiate the query to the blockchain.

1. Both the approaches above are inefficient. 

The first approach, thought it employs an automated process has the following disadvantages:

• The time for confirmation depends on various factors like network congestion and size of the transaction. By the time it's on chain a dApp might have already done multiple queries to the blockchain without any result. This is inefficient. Also, even when the tx is confirmed that dApp does not get an update till it does the next query.
• The dApp developer has to develop the component for polling the blockchain along with their dApp. This increases development and maintenance cost of the dApp.

The second approach avoids periodic query to the blockchain but it comes with the trade off of

• Bad user experience as the user needs to manually check transaction confirmation and perform actions on the dApp.
• Larger delays if the user is not able to wait till the tx is on chain. We have already seen an hour-long waiting period for ADA transactions when network congestion is at its peak. Further if the dApp performs a smart contract based transaction then waiting period can be longer due to queuing of transactions (eg: Swaps on DEXes) and larger transaction sizes (large tx may not fit into the current block).

2. Though there are different ways to query the blockchain, dApps usually use a third party service to save on integration costs and related code on the dApp side. If the dApp needs to switch to a different query service in future for considerations like performance or cost, additional development work might be needed on the dApp side.

This solution is to solve the above drawbacks.

How it works:

1. DApps subscribe to this transaction monitoring service and configure properties for receiving real time push updates from the service.
2. Registered dApps submit the transaction ids to the monitoring service over HTTP APIs. 
3. Once the tx status is updated on the blockchain the monitoring app publishes the message to the dApp that submitted the transaction.
4. DApps and teams can create multiple communication channels with the monitoring service. They can choose their preferred set of querying services to be used for better resiliency.

Please see this diagramatic illustration also.

Benefits of this solution:

1. Transaction monitoring as a service for dApps to use to get updates about their transaction. DApps avoid duplicated work of periodically querying the blockchain and save effort in writing their own code for it. This service will offer Webhook based communication so that the dApps can receive updates without having to initiate a query. The service reduces what can be a few weeks' work to just a few minutes for dApp dev teams. If you would like to read about webhooks, please read here: https://webhook.net/?gclid=Cj0KCQjwz7uRBhDRARIsAFqjuln4DfhVaVWM4fruSXls_mDOiZc8ZTyfNuNS29GviFE0lxlx5qS2dJEaAvAmEALw_wcB
2. Aggregation and abstraction of blockchain query services (Blockfrost, Dandelion APIS from Gimbalabs etc) so that dApps can pick and choose services behind the scenes for accessing blockchain data and switch among such services without refactoring the dApp. 
3. Cost optimization for dApps by sharing paid services. Many blockchain query services offer paid plans when significant usage is involved. Abstracting the query will optimize cost for each dApp that requires paid plans by sharing costs across a larger pool of dApps. 
4. Better scalability through abstraction. New blockchain query services that may provide better cost and performance may arrive in the market and by delegating the integration mechanism to the monitoring service dApp developers can pick or switch services with just a configuration change. 

We have highlighted savings through better time efficiency for tracking transactions enabled by proactive call back to the dAapp as well as better cost efficiency of dApp development by avoiding duplicated development work. There are 500+ projects being developed in Cardano today and will likely have thousands more especially once the scaling solutions are implemented. The time & cost savings that can be achieved as a whole for the ecosystem of dApp on Cardano with this solution is big.

Use cases:

Usable in all cases where a transaction is to be tracked by a dApp. In the case of marketplaces where users make purchases with ADA, the marketplace backend should ideally check that the transaction has been confirmed before issuing the purchase (NFT, native tokens or other items). DEXes can use this solution for more efficient transaction monitoring to improve their transaction pipelines. Similarly token faucets and other dApps can also make use of this service.

Design considerations:

• Best practices for large scale application platforms need to be followed in the design of this service. Modularising components will allow to compose and maintain different modules as well as integrations of third party query services.
• Horizontal scalability of throughput within dApps and across dApps will be ensured by adopting service oriented architecture. Deployment models will be akin to what’s used for micro services.
• We will choose a modern tech stack, widely adopted by the IT industry for implementation.
• For quality assurance each module will be unit tested with automated tests during the course of development. During the integration phase of modules end to end integration tests will be used for ensuring quality. Additionally, static code analysis wherever relevant will be employed.

Sustainability

• Hardware costs for a year have been factored into the budget. 
• We will provide support and maintenance for the considerable future.
• If need for new functionalities is discovered funding through future Catalyst funds might be requested. 
• A platform fee to sustain the running costs is expected only after a year or more after the service is launched and even then would be managed without charging dApps with small usage.

Future scope:

• Integration with new services that read data from the Cardano blockchain. 
• Further cost optimization for dApps using this service by usage metric driven resource allocation for each dApp. This is useful but requires a larger effort to implement. We would like to keep such finer optimisations outside the scope for now to manage the delivery timeline.
• Enhance webhook based communication to allow dApps to define conditions in which notifications about transactions are to be sent.

The outcome of this project is to provide service to dApps for monitoring on chain status of transactions, improve time efficiency of dApp operation through proactive call backs, avoid duplicated development effort for dApps and reduce costs by pooling of resources across dApps.

We believe the outcome suits the intent of this ‘Developer Ecosystem’ challenge.

The guiding question of this challenge that this project aligns to is ‘What are enterprise dev managers looking for in order to be able to build out enterprise projects - either internal or commercial?’

The key metric of this challenge is to check whether it has made it easier for developers to build on top of Cardano? This project of ours, like many other good proposals, will do exactly that by helping achieve better productivity and costing for their dApp development on Cardano.

Health challenges like Covid can reduce availability of human resources and delay our delivery timeline.

[FEASIBILITY]

The development (and launch) timeframe will span 4 months divided into 8 sprints. As indicated previously, the components will be modularised. This allows for parallely development and testing of various modules in the first two months and subsequently composition to form the larger service in the third month. Depending on when the voting results are announced we may be able to start off in May/June.

High level architecture is as per the diagramatic illustration. We will prepare a formally designed and reviewed architecture.

Roadmap

Sprints 1 - 2 : (expected June 2022)

• System architecture. Finalize tech stack.
• UI mockups and design 
• Develop authentication modules
• Short list services that will be interfaced with for reading data from Cardano blockchain.

Sprints 3 - 4 : (July 2022)

• Develop user interface for the dApp facing side of this solution.
• Develop Integration modules for interfacing with blockchain query services.
• Develop subscription modules for dApps to communicate to (dApp side endpoints).
• Develop the webhook module
• Code review and unit testing of modules.

Sprints 5 - 6 : (August 2022)

• Middleware layer for managing data flow and queues.
• Develop the webhook module.
• Integrate modules. Develop integration tests for testing end to end flows.
• Set up hosting environment (Dev Ops)
• Deploy on testnet
• Simulate end use (dApp) usage with manual test cases.

Sprints 7 - 8 : (September 2022)

• Deploy on mainnet
• Publish repository with user guide.
• Publicity by means of social media, blogs, Catalyst Town Hall events.
• Integrate with one of the other Cardano projects our extended team is working on.

~ October - December 2022

• Assist teams to register and use the service. 
• Maintenance and support activities.

2023 +

• Assist dApp teams, routine maintenance.
• Interface with new blockchain services.
• Drive enhancements with the help of Catalyst/community contribution.

Budget

The following are our expenses:

• 1 Technical Architect for 2 months for system design and review ($5000 x 2) = $10000
• 2 Backend developers for 3 months ($4000 x 2 x 3) = $24000
• 1 Frontend developer for 2 months ($3000 x 2) = $6000
• 1 DevOps engineer, part time for 2 months ($1500 x 2) = $3000
• Tech infrastructure for Web hosting, middleware infra, GCP etc.: $6000

Total = $49000

We are a team of experienced Software Engineers, Plutus and Prism Pioneers, Gimbalabs PPBL member, members of a delivered Fund 6 project, funded proposer from Fund 7 as well as an ambassador in the Cardano ecosystem. Notable projects on Cardano where our teammates are part of the development team are:

• Adatar.me ( https://adatar.me/ ) funded through Catalyst in Fund 6
• Unsigs NFT marketplace ( https://unsigs-frontend.vercel.app/ ) through Gimbalabs project based learning initiative.

We have been building in Cardano for close to a year. We have developed various open source repositories for Plutus/Cardano which can be found in our Github repo added in this proposal.

Reshma Mohan

• Professional Software Engineer
• Plutus Pioneer (1st cohort)
• Prism Pioneer (currently undergoing)
• Bachelor of Technology degree in Electrical and Electronics Engineering
• Ambassador to ‘Genius Yield’.
• Proposer of multiple funded proposals in Fund 7.

LinkedIn: https://www.linkedin.com/in/reshmhn/

Twitter: https://twitter.com/reshmhn

Karthik Vishwanathan

• Lead Software engineer with 11 years of experience.
• Bachelor of Technology degree in Computer Science
• Ex engineer at Amazon
• Demonstrated history of building software for e-commerce, travel & tourism industry.

LinkedIn: https://www.linkedin.com/in/karthik-viswanathan-90614052

Ganesh N

• Professional Software Engineer for over 8 years
• Plutus & Prism Pioneer (1st cohorts)
• Gimbalabs Plutus PBL Team Member
• Bachelor of Technology degree in Electronics Engineering
• Has lead development of large scale applications
• Experience in the financial services industry

Github: https://github.com/gannith

[AUDITABILITY]

We have illustrated the solution in previous sections. Progress of the development will be measured by tracking the milestones listed in the roadmap.

Agile methodology will be adopted for the development. Each module to be developed will be mapped to its work breakdown structure and assigned to sprints spanning 2 or 3 weeks.

Our success metrics are as follows.

In 1 month:

At least 2 dApps using the service on testnet.

In 3 months:

At least 2 dApps using the serivice on mainnet and 10 dApps on testnet using the service.

In 6 months: 

At least 10 dApps using the service on mainnet.

In 12 months: 

Wide adoption as the de facto solution for transaction monitoring for dApps.

KPIs

• Number of projects registered for the service.
• Number of projects using the service on mainnet.
• Number of unique query services chosen by teams to read transaction status from Cardano blockchain.

This is a new proposal.

SDG Rating

SDG goals:

Goal 8. Promote sustained, inclusive and sustainable economic growth, full and productive employment and decent work for all

Goal 9. Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation

SDG subgoals:

8.2 Achieve higher levels of economic productivity through diversification, technological upgrading and innovation, including through a focus on high-value added and labour-intensive sectors

9.4 By 2030, upgrade infrastructure and retrofit industries to make them sustainable, with increased resource-use efficiency and greater adoption of clean and environmentally sound technologies and industrial processes, with all countries taking action in accordance with their respective capabilities

Key Performance Indicator (KPI):

9.4.1 CO2 emission per unit of value added

#proposertoolsdg