Imagine the peace of mind of being able to connect with a pediatrician within just 15 minutes, around the clock, as a parent. That's exactly what we provide to our caregivers. In this blog post, I will describe how we designed the system to achieve this goal.
The Challenge
Our top priority is to promptly answer any questions or concerns raised by caregivers. However, we understand that our providers may not always be available to respond immediately. This presented us with a challenge: how do we ensure quick response times while accommodating our providers' availability?
The Solution: An Idle Monitoring System
To address this challenge, we developed an idle monitoring system that enhances communication between caregivers and providers. This system operates according to a set of rules that automatically trigger notifications based on the duration of inactivity.
Let's take a closer look at these rules:
- 15-Minute Reminder: After the last message from a caregiver, an SMS should nudge the provider.
- 30-Minute Alert: If there's been no response for 30 minutes, an internal Slack notification should be sent to the operations team, initiating manual backup provider allocation.
- 60-Minute On-call Handover: After a full hour of silence, the on-call provider should receive an SMS and take over the conversation.
The Initial Approach
Our first design utilized Cloud Tasks to schedule tasks. Here's how it worked:
- When the idle window began, we scheduled a task to trigger an event in 15 minutes.
- Once the task timer fired, we checked the conversation's status and notified the provider accordingly.
- We performed a series of checks, such as fetching conversation messages and verifying if the conversation was indeed in the idle window.
- If confirmed, we sent a notification to the provider; otherwise, we skipped it.
- Complex calculations were required to determine when the idle window commenced.
Challenges Faced
However, this initial approach brought about some challenges:
- Task Overload: Multiple tasks were scheduled for various conversations, but most remained unused and ignored.
- Resource-Intensive: Upon task trigger, we had to fetch all conversation messages to determine if the conversation was idle. This proved resource-intensive and inefficient.
The New Design
In our quest to improve our idle monitoring system, we needed an easy way to manage state, timers, and retries. Enter workflow engines, the game-changer that met all our requirements.
The Power of Workflow Engines
1. Simplified State Management
Workflow engines provide isolated environments, simplifying state management. Each workflow operates independently, making state management a breeze.
2. Effortless Timer Management
With workflow engines, timers become worry-free. Isolation ensures timers won't disrupt the application, enhancing precision and reliability.
3. Robust Retry Mechanisms
Workflow engines offer robust retry capabilities, ensuring system resilience. We can handle failures gracefully, maintaining system integrity.
4. Streamlined Code Maintenance
We separate state and timer code into distinct workflows, simplifying code maintenance, organization, and future improvements.
Visualizing Workflow Engine in Action
When it came to choosing a workflow engine for our system, we opted for Temporal due to its dynamic features and user-friendly approach in conceptualizing workflow engines. Let's dive into the fundamental aspects of Temporal that played a pivotal role in our implementation.
Signals and Activities: Powering Workflow Execution
In Temporal, a Signal represents an event triggered by the application,
acting as an input for the workflow. Conversely, an Activity represents
an action that the workflow can execute on the application.
These two elements serve as the backbone of our system,
enabling seamless interaction and coordination between the workflow and the application.
Crafting a Simplified Workflow Interface
To encapsulate the logic of our idle monitoring system efficiently, we crafted a streamlined and uncluttered workflow interface. By keeping things simple, we established a solid foundation for managing the flow and interactions within our system.
Here's an example of how our workflow interface looks:
This straightforward interface acts as a wrapper for the idle monitoring system's logic, ensuring easy management and maintenance.
Streamlining Workflow Logic
Temporal excels in simplifying workflow logic. Within our workflow, we focused solely on tracking state changes and reacting to timers, making the implementation straightforward and efficient. By abstracting the complexities of managing workflow execution and coordination, Temporal empowered us to concentrate on the core functionality of our system.
By harnessing Temporal's capabilities, we created a highly efficient and streamlined workflow engine that powers our idle monitoring system. Easing the implementation process with its user-friendly approach, Temporal allowed us to focus on delivering a reliable and effective solution.
Real-World Applications
Workflow engines have proven to be incredibly versatile, offering benefits beyond just idle monitoring. By incorporating workflow engines into our system, we have not only optimized our processes but also unlocked new possibilities for efficient and resilient system design.
Let's explore some real-world applications that we are currently using:
Long-Running Data Exports: Workflow engines excel in managing complex data export tasks that require substantial time and resources. By leveraging the capabilities of a workflow engine, we can efficiently handle and track the progress of these lengthy data exports, ensuring their successful completion.
Data Quality Audits and Reconciliations: Workflow engines provide a systematic approach to performing data quality audits and reconciliations. With the ability to define and execute a series of steps, workflow engines enable us to validate data integrity, identify discrepancies, and reconcile any inconsistencies quickly and accurately.
Idempotent Multi-Step Transactions: Workflow engines are adept at handling multi-step transactions that need to be executed reliably and idempotently. Whether it's managing subscription and registration processes or coordinating complex financial transactions, workflow engines ensure that each step is executed atomically, maintaining data consistency and preventing any unintended side effects.
Conclusion
By embracing workflow engines, we have not only optimized our existing system but also opened doors to new possibilities in designing efficient and resilient systems. The versatility and flexibility of workflow engines make them indispensable tools in solving complex problems and orchestrating intricate processes.