Design - Notification System

 To design a notification system we first understand the different third party services.

How do these third-party services work?

Third Party Services:

• A provider builds and sends notification request to the required third part service.
• The provider provides the third-party service with device token and the payload.

 

Gathering Contact Information:

 

To send notifications, we need to gather device tokens, phone numbers, or email addresses. To do this, when a user installs our application or signs up for the first time, API servers collect the user contact info and store them in the database.

FUNCTIONAL REQUIREMENTS:

•  Should be able to send notifications.
• Pluggable: Should be extensible to send different types of notification.
• SaaS – Should be built as SaaS product with rate limiting even it is a internal tool. Because we should limit the number of requests from a service.
• Prioritization – We must send notifications on bases of priority, as OTP is high priority whereas a promotional message is a low priority notification.

 

 

NON-FUNCTIONAL REQUIREMENTS:

•  High Availability
• Many Clients – Should be built in such a way that it is easy enough to add more clients.

 

Now let’s deep dive into all the individual components:

 

Notification Service: The notification service handles all the requests from different clients and puts an event to kafka topic. It returns the response to the client like “The notification will be sent in 10secs”. We can also make this a synchronous flow by making a direct API call but as we are focusing on making this a SaaS offering so we don’t want our client to wait.

Validator & Prioritizer Service: This service validates the request for any incorrect parameters like email id or user id should not be null etc. It also priotitizes the message based on some request parameter and then puts the event to a kafka topic.

 

Kafka (2): It has different topics specific for each priority. The consumers will consume the messages based on the priority because we don’t want any lag on the high priority messages.

 

Rate Limiter: It checks whether the client has exceeded the number of notifications for a given time range or not. It can also do request counting if we have pay per every read king of model.

 

Notification Handler Service: A user might have given us some preferences like they don’t want to receive emails, or they have unsubscribed from all the promotional messages. It talks to User preference DB which has all the user preference data and to the User Service which has the user data like name, gender, contact info etc.
It then puts the event in kafka topic as each handler will handle the requests at their own pace.

 

Handler: Each Handler can have multiple vendors for different regions across the globe. All these vendor integrations can be done by the specific handler.

It also has a retry mechanism like if there is some notification failure it puts that request again in the kafka topic.

 

Notification Template: To avoid building every notification from scratch we can use predesigned templates that will be provide consistent and efficient notification creation process.

 

Notification Tracker: We need to keep track of all the notification logs that we have sent for any audit jobs that can take place. As this a write only thing with very few reads so we have used Cassandra.

 

BULK NOTIFICATION:

 

Bulk Notification UI: We need to have a separate UI as it can diff pass request parameters to bulk notification service like “you want to send a reminder message to all users who have bought milk 2 days earlier”.

 

User Transactional Service: It takes data from different service where user is involved in a transaction. All these different transactional data is put into various kafka topics.

 

Transactional Data Parser: It parses all the transactional data as the data can be in multiple formats and then puts that into a ElasticSearch or MongoDB.

 

Query Engine: It generates query based on diff parameters passed by the bulk notification service. It would have its own DSL(Domain Specific Language), which would be a format on how we would want to structure our query. It queries the data store and returns the list of users that match our criteria.

It can also have different consumers like rule engine (“deactivate A/C of all users who have cancelled consecutive 5 orders”) or a fraud engine (“which detects frauds on basis of some criteria”)

 

Bulk Notification Service: It takes the list of all the users that match the request criteria and then puts that into the same kafka topic as of notification service.

 

How to handle scheduled notifications?

Scheduler Service that stores notifications in a database

We can add a Scheduler Service that stores notifications in a database(preferably time series database) along with their scheduled delivery times.

Partition the database

Partition the database by scheduled time so that the system can efficiently retrieve notifications within a specific time range.

Query the database

The Scheduler Service queries the database for notifications that are ready to be delivered.

Put notifications in Kafka

When the scheduled time arrives, the Scheduler Service puts the notification in the kafka topic so it can be consumed by the appropriate handler.