Why do we use different types of databases for an online service
The intended audience of this article is non-technical directors and managers who’ve been wondering why reliable online services and platforms use different types of databases.
Applications fetch and store data in databases. Even your website’s content management is very likely using a database to store content and user information. When designing and building highly available services for web and mobile platforms, we often utilize more than one type of databases, each specialized for a specific kind of data storage.
Online services that we use every day consist of Client apps such as the ones working on your smartphones, browsers, and desktop applications, and Server apps that are running in the cloud. Software architects call this a Client-Server architecture, where the client apps deliver the user experience (UX) while the server apps do the heavy work of processing, storing, fetching, and serving large sums of data and files.
Serverside databases most commonly manage three types of data: documents, live user sessions, and everything else.
In the past, we used relational databases for all three data types, but when building online services and platforms with High Availability (HA) architecture, we need more specialized data storage tools. For our Software As a Service (SAAS) projects, we favour using NoSQL databases for storing and managing documents and Im-Memory Key-Value databases for storing live user sessions.
Storing & Managing Documents
Relational databases such as MySql and PostgreSQL are most commonly uses in many web applications and services. We call them relational because they store data in tables, which is convenient if you are storing a collection of well-defined attributes and using query languages to fetch the data you need. We usually obtain user data using forms in browsers and mobile apps, and if the forms have a relatively small set of fields, they are easy to manage in tables. Now imagine if a form contains more than a hundred fields such as an assessment worksheet, electronic invoice, or medical report. In those cases, mapping each form field to a table field and keeping track of them could add a great deal of complexity and performance issues in the application.
A NoSQL database is, however, quite suitable for such scenarios because it stores the form attributes as a document object. Not only that, but a NoSQL database also keeps track of revisions every time the user edits a document. NoSQL databases can form an ad-hoc network and synchronize their data with each other. This concept opens up many opportunities for designing systems that are reliable and globally accessible. MongoDB and CouchDB are two of our favourite NoSQL databases that are now common in the industry.
If you keep reading, this article tells you about a different type of NoSQL database PouchDB that is available on all modern browsers, mobile devices, and cross-platform desktop apps. PouchDB easily connects and synchronizes data with a CouchDB instance in the cloud.
Storing Live Sessions
User sessions are used to help applications recognize you the next time that you use the app. Sessions are often unique tokens that the server application creates and hands out to every user that logs in to use an application. The server app stores the sessions in the file system or database. It is possible to store user sessions in a relational database, in fact, commonly used content management systems such as WordPress, store user sessions in the same database where the rest of the user data and content resides.
In HA platforms, where a large number of people are using your service online, there are two main issues that we need to consider: persistence and performance.
We want a database that holds on to the user session even if the application is updated or restarted. We also want the process of reading and writing user session to happen as fast as possible, which is essential for superior user experience. For this purpose, we use an in-memory key-value database. In memory refers to the fact that such a database runs in the server’s Random Access Memory or RAM, which is the fastest memory on the machine. A key-value or dictionary database stores data as key-value pairs, for example:
- name=morgan
- [email protected]
- city=vancouver
Our favourite in-memory key-value database is currently Redis, which is a commonly used database of this category in the industry.
Another common usage of Redis is for caching data and improving the performance of a web application in the cloud.
Databases in Mobile and Browser Apps
Having a database in the client app is particularly important when users don’t have a network connection at all times, but still need to use the application. The user might be a field researcher, inspector, or medical practitioner within an air-gapped facility that doesn’t allow connecting to the world wide web for security reasons. For this type of users, we develop offline-first clientside apps.
Clientside apps have a database for when the app needs to work offline, in which case the app temporarily stores the user data on a lightweight database on the mobile device, browser, or desktop application until an internet connection becomes available again. Once the device has network connectivity, it attempts to sync its local database with those in the cloud via the cloud API.
One of the most commonly used in iOS mobile devices is Core Data that comes with the iOS software development kit (SDK). There are also cross-platform technologies that are good for both native Android and iOS apps. One of our favourite clientside databases that we like to use in our cross-platform projects is PouchDB. The reason we like this particular database is that we can use it on iOS, Android, and Desktop applications and it synchronizes seamlessly with CouchDB in the cloud.
As you can see, most of these scenarios can use a pure relational or NoSQL database, and perhaps your system can scale up just fine for several thousands of users, but if you are investing in an architecture that is reliable and scalable, then your data storage tools need to be specialized.