Trends of DB Engines in 2017 and What to Expect in 2018

2018 is here already and it's time to look at the DB engines ranking from "solid IT". The ranking measures the popularity of over 300 database engines. The ranking is based on various parameters such as the number of times a DB was mentioned on the web, Google Trends ranking, number of technical discussions, amount of job offers, amount of references in LinkedIn profiles etc. It is interesting to see how trends have been changed a bit in 2017.

The top 3 DB engines have declined by ~6%

Oracle, MySQL and Microsoft SQL Server are still dominating the top although all of them have lost some rating. Oracle has decline by 5.9%, MySQL has declined by 5.4% and MS SQL Server has declined by 6.8%. So which system gained the "lost" popularity from the top 3? Here are a few.

PostgreSQL is up 12.6% and MariaDB is up 18.5%

PostgreSQL has gained more and more attention in 2017 and has positive momentum in 2018. Being open sourced, PostgreSQL seems to be constantly improving by its developers’ community and it keeps presenting new features. Version 10 which was released on October 5th, 2017 includes an impressive list of new features and enhancements.

MariaDB probably attracts former users of MySQL as MariaDB is a fork from MySQL and the migration is pretty simple. MariaDB offers some unique niche features and there are some benchmarks that claim MariaDB has better performance in some scenarios.

ElasticSearch is up 11.8%, Splunk is up 11.7%

ElasticSearch has established itself as the go-to DB for search engines and as the leader of full text search engines - twice more popular than Apache Solr although both are based on Lucene core. Its popularity probably comes from the "ELK stack" which includes Elasticsearch, Logstash (data collection and log parsing engine) and Kibana (analytics and visualization platform). These 3 works together in harmony and make a rather comfortable and easy to use free package. Splunk is also a direct competitor of Elasticsearch but is not an open source system and has a price tag on it. Splunk offers more functionality than ELK with 2 major differences:

Logstash needs to be configured so that each field is identified before the data is shipped to Elasticsearch while in Splunk it is not necessarily required.
Splunk offers dynamic data exploration to help users to find and extract everything as a searchable field. Elasticsearch fields, on the other hand, need to be defined in advance to use aggregation over the log properties.

In the rate ELK is improving, the functionality gaps with Splunk might be closed soon and the fact ELK is free surely helps.

Rising stars - The cloud DBs

Amazon DynamoDB (+ 15.37%) - DynamoDB is the top NoSQL DB on AWS environment and in the cloud entirely. Its first version was released 6 years ago and today it is a mature, relatively cheap and fast engine which can scale automatically and integrates well with many other tools in the AWS big toolbox.
Azure Cosmos DB (+43.3%) - Cosmos DB is a rather new product and an incarnation of Azure DocumentDB on Azure. Unlike Dynamo, Cosmos offers a multi-model approach to the data inside it: An SQL API (SQL statements on the JSON documents in the DB), a MongoDB style API (JavaScript), a Cassandra API (CQL), a Table API (a key-value storing with LINQ or REST API). Cosmos is expected to offer additional models in the future. The unique multi-model approach is expected to attract new users this year and Cosmos might be a fierce competitor of Dynamo.
RDBMS as a Service - As the cloud keeps growing, AWS Aurora and SQL Azure will gain more popularity this year. Azure now offers a DBaaS for MySQL and for PostgreSQL as well, trying to compete with AWS. Google Cloud Platform (GCP) is also trying to compete with an offering for MySQL (PostgreSQL is in beta). This fierce competition between the 3 giants might ignite a price war, and lower prices will attract more users and more popularity.

What to expect in 2018

Oracle will still be the market leader, but it might have some difficulties in recruiting new followers as it's an expensive product. Oracle is like a Mercedes - a very good product but not everyone can afford it.

Microsoft SQL Server is now available on Linux and that might boost it a little as Microsoft tries to step into Oracle's turf.

As more and more systems are migrated to the cloud, we can expect the cloud databases to gain more and more traction.

PostgreSQL is also expected to gain more users sailing on version 10's wind.

Elasticsearch will get stronger as it turns to be the preferred choice for a full text search engine.

For more information: https://www.ceymplon.lk/service/it-service/mobile-applications