Redis Data Structures: Comparing Memory, CPU and Latency Benchmark

At KKBOX, we have now been using Redis for over a year now to help improve data query performance for mostly our social media data. I increasingly notices that our developers, just they have a tendency to do with MySQL, created queries that where not very efficient. So i was asked by our director to dive into Redis data structures and create a training for our developers on how to appropriately choose and create queries on our Redis cluster.

The Benchmark results below focus on Memory utilization, CPU usage, and Latency for the Data Structures Strings, Sets, Lists and Sorted Sets.

In my test, instead of using a single key with a million values, i broke down the key to 10K keys each containing 100 values. This is because of the following fact:

Redis uses a special memory-efficient encoding when a list, sorted set, or hash set meets the following criteria:
 - Number of elements is less than the setting list-max-ziplist-entries 200
 - Size and byte of each and every element in the list, sorted set, or hash is less that the setting <data-type>-max-ziplist-value

So the goal was to see if actually using small data sets will actually improve performance, in particular memory utilization.

The config in Redis config file during the time the benchmarks were being carried were as follows:

hash-max-zipmap-entries 512 
hash-max-zipmap-value 64  
list-max-ziplist-entries 512
list-max-ziplist-value 64
zset-max-ziplist-entries 128
zset-max-ziplist-value 64

So with the List, Sorted Sets and Hash sets having elements limited to 100 only for each key, we expect that they perform better that Sets and Strings when it comes to utilizing memory.

So lets look at what the graph tells us:

The last column for Latency has two values "val1|val2", The first value is a results of querying all the data on the data structure, and the second value is for querying just the cardinality of the data structure.

Conclusion:

We can see that when you tune the Redis config file so that Lists, Sorted Sets and Hash Sets utilize ziplists, they perform almost 5 times better that Sets and Strings when it comes to Memory utilization.This is something to really keep in mind when choosing which data type to use for a given task.

Below is some tips on using Redis data structures:

 - Name keys appropriately for easy tracking.
 - Keep sets to a minimal size. Break large sets down.
 - Learn how to use which data structure for what purpose.
 - Redis uses a special memory-efficient encoding when a list, sorted set, or hash set meets the following criteria:
 - Number of elements is less than the setting list-max-ziplist-entries 200
 - Size and byte of each and every element in the list, sorted set, or hash is less that the setting <data-t