You are viewing our old blog site. For latest posts, please visit us at the new space. Follow our publication there to stay updated with tech articles, tutorials, events & more.

Redis with a distribution twist

Tuesday, May 5, 2015 | By
0.00 avg. rating (0% score) - 0 votes
We’ve all heard that, Redis is the next big thing when talking about noSQL data storage systems. We had 2 scenarios where we needed a caching mechanism, efficient in terms of, both memory and time. We tried out Redis with a distribution twist to solve both of our problems. Lets begin with the problem statements.
Problem 1 :-
We were caching the job-details and the job-description page-footer details for our jobs on the server memory directly as a local cache. This was being done for each server separately i.e if, I open the job-details page which is being served by server 1, it will be cached on Server 1 and if, I hit the same page from server 2, the same job will be cached on server 2 as well, leading to pointless redundancy and cumbersome cache management.
Problem 2 :-
We had a huge static php associative array which was being loaded very frequently and eating up a lot of memory, although only a few values, depending on some dynamic parameters, were needed on the page.
As stated above, we implemented Redis as a solution to both of our problems.
Before moving ahead, let us understand a bit more about Redis
What is Redis ?
Redis is basically a NoSql data storage system . It is an open sourced, in-memory advanced form of a key value store. There are 6 data types in Redis – Strings, Sets, Hash, Lists ,Sorted Sets and HyperLogLogs. Each data type exposes various operations .
Refer http://redis.io/topics/data-types-intro to know more about redis data types.
We used phpRedis, a php extension that integrates Redis commands in php, to implement Redis functionality in our code.
Refer https://github.com/phpredis/phpredis to know more about phpRedis
The Distribution Twist !
Memcached has a more stable infrastructure for horizontal scaling that is distributing cache across multiple servers while Redis is coming up with clusters for serving the same purpose but it is still in its beta stage. We used RedisArray to distribute keys across a number of Redis instances using consistent hashing.
Redis array is an isolated namespace (available in phpRedis extension ) in which the keys are related in some manner.
The array is composed of a list of hosts (redis servers ) and an optional key extraction function to define the part of the key on which the hash is to be calculated. By default and in order to be compatible with other libraries, phpredis tries to find a substring enclosed in curly braces within the key name, and uses it to distribute the data. The computed hash value of a key decides the target location for that particular key. We can define our own custom key distribution function that returns the server number, which is the index in the array of servers that we created the RedisArray object with, to control data distribution.
All phpRedis functions for a normal Redis object are available for a RedisArray object too.
Using Redis Arrays, we were able to solve our cache management (across multiple servers) problem to a great extent. The cache can be updated / deleted with a single command and the cache is common across all servers hence there is absolutely no data redundancy.
Also using redis hashes for storing static data proved to be very beneficial in terms of memory consumption and time latency .
Implementation
PhpRedis is an extension that provides support for redis as well as redis arrays. Let us see the implementation steps :-
  1. Install phpRedis extension for php.
  2. Install the redis server on all the servers you wish to cache stuff on.
Run redis-cli to check if the server has been installed or not.
  1. Use Redis to create a single instance of Redis and use RedisArray for distributed Redis .
Example 1 (Redis )
$redisObject = new Redis();
$redisObject->connect(‘172.16.3.158’,’6379’); //host address and port number
$redisObject->set(‘1’,’Hello’);
$redisObject->get(‘1’); //Fetches Hello
Example 2 (RedisArray)
$redisArray = new RedisArray(‘172.16.3.158:6379’,’172.16.3.178:6379’);
//Distribute the cache over 2 servers
$redisArray->set(‘1’,’Hello’);
$redisArray->_target(‘1’)
//Returns one of the 2 servers where this key will be stored
$redisArray->get(‘1’);
//Fetches hello from the targeted server
So with its richer functionality and advanced design Redis can be the first choice for caching static, semi dynamic or dynamic stuff or even as a simple non relational database . Three cheers to Redis !!!
Posted in General