Similar to many other projects that we maintain or co-maintainer, we became active around 11 years ago, when we were working on porting important contributed projects to Drupal 8 as part of our work on the NP8 distribution.
We inherited the general architecture from the previous 7.x-2.x release. Already back then, the module supported both the phpredis extension and the predis library. This was done with a largely separate classes for each supported API and supported feature (cache, lock, flood, queue). Fairly recently, we also added support for the Relay extension.
Limitations of the existing architecture
Over the years, the current approach resulted in some issues as there were slight differences between the different implementations, causing bugs and different behavior.
Additionally, the way those classes were initialized was not very extensible, causing conflicts between different issue trying to add additional features such as Cluster support and new authentication mechanisms, which was one reason that many of those features lingered in the issue queue for years.
New major version in development with new adapter approach
A decision was made to start the development of a new 2.x branch and refactor the architecture to resolve these issues.
The new approach uses a single adapter class and corresponding factory for each supported implementation. Those implementations are mostly very similar. That means that those adapters can pass through most calls directly to the underlying implementation, and only need to intervene when there are differences, such as a different argument structure.
This quickly progressed from vague ideas that we've been thinking about for a long time to a proof of concept and then a working and tested implementation. Previous investments into allowing the tests to run against all supported backends on Gitlab CI, partially sponsored by CacheWerk, the company behind the relay extension, were extremely valuable to ensure that all functionality was working on all supported implementations.
See https://www.drupal.org/node/3541244 for the change record for this change.
MULTI vs. Pipeline
The area with the biggest differences between the different implementations was their support and behavior around MULTI/PIPELINE. Different methods and different approaches in how multiple API calls could be chained.
The chosen approach was to disallow method chains so that the adapter has easier control over the full process.
As part of that, the current use of MULTI and Pipelines was carefully reviewed as the different implementations didn't always use the same approach. MULTI can be compared to a database transaction, in that Redis will ensure an atomic execution of all commands wrapped in a MULTI...EXEC chain. Pipelines on the other hand are a tool to send and execute multiple commands at once and are generally much faster than a MULTI block.
While some backends such as lock must use MULTI in some cases, the cache backend was refactored to only use pipelines. Previously, at least for the phpredis implementation, MULTI was used when writing caches and bulk cache tag invalidations.
Performance testing
We did run some basic performance comparisons to ensure that there is no regression when reading caches due to the new approach and verify the improvements that switching to pipelines brings.
The test script writes and then reads 10'000 small cache entries and then we compared the performance between Redis 1, 2 and the default database by executing it 3 times. the tests were run on a local DDEV project and the absolute numbers likely vary greatly between environments.
| Redis 1 | Redis 2 | Database | |
|---|---|---|---|
| Write | 1263 | 339 | 1095 |
| 1171 | 334 | 1162 | |
| 998 | 417 | 1128 | |
| Total | 1144 | 363 | 1128 |
| Read | 407 | 383 | 915 |
| 336 | 357 | 900 | |
| 317 | 372 | 1047 | |
| Total | 353 | 371 | 954 |
The difference in read performance is smaller than the variations between each run and Redis is almost 3x as fast as the default Database cache backend in this specific setup.
The results in write performance are better than expected. While Redis 1 has a similar performance to the database when writing caches, Redis 2 is roughly 3x faster. this should result in a considerable improvement for requests that have to write a lot of caches. the relative improvements might be even better when the connection to Redis is slower, such as Redis being on a different server.
<?phpuse Drush\Drush;$cache = \Drupal::cache();var_dump(get_class($cache));$start = microtime(TRUE);foreach (range(1, 10000) as $i) { $cache->set('test' . $i, 'test' . $i);}$end = microtime(TRUE);Drush::logger()->notice('Write: ' . round(($end - $start) * 1000) . 'ms');$start = microtime(TRUE);foreach (range(1, 10000) as $i) { $cache->get('test' . $i);}$end = microtime(TRUE);Drush::logger()->notice('Read: ' . round(($end - $start) * 1000) . 'ms');
Current status and outlook
Redis 2.x is now available as a development snapshot. We recommend against using it in production as it has not been heavily tested yet and more API changes are expected. We do plan to start creating releases in the not-so-far future, as we are of course interested in benefiting from these improvements on our own projects as well.
If there are features that you would like to see, we are currently reviewing what can be included as well and are looking for sponsoring to support our work around that and making this available to everyone soon.