Redis saves my night
How I overcome the quick fix with the Redis
Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache, and message broker. Redis provides data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes, and streams. Redis has built-in replication, Lua scripting, LRU eviction, transactions, and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster
The performance problems in the production environment
It was a Friday we happily published the new features of our app and we enjoyed the TGIF. After I got back home, I got a message that some of our services of the app are down sometimes. We performed the app testing very well before the release of the new features but missed checking some parts of the code from git.
As usual, the bug hunter me to open the Azure portal and found that one of our resource groups is high in CPU consumption. I check the CPU usages of the app services and found that one service is actually causing the whole resource group to fail. So it is time to open the Application insights to know which API is really causing the bug and to slay it. Finally, I found that only one API is causing the CPU 100% when users requested and I was really surprised about that API.
Let’s see together the code which gave me surprise.
As in the comments in the code, the thousands of records in somecat will loop and about 500K of records will join happily each other and they will pull out from the dB to the ram by calling ToList() which is not necessary. This code actually slays the server by one request 1 time 100% CPU.
I have a few minutes to solve this problem and the actual fix is to modify some structure of the table to save the join query cost and to wipe out the loop with just one simple query. But this will take time. So I opt-in Redis as fresh data is not required in every request.
The master comes. Redis saves my night
I quickly installed the Redis extension and added some helper classes. And I made some modifications to the code.
Now the code works like magic the CPU problem is wiped out. And all the services become stable again. I didn't know where my TGIF happiness is gone. But finally, I was able to sleep peacefully that night. Redis saves my night.
