gatus/vendor/github.com/TwinProduction/gocache

gocache

gocache is an easy-to-use, high-performance, lightweight and thread-safe (goroutine-safe) in-memory key-value cache with support for LRU and FIFO eviction policies as well as expiration, bulk operations and even persistence to file.

Table of Contents

• Features
• Usage
  • Initializing the cache
  • Functions
  • Examples
    • Creating or updating an entry
    • Getting an entry
    • Deleting an entry
    • Complex example
• Persistence
  • Limitations
• Eviction
  • MaxSize
  • MaxMemoryUsage
• Server
• Running the server with Docker
• Performance
  • Summary
  • Results
• FAQ
  • Why does the memory usage not go down?

Features

gocache supports the following cache eviction policies:

• First in first out (FIFO)
• Least recently used (LRU)

It also supports cache entry TTL, which is both active and passive. Active expiration means that if you attempt to retrieve a cache key that has already expired, it will delete it on the spot and the behavior will be as if the cache key didn't exist. As for passive expiration, there's a background task that will take care of deleting expired keys.

It also includes what you'd expect from a cache, like bulk operations, persistence and patterns.

While meant to be used as a library, there's a Redis-compatible cache server included. See the Server section. It may also serve as a good reference to use in order to implement gocache in your own applications.

Usage

go get -u github.com/TwinProduction/gocache

Initializing the cache

cache := gocache.NewCache().WithMaxSize(1000).WithEvictionPolicy(gocache.LeastRecentlyUsed)

If you're planning on using expiration (SetWithTTL or Expire) and you want expired entries to be automatically deleted in the background, make sure to start the janitor when you instantiate the cache:

cache.StartJanitor()

Functions

Function	Description
WithMaxSize	Sets the max size of the cache. gocache.NoMaxSize means there is no limit. If not set, the default max size is gocache.DefaultMaxSize.
WithMaxMemoryUsage	Sets the max memory usage of the cache. gocache.NoMaxMemoryUsage means there is no limit. The default behavior is to not evict based on memory usage.
WithEvictionPolicy	Sets the eviction algorithm to be used when the cache reaches the max size. If not set, the default eviction policy is gocache.FirstInFirstOut (FIFO).
StartJanitor	Starts the janitor, which is in charge of deleting expired cache entries in the background.
StopJanitor	Stops the janitor.
Set	Same as SetWithTTL, but with no expiration (gocache.NoExpiration)
SetAll	Same as Set, but in bulk
SetWithTTL	Creates or updates a cache entry with the given key, value and expiration time. If the max size after the aforementioned operation is above the configured max size, the tail will be evicted. Depending on the eviction policy, the tail is defined as the oldest
Get	Gets a cache entry by its key.
GetAll	Gets a map of entries by their keys. The resulting map will contain all keys, even if some of the keys in the slice passed as parameter were not present in the cache.
GetKeysByPattern	Retrieves a slice of keys that matches a given pattern.
Delete	Removes a key from the cache.
DeleteAll	Removes multiple keys from the cache.
Count	Gets the size of the cache. This includes cache keys which may have already expired, but have not been removed yet.
Clear	Wipes the cache.
TTL	Gets the time until a cache key expires.
Expire	Sets the expiration time of an existing cache key.
SaveToFile	Stores the content of the cache to a file so that it can be read using ReadFromFile. See persistence.
ReadFromFile	Populates the cache using a file created using SaveToFile. See persistence.

Examples

Creating or updating an entry

cache.Set("key", "value") 
cache.Set("key", 1)
cache.Set("key", struct{ Text string }{Test: "value"})

Getting an entry

value, ok := cache.Get("key")

You can also get multiple entries by using cache.GetAll([]string{"key1", "key2"})

Deleting an entry

cache.Delete("key")

You can also delete multiple entries by using cache.DeleteAll([]string{"key1", "key2"})

Complex example

package main

import (
	"fmt"
	"github.com/TwinProduction/gocache"
	"time"
)

func main() {
	cache := gocache.NewCache().WithEvictionPolicy(gocache.LeastRecentlyUsed).WithMaxSize(10000)
	cache.StartJanitor() // Passively manages expired entries

	cache.Set("key", "value")
	cache.SetWithTTL("key-with-ttl", "value", 60*time.Minute)
	cache.SetAll(map[string]interface{}{"k1": "v1", "k2": "v2", "k3": "v3"})

	value, exists := cache.Get("key")
	fmt.Printf("[Get] key=key; value=%s; exists=%v\n", value, exists)
	for key, value := range cache.GetAll([]string{"k1", "k2", "k3"}) {
		fmt.Printf("[GetAll] key=%s; value=%s\n", key, value)
	}
	for _, key := range cache.GetKeysByPattern("key*", 0) {
		fmt.Printf("[GetKeysByPattern] key=%s\n", key)
	}

	fmt.Println("Cache size before persisting cache to file:", cache.Count())
	err := cache.SaveToFile("cache.bak")
	if err != nil {
		panic(fmt.Sprintf("failed to persist cache to file: %s", err.Error()))
	}

	cache.Expire("key", time.Hour)
	time.Sleep(500*time.Millisecond)
	timeUntilExpiration, _ := cache.TTL("key")
	fmt.Println("Number of minutes before 'key' expires:", int(timeUntilExpiration.Seconds()))

	cache.Delete("key")
	cache.DeleteAll([]string{"k1", "k2", "k3"})

	fmt.Println("Cache size before restoring cache from file:", cache.Count())
	_, err = cache.ReadFromFile("cache.bak")
	if err != nil {
		panic(fmt.Sprintf("failed to restore cache from file: %s", err.Error()))
	}

	fmt.Println("Cache size after restoring cache from file:", cache.Count())
	cache.Clear()
	fmt.Println("Cache size after clearing the cache:", cache.Count())
}

Output

[Get] key=key; value=value; exists=true
[GetAll] key=k2; value=v2
[GetAll] key=k3; value=v3
[GetAll] key=k1; value=v1
[GetKeysByPattern] key=key
[GetKeysByPattern] key=key-with-ttl
Cache size before persisting cache to file: 5
Number of minutes before 'key' expires: 3599
Cache size before restoring cache from file: 1
Cache size after restoring cache from file: 5
Cache size after clearing the cache: 0

Persistence

While gocache is an in-memory cache, you can still save the content of the cache in a file and vice versa.

To save the content of the cache to a file:

err := cache.SaveToFile(TestCacheFile)

To retrieve the content of the cache from a file:

numberOfEntriesEvicted, err := newCache.ReadFromFile(TestCacheFile)

The numberOfEntriesEvicted will be non-zero only if the number of entries in the file is higher than the cache's configured MaxSize.

Limitations

While you can cache structs in memory out of the box, persisting structs to a file requires you to register the custom interfaces that your application uses with the gob package.

type YourCustomStruct struct {
	A string
	B int
}

// ...
cache.Set("key", YourCustomStruct{A: "test", B: 123})

To persist your custom struct properly:

gob.Register(YourCustomStruct{})
cache.SaveToFile("gocache.bak")

The same applies for restoring the cache from a file:

cache := NewCache()
gob.Register(YourCustomStruct{})
cache.ReadFromFile(TestCacheFile)
value, _ := cache.Get("key")
fmt.Println(value.(YourCustomStruct))

You only need to persist the struct once, so adding the following function in a file would suffice:

func init() {
    gob.Register(YourCustomStruct{})
}

Failure to register your custom structs will prevent gocache from persisting and/or parsing the value of each keys that use said custom structs.

That being said, assuming that you're using gocache as a cache, this shouldn't create any bugs on your end, because every key that cannot be parsed are not populated into the cache by ReadFromFile.

In other words, if you're falling back to a database or something similar when the cache doesn't have the key requested, you'll be fine.

Eviction

MaxSize

Eviction by MaxSize is the default behavior, and is also the most efficient.

The code below will create a cache that has a maximum size of 1000:

cache := gocache.NewCache().WithMaxSize(1000)

This means that whenever an operation causes the total size of the cache to go above 1000, the tail will be evicted.

MaxMemoryUsage

Eviction by MaxMemoryUsage is disabled by default, and is still a work in progress.

The code below will create a cache that has a maximum memory usage of 50MB:

cache := gocache.NewCache().WithMaxSize(0).WithMaxMemoryUsage(50*gocache.Megabyte)

This means that whenever an operation causes the total memory usage of the cache to go above 50MB, one or more tails will be evicted.

Unlike evictions caused by reaching the MaxSize, evictions triggered by MaxMemoryUsage may lead to multiple entries being evicted in a row. The reason for this is that if, for instance, you had 500 entries of 0.1MB each and you suddenly added a single entry of 10MB, 100 entries would need to be evicted to make enough space for that new big entry.

It's very important to keep in mind that eviction by MaxMemoryUsage is approximate.

The only memory taken into consideration is the size of the cache, not the size of the entire application. If you pass along 100MB worth of data in a matter of seconds, even though the cache's memory usage will remain under 50MB (or whatever you configure the MaxMemoryUsage to), the memory footprint generated by that 100MB will still exist until the next GC cycle.

As previously mentioned, this is a work in progress, and here's a list of the things you should keep in mind:

• The memory usage of structs are a gross estimation and may not reflect the actual memory usage.
• Native types (string, int, bool, []byte, etc.) are the most accurate for calculating the memory usage.
• Adding an entry bigger than the configured MaxMemoryUsage will work, but it will evict all other entries.

Server

For the sake of convenience, a ready-to-go cache server is available through the gocacheserver package.

The reason why the server is in a different package is because gocache does not use any external dependencies, but rather than re-inventing the wheel, the server implementation uses redcon, which is a Redis server framework for Go.

That way, those who desire to use gocache without the server will not add any extra dependencies as long as they don't import the gocacheserver package.

package main

import (
	"github.com/TwinProduction/gocache"
	"github.com/TwinProduction/gocache/gocacheserver"
)

func main() {
	cache := gocache.NewCache().WithEvictionPolicy(gocache.LeastRecentlyUsed).WithMaxSize(100000)
	server := gocacheserver.NewServer(cache)
	server.Start()
}

Any Redis client should be able to interact with the server, though only the following instructions are supported:

• GET
• SET
• DEL
• PING
• QUIT
• INFO
• EXPIRE
• SETEX
• TTL
• FLUSHDB
• EXISTS
• ECHO
• MGET
• MSET
• SCAN (kind of - cursor is not currently supported)
• KEYS

Running the server with Docker

To build it locally, refer to the Makefile's docker-build and docker-run steps.

Note that the server version of gocache is still under development.

docker run --name gocache-server -p 6379:6379 twinproduction/gocache-server:v0.1.0

Performance

Summary

• Set: Both map and gocache have the same performance.
• Get: Map is faster than gocache.

This is because gocache keeps track of the head and the tail for eviction and expiration/TTL.

Ultimately, the difference is negligible.

We could add a way to disable eviction or disable expiration altogether just to match the map's performance, but if you're looking into using a library like gocache, odds are, you want more than just a map.

Results

key	value
goos	windows
goarch	amd64
cpu	i7-9700K
mem	32G DDR4

BenchmarkMap_Get-8                                                         	47943618	       26.6 ns/op
BenchmarkMap_SetSmallValue-8                                               	 3800810	       394 ns/op
BenchmarkMap_SetMediumValue-8                                              	 3904794	       400 ns/op
BenchmarkMap_SetLargeValue-8                                               	 3934033	       383 ns/op
BenchmarkCache_Get-8                                                       	27254640	       45.0 ns/op
BenchmarkCache_SetSmallValue-8                                             	 2991620	       401 ns/op
BenchmarkCache_SetMediumValue-8                                            	 3051128	       381 ns/op
BenchmarkCache_SetLargeValue-8                                             	 2995904	       382 ns/op
BenchmarkCache_SetSmallValueWhenUsingMaxMemoryUsage-8                      	 2752288	       428 ns/op
BenchmarkCache_SetMediumValueWhenUsingMaxMemoryUsage-8                     	 2744899	       436 ns/op
BenchmarkCache_SetLargeValueWhenUsingMaxMemoryUsage-8                      	 2756816	       430 ns/op
BenchmarkCache_SetSmallValueWithMaxSize10-8                                	 5308886	       226 ns/op
BenchmarkCache_SetMediumValueWithMaxSize10-8                               	 5304098	       226 ns/op
BenchmarkCache_SetLargeValueWithMaxSize10-8                                	 5277986	       227 ns/op
BenchmarkCache_SetSmallValueWithMaxSize1000-8                              	 5130580	       236 ns/op
BenchmarkCache_SetMediumValueWithMaxSize1000-8                             	 5102404	       237 ns/op
BenchmarkCache_SetLargeValueWithMaxSize1000-8                              	 5084695	       237 ns/op
BenchmarkCache_SetSmallValueWithMaxSize100000-8                            	 3858066	       315 ns/op
BenchmarkCache_SetMediumValueWithMaxSize100000-8                           	 3909277	       315 ns/op
BenchmarkCache_SetLargeValueWithMaxSize100000-8                            	 3870913	       315 ns/op
BenchmarkCache_SetSmallValueWithMaxSize100000AndLRU-8                      	 3856012	       316 ns/op
BenchmarkCache_SetMediumValueWithMaxSize100000AndLRU-8                     	 3809518	       316 ns/op
BenchmarkCache_SetLargeValueWithMaxSize100000AndLRU-8                      	 3834754	       318 ns/op
BenchmarkCache_GetAndSetConcurrently-8                                     	 1779258	       672 ns/op
BenchmarkCache_GetAndSetConcurrentlyWithRandomKeysAndLRU-8                 	 2569590	       487 ns/op
BenchmarkCache_GetAndSetConcurrentlyWithRandomKeysAndFIFO-8                	 2608369	       474 ns/op
BenchmarkCache_GetAndSetConcurrentlyWithRandomKeysAndNoEvictionAndLRU-8    	 2185795	       582 ns/op
BenchmarkCache_GetAndSetConcurrentlyWithRandomKeysAndNoEvictionAndFIFO-8   	 2238811	       568 ns/op
BenchmarkCache_GetAndSetConcurrentlyWithFrequentEvictionsAndLRU-8          	 3726714	       320 ns/op
BenchmarkCache_GetAndSetConcurrentlyWithFrequentEvictionsAndFIFO-8         	 3682808	       325 ns/op
BenchmarkCache_GetConcurrentlyWithLRU-8                                    	 1536589	       739 ns/op
BenchmarkCache_GetConcurrentlyWithFIFO-8                                   	 1558513	       737 ns/op
BenchmarkCache_GetKeysThatDoNotExistConcurrently-8                         	10173138	       119 ns/op

FAQ

Why does the memory usage not go down?

By default, Go uses MADV_FREE if the kernel supports it to release memory, which is significantly more efficient than using MADV_DONTNEED. Unfortunately, this means that RSS doesn't go down unless the OS actually needs the memory.

Technically, the memory is available to the kernel, even if it shows a high memory usage, but the OS will only use that memory if it needs to. In the case that the OS does need the freed memory, the RSS will go down and you'll notice the memory usage lowering.

reference

You can reproduce this by following the steps below:

• Start gocacheserver
• Note the memory usage
• Create 500k keys
• Note the memory usage
• Flush the cache
• Note that the memory usage has not decreased, despite the cache being empty.

Substituting gocache for a normal map will yield the same result.

If the released memory still appearing as used is a problem for you, you can set the environment variable GODEBUG to madvdontneed=1.