Optimize Memory usage of GO Applications by setting Memory Limit

Go is one of the most efficient programming languages, and not every Go developer is aware of the Go language garbage collection. Setting the garbage collection frequency and memory limit for applications plays an important role for optimizing memory.

Data that changes dynamically during execution or requires access beyond the scope of a function is stored on the heap. If an allocated memory space is no longer needed, then that memory needs to be deallocated so that further allocation can be done on the same space. The Garbage Collector(GC) is a system designed specifically to identify and free dynamically allocated memory. GO GC is neither triggered at regular intervals nor when CPU is idle. The only parameter that trigger Go GC is the previous heap allocation.

We have faced a production issue where applications are getting killed by the operating system with an Out of Memory (OOM) exception. This situation occurs when the application requests more memory and the host memory is close to exhaustion. As the operating system is unable to provide additional memory, it kills the application. On debugging, we came to know that the application is not releasing/reusing the allocated memory, and Go Garbage Collection is not running in our scenario.
With default configuration (GOGC=100), Go’s runtime garbage collector (GC) runs only when the heap size is twice the size of the retained heap since the last collection. For example., initially 1 GB was allocated. Then GC would run only after 2GB was allocated. Let’s say if the application is running at 8 GB, then GOGC would run only after 16 GB was allocated. Anything less than 16 GB would not trigger GC. This caused the production issue. This problem can we overcome using two methods. 1. Setting GOGC to a lower value    2. Setting GOMEMLIMIT
I wrote a paper that speaks on the ways to overcome this and the performance measurements of setting Memory limit to different values.
 http://doi.org/10.51219/JAIMLD/pallavi-priya-patharlagadda/213
In our project, we proceed by setting GOMEMLIMIT and avoid application being killed by OS due to OOM exception.