1

My application randomly returned OOM errors trying to allocate 16M chunks, while Linux had plenty memory used by disk cache (20G).

Swapping disabled. All OS limits seem fine.

After clearing Linux cache with drop_caches error disappeared.

Any idea what to check or is it somehow expected behavior?

nonobe
  • 11
  • 1
  • 2

2 Answers2

2

There was plenty of memory, but probably fragmented so you couldn't get a 16mb continuous chunk. Drop_caches will have triggered a defrag of memory so afterwards there is sufficient continuous memory available to honor your malloc request.

(This question is probably more suited to one of the programming forums.)

Tonny
  • 6,360
  • 1
  • 20
  • 31
2

Malloc() does not allocate physical memory, it allocates virtual memory. Malloc() can fail due to lack of (continuous free chunk of) virtual memory or exceeded commit limit.

  1. Check the virtual memory usage of the process using ps, top or pmamp commands. 64bit architectures (amd64) have extremely large virtual memory and is basically impossible to exhaust that, but 32bit process would be limited to at most 4GB of virtual memory.
  2. Check /proc/sys/vm/overcommit_memory and Committed_AS and CommitLimit rows in /proc/meminfo. If overcommit_memory is 1, exceeding CommitLimit will cause malloc() to fail.
x22
  • 181