I have an AKS windows node pool. Here is its host memory breakdown before running the heavy RAM intensive workload for over 4 hours:
PS C:\hpc> Get-Counter "\Memory\Pool Nonpaged Bytes", "\Memory\Pool Paged Bytes", "\Memory\System Cache Resident Bytes", "\Memory\Modified Page List Bytes", "\Process(_Total)\Working Set" |
>> Select-Object -ExpandProperty CounterSamples |
>> Select-Object Path, @{Name = 'SizeGB'; Expression = { '{0:N2} GB' -f ($_.CookedValue / 1GB) } }
Path SizeGB
\akswin2000001\memory\pool nonpaged bytes 0.56 GB
\akswin2000001\memory\pool paged bytes 0.85 GB
\akswin2000001\memory\system cache resident bytes 0.38 GB
\akswin2000001\memory\modified page list bytes 0.14 GB
\akswin2000001\process(_total)\working set 4.35 GB
Here it is 25 hours after the heavy workload finished:
Get-Counter "\Memory\Pool Nonpaged Bytes", "\Memory\Pool Paged Bytes", "\Memory\System Cache Resident Bytes", "\Memory\Modified Page List Bytes", "\Process(_Total)\Working Set" |
Select-Object -ExpandProperty CounterSamples |
Select-Object Path, @{Name = 'SizeGB'; Expression = { '{0:N2} GB' -f ($_.CookedValue / 1GB) } }
Path SizeGB
\akswin000001\memory\pool nonpaged bytes 5.80 GB
\akswin000001\memory\pool paged bytes 11.38 GB
\akswin000001\memory\system cache resident bytes 2.84 GB
\akswin000001\memory\modified page list bytes 0.14 GB
\akswin000001\process(_total)\working set 6.01 GB
The extra 2GB of the total process memory is not a concern (the heavy container is idle, but still takes up memory)
What is a concern is that the kernel memory is not released. And I have no idea how to deal with it except restart the node.
Are there any other options here?
P.S.
Please, do not recommend to migrate to Linux or reduce the workload.
