We have Windows Server 2022 physical terminal servers, where users log in and run applications, including finite elements computations. Depending on what the users do, applications can consume all processor and RAM, and Windows becomes unstable.
The System log contains errors like:
Application popup: dwm.exe - System Error : The process has terminated because it could not allocate additional memory.
And low-memory warnings:
Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: fl_mpi2320.exe (7256) consumed 8,306,704,384 bytes, fl_mpi2320.exe (6020) consumed 6,652,760,064 bytes, and fl_mpi2320.exe (11188) consumed 6,635,606,016 bytes.
This is problematic because system processes crash. Remote desktop connections don't work any more, and we need to remotely restart the hosts to recover.
I searched for solutions, but did not find a way to ensure that user processes can't affect the system in this way (no system settings nor GPOs). Increasing the page file could make the problem appear a bit later, but wont solve it.
Adding RAM is not an option. The memory is enough for what users need to do, with minor exceptions, and there will be always someone who will try to compute something too big to fit in memory. User education would be the best solution, but it doesn't work well enough, as many people are just wanting to see their results as quickly as possible.
It's possible to write a Windows service that kills the user processes when they use too much memory, but it might be too late for the system. Remotely killing them doesn't resurrect the machines when the memory is depleted.
How can I prevent the system from allocating too much memory?
