Home Command line utilities How To Kill The Largest Process In An Unresponsive Linux System

How To Kill The Largest Process In An Unresponsive Linux System

By sk
Published: Last Updated on 1.5K views

When I open a lot of tabs in my browser, my system goes unresponsive for several minutes. I couldn't move the mouse cursor, or kill a process or close any opened tabs. At that times, I had no choice but to forcibly reset the system. Of course, I use OneTab and The Great Suspender extensions, but they didn't help much either. I often ran out of memory. While looking for a solution to fix this, I came across Early OOM. For those wondering, Early OOM will kill the largest process in an unresponsive system when it has absolutely no other choices. Early OOM checks the amount of available memory and free swap 10 times a second. If both are below 10%, it will kill the largest process.

Why Early OOM? Why not built-in OOM killer?

Before going into further, let me you give a short explanation of OOM killer, also known as Out Of Memory killer. OOM killer is a process that the Kernel uses when the system is critically low on memory. The main task of OOM killer is to continue killing processes until enough memory is freed for the smooth functioning of the rest of the process that the Kernel is attempting to run. OOM killer will choose the best processes that are least important to the system and free up maximum memory and kill them. We can view the oom_score of each process in /proc directory under pid section.

Example:

$ cat /proc/10299/oom_score
1

The higher the value of oom_score of any process, the higher is its likelihood of getting killed by the OOM Killer when the system is running out of memory.

The developer of Early OOM claims that it has one big advantage over the in-kernel OOM killer. As I stated earlier, the Linux oom killer kills the process with the highest score, so the Chrome browser will always be the first victim of the oom killer. To avoid this, Early OOM uses /proc/*/status  instead of echo f > /proc/sysrq-trigger. He also claims that triggering the oom killer manually may not work at all in latest Linux Kernel.

Install Early OOM

Early OOM is available in AUR, so you can install it using any AUR helper programs in Arch Linux and its derivatives.

Using Yay:

$ yay -S earlyoom

Enable and start Early OOM daemon:

$ sudo systemctl enable earlyoom
$ sudo systemctl start earlyoom

On other Linux distributions, compile and install it manually as shown below.

$ git clone https://github.com/rfjakob/earlyoom.git
$ cd earlyoom
$ make
$ sudo make install

Kill The Largest Process In An Unresponsive Linux System Using Early OOM

Run the following command to start Early OOM:

$ earlyoom

If you compiled it from source, run the following command to start Early OOM:

$ ./earlyoom

The sample output would be:

earlyoom 0.12
mem total: 3863 MiB, min: 386 MiB (10 %)
swap total: 2047 MiB, min: 204 MiB (10 %)
mem avail: 1770 MiB (45 %), swap free: 2047 MiB (99 %)
mem avail: 1773 MiB (45 %), swap free: 2047 MiB (99 %)
mem avail: 1772 MiB (45 %), swap free: 2047 MiB (99 %)
mem avail: 1773 MiB (45 %), swap free: 2047 MiB (99 %)
mem avail: 1772 MiB (45 %), swap free: 2047 MiB (99 %)
mem avail: 1773 MiB (45 %), swap free: 2047 MiB (99 %)
mem avail: 1771 MiB (45 %), swap free: 2047 MiB (99 %)
mem avail: 1773 MiB (45 %), swap free: 2047 MiB (99 %)
mem avail: 1784 MiB (46 %), swap free: 2047 MiB (99 %)
[...]

As you see in the above output, Early OOM will display how much memory and swap you have, what the minimum is, how much memory is available and how much swap is free. Remember it will keep running until you manually stop by pressing CTRL+C.

If both memory and swap reaches below 10%, Early OOM will automatically kill the largest processes until the system has enough memory to run smoothly. You can also configure the minimum percentage value as per your requirement.

To set available memory minimum to PERCENT of total, run:

$ earlyoom -m <PERCENT_HERE>

To set available swap minimum to PERCENT of total, run:

$ earlyoom -s <PERCENT_HERE>

For more details, refer the help section.

$ earlyoom -h
earlyoom 0.12
Usage: earlyoom [OPTION]...

 -m PERCENT set available memory minimum to PERCENT of total (default 10 %)
 -s PERCENT set free swap minimum to PERCENT of total (default 10 %)
 -M SIZE set available memory minimum to SIZE KiB
 -S SIZE set free swap minimum to SIZE KiB
 -k use kernel oom killer instead of own user-space implementation
 -i user-space oom killer should ignore positive oom_score_adj values
 -d enable debugging messages
 -v print version information and exit
 -r INTERVAL memory report interval in seconds (default 1), set to 0 to
 disable completely
 -p set niceness of earlyoom to -20 and oom_score_adj to -1000
 -h this help text

Now, you don't need to worry about highest memory consumption processes.

Resources:

You May Also Like

4 comments

Aditya Goturu November 22, 2017 - 7:47 am

Its going to kill Apache isn’t it?

Reply
IJK November 22, 2017 - 8:11 pm

Far less useful than one might think – in a scarce memory situation, you might not even be able to issue any commands, much less spawning a shell to do so.

Reply
Mike November 16, 2018 - 3:38 pm

Isn’t the point that it kills the largest processes automatically? Usually, when a system freezes, we realize what is happening and we want to kill some programs manually but we can’t. This will do that and then we can, for example, restore browser session afterward if the browser will be killed, despite what it says in the article.

Reply
Bobo November 16, 2018 - 1:28 pm

See like a interesting solution for unresponsive system. My only concern is if there is maybe something like time option – if system is more then 5 minutes with very high memory usage then kill that procces. But it’s a nice thing to have definitly!

Reply

Leave a Comment

* By using this form you agree with the storage and handling of your data by this website.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

This website uses cookies to improve your experience. By using this site, we will assume that you're OK with it. Accept Read More