Apache: No space left on device: Couldn’t create accept lock

Recently I faced on issue one of my web server which was running on Apache. I checked Apache syntax error and attempted to restart it, but it refused to start properly and didn’t bind to any ports.

In Apache error logs, this message appeared over and over:

[emerg] (28)No space left on device: Couldn't create accept lock

Above error you can see Apache is trying write something but no space left on device. If this happens, you can check these things in order:

1. Check Disk Space On The System

First it comes in mind, to check the disk space, if you are out of disk space then you will need to fix this issue.

2. Check File System Quotas

Check the your filesystem uses quotas you might be reaching a quota limit rather than a disk space limit. Run repquota / to check your quotas on the root partition. If you are crossed limit, increase your quota or clear up some disk space. In most cases Apache logs are usually the culprit in these situations.

# repquota /

3. Clear Out Your Active Semaphores

What is a semaphore? It is actually an apparatus for conveying information by means of visual signals. But, when it comes to programming, semaphores are used for communicating between the active processes of a certain application. In this situation Apache, used to communicate between the parent and child processes. If Apache can not write these things down, then it can not communicate properly with all of the processes it starts.

Run blow command as root to see list of semaphores, Apache has not cleaned up after itself, and some semaphores are stuck.

# ipcs -s

Sample output:

------ Semaphore Arrays --------
key        semid      owner      perms      nsems
0x00000000 3866626    -1       600        1
0x00000000 98307      -1       600        1
0x00000000 131076     -1       600        1
0x00000000 163845     -1       600        1
0x00000000 196614     -1       600        1
0x00000000 229383     -1       600        1
0x00000000 262152     -1       600        1
0x00000000 294921     -1       600        1
0x00000000 327690     -1       600        1
0x00000000 425995     -1       600        1
0x00000000 458764     -1       600        1
0x00000000 1998861    -1       600        1
0x00000000 2031630    -1       600        1
0x00000000 2457615    -1       600        1
0x00000000 2490384    -1       600        1
0x00000000 2523153    -1       600        1
0x00000000 2555922    -1       600        1
0x00000000 2588691    -1       600        1
0x00000000 2621460    -1       600        1
0x00000000 2654229    -1       600        1
0x00000000 2686998    -1       600        1
0x00000000 2719767    -1       600        1

If you see list of semaphores, run the blow command to clear them out:

# for i in `ipcs -s | awk '/httpd/ {print $2}'`; do (ipcrm -s $i); done

Now Apache should start properly, If it does not, you may just be completely out of available semaphores.

You can also increase your available semaphores, tickling your kernel to do so. Add this to /etc/sysctl.conf .

# vim /etc/sysctl.conf

kernel.msgmni = 1024
kernel.sem = 250 256000 32 1024

Save and close file.

Now then run sysctl -p command to puck up the new changes.

# sysctl -p

Thanks:)