A few days ago I noticed that my Mac Mini locked up and became unresponsive. So I power-cycled it but didn’t want to spend a lot of time investigating what the problem was. Today it locked up again and fortunately I had an Apple Remote Desktop session open at the time.
To my surprise OS X claimed that it ran out of application memory. My Mac Mini has 16GB of memory and there aren’t any resource hungry tasks running on the Mini. I thought about what changes I had made to the Mini in the past couple of days and came up with two potential culprits:
- I re-installed OS X 10.9 (since the official release has a slightly higher build number than the GM I had installed)
- I upgraded OS X Server from a 2.x Beta to 3.0 GM
To see what exact process was eating up my memory I recycled the Mini, opened up Activity Monitor and let it just sit there. When I returned a few hours later I noticed that a process called “devicemgrd” was hogging memory.
I googled to see if I was the only one having this issue and sure enough I came across a thread in the Apple Support Forum addressing this very same problem. One of the users pointed to the “Profile Manager” as being the cause of the problem. To prevent the system from running out of memory completely I killed the offending process and memory usage returned to normal levels immediately. You can see the immediate drop in “Memory Pressure” in the Activity Monitor.
A search on the command line for “devicemgrd” revealed that it’s indeed part of the OS X Server App I just upgraded.
I guess I’ll do some cleanup tomorrow and re-install the application. If that doesn’t work I’ll call my friends at Apple Support again.
Devicemgrd kept restarting automatically so I looked in /var/log/devicemgr/devicemgrd.log and noticed that there was an issue with the underlying Postgres database filling up said log file.
So I called Apple Support and an enterprise advisor had me run a couple of command in the terminal.
sh-3.2# serveradmin fullstatus devicemgr
devicemgr:state = “STOPPED”
sh-3.2# serveradmin fullstatus postgres_server
postgres_server:error = “FILE_NOT_FOUND_ERR”
Next we ran the following SQL command, which effectively stopped the resource starvation and memory build up:
sudo -u _devicemgr /Applications/Server.app/Contents/ServerRoot/usr/bin/psql -d devicemgr_v2m0 -h /Library/Server/ProfileManager/Config/var/PostgreSQL -c “DELETE FROM db_notifications WHERE message LIKE ‘[DMDevicemgrd updateAllProfilesForID:%’;”
As soon as we executed the above command memory pressure went down. However devicemgrd still consumed over 100% of CPU cycles. So I restarted the Mini and voilà, devicemgrd was behaving correctly.
After I got off the phone with Apple I ran the “serveradmin fullstatus postgres_server” command again and it was still showing “FILE_NOT_FOUND_ERR“. So there is still something not quite right, but at least the rest of the system is running normal now.
On November 11th Apple released an update for OS X Server fixing “various Profile Manager migration issues”