Memory usage seems to be incorrect

[inspiredearth]

I use WP Server Stats on every site I work with.
On one particular site it appears to be showing incorrect memory usage. Oddly, this has occurred in two different hosting environments (a shared host, and a VPS I've moved the site to).

The VPS is with DigialOcean. Looking at the monitoring data on DO for this droplet, the memory usage is sitting around 25% of 2GB. Yet in WP Server Stats it's constantly showing around 92% (2GB in use). It did exactly the same thing on the other shared hosting environment. Sat around 92%, although in that case it was showing "5 GB" in use (which was the available RAM on that server).

It seems like a bug.

I get the same result with all plug-ins (except WP Server Stats) disabled, and on a default WP theme.

[inspiredearth]

Here's a screenshot showing what I mean:

Here's the actual RAM usage reported in DigitalOcean monitor:

And here is the RAM usage reported in HTOP on linux

As you can see, WP Server Stats is reporting something completely different.

[inspiredearth]

Here's a note I posted on a related discussion on wordpress.org

I found this thread, after posting an Issue report on Github.

What you've written here seems to address the issue I have reported. However, I am wondering about which data from /proc/meminfo is most relevant. I see it reporting:

MemTotal:        2048112 kB
MemFree:          325808 kB
MemAvailable:    1363144 kB

Based on the memory usage reported by top/htop, and also by the monitor on my server, the MemAvailable appears to be the more accurate indication of RAM available. It's the one that matches top/htop and the server RAM monitor. It shows that 684968 is in use, which is about 33%.

This matches what TOP shows:

KiB Mem :  2048112 total,   310804 free,   385364 used,  1351944 buff/cache
KiB Swap:        0 total,        0 free,        0 used.  1349148 avail Mem

and HTOP, and the RAM monitor (on Digital Ocean dashboard).

Perhaps your plug-in needs to use the MemAvailable number for its calculation?

[isaumya]

@jonathannz Hi,
first of all, thanks a lot for doing the research. I completely understand that you want all the calculations to be converted to MemAvailable instead of MemFree. Here is the catch, I have no issue doing that, but the problem is I still will be needing the MemTotal value for doing the math.

I'll test it out and push an update to you first so that you can test it out and let me know. Wait for the next update.

[isaumya]

Really sorry for the delayed reply. Unfortunately, GitHub doesn't send me emails when you posted the thread.

[inspiredearth]

Thanks for getting back to me.
I took a quick look through your code... and was thinking, wouldn't it be as simple as replacing what you grep for, where you have "grep -w 'MemFree' /proc/meminfo | grep -o -E '[0-9]+'" this would change to "grep -w 'MemAvailable' /proc/meminfo | grep -o -E '[0-9]+'", and then you can let your calculations continue as before?

As an aside... Something I am going to look into (just as soon as I have a little time for it) is what the MemFree is actually referring to. It's unclear to me by Linux reports MemFree and MemAvailable as two very different figures.

[inspiredearth]

Here's an explanation of what MemAvailable refers to. Looks to me like it's the better choice, over MemFree, as it is derived from a calculation that's already taking numerous factors into account. The result is...

MemAvailable: An estimate of how much memory is available for starting new applications, without swapping. Calculated from MemFree, SReclaimable, the size of the file LRU lists, and the low watermarks in each zone. The estimate takes into account that the system needs some page cache to function well, and that not all reclaimable slab will be reclaimable, due to items being in use. The impact of those factors will vary from system to system.

Some helpful info here, and here

[isaumya]

@jonathannz Can you replace the wp-server-stats.php file with the one in the GitHub? Let me know if that fixes the issue.

[inspiredearth]

It looks like it might have. It's hard to tell, as the plug-in is rounding up/down quite substaintially. I've also since made changes to the PHP and NGINX config on this server, which has freed up a large amount of RAM. Which means my MemFree and MemAvailable are now much closer to each other, as opposed to before.

Here's my mem data:

MemTotal:        2048096 kB
MemFree:         1300336 kB
MemAvailable:    1540252 kB

Here's the reading from the plug-in.

So, yes, if I deduct 1540252 from 2048096 (= 497844), then it's around what the Real time RAM usage is reporting (=496M)

Thanks very much.

[isaumya]

@jonathannz I am sorry to say that I had to take out MemAvailable from the code. Here is the case: after your research, I did change the MemFree to MemAvailable without realizing one key thing i.e. MemAvailable is not a generic global value that is available across all system, distros, and servers. As it turns out only in few distros your will get to see that MemAvailable flag, for other that doesn't exist.

Whereas MemFree is a global tag available across all distros and system no matter what you use. So, maybe it is not 100% accurate but it will at least work on all Linux servers, across all system.

For example, this is the output of one of my server's meminfo file:

cat /proc/meminfo
MemTotal:        5683284 kB
MemFree:         2510052 kB
Cached:           120284 kB
Buffers:               0 kB
Active:           245204 kB
Inactive:         199644 kB
Active(anon):     170008 kB
Inactive(anon):   154556 kB
Active(file):      75196 kB
Inactive(file):    45088 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:      11366564 kB
SwapFree:       10770844 kB
Dirty:               112 kB
Writeback:             0 kB
AnonPages:        324564 kB
Shmem:            350868 kB
Slab:            2377496 kB
SReclaimable:    2353688 kB
SUnreclaim:        23808 kB

Whereas here is the output from another server:

MemTotal:        8162232 kB
MemFree:          865056 kB
Buffers:           63320 kB
Cached:          5790844 kB
SwapCached:          264 kB
Active:          3559352 kB
Inactive:        2677916 kB
Active(anon):     356544 kB
Inactive(anon):   303672 kB
Active(file):    3202808 kB
Inactive(file):  2374244 kB
Unevictable:        6428 kB
Mlocked:            6428 kB
SwapTotal:       3998716 kB
SwapFree:        3994964 kB
Dirty:              4644 kB
Writeback:             0 kB
AnonPages:        389652 kB
Mapped:           305536 kB
Shmem:            273600 kB
Slab:             744592 kB
SReclaimable:     671484 kB
SUnreclaim:        73108 kB
KernelStack:        3552 kB
PageTables:        27500 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     8079832 kB
Committed_AS:    2727108 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      266668 kB
VmallocChunk:   34359337040 kB
AnonHugePages:    200704 kB
DirectMap4k:        3556 kB
DirectMap2M:     3129344 kB
DirectMap1G:     5242880 kB

Now in both of these cases, you won't see the MemAvailable value in them. So, I cannot use that in a plugin that is being used in all sorts of servers. In v1.6.2 (just pushed) the code has been rolled back to using MemFree instead of MemAvailable.

[inspiredearth]

It’s good you discovered that. I didn’t realise different servers product that differently. What about doing a grep for `MemAvailable`. If it is found, then use it, and it is not found, then use `MemFree `? That way, people with servers that provide `MemAvailable` can get the more useful result it provides, and people whose servers uses `MemFree` will get also the best result they can get for their environment.

…

On 6/04/2018, at 6:47 PM, Saumya Majumder ***@***.*** ***@***.***>> wrote: @jonathannz <https://github.com/jonathannz> I am sorry to say that I had to take out MemAvailable from the code. Here is the case: after your research, I did change the MemFree to MemAvailable without realizing one key thing i.e. MemAvailable is not a generic global value that is available across all system, distros, and servers. As it turns out only in few distros your will get to see that MemAvailable flag, for other that doesn't exist. Whereas MemFree is a global tag available across all distros and system no matter what you use. So, maybe it is not 100% accurate but it will at least work on all Linux servers, across all system. For example, this is the output of one of my server's meminfo file: cat /proc/meminfo MemTotal: 5683284 kB MemFree: 2510052 kB Cached: 120284 kB Buffers: 0 kB Active: 245204 kB Inactive: 199644 kB Active(anon): 170008 kB Inactive(anon): 154556 kB Active(file): 75196 kB Inactive(file): 45088 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 11366564 kB SwapFree: 10770844 kB Dirty: 112 kB Writeback: 0 kB AnonPages: 324564 kB Shmem: 350868 kB Slab: 2377496 kB SReclaimable: 2353688 kB SUnreclaim: 23808 kB Whereas here is the output from another server: MemTotal: 8162232 kB MemFree: 865056 kB Buffers: 63320 kB Cached: 5790844 kB SwapCached: 264 kB Active: 3559352 kB Inactive: 2677916 kB Active(anon): 356544 kB Inactive(anon): 303672 kB Active(file): 3202808 kB Inactive(file): 2374244 kB Unevictable: 6428 kB Mlocked: 6428 kB SwapTotal: 3998716 kB SwapFree: 3994964 kB Dirty: 4644 kB Writeback: 0 kB AnonPages: 389652 kB Mapped: 305536 kB Shmem: 273600 kB Slab: 744592 kB SReclaimable: 671484 kB SUnreclaim: 73108 kB KernelStack: 3552 kB PageTables: 27500 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 8079832 kB Committed_AS: 2727108 kB VmallocTotal: 34359738367 kB VmallocUsed: 266668 kB VmallocChunk: 34359337040 kB AnonHugePages: 200704 kB DirectMap4k: 3556 kB DirectMap2M: 3129344 kB DirectMap1G: 5242880 kB Now in both of these cases, you won't see the MemAvailable value in them. So, I cannot use that in a plugin that is being used in all sorts of servers. In v1.6.2 (just pushed) the code has been rolled back to using MemFree instead of MemAvailable. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAovA1GG4oPZYAbiy5Xg7zPJbxtgzRDjks5tlw9_gaJpZM4TDDJn>. <https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png> <https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png> <https://github.com/isaumya/wp-server-stats> <#3 (comment)>

[isaumya]

That is exactly what I did in v1.6.3
But can you please let me know what your server setup is i.e. OS, server software etc. So, that I can have a clear understanding of who gets the MemAvailable

[inspiredearth]

Ubuntu 16.04 Which is what I would think is providing the info. It is a fairly normal /default set up. On 6 April 2018 8:19:15 PM Saumya Majumder <notifications@github.com> wrote: That is exactly what I did in v1.6.3 But can you please let me know what your server setup is i.e. OS, server software etc. So, that I can have a clear understanding of who gets the MemAvailable — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.