11:05 a.m.
Dear all,
I just noticed a very weird problem on my workstation. Although the
machine has 64 GB of RAM installed,
$ head -n 1 /proc/meminfo
MemTotal: 2400748 kB
that is, only a few gigabytes are actually available.
Unfortunately I don't have physical access to the machine, which is a HP
desktop bought last fall. According to dmesg, the model is something like
[ 0.000000] DMI: HP HP EliteDesk 800 G3 TWR/8298, BIOS P01 Ver. 02.06
06/09/2017
Also according to dmesg, it appears that the kernel *does* see all 64
gigs, BUT OVER 60 GB IS RESERVED MEMORY!!
[ 0.000000] Memory: 2326720K/66985176K available (12300K kernel code,
1568K rwdata, 3836K rodata, 2124K init, 1368K bss, 64658456K reserved,
0K cma-reserved)
Does anyone have any idea of what could be going on?
I am running an up-to-date Fedora 27 x86_64 with kernel
4.16.15-200.fc27.x86_64.--
Susi Lehtola
Fedora Project Contributor
jussilehtola(a)fedoraproject.org
11:51 a.m.
On Thu, 2018-06-21 at 19:05 +0300, Susi Lehtola wrote:
Dear all,
I just noticed a very weird problem on my workstation. Although the
machine has 64 GB of RAM installed,
$ head -n 1 /proc/meminfo
MemTotal: 2400748 kB
that is, only a few gigabytes are actually available.
Unfortunately I don't have physical access to the machine, which is a HP
desktop bought last fall. According to dmesg, the model is something like
[ 0.000000] DMI: HP HP EliteDesk 800 G3 TWR/8298, BIOS P01 Ver. 02.06
06/09/2017
Also according to dmesg, it appears that the kernel *does* see all 64
gigs, BUT OVER 60 GB IS RESERVED MEMORY!!
[ 0.000000] Memory: 2326720K/66985176K available (12300K kernel code,
1568K rwdata, 3836K rodata, 2124K init, 1368K bss, 64658456K reserved,
0K cma-reserved)
Does anyone have any idea of what could be going on?
I am running an up-to-date Fedora 27 x86_64 with kernel
4.16.15-200.fc27.x86_64.--
AFAIK Fedora uses the available RAM as a) buffer space, and b) the /tmp
filesystem.
BTW a slightly more user-friendly way to get that info is with the
free(1) command.
poc
1:13 p.m.
On 06/21/2018 07:51 PM, Patrick O'Callaghan wrote:
AFAIK Fedora uses the available RAM as a) buffer space, and b) the
/tmp
filesystem.
BTW a slightly more user-friendly way to get that info is with the
free(1) command.
The joke is, the >60GB of missing memory doesn't even show up in free:
$ free
total used free shared buff/cache
available
Mem: 2400708 1093732 284844 121920 1022132 957848
Swap: 62499836 0 62499836
For some reason the kernel marks it as reserved.
On 06/21/2018 07:55 PM, Rick Stevens wrote:
Try installing htop ("$ sudo dnf install htop"), run it,
press F6 and
select which memory item you want to sort by. That'll tell you what
process is sucking up memory.
Same goes for htop. It's showing I have 2.29G of memory.
Update: I upgraded from Fedora 27 to Fedora 28, and the same issue
persists. Of course, the kernel is almost the same.
This has to be some weird kernel / BIOS bug...
--
Susi Lehtola
Fedora Project Contributor
jussilehtola(a)fedoraproject.org
4 p.m.
On 06/21/2018 11:13 AM, Susi Lehtola wrote:
On 06/21/2018 07:51 PM, Patrick O'Callaghan wrote:
> AFAIK Fedora uses the available RAM as a) buffer space, and b) the /tmp
> filesystem.
>
> BTW a slightly more user-friendly way to get that info is with the
> free(1) command.
The joke is, the >60GB of missing memory doesn't even show up in free:
$ free
total used free shared buff/cache
available
Mem: 2400708 1093732 284844 121920 1022132 957848
Swap: 62499836 0 62499836
For some reason the kernel marks it as reserved.
On 06/21/2018 07:55 PM, Rick Stevens wrote:
> Try installing htop ("$ sudo dnf install htop"), run it, press F6 and
> select which memory item you want to sort by. That'll tell you what
> process is sucking up memory.
Same goes for htop. It's showing I have 2.29G of memory.
Update: I upgraded from Fedora 27 to Fedora 28, and the same issue
persists. Of course, the kernel is almost the same.
This has to be some weird kernel / BIOS bug...
Can you send us the content of /proc/cmdline?
Also check any RAMdisks you may have (e.g. "df -h | grep tmpfs"). See if
any of them are possibly eating up RAM. I'd suspect /tmp and (possibly)
your swap partition.
----------------------------------------------------------------------
- Rick Stevens, Systems Engineer, AllDigital ricks(a)alldigital.com -
- AIM/Skype: therps2 ICQ: 226437340 Yahoo: origrps2 -
- -
- To understand recursion, you must first understand recursion. -
----------------------------------------------------------------------
5:24 p.m.
On 06/22/2018 12:00 AM, Rick Stevens wrote:
On 06/21/2018 11:13 AM, Susi Lehtola wrote:
> Update: I upgraded from Fedora 27 to Fedora 28, and the same issue
> persists. Of course, the kernel is almost the same.
>
> This has to be some weird kernel / BIOS bug...
Can you send us the content of /proc/cmdline?
$ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-4.16.16-300.fc28.x86_64
root=/dev/mapper/luks-1d176bad-1b27-4e38-8c18-61a8b96517db ro
rd.luks.uuid=luks-1d176bad-1b27-4e38-8c18-61a8b96517db
rd.luks.uuid=luks-aae08a03-2832-4b82-8d74-598e02c6c445 rhgb quiet
LANG=en_US.UTF-8
Also check any RAMdisks you may have (e.g. "df -h | grep
tmpfs"). See
if any of them are possibly eating up RAM. I'd suspect /tmp and
(possibly) your swap partition.
$ df -h |grep tmpfs
devtmpfs 3.9G 0 3.9G 0% /dev
tmpfs 3.9G 0 3.9G 0% /dev/shm
tmpfs 3.9G 1.8M 3.9G 1% /run
tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
tmpfs 3.9G 104K 3.9G 1% /tmp
tmpfs 789M 16K 789M 1% /run/user/42
tmpfs 789M 24K 789M 1% /run/user/1000
so this still would leave something like 40GB missing.
I opened up a bug on the kernel tracker at
https://bugzilla.redhat.com/show_bug.cgi?id=1593878
--
Susi Lehtola
Fedora Project Contributor
jussilehtola(a)fedoraproject.org
6 p.m.
On 06/21/2018 03:24 PM, Susi Lehtola wrote:
On 06/22/2018 12:00 AM, Rick Stevens wrote:
> On 06/21/2018 11:13 AM, Susi Lehtola wrote:
>> Update: I upgraded from Fedora 27 to Fedora 28, and the same issue
>> persists. Of course, the kernel is almost the same.
>>
>> This has to be some weird kernel / BIOS bug...
>
> Can you send us the content of /proc/cmdline?
$ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-4.16.16-300.fc28.x86_64
root=/dev/mapper/luks-1d176bad-1b27-4e38-8c18-61a8b96517db ro
rd.luks.uuid=luks-1d176bad-1b27-4e38-8c18-61a8b96517db
rd.luks.uuid=luks-aae08a03-2832-4b82-8d74-598e02c6c445 rhgb quiet
LANG=en_US.UTF-8
Ok, I don't see anything obnoxious there.
> Also check any RAMdisks you may have (e.g. "df -h | grep
tmpfs"). See
> if any of them are possibly eating up RAM. I'd suspect /tmp and
> (possibly) your swap partition.
$ df -h |grep tmpfs
devtmpfs 3.9G 0 3.9G 0% /dev
tmpfs 3.9G 0 3.9G 0% /dev/shm
tmpfs 3.9G 1.8M 3.9G 1% /run
tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
tmpfs 3.9G 104K 3.9G 1% /tmp
tmpfs 789M 16K 789M 1% /run/user/42
tmpfs 789M 24K 789M 1% /run/user/1000
Yeah, I don't see anything there truly worrying either. Don't know
what swap looks like (try "swapon" to see). If it's a partition
(like "dm-1" or something), that's not it, either.
so this still would leave something like 40GB missing.
Yup. It is odd. I've never seen anything like this unless you have
something weird starting up early in the boot process. Perhaps a device
driver that's got weird stuff in its configuration and is part of your
initramfs. Do an "lsmod" and see if one of the modules listed is using a
boatload of memory. If so, then check the /etc/modprobe.d/* file for it
and see if there's something odd in there.
I opened up a bug on the kernel tracker at
https://bugzilla.redhat.com/show_bug.cgi?id=1593878
Yeah, that might be your best bet.
----------------------------------------------------------------------
- Rick Stevens, Systems Engineer, AllDigital ricks(a)alldigital.com -
- AIM/Skype: therps2 ICQ: 226437340 Yahoo: origrps2 -
- -
- You possess a mind not merely twisted, but actually sprained. -
----------------------------------------------------------------------
6:50 a.m.
On Thu, Jun 21, 2018 at 09:13:00PM +0300, Susi Lehtola wrote:
On 06/21/2018 07:51 PM, Patrick O'Callaghan wrote:
> AFAIK Fedora uses the available RAM as a) buffer space, and b) the /tmp
> filesystem.
>
> BTW a slightly more user-friendly way to get that info is with the
> free(1) command.
The joke is, the >60GB of missing memory doesn't even show up in free:
$ free
total used free shared buff/cache
available
Mem: 2400708 1093732 284844 121920 1022132 957848
Swap: 62499836 0 62499836
For some reason the kernel marks it as reserved.
On 06/21/2018 07:55 PM, Rick Stevens wrote:
> Try installing htop ("$ sudo dnf install htop"), run it, press F6 and
> select which memory item you want to sort by. That'll tell you what
> process is sucking up memory.
Same goes for htop. It's showing I have 2.29G of memory.
Update: I upgraded from Fedora 27 to Fedora 28, and the same issue
persists. Of course, the kernel is almost the same.
I'd try a newer kernel from the 4.17 series from the non-official
kernel repo for Fedora, just to narrow down the issue and see, whether
it's really a kernel thing:
https://fedoraproject.org/wiki/Kernel_Vanilla_Repositories
I've had issues here, too, on F27, and at least one of them seems to
be gone with such a kernel:
% uname -r
4.17.2-250.vanilla.knurd.1.fc27.x86_64
HTH, and Regards.
Good luck!
Wolfgang
5:09 a.m.
New subject: [SOLVED] Re: >60gb of reserved memory??
On 06/22/2018 02:50 PM, Wolfgang Pfeiffer wrote:
I'd try a newer kernel from the 4.17 series from the
non-official
kernel repo for Fedora, just to narrow down the issue and see, whether
it's really a kernel thing:
https://fedoraproject.org/wiki/Kernel_Vanilla_Repositories
This turned out to be a bad proposal, since the workstation didn't boot
up anymore with the vanilla kernel.
Fortunately, the local IT support went in and updated the BIOS which
fixed the memory issue
$ free
total used free shared buff/cache
available
Mem: 65845468 690880 64087868 161676 1066720
64368104
Swap: 62499836 0 62499836
and they also booted the computer in a Fedora kernel. (I'm off-site for
a couple of months.)
I.e. the BIOS was the problem, as I had suspected.
--
Susi Lehtola
Fedora Project Contributor
jussilehtola(a)fedoraproject.org
9:05 a.m.
New subject: [SOLVED] Re: >60gb of reserved memory??
On Mon, Jun 25, 2018 at 01:09:55PM +0300, Susi Lehtola wrote:
On 06/22/2018 02:50 PM, Wolfgang Pfeiffer wrote:
>I'd try a newer kernel from the 4.17 series from the non-official
>kernel repo for Fedora, just to narrow down the issue and see, whether
>it's really a kernel thing:
>
>https://fedoraproject.org/wiki/Kernel_Vanilla_Repositories
This turned out to be a bad proposal, since the workstation didn't
boot up anymore with the vanilla kernel.
Perhaps I should have explicitly quoted here what the page also
says:
" Important notes ...
most systems work better and run in a more secure manner with the
official Fedora kernels"
Next time I might just mention exactly this ... ;)
Fortunately, the local IT support went in and updated the BIOS which
fixed the memory issue
Congratulations!
And thanks for the feed-back!
Wolfgang
11:55 a.m.
On 06/21/2018 09:05 AM, Susi Lehtola wrote:
Dear all,
I just noticed a very weird problem on my workstation. Although the
machine has 64 GB of RAM installed,
$ head -n 1 /proc/meminfo
MemTotal: 2400748 kB
that is, only a few gigabytes are actually available.
Unfortunately I don't have physical access to the machine, which is a HP
desktop bought last fall. According to dmesg, the model is something like
[ 0.000000] DMI: HP HP EliteDesk 800 G3 TWR/8298, BIOS P01 Ver. 02.06
06/09/2017
Also according to dmesg, it appears that the kernel *does* see all 64
gigs, BUT OVER 60 GB IS RESERVED MEMORY!!
[ 0.000000] Memory: 2326720K/66985176K available (12300K kernel code,
1568K rwdata, 3836K rodata, 2124K init, 1368K bss, 64658456K reserved,
0K cma-reserved)
Does anyone have any idea of what could be going on?
Try installing htop ("$ sudo dnf install htop"), run it, press F6 and
select which memory item you want to sort by. That'll tell you what
process is sucking up memory.
Not knowing what your system is doing, it's hard to predict what it'd
be. Stuff like Java and memcached are candidates, as would a big tempfs
RAMdisk (e.g. /tmp). By default, Fedora uses 50% of your RAM for /tmp
(an absolutely idiotic idea, IMHO, which is why I turn that off and use
a normal filesystem directory for /tmp).
----------------------------------------------------------------------
- Rick Stevens, Systems Engineer, AllDigital ricks(a)alldigital.com -
- AIM/Skype: therps2 ICQ: 226437340 Yahoo: origrps2 -
- -
- Give me ambiguity or give me something else! -
----------------------------------------------------------------------
5:37 p.m.
On 06/21/2018 09:55 AM, Rick Stevens wrote:
RAMdisk (e.g. /tmp). By default, Fedora uses 50% of your RAM for
/tmp
(an absolutely idiotic idea, IMHO, which is why I turn that off and use
a normal filesystem directory for /tmp).
Are you aware that the tmpfs memory is not reserved, that the size is
only a limit for how much it can use? Also, the used memory can be
swapped out if the memory pressure gets too high.
1:24 p.m.
On Thu, Jun 21, 2018 at 07:05:16PM +0300, Susi Lehtola wrote:
I just noticed a very weird problem on my workstation. Although the
machine has 64 GB of RAM installed,
$ head -n 1 /proc/meminfo
MemTotal: 2400748 kB
that is, only a few gigabytes are actually available.
Unfortunately I don't have physical access to the machine, which is
a HP desktop bought last fall. According to dmesg, the model is
something like
Is there anything about memory in /proc/cmdline?
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
1:41 p.m.
You didn't install a 32 bit version of fedora did you?
With no more PAE kernels being provided the only way to
get to big memory is with a 64 bit kernel.
3:28 p.m.
On Thu, Jun 21, 2018 at 02:41:32PM -0400, Tom Horsley wrote:
You didn't install a 32 bit version of fedora did you?
With no more PAE kernels being provided the only way to
get to big memory is with a 64 bit kernel.
From the original post, looks like 64 bit:
"I am running an up-to-date Fedora 27 x86_64 with kernel
4.16.15-200.fc27.x86_64"
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
2131
days inactive
2135
days old
14 comments
8 participants
participants (8)
-
Joe Zeff -
Matthew Miller -
Patrick O'Callaghan -
Rick Stevens -
Samuel Sieb -
Susi Lehtola -
Tom Horsley -
Wolfgang Pfeiffer