Using huge pages on RHEL 7 and PostgreSQL 9.4
PostgreSQL 9.4 will finally support huge pages. This article will cover about configuring huge pages on RHEL 7 box.
For some info about huge pages, please refer to this article :
As noted in the article, "Hugepages allows large amounts of memory to be utilized with a reduced overhead." The default page size is 4kB on Red Hat releases (and its derivatives). Given that PostgreSQL uses large chunks of pages in the memory, enabling/using huge pages will help improve the performance, which will help the kernel to look up less pages in total.
By default, huge pages is disabled:
In PostgreSQL 9.4, there is a new GUC called huge_pages, that controls the behaviour.
There are 3 values for this GUC: on, off and try. By default, it is set to try, which means PostgreSQL will try to use huge pages, if there are enough huge pages in the kernel, and otherwise will not use it. Off will disable, and on will force using huge pages. When this parameter is set to on, and if kernel does not have enough huge pages, PostgreSQL will fail to start:
After setting huge_pages to on in /var/lib/pgsql/9.4/data/postgresql.conf, let's try starting PostgreSQL:
Ok, it failed as expected. Let's see the status:
Ok, now, please revert this setting to off or try, and start PostgreSQL:
Great. PostgreSQL 9.4 started, with huge pages disabled. In order to see how many pages you need, first let's find the pid of postmaster process:
Now, let's find VmPeak in the proc status:
As I wrote above, the default PAGE_SIZE is 4kB on Red Hat. Let's confirm that:
Let's divide VmPeak by PAGE_SIZE. It will be 165.0859375 huge pages, so PostgreSQL will need at least 166 huge pages. On RHEL 7, the good way to enable this is using /etc/sysctl.d directory:
Please note that you will need to increase this value depending on the other kernel parameters, like increasing shared_buffers. This is a default PostgreSQL 9.4 installation, so the values are small.
Let's activate it:
... and verify it:
Ok, let's edit /var/lib/pgsql/9.4/data/postgresql.conf , change huge_pages to on, and start PostgreSQL 9.4:
PostgreSQL will start on your RHEL 7 box, and will be able to benefit from huge pages.
Enjoy!
As noted in the article, "Hugepages allows large amounts of memory to be utilized with a reduced overhead." The default page size is 4kB on Red Hat releases (and its derivatives). Given that PostgreSQL uses large chunks of pages in the memory, enabling/using huge pages will help improve the performance, which will help the kernel to look up less pages in total.
By default, huge pages is disabled:
sysctl vm.nr_hugepages
vm.nr_hugepages = 0
In PostgreSQL 9.4, there is a new GUC called huge_pages, that controls the behaviour.
There are 3 values for this GUC: on, off and try. By default, it is set to try, which means PostgreSQL will try to use huge pages, if there are enough huge pages in the kernel, and otherwise will not use it. Off will disable, and on will force using huge pages. When this parameter is set to on, and if kernel does not have enough huge pages, PostgreSQL will fail to start:
After setting huge_pages to on in /var/lib/pgsql/9.4/data/postgresql.conf, let's try starting PostgreSQL:
# systemctl start postgresql-9.4.service
Job for postgresql-9.4.service failed. See 'systemctl status postgresql-9.4.service' and 'journalctl -xn' for details.
Ok, it failed as expected. Let's see the status:
# systemctl status postgresql-9.4.service -l
postgresql-9.4.service - PostgreSQL 9.4 database server
Loaded: loaded (/usr/lib/systemd/system/postgresql-9.4.service; disabled)
Active: failed (Result: exit-code) since Fri 2014-07-04 12:46:01 EEST; 45s ago
Process: 1959 ExecStart=/usr/pgsql-9.4/bin/pg_ctl start -D ${PGDATA} -s -w -t 300 (code=exited, status=1/FAILURE)
Process: 1954 ExecStartPre=/usr/pgsql-9.4/bin/postgresql94-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
Jul 04 12:45:55 rhel-7-x86-64 systemd[1]: Starting PostgreSQL 9.4 database server...
Jul 04 12:45:56 rhel-7-x86-64 pg_ctl[1959]: < 2014-07-04 12:45:56.113 EEST >FATAL: could not map anonymous shared memory: Cannot allocate memory
Jul 04 12:45:56 rhel-7-x86-64 pg_ctl[1959]: < 2014-07-04 12:45:56.113 EEST >HINT: This error usually means that PostgreSQL's request for a shared memory segment exceeded available memory, swap space or huge pages. To reduce the request size (currently 148324352 bytes), reduce PostgreSQL's shared memory usage, perhaps by reducing shared_buffers or max_connections.
Jul 04 12:46:01 rhel-7-x86-64 pg_ctl[1959]: pg_ctl: could not start server
Jul 04 12:46:01 rhel-7-x86-64 pg_ctl[1959]: Examine the log output.
Jul 04 12:46:01 rhel-7-x86-64 systemd[1]: postgresql-9.4.service: control process exited, code=exited status=1
Jul 04 12:46:01 rhel-7-x86-64 systemd[1]: Failed to start PostgreSQL 9.4 database server.
Jul 04 12:46:01 rhel-7-x86-64 systemd[1]: Unit postgresql-9.4.service entered failed state.
Ok, now, please revert this setting to off or try, and start PostgreSQL:
# systemctl start postgresql-9.4.service
Great. PostgreSQL 9.4 started, with huge pages disabled. In order to see how many pages you need, first let's find the pid of postmaster process:
# head -n 1 /var/lib/pgsql/9.4/data/postmaster.pid
2979
Now, let's find VmPeak in the proc status:
# grep ^VmPeak /proc/2979/status
VmPeak: 338096 kB
As I wrote above, the default PAGE_SIZE is 4kB on Red Hat. Let's confirm that:
# getconf PAGE_SIZE
4096
Let's divide VmPeak by PAGE_SIZE. It will be 165.0859375 huge pages, so PostgreSQL will need at least 166 huge pages. On RHEL 7, the good way to enable this is using /etc/sysctl.d directory:
# echo "vm.nr_hugepages=166" >> /etc/sysctl.d/postgresql-9.4.conf
Please note that you will need to increase this value depending on the other kernel parameters, like increasing shared_buffers. This is a default PostgreSQL 9.4 installation, so the values are small.
Let's activate it:
# sysctl -p /etc/sysctl.d/postgresql-9.4.conf
... and verify it:
# sysctl vm.nr_hugepages
vm.nr_hugepages = 166
Ok, let's edit /var/lib/pgsql/9.4/data/postgresql.conf , change huge_pages to on, and start PostgreSQL 9.4:
systemctl start postgresql-9.4.service
PostgreSQL will start on your RHEL 7 box, and will be able to benefit from huge pages.
Enjoy!
Comments
Display comments as Linear | Threaded
Marc Linster on :
Andres Freund on :
Devrim Gündüz on :
http://www.postgresql.org/docs/9.4/static/kernel-resources.html#LINUX-HUGE-PAGES
Shish on :
Is there any problem with setting nr_hugepages too high?
Andres Freund on :
Well, VmPeak is the biggest overall size of the virtual memory allocated to a process. Including process private memory and shared memory. Now, it won't grow much for the postmaster because there seldomly will be memory allocations from it.
> Is there any problem with setting nr_hugepages too high?
The set aside memory won't be used for anything else. I.e. it's pretty much wasted.
Douglas Hunley on :
Devrim Gündüz on :
Daniel Swarbrick on :
Andrew Yoder on :
Thanks.
Douglas Hunley on :