diff --git a/doc/source/_extra/.htaccess b/doc/source/_extra/.htaccess index 07e56bb782..3f4995840e 100644 --- a/doc/source/_extra/.htaccess +++ b/doc/source/_extra/.htaccess @@ -77,3 +77,4 @@ redirectmatch 301 ^/nova/([^/]+)/admin/adv-config.html$ /nova/$1/admin/index.htm redirectmatch 301 ^/nova/([^/]+)/admin/system-admin.html$ /nova/$1/admin/index.html redirectmatch 301 ^/nova/([^/]+)/admin/port_with_resource_request.html$ /nova/$1/admin/ports-with-resource-requests.html redirectmatch 301 ^/nova/([^/]+)/admin/manage-users.html$ /nova/$1/admin/arch.html +redirectmatch 301 ^/nova/([^/]+)/admin/mitigation-for-Intel-MDS-security-flaws.html /nova/$1/admin/cpu-models.html diff --git a/doc/source/admin/cpu-models.rst b/doc/source/admin/cpu-models.rst index 68680dd847..06ffdb61b6 100644 --- a/doc/source/admin/cpu-models.rst +++ b/doc/source/admin/cpu-models.rst @@ -191,3 +191,130 @@ example: As ``Haswell`` is the first CPU model supporting both of these CPU features, the instance would be configured with this model. + +.. _mitigation-for-Intel-MDS-security-flaws: + +Mitigation for MDS ("Microarchitectural Data Sampling") Security Flaws +---------------------------------------------------------------------- + +In May 2019, four new microprocessor flaws, known as `MDS`__ and also referred +to as `RIDL and Fallout`__ or `ZombieLoad`__, were discovered. +These flaws affect unpatched Nova compute nodes and instances running on Intel +x86_64 CPUs. + +.. __: https://access.redhat.com/security/vulnerabilities/mds +.. __: https://mdsattacks.com/ +.. __: https://zombieloadattack.com + +Resolution +~~~~~~~~~~ + +To get mitigation for the said MDS security flaws, a new CPU flag, +``md-clear``, needs to be exposed to the Nova instances. This can be done as +follows. + +#. Update the following components to the versions from your Linux + distribution that have fixes for the MDS flaws, on all compute nodes + with Intel x86_64 CPUs: + + - ``microcode_ctl`` + - ``kernel`` + - ``qemu-system-x86`` + - ``libvirt`` + +#. When using the libvirt driver, ensure that the CPU flag ``md-clear`` + is exposed to the Nova instances. This can be done in one of three ways, + depending on your configured CPU mode: + + #. :oslo.config:option:`libvirt.cpu_mode`\ =host-model + + When using the ``host-model`` CPU mode, the ``md-clear`` CPU flag + will be passed through to the Nova guests automatically. + + This mode is the default, when + :oslo.config:option:`libvirt.virt_type`\ =kvm|qemu is set in + ``/etc/nova/nova-cpu.conf`` on compute nodes. + + #. :oslo.config:option:`libvirt.cpu_mode`\ =host-passthrough + + When using the ``host-passthrough`` CPU mode, the ``md-clear`` CPU + flag will be passed through to the Nova guests automatically. + + #. :oslo.config:option:`libvirt.cpu_mode`\ =custom + + When using the ``custom`` CPU mode, you must *explicitly* enable the + CPU flag ``md-clear`` to the Nova instances, in addition to the + flags required for previous vulnerabilities, using the + :oslo.config:option:`libvirt.cpu_model_extra_flags`. For example: + + .. code-block:: ini + + [libvirt] + cpu_mode = custom + cpu_models = IvyBridge + cpu_model_extra_flags = spec-ctrl,ssbd,md-clear + +#. Reboot the compute node for the fixes to take effect. + + To minimize workload downtime, you may wish to live migrate all guests to + another compute node first. + +Once the above steps have been taken on every vulnerable compute node in the +deployment, each running guest in the cluster must be fully powered down, and +cold-booted (i.e. an explicit stop followed by a start), in order to activate +the new CPU models. This can be done by the guest administrators at a time of +their choosing. + +Validation +~~~~~~~~~~ + +After applying relevant updates, administrators can check the kernel's +``sysfs`` interface to see what mitigation is in place, by running the +following command on the host: + +.. code-block:: bash + + # cat /sys/devices/system/cpu/vulnerabilities/mds + Mitigation: Clear CPU buffers; SMT vulnerable + +To unpack the message "Mitigation: Clear CPU buffers; SMT vulnerable": + +- ``Mitigation: Clear CPU buffers`` means you have the "CPU buffer clearing" + mitigation enabled, which is mechanism to invoke a flush of various + exploitable CPU buffers by invoking a CPU instruction called "VERW". + +- ``SMT vulnerable`` means, depending on your workload, you may still be + vulnerable to SMT-related problems. You need to evaluate whether your + workloads need SMT (also called "Hyper-Threading") to be disabled or not. + Refer to the guidance from your Linux distribution and processor vendor. + +To see the other possible values for +``/sys/devices/system/cpu/vulnerabilities/mds``, refer to the `MDS system +information`__ section in Linux kernel's documentation for MDS. + +On the host, validate that KVM is capable of exposing the ``md-clear`` flag to +guests: + +.. code-block:: bash + + # virsh domcapabilities kvm | grep md-clear + + +More information can be found on the 'Diagnosis' tab of `this security notice +document`__. + +.. __: https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html#mds-system-information +.. __: https://access.redhat.com/security/vulnerabilities/mds + +Performance Impact +~~~~~~~~~~~~~~~~~~ + +Refer to this section titled "Performance Impact and Disabling MDS" from +`this security notice document`__, under the *Resolve* tab. + +.. note:: + + Although the article referred to is from Red Hat, the findings and + recommendations about performance impact apply for other distributions also. + +.. __: https://access.redhat.com/security/vulnerabilities/mds diff --git a/doc/source/admin/index.rst b/doc/source/admin/index.rst index ef7fb902a8..39dc76ace7 100644 --- a/doc/source/admin/index.rst +++ b/doc/source/admin/index.rst @@ -151,6 +151,5 @@ Additional guides ssh-configuration support-compute secure-live-migration-with-qemu-native-tls - mitigation-for-Intel-MDS-security-flaws vendordata hw-machine-type diff --git a/doc/source/admin/mitigation-for-Intel-MDS-security-flaws.rst b/doc/source/admin/mitigation-for-Intel-MDS-security-flaws.rst deleted file mode 100644 index b5454e8ba3..0000000000 --- a/doc/source/admin/mitigation-for-Intel-MDS-security-flaws.rst +++ /dev/null @@ -1,128 +0,0 @@ -====================================================================== -Mitigation for MDS ("Microarchitectural Data Sampling") Security Flaws -====================================================================== - -Issue -~~~~~ - -In May 2019, four new microprocessor flaws, known as `MDS -`_ , have been -discovered. These flaws affect unpatched Nova compute nodes and -instances running on Intel x86_64 CPUs. (The said MDS security flaws -are also referred to as `RIDL and Fallout `_ or -`ZombieLoad `_). - - -Resolution -~~~~~~~~~~ - -To get mitigation for the said MDS security flaws, a new CPU flag, -`md-clear`, needs to be exposed to the Nova instances. It can be done -as follows. - -(1) Update the following components to the versions from your Linux - distribution that have fixes for the MDS flaws, on all compute nodes - with Intel x86_64 CPUs: - - - microcode_ctl - - kernel - - qemu-system-x86 - - libvirt - -(2) When using the libvirt driver, ensure that the CPU flag ``md-clear`` - is exposed to the Nova instances. It can be done so in one of the - three following ways, given that Nova supports three distinct CPU - modes: - - a. :oslo.config:option:`libvirt.cpu_mode`\ =host-model - - When using ``host-model`` CPU mode, the ``md-clear`` CPU flag - will be passed through to the Nova guests automatically. - - This mode is the default, when - :oslo.config:option:`libvirt.virt_type`\ =kvm|qemu is set in - ``/etc/nova/nova-cpu.conf`` on compute nodes. - - b. :oslo.config:option:`libvirt.cpu_mode`\ =host-passthrough - - When using ``host-passthrough`` CPU mode, the ``md-clear`` CPU - flag will be passed through to the Nova guests automatically. - - c. Specific custom CPU models — this can be enabled using the - Nova config attributes :oslo.config:option:`libvirt.cpu_mode`\ =custom - plus particular named CPU models, e.g. - :oslo.config:option:`libvirt.cpu_models`\ =IvyBridge. - - (The list of all valid named CPU models that are supported by - your host, QEMU, and libvirt can be found by running the - command ``virsh domcapabilities``.) - - When using a custom CPU mode, you must *explicitly* enable the - CPU flag ``md-clear`` to the Nova instances, in addition to the - flags required for previous vulnerabilities, using the - :oslo.config:option:`libvirt.cpu_model_extra_flags`. E.g.:: - - [libvirt] - cpu_mode = custom - cpu_models = IvyBridge - cpu_model_extra_flags = spec-ctrl,ssbd,md-clear - -(3) Reboot the compute node for the fixes to take effect. (To minimize - workload downtime, you may wish to live migrate all guests to - another compute node first.) - -Once the above steps have been taken on every vulnerable compute -node in the deployment, each running guest in the cluster must be -fully powered down, and cold-booted (i.e. an explicit stop followed -by a start), in order to activate the new CPU models. This can be done -by the guest administrators at a time of their choosing. - - -Validate that the fixes are in effect -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -After applying relevant updates, administrators can check the kernel's -"sysfs" interface to see what mitigation is in place, by running the -following command (on the host):: - - # cat /sys/devices/system/cpu/vulnerabilities/mds - Mitigation: Clear CPU buffers; SMT vulnerable - -To unpack the message "Mitigation: Clear CPU buffers; SMT vulnerable": - -- The ``Mitigation: Clear CPU buffers`` bit means, you have the "CPU - buffer clearing" mitigation enabled (which is mechanism to invoke a - flush of various exploitable CPU buffers by invoking a CPU - instruction called "VERW"). - -- The ``SMT vulnerable`` bit means, depending on your workload, you may - still be vulnerable to SMT-related problems. You need to evaluate - whether your workloads need SMT (also called "Hyper-Threading") to - be disabled or not. Refer to the guidance from your Linux - distribution and processor vendor. - -To see the other possible values for the sysfs file, -``/sys/devices/system/cpu/vulnerabilities/mds``, refer to the `MDS -system information -`_ -section in Linux kernel's documentation for MDS. - -On the host, validate that KVM is capable of exposing the ``md-clear`` -flag to guests:: - - # virsh domcapabilities kvm | grep md-clear - - -Also, refer to the 'Diagnosis' tab in this security notice document -`here `_ - - -Performance Impact -~~~~~~~~~~~~~~~~~~ - -Refer to this section titled "Performance Impact and Disabling MDS" from -the security notice document `here -`_, under the -'Resolve' tab. (Note that although the article referred to is from Red -Hat, the findings and recommendations about performance impact apply -for other distributions as well.) diff --git a/doc/source/admin/security.rst b/doc/source/admin/security.rst index a7300f8eef..5743023e73 100644 --- a/doc/source/admin/security.rst +++ b/doc/source/admin/security.rst @@ -56,5 +56,4 @@ Mitigation for MDS (Microarchitectural Data Sampling) security flaws It is strongly recommended to patch all compute nodes and nova instances against the processor-related security flaws, such as MDS (and other previous vulnerabilities). For details on applying mitigation for the -MDS flaws, refer to the :doc:`mitigation-for-Intel-MDS-security-flaws` -document. +MDS flaws, refer to :ref:`mitigation-for-Intel-MDS-security-flaws`. diff --git a/doc/test/redirect-tests.txt b/doc/test/redirect-tests.txt index 283c9731e3..6731ceb7f5 100644 --- a/doc/test/redirect-tests.txt +++ b/doc/test/redirect-tests.txt @@ -77,3 +77,4 @@ /nova/latest/admin/system-admin.html 301 /nova/latest/admin/index.html /nova/latest/admin/port_with_resource_request.html 301 /nova/latest/admin/ports-with-resource-requests.html /nova/latest/admin/manage-users.html 301 /nova/latest/admin/arch.html +/nova/latest/admin/mitigation-for-Intel-MDS-security-flaws.html 301 /nova/latest/admin/cpu-models.html