Commit Graph

57940 Commits

Author SHA1 Message Date
Zuul 261de76104 Merge "Follow up for cherry-pick check for merge patch" 2020-10-12 07:26:50 +00:00
Zuul 3756f4ffa6 Merge "libvirt: 'video.vram' property must be an integer" 2020-10-10 14:40:49 +00:00
Zuul 745f835e9d Merge "libvirt: Increase incremental and max sleep time during device detach" 2020-10-10 14:40:42 +00:00
melanie witt 1e10461c71 Follow up for cherry-pick check for merge patch
This is a follow up to change
I8e4e5afc773d53dee9c1c24951bb07a45ddc2f1a which fixed an issue with
validation when the topmost patch after a Zuul rebase is a merge
patch.

We need to also use the $commit_hash variable for the check for
stable-only patches, else it will incorrectly fail because it is
checking the merge patch's commit message.

Change-Id: Ia725346b65dd5e2f16aa049c74b45d99e22b3524
2020-10-08 05:45:24 +00:00
Zuul 2745e68537 Merge "Add Python3 wallaby unit tests" 2020-10-06 23:01:27 +00:00
Stephen Finucane f2ca089bce libvirt: 'video.vram' property must be an integer
The 'vram' property of the 'video' device must be an integer else
libvirt will spit the dummy out, e.g.

  libvirt.libvirtError: XML error: cannot parse video vram '8192.0'

The division operator in Python 3 results in a float, not an integer
like in Python 2. Use the truncation division operator instead.

Change-Id: Iebf678c229da4f455459d068cafeee5f241aea1f
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Closes-Bug: #1896496
2020-10-06 09:05:23 +00:00
Lee Yarwood dd1e6d4b0c libvirt: Increase incremental and max sleep time during device detach
Bug #1894804 outlines how DEVICE_DELETED events were often missing from
QEMU on Focal based OpenStack CI hosts as originally seen in bug
 #1882521. This has eventually been tracked down to some undefined QEMU
behaviour when a new device_del QMP command is received while another is
still being processed, causing the original attempt to be aborted.

We hit this race in slower OpenStack CI envs as n-cpu rather crudely
retries attempts to detach devices using the RetryDecorator from
oslo.service. The default incremental sleep time currently being tight
enough to ensure QEMU is still processing the first device_del request
on these slower CI hosts when n-cpu asks libvirt to retry the detach,
sending another device_del to QEMU hitting the above behaviour.

Additionally we have also seen the following check being hit when
testing with QEMU >= v5.0.0. This check now rejects overlapping
device_del requests in QEMU rather than aborting the original:

https://github.com/qemu/qemu/commit/cce8944cc9efab47d4bf29cfffb3470371c3541b

This change aims to avoid this situation entirely by raising the default
incremental sleep time between detach requests from 2 seconds to 10,
leaving enough time for the first attempt to complete. The overall
maximum sleep time is also increased from 30 to 60 seconds.

Future work will aim to entirely remove this retry logic with a libvirt
event driven approach, polling for the the
VIR_DOMAIN_EVENT_ID_DEVICE_REMOVED and
VIR_DOMAIN_EVENT_ID_DEVICE_REMOVAL_FAILED events before retrying.

Finally, the cleanup of unused arguments in detach_device_with_retry is
left for a follow up change in order to keep this initial change small
enough to quickly backport.

Closes-Bug: #1882521
Related-Bug: #1894804
Change-Id: Ib9ed7069cef5b73033351f7a78a3fb566753970d
2020-10-06 09:42:26 +01:00
Zuul 655cd30c9f Merge "Update master for stable/victoria" 2020-10-06 02:28:54 +00:00
Zuul 07c68e45dd Merge "Log stack trace when fails launching a defined domain" 2020-10-06 00:30:48 +00:00
Zuul 43801cf24a Merge "libvirt: Remove MIN_{LIBVIRT,QEMU}_PMEM_SUPPORT" 2020-10-05 14:08:13 +00:00
Zuul 8bb44fd545 Merge "nova-evacuate: Disable libvirtd service and sockets during negative tests" 2020-10-04 18:14:56 +00:00
Lee Yarwood 015f7dadcc libvirt: Remove MIN_{LIBVIRT,QEMU}_PMEM_SUPPORT
I8e349849db0b1a540d295c903f1470917b82fd97 has bumped
MIN_{LIBVIRT,QEMU}_VERSION past this so remove the constant and
associated logic.

Change-Id: Ibe0af519b4428ef05f83248679fe66382d20cfa8
2020-10-02 17:02:50 +00:00
Lee Yarwood 0e7cd9d1a9 libvirt: Remove MIN_LIBVIRT_VIDEO_MODEL_VERSIONS
I8e349849db0b1a540d295c903f1470917b82fd97 has bumped MIN_LIBVIRT_VERSION
past this so remove the constant and associated logic.

Change-Id: If082b869238d08c0b713e60697185fc61c3ff99b
2020-10-02 17:02:25 +00:00
Lee Yarwood 226250beb6 nova-evacuate: Disable libvirtd service and sockets during negative tests
The recent switch to Focal introduced a change in behaviour for the
libvirtd service that can now be restarted through new systemd socket
services associated with it once stopped. As we need it to remain
stopped during the initial negative evacuation tests on the controller
we now need to also stop these socket services and then later restart
them.

Change-Id: I2333872670e9e6c905efad7461af4d149f8216b6
2020-10-02 17:01:57 +00:00
Balazs Gibizer d309e3cdf5 Temporarily disable parts of heal port allocation test
Due to bug 1894825 the nova-next job fails on master as the allocation
key cannot be deleted from the binding:profile of a neutron port. This
patch temporarily disable this test step while the bug is being fixed
and a new neutronlib is released with the fix.

Change-Id: I4dfebfb5c92dd8a5cdc779aac587e7477cd5fad6
Related-Bug: #1894825
Closes-Bug: #1898035
2020-10-02 09:39:01 +02:00
Zuul 2beb18444f Merge "libvirt: Remove MIN_LIBVIRT_BETTER_SIGKILL_HANDLING" 2020-09-29 15:19:46 +00:00
Zuul c8133fafe8 Merge "libvirt: Remove MIN_{LIBVIRT,QEMU}_NATIVE_TLS_VERSION" 2020-09-29 12:49:16 +00:00
Zuul 7832dc3311 Merge "Add reproducer for bug #1894095" 2020-09-28 17:26:52 +00:00
Lee Yarwood 6b22b697f7 libvirt: Remove MIN_LIBVIRT_BETTER_SIGKILL_HANDLING
I8e349849db0b1a540d295c903f1470917b82fd97 has bumped MIN_LIBVIRT_VERSION
past this so remove the constant and associated logic.

Change-Id: Icb2001316aa154ee7975cec806b0eaed0d953591
2020-09-28 13:17:55 +01:00
Lee Yarwood 6c8e287156 libvirt: Remove MIN_{LIBVIRT,QEMU}_NATIVE_TLS_VERSION
I8e349849db0b1a540d295c903f1470917b82fd97 has bumped
MIN_{LIBVIRT,QEMU}_VERSION past this so remove the constant and
associated logic.

Change-Id: I83f9198c6ddfd8c9ac007dafdc085ce862ed0ec2
2020-09-28 13:17:55 +01:00
Zuul 4acc112a30 Merge "libvirt: Use local variable instead of CONF" 2020-09-28 12:03:05 +00:00
Stephen Finucane 99850a0481 Add reproducer for bug #1894095
As with the resize/cold migration confirm/resize flow, the rolling back
of live migrations can be racy due to us doing various things without
the benefit of locks. Add a reproducer demonstrating this.

Change-Id: I276d2bbce4a0390a6cfd597bb51481cc53eca918
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Related-Bug: #1894095
Related-Bug: #1879878
2020-09-28 10:26:36 +01:00
Zuul d01972b272 Merge "zuul: Introduce nova-evacuate" 2020-09-25 19:40:16 +00:00
Zuul 3dbb6c18a5 Merge "libvirt: Remove MIN_LIBVIRT_FILE_BACKED_DISCARD_VERSION" 2020-09-25 15:41:30 +00:00
Zuul 1304773871 Merge "Use absolute path during qemu img rebase" 2020-09-25 15:41:13 +00:00
Zuul 162c6b67f3 Merge "libvirt: make mdev types name attribute be optional" 2020-09-25 13:33:34 +00:00
OpenStack Release Bot ed4a748bda Add Python3 wallaby unit tests
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for wallaby.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: Ia50ca4e32e0d1ebb4d98ef966c117b7c85002855
2020-09-25 08:40:43 +00:00
OpenStack Release Bot 851cc12f67 Update master for stable/victoria
Add file to the reno documentation build to show release notes for
stable/victoria.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/victoria.

Change-Id: I90b2702b83512edd04e5a85a4b8ba2fbd0f0fd6e
Sem-Ver: feature
2020-09-25 08:40:38 +00:00
Zuul 2931516b75 Merge "libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION and NEXT_MIN_{LIBVIRT,QEMU}_VERSION" 2020-09-23 22:15:00 +00:00
Lee Yarwood f357d80407 zuul: Introduce nova-evacuate
This change reworks the evacuation parts of the original
nova-live-migration job into a zuulv3 native ansible role and initial
job covering local ephemeral and iSCSI/LVM volume attached instance
evacuation. Future jobs will cover ceph and other storage backends.

Change-Id: I380e9ca1e6a84da2b2ae577fb48781bf5c740e23
2020-09-23 16:47:47 +01:00
Zuul 167472c9ac Merge "test_evacuate.sh: Stop using libvirt-bin" 2020-09-23 12:51:39 +00:00
Federico Ressi 73cc85d910 Log stack trace when fails launching a defined domain
Change-Id: I5165cebf8928c2214e28c61231baca7793bb421e
2020-09-23 12:18:12 +00:00
Zuul e1842ee739 Merge "Test for disabling greendns" 2020-09-23 11:33:53 +00:00
Sylvain Bauza 416cd1ab18 libvirt: make mdev types name attribute be optional
Some GPU drivers like i915 don't provide a name attribute for mdev types.
As we don't use this attribute yet, let's just make sure we support the fact
it's optional.

Change-Id: Ia745ed7095c74e2bfba38379e623a3f81e7799eb
Closes-Bug: #1896741
2020-09-23 12:52:52 +02:00
Zuul 5441a76b0e Merge "docs: fix aggregate weight multiplier property names" 2020-09-22 15:27:51 +00:00
Zuul 3986c5c35c Merge "doc: correct the link to user/flavor.rst" 2020-09-22 15:22:24 +00:00
Zuul ed5de21dd8 Merge "doc: Fix rendering in the accelerator support doc" 2020-09-22 15:11:43 +00:00
Artom Lifshitz 6f35e4fd2a Test for disabling greendns
In commit 7c1d964faa we fixed how we disable greendns. This patch adds
a test for this. It also lays down the groundwork for future tests
of how we manage eventlet's monkeypatching.

How and what eventlet monkeypatches can be controlled by environment
variables that are processed by eventlet at import-time (for exmaple,
EVENTLET_NO_GREENDNS). Nova manages all of this in nova.monkey_patch.
Therefore, nova.monkey_patch must be the first thing to import
eventlet. As nova.tests.functional.__init__ imports nova.monkey_patch,
our new test can go in the functional tree.

Related-bug: 1895322
Change-Id: I5b6c45b7b9a9eca3c13ecfaa5f50942922b69270
2020-09-22 10:56:42 -04:00
Lee Yarwood 34cc68ac6e libvirt: Remove MIN_LIBVIRT_FILE_BACKED_DISCARD_VERSION
I8e349849db0b1a540d295c903f1470917b82fd97 has bumped MIN_LIBVIRT_VERSION
past this so remove the constant and associated logic.

Change-Id: I952f36fe201aad37297c7630ed562b579c90f64a
2020-09-22 10:29:37 +01:00
Lee Yarwood 95103c3bc9 libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION and NEXT_MIN_{LIBVIRT,QEMU}_VERSION
Ia18e9be4d07b732afd9db0007c541253f3cdaf82 bumped these versions in
Ussuri and while it's late in the Victoria cycle we now need to do the
same. Based on the LibvirtDistroSupportMatrix wiki page [1] the
following versions have been selected for the next min versions during
the Wallaby release:

NEXT_MIN_LIBVIRT_VERSION = (6, 0, 0)
NEXT_MIN_QEMU_VERSION = (4, 2, 0)

MIN_QEMU_VTPM is removed as part of this patch as it also requires
changes to functional tests that would otherwise fail.

Follow up patches will remove the following constants after this change:

- MIN_LIBVIRT_FILE_BACKED_DISCARD_VERSION
- MIN_{LIBVIRT,QEMU}_NATIVE_TLS_VERSION
- MIN_LIBVIRT_BETTER_SIGKILL_HANDLING
- MIN_LIBVIRT_VIDEO_MODEL_VERSIONS
- MIN_{LIBVIRT,QEMU}_PMEM_SUPPORT

This change depends on the following devstack change that introduces the
use of the Train UCA on Bionic nodes, allowing us to bump versions here
ahead of a move to Focal.

Depends-On: https://review.opendev.org/#/c/747123/

[1] https://wiki.openstack.org/wiki/LibvirtDistroSupportMatrix

Change-Id: I8e349849db0b1a540d295c903f1470917b82fd97
2020-09-22 10:29:37 +01:00
Lee Yarwood 5ab9b28161 test_evacuate.sh: Stop using libvirt-bin
I49dc963ada17a595232d3eb329d94632d07b874b missed that
call_hook_if_defined will actually cause the entire run to fail [1] if
we attempt to stop the non-existent libvirt-bin service so just remove
it now we are using the train UCA.

[1] https://opendev.org/openstack/devstack-gate/src/commit/7a70f559c559e22b498d735b4ed20aadc71b7f39/functions.sh#L74

Change-Id: Ife26f1ceb6208e12328ccdccbab0681ee55d5a2a
2020-09-22 10:29:37 +01:00
Zuul b5330a97ae Merge "test_evacuate.sh: Support libvirt-bin and libvirtd systemd services" 2020-09-18 23:03:01 +00:00
Zuul e3acb21641 Merge "test_evacuate: Wait until subnode is down before starting tests" 2020-09-18 17:03:15 +00:00
Zuul 0a2403acd9 Merge "releasenote: Add known issue for bug #1894804" 2020-09-18 13:22:18 +00:00
Lee Yarwood 6c62830ae8 test_evacuate.sh: Support libvirt-bin and libvirtd systemd services
The systemd service unit for libvirtd has changed name from libvirt-bin
to libvirtd, as such the evacuation test script needs to be changed to
support both as we move between these versions.

Change-Id: I49dc963ada17a595232d3eb329d94632d07b874b
2020-09-18 13:45:39 +01:00
Zuul c49cd3b966 Merge "Victoria 22.0.0 prelude section" 2020-09-18 10:03:41 +00:00
Lee Yarwood 57ac83d4d7 releasenote: Add known issue for bug #1894804
Related-Bug: #1882521
Change-Id: Ib9059dde41b0a07144ffa26552577308b1ffc9e1
2020-09-18 10:45:43 +01:00
Takashi Natsume e3cfcb0b37 doc: Fix rendering in the accelerator support doc
Fix list rendering in the accelerator support document.

Change-Id: Ic58c5604b2e63daab7a30a95ad6a3481d96f6f63
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
2020-09-18 09:03:15 +00:00
Wang Huaqiang db7f263406 doc: correct the link to user/flavor.rst
In commit a76277f81a, the introduction
of flavor extra specs, such as 'hw:cpu_policy', 'hw:cpu_thread_policy'
and ... , have been moved to 'doc/source/user/flavors.rst' from
'doc/source/admin/flavors.rst', while in 'cpu-topologiest.rst' this
change hasn't been updated. Apply this change.

Change-Id: I031a5ea6de00a8c5cf67897ddb78075c8bc79c0b
Signed-off-by: Wang Huaqiang <huaqiang.wang@intel.com>
2020-09-18 09:02:58 +00:00
Lee Yarwood 75250d6b17 fakelibvirt: Use versionutils to set min versions found in the driver
These were previously manually converted ints based on the version
tuples present in the driver. To avoid ever needing to worry about these
again this change simply uses versionutils to covert the tuples found in
the driver into ints used by fakelibvirt.

Change-Id: I864494e11ff697788167c996a39f38d3d833d0d7
2020-09-17 21:21:28 +01:00