Commit Graph

55011 Commits

Author SHA1 Message Date
Eric Fried d91c1626ff Process [compute] in $NOVA_CPU_CONF in nova-next
Due to bug 1802143, devstack was ignoring variables set in the
$NOVA_CPU_CONF section of local_conf. Now that that bug has been fixed
(see the Depends-On) we can put those back in their rightful place.

This patch moves the [compute]resource_provider_association_refresh
setting, introduced via [1], accordingly.

[1] https://review.opendev.org/616033

Depends-On: https://review.opendev.org/672805
Change-Id: Ifa50748b6d6e91ac954d0a7a6060505fd45716d4
Related-Bug: #1802143
2019-08-23 14:11:48 +00:00
Zuul af3433a255 Merge "claims: Remove useless caching" 2019-08-23 03:55:20 +00:00
Zuul 2c38ef3396 Merge "libvirt: Remove unnecessary argument" 2019-08-23 03:55:11 +00:00
Zuul bfb9bee84c Merge "libvirt: Remove unnecessary try-catch around 'getCPUMap'" 2019-08-23 03:55:02 +00:00
Zuul ce487d2a08 Merge "objects: Rename 'nova.objects.instance_numa_topology'" 2019-08-23 03:54:55 +00:00
Zuul 8f2626ff7b Merge "Add docstring to check_availability_zone function" 2019-08-23 02:11:56 +00:00
Zuul 0c861c29c1 Merge "Handle websockify v0.9.0 in console proxy" 2019-08-22 23:49:25 +00:00
Zuul 170fd5a7d5 Merge "Address nits from privsep series" 2019-08-22 21:27:01 +00:00
Zuul b6774c25e9 Merge "Update docstring of 'revert_resize' function" 2019-08-22 21:07:52 +00:00
Zuul a55ae413ea Merge "doc: remove confusing docs about aggregate allocation ratios" 2019-08-22 20:13:47 +00:00
Zuul be4d0f087c Merge "Add a hacking rule for useless assertions" 2019-08-22 20:13:39 +00:00
Zuul 5559236b11 Merge "Limit get_sdk_adapter to requested service type" 2019-08-22 20:13:32 +00:00
Zuul efbe94cd7d Merge "Remove deprecated [neutron]/url option" 2019-08-22 19:57:32 +00:00
zhangbailin c77e75824f Add docstring to check_availability_zone function
Change-Id: Idc76dd50cb4808c10352f01329dab01c724c05b6
2019-08-22 18:38:51 +00:00
melanie witt 791fa595e6 Handle websockify v0.9.0 in console proxy
In websockify v0.9.0, the 'socket' staticmethod  moved from the
websockfiy.websocket.WebSocketServer class to the
websockify.websockifyserver.WebSockifyServer class [1][2], so our
import of the top-level module is no longer sufficient for accessing
the 'socket' method [3] when using v0.9.0:

 AttributeError: module 'websockify' has no attribute 'WebSocketServer'

This adds a try_import from the v0.9.0 location and uses
WebSockifyServer.socket if the module is present. Otherwise, it
will fall back on the old location. This way, we are able to run with
websockify v0.9.0 and earlier versions with the same code.

Partial-Bug: #1840788

[1] https://github.com/novnc/websockify/commit/8a697622495fd319582cd1c604e7eb2cc0ac0ef6
[2] https://github.com/novnc/websockify/commit/e47591f4aaa0221a187d3ea2f61c7ab5bb93ed54
[3] https://github.com/novnc/websockify/blob/v0.9.0/websockify/__init__.py

Change-Id: I4a50e2f772101315140df43910be2e3f69a63b73
2019-08-22 15:53:29 +00:00
Zuul 0c9a554d3f Merge "Extract new base class for provider usage functional tests" 2019-08-22 15:41:10 +00:00
Zuul f5dfce0c87 Merge "Track libvirt host/domain capabilities for multiple machine types" 2019-08-22 15:41:02 +00:00
Stephen Finucane 2d4a2c6882 claims: Remove useless caching
Nothing was ever setting the '_numa_topology_loaded' attribute which
means this caching has never worked. Just drop the idea entirely.

Change-Id: I67c1d96f5b7c7c2c171869fbd54392d118b735af
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-08-22 16:10:36 +01:00
Stephen Finucane b86670caab Update docstring of 'revert_resize' function
Migrations are resizes but with the same flavor so we use the
'revert_resize' function to revert a migration also. This isn't at all
obvious though so update the docstring to highlight this fact.

Change-Id: I96bc890a28991ddee3c672fc127956ec267afea0
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-08-22 13:55:47 +00:00
Zuul f9ba1192c4 Merge "fakelibvirt: Stop distinguishing between NUMA, non-NUMA" 2019-08-22 13:10:50 +00:00
Zuul 9e48293ec7 Merge "Make memtune parameters consistent with libvirt docs and code" 2019-08-22 13:10:42 +00:00
Zuul 44e5c88b22 Merge "FUP for I5576fa2a67d2771614266022428b4a95487ab6d5" 2019-08-22 12:15:08 +00:00
Stephen Finucane 8f6c3049a4 Address nits from privsep series
- Add missing coverage for 'iptables_set_rules' and 'iptables_get_rules'
- Use a class-level mock for common mocks
- Use 'fixtures.MockPatch' instead of 'fixtures.MonkeyPatch'
- Move exception handling for the 'ovs_plug' and 'ovs_unplug' functions
  to the functions themselves, allowing us to include the full command
  attempted in the exception message
- OvsConfigurationFailure is renamed to OVSConfigurationFailure

Change-Id: Ib8bb847598db2a6b8de95fdb222279cd22eaabcc
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-08-22 11:37:02 +01:00
Zuul 52af043940 Merge "Split fake host capabilities into reusable variables" 2019-08-22 08:13:58 +00:00
Zuul b246a592e1 Merge "Make a failure to purge_db fail in post_test_hook.sh" 2019-08-22 08:13:47 +00:00
Zuul baaf28642f Merge "Cleanup no longer required filters and add a release note." 2019-08-22 08:13:34 +00:00
Zuul 23d8316c65 Merge "We no longer need rootwrap." 2019-08-22 08:13:14 +00:00
Zuul e43840c806 Merge "Cleanup the _execute shim in nova/network." 2019-08-22 08:13:00 +00:00
Zuul 02dc8da3a8 Merge "Move final bridge commands to privsep." 2019-08-22 08:12:52 +00:00
Zuul 04046a3d88 Merge "Add a hacking rule for non-existent assertions" 2019-08-22 07:05:55 +00:00
Zuul 005a565e3f Merge "Fix missing rule description in HACKING.rst" 2019-08-22 05:28:25 +00:00
Zuul 772c911f55 Merge "Bump minimum ksa (3.16.0) and sdk (0.34.0)" 2019-08-22 00:51:15 +00:00
Zuul 7ef3774128 Merge "Re-use DB MetaData during archive_deleted_rows" 2019-08-21 22:27:40 +00:00
Zuul 1ae57fe04d Merge "Deprecate Aggregate[Core|Ram|Disk]Filters" 2019-08-21 22:27:13 +00:00
melanie witt f32671359e Make a failure to purge_db fail in post_test_hook.sh
Currently, the 'purge_db' call occurs before 'set -e', so if and when
the database purge fails (return non-zero) it does not cause the script
to exit with a failure.

This moves the call after 'set -e' to make the script exit with a
failure if the database purge step fails.

Closes-Bug: #1840967

Change-Id: I6ae27c4e11acafdc0bba8813f47059d084758b4e
2019-08-21 19:21:55 +00:00
Matt Riedemann 8f53a051cc Remove deprecated [neutron]/url option
The url option was deprecated in Queens:

  I41724a612a5f3eabd504f3eaa9d2f9d141ca3f69

The same functionality is available in the
endpoint_override option so tests and docs
are updated to use that where they were using
url before.

Note that because the logic in the get_client
method changed, some small changes were made to
the test_withtoken and test_withtoken_context_is_admin
unit tests to differentiate from when there is a
context with a token that is not admin and an
admin context that does not have a token which
was otherwise determined by asserting the default
region name.

Change-Id: I6c068a84c4c0bd88f088f9328d7897bfc1f843f1
2019-08-21 17:55:56 +00:00
Zuul 6ddfad5935 Merge "Move setting of device trust to privsep." 2019-08-21 17:31:59 +00:00
Zuul 9950b6b415 Merge "Move calls to ovs-vsctl to privsep." 2019-08-21 17:29:57 +00:00
Matt Riedemann 69faaddc6c FUP for I5576fa2a67d2771614266022428b4a95487ab6d5
This addresses a single review comment for the vPMU
flavor extra spec docs [1].

[1] https://review.opendev.org/#/c/671338/13/doc/source/user/flavors.rst@379

Change-Id: Id8ba982b0efa92630be60f79c6ecb67f7fd79100
2019-08-21 11:45:51 -04:00
Adam Spiers e4473c54de Extract new base class for provider usage functional tests
The functional test setup already implemented in
test_shared_resource_provider.py and test_report_cpu_traits.py were
near identical, and the same approach is also needed by future tests
in the AMD SEV patch series, so extract it into a new base class
called LibvirtProviderUsageBaseTestCase in a new file:

    nova/tests/functional/libvirt/integrated_helpers.py

and change the existing tests to use that.

blueprint: amd-sev-libvirt-support
Change-Id: I84375954cd1e6338b5ef857cf620616aee32bcc7
2019-08-21 15:06:20 +01:00
Adam Spiers a53c867913 Track libvirt host/domain capabilities for multiple machine types
Currently we're only calling libvirt's getDomainCapabilities API once
per architecture, with the assumption that covering a single machine
type (the default) for that architecture is enough.  However, the
default for x86_64 is 'pc', but we need domain capabilities for 'q35'
in order to allow guests with SEV or Secure Boot enabled.  So for
x86_64, we need domain capabilities for at least two machine types:
'pc' and 'q35'.

We can obtain a sensibly small list of machine types with which to
call getDomainCapabilities by looking for the 'canonical' attribute in
machine types returned from getCapabilities (N.B. not
getDomainCapabilities). For example, getCapabilities returns these:

    <machine canonical='pc-i440fx-2.11' maxCpus='255'>pc</machine>
    <machine canonical='pc-q35-2.11' maxCpus='288'>q35</machine>

So change Host.get_domain_capabilities() to call the API not just with
the default machine type, but also once per canonical machine type.

In order to obtain the canonical machine types, enhance config.py so
that it can extract them from the capabilities XML, modernize the
fixtures for i686 and x86_64 capabilities so that they include recent
versions of the 'pc' and 'q35' families of machine types, and add
corresponding tests.  A new class LibvirtConfigCapsGuestDomain is
introduced to model the elements of the capabilities XML such as
<domain type='kvm'>, and their associated machine types.  This
supersedes the previous domtype attribute of LibvirtConfigCapsGuest
objects.  Canonical machine types are tracked separately from the
others in order to allow Host.get_domain_capabilities to invoke
libvirt's getDomainCapabilities only on those types.

As before, we register both the shortened canonical (alias) machine
type (e.g. 'q35') plus its full expanded counterpart (e.g. 'pc-q35-2.11')
if that is different.

As Host.get_domain_capabilities() is already long and complex, and
needs more functionality adding to support these changes, split out
much of the code into smaller methods:

    - _get_machine_types()
    - _add_to_domain_capabilities()

The new tests require the allow_mixed_nodes option of assertXmlEqual
to be enabled to compare XML fragments with elements in different
orders, since the children of <arch> generated by
LibvirtConfigCapsGuest.format_dom have the non-canonical machine types
first, followed by the canonical ones.

The tests also require removing the 'bamboo' canonical machine type
for the ppc architecture, since the getDomainCapabilities fixture for
this architecture is static and only returns results with a fixed
'g3beige' machine type which would cause a mismatch when called with
the 'bamboo' machine type.  The same applies with the 'q35' canonical
machine type for the i686 architecture, since the static fixture for
i686 only returns results with a fixed 'pc-i440fx-2.11' machine type.

However, the x86_64 fixtures are sufficient to test these code paths,
so this is a simpler alternative to adding complexity to the i686 and
ppc fixtures.

blueprint: amd-sev-libvirt-support
blueprint: allow-secure-boot-for-qemu-kvm-guests
Change-Id: I9da9ce682dc8c8b72fb31dd4e732b556b2ed7f90
2019-08-21 14:49:43 +01:00
Boris Bobrov 7fe5f6e4fb Make memtune parameters consistent with libvirt docs and code
LibvirtConfigGuestMemoryTune created a <memtune> element with
attribute "units", although the libvirt docs mention that it should be
"unit":

    https://libvirt.org/formatdomain.html#elementsMemoryTuning

It did not cause any issues because libvirt actually accepts both,
but domains defined using "units" are automatically converted to
use "unit".

Similarly, libvirt will automatically convert "K" to "KiB".

So rename the attribute from "units" to "unit", and use the same "KiB"
unit format as elsewhere in the code, e.g. LibvirtConfigCapsNUMACell
and LibvirtConfigGuestMemoryBackingPage.

Change-Id: I1e5a28cd30124a2b08ed732d45e5504b31c97284
2019-08-21 14:37:16 +01:00
Adam Spiers 415ed543dd Split fake host capabilities into reusable variables
fakelibvirt.Connection.getCapabilities() spoofs a response to the
libvirt getCapabilities API call by hardcoding a large XML response
within itself.  Split this XML response into smaller fragments and
move them to separate variables within the recently introduced
fake_libvirt_data class so that other tests (in particular an imminent
patch for the benefit of SEV and Secure Boot which tracks libvirt
host/domain capabilities for multiple machine types) can reuse them
individually.

This has some side benefits on readability: it makes much easier to
see the overall structure of the fake <capabilities> element at a
single glance without having to scroll through pages of XML, it avoids
bloating code by mixing in large chunks of static data, and it also
places the fake capabilities data next to the fake domain capabilities
data, which is useful given the relationships between the two.

blueprint: amd-sev-libvirt-support
Change-Id: I08f9069a7735bf3a705e6bedf9defad58b9ac7e8
2019-08-21 12:54:08 +01:00
Zuul 7bf7597601 Merge "Libvirt: add support for vPMU configuration." 2019-08-21 11:15:30 +00:00
Zuul 3c6cea360b Merge "[Trivial]Remove unused helper _get_instances_by_filters" 2019-08-21 09:56:26 +00:00
Zuul 11098edeca Merge "Add blocker migration for completing services.uuid migration" 2019-08-21 05:50:15 +00:00
Zuul a64d6f98d8 Merge "doc: Trivial fixes to API version history" 2019-08-21 05:46:15 +00:00
Takashi NATSUME 2967897fa0 Add a hacking rule for useless assertions
Add a hacking rule for useless assertions in tests.

[N365] Misuse of assertTrue/assertIsNone

Change-Id: I3f76d57d75a266eddf7a4100c0b39fabe346e71c
2019-08-21 14:42:53 +09:00
Takashi NATSUME d4ed9ed93f Add a hacking rule for non-existent assertions
Add a hacking rule for non-existent mock assertion methods and
attributes.

[N364] Non existent mock assertion method or attribute (<name>) is used.
       Check a typo or whether the assertion method should begin with
       'assert_'.

Change-Id: Ic6860e373120086a1a2ae9953f09a7bbaa032a7b
2019-08-21 05:23:02 +00:00
Zuul 14e5042fa3 Merge "[Trivial]Remove unused helper filter_and_format_resource_metadata" 2019-08-21 02:21:04 +00:00