Commit Graph

59935 Commits

Author SHA1 Message Date
Zuul b542b53585 Merge "Doc follow up for PCI in placement" 2022-09-06 16:30:19 +00:00
Zuul 36091a7ed7 Merge "Follow up for the PCI in placement series" 2022-09-06 16:30:11 +00:00
Zuul 58be0ca8b8 Merge "Generate request_id for Flavor based InstancePCIRequest" 2022-09-02 13:12:56 +00:00
Balazs Gibizer efb6fd834e Doc follow up for PCI in placement
This fixes the doc comments for the already merged (or being merged)
patches in the series.

blueprint: pci-device-tracking-in-placement
Change-Id: Ia99138d603722a66c9a6ac61b035384d86ccca75
2022-09-02 14:20:18 +02:00
Balazs Gibizer 11bcf2676c Follow up for the PCI in placement series
Fixed various small issues from the already merged (or being merged)
patches.

The logic behind the dropped FIXME and empty condition are handled in
the update_allocations() calls of the translator already.

The removal of  excutils.save_and_reraise_exception from the report
client is safe as self._clear_provider_cache_for_tree does not raise.

blueprint: pci-device-tracking-in-placement
Change-Id: If87dedc6a14f7b116c4238e7534b67574428c01c
2022-09-02 14:07:32 +02:00
Zuul 90e2a5e50f Merge "Add traits for viommu model" 2022-09-02 00:26:47 +00:00
Zuul 457806e206 Merge "libvirt: Add vIOMMU device to guest" 2022-09-01 19:56:09 +00:00
Zuul 8b55b44cc6 Merge "Allow enabling PCI tracking in Placement" 2022-09-01 18:21:52 +00:00
Zuul 82498dfe4b Merge "Handle PCI dev reconf with allocations" 2022-09-01 18:21:45 +00:00
Zuul 2925fd1427 Merge "Heal allocation for same host resize" 2022-09-01 18:16:11 +00:00
Zuul 40ca5e169a Merge "Heal PCI allocation during resize" 2022-09-01 18:16:04 +00:00
Zuul 4b7c31d103 Merge "Heal missing simple PCI allocation in the resource tracker" 2022-09-01 18:15:56 +00:00
Rico Lin dc05564435 Add traits for viommu model
* Add iommu model trait for viommu model

* Add ``hw_viommu_model`` to request_filter, this will extend the
  transform_image_metadata prefilter to select host with the correct model.

* Provide new compute ``COMPUTE_VIOMMU_MODEL_*`` capablity trait for each model
  it supports in driver.

Implements: blueprint libvirt-viommu-device
Depends-On: https://review.opendev.org/c/openstack/os-traits/+/844336
Change-Id: I2caa1a9c473a2b11c287061280b4a78edb96f859
2022-09-02 02:13:20 +08:00
Zuul bcdf5988f6 Merge "Add API support for rebuilding BFV instances" 2022-09-01 07:53:12 +00:00
Zuul 7e29109edb Merge "Add conductor RPC interface for rebuild" 2022-09-01 07:51:30 +00:00
Zuul ea34d9a707 Merge "Add support for volume backed server rebuild" 2022-09-01 07:44:39 +00:00
Zuul f465bf6cd4 Merge "Add documentation and releasenotes for RBAC change" 2022-08-31 23:27:35 +00:00
Stephen Finucane 14e3b352c2 libvirt: Add vIOMMU device to guest
Implementation for BP/libvirt-viommu-device.
With provide `hw:viommu_model` property to extra_specs or
`hw_viommu_model` to image property. will enable viommu to libvirt
guest.

[1] https://www.berrange.com/posts/2017/02/16/setting-up-a-nested-kvm-guest-for-developing-testing-pci-device-assignment-with-numa/
[2] https://review.opendev.org/c/openstack/nova-specs/+/840310

Implements: blueprint libvirt-viommu-device
Change-Id: Ief9c550292788160433a28a7a1c36ba38a6bc849
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2022-09-01 04:50:16 +08:00
Balazs Gibizer 1bc1b599df Skip UnshelveToHostMultiNodesTest in nova-multi-cell
The I303a28afe69d5d17261a07fd45c4fa92bbad5598 added tempest test coverage
for the new unshelve-to-host nova feature. However the test fails in the
multi cell job as it tries to unshelve the instance to another cell
which is clearly not supported.

So this patch skips the unshelve to host test cases to unblock the gate.

Related-Bug: #1988316
Change-Id: I50c08a5dcffbf7c31bf02bdfb8615966f9271791
2022-08-31 16:52:21 +02:00
Dan Smith 45c5b80fd0 Add API support for rebuilding BFV instances
This adds a microversion and API support for triggering a rebuild
of volume-backed instances by leveraging cinder functionality to
do so.

Implements: blueprint volume-backed-server-rebuild
Closes-Bug: #1482040

Co-Authored-By: Rajat Dhasmana <rajatdhasmana@gmail.com>

Change-Id: I211ad6b8aa7856eb94bfd40e4fdb7376a7f5c358
2022-08-31 18:05:03 +05:30
whoami-rajat 6919db5612 Add conductor RPC interface for rebuild
This patch adds support for passing the ``reimage_boot_volume``
flag from the API layer through the conductor layer to the
computer layer and also includes RPC bump as necessary.

Related blueprint volume-backed-server-rebuild

Change-Id: I8daf177eb67d08112a16fe788910644abf338fa6
2022-08-31 16:38:50 +05:30
Rajat Dhasmana 30aab9c234 Add support for volume backed server rebuild
This patch adds the plumbing for rebuilding a volume backed
instance in compute code. This functionality will be enabled
in a subsequent patch which adds a new microversion and the
external support for requesting it.

The flow of the operation is as follows:

1) Create an empty attachment
2) Detach the volume
3) Request cinder to reimage the volume
4) Wait for cinder to notify success to nova (via external events)
5) Update and complete the attachment

Related blueprint volume-backed-server-rebuild

Change-Id: I0d889691de1af6875603a9f0f174590229e7be18
2022-08-31 16:38:37 +05:30
Zuul 733a87e612 Merge "Adapt websocketproxy tests for SimpleHTTPServer fix" 2022-08-30 15:24:39 +00:00
ghanshyam mann 6903456820 Add documentation and releasenotes for RBAC change
We have droped the system scope from Nova policy
and keeping the legacy admin behaviour same. This
commit adds the releasenotes and update the policy
configuration documentation accordingly.

Also, remove the upgrade check for policy which was
added for the system scope configuration protection.

Change-Id: I127cc4da689a82dbde07059de90c451eb09ea4cf
2022-08-30 01:44:33 +05:30
Balazs Gibizer ccab6fed46 Generate request_id for Flavor based InstancePCIRequest
The InstancePCIRequest.request_id is used to correlate allocated
PciDevice objects with the InstancePCIRequest object triggered the PCI
allocation. For neutron port based PCI requests the
IstancePCIRequest.request_id was already set to a generated UUID by
nova. But for Flavor based request the request_id was kept None. The
placement PCI scheduling code depends on the request_id to be a unique
identifier of the request. So this patch starts filling the request_id
for flavor based requests as well.

This change showed than in some places nova still uses the request_id ==
None condition to distinguish between flavor based and neutron based
requests. This logic is now adapted to use the newer and better
InstancePCIRequest.source based approach. Also we took the opportunity
to move the logic of querying PCI devices allocated to an instance to the
Instance ovo.

This change fills the request_id for newly created flavor based
InstancePCIRequest ovos. But the change in logic to use the
InstancePCIRequest.source property instead of the request_id == None
condition works even if the request_id is None for already existing
InstancePCIRequest objects. So this patch does not include a data
migration logic to fill request_id for existing objects.

blueprint: pci-device-tracking-in-placement
Change-Id: I53e03ff7a0221db682b043fb6d5adba3f5c9fdbe
2022-08-27 12:44:11 +02:00
Balazs Gibizer 06389f8d84 Allow enabling PCI tracking in Placement
This patch introduces the [pci]report_in_placement config option that is
False by default but if set to True will enable reporting of the PCI
passthrough inventories to Placement.

blueprint: pci-device-tracking-in-placement
Change-Id: I49a3dbf4c5708d2d92dedd29a9dc3ef25b6cd66c
2022-08-27 12:41:20 +02:00
Zuul 0bab2e5a88 Merge "Retry /reshape at provider generation conflict" 2022-08-26 17:07:56 +00:00
Balazs Gibizer 9268bc36a3 Handle PCI dev reconf with allocations
PCI devices which are allocated to instances can be removed from the
[pci]device_spec configuration or can be removed from the hypervisor
directly. The existing PciTracker code handle this cases by keeping the
PciDevice in the nova DB exists and allocated and issue a warning in the
logs during the compute service startup that nova is in an inconsistent
state. Similar behavior is now added to the PCI placement tracking code
so the PCI inventories and allocations in placement is kept in such
situation.

There is one case where we cannot simply accept the PCI device
reconfiguration by keeping the existing allocations and applying the new
config. It is when a PF that is configured and allocated is removed and
VFs from this PF is now configured in the [pci]device_spec. And vice
versa when VFs are removed and its parent PF is configured. In this case
keeping the existing inventory and allocations and adding the new inventory
to placement would result in placement model where a single PCI device
would provide both PF and VF inventories. This dependent device
configuration is not supported as it could lead to double consumption.
In such situation the compute service will refuse to start.

blueprint: pci-device-tracking-in-placement
Change-Id: Id130893de650cc2d38953cea7cf9f53af71ced93
2022-08-26 19:05:45 +02:00
Balazs Gibizer ab439dadb1 Heal allocation for same host resize
Same host resize needs special handling in the allocation healing logic
as both the source and the dest host PCI devices are visible to the
healing code as the PciDevice.instance_uuid points to the healed
instance in both cases.

blueprint: pci-device-tracking-in-placement
Change-Id: Id02e445c55fc956965b7d725f0260876d42422f2
2022-08-26 19:05:23 +02:00
Zuul e4328ed7cd Merge "Add locked_memory extra spec and image property" 2022-08-26 15:22:08 +00:00
Zuul 1dd1342d83 Merge "libvirt: Consolidate create_cow_image and create_image" 2022-08-26 14:26:48 +00:00
Zuul 86dd1d3be3 Merge "imagebackend: Add disk_info_mapping as an optional attribute of Image" 2022-08-26 14:26:40 +00:00
Zuul d405bd1194 Merge "blockinfo: Add encryption details to the disk_info mappings when provided" 2022-08-26 14:26:32 +00:00
Zuul 75e27f8c80 Merge "virt: Add block_device_info helper to find encrypted disks" 2022-08-26 14:08:35 +00:00
Zuul b17d8e4384 Merge "Move provider_tree RP creation to PciResourceProvider" 2022-08-26 05:01:37 +00:00
Zuul bac2cd4cf9 Merge "Stop if tracking is disable after it was enabled before" 2022-08-26 04:59:41 +00:00
Zuul 7175aa4c50 Merge "Support [pci]device_spec reconfiguration" 2022-08-26 04:58:50 +00:00
Zuul 8984e74112 Merge "Reject devname based device_spec config" 2022-08-26 04:58:43 +00:00
Zuul a28afc29db Merge "Ignore PCI devs with physical_network tag" 2022-08-26 04:51:12 +00:00
Zuul 8d202c13a0 Merge "Reject mixed VF rc and trait config" 2022-08-26 04:46:36 +00:00
Zuul e55e3657ea Merge "Reject PCI dependent device config" 2022-08-26 04:42:58 +00:00
Zuul baef27f504 Merge "Extend device_spec with resource_class and traits" 2022-08-25 22:54:02 +00:00
Zuul ff9041e962 Merge "Basics for PCI Placement reporting" 2022-08-25 22:33:22 +00:00
Zuul ccc06ac808 Merge "Trigger reschedule if PCI consumption fail on compute" 2022-08-25 18:06:28 +00:00
Zuul 9316a156d4 Merge "Reproduce bug 1986838" 2022-08-25 12:23:06 +00:00
Zuul 76d95d5efc Merge "Keep legacy admin behaviour in new RBAC" 2022-08-25 09:12:16 +00:00
Balazs Gibizer 98e9989cad Heal PCI allocation during resize
During resize an instance with existing PCI allocation can be changed to
consume less, more, or different PCI devices. So the heal allocation
logic needs to handle the case when an existing instance is changed to
consume different PCI devices.

This patch adds support to change existing PCI allocations in placement
during resize.

There is one limitation of the healing logic. It assumes that there is
no in-progress migration when nova is upgraded. If there is an in
progress migration, then the PCI usage will not be healed in the
migration allocation. The placement view will be consistent after such
migration is completed or reverted.

blueprint: pci-device-tracking-in-placement
Change-Id: Icc968c567f9967d7449d6c6c1f57783098e63f55
2022-08-25 10:00:11 +02:00
Balazs Gibizer d483a696a5 Heal missing simple PCI allocation in the resource tracker
There can be existing instances with PCI allocation today. When the PCI
tracking in Placement is enabled the instance allocations in Placement
needs to be updated to hold the PCI allocations as well.

This patch adds support to gathering such missing allocation and issuing
a reshape to update them in Placement. This logic will be kept in the
resource tracker until the scheduler gains support for creating PCI
allocations and such scheduler logic is made mandatory.

This patch only handles the simple cases where healing is needed because
an instance exists with PCI allocation and the whole PCI allocation is
missing from placement. More complex cases (i.e. resize to different
amount of PCI devs) will be handled by subsequent patches.

blueprint: pci-device-tracking-in-placement
Change-Id: I6d4ebcd3a4f83d4a2f937fac0b5249654cc46bc9
2022-08-25 10:00:10 +02:00
Balazs Gibizer 48229b46b4 Retry /reshape at provider generation conflict
During a normal update_available_resources run if the local provider
tree caches is invalid (i.e. due to the scheduler made an allocation
bumping the generation of the RPs) and the virt driver try to update the
inventory of an RP based on the cache Placement will report conflict,
the report client will invalidate the caches and the retry decorator
on ResourceTracker._update_to_placement will re-drive the top of the
fresh RP data.

However the same thing can happen during reshape as well but the retry
mechanism is missing in that code path so the stale caches can cause
reshape failures.

This patch adds specific error handling in the reshape code path to
implement the same retry mechanism as exists for inventory update.

blueprint: pci-device-tracking-in-placement
Change-Id: Ieb954a04e6aba827611765f7f401124a1fe298f3
2022-08-25 10:00:10 +02:00
Balazs Gibizer bdec962fd6 Move provider_tree RP creation to PciResourceProvider
It is a simple refactor to move the actual RP creation under the scope
of PciResourceProvider.update_provider_tree to remove the assumption the
RPs are always pre-created by the caller in the provider_tree.

blueprint: pci-device-tracking-in-placement
Change-Id: I375a1fb013a8e877c159b907484ba16871935879
2022-08-25 10:00:10 +02:00