Commit Graph

55068 Commits

Author SHA1 Message Date
Yongli He 3dcb404b1f Add server sub-resource topology API
Add a new server topology API to show server NUMA information:
  - GET /servers/{server_id}/topology

Add new policy to control the default behavior:
  - compute:server:topology:index
  - compute:server:topology:host:index

Change-Id: Ie647ef96597195b0ef00f77cece16c2bef8a78d4
Implements: blueprint show-server-numa-topology
Signed-off-by: Yongli He <yongli.he@intel.com>
2019-08-30 10:04:56 +08:00
Zuul eb6fcb2191 Merge "Change HostManager to allow scheduling to other cells" 2019-08-29 05:47:29 +00:00
Zuul 253d030c7f Merge "Remove unused args from archive_deleted_rows calls" 2019-08-29 02:45:18 +00:00
Zuul 5364b26f60 Merge "[Trivial]Remove unused helper get_allocated_disk_size" 2019-08-29 02:45:09 +00:00
Zuul 9903c8c068 Merge "[Trivial]Remove unused helper check_temp_folder" 2019-08-28 23:27:34 +00:00
Zuul 54aa420ca9 Merge "Verify archive_deleted_rows --all-cells in post test hook" 2019-08-28 20:59:27 +00:00
Zuul 9c2b1be0d6 Merge "nova-manage db archive_deleted_rows is not multi-cell aware" 2019-08-28 20:59:18 +00:00
Zuul a7be343c87 Merge "Add Destination.allow_cross_cell_move field" 2019-08-28 20:39:16 +00:00
Zuul 2f69bf6db1 Merge "Add power_on kwarg to ComputeDriver.spawn() method" 2019-08-28 20:39:06 +00:00
Zuul 3fe79c0bdb Merge "Refactor ComputeManager.remove_volume_connection" 2019-08-28 16:58:03 +00:00
Zuul 49312fe3a3 Merge "Add nova.compute.utils.delete_image" 2019-08-28 16:57:54 +00:00
Zuul 67ee187c06 Merge "Remove 'hw:cpu_policy', 'hw:mem_page_size' extra specs from API samples" 2019-08-28 16:12:39 +00:00
Zuul 1066561eba Merge "scheduler: Flatten 'ResourceRequest.from_extra_specs', 'from_image_props'" 2019-08-28 16:12:31 +00:00
zhufl d4d7831236 [Trivial]Remove unused helper get_allocated_disk_size
Helper get_allocated_disk_size is no longer used after
d41ea9d878, this is to remove it.

Change-Id: Ib7a347b4fdb49fce893e227c72bdfbee4cea00ef
2019-08-28 13:36:52 +08:00
melanie witt f5c2430876 Remove unused args from archive_deleted_rows calls
As of commit 1c9de9c777, we no longer
pass any args to the archive_deleted_rows function, so we can remove
the argument list from the function.

Change-Id: I73b2f716908088b137102631f9360939a1d7341a
2019-08-28 05:14:18 +00:00
zhufl 307cd38bbc [Trivial]Remove unused helper check_temp_folder
Helper check_temp_folder is no longer used after
806bdaf947, this is to remove it.

Change-Id: I2d965a5bae8ab7f023edcac643b989c10a085c15
2019-08-28 09:15:43 +08:00
Matt Riedemann 9fe5a17730 Change HostManager to allow scheduling to other cells
As part of the cross-cell resize work, we need to be able
to tell the scheduler that it's OK to select destinations
from cells in which the instance does not currently live.
This happens via the "allow_cross_cell_move" flag in the
RequestSpec.requested_destination field, which is not set
at this time (this is just plumbing which will be used later)
and defaults to False for backward compatibility.

Part of blueprint cross-cell-resize

Change-Id: Ib29a17f7fcc3280871d5c9ba31e1374eff98cf82
2019-08-27 14:26:33 -04:00
Matt Riedemann 656f53edc1 Add Destination.allow_cross_cell_move field
This adds a new boolean field, allow_cross_cell_move, to
the Destination object used within a RequestSpec.

This will be used during a cross-cell resize when asking
the scheduler for a target host whether or not that host
should be restricted to the current cell or if the host
can come from another cell. Furthermore, it will be used
in a scheduler weigher which will prefer candidate target
hosts in the current cell rather than a different cell.

Part of blueprint cross-cell-resize

Change-Id: I6e226321348f5d64d16b0bae26ad31c52debf0bc
2019-08-27 14:26:33 -04:00
Matt Riedemann a958dc5fcc Add power_on kwarg to ComputeDriver.spawn() method
For a cross-cell resize, the equivalent of the "finish_resize"
method on the destination compute is going to call the
driver spawn() method rather than the finish_migration()
method and needs to pass through the power_on value, similar
to finish_migration, so that when resizing a stopped server
it remains stopped once it is resized.

The finish_migration method in the driver behaves very similar
to spawn so the semantics are the same.

This change updates the spawn method signature for all in-tree
compute drivers but only implements the logic for the libvirt
driver as that is the only driver (currently) which supports
cross-cell resize (note the can_connect_volume method is also
necessary for cross-cell resize implementation in the driver).

Part of blueprint cross-cell-resize

Change-Id: I6929c588dd2e0e805f2e30b2e30d29967469d756
2019-08-27 14:26:33 -04:00
Matt Riedemann f7e7a07bdd Refactor ComputeManager.remove_volume_connection
This change does two things:

First is a simple refactor to allow passing a BDM to
_remove_volume_connection. The interface on the public
remove_volume_connection cannot be easily changed since
it's an RPC API call.

Second, the new private method gains a delete_attachment
kwarg which will tell the method whether or not to delete
a v3-style attachment, i.e. bdm.attachment_id is not None.

This is prep work for an upcoming change that is going
to call the new private method locally rather than over
RPC.

Change-Id: Ice6d5b4757134b7d1b36514744a8f958528a6a47
2019-08-27 14:26:33 -04:00
Matt Riedemann 44130b03a6 Add nova.compute.utils.delete_image
This refactors the meat from delete_image_on_error()
to nova.compute.utils so it can be re-used in other
places.

Change-Id: Idee91453289dd512eddfc4903ce00e4e20b76df8
2019-08-27 14:26:33 -04:00
zhangbailin 27b6c18c66 Specify availability_zone to unshelve
This adds support, in a new microversion, for specifying an availability
zone to the unshelve server action when the server is shelved offloaded.

Note that the functional test changes are due to those tests using the
"latest" microversion where an empty dict is not allowed for unshelve
with 2.77 so the value is changed from an empty dict to None.

Implements: blueprint support-specifying-az-when-restore-shelved-server
Closes-Bug: #1723880

Change-Id: I4b13483eef42bed91d69eabf1f30762d6866f957
2019-08-27 12:57:10 -04:00
Stephen Finucane b01676c418 Remove 'hw:cpu_policy', 'hw:mem_page_size' extra specs from API samples
In a future change, the use of 'hw:cpu_policy' will require a host to
report PCPU inventory. Rather than modify the fake driver used in these
tests to report such inventory, just use a different extra spec,
'hw:numa_nodes'. This has the added bonus of being supported by both the
libvirt and Hyper-V virt drivers, unlike 'hw:cpu_policy' and
'hw:mem_page_size', which are only supported by the libvirt virt driver.

Change-Id: Id203dc07f08557b1b094ec72e1df3493ec9524b1
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-08-27 17:00:46 +01:00
Stephen Finucane 7abe83f646 scheduler: Flatten 'ResourceRequest.from_extra_specs', 'from_image_props'
The 'ResourceRequest' object sources information from three different
attributes of an instance: the instance's image metadata properties,
the instance's flavor, this flavor's extra specs. It's possible for a
user to override resources requested via the flavor using flavor extra
specs (e.g. using the 'resources:VCPU=N' extra spec), and it's possible
to override traits requested via the flavor extra specs using image
metadata (e.g. using the 'traits_required=foo' metadata property). This
means there's an implicit hierarchy present:

- Traits: image metadata > flavor extra specs
- Resources: flavor extra specs > flavor

Previously, we pulled information from the flavor extra specs and image
metadata using two classmethods, 'from_extra_specs' and
'from_image_props', but this required a lot of glue code in between to
ensure this hierarchy was maintained. Stop doing this, preferring to
centralize everything in one location. This results in fewer LoC and a
more grokable implementation, and will make things much easier when we
start handling 'PCPU's here.

Part of blueprint cpu-resources

Change-Id: Ic0e6bc47b79711b38b2d4dabaeb5ae1dbaf2b18a
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-08-27 17:00:03 +01:00
Zuul 912a46c9d4 Merge "[Trivial]Remove unused helper get_vm_ref_from_name" 2019-08-27 11:10:56 +00:00
melanie witt 1c9de9c777 Verify archive_deleted_rows --all-cells in post test hook
We are already running archive_deleted_rows in the gate, but we are
not verifying whether all instance records, for example, were actually
successfully removed from the databases (cell0 and cell1).

This adds the --all-cells option to our archive_deleted_rows runs and
verifies that instance records were successfully removed from all cell
databases.

It is not sufficient to check only for return code 0 because
archive_deleted_rows will still return 0 when it misses archiving
records in cell databases.

Related-Bug: #1719487

Change-Id: If133b12bf02d708c099504a88b474dce0bdb0f00
2019-08-27 06:16:24 +00:00
Kevin_Zheng 97b8cb3f58 nova-manage db archive_deleted_rows is not multi-cell aware
The archive_deleted_rows cmd depend on DB connection config from config
file, and when applying super-conductor mode, there are several config
files for different cells. If so, the command can only archive rows in
cell0 DB as it only reads the nova.conf

This patch added code that provides --all-cells parameter to the
command and read info for all cells from the api_db and then archive
rows across all cells.

The --all-cells parameter is passed on to the purge command when
archive_deleted_rows is called with both --all-cells and --purge.

Co-Authored-By: melanie witt <melwittt@gmail.com>

Change-Id: Id16c3d91d9ce5db9ffd125b59fffbfedf4a6843d
Closes-Bug: #1719487
2019-08-27 06:01:58 +00:00
Zuul 90401df566 Merge "[Trivial]Remove unused helper _get_min_service_version" 2019-08-27 04:35:48 +00:00
Zuul 4b8b4217fe Merge "[Trivial]Remove unused helper get_vif_devname_with_prefix" 2019-08-27 01:34:58 +00:00
Zuul b5c86ebb0c Merge "Add test for create server with integer AZ" 2019-08-26 23:36:34 +00:00
Zuul 71478c3eed Merge "tests: Split NUMA object tests" 2019-08-26 20:41:56 +00:00
Zuul 5e740290ae Merge "Remove descriptions of nonexistent hacking rules" 2019-08-26 20:29:34 +00:00
Zuul 320175c58d Merge "Update SDK fixture for openstacksdk 0.35.0" 2019-08-26 18:46:57 +00:00
Zuul 781f47b243 Merge "Document map_instances return codes in table format" 2019-08-26 14:31:23 +00:00
Zuul 5e1de8ac24 Merge "Change nova-manage unexpected error return code to 255" 2019-08-26 14:31:15 +00:00
Zuul f4343f7b5c Merge "Document archive_deleted_rows return codes" 2019-08-26 14:26:38 +00:00
Takashi NATSUME 97a5f0e216 Remove descriptions of nonexistent hacking rules
N321, N328, N329, N330 hacking rules have been removed
since I9c334162fe1799e7b24563fdc11256b91bbafc9f.
However the descriptions are still in HACKING.rst.
So remove them.
The rule number N307 is missing in HACKING.rst.
So add it.

Change-Id: I868c421a0f5a3329ab36f786f8519accae623f1a
Closes-Bug: #1841400
2019-08-26 14:55:51 +09:00
zhufl 8cbd0bb8f8 [Trivial]Remove unused helper get_vm_ref_from_name
get_vm_ref_from_name is no longer used after 86f385b955,
this is to remove it.

Change-Id: I75897b9dcfd27bfc2277856471e4b6809859026b
2019-08-26 13:39:28 +08:00
zhufl 6723909f52 [Trivial]Remove unused helper _get_min_service_version
_get_min_service_version is no longer used after bedaeab074,
this is to remove it.

Change-Id: I9d9aa81d4bbc6c0e8935453d3c116f4b81ba147e
2019-08-26 13:31:27 +08:00
Stephen Finucane 66bc6702f0 tests: Split NUMA object tests
This makes it a little easier to reason about tests and helps us better
see where the gaps are.

Change-Id: Ib59970cce77d0276a53cfee6ff07ff64347673fc
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-08-26 10:23:29 +09:00
Ghanshyam Mann 5488c34375 Add test for create server with integer AZ
While reviewing the AZ in unshelve request BP,
matt found that there is no test case for passing
the AZ as integer value in create server request.

- https://review.opendev.org/#/c/663851/40

This commit adds the missing test case and make sure
code raise ValidationError.

Change-Id: I1cf628c490e8937e2e822957a1de173f02c69bbb
2019-08-24 03:53:17 +00:00
Zuul e493d33128 Merge "Process [compute] in $NOVA_CPU_CONF in nova-next" 2019-08-24 00:20:10 +00:00
Zuul e8a9a5798b Merge "Rework 'hardware.numa_usage_from_instances'" 2019-08-23 16:23:54 +00:00
Zuul 49572cedb9 Merge "Remove 'hardware.instance_topology_from_instance'" 2019-08-23 16:23:43 +00:00
Zuul f9b11689f4 Merge "Remove 'hardware.host_topology_and_format_from_host'" 2019-08-23 16:23:31 +00:00
Zuul 44c1430584 Merge "Remove 'hardware.get_host_numa_usage_from_instance'" 2019-08-23 16:23:19 +00:00
Zuul 9231dca5cb Merge "trivial: Rename exception argument" 2019-08-23 16:23:11 +00:00
Eric Fried 78b15e19ce Update SDK fixture for openstacksdk 0.35.0
Version 0.35.0 of openstacksdk moved a get_endpoint call out from
_make_proxy, rebreaking all our tests that don't care about SDK adapters
beyend default mocking.

This commit stubs out that get_endpoint call.

We get to stop doing this soon, I promise.

Change-Id: I18b13609768d557f2db3b5ab70d913494933dc43
2019-08-23 09:50:19 -05:00
Eric Fried d91c1626ff Process [compute] in $NOVA_CPU_CONF in nova-next
Due to bug 1802143, devstack was ignoring variables set in the
$NOVA_CPU_CONF section of local_conf. Now that that bug has been fixed
(see the Depends-On) we can put those back in their rightful place.

This patch moves the [compute]resource_provider_association_refresh
setting, introduced via [1], accordingly.

[1] https://review.opendev.org/616033

Depends-On: https://review.opendev.org/672805
Change-Id: Ifa50748b6d6e91ac954d0a7a6060505fd45716d4
Related-Bug: #1802143
2019-08-23 14:11:48 +00:00
zhufl b81d577da0 [Trivial]Remove unused helper get_vif_devname_with_prefix
get_vif_devname_with_prefix is no longer used after 54b1af686d,
so this is to remove it for code clean.

Change-Id: Iab16a15a5f141a7f7aae4172b092c50ee7964597
2019-08-23 13:28:38 +08:00