Commit Graph

51802 Commits

Author SHA1 Message Date
Zuul d07694b28c Merge "Make host_aggregate_map dictionary case-insensitive" 2018-07-24 12:24:05 +00:00
Zuul 9cef64efab Merge "Fix confusing log message in scheduler" 2018-07-24 11:16:45 +00:00
Zuul 5486f17883 Merge "libvirt: Remove reference to transient domain when detaching devices" 2018-07-24 07:41:33 +00:00
Zuul c7144377dc Merge "Heal RequestSpec.is_bfv for legacy instances during moves" 2018-07-24 06:52:50 +00:00
Zuul 2392b4da81 Merge "Report 0 root_gb in resource tracker if instance is bfv." 2018-07-24 06:44:11 +00:00
Zuul 83057b8154 Merge "Enhance doc to guide user to use nova user" 2018-07-24 06:21:56 +00:00
Zuul eaf2a2ca65 Merge "update tox venv env to install all requirements" 2018-07-24 06:21:50 +00:00
Zuul 9d07f4f77c Merge "ironic: add instance_uuid before any other spawn activity" 2018-07-24 05:21:15 +00:00
Zuul 23695627de Merge "Move legacy-tempest-dsvm-nova-os-vif in repo" 2018-07-24 05:15:22 +00:00
Zuul 0502b0ffaf Merge "Update RequestSpec.instance_uuid during scheduling" 2018-07-24 03:48:22 +00:00
Zuul 252856ba80 Merge "Use consumer generation in _heal_allocations_for_instance" 2018-07-24 03:21:53 +00:00
Zuul f27cde5406 Merge "libvirt: use dest host port bindings during pre_live_migration" 2018-07-24 03:21:45 +00:00
Zuul a9771d118c Merge "libvirt: factor out pre_live_migration plug_vifs call" 2018-07-24 03:21:39 +00:00
Zuul 809ac51926 Merge "Add VIFMigrateData.get_dest_vif" 2018-07-24 03:14:32 +00:00
Zuul 47bab7588e Merge "Address nits from consumer generation" 2018-07-24 02:19:42 +00:00
Zuul e49c4623a7 Merge "Fix missing print format error" 2018-07-24 02:19:35 +00:00
jichenjc e0546870cd Enhance doc to guide user to use nova user
The document doesn't follow previous pattern to guide use
in using nova user to execute command and lead to issue

Change-Id: I795706a8f78ab4154bd39ce6259901800b34890e
Closes-Bug: 1781573
2018-07-23 23:48:33 +00:00
Zuul 6690d00ebf Merge "Add VIFMigrateData object for live migration" 2018-07-23 22:10:22 +00:00
Jim Rollenhagen e45c5ec819 ironic: add instance_uuid before any other spawn activity
In the baremetal use case, it is necessary to allow a remote
source of truth assert information about a physical node,
as opposed to a client asserting desired configuration.

As such, the prepare_networks_before_block_device_mapping virt
driver method was added, however in doing so network VIF attaching
was moved before actual hardware reservation. As ironic only
supports users to have a number of VIFs limited by the number
of network interfaces in the physical node, VIF attach actions
cannot be performed without first asserting control over the node.

Adding an "instance_uuid" upfront allows other ironic API consumers
to be aware that a node is now in use. Alternatively it also allows
nova to become aware prior to adding network information on the node
that the node may already be in use by an external user.

Co-Authored-By: Julia Kreger <juliaashleykreger@gmail.com>

Change-Id: I87f085589bb663c519650f307f25d087c88bbdb1
Closes-Bug: #1766301
2018-07-23 20:58:53 +00:00
Zuul a79d0e09a0 Merge "[placement] disallow additional fields in allocations" 2018-07-23 18:56:26 +00:00
Zuul b9fa24fb61 Merge "Func test for improper cn local DISK_GB reporting" 2018-07-23 18:55:03 +00:00
Zuul 43fd48ed6c Merge "Rename auth_uri to www_authenticate_uri" 2018-07-23 18:27:37 +00:00
Matt Riedemann 660e328a25 Use consumer generation in _heal_allocations_for_instance
If we're updating existing allocations for an instance due
to the project_id/user_id not matching the instance, we should
use the consumer_generation parameter, new in placement 1.28,
to ensure we don't overwrite the allocations while another
process is updating them.

As a result, the include_project_user kwarg to method
get_allocations_for_consumer is removed since nothing else
is using it now, and the minimum required version of placement
checked by nova-status is updated to 1.28.

Change-Id: I4d5f26061594fa9863c1110e6152069e44168cc3
2018-07-23 14:09:55 -04:00
Zuul 838f6dfb03 Merge "Refactor _heal_instances_in_cell" 2018-07-23 16:57:51 +00:00
Zuul c758dc6a44 Merge "perform reshaper operations in single transaction" 2018-07-23 16:57:45 +00:00
Matt Riedemann 9344c1995a Heal RequestSpec.is_bfv for legacy instances during moves
Change I9c2111f7377df65c1fc3c72323f85483b3295989 sets the
RequestSpec.is_bfv flag for newly created instances so
that scheduling (using placement) does not allocate DISK_GB
resources for the Flavor's root_gb when booting from volume.

RequestSpecs for old instances created before that change will
not have the is_bfv field set, so this change adds a check for
that in the various move operations (evacuate, unshelve, cold
migrate and live migrate) and sets the RequestSpec.is_bfv flag
accordingly.

The related functional test is updated for the legacy cold
migrate and heal scenario.

Change-Id: I8e529ad4d707b2ad012328993892db83ce464c4b
Closes-Bug: #1469179
2018-07-23 12:24:10 -04:00
Kevin_Zheng f63fd14975 Report 0 root_gb in resource tracker if instance is bfv.
hypervisor local_gb_used still reports usage even with volume-backed
instances after fix for bug 1469179.

This is because of some code in the resource tracker that is not
boot-from-volume aware when reporting disk usage, it just gets the
root_gb off the flavor:

https://github.com/openstack/nova/blob/eb4f65a7951e921b1cd8d05713e144e72f2f254f/nova/compute/resource_tracker.py#L1455

This patch simply check if the instance is bfv before returning
the result. If it's a volume-backed instance and report 0 for root_gb if
that's the case.

Change-Id: I0839470c4bcfb16590a0d87b306d683b059bf8a9
Closes-bug: #1782393
2018-07-23 10:16:39 -04:00
Lee Yarwood 636c7461de libvirt: Remove reference to transient domain when detaching devices
When detaching a device from a domain we first attempt to remove the
device from both the persistent and live configs before looping to
ensure the device has really been detached from the running live config.

Previously when this failed we logged an error message that suggested
that this was due to issues detaching the device from a transient
domain, however this is not the case as the domain is persistent.

This change simply updates the error and associated comments to only
reference the live config of the domain.

Additionally a DEBUG line claiming that a device has been successfully
detached is now only logged once the device is removed from the live
config, hopefully avoiding any confusion from this line been logged
each time an attempt is made to detach the device.

Change-Id: If869470216600c303d47cf79f12c4fc88abcf813
2018-07-23 12:34:27 +01:00
Zuul f1ec4ebe64 Merge "Merge server create for multiple-create extension" 2018-07-23 04:54:43 +00:00
Zuul c9143802b1 Merge "Add regression test for bug 1781710" 2018-07-23 02:38:43 +00:00
yanpuqing 59d5eb125b Rename auth_uri to www_authenticate_uri
Keystone deprecated 'auth_uri' to bring 'www_authenticate_uri' in
https://review.openstack.org/#/c/508522/, so 'auth_uri'
should be removed from all openstack projects.
The auth_url is the parameter passed to the keystoneauth plugin
and the auth_uri is the parameter specific to keystonemiddleware,
so the auth_url would be appropriate in api/client since that's
going to be passed straight to keystoneauth, and it should be
the same in api/client and in the fixture. So we change the
parameter to auth_url in api/client instead of www_authenticate_uri.

Change-Id: I7f878974fe8bebb8fe5803ca4e2f8839cdc971d0
Related-Bug: 1755728
2018-07-23 02:19:35 +00:00
Eric Fried ece289fa5a Func test for improper cn local DISK_GB reporting
Change I9c2111f7377df65c1fc3c72323f85483b3295989 fixed it so we stop
allocating the flavor's root_gb if we're booting from volume. However,
we're still reporting it against the compute node's capacity.

Furthermore, we're not reporting swap space at all.

This adds functional test coverage demonstrating these bugs.

Change-Id: Ia46754d8ff65c501545853e6c2909a5a900d45c0
Related-Bug: #1782393
Related-Bug: #1782386
2018-07-23 01:34:15 +00:00
Jay Pipes 19eed8592b perform reshaper operations in single transaction
Adds a nova.api.openstack.placement.objects.resource_provider.reshape()
function that accepts a dict, keyed by provider UUID, of inventory
information for a set of providers and an AllocationList object that
contains all of the rejiggered allocation records for all consumers on
the providers involved in the reshape operation.

The reshape() function is decorated with the placement API's DB
transaction context manager which will catch all exceptions and issue a
ROLLBACK of the single writer transaction that is involved in the myriad
sub-operations that happen inside reshape(). Likewise, a single COMMIT
will be executed in the writer transaction when reshape() completes
without an exception.

Change-Id: I527de486eda63b8272ffbfe42f6475907304556c
blueprint: reshape-provider-tree
2018-07-22 08:41:14 -05:00
Zuul 24bd27fcfd Merge "Rename recreate to evacuate in driver signatures" 2018-07-21 23:19:40 +00:00
Zuul 1daa6f9090 Merge "Check provider generation and retry on conflict" 2018-07-21 19:22:06 +00:00
Zuul 0807b94080 Merge "Avoid requesting DISK_GB allocation for root_gb on BFV instances" 2018-07-21 16:17:41 +00:00
Zuul 6021988a24 Merge "Disable limits if force_hosts or force_nodes is set" 2018-07-21 15:51:03 +00:00
Zuul 437cfc2c20 Merge "docs: add nova host-evacuate command to evacuate documentation" 2018-07-21 07:56:36 +00:00
Zuul 3b92f20316 Merge "Skip test_resize_server_revert_with_volume_attached in nova-lvm" 2018-07-21 02:36:34 +00:00
Dominic Schlegel 7b8e898807 docs: add nova host-evacuate command to evacuate documentation
Split up the evacuate instance documentation into two parts. One
for the existing single instance evacuation and a second new part
for the nova host-evacuate procedure.

Change-Id: Ibcdc2bc3f08e2fab23b9821feae0f489fb64a8f7
Closes-Bug: #1763039
2018-07-20 22:05:41 +00:00
Eric Fried 3518ccb665 Check provider generation and retry on conflict
Update aggregate-related scheduler report client methods to use
placement microversion 1.19, which returns provider generation in GET
/rps/{u}/aggregates and handles generation conflicts in PUT
/rps/{u}/aggregates. Helper methods previously returning aggregates and
traits now also return the generation, which is fed through
appropriately to subsequent calls. As a result, the generation kwarg is
no longer needed in _refresh_associations, so it is removed.

Doing this exposes the race described in the cited bug, so we add a
retry decorator to the resource tracker's _update and the report
client's aggregate_{add|remove}_host methods.

Related to blueprint placement-aggregate-generation
Closes-Bug: #1779931

Change-Id: I3c5fbb18297db71e682fcddb5bf4536595d92383
2018-07-20 10:09:44 -05:00
zhufl 7b8dd39985 Fix missing print format error
Missing print format will cause 'ValueError: unsupported
format character' error, this is to fix it.

Change-Id: I1a18a6357056a3a7bfa729b9fccb288437564937
2018-07-20 16:29:53 +08:00
Matt Riedemann c22b53c248 Update RequestSpec.instance_uuid during scheduling
Before change I4b67ec9dd4ce846a704d0f75ad64c41e693de0fb
the ServerGroupAntiAffinityFilter did not rely on the
HostState.instances dict to determine **within the same
request** how many members of the same anti-affinity
group were on a given host. In fact, at that time, the
HostState.instances dict wasn't updated between filter
runs in the same multi-create request. That was fixed with
change Iacc636fa8a59a9e8670a8d683c10bdbb0dc8237b so that
as we select a host for each group member being created
within a single request, we also update the HostState.instances
dict so the ServerGroupAntiAffinityFilter would have
accurate tracking of which group members were on which
hosts.

However, that did not account for a wrinkle in the filter
added in change Ie016f59f5b98bb9c70b3e33556bd747f79fc77bd
which is needed to allow resizing a server to the same host
when that server is in an anti-affinity group. That wrinkle,
combined with the fact the RequestSpec the filter is acting
upon for a given instance in a multi-create request might
not actually have the same instance_uuid can cause the filter
to think it's in a resize situation and accept a host which
already has another member from the same anti-affinity group
on it, which breaks the anti-affinity policy.

For background, during a multi-create request, we create a
RequestSpec per instance being created, but conductor only
sends the first RequestSpec for the first instance to the
scheduler. This means RequestSpec.num_instances can be >1
and we can be processing the Nth instance in the list during
scheduling but working on a RequestSpec for the first instance.
That is what breaks the resize check in ServerGroupAntiAffinityFilter
with regard to multi-create.

To resolve this, we update the RequestSpec.instance_uuid when
filtering hosts for a given instance but we don't persist that
change to the RequestSpec.

This is a bit clunky, but it kind of comes with the territory of
how we hack scheduling requests together using a single RequestSpec
for multi-create requests. An alternative to this is found in change
I0dd1fa5a70ac169efd509a50b5d69ee5deb8deb7 where we rely on the
RequestSpec.num_instances field to determine if we're in a multi-create
situation, but that has its own flaws because num_instances is
persisted with the RequestSpec which might cause us to re-introduce
bug 1558532 if we're not careful. In that case we'd have to either
(1) stop persisting RequestSpec.num_instances or (2) temporarily
unset it like we do using RequestSpec.reset_forced_destinations()
during move operations.

Co-Authored-By: Sean Mooney <work@seanmooney.info>

Closes-Bug: #1781710

Change-Id: Icba22060cb379ebd5e906981ec283667350b8c5a
2018-07-19 14:50:09 -04:00
Matt Riedemann 5e262d673b Add regression test for bug 1781710
Co-Authored-By: Sean Mooney <work@seanmooney.info>

Related-Bug: #1781710
Change-Id: I411eb17524a44da4f765fd1463e404dc15075e2c
2018-07-19 14:37:30 -04:00
Matt Riedemann c9589ed950 Skip test_resize_server_revert_with_volume_attached in nova-lvm
The libvirt driver doesn't support resize for lvm-backed instances
so we need to skip this test to get the nova-lvm job to pass again.

Change-Id: Id752b539babadd187b4c999039cc4ca655437d47
Closes-Bug: #1782607
2018-07-19 12:22:18 -04:00
xulei 1d91811ad4 Disable limits if force_hosts or force_nodes is set
Setting max_placement_results will make force_host invaild sometimes,
especially in large-scale enviroment.
Disable limit param in GET /allocation_candidates if force_hosts
or force_nodes is set.

Change-Id: Iff1b49fe7e6347e3c2bb5992494b2450809719a2
Closes-Bug: #1777591
2018-07-19 14:05:05 +00:00
Matt Riedemann a81811e111 libvirt: use dest host port bindings during pre_live_migration
This uses the destination host port binding information to plug
vifs on the destination host during pre_live_migration.

This gets "turned on" later in the series when conductor sets the
migrate_data.vifs field.

Part of blueprint neutron-new-port-binding-api

Change-Id: Ia803978401a0a9e8e28af006ffda1f4897ecd9f0
2018-07-19 09:08:12 -04:00
Matt Riedemann 5b554ed90d libvirt: factor out pre_live_migration plug_vifs call
The pre_live_migration() method is already large so this
factors out the plug_vifs() call into a separate method
since we're going to build on it in an upcoming patch.

Part of blueprint neutron-new-port-binding-api

Change-Id: Iad4edecee9866d4eadd9da2d81b6fba214578c91
2018-07-19 09:08:12 -04:00
Matt Riedemann 8d3a049507 Add VIFMigrateData.get_dest_vif
This adds a simple helper function to VIFMigrateData to
get a destination VIF object based on the set source_vif
but updated with destination port binding details. This
will be used by the libvirt driver in later patches in
the series.

Part of blueprint neutron-new-port-binding-api

Change-Id: I4c5b1fee4a4f6b6ab7b0b1bb2c1049e23a981b48
2018-07-19 09:08:12 -04:00
Sean Mooney 0a71a542e0 Add VIFMigrateData object for live migration
- This change adds a new VIFMigrateData object.
- This change adds a new vifs field to the
  base LiveMigrateData object.

Later patches will populate this information during
live migration where nova uses the neutron port
binding extended API to bind the port on the destination
host, and store the destination host binding details
in this object to be leveraged on the source host for
things like defining vif details in the guest before
starting the live migration in the hypervisor.

Note that vif_details and profile must be stored as
serialized dicts similar to
LibvirtLiveMigrateBDMInfo.connection_info_json.

Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>

Part of blueprint neutron-new-port-binding-api

Change-Id: I174ae21f143016bb23b8043fa22465dac6c71314
2018-07-19 09:08:12 -04:00