Commit Graph

50653 Commits

Author SHA1 Message Date
Chris Dent 4e07d81260 [placement] Support forbidden traits in API
In a new microversion (1.22) expose support for processing
forbidden traits in GET /resource_providers and GET
/allocation_candidates. A forbidden trait is expressed as
part of the required parameter with a "!" prefix:

    required=CUSTOM_FAST,!CUSTOM_SLOW

This change uses db and query processing code adjustments
already present in the code but guarded by a flag. If the
currently requested microversion matches 1.22 or beyond
that flag is True, otherwise False.

Reno, api-ref update and api history update are included.
Because this microversion changes the value of an existing
parameter it was unclear how to best express that in the
api-ref. In this case existing parameter references were
annotated.

Partially implements blueprint placement-forbidden-traits

Change-Id: I43e92bc5f97db7a2b09e64c6cb953c07d0561e63
2018-04-13 19:24:08 +01:00
Chris Dent 6f31523b31 [placement] Filter allocation candidates by forbidden traits in db
Adjust the generated SQL and surrounding python code to support
expressing forbidden traits when requesting allocation candidates.
Any resource provider which has a forbidden trait will not be
included.

The RequestGroup object now supports a forbidden_traits attribute.
Code in forthcoming patches will adjust the query string processing
that creates a RequestGroup to set that attribute if the calling code
is using the right microversion.

Partially implements blueprint placement-forbidden-traits
Change-Id: I53f6fce0810c53551710407df7ab1689282e73f3
2018-04-13 19:16:44 +01:00
Chris Dent e980f6198c [placement] Filter resource providers by forbidden traits in db
This is the db-side work for filtering resource providers by
forbidden traits. Since we control whether forbidden is allowed
or not at the API layer, we can add support here in the DB layer
such that it works all the time.

Subsequent patches will turn on the the functionality in a microversion.

Partially implements blueprint placement-forbidden-traits
Change-Id: I46796d49931df20b002d1a7e6bb3a6be34dabefa
2018-04-13 19:16:38 +01:00
Chris Dent 8bde4042da [placement] Parse forbidden traits in query strings
Add support to parse_qs_request_groups to optionally process the
"required" parameter for forbidden traits, expressed as the
normal trait form prefixed with "!". The parsing removes "!"
prefixed traits from the required_traits attribute on a
RequestGroup and puts them (without the "!") in a new
forbidden_traits attribute.

The optionality allows the caller to control the processing
based on the microversion. Later code will add that to
the list resource providers and list allocation canidates
handler code, in the same microversion.

This allows declaring forbidden handling only at the API layer.
Subsequent patches will turn on forbidden handling in the object/
data layer but no forbidden traits will reach that layer until
the microversion turns on allow_forbidden in the API.

Partially implements blueprint placement-forbidden-traits
Change-Id: Icc9dc2631a0eca9e998b9ce8706c7b6920e0cb69
2018-04-13 19:16:07 +01:00
Zuul bb4e304407 Merge "libvirt: refactor get_base_config to accept host arg" 2018-04-13 18:15:31 +00:00
Zuul 037a51d4bc Merge "libvirt: move version to string in utils" 2018-04-13 18:15:10 +00:00
Zuul cf6324f0d6 Merge "Remove the branch specifier from the nova-multiattach job" 2018-04-13 18:14:58 +00:00
Zuul 4c1a1be88a Merge "Make ResourceClass.normalize_name handle sharp S" 2018-04-13 15:21:37 +00:00
Zuul e93be26907 Merge "Test case: ResourceClass.normalize_name with ß" 2018-04-13 15:19:28 +00:00
Zuul 39506303c6 Merge "Update link of metadata" 2018-04-13 15:07:41 +00:00
Zuul 04a2bfc069 Merge "tests: Fix how context managers are mocked" 2018-04-13 15:07:14 +00:00
Yikun Jiang ee29fe6d7c Address nits in Idf57fb5fbc611abb83943bd7e36d3cebf03b3977
Change-Id: Idfca23d3b65537f58e906334109112f784eb8718
2018-04-13 12:15:28 +00:00
Zuul cebc18015e Merge "trivial: fix a comment typo" 2018-04-13 10:47:53 +00:00
Zuul 2dc98cdb5d Merge "only increment disk address unit for scsi devices" 2018-04-13 10:42:14 +00:00
Stephen Finucane 5c98984bb6 tests: Fix how context managers are mocked
Two issues here:

- The mock library provides a 'mock_open' function to mock the 'open'
  builtin's context manager. We should have been using this but were
  not.
- We were mocking another context manager in a fairly strange way. This
  seems to be causing issues, possibly down to race. We shouldn't do
  that.

Fix both issues.

Change-Id: I289b56ae3f9ccd16d72b2ca317e68e6c18b8b313
Fixes-Bug: #1763535
2018-04-13 11:29:55 +01:00
Sahid Orentino Ferdjaoui 154c05f27b libvirt: refactor get_base_config to accept host arg
host arg will be used to check libvirt and QEMU version.

Change-Id: Ib592065e8088a0cb45e95eb1501ff4aa32715206
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@redhat.com>
2018-04-13 03:16:40 -04:00
Sahid Orentino Ferdjaoui d6c730cc8f libvirt: move version to string in utils
This will be used outside the libvirt driver itself in subsequent
patches, so this is factoring it out so it can be used elsewhere
instead of duplicating it.

Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@redhat.com>
Change-Id: I2e77d7ae3666a4ce9e256f7757b5d3c6ba2495dd
2018-04-13 03:16:40 -04:00
Zuul 6846d995b7 Merge "Get rid of 406 paths in report client" 2018-04-13 06:19:01 +00:00
Zuul 3eb0b6662c Merge "Move xenapi disk resizing to privsep." 2018-04-13 05:14:35 +00:00
jichen 68c32a552b Update link of metadata
commit 43f91a87cb
enhanced the metadata doc, so we can adjust other links to
this new doc.

Change-Id: I35803472666e39f3e7d947e65760d97c0a7959bb
2018-04-13 11:15:39 +08:00
Zuul dd13f62e12 Merge "Remove mox in test_neutron_security_groups.py" 2018-04-13 03:01:35 +00:00
Zuul a7b0f91e56 Merge "ProviderTree.{add|remove}_{traits|aggregates}" 2018-04-13 01:14:10 +00:00
Zuul 57d3b70932 Merge "Move get_stashed_volume_connector to compute.utils" 2018-04-13 01:05:33 +00:00
Zuul 8ca3bfa910 Merge "Remove :return from update_provider_tree docstring" 2018-04-12 23:51:38 +00:00
Michael Still fd154e71c2 Move xenapi partition copies to privsep.
I am unsure if I should be concerned about the lack of error
checking here, but this is how it was originally.

Change-Id: I9c1dfd5a90a53d90a44baae4874965bf8f48cea1
blueprint: hurrah-for-privsep
2018-04-13 07:19:39 +10:00
Michael Still e2926a2046 Sync xenapi and libvirt on what flags to pass e2fsck.
I can't see a good reason for the two drivers to use different
flags, and this simplifies the code a little.

Change-Id: I55cdc22ce8f95dff0cfd1a31b58582fc4b3ea48f
blueprint: hurrah-for-privsep-again
2018-04-13 07:15:44 +10:00
Michael Still 7b43fb4ebd Move xenapi disk resizing to privsep.
The same pattern as the rest of the changes. This means that privsep now
needs to let you pass flags to e2fsck, which I don't love and will remove
in a later patch.

Change-Id: I6c695c04ae586fec6adc354257638116277dda88
blueprint: hurrah-for-privsep
2018-04-13 07:09:58 +10:00
Zuul 159df07789 Merge "Remove mox in unit/virt/xenapi/test_vm_utils.py (3)" 2018-04-12 19:15:27 +00:00
Zuul 48c0201d9f Merge "Remove mox in unit/virt/xenapi/test_vm_utils.py (2)" 2018-04-12 19:15:15 +00:00
Zuul c701dd1a94 Merge "xenapi: Support live migration in pooled multi-nodes environment" 2018-04-12 19:12:52 +00:00
Zuul 17193fd314 Merge "uncap eventlet in nova" 2018-04-12 19:12:39 +00:00
Hongbin Lu c3a894b6af Skip placement on rebuild in same host
If the instance is rebuilt with a different image in the same
host, we don't need to call placement because there is no change
in resource consumption.

Change-Id: Ie252271ecfd38a0a1c61c26e323cc03869889f0a
Closes-Bug: #1750623
2018-04-12 19:09:52 +00:00
Matt Riedemann 406c9d13db Remove the branch specifier from the nova-multiattach job
For in-tree job definitions, the job is branch-specific
by design, so we don't need the stable branch exclusion
specifier as it won't run on those branches anyway since
it's not defined for those branches.

Related to https://storyboard.openstack.org/#!/story/2001839
Related-Bug: #1763382

Change-Id: I78dead482e2ed8262745cb08c9f4ab71035adb33
2018-04-12 10:55:48 -04:00
Matt Riedemann cb6c8ca1a7 Make the nova-multiattach job non-voting temporarily
Since Icb971831c8d4fe5f940d9e7993d53f1c3765e30f merged
in devstack to use the Queens UCA, the nova-multiattach
job has been failing to start nova-compute due to permission
errors connecting to libvirt, which probably has something
to do with the stack user not having access to the proper
libvirt(d) group.

It's weird since this was working in change
I0962474ff6dfc5fa97670c09a1af97a0f34cd54f to make the
nova-multiattach job use the Queens UCA.

Anyway, let's make it non-voting to stop the bleeding while
we can figure out the actual problem.

Change-Id: I3911c902cecfa8e66fccffd9fe0e4f462eacd588
Related-Bug: #1763382
2018-04-12 09:52:42 -04:00
Zuul 2251c6e0c9 Merge "Remove mox in unit/virt/xenapi/test_vm_utils.py (1)" 2018-04-12 11:20:39 +00:00
Zuul 52ed9866d8 Merge "Remove mox in test_virt_drivers.py" 2018-04-12 11:04:07 +00:00
jichen 6db94dae98 remove ec2 in service and cmd
ec2 was deprecated in commit
f098398a83
back to mitaka release, this patch
starts to remove ec2 services in nova.

Change-Id: I5939ef73904f1bb66ed8453c900bfb50cdb5b8b6
2018-04-12 13:10:23 +08:00
Zuul d20372445f Merge "Fix race fail in test_resize_with_reschedule_then_live_migrate" 2018-04-12 04:40:05 +00:00
Takashi NATSUME 57149d9348 Remove mox in test_neutron_security_groups.py
Replace stubs.Set with stub_out or mock in
nova/tests/unit/api/openstack/compute/test_neutron_security_groups.py.

The 'fake_get_instances_security_groups_bindings' method
in nova/tests/unit/api/openstack/compute/test_security_groups.py
is only used in test_neutron_security_groups.py.
So the method is moved to test_neutron_security_groups.py.

Change-Id: Ia2857d51f65f2acb0d22bb873461a50f8c9f9ba4
Implements: blueprint mox-removal
2018-04-11 23:51:30 +00:00
Zuul fa73e8e63d Merge "Remove mox in tests/unit/test_utils.py" 2018-04-11 21:22:39 +00:00
Zuul 2c5da2212c Merge "Imported Translations from Zanata" 2018-04-11 17:57:13 +00:00
Zuul 7891acf809 Merge "Marker reset option for nova-manage map_instances" 2018-04-11 17:57:04 +00:00
Zuul 5244d25f89 Merge "Expose shutdown retry interval as config setting" 2018-04-11 16:41:42 +00:00
Matt Riedemann 28f29ca3df Remove RequestContext.instance_lock_checked
This was initially added in change I1127e31d86a061a93a64ee1eb4a4d900d8bf49b5
but is no longer used anywhere in nova since change
Ic18017a16c5bffee85a43db65ff17283599a27ba.

Anyway, this change still allows it to be passed as a kwarg for now but will
emit a UserWarning if it is, and then we can make it a hard failure starting
in the S release.

Change-Id: Ib0b23a47c2e833073108af6700b16f8026631a83
2018-04-11 11:46:19 -04:00
Zuul cc5ac1e41f Merge "api-ref: Fix parameter order in rebuild" 2018-04-11 14:58:50 +00:00
Zuul 205286eb00 Merge "api-ref: Parameter verification for servers.inc (3/3)" 2018-04-11 14:55:16 +00:00
Zuul cbbace7177 Merge "api-ref: Parameter verification for servers.inc (2/3)" 2018-04-11 14:51:54 +00:00
Matt Riedemann 4afa8c2b97 Fix race fail in test_resize_with_reschedule_then_live_migrate
The assertion in the test that the migration status is 'completed'
is flawed in that it assumes when the instance status is 'ACTIVE'
the migration is completed, which isn't True, since the instance
status gets changed before the migration is completed, but they are
very close in time so there is a race, which is how this test slipped
by. This fixes the issue by polling the migration status until it
is actually completed or we timeout.

Change-Id: I61f745667f4c003d7e3ca6f2f9a99194930ac892
Closes-Bug: #1762876
2018-04-11 10:43:34 -04:00
Eric Fried e6dc26fe74 Remove :return from update_provider_tree docstring
During development of update_provider_tree, it was decided that we don't
need it to return anything - deciding whether things have changed is
handled by the report client.

But somehow we missed removing that bit from the docstring.  This patch
remedies that.

related to blueprint update-provider-tree

Change-Id: I199a13b0a3de9f1cbe3f6419506fe7b37ed6c4ae
2018-04-11 09:23:11 -05:00
Zuul fe976dcc55 Merge "Update wording in @safe_connect placement warnings" 2018-04-11 14:06:58 +00:00