Commit Graph

51962 Commits

Author SHA1 Message Date
Zuul c52cd8dac4 Merge "Fix the incorrect description and sample" 2018-08-01 14:43:09 +00:00
Zuul 1adc6ad533 Merge "Ensure the order of AllocationRequestResources" 2018-08-01 14:43:02 +00:00
Zuul e9c96374d6 Merge "[placement] Use own set_middleware_defaults" 2018-08-01 14:36:02 +00:00
Zuul 47ef500f44 Merge "Convert 'placement_api_docs' into a Sphinx extension" 2018-08-01 13:45:35 +00:00
Zuul 392b38b6f9 Merge "Fix comments in _anchors_for_sharing_providers and related test" 2018-08-01 13:45:28 +00:00
Zuul 5a76921712 Merge "[placement] Use a simplified WarningsFixture" 2018-08-01 13:32:37 +00:00
Zuul fbfe876423 Merge "[placement] Use a non-nova log capture fixture" 2018-08-01 13:32:31 +00:00
Zuul 72226f95d8 Merge "[placement] Use oslotest CaptureOutput fixture" 2018-08-01 13:31:38 +00:00
Zuul 18dcfbb0b6 Merge "Add placement.concurrent_udpate to generation pre-checks" 2018-08-01 12:19:49 +00:00
Zuul cc436c2b2a Merge "Change deprecated policies to policy" 2018-08-01 09:53:03 +00:00
Zuul 07f2fae3e1 Merge "Define common variables for irrelevant-files" 2018-08-01 09:33:51 +00:00
Zuul 6734eae416 Merge "compute node local_gb_used include swap disks" 2018-08-01 09:30:09 +00:00
Yikun Jiang 04afe2aae9 Fix comments in _anchors_for_sharing_providers and related test
1. Remove "GROUP by" line in _anchors_for_sharing_providers, because
this has been removed in Ib1738fb4a4664aa7b78398655fd23159a54f5f69.
2. Add reminder note when we are sure all root_provider_id values are
NOT NULL.
3. Fix note in test_anchors_for_sharing_providers, s1 get r3 only via
agg3.

trivialfix

Change-Id: Id8bfd83db58366047267ff0eeb2930a19bddbf4e
2018-08-01 16:34:50 +08:00
Tetsuro Nakamura 0b7946f5a3 Ensure the order of AllocationRequestResources
Getting allocation candidates with sharing providers, placement
creates a list of AllocationRequestResources to get all the
possible combinations of resource providers in the same aggregate.

However, the order of the list was arbitrary, which could cause
a bug later in duplicate check of the combination.

This patch ensures that the list is ordered by the resource
class id.

Note:
  This bug is only exposed when it is tested with python3.6,
  where order-preserving aspect is added to the dict object.

Change-Id: I2e236fbbc3a4cfd3bd66d50198de643e06d62331
Closes-Bug: #1784577
2018-08-01 15:10:41 +09:00
Zuul 2f5513172e Merge "Extract _update_to_placement method in resource tracker" 2018-08-01 05:02:54 +00:00
Zuul 372f624893 Merge "Delete orphan compute nodes before updating resources" 2018-08-01 01:41:40 +00:00
Zuul c90553b388 Merge "libvirt: Remove usage of migrateToURI{2} APIs" 2018-08-01 01:27:16 +00:00
Zuul 2cdc622799 Merge "Delete checking a bool opt of None condition" 2018-08-01 01:22:21 +00:00
Zuul 882a66427c Merge "Remove unnecessary PlacementFixture setups" 2018-07-31 23:00:34 +00:00
Zuul 5164517ce7 Merge "Don't poison Host._init_events if it's already mocked" 2018-07-31 18:28:54 +00:00
Zuul c90d6101f4 Merge "Remove redundant join in _anchors_for_sharing_providers" 2018-07-31 18:28:39 +00:00
Zuul 4456da5ad7 Merge "[placement] Retry allocation writes server side" 2018-07-31 18:28:30 +00:00
Kashyap Chamarthy 4b3e877210 libvirt: Remove usage of migrateToURI{2} APIs
The recently updated minimum required libvirt version (1.3.1; in commit
403320b -- libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION for "Rocky") brings
in the newer libvirt migration API, migrateToURI3().  The newer API was
explicitly designed[*] to be backward compatible with the older variant.

So remove the usage of the older variants:

    migrateToURI()
    migrateToURI2()

And just stick to the newer API -- migrateToURI3().

Clean up the following:

 - Add the 'migrate_disks' and 'destination_xml' paramters, and remove
   the no longer needed 'domain_xml' from the Nova migrate() method.

 - Remove or fix various unit tests to use migrateToURI3().

 - Stub nova.virt.libvirt.guest.Guest.migrate() correctly in
   nova/tests/unit/virt/test_virt_drivers.py.

[*] https://libvirt.org/git/?p=libvirt.git;a=commit;h=4bf62f4 --
    Extensible migration APIs

Change-Id: Id9ee1feeadf612fa79c3d280cee3a614a74a00a7
Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com>
2018-07-31 18:57:28 +02:00
Zuul 7b3a90f545 Merge "[placement] api-ref: add traits parameter" 2018-07-31 15:13:22 +00:00
Zuul 965eb3ff9a Merge "Fix formatting for vcpu_pin_set and reserved_huge_pages" 2018-07-31 15:13:14 +00:00
Zuul 2d0f0c64e9 Merge "Remove superfluous network stubbing in func test" 2018-07-31 14:54:39 +00:00
Balazs Gibizer 06e854bc0c Remove unnecessary PlacementFixture setups
The _IntegratedTestBase functional test base class sets up the
PlacementFixture unconditionally so the deriver test class doesn't need
to do that. This patch removes the extra setups.

Change-Id: I68b939f09fdecc59ccd258595a9b80e6f42a5df1
2018-07-31 16:31:59 +02:00
Matthew Booth 2c9fe11e8d Don't poison Host._init_events if it's already mocked
PoisonFunctions poisons Host._init_events without checking if it's
already mocked. This creates a proper headscratcher when you try to
use FakeLibvirtFixture with ServersTestBase: if you use
FakeLibvirtFixture before calling super.setUp() PoisonFunctions will
poison the function we already mocked, if we call it in the other
order setUp() will fail when it tries to start nova-compute.

Change-Id: I6d8214c75a67c87ec5a7406f3cc2e8b923aa4d84
2018-07-31 12:17:15 +00:00
Zuul 03ccb1f933 Merge "Update install guide for placement database configuration" 2018-07-31 10:32:49 +00:00
Zuul 7f9cdb5d6d Merge "Fix being able to hard reboot a pausing instance" 2018-07-31 10:32:41 +00:00
Yikun Jiang 1c66a4b029 Remove redundant join in _anchors_for_sharing_providers
There is a redundant join when we want to get id from
_anchors_for_sharing_providers. The last outerjoin is used to get the
rp.UUID according rp.id, if we set get_id=True, we no longer need this
outer join.

So, we remove the redundant join in this patch.

Change-Id: Ib5fc6e4efae29dd88ce92df834700d2121ed8076
Closes-bug: #1784604
2018-07-31 18:27:39 +08:00
Chris Dent 72e4c4c8d7 [placement] Retry allocation writes server side
This change adds a fast retry loop around
AllocationList._set_allocations if a resource provider generation
conflict happens. It turns out that under high concurrency of allocation
claims being made on the same resource provider conflicts can be quite
common and client side retries are insufficient.

Because both consumer generation and resource provider generations had
raised the same exception there was no way to distinguish between the
two so a child of ConcurrentUpdateDetected has been created as
ResourceProviderConcurrentUpdateDetected. In the future this will allow
us to send different error codes to the client as well, but that change
is not done here.

When the conflict is detected, all the resource providers in the
AllocationList are reloaded and the list objects refreshed.

Logging is provided to indicate:

* at debug that a retry is going to happen
* at warning that all the retries failed and the client is going to
  see the conflict

The tests for this are a bit funky: Some mocks are used to cause the
conflicts, then the real actions after a couple of iterations.

Change-Id: Id614d609fc8f3ed2d2ff29a2b52143f53b3b1b9a
Closes-Bug: #1719933
2018-07-31 10:57:16 +01:00
Takashi NATSUME cc7d5f836f [placement] api-ref: add traits parameter
In the "List allocation candidates" API,
the 'traits' parameter is missing.
So add it and update examples.

Change-Id: I4a307759f9e8fa80d003d92142f8e99b2c09c9fe
Partial-Bug: #1778670
2018-07-31 14:36:44 +09:00
Zuul 3ce7fa51e9 Merge "Set default of oslo.privsep.daemon logging to INFO level" 2018-07-31 02:01:17 +00:00
Zuul f6ce98749a Merge "Retry decorator fix for instances which go into ERROR state during bulk delete" 2018-07-31 01:20:01 +00:00
Zuul 4370aa9c5e Merge "Updated AggregateImagePropertiesIsolation filter illustration" 2018-07-30 21:21:53 +00:00
MultipleCrashes c6dda9f806 Retry decorator fix for instances which go into ERROR state during bulk delete
During bulk delete, some of the machines go into ERROR state
rather than being deleted. This happens once in a while during
deletion of huge number of machines concurrently. The failure occurs
during deallocation of network. At a later point in time the ERROR
state gets cleared if the user deletes the instance manually later
on. This fix proposes for retries for certain number of
time with variable delay to allow proper deallocation of network.

Change-Id: I32212b4d8180e947fdc958449aebd822f50e97fd
Closes-Bug: #1765942
2018-07-30 16:56:47 -04:00
Zuul 849ccb4bc5 Merge "Add description for placement 1.26" 2018-07-30 20:53:52 +00:00
Matt Riedemann f53d28f9be Fix formatting for vcpu_pin_set and reserved_huge_pages
This just fixes the rst formatting for the help on
each option. I'm sure there are tons more like this,
but this change isn't an attempt to fix all of the
formatting issues in our config options.

Change-Id: If52cbfc44234ce94427bd2ba78b79270c3a873d3
2018-07-30 15:36:00 -04:00
karimull a9083d13b0 Updated AggregateImagePropertiesIsolation filter illustration
The example given in this section was not matching the explanation
of how the filter works.
Updated the example with right properties and aggregate name

Change-Id: Ieadeb0d736cc83a41093e6f4dfeb75d2396976ec
Closes-Bug: #1684261
2018-07-30 12:31:35 -07:00
Chris Dent 4b1d38e88a [placement] Use a simplified WarningsFixture
Use a WarningsFixture specific to placement that worries about fewer
warnings and is not dependent on nova.

blueprint: placement-extract

Change-Id: Idfcc6882d7fe5141dcc793f0409f75c51fd26234
2018-07-30 19:38:35 +01:00
Chris Dent 1de7ac302a [placement] Use a non-nova log capture fixture
We want to avoid using the nova StandardLogging fixture, to limit
imports from nova, but it has two useful features that we want:

* always capture
* if the chosen log level is more than DEBUG, format DEBUG message
  anyway, but don't output

blueprint: placement-extract

Change-Id: Iadd32c731ebfb5a62308a4d5f907a69f93590935
2018-07-30 19:38:22 +01:00
Chris Dent bee0a133e5 [placement] Use oslotest CaptureOutput fixture
Instead of the nova fixture OutputStreamCapture. They do effectively
the same thing and once placement is extracted we'd like to not
have duplication.

blueprint: placement-extract

Change-Id: I4636533b1262f819e34ea78cca33ad9f90a35702
2018-07-30 19:12:19 +01:00
Chris Dent 45dbccd1fa [placement] Use own set_middleware_defaults
The CORS middleware needs to have its defaults set to something
reaonable. Placement had been using some code in nova to do so.
This change duplicates that code into the placement WSGI app
which had been calling it.

Change-Id: I025a2d14c4126f68e199efa54329797576c50677
blueprint: placement-extract
2018-07-30 19:00:09 +01:00
Zuul d7a87fad35 Merge "Add additional functional tests for NUMA networks" 2018-07-30 17:55:13 +00:00
Chris Dent 06ea35e36f Extract _update_to_placement method in resource tracker
Pull the placement related code in _update into its own method.

This is a nice refactoring to keep things in conceptual chunks
and will keep the diff of a forthcoming change much much smaller.

Change-Id: Ic4585ec059ab9a0c372d7a2de7324cddc4778181
2018-07-30 18:15:09 +01:00
Dan Smith 86a8aac0d7 Set default of oslo.privsep.daemon logging to INFO level
This avoids us logging the output of potentially-sensitive operations
performed with privilege.

Change-Id: I706fbd883f7640bebb3ba0b5848b863a157a9794
Closes-Bug: #1784062
2018-07-30 17:06:10 +00:00
Balazs Gibizer 25b3e2d69b Remove superfluous network stubbing in func test
The fake_network.set_stub_network_methods has been called in
ServersTestBase.setUp() and then in the derived test class
NUMAServersTest._run_build_test() method. The second call is
not needed.

Change-Id: I68d5b062c6963a37a30e0cd7520f658b1fbccd12
2018-07-30 18:40:32 +02:00
Zuul 3c6f0eb49e Merge "tox: Reuse envdirs" 2018-07-30 15:02:21 +00:00
Stephen Finucane 45662d77a2 Add additional functional tests for NUMA networks
Cover the move and move-like operations through cold migration and
rebuild tests. This requires extensive filling of gaps in the
NeutronFixture and fake libvirt driver.

Part of blueprint numa-aware-vswitches

Change-Id: I18acbda74dd8ac4e9a5d439f1393765a9a6a60f3
Co-Authored-By: Sean Mooney <work@seanmooney.info>
Co-Authored-By: Dan Smith <dms@danplanet.com>
2018-07-30 07:46:57 -07:00