Commit Graph

51669 Commits

Author SHA1 Message Date
Zuul 30b552da76 Merge "z/VM Driver: Initial change set of z/VM driver" 2018-07-16 16:34:00 +00:00
Zuul 529621bbb1 Merge "Transform instance.live_migration_force_complete notification" 2018-07-16 16:33:35 +00:00
Zuul d8319d8339 Merge "do not assume 1 consumer in AllocList.delete_all()" 2018-07-16 15:43:52 +00:00
Zuul ae40af621f Merge "Microversion 2.64 - Use new format policy in server group" 2018-07-16 15:41:57 +00:00
Zuul 16a2db6f23 Merge "Transform aggregate.update_prop notification" 2018-07-16 15:15:32 +00:00
Zuul 6ca30fd832 Merge "Resource_provider API handler does not return specific error codes" 2018-07-16 14:52:50 +00:00
Zuul f2b96588ef Merge "network: Always retrieve network information if available" 2018-07-16 12:18:53 +00:00
Huang Rui e5acf4f961 z/VM Driver: Initial change set of z/VM driver
This is the first change that implements basic virt.driver methods
to allow nova-compute process start successfully.

A set of subsequent changes will implement spawn, snapshot, destroy
and instance power actions.

Change-Id: Ica6117c2c64f7518b78b7fb02487622250638e88
blueprint: add-zvm-driver-rocky
2018-07-16 11:35:10 +08:00
Zuul 6ee633a971 Merge "Skip more rebuild tests for cells v1 job" 2018-07-15 15:17:21 +00:00
Gábor Antal 23efc549a1 Transform instance.live_migration_force_complete notification
The instance.live_migration_force_complete.start and
instance.live_migration_force_complete.end notifications have been
transformed to the versioned notification framework.

Co-Authored-By: Takashi Natsume <natsume.takashi@lab.ntt.co.jp>
Change-Id: I2e9a474457a7597a9febc111b13a67ddcf45f29e
Implements: bp versioned-notification-transformation-rocky
2018-07-16 00:12:37 +09:00
Gábor Antal 6322cb6a18 Transform aggregate.update_prop notification
The aggregate.update_prop.start and aggregate.update_prop.end
notifications has been transformed to the versioned notification
framework.

Co-Authored-By: Takashi Natsume <natsume.takashi@lab.ntt.co.jp>
Change-Id: I37b19573b6d0e1131c446fcec361f01fa2560f82
Implements: bp versioned-notification-transformation-rocky
2018-07-15 23:36:31 +09:00
Zuul 202c147f3c Merge "Transform instance.live_migration_rollback_dest notification" 2018-07-15 13:25:45 +00:00
Zuul 13dbe1a58d Merge "Transform instance.live_migration_post notification" 2018-07-15 13:14:54 +00:00
Zuul cb0bf8a5e1 Merge "ironic: Log an error when API version is not available" 2018-07-15 07:49:44 +00:00
Zuul 629dc4d008 Merge "virt/ironic: Implement rescue and unrescue" 2018-07-15 07:49:36 +00:00
Zuul f75fcc6ce5 Merge "ironic: provide facilities to gracefully navigate versions" 2018-07-15 01:19:13 +00:00
Zuul 039f7e055e Merge "Transform instance.rebuild_scheduled notification" 2018-07-14 17:20:16 +00:00
Zuul 21a368e1a6 Merge "Heal allocations with incomplete consumer information" 2018-07-13 19:26:59 +00:00
Matt Riedemann 4a1b08365c Skip more rebuild tests for cells v1 job
This skips a couple more tempest rebuild tests for
latent intermittent rebuild race failures
due to status sync delays with cells v1.

Change-Id: Ib2dcbba7f447f54c36877a4e7c29d1b6839a0a80
Related-Bug: #1709985
2018-07-13 15:53:11 +00:00
Matt Riedemann 6b6d81cf2b Heal allocations with incomplete consumer information
Allocations created before microversion 1.8 didn't have project_id
/ user_id consumer information. In Rocky those will be migrated
to have consumer records, but using configurable sentinel values.

As part of heal_allocations, we can detect this and heal the
allocations using the instance.project_id/user_id information.

This is something we'd need if we ever use Placement allocation
information counting quotas.

Note that we should be using Placement API version 1.28 with
consumer_generation when updating the allocations, but since
people might backport this change the usage of consumer
generations is left for a follow up patch.

Related to blueprint add-consumer-generation

Change-Id: Idba40838b7b1d5389ab308f2ea40e28911aecffa
2018-07-13 11:29:54 -04:00
Zuul 706257d20b Merge "Adapt _validate_instance_group_policy to new policy model" 2018-07-13 13:30:59 +00:00
OpenStack Proposal Bot 9b5d1d9ae8 Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I3a294bcbbedea827d4cc98419c9ffb5dbc3f7f97
2018-07-13 06:58:20 +00:00
Hironori Shiina 1db9f80a29 ironic: Log an error when API version is not available
This patch adds logging when ironic doesn't support a required API
version for rescuing and unrescuing.

Change-Id: Ibabfc439e9d426aeabd54877a82e27f02f6ddb5a
Implements: blueprint ironic-rescue-mode
2018-07-13 15:41:48 +09:00
Zuul 00fa50c8da Merge "Change the ServerGroupAntiAffinityFilter to adapt to new policy" 2018-07-13 02:59:44 +00:00
Zuul e13db24c48 Merge "Add policy field to ServerGroup notification object" 2018-07-13 02:45:55 +00:00
Zuul 63bc42f6b9 Merge "Follow up for Ie49d605c66062d2548241d7e04f5a2a6b98c011e" 2018-07-13 02:45:44 +00:00
Yikun Jiang 5cdb1ce26b Microversion 2.64 - Use new format policy in server group
Enable users to define the policy rules on server group policy
to meet more advanced policy requirement. This microversion
brings the following changes in server group APIs:

* Add  ``policy`` and ``rules`` fields in the request of POST
  ``/os-server-groups``.
* The ``policy`` and ``rules`` fields will be
  returned in response body of POST, GET ``/os-server-groups``
  API and GET ``/os-server-groups/{server_group_id}`` API.
* The ``policies`` and ``metadata`` fields have been removed
  from the response body of POST, GET ``/os-server-groups`` API
  and GET ``/os-server-groups/{server_group_id}`` API.

Part of blueprint: complex-anti-affinity-policies

Change-Id: I6911e97bd7f8df92511e90518dba21c127e106a5
2018-07-13 10:43:42 +08:00
Taku Izumi a07b68ea92 virt/ironic: Implement rescue and unrescue
This patch adds implementation of rescue and unrescue for ironic virt
driver.

Implements: blueprint ironic-rescue-mode

Change-Id: I7c20a0c5f566c3255350fd494d1a2cde84a99440
Signed-off-by: Taku Izumi <izumi.taku@jp.fujitsu.com>
Co-Authored-By: Hironori Shiina <shiina.hironori@jp.fujitsu.com>
2018-07-13 01:56:13 +00:00
Julia Kreger 4acbf4fee3 ironic: provide facilities to gracefully navigate versions
For quite some time, the nova virt driver has invoked the ironic
python client library with a static microversion pin which causes
operators to have a failed nova-compute process upon out of order
upgrades, as well as causes ironic's grenade CI jobs to break every
cycle when the pin is changed to a version released in the current
cycle due to established community agreement as well as the grenade
testing framework limitations.

In order to gracefully navigate this, python-ironicclient has
accepted a list of possible versions to negotiate since 2.2.0.

This patch provies a mechanism to validate if we are able to
send a request that was added at a specific version, allowing for
minimal code changes and for logic to only be centered around newer
features being added.

Change-Id: I440689a246538fbc8200687e40480d837b87eb7b
Closes-Bug: #1739440
2018-07-12 18:53:36 -07:00
Zuul fff9335bed Merge "Handle rebuild of instances with image traits" 2018-07-13 00:43:12 +00:00
Zuul 3dc4e61f6e Merge "Update process doc to be more generic about point of contact" 2018-07-13 00:30:18 +00:00
Jay Pipes 11c29ae470 do not assume 1 consumer in AllocList.delete_all()
Ever since we introduced support for setting multiple consumers in a
single POST /allocations, the AllocationList.delete_all() method has
been housing a latent bad assumption and bug.

The AllocationList.delete_all() method used to assume that the
AllocationList's Allocation objects were only ever for a single
consumer, and took a shortcut in deleting the allocation by deleting all
allocations with the "first" Allocation's consumer UUID:

```python
    def delete_all(self):
        # Allocations can only have a single consumer, so take advantage of
        # that fact and do an efficient batch delete
        consumer_uuid = self.objects[0].consumer.uuid
        _delete_allocations_for_consumer(self._context, consumer_uuid)
        consumer_obj.delete_consumers_if_no_allocations(
            self._context, [consumer_uuid])
```

The problem with the above is that if you get all the allocations for a
single resource provider, using
AllocationList.get_all_by_resource_provider() and there are more than
one consumer allocating resources against that provider, then calling
AllocationList.delete_all() will only delete *some* of the resource
provider's allocations, not all of them.

Luckily, the handler code has never used AllocationList.delete_all()
after calling AllocationList.get_all_by_resource_provider(), and so
we've not hit this latent bug in production.

However, in the next patch in this series (the reshaper DB work), we
*do* call AllocationList.delete_all() for allocation lists for each
provider involved in the reshape operation, which is why this fix is
important to get done correctly.

Note that this patch renames AllocationList.create_all() to
AllocationList.replace_all() to make it absolutely clear that all of
the allocations for all consumers in the list are first *deleted* by the
codebase and then re-created. We also remove the check in
AllocationList.create_all() that the Allocation objects in the list must
not have an 'id' field set. The reason for that is because in order to
properly implement AllocationList.delete_all() to call DELETE FROM
allocations WHERE id IN (<...>) we need the list of allocation record
internal IDs. These id field values are now properly set on the
Allocation objects when AllocationList.get_all_by_resource_provider()
and AllocationList.get_all_by_consumer_id() are called. This allows that
returned object to have delete_all() called on it and the DELETE
statement to work properly.

Change-Id: I12393b033054683bcc3e6f20da14e6243b4d5577
Closes-bug: #1781430
2018-07-12 16:57:31 -04:00
Zuul 536e5fa57f Merge "Mention osc-placement for managing traits in docs" 2018-07-12 18:49:06 +00:00
Zuul 45c58c0b70 Merge "Revert "docs: Disable smartquotes"" 2018-07-12 18:43:22 +00:00
melanie witt d76c54a70e Update process doc to be more generic about point of contact
The docs currently name a past PTL by IRC nick -- update it to be
more generic so we don't have to update it every time the PTL
changes.

Change-Id: I31dbf0ee4f7650389e824c25c3aea7529d63eaa0
2018-07-12 17:23:03 +00:00
Matt Riedemann 0d9e140df3 Follow up for Ie49d605c66062d2548241d7e04f5a2a6b98c011e
This just addresses the comments from said change.

Change-Id: Iec9a54c70ac4c1e99d320b5d9db05dfbb84aa529
2018-07-12 12:12:09 -04:00
Matt Riedemann 7ac21382f9 Mention osc-placement for managing traits in docs
This just links to the osc-placement plugin docs
for managing required and forbidden traits in the
flavor extra specs docs.

Change-Id: I8549dc404a62a05d327a2c7a4813e7cc505d6b06
2018-07-12 11:25:31 -04:00
Zuul da16690f4d Merge "Add policy to InstanceGroup object" 2018-07-12 14:34:04 +00:00
Author Name 012ea7439b Handle rebuild of instances with image traits
Update the conductor to handle rebuilding of an instance. If a rebuild
is requested with an image with traits, we check the current allocations
of the instance against the image required traits and allow the action
only if the current allocations can satisfy the image requirements.

Change-Id: Ie49d605c66062d2548241d7e04f5a2a6b98c011e
Implements: blueprint glance-image-traits
2018-07-12 04:24:55 -07:00
Yikun Jiang 09f67d7789 Adapt _validate_instance_group_policy to new policy model
To solve the race condition problem, we do anti-affinity policy
sanity check in the instance build code of compute manager, but
this check is only valid in original policy.

Specifically, this change adds validation of the new
max_server_per_host policy rule which can be used to allow >1
anti-affinity group member on the same host.

blueprint: complex-anti-affinity-policies

Change-Id: I755b6fdddc9d754326cd9c81b6880581641f73e8
2018-07-12 10:32:14 +08:00
Yikun Jiang a77f57acf7 Change the ServerGroupAntiAffinityFilter to adapt to new policy
In this patch, the anti-affinity filter will get the
max_server_per_host limit from policy rules, and compare it
against the number of servers within the same group on a given
host. If the filter finds the number is not satisfied with the
limit, it will filter out this host.

The default ``max_server_per_host`` for the anti-affinity filter
is 1 for backward compatibility.

Change-Id: I4b67ec9dd4ce846a704d0f75ad64c41e693de0fb
blueprint: complex-anti-affinity-policies
2018-07-12 10:30:10 +08:00
Yikun Jiang c67ab38ea0 Add policy field to ServerGroup notification object
In this patch, the ServerGroupPayload is updated to include
the new ``policy`` field; the ``policies`` field is deprecated
for removal but still put into the notification payload for
backward compatibility.

Related to blueprint complex-anti-affinity-policies

Change-Id: Ie739ee8dec4685cd70e735ff83f7f30bc7e95a57
2018-07-12 10:00:05 +08:00
Yikun Jiang 8fa70e5dfc Add policy to InstanceGroup object
The change items in this patch as below:
1. Add policy and rules to InstanceGroup, deprecate
the policies field.
2. Also, make _from_db_object and _create_in_db method to
support new "policy" field.

The internal usage of InstanceGroup.policies is converted to
use the new model in the REST API and RequestSpec compat code.

Related to blueprint complex-anti-affinity-policies

Change-Id: Ib33719a4b9599d86848c618a6e142c71ece79ca5
2018-07-12 09:30:41 +08:00
Zuul 39b05ee9e3 Merge "Fix server_group_members quota check" 2018-07-11 21:33:06 +00:00
Zuul 6235735ba8 Merge "Add functional regressions tests for server_group_members OverQuota" 2018-07-11 21:32:58 +00:00
Zuul d620b0fdc3 Merge "Fix TypeError in prep_resize allocation cleanup" 2018-07-11 21:32:46 +00:00
Zuul d2954aed7c Merge "Add queued for delete to instance_mappings table." 2018-07-11 21:32:30 +00:00
Zuul 746a8c8d29 Merge "[placement] add error.code on a ConcurrentUpdateDetected" 2018-07-11 18:20:54 +00:00
Zuul 8136772f28 Merge "Test for unsanitized consumer UUID" 2018-07-11 16:27:36 +00:00
Zuul 90cdf80750 Merge "Remove irrelevant comment" 2018-07-11 14:12:32 +00:00