Commit Graph

1320 Commits

Author SHA1 Message Date
Stephen Finucane db294b1e33 Remove support for /os-floating-ip-dns REST API
Drop support for the os-floating-ip-dns API which has been deprecated
since Newton:

  Idca478c566f9a7b5b30a3172453ce7c66d9fd8f0

This API now returns a 410 response for all routes.

Unit tests are removed and the functional API sample tests are just
asserting the 410 response now.

The API sample docs are left intact since the API reference still builds
from those and can be considered more or less branchless, so people
looking at the API reference can apply it to older deployments of nova
before os-floating-ip-dns was removed.

The release note added for previous nova-network API removals is
amended to note this additional change.

Part of blueprint remove-nova-network

Change-Id: I0c4b586292814b8483226aee315f41cbefc86a1e
2018-07-18 22:23:45 -04:00
Stephen Finucane 823c4e840d Remove support for /os-floating-ips-bulk REST API
Drop support for the os-floating-ips-bulk API which has been deprecated
since Newton:

  Idca478c566f9a7b5b30a3172453ce7c66d9fd8f0

This API now returns a 410 response for all routes.

Unit tests are removed and the functional API sample tests are just
asserting the 410 response now.

The API sample docs are left intact since the API reference still builds
from those and can be considered more or less branchless, so people
looking at the API reference can apply it to older deployments of nova
before os-floating-ips-bulk was removed.

The release note added for previous nova-network API removals is
amended to note this additional change.

Part of blueprint remove-nova-network

Change-Id: I89d081108b398d8efba9636279088c61349b21e6
Depends-On: https://review.openstack.org/582945
2018-07-18 22:23:45 -04:00
Zuul 5e045984bb Merge "API: add support to abort queued live migration in microversion 2.65" 2018-07-18 17:45:24 +00:00
Zuul a86604dc60 Merge "Address nits in server group policy series" 2018-07-17 06:23:25 +00:00
Kevin_Zheng 4cae503767 API: add support to abort queued live migration in microversion 2.65
This patch bumped API microversion to 2.65 to add support for
abort live migrations in ``queued`` and ``preparing`` status.

Part of blueprint abort-live-migration-in-queued-status

Change-Id: I4636a8d270ce01c1831bc951c4497ad472bc9aa8
2018-07-16 13:57:11 -04:00
Zuul ae40af621f Merge "Microversion 2.64 - Use new format policy in server group" 2018-07-16 15:41:57 +00:00
Yikun Jiang d1ccea4dd7 Address nits in server group policy series
Change-Id: I126b51892056e3241640bbf05acf16d7f72b4329
2018-07-16 17:28:46 +08:00
Zuul 629dc4d008 Merge "virt/ironic: Implement rescue and unrescue" 2018-07-15 07:49:36 +00:00
Zuul e13db24c48 Merge "Add policy field to ServerGroup notification object" 2018-07-13 02:45:55 +00:00
Yikun Jiang 5cdb1ce26b Microversion 2.64 - Use new format policy in server group
Enable users to define the policy rules on server group policy
to meet more advanced policy requirement. This microversion
brings the following changes in server group APIs:

* Add  ``policy`` and ``rules`` fields in the request of POST
  ``/os-server-groups``.
* The ``policy`` and ``rules`` fields will be
  returned in response body of POST, GET ``/os-server-groups``
  API and GET ``/os-server-groups/{server_group_id}`` API.
* The ``policies`` and ``metadata`` fields have been removed
  from the response body of POST, GET ``/os-server-groups`` API
  and GET ``/os-server-groups/{server_group_id}`` API.

Part of blueprint: complex-anti-affinity-policies

Change-Id: I6911e97bd7f8df92511e90518dba21c127e106a5
2018-07-13 10:43:42 +08:00
Taku Izumi a07b68ea92 virt/ironic: Implement rescue and unrescue
This patch adds implementation of rescue and unrescue for ironic virt
driver.

Implements: blueprint ironic-rescue-mode

Change-Id: I7c20a0c5f566c3255350fd494d1a2cde84a99440
Signed-off-by: Taku Izumi <izumi.taku@jp.fujitsu.com>
Co-Authored-By: Hironori Shiina <shiina.hironori@jp.fujitsu.com>
2018-07-13 01:56:13 +00:00
Yikun Jiang c67ab38ea0 Add policy field to ServerGroup notification object
In this patch, the ServerGroupPayload is updated to include
the new ``policy`` field; the ``policies`` field is deprecated
for removal but still put into the notification payload for
backward compatibility.

Related to blueprint complex-anti-affinity-policies

Change-Id: Ie739ee8dec4685cd70e735ff83f7f30bc7e95a57
2018-07-12 10:00:05 +08:00
Zuul f92b1326de Merge "Allow templated cell_mapping URLs" 2018-07-10 06:10:07 +00:00
shuangyang.qian 2bca6431e6 Add action initiator attribute to the instance payload
The instance action notifications contain the user id and the
project id of the owner of the instance. However an instance
action might be initiated by another user. It could be another
user from the same project or can be an admin from the admin project.
To be able to distinguish between the user who initiated the instance
action from the user owning the instance we need to add two new
fields to the instance action notifications, action_initiator_user
and action_initiator_project

Change-Id: I649d8a27baa8840bc1bb567fef027c749c663432
Closes-bug: #1744658
Blueprint: add-action-initiator-to-instance-action-notifications
2018-07-03 18:45:44 +00:00
Zuul 162e55d07e Merge "Add microversion for nested allocation candidate" 2018-06-29 15:03:02 +00:00
Tetsuro Nakamura 5b4aa78459 Add microversion for nested allocation candidate
This patch adds a microversion with a release note for allocation
candidates with nested resource provider trees.

From now on we support allocation candidates with nested resource
providers with the following features.

1) ``GET /allocation_candidates`` is aware of nested providers.
   Namely, when provider trees are present, ``allocation_requests``
   in the response of ``GET /allocation_candidates`` can include
   allocations on combinations of multiple resource providers
   in the same tree.
2) ``root_provider_uuid`` and ``parent_provider_uuid`` fields are
    added to ``provider_summaries`` in the response of
   ``GET /allocation_candidates``.

Change-Id: I6cecb25c6c16cecc23d4008474d150b1f15f7d8a
Blueprint: nested-resource-providers-allocation-candidates
2018-06-29 17:38:10 +09:00
Zuul b2760300b8 Merge "conf: libvirt: Make /dev/urandom the default for 'rng_dev_path'" 2018-06-29 07:18:49 +00:00
Dan Smith 50658eee4f Allow templated cell_mapping URLs
The way we store DB and MQ URLs in the API database causes issues for
some deployments (and deployment tools) which want to use per-host
credentials or remote hostnames. Since all the URLs loaded from the
database are the same on all systems, this becomes very difficult and
some have even resorted to using client-based aliasing underneath Nova
and just providing URLs that reference those aliases.

This makes our CellMapping object load the URLs out of the database,
and apply variable substitution from the CONF-resident base URLs
for any fields provided. Such functionality will let operators
define per-host credentials in [database]/connection, for example,
and have those applied to the database_connection URLs loaded from
CellMapping records.

Change-Id: Iab296c27bcd56162e2efca5fb232cae0aea1160e
2018-06-27 12:54:44 -07:00
Zuul 351faa959f Merge "Remove support for /os-fixed-ips REST API" 2018-06-26 21:01:35 +00:00
Zuul 46bfd6f1c2 Merge "xenapi: drop deprecated vif_driver config option" 2018-06-26 16:14:54 +00:00
Kashyap Chamarthy 814bfd9372 conf: libvirt: Make /dev/urandom the default for 'rng_dev_path'
Since libvirt 1.3.4, any RNG (Random Number Generator) device path (that
returns random numbers when read!) is accepted.  However, the
recommended source of entropy is `/dev/urandom` (it is non-blocking; and
doesn't have the same limitations of `dev/random`, which is a legacy
interface).

Therefore, make `/dev/urandom` the default RNG for 'rng_dev_path' config
attribute; adjust the relevant tests.  Also update the documention to
reflect this change.

Change-Id: Ia39402a045ffb1943463b5741655d84071613e8c
Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com>
Reported-by: Daniel P. Berrangé <berrange@redhat.com>
2018-06-26 17:55:11 +02:00
jichen 5097309f89 Remove support for /os-fixed-ips REST API
This drops support for the os-fixed-ips compute REST API which has been
deprecated since
Newton: I1a8a44530be29292561e90d6f7bd7ed512a88ee3

Now it returns 410 response. Unit tests are removed and the functional API
sample test is just asserting the 410 response now. The API sample docs are
left intact since the API reference still builds from those and can be
considered more or less branchless, so people looking at the API reference
can apply it to older deployments of nova before os-fixed-ips was removed.

Part of blueprint remove-nova-network

Change-Id: I61f758ff9285448d431b45f67c70286082b4ee90
2018-06-26 09:20:28 -04:00
Zuul c86691a86d Merge "Make nova list and migration-list ignore down cells" 2018-06-26 05:07:38 +00:00
Zuul 4e7672d60f Merge "Add instance.unlock notification" 2018-06-25 23:01:14 +00:00
Surya Seetharaman ee461b5bf6 Make nova list and migration-list ignore down cells
This patch makes InstanceLister and MigrationLister ignore down
cells and list the instances/records from the up cell as opposed to
giving 500 to the users as is the current situation.

Change-Id: I308b494ab07f6936bef94f4c9da45e9473e3534d
Partial-Bug: #1726301
2018-06-25 17:28:57 -04:00
Yikun Jiang 0b9b37fe9a Add instance.unlock notification
The instance.unlock versioned notification is introduced in this
patch.

The unlock operation just changes the instance.locked to False in
API, we send the notification after db operation.

Change-Id: Ic750c33b4f88ba9c62ea8cba86915c6010f2cd6f
blueprint: trigger-notifications-when-lock-unlock-instances
2018-06-25 19:25:39 +00:00
Zuul 867d8e8d6f Merge "libvirt: Log breadcrumb for known encryption bug" 2018-06-22 21:41:15 +00:00
Lee Yarwood 78891c2305 libvirt: Log breadcrumb for known encryption bug
The initial implementation of native LUKS support within Libvirt
introduced a small issue when using a passphrase that is a multiple of
16 bytes in size. This is documented in the following bug and associated
patch posted to the Libvirt development list:

Unable to use LUKS passphrase that is exactly 16 bytes long
https://bugzilla.redhat.com/show_bug.cgi?id=1447297

[libvirt] [PATCH] Fix padding of encrypted data
https://www.redhat.com/archives/libvir-list/2017-May/msg00030.html

This change introduces a known issue release note and logs an additional
breadcrumb when we appear to hit this with pointers to the above.

Closes-Bug: #1778044
Change-Id: Id346bce6e47431988cce7001abcf29a9faf2936a
2018-06-22 09:24:53 +01:00
Zuul 6635924ebf Merge "Clarify placement DB schema migration" 2018-06-22 02:02:51 +00:00
Zuul 88213a0352 Merge "Implement file backed memory for instances in libvirt" 2018-06-20 20:57:59 +00:00
Zuul 2e0a7280db Merge "Add a microversion for consumer generation support" 2018-06-20 16:43:01 +00:00
Zack Cornelius cbc28f0d15 Implement file backed memory for instances in libvirt
File backed memory is enabled per Nova compute host. When enabled, host
will report 'file_backed_memory_capacity' for available memory.

When enabled, instances will create memory backing files in the
directory specified in libvirt's qemu.conf file 'memory_backing_dir'
config option.

This feature is not compatible with memory overcommit, and requires
'ram_allocation_ratio' to be set to 1.0

Change-Id: I676291ec0faa1dea0bd5050ef8e3426d171de4c6
Implements: blueprint libvirt-file-backed-memory
2018-06-20 11:25:06 -05:00
Jay Pipes 092820939d Add a microversion for consumer generation support
This patch adds new placement API microversion for handling consumer
generations.

Change-Id: I978fdea51f2d6c2572498ef80640c92ab38afe65
Co-Authored-By: Ed Leafe <ed@leafe.com>
Blueprint: add-consumer-generation
2018-06-20 12:11:09 +01:00
Matt Riedemann 5aadff75c3 Wait for network-vif-plugged before starting live migration
This adds a new config option which is read on the destination host
during pre_live_migration and the value is returned back to the
source host, which can be used to determine, from the source host,
if it should wait for a "network-vif-plugged" event due to VIFs
being plugged on the destination host. This helps us to
avoid the guest transfer at all if vif plugging failed on the dest
host, which we just wouldn't find out until post live migration
and then we have to rollback.

The option is disabled by default for backward compatibility and
also because certain networking backends, like OpenDaylight, are
known to not send network-vif-plugged events unless the port host
binding information changes, which for live migration doesn't happen
until after the guest is transferred to the destination host.

We could arguably avoid the changes to the live migrate data
versioned object and just assume the same networking backend is
used within each cell, but this does allow the deployer to have
the flexibility of live migrating between different network
backends (eventually anyway). The ability to live migrate between
different VIF types is being worked on as part of blueprint
neutron-new-port-binding-api.

Related to blueprint neutron-new-port-binding-api

Change-Id: I0f3ab6604d8b79bdb75cf67571e359cfecc039d8
2018-06-19 18:21:29 -04:00
Zuul ecaadf6d6d Merge "Add full traceback to ExceptionPayload in versioned notifications" 2018-06-19 20:18:32 +00:00
Zuul afcf078715 Merge "Add policy rule to block image-backed servers with 0 root disk flavor" 2018-06-19 12:01:13 +00:00
Kevin_Zheng 2a0f2a0d27 Add full traceback to ExceptionPayload in versioned notifications
This patch adds full traceback to ExceptionPayload in versioned
notifications.

The instance fault field and instance-action REST API has already
provide the traceback to the admin users (controlable through policy)
and the notifications are also admin only things as they are emitted
to the message bus by default. So it is assumed that security is not
a bigger concern for the notification than for the REST API.

On the ML [1] post there was no objection to add new string field to the
ExceptionPayload that will hold the serialized traceback object.

[1] http://lists.openstack.org/pipermail/openstack-dev/2018-March/128105.html

Implements: blueprint add-full-traceback-to-error-notifications

Change-Id: Id587967ea4f9980c292492e2f659bf55fb037b28
2018-06-19 16:46:46 +08:00
Matt Riedemann ae4285c837 Clarify placement DB schema migration
This just clarifies in the release note for the optional
placement database that the database itself is not created
when running "nova-manage api_db sync", but rather the
database schema is created. This is important since a
non-trivial number of people over the years have thought
that the db sync commands actually create a database, which
they do not.

Change-Id: Ie6c3a5dc61a288935829276cc72f7f7563e20420
2018-06-18 16:56:24 -04:00
Matt Riedemann 763fd62464 Add policy rule to block image-backed servers with 0 root disk flavor
This adds a new policy rule which defaults to behave in a
backward compatible way, but will allow operators to enforce
that servers created with a zero disk flavor must also be
volume-backed servers.

Allowing users to upload their own images and create image-backed
servers on local disk with zero root disk size flavors can be
potentially hazardous if the size of the image is unexpectedly
large, since it can consume the local disk (or shared storage pool).

It should be noted that disabling the new policy rule will
result in a non-backward compatible API behavior change and no
microversion is being introduced for this because enforcement via
a new microversion would not close the security gap on any previous
microversions.

Related compute API reference and user documentation is updated
to mention the policy rule along with a release note since
this is tied to a security bug, which will be backported to stable
branches.

Change-Id: Id67e1285a0522474844de130c9263e11868f67fb
Closes-Bug: #1739646
2018-06-15 17:10:11 -04:00
Zuul 93f2ca64e0 Merge "Fix nits from trusted certs notification change" 2018-06-15 16:58:45 +00:00
Matt Riedemann dc9fb5842c Fix nits from trusted certs notification change
This is a follow up to address a few nits in change
Ib5b50a3889ab15d5aac992f92e9be372a915eeff.

Change-Id: I08ce4a9f5bb33761d8c5ff82ef9faf874ac97fcc
2018-06-15 08:34:12 -04:00
Chris Dent 0044beb358 Optional separate database for placement API
If 'connection' is set in the 'placement_database' conf group use
that as the connection URL for the placement database. Otherwise if
it is None, the default, then use the entire api_database conf group
to configure a database connection.

When placement_database.connection is not None a replica of the
structure of the API database is used, using the same migrations
used for the API database.

A placement_context_manager is added and used by the OVO objects in
nova.api.openstack.placement.objects.*. If there is no separate
placement database, this is still used, but points to the API
database.

nova.test and nova.test.fixtures are adjusted to add awareness of
the placement database.

This functionality is being provided to allow deployers to choose
between establishing a new database now or requiring a migration
later. The default is migration later. A reno is added to explain
the existence of the configuration setting.

This change returns the behavior removed by the revert in commit
39fb302fd9 but done in a more
appropriate way.

Note that with the advent of the nova-status command, which checks
to see if placement is "ready" the tests here had to be adjusted.
If we do allow a separate database the code will now check the
separate database (if configured), but nothing is done with regard
to migrating from the api to placement database or checking that.

blueprint placement-extract

Change-Id: I7e1e89cd66397883453935dcf7172d977bf82e84
Implements: blueprint optional-placement-database
Co-Authored-By: Roman Podoliaka <rpodolyaka@mirantis.com>
2018-06-15 13:01:50 +01:00
Zuul 2d6a838a28 Merge "Add notification support for trusted_certs" 2018-06-14 22:03:44 +00:00
Zuul 5b56434517 Merge "Use oslo.messaging per-call monitoring" 2018-06-14 21:55:37 +00:00
Zuul bc480656b8 Merge "Add enhanced KVM storage QoS quotas" 2018-06-14 15:47:45 +00:00
Zuul 2bcbea5439 Merge "XenAPI: define a new image handler to use vdi streaming" 2018-06-14 15:08:25 +00:00
Brianna Poulos e8ed9aacf8 Add notification support for trusted_certs
Add the 'trusted_image_certificates' field to InstanceCreatePayload
and InstanceActionRebuildPayload notifications.

Change-Id: Ib5b50a3889ab15d5aac992f92e9be372a915eeff
2018-06-14 16:16:00 +02:00
Brianna Poulos 8c7ca368b1 Add trusted_image_certificates to REST API
This change adds support for the trusted_image_certificates parameter,
which is used to define a list of trusted certificate IDs that can be
used during image signature verification and certificate validation. The
parameter may contain a list of strings, each string representing the ID
of a trusted certificate. The list is restricted to a maximum of 50 IDs.
The list of certificate IDs will be stored in the trusted_certs field of
the instance InstanceExtra and will be used to verify the validity of
the signing certificate of a signed instance image.

The trusted_image_certificates request parameter can be passed to
the server create and rebuild APIs (if allowed by policy):

* POST /servers
* POST /servers/{server_id}/action (rebuild)

The following policy rules were added to restrict the usage of the
``trusted_image_certificates`` request parameter in the server create
and rebuild APIs:

* os_compute_api:servers:create:trusted_certs
* os_compute_api:servers:rebuild:trusted_certs

The trusted_image_certificates parameter will be in the response
body of the following APIs (not restricted by policy):

* GET /servers/detail
* GET /servers/{server_id}
* PUT /servers/{server_id}
* POST /servers/{server_id}/action (rebuild)

APIImpact

Implements blueprint: nova-validate-certificates
Change-Id: Iedd3fea0e86648fae364f075915555dcb2c4f199
2018-06-13 15:52:59 -04:00
jianghua wang f7593ded8f XenAPI: define a new image handler to use vdi streaming
With the new image handler, it creates an image proxy which
will use the vdi streaming function from os-xenapi to
remotely export VHD from XenServer(image upload) or import
VHD to Xenerver(image download).
The existing GlanceStore uses custom functionality to directly
manipulate files on-disk, so it has the restriction that SR's
type must be file system based: e.g. ext or nfs. The new
image handler invokes APIs formally supported by XenServer
to export/import VDI remotely, it can support other SR
types also e.g. lvm, iscsi, etc.

Note:
vdi streaming would be supported by XenServer 6.5 or above.
The function of image handler depends on os-xenapi 0.3.3 or
above, so bump os-xenapi's version to 0.3.3 and also declare
depends on the patch which bump version in openstack/requirements.

Blueprint: xenapi-image-handler-option-improvement
Change-Id: I0ad8e34808401ace9b85e1b937a542f4c4e61690
Depends-On: Ib8bc0f837c55839dc85df1d1f0c76b320b9d97b8
2018-06-13 01:59:07 +00:00
Simon Dodsley 37aea88845 Add enhanced KVM storage QoS quotas
This will add additonal KVM-based qoutas for Cinder front-end QoS

The new quotas are:

 - read_bytes_sec_max
 - write_bytes_sec_max
 - total_bytes_sec_max
 - read_iops_sec_max
 - write_iops_sec_max
 - total_iops_sec_max
 - size_iops_sec

Depends-On: https://review.openstack.org/574804
Change-Id: I5a8f34c7ed3dea940700dffd54bc7d44f7518ab4
Implements: blueprint enhanced-kvm-storage-qos
2018-06-12 23:00:12 +00:00