Commit Graph

58023 Commits

Author SHA1 Message Date
Ghanshyam Mann 454b8d30f4 Fix config option default value for sample config file
There are multiple places where nova set the new default
value for other (than nova) namespace config option for
example oslo lib config options. Examples:

- https://github.com/openstack/nova/blob/20572809f2d8fefd72a7a059c4e82462a0d66262/nova/config.py#L62
- https://github.com/openstack/nova/blob/20572809f2d8fefd72a7a059c4e82462a0d66262/nova/policy.py#L48

and these defaults are reflected in code also which is working
as expected. But when config sample file is generated via
oslo-config-generator tool (tox -egenconfig) then these defaults
which are set by Nova are not reflected as this tool take the
raw defaults. To solve this issue oslo config provide a option
to add hook to reflect the new default in config generator
- https://docs.openstack.org/oslo.config/latest/cli/generator.html#modifying-defaults-from-other-namespaces

We already doing it for middleware cors option
- https://github.com/openstack/nova/blob/20572809f2d8fefd72a7a059c4e82462a0d66262/setup.cfg#L41

Fixing existing default value and if we find more or any future
change for default value, we need to add those config generator hook.

Change-Id: Ib0e926aec069dfb4ffd1bbfe8506d62464511b51
Closes-Bug: #1904250
2020-11-25 00:05:08 +00:00
Zuul 20572809f2 Merge "functional: Wait for revert resize to complete" 2020-11-13 13:40:14 +00:00
Stephen Finucane 1d23b358d6 functional: Wait for revert resize to complete
When reverting a resize, we need to wait for the migration status to
change to 'reverted', but we also need to wait for the relevant
versioned notification to be emitted. The reason for this is noted in a
couple of places, including the '_revert_resize' helper in the
'nova.tests.functional.integrated_helpers.InstanceHelperMixin' module:

  [T]he migration status is changed to "reverted" in the dest host
  revert_resize method but the allocations are cleaned up in the source
  host finish_revert_resize method so we need to wait for the
  finish_revert_resize method to complete.

Two tests in the 'test_cross_cell_migrate' test module were not doing
this wait, resulting in intermittent failures in CI due to the races.
Resolve this now.

Change-Id: I3ec6cae19b362ac9cc311a979f680cf64db4f458
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Closes-Bug: #1904051
2020-11-12 19:41:27 +00:00
Zuul eb279e9a56 Merge "Update contributor guide for Wallaby" 2020-11-11 22:16:03 +00:00
Takashi Natsume bf47afd70a Update contributor guide for Wallaby
Change-Id: I8ce10c7dad73da2c2269282b9f2350a230e3aa84
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
2020-11-11 12:54:35 +00:00
Balazs Gibizer 3b44275868 Improve error handling during service level check
The service level check introduced in
Ie15ec8299ae52ae8f5334d591ed3944e9585cf71 should handle the case when a
compute service is wrongly configured with DB credentials. The previous
patch did not handle this and it caused a misleading error at compute
service startup. This patch makes sure that a user friendly warning is
logged in this case then the service level check is done ignoring the DB
configuration and only checking the local cell.

A subsequent patch will add a separate check that fails the compute
service startup in such invalid configuration.

Change-Id: I89cdf3852266ed93a2ac7cd6261fe269932026ac
Related-Bug: #1871482
2020-11-10 17:39:50 +01:00
Zuul dc93e3b510 Merge "rbd: Only log import failures when the RbdDriver is used" 2020-11-09 23:51:46 +00:00
Zuul 4f2540a7a6 Merge "virt: Remove 'get_per_instance_usage' API" 2020-11-09 18:50:31 +00:00
Zuul d6689e316c Merge "Remove six.byte2int/int2byte" 2020-11-09 16:34:22 +00:00
Zuul 6898cf2765 Merge "Remove six.iteritems/itervalues/iterkeys" 2020-11-09 16:34:13 +00:00
Zuul 80b807a4c5 Merge "Remove six.moves" 2020-11-09 16:05:06 +00:00
Lee Yarwood 4916ab7a4c rbd: Only log import failures when the RbdDriver is used
Rework Ie8bb5e5622bd37dfe8073cca12f77174e8e7d98c so we only log failures
to import the rbd or rados modules when the RbdDriver is used.

This should reduce noise in the logs at runtime as well as during unit
and functional tests runs where these modules are not present.

Change-Id: I150e70629f6ae579ccfe0bf585c8a27df14fb51d
Closes-Bug: #1903316
2020-11-09 10:22:12 +00:00
Lee Yarwood be752b8175 zuul: Reintroduce nova-dsvm-multinode-base
This was incorrectly removed by
Ib342e2d3c395830b4667a60de7e492d3b9de2f0a while still being used by the
nova-grenade-multinode job. This was missed as the check queue appears
to default to silently skipping jobs where it can't find the parent
instead of failing.

Change-Id: I3ece71ab75a28a0ba662c56fb140525e8ce4aa6c
2020-11-09 08:52:41 +00:00
Zuul 321d2906f8 Merge "Add upgrade check about old computes" 2020-11-08 12:27:47 +00:00
Zuul bf0f9b1bd7 Merge "trivial: Remove irrelevant comment" 2020-11-08 09:53:00 +00:00
Zuul 8584a82e25 Merge "virt: Remove various aggregate APIs" 2020-11-08 09:45:13 +00:00
Zuul d97cb3b9bd Merge "Prevent starting services with older than N-1 computes" 2020-11-08 04:30:15 +00:00
Zuul ccb0ab1d0a Merge "virt: Remove 'set_bootable' API" 2020-11-07 14:55:57 +00:00
Zuul 120d866df7 Merge "Allow excluding image type capabilities" 2020-11-07 08:27:51 +00:00
Takashi Natsume f6d74eabfc Remove six.byte2int/int2byte
Replace the following items with Python 3 style code.

- six.byte2int
- six.int2byte
- six.u
- six.b
- six.unichr
- six.get_method_self
- six.wraps

Subsequent patches will replace other six usages.

Change-Id: I931e717cd18b866c9577089b1237b663513c173e
Implements: blueprint six-removal
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
2020-11-07 03:25:40 +00:00
Takashi Natsume 28ad7c517d Remove six.iteritems/itervalues/iterkeys
Replace the following items with Python 3 style code.

- six.iteritems
- six.itervalues
- six.iterkeys

Subsequent patches will replace other six usages.

Change-Id: Id55de3a105bedcf61bcfc797aabe86d6e75709c8
Implements: blueprint six-removal
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
2020-11-07 03:25:26 +00:00
Takashi Natsume 1d0a0e8c20 Remove six.moves
Replace the following items with Python 3 style code.

- six.moves.configparser
- six.moves.StringIO
- six.moves.cStringIO
- six.moves.urllib
- six.moves.builtins
- six.moves.range
- six.moves.xmlrpc_client
- six.moves.http_client
- six.moves.http_cookies
- six.moves.queue
- six.moves.zip
- six.moves.reload_module
- six.StringIO
- six.BytesIO

Subsequent patches will replace other six usages.

Change-Id: Ib2c406327fef2fb4868d8050fc476a7d17706e23
Implements: blueprint six-removal
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
2020-11-07 03:25:02 +00:00
Lee Yarwood 4ac4a04d18 zuul: Replace nova-live-migration with zuulv3 jobs
This change removes the original nova-live-migration job and replaces it
directly with the new Focal based zuulv3 native job.

The nova-dsvm-multinode-base base job is no longer used and so also
removed as part of this change.

Note that this new nova-live-migration job does not yet contain any
ceph coverage like the original, this is still pending and will be
completed early in the W cycle.

This change is being merged ahead of this to resolve bug #1901739, a
known QEMU -drive issue caused by the previous jobs use of libvirt 5.4.0
as provided by Ubuntu Bionic. The fix here being the migration to Ubuntu
Focal based jobs and libvirt 6.0.0 that now defaults to using QEMU
-blockdev.

Closes-Bug: #1901739
Change-Id: Ib342e2d3c395830b4667a60de7e492d3b9de2f0a
2020-11-06 10:55:52 +00:00
Lee Yarwood c0fe95fcc5 zuul: Merge nova-evacuate into nova-multinode-live-migration
Both jobs deploy a multinode env before running their tests so just
append the new nova-evacuate tests to the end of the job.

Change-Id: If64cdf1002eec1504fa76eb4df39b6b2e4ff3728
2020-11-06 10:49:25 +00:00
Zuul 7134cbdc7a Merge "Update pci stat pools based on PCI device changes" 2020-11-06 05:23:46 +00:00
Zuul 89cbd9de10 Merge "Fix the compute RPC 5.12 issue" 2020-11-05 21:55:29 +00:00
Zuul a526a97785 Merge "Add a regression test for 5.12 compute API issue" 2020-11-05 17:00:11 +00:00
Sylvain Bauza 8d9f298f4c Fix the compute RPC 5.12 issue
In I147bf4d95e6d86ff1f967a8ce37260730f21d236 we added a new argument for
the rebuild_instance() RPC method. Unfortunately, we missed to that it
needs to be optional for older versions.

Adding a default none value for it so rolling upgrades would work.

Change-Id: I59c5e56b00114fea5ec19fa63ec73f032dc3bd5c
Closes-Bug: #1902925
2020-11-05 16:58:03 +01:00
Sylvain Bauza 8f79afd448 Add a regression test for 5.12 compute API issue
In I147bf4d95e6d86ff1f967a8ce37260730f21d236 we wrote a breaking RPC change
for the 5.12 version as the accel_uuids parameter is not optional.

Adding a regression test to check the issue.

Change-Id: I1f3914e16294c99a625b3984ca0098d835cd9b92
Related-Bug: #1902925
2020-11-05 10:45:04 +01:00
Zuul f3c2780e98 Merge "Use subqueryload() instead of joinedload() for (system_)metadata" 2020-11-05 08:22:25 +00:00
Balazs Gibizer 3b8257cd23 Add upgrade check about old computes
Report a warning during upgrade checks if there are computes older than
the previous major nova release in the system.

So if code is upgraded to Wallaby and the upgrade check was run before
the restart of the services with W code then the check warns for Ussuri
computes in the system.

Change-Id: I873b0c1e6e695ae88241bbf75ac9f80ecc6f5664
2020-11-04 15:43:09 +01:00
Balazs Gibizer aa7c6f8769 Prevent starting services with older than N-1 computes
Nova services only support computes that are not older than
the previous major release. This patch introduces a check in the
service startup that prevents staring the service if too old computes
are detected.

Change-Id: Ie15ec8299ae52ae8f5334d591ed3944e9585cf71
2020-11-04 14:05:52 +01:00
Zuul edd8fefe3f Merge "optimize the shelve code flow" 2020-11-03 09:22:00 +00:00
Zuul 527d6a5b7d Merge "Modify glance's copy_image permission for nova-ceph-multistore" 2020-11-02 22:15:22 +00:00
Zuul 926af1faf1 Merge "objects: Fix issue in exception type" 2020-10-31 11:20:21 +00:00
Zuul 0b7c00b566 Merge "Add regression test for bug #1899649" 2020-10-31 08:14:30 +00:00
Zuul 74bf2a714c Merge "Fix virsh domifstat to get vhostuser vif statistics" 2020-10-31 06:31:40 +00:00
Ghanshyam Mann e7c7a2851e Modify glance's copy_image permission for nova-ceph-multistore
nova-ceph-multistore setup needs non-admin users to copy the image.
To allow that glance's policy was overriden to allow public
images to copy. This restriction again can cause issue if there
is any new copy image tempest test try to copy private image with
admin users.
- https://review.opendev.org/#/c/742546/

Let's allow everyone to copy every image to make it work
for all type of test credentials.

Change-Id: Ia65afdfb8989909441dba55faeed2d78cc7f1ee7
2020-10-29 12:20:17 -05:00
Zuul 7dcc4cfea7 Merge "Add placeholder migrations for Victoria backports" 2020-10-28 16:28:56 +00:00
Lee Yarwood 0d2ca53bb8 libvirt: Only ask tpool.Proxy to autowrap vir* classes
I668643c836d46a25df46d4c99a973af5e50a39db attempted to fix service wide
pauses by providing a more complete list of classes to tpool.Proxy.

While this excluded libvirtError it can include internal libvirt-python
classes pointed to by private globals that have been introduced with the
use of type checking within the module.

Any attempt to wrap these internal classes will result in the failure
seen in bug #1901383. As a result this change simply ignores any class
found during inspection that doesn't start with the `vir` string, used
by libvirt to denote public methods and classes.

Closes-Bug: #1901383
Co-Authored-By: Daniel Berrange <berrange@redhat.com>
Change-Id: I568b0c4fd6069b9118ff116532f14abb46cc42ab
2020-10-27 09:38:18 +00:00
Zuul a83001a903 Merge "CinderFixture: Return a driver_type of fake within connection_info" 2020-10-24 01:46:23 +00:00
Zuul e2a1bc7641 Merge "[doc]: Fix glance image_metadata link" 2020-10-24 01:23:53 +00:00
Stephen Finucane b32a685595 docs: Make JSON valid
Pygments 2.7.x is stricter in how it validates JSON escapes, aligning it
closer with the spec [1]. Turns out we have some invalid JSON in our
docs, meaning builds are now failing with the following error:

  doc/source/user/metadata.rst:262: WARNING: Could not lex literal_block
  as "json". Highlighting skipped.

Resolve this.

[1] https://github.com/pygments/pygments/commit/9514e794e0c2a5c7c048df97fcfef4a099e05ac3

Change-Id: Ic50e29e9c7817744ad0b4f9de309aa3e96a09505
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2020-10-23 17:15:43 +01:00
Zuul 6a5e158756 Merge "Remove self.server dependency form _evacuate_server" 2020-10-22 13:13:04 +00:00
Zuul 2e684ca9f6 Merge "fixtures: Handle binding of first port" 2020-10-22 13:12:57 +00:00
Zuul 5f7e622448 Merge "virt: Remove 'post_interrupted_snapshot_cleanup' API" 2020-10-22 10:41:20 +00:00
Zuul f62b87cb07 Merge "virt: Remove 'get_console_pool_info' API" 2020-10-21 23:31:50 +00:00
Stephen Finucane 5b9d587e74 trivial: Remove irrelevant comment
This should have been removed in change
I18a01032a89bff84d71e879c5207157393849b7e. Remove it now.

Change-Id: I29d1cd2f043bd2244c6fb0410c94f6612de14dbc
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2020-10-21 18:30:35 +01:00
Balazs Gibizer f5a68826c7 [doc]: Fix glance image_metadata link
Change-Id: Ib1e3798992696cf5f271b20bc731b8b766d173ac
Closes-Bug: #1900731
2020-10-21 18:19:13 +02:00
melanie witt e728fe668a Use subqueryload() instead of joinedload() for (system_)metadata
Currently, when we "get" a single instance from the database and we
load metadata and system_metadata, we do so using a joinedload() which
does JOINs with the respective tables. Because of the one-to-many
relationship between an instance and (system_)metadata records, doing
the database query this way can result in a large number of additional
rows being returned unnecessarily and cause a large data transfer.

This is similar to the problem addressed by change
I0610fb16ccce2ee95c318589c8abcc30613a3fe9 which added separate queries
for (system_)metadata when we "get" multiple instances. We don't,
however, reuse the same code for this change because
_instances_fill_metadata converts the instance database object to a
dict, and some callers of _instance_get_by_uuid need to be able to
access an instance database object attached to the session (example:
instance_update_and_get_original).

By using subqueryload() [1], we can perform the additional queries for
(system_)metadata to solve the problem with a similar approach.

Closes-Bug: #1799298

[1] https://docs.sqlalchemy.org/en/13/orm/loading_relationships.html#subquery-eager-loading

Change-Id: I5c071f70f669966e9807b38e99077c1cae5b4606
2020-10-20 22:09:50 +00:00