Commit Graph

59349 Commits

Author SHA1 Message Date
melanie witt 887c445a7a Add wrapper for oslo.concurrency lockutils.ReaderWriterLock()
This is a follow up change to I168fffac8002f274a905cfd53ac4f6c9abe18803
which added a hackaround to enable our tests to pass with
fasteners>=0.15 which was upgraded recently as part of a
openstack/requirements update.

The ReaderWriterLock from fasteners (and thus lockutils) cannot work
correctly with eventlet patched code, so this adds a wrapper containing
the aforementioned hackaround along with a hacking check to do our best
to ensure that future use of ReaderWriterLock will be through the
wrapper.

Change-Id: Ia7bcb40a21a804c7bc6b74f501d95ce2a88b09b5
2022-01-12 04:15:26 +00:00
Zuul f2b364df64 Merge "Fill the exception msg" 2022-01-11 17:34:06 +00:00
Zuul f37a368e4e Merge "Make the CellDatabases fixture work with fasteners >= 0.15" 2022-01-11 17:19:44 +00:00
melanie witt 50bf252250 Make the CellDatabases fixture work with fasteners >= 0.15
As of fasteners >= 0.15, the workaround code to use eventlet.getcurrent
if eventlet patching is detected has been removed and
threading.current_thread is being used instead [1]. Although we are
running in a greenlet in our test environment, we are not running in a
greenlet of type GreenThread. A GreenThread is created by calling
eventlet.spawn and spawn is not used to run our tests. At the time of
this writing, the eventlet patched threading.current_thread method
falls back to the original unpatched current_thread method if it is not
called from a GreenThead [2] and that breaks our tests involving this
fixture. We can work around this by patching threading.current_thread
with eventlet.getcurrent during creation of the lock object.

[1] https://github.com/harlowja/fasteners/commit/467ed75ee1e9465ebff8b5edf452770befb93913
[2] https://github.com/eventlet/eventlet/blob/v0.32.0/eventlet/green/threading.py#L128

Change-Id: I168fffac8002f274a905cfd53ac4f6c9abe18803
2022-01-06 17:33:07 +00:00
Zuul b49b7663e1 Merge "Re-add python 3.6 functional testing" 2021-12-20 19:32:10 +00:00
Zuul 14bfbaa50e Merge "[rt] Apply migration context for incoming migrations" 2021-12-15 13:32:58 +00:00
Ghanshyam Mann cb1a52a828 Re-add python 3.6 functional testing
We have updated the yoga testing runtime to keep the
py36 testing.

- https://review.opendev.org/c/openstack/governance/+/820195

Unit tests job template is also updated to keep python
3.6 as a voting job.

- https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/820286

This commit re-add the python 3.6 functional tests also which we
removed in https://review.opendev.org/c/openstack/nova/+/819194
also, re-adding py3.6/3.7 versions in setup.cfg classifier.

Change-Id: I7c83f1d64f31741dc51cc736265cf50fd0dad7af
2021-12-13 19:26:19 -06:00
songwenping 2d4c77aa68 Fill the exception msg
Change-Id: Ia3fde144fafbe4f036a14258e713ac830239592f
2021-12-10 10:26:47 +00:00
Zuul 755aa11e0c Merge "Reattach mdevs to guest on resume" 2021-12-08 16:08:24 +00:00
Balazs Gibizer 32c1044d86 [rt] Apply migration context for incoming migrations
There is a race condition between an incoming resize and an
update_available_resource periodic in the resource tracker. The race
window starts when the resize_instance RPC finishes  and ends when the
finish_resize compute RPC finally applies the migration context on the
instance.

In the race window, if the update_available_resource periodic is run on
the destination node, then it will see the instance as being tracked on
this host as the instance.node is already pointing to the dest. But the
instance.numa_topology still points to the source host topology as the
migration context is not applied yet. This leads to CPU pinning error if
the source topology does not fit to the dest topology. Also it stops the
periodic task and leaves the tracker in an inconsistent state. The
inconsistent state only cleanup up after the periodic is run outside of
the race window.

This patch applies the migration context temporarily to the specific
instances during the periodic to keep resource accounting correct.

Change-Id: Icaad155e22c9e2d86e464a0deb741c73f0dfb28a
Closes-Bug: #1953359
Closes-Bug: #1952915
2021-12-07 13:32:26 +01:00
Balazs Gibizer 9f296d775d Extend the reproducer for 1953359 and 1952915
This patch extends the original reproduction
I4be429c56aaa15ee12f448978c38214e741eae63 to cover
bug 1952915 as well as they have a common root cause.

Change-Id: I57982131768d87e067d1413012b96f1baa68052b
Related-Bug: #1953359
Related-Bug: #1952915
2021-12-07 13:32:07 +01:00
Balazs Gibizer c59224d715 Reproduce bug 1953359
This patch adds a functional test that reproduces a race between
incoming migration and the update_available_resource periodic

Change-Id: I4be429c56aaa15ee12f448978c38214e741eae63
Related-Bug: #1953359
2021-12-06 16:36:41 +01:00
Gustavo Santos 16f7c601b6 Reattach mdevs to guest on resume
When suspending a VM in OpenStack, Nova detaches all the mediated
devices from the guest machine, but does not reattach them on the resume
operation. This patch makes Nova reattach the mdevs that were detached
when the guest was suspended.

This behavior is due to libvirt not supporting the hot-unplug of
mediated devices at the time the feature was being developed. The
limitation has been lifted since then, and now we have to amend the
resume function so it will reattach the mediated devices that were
detached on suspension.

Closes-bug: #1948705

Signed-off-by: Gustavo Santos <gustavofaganello.santos@windriver.com>
Change-Id: I083929f36d9e78bf7713a87cae6d581e0d946867
2021-12-06 12:00:34 -03:00
Zuul 7670303aab Merge "Remove broken tempest-full-py3-opensuse15 job" 2021-12-01 19:48:17 +00:00
Zuul c5c2cc0341 Merge "docs: Add notes about IoOpsWeigher, BuildFailureWeigher" 2021-12-01 15:32:28 +00:00
Zuul d630615a02 Merge "Updating tests with Yoga testing runtime" 2021-11-26 11:13:36 +00:00
Zuul 4a359d576d Merge "db: Don't use legacy 'Row()' methods" 2021-11-25 19:50:01 +00:00
Zuul f1258bca05 Merge "docs: Update libvirt distro support matrix for Xena" 2021-11-25 17:40:49 +00:00
Ghanshyam Mann 9dd0070ce6 Updating tests with Yoga testing runtime
Yoga testing runtime is updated now
- https://governance.openstack.org/tc/reference/runtimes/yoga.html

which needs to test py38 and py39. Unit tests update are
handled by the job template change in openstack-zuul-job and
this commit makes changes to fucntional job to run py39 as voting
and updating the metdata in setup file.

Change-Id: I314fd61f20f2c3551f6231d191b7dcaaefabd2b5
2021-11-25 17:38:32 +00:00
Zuul 3047cde210 Merge "Retry image download if it's corrupted" 2021-11-25 13:05:55 +00:00
Zuul 8f250f5044 Merge "db: Don't pass strings to 'Connection.execute'" 2021-11-23 11:35:30 +00:00
Zuul e39690d792 Merge "db: Replace 'insert.inline' parameter with 'Insert.inline()' method" 2021-11-23 11:35:23 +00:00
Zuul 5168d74281 Merge "db: Replace use of legacy select() calling style" 2021-11-23 11:26:51 +00:00
Zuul 55210a5faa Merge "db: Replace use of 'autoload' parameter" 2021-11-23 11:26:42 +00:00
Zuul 841e62f4d4 Merge "db: Replace use of Executable.scalar(), Executable.execute()" 2021-11-23 11:09:58 +00:00
sdmitriev1 ce493273b9 Retry image download if it's corrupted
Adding IOError in list of catching exceptions in order to
fix behavior when nova-compute wouldn't retry image download
when got "Corrupt image download" error from glanceclient
and had num_retries config option set.

Closes-Bug: #1950657
Change-Id: Iae4fd0579f71d3ba6793dbdb037275352d7e57b0
2021-11-22 16:15:27 -05:00
Zuul 07dfd853b2 Merge "nova-manage: Always get BDMs using get_by_volume_and_instance" 2021-11-22 18:39:03 +00:00
Zuul a41783ed49 Merge "Log instance event wait times" 2021-11-22 17:35:48 +00:00
Zuul 69b0d31d20 Merge "Add a WA flag waiting for vif-plugged event during reboot" 2021-11-18 20:06:43 +00:00
Zuul 7aa3a0f558 Merge "libvirt: Switch the default video model from 'cirrus' to 'virtio'" 2021-11-18 15:54:13 +00:00
Lee Yarwood 1241d3440b docs: Update libvirt distro support matrix for Xena
This is mostly distro updates as our MIN versions didn't change during
the cycle.

Change-Id: If30d75c8fa96aa6ed7f0483d791d1dea86f58c08
2021-11-18 14:12:49 +00:00
Stephen Finucane be4e01c101 db: Don't use legacy 'Row()' methods
Resolve the following RemovedIn20Warning warnings:

  The Row.keys() method is considered legacy as of the 1.x series of
  SQLAlchemy and will be removed in 2.0. Use the namedtuple standard
  accessor Row._fields, or for full mapping behavior use
  row._mapping.keys()

An additional warning that appears to have been resolved in the interim
is also removed.

Change-Id: I0c33130a745b986f1bcd2ec177f78e3bb68d2271
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2021-11-12 10:44:04 +00:00
Stephen Finucane b88ea30701 db: Don't pass strings to 'Connection.execute'
Resolve the following RemovedIn20Warning warning:

  Passing a string to Connection.execute() is deprecated and will be
  removed in version 2.0.  Use the text() construct, or the
  Connection.exec_driver_sql() method to invoke a driver-level SQL
  string.

Change-Id: I44d6bf1ebfaf24f00a21389364456bceaae7c4d1
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-11-12 09:58:42 +00:00
Stephen Finucane 5d2399e210 db: Replace 'insert.inline' parameter with 'Insert.inline()' method
Resolve the following RemovedIn20Warning warning:

  The insert.inline parameter will be removed in SQLAlchemy 2.0.  Please
  use the Insert.inline() method.

Change-Id: Ib5fcb841294b4e20fe085e8603d4132e97be7db9
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-11-12 09:58:42 +00:00
Stephen Finucane c72068d466 db: Replace use of legacy select() calling style
Resolve the following RemovedIn20Warning warning:

  The legacy calling style of select() is deprecated and will be removed
  in SQLAlchemy 2.0.  Please use the new calling style described at
  select().

Change-Id: I36e43e30e07f4904c7b49925cefe804af45cff6c
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-11-12 09:58:42 +00:00
Stephen Finucane cd9b792ea6 db: Replace use of 'autoload' parameter
Resolve the following RemovedIn20Warning warnings:

  The autoload parameter is deprecated and will be removed in version
  2.0.  Please use the autoload_with parameter, passing an engine or
  connection.

Change-Id: I10e9bbf14706aece2a59ebb5861004c5b852a592
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-11-12 09:58:42 +00:00
Stephen Finucane cd3e5a23c9 db: Replace use of Executable.scalar(), Executable.execute()
Resolve the following RemovedIn20Warning warnings:

  The Executable.scalar() method is considered legacy as of the 1.x
  series of SQLAlchemy and will be removed in 2.0. Scalar execution in
  SQLAlchemy 2.0 is performed by the Connection.scalar() method of
  Connection, or in the ORM by the Session.scalar() method of Session.

  The Executable.execute() method is considered legacy as of the 1.x
  series of SQLAlchemy and will be removed in 2.0. All statement
  execution in SQLAlchemy 2.0 is performed by the Connection.execute()
  method of Connection, or in the ORM by the Session.execute() method of
  Session.

Note that while we're resolving these issues, we also head off other
issues that are currently ignored but will ultimately need to be
resolved such as switching to the modern calling style of 'select()'

Change-Id: Idebcba02a6704df21e5520fdbf60296b8187e79c
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2021-11-12 09:58:16 +00:00
Stephen Finucane 52bd1e51d6 tests: Enable SQLAlchemy 2.0 deprecation warnings
Well, sort of. We enable them but immediately filter out the ones we're
actually seeing, the rationale being that we can address these in a
piecemeal fashion without the risk of introducing new issues.

There's a lot more to be done here. However, the work done in oslo.db
[1] should provide a guide for how to resolve the outstanding issues.

[1] https://review.opendev.org/q/topic:%2522sqlalchemy-20%2522+project:openstack/oslo.db

Change-Id: Iafe726d3819031c357460fd131bb2bb58babb4e2
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2021-11-12 09:43:38 +00:00
Stephen Finucane e28afc5647 tests: Restore - don't reset - warning filters
There are more various warning filters pre-configured in a typical
Python environment, including a few from third-party libraries such as
requests [1][2] and urllib3 [3] as well as stdlib [4]. Our fixture to
configure warnings, 'WarningsFixture', called 'warnings.resetwarnings'
which *reset* all the warning filters [5]. This is clearly not something
we want to do, and resulted in tests puking warnings after the initial
test run.

Resolve this by backing up the existing warning filters before applying
the filter, and then *restoring* this original list of warning filters
after the test run.

[1] https://github.com/psf/requests/blob/v2.26.0/requests/__init__.py#L127
[2] https://github.com/psf/requests/blob/v2.26.0/requests/__init__.py#L152
[3] https://github.com/urllib3/urllib3/blob/1.26.7/src/urllib3/__init__.py#L68-L78
[4] https://docs.python.org/3.8/library/warnings.html#default-warning-filter
[5] https://docs.python.org/3.8/library/warnings.html#warnings.resetwarnings

Change-Id: I63f57980e01f472a25821790610f0836f1882a7f
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2021-11-11 19:27:51 +00:00
Zuul 79436a84cc Merge "Set "cache_ok=True" in "TypeDecorator" inheriting classes" 2021-11-11 14:37:56 +00:00
Zuul 20c8983761 Merge "Log failed metadata retrievals" 2021-11-11 14:37:33 +00:00
Dan Smith af7762713f Log instance event wait times
When trying to debug failed neutron VIF plugging events, it can be
tempting to just increase the timeout from the default to allow more
runway for those events to come in. In most cases, this is a fool's
errand because something is preventing them from coming at all. This
patch makes us debug log the event times regularly so that it is easy
to look at the history and see how long we normally wait on events,
to see if increasing the timeout is warranted or not.

Change-Id: I1be011f4dbcace78a698f9700170b8884e98a49b
2021-11-10 11:08:25 -08:00
Zuul a1e7ed3670 Merge "Apply common irrelevant_files for centos 8 job" 2021-11-10 18:35:54 +00:00
Zuul 70998b27d3 Merge "db: Remove nova-network models" 2021-11-10 18:04:36 +00:00
Zuul 0edaf2b9d4 Merge "db: Remove models for removed services, features" 2021-11-10 18:02:59 +00:00
Zuul 2761cdac3f Merge "objects: Remove 'bandwidth' fields from notifications" 2021-11-10 18:02:50 +00:00
Zuul dd11972b1b Merge "db: Remove models that were moved to the API database" 2021-11-10 17:56:48 +00:00
Balazs Gibizer ed3abea3b2 Apply common irrelevant_files for centos 8 job
Apply the common nova irrelevant files filter for the new
tempest-integrated-compute-centos-8-stream job

Change-Id: I0bacb8884a75b5ae604383d73d60fc618123a8d3
2021-11-09 17:46:12 +01:00
Zuul 78d398ad91 Merge "Fix interference in db unit test" 2021-11-09 13:59:03 +00:00
Zuul bae60995e3 Merge "Use ReplaceEngineFacade fixture" 2021-11-09 13:58:49 +00:00