Commit Graph

59337 Commits

Author SHA1 Message Date
Lee Yarwood fcbba7d2ae libvirt: Ensure all volume drivers log the instance whenever possible
This trivial change ensures the instance is logged within the volume
drivers whenever possible to ease debugging.

Change-Id: Ib61ba7266ad58b311adcac566a96149839cb688e
2021-12-03 12:07:02 +00:00
Zuul 7670303aab Merge "Remove broken tempest-full-py3-opensuse15 job" 2021-12-01 19:48:17 +00:00
Zuul c5c2cc0341 Merge "docs: Add notes about IoOpsWeigher, BuildFailureWeigher" 2021-12-01 15:32:28 +00:00
Zuul d630615a02 Merge "Updating tests with Yoga testing runtime" 2021-11-26 11:13:36 +00:00
Zuul 4a359d576d Merge "db: Don't use legacy 'Row()' methods" 2021-11-25 19:50:01 +00:00
Zuul f1258bca05 Merge "docs: Update libvirt distro support matrix for Xena" 2021-11-25 17:40:49 +00:00
Ghanshyam Mann 9dd0070ce6 Updating tests with Yoga testing runtime
Yoga testing runtime is updated now
- https://governance.openstack.org/tc/reference/runtimes/yoga.html

which needs to test py38 and py39. Unit tests update are
handled by the job template change in openstack-zuul-job and
this commit makes changes to fucntional job to run py39 as voting
and updating the metdata in setup file.

Change-Id: I314fd61f20f2c3551f6231d191b7dcaaefabd2b5
2021-11-25 17:38:32 +00:00
Zuul 3047cde210 Merge "Retry image download if it's corrupted" 2021-11-25 13:05:55 +00:00
Zuul 8f250f5044 Merge "db: Don't pass strings to 'Connection.execute'" 2021-11-23 11:35:30 +00:00
Zuul e39690d792 Merge "db: Replace 'insert.inline' parameter with 'Insert.inline()' method" 2021-11-23 11:35:23 +00:00
Zuul 5168d74281 Merge "db: Replace use of legacy select() calling style" 2021-11-23 11:26:51 +00:00
Zuul 55210a5faa Merge "db: Replace use of 'autoload' parameter" 2021-11-23 11:26:42 +00:00
Zuul 841e62f4d4 Merge "db: Replace use of Executable.scalar(), Executable.execute()" 2021-11-23 11:09:58 +00:00
sdmitriev1 ce493273b9 Retry image download if it's corrupted
Adding IOError in list of catching exceptions in order to
fix behavior when nova-compute wouldn't retry image download
when got "Corrupt image download" error from glanceclient
and had num_retries config option set.

Closes-Bug: #1950657
Change-Id: Iae4fd0579f71d3ba6793dbdb037275352d7e57b0
2021-11-22 16:15:27 -05:00
Zuul 07dfd853b2 Merge "nova-manage: Always get BDMs using get_by_volume_and_instance" 2021-11-22 18:39:03 +00:00
Zuul a41783ed49 Merge "Log instance event wait times" 2021-11-22 17:35:48 +00:00
Zuul 69b0d31d20 Merge "Add a WA flag waiting for vif-plugged event during reboot" 2021-11-18 20:06:43 +00:00
Zuul 7aa3a0f558 Merge "libvirt: Switch the default video model from 'cirrus' to 'virtio'" 2021-11-18 15:54:13 +00:00
Lee Yarwood 1241d3440b docs: Update libvirt distro support matrix for Xena
This is mostly distro updates as our MIN versions didn't change during
the cycle.

Change-Id: If30d75c8fa96aa6ed7f0483d791d1dea86f58c08
2021-11-18 14:12:49 +00:00
Stephen Finucane be4e01c101 db: Don't use legacy 'Row()' methods
Resolve the following RemovedIn20Warning warnings:

  The Row.keys() method is considered legacy as of the 1.x series of
  SQLAlchemy and will be removed in 2.0. Use the namedtuple standard
  accessor Row._fields, or for full mapping behavior use
  row._mapping.keys()

An additional warning that appears to have been resolved in the interim
is also removed.

Change-Id: I0c33130a745b986f1bcd2ec177f78e3bb68d2271
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2021-11-12 10:44:04 +00:00
Stephen Finucane b88ea30701 db: Don't pass strings to 'Connection.execute'
Resolve the following RemovedIn20Warning warning:

  Passing a string to Connection.execute() is deprecated and will be
  removed in version 2.0.  Use the text() construct, or the
  Connection.exec_driver_sql() method to invoke a driver-level SQL
  string.

Change-Id: I44d6bf1ebfaf24f00a21389364456bceaae7c4d1
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-11-12 09:58:42 +00:00
Stephen Finucane 5d2399e210 db: Replace 'insert.inline' parameter with 'Insert.inline()' method
Resolve the following RemovedIn20Warning warning:

  The insert.inline parameter will be removed in SQLAlchemy 2.0.  Please
  use the Insert.inline() method.

Change-Id: Ib5fcb841294b4e20fe085e8603d4132e97be7db9
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-11-12 09:58:42 +00:00
Stephen Finucane c72068d466 db: Replace use of legacy select() calling style
Resolve the following RemovedIn20Warning warning:

  The legacy calling style of select() is deprecated and will be removed
  in SQLAlchemy 2.0.  Please use the new calling style described at
  select().

Change-Id: I36e43e30e07f4904c7b49925cefe804af45cff6c
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-11-12 09:58:42 +00:00
Stephen Finucane cd9b792ea6 db: Replace use of 'autoload' parameter
Resolve the following RemovedIn20Warning warnings:

  The autoload parameter is deprecated and will be removed in version
  2.0.  Please use the autoload_with parameter, passing an engine or
  connection.

Change-Id: I10e9bbf14706aece2a59ebb5861004c5b852a592
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-11-12 09:58:42 +00:00
Stephen Finucane cd3e5a23c9 db: Replace use of Executable.scalar(), Executable.execute()
Resolve the following RemovedIn20Warning warnings:

  The Executable.scalar() method is considered legacy as of the 1.x
  series of SQLAlchemy and will be removed in 2.0. Scalar execution in
  SQLAlchemy 2.0 is performed by the Connection.scalar() method of
  Connection, or in the ORM by the Session.scalar() method of Session.

  The Executable.execute() method is considered legacy as of the 1.x
  series of SQLAlchemy and will be removed in 2.0. All statement
  execution in SQLAlchemy 2.0 is performed by the Connection.execute()
  method of Connection, or in the ORM by the Session.execute() method of
  Session.

Note that while we're resolving these issues, we also head off other
issues that are currently ignored but will ultimately need to be
resolved such as switching to the modern calling style of 'select()'

Change-Id: Idebcba02a6704df21e5520fdbf60296b8187e79c
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2021-11-12 09:58:16 +00:00
Stephen Finucane 52bd1e51d6 tests: Enable SQLAlchemy 2.0 deprecation warnings
Well, sort of. We enable them but immediately filter out the ones we're
actually seeing, the rationale being that we can address these in a
piecemeal fashion without the risk of introducing new issues.

There's a lot more to be done here. However, the work done in oslo.db
[1] should provide a guide for how to resolve the outstanding issues.

[1] https://review.opendev.org/q/topic:%2522sqlalchemy-20%2522+project:openstack/oslo.db

Change-Id: Iafe726d3819031c357460fd131bb2bb58babb4e2
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2021-11-12 09:43:38 +00:00
Stephen Finucane e28afc5647 tests: Restore - don't reset - warning filters
There are more various warning filters pre-configured in a typical
Python environment, including a few from third-party libraries such as
requests [1][2] and urllib3 [3] as well as stdlib [4]. Our fixture to
configure warnings, 'WarningsFixture', called 'warnings.resetwarnings'
which *reset* all the warning filters [5]. This is clearly not something
we want to do, and resulted in tests puking warnings after the initial
test run.

Resolve this by backing up the existing warning filters before applying
the filter, and then *restoring* this original list of warning filters
after the test run.

[1] https://github.com/psf/requests/blob/v2.26.0/requests/__init__.py#L127
[2] https://github.com/psf/requests/blob/v2.26.0/requests/__init__.py#L152
[3] https://github.com/urllib3/urllib3/blob/1.26.7/src/urllib3/__init__.py#L68-L78
[4] https://docs.python.org/3.8/library/warnings.html#default-warning-filter
[5] https://docs.python.org/3.8/library/warnings.html#warnings.resetwarnings

Change-Id: I63f57980e01f472a25821790610f0836f1882a7f
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2021-11-11 19:27:51 +00:00
Zuul 79436a84cc Merge "Set "cache_ok=True" in "TypeDecorator" inheriting classes" 2021-11-11 14:37:56 +00:00
Zuul 20c8983761 Merge "Log failed metadata retrievals" 2021-11-11 14:37:33 +00:00
Dan Smith af7762713f Log instance event wait times
When trying to debug failed neutron VIF plugging events, it can be
tempting to just increase the timeout from the default to allow more
runway for those events to come in. In most cases, this is a fool's
errand because something is preventing them from coming at all. This
patch makes us debug log the event times regularly so that it is easy
to look at the history and see how long we normally wait on events,
to see if increasing the timeout is warranted or not.

Change-Id: I1be011f4dbcace78a698f9700170b8884e98a49b
2021-11-10 11:08:25 -08:00
Zuul a1e7ed3670 Merge "Apply common irrelevant_files for centos 8 job" 2021-11-10 18:35:54 +00:00
Zuul 70998b27d3 Merge "db: Remove nova-network models" 2021-11-10 18:04:36 +00:00
Zuul 0edaf2b9d4 Merge "db: Remove models for removed services, features" 2021-11-10 18:02:59 +00:00
Zuul 2761cdac3f Merge "objects: Remove 'bandwidth' fields from notifications" 2021-11-10 18:02:50 +00:00
Zuul dd11972b1b Merge "db: Remove models that were moved to the API database" 2021-11-10 17:56:48 +00:00
Balazs Gibizer ed3abea3b2 Apply common irrelevant_files for centos 8 job
Apply the common nova irrelevant files filter for the new
tempest-integrated-compute-centos-8-stream job

Change-Id: I0bacb8884a75b5ae604383d73d60fc618123a8d3
2021-11-09 17:46:12 +01:00
Zuul 78d398ad91 Merge "Fix interference in db unit test" 2021-11-09 13:59:03 +00:00
Zuul bae60995e3 Merge "Use ReplaceEngineFacade fixture" 2021-11-09 13:58:49 +00:00
Zuul 59e06f9f3a Merge "Refactor Database fixture" 2021-11-09 13:58:38 +00:00
Zuul 9123ec52ff Merge "Remove SESSION_CONFIGURED global from DB fixture" 2021-11-09 13:21:09 +00:00
Zuul e537d90d6f Merge "Ignore plug_vifs on the ironic driver" 2021-11-09 04:36:51 +00:00
Zuul 0e0196d979 Merge "Add autopep8 to tox and pre-commit" 2021-11-08 23:54:03 +00:00
Zuul c62616963a Merge "Revert "tox: Encode specific Python versions"" 2021-11-08 14:23:55 +00:00
Zuul cb4ba89f78 Merge "objects: Stop querying the main DB for keypairs" 2021-11-08 13:26:21 +00:00
Zuul 83d01e64d8 Merge "db: Remove legacy placement models" 2021-11-08 13:26:11 +00:00
Sean Mooney f3d48000b1 Add autopep8 to tox and pre-commit
autopep8 is a code formating tool that makes python code pep8
compliant without changing everything. Unlike black it will
not radically change all code and the primary change to the
existing codebase is adding a new line after class level doc strings.

This change adds a new tox autopep8 env to manually run it on your
code before you submit a patch, it also adds autopep8 to pre-commit
so if you use pre-commit it will do it for you automatically.

This change runs autopep8 in diff mode with --exit-code in the pep8
tox env so it will fail if autopep8 would modify your code if run
in in-place mode. This allows use to gate on autopep8 not modifying
patches that are submited. This will ensure authorship of patches is
maintianed.

The intent of this change is to save the large amount of time we spend
on ensuring style guidlines are followed automatically to make it
simpler for both new and old contibutors to work on nova and save
time and effort for all involved.

Change-Id: Idd618d634cc70ae8d58fab32f322e75bfabefb9d
2021-11-08 12:37:27 +00:00
Balazs Gibizer a7bccff06b Fix interference in db unit test
We discovered that two unit test cases added in
I0647bb8545c1464b521a1d866cf5ee674aea2eae cause errors like
  oslo_db.sqlalchemy.enginefacade.AlreadyStartedError:
  this TransactionFactory is already started

when the db tests run selectively with tox -e py38 nova.tests.unit.db
but does not cause errors if the whole unit test suit is run.

This error happened because our db code uses two global transaction
factory, one for the api DB and one for the main DB. There was a global
flag SESSION_CONFIGURED in our Database fixture that guarded against
double initialization of the factory. But the faulty test cases in
question do not use our Database fixture but use the
OpportunisticDBTestMixin from oslo_db. Obviously that fixture does not
know about our SESSION_CONFIGURED global. So if one of the offending
test case ran first in an executor then that initialized the
transaction factory globally and a later test that uses our Database
fixture tried to configure it again leading to the error. For some
unknown reason if these tests were run in the opposite order the faulty
re-initialization did not happen. Probably the OpportunisticDBTestMixin
was able to prevent that.

A previous patch already removed the global SESSION_CONFIGURED flag
from our fixture and replaced it with a per DB specific patch_factory
calls that allow resetting the state of the factory at the end of each
test case. This would already solve the current test case issue as only
our offending test cases would initialize the global factory without
cleanup and we have one test case per DB. So there would be no
interference. However if in the future we add similar test cases then
those can still interfere through the global factory.

So this patch fixes the two offending test case. Also it extends the
DatabasePoisonFixture used for the NoDbTestCase tesst. The poison now
detects if the test case starts any transaction factory.

This poison caught another offender test case,
test_db_sync_with_special_symbols_in_connection_string, that was marked
NoDb but actually using the database. It is now changed to declare
itself as a test that sets up the DB manually and also it is changed to
use the Database fixture instead of touching the global factory
directly.

Closes-Bug: #1948963

Change-Id: Id96f1795034490c13125ebbab49b029fb96af1c7
2021-11-05 11:41:27 +01:00
Balazs Gibizer 063ed41174 Use ReplaceEngineFacade fixture
Now that the previous patch Ifc070d19a18a2d66f1a7bd5898428b12901dfe9e moved
most of the logic to the setUp of the Database fixture we can replace
our direct factory patching with the ReplaceEngineFacade fixture from
oslo_db.

Change-Id: Icd25adcc931cae2126e03c00af7e4420d3781b9a
2021-11-05 11:39:17 +01:00
zhangbailin f024490e95 [Trival] Fix wrong microversion in TestClass name
Change-Id: I0dce3963c7726cf126ad2c4b3e25f005fdd12790
2021-11-05 10:10:50 +08:00
Balazs Gibizer c7ccbfe403 Refactor Database fixture
This patch apply the following changes to the fixture to make the
intention clearer and remove some unnecessary complexity:

* the fixture does a lot of dynamic thing in its __init__, these are
  moved to setUp() instead to facilitate proper reset functionality of
  the fixture

* the caching and applying of the DB schema is made a explicit and
  moved to setUp() too

* the explicit reset() function is removed as it is probably
  unintentionally overwrote the Fixture.reset(). Now the Fixture can be
  properly reset by calling the Fixture.reset() which is by default
  implemented by calling cleanUp() and setUp()

Change-Id: Ic58e93d6aafb88be4abeb6e52089f7ee43d8db01
2021-11-04 17:18:54 +01:00