Commit Graph

60919 Commits

Author SHA1 Message Date
Zuul bc1febbc07 Merge "tweak emulation job to avoid OOM errors" 2024-06-27 19:07:50 +00:00
melanie witt 438c405394 pre-commit: Bump autopep8 version and update repo
Attempts to run autopep8 via pre-commit fail for Python 3.12 with
error: "ModuleNotFoundError: No module named 'lib2to3'".

The issue was fixed in v2.1.0:

https://github.com/hhatto/autopep8/issues/581

but v2.1.0 is not available at
https://github.com/pre-commit/mirrors-autopep8 as the mirror has been
deprecated.

The README instructs to use https://github.com/hhatto/autopep8 directly
instead.

Change-Id: Ifdac3dad368298720682ab256e9dbf0f2624884e
2024-06-27 08:11:33 +00:00
Stephen Finucane be029783dd api: Add response body validation helper
No users yet: those will come soon.

Change-Id: I935f43518aa202631960beca44eeeef1db39b8bd
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2024-06-24 11:53:35 +01:00
Stephen Finucane 1a93703623 conf: Add '[api] response_validation' option
We have no schemas yet and thus nothing to use this, but we enable it in
our tests for when we start adding these.

Change-Id: I160ee724459403a10516a356ee860831545d7d65
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2024-06-24 11:42:50 +01:00
Stephen Finucane 0021b840a2 tests: Ensure all APIs have a request query schema
Change-Id: Ib920c9846f69b3af2a525abe60b02135e998e03a
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2024-06-24 11:42:44 +01:00
Stephen Finucane 41b436bff7 api: Add remaining missing query parameter schema
We already handled request bodies. Now we tackle query parameters. Note
that we're only handling GET requests currently since these are the APIs
where query parameters are most likely to be present. A future change
will add a test to verify that all methods handling GET requests have
schemas present.

This patch might look huge but every one of these schemas is effectively
a no-op since they're empty and have 'additionalProperties = True'. This
means there are lots of TODOs left in here which would be nice to tackle
in a future API microversion.

Change-Id: I80fc84df7c2be908856e6d4033b4922d7b0e9af1
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2024-06-24 11:40:48 +01:00
Stephen Finucane ff040ada4f api: Don't do version check if nothing required
There's no reason to inspect the API version requested (and reject
things if unset) if we don't have version constraints on the API
resource. This lets us add schema decorators to unversioned API
resources (i.e. the "versions" API at '/').

Change-Id: Ib4d4585e3e4664eba8cd3daa47b4349eb2bfed74
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2024-06-24 11:38:55 +01:00
Stephen Finucane b1824c5c97 api: Add 'removed' decorator
For APIs and resource actions that have been wholly removed and now
return a HTTP 4xx request regardless of microversion.

Change-Id: Ie3e90f4628dab4cc3e5983e0e8d8799dcf0465d8
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2024-06-24 11:38:27 +01:00
Elod Illes 9b77bae8a3 [CI] Replace deprecated regex
Latest Zuul drops the following warnings:

  All regular expressions must conform to RE2 syntax, but an
  expression using the deprecated Perl-style syntax has been detected.
  Adjust the configuration to conform to RE2 syntax.

  The RE2 syntax error is: invalid perl operator: (?!

This patch replaces the 'irrelevant-files' to 'files' with explicitly
listing the pattern which files should be the tests run against.

Change-Id: If287e800fb9ff428dbe6f9c4c046627f22afe3df
2024-06-18 16:18:33 +02:00
Stephen Finucane 7491417323 api: Migrate to JSON Schema Draft 2020-12
OpenAPI 3.1 is a superset of JSON Schema Draft 2020-12. As a result, we
wish to migrate our current schemas to this. There are a couple of
issues to address:

- 'exclusiveMinimum' is now an integer and allows you to define an
  minimum for an exclusive range, to complement 'minimum' which is used
  for an inclusive range. We can drop it and use 'minimum' with a larger
  number (since draft 6 [1]).

- 'integer' types can now accept a fractional part but only if it's zero
  (i.e. '1.0' is permissible, '1.1' is not) (since draft 6 [1]).

- 'items' has been replaced with 'prefixItems' for describing the format
  of an array item (since draft 2020-12 [2])

[1] https://json-schema.org/draft-06/json-schema-release-notes
[2] https://json-schema.org/draft/2020-12/release-notes

Change-Id: I1486701786960eef95c5c42674bff1b2d7d686e2
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2024-06-17 16:19:17 +00:00
Stephen Finucane c382f036c3 tests: Ensure API schemas are valid
Validate the validation by validating our schemas against the JSON
Schema meta schema. This is an important first step in getting us
of JSON Schema Draft 4 and onto Draft 2019-09, which OpenAPI is a
superset of.

Change-Id: I3b5a05aa0aa058e92c6927c9e3bee3cdd4477f8f
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2024-06-17 16:19:10 +00:00
Stephen Finucane 7e78623ac5 api: Fix bugs in schemas
None of these should affect us at runtime but they are not technically
valid against JSON Schema Draft 4.

Change-Id: Iaae8998a34ff812b9f76a0cd1fa9be4555c89b98
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2024-06-17 16:19:03 +00:00
Stephen Finucane 917c4f0721 tests: Ensure all APIs have a request body schema
The idea here is to test that we are doing schema validation against
all routes that accept a request body (i.e. POST, PUT, PATCH). We can
use these schemas down the line to generate OpenAPI docs like all the
cool kids do.

Change-Id: Icdf20e7e5b38c9f5324eac39ec1d6327609bf6d4
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2024-06-17 16:18:57 +00:00
Stephen Finucane f86cd38dfe api: Add remaining missing request body schemas
This provides us a path to generate OpenAPI schemas for the nova API.
Thankfully most of the API documentation is pretty darn simple. Note
that we need to fix a lot of unit tests so that they pass correct bodies
to the controller methods under test. Also note that the
'reset_server_state' module in 'nova.api.openstack.compute.schemas' is
renamed to 'admin_actions' to match the name of the module containing
the controller itself, as is the case for all other schemas.

Change-Id: I39098fb621b120589ed6a923ce64207ee26e91aa
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2024-06-17 16:18:51 +00:00
Stephen Finucane 847608e75a api: Add request body schemas for SG APIs
These are deprecated but there's value in having a proper - if loose -
schema in place for API documentation purposes. Also, doing things this
way allows us to remove a whole load of hand-rolled stuff.

Change-Id: I4106cfa2a09d135f12892ed6d1f42f4151dc72e4
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2024-06-17 17:18:25 +01:00
Zuul 7dc4b1ea62 Merge "add functional repoducer for bug 2065927" 2024-05-31 10:44:09 +00:00
Zuul 96268d4e7a Merge "libvirt: Ensure both swtpm and swtpm_setup exist for vTPM support" 2024-05-28 09:07:13 +00:00
Sean Mooney 44c1b48b31 retry write_sys call on device busy
This change adds a retry_if_busy decorator
to the read_sys and write_sys functions in the filesystem
module that will retry reads and writes up to 5 times with
an linear backoff.

This allows nova to tolerate short periods of time where
sysfs retruns device busy. If the reties are exausted
and offlineing a core fails a warning is log and the failure is
ignored. onling a core is always treated as a hard error if
retries are exausted.

Closes-Bug: #2065927
Change-Id: I2a6a9f243cb403167620405e167a8dd2bbf3fa79
2024-05-27 18:31:31 +01:00
Sean Mooney 3811c7f648 tweak emulation job to avoid OOM errors
This change increases the swap in the emulation job form
1G to 8G
This change updates the default cirrors image form 0.5.2
to 0.5.3 to avoid know kernel bugs
this change reduces the tb_cache_size: to 128
The tempest concurrency is reduced to 4 to avoid
no valid host error in the resize tests

Change-Id: Ic1dde3d54f5ca12408ef53218773a27d55760705
2024-05-27 14:19:02 +01:00
Zuul ac4a67cbda Merge "docs: Follow up for persistent mdevs" 2024-05-27 11:01:32 +00:00
Zuul 3dfdc10f63 Merge "libvirt: Create persistent mdevs" 2024-05-27 11:01:26 +00:00
Zuul bded279a00 Merge "docs: Add more information about unified limits" 2024-05-23 19:48:39 +00:00
melanie witt c7e49dfa16 docs: Follow up for persistent mdevs
This address review feedback on change
I7e1d10e66a260efd0a3f2d6522aeb246c7582178 to add some clarifying text
to the docs and release note.

Related to blueprint persistent-mdevs

Change-Id: I472552c64cc2c2ce06896158664faac0199d90bd
2024-05-23 18:29:04 +00:00
Zuul d7d2fb1edd Merge "scheduler: AggregateMultitenancyIsolation to support unlimited tenant" 2024-05-23 15:35:54 +00:00
Zuul 4e3a41f0a4 Merge "Stop using split UEC image (mostly)" 2024-05-22 19:03:32 +00:00
Zuul 3a53d715cd Merge "[doc] Improve description for nova-manage db purge" 2024-05-22 02:44:50 +00:00
Alexey Stupnikov ac8729ac87 [doc] Improve description for nova-manage db purge
--before argument is currently described in ambiguous way: it
is not actually used to filter entries ARCHIVED before specified
date. Instead, it compares provided data with "deleted_at" value
for most rows and "updated_at" or "created_at" for remaining ones.

Since we already talk about time of deletion when describing
--before argument of "nova-manage db archive_deleted_rows" rows,
it make sense to not provide extra details here as well.

Change-Id: Ib5940e88a52dc8d32303e27237e567c3481fc3dc
2024-05-21 20:19:18 +02:00
Zuul 4bc5ff1c99 Merge "fix py312 tox definitions" 2024-05-21 17:43:27 +00:00
Sean Mooney ee581a5c9d add functional repoducer for bug 2065927
Today if the write sys call to offline a cpu when
deleting an instnace fails due to an OSERROR or ValueERROR
the instance delete fails and the instance goes to error.

as reported in bug: #2065927 this can happen as a result of
OSError: [Errno 16] Device or resource busy if the vm is
deleted shortly after its started.

Related-Bug: #2065927
Change-Id: I1352a3a1e28cfe14ec8f32042ed35cb25e70338e
2024-05-21 17:57:29 +01:00
Zuul c60b81fa4b Merge "tox: Drop envdir" 2024-05-21 15:17:45 +00:00
Sean Mooney 7ff24958ee fix py312 tox definitions
change I6de86f3e3e283ba404f927ea4c8164f791df3989
added the py312 funtional job definition but did not
update the tox.ini to define it.
As a result it is runing the unit tests not the functional tests.
This change simpley corrects that.

Change-Id: Id6ee76e0190469ac09baf0bc56a9022317c6f881
2024-05-21 12:03:41 +01:00
Zuul 61ad4f1f27 Merge "Enable virtio-scsi in nova-next" 2024-05-20 21:41:24 +00:00
Dan Smith eed3e2b47f Stop using split UEC image (mostly)
This reverts us back to using the standard disk image for most of our
tests, which is more representative of how people actually use nova.
This leaves the UEC image on a few jobs for the sake of comparison
data for the time being, and because we should actually test that
code path if we're going to say we support it.

Change-Id: I16ed92d342464325d4bef33c1e22b328bcfbe7d6
2024-05-20 08:24:18 -07:00
Dan Smith 84b0a481fe Enable OCaaS for several nova jobs
This uses the OCaaS feature in devstack which saves *minutes* of time
running devstack by effectively caching the openstackclient startup
instead of taking that hit for every invocation.

Change-Id: I78308128c6249f7f871e0231ce717b1ec0f88509
2024-05-20 08:24:18 -07:00
melanie witt d45379c6b4 docs: Add more information about unified limits
The admin docs are missing some details about enabling unified limits,
like oslo.limit configuration and Keystone roles. This adds more
information about what roles are needed for what actions, how to set
quota limits, quota enforcement, and unified limits in general.

This also removes a couple of tables from the user docs that show
obsolete/deprecated quota limits because they may be more confusing
than helpful considering we don't want new deployments to use them and
they add more clutter to the page.

More info is also added regarding the CLI commands for unified limits
and makes it consistent between the user and admin docs.

Change-Id: Id93f9997d1b217e0c2151c88323564f7a7fefc02
2024-05-17 19:17:05 +00:00
Zuul ab3ca1e205 Merge "Make python 3.12 unit and functional voting" 2024-05-16 15:42:38 +00:00
Zuul b6a846c6c8 Merge "Fix hacking test with syntax error" 2024-05-16 14:26:26 +00:00
Zuul fa44978141 Merge "Fix notification object hashes for python 3.12" 2024-05-16 14:26:18 +00:00
Pierre Riteau bfd3525863 Fix formatting issues in extra-specs docs
Change-Id: I693f70ffb7630fe99336fef52783ea55c492624d
2024-05-16 11:08:19 +02:00
Zuul d467edac49 Merge "do not use str(url) to stringify a URL for subsequent use" 2024-05-15 20:05:28 +00:00
Dan Smith 50b180023f Make python 3.12 unit and functional voting
Change-Id: I6de86f3e3e283ba404f927ea4c8164f791df3989
2024-05-15 12:29:14 -07:00
Dan Smith 3f0879ccc3 Fix hacking test with syntax error
This hacking test has a syntax error in it. On older pythons, this
does not prevent us from finding the second popen() use, but on
python 3.12 it does.

Change-Id: Ib74dc030118e0cb9fab548b112d32ce080969a15
2024-05-15 12:12:55 -07:00
Dan Smith 6ee938fd22 Fix notification object hashes for python 3.12
Python 3.12 changes the repr() syntax for the OrderedDict object,
which causes us to calculate different hashes for the notification
objects across the version gap. The get_extra_data() function wraps
a dict in OrderedDict after sorting the elements of the original dict,
but that is not entirely necessary. The sorted() list of dict.items()
is a perfectly reasonable representation of the things it is capturing,
so we can just eliminate the use of OrderedDict there and thus end
up calculating the same hashes on <3.12 and >3.12.

NOTE: This changes a bunch of the hashes of notification objects
without bumping or changing versions. This is expected purely because
we are changing the hashing method, and thus this is not a violation
or upgrade concern.

Change-Id: I242150138deed7fe74b13d9c44b333293cd24ffa
2024-05-15 12:12:54 -07:00
Zuul 2e6041a76e Merge "Remove SQLAlchemy tips jobs" 2024-05-15 16:51:17 +00:00
Mike Bayer acbe3e28e5 do not use str(url) to stringify a URL for subsequent use
The str(url) function in SQLAlchemy hides the password.
For a URL string that is to be re-used, use
render_as_string(hide_password=False).

Change-Id: I2ab28da5cc2b9ed3a1588259b2e94320662816bb
2024-05-15 16:23:58 +01:00
Zuul 5095336689 Merge "Upload glance image with --file in ceph job" 2024-05-14 16:40:32 +00:00
Zuul 7096423b34 Merge "Reject AZ changes during aggregate add / remove host" 2024-05-09 20:17:32 +00:00
Zuul 5470dedd4d Merge "Fix device_type=lun with boot_index" 2024-05-09 17:32:28 +00:00
Zuul 67119b7de3 Merge "Avoid setting serial on raw LUN devices" 2024-05-08 18:36:11 +00:00
Dan Smith a4e72f71fc Upload glance image with --file in ceph job
This enables our use of the OCaaS devstack feature, which can't support
image upload from stdin.

Change-Id: Idc5646ef6763447e3c1c68de03dea8197c305f6c
2024-05-08 10:57:06 -07:00